2023-10-06T02:48:12.442 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T02:48:12.442 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176 --name yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} -- /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/orig.config.yaml 2023-10-06T02:48:12.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T02:48:12.575 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176 branch: wip-yuri-testing-2023-10-05-0730-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: yweinste@redhat.com first_in_suite: false job_id: '7412176' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-05-0730-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-05-0730-quincy sha1: f1e315a9732d36c05ec316678e2397dc58561c7f owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7519 sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-05-0730-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f1e315a9732d36c05ec316678e2397dc58561c7f targets: smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNySbKwKtbn23kaOpdUEXi+78qW6CwALxFpT/RCDrTx+bL/kB6CcHKIvUERPjiPH16MiOBLWYiifw3LJjrmqQLo= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKH/OkfSWXHIbERePHsgBONEAbvU0eMC2nwTK72hhDG2ASmy/zmAyTPj/6R828c1bvdoAMZ2gsV8tNbd6Z76RR4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:52:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T02:48:12.575 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa; will attempt to use it 2023-10-06T02:48:12.576 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks 2023-10-06T02:48:12.576 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T02:48:12.578 INFO:teuthology.task.internal:Checking packages... 2023-10-06T02:48:12.603 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2' 2023-10-06T02:48:12.603 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T02:48:12.603 INFO:teuthology.packaging:ref: None 2023-10-06T02:48:12.603 INFO:teuthology.packaging:tag: None 2023-10-06T02:48:12.603 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T02:48:12.603 INFO:teuthology.packaging:sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:48:12.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T02:48:12.828 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1296.gf1e315a9 2023-10-06T02:48:12.829 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T02:48:12.835 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T02:48:12.835 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T02:48:12.882 INFO:teuthology.task.internal:Saving configuration 2023-10-06T02:48:12.898 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T02:48:12.915 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T02:48:12.939 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:43:07.402300', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNySbKwKtbn23kaOpdUEXi+78qW6CwALxFpT/RCDrTx+bL/kB6CcHKIvUERPjiPH16MiOBLWYiifw3LJjrmqQLo='} 2023-10-06T02:48:12.963 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:43:07.401011', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKH/OkfSWXHIbERePHsgBONEAbvU0eMC2nwTK72hhDG2ASmy/zmAyTPj/6R828c1bvdoAMZ2gsV8tNbd6Z76RR4='} 2023-10-06T02:48:12.963 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T02:48:12.969 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T02:48:12.969 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T02:48:12.969 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T02:48:13.067 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd9f70b3b80>, signals=[15]) 2023-10-06T02:48:13.067 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T02:48:13.073 INFO:teuthology.task.internal:Opening connections... 2023-10-06T02:48:13.073 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T02:48:13.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:48:13.143 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T02:48:13.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:48:13.210 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T02:48:13.216 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2023-10-06T02:48:13.233 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2023-10-06T02:48:13.233 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2023-10-06T02:48:13.292 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T02:48:13.292 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T02:48:13.292 INFO:teuthology.orchestra.run.smithi092.stdout:ID="rhel" 2023-10-06T02:48:13.292 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="fedora" 2023-10-06T02:48:13.292 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="8.4" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el8" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T02:48:13.293 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T02:48:13.294 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2023-10-06T02:48:13.323 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2023-10-06T02:48:13.339 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2023-10-06T02:48:13.340 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2023-10-06T02:48:13.395 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T02:48:13.395 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:ID="rhel" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="fedora" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="8.4" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el8" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T02:48:13.396 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T02:48:13.397 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T02:48:13.397 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T02:48:13.397 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T02:48:13.397 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T02:48:13.397 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2023-10-06T02:48:13.423 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T02:48:13.432 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T02:48:13.457 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T02:48:13.458 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2023-10-06T02:48:13.460 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2023-10-06T02:48:13.475 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T02:48:13.491 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T02:48:13.491 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2023-10-06T02:48:13.520 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2023-10-06T02:48:13.534 INFO:teuthology.orchestra.run.smithi092.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T02:48:13.537 INFO:teuthology.orchestra.run.smithi136.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T02:48:13.538 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T02:48:13.593 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T02:48:13.607 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T02:48:13.607 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-06T02:48:13.607 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-06T02:48:13.607 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T02:48:13.608 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T02:48:13.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:48:13.623 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-10-06T02:48:13.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:48:13.625 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2023-10-06T02:48:13.681 INFO:teuthology.orchestra.run.smithi136.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T02:48:13.681 INFO:teuthology.task.kernel:Running kernel on smithi136: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T02:48:13.681 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2023-10-06T02:48:13.682 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T02:48:13.682 INFO:teuthology.task.kernel:Running kernel on smithi092: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T02:48:13.683 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum install -y kernel 2023-10-06T02:48:15.066 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T02:48:15.066 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-10-06T02:48:15.173 INFO:teuthology.orchestra.run.smithi136.stdout:Updating Subscription Management repositories. 2023-10-06T02:48:15.173 INFO:teuthology.orchestra.run.smithi136.stdout:Unable to read consumer identity 2023-10-06T02:48:16.489 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-10-06T02:48:16.519 INFO:teuthology.orchestra.run.smithi136.stdout:Copr repo for python3-asyncssh owned by ceph 35 kB/s | 3.5 kB 00:00 2023-10-06T02:48:17.045 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 31 MB/s | 16 MB 00:00 2023-10-06T02:48:17.296 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-10-06T02:48:20.322 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 4.9 MB/s | 24 kB 00:00 2023-10-06T02:48:20.689 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-10-06T02:48:21.004 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T02:48:21.004 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T02:48:21.004 INFO:teuthology.orchestra.run.smithi092.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T02:48:21.164 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-10-06T02:48:21.165 INFO:teuthology.orchestra.run.smithi092.stdout:Nothing to do. 2023-10-06T02:48:21.165 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-10-06T02:48:21.276 DEBUG:teuthology.orchestra.run.smithi092:> echo no | sudo yum reinstall kernel || true 2023-10-06T02:48:21.438 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T02:48:21.438 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T02:48:21.438 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T02:48:21.597 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2023-10-06T02:48:21.598 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2023-10-06T02:48:21.598 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2023-10-06T02:48:21.620 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T02:48:21.620 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-10-06T02:48:21.730 DEBUG:teuthology.orchestra.run.smithi136:> echo no | sudo yum reinstall kernel || true 2023-10-06T02:48:21.906 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:01 ago on Fri 06 Oct 2023 02:48:20 AM UTC. 2023-10-06T02:48:22.034 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T02:48:22.034 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:22.034 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:22.060 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-10-06T02:48:22.073 INFO:teuthology.orchestra.run.smithi136.stdout:Updating Subscription Management repositories. 2023-10-06T02:48:22.073 INFO:teuthology.orchestra.run.smithi136.stdout:Unable to read consumer identity 2023-10-06T02:48:22.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum reinstall -y kernel || true 2023-10-06T02:48:22.369 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 02:48:20 AM UTC. 2023-10-06T02:48:22.454 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T02:48:22.454 INFO:teuthology.orchestra.run.smithi092.stdout:Unable to read consumer identity 2023-10-06T02:48:22.520 INFO:teuthology.orchestra.run.smithi136.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T02:48:22.520 INFO:teuthology.orchestra.run.smithi136.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:22.520 INFO:teuthology.orchestra.run.smithi136.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:22.557 INFO:teuthology.orchestra.run.smithi136.stderr:Error: No packages marked for reinstall. 2023-10-06T02:48:22.607 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum reinstall -y kernel || true 2023-10-06T02:48:22.747 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 02:48:20 AM UTC. 2023-10-06T02:48:22.891 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T02:48:22.891 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:22.892 INFO:teuthology.orchestra.run.smithi092.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:Error: No packages marked for reinstall. 2023-10-06T02:48:22.953 INFO:teuthology.orchestra.run.smithi136.stdout:Updating Subscription Management repositories. 2023-10-06T02:48:22.954 INFO:teuthology.orchestra.run.smithi136.stdout:Unable to read consumer identity 2023-10-06T02:48:22.961 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T02:48:23.085 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.086 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.086 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.086 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T02:48:23.086 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T02:48:23.086 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-06T02:48:23.086 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T02:48:23.087 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T02:48:23.211 INFO:teuthology.orchestra.run.smithi092.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.211 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo rpm -qi grub2-tools 2023-10-06T02:48:23.237 INFO:teuthology.orchestra.run.smithi092.stdout:Name : grub2-tools 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Epoch : 1 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Version : 2.02 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Release : 99.el8_4.1 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: x86_64 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Group : System Environment/Base 2023-10-06T02:48:23.238 INFO:teuthology.orchestra.run.smithi092.stdout:Size : 9264631 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:License : GPLv3+ 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Relocations : (not relocatable) 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Packager : Red Hat, Inc. 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Vendor : Red Hat, Inc. 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T02:48:23.239 INFO:teuthology.orchestra.run.smithi092.stdout:Summary : Support tools for GRUB. 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:Description : 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:hardware devices. 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T02:48:23.240 INFO:teuthology.orchestra.run.smithi092.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T02:48:23.241 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T02:48:23.241 INFO:teuthology.task.kernel:Updating grub on smithi092 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T02:48:23.245 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:03 ago on Fri 06 Oct 2023 02:48:20 AM UTC. 2023-10-06T02:48:23.402 INFO:teuthology.orchestra.run.smithi136.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T02:48:23.402 INFO:teuthology.orchestra.run.smithi136.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:23.403 INFO:teuthology.orchestra.run.smithi136.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T02:48:23.422 INFO:teuthology.orchestra.run.smithi136.stderr:Error: No packages marked for reinstall. 2023-10-06T02:48:23.478 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T02:48:23.603 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.603 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.603 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.603 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T02:48:23.603 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T02:48:23.603 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-06T02:48:23.603 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T02:48:23.604 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T02:48:23.728 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.729 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.729 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2023-10-06T02:48:23.755 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2023-10-06T02:48:23.755 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.02 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 99.el8_4.1 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Group : System Environment/Base 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 9264631 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T02:48:23.756 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:Relocations : (not relocatable) 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : Red Hat, Inc. 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : Red Hat, Inc. 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T02:48:23.757 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T02:48:23.758 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T02:48:23.758 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:23.758 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T02:48:24.543 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2023-10-06T02:48:25.223 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2023-10-06T02:48:26.618 INFO:teuthology.orchestra.run.smithi092.stderr:done 2023-10-06T02:48:26.619 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-10-06T02:48:26.638 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.R3hJusvec1 2023-10-06T02:48:26.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.R3hJusvec1 2023-10-06T02:48:26.724 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.R3hJusvec1 2023-10-06T02:48:26.847 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.R3hJusvec1 is 6KB 2023-10-06T02:48:26.895 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.R3hJusvec1 2023-10-06T02:48:26.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T02:48:26.970 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T02:48:26.970 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T02:48:26.970 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T02:48:26.970 INFO:teuthology.orchestra.run.smithi092.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T02:48:26.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:27.124 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2023-10-06T02:48:27.157 INFO:teuthology.orchestra.run.smithi136.stderr:done 2023-10-06T02:48:27.159 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2023-10-06T02:48:27.173 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.3Of2EX38ZO 2023-10-06T02:48:27.173 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3Of2EX38ZO 2023-10-06T02:48:27.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.3Of2EX38ZO 2023-10-06T02:48:27.399 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.3Of2EX38ZO is 6KB 2023-10-06T02:48:27.446 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.3Of2EX38ZO 2023-10-06T02:48:27.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T02:48:27.527 INFO:teuthology.orchestra.run.smithi136.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T02:48:27.527 INFO:teuthology.orchestra.run.smithi136.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T02:48:27.527 INFO:teuthology.orchestra.run.smithi136.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T02:48:27.527 INFO:teuthology.orchestra.run.smithi136.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T02:48:27.528 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T02:48:27.701 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2023-10-06T02:48:57.154 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T02:48:57.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T02:48:57.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:48:57.704 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T02:48:57.705 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:48:57.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:49:15.646 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-10-06T02:49:16.154 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-10-06T02:49:24.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T02:49:24.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:49:25.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:49:25.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:49:27.710 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2023-10-06T02:49:28.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-10-06T02:49:39.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T02:49:39.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:49:40.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:49:40.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:49:58.650 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2023-10-06T02:50:11.639 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-10-06T02:50:11.931 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T02:50:11.931 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T02:50:11.931 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2023-10-06T02:50:11.945 INFO:teuthology.orchestra.run.smithi092.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T02:50:11.945 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:50:11.945 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T02:50:11.946 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T02:50:12.947 DEBUG:teuthology.parallel:result is None 2023-10-06T02:50:13.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:50:13.652 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:50:13.967 DEBUG:teuthology.orchestra.run.smithi136:> true 2023-10-06T02:50:14.255 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:50:14.255 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T02:50:14.255 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2023-10-06T02:50:14.269 INFO:teuthology.orchestra.run.smithi136.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T02:50:14.269 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T02:50:14.269 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T02:50:14.269 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T02:50:15.271 DEBUG:teuthology.parallel:result is None 2023-10-06T02:50:15.554 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T02:50:15.561 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T02:50:15.766 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T02:50:15.822 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T02:50:16.086 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T02:50:16.093 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T02:50:16.143 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T02:50:16.143 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T02:50:16.181 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T02:50:16.507 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T02:50:16.514 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T02:50:16.514 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T02:50:16.516 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T02:50:17.109 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:50:17.204 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:50:17.230 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:50:17.430 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T02:50:17.453 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T02:50:17.461 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T02:50:17.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T02:50:17.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T02:50:17.873 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T02:50:17.883 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T02:50:17.883 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T02:50:17.885 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T02:50:17.916 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:50:17.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:50:18.020 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:50:18.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:50:18.141 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T02:50:18.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T02:50:18.205 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:50:18.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T02:50:18.291 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:50:18.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T02:50:18.428 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T02:50:18.428 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T02:50:18.493 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2023-10-06T02:50:18.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2023-10-06T02:50:18.551 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T02:50:18.788 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T02:50:18.931 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T02:50:18.938 INFO:teuthology.task.internal:Starting timer... 2023-10-06T02:50:18.939 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T02:50:18.995 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T02:50:19.052 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T02:50:19.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2023-10-06T02:50:19.107 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: [ OK ] 2023-10-06T02:50:19.109 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2023-10-06T02:50:19.178 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: [ OK ] 2023-10-06T02:50:19.179 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T02:50:19.179 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2023-10-06T02:50:19.205 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2023-10-06T02:50:19.205 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2023-10-06T02:50:19.227 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2023-10-06T02:50:19.227 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2023-10-06T02:50:19.228 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T02:50:19.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:50:19.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T02:50:19.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T02:50:19.294 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T02:50:19.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2023-10-06T02:50:19.333 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2023-10-06T02:50:19.365 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T02:50:19.377 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T02:50:19.378 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T02:50:19.397 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T02:50:19.398 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T02:58:34.753 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2023-10-06T02:58:34.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T02:58:34.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:58:34.822 DEBUG:teuthology.orchestra.run.smithi092:> true 2023-10-06T02:58:34.885 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2023-10-06T02:58:34.885 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:58:34.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T02:58:34.954 DEBUG:teuthology.orchestra.run.smithi136:> true 2023-10-06T02:58:35.020 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2023-10-06T02:58:35.020 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T02:58:35.031 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T02:58:35.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T02:58:35.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:58:35.034 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T02:58:35.034 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T02:58:35.061 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T02:58:35.075 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T02:58:35.089 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T02:58:35.105 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T02:58:35.106 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2023-10-06T02:58:35.116 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2023-10-06T02:58:35.130 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T02:58:35.141 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2023-10-06T02:58:35.143 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T02:58:35.152 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2023-10-06T02:58:35.166 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T02:58:35.179 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T02:58:35.212 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-10-06T02:58:35.215 INFO:teuthology.orchestra.run.smithi092.stdout:210 Number of sources = 4 2023-10-06T02:58:35.215 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T02:58:35.215 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-10-06T02:58:35.215 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.215 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.216 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.216 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.248 INFO:teuthology.orchestra.run.smithi136.stderr:bash: ntpq: command not found 2023-10-06T02:58:35.251 INFO:teuthology.orchestra.run.smithi136.stdout:210 Number of sources = 4 2023-10-06T02:58:35.251 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T02:58:35.251 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2023-10-06T02:58:35.252 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.252 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.252 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.252 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T02:58:35.252 INFO:teuthology.run_tasks:Running task install... 2023-10-06T02:58:35.261 DEBUG:teuthology.task.install:project ceph 2023-10-06T02:58:35.261 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}} 2023-10-06T02:58:35.261 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'} 2023-10-06T02:58:35.261 INFO:teuthology.task.install:Using flavor: default 2023-10-06T02:58:35.267 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T02:58:35.267 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T02:58:35.267 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T02:58:35.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:58:35.270 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T02:58:35.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T02:58:35.414 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T02:58:35.414 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T02:58:35.482 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T02:58:35.482 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T02:58:35.533 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T02:58:35.534 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T02:58:35.534 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T02:58:35.559 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T02:58:35.559 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T02:58:35.625 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T02:58:35.649 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T02:58:35.649 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T02:58:35.649 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T02:58:35.674 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T02:58:35.674 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T02:58:35.699 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T02:58:35.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T02:58:35.762 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2023-10-06T02:58:35.764 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2023-10-06T02:58:35.813 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T02:58:35.877 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2023-10-06T02:58:35.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2023-10-06T02:58:36.154 INFO:teuthology.orchestra.run.smithi136.stdout:Updating Subscription Management repositories. 2023-10-06T02:58:36.271 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T02:58:36.490 INFO:teuthology.orchestra.run.smithi136.stdout:44 files removed 2023-10-06T02:58:36.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T02:58:36.615 INFO:teuthology.orchestra.run.smithi092.stdout:44 files removed 2023-10-06T02:58:36.667 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T02:58:36.889 INFO:teuthology.orchestra.run.smithi136.stdout:Updating Subscription Management repositories. 2023-10-06T02:58:37.015 INFO:teuthology.orchestra.run.smithi092.stdout:Updating Subscription Management repositories. 2023-10-06T02:58:37.751 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 249 kB/s | 78 kB 00:00 2023-10-06T02:58:37.761 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 352 kB/s | 78 kB 00:00 2023-10-06T02:58:38.042 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-06T02:58:38.098 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-06T02:58:38.259 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 7.8 kB/s | 1.4 kB 00:00 2023-10-06T02:58:38.398 INFO:teuthology.orchestra.run.smithi092.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-10-06T02:58:38.434 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 4.7 kB/s | 1.4 kB 00:00 2023-10-06T02:58:38.548 INFO:teuthology.orchestra.run.smithi136.stdout:Copr repo for python3-asyncssh owned by ceph 41 kB/s | 3.5 kB 00:00 2023-10-06T02:58:39.618 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-10-06T02:58:40.057 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 16 MB 00:01 2023-10-06T02:58:42.872 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-06T02:58:43.265 INFO:teuthology.orchestra.run.smithi136.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 122 MB/s | 38 MB 00:00 2023-10-06T02:58:43.411 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 2.9 MB/s | 24 kB 00:00 2023-10-06T02:58:43.790 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 129 MB/s | 38 MB 00:00 2023-10-06T02:58:48.761 INFO:teuthology.orchestra.run.smithi136.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 116 MB/s | 33 MB 00:00 2023-10-06T02:58:49.284 INFO:teuthology.orchestra.run.smithi092.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 123 MB/s | 33 MB 00:00 2023-10-06T02:58:56.673 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:58:56.674 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:58:56.685 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T02:58:56.685 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T02:58:56.686 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T02:58:56.687 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T02:58:56.689 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T02:58:56.691 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T02:58:56.692 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T02:58:57.178 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2023-10-06T02:58:57.189 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:58:57.189 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T02:58:57.190 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================================================== 2023-10-06T02:58:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2023-10-06T02:58:57.190 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================================================== 2023-10-06T02:58:57.190 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2023-10-06T02:58:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T02:58:57.190 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T02:58:57.191 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T02:58:57.192 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2023-10-06T02:58:57.193 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T02:58:57.194 INFO:teuthology.orchestra.run.smithi136.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T02:58:57.195 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T02:58:57.196 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T02:58:57.197 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T02:58:57.198 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T02:58:57.199 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T02:58:57.200 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================================================== 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout:Install 118 Packages 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 2 Packages 2023-10-06T02:58:57.201 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T02:58:57.202 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 201 M 2023-10-06T02:58:57.202 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2023-10-06T02:58:57.202 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T02:58:57.202 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T02:58:57.203 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T02:58:57.203 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T02:58:57.203 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T02:58:57.205 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T02:58:57.206 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T02:58:57.418 INFO:teuthology.orchestra.run.smithi136.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-10-06T02:58:57.694 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2023-10-06T02:58:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-10-06T02:58:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2023-10-06T02:58:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-10-06T02:58:57.705 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2023-10-06T02:58:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T02:58:57.705 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T02:58:57.706 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T02:58:57.707 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2023-10-06T02:58:57.708 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T02:58:57.709 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T02:58:57.710 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T02:58:57.711 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T02:58:57.712 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T02:58:57.713 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T02:58:57.714 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T02:58:57.715 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout:====================================================================================================================== 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout:Install 118 Packages 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 2 Packages 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 201 M 2023-10-06T02:58:57.716 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2023-10-06T02:58:57.769 INFO:teuthology.orchestra.run.smithi136.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.4 MB/s | 876 kB 00:00 2023-10-06T02:58:57.812 INFO:teuthology.orchestra.run.smithi136.stdout:(3/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 9.5 MB/s | 5.2 MB 00:00 2023-10-06T02:58:57.826 INFO:teuthology.orchestra.run.smithi136.stdout:(4/120): ceph-immutable-object-cache-17.2.6-129 2.8 MB/s | 162 kB 00:00 2023-10-06T02:58:58.023 INFO:teuthology.orchestra.run.smithi136.stdout:(5/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 11 MB/s | 2.3 MB 00:00 2023-10-06T02:58:58.033 INFO:teuthology.orchestra.run.smithi092.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-10-06T02:58:58.287 INFO:teuthology.orchestra.run.smithi136.stdout:(6/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 3.4 MB/s | 1.6 MB 00:00 2023-10-06T02:58:58.400 INFO:teuthology.orchestra.run.smithi136.stdout:(7/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 10 MB/s | 3.9 MB 00:00 2023-10-06T02:58:59.096 INFO:teuthology.orchestra.run.smithi092.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 825 kB/s | 876 kB 00:01 2023-10-06T02:58:59.279 INFO:teuthology.orchestra.run.smithi092.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 882 kB/s | 162 kB 00:00 2023-10-06T02:58:59.481 INFO:teuthology.orchestra.run.smithi136.stdout:(8/120): ceph-common-17.2.6-1296.g736ac5ba.el8. 11 MB/s | 23 MB 00:02 2023-10-06T02:58:59.551 INFO:teuthology.orchestra.run.smithi136.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 354 kB/s | 24 kB 00:00 2023-10-06T02:59:00.317 INFO:teuthology.orchestra.run.smithi136.stdout:(10/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 8.8 MB/s | 17 MB 00:01 2023-10-06T02:59:00.377 INFO:teuthology.orchestra.run.smithi136.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 4.2 MB/s | 255 kB 00:00 2023-10-06T02:59:00.408 INFO:teuthology.orchestra.run.smithi092.stdout:(4/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-10-06T02:59:00.432 INFO:teuthology.orchestra.run.smithi136.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 458 kB/s | 25 kB 00:00 2023-10-06T02:59:00.513 INFO:teuthology.orchestra.run.smithi092.stdout:(5/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 1.9 MB/s | 5.2 MB 00:02 2023-10-06T02:59:00.543 INFO:teuthology.orchestra.run.smithi136.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 7.0 MB/s | 797 kB 00:00 2023-10-06T02:59:00.610 INFO:teuthology.orchestra.run.smithi136.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 2.6 MB/s | 174 kB 00:00 2023-10-06T02:59:00.675 INFO:teuthology.orchestra.run.smithi136.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 2.0 MB/s | 130 kB 00:00 2023-10-06T02:59:00.755 INFO:teuthology.orchestra.run.smithi136.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 6.2 MB/s | 507 kB 00:00 2023-10-06T02:59:01.054 INFO:teuthology.orchestra.run.smithi092.stdout:(6/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 2.4 MB/s | 1.6 MB 00:00 2023-10-06T02:59:01.696 INFO:teuthology.orchestra.run.smithi136.stdout:(17/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x8 4.9 MB/s | 17 MB 00:03 2023-10-06T02:59:01.781 INFO:teuthology.orchestra.run.smithi136.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 530 kB/s | 45 kB 00:00 2023-10-06T02:59:01.874 INFO:teuthology.orchestra.run.smithi136.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 1.2 MB/s | 115 kB 00:00 2023-10-06T02:59:01.962 INFO:teuthology.orchestra.run.smithi136.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 2.1 MB/s | 190 kB 00:00 2023-10-06T02:59:02.050 INFO:teuthology.orchestra.run.smithi136.stdout:(21/120): python3-rados-17.2.6-1296.g736ac5ba.e 4.1 MB/s | 371 kB 00:00 2023-10-06T02:59:02.144 INFO:teuthology.orchestra.run.smithi136.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 3.7 MB/s | 358 kB 00:00 2023-10-06T02:59:02.170 INFO:teuthology.orchestra.run.smithi136.stdout:(23/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 3.9 MB/s | 5.6 MB 00:01 2023-10-06T02:59:02.231 INFO:teuthology.orchestra.run.smithi136.stdout:(24/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 1.2 MB/s | 107 kB 00:00 2023-10-06T02:59:02.247 INFO:teuthology.orchestra.run.smithi136.stdout:(25/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.2 MB/s | 90 kB 00:00 2023-10-06T02:59:02.590 INFO:teuthology.orchestra.run.smithi092.stdout:(7/120): ceph-common-17.2.6-1296.g736ac5ba.el8. 4.9 MB/s | 23 MB 00:04 2023-10-06T02:59:02.602 INFO:teuthology.orchestra.run.smithi092.stdout:(8/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 1.9 MB/s | 3.9 MB 00:02 2023-10-06T02:59:02.696 INFO:teuthology.orchestra.run.smithi092.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 259 kB/s | 24 kB 00:00 2023-10-06T02:59:02.770 INFO:teuthology.orchestra.run.smithi136.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 330 kB/s | 173 kB 00:00 2023-10-06T02:59:02.850 INFO:teuthology.orchestra.run.smithi136.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 290 kB/s | 23 kB 00:00 2023-10-06T02:59:02.928 INFO:teuthology.orchestra.run.smithi136.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 1.5 MB/s | 121 kB 00:00 2023-10-06T02:59:03.177 INFO:teuthology.orchestra.run.smithi136.stdout:(29/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 3.7 MB/s | 3.5 MB 00:00 2023-10-06T02:59:03.630 INFO:teuthology.orchestra.run.smithi136.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 2.4 MB/s | 1.7 MB 00:00 2023-10-06T02:59:03.644 INFO:teuthology.orchestra.run.smithi092.stdout:(10/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x8 6.5 MB/s | 17 MB 00:02 2023-10-06T02:59:03.705 INFO:teuthology.orchestra.run.smithi136.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 3.1 MB/s | 238 kB 00:00 2023-10-06T02:59:03.712 INFO:teuthology.orchestra.run.smithi092.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 3.7 MB/s | 255 kB 00:00 2023-10-06T02:59:03.749 INFO:teuthology.orchestra.run.smithi136.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 1.1 MB/s | 47 kB 00:00 2023-10-06T02:59:03.769 INFO:teuthology.orchestra.run.smithi092.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 433 kB/s | 25 kB 00:00 2023-10-06T02:59:03.786 INFO:teuthology.orchestra.run.smithi136.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 392 kB/s | 14 kB 00:00 2023-10-06T02:59:03.894 INFO:teuthology.orchestra.run.smithi136.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 2.4 MB/s | 267 kB 00:00 2023-10-06T02:59:03.899 INFO:teuthology.orchestra.run.smithi092.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 6.0 MB/s | 797 kB 00:00 2023-10-06T02:59:03.938 INFO:teuthology.orchestra.run.smithi136.stdout:(35/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 1.8 MB/s | 80 kB 00:00 2023-10-06T02:59:03.962 INFO:teuthology.orchestra.run.smithi092.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 2.7 MB/s | 174 kB 00:00 2023-10-06T02:59:03.982 INFO:teuthology.orchestra.run.smithi136.stdout:(36/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 416 kB/s | 18 kB 00:00 2023-10-06T02:59:04.018 INFO:teuthology.orchestra.run.smithi092.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 2.3 MB/s | 130 kB 00:00 2023-10-06T02:59:04.087 INFO:teuthology.orchestra.run.smithi092.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 7.1 MB/s | 507 kB 00:00 2023-10-06T02:59:04.112 INFO:teuthology.orchestra.run.smithi136.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 MB/s | 306 kB 00:00 2023-10-06T02:59:04.134 INFO:teuthology.orchestra.run.smithi136.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 2.7 MB/s | 59 kB 00:00 2023-10-06T02:59:04.156 INFO:teuthology.orchestra.run.smithi136.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 3.3 MB/s | 75 kB 00:00 2023-10-06T02:59:04.201 INFO:teuthology.orchestra.run.smithi136.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 10 MB/s | 476 kB 00:00 2023-10-06T02:59:04.220 INFO:teuthology.orchestra.run.smithi136.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 2.2 MB/s | 44 kB 00:00 2023-10-06T02:59:04.244 INFO:teuthology.orchestra.run.smithi136.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 7.8 MB/s | 185 kB 00:00 2023-10-06T02:59:04.263 INFO:teuthology.orchestra.run.smithi136.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.6 MB/s | 33 kB 00:00 2023-10-06T02:59:04.283 INFO:teuthology.orchestra.run.smithi136.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 833 kB/s | 16 kB 00:00 2023-10-06T02:59:04.306 INFO:teuthology.orchestra.run.smithi136.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 7.6 MB/s | 173 kB 00:00 2023-10-06T02:59:04.333 INFO:teuthology.orchestra.run.smithi136.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 14 MB/s | 384 kB 00:00 2023-10-06T02:59:04.353 INFO:teuthology.orchestra.run.smithi136.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 4.0 MB/s | 81 kB 00:00 2023-10-06T02:59:04.372 INFO:teuthology.orchestra.run.smithi136.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 579 kB/s | 11 kB 00:00 2023-10-06T02:59:04.391 INFO:teuthology.orchestra.run.smithi136.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 913 kB/s | 18 kB 00:00 2023-10-06T02:59:04.422 INFO:teuthology.orchestra.run.smithi092.stdout:(17/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 9.2 MB/s | 17 MB 00:01 2023-10-06T02:59:04.436 INFO:teuthology.orchestra.run.smithi136.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 27 MB/s | 1.2 MB 00:00 2023-10-06T02:59:04.455 INFO:teuthology.orchestra.run.smithi136.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 2.4 MB/s | 49 kB 00:00 2023-10-06T02:59:04.475 INFO:teuthology.orchestra.run.smithi136.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 2.9 MB/s | 59 kB 00:00 2023-10-06T02:59:04.475 INFO:teuthology.orchestra.run.smithi092.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 835 kB/s | 45 kB 00:00 2023-10-06T02:59:04.494 INFO:teuthology.orchestra.run.smithi136.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 3.0 MB/s | 60 kB 00:00 2023-10-06T02:59:04.517 INFO:teuthology.orchestra.run.smithi136.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 12 MB/s | 283 kB 00:00 2023-10-06T02:59:04.534 INFO:teuthology.orchestra.run.smithi092.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 1.9 MB/s | 115 kB 00:00 2023-10-06T02:59:04.536 INFO:teuthology.orchestra.run.smithi136.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 810 kB/s | 16 kB 00:00 2023-10-06T02:59:04.556 INFO:teuthology.orchestra.run.smithi136.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.7 MB/s | 34 kB 00:00 2023-10-06T02:59:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 9.3 MB/s | 196 kB 00:00 2023-10-06T02:59:04.593 INFO:teuthology.orchestra.run.smithi092.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 3.1 MB/s | 190 kB 00:00 2023-10-06T02:59:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 3.2 MB/s | 64 kB 00:00 2023-10-06T02:59:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 980 kB/s | 19 kB 00:00 2023-10-06T02:59:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:(21/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 10 MB/s | 5.6 MB 00:00 2023-10-06T02:59:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 1.2 MB/s | 24 kB 00:00 2023-10-06T02:59:04.642 INFO:teuthology.orchestra.run.smithi092.stdout:(22/120): python3-rados-17.2.6-1296.g736ac5ba.e 7.4 MB/s | 371 kB 00:00 2023-10-06T02:59:04.654 INFO:teuthology.orchestra.run.smithi136.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 1.5 MB/s | 29 kB 00:00 2023-10-06T02:59:04.674 INFO:teuthology.orchestra.run.smithi092.stdout:(23/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 6.2 MB/s | 358 kB 00:00 2023-10-06T02:59:04.674 INFO:teuthology.orchestra.run.smithi136.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.4 MB/s | 27 kB 00:00 2023-10-06T02:59:04.691 INFO:teuthology.orchestra.run.smithi092.stdout:(24/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 2.2 MB/s | 107 kB 00:00 2023-10-06T02:59:04.695 INFO:teuthology.orchestra.run.smithi136.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 12 MB/s | 255 kB 00:00 2023-10-06T02:59:04.715 INFO:teuthology.orchestra.run.smithi092.stdout:(25/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.2 MB/s | 90 kB 00:00 2023-10-06T02:59:04.717 INFO:teuthology.orchestra.run.smithi136.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 12 MB/s | 251 kB 00:00 2023-10-06T02:59:04.736 INFO:teuthology.orchestra.run.smithi136.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 3.1 MB/s | 61 kB 00:00 2023-10-06T02:59:04.756 INFO:teuthology.orchestra.run.smithi136.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 4.3 MB/s | 85 kB 00:00 2023-10-06T02:59:04.773 INFO:teuthology.orchestra.run.smithi092.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 2.9 MB/s | 173 kB 00:00 2023-10-06T02:59:04.778 INFO:teuthology.orchestra.run.smithi136.stdout:(67/120): ceph-mgr-diskprediction-local-17.2.6- 4.6 MB/s | 7.4 MB 00:01 2023-10-06T02:59:04.778 INFO:teuthology.orchestra.run.smithi136.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.0 MB/s | 23 kB 00:00 2023-10-06T02:59:04.782 INFO:teuthology.orchestra.run.smithi136.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-10-06T02:59:04.814 INFO:teuthology.orchestra.run.smithi092.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 574 kB/s | 23 kB 00:00 2023-10-06T02:59:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 507 kB/s | 19 kB 00:00 2023-10-06T02:59:04.836 INFO:teuthology.orchestra.run.smithi136.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.5 MB/s | 43 kB 00:00 2023-10-06T02:59:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.8 MB/s | 155 kB 00:00 2023-10-06T02:59:04.863 INFO:teuthology.orchestra.run.smithi092.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 2.4 MB/s | 121 kB 00:00 2023-10-06T02:59:04.871 INFO:teuthology.orchestra.run.smithi136.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 12 MB/s | 220 kB 00:00 2023-10-06T02:59:04.887 INFO:teuthology.orchestra.run.smithi136.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.1 MB/s | 48 kB 00:00 2023-10-06T02:59:04.905 INFO:teuthology.orchestra.run.smithi136.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 13 MB/s | 229 kB 00:00 2023-10-06T02:59:04.920 INFO:teuthology.orchestra.run.smithi136.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 2.7 MB/s | 43 kB 00:00 2023-10-06T02:59:04.964 INFO:teuthology.orchestra.run.smithi136.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 54 MB/s | 2.3 MB 00:00 2023-10-06T02:59:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:(78/120): thrift-0.13.0-2.el8.x86_64.rpm 9.1 MB/s | 1.7 MB 00:00 2023-10-06T02:59:04.971 INFO:teuthology.orchestra.run.smithi092.stdout:(29/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 12 MB/s | 3.5 MB 00:00 2023-10-06T02:59:04.980 INFO:teuthology.orchestra.run.smithi136.stdout:(79/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.1 MB/s | 18 kB 00:00 2023-10-06T02:59:04.993 INFO:teuthology.orchestra.run.smithi136.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 6.3 MB/s | 83 kB 00:00 2023-10-06T02:59:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:(81/120): pcre2-devel-10.32-2.el8.x86_64.rpm 15 MB/s | 605 kB 00:00 2023-10-06T02:59:05.010 INFO:teuthology.orchestra.run.smithi136.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.3 MB/s | 59 kB 00:00 2023-10-06T02:59:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:(83/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.9 MB/s | 38 kB 00:00 2023-10-06T02:59:05.027 INFO:teuthology.orchestra.run.smithi136.stdout:(84/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.4 MB/s | 169 kB 00:00 2023-10-06T02:59:05.041 INFO:teuthology.orchestra.run.smithi136.stdout:(85/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.2 MB/s | 58 kB 00:00 2023-10-06T02:59:05.041 INFO:teuthology.orchestra.run.smithi136.stdout:(86/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.7 MB/s | 86 kB 00:00 2023-10-06T02:59:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-10-06T02:59:05.062 INFO:teuthology.orchestra.run.smithi092.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 8.5 MB/s | 1.7 MB 00:00 2023-10-06T02:59:05.064 INFO:teuthology.orchestra.run.smithi136.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 24 MB/s | 559 kB 00:00 2023-10-06T02:59:05.077 INFO:teuthology.orchestra.run.smithi136.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 29 MB/s | 639 kB 00:00 2023-10-06T02:59:05.081 INFO:teuthology.orchestra.run.smithi136.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.2 MB/s | 70 kB 00:00 2023-10-06T02:59:05.095 INFO:teuthology.orchestra.run.smithi136.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 11 MB/s | 200 kB 00:00 2023-10-06T02:59:05.097 INFO:teuthology.orchestra.run.smithi136.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 5.7 MB/s | 97 kB 00:00 2023-10-06T02:59:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.2 MB/s | 44 kB 00:00 2023-10-06T02:59:05.115 INFO:teuthology.orchestra.run.smithi092.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 4.5 MB/s | 238 kB 00:00 2023-10-06T02:59:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-06T02:59:05.143 INFO:teuthology.orchestra.run.smithi136.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 9.0 MB/s | 161 kB 00:00 2023-10-06T02:59:05.161 INFO:teuthology.orchestra.run.smithi136.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-06T02:59:05.164 INFO:teuthology.orchestra.run.smithi092.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 960 kB/s | 47 kB 00:00 2023-10-06T02:59:05.181 INFO:teuthology.orchestra.run.smithi136.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 23 MB/s | 457 kB 00:00 2023-10-06T02:59:05.204 INFO:teuthology.orchestra.run.smithi092.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 366 kB/s | 14 kB 00:00 2023-10-06T02:59:05.261 INFO:teuthology.orchestra.run.smithi136.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 60 MB/s | 4.8 MB 00:00 2023-10-06T02:59:05.263 INFO:teuthology.orchestra.run.smithi092.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 4.4 MB/s | 267 kB 00:00 2023-10-06T02:59:05.312 INFO:teuthology.orchestra.run.smithi092.stdout:(35/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 1.6 MB/s | 80 kB 00:00 2023-10-06T02:59:05.364 INFO:teuthology.orchestra.run.smithi092.stdout:(36/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 345 kB/s | 18 kB 00:00 2023-10-06T02:59:05.398 INFO:teuthology.orchestra.run.smithi136.stdout:(99/120): python3-scipy-1.0.0-20.module+el8.1.0 48 MB/s | 15 MB 00:00 2023-10-06T02:59:05.398 INFO:teuthology.orchestra.run.smithi136.stdout:(100/120): python3-markupsafe-0.23-19.el8.x86_6 285 kB/s | 39 kB 00:00 2023-10-06T02:59:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-06T02:59:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:(102/120): bison-3.0.4-10.el8.x86_64.rpm 27 MB/s | 688 kB 00:00 2023-10-06T02:59:05.432 INFO:teuthology.orchestra.run.smithi136.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 12 MB/s | 225 kB 00:00 2023-10-06T02:59:05.439 INFO:teuthology.orchestra.run.smithi136.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 7.0 MB/s | 126 kB 00:00 2023-10-06T02:59:05.458 INFO:teuthology.orchestra.run.smithi136.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.8 MB/s | 187 kB 00:00 2023-10-06T02:59:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 24 MB/s | 538 kB 00:00 2023-10-06T02:59:05.493 INFO:teuthology.orchestra.run.smithi136.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 62 MB/s | 3.7 MB 00:00 2023-10-06T02:59:05.499 INFO:teuthology.orchestra.run.smithi136.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 16 MB/s | 302 kB 00:00 2023-10-06T02:59:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.3 MB/s | 157 kB 00:00 2023-10-06T02:59:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 6.4 MB/s | 110 kB 00:00 2023-10-06T02:59:05.527 INFO:teuthology.orchestra.run.smithi136.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-06T02:59:05.533 INFO:teuthology.orchestra.run.smithi136.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.2 MB/s | 20 kB 00:00 2023-10-06T02:59:05.537 INFO:teuthology.orchestra.run.smithi092.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 13 MB/s | 7.4 MB 00:00 2023-10-06T02:59:05.550 INFO:teuthology.orchestra.run.smithi136.stdout:(113/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-10-06T02:59:05.551 INFO:teuthology.orchestra.run.smithi136.stdout:(114/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.0 MB/s | 54 kB 00:00 2023-10-06T02:59:05.629 INFO:teuthology.orchestra.run.smithi136.stdout:(115/120): openblas-0.3.12-1.el8.x86_64.rpm 59 MB/s | 4.6 MB 00:00 2023-10-06T02:59:05.642 INFO:teuthology.orchestra.run.smithi136.stdout:(116/120): openblas-threads-0.3.12-1.el8.x86_64 51 MB/s | 4.7 MB 00:00 2023-10-06T02:59:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 30 MB/s | 713 kB 00:00 2023-10-06T02:59:05.725 INFO:teuthology.orchestra.run.smithi092.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 852 kB/s | 306 kB 00:00 2023-10-06T02:59:05.784 INFO:teuthology.orchestra.run.smithi092.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.2 MB/s | 75 kB 00:00 2023-10-06T02:59:05.791 INFO:teuthology.orchestra.run.smithi092.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 232 kB/s | 59 kB 00:00 2023-10-06T02:59:05.859 INFO:teuthology.orchestra.run.smithi092.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 646 kB/s | 44 kB 00:00 2023-10-06T02:59:05.889 INFO:teuthology.orchestra.run.smithi092.stdout:(42/120): python3-asyncssh-2.7.0-2.el8.noarch.r 4.4 MB/s | 476 kB 00:00 2023-10-06T02:59:05.938 INFO:teuthology.orchestra.run.smithi092.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 678 kB/s | 33 kB 00:00 2023-10-06T02:59:05.986 INFO:teuthology.orchestra.run.smithi092.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 335 kB/s | 16 kB 00:00 2023-10-06T02:59:05.990 INFO:teuthology.orchestra.run.smithi092.stdout:(45/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-10-06T02:59:06.043 INFO:teuthology.orchestra.run.smithi092.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.0 MB/s | 173 kB 00:00 2023-10-06T02:59:06.094 INFO:teuthology.orchestra.run.smithi092.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.5 MB/s | 81 kB 00:00 2023-10-06T02:59:06.109 INFO:teuthology.orchestra.run.smithi092.stdout:(48/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.2 MB/s | 384 kB 00:00 2023-10-06T02:59:06.144 INFO:teuthology.orchestra.run.smithi092.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 227 kB/s | 11 kB 00:00 2023-10-06T02:59:06.158 INFO:teuthology.orchestra.run.smithi092.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 357 kB/s | 18 kB 00:00 2023-10-06T02:59:06.199 INFO:teuthology.orchestra.run.smithi136.stdout:(118/120): librados2-17.2.6-1296.g736ac5ba.el8. 6.2 MB/s | 3.5 MB 00:00 2023-10-06T02:59:06.210 INFO:teuthology.orchestra.run.smithi092.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 942 kB/s | 49 kB 00:00 2023-10-06T02:59:06.250 INFO:teuthology.orchestra.run.smithi092.stdout:(52/120): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-10-06T02:59:06.263 INFO:teuthology.orchestra.run.smithi092.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-10-06T02:59:06.290 INFO:teuthology.orchestra.run.smithi136.stdout:(119/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 5.6 MB/s | 3.6 MB 00:00 2023-10-06T02:59:06.299 INFO:teuthology.orchestra.run.smithi092.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-10-06T02:59:06.332 INFO:teuthology.orchestra.run.smithi092.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-10-06T02:59:06.348 INFO:teuthology.orchestra.run.smithi092.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 320 kB/s | 16 kB 00:00 2023-10-06T02:59:06.382 INFO:teuthology.orchestra.run.smithi092.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2023-10-06T02:59:06.403 INFO:teuthology.orchestra.run.smithi092.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 3.5 MB/s | 196 kB 00:00 2023-10-06T02:59:06.435 INFO:teuthology.orchestra.run.smithi092.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 1.2 MB/s | 64 kB 00:00 2023-10-06T02:59:06.455 INFO:teuthology.orchestra.run.smithi092.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 361 kB/s | 19 kB 00:00 2023-10-06T02:59:06.484 INFO:teuthology.orchestra.run.smithi092.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 486 kB/s | 24 kB 00:00 2023-10-06T02:59:06.504 INFO:teuthology.orchestra.run.smithi092.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 598 kB/s | 29 kB 00:00 2023-10-06T02:59:06.535 INFO:teuthology.orchestra.run.smithi092.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-10-06T02:59:06.558 INFO:teuthology.orchestra.run.smithi092.stdout:(64/120): python3-waitress-1.4.3-1.el8.noarch.r 4.7 MB/s | 255 kB 00:00 2023-10-06T02:59:06.601 INFO:teuthology.orchestra.run.smithi092.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-10-06T02:59:06.607 INFO:teuthology.orchestra.run.smithi092.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-10-06T02:59:06.655 INFO:teuthology.orchestra.run.smithi092.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.5 MB/s | 85 kB 00:00 2023-10-06T02:59:06.656 INFO:teuthology.orchestra.run.smithi092.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 471 kB/s | 23 kB 00:00 2023-10-06T02:59:06.660 INFO:teuthology.orchestra.run.smithi092.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-10-06T02:59:06.697 INFO:teuthology.orchestra.run.smithi092.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 505 kB/s | 19 kB 00:00 2023-10-06T02:59:06.710 INFO:teuthology.orchestra.run.smithi092.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.2 MB/s | 43 kB 00:00 2023-10-06T02:59:06.725 INFO:teuthology.orchestra.run.smithi092.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 11 MB/s | 155 kB 00:00 2023-10-06T02:59:06.740 INFO:teuthology.orchestra.run.smithi092.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 15 MB/s | 220 kB 00:00 2023-10-06T02:59:06.753 INFO:teuthology.orchestra.run.smithi092.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.6 MB/s | 48 kB 00:00 2023-10-06T02:59:06.767 INFO:teuthology.orchestra.run.smithi136.stdout:(120/120): ceph-test-17.2.6-1296.g736ac5ba.el8. 7.2 MB/s | 52 MB 00:07 2023-10-06T02:59:06.768 INFO:teuthology.orchestra.run.smithi092.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-10-06T02:59:06.772 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2023-10-06T02:59:06.773 INFO:teuthology.orchestra.run.smithi136.stdout:Total 21 MB/s | 201 MB 00:09 2023-10-06T02:59:06.783 INFO:teuthology.orchestra.run.smithi092.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 2.8 MB/s | 43 kB 00:00 2023-10-06T02:59:06.811 INFO:teuthology.orchestra.run.smithi092.stdout:(77/120): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-10-06T02:59:06.827 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2023-10-06T02:59:06.828 INFO:teuthology.orchestra.run.smithi092.stdout:(78/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 52 MB/s | 2.3 MB 00:00 2023-10-06T02:59:06.849 INFO:teuthology.orchestra.run.smithi092.stdout:(79/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 472 kB/s | 18 kB 00:00 2023-10-06T02:59:06.856 INFO:teuthology.orchestra.run.smithi092.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 27 MB/s | 605 kB 00:00 2023-10-06T02:59:06.865 INFO:teuthology.orchestra.run.smithi092.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 5.3 MB/s | 83 kB 00:00 2023-10-06T02:59:06.873 INFO:teuthology.orchestra.run.smithi092.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 3.4 MB/s | 59 kB 00:00 2023-10-06T02:59:06.880 INFO:teuthology.orchestra.run.smithi092.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-10-06T02:59:06.891 INFO:teuthology.orchestra.run.smithi092.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.1 MB/s | 38 kB 00:00 2023-10-06T02:59:06.895 INFO:teuthology.orchestra.run.smithi092.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.7 MB/s | 86 kB 00:00 2023-10-06T02:59:06.911 INFO:teuthology.orchestra.run.smithi092.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.9 MB/s | 58 kB 00:00 2023-10-06T02:59:06.918 INFO:teuthology.orchestra.run.smithi092.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.0 MB/s | 47 kB 00:00 2023-10-06T02:59:06.934 INFO:teuthology.orchestra.run.smithi092.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 23 MB/s | 559 kB 00:00 2023-10-06T02:59:06.943 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2023-10-06T02:59:06.943 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2023-10-06T02:59:06.946 INFO:teuthology.orchestra.run.smithi092.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 23 MB/s | 639 kB 00:00 2023-10-06T02:59:06.948 INFO:teuthology.orchestra.run.smithi092.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.8 MB/s | 70 kB 00:00 2023-10-06T02:59:06.965 INFO:teuthology.orchestra.run.smithi092.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 10 MB/s | 200 kB 00:00 2023-10-06T02:59:06.966 INFO:teuthology.orchestra.run.smithi092.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 5.4 MB/s | 97 kB 00:00 2023-10-06T02:59:06.983 INFO:teuthology.orchestra.run.smithi092.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 2.6 MB/s | 44 kB 00:00 2023-10-06T02:59:07.000 INFO:teuthology.orchestra.run.smithi092.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.6 MB/s | 27 kB 00:00 2023-10-06T02:59:07.016 INFO:teuthology.orchestra.run.smithi092.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 9.8 MB/s | 161 kB 00:00 2023-10-06T02:59:07.034 INFO:teuthology.orchestra.run.smithi092.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-10-06T02:59:07.055 INFO:teuthology.orchestra.run.smithi092.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 22 MB/s | 457 kB 00:00 2023-10-06T02:59:07.217 INFO:teuthology.orchestra.run.smithi092.stdout:(98/120): python3-scipy-1.0.0-20.module+el8.1.0 58 MB/s | 15 MB 00:00 2023-10-06T02:59:07.250 INFO:teuthology.orchestra.run.smithi092.stdout:(99/120): python3-babel-2.5.1-5.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-10-06T02:59:07.251 INFO:teuthology.orchestra.run.smithi092.stdout:(100/120): python3-markupsafe-0.23-19.el8.x86_6 2.1 MB/s | 39 kB 00:00 2023-10-06T02:59:07.269 INFO:teuthology.orchestra.run.smithi092.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 14 MB/s | 259 kB 00:00 2023-10-06T02:59:07.288 INFO:teuthology.orchestra.run.smithi092.stdout:(102/120): bison-3.0.4-10.el8.x86_64.rpm 27 MB/s | 688 kB 00:00 2023-10-06T02:59:07.288 INFO:teuthology.orchestra.run.smithi092.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 13 MB/s | 225 kB 00:00 2023-10-06T02:59:07.293 INFO:teuthology.orchestra.run.smithi092.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 7.1 MB/s | 126 kB 00:00 2023-10-06T02:59:07.311 INFO:teuthology.orchestra.run.smithi092.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 10 MB/s | 187 kB 00:00 2023-10-06T02:59:07.332 INFO:teuthology.orchestra.run.smithi092.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 26 MB/s | 538 kB 00:00 2023-10-06T02:59:07.347 INFO:teuthology.orchestra.run.smithi092.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 61 MB/s | 3.7 MB 00:00 2023-10-06T02:59:07.351 INFO:teuthology.orchestra.run.smithi092.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 16 MB/s | 302 kB 00:00 2023-10-06T02:59:07.363 INFO:teuthology.orchestra.run.smithi092.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 9.6 MB/s | 157 kB 00:00 2023-10-06T02:59:07.374 INFO:teuthology.orchestra.run.smithi092.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 4.7 MB/s | 110 kB 00:00 2023-10-06T02:59:07.390 INFO:teuthology.orchestra.run.smithi092.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 13 MB/s | 353 kB 00:00 2023-10-06T02:59:07.390 INFO:teuthology.orchestra.run.smithi092.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.2 MB/s | 20 kB 00:00 2023-10-06T02:59:07.406 INFO:teuthology.orchestra.run.smithi092.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.6 MB/s | 54 kB 00:00 2023-10-06T02:59:07.412 INFO:teuthology.orchestra.run.smithi092.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-10-06T02:59:07.478 INFO:teuthology.orchestra.run.smithi092.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 66 MB/s | 4.7 MB 00:00 2023-10-06T02:59:07.499 INFO:teuthology.orchestra.run.smithi092.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 53 MB/s | 4.6 MB 00:00 2023-10-06T02:59:07.507 INFO:teuthology.orchestra.run.smithi092.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 24 MB/s | 713 kB 00:00 2023-10-06T02:59:07.940 INFO:teuthology.orchestra.run.smithi092.stdout:(118/120): librados2-17.2.6-1296.g736ac5ba.el8. 7.9 MB/s | 3.5 MB 00:00 2023-10-06T02:59:07.961 INFO:teuthology.orchestra.run.smithi092.stdout:(119/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 7.9 MB/s | 3.6 MB 00:00 2023-10-06T02:59:08.624 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2023-10-06T02:59:08.628 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2023-10-06T02:59:13.512 INFO:teuthology.orchestra.run.smithi092.stdout:(120/120): ceph-test-17.2.6-1296.g736ac5ba.el8. 4.8 MB/s | 52 MB 00:10 2023-10-06T02:59:13.518 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2023-10-06T02:59:13.519 INFO:teuthology.orchestra.run.smithi092.stdout:Total 13 MB/s | 201 MB 00:15 2023-10-06T02:59:13.571 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2023-10-06T02:59:13.609 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2023-10-06T02:59:13.685 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2023-10-06T02:59:13.685 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2023-10-06T02:59:13.859 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T02:59:14.084 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T02:59:14.223 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T02:59:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T02:59:14.842 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T02:59:15.076 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T02:59:15.434 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2023-10-06T02:59:15.438 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2023-10-06T02:59:15.478 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T02:59:16.150 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:59:16.284 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:59:16.467 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:59:16.571 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:59:16.755 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:59:17.085 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T02:59:17.769 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T02:59:17.925 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:59:18.035 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:59:18.312 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T02:59:18.507 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T02:59:18.725 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:59:18.819 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:59:19.030 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:59:19.105 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:59:19.321 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T02:59:19.563 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T02:59:19.756 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T02:59:19.935 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T02:59:20.083 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T02:59:20.219 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2023-10-06T02:59:20.254 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T02:59:20.408 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T02:59:20.477 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T02:59:20.678 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T02:59:20.801 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T02:59:20.988 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T02:59:21.020 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T02:59:21.433 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T02:59:21.641 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T02:59:21.661 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T02:59:22.050 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T02:59:22.185 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T02:59:22.294 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:59:22.391 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T02:59:22.407 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T02:59:22.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:59:22.709 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T02:59:22.924 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T02:59:23.102 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T02:59:23.273 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T02:59:23.369 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T02:59:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T02:59:23.910 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T02:59:23.979 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T02:59:24.138 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T02:59:24.169 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:59:24.284 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T02:59:24.293 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T02:59:24.515 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T02:59:24.565 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T02:59:24.700 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T02:59:24.821 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T02:59:24.910 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:59:25.016 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T02:59:25.114 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T02:59:25.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:59:25.287 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T02:59:25.365 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T02:59:25.498 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T02:59:25.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T02:59:26.067 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T02:59:26.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T02:59:26.410 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T02:59:26.552 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T02:59:26.701 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T02:59:27.281 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T02:59:27.426 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T02:59:27.426 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T02:59:27.787 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T02:59:27.988 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T02:59:28.201 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T02:59:28.798 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T02:59:28.922 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T02:59:29.160 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T02:59:29.222 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T02:59:29.418 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T02:59:29.504 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T02:59:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T02:59:29.764 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T02:59:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T02:59:29.966 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T02:59:30.094 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T02:59:30.137 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T02:59:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T02:59:30.419 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T02:59:30.589 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T02:59:30.658 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T02:59:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T02:59:30.943 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T02:59:31.054 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T02:59:31.209 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T02:59:31.235 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T02:59:31.442 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T02:59:31.589 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T02:59:31.702 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T02:59:31.855 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T02:59:32.067 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T02:59:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T02:59:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T02:59:33.016 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T02:59:33.199 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T02:59:33.307 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T02:59:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T02:59:33.547 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T02:59:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T02:59:33.725 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T02:59:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T02:59:34.300 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T02:59:34.572 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T02:59:34.766 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T02:59:34.857 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T02:59:35.063 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T02:59:35.123 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T02:59:35.242 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T02:59:35.412 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T02:59:35.509 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T02:59:35.681 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T02:59:35.771 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T02:59:35.895 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T02:59:35.994 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T02:59:36.118 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T02:59:36.207 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T02:59:36.337 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T02:59:36.365 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T02:59:36.597 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T02:59:36.645 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T02:59:36.833 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T02:59:36.857 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T02:59:37.048 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T02:59:37.049 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T02:59:37.280 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T02:59:37.282 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T02:59:37.460 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T02:59:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T02:59:37.688 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T02:59:37.691 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T02:59:37.770 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T02:59:37.901 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T02:59:37.967 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T02:59:38.175 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T02:59:38.194 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T02:59:38.396 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T02:59:38.510 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T02:59:38.579 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T02:59:38.742 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T02:59:38.762 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T02:59:38.935 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T02:59:38.979 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T02:59:39.096 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T02:59:39.257 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T02:59:39.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T02:59:39.470 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T02:59:39.785 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T02:59:40.098 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T02:59:40.397 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T02:59:40.428 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T02:59:40.709 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T02:59:40.784 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T02:59:40.882 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T02:59:41.030 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T02:59:41.070 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T02:59:41.353 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T02:59:41.381 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T02:59:41.574 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T02:59:41.667 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T02:59:41.772 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T02:59:42.009 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T02:59:42.010 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T02:59:42.218 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T02:59:42.253 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T02:59:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T02:59:42.451 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T02:59:42.670 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T02:59:42.703 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T02:59:42.969 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T02:59:43.185 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T02:59:43.388 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T02:59:43.667 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T02:59:43.921 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T02:59:44.138 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T02:59:44.453 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T02:59:44.731 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T02:59:44.956 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T02:59:45.236 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T02:59:46.428 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T02:59:46.745 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T02:59:46.990 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T02:59:47.376 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T02:59:47.675 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T02:59:47.773 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:59:48.044 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:59:48.046 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T02:59:48.303 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T02:59:48.530 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T02:59:48.772 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T02:59:48.822 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:59:49.138 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:59:49.251 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:59:49.401 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T02:59:53.965 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:59:54.205 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:59:54.799 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T02:59:55.140 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:59:55.229 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T02:59:55.403 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:00:07.466 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:00:07.466 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2023-10-06T03:00:07.466 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /dev 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /run 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /dev 2023-10-06T03:00:07.467 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:07.856 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:00:09.339 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:00:09.492 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:00:09.714 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:00:09.881 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:00:10.288 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:00:10.472 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:00:10.488 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:00:13.033 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:00:13.033 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /run 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /dev 2023-10-06T03:00:13.034 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:13.395 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:00:13.484 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:00:13.578 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:00:13.794 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T03:00:14.018 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T03:00:14.234 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:00:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:00:14.690 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:00:14.973 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:00:15.177 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:00:15.399 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:00:15.579 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:00:15.661 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:00:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:00:15.936 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:00:15.986 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:00:16.158 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:00:16.175 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:00:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T03:00:16.457 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:00:16.494 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:00:16.704 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:00:16.715 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:00:18.359 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T03:00:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:00:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:00:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:00:18.905 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:00:19.070 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:00:19.157 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:00:19.338 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:00:19.418 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:00:19.538 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T03:00:19.612 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:00:19.740 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T03:00:19.974 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:00:20.194 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:00:20.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:00:21.412 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:00:21.648 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:00:21.749 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:00:22.131 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T03:00:22.334 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:00:22.371 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:00:22.564 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:00:22.575 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:00:24.201 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T03:00:24.467 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:00:24.529 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:00:24.543 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:00:24.689 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:00:24.733 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:00:24.906 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:00:25.254 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:00:25.431 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:00:25.480 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T03:00:25.879 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T03:00:26.084 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 110/122 2023-10-06T03:00:26.319 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T03:00:26.493 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:00:26.505 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:00:26.689 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T03:00:26.840 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T03:00:27.010 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T03:00:27.193 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T03:00:27.460 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T03:00:27.676 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:00:27.737 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:00:27.960 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:00:27.969 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:00:28.120 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T03:00:28.121 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:00:28.312 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:00:28.313 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:00:29.197 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:00:29.197 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T03:00:29.197 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T03:00:29.197 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T03:00:29.197 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T03:00:29.198 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T03:00:29.199 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T03:00:29.200 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T03:00:29.209 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T03:00:29.210 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T03:00:29.211 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T03:00:29.212 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T03:00:29.213 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T03:00:29.214 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T03:00:29.215 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T03:00:29.216 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T03:00:29.217 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T03:00:29.218 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T03:00:29.218 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T03:00:29.218 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T03:00:29.218 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T03:00:29.218 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T03:00:30.343 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:00:31.306 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T03:00:31.422 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:00:31.423 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:01:43 ago on Fri 06 Oct 2023 02:58:48 AM UTC. 2023-10-06T03:00:31.737 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T03:00:31.949 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 110/122 2023-10-06T03:00:32.207 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T03:00:32.412 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:00:32.423 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:00:32.616 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T03:00:32.817 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T03:00:32.978 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T03:00:33.194 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T03:00:33.456 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T03:00:33.663 INFO:teuthology.orchestra.run.smithi136.stdout:Installed products updated. 2023-10-06T03:00:33.670 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:00:33.722 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:00:33.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:33.829 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.830 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.831 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:33.832 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: jq-1.5-12.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.833 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T03:00:33.834 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T03:00:33.835 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.836 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T03:00:33.837 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T03:00:33.838 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T03:00:33.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T03:00:33.840 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T03:00:33.841 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T03:00:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T03:00:33.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:33.842 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2023-10-06T03:00:33.928 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:00:33.937 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:00:34.072 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T03:00:34.072 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:00:34.145 DEBUG:teuthology.parallel:result is None 2023-10-06T03:00:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:00:34.298 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:00:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:00:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T03:00:35.185 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:00:35.186 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T03:00:35.187 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T03:00:35.188 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T03:00:35.189 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T03:00:35.190 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T03:00:35.191 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T03:00:35.192 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T03:00:35.193 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T03:00:35.194 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T03:00:35.195 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T03:00:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T03:00:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T03:00:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T03:00:35.196 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T03:00:37.463 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:00:37.463 INFO:teuthology.orchestra.run.smithi092.stdout:Last metadata expiration check: 0:01:48 ago on Fri 06 Oct 2023 02:58:49 AM UTC. 2023-10-06T03:00:39.449 INFO:teuthology.orchestra.run.smithi092.stdout:Installed products updated. 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.578 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.579 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T03:00:39.580 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: jq-1.5-12.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T03:00:39.581 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T03:00:39.582 INFO:teuthology.orchestra.run.smithi092.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T03:00:39.583 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T03:00:39.584 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T03:00:39.585 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.586 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T03:00:39.587 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T03:00:39.588 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:39.589 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2023-10-06T03:00:39.871 DEBUG:teuthology.parallel:result is None 2023-10-06T03:00:39.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:00:40.089 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T03:00:40.110 INFO:teuthology.orchestra.run.smithi092.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T03:00:40.111 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T03:00:40.111 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T03:00:40.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:00:40.264 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T03:00:40.289 INFO:teuthology.orchestra.run.smithi136.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T03:00:40.289 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T03:00:40.289 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T03:00:40.289 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T03:00:40.289 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:40.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T03:00:40.327 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:40.327 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T03:00:40.372 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T03:00:40.372 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:40.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T03:00:40.397 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T03:00:40.462 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:40.462 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T03:00:40.488 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T03:00:40.553 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T03:00:40.554 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:40.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T03:00:40.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T03:00:40.642 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:40.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T03:00:40.666 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T03:00:40.731 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T03:00:40.731 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:40.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T03:00:40.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T03:00:40.820 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:40.820 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T03:00:40.844 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T03:00:40.909 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T03:00:40.973 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T03:00:40.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2023-10-06T03:00:40.976 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2023-10-06T03:00:40.999 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T03:00:41.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T03:00:41.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T03:00:41.063 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T03:00:41.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T03:00:41.104 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T03:00:41.135 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T03:00:41.135 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T03:00:41.138 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T03:00:41.138 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:41.138 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T03:00:41.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T03:00:41.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T03:00:41.319 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:41.319 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T03:00:41.333 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T03:00:41.443 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T03:00:41.472 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T03:00:41.472 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T03:00:41.472 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176', 'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412176', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7519, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f', 'targets': {'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNySbKwKtbn23kaOpdUEXi+78qW6CwALxFpT/RCDrTx+bL/kB6CcHKIvUERPjiPH16MiOBLWYiifw3LJjrmqQLo=', 'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKH/OkfSWXHIbERePHsgBONEAbvU0eMC2nwTK72hhDG2ASmy/zmAyTPj/6R828c1bvdoAMZ2gsV8tNbd6Z76RR4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:52:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T03:00:41.473 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T03:00:41.475 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T03:00:41.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T03:00:41.529 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T03:00:41.586 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:41.586 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T03:00:41.606 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T03:00:41.606 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2023-10-06T03:00:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T03:00:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:41.661 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 218709 Links: 1 2023-10-06T03:00:41.661 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:41.661 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:41.661 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 03:00:34.730939212 +0000 2023-10-06T03:00:41.662 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 02:58:05.241239017 +0000 2023-10-06T03:00:41.662 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 02:58:05.241239017 +0000 2023-10-06T03:00:41.662 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T03:00:41.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T03:00:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T03:00:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T03:00:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000251845 s, 2.0 MB/s 2023-10-06T03:00:41.725 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T03:00:41.781 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2023-10-06T03:00:41.836 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T03:00:41.836 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:41.836 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 217860 Links: 1 2023-10-06T03:00:41.837 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:41.837 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:41.837 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 03:00:34.730939212 +0000 2023-10-06T03:00:41.837 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 02:58:05.741225071 +0000 2023-10-06T03:00:41.837 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 02:58:05.741225071 +0000 2023-10-06T03:00:41.837 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T03:00:41.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T03:00:41.899 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T03:00:41.899 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T03:00:41.899 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000245262 s, 2.1 MB/s 2023-10-06T03:00:41.900 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T03:00:41.956 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2023-10-06T03:00:42.011 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T03:00:42.011 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:42.011 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 214708 Links: 1 2023-10-06T03:00:42.011 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:42.012 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:42.012 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 03:00:34.730939212 +0000 2023-10-06T03:00:42.012 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 02:58:06.242211098 +0000 2023-10-06T03:00:42.012 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 02:58:06.242211098 +0000 2023-10-06T03:00:42.012 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T03:00:42.012 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T03:00:42.074 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T03:00:42.075 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T03:00:42.075 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000258112 s, 2.0 MB/s 2023-10-06T03:00:42.076 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T03:00:42.131 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2023-10-06T03:00:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T03:00:42.186 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:42.186 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 222213 Links: 1 2023-10-06T03:00:42.186 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2023-10-06 03:00:34.730939212 +0000 2023-10-06T03:00:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2023-10-06 02:58:06.742197153 +0000 2023-10-06T03:00:42.187 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2023-10-06 02:58:06.742197153 +0000 2023-10-06T03:00:42.187 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2023-10-06T03:00:42.187 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T03:00:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2023-10-06T03:00:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2023-10-06T03:00:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000194257 s, 2.6 MB/s 2023-10-06T03:00:42.250 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T03:00:42.306 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:42.306 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:42.306 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T03:00:42.322 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T03:00:42.322 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2023-10-06T03:00:42.378 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T03:00:42.378 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:42.378 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 222290 Links: 1 2023-10-06T03:00:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2023-10-06 03:00:28.756716150 +0000 2023-10-06T03:00:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2023-10-06 02:58:05.228921845 +0000 2023-10-06T03:00:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2023-10-06 02:58:05.228921845 +0000 2023-10-06T03:00:42.379 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2023-10-06T03:00:42.379 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T03:00:42.442 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2023-10-06T03:00:42.443 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2023-10-06T03:00:42.443 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000306073 s, 1.7 MB/s 2023-10-06T03:00:42.444 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T03:00:42.501 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2023-10-06T03:00:42.556 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 218823 Links: 1 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2023-10-06 03:00:28.756716150 +0000 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2023-10-06 02:58:05.710911214 +0000 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2023-10-06 02:58:05.710911214 +0000 2023-10-06T03:00:42.557 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2023-10-06T03:00:42.557 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T03:00:42.619 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2023-10-06T03:00:42.620 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2023-10-06T03:00:42.620 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000234625 s, 2.2 MB/s 2023-10-06T03:00:42.621 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T03:00:42.677 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2023-10-06T03:00:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T03:00:42.732 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:42.732 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 223282 Links: 1 2023-10-06T03:00:42.732 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:42.732 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:42.733 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2023-10-06 03:00:28.756716150 +0000 2023-10-06T03:00:42.733 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2023-10-06 02:58:06.222899922 +0000 2023-10-06T03:00:42.733 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2023-10-06 02:58:06.222899922 +0000 2023-10-06T03:00:42.733 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2023-10-06T03:00:42.733 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T03:00:42.795 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2023-10-06T03:00:42.795 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2023-10-06T03:00:42.796 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000210037 s, 2.4 MB/s 2023-10-06T03:00:42.796 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T03:00:42.851 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2023-10-06T03:00:42.906 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T03:00:42.906 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:00:42.906 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 6h/6d Inode: 223414 Links: 1 2023-10-06T03:00:42.906 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:00:42.906 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:00:42.906 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2023-10-06 03:00:28.757716128 +0000 2023-10-06T03:00:42.907 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2023-10-06 02:58:06.713889093 +0000 2023-10-06T03:00:42.907 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2023-10-06 02:58:06.713889093 +0000 2023-10-06T03:00:42.907 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2023-10-06T03:00:42.907 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T03:00:42.969 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2023-10-06T03:00:42.969 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2023-10-06T03:00:42.969 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000225484 s, 2.3 MB/s 2023-10-06T03:00:42.970 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T03:00:43.025 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:43.026 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T03:00:43.026 INFO:tasks.ceph:Generating config... 2023-10-06T03:00:43.027 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T03:00:43.027 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T03:00:43.027 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T03:00:43.027 INFO:tasks.ceph:[client] pid file = 2023-10-06T03:00:43.027 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T03:00:43.027 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T03:00:43.028 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T03:00:43.028 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T03:00:43.029 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T03:00:43.030 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T03:00:43.030 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T03:00:43.031 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T03:00:43.031 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T03:00:43.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T03:00:43.086 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T03:00:43.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T03:00:43.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:00:43.192 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.92'), ('mon.b', '172.21.15.136'), ('mon.c', '[v2:172.21.15.136:3301,v1:172.21.15.136:6790]')] 2023-10-06T03:00:43.192 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92,172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T03:00:43.193 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:43.193 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T03:00:43.247 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --add b 172.21.15.136 --addv c '[v2:172.21.15.136:3301,v1:172.21.15.136:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid e5cba154-9a64-4fb6-add9-5855ed5b5b5a 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:setting min_mon_release = octopus 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:fsid e5cba154-9a64-4fb6-add9-5855ed5b5b5a 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2023-10-06T03:00:43.329103+0000 2023-10-06T03:00:43.329 INFO:teuthology.orchestra.run.smithi092.stdout:created 2023-10-06T03:00:43.329103+0000 2023-10-06T03:00:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 15 (octopus) 2023-10-06T03:00:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2023-10-06T03:00:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2023-10-06T03:00:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2023-10-06T03:00:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2023-10-06T03:00:43.330 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T03:00:43.331 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T03:00:43.386 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e5cba154-9a64-4fb6-add9-5855ed5b5b5a... 2023-10-06T03:00:43.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T03:00:43.428 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2023-10-06T03:00:43.468 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.469 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2023-10-06T03:00:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.471 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.92,172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790]" 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on skipped message = False 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = e5cba154-9a64-4fb6-add9-5855ed5b5b5a 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.472 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2023-10-06T03:00:43.473 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2023-10-06T03:00:43.474 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2023-10-06T03:00:43.475 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 1/20 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 1/20 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2023-10-06T03:00:43.476 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 4/10 2023-10-06T03:00:43.477 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T03:00:43.477 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2023-10-06T03:00:43.477 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2023-10-06T03:00:43.477 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2023-10-06T03:00:43.478 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2023-10-06T03:00:43.478 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-10-06T03:00:43.478 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.478 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2023-10-06T03:00:43.478 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T03:00:43.478 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2023-10-06T03:00:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-10-06T03:00:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-10-06T03:00:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:00:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:00:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2023-10-06T03:00:43.479 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug finisher = 20 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2023-10-06T03:00:43.480 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2023-10-06T03:00:43.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2023-10-06T03:00:43.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2023-10-06T03:00:43.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:00:43.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2023-10-06T03:00:43.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2023-10-06T03:00:43.481 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:00:43.482 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.482 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T03:00:43.482 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T03:00:43.482 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T03:00:43.482 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.482 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T03:00:43.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T03:00:43.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T03:00:43.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.483 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2023-10-06T03:00:43.483 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2023-10-06T03:00:43.483 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2023-10-06T03:00:43.484 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = "" 2023-10-06T03:00:43.485 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-10-06T03:00:43.486 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-10-06T03:00:43.486 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2023-10-06T03:00:43.486 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2023-10-06T03:00:43.486 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2023-10-06T03:00:43.486 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-10-06T03:00:43.486 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-10-06T03:00:43.487 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-10-06T03:00:43.487 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-10-06T03:00:43.487 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2023-10-06T03:00:43.487 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2023-10-06T03:00:43.487 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2023-10-06T03:00:43.487 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2023-10-06T03:00:43.488 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2023-10-06T03:00:43.488 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2023-10-06T03:00:43.488 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2023-10-06T03:00:43.488 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2023-10-06T03:00:43.489 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2023-10-06T03:00:43.489 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2023-10-06T03:00:43.491 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2023-10-06T03:00:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2023-10-06T03:00:43.491 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T03:00:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2023-10-06T03:00:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2023-10-06T03:00:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.492 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.493 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2023-10-06T03:00:43.494 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2023-10-06T03:00:43.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.92,172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790]" 2023-10-06T03:00:43.496 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on skipped message = False 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = e5cba154-9a64-4fb6-add9-5855ed5b5b5a 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2023-10-06T03:00:43.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2023-10-06T03:00:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2023-10-06T03:00:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2023-10-06T03:00:43.498 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2023-10-06T03:00:43.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: bdev async discard = True 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: bdev enable discard = True 2023-10-06T03:00:43.500 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore allocator = bitmap 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 1/20 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 1/20 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 4/10 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2023-10-06T03:00:43.501 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2023-10-06T03:00:43.502 INFO:teuthology.orchestra.run.smithi136.stdout: debug client = 20 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: debug finisher = 20 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2023-10-06T03:00:43.503 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T03:00:43.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: mon op complaint time = 120 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2023-10-06T03:00:43.505 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: client mount timeout = 600 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: debug client = 20 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = "" 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 900 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 900 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2023-10-06T03:00:43.506 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.c] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.d] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout:[mds] 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: debug mds = 20 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug frag = True 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug scatterstat = True 2023-10-06T03:00:43.507 INFO:teuthology.orchestra.run.smithi136.stdout: mds op complaint time = 180 2023-10-06T03:00:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: mds verify scatter = True 2023-10-06T03:00:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2023-10-06T03:00:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 900 2023-10-06T03:00:43.508 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 900 2023-10-06T03:00:43.508 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T03:00:43.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T03:00:43.583 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T03:00:43.583 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:43.583 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T03:00:43.598 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:43.598 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T03:00:43.652 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:00:43.652 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:43.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T03:00:43.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:00:43.726 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:00:43.726 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:43.781 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:00:43.782 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:43.782 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T03:00:43.782 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:00:43.823 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:00:43.823 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:43.880 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T03:00:43.880 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T03:00:43.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T03:00:43.928 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T03:00:43.930 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T03:00:44.067 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T03:00:44.070 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T03:00:44.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T03:00:44.117 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T03:00:44.120 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T03:00:44.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T03:00:44.230 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T03:00:44.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T03:00:44.257 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T03:00:44.306 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T03:00:44.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T03:00:44.333 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T03:00:44.421 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T03:00:44.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T03:00:44.449 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T03:00:44.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T03:00:44.487 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T03:00:44.499 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T03:00:44.577 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T03:00:44.590 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T03:00:44.627 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T03:00:44.639 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T03:00:44.717 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T03:00:44.729 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T03:00:44.730 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T03:00:44.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T03:00:44.754 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:44.754 INFO:tasks.ceph:role: osd.0 2023-10-06T03:00:44.754 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:00:44.755 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T03:00:44.826 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:00:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:00:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:00:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T03:00:44.826 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:00:44.826 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:00:44.827 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:00:44.827 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:00:44.827 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:00:44.827 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:00:45.254 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T03:00:45.256 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T03:00:45.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T03:00:45.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T03:00:45.448 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T03:00:45.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:45.512 INFO:tasks.ceph:role: osd.1 2023-10-06T03:00:45.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:00:45.513 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T03:00:45.584 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:00:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:00:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:00:45.584 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T03:00:45.584 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:00:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:00:45.585 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:00:45.585 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:00:45.585 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:00:45.585 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:00:46.013 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T03:00:46.015 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T03:00:46.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T03:00:46.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T03:00:46.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T03:00:46.208 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:46.208 INFO:tasks.ceph:role: osd.2 2023-10-06T03:00:46.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:00:46.208 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T03:00:46.279 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:00:46.279 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:00:46.280 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:00:46.729 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T03:00:46.730 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T03:00:46.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T03:00:46.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T03:00:46.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T03:00:47.004 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:47.004 INFO:tasks.ceph:role: osd.3 2023-10-06T03:00:47.004 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:00:47.004 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T03:00:47.077 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:00:47.078 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:00:47.521 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2023-10-06T03:00:47.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2023-10-06T03:00:47.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T03:00:47.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T03:00:47.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.013+0000 7fc59ece9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T03:00:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.014+0000 7fc59ece9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T03:00:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.014+0000 7fc59ece9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:00:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.014+0000 7fc59ece9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:00:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.014+0000 7fc59ece9540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:00:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.014+0000 7fc59ece9540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T03:00:48.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:48.541+0000 7fc59ece9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:49.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:49.304+0000 7fc59ece9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:50.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.066+0000 7fc59ece9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:50.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T03:00:50.894 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.981+0000 7f06dc30f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T03:00:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.981+0000 7f06dc30f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T03:00:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.981+0000 7f06dc30f540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:00:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.981+0000 7f06dc30f540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:00:50.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.981+0000 7f06dc30f540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:00:50.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:50.982+0000 7f06dc30f540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T03:00:51.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:51.508+0000 7f06dc30f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:52.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:52.275+0000 7f06dc30f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:53.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.037+0000 7f06dc30f540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:53.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T03:00:53.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:53.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.945+0000 7fe975bc4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T03:00:53.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.945+0000 7fe975bc4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T03:00:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.945+0000 7fe975bc4540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:00:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.945+0000 7fe975bc4540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:00:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.945+0000 7fe975bc4540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:00:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:53.946+0000 7fe975bc4540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T03:00:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:54.479+0000 7fe975bc4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:55.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:55.242+0000 7fe975bc4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:56.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.004+0000 7fe975bc4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:56.799 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T03:00:56.869 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:00:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.953+0000 7fbeedfdb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T03:00:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.953+0000 7fbeedfdb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T03:00:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.953+0000 7fbeedfdb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:00:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.953+0000 7fbeedfdb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:00:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.953+0000 7fbeedfdb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:00:56.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:56.954+0000 7fbeedfdb540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T03:00:57.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:57.477+0000 7fbeedfdb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:58.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:58.238+0000 7fbeedfdb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:59.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:00:59.001+0000 7fbeedfdb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:00:59.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T03:00:59.824 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T03:00:59.855 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:00:59.855 INFO:tasks.ceph:role: osd.4 2023-10-06T03:00:59.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:00:59.856 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T03:00:59.931 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:00:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:00:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:00:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2023-10-06T03:00:59.932 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:00:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:00:59.932 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:00:59.932 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:00:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:00:59.932 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:01:00.407 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2023-10-06T03:01:00.409 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2023-10-06T03:01:00.410 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T03:01:00.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T03:01:00.566 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T03:01:00.634 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:01:00.634 INFO:tasks.ceph:role: osd.5 2023-10-06T03:01:00.634 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:01:00.634 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T03:01:00.708 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:01:00.708 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:01:00.708 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:01:00.708 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2023-10-06T03:01:00.708 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:01:00.708 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:01:00.709 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:01:00.709 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:01:00.709 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:01:00.709 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:01:01.139 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2023-10-06T03:01:01.140 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2023-10-06T03:01:01.141 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T03:01:01.270 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T03:01:01.301 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T03:01:01.367 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:01:01.367 INFO:tasks.ceph:role: osd.6 2023-10-06T03:01:01.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:01:01.367 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T03:01:01.441 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:01:01.442 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:01:01.934 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2023-10-06T03:01:01.936 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2023-10-06T03:01:01.936 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T03:01:02.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T03:01:02.112 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T03:01:02.176 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:01:02.176 INFO:tasks.ceph:role: osd.7 2023-10-06T03:01:02.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:01:02.177 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T03:01:02.248 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:01:02.249 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:01:02.732 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2023-10-06T03:01:02.733 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2023-10-06T03:01:02.734 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T03:01:02.847 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T03:01:02.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:01:02.974 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:02.972+0000 7f8768958540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T03:01:02.975 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:02.974+0000 7f8768958540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T03:01:02.976 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:02.974+0000 7f8768958540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:01:02.976 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:02.974+0000 7f8768958540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:01:02.976 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:02.974+0000 7f8768958540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:01:02.976 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:02.975+0000 7f8768958540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T03:01:03.505 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:03.503+0000 7f8768958540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:04.269 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:04.267+0000 7f8768958540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:05.030 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.029+0000 7f8768958540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:05.824 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T03:01:05.852 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:01:05.937 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.935+0000 7f427c1e9540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T03:01:05.937 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.935+0000 7f427c1e9540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T03:01:05.937 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.935+0000 7f427c1e9540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:01:05.937 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.935+0000 7f427c1e9540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:01:05.937 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.935+0000 7f427c1e9540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:01:05.937 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:05.935+0000 7f427c1e9540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T03:01:06.464 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:06.462+0000 7f427c1e9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:07.233 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:07.231+0000 7f427c1e9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:07.995 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:07.994+0000 7f427c1e9540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:08.806 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T03:01:08.872 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:01:08.954 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:08.952+0000 7fafbc113540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T03:01:08.954 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:08.952+0000 7fafbc113540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T03:01:08.955 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:08.952+0000 7fafbc113540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:01:08.955 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:08.952+0000 7fafbc113540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:01:08.955 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:08.952+0000 7fafbc113540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:01:08.955 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:08.953+0000 7fafbc113540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T03:01:09.481 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:09.479+0000 7fafbc113540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:10.247 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:10.245+0000 7fafbc113540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:11.013 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.011+0000 7fafbc113540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:11.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:01:11.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:01:11.803 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T03:01:11.827 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:01:11.909 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.907+0000 7fd2c5f61540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T03:01:11.909 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.907+0000 7fd2c5f61540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T03:01:11.909 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.907+0000 7fd2c5f61540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:01:11.909 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.907+0000 7fd2c5f61540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:01:11.909 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.907+0000 7fd2c5f61540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:01:11.910 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:11.908+0000 7fd2c5f61540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T03:01:12.432 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:12.431+0000 7fd2c5f61540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:13.194 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:13.193+0000 7fd2c5f61540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:13.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:13.960+0000 7fd2c5f61540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:01:14.753 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T03:01:14.820 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T03:01:14.820 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:14.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T03:01:14.847 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:14.847 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T03:01:14.914 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:14.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T03:01:14.981 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:14.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T03:01:15.047 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:15.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T03:01:15.112 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:15.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T03:01:15.177 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:15.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T03:01:15.242 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.242 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T03:01:15.269 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.269 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T03:01:15.336 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T03:01:15.402 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.402 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T03:01:15.467 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.467 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T03:01:15.532 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.532 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T03:01:15.596 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.596 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T03:01:15.660 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:15.660 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T03:01:15.676 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:15.676 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T03:01:15.730 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.731 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T03:01:15.745 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:15.745 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T03:01:15.800 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T03:01:15.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T03:01:15.802 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T03:01:15.842 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2023-10-06T03:01:15.842 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg== 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww== 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ== 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw== 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ== 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA== 2023-10-06T03:01:15.843 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA== 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g== 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA== 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ== 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg== 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA== 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2023-10-06T03:01:15.844 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA== 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA== 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw== 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout:[client.2] 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ== 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout:[client.1] 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg== 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout:[client.3] 2023-10-06T03:01:15.845 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg== 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg== 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww== 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.c] 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ== 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw== 2023-10-06T03:01:15.862 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ== 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA== 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA== 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g== 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA== 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.d] 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ== 2023-10-06T03:01:15.863 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg== 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA== 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA== 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA== 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw== 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout:[client.2] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ== 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout:[client.1] 2023-10-06T03:01:15.864 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg== 2023-10-06T03:01:15.865 INFO:teuthology.orchestra.run.smithi136.stdout:[client.3] 2023-10-06T03:01:15.865 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg== 2023-10-06T03:01:15.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:01:15.889 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:01:15.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:15.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.006 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.058 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.108 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.154 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.205 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.252 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:01:16.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:01:16.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.354 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:01:16.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.407 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.453 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.455 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.554 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:01:16.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.603 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.706 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.759 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:01:16.809 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T03:01:16.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T03:01:16.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:01:16.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T03:01:16.947 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T03:01:16.972 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:01:17.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T03:01:17.086 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T03:01:17.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:01:17.241 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T03:01:17.265 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:01:17.268 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:01:17.320 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T03:01:17.320 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T03:01:17.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T03:01:17.323 INFO:tasks.ceph.mon.a:Started 2023-10-06T03:01:17.323 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T03:01:17.323 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T03:01:17.362 INFO:tasks.ceph.mon.b:Started 2023-10-06T03:01:17.362 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T03:01:17.362 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T03:01:17.364 INFO:tasks.ceph.mon.c:Started 2023-10-06T03:01:17.364 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T03:01:17.364 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T03:01:17.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T03:01:17.367 INFO:tasks.ceph.mgr.y:Started 2023-10-06T03:01:17.367 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T03:01:17.367 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T03:01:17.369 INFO:tasks.ceph.mgr.x:Started 2023-10-06T03:01:17.369 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T03:01:17.369 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T03:01:17.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.484+0000 7f5f7bfff700 1 Processor -- start 2023-10-06T03:01:17.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.484+0000 7f5f7bfff700 1 -- start start 2023-10-06T03:01:17.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f7c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 0x7f5f7c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7bfff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c0b2080 con 0x7f5f7c15d240 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7bfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c0cfb00 con 0x7f5f7c0b16d0 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7bfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0b5c40 con 0x7f5f7c0b4e70 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 0x7f5f7c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 0x7f5f7c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40426/0 (socket says 172.21.15.92:40426) 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 -- 172.21.15.92:0/2232155539 learned_addr learned my addr 172.21.15.92:0/2232155539 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 -- 172.21.15.92:0/2232155539 >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 msgr2=0x7f5f7c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 -- 172.21.15.92:0/2232155539 >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 msgr2=0x7f5f7c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/2232155539 >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 0x7f5f7c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7affd700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f7c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/2232155539 >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 0x7f5f7c0b1b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:01:17.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7affd700 1 -- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 msgr2=0x7f5f7c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T03:01:17.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7affd700 1 -- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 msgr2=0x7f5f7c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T03:01:17.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7affd700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f7c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T03:01:17.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.485+0000 7f5f7affd700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f7c0ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:01:17.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.675+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078767933 0 0) 0x7f5f7c0b2080 con 0x7f5f7c15d240 2023-10-06T03:01:17.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.675+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f68003570 con 0x7f5f7c15d240 2023-10-06T03:01:17.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 <== mon.1 v1:172.21.15.136:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 412071211 0 0) 0x7f5f68003570 con 0x7f5f7c15d240 2023-10-06T03:01:17.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 msgr2=0x7f5f7c0ccff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:01:17.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f7c0ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 msgr2=0x7f5f7c0b1b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 --2- 172.21.15.92:0/2232155539 >> v2:172.21.15.92:3300/0 conn(0x7f5f7c0b16d0 0x7f5f7c0b1b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 --> v1:172.21.15.136:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0b5f20 con 0x7f5f7c15d240 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 <== mon.1 v1:172.21.15.136:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5f74001470 con 0x7f5f7c15d240 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 >> v1:172.21.15.136:6789/0 conn(0x7f5f7c15d240 legacy=0x7f5f7c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f68003ae0 0x7f5f68003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f68005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0cfb00 con 0x7f5f68004810 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0b5c40 con 0x7f5f68003ae0 2023-10-06T03:01:17.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6800bc70 con 0x7f5f7c0b4e70 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.680+0000 7f5f7affd700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f68003ae0 0x7f5f68003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7b7fe700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f68005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7a7fc700 1 -- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 msgr2=0x7f5f68005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0b4e70 0x7f5f68005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7a7fc700 1 -- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f68003ae0 msgr2=0x7f5f68003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f68003ae0 0x7f5f68003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7a7fc700 1 -- 172.21.15.92:0/2232155539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f68003570 con 0x7f5f68004810 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f68004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009060 tx=0x7f5f64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e4aaf09993221c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400c070 con 0x7f5f68004810 2023-10-06T03:01:17.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6400e5e0 con 0x7f5f68004810 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.681+0000 7f5f79ffb700 1 -- 172.21.15.92:0/2232155539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400e770 con 0x7f5f68004810 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7bfff700 1 -- 172.21.15.92:0/2232155539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 msgr2=0x7f5f68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7bfff700 1 --2- 172.21.15.92:0/2232155539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f68004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009060 tx=0x7f5f64008040 comp rx=0 tx=0).stop 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7b7fe700 1 -- 172.21.15.92:0/2232155539 reap_dead start 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7bfff700 1 -- 172.21.15.92:0/2232155539 shutdown_connections 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7bfff700 1 -- 172.21.15.92:0/2232155539 >> 172.21.15.92:0/2232155539 conn(0x7f5f7c0d10c0 msgr2=0x7f5f7c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7bfff700 1 -- 172.21.15.92:0/2232155539 shutdown_connections 2023-10-06T03:01:17.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.682+0000 7f5f7bfff700 1 -- 172.21.15.92:0/2232155539 wait complete. 2023-10-06T03:01:17.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.683+0000 7f5f7bfff700 1 Processor -- start 2023-10-06T03:01:17.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.683+0000 7f5f7bfff700 1 -- start start 2023-10-06T03:01:17.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f7c0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0c58d0 0x7f5f7c15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f7c15c320 0x7f5f7c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7bfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c7470 con 0x7f5f68004810 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7bfff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c5d70 con 0x7f5f7c15c320 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7bfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c6050 con 0x7f5f7c0c58d0 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f7c0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f7c0c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40468/0 (socket says 172.21.15.92:40468) 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7affd700 1 -- 172.21.15.92:0/3702932723 learned_addr learned my addr 172.21.15.92:0/3702932723 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7b7fe700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f7c15c320 0x7f5f7c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.684+0000 7f5f7a7fc700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0c58d0 0x7f5f7c15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:17.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7affd700 1 -- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0c58d0 msgr2=0x7f5f7c15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7affd700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0c58d0 0x7f5f7c15bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7affd700 1 -- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f7c15c320 msgr2=0x7f5f7c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7affd700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f7c15c320 0x7f5f7c157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7affd700 1 -- 172.21.15.92:0/3702932723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0c6330 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7affd700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f7c0c5390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00f2d0 tx=0x7f5f6c009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c019070 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c00fe10 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c0c70e0 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0156a0 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.685+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c0cb9d0 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.686+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c0c7750 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.688+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5f6c003020 con 0x7f5f68004810 2023-10-06T03:01:17.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.688+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5f6c00a380 con 0x7f5f68004810 2023-10-06T03:01:17.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.690+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5f6c015d50 con 0x7f5f68004810 2023-10-06T03:01:17.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.766+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5f7c0cd5a0 con 0x7f5f68004810 2023-10-06T03:01:17.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.772+0000 7f5f72ffd700 1 -- 172.21.15.92:0/3702932723 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5f6c009cf0 con 0x7f5f68004810 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.773+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 msgr2=0x7f5f7c0c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.773+0000 7f5f7bfff700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f7c0c5390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00f2d0 tx=0x7f5f6c009720 comp rx=0 tx=0).stop 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 shutdown_connections 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f68004810 0x7f5f7c0c5390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f7c15c320 0x7f5f7c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 --2- 172.21.15.92:0/3702932723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f7c0c58d0 0x7f5f7c15bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 >> 172.21.15.92:0/3702932723 conn(0x7f5f7c0d10c0 msgr2=0x7f5f7c15c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:17.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 shutdown_connections 2023-10-06T03:01:17.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:17.774+0000 7f5f7bfff700 1 -- 172.21.15.92:0/3702932723 wait complete. 2023-10-06T03:01:17.777 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2023-10-06T03:01:17.787 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T03:01:17.788 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:17.788 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T03:01:17.812 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:17.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T03:01:17.877 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:17.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T03:01:17.942 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2023-10-06T03:01:17.942 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T03:01:18.007 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:18.007 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T03:01:18.032 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:18.032 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T03:01:18.097 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:18.097 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T03:01:18.161 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2023-10-06T03:01:18.161 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T03:01:18.226 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new b6c8c1e3-7b81-48e7-968a-7d3f12021a11 0 2023-10-06T03:01:18.381 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.380+0000 7fae6787e700 1 Processor -- start 2023-10-06T03:01:18.382 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.380+0000 7fae6787e700 1 -- start start 2023-10-06T03:01:18.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae6787e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fae600caa90 0x7fae600c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae6787e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6015d7e0 con 0x7fae6006bcd0 2023-10-06T03:01:18.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae6787e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6015dac0 con 0x7fae6006c0a0 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae6787e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae600caef0 con 0x7fae600caa90 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae65e1b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fae600caa90 0x7fae600c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae64e19700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fae6006c0a0 0x7fae6015df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:37396/0 (socket says 172.21.15.136:37396) 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae64e19700 1 -- 172.21.15.136:0/1426068742 learned_addr learned my addr 172.21.15.136:0/1426068742 (peer_addr_for_me v1:172.21.15.136:0/0) 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae65e1b700 1 -- 172.21.15.136:0/1426068742 >> v1:172.21.15.136:6789/0 conn(0x7fae6006c0a0 legacy=0x7fae6015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae65e1b700 1 -- 172.21.15.136:0/1426068742 >> v1:172.21.15.92:6789/0 conn(0x7fae6006bcd0 legacy=0x7fae6015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae57fff700 1 -- 172.21.15.136:0/1426068742 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980021047 0 0) 0x7fae6015dac0 con 0x7fae6006c0a0 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae65e1b700 1 -- 172.21.15.136:0/1426068742 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6015aa30 con 0x7fae600caa90 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.381+0000 7fae65e1b700 1 --2- 172.21.15.136:0/1426068742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae600caa90 0x7fae600c8c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae58009e60 tx=0x7fae58009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177b256fcdc17c1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:18.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae57fff700 1 -- 172.21.15.136:0/1426068742 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800e070 con 0x7fae600caa90 2023-10-06T03:01:18.385 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae57fff700 1 -- 172.21.15.136:0/1426068742 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5800bc30 con 0x7fae600caa90 2023-10-06T03:01:18.385 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae57fff700 1 -- 172.21.15.136:0/1426068742 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58012430 con 0x7fae600caa90 2023-10-06T03:01:18.387 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 -- 172.21.15.136:0/1426068742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae600caa90 msgr2=0x7fae600c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 --2- 172.21.15.136:0/1426068742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae600caa90 0x7fae600c8c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae58009e60 tx=0x7fae58009780 comp rx=0 tx=0).stop 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 -- 172.21.15.136:0/1426068742 shutdown_connections 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 --2- 172.21.15.136:0/1426068742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae600caa90 0x7fae600c8c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 -- 172.21.15.136:0/1426068742 >> 172.21.15.136:0/1426068742 conn(0x7fae600ce600 msgr2=0x7fae600cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 -- 172.21.15.136:0/1426068742 shutdown_connections 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.382+0000 7fae6787e700 1 -- 172.21.15.136:0/1426068742 wait complete. 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 Processor -- start 2023-10-06T03:01:18.388 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 -- start start 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 0x7fae6014b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6006c0a0 0x7fae600b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae600caa90 0x7fae600b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae600b1770 con 0x7fae600caa90 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae600b1440 con 0x7fae6006c0a0 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.384+0000 7fae6787e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae600b4360 con 0x7fae6006bcd0 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 0x7fae6014b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 0x7fae6014b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41000/0 (socket says 172.21.15.136:41000) 2023-10-06T03:01:18.389 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 -- 172.21.15.136:0/2241283448 learned_addr learned my addr 172.21.15.136:0/2241283448 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae64e19700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6006c0a0 0x7fae600b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae65e1b700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae600caa90 0x7fae600b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 -- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6006c0a0 msgr2=0x7fae600b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6006c0a0 0x7fae600b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 -- 172.21.15.136:0/2241283448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae600caa90 msgr2=0x7fae600b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae600caa90 0x7fae600b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 -- 172.21.15.136:0/2241283448 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009a20 con 0x7fae6006bcd0 2023-10-06T03:01:18.390 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6561a700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 0x7fae6014b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c4e0 tx=0x7fae5c00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c011590 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5c011710 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae600c73f0 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c011ad0 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae600c79b0 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.385+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fae5c01c6c0 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.386+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fae5c012770 con 0x7fae6006bcd0 2023-10-06T03:01:18.391 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.386+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae600b1a50 con 0x7fae6006bcd0 2023-10-06T03:01:18.392 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.390+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fae5c017070 con 0x7fae6006bcd0 2023-10-06T03:01:18.475 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.473+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b6c8c1e3-7b81-48e7-968a-7d3f12021a11", "id": 0} v 0) v1 -- 0x7fae6004c3c0 con 0x7fae6006bcd0 2023-10-06T03:01:18.480 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.478+0000 7fae557fa700 1 -- 172.21.15.136:0/2241283448 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b6c8c1e3-7b81-48e7-968a-7d3f12021a11", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae5c01cba0 con 0x7fae6006bcd0 2023-10-06T03:01:18.480 INFO:teuthology.orchestra.run.smithi136.stdout:0 2023-10-06T03:01:18.481 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 msgr2=0x7fae6014b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.481 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 0x7fae6014b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c4e0 tx=0x7fae5c00c7f0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.481 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 shutdown_connections 2023-10-06T03:01:18.482 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae600caa90 0x7fae600b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.482 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae6006bcd0 0x7fae6014b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.482 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 --2- 172.21.15.136:0/2241283448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae6006c0a0 0x7fae600b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.482 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 >> 172.21.15.136:0/2241283448 conn(0x7fae600ce600 msgr2=0x7fae600c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:18.482 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 shutdown_connections 2023-10-06T03:01:18.482 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.479+0000 7fae6787e700 1 -- 172.21.15.136:0/2241283448 wait complete. 2023-10-06T03:01:18.493 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb 1 2023-10-06T03:01:18.606 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.605+0000 7fd934e41700 1 Processor -- start 2023-10-06T03:01:18.607 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.605+0000 7fd934e41700 1 -- start start 2023-10-06T03:01:18.609 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd934e41700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 0x7fd93015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.609 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd934e41700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd930159c60 0x7fd93015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.609 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd934e41700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd93015db10 0x7fd930069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.609 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd934e41700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93015a160 con 0x7fd930159c60 2023-10-06T03:01:18.609 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd934e41700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9300d13a0 con 0x7fd93015db10 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd934e41700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93015e140 con 0x7fd930159890 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92e59c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 0x7fd93015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd93015db10 0x7fd930069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd93015db10 0x7fd930069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38838/0 (socket says 172.21.15.136:38838) 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 -- 172.21.15.136:0/2924922242 learned_addr learned my addr 172.21.15.136:0/2924922242 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92e59c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 0x7fd93015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41006/0 (socket says 172.21.15.136:41006) 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92dd9b700 1 --2- 172.21.15.136:0/2924922242 >> v2:172.21.15.92:3300/0 conn(0x7fd930159c60 0x7fd93015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 -- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 msgr2=0x7fd93015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 --2- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 0x7fd93015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 -- 172.21.15.136:0/2924922242 >> v2:172.21.15.92:3300/0 conn(0x7fd930159c60 msgr2=0x7fd93015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:18.610 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 --2- 172.21.15.136:0/2924922242 >> v2:172.21.15.92:3300/0 conn(0x7fd930159c60 0x7fd93015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 -- 172.21.15.136:0/2924922242 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93015a3d0 con 0x7fd93015db10 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92e59c700 1 --2- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 0x7fd93015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92ed9d700 1 --2- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd93015db10 0x7fd930069560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd930089b50 tx=0x7fd928009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f81cd8387b15cb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92d59a700 1 -- 172.21.15.136:0/2924922242 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800e070 con 0x7fd93015db10 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92d59a700 1 -- 172.21.15.136:0/2924922242 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92800b970 con 0x7fd93015db10 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.606+0000 7fd92d59a700 1 -- 172.21.15.136:0/2924922242 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280092e0 con 0x7fd93015db10 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 -- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd93015db10 msgr2=0x7fd930069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 --2- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd93015db10 0x7fd930069560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd930089b50 tx=0x7fd928009840 comp rx=0 tx=0).stop 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 -- 172.21.15.136:0/2924922242 shutdown_connections 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 --2- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd930159890 0x7fd93015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 --2- 172.21.15.136:0/2924922242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd93015db10 0x7fd930069560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.611 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 --2- 172.21.15.136:0/2924922242 >> v2:172.21.15.92:3300/0 conn(0x7fd930159c60 0x7fd93015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.612 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 -- 172.21.15.136:0/2924922242 >> 172.21.15.136:0/2924922242 conn(0x7fd930068e20 msgr2=0x7fd9300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:18.612 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 -- 172.21.15.136:0/2924922242 shutdown_connections 2023-10-06T03:01:18.612 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.607+0000 7fd934e41700 1 -- 172.21.15.136:0/2924922242 wait complete. 2023-10-06T03:01:18.614 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.608+0000 7fd934e41700 1 Processor -- start 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.608+0000 7fd934e41700 1 -- start start 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd934e41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd930159890 0x7fd93014c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd934e41700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 0x7fd9300ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd934e41700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd93015db10 0x7fd930150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd934e41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300cdad0 con 0x7fd930159890 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd934e41700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93014c7a0 con 0x7fd930159c60 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd934e41700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300cdd20 con 0x7fd93015db10 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92ed9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd93015db10 0x7fd930150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 0x7fd9300ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.615 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92ed9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd93015db10 0x7fd930150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41022/0 (socket says 172.21.15.136:41022) 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 0x7fd9300ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38846/0 (socket says 172.21.15.136:38846) 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92ed9d700 1 -- 172.21.15.136:0/1970201504 learned_addr learned my addr 172.21.15.136:0/1970201504 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92e59c700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd930159890 0x7fd93014c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 -- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd93015db10 msgr2=0x7fd930150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd93015db10 0x7fd930150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 -- 172.21.15.136:0/1970201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd930159890 msgr2=0x7fd93014c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd930159890 0x7fd93014c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.609+0000 7fd92dd9b700 1 -- 172.21.15.136:0/1970201504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920009420 con 0x7fd930159c60 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd92dd9b700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 0x7fd9300ceae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd92400ea20 tx=0x7fd9240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400ee60 con 0x7fd930159c60 2023-10-06T03:01:18.616 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92400c7f0 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92400cb90 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd928009ae0 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9300ce2e0 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd924005c80 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.610+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fd924013470 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.611+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93014c8e0 con 0x7fd930159c60 2023-10-06T03:01:18.617 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.613+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd93014c8e0 con 0x7fd930159c60 2023-10-06T03:01:18.698 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.696+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb", "id": 1} v 0) v1 -- 0x7fd93004c3c0 con 0x7fd930159c60 2023-10-06T03:01:18.703 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.701+0000 7fd91e7fc700 1 -- 172.21.15.136:0/1970201504 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd924021060 con 0x7fd930159c60 2023-10-06T03:01:18.703 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 msgr2=0x7fd9300ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 0x7fd9300ceae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd92400ea20 tx=0x7fd9240091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 shutdown_connections 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd930159890 0x7fd93014c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd93015db10 0x7fd930150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 --2- 172.21.15.136:0/1970201504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd930159c60 0x7fd9300ceae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 >> 172.21.15.136:0/1970201504 conn(0x7fd930068e20 msgr2=0x7fd930156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:18.705 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 shutdown_connections 2023-10-06T03:01:18.705 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.702+0000 7fd934e41700 1 -- 172.21.15.136:0/1970201504 wait complete. 2023-10-06T03:01:18.715 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 14ab28bc-0084-45c5-bb6c-4a7b469ce151 2 2023-10-06T03:01:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.832+0000 7f4e0efaa700 1 Processor -- start 2023-10-06T03:01:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.832+0000 7f4e0efaa700 1 -- start start 2023-10-06T03:01:18.836 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e0efaa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e08141500 0x7f4e080c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e0efaa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0815c2a0 con 0x7f4e0806bcd0 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e0efaa700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0815b690 con 0x7f4e0806c0a0 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e0efaa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0815b970 con 0x7f4e08141500 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e07fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e08141500 0x7f4e080c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e0d547700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f4e0806c0a0 0x7f4e080c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:37410/0 (socket says 172.21.15.136:37410) 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e0d547700 1 -- 172.21.15.136:0/2643760874 learned_addr learned my addr 172.21.15.136:0/2643760874 (peer_addr_for_me v1:172.21.15.136:0/0) 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e077fe700 1 -- 172.21.15.136:0/2643760874 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233061868 0 0) 0x7f4e0815b690 con 0x7f4e0806c0a0 2023-10-06T03:01:18.837 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e077fe700 1 -- 172.21.15.136:0/2643760874 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df4003570 con 0x7f4e0806c0a0 2023-10-06T03:01:18.838 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e07fff700 1 -- 172.21.15.136:0/2643760874 >> v1:172.21.15.136:6789/0 conn(0x7f4e0806c0a0 legacy=0x7f4e080c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.838 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e07fff700 1 -- 172.21.15.136:0/2643760874 >> v1:172.21.15.92:6789/0 conn(0x7f4e0806bcd0 legacy=0x7f4e08140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.838 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.833+0000 7f4e07fff700 1 -- 172.21.15.136:0/2643760874 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0815bc50 con 0x7f4e08141500 2023-10-06T03:01:18.838 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e07fff700 1 --2- 172.21.15.136:0/2643760874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e08141500 0x7f4e080c8e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009910 tx=0x7f4df80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d142ade99c5957a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:18.838 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e077fe700 1 -- 172.21.15.136:0/2643760874 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800b680 con 0x7f4e08141500 2023-10-06T03:01:18.841 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e077fe700 1 -- 172.21.15.136:0/2643760874 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df800b7e0 con 0x7f4e08141500 2023-10-06T03:01:18.841 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e077fe700 1 -- 172.21.15.136:0/2643760874 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800bba0 con 0x7f4e08141500 2023-10-06T03:01:18.841 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e0efaa700 1 -- 172.21.15.136:0/2643760874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e08141500 msgr2=0x7f4e080c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.841 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e0efaa700 1 --2- 172.21.15.136:0/2643760874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e08141500 0x7f4e080c8e00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009910 tx=0x7f4df80091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.841 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e0efaa700 1 -- 172.21.15.136:0/2643760874 shutdown_connections 2023-10-06T03:01:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e0efaa700 1 --2- 172.21.15.136:0/2643760874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e08141500 0x7f4e080c8e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.834+0000 7f4e0efaa700 1 -- 172.21.15.136:0/2643760874 >> 172.21.15.136:0/2643760874 conn(0x7f4e080ce600 msgr2=0x7f4e080cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.835+0000 7f4e0efaa700 1 -- 172.21.15.136:0/2643760874 shutdown_connections 2023-10-06T03:01:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.835+0000 7f4e0efaa700 1 -- 172.21.15.136:0/2643760874 wait complete. 2023-10-06T03:01:18.843 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:18.837+0000 7fdd950bd180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:01:18.845 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.836+0000 7f4e0efaa700 1 Processor -- start 2023-10-06T03:01:18.845 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.836+0000 7f4e0efaa700 1 -- start start 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.836+0000 7f4e0efaa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 0x7f4e080c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.836+0000 7f4e0efaa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0806c0a0 0x7f4e0813f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.836+0000 7f4e0efaa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e08141500 0x7f4e080b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0efaa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e080c7c80 con 0x7f4e08141500 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0efaa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e080c7390 con 0x7f4e0806c0a0 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0efaa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0813f650 con 0x7f4e0806bcd0 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 0x7f4e080c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e07fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0806c0a0 0x7f4e0813f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 0x7f4e080c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41050/0 (socket says 172.21.15.136:41050) 2023-10-06T03:01:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 -- 172.21.15.136:0/1052248381 learned_addr learned my addr 172.21.15.136:0/1052248381 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0d547700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e08141500 0x7f4e080b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 -- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0806c0a0 msgr2=0x7f4e0813f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0806c0a0 0x7f4e0813f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 -- 172.21.15.136:0/1052248381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e08141500 msgr2=0x7f4e080b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e08141500 0x7f4e080b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 -- 172.21.15.136:0/1052248381 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df8009420 con 0x7f4e0806bcd0 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e07fff700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0806c0a0 0x7f4e0813f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0d547700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e08141500 0x7f4e080b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0cd46700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 0x7f4e080c8110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000cb10 tx=0x7f4e000017b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000ce20 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e00006ba0 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000a7e0 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e080c7630 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.837+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e080b2ce0 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.838+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4e0000abd0 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.838+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f4e00012d00 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.838+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0804c3c0 con 0x7f4e0806bcd0 2023-10-06T03:01:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.841+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4e00016030 con 0x7f4e0806bcd0 2023-10-06T03:01:18.879 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:18.878+0000 7f11fdf55180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:01:18.923 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.922+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "14ab28bc-0084-45c5-bb6c-4a7b469ce151", "id": 2} v 0) v1 -- 0x7f4e0815bf30 con 0x7f4e0806bcd0 2023-10-06T03:01:18.929 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.927+0000 7f4e04ff9700 1 -- 172.21.15.136:0/1052248381 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14ab28bc-0084-45c5-bb6c-4a7b469ce151", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e00012420 con 0x7f4e0806bcd0 2023-10-06T03:01:18.929 INFO:teuthology.orchestra.run.smithi136.stdout:2 2023-10-06T03:01:18.930 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 msgr2=0x7f4e080c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:18.930 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 0x7f4e080c8110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000cb10 tx=0x7f4e000017b0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.930 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 shutdown_connections 2023-10-06T03:01:18.930 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e08141500 0x7f4e080b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.930 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0806bcd0 0x7f4e080c8110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.930 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 --2- 172.21.15.136:0/1052248381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0806c0a0 0x7f4e0813f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:18.931 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 >> 172.21.15.136:0/1052248381 conn(0x7f4e080ce600 msgr2=0x7f4e080cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:18.931 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 shutdown_connections 2023-10-06T03:01:18.931 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:18.928+0000 7f4e0efaa700 1 -- 172.21.15.136:0/1052248381 wait complete. 2023-10-06T03:01:18.941 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 54c6fa79-9c4a-4035-afec-92597ecdc604 3 2023-10-06T03:01:18.971 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:18.970+0000 7fdd950bd180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:01:19.018 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.017+0000 7f11fdf55180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:01:19.046 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:19.045+0000 7fdd950bd180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:01:19.057 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.055+0000 7fe24bafc700 1 Processor -- start 2023-10-06T03:01:19.057 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.055+0000 7fe24bafc700 1 -- start start 2023-10-06T03:01:19.059 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe24bafc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe244159c60 0x7fe24415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.059 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe24bafc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe24bafc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24415a160 con 0x7fe244159890 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe24bafc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24415df50 con 0x7fe244159c60 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe24bafc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440d13a0 con 0x7fe24415db10 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe249097700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe244159c60 0x7fe24415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe249898700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fe244159890 0x7fe24415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:37424/0 (socket says 172.21.15.136:37424) 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe249898700 1 -- 172.21.15.136:0/2100701026 learned_addr learned my addr 172.21.15.136:0/2100701026 (peer_addr_for_me v1:172.21.15.136:0/0) 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.056+0000 7fe24a099700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24a099700 1 -- 172.21.15.136:0/2100701026 >> v2:172.21.15.136:3300/0 conn(0x7fe244159c60 msgr2=0x7fe24415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24a099700 1 --2- 172.21.15.136:0/2100701026 >> v2:172.21.15.136:3300/0 conn(0x7fe244159c60 0x7fe24415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24a099700 1 -- 172.21.15.136:0/2100701026 >> v1:172.21.15.136:6789/0 conn(0x7fe244159890 legacy=0x7fe24415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24a099700 1 -- 172.21.15.136:0/2100701026 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24415e140 con 0x7fe24415db10 2023-10-06T03:01:19.060 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe249097700 1 --2- 172.21.15.136:0/2100701026 >> v2:172.21.15.136:3300/0 conn(0x7fe244159c60 0x7fe24415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24a099700 1 --2- 172.21.15.136:0/2100701026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244069560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e60 tx=0x7fe23c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5600a479b05b5815 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe248896700 1 -- 172.21.15.136:0/2100701026 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00e070 con 0x7fe24415db10 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe248896700 1 -- 172.21.15.136:0/2100701026 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c00bb60 con 0x7fe24415db10 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe248896700 1 -- 172.21.15.136:0/2100701026 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c012430 con 0x7fe24415db10 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24bafc700 1 -- 172.21.15.136:0/2100701026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 msgr2=0x7fe244069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.057+0000 7fe24bafc700 1 --2- 172.21.15.136:0/2100701026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244069560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e60 tx=0x7fe23c009780 comp rx=0 tx=0).stop 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.058+0000 7fe24bafc700 1 -- 172.21.15.136:0/2100701026 shutdown_connections 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.058+0000 7fe24bafc700 1 --2- 172.21.15.136:0/2100701026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244069560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.058+0000 7fe24bafc700 1 --2- 172.21.15.136:0/2100701026 >> v2:172.21.15.136:3300/0 conn(0x7fe244159c60 0x7fe24415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.058+0000 7fe24bafc700 1 -- 172.21.15.136:0/2100701026 >> 172.21.15.136:0/2100701026 conn(0x7fe244068e20 msgr2=0x7fe2440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.058+0000 7fe24bafc700 1 -- 172.21.15.136:0/2100701026 shutdown_connections 2023-10-06T03:01:19.061 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.058+0000 7fe24bafc700 1 -- 172.21.15.136:0/2100701026 wait complete. 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 Processor -- start 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 -- start start 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe244159890 0x7fe2440c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 0x7fe2440cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440c98b0 con 0x7fe244159890 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2440cffa0 con 0x7fe244159c60 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe24bafc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24415bb00 con 0x7fe24415db10 2023-10-06T03:01:19.064 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.059+0000 7fe249097700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 0x7fe2440cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe24a099700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 0x7fe2440cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38886/0 (socket says 172.21.15.136:38886) 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 -- 172.21.15.136:0/64668586 learned_addr learned my addr 172.21.15.136:0/64668586 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249898700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe244159890 0x7fe2440c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 -- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 msgr2=0x7fe244149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 -- 172.21.15.136:0/64668586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe244159890 msgr2=0x7fe2440c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe244159890 0x7fe2440c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 -- 172.21.15.136:0/64668586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe240009420 con 0x7fe244159c60 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe24a099700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:19.065 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249898700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe244159890 0x7fe2440c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe249097700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 0x7fe2440cf200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe24000ec80 tx=0x7fe2400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000c890 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24000c9f0 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000cd70 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c009a20 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.060+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2440cfcc0 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.061+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fe240005c80 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.061+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fe240013c70 con 0x7fe244159c60 2023-10-06T03:01:19.066 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.061+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24404c3c0 con 0x7fe244159c60 2023-10-06T03:01:19.067 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.065+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fe240021060 con 0x7fe244159c60 2023-10-06T03:01:19.100 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.098+0000 7f11fdf55180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:01:19.148 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.147+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "54c6fa79-9c4a-4035-afec-92597ecdc604", "id": 3} v 0) v1 -- 0x7fe24406a5b0 con 0x7fe244159c60 2023-10-06T03:01:19.156 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.154+0000 7fe239ffb700 1 -- 172.21.15.136:0/64668586 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "54c6fa79-9c4a-4035-afec-92597ecdc604", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe240017070 con 0x7fe244159c60 2023-10-06T03:01:19.156 INFO:teuthology.orchestra.run.smithi136.stdout:3 2023-10-06T03:01:19.157 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 msgr2=0x7fe2440cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.157 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 0x7fe2440cf200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe24000ec80 tx=0x7fe2400091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.157 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 shutdown_connections 2023-10-06T03:01:19.157 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe244159890 0x7fe2440c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.158 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24415db10 0x7fe244149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.158 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 --2- 172.21.15.136:0/64668586 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe244159c60 0x7fe2440cf200 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.158 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 >> 172.21.15.136:0/64668586 conn(0x7fe244068e20 msgr2=0x7fe2440b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.158 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 shutdown_connections 2023-10-06T03:01:19.158 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.155+0000 7fe24bafc700 1 -- 172.21.15.136:0/64668586 wait complete. 2023-10-06T03:01:19.169 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new c338a9f5-ade8-40aa-b512-dbe9e4c9d29b 4 2023-10-06T03:01:19.231 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:19.230+0000 7fdd950bd180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:01:19.284 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.282+0000 7f499f04a700 1 Processor -- start 2023-10-06T03:01:19.284 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.282+0000 7f499f04a700 1 -- start start 2023-10-06T03:01:19.286 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499f04a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 0x7f49980ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.286 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499f04a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49980b1b40 0x7f499815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.286 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499f04a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49980cfb00 con 0x7f49980b16d0 2023-10-06T03:01:19.286 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499f04a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49980b5c40 con 0x7f49980b1b40 2023-10-06T03:01:19.286 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499f04a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980b5f20 con 0x7f49980b4e70 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 0x7f49980ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 0x7f49980ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41080/0 (socket says 172.21.15.136:41080) 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 -- 172.21.15.136:0/3258564171 learned_addr learned my addr 172.21.15.136:0/3258564171 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e849700 1 --2- 172.21.15.136:0/3258564171 >> v2:172.21.15.92:3300/0 conn(0x7f49980b1b40 0x7f499815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 -- 172.21.15.136:0/3258564171 >> v2:172.21.15.92:3300/0 conn(0x7f49980b1b40 msgr2=0x7f499815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 --2- 172.21.15.136:0/3258564171 >> v2:172.21.15.92:3300/0 conn(0x7f49980b1b40 0x7f499815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 -- 172.21.15.136:0/3258564171 >> v1:172.21.15.92:6789/0 conn(0x7f49980b16d0 legacy=0x7f499815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e048700 1 -- 172.21.15.136:0/3258564171 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980b6200 con 0x7f49980b4e70 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.283+0000 7f499e849700 1 --2- 172.21.15.136:0/3258564171 >> v2:172.21.15.92:3300/0 conn(0x7f49980b1b40 0x7f499815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499e048700 1 --2- 172.21.15.136:0/3258564171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 0x7f49980ccff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49880099b0 tx=0x7f4988009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f9a8df8397fb0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499d046700 1 -- 172.21.15.136:0/3258564171 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800b860 con 0x7f49980b4e70 2023-10-06T03:01:19.287 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499d046700 1 -- 172.21.15.136:0/3258564171 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498800b9c0 con 0x7f49980b4e70 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499d046700 1 -- 172.21.15.136:0/3258564171 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498800bd60 con 0x7f49980b4e70 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 -- 172.21.15.136:0/3258564171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 msgr2=0x7f49980ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 --2- 172.21.15.136:0/3258564171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 0x7f49980ccff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49880099b0 tx=0x7f4988009250 comp rx=0 tx=0).stop 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 -- 172.21.15.136:0/3258564171 shutdown_connections 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 --2- 172.21.15.136:0/3258564171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b4e70 0x7f49980ccff0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 --2- 172.21.15.136:0/3258564171 >> v2:172.21.15.92:3300/0 conn(0x7f49980b1b40 0x7f499815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 -- 172.21.15.136:0/3258564171 >> 172.21.15.136:0/3258564171 conn(0x7f49980d10c0 msgr2=0x7f49980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 -- 172.21.15.136:0/3258564171 shutdown_connections 2023-10-06T03:01:19.288 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.284+0000 7f499f04a700 1 -- 172.21.15.136:0/3258564171 wait complete. 2023-10-06T03:01:19.291 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.285+0000 7f499f04a700 1 Processor -- start 2023-10-06T03:01:19.291 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.285+0000 7f499f04a700 1 -- start start 2023-10-06T03:01:19.291 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499f04a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 0x7f49981449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.291 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499f04a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49980b1b40 0x7f499815f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.291 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499f04a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49980b4e70 0x7f499815fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499f04a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499816d280 con 0x7f49980b4e70 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499f04a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499816d560 con 0x7f49980b1b40 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499f04a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499816d840 con 0x7f49980b16d0 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499e048700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 0x7f49981449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499e048700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 0x7f49981449d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41088/0 (socket says 172.21.15.136:41088) 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499e048700 1 -- 172.21.15.136:0/1644098726 learned_addr learned my addr 172.21.15.136:0/1644098726 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499d847700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49980b1b40 0x7f499815f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.286+0000 7f499e849700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49980b4e70 0x7f499815fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499e048700 1 -- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49980b1b40 msgr2=0x7f499815f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499e048700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49980b1b40 0x7f499815f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499e048700 1 -- 172.21.15.136:0/1644098726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49980b4e70 msgr2=0x7f499815fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.292 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499e048700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49980b4e70 0x7f499815fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499e048700 1 -- 172.21.15.136:0/1644098726 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880094f0 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499d847700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49980b1b40 0x7f499815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499e048700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 0x7f49981449d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4988009290 tx=0x7f4988017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988011e00 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498801f070 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499816db20 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880266d0 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4998144f10 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4988026cc0 con 0x7f49980b16d0 2023-10-06T03:01:19.293 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.287+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f4988021070 con 0x7f49980b16d0 2023-10-06T03:01:19.294 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.288+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499816e040 con 0x7f49980b16d0 2023-10-06T03:01:19.294 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.290+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4988015020 con 0x7f49980b16d0 2023-10-06T03:01:19.294 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.293+0000 7f11fdf55180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:01:19.374 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.373+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c338a9f5-ade8-40aa-b512-dbe9e4c9d29b", "id": 4} v 0) v1 -- 0x7f499804bb80 con 0x7f49980b16d0 2023-10-06T03:01:19.381 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.380+0000 7f498e7fc700 1 -- 172.21.15.136:0/1644098726 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c338a9f5-ade8-40aa-b512-dbe9e4c9d29b", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f498801adf0 con 0x7f49980b16d0 2023-10-06T03:01:19.381 INFO:teuthology.orchestra.run.smithi136.stdout:4 2023-10-06T03:01:19.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 msgr2=0x7f49981449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 0x7f49981449d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4988009290 tx=0x7f4988017000 comp rx=0 tx=0).stop 2023-10-06T03:01:19.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 shutdown_connections 2023-10-06T03:01:19.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49980b4e70 0x7f499815fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.383 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49980b16d0 0x7f49981449d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 --2- 172.21.15.136:0/1644098726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49980b1b40 0x7f499815f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 >> 172.21.15.136:0/1644098726 conn(0x7f49980d10c0 msgr2=0x7f4998159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 shutdown_connections 2023-10-06T03:01:19.384 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.381+0000 7f499f04a700 1 -- 172.21.15.136:0/1644098726 wait complete. 2023-10-06T03:01:19.394 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 23217f93-1916-413e-8251-9258ed175816 5 2023-10-06T03:01:19.404 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:19.403+0000 7fdd950bd180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:01:19.476 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.475+0000 7f11fdf55180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:01:19.481 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:19.479+0000 7fdd950bd180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:01:19.510 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.508+0000 7f5c4b303700 1 Processor -- start 2023-10-06T03:01:19.510 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.508+0000 7f5c4b303700 1 -- start start 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c4b303700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5c4406b520 0x7f5c440cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c4b303700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440cc210 con 0x7f5c4406aa40 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c4b303700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440cc4f0 con 0x7f5c440c91e0 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c4b303700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c440cc7d0 con 0x7f5c4406b520 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c4889e700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f5c440c91e0 0x7f5c4406ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:37428/0 (socket says 172.21.15.136:37428) 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c4889e700 1 -- 172.21.15.136:0/4281945280 learned_addr learned my addr 172.21.15.136:0/4281945280 (peer_addr_for_me v1:172.21.15.136:0/0) 2023-10-06T03:01:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c498a0700 1 --2- 172.21.15.136:0/4281945280 >> v2:172.21.15.92:3300/0 conn(0x7f5c4406b520 0x7f5c440cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c3bfff700 1 -- 172.21.15.136:0/4281945280 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365306667 0 0) 0x7f5c440cc4f0 con 0x7f5c440c91e0 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c3bfff700 1 -- 172.21.15.136:0/4281945280 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c30003570 con 0x7f5c440c91e0 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c498a0700 1 -- 172.21.15.136:0/4281945280 >> v1:172.21.15.136:6789/0 conn(0x7f5c440c91e0 legacy=0x7f5c4406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c498a0700 1 -- 172.21.15.136:0/4281945280 >> v1:172.21.15.92:6789/0 conn(0x7f5c4406aa40 legacy=0x7f5c4406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.509+0000 7f5c498a0700 1 -- 172.21.15.136:0/4281945280 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440ccab0 con 0x7f5c4406b520 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c498a0700 1 --2- 172.21.15.136:0/4281945280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c440cbc00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009e60 tx=0x7f5c3c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c65674b5953ff2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c3bfff700 1 -- 172.21.15.136:0/4281945280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00e070 con 0x7f5c4406b520 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c3bfff700 1 -- 172.21.15.136:0/4281945280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00bb60 con 0x7f5c4406b520 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c3bfff700 1 -- 172.21.15.136:0/4281945280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c012430 con 0x7f5c4406b520 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 -- 172.21.15.136:0/4281945280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 msgr2=0x7f5c440cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 --2- 172.21.15.136:0/4281945280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c440cbc00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009e60 tx=0x7f5c3c009780 comp rx=0 tx=0).stop 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 -- 172.21.15.136:0/4281945280 shutdown_connections 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 --2- 172.21.15.136:0/4281945280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c440cbc00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 -- 172.21.15.136:0/4281945280 >> 172.21.15.136:0/4281945280 conn(0x7f5c440cd670 msgr2=0x7f5c440cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 -- 172.21.15.136:0/4281945280 shutdown_connections 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.510+0000 7f5c4b303700 1 -- 172.21.15.136:0/4281945280 wait complete. 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.511+0000 7f5c4b303700 1 Processor -- start 2023-10-06T03:01:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.511+0000 7f5c4b303700 1 -- start start 2023-10-06T03:01:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4b303700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 0x7f5c44145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4b303700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c4414dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4b303700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 0x7f5c44166260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4b303700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c441667a0 con 0x7f5c4406b520 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4b303700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4414f010 con 0x7f5c4406aa40 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4b303700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4414e400 con 0x7f5c440c91e0 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4909f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 0x7f5c44145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c498a0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 0x7f5c44166260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c498a0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 0x7f5c44166260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41102/0 (socket says 172.21.15.136:41102) 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c4909f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 0x7f5c44145280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38898/0 (socket says 172.21.15.136:38898) 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.512+0000 7f5c498a0700 1 -- 172.21.15.136:0/3493235440 learned_addr learned my addr 172.21.15.136:0/3493235440 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c4889e700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c4414dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c498a0700 1 -- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 msgr2=0x7f5c44145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c498a0700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 0x7f5c44145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c498a0700 1 -- 172.21.15.136:0/3493235440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 msgr2=0x7f5c4414dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c498a0700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c4414dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c498a0700 1 -- 172.21.15.136:0/3493235440 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c009a20 con 0x7f5c440c91e0 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c4909f700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 0x7f5c44145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c498a0700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 0x7f5c44166260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c013000 tx=0x7f5c3c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c027470 con 0x7f5c440c91e0 2023-10-06T03:01:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c0275f0 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4414e6e0 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0279b0 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4414eca0 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5c3c024070 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.513+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f5c3c01dd60 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.514+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4414f180 con 0x7f5c440c91e0 2023-10-06T03:01:19.520 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.516+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5c3c0466d0 con 0x7f5c440c91e0 2023-10-06T03:01:19.558 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.556+0000 7f11fdf55180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:01:19.601 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.599+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "23217f93-1916-413e-8251-9258ed175816", "id": 5} v 0) v1 -- 0x7f5c4404c3c0 con 0x7f5c440c91e0 2023-10-06T03:01:19.605 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.603+0000 7f5c397fa700 1 -- 172.21.15.136:0/3493235440 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23217f93-1916-413e-8251-9258ed175816", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c3c035030 con 0x7f5c440c91e0 2023-10-06T03:01:19.605 INFO:teuthology.orchestra.run.smithi136.stdout:5 2023-10-06T03:01:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 msgr2=0x7f5c44166260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 0x7f5c44166260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c013000 tx=0x7f5c3c019000 comp rx=0 tx=0).stop 2023-10-06T03:01:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 shutdown_connections 2023-10-06T03:01:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c440c91e0 0x7f5c44166260 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c4406aa40 0x7f5c44145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 --2- 172.21.15.136:0/3493235440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5c4406b520 0x7f5c4414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 >> 172.21.15.136:0/3493235440 conn(0x7f5c440cd670 msgr2=0x7f5c44150310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 shutdown_connections 2023-10-06T03:01:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.604+0000 7f5c4b303700 1 -- 172.21.15.136:0/3493235440 wait complete. 2023-10-06T03:01:19.618 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new d5701781-828b-4f78-ace3-f8445084bfeb 6 2023-10-06T03:01:19.619 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:19.617+0000 7fdd950bd180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:01:19.702 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.700+0000 7f11fdf55180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:01:19.714 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:19.713+0000 7fdd950bd180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:01:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.733+0000 7ff5005fe700 1 Processor -- start 2023-10-06T03:01:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.734+0000 7ff5005fe700 1 -- start start 2023-10-06T03:01:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.734+0000 7ff5005fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff4f815ac80 0x7ff4f8069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.734+0000 7ff5005fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4f8069cb0 0x7ff4f80c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.734+0000 7ff5005fe700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80af2f0 con 0x7ff4f815b050 2023-10-06T03:01:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.734+0000 7ff5005fe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80af5d0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.734+0000 7ff5005fe700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f806a0e0 con 0x7ff4f815ac80 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff4f815ac80 0x7ff4f8069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff4f815ac80 0x7ff4f8069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:38900/0 (socket says 172.21.15.136:38900) 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fdb99700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7ff4f815b050 0x7ff4f80afa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:37430/0 (socket says 172.21.15.136:37430) 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 -- 172.21.15.136:0/2798354574 learned_addr learned my addr 172.21.15.136:0/2798354574 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4feb9b700 1 --2- 172.21.15.136:0/2798354574 >> v2:172.21.15.92:3300/0 conn(0x7ff4f8069cb0 0x7ff4f80c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 -- 172.21.15.136:0/2798354574 >> v2:172.21.15.92:3300/0 conn(0x7ff4f8069cb0 msgr2=0x7ff4f80c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 --2- 172.21.15.136:0/2798354574 >> v2:172.21.15.92:3300/0 conn(0x7ff4f8069cb0 0x7ff4f80c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 -- 172.21.15.136:0/2798354574 >> v1:172.21.15.136:6789/0 conn(0x7ff4f815b050 legacy=0x7ff4f80afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 -- 172.21.15.136:0/2798354574 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80c8660 con 0x7ff4f815ac80 2023-10-06T03:01:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fd398700 1 -- 172.21.15.136:0/2798354574 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317176140 0 0) 0x7ff4f80af2f0 con 0x7ff4f815b050 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fe39a700 1 --2- 172.21.15.136:0/2798354574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815ac80 0x7ff4f8069770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4f806fdc0 tx=0x7ff4e8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab0cbaaef5733b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fd398700 1 -- 172.21.15.136:0/2798354574 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8009e70 con 0x7ff4f815ac80 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fd398700 1 -- 172.21.15.136:0/2798354574 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e800b860 con 0x7ff4f815ac80 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.735+0000 7ff4fd398700 1 -- 172.21.15.136:0/2798354574 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800bc00 con 0x7ff4f815ac80 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 -- 172.21.15.136:0/2798354574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815ac80 msgr2=0x7ff4f8069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 --2- 172.21.15.136:0/2798354574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815ac80 0x7ff4f8069770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4f806fdc0 tx=0x7ff4e8009250 comp rx=0 tx=0).stop 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 -- 172.21.15.136:0/2798354574 shutdown_connections 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 --2- 172.21.15.136:0/2798354574 >> v2:172.21.15.92:3300/0 conn(0x7ff4f8069cb0 0x7ff4f80c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 --2- 172.21.15.136:0/2798354574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815ac80 0x7ff4f8069770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 -- 172.21.15.136:0/2798354574 >> 172.21.15.136:0/2798354574 conn(0x7ff4f8068e10 msgr2=0x7ff4f80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 -- 172.21.15.136:0/2798354574 shutdown_connections 2023-10-06T03:01:19.741 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.736+0000 7ff5005fe700 1 -- 172.21.15.136:0/2798354574 wait complete. 2023-10-06T03:01:19.742 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.737+0000 7ff5005fe700 1 Processor -- start 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.737+0000 7ff5005fe700 1 -- start start 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff5005fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 0x7ff4f80acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff5005fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f815ac80 0x7ff4f80c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff5005fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815b050 0x7ff4f815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff5005fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80c49d0 con 0x7ff4f815ac80 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff5005fe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f815c6f0 con 0x7ff4f815b050 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff5005fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f815c9d0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4feb9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815b050 0x7ff4f815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 0x7ff4f80acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.743 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 0x7ff4f80acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41106/0 (socket says 172.21.15.136:41106) 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 -- 172.21.15.136:0/363214910 learned_addr learned my addr 172.21.15.136:0/363214910 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fdb99700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f815ac80 0x7ff4f80c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 -- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815b050 msgr2=0x7ff4f815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815b050 0x7ff4f815d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 -- 172.21.15.136:0/363214910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f815ac80 msgr2=0x7ff4f80c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f815ac80 0x7ff4f80c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 -- 172.21.15.136:0/363214910 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80094f0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4feb9b700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815b050 0x7ff4f815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4fe39a700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 0x7ff4f80acbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80094c0 tx=0x7ff4e8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.738+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800bbe0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8000c90 con 0x7ff4f8069cb0 2023-10-06T03:01:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f815ccb0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.745 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8011650 con 0x7ff4f8069cb0 2023-10-06T03:01:19.745 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80ad100 con 0x7ff4f8069cb0 2023-10-06T03:01:19.745 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff4e8011cc0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.745 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7ff4e802ee00 con 0x7ff4f8069cb0 2023-10-06T03:01:19.745 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.739+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f804c3c0 con 0x7ff4f8069cb0 2023-10-06T03:01:19.745 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.741+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff4e8033060 con 0x7ff4f8069cb0 2023-10-06T03:01:19.802 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:19.800+0000 7f11fdf55180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:01:19.825 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.823+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d5701781-828b-4f78-ace3-f8445084bfeb", "id": 6} v 0) v1 -- 0x7ff4f80c8940 con 0x7ff4f8069cb0 2023-10-06T03:01:19.830 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.829+0000 7ff4ee7fc700 1 -- 172.21.15.136:0/363214910 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5701781-828b-4f78-ace3-f8445084bfeb", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff4e8015020 con 0x7ff4f8069cb0 2023-10-06T03:01:19.831 INFO:teuthology.orchestra.run.smithi136.stdout:6 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 msgr2=0x7ff4f80acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 0x7ff4f80acbc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80094c0 tx=0x7ff4e8017000 comp rx=0 tx=0).stop 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 shutdown_connections 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4f815b050 0x7ff4f815d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4f8069cb0 0x7ff4f80acbc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 --2- 172.21.15.136:0/363214910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4f815ac80 0x7ff4f80c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 >> 172.21.15.136:0/363214910 conn(0x7ff4f8068e10 msgr2=0x7ff4f80b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 shutdown_connections 2023-10-06T03:01:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.830+0000 7ff5005fe700 1 -- 172.21.15.136:0/363214910 wait complete. 2023-10-06T03:01:19.843 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new d78abc86-39b3-454c-b242-09e42328acc6 7 2023-10-06T03:01:19.958 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.956+0000 7f64e62e2700 1 Processor -- start 2023-10-06T03:01:19.958 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.956+0000 7f64e62e2700 1 -- start start 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e62e2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f64e0159b10 0x7f64e00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e62e2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 0x7f64e015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e62e2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00c74b0 con 0x7f64e015d730 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e62e2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e00c8700 con 0x7f64e0159b10 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e62e2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00c9070 con 0x7f64e006b420 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 0x7f64e015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.960 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 0x7f64e015dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41122/0 (socket says 172.21.15.136:41122) 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 -- 172.21.15.136:0/4222468580 learned_addr learned my addr 172.21.15.136:0/4222468580 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64dffff700 1 --2- 172.21.15.136:0/4222468580 >> v2:172.21.15.92:3300/0 conn(0x7f64e0159b10 0x7f64e00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 -- 172.21.15.136:0/4222468580 >> v2:172.21.15.92:3300/0 conn(0x7f64e0159b10 msgr2=0x7f64e00cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 --2- 172.21.15.136:0/4222468580 >> v2:172.21.15.92:3300/0 conn(0x7f64e0159b10 0x7f64e00cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 -- 172.21.15.136:0/4222468580 >> v1:172.21.15.92:6789/0 conn(0x7f64e015d730 legacy=0x7f64e0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 -- 172.21.15.136:0/4222468580 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e015e0e0 con 0x7f64e006b420 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64e487f700 1 --2- 172.21.15.136:0/4222468580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 0x7f64e015dba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009f20 tx=0x7f64d8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9713e748db955f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64deffd700 1 -- 172.21.15.136:0/4222468580 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d800e070 con 0x7f64e006b420 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64deffd700 1 -- 172.21.15.136:0/4222468580 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d800ba60 con 0x7f64e006b420 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.957+0000 7f64deffd700 1 -- 172.21.15.136:0/4222468580 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80092e0 con 0x7f64e006b420 2023-10-06T03:01:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 -- 172.21.15.136:0/4222468580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 msgr2=0x7f64e015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 --2- 172.21.15.136:0/4222468580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 0x7f64e015dba0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009f20 tx=0x7f64d8009840 comp rx=0 tx=0).stop 2023-10-06T03:01:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 -- 172.21.15.136:0/4222468580 shutdown_connections 2023-10-06T03:01:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 --2- 172.21.15.136:0/4222468580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e006b420 0x7f64e015dba0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 --2- 172.21.15.136:0/4222468580 >> v2:172.21.15.92:3300/0 conn(0x7f64e0159b10 0x7f64e00cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 -- 172.21.15.136:0/4222468580 >> 172.21.15.136:0/4222468580 conn(0x7f64e00c9870 msgr2=0x7f64e00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 -- 172.21.15.136:0/4222468580 shutdown_connections 2023-10-06T03:01:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.958+0000 7f64e62e2700 1 -- 172.21.15.136:0/4222468580 wait complete. 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.959+0000 7f64e62e2700 1 Processor -- start 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.959+0000 7f64e62e2700 1 -- start start 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e62e2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64e006b420 0x7f64e0152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e62e2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e0159b10 0x7f64e00c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e62e2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 0x7f64e0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e62e2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0152870 con 0x7f64e006b420 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e62e2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00b0b70 con 0x7f64e0159b10 2023-10-06T03:01:19.966 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e62e2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00c71b0 con 0x7f64e015d730 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 0x7f64e0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 0x7f64e0154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:41124/0 (socket says 172.21.15.136:41124) 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 -- 172.21.15.136:0/1154585770 learned_addr learned my addr 172.21.15.136:0/1154585770 (peer_addr_for_me v2:172.21.15.136:0/0) 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64df7fe700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e0159b10 0x7f64e00c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64dffff700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64e006b420 0x7f64e0152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 -- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e0159b10 msgr2=0x7f64e00c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e0159b10 0x7f64e00c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 -- 172.21.15.136:0/1154585770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64e006b420 msgr2=0x7f64e0152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64e006b420 0x7f64e0152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64e487f700 1 -- 172.21.15.136:0/1154585770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d8009ae0 con 0x7f64e015d730 2023-10-06T03:01:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.960+0000 7f64df7fe700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e0159b10 0x7f64e00c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64e487f700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 0x7f64e0154220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f64d800b610 tx=0x7f64d8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80032f0 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d8008d20 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e00aff60 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d801d630 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e00b0520 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f64d8003040 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.961+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f64d801dd90 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.962+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e00b1640 con 0x7f64e015d730 2023-10-06T03:01:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:19.964+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f64d802c060 con 0x7f64e015d730 2023-10-06T03:01:20.062 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.061+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d78abc86-39b3-454c-b242-09e42328acc6", "id": 7} v 0) v1 -- 0x7f64e004c3c0 con 0x7f64e015d730 2023-10-06T03:01:20.070 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.069+0000 7f64c7fff700 1 -- 172.21.15.136:0/1154585770 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d78abc86-39b3-454c-b242-09e42328acc6", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f64d8022070 con 0x7f64e015d730 2023-10-06T03:01:20.071 INFO:teuthology.orchestra.run.smithi136.stdout:7 2023-10-06T03:01:20.072 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 msgr2=0x7f64e0154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:20.072 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 0x7f64e0154220 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f64d800b610 tx=0x7f64d8018000 comp rx=0 tx=0).stop 2023-10-06T03:01:20.073 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 shutdown_connections 2023-10-06T03:01:20.073 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f64e006b420 0x7f64e0152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.073 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e015d730 0x7f64e0154220 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.073 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 --2- 172.21.15.136:0/1154585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e0159b10 0x7f64e00c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.073 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 >> 172.21.15.136:0/1154585770 conn(0x7f64e00c9870 msgr2=0x7f64e00c7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:20.073 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 shutdown_connections 2023-10-06T03:01:20.074 INFO:teuthology.orchestra.run.smithi136.stderr:2023-10-06T03:01:20.070+0000 7f64e62e2700 1 -- 172.21.15.136:0/1154585770 wait complete. 2023-10-06T03:01:20.088 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T03:01:20.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T03:01:20.090 INFO:tasks.ceph.osd.0:Started 2023-10-06T03:01:20.090 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T03:01:20.091 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T03:01:20.093 INFO:tasks.ceph.osd.1:Started 2023-10-06T03:01:20.093 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T03:01:20.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T03:01:20.095 INFO:tasks.ceph.osd.2:Started 2023-10-06T03:01:20.095 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T03:01:20.096 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T03:01:20.097 INFO:tasks.ceph.osd.3:Started 2023-10-06T03:01:20.098 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T03:01:20.098 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T03:01:20.100 INFO:tasks.ceph.osd.4:Started 2023-10-06T03:01:20.100 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T03:01:20.100 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T03:01:20.102 INFO:tasks.ceph.osd.5:Started 2023-10-06T03:01:20.102 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T03:01:20.103 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T03:01:20.105 INFO:tasks.ceph.osd.6:Started 2023-10-06T03:01:20.105 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T03:01:20.105 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T03:01:20.107 INFO:tasks.ceph.osd.7:Started 2023-10-06T03:01:20.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.249+0000 7f0ad4053700 1 Processor -- start 2023-10-06T03:01:20.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.250+0000 7f0ad4053700 1 -- start start 2023-10-06T03:01:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.250+0000 7f0ad4053700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0acc0aca80 0x7f0acc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.250+0000 7f0ad4053700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.250+0000 7f0ad4053700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc06bdb0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.250+0000 7f0ad4053700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0acc0c8670 con 0x7f0acc0aca80 2023-10-06T03:01:20.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.250+0000 7f0ad4053700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc0cb370 con 0x7f0acc069d60 2023-10-06T03:01:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad25f0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0acc0c81f0 0x7f0acc068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41740/0 (socket says 172.21.15.92:41740) 2023-10-06T03:01:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad25f0700 1 -- 172.21.15.92:0/4071104845 learned_addr learned my addr 172.21.15.92:0/4071104845 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad1def700 1 --2- 172.21.15.92:0/4071104845 >> v2:172.21.15.136:3300/0 conn(0x7f0acc0aca80 0x7f0acc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad15ee700 1 --2- 172.21.15.92:0/4071104845 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:20.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad0ded700 1 -- 172.21.15.92:0/4071104845 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053003222 0 0) 0x7f0acc06bdb0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad0ded700 1 -- 172.21.15.92:0/4071104845 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab8003570 con 0x7f0acc0c81f0 2023-10-06T03:01:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad15ee700 1 -- 172.21.15.92:0/4071104845 >> v2:172.21.15.136:3300/0 conn(0x7f0acc0aca80 msgr2=0x7f0acc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad15ee700 1 --2- 172.21.15.92:0/4071104845 >> v2:172.21.15.136:3300/0 conn(0x7f0acc0aca80 0x7f0acc06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad15ee700 1 -- 172.21.15.92:0/4071104845 >> v1:172.21.15.92:6789/0 conn(0x7f0acc0c81f0 legacy=0x7f0acc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad15ee700 1 -- 172.21.15.92:0/4071104845 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc159f60 con 0x7f0acc069d60 2023-10-06T03:01:20.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad1def700 1 --2- 172.21.15.92:0/4071104845 >> v2:172.21.15.136:3300/0 conn(0x7f0acc0aca80 0x7f0acc06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad15ee700 1 --2- 172.21.15.92:0/4071104845 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc0c7cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80098e0 tx=0x7f0ac80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d3675675009aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad0ded700 1 -- 172.21.15.92:0/4071104845 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8011630 con 0x7f0acc069d60 2023-10-06T03:01:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad0ded700 1 -- 172.21.15.92:0/4071104845 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac8011790 con 0x7f0acc069d60 2023-10-06T03:01:20.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.251+0000 7f0ad0ded700 1 -- 172.21.15.92:0/4071104845 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8011b30 con 0x7f0acc069d60 2023-10-06T03:01:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 -- 172.21.15.92:0/4071104845 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 msgr2=0x7f0acc0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:20.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 --2- 172.21.15.92:0/4071104845 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc0c7cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80098e0 tx=0x7f0ac80091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 -- 172.21.15.92:0/4071104845 shutdown_connections 2023-10-06T03:01:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 --2- 172.21.15.92:0/4071104845 >> v2:172.21.15.136:3300/0 conn(0x7f0acc0aca80 0x7f0acc06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 --2- 172.21.15.92:0/4071104845 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc0c7cb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 -- 172.21.15.92:0/4071104845 >> 172.21.15.92:0/4071104845 conn(0x7f0acc0cd670 msgr2=0x7f0acc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:20.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 -- 172.21.15.92:0/4071104845 shutdown_connections 2023-10-06T03:01:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.252+0000 7f0ad4053700 1 -- 172.21.15.92:0/4071104845 wait complete. 2023-10-06T03:01:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.253+0000 7f0ad4053700 1 Processor -- start 2023-10-06T03:01:20.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.253+0000 7f0ad4053700 1 -- start start 2023-10-06T03:01:20.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad4053700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:20.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad4053700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0acc0aca80 0x7f0acc150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:20.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad4053700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 0x7f0acc16d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:20.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad4053700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc0ad8b0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad4053700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc0afe10 con 0x7f0acc0aca80 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad4053700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc157fb0 con 0x7f0acc069d60 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 0x7f0acc16d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad1def700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 0x7f0acc16d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36032/0 (socket says 172.21.15.92:36032) 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 -- 172.21.15.92:0/61679905 learned_addr learned my addr 172.21.15.92:0/61679905 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad15ee700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0acc0aca80 0x7f0acc150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 -- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 msgr2=0x7f0acc150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 -- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0acc0aca80 msgr2=0x7f0acc150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0acc0aca80 0x7f0acc150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.254+0000 7f0ad25f0700 1 -- 172.21.15.92:0/61679905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc0093c0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ad25f0700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 0x7f0acc16d670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4001460 tx=0x7f0ac4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400a870 con 0x7f0acc0c81f0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac400a9d0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac400ad50 con 0x7f0acc0c81f0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac8009420 con 0x7f0acc0c81f0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0ac4007580 con 0x7f0acc0c81f0 2023-10-06T03:01:20.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc0ae0f0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.255+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f0ac4007a10 con 0x7f0acc0c81f0 2023-10-06T03:01:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.256+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc04c3c0 con 0x7f0acc0c81f0 2023-10-06T03:01:20.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.258+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0ac4024430 con 0x7f0acc0c81f0 2023-10-06T03:01:20.277 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:20.275+0000 7fdd950bd180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:01:20.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.326+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0acc06b510 con 0x7f0acc0c81f0 2023-10-06T03:01:20.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.326+0000 7f0ac1ffb700 1 -- 172.21.15.92:0/61679905 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f0ac4012400 con 0x7f0acc0c81f0 2023-10-06T03:01:20.329 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:20.329 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:20.063609+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:20.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.327+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 msgr2=0x7f0acc16d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:20.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.327+0000 7f0ad4053700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 0x7f0acc16d670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4001460 tx=0x7f0ac4001a40 comp rx=0 tx=0).stop 2023-10-06T03:01:20.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 shutdown_connections 2023-10-06T03:01:20.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0acc0c81f0 0x7f0acc16d670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0acc069d60 0x7f0acc150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 --2- 172.21.15.92:0/61679905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0acc0aca80 0x7f0acc150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:20.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 >> 172.21.15.92:0/61679905 conn(0x7f0acc0cd670 msgr2=0x7f0acc0c2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:20.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 shutdown_connections 2023-10-06T03:01:20.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:20.328+0000 7f0ad4053700 1 -- 172.21.15.92:0/61679905 wait complete. 2023-10-06T03:01:20.342 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T03:01:20.342 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T03:01:20.357 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:20.355+0000 7fdd950bd180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:01:20.359 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:20.358+0000 7f11fdf55180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:01:20.435 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:20.433+0000 7fdd950bd180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:01:20.437 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:20.435+0000 7f11fdf55180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:01:20.515 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:20.514+0000 7f11fdf55180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:01:20.703 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T03:01:20.701+0000 7fd132700540 -1 Falling back to public interface 2023-10-06T03:01:20.705 INFO:tasks.ceph.osd.5.smithi136.stderr:2023-10-06T03:01:20.703+0000 7f9bc111f540 -1 Falling back to public interface 2023-10-06T03:01:20.709 INFO:tasks.ceph.osd.6.smithi136.stderr:2023-10-06T03:01:20.707+0000 7f2cfcf0f540 -1 Falling back to public interface 2023-10-06T03:01:20.711 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T03:01:20.709+0000 7f0aefb78540 -1 Falling back to public interface 2023-10-06T03:01:20.711 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T03:01:20.710+0000 7fdb3f905540 -1 Falling back to public interface 2023-10-06T03:01:20.711 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T03:01:20.710+0000 7f7a527b3540 -1 Falling back to public interface 2023-10-06T03:01:20.712 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T03:01:20.710+0000 7fdb39d2e700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6811/0: (98) Address already in use 2023-10-06T03:01:20.712 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T03:01:20.710+0000 7fdb39d2e700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6815/0: (98) Address already in use 2023-10-06T03:01:20.717 INFO:tasks.ceph.osd.4.smithi136.stderr:2023-10-06T03:01:20.715+0000 7fee30296540 -1 Falling back to public interface 2023-10-06T03:01:20.722 INFO:tasks.ceph.osd.7.smithi136.stderr:2023-10-06T03:01:20.721+0000 7f6e8eaa0540 -1 Falling back to public interface 2023-10-06T03:01:21.017 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:21.015+0000 7fdd950bd180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:01:21.094 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:21.092+0000 7f11fdf55180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:01:21.147 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:21.154 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:21.152+0000 7fdd950bd180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:01:21.231 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:21.230+0000 7f11fdf55180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:01:21.254 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.252+0000 7ff9fd182700 1 Processor -- start 2023-10-06T03:01:21.255 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 -- start start 2023-10-06T03:01:21.260 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff9f80af350 0x7ff9f80ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:21.260 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9f80ad420 0x7ff9f80b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:21.260 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80af720 0x7ff9f80afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80ae560 con 0x7ff9f80ad420 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80ad950 con 0x7ff9f80af350 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.253+0000 7ff9fd182700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80adc30 con 0x7ff9f80af720 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9f80ad420 0x7ff9f80b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9f80ad420 0x7ff9f80b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36034/0 (socket says 172.21.15.92:36034) 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 -- 172.21.15.92:0/1729588266 learned_addr learned my addr 172.21.15.92:0/1729588266 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f759e700 1 --2- 172.21.15.92:0/1729588266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80af720 0x7ff9f80afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f6d9d700 1 --2- 172.21.15.92:0/1729588266 >> v2:172.21.15.136:3300/0 conn(0x7ff9f80af350 0x7ff9f80ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 -- 172.21.15.92:0/1729588266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80af720 msgr2=0x7ff9f80afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 --2- 172.21.15.92:0/1729588266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80af720 0x7ff9f80afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 -- 172.21.15.92:0/1729588266 >> v2:172.21.15.136:3300/0 conn(0x7ff9f80af350 msgr2=0x7ff9f80ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:21.261 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 --2- 172.21.15.92:0/1729588266 >> v2:172.21.15.136:3300/0 conn(0x7ff9f80af350 0x7ff9f80ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 -- 172.21.15.92:0/1729588266 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80adf10 con 0x7ff9f80ad420 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f659c700 1 --2- 172.21.15.92:0/1729588266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80ad420 0x7ff9f80b15f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8088c10 tx=0x7ff9e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14db19f225d0bbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f5d9b700 1 -- 172.21.15.92:0/1729588266 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000b640 con 0x7ff9f80ad420 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f5d9b700 1 -- 172.21.15.92:0/1729588266 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e000b7a0 con 0x7ff9f80ad420 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.254+0000 7ff9f5d9b700 1 -- 172.21.15.92:0/1729588266 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e000bb60 con 0x7ff9f80ad420 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 -- 172.21.15.92:0/1729588266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80ad420 msgr2=0x7ff9f80b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 --2- 172.21.15.92:0/1729588266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80ad420 0x7ff9f80b15f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8088c10 tx=0x7ff9e0009250 comp rx=0 tx=0).stop 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 -- 172.21.15.92:0/1729588266 shutdown_connections 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 --2- 172.21.15.92:0/1729588266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80af720 0x7ff9f80afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.262 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 --2- 172.21.15.92:0/1729588266 >> v2:172.21.15.136:3300/0 conn(0x7ff9f80af350 0x7ff9f80ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 --2- 172.21.15.92:0/1729588266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80ad420 0x7ff9f80b15f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 -- 172.21.15.92:0/1729588266 >> 172.21.15.92:0/1729588266 conn(0x7ff9f8099ab0 msgr2=0x7ff9f806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 -- 172.21.15.92:0/1729588266 shutdown_connections 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.255+0000 7ff9fd182700 1 -- 172.21.15.92:0/1729588266 wait complete. 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.256+0000 7ff9fd182700 1 Processor -- start 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.256+0000 7ff9fd182700 1 -- start start 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80ad420 0x7ff9f80cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9f80af350 0x7ff9f8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 0x7ff9f81521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f809d960 con 0x7ff9f80af720 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f809dc40 con 0x7ff9f80af350 2023-10-06T03:01:21.263 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f809df20 con 0x7ff9f80ad420 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 0x7ff9f81521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f6d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80ad420 0x7ff9f80cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 0x7ff9f81521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36042/0 (socket says 172.21.15.92:36042) 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 -- 172.21.15.92:0/944088821 learned_addr learned my addr 172.21.15.92:0/944088821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f659c700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9f80af350 0x7ff9f8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 -- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80ad420 msgr2=0x7ff9f80cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80ad420 0x7ff9f80cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 -- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9f80af350 msgr2=0x7ff9f8151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9f80af350 0x7ff9f8151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 -- 172.21.15.92:0/944088821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e00094f0 con 0x7ff9f80af720 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9f759e700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 0x7ff9f81521a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009df0 tx=0x7ff9f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:21.264 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000caa0 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000f040 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000ce70 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.257+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f809e200 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.258+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f80cd7e0 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.258+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ff9f0014690 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.258+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7ff9f0013ac0 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.258+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f80cd9d0 con 0x7ff9f80af720 2023-10-06T03:01:21.265 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.261+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ff9f0018020 con 0x7ff9f80af720 2023-10-06T03:01:21.266 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:21.262+0000 7fdd950bd180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:01:21.320 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:21.318+0000 7f11fdf55180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:01:21.332 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.330+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9f80ae1f0 con 0x7ff9f80af720 2023-10-06T03:01:21.332 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.331+0000 7ff9eeffd700 1 -- 172.21.15.92:0/944088821 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7ff9f0013e60 con 0x7ff9f80af720 2023-10-06T03:01:21.333 INFO:teuthology.misc.health.smithi092.stdout: 2023-10-06T03:01:21.333 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":10,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:20.063609+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 msgr2=0x7ff9f81521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 0x7ff9f81521a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009df0 tx=0x7ff9f000b040 comp rx=0 tx=0).stop 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 shutdown_connections 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9f80af720 0x7ff9f81521a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9f80ad420 0x7ff9f80cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 --2- 172.21.15.92:0/944088821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9f80af350 0x7ff9f8151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 >> 172.21.15.92:0/944088821 conn(0x7ff9f8099ab0 msgr2=0x7ff9f8158ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:21.334 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 shutdown_connections 2023-10-06T03:01:21.335 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:21.332+0000 7ff9fd182700 1 -- 172.21.15.92:0/944088821 wait complete. 2023-10-06T03:01:21.345 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T03:01:21.514 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:21.512+0000 7fdd950bd180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:01:21.572 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:21.570+0000 7f11fdf55180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:01:21.593 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:21.591+0000 7fdd950bd180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:01:21.650 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:21.649+0000 7f11fdf55180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:01:22.423 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:22.422+0000 7fdd950bd180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:01:22.469 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:22.467+0000 7f11fdf55180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:01:22.496 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:22.494+0000 7fdd950bd180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:01:22.543 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:22.541+0000 7f11fdf55180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:01:22.569 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:22.567+0000 7fdd950bd180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:01:22.617 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:22.615+0000 7f11fdf55180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:01:22.702 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:22.700+0000 7fdd950bd180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:01:22.751 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:22.749+0000 7f11fdf55180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:01:22.786 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:22.784+0000 7fdd950bd180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:01:22.834 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:22.833+0000 7f11fdf55180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:01:22.937 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:22.935+0000 7fdd950bd180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:01:22.987 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:22.986+0000 7f11fdf55180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:01:23.013 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:23.010+0000 7fdd950bd180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:01:23.062 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:23.061+0000 7f11fdf55180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:01:23.085 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:23.084+0000 7fdd950bd180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:01:23.137 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:23.136+0000 7f11fdf55180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:01:23.161 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:23.160+0000 7fdd950bd180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:01:23.221 INFO:tasks.ceph.mgr.x.smithi136.stderr:2023-10-06T03:01:23.220+0000 7f11fdf55180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:01:24.185 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:24.183+0000 7fdd4e4be700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T03:01:24.185 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:01:24.184+0000 7fdd4e4be700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T03:01:24.201 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T03:01:24.199+0000 7fd132700540 -1 osd.1 0 log_to_monitors true 2023-10-06T03:01:24.246 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T03:01:24.244+0000 7fdb3f905540 -1 osd.2 0 log_to_monitors true 2023-10-06T03:01:24.246 INFO:tasks.ceph.osd.6.smithi136.stderr:2023-10-06T03:01:24.245+0000 7f2cfcf0f540 -1 osd.6 0 log_to_monitors true 2023-10-06T03:01:24.255 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T03:01:24.254+0000 7f7a527b3540 -1 osd.3 0 log_to_monitors true 2023-10-06T03:01:24.268 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T03:01:24.267+0000 7f0aefb78540 -1 osd.0 0 log_to_monitors true 2023-10-06T03:01:24.274 INFO:tasks.ceph.osd.4.smithi136.stderr:2023-10-06T03:01:24.272+0000 7fee30296540 -1 osd.4 0 log_to_monitors true 2023-10-06T03:01:24.281 INFO:tasks.ceph.osd.7.smithi136.stderr:2023-10-06T03:01:24.280+0000 7f6e8eaa0540 -1 osd.7 0 log_to_monitors true 2023-10-06T03:01:24.285 INFO:tasks.ceph.osd.5.smithi136.stderr:2023-10-06T03:01:24.284+0000 7f9bc111f540 -1 osd.5 0 log_to_monitors true 2023-10-06T03:01:27.625 INFO:tasks.ceph.osd.4.smithi136.stderr:2023-10-06T03:01:27.624+0000 7fee26eb8700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T03:01:27.630 INFO:tasks.ceph.osd.4.smithi136.stderr:2023-10-06T03:01:27.629+0000 7fee1fcaf700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:27.657 INFO:tasks.ceph.osd.5.smithi136.stderr:2023-10-06T03:01:27.655+0000 7f9bb7d41700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T03:01:27.661 INFO:tasks.ceph.osd.5.smithi136.stderr:2023-10-06T03:01:27.660+0000 7f9bb0b38700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:27.666 INFO:tasks.ceph.osd.7.smithi136.stderr:2023-10-06T03:01:27.665+0000 7f6e856c2700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T03:01:27.670 INFO:tasks.ceph.osd.7.smithi136.stderr:2023-10-06T03:01:27.669+0000 7f6e7e4b9700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:27.681 INFO:tasks.ceph.osd.6.smithi136.stderr:2023-10-06T03:01:27.679+0000 7f2cf3b31700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T03:01:27.684 INFO:tasks.ceph.osd.6.smithi136.stderr:2023-10-06T03:01:27.682+0000 7f2cec928700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:28.152 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:28.193 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T03:01:28.192+0000 7fd129322700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T03:01:28.201 INFO:tasks.ceph.osd.1.smithi092.stderr:2023-10-06T03:01:28.200+0000 7fd122119700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:28.262 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T03:01:28.260+0000 7f0ae679a700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T03:01:28.271 INFO:tasks.ceph.osd.0.smithi092.stderr:2023-10-06T03:01:28.270+0000 7f0adf591700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:28.282 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T03:01:28.281+0000 7f7a493d5700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T03:01:28.288 INFO:tasks.ceph.osd.3.smithi092.stderr:2023-10-06T03:01:28.286+0000 7f7a421cc700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:28.288 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T03:01:28.286+0000 7fdb36527700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T03:01:28.293 INFO:tasks.ceph.osd.2.smithi092.stderr:2023-10-06T03:01:28.291+0000 7fdb2f31e700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:01:28.308 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.306+0000 7fbad2bf3700 1 Processor -- start 2023-10-06T03:01:28.308 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.306+0000 7fbad2bf3700 1 -- start start 2023-10-06T03:01:28.310 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad2bf3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0c9a20 0x7fbacc0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.310 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad2bf3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbacc0ce1f0 0x7fbacc0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.310 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad2bf3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbacc0c9df0 0x7fbacc15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.310 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad2bf3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc0cdd60 con 0x7fbacc0c9df0 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad2bf3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbacc0cd470 con 0x7fbacc0ce1f0 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad2bf3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc157da0 con 0x7fbacc0c9a20 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad1190700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbacc0c9df0 0x7fbacc15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad1190700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbacc0c9df0 0x7fbacc15b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36174/0 (socket says 172.21.15.92:36174) 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.307+0000 7fbad1190700 1 -- 172.21.15.92:0/3024070799 learned_addr learned my addr 172.21.15.92:0/3024070799 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbacbfff700 1 --2- 172.21.15.92:0/3024070799 >> v2:172.21.15.136:3300/0 conn(0x7fbacc0ce1f0 0x7fbacc0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad098f700 1 --2- 172.21.15.92:0/3024070799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0c9a20 0x7fbacc0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad1190700 1 -- 172.21.15.92:0/3024070799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0c9a20 msgr2=0x7fbacc0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad1190700 1 --2- 172.21.15.92:0/3024070799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0c9a20 0x7fbacc0b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad1190700 1 -- 172.21.15.92:0/3024070799 >> v2:172.21.15.136:3300/0 conn(0x7fbacc0ce1f0 msgr2=0x7fbacc0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:28.311 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad1190700 1 --2- 172.21.15.92:0/3024070799 >> v2:172.21.15.136:3300/0 conn(0x7fbacc0ce1f0 0x7fbacc0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad1190700 1 -- 172.21.15.92:0/3024070799 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc0cd710 con 0x7fbacc0c9df0 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbacbfff700 1 --2- 172.21.15.92:0/3024070799 >> v2:172.21.15.136:3300/0 conn(0x7fbacc0ce1f0 0x7fbacc0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad1190700 1 --2- 172.21.15.92:0/3024070799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc15b5f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0aff90 tx=0x7fbac4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f6c0c60bfef23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbacb7fe700 1 -- 172.21.15.92:0/3024070799 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac400e070 con 0x7fbacc0c9df0 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbacb7fe700 1 -- 172.21.15.92:0/3024070799 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac400ba50 con 0x7fbacc0c9df0 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbacb7fe700 1 -- 172.21.15.92:0/3024070799 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40092e0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad2bf3700 1 -- 172.21.15.92:0/3024070799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 msgr2=0x7fbacc15b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/3024070799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc15b5f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0aff90 tx=0x7fbac4009840 comp rx=0 tx=0).stop 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.308+0000 7fbad2bf3700 1 -- 172.21.15.92:0/3024070799 shutdown_connections 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.309+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/3024070799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0c9a20 0x7fbacc0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.309+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/3024070799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc15b5f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.312 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.309+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/3024070799 >> v2:172.21.15.136:3300/0 conn(0x7fbacc0ce1f0 0x7fbacc0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.313 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.309+0000 7fbad2bf3700 1 -- 172.21.15.92:0/3024070799 >> 172.21.15.92:0/3024070799 conn(0x7fbacc067610 msgr2=0x7fbacc067a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:28.313 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.309+0000 7fbad2bf3700 1 -- 172.21.15.92:0/3024070799 shutdown_connections 2023-10-06T03:01:28.313 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.309+0000 7fbad2bf3700 1 -- 172.21.15.92:0/3024070799 wait complete. 2023-10-06T03:01:28.315 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.310+0000 7fbad2bf3700 1 Processor -- start 2023-10-06T03:01:28.315 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.310+0000 7fbad2bf3700 1 -- start start 2023-10-06T03:01:28.315 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.310+0000 7fbad2bf3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc0c9a20 0x7fbacc0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.315 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.310+0000 7fbad2bf3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc0c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.315 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.310+0000 7fbad2bf3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0ce1f0 0x7fbacc06acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.310+0000 7fbad2bf3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc14f030 con 0x7fbacc0c9df0 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbad2bf3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc14f1c0 con 0x7fbacc0c9a20 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbad2bf3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc0c75d0 con 0x7fbacc0ce1f0 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc0c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc0c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36186/0 (socket says 172.21.15.92:36186) 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 -- 172.21.15.92:0/2329819468 learned_addr learned my addr 172.21.15.92:0/2329819468 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbad098f700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc0c9a20 0x7fbacc0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbad1190700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0ce1f0 0x7fbacc06acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 -- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0ce1f0 msgr2=0x7fbacc06acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0ce1f0 0x7fbacc06acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 -- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc0c9a20 msgr2=0x7fbacc0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc0c9a20 0x7fbacc0b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.316 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 -- 172.21.15.92:0/2329819468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4009ae0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbacbfff700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc0c7090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbab800d8c0 tx=0x7fbab8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab80125f0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab8012750 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8012b10 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc0c7850 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.311+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc0c7e10 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.312+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc069f10 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.312+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fbab8014430 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.313+0000 7fbac8ff9700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbab0066ea0 0x7fbab0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.313+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fbab80838f0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.313+0000 7fbad098f700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbab0066ea0 0x7fbab0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:28.317 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.313+0000 7fbad098f700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbab0066ea0 0x7fbab0069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbac0006fd0 tx=0x7fbac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:28.319 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.317+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbab80519d0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.502 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.500+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbacc0c81b0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.502 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.501+0000 7fbac8ff9700 1 -- 172.21.15.92:0/2329819468 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fbab8054ff0 con 0x7fbacc0c9df0 2023-10-06T03:01:28.503 INFO:teuthology.misc.health.smithi092.stdout: 2023-10-06T03:01:28.503 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":12,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:26.186140+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbab0066ea0 msgr2=0x7fbab0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbab0066ea0 0x7fbab0069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbac0006fd0 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 msgr2=0x7fbacc0c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc0c7090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbab800d8c0 tx=0x7fbab8008040 comp rx=0 tx=0).stop 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 shutdown_connections 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbab0066ea0 0x7fbab0069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbacc0ce1f0 0x7fbacc06acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.505 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbacc0c9a20 0x7fbacc0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.506 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 --2- 172.21.15.92:0/2329819468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbacc0c9df0 0x7fbacc0c7090 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:28.506 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 >> 172.21.15.92:0/2329819468 conn(0x7fbacc067610 msgr2=0x7fbacc154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:28.506 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 shutdown_connections 2023-10-06T03:01:28.506 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:28.503+0000 7fbad2bf3700 1 -- 172.21.15.92:0/2329819468 wait complete. 2023-10-06T03:01:28.516 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T03:01:35.322 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:35.435 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.434+0000 7f2c4e2ad700 1 Processor -- start 2023-10-06T03:01:35.435 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.434+0000 7f2c4e2ad700 1 -- start start 2023-10-06T03:01:35.437 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c4e2ad700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c480ad420 0x7f2c480b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.437 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c4e2ad700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480ae560 con 0x7f2c480af720 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c4e2ad700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480af100 con 0x7f2c480af350 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c4e2ad700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c480ad950 con 0x7f2c480ad420 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c480ad420 0x7f2c480b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c480ad420 0x7f2c480b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59122/0 (socket says 172.21.15.92:59122) 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c477fe700 1 -- 172.21.15.92:0/1989320730 learned_addr learned my addr 172.21.15.92:0/1989320730 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.435+0000 7f2c4c84a700 1 --1- 172.21.15.92:0/1989320730 >> v1:172.21.15.92:6789/0 conn(0x7f2c480af720 0x7f2c480afb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53342/0 (socket says 172.21.15.92:53342) 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c477fe700 1 -- 172.21.15.92:0/1989320730 >> v1:172.21.15.136:6789/0 conn(0x7f2c480af350 legacy=0x7f2c480ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c477fe700 1 -- 172.21.15.92:0/1989320730 >> v1:172.21.15.92:6789/0 conn(0x7f2c480af720 legacy=0x7f2c480afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c46ffd700 1 -- 172.21.15.92:0/1989320730 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277288177 0 0) 0x7f2c480ae560 con 0x7f2c480af720 2023-10-06T03:01:35.438 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c477fe700 1 -- 172.21.15.92:0/1989320730 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c480adc30 con 0x7f2c480ad420 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c477fe700 1 --2- 172.21.15.92:0/1989320730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480ad420 0x7f2c480b15f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0099b0 tx=0x7f2c3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac17fb220495953 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c46ffd700 1 -- 172.21.15.92:0/1989320730 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00b860 con 0x7f2c480ad420 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c46ffd700 1 -- 172.21.15.92:0/1989320730 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c00b9c0 con 0x7f2c480ad420 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c46ffd700 1 -- 172.21.15.92:0/1989320730 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00bd60 con 0x7f2c480ad420 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1989320730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480ad420 msgr2=0x7f2c480b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1989320730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480ad420 0x7f2c480b15f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0099b0 tx=0x7f2c3c009250 comp rx=0 tx=0).stop 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1989320730 shutdown_connections 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1989320730 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480ad420 0x7f2c480b15f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1989320730 >> 172.21.15.92:0/1989320730 conn(0x7f2c48099ab0 msgr2=0x7f2c4806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.436+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1989320730 shutdown_connections 2023-10-06T03:01:35.439 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.437+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1989320730 wait complete. 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 Processor -- start 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 -- start start 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c480ad420 0x7f2c4815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 0x7f2c480b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c480af720 0x7f2c4809b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4815b950 con 0x7f2c480af350 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4815ade0 con 0x7f2c480ad420 2023-10-06T03:01:35.442 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.438+0000 7f2c4e2ad700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4815b0b0 con 0x7f2c480af720 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 0x7f2c480b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 0x7f2c480b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59124/0 (socket says 172.21.15.92:59124) 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 -- 172.21.15.92:0/1828950518 learned_addr learned my addr 172.21.15.92:0/1828950518 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c47fff700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c480ad420 0x7f2c4815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c4c84a700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c480af720 0x7f2c4809b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 -- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c480af720 msgr2=0x7f2c4809b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c480af720 0x7f2c4809b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 -- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c480ad420 msgr2=0x7f2c4815a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c480ad420 0x7f2c4815a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 -- 172.21.15.92:0/1828950518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0094f0 con 0x7f2c480af350 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c4c84a700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c480af720 0x7f2c4809b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:35.443 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c477fe700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 0x7f2c480b05b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c006f50 tx=0x7f2c3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c01f070 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c011c60 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4815b390 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c025420 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.439+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4815c440 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.440+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4815c630 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.440+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c3c023070 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.441+0000 7f2c2ffff700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c34066f30 0x7f2c340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.441+0000 7f2c47fff700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c34066f30 0x7f2c340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.441+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2c3c085770 con 0x7f2c480af350 2023-10-06T03:01:35.444 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.441+0000 7f2c47fff700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c34066f30 0x7f2c340693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c38003a80 tx=0x7f2c38007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:35.446 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.445+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c3c052d30 con 0x7f2c480af350 2023-10-06T03:01:35.629 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.628+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c480adf10 con 0x7f2c480af350 2023-10-06T03:01:35.630 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.629+0000 7f2c2ffff700 1 -- 172.21.15.92:0/1828950518 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12127 (secure 0 0 0) 0x7f2c3c015090 con 0x7f2c480af350 2023-10-06T03:01:35.630 INFO:teuthology.misc.health.smithi092.stdout: 2023-10-06T03:01:35.630 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":16,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:31.660036+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c34066f30 msgr2=0x7f2c340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c34066f30 0x7f2c340693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2c38003a80 tx=0x7f2c38007ec0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 msgr2=0x7f2c480b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 0x7f2c480b05b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c006f50 tx=0x7f2c3c017000 comp rx=0 tx=0).stop 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 shutdown_connections 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c34066f30 0x7f2c340693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c480af720 0x7f2c4809b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c480ad420 0x7f2c4815a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 --2- 172.21.15.92:0/1828950518 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c480af350 0x7f2c480b05b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.633 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 >> 172.21.15.92:0/1828950518 conn(0x7f2c48099ab0 msgr2=0x7f2c480b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:35.634 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 shutdown_connections 2023-10-06T03:01:35.634 INFO:teuthology.misc.health.smithi092.stderr:2023-10-06T03:01:35.632+0000 7f2c4e2ad700 1 -- 172.21.15.92:0/1828950518 wait complete. 2023-10-06T03:01:35.643 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T03:01:35.643 INFO:tasks.ceph:Creating RBD pool 2023-10-06T03:01:35.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T03:01:35.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.756+0000 7f20c6f1a700 1 Processor -- start 2023-10-06T03:01:35.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.756+0000 7f20c6f1a700 1 -- start start 2023-10-06T03:01:35.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20c6f1a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f20c0159b10 0x7f20c00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20c6f1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c00c74b0 con 0x7f20c015d730 2023-10-06T03:01:35.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20c6f1a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c00c8700 con 0x7f20c006b420 2023-10-06T03:01:35.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20c6f1a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c00c9070 con 0x7f20c0159b10 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20b7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f20c015d730 0x7f20c0159ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53358/0 (socket says 172.21.15.92:53358) 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20b7fff700 1 -- 172.21.15.92:0/3490414861 learned_addr learned my addr 172.21.15.92:0/3490414861 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.757+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3490414861 >> v2:172.21.15.136:3300/0 conn(0x7f20c0159b10 0x7f20c00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157117590 0 0) 0x7f20c00c74b0 con 0x7f20c015d730 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a8003570 con 0x7f20c015d730 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424649571 0 0) 0x7f20c00c8700 con 0x7f20c006b420 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c00c74b0 con 0x7f20c006b420 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3619712318 0 0) 0x7f20a8003570 con 0x7f20c015d730 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 >> v2:172.21.15.136:3300/0 conn(0x7f20c0159b10 msgr2=0x7f20c00cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 --2- 172.21.15.92:0/3490414861 >> v2:172.21.15.136:3300/0 conn(0x7f20c0159b10 0x7f20c00cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 >> v1:172.21.15.136:6789/0 conn(0x7f20c006b420 legacy=0x7f20c015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00c80f0 con 0x7f20c015d730 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f20b0001470 con 0x7f20c015d730 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 >> v1:172.21.15.92:6789/0 conn(0x7f20c015d730 legacy=0x7f20c0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8003c40 0x7f20a8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a8004810 0x7f20a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8005210 0x7f20a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00c9070 con 0x7f20a8003c40 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a800c130 con 0x7f20a8004810 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a800c2c0 con 0x7f20a8005210 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8003c40 0x7f20a8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b7fff700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a8004810 0x7f20a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c54b7700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8005210 0x7f20a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 -- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8005210 msgr2=0x7f20a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8005210 0x7f20a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 -- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a8004810 msgr2=0x7f20a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a8004810 0x7f20a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 -- 172.21.15.92:0/3490414861 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a8003570 con 0x7f20a8003c40 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20b7fff700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20a8004810 0x7f20a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c54b7700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8005210 0x7f20a800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c54b7700 1 -- 172.21.15.92:0/3490414861 reap_dead start 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.758+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8003c40 0x7f20a8004030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e240 tx=0x7f20bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed98ac87ce77b39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc009ac0 con 0x7f20a8003c40 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc00f040 con 0x7f20a8003c40 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20b77fe700 1 -- 172.21.15.92:0/3490414861 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc00ed40 con 0x7f20a8003c40 2023-10-06T03:01:35.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3490414861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8003c40 msgr2=0x7f20a8004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8003c40 0x7f20a8004030 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e240 tx=0x7f20bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3490414861 shutdown_connections 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3490414861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20a8003c40 0x7f20a8004030 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3490414861 >> 172.21.15.92:0/3490414861 conn(0x7f20c00c9870 msgr2=0x7f20c00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3490414861 shutdown_connections 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.759+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3490414861 wait complete. 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.760+0000 7f20c6f1a700 1 Processor -- start 2023-10-06T03:01:35.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.760+0000 7f20c6f1a700 1 -- start start 2023-10-06T03:01:35.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c6f1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8003c40 0x7f20c00c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c6f1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 0x7f20c0155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c6f1a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20c00c4b10 0x7f20c00c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c6f1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00afba0 con 0x7f20c00af2b0 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c6f1a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00c54f0 con 0x7f20c00c4b10 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c6f1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b0f10 con 0x7f20a8003c40 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 0x7f20c0155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 0x7f20c0155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59146/0 (socket says 172.21.15.92:59146) 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20b7fff700 1 -- 172.21.15.92:0/3132339647 learned_addr learned my addr 172.21.15.92:0/3132339647 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8003c40 0x7f20c00c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.761+0000 7f20c54b7700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20c00c4b10 0x7f20c00c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b7fff700 1 -- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8003c40 msgr2=0x7f20c00c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b7fff700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8003c40 0x7f20c00c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b7fff700 1 -- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20c00c4b10 msgr2=0x7f20c00c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b7fff700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20c00c4b10 0x7f20c00c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b7fff700 1 -- 172.21.15.92:0/3132339647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00afd80 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b7fff700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 0x7f20c0155cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f20b000eb90 tx=0x7f20c00c8280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000c8b0 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b000ca10 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c00b0060 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000cdb0 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.762+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c00b0620 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.763+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c004c3c0 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.763+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20b0013630 con 0x7f20c00af2b0 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.764+0000 7f20b4ff9700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20ac066f30 0x7f20ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.764+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f20b0083d70 con 0x7f20c00af2b0 2023-10-06T03:01:35.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.764+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20ac066f30 0x7f20ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:35.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.764+0000 7f20c4cb6700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20ac066f30 0x7f20ac0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e240 tx=0x7f20bc01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:35.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.768+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20b0051330 con 0x7f20c00af2b0 2023-10-06T03:01:35.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:35.951+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f20c00633e0 con 0x7f20c00af2b0 2023-10-06T03:01:36.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.240+0000 7f20b4ff9700 1 -- 172.21.15.92:0/3132339647 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f20b0054950 con 0x7f20c00af2b0 2023-10-06T03:01:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20ac066f30 msgr2=0x7f20ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:36.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20ac066f30 0x7f20ac0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00e240 tx=0x7f20bc01a000 comp rx=0 tx=0).stop 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 msgr2=0x7f20c0155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 0x7f20c0155cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f20b000eb90 tx=0x7f20c00c8280 comp rx=0 tx=0).stop 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 shutdown_connections 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20ac066f30 0x7f20ac0693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20c00c4b10 0x7f20c00c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20a8003c40 0x7f20c00c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 --2- 172.21.15.92:0/3132339647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20c00af2b0 0x7f20c0155cc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 >> 172.21.15.92:0/3132339647 conn(0x7f20c00c9870 msgr2=0x7f20c00689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 shutdown_connections 2023-10-06T03:01:36.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.243+0000 7f20c6f1a700 1 -- 172.21.15.92:0/3132339647 wait complete. 2023-10-06T03:01:36.246 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2023-10-06T03:01:36.259 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T03:01:36.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.374+0000 7fa08535e700 1 Processor -- start 2023-10-06T03:01:36.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.374+0000 7fa08535e700 1 -- start start 2023-10-06T03:01:36.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa08535e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa0801563b0 0x7fa080156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa08535e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa08015a440 0x7fa0800acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa08535e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa080068890 0x7fa0800c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa08535e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08015a9b0 con 0x7fa0801563b0 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa08535e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08015ac90 con 0x7fa08015a440 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa08535e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08015af70 con 0x7fa080068890 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa07effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa0801563b0 0x7fa080156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa07effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa0801563b0 0x7fa080156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59162/0 (socket says 172.21.15.92:59162) 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa07effd700 1 -- 172.21.15.92:0/1513853013 learned_addr learned my addr 172.21.15.92:0/1513853013 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa0767fc700 1 --2- 172.21.15.92:0/1513853013 >> v2:172.21.15.136:3300/0 conn(0x7fa08015a440 0x7fa0800acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.375+0000 7fa07f7fe700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa080068890 0x7fa0800c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07effd700 1 -- 172.21.15.92:0/1513853013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa080068890 msgr2=0x7fa0800c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:36.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07effd700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa080068890 0x7fa0800c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07effd700 1 -- 172.21.15.92:0/1513853013 >> v2:172.21.15.136:3300/0 conn(0x7fa08015a440 msgr2=0x7fa0800acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07effd700 1 --2- 172.21.15.92:0/1513853013 >> v2:172.21.15.136:3300/0 conn(0x7fa08015a440 0x7fa0800acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07effd700 1 -- 172.21.15.92:0/1513853013 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08015b250 con 0x7fa0801563b0 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa0767fc700 1 --2- 172.21.15.92:0/1513853013 >> v2:172.21.15.136:3300/0 conn(0x7fa08015a440 0x7fa0800acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07f7fe700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa080068890 0x7fa0800c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07effd700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa080156780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa08006fb10 tx=0x7fa070009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1e8abdbfd4cd80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07e7fc700 1 -- 172.21.15.92:0/1513853013 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07000b860 con 0x7fa0801563b0 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07e7fc700 1 -- 172.21.15.92:0/1513853013 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07000b9c0 con 0x7fa0801563b0 2023-10-06T03:01:36.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa07e7fc700 1 -- 172.21.15.92:0/1513853013 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07000bd60 con 0x7fa0801563b0 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 -- 172.21.15.92:0/1513853013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 msgr2=0x7fa080156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa080156780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa08006fb10 tx=0x7fa070009250 comp rx=0 tx=0).stop 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 -- 172.21.15.92:0/1513853013 shutdown_connections 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa080068890 0x7fa0800c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 --2- 172.21.15.92:0/1513853013 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa080156780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 --2- 172.21.15.92:0/1513853013 >> v2:172.21.15.136:3300/0 conn(0x7fa08015a440 0x7fa0800acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 -- 172.21.15.92:0/1513853013 >> 172.21.15.92:0/1513853013 conn(0x7fa0800c8f90 msgr2=0x7fa0800c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.376+0000 7fa08535e700 1 -- 172.21.15.92:0/1513853013 shutdown_connections 2023-10-06T03:01:36.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.377+0000 7fa08535e700 1 -- 172.21.15.92:0/1513853013 wait complete. 2023-10-06T03:01:36.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 Processor -- start 2023-10-06T03:01:36.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 -- start start 2023-10-06T03:01:36.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa080068890 0x7fa08014bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa08014d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa08015a440 0x7fa080169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08014c290 con 0x7fa0801563b0 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08014dc60 con 0x7fa080068890 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.378+0000 7fa08535e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08014df40 con 0x7fa08015a440 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa08014d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa08014d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59172/0 (socket says 172.21.15.92:59172) 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 -- 172.21.15.92:0/236555829 learned_addr learned my addr 172.21.15.92:0/236555829 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa07effd700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa080068890 0x7fa08014bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa07f7fe700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa08015a440 0x7fa080169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 -- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa08015a440 msgr2=0x7fa080169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa08015a440 0x7fa080169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 -- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa080068890 msgr2=0x7fa08014bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:36.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa080068890 0x7fa08014bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 -- 172.21.15.92:0/236555829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700094f0 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0767fc700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa08014d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa06800d860 tx=0x7fa068008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0680125f0 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa068012750 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08014e220 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068012af0 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.379+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08014e7e0 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.380+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0800718b0 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.381+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa068014430 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.381+0000 7fa0777fe700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa060066f30 0x7fa0600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.381+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fa068084620 con 0x7fa0801563b0 2023-10-06T03:01:36.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.381+0000 7fa07effd700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa060066f30 0x7fa0600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:36.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.381+0000 7fa07effd700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa060066f30 0x7fa0600693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa08006fb10 tx=0x7fa070017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:36.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.385+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa068055520 con 0x7fa0801563b0 2023-10-06T03:01:36.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:36.569+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa08006d6a0 con 0x7fa0801563b0 2023-10-06T03:01:37.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.246+0000 7fa0777fe700 1 -- 172.21.15.92:0/236555829 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa0680550b0 con 0x7fa0801563b0 2023-10-06T03:01:37.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa060066f30 msgr2=0x7fa0600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa060066f30 0x7fa0600693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa08006fb10 tx=0x7fa070017000 comp rx=0 tx=0).stop 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 msgr2=0x7fa08014d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa08014d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa06800d860 tx=0x7fa068008040 comp rx=0 tx=0).stop 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 shutdown_connections 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa060066f30 0x7fa0600693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa08015a440 0x7fa080169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa080068890 0x7fa08014bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 --2- 172.21.15.92:0/236555829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0801563b0 0x7fa08014d3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 >> 172.21.15.92:0/236555829 conn(0x7fa0800c8f90 msgr2=0x7fa0800b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.249+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 shutdown_connections 2023-10-06T03:01:37.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.250+0000 7fa08535e700 1 -- 172.21.15.92:0/236555829 wait complete. 2023-10-06T03:01:37.252 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T03:01:37.264 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T03:01:37.265 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:01:37.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:01:37.267 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:01:37.267 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:01:37.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:01:37.270 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:01:37.270 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:01:37.270 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:01:37.273 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:01:37.273 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:01:37.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:01:37.275 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:01:37.275 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T03:01:37.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:37.344 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:01:37.345 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:01:37.345 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:01:37.344+0000 7f810b844700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T03:01:37.355 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:01:37.355 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:01:37.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.399+0000 7faa56508700 1 Processor -- start 2023-10-06T03:01:37.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.399+0000 7faa56508700 1 -- start start 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa56508700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7faa5006bcd0 0x7faa500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa56508700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faa5006c0a0 0x7faa5015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa56508700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5015d7e0 con 0x7faa5015df50 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa56508700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5015dac0 con 0x7faa5006c0a0 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa56508700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5015aa30 con 0x7faa5006bcd0 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faa5006c0a0 0x7faa5015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faa5006c0a0 0x7faa5015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59218/0 (socket says 172.21.15.92:59218) 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa4f7fe700 1 -- 172.21.15.92:0/1442916422 learned_addr learned my addr 172.21.15.92:0/1442916422 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.400+0000 7faa4ffff700 1 --2- 172.21.15.92:0/1442916422 >> v2:172.21.15.136:3300/0 conn(0x7faa5006bcd0 0x7faa500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4f7fe700 1 -- 172.21.15.92:0/1442916422 >> v2:172.21.15.136:3300/0 conn(0x7faa5006bcd0 msgr2=0x7faa500caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4f7fe700 1 --2- 172.21.15.92:0/1442916422 >> v2:172.21.15.136:3300/0 conn(0x7faa5006bcd0 0x7faa500caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4f7fe700 1 -- 172.21.15.92:0/1442916422 >> v1:172.21.15.136:6789/0 conn(0x7faa5015df50 legacy=0x7faa500c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4f7fe700 1 -- 172.21.15.92:0/1442916422 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5015ad10 con 0x7faa5006c0a0 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4f7fe700 1 --2- 172.21.15.92:0/1442916422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006c0a0 0x7faa5015a1e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa500691a0 tx=0x7faa44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46ce681417260bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4effd700 1 -- 172.21.15.92:0/1442916422 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44009e70 con 0x7faa5006c0a0 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4effd700 1 -- 172.21.15.92:0/1442916422 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4400b860 con 0x7faa5006c0a0 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa4effd700 1 -- 172.21.15.92:0/1442916422 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4400bc20 con 0x7faa5006c0a0 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa56508700 1 -- 172.21.15.92:0/1442916422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006c0a0 msgr2=0x7faa5015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa56508700 1 --2- 172.21.15.92:0/1442916422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006c0a0 0x7faa5015a1e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa500691a0 tx=0x7faa44009250 comp rx=0 tx=0).stop 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa56508700 1 -- 172.21.15.92:0/1442916422 shutdown_connections 2023-10-06T03:01:37.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa56508700 1 --2- 172.21.15.92:0/1442916422 >> v2:172.21.15.136:3300/0 conn(0x7faa5006bcd0 0x7faa500caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa56508700 1 --2- 172.21.15.92:0/1442916422 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006c0a0 0x7faa5015a1e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.401+0000 7faa56508700 1 -- 172.21.15.92:0/1442916422 >> 172.21.15.92:0/1442916422 conn(0x7faa500ce600 msgr2=0x7faa500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.402+0000 7faa56508700 1 -- 172.21.15.92:0/1442916422 shutdown_connections 2023-10-06T03:01:37.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.402+0000 7faa56508700 1 -- 172.21.15.92:0/1442916422 wait complete. 2023-10-06T03:01:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 Processor -- start 2023-10-06T03:01:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 -- start start 2023-10-06T03:01:37.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 0x7faa500c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5006c0a0 0x7faa5016a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faa5015df50 0x7faa500afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500c7470 con 0x7faa5006bcd0 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500c7900 con 0x7faa5006c0a0 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.403+0000 7faa56508700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa500c6860 con 0x7faa5015df50 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 0x7faa500c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 0x7faa500c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59228/0 (socket says 172.21.15.92:59228) 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 -- 172.21.15.92:0/3235936658 learned_addr learned my addr 172.21.15.92:0/3235936658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4f7fe700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5006c0a0 0x7faa5016a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa54aa5700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faa5015df50 0x7faa500afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 -- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faa5015df50 msgr2=0x7faa500afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faa5015df50 0x7faa500afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 -- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5006c0a0 msgr2=0x7faa5016a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5006c0a0 0x7faa5016a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 -- 172.21.15.92:0/3235936658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440094f0 con 0x7faa5006bcd0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa54aa5700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faa5015df50 0x7faa500afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa4ffff700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 0x7faa500c8d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa4000e930 tx=0x7faa400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40013680 con 0x7faa5006bcd0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa400137e0 con 0x7faa5006bcd0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40013b80 con 0x7faa5006bcd0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa500c6b40 con 0x7faa5006bcd0 2023-10-06T03:01:37.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.404+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa500c7100 con 0x7faa5006bcd0 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.405+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5004c3c0 con 0x7faa5006bcd0 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.405+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa4000c460 con 0x7faa5006bcd0 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.406+0000 7faa37fff700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faa3c066f30 0x7faa3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.406+0000 7faa4f7fe700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faa3c066f30 0x7faa3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.406+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7faa400844f0 con 0x7faa5006bcd0 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.406+0000 7faa4f7fe700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faa3c066f30 0x7faa3c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faa500691a0 tx=0x7faa44017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:37.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.410+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa40051950 con 0x7faa5006bcd0 2023-10-06T03:01:37.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.595+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa5015aff0 con 0x7faa5006bcd0 2023-10-06T03:01:37.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.596+0000 7faa37fff700 1 -- 172.21.15.92:0/3235936658 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13660 (secure 0 0 0) 0x7faa40054f70 con 0x7faa5006bcd0 2023-10-06T03:01:37.597 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:37.597 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":18,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:37.239688+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:37.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faa3c066f30 msgr2=0x7faa3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faa3c066f30 0x7faa3c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faa500691a0 tx=0x7faa44017000 comp rx=0 tx=0).stop 2023-10-06T03:01:37.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 msgr2=0x7faa500c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 0x7faa500c8d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa4000e930 tx=0x7faa400091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 shutdown_connections 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faa3c066f30 0x7faa3c0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faa5006bcd0 0x7faa500c8d60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faa5015df50 0x7faa500afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 --2- 172.21.15.92:0/3235936658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa5006c0a0 0x7faa5016a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 >> 172.21.15.92:0/3235936658 conn(0x7faa500ce600 msgr2=0x7faa500cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 shutdown_connections 2023-10-06T03:01:37.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.598+0000 7faa56508700 1 -- 172.21.15.92:0/3235936658 wait complete. 2023-10-06T03:01:37.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:01:37.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:01:37.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.728+0000 7f5d57fff700 1 Processor -- start 2023-10-06T03:01:37.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.728+0000 7f5d57fff700 1 -- start start 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d57fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5d58157a50 0x7f5d580c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d57fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d580cd240 con 0x7f5d58157680 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d57fff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d580c9a20 con 0x7f5d580cb590 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d57fff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d580cc700 con 0x7f5d58157a50 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d56ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5d58157680 0x7f5d580cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53380/0 (socket says 172.21.15.92:53380) 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d56ffd700 1 -- 172.21.15.92:0/2414720753 learned_addr learned my addr 172.21.15.92:0/2414720753 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d567fc700 1 --2- 172.21.15.92:0/2414720753 >> v2:172.21.15.136:3300/0 conn(0x7f5d58157a50 0x7f5d580c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973202109 0 0) 0x7f5d580cd240 con 0x7f5d58157680 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d40003570 con 0x7f5d58157680 2023-10-06T03:01:37.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 455065617 0 0) 0x7f5d40003570 con 0x7f5d58157680 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 >> v2:172.21.15.136:3300/0 conn(0x7f5d58157a50 msgr2=0x7f5d580c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 --2- 172.21.15.92:0/2414720753 >> v2:172.21.15.136:3300/0 conn(0x7f5d58157a50 0x7f5d580c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 >> v1:172.21.15.136:6789/0 conn(0x7f5d580cb590 legacy=0x7f5d580c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d580cc910 con 0x7f5d58157680 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5d48001470 con 0x7f5d58157680 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 >> v1:172.21.15.92:6789/0 conn(0x7f5d58157680 legacy=0x7f5d580cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d40003a60 0x7f5d40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d40004810 0x7f5d40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d400051c0 0x7f5d4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d580cc700 con 0x7f5d400051c0 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000c0d0 con 0x7f5d40004810 2023-10-06T03:01:37.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000c290 con 0x7f5d40003a60 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d400051c0 0x7f5d4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d56ffd700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d40003a60 0x7f5d40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.729+0000 7f5d567fc700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d40004810 0x7f5d40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 -- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d40003a60 msgr2=0x7f5d40003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d40003a60 0x7f5d40003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 -- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d40004810 msgr2=0x7f5d40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d40004810 0x7f5d40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 -- 172.21.15.92:0/2414720753 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40003570 con 0x7f5d400051c0 2023-10-06T03:01:37.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 -- 172.21.15.92:0/2414720753 reap_dead start 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d400051c0 0x7f5d4000ba60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d50001fc0 tx=0x7f5d50001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaac17f3462fd19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50001950 con 0x7f5d400051c0 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5000c8b0 con 0x7f5d400051c0 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d55ffb700 1 -- 172.21.15.92:0/2414720753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000cc90 con 0x7f5d400051c0 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 -- 172.21.15.92:0/2414720753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d400051c0 msgr2=0x7f5d4000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d400051c0 0x7f5d4000ba60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5d50001fc0 tx=0x7f5d50001640 comp rx=0 tx=0).stop 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 -- 172.21.15.92:0/2414720753 shutdown_connections 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2414720753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d400051c0 0x7f5d4000ba60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 -- 172.21.15.92:0/2414720753 >> 172.21.15.92:0/2414720753 conn(0x7f5d5806baa0 msgr2=0x7f5d5806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 -- 172.21.15.92:0/2414720753 shutdown_connections 2023-10-06T03:01:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.730+0000 7f5d57fff700 1 -- 172.21.15.92:0/2414720753 wait complete. 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 Processor -- start 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 -- start start 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d400051c0 0x7f5d580c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d580c7880 0x7f5d581598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 0x7f5d58159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d580c6a60 con 0x7f5d58153560 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d580c8930 con 0x7f5d580c7880 2023-10-06T03:01:37.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.732+0000 7f5d57fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d580c7d20 con 0x7f5d400051c0 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 0x7f5d58159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 0x7f5d58159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59250/0 (socket says 172.21.15.92:59250) 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 -- 172.21.15.92:0/2775324108 learned_addr learned my addr 172.21.15.92:0/2775324108 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d567fc700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d580c7880 0x7f5d581598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d56ffd700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d400051c0 0x7f5d580c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 -- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d400051c0 msgr2=0x7f5d580c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d400051c0 0x7f5d580c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 -- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d580c7880 msgr2=0x7f5d581598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d580c7880 0x7f5d581598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 -- 172.21.15.92:0/2775324108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d580c8000 con 0x7f5d58153560 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d577fe700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 0x7f5d58159e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c6a0 tx=0x7f5d50015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:37.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50001950 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5000c6d0 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d580c82e0 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5001e850 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.733+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d580c4940 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.734+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d580c4b60 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.734+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d5001ec40 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.735+0000 7f5d3effd700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d44066f30 0x7f5d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.735+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5d50085820 con 0x7f5d58153560 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.735+0000 7f5d56ffd700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d44066f30 0x7f5d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:37.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.735+0000 7f5d56ffd700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d44066f30 0x7f5d440693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d48007e90 tx=0x7f5d48001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:37.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.739+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d50052c80 con 0x7f5d58153560 2023-10-06T03:01:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.920+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d580ccbf0 con 0x7f5d58153560 2023-10-06T03:01:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.921+0000 7f5d3effd700 1 -- 172.21.15.92:0/2775324108 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d50017070 con 0x7f5d58153560 2023-10-06T03:01:37.922 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:01:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d44066f30 msgr2=0x7f5d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d44066f30 0x7f5d440693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d48007e90 tx=0x7f5d48001e50 comp rx=0 tx=0).stop 2023-10-06T03:01:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 msgr2=0x7f5d58159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 0x7f5d58159e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c6a0 tx=0x7f5d50015000 comp rx=0 tx=0).stop 2023-10-06T03:01:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 shutdown_connections 2023-10-06T03:01:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d44066f30 0x7f5d440693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d58153560 0x7f5d58159e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d400051c0 0x7f5d580c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 --2- 172.21.15.92:0/2775324108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d580c7880 0x7f5d581598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 >> 172.21.15.92:0/2775324108 conn(0x7f5d5806baa0 msgr2=0x7f5d58156410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 shutdown_connections 2023-10-06T03:01:37.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:37.923+0000 7f5d57fff700 1 -- 172.21.15.92:0/2775324108 wait complete. 2023-10-06T03:01:37.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:01:38.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.053+0000 7f85af59e700 1 Processor -- start 2023-10-06T03:01:38.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.053+0000 7f85af59e700 1 -- start start 2023-10-06T03:01:38.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85af59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85b006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85af59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00cb5d0 con 0x7f85b00c7fd0 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85af59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b00af490 con 0x7f85b00c9770 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85af59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00af770 con 0x7f85b00c93a0 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85aed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f85b00c7fd0 0x7f85b00c8410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53396/0 (socket says 172.21.15.92:53396) 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85aed9d700 1 -- 172.21.15.92:0/1940634340 learned_addr learned my addr 172.21.15.92:0/1940634340 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ae59c700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85b006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697201875 0 0) 0x7f85b00cb5d0 con 0x7f85b00c7fd0 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8598003570 con 0x7f85b00c7fd0 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085575245 0 0) 0x7f85b00af490 con 0x7f85b00c9770 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b00cb5d0 con 0x7f85b00c9770 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2588460132 0 0) 0x7f8598003570 con 0x7f85b00c7fd0 2023-10-06T03:01:38.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 msgr2=0x7f85b006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85b006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 >> v1:172.21.15.136:6789/0 conn(0x7f85b00c9770 legacy=0x7f85b00caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00afa50 con 0x7f85b00c7fd0 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f85a80017f0 con 0x7f85b00c7fd0 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ae59c700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85b006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 >> v1:172.21.15.92:6789/0 conn(0x7f85b00c7fd0 legacy=0x7f85b00c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8598003c20 0x7f8598004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f8598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00af770 con 0x7f8598004810 2023-10-06T03:01:38.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859800bb10 con 0x7f8598003c20 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859800bcd0 con 0x7f85b00c93a0 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.054+0000 7f85add9b700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f8598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85ae59c700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8598003c20 0x7f8598004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85aed9d700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85add9b700 1 -- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 msgr2=0x7f85980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85add9b700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b00c93a0 0x7f85980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85add9b700 1 -- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8598003c20 msgr2=0x7f8598004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85add9b700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8598003c20 0x7f8598004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85add9b700 1 -- 172.21.15.92:0/1940634340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0009420 con 0x7f8598004810 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85add9b700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f8598004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001a30 tx=0x7f85a4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2543e30a83782c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a40013f0 con 0x7f8598004810 2023-10-06T03:01:38.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400c5d0 con 0x7f8598004810 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.055+0000 7f85ad59a700 1 -- 172.21.15.92:0/1940634340 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a400c990 con 0x7f8598004810 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85af59e700 1 -- 172.21.15.92:0/1940634340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 msgr2=0x7f8598004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85af59e700 1 --2- 172.21.15.92:0/1940634340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f8598004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001a30 tx=0x7f85a4007d80 comp rx=0 tx=0).stop 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85aed9d700 1 -- 172.21.15.92:0/1940634340 reap_dead start 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85af59e700 1 -- 172.21.15.92:0/1940634340 shutdown_connections 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85af59e700 1 -- 172.21.15.92:0/1940634340 >> 172.21.15.92:0/1940634340 conn(0x7f85b00cddc0 msgr2=0x7f85b00c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85af59e700 1 -- 172.21.15.92:0/1940634340 shutdown_connections 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.056+0000 7f85af59e700 1 -- 172.21.15.92:0/1940634340 wait complete. 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 Processor -- start 2023-10-06T03:01:38.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 -- start start 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f85b014a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b014d940 0x7f85b00c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b00c5250 0x7f85b00b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b014a960 con 0x7f8598004810 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00c56d0 con 0x7f85b00c5250 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85af59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00b0310 con 0x7f85b014d940 2023-10-06T03:01:38.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.057+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f85b014a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f85b014a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59256/0 (socket says 172.21.15.92:59256) 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 -- 172.21.15.92:0/964667710 learned_addr learned my addr 172.21.15.92:0/964667710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85add9b700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b014d940 0x7f85b00c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85aed9d700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b00c5250 0x7f85b00b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 -- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b014d940 msgr2=0x7f85b00c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b014d940 0x7f85b00c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 -- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b00c5250 msgr2=0x7f85b00b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b00c5250 0x7f85b00b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 -- 172.21.15.92:0/964667710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b014ddd0 con 0x7f8598004810 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85add9b700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b014d940 0x7f85b00c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85aed9d700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b00c5250 0x7f85b00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:38.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85ae59c700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f85b014a420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85a0011da0 tx=0x7f85a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a000fcd0 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a000fe30 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b014e0b0 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0014690 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.058+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b014e670 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.059+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b006d560 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.059+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85a0014a40 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.060+0000 7f859e7fc700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8590066f30 0x7f85900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.060+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f85a0018030 con 0x7f8598004810 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.060+0000 7f85add9b700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8590066f30 0x7f85900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.060+0000 7f85add9b700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8590066f30 0x7f85900693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85b01690c0 tx=0x7f85a4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.064+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85a00523c0 con 0x7f8598004810 2023-10-06T03:01:38.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.247+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85b006b500 con 0x7f8598004810 2023-10-06T03:01:38.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.247+0000 7f859e7fc700 1 -- 172.21.15.92:0/964667710 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85a0022020 con 0x7f8598004810 2023-10-06T03:01:38.249 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8590066f30 msgr2=0x7f85900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8590066f30 0x7f85900693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f85b01690c0 tx=0x7f85a4015000 comp rx=0 tx=0).stop 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 msgr2=0x7f85b014a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f85b014a420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85a0011da0 tx=0x7f85a00091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 shutdown_connections 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8590066f30 0x7f85900693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8598004810 0x7f85b014a420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85b00c5250 0x7f85b00b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 --2- 172.21.15.92:0/964667710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85b014d940 0x7f85b00c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 >> 172.21.15.92:0/964667710 conn(0x7f85b00cddc0 msgr2=0x7f85b01551d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:38.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 shutdown_connections 2023-10-06T03:01:38.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.250+0000 7f85af59e700 1 -- 172.21.15.92:0/964667710 wait complete. 2023-10-06T03:01:38.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:01:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.377+0000 7f728c810700 1 Processor -- start 2023-10-06T03:01:38.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.377+0000 7f728c810700 1 -- start start 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f728c810700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 0x7f72880caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f728c810700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 0x7f728815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f728c810700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728815d7e0 con 0x7f728806bcd0 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f728c810700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728815dac0 con 0x7f728806c0a0 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f728c810700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728815aa30 con 0x7f728815df50 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f727dd9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 0x7f72880caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.378+0000 7f7286d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 0x7f728815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 0x7f728815e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:53716/0 (socket says 172.21.15.92:53716) 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f727dd9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 0x7f72880caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:41448/0 (socket says 172.21.15.92:41448) 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 -- 172.21.15.92:0/1353554480 learned_addr learned my addr 172.21.15.92:0/1353554480 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 -- 172.21.15.92:0/1353554480 >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 msgr2=0x7f72880caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 --2- 172.21.15.92:0/1353554480 >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 0x7f72880caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 -- 172.21.15.92:0/1353554480 >> v1:172.21.15.136:6789/0 conn(0x7f728806bcd0 legacy=0x7f728815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 -- 172.21.15.92:0/1353554480 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728815ad10 con 0x7f728815df50 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f727dd9b700 1 --2- 172.21.15.92:0/1353554480 >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 0x7f72880caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7286d9d700 1 --2- 172.21.15.92:0/1353554480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 0x7f728815e3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7280009f20 tx=0x7f7280009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121a069d0ce8949a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7285d9b700 1 -- 172.21.15.92:0/1353554480 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728000e070 con 0x7f728815df50 2023-10-06T03:01:38.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7285d9b700 1 -- 172.21.15.92:0/1353554480 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728000b9d0 con 0x7f728815df50 2023-10-06T03:01:38.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.379+0000 7f7285d9b700 1 -- 172.21.15.92:0/1353554480 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800092e0 con 0x7f728815df50 2023-10-06T03:01:38.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 -- 172.21.15.92:0/1353554480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 msgr2=0x7f728815e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 --2- 172.21.15.92:0/1353554480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 0x7f728815e3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7280009f20 tx=0x7f7280009840 comp rx=0 tx=0).stop 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 -- 172.21.15.92:0/1353554480 shutdown_connections 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 --2- 172.21.15.92:0/1353554480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728815df50 0x7f728815e3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 --2- 172.21.15.92:0/1353554480 >> v2:172.21.15.136:3300/0 conn(0x7f728806c0a0 0x7f72880caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 -- 172.21.15.92:0/1353554480 >> 172.21.15.92:0/1353554480 conn(0x7f72880ce600 msgr2=0x7f72880cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 -- 172.21.15.92:0/1353554480 shutdown_connections 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.380+0000 7f728c810700 1 -- 172.21.15.92:0/1353554480 wait complete. 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.381+0000 7f728c810700 1 Processor -- start 2023-10-06T03:01:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.381+0000 7f728c810700 1 -- start start 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 0x7f72881481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728806c0a0 0x7f728816a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f728815df50 0x7f7288156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288153360 con 0x7f728806bcd0 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288154150 con 0x7f728815df50 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7288153540 con 0x7f728806c0a0 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 0x7f72881481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 0x7f72881481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59270/0 (socket says 172.21.15.92:59270) 2023-10-06T03:01:38.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 -- 172.21.15.92:0/3082280544 learned_addr learned my addr 172.21.15.92:0/3082280544 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f727dd9b700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728806c0a0 0x7f728816a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f7286d9d700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f728815df50 0x7f7288156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 -- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728806c0a0 msgr2=0x7f728816a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728806c0a0 0x7f728816a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 -- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f728815df50 msgr2=0x7f7288156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f728815df50 0x7f7288156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 -- 172.21.15.92:0/3082280544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7270009420 con 0x7f728806bcd0 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728659c700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 0x7f72881481a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f727800c590 tx=0x7f7278007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278011590 con 0x7f728806bcd0 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7278011730 con 0x7f728806bcd0 2023-10-06T03:01:38.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.382+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7280009ae0 con 0x7f728806bcd0 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.383+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7278011af0 con 0x7f728806bcd0 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.383+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72881538d0 con 0x7f728806bcd0 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.383+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72880cf510 con 0x7f728806bcd0 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.384+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7278012420 con 0x7f728806bcd0 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.384+0000 7f727effd700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7268066f30 0x7f72680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.384+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7278083570 con 0x7f728806bcd0 2023-10-06T03:01:38.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.384+0000 7f727dd9b700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7268066f30 0x7f72680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.385+0000 7f727dd9b700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7268066f30 0x7f72680693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72880c9890 tx=0x7f72700091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.388+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7278050a80 con 0x7f728806bcd0 2023-10-06T03:01:38.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.613+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f728804c3c0 con 0x7f728806bcd0 2023-10-06T03:01:38.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.613+0000 7f727effd700 1 -- 172.21.15.92:0/3082280544 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f7278017020 con 0x7f728806bcd0 2023-10-06T03:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:38.614 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4306,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T03:01:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7268066f30 msgr2=0x7f72680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7268066f30 0x7f72680693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72880c9890 tx=0x7f72700091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 msgr2=0x7f72881481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 0x7f72881481a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f727800c590 tx=0x7f7278007f30 comp rx=0 tx=0).stop 2023-10-06T03:01:38.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 shutdown_connections 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7268066f30 0x7f72680693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f728806bcd0 0x7f72881481a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f728815df50 0x7f7288156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 --2- 172.21.15.92:0/3082280544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f728806c0a0 0x7f728816a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 >> 172.21.15.92:0/3082280544 conn(0x7f72880ce600 msgr2=0x7f72880c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 shutdown_connections 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.615+0000 7f728c810700 1 -- 172.21.15.92:0/3082280544 wait complete. 2023-10-06T03:01:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 3 2023-10-06T03:01:38.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:38.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.741+0000 7f73b3fff700 1 Processor -- start 2023-10-06T03:01:38.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.741+0000 7f73b3fff700 1 -- start start 2023-10-06T03:01:38.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b3fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b41574d0 0x7f73b40cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73b40cc3c0 0x7f73b406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b3fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b406c380 con 0x7f73b406bf20 2023-10-06T03:01:38.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b3fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b40b16a0 con 0x7f73b40cc3c0 2023-10-06T03:01:38.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b3fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40b1c60 con 0x7f73b41574d0 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73aa7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73b40cc3c0 0x7f73b406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f73b406bf20 0x7f73b406cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53406/0 (socket says 172.21.15.92:53406) 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b37fe700 1 -- 172.21.15.92:0/2615022970 learned_addr learned my addr 172.21.15.92:0/2615022970 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b2ffd700 1 --2- 172.21.15.92:0/2615022970 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b41574d0 0x7f73b40cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73aa7fc700 1 -- 172.21.15.92:0/2615022970 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b41574d0 msgr2=0x7f73b40cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73aa7fc700 1 --2- 172.21.15.92:0/2615022970 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b41574d0 0x7f73b40cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73aa7fc700 1 -- 172.21.15.92:0/2615022970 >> v1:172.21.15.92:6789/0 conn(0x7f73b406bf20 legacy=0x7f73b406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73aa7fc700 1 -- 172.21.15.92:0/2615022970 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b41578a0 con 0x7f73b40cc3c0 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.742+0000 7f73b27fc700 1 -- 172.21.15.92:0/2615022970 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949011625 0 0) 0x7f73ac001490 con 0x7f73b406bf20 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73aa7fc700 1 --2- 172.21.15.92:0/2615022970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b40cc3c0 0x7f73b406b9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f739c00dd10 tx=0x7f739c00d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3e745c339b661a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b27fc700 1 -- 172.21.15.92:0/2615022970 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c012070 con 0x7f73b40cc3c0 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b27fc700 1 -- 172.21.15.92:0/2615022970 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c00d2e0 con 0x7f73b40cc3c0 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b27fc700 1 -- 172.21.15.92:0/2615022970 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00fb60 con 0x7f73b40cc3c0 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 -- 172.21.15.92:0/2615022970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b40cc3c0 msgr2=0x7f73b406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2615022970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b40cc3c0 0x7f73b406b9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f739c00dd10 tx=0x7f739c00d630 comp rx=0 tx=0).stop 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 -- 172.21.15.92:0/2615022970 shutdown_connections 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2615022970 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b41574d0 0x7f73b40cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2615022970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b40cc3c0 0x7f73b406b9e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 -- 172.21.15.92:0/2615022970 >> 172.21.15.92:0/2615022970 conn(0x7f73b40ce430 msgr2=0x7f73b40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 -- 172.21.15.92:0/2615022970 shutdown_connections 2023-10-06T03:01:38.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.743+0000 7f73b3fff700 1 -- 172.21.15.92:0/2615022970 wait complete. 2023-10-06T03:01:38.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 Processor -- start 2023-10-06T03:01:38.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 -- start start 2023-10-06T03:01:38.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73b406bf20 0x7f73b40c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b40cc3c0 0x7f73b40c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 0x7f73b40c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40ca0b0 con 0x7f73b41574d0 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b4168da0 con 0x7f73b406bf20 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b3fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b4149e40 con 0x7f73b40cc3c0 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 0x7f73b40c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 0x7f73b40c53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59300/0 (socket says 172.21.15.92:59300) 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.745+0000 7f73b37fe700 1 -- 172.21.15.92:0/2610885277 learned_addr learned my addr 172.21.15.92:0/2610885277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73aa7fc700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b40cc3c0 0x7f73b40c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b2ffd700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73b406bf20 0x7f73b40c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b37fe700 1 -- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b40cc3c0 msgr2=0x7f73b40c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b37fe700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b40cc3c0 0x7f73b40c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b37fe700 1 -- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73b406bf20 msgr2=0x7f73b40c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b37fe700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73b406bf20 0x7f73b40c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b37fe700 1 -- 172.21.15.92:0/2610885277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c00d8d0 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b37fe700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 0x7f73b40c53b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00cc90 tx=0x7f73ac001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac00cea0 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac004bb0 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b40c9810 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac00ab70 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.746+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b40c9dd0 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.747+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b4168b70 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.747+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73ac0070f0 con 0x7f73b41574d0 2023-10-06T03:01:38.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.748+0000 7f73ab7fe700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7394066f30 0x7f73940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.748+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f73ac0840c0 con 0x7f73b41574d0 2023-10-06T03:01:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.748+0000 7f73b2ffd700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7394066f30 0x7f73940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.748+0000 7f73b2ffd700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7394066f30 0x7f73940693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73a4006520 tx=0x7f73a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:38.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.752+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73b4168b70 con 0x7f73b41574d0 2023-10-06T03:01:38.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.938+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73b406c380 con 0x7f73b41574d0 2023-10-06T03:01:38.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.939+0000 7f73ab7fe700 1 -- 172.21.15.92:0/2610885277 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13647 (secure 0 0 0) 0x7f73b406c380 con 0x7f73b41574d0 2023-10-06T03:01:38.940 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:38.940 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":19,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:38.244359+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7394066f30 msgr2=0x7f73940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7394066f30 0x7f73940693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f73a4006520 tx=0x7f73a4008040 comp rx=0 tx=0).stop 2023-10-06T03:01:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 msgr2=0x7f73b40c53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 0x7f73b40c53b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00cc90 tx=0x7f73ac001a50 comp rx=0 tx=0).stop 2023-10-06T03:01:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 shutdown_connections 2023-10-06T03:01:38.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7394066f30 0x7f73940693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73b41574d0 0x7f73b40c53b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73b406bf20 0x7f73b40c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 --2- 172.21.15.92:0/2610885277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73b40cc3c0 0x7f73b40c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 >> 172.21.15.92:0/2610885277 conn(0x7f73b40ce430 msgr2=0x7f73b40b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 shutdown_connections 2023-10-06T03:01:38.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:38.941+0000 7f73b3fff700 1 -- 172.21.15.92:0/2610885277 wait complete. 2023-10-06T03:01:38.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:01:38.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:01:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.070+0000 7f1b4f232700 1 Processor -- start 2023-10-06T03:01:39.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.070+0000 7f1b4f232700 1 -- start start 2023-10-06T03:01:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b4f232700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480cb710 0x7f1b48159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b4f232700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b480689e0 con 0x7f1b48159930 2023-10-06T03:01:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b4f232700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b480cbcc0 con 0x7f1b480cd1c0 2023-10-06T03:01:39.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b4f232700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480cbfa0 con 0x7f1b480cb710 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b3ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480cb710 0x7f1b48159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b4cfce700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b48159930 0x7f1b480b3ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53416/0 (socket says 172.21.15.92:53416) 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.071+0000 7f1b4cfce700 1 -- 172.21.15.92:0/4194816099 learned_addr learned my addr 172.21.15.92:0/4194816099 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3ffff700 1 -- 172.21.15.92:0/4194816099 >> v1:172.21.15.136:6789/0 conn(0x7f1b480cd1c0 legacy=0x7f1b48068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3ffff700 1 -- 172.21.15.92:0/4194816099 >> v1:172.21.15.92:6789/0 conn(0x7f1b48159930 legacy=0x7f1b480b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3f7fe700 1 -- 172.21.15.92:0/4194816099 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888030990 0 0) 0x7f1b480689e0 con 0x7f1b48159930 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3ffff700 1 -- 172.21.15.92:0/4194816099 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48068390 con 0x7f1b480cb710 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3ffff700 1 --2- 172.21.15.92:0/4194816099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b48159d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098e0 tx=0x7f1b380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbd5cfb9f54529a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3f7fe700 1 -- 172.21.15.92:0/4194816099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38009e80 con 0x7f1b480cb710 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3f7fe700 1 -- 172.21.15.92:0/4194816099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3800b8b0 con 0x7f1b480cb710 2023-10-06T03:01:39.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b3f7fe700 1 -- 172.21.15.92:0/4194816099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800bc30 con 0x7f1b480cb710 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b4f232700 1 -- 172.21.15.92:0/4194816099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 msgr2=0x7f1b48159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b4f232700 1 --2- 172.21.15.92:0/4194816099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b48159d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098e0 tx=0x7f1b380091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b4f232700 1 -- 172.21.15.92:0/4194816099 shutdown_connections 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b4f232700 1 --2- 172.21.15.92:0/4194816099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b48159d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.072+0000 7f1b4f232700 1 -- 172.21.15.92:0/4194816099 >> 172.21.15.92:0/4194816099 conn(0x7f1b480999d0 msgr2=0x7f1b480b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.073+0000 7f1b4f232700 1 -- 172.21.15.92:0/4194816099 shutdown_connections 2023-10-06T03:01:39.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.073+0000 7f1b4f232700 1 -- 172.21.15.92:0/4194816099 wait complete. 2023-10-06T03:01:39.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.074+0000 7f1b4f232700 1 Processor -- start 2023-10-06T03:01:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.074+0000 7f1b4f232700 1 -- start start 2023-10-06T03:01:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.074+0000 7f1b4f232700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b480c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.074+0000 7f1b4f232700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480cd1c0 0x7f1b48156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.074+0000 7f1b4f232700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b48159930 0x7f1b4815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.074+0000 7f1b4f232700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48157060 con 0x7f1b480cb710 2023-10-06T03:01:39.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4f232700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ca0b0 con 0x7f1b480cd1c0 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4f232700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4815df80 con 0x7f1b48159930 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b480c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b480c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59316/0 (socket says 172.21.15.92:59316) 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 -- 172.21.15.92:0/545857383 learned_addr learned my addr 172.21.15.92:0/545857383 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b3ffff700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480cd1c0 0x7f1b48156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4d7cf700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b48159930 0x7f1b4815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 -- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b48159930 msgr2=0x7f1b4815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b48159930 0x7f1b4815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 -- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480cd1c0 msgr2=0x7f1b48156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480cd1c0 0x7f1b48156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 -- 172.21.15.92:0/545857383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b38009420 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b3ffff700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480cd1c0 0x7f1b48156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4cfce700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b480c8e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c5e0 tx=0x7f1b44007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440072e0 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44007440 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440057f0 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480c94a0 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.075+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480c9a60 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.076+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4804c3c0 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.076+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b44005c00 con 0x7f1b480cb710 2023-10-06T03:01:39.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.077+0000 7f1b3cff9700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b34066f30 0x7f1b340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.077+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1b440838e0 con 0x7f1b480cb710 2023-10-06T03:01:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.077+0000 7f1b3ffff700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b34066f30 0x7f1b340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.077+0000 7f1b3ffff700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b34066f30 0x7f1b340693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b380030c0 tx=0x7f1b38016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.081+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b44050d40 con 0x7f1b480cb710 2023-10-06T03:01:39.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.268+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b48099da0 con 0x7f1b480cb710 2023-10-06T03:01:39.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.268+0000 7f1b3cff9700 1 -- 172.21.15.92:0/545857383 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b44017070 con 0x7f1b480cb710 2023-10-06T03:01:39.269 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:01:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.270+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b34066f30 msgr2=0x7f1b340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.270+0000 7f1b4f232700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b34066f30 0x7f1b340693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1b380030c0 tx=0x7f1b38016000 comp rx=0 tx=0).stop 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 msgr2=0x7f1b480c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b480c8e50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c5e0 tx=0x7f1b44007e80 comp rx=0 tx=0).stop 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 shutdown_connections 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b34066f30 0x7f1b340693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480cb710 0x7f1b480c8e50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b48159930 0x7f1b4815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 --2- 172.21.15.92:0/545857383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480cd1c0 0x7f1b48156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 >> 172.21.15.92:0/545857383 conn(0x7f1b480999d0 msgr2=0x7f1b480cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:39.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 shutdown_connections 2023-10-06T03:01:39.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.271+0000 7f1b4f232700 1 -- 172.21.15.92:0/545857383 wait complete. 2023-10-06T03:01:39.283 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:01:39.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.400+0000 7fc5350b3700 1 Processor -- start 2023-10-06T03:01:39.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.400+0000 7fc5350b3700 1 -- start start 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc5350b3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5300c91e0 0x7fc5300cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc5350b3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc53006ae10 0x7fc53006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc5350b3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5300cc210 con 0x7fc53006aa40 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc5350b3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5300cc4f0 con 0x7fc5300c91e0 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc5350b3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5300cc7d0 con 0x7fc53006ae10 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5300c91e0 0x7fc5300cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5300c91e0 0x7fc5300cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44090/0 (socket says 172.21.15.92:44090) 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 -- 172.21.15.92:0/3050333349 learned_addr learned my addr 172.21.15.92:0/3050333349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:39.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 -- 172.21.15.92:0/3050333349 >> v2:172.21.15.136:3300/0 conn(0x7fc53006ae10 msgr2=0x7fc53006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 --2- 172.21.15.92:0/3050333349 >> v2:172.21.15.136:3300/0 conn(0x7fc53006ae10 0x7fc53006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 -- 172.21.15.92:0/3050333349 >> v1:172.21.15.136:6789/0 conn(0x7fc53006aa40 legacy=0x7fc53006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.401+0000 7fc52e59c700 1 -- 172.21.15.92:0/3050333349 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5300ccab0 con 0x7fc5300c91e0 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc52e59c700 1 --2- 172.21.15.92:0/3050333349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5300c91e0 0x7fc5300cbc00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5240099b0 tx=0x7fc524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660f508156a0f938 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc52dd9b700 1 -- 172.21.15.92:0/3050333349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400b7e0 con 0x7fc5300c91e0 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc52dd9b700 1 -- 172.21.15.92:0/3050333349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400b940 con 0x7fc5300c91e0 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc52dd9b700 1 -- 172.21.15.92:0/3050333349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400bce0 con 0x7fc5300c91e0 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 -- 172.21.15.92:0/3050333349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5300c91e0 msgr2=0x7fc5300cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 --2- 172.21.15.92:0/3050333349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5300c91e0 0x7fc5300cbc00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5240099b0 tx=0x7fc524009250 comp rx=0 tx=0).stop 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 -- 172.21.15.92:0/3050333349 shutdown_connections 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 --2- 172.21.15.92:0/3050333349 >> v2:172.21.15.136:3300/0 conn(0x7fc53006ae10 0x7fc53006b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 --2- 172.21.15.92:0/3050333349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5300c91e0 0x7fc5300cbc00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 -- 172.21.15.92:0/3050333349 >> 172.21.15.92:0/3050333349 conn(0x7fc5300cd670 msgr2=0x7fc5300cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 -- 172.21.15.92:0/3050333349 shutdown_connections 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.402+0000 7fc5350b3700 1 -- 172.21.15.92:0/3050333349 wait complete. 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 Processor -- start 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 -- start start 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc53006aa40 0x7fc530159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 0x7fc53015bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5300c91e0 0x7fc5300c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc530168670 con 0x7fc53006ae10 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc530167a80 con 0x7fc5300c91e0 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc5350b3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53014e710 con 0x7fc53006aa40 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc52e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 0x7fc53015bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc52e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 0x7fc53015bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44104/0 (socket says 172.21.15.92:44104) 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.404+0000 7fc52e59c700 1 -- 172.21.15.92:0/4189694320 learned_addr learned my addr 172.21.15.92:0/4189694320 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52f59e700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5300c91e0 0x7fc5300c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52e59c700 1 -- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc53006aa40 msgr2=0x7fc530159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52ed9d700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc53006aa40 0x7fc530159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52e59c700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc53006aa40 0x7fc530159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52e59c700 1 -- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5300c91e0 msgr2=0x7fc5300c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52e59c700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5300c91e0 0x7fc5300c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52e59c700 1 -- 172.21.15.92:0/4189694320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240094f0 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52f59e700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5300c91e0 0x7fc5300c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc52e59c700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 0x7fc53015bb90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc524009290 tx=0x7fc524016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524026480 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52401f070 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc530167d40 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524026830 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.405+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc530168300 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.406+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc530168820 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.406+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc52401a440 con 0x7fc53006ae10 2023-10-06T03:01:39.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.407+0000 7fc516ffd700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc51c066f30 0x7fc51c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.407+0000 7fc52ed9d700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc51c066f30 0x7fc51c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.407+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc524021070 con 0x7fc53006ae10 2023-10-06T03:01:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.407+0000 7fc52ed9d700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc51c066f30 0x7fc51c0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc520006fd0 tx=0x7fc520008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.411+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc530168820 con 0x7fc53006ae10 2023-10-06T03:01:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.591+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc530168a00 con 0x7fc53006ae10 2023-10-06T03:01:39.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.592+0000 7fc516ffd700 1 -- 172.21.15.92:0/4189694320 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc52402b0a0 con 0x7fc53006ae10 2023-10-06T03:01:39.593 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:01:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc51c066f30 msgr2=0x7fc51c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc51c066f30 0x7fc51c0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc520006fd0 tx=0x7fc520008040 comp rx=0 tx=0).stop 2023-10-06T03:01:39.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 msgr2=0x7fc53015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 0x7fc53015bb90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc524009290 tx=0x7fc524016000 comp rx=0 tx=0).stop 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 shutdown_connections 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc51c066f30 0x7fc51c0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5300c91e0 0x7fc5300c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc53006aa40 0x7fc530159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 --2- 172.21.15.92:0/4189694320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc53006ae10 0x7fc53015bb90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 >> 172.21.15.92:0/4189694320 conn(0x7fc5300cd670 msgr2=0x7fc530150370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 shutdown_connections 2023-10-06T03:01:39.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.594+0000 7fc5350b3700 1 -- 172.21.15.92:0/4189694320 wait complete. 2023-10-06T03:01:39.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:01:39.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.720+0000 7f4eccb11700 1 Processor -- start 2023-10-06T03:01:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.720+0000 7f4eccb11700 1 -- start start 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4eccb11700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ec8069560 0x7f4ec80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4eccb11700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec8069ee0 con 0x7f4ec8159890 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4eccb11700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec815a160 con 0x7f4ec8159c60 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4eccb11700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec80d13a0 con 0x7f4ec8069560 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ec8069560 0x7f4ec80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4ec8159890 0x7f4ec815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34250/0 (socket says 172.21.15.92:34250) 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec659c700 1 -- 172.21.15.92:0/2799824542 learned_addr learned my addr 172.21.15.92:0/2799824542 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec6d9d700 1 -- 172.21.15.92:0/2799824542 >> v1:172.21.15.136:6789/0 conn(0x7f4ec8159c60 legacy=0x7f4ec815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:39.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec6d9d700 1 -- 172.21.15.92:0/2799824542 >> v1:172.21.15.92:6789/0 conn(0x7f4ec8159890 legacy=0x7f4ec815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec6d9d700 1 -- 172.21.15.92:0/2799824542 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec815e140 con 0x7f4ec8069560 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec5d9b700 1 -- 172.21.15.92:0/2799824542 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375049823 0 0) 0x7f4ec8069ee0 con 0x7f4ec8159890 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec6d9d700 1 --2- 172.21.15.92:0/2799824542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec80699a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009e60 tx=0x7f4ec0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5604f72a67eb909e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec5d9b700 1 -- 172.21.15.92:0/2799824542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000e070 con 0x7f4ec8069560 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec5d9b700 1 -- 172.21.15.92:0/2799824542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec000bbf0 con 0x7f4ec8069560 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.721+0000 7f4ec5d9b700 1 -- 172.21.15.92:0/2799824542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0012430 con 0x7f4ec8069560 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 -- 172.21.15.92:0/2799824542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 msgr2=0x7f4ec80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 --2- 172.21.15.92:0/2799824542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec80699a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009e60 tx=0x7f4ec0009780 comp rx=0 tx=0).stop 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 -- 172.21.15.92:0/2799824542 shutdown_connections 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 --2- 172.21.15.92:0/2799824542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec80699a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 -- 172.21.15.92:0/2799824542 >> 172.21.15.92:0/2799824542 conn(0x7f4ec8068e20 msgr2=0x7f4ec80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 -- 172.21.15.92:0/2799824542 shutdown_connections 2023-10-06T03:01:39.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.722+0000 7f4eccb11700 1 -- 172.21.15.92:0/2799824542 wait complete. 2023-10-06T03:01:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.723+0000 7f4eccb11700 1 Processor -- start 2023-10-06T03:01:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.723+0000 7f4eccb11700 1 -- start start 2023-10-06T03:01:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec81534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ec8159890 0x7f4ec80b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8159c60 0x7f4ec8154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80cd0c0 con 0x7f4ec8069560 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80cd960 con 0x7f4ec8159c60 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8153a30 con 0x7f4ec8159890 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec81534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec81534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44126/0 (socket says 172.21.15.92:44126) 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 -- 172.21.15.92:0/4038676671 learned_addr learned my addr 172.21.15.92:0/4038676671 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec6d9d700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8159c60 0x7f4ec8154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ebffff700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ec8159890 0x7f4ec80b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 -- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ec8159890 msgr2=0x7f4ec80b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ec8159890 0x7f4ec80b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 -- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8159c60 msgr2=0x7f4ec8154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8159c60 0x7f4ec8154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 -- 172.21.15.92:0/4038676671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009a20 con 0x7f4ec8069560 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec6d9d700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8159c60 0x7f4ec8154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ec659c700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec81534f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c810 tx=0x7f4eb8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8005b30 con 0x7f4ec8069560 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8005c90 con 0x7f4ec8069560 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec80b62c0 con 0x7f4ec8069560 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8007300 con 0x7f4ec8069560 2023-10-06T03:01:39.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.724+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec80cd5f0 con 0x7f4ec8069560 2023-10-06T03:01:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.725+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec804c3c0 con 0x7f4ec8069560 2023-10-06T03:01:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.726+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4eb801b400 con 0x7f4ec8069560 2023-10-06T03:01:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.726+0000 7f4ebe7fc700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.726+0000 7f4ebffff700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.726+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4eb8083e10 con 0x7f4ec8069560 2023-10-06T03:01:39.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.726+0000 7f4ebffff700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ea8066f30 0x7f4ea80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80cbc00 tx=0x7f4eb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:39.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.730+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4eb80512f0 con 0x7f4ec8069560 2023-10-06T03:01:39.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.914+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4ec806a5b0 con 0x7f4ec8069560 2023-10-06T03:01:39.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.915+0000 7f4ebe7fc700 1 -- 172.21.15.92:0/4038676671 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4eb8054910 con 0x7f4ec8069560 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:01:39.916 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:01:39.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:01:39.917 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:01:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ea8066f30 msgr2=0x7f4ea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ea8066f30 0x7f4ea80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80cbc00 tx=0x7f4eb0008040 comp rx=0 tx=0).stop 2023-10-06T03:01:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 msgr2=0x7f4ec81534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec81534f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c810 tx=0x7f4eb8007df0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 shutdown_connections 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ea8066f30 0x7f4ea80693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ec8069560 0x7f4ec81534f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8159c60 0x7f4ec8154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 --2- 172.21.15.92:0/4038676671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ec8159890 0x7f4ec80b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 >> 172.21.15.92:0/4038676671 conn(0x7f4ec8068e20 msgr2=0x7f4ec80d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 shutdown_connections 2023-10-06T03:01:39.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:39.917+0000 7f4eccb11700 1 -- 172.21.15.92:0/4038676671 wait complete. 2023-10-06T03:01:39.929 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:01:39.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:01:40.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.047+0000 7fb508f1a700 1 Processor -- start 2023-10-06T03:01:40.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.047+0000 7fb508f1a700 1 -- start start 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb508f1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb504164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb508f1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040cfa60 con 0x7fb5040cd910 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb508f1a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040b5b00 con 0x7fb504159ec0 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb508f1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040b5da0 con 0x7fb5040b55a0 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb4f9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5040cd910 0x7fb5040ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34256/0 (socket says 172.21.15.92:34256) 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1904563516 learned_addr learned my addr 172.21.15.92:0/1904563516 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb502d9d700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb504164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49679610 0 0) 0x7fb5040cfa60 con 0x7fb5040cd910 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f0003570 con 0x7fb5040cd910 2023-10-06T03:01:40.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253844585 0 0) 0x7fb5040b5b00 con 0x7fb504159ec0 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5040cfa60 con 0x7fb504159ec0 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.048+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2150429352 0 0) 0x7fb4f0003570 con 0x7fb5040cd910 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 msgr2=0x7fb504164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb504164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 >> v1:172.21.15.136:6789/0 conn(0x7fb504159ec0 legacy=0x7fb5040b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040b6080 con 0x7fb5040cd910 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb4ec001470 con 0x7fb5040cd910 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb502d9d700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb504164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 >> v1:172.21.15.92:6789/0 conn(0x7fb5040cd910 legacy=0x7fb5040ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f0003c20 0x7fb4f0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0004810 0x7fb4f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb4f0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040b5da0 con 0x7fb4f0003c20 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f000bbc0 con 0x7fb4f0004810 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f000bd80 con 0x7fb5040b55a0 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f0003c20 0x7fb4f0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0004810 0x7fb4f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb502d9d700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb4f0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 -- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 msgr2=0x7fb4f0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb4f0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 -- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0004810 msgr2=0x7fb4f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0004810 0x7fb4f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 -- 172.21.15.92:0/1904563516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc009a20 con 0x7fb4f0003c20 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb502d9d700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5040b55a0 0x7fb4f0005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb50259c700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f0003c20 0x7fb4f0004010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4001a30 tx=0x7fb4f4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5b388df452779e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40013f0 con 0x7fb4f0003c20 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f400c5d0 con 0x7fb4f0003c20 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.049+0000 7fb501d9b700 1 -- 172.21.15.92:0/1904563516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400c990 con 0x7fb4f0003c20 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb508f1a700 1 -- 172.21.15.92:0/1904563516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f0003c20 msgr2=0x7fb4f0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1904563516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4f0003c20 0x7fb4f0004010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4001a30 tx=0x7fb4f4007d80 comp rx=0 tx=0).stop 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb502d9d700 1 -- 172.21.15.92:0/1904563516 reap_dead start 2023-10-06T03:01:40.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb508f1a700 1 -- 172.21.15.92:0/1904563516 shutdown_connections 2023-10-06T03:01:40.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb508f1a700 1 -- 172.21.15.92:0/1904563516 >> 172.21.15.92:0/1904563516 conn(0x7fb5040999f0 msgr2=0x7fb5040b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:40.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb508f1a700 1 -- 172.21.15.92:0/1904563516 shutdown_connections 2023-10-06T03:01:40.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.050+0000 7fb508f1a700 1 -- 172.21.15.92:0/1904563516 wait complete. 2023-10-06T03:01:40.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.051+0000 7fb508f1a700 1 Processor -- start 2023-10-06T03:01:40.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.051+0000 7fb508f1a700 1 -- start start 2023-10-06T03:01:40.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0003c20 0x7fb504153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 0x7fb50406b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb504157110 0x7fb504157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50409dea0 con 0x7fb50406ab90 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb504156090 con 0x7fb504157110 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb504156370 con 0x7fb4f0003c20 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 0x7fb50406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 0x7fb50406b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44142/0 (socket says 172.21.15.92:44142) 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1280311124 learned_addr learned my addr 172.21.15.92:0/1280311124 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb502d9d700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb504157110 0x7fb504157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb50259c700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0003c20 0x7fb504153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0003c20 msgr2=0x7fb504153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0003c20 0x7fb504153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb504157110 msgr2=0x7fb504157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb504157110 0x7fb504157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1280311124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504156650 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 0x7fb50406b000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0073c0 tx=0x7fb4ec001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec000c40 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec00f7f0 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb504156930 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.052+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec00fbb0 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.053+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb504153bb0 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.053+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb504156f80 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.054+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4ec01d460 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.054+0000 7fb4faffd700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4066f30 0x7fb4e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.054+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb4ec084420 con 0x7fb50406ab90 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.054+0000 7fb50259c700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4066f30 0x7fb4e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.055+0000 7fb50259c700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4066f30 0x7fb4e40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4001a30 tx=0x7fb4f4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:40.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.058+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4ec0517e0 con 0x7fb50406ab90 2023-10-06T03:01:40.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.242+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb504063c90 con 0x7fb50406ab90 2023-10-06T03:01:40.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.359+0000 7fb4faffd700 1 -- 172.21.15.92:0/1280311124 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb4ec054e00 con 0x7fb50406ab90 2023-10-06T03:01:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4066f30 msgr2=0x7fb4e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4066f30 0x7fb4e40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4001a30 tx=0x7fb4f4015000 comp rx=0 tx=0).stop 2023-10-06T03:01:40.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 msgr2=0x7fb50406b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 0x7fb50406b000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0073c0 tx=0x7fb4ec001d20 comp rx=0 tx=0).stop 2023-10-06T03:01:40.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 shutdown_connections 2023-10-06T03:01:40.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4066f30 0x7fb4e40693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb504157110 0x7fb504157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0003c20 0x7fb504153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 --2- 172.21.15.92:0/1280311124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406ab90 0x7fb50406b000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 >> 172.21.15.92:0/1280311124 conn(0x7fb5040999f0 msgr2=0x7fb504155770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.362+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 shutdown_connections 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.363+0000 7fb508f1a700 1 -- 172.21.15.92:0/1280311124 wait complete. 2023-10-06T03:01:40.364 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:01:40.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:01:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.496+0000 7f002dc36700 1 Processor -- start 2023-10-06T03:01:40.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.496+0000 7f002dc36700 1 -- start start 2023-10-06T03:01:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f002dc36700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f00280af1c0 0x7f0028157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f002dc36700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0028063830 con 0x7f0028069920 2023-10-06T03:01:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f002dc36700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00280c90e0 con 0x7f00280cc670 2023-10-06T03:01:40.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f002dc36700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0028157cb0 con 0x7f00280af1c0 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f00277fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0028069920 0x7f00280cbf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34268/0 (socket says 172.21.15.92:34268) 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f00277fe700 1 -- 172.21.15.92:0/2978605509 learned_addr learned my addr 172.21.15.92:0/2978605509 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f0027fff700 1 --2- 172.21.15.92:0/2978605509 >> v2:172.21.15.136:3300/0 conn(0x7f00280af1c0 0x7f0028157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.497+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938317633 0 0) 0x7f0028063830 con 0x7f0028069920 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0010003570 con 0x7f0028069920 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2985672696 0 0) 0x7f0010003570 con 0x7f0028069920 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 >> v2:172.21.15.136:3300/0 conn(0x7f00280af1c0 msgr2=0x7f0028157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 --2- 172.21.15.92:0/2978605509 >> v2:172.21.15.136:3300/0 conn(0x7f00280af1c0 0x7f0028157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 >> v1:172.21.15.136:6789/0 conn(0x7f00280cc670 legacy=0x7f00280ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028069cf0 con 0x7f0028069920 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f00180017f0 con 0x7f0028069920 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 >> v1:172.21.15.92:6789/0 conn(0x7f0028069920 legacy=0x7f00280cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0010003a60 0x7f0010003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0010004810 0x7f0010004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00100051c0 0x7f001000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028157cb0 con 0x7f00100051c0 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001000c0d0 con 0x7f0010003a60 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001000c290 con 0x7f0010004810 2023-10-06T03:01:40.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00100051c0 0x7f001000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00277fe700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0010003a60 0x7f0010003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0026ffd700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0010004810 0x7f0010004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 -- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0010004810 msgr2=0x7f0010004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0010004810 0x7f0010004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 -- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0010003a60 msgr2=0x7f0010003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0010003a60 0x7f0010003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 -- 172.21.15.92:0/2978605509 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010003570 con 0x7f00100051c0 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 -- 172.21.15.92:0/2978605509 reap_dead start 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f0027fff700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00100051c0 0x7f001000ba60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0020009c00 tx=0x7f00200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b934dc1915eb3564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:40.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.498+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000e710 con 0x7f00100051c0 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002000e890 con 0x7f00100051c0 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f00267fc700 1 -- 172.21.15.92:0/2978605509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000ec50 con 0x7f00100051c0 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 -- 172.21.15.92:0/2978605509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00100051c0 msgr2=0x7f001000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00100051c0 0x7f001000ba60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0020009c00 tx=0x7f00200091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 -- 172.21.15.92:0/2978605509 shutdown_connections 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 --2- 172.21.15.92:0/2978605509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00100051c0 0x7f001000ba60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 -- 172.21.15.92:0/2978605509 >> 172.21.15.92:0/2978605509 conn(0x7f00280c9870 msgr2=0x7f00280c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 -- 172.21.15.92:0/2978605509 shutdown_connections 2023-10-06T03:01:40.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.499+0000 7f002dc36700 1 -- 172.21.15.92:0/2978605509 wait complete. 2023-10-06T03:01:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.500+0000 7f002dc36700 1 Processor -- start 2023-10-06T03:01:40.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.500+0000 7f002dc36700 1 -- start start 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f002dc36700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00100051c0 0x7f002814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f002dc36700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f002814e4f0 0x7f002814e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f002dc36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 0x7f00280acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f002dc36700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028167440 con 0x7f00280ac740 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f002dc36700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028167720 con 0x7f002814e4f0 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f002dc36700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0028167a00 con 0x7f00100051c0 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 0x7f00280acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 0x7f00280acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44166/0 (socket says 172.21.15.92:44166) 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 -- 172.21.15.92:0/113881016 learned_addr learned my addr 172.21.15.92:0/113881016 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0026ffd700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f002814e4f0 0x7f002814e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f00277fe700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00100051c0 0x7f002814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 -- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00100051c0 msgr2=0x7f002814c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00100051c0 0x7f002814c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 -- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f002814e4f0 msgr2=0x7f002814e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f002814e4f0 0x7f002814e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f0027fff700 1 -- 172.21.15.92:0/113881016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028167ce0 con 0x7f00280ac740 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.501+0000 7f00277fe700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00100051c0 0x7f002814c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:40.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.502+0000 7f0027fff700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 0x7f00280acb90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f002806cc40 tx=0x7f0020019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.502+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020024070 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.502+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002000e710 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.502+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002814cef0 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.502+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020006e50 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.502+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028168210 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.503+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0020028070 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.503+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0028167860 con 0x7f00280ac740 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.504+0000 7f000f7fe700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0014066f30 0x7f00140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.504+0000 7f00277fe700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0014066f30 0x7f00140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:40.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.504+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f0020087eb0 con 0x7f00280ac740 2023-10-06T03:01:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.504+0000 7f00277fe700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0014066f30 0x7f00140693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f001800a0c0 tx=0x7f0018006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:40.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.508+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0028167860 con 0x7f00280ac740 2023-10-06T03:01:40.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:40.692+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0028167860 con 0x7f00280ac740 2023-10-06T03:01:41.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.364+0000 7f000f7fe700 1 -- 172.21.15.92:0/113881016 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0028167860 con 0x7f00280ac740 2023-10-06T03:01:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.369+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0014066f30 msgr2=0x7f00140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.369+0000 7f002dc36700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0014066f30 0x7f00140693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f001800a0c0 tx=0x7f0018006040 comp rx=0 tx=0).stop 2023-10-06T03:01:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.369+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 msgr2=0x7f00280acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.369+0000 7f002dc36700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 0x7f00280acb90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f002806cc40 tx=0x7f0020019000 comp rx=0 tx=0).stop 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 shutdown_connections 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0014066f30 0x7f00140693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00280ac740 0x7f00280acb90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00100051c0 0x7f002814c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 --2- 172.21.15.92:0/113881016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f002814e4f0 0x7f002814e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 >> 172.21.15.92:0/113881016 conn(0x7f00280c9870 msgr2=0x7f00280c1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.370+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 shutdown_connections 2023-10-06T03:01:41.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.371+0000 7f002dc36700 1 -- 172.21.15.92:0/113881016 wait complete. 2023-10-06T03:01:41.382 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:01:41.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:01:41.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.514+0000 7f09a7fff700 1 Processor -- start 2023-10-06T03:01:41.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.514+0000 7f09a7fff700 1 -- start start 2023-10-06T03:01:41.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 0x7f09a80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a7fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80c9cb0 con 0x7f09a80cb230 2023-10-06T03:01:41.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a7fff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80c9f30 con 0x7f09a80ccc00 2023-10-06T03:01:41.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a7fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a806dfc0 con 0x7f09a8159a60 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09a80cb230 0x7f09a806d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34278/0 (socket says 172.21.15.92:34278) 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a77fe700 1 -- 172.21.15.92:0/1505118899 learned_addr learned my addr 172.21.15.92:0/1505118899 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a6ffd700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 0x7f09a80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228725837 0 0) 0x7f09a80c9cb0 con 0x7f09a80cb230 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.515+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0994003570 con 0x7f09a80cb230 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745214015 0 0) 0x7f0994003570 con 0x7f09a80cb230 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 msgr2=0x7f09a80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 0x7f09a80c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 >> v1:172.21.15.136:6789/0 conn(0x7f09a80ccc00 legacy=0x7f09a8069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80ca470 con 0x7f09a80cb230 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f09a00017f0 con 0x7f09a80cb230 2023-10-06T03:01:41.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 >> v1:172.21.15.92:6789/0 conn(0x7f09a80cb230 legacy=0x7f09a806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 0x7f0994003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0994003fd0 0x7f0994004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994004d50 0x7f09940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a806dfc0 con 0x7f0994004d50 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099400bb40 con 0x7f0994003fd0 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099400bd00 con 0x7f09a8159a60 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994004d50 0x7f09940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a6ffd700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 0x7f0994003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f099e7fc700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0994003fd0 0x7f0994004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 -- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 msgr2=0x7f0994003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a8159a60 0x7f0994003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 -- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0994003fd0 msgr2=0x7f0994004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0994003fd0 0x7f0994004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 -- 172.21.15.92:0/1505118899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994003570 con 0x7f0994004d50 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a77fe700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994004d50 0x7f09940051c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a000a950 tx=0x7f09a00021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857f4fb23c9773ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00070d0 con 0x7f0994004d50 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0007230 con 0x7f0994004d50 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.516+0000 7f09a67fc700 1 -- 172.21.15.92:0/1505118899 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00075f0 con 0x7f0994004d50 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a7fff700 1 -- 172.21.15.92:0/1505118899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994004d50 msgr2=0x7f09940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a7fff700 1 --2- 172.21.15.92:0/1505118899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0994004d50 0x7f09940051c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a000a950 tx=0x7f09a00021f0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a77fe700 1 -- 172.21.15.92:0/1505118899 reap_dead start 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a7fff700 1 -- 172.21.15.92:0/1505118899 shutdown_connections 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a7fff700 1 -- 172.21.15.92:0/1505118899 >> 172.21.15.92:0/1505118899 conn(0x7f09a806baa0 msgr2=0x7f09a806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:41.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a7fff700 1 -- 172.21.15.92:0/1505118899 shutdown_connections 2023-10-06T03:01:41.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.517+0000 7f09a7fff700 1 -- 172.21.15.92:0/1505118899 wait complete. 2023-10-06T03:01:41.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.518+0000 7f09a7fff700 1 Processor -- start 2023-10-06T03:01:41.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.518+0000 7f09a7fff700 1 -- start start 2023-10-06T03:01:41.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 0x7f09a80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09a815ec00 0x7f09a80c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a815c2a0 0x7f09a815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a7fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a815cc10 con 0x7f09a815e830 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a7fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80c5f90 con 0x7f09a815ec00 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a7fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80c6230 con 0x7f09a815c2a0 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 0x7f09a80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 0x7f09a80c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44182/0 (socket says 172.21.15.92:44182) 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 -- 172.21.15.92:0/90740138 learned_addr learned my addr 172.21.15.92:0/90740138 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a77fe700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a815c2a0 0x7f09a815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 -- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a815c2a0 msgr2=0x7f09a815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a815c2a0 0x7f09a815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 -- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09a815ec00 msgr2=0x7f09a80c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09a815ec00 0x7f09a80c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 -- 172.21.15.92:0/90740138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80c6510 con 0x7f09a815e830 2023-10-06T03:01:41.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a77fe700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a815c2a0 0x7f09a815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f09a6ffd700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 0x7f09a80c9150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0998006ac0 tx=0x7f09980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.519+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800fa10 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.520+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099800fb70 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.520+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a80c67f0 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.520+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998014440 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.520+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a80c6db0 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.520+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a80c6fa0 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.521+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0998014870 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.521+0000 7f099f7fe700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0988066f30 0x7f09880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.521+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f0998018030 con 0x7f09a815e830 2023-10-06T03:01:41.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.521+0000 7f099e7fc700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0988066f30 0x7f09880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:41.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.522+0000 7f099e7fc700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0988066f30 0x7f09880693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0990007e20 tx=0x7f0990007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:41.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.526+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09980522a0 con 0x7f09a815e830 2023-10-06T03:01:41.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:01:41.573 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:01:41.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:41.756+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f09a80ca750 con 0x7f09a815e830 2023-10-06T03:01:42.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.364+0000 7f099f7fe700 1 -- 172.21.15.92:0/90740138 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f09980558c0 con 0x7f09a815e830 2023-10-06T03:01:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.366+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0988066f30 msgr2=0x7f09880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.366+0000 7f09a7fff700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0988066f30 0x7f09880693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0990007e20 tx=0x7f0990007d10 comp rx=0 tx=0).stop 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.366+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 msgr2=0x7f09a80c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.366+0000 7f09a7fff700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 0x7f09a80c9150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0998006ac0 tx=0x7f09980091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 shutdown_connections 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0988066f30 0x7f09880693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09a815e830 0x7f09a80c9150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09a815c2a0 0x7f09a815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 --2- 172.21.15.92:0/90740138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09a815ec00 0x7f09a80c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 >> 172.21.15.92:0/90740138 conn(0x7f09a806baa0 msgr2=0x7f09a815f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 shutdown_connections 2023-10-06T03:01:42.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.367+0000 7f09a7fff700 1 -- 172.21.15.92:0/90740138 wait complete. 2023-10-06T03:01:42.369 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T03:01:42.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:42.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.495+0000 7f70dc65c700 1 Processor -- start 2023-10-06T03:01:42.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.495+0000 7f70dc65c700 1 -- start start 2023-10-06T03:01:42.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70dc65c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70d4159890 0x7f70d415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70dc65c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d4159c60 0x7f70d415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70dc65c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d415a160 con 0x7f70d415db10 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70dc65c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40d13a0 con 0x7f70d4159890 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70dc65c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415e140 con 0x7f70d4159c60 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70da3f8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70d4159890 0x7f70d415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70da3f8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f70d4159890 0x7f70d415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44198/0 (socket says 172.21.15.92:44198) 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.496+0000 7f70da3f8700 1 -- 172.21.15.92:0/3297943404 learned_addr learned my addr 172.21.15.92:0/3297943404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70d9bf7700 1 --2- 172.21.15.92:0/3297943404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d4159c60 0x7f70d415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70da3f8700 1 -- 172.21.15.92:0/3297943404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d4159c60 msgr2=0x7f70d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70da3f8700 1 --2- 172.21.15.92:0/3297943404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d4159c60 0x7f70d415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70da3f8700 1 -- 172.21.15.92:0/3297943404 >> v1:172.21.15.136:6789/0 conn(0x7f70d415db10 legacy=0x7f70d4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70da3f8700 1 -- 172.21.15.92:0/3297943404 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d415a3d0 con 0x7f70d4159890 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70da3f8700 1 --2- 172.21.15.92:0/3297943404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159890 0x7f70d415abd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009f20 tx=0x7f70d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8299e79d02049be4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:42.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70d93f6700 1 -- 172.21.15.92:0/3297943404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d000e070 con 0x7f70d4159890 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70d93f6700 1 -- 172.21.15.92:0/3297943404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d000ba80 con 0x7f70d4159890 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.497+0000 7f70d93f6700 1 -- 172.21.15.92:0/3297943404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d00092e0 con 0x7f70d4159890 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 -- 172.21.15.92:0/3297943404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159890 msgr2=0x7f70d415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 --2- 172.21.15.92:0/3297943404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159890 0x7f70d415abd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009f20 tx=0x7f70d0009840 comp rx=0 tx=0).stop 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 -- 172.21.15.92:0/3297943404 shutdown_connections 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 --2- 172.21.15.92:0/3297943404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159890 0x7f70d415abd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 --2- 172.21.15.92:0/3297943404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d4159c60 0x7f70d415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 -- 172.21.15.92:0/3297943404 >> 172.21.15.92:0/3297943404 conn(0x7f70d4068e20 msgr2=0x7f70d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 -- 172.21.15.92:0/3297943404 shutdown_connections 2023-10-06T03:01:42.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.498+0000 7f70dc65c700 1 -- 172.21.15.92:0/3297943404 wait complete. 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 Processor -- start 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 -- start start 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d4159890 0x7f70d4150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 0x7f70d414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d415db10 0x7f70d4169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40b49b0 con 0x7f70d4159c60 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40b4c90 con 0x7f70d4159890 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.499+0000 7f70dc65c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40b4f70 con 0x7f70d415db10 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 0x7f70d414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 0x7f70d414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44204/0 (socket says 172.21.15.92:44204) 2023-10-06T03:01:42.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 -- 172.21.15.92:0/2728849303 learned_addr learned my addr 172.21.15.92:0/2728849303 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70da3f8700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d4159890 0x7f70d4150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70dabf9700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d415db10 0x7f70d4169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 -- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d415db10 msgr2=0x7f70d4169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d415db10 0x7f70d4169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 -- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d4159890 msgr2=0x7f70d4150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d4159890 0x7f70d4150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 -- 172.21.15.92:0/2728849303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d0009ae0 con 0x7f70d4159c60 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70d9bf7700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 0x7f70d414d1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f70c400d860 tx=0x7f70c4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40125f0 con 0x7f70d4159c60 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c4012750 con 0x7f70d4159c60 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d40b5250 con 0x7f70d4159c60 2023-10-06T03:01:42.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4012af0 con 0x7f70d4159c60 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.500+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d40b4610 con 0x7f70d4159c60 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.501+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d40b56f0 con 0x7f70d4159c60 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.501+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70c4013480 con 0x7f70d4159c60 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.502+0000 7f70ca7fc700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.502+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f70c4084a10 con 0x7f70d4159c60 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.502+0000 7f70da3f8700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.502+0000 7f70da3f8700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f70b8066f30 0x7f70b80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009880 tx=0x7f70d0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:42.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.506+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70c40555b0 con 0x7f70d4159c60 2023-10-06T03:01:42.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.691+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70d404c3c0 con 0x7f70d4159c60 2023-10-06T03:01:42.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.691+0000 7f70ca7fc700 1 -- 172.21.15.92:0/2728849303 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f70c4055140 con 0x7f70d4159c60 2023-10-06T03:01:42.692 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:42.692 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:41.756921+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f70b8066f30 msgr2=0x7f70b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f70b8066f30 0x7f70b80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009880 tx=0x7f70d0018000 comp rx=0 tx=0).stop 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 msgr2=0x7f70d414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 0x7f70d414d1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f70c400d860 tx=0x7f70c4008040 comp rx=0 tx=0).stop 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 shutdown_connections 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f70b8066f30 0x7f70b80693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70d415db10 0x7f70d4169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d4159890 0x7f70d4150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 --2- 172.21.15.92:0/2728849303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70d4159c60 0x7f70d414d1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 >> 172.21.15.92:0/2728849303 conn(0x7f70d4068e20 msgr2=0x7f70d415d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:42.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 shutdown_connections 2023-10-06T03:01:42.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.694+0000 7f70dc65c700 1 -- 172.21.15.92:0/2728849303 wait complete. 2023-10-06T03:01:42.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:42.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 Processor -- start 2023-10-06T03:01:42.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 -- start start 2023-10-06T03:01:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb806bcd0 0x7f3bb8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bb806c0a0 0x7f3bb8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb815c2a0 con 0x7f3bb80c8e00 2023-10-06T03:01:42.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb815b690 con 0x7f3bb806c0a0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.820+0000 7f3bbdf31700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb815b970 con 0x7f3bb806bcd0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bb806c0a0 0x7f3bb8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3bb806c0a0 0x7f3bb8141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44210/0 (socket says 172.21.15.92:44210) 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 -- 172.21.15.92:0/974447196 learned_addr learned my addr 172.21.15.92:0/974447196 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3bb77fe700 1 --2- 172.21.15.92:0/974447196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb806bcd0 0x7f3bb8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 -- 172.21.15.92:0/974447196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb806bcd0 msgr2=0x7f3bb8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 --2- 172.21.15.92:0/974447196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb806bcd0 0x7f3bb8140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 -- 172.21.15.92:0/974447196 >> v1:172.21.15.136:6789/0 conn(0x7f3bb80c8e00 legacy=0x7f3bb80c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 -- 172.21.15.92:0/974447196 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb815bc50 con 0x7f3bb806c0a0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3baeffd700 1 --2- 172.21.15.92:0/974447196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806c0a0 0x7f3bb8141330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00099b0 tx=0x7f3ba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d2b4ed2b81a478 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3bb6ffd700 1 -- 172.21.15.92:0/974447196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000b820 con 0x7f3bb806c0a0 2023-10-06T03:01:42.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3bb6ffd700 1 -- 172.21.15.92:0/974447196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba000b980 con 0x7f3bb806c0a0 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.821+0000 7f3bb6ffd700 1 -- 172.21.15.92:0/974447196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000bd20 con 0x7f3bb806c0a0 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 -- 172.21.15.92:0/974447196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806c0a0 msgr2=0x7f3bb8141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/974447196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806c0a0 0x7f3bb8141330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00099b0 tx=0x7f3ba0009250 comp rx=0 tx=0).stop 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 -- 172.21.15.92:0/974447196 shutdown_connections 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/974447196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb806bcd0 0x7f3bb8140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/974447196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806c0a0 0x7f3bb8141330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 -- 172.21.15.92:0/974447196 >> 172.21.15.92:0/974447196 conn(0x7f3bb80ce600 msgr2=0x7f3bb80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 -- 172.21.15.92:0/974447196 shutdown_connections 2023-10-06T03:01:42.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.822+0000 7f3bbdf31700 1 -- 172.21.15.92:0/974447196 wait complete. 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.823+0000 7f3bbdf31700 1 Processor -- start 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.823+0000 7f3bbdf31700 1 -- start start 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 0x7f3bb80c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb806c0a0 0x7f3bb813bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb80c8e00 0x7f3bb80c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb813d9a0 con 0x7f3bb806bcd0 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb813e1d0 con 0x7f3bb806c0a0 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80b3340 con 0x7f3bb80c8e00 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 0x7f3bb80c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 0x7f3bb80c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44220/0 (socket says 172.21.15.92:44220) 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 -- 172.21.15.92:0/2114017378 learned_addr learned my addr 172.21.15.92:0/2114017378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb806c0a0 0x7f3bb813bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb7fff700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb80c8e00 0x7f3bb80c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 -- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb80c8e00 msgr2=0x7f3bb80c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb80c8e00 0x7f3bb80c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 -- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb806c0a0 msgr2=0x7f3bb813bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb806c0a0 0x7f3bb813bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 -- 172.21.15.92:0/2114017378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba00094f0 con 0x7f3bb806bcd0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bb77fe700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 0x7f3bb80c8650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c880 tx=0x7f3ba8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8011600 con 0x7f3bb806bcd0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba8011760 con 0x7f3bb806bcd0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb813db80 con 0x7f3bb806bcd0 2023-10-06T03:01:42.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8011b00 con 0x7f3bb806bcd0 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.824+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb813c2f0 con 0x7f3bb806bcd0 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.825+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb804c3c0 con 0x7f3bb806bcd0 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.825+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ba8018360 con 0x7f3bb806bcd0 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.826+0000 7f3baffff700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b98066f30 0x7f3b980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.826+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ba8084540 con 0x7f3bb806bcd0 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.826+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b98066f30 0x7f3b980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:42.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.826+0000 7f3baeffd700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b98066f30 0x7f3b980693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00099b0 tx=0x7f3ba0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:42.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:42.830+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ba8051650 con 0x7f3bb806bcd0 2023-10-06T03:01:43.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.014+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bb806c9d0 con 0x7f3bb806bcd0 2023-10-06T03:01:43.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.015+0000 7f3baffff700 1 -- 172.21.15.92:0/2114017378 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f3ba8054c70 con 0x7f3bb806bcd0 2023-10-06T03:01:43.015 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:43.016 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":22,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:41.756921+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:43.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b98066f30 msgr2=0x7f3b980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b98066f30 0x7f3b980693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00099b0 tx=0x7f3ba0016000 comp rx=0 tx=0).stop 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 msgr2=0x7f3bb80c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 0x7f3bb80c8650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c880 tx=0x7f3ba8008040 comp rx=0 tx=0).stop 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 shutdown_connections 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b98066f30 0x7f3b980693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bb806bcd0 0x7f3bb80c8650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bb80c8e00 0x7f3bb80c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 --2- 172.21.15.92:0/2114017378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb806c0a0 0x7f3bb813bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 >> 172.21.15.92:0/2114017378 conn(0x7f3bb80ce600 msgr2=0x7f3bb80cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 shutdown_connections 2023-10-06T03:01:43.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.017+0000 7f3bbdf31700 1 -- 172.21.15.92:0/2114017378 wait complete. 2023-10-06T03:01:43.029 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:01:43.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.145+0000 7f09460e1700 1 Processor -- start 2023-10-06T03:01:43.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.145+0000 7f09460e1700 1 -- start start 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f09460e1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0940159c60 0x7f094015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f09460e1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094015a160 con 0x7f0940159890 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f09460e1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09400d13a0 con 0x7f094015db10 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f09460e1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094015df50 con 0x7f0940159c60 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0940159c60 0x7f094015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0940159890 0x7f094015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34294/0 (socket says 172.21.15.92:34294) 2023-10-06T03:01:43.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093f7fe700 1 -- 172.21.15.92:0/1714912840 learned_addr learned my addr 172.21.15.92:0/1714912840 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093effd700 1 -- 172.21.15.92:0/1714912840 >> v1:172.21.15.136:6789/0 conn(0x7f094015db10 legacy=0x7f0940069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093effd700 1 -- 172.21.15.92:0/1714912840 >> v1:172.21.15.92:6789/0 conn(0x7f0940159890 legacy=0x7f094015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093effd700 1 -- 172.21.15.92:0/1714912840 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094015e140 con 0x7f0940159c60 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093e7fc700 1 -- 172.21.15.92:0/1714912840 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037708735 0 0) 0x7f094015a160 con 0x7f0940159890 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093effd700 1 --2- 172.21.15.92:0/1714912840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0940159c60 0x7f094015d5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f09280098e0 tx=0x7f09280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6190d8325281c0c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093e7fc700 1 -- 172.21.15.92:0/1714912840 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928009e80 con 0x7f0940159c60 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093e7fc700 1 -- 172.21.15.92:0/1714912840 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092800b820 con 0x7f0940159c60 2023-10-06T03:01:43.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.146+0000 7f093e7fc700 1 -- 172.21.15.92:0/1714912840 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092800bba0 con 0x7f0940159c60 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 -- 172.21.15.92:0/1714912840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0940159c60 msgr2=0x7f094015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 --2- 172.21.15.92:0/1714912840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0940159c60 0x7f094015d5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f09280098e0 tx=0x7f09280091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 -- 172.21.15.92:0/1714912840 shutdown_connections 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 --2- 172.21.15.92:0/1714912840 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0940159c60 0x7f094015d5d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 -- 172.21.15.92:0/1714912840 >> 172.21.15.92:0/1714912840 conn(0x7f0940068e20 msgr2=0x7f09400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 -- 172.21.15.92:0/1714912840 shutdown_connections 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.147+0000 7f09460e1700 1 -- 172.21.15.92:0/1714912840 wait complete. 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.148+0000 7f09460e1700 1 Processor -- start 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.148+0000 7f09460e1700 1 -- start start 2023-10-06T03:01:43.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f09460e1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0940159890 0x7f09401504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f09460e1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0940159c60 0x7f09400cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f09460e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 0x7f09401584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f09460e1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400ce610 con 0x7f094015db10 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f09460e1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400cda00 con 0x7f0940159c60 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f09460e1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09400cdce0 con 0x7f0940159890 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 0x7f09401584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 0x7f09401584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44240/0 (socket says 172.21.15.92:44240) 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 -- 172.21.15.92:0/2537746452 learned_addr learned my addr 172.21.15.92:0/2537746452 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093f7fe700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0940159890 0x7f09401504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093effd700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0940159c60 0x7f09400cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 -- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0940159890 msgr2=0x7f09401504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0940159890 0x7f09401504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 -- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0940159c60 msgr2=0x7f09400cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0940159c60 0x7f09400cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 -- 172.21.15.92:0/2537746452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0928009420 con 0x7f094015db10 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.149+0000 7f093ffff700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 0x7f09401584b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f093000c5b0 tx=0x7f0930007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.150+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930011590 con 0x7f094015db10 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.150+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09300116f0 con 0x7f094015db10 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.150+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09400cdfc0 con 0x7f094015db10 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.150+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930011ab0 con 0x7f094015db10 2023-10-06T03:01:43.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.150+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0940150a00 con 0x7f094015db10 2023-10-06T03:01:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.151+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094004c3c0 con 0x7f094015db10 2023-10-06T03:01:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.151+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f093001c460 con 0x7f094015db10 2023-10-06T03:01:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.151+0000 7f09377fe700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0920066f30 0x7f09200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.151+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09300842c0 con 0x7f094015db10 2023-10-06T03:01:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.151+0000 7f093f7fe700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0920066f30 0x7f09200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:43.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.152+0000 7f093f7fe700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0920066f30 0x7f09200693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0938002180 tx=0x7f0938009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:43.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.155+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09300513d0 con 0x7f094015db10 2023-10-06T03:01:43.356 INFO:tasks.ceph.mds.d.smithi136.stderr:2023-10-06T03:01:43.356+0000 7fabfec2c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:01:43.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:43.385+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f094006a5b0 con 0x7f094015db10 2023-10-06T03:01:44.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.275+0000 7f09377fe700 1 -- 172.21.15.92:0/2537746452 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f09300549f0 con 0x7f094015db10 2023-10-06T03:01:44.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.277+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0920066f30 msgr2=0x7f09200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.277+0000 7f09460e1700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0920066f30 0x7f09200693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0938002180 tx=0x7f0938009040 comp rx=0 tx=0).stop 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.277+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 msgr2=0x7f09401584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.277+0000 7f09460e1700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 0x7f09401584b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f093000c5b0 tx=0x7f0930007f60 comp rx=0 tx=0).stop 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 shutdown_connections 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0920066f30 0x7f09200693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f094015db10 0x7f09401584b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0940159890 0x7f09401504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 --2- 172.21.15.92:0/2537746452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0940159c60 0x7f09400cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 >> 172.21.15.92:0/2537746452 conn(0x7f0940068e20 msgr2=0x7f0940153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 shutdown_connections 2023-10-06T03:01:44.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.278+0000 7f09460e1700 1 -- 172.21.15.92:0/2537746452 wait complete. 2023-10-06T03:01:44.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:01:44.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.415+0000 7faf5edd2700 1 Processor -- start 2023-10-06T03:01:44.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.415+0000 7faf5edd2700 1 -- start start 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.416+0000 7faf5edd2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf58141500 0x7faf580c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.416+0000 7faf5edd2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5815c2a0 con 0x7faf5806bcd0 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.416+0000 7faf5edd2700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5815b690 con 0x7faf5806c0a0 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.416+0000 7faf5edd2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5815b970 con 0x7faf58141500 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.416+0000 7faf5cb6e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faf5806bcd0 0x7faf58140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34310/0 (socket says 172.21.15.92:34310) 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.416+0000 7faf5cb6e700 1 -- 172.21.15.92:0/2812461415 learned_addr learned my addr 172.21.15.92:0/2812461415 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf58141500 0x7faf580c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940816044 0 0) 0x7faf5815c2a0 con 0x7faf5806bcd0 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf40003570 con 0x7faf5806bcd0 2023-10-06T03:01:44.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722228960 0 0) 0x7faf5815b690 con 0x7faf5806c0a0 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf5815c2a0 con 0x7faf5806c0a0 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620113923 0 0) 0x7faf40003570 con 0x7faf5806bcd0 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf58141500 msgr2=0x7faf580c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf58141500 0x7faf580c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 >> v1:172.21.15.136:6789/0 conn(0x7faf5806c0a0 legacy=0x7faf580c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5815bc50 con 0x7faf5806bcd0 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7faf540017f0 con 0x7faf5806bcd0 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 >> v1:172.21.15.92:6789/0 conn(0x7faf5806bcd0 legacy=0x7faf58140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf40003c20 0x7faf40004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf400051c0 0x7faf4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5815b970 con 0x7faf40004810 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4000c020 con 0x7faf400051c0 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4000c1e0 con 0x7faf40003c20 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf58141500 0x7faf580c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf5cb6e700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf40003c20 0x7faf40004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf5d36f700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf400051c0 0x7faf4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 -- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf40003c20 msgr2=0x7faf40004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf40003c20 0x7faf40004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 -- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf400051c0 msgr2=0x7faf4000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf400051c0 0x7faf4000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.417+0000 7faf4ffff700 1 -- 172.21.15.92:0/2812461415 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40003570 con 0x7faf40004810 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf4ffff700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf40004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf580b4600 tx=0x7faf480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83581e138edefe68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800e710 con 0x7faf40004810 2023-10-06T03:01:44.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4800e870 con 0x7faf40004810 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf4f7fe700 1 -- 172.21.15.92:0/2812461415 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800ec10 con 0x7faf40004810 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5edd2700 1 -- 172.21.15.92:0/2812461415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 msgr2=0x7faf40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5edd2700 1 --2- 172.21.15.92:0/2812461415 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf40004c80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faf580b4600 tx=0x7faf480091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5d36f700 1 -- 172.21.15.92:0/2812461415 reap_dead start 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5edd2700 1 -- 172.21.15.92:0/2812461415 shutdown_connections 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5edd2700 1 -- 172.21.15.92:0/2812461415 >> 172.21.15.92:0/2812461415 conn(0x7faf580ce600 msgr2=0x7faf580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5edd2700 1 -- 172.21.15.92:0/2812461415 shutdown_connections 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.418+0000 7faf5edd2700 1 -- 172.21.15.92:0/2812461415 wait complete. 2023-10-06T03:01:44.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.419+0000 7faf5edd2700 1 Processor -- start 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 -- start start 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf5813c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf5813c930 0x7faf580b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580b2b00 0x7faf5815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580b2f80 con 0x7faf40004810 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580b15b0 con 0x7faf580b2b00 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5edd2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf580b2130 con 0x7faf5813c930 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5cb6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf5813c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5cb6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf5813c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44254/0 (socket says 172.21.15.92:44254) 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.420+0000 7faf5cb6e700 1 -- 172.21.15.92:0/3041436929 learned_addr learned my addr 172.21.15.92:0/3041436929 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:44.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf4ffff700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf5813c930 0x7faf580b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5d36f700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580b2b00 0x7faf5815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5cb6e700 1 -- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf5813c930 msgr2=0x7faf580b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5cb6e700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf5813c930 0x7faf580b25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5cb6e700 1 -- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580b2b00 msgr2=0x7faf5815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5cb6e700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580b2b00 0x7faf5815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5cb6e700 1 -- 172.21.15.92:0/3041436929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580b1800 con 0x7faf40004810 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5d36f700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580b2b00 0x7faf5815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf4ffff700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf5813c930 0x7faf580b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:44.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5cb6e700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf5813c3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf54007ac0 tx=0x7faf5806e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:44.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5400aca0 con 0x7faf40004810 2023-10-06T03:01:44.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5400ae20 con 0x7faf40004810 2023-10-06T03:01:44.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf580b1ae0 con 0x7faf40004810 2023-10-06T03:01:44.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54013640 con 0x7faf40004810 2023-10-06T03:01:44.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.421+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf58156560 con 0x7faf40004810 2023-10-06T03:01:44.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.422+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf581566f0 con 0x7faf40004810 2023-10-06T03:01:44.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.422+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf540139f0 con 0x7faf40004810 2023-10-06T03:01:44.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.423+0000 7faf4cff9700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faf44066f30 0x7faf440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:44.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.423+0000 7faf4ffff700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faf44066f30 0x7faf440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:44.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.423+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf54085320 con 0x7faf40004810 2023-10-06T03:01:44.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.423+0000 7faf4ffff700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faf44066f30 0x7faf440693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf480091e0 tx=0x7faf48019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:44.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.427+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf54052430 con 0x7faf40004810 2023-10-06T03:01:44.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:44.652+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faf5804c3c0 con 0x7faf40004810 2023-10-06T03:01:45.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.280+0000 7faf4cff9700 1 -- 172.21.15.92:0/3041436929 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7faf54055a50 con 0x7faf40004810 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.282+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faf44066f30 msgr2=0x7faf440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.282+0000 7faf5edd2700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faf44066f30 0x7faf440693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf480091e0 tx=0x7faf48019000 comp rx=0 tx=0).stop 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.282+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 msgr2=0x7faf5813c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.282+0000 7faf5edd2700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf5813c3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf54007ac0 tx=0x7faf5806e470 comp rx=0 tx=0).stop 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 shutdown_connections 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faf44066f30 0x7faf440693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf40004810 0x7faf5813c3f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf580b2b00 0x7faf5815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 --2- 172.21.15.92:0/3041436929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf5813c930 0x7faf580b25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 >> 172.21.15.92:0/3041436929 conn(0x7faf580ce600 msgr2=0x7faf58158090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:45.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 shutdown_connections 2023-10-06T03:01:45.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.283+0000 7faf5edd2700 1 -- 172.21.15.92:0/3041436929 wait complete. 2023-10-06T03:01:45.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:01:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.411+0000 7f4634055700 1 Processor -- start 2023-10-06T03:01:45.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.411+0000 7f4634055700 1 -- start start 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4634055700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f462c06aa40 0x7f462c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4634055700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4634055700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f462c06b350 0x7f462c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4634055700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c0cc210 con 0x7f462c06aa40 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4634055700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c0cc4f0 con 0x7f462c06b350 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4634055700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0cc7d0 con 0x7f462c06bac0 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4631df1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f462c06aa40 0x7f462c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4631df1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f462c06aa40 0x7f462c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44264/0 (socket says 172.21.15.92:44264) 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.412+0000 7f4631df1700 1 -- 172.21.15.92:0/1399608241 learned_addr learned my addr 172.21.15.92:0/1399608241 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:45.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f46325f2700 1 --2- 172.21.15.92:0/1399608241 >> v2:172.21.15.136:3300/0 conn(0x7f462c06b350 0x7f462c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f46315f0700 1 --2- 172.21.15.92:0/1399608241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4631df1700 1 -- 172.21.15.92:0/1399608241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 msgr2=0x7f462c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4631df1700 1 --2- 172.21.15.92:0/1399608241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4631df1700 1 -- 172.21.15.92:0/1399608241 >> v2:172.21.15.136:3300/0 conn(0x7f462c06b350 msgr2=0x7f462c0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4631df1700 1 --2- 172.21.15.92:0/1399608241 >> v2:172.21.15.136:3300/0 conn(0x7f462c06b350 0x7f462c0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4631df1700 1 -- 172.21.15.92:0/1399608241 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c0ccab0 con 0x7f462c06aa40 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4631df1700 1 --2- 172.21.15.92:0/1399608241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06aa40 0x7f462c0cbc00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f462c0c7c40 tx=0x7f4628009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9eb4f71c7d72a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4630def700 1 -- 172.21.15.92:0/1399608241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800e070 con 0x7f462c06aa40 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4630def700 1 -- 172.21.15.92:0/1399608241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462800bad0 con 0x7f462c06aa40 2023-10-06T03:01:45.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4630def700 1 -- 172.21.15.92:0/1399608241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46280092e0 con 0x7f462c06aa40 2023-10-06T03:01:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.413+0000 7f4634055700 1 -- 172.21.15.92:0/1399608241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06aa40 msgr2=0x7f462c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 --2- 172.21.15.92:0/1399608241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06aa40 0x7f462c0cbc00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f462c0c7c40 tx=0x7f4628009840 comp rx=0 tx=0).stop 2023-10-06T03:01:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 -- 172.21.15.92:0/1399608241 shutdown_connections 2023-10-06T03:01:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 --2- 172.21.15.92:0/1399608241 >> v2:172.21.15.136:3300/0 conn(0x7f462c06b350 0x7f462c0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 --2- 172.21.15.92:0/1399608241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06aa40 0x7f462c0cbc00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 --2- 172.21.15.92:0/1399608241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 -- 172.21.15.92:0/1399608241 >> 172.21.15.92:0/1399608241 conn(0x7f462c0cd670 msgr2=0x7f462c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 -- 172.21.15.92:0/1399608241 shutdown_connections 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.414+0000 7f4634055700 1 -- 172.21.15.92:0/1399608241 wait complete. 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 Processor -- start 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 -- start start 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c06aa40 0x7f462c168790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 0x7f462c168df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c13d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c169330 con 0x7f462c06b350 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c16a170 con 0x7f462c06aa40 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.415+0000 7f4634055700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c169560 con 0x7f462c06bac0 2023-10-06T03:01:45.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 0x7f462c168df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 0x7f462c168df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44266/0 (socket says 172.21.15.92:44266) 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 -- 172.21.15.92:0/2955835303 learned_addr learned my addr 172.21.15.92:0/2955835303 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f4631df1700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c06aa40 0x7f462c168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46325f2700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c13d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 -- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 msgr2=0x7f462c13d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c13d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 -- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c06aa40 msgr2=0x7f462c168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c06aa40 0x7f462c168790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 -- 172.21.15.92:0/2955835303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628009ae0 con 0x7f462c06b350 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f46315f0700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 0x7f462c168df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d860 tx=0x7f461c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c012630 con 0x7f462c06b350 2023-10-06T03:01:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c012790 con 0x7f462c06b350 2023-10-06T03:01:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c012b50 con 0x7f462c06b350 2023-10-06T03:01:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c169840 con 0x7f462c06b350 2023-10-06T03:01:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.416+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c169e00 con 0x7f462c06b350 2023-10-06T03:01:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.417+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c16a2e0 con 0x7f462c06b350 2023-10-06T03:01:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.417+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f461c01d400 con 0x7f462c06b350 2023-10-06T03:01:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.418+0000 7f4621ffb700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.418+0000 7f4631df1700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4610066f30 0x7f46100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.418+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f461c084a50 con 0x7f462c06b350 2023-10-06T03:01:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.418+0000 7f4631df1700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f4628018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:45.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.422+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f461c0555f0 con 0x7f462c06b350 2023-10-06T03:01:45.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.644+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f462c04c3c0 con 0x7f462c06b350 2023-10-06T03:01:45.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.644+0000 7f4621ffb700 1 -- 172.21.15.92:0/2955835303 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f461c055180 con 0x7f462c06b350 2023-10-06T03:01:45.646 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:45.646 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:01:45.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4610066f30 msgr2=0x7f46100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f4628018000 comp rx=0 tx=0).stop 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 msgr2=0x7f462c168df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 0x7f462c168df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d860 tx=0x7f461c008040 comp rx=0 tx=0).stop 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 shutdown_connections 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4610066f30 0x7f46100693e0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f4628018000 comp rx=0 tx=0).stop 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f462c06bac0 0x7f462c13d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f462c06aa40 0x7f462c168790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 --2- 172.21.15.92:0/2955835303 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f462c06b350 0x7f462c168df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 >> 172.21.15.92:0/2955835303 conn(0x7f462c0cd670 msgr2=0x7f462c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.646+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 shutdown_connections 2023-10-06T03:01:45.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.647+0000 7f4634055700 1 -- 172.21.15.92:0/2955835303 wait complete. 2023-10-06T03:01:45.650 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:01:45.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:45.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.773+0000 7fe0c3076700 1 Processor -- start 2023-10-06T03:01:45.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.773+0000 7fe0c3076700 1 -- start start 2023-10-06T03:01:45.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.773+0000 7fe0c3076700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0bc159c60 0x7fe0bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0c3076700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc15a160 con 0x7fe0bc15db10 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0c3076700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0d13a0 con 0x7fe0bc159890 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0c3076700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc15df50 con 0x7fe0bc159c60 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe0bc159c60 0x7fe0bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0c1613700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe0bc15db10 0x7fe0bc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34316/0 (socket says 172.21.15.92:34316) 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0c1613700 1 -- 172.21.15.92:0/3333289725 learned_addr learned my addr 172.21.15.92:0/3333289725 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:45.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b3fff700 1 -- 172.21.15.92:0/3333289725 >> v1:172.21.15.136:6789/0 conn(0x7fe0bc159890 legacy=0x7fe0bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b3fff700 1 -- 172.21.15.92:0/3333289725 >> v1:172.21.15.92:6789/0 conn(0x7fe0bc15db10 legacy=0x7fe0bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b37fe700 1 -- 172.21.15.92:0/3333289725 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851110541 0 0) 0x7fe0bc15a160 con 0x7fe0bc15db10 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b3fff700 1 -- 172.21.15.92:0/3333289725 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc15e140 con 0x7fe0bc159c60 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b3fff700 1 --2- 172.21.15.92:0/3333289725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc159c60 0x7fe0bc15d5d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0098e0 tx=0x7fe0ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a911ad22eac37b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b37fe700 1 -- 172.21.15.92:0/3333289725 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac009e80 con 0x7fe0bc159c60 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b37fe700 1 -- 172.21.15.92:0/3333289725 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00b860 con 0x7fe0bc159c60 2023-10-06T03:01:45.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.774+0000 7fe0b37fe700 1 -- 172.21.15.92:0/3333289725 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00bbe0 con 0x7fe0bc159c60 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 -- 172.21.15.92:0/3333289725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc159c60 msgr2=0x7fe0bc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 --2- 172.21.15.92:0/3333289725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc159c60 0x7fe0bc15d5d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0098e0 tx=0x7fe0ac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 -- 172.21.15.92:0/3333289725 shutdown_connections 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 --2- 172.21.15.92:0/3333289725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc159c60 0x7fe0bc15d5d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 -- 172.21.15.92:0/3333289725 >> 172.21.15.92:0/3333289725 conn(0x7fe0bc068e20 msgr2=0x7fe0bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 -- 172.21.15.92:0/3333289725 shutdown_connections 2023-10-06T03:01:45.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.775+0000 7fe0c3076700 1 -- 172.21.15.92:0/3333289725 wait complete. 2023-10-06T03:01:45.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.776+0000 7fe0c3076700 1 Processor -- start 2023-10-06T03:01:45.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.776+0000 7fe0c3076700 1 -- start start 2023-10-06T03:01:45.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0bc159890 0x7fe0bc0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0bc159c60 0x7fe0bc0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 0x7fe0bc0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0ca650 con 0x7fe0bc15db10 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc16d0e0 con 0x7fe0bc159890 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc16a880 con 0x7fe0bc159c60 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 0x7fe0bc0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 0x7fe0bc0c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44282/0 (socket says 172.21.15.92:44282) 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 -- 172.21.15.92:0/2923160622 learned_addr learned my addr 172.21.15.92:0/2923160622 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:45.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0b3fff700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0bc159c60 0x7fe0bc0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c0e12700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0bc159890 0x7fe0bc0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 -- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0bc159c60 msgr2=0x7fe0bc0c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0bc159c60 0x7fe0bc0c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 -- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0bc159890 msgr2=0x7fe0bc0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0bc159890 0x7fe0bc0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 -- 172.21.15.92:0/2923160622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac009420 con 0x7fe0bc15db10 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0b3fff700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0bc159c60 0x7fe0bc0c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:45.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c1613700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 0x7fe0bc0c6b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400c4f0 tx=0x7fe0b4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4005b30 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4005cb0 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc16d330 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.777+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4007320 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.778+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc16d8f0 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.779+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc04c3c0 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.779+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0b401b3b0 con 0x7fe0bc15db10 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.779+0000 7fe0b0ff9700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe0a8066f30 0x7fe0a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:45.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.779+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe0b4083bf0 con 0x7fe0bc15db10 2023-10-06T03:01:45.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.779+0000 7fe0c0e12700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe0a8066f30 0x7fe0a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:45.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.780+0000 7fe0c0e12700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe0a8066f30 0x7fe0a80693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8007fd0 tx=0x7fe0b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:45.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.783+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0b4050c70 con 0x7fe0bc15db10 2023-10-06T03:01:45.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.966+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0bc16dab0 con 0x7fe0bc15db10 2023-10-06T03:01:45.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.966+0000 7fe0b0ff9700 1 -- 172.21.15.92:0/2923160622 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fe0b4017020 con 0x7fe0bc15db10 2023-10-06T03:01:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:45.968 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe0a8066f30 msgr2=0x7fe0a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe0a8066f30 0x7fe0a80693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8007fd0 tx=0x7fe0b8009040 comp rx=0 tx=0).stop 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 msgr2=0x7fe0bc0c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 0x7fe0bc0c6b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400c4f0 tx=0x7fe0b4007f50 comp rx=0 tx=0).stop 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 shutdown_connections 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe0a8066f30 0x7fe0a80693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0bc15db10 0x7fe0bc0c6b70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0bc159890 0x7fe0bc0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 --2- 172.21.15.92:0/2923160622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0bc159c60 0x7fe0bc0c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 >> 172.21.15.92:0/2923160622 conn(0x7fe0bc068e20 msgr2=0x7fe0bc153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 shutdown_connections 2023-10-06T03:01:45.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:45.969+0000 7fe0c3076700 1 -- 172.21.15.92:0/2923160622 wait complete. 2023-10-06T03:01:45.983 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:01:45.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:01:46.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.095+0000 7f63ece38700 1 Processor -- start 2023-10-06T03:01:46.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.095+0000 7f63ece38700 1 -- start start 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ece38700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63e8159890 0x7f63e815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ece38700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f63e8159c60 0x7f63e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ece38700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ece38700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e815df50 con 0x7f63e8159890 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ece38700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e815a160 con 0x7f63e8159c60 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ece38700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e80d13a0 con 0x7f63e815db10 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63e8159890 0x7f63e815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f63e8159890 0x7f63e815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44286/0 (socket says 172.21.15.92:44286) 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 -- 172.21.15.92:0/4082359842 learned_addr learned my addr 172.21.15.92:0/4082359842 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e6d9d700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63ddd9b700 1 --2- 172.21.15.92:0/4082359842 >> v2:172.21.15.136:3300/0 conn(0x7f63e8159c60 0x7f63e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 -- 172.21.15.92:0/4082359842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 msgr2=0x7f63e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 -- 172.21.15.92:0/4082359842 >> v2:172.21.15.136:3300/0 conn(0x7f63e8159c60 msgr2=0x7f63e815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 --2- 172.21.15.92:0/4082359842 >> v2:172.21.15.136:3300/0 conn(0x7f63e8159c60 0x7f63e815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e659c700 1 -- 172.21.15.92:0/4082359842 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e815e140 con 0x7f63e8159890 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.096+0000 7f63e6d9d700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63e659c700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159890 0x7f63e815abd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63e80b67f0 tx=0x7f63d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc4aef36c62f416 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63e5d9b700 1 -- 172.21.15.92:0/4082359842 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800b860 con 0x7f63e8159890 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63e5d9b700 1 -- 172.21.15.92:0/4082359842 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d800b9c0 con 0x7f63e8159890 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63e5d9b700 1 -- 172.21.15.92:0/4082359842 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800bd60 con 0x7f63e8159890 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 -- 172.21.15.92:0/4082359842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159890 msgr2=0x7f63e815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159890 0x7f63e815abd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f63e80b67f0 tx=0x7f63d8009250 comp rx=0 tx=0).stop 2023-10-06T03:01:46.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 -- 172.21.15.92:0/4082359842 shutdown_connections 2023-10-06T03:01:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 --2- 172.21.15.92:0/4082359842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159890 0x7f63e815abd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 --2- 172.21.15.92:0/4082359842 >> v2:172.21.15.136:3300/0 conn(0x7f63e8159c60 0x7f63e815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 -- 172.21.15.92:0/4082359842 >> 172.21.15.92:0/4082359842 conn(0x7f63e8068e20 msgr2=0x7f63e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 -- 172.21.15.92:0/4082359842 shutdown_connections 2023-10-06T03:01:46.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.097+0000 7f63ece38700 1 -- 172.21.15.92:0/4082359842 wait complete. 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.098+0000 7f63ece38700 1 Processor -- start 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 -- start start 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e8159890 0x7f63e8151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 0x7f63e80b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e80b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8169b40 con 0x7f63e8159c60 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e8169d50 con 0x7f63e8159890 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ece38700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e816a030 con 0x7f63e815db10 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ddd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 0x7f63e80b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ddd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 0x7f63e80b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44292/0 (socket says 172.21.15.92:44292) 2023-10-06T03:01:46.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63ddd9b700 1 -- 172.21.15.92:0/3571718583 learned_addr learned my addr 172.21.15.92:0/3571718583 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63e6d9d700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e80b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.099+0000 7f63e659c700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e8159890 0x7f63e8151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ddd9b700 1 -- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 msgr2=0x7f63e80b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ddd9b700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e80b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ddd9b700 1 -- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e8159890 msgr2=0x7f63e8151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ddd9b700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e8159890 0x7f63e8151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ddd9b700 1 -- 172.21.15.92:0/3571718583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80094f0 con 0x7f63e8159c60 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ddd9b700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 0x7f63e80b5940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63d000d890 tx=0x7f63d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d00125f0 con 0x7f63e8159c60 2023-10-06T03:01:46.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d0012750 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e816a310 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0012af0 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.100+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e816a8d0 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.101+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e816aaf0 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.101+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63d001e460 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.102+0000 7f63deffd700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f63c8066f30 0x7f63c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.102+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f63d0084c00 con 0x7f63e8159c60 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.102+0000 7f63e659c700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f63c8066f30 0x7f63c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.102+0000 7f63e659c700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f63c8066f30 0x7f63c80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e80b67f0 tx=0x7f63d8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.106+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63d008b080 con 0x7f63e8159c60 2023-10-06T03:01:46.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.286+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63e804c3c0 con 0x7f63c8066f30 2023-10-06T03:01:46.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.294+0000 7f63deffd700 1 -- 172.21.15.92:0/3571718583 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286724 (secure 0 0 0) 0x7f63e804c3c0 con 0x7f63c8066f30 2023-10-06T03:01:46.299 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.297+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f63c8066f30 msgr2=0x7f63c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.297+0000 7f63ece38700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f63c8066f30 0x7f63c80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f63e80b67f0 tx=0x7f63d8017000 comp rx=0 tx=0).stop 2023-10-06T03:01:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 msgr2=0x7f63e80b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 0x7f63e80b5940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f63d000d890 tx=0x7f63d0008040 comp rx=0 tx=0).stop 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 shutdown_connections 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f63c8066f30 0x7f63c80693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63e815db10 0x7f63e80b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63e8159890 0x7f63e8151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 --2- 172.21.15.92:0/3571718583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f63e8159c60 0x7f63e80b5940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 >> 172.21.15.92:0/3571718583 conn(0x7f63e8068e20 msgr2=0x7f63e80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 shutdown_connections 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.298+0000 7f63ece38700 1 -- 172.21.15.92:0/3571718583 wait complete. 2023-10-06T03:01:46.302 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:01:46.311 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T03:01:45.183022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381380,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593340,"statfs":{"total":773094113280,"available":770655580160,"internally_reserved":0,"allocated":3149824,"data_stored":1879977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.825470"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280436+0000","last_change":"2023-10-06T03:01:42.278647+0000","last_active":"2023-10-06T03:01:42.280436+0000","last_peered":"2023-10-06T03:01:42.280436+0000","last_clean":"2023-10-06T03:01:42.280436+0000","last_became_active":"2023-10-06T03:01:42.278364+0000","last_became_peered":"2023-10-06T03:01:42.278364+0000","last_unstale":"2023-10-06T03:01:42.280436+0000","last_undegraded":"2023-10-06T03:01:42.280436+0000","last_fullsized":"2023-10-06T03:01:42.280436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:06.498462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282043+0000","last_change":"2023-10-06T03:01:42.277062+0000","last_active":"2023-10-06T03:01:42.282043+0000","last_peered":"2023-10-06T03:01:42.282043+0000","last_clean":"2023-10-06T03:01:42.282043+0000","last_became_active":"2023-10-06T03:01:42.276189+0000","last_became_peered":"2023-10-06T03:01:42.276189+0000","last_unstale":"2023-10-06T03:01:42.282043+0000","last_undegraded":"2023-10-06T03:01:42.282043+0000","last_fullsized":"2023-10-06T03:01:42.282043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284329+0000","last_change":"2023-10-06T03:01:42.280530+0000","last_active":"2023-10-06T03:01:42.284329+0000","last_peered":"2023-10-06T03:01:42.284329+0000","last_clean":"2023-10-06T03:01:42.284329+0000","last_became_active":"2023-10-06T03:01:42.280239+0000","last_became_peered":"2023-10-06T03:01:42.280239+0000","last_unstale":"2023-10-06T03:01:42.284329+0000","last_undegraded":"2023-10-06T03:01:42.284329+0000","last_fullsized":"2023-10-06T03:01:42.284329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284290+0000","last_change":"2023-10-06T03:01:42.279732+0000","last_active":"2023-10-06T03:01:42.284290+0000","last_peered":"2023-10-06T03:01:42.284290+0000","last_clean":"2023-10-06T03:01:42.284290+0000","last_became_active":"2023-10-06T03:01:42.279445+0000","last_became_peered":"2023-10-06T03:01:42.279445+0000","last_unstale":"2023-10-06T03:01:42.284290+0000","last_undegraded":"2023-10-06T03:01:42.284290+0000","last_fullsized":"2023-10-06T03:01:42.284290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:08.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285662+0000","last_change":"2023-10-06T03:01:42.279674+0000","last_active":"2023-10-06T03:01:42.285662+0000","last_peered":"2023-10-06T03:01:42.285662+0000","last_clean":"2023-10-06T03:01:42.285662+0000","last_became_active":"2023-10-06T03:01:42.279435+0000","last_became_peered":"2023-10-06T03:01:42.279435+0000","last_unstale":"2023-10-06T03:01:42.285662+0000","last_undegraded":"2023-10-06T03:01:42.285662+0000","last_fullsized":"2023-10-06T03:01:42.285662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284581+0000","last_change":"2023-10-06T03:01:42.282579+0000","last_active":"2023-10-06T03:01:42.284581+0000","last_peered":"2023-10-06T03:01:42.284581+0000","last_clean":"2023-10-06T03:01:42.284581+0000","last_became_active":"2023-10-06T03:01:42.282276+0000","last_became_peered":"2023-10-06T03:01:42.282276+0000","last_unstale":"2023-10-06T03:01:42.284581+0000","last_undegraded":"2023-10-06T03:01:42.284581+0000","last_fullsized":"2023-10-06T03:01:42.284581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286199+0000","last_change":"2023-10-06T03:01:42.282922+0000","last_active":"2023-10-06T03:01:42.286199+0000","last_peered":"2023-10-06T03:01:42.286199+0000","last_clean":"2023-10-06T03:01:42.286199+0000","last_became_active":"2023-10-06T03:01:42.282688+0000","last_became_peered":"2023-10-06T03:01:42.282688+0000","last_unstale":"2023-10-06T03:01:42.286199+0000","last_undegraded":"2023-10-06T03:01:42.286199+0000","last_fullsized":"2023-10-06T03:01:42.286199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285998+0000","last_change":"2023-10-06T03:01:42.279085+0000","last_active":"2023-10-06T03:01:42.285998+0000","last_peered":"2023-10-06T03:01:42.285998+0000","last_clean":"2023-10-06T03:01:42.285998+0000","last_became_active":"2023-10-06T03:01:42.278311+0000","last_became_peered":"2023-10-06T03:01:42.278311+0000","last_unstale":"2023-10-06T03:01:42.285998+0000","last_undegraded":"2023-10-06T03:01:42.285998+0000","last_fullsized":"2023-10-06T03:01:42.285998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285685+0000","last_change":"2023-10-06T03:01:42.280111+0000","last_active":"2023-10-06T03:01:42.285685+0000","last_peered":"2023-10-06T03:01:42.285685+0000","last_clean":"2023-10-06T03:01:42.285685+0000","last_became_active":"2023-10-06T03:01:42.279534+0000","last_became_peered":"2023-10-06T03:01:42.279534+0000","last_unstale":"2023-10-06T03:01:42.285685+0000","last_undegraded":"2023-10-06T03:01:42.285685+0000","last_fullsized":"2023-10-06T03:01:42.285685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286459+0000","last_change":"2023-10-06T03:01:42.281699+0000","last_active":"2023-10-06T03:01:42.286459+0000","last_peered":"2023-10-06T03:01:42.286459+0000","last_clean":"2023-10-06T03:01:42.286459+0000","last_became_active":"2023-10-06T03:01:42.279098+0000","last_became_peered":"2023-10-06T03:01:42.279098+0000","last_unstale":"2023-10-06T03:01:42.286459+0000","last_undegraded":"2023-10-06T03:01:42.286459+0000","last_fullsized":"2023-10-06T03:01:42.286459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283045+0000","last_change":"2023-10-06T03:01:42.277096+0000","last_active":"2023-10-06T03:01:42.283045+0000","last_peered":"2023-10-06T03:01:42.283045+0000","last_clean":"2023-10-06T03:01:42.283045+0000","last_became_active":"2023-10-06T03:01:42.276719+0000","last_became_peered":"2023-10-06T03:01:42.276719+0000","last_unstale":"2023-10-06T03:01:42.283045+0000","last_undegraded":"2023-10-06T03:01:42.283045+0000","last_fullsized":"2023-10-06T03:01:42.283045+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:13.143094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275038+0000","last_change":"2023-10-06T03:01:42.272031+0000","last_active":"2023-10-06T03:01:42.275038+0000","last_peered":"2023-10-06T03:01:42.275038+0000","last_clean":"2023-10-06T03:01:42.275038+0000","last_became_active":"2023-10-06T03:01:42.271796+0000","last_became_peered":"2023-10-06T03:01:42.271796+0000","last_unstale":"2023-10-06T03:01:42.275038+0000","last_undegraded":"2023-10-06T03:01:42.275038+0000","last_fullsized":"2023-10-06T03:01:42.275038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281109+0000","last_change":"2023-10-06T03:01:42.279147+0000","last_active":"2023-10-06T03:01:42.281109+0000","last_peered":"2023-10-06T03:01:42.281109+0000","last_clean":"2023-10-06T03:01:42.281109+0000","last_became_active":"2023-10-06T03:01:42.278911+0000","last_became_peered":"2023-10-06T03:01:42.278911+0000","last_unstale":"2023-10-06T03:01:42.281109+0000","last_undegraded":"2023-10-06T03:01:42.281109+0000","last_fullsized":"2023-10-06T03:01:42.281109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:14.170785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286021+0000","last_change":"2023-10-06T03:01:42.277711+0000","last_active":"2023-10-06T03:01:42.286021+0000","last_peered":"2023-10-06T03:01:42.286021+0000","last_clean":"2023-10-06T03:01:42.286021+0000","last_became_active":"2023-10-06T03:01:42.276996+0000","last_became_peered":"2023-10-06T03:01:42.276996+0000","last_unstale":"2023-10-06T03:01:42.286021+0000","last_undegraded":"2023-10-06T03:01:42.286021+0000","last_fullsized":"2023-10-06T03:01:42.286021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.289013+0000","last_change":"2023-10-06T03:01:42.288226+0000","last_active":"2023-10-06T03:01:42.289013+0000","last_peered":"2023-10-06T03:01:42.289013+0000","last_clean":"2023-10-06T03:01:42.289013+0000","last_became_active":"2023-10-06T03:01:42.287944+0000","last_became_peered":"2023-10-06T03:01:42.287944+0000","last_unstale":"2023-10-06T03:01:42.289013+0000","last_undegraded":"2023-10-06T03:01:42.289013+0000","last_fullsized":"2023-10-06T03:01:42.289013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284504+0000","last_change":"2023-10-06T03:01:42.281156+0000","last_active":"2023-10-06T03:01:42.284504+0000","last_peered":"2023-10-06T03:01:42.284504+0000","last_clean":"2023-10-06T03:01:42.284504+0000","last_became_active":"2023-10-06T03:01:42.280924+0000","last_became_peered":"2023-10-06T03:01:42.280924+0000","last_unstale":"2023-10-06T03:01:42.284504+0000","last_undegraded":"2023-10-06T03:01:42.284504+0000","last_fullsized":"2023-10-06T03:01:42.284504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285706+0000","last_change":"2023-10-06T03:01:42.280272+0000","last_active":"2023-10-06T03:01:42.285706+0000","last_peered":"2023-10-06T03:01:42.285706+0000","last_clean":"2023-10-06T03:01:42.285706+0000","last_became_active":"2023-10-06T03:01:42.279857+0000","last_became_peered":"2023-10-06T03:01:42.279857+0000","last_unstale":"2023-10-06T03:01:42.285706+0000","last_undegraded":"2023-10-06T03:01:42.285706+0000","last_fullsized":"2023-10-06T03:01:42.285706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284102+0000","last_change":"2023-10-06T03:01:42.278122+0000","last_active":"2023-10-06T03:01:42.284102+0000","last_peered":"2023-10-06T03:01:42.284102+0000","last_clean":"2023-10-06T03:01:42.284102+0000","last_became_active":"2023-10-06T03:01:42.277860+0000","last_became_peered":"2023-10-06T03:01:42.277860+0000","last_unstale":"2023-10-06T03:01:42.284102+0000","last_undegraded":"2023-10-06T03:01:42.284102+0000","last_fullsized":"2023-10-06T03:01:42.284102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:23.395145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283307+0000","last_change":"2023-10-06T03:01:42.281950+0000","last_active":"2023-10-06T03:01:42.283307+0000","last_peered":"2023-10-06T03:01:42.283307+0000","last_clean":"2023-10-06T03:01:42.283307+0000","last_became_active":"2023-10-06T03:01:42.281696+0000","last_became_peered":"2023-10-06T03:01:42.281696+0000","last_unstale":"2023-10-06T03:01:42.283307+0000","last_undegraded":"2023-10-06T03:01:42.283307+0000","last_fullsized":"2023-10-06T03:01:42.283307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:22.656089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284642+0000","last_change":"2023-10-06T03:01:42.281651+0000","last_active":"2023-10-06T03:01:42.284642+0000","last_peered":"2023-10-06T03:01:42.284642+0000","last_clean":"2023-10-06T03:01:42.284642+0000","last_became_active":"2023-10-06T03:01:42.278739+0000","last_became_peered":"2023-10-06T03:01:42.278739+0000","last_unstale":"2023-10-06T03:01:42.284642+0000","last_undegraded":"2023-10-06T03:01:42.284642+0000","last_fullsized":"2023-10-06T03:01:42.284642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:22.011159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286267+0000","last_change":"2023-10-06T03:01:42.283331+0000","last_active":"2023-10-06T03:01:42.286267+0000","last_peered":"2023-10-06T03:01:42.286267+0000","last_clean":"2023-10-06T03:01:42.286267+0000","last_became_active":"2023-10-06T03:01:42.283048+0000","last_became_peered":"2023-10-06T03:01:42.283048+0000","last_unstale":"2023-10-06T03:01:42.286267+0000","last_undegraded":"2023-10-06T03:01:42.286267+0000","last_fullsized":"2023-10-06T03:01:42.286267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285977+0000","last_change":"2023-10-06T03:01:42.282637+0000","last_active":"2023-10-06T03:01:42.285977+0000","last_peered":"2023-10-06T03:01:42.285977+0000","last_clean":"2023-10-06T03:01:42.285977+0000","last_became_active":"2023-10-06T03:01:42.282269+0000","last_became_peered":"2023-10-06T03:01:42.282269+0000","last_unstale":"2023-10-06T03:01:42.285977+0000","last_undegraded":"2023-10-06T03:01:42.285977+0000","last_fullsized":"2023-10-06T03:01:42.285977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281082+0000","last_change":"2023-10-06T03:01:42.279886+0000","last_active":"2023-10-06T03:01:42.281082+0000","last_peered":"2023-10-06T03:01:42.281082+0000","last_clean":"2023-10-06T03:01:42.281082+0000","last_became_active":"2023-10-06T03:01:42.279246+0000","last_became_peered":"2023-10-06T03:01:42.279246+0000","last_unstale":"2023-10-06T03:01:42.281082+0000","last_undegraded":"2023-10-06T03:01:42.281082+0000","last_fullsized":"2023-10-06T03:01:42.281082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286308+0000","last_change":"2023-10-06T03:01:42.283900+0000","last_active":"2023-10-06T03:01:42.286308+0000","last_peered":"2023-10-06T03:01:42.286308+0000","last_clean":"2023-10-06T03:01:42.286308+0000","last_became_active":"2023-10-06T03:01:42.283581+0000","last_became_peered":"2023-10-06T03:01:42.283581+0000","last_unstale":"2023-10-06T03:01:42.286308+0000","last_undegraded":"2023-10-06T03:01:42.286308+0000","last_fullsized":"2023-10-06T03:01:42.286308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287866+0000","last_change":"2023-10-06T03:01:42.277988+0000","last_active":"2023-10-06T03:01:42.287866+0000","last_peered":"2023-10-06T03:01:42.287866+0000","last_clean":"2023-10-06T03:01:42.287866+0000","last_became_active":"2023-10-06T03:01:42.277260+0000","last_became_peered":"2023-10-06T03:01:42.277260+0000","last_unstale":"2023-10-06T03:01:42.287866+0000","last_undegraded":"2023-10-06T03:01:42.287866+0000","last_fullsized":"2023-10-06T03:01:42.287866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287998+0000","last_change":"2023-10-06T03:01:42.280682+0000","last_active":"2023-10-06T03:01:42.287998+0000","last_peered":"2023-10-06T03:01:42.287998+0000","last_clean":"2023-10-06T03:01:42.287998+0000","last_became_active":"2023-10-06T03:01:42.280427+0000","last_became_peered":"2023-10-06T03:01:42.280427+0000","last_unstale":"2023-10-06T03:01:42.287998+0000","last_undegraded":"2023-10-06T03:01:42.287998+0000","last_fullsized":"2023-10-06T03:01:42.287998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:05.130474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286079+0000","last_change":"2023-10-06T03:01:42.280012+0000","last_active":"2023-10-06T03:01:42.286079+0000","last_peered":"2023-10-06T03:01:42.286079+0000","last_clean":"2023-10-06T03:01:42.286079+0000","last_became_active":"2023-10-06T03:01:42.279744+0000","last_became_peered":"2023-10-06T03:01:42.279744+0000","last_unstale":"2023-10-06T03:01:42.286079+0000","last_undegraded":"2023-10-06T03:01:42.286079+0000","last_fullsized":"2023-10-06T03:01:42.286079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373578+0000","last_change":"2023-10-06T03:01:41.380874+0000","last_active":"2023-10-06T03:01:42.373578+0000","last_peered":"2023-10-06T03:01:42.373578+0000","last_clean":"2023-10-06T03:01:42.373578+0000","last_became_active":"2023-10-06T03:01:41.379061+0000","last_became_peered":"2023-10-06T03:01:41.379061+0000","last_unstale":"2023-10-06T03:01:42.373578+0000","last_undegraded":"2023-10-06T03:01:42.373578+0000","last_fullsized":"2023-10-06T03:01:42.373578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280483+0000","last_change":"2023-10-06T03:01:42.276652+0000","last_active":"2023-10-06T03:01:42.280483+0000","last_peered":"2023-10-06T03:01:42.280483+0000","last_clean":"2023-10-06T03:01:42.280483+0000","last_became_active":"2023-10-06T03:01:42.276378+0000","last_became_peered":"2023-10-06T03:01:42.276378+0000","last_unstale":"2023-10-06T03:01:42.280483+0000","last_undegraded":"2023-10-06T03:01:42.280483+0000","last_fullsized":"2023-10-06T03:01:42.280483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270139+0000","last_change":"2023-10-06T03:01:41.379177+0000","last_active":"2023-10-06T03:01:42.270139+0000","last_peered":"2023-10-06T03:01:42.270139+0000","last_clean":"2023-10-06T03:01:42.270139+0000","last_became_active":"2023-10-06T03:01:41.378913+0000","last_became_peered":"2023-10-06T03:01:41.378913+0000","last_unstale":"2023-10-06T03:01:42.270139+0000","last_undegraded":"2023-10-06T03:01:42.270139+0000","last_fullsized":"2023-10-06T03:01:42.270139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286002+0000","last_change":"2023-10-06T03:01:42.284965+0000","last_active":"2023-10-06T03:01:42.286002+0000","last_peered":"2023-10-06T03:01:42.286002+0000","last_clean":"2023-10-06T03:01:42.286002+0000","last_became_active":"2023-10-06T03:01:42.284715+0000","last_became_peered":"2023-10-06T03:01:42.284715+0000","last_unstale":"2023-10-06T03:01:42.286002+0000","last_undegraded":"2023-10-06T03:01:42.286002+0000","last_fullsized":"2023-10-06T03:01:42.286002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268732+0000","last_change":"2023-10-06T03:01:41.383082+0000","last_active":"2023-10-06T03:01:42.268732+0000","last_peered":"2023-10-06T03:01:42.268732+0000","last_clean":"2023-10-06T03:01:42.268732+0000","last_became_active":"2023-10-06T03:01:41.382776+0000","last_became_peered":"2023-10-06T03:01:41.382776+0000","last_unstale":"2023-10-06T03:01:42.268732+0000","last_undegraded":"2023-10-06T03:01:42.268732+0000","last_fullsized":"2023-10-06T03:01:42.268732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285524+0000","last_change":"2023-10-06T03:01:42.283247+0000","last_active":"2023-10-06T03:01:42.285524+0000","last_peered":"2023-10-06T03:01:42.285524+0000","last_clean":"2023-10-06T03:01:42.285524+0000","last_became_active":"2023-10-06T03:01:42.282951+0000","last_became_peered":"2023-10-06T03:01:42.282951+0000","last_unstale":"2023-10-06T03:01:42.285524+0000","last_undegraded":"2023-10-06T03:01:42.285524+0000","last_fullsized":"2023-10-06T03:01:42.285524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269263+0000","last_change":"2023-10-06T03:01:41.381863+0000","last_active":"2023-10-06T03:01:42.269263+0000","last_peered":"2023-10-06T03:01:42.269263+0000","last_clean":"2023-10-06T03:01:42.269263+0000","last_became_active":"2023-10-06T03:01:41.381602+0000","last_became_peered":"2023-10-06T03:01:41.381602+0000","last_unstale":"2023-10-06T03:01:42.269263+0000","last_undegraded":"2023-10-06T03:01:42.269263+0000","last_fullsized":"2023-10-06T03:01:42.269263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280585+0000","last_change":"2023-10-06T03:01:42.276543+0000","last_active":"2023-10-06T03:01:42.280585+0000","last_peered":"2023-10-06T03:01:42.280585+0000","last_clean":"2023-10-06T03:01:42.280585+0000","last_became_active":"2023-10-06T03:01:42.276256+0000","last_became_peered":"2023-10-06T03:01:42.276256+0000","last_unstale":"2023-10-06T03:01:42.280585+0000","last_undegraded":"2023-10-06T03:01:42.280585+0000","last_fullsized":"2023-10-06T03:01:42.280585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267847+0000","last_change":"2023-10-06T03:01:41.378556+0000","last_active":"2023-10-06T03:01:42.267847+0000","last_peered":"2023-10-06T03:01:42.267847+0000","last_clean":"2023-10-06T03:01:42.267847+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.267847+0000","last_undegraded":"2023-10-06T03:01:42.267847+0000","last_fullsized":"2023-10-06T03:01:42.267847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:30.948926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287982+0000","last_change":"2023-10-06T03:01:42.283975+0000","last_active":"2023-10-06T03:01:42.287982+0000","last_peered":"2023-10-06T03:01:42.287982+0000","last_clean":"2023-10-06T03:01:42.287982+0000","last_became_active":"2023-10-06T03:01:42.283734+0000","last_became_peered":"2023-10-06T03:01:42.283734+0000","last_unstale":"2023-10-06T03:01:42.287982+0000","last_undegraded":"2023-10-06T03:01:42.287982+0000","last_fullsized":"2023-10-06T03:01:42.287982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.374949+0000","last_change":"2023-10-06T03:01:41.384246+0000","last_active":"2023-10-06T03:01:42.374949+0000","last_peered":"2023-10-06T03:01:42.374949+0000","last_clean":"2023-10-06T03:01:42.374949+0000","last_became_active":"2023-10-06T03:01:41.383977+0000","last_became_peered":"2023-10-06T03:01:41.383977+0000","last_unstale":"2023-10-06T03:01:42.374949+0000","last_undegraded":"2023-10-06T03:01:42.374949+0000","last_fullsized":"2023-10-06T03:01:42.374949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.372963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287980+0000","last_change":"2023-10-06T03:01:42.280871+0000","last_active":"2023-10-06T03:01:42.287980+0000","last_peered":"2023-10-06T03:01:42.287980+0000","last_clean":"2023-10-06T03:01:42.287980+0000","last_became_active":"2023-10-06T03:01:42.277563+0000","last_became_peered":"2023-10-06T03:01:42.277563+0000","last_unstale":"2023-10-06T03:01:42.287980+0000","last_undegraded":"2023-10-06T03:01:42.287980+0000","last_fullsized":"2023-10-06T03:01:42.287980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.379545+0000","last_change":"2023-10-06T03:01:41.381401+0000","last_active":"2023-10-06T03:01:42.379545+0000","last_peered":"2023-10-06T03:01:42.379545+0000","last_clean":"2023-10-06T03:01:42.379545+0000","last_became_active":"2023-10-06T03:01:41.379277+0000","last_became_peered":"2023-10-06T03:01:41.379277+0000","last_unstale":"2023-10-06T03:01:42.379545+0000","last_undegraded":"2023-10-06T03:01:42.379545+0000","last_fullsized":"2023-10-06T03:01:42.379545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284666+0000","last_change":"2023-10-06T03:01:42.281115+0000","last_active":"2023-10-06T03:01:42.284666+0000","last_peered":"2023-10-06T03:01:42.284666+0000","last_clean":"2023-10-06T03:01:42.284666+0000","last_became_active":"2023-10-06T03:01:42.278708+0000","last_became_peered":"2023-10-06T03:01:42.278708+0000","last_unstale":"2023-10-06T03:01:42.284666+0000","last_undegraded":"2023-10-06T03:01:42.284666+0000","last_fullsized":"2023-10-06T03:01:42.284666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.231633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377978+0000","last_change":"2023-10-06T03:01:41.384981+0000","last_active":"2023-10-06T03:01:42.377978+0000","last_peered":"2023-10-06T03:01:42.377978+0000","last_clean":"2023-10-06T03:01:42.377978+0000","last_became_active":"2023-10-06T03:01:41.384739+0000","last_became_peered":"2023-10-06T03:01:41.384739+0000","last_unstale":"2023-10-06T03:01:42.377978+0000","last_undegraded":"2023-10-06T03:01:42.377978+0000","last_fullsized":"2023-10-06T03:01:42.377978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280582+0000","last_change":"2023-10-06T03:01:42.276570+0000","last_active":"2023-10-06T03:01:42.280582+0000","last_peered":"2023-10-06T03:01:42.280582+0000","last_clean":"2023-10-06T03:01:42.280582+0000","last_became_active":"2023-10-06T03:01:42.276163+0000","last_became_peered":"2023-10-06T03:01:42.276163+0000","last_unstale":"2023-10-06T03:01:42.280582+0000","last_undegraded":"2023-10-06T03:01:42.280582+0000","last_fullsized":"2023-10-06T03:01:42.280582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.272049+0000","last_change":"2023-10-06T03:01:41.381162+0000","last_active":"2023-10-06T03:01:42.272049+0000","last_peered":"2023-10-06T03:01:42.272049+0000","last_clean":"2023-10-06T03:01:42.272049+0000","last_became_active":"2023-10-06T03:01:41.380885+0000","last_became_peered":"2023-10-06T03:01:41.380885+0000","last_unstale":"2023-10-06T03:01:42.272049+0000","last_undegraded":"2023-10-06T03:01:42.272049+0000","last_fullsized":"2023-10-06T03:01:42.272049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286056+0000","last_change":"2023-10-06T03:01:42.279843+0000","last_active":"2023-10-06T03:01:42.286056+0000","last_peered":"2023-10-06T03:01:42.286056+0000","last_clean":"2023-10-06T03:01:42.286056+0000","last_became_active":"2023-10-06T03:01:42.279409+0000","last_became_peered":"2023-10-06T03:01:42.279409+0000","last_unstale":"2023-10-06T03:01:42.286056+0000","last_undegraded":"2023-10-06T03:01:42.286056+0000","last_fullsized":"2023-10-06T03:01:42.286056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275415+0000","last_change":"2023-10-06T03:01:41.382403+0000","last_active":"2023-10-06T03:01:42.275415+0000","last_peered":"2023-10-06T03:01:42.275415+0000","last_clean":"2023-10-06T03:01:42.275415+0000","last_became_active":"2023-10-06T03:01:41.382119+0000","last_became_peered":"2023-10-06T03:01:41.382119+0000","last_unstale":"2023-10-06T03:01:42.275415+0000","last_undegraded":"2023-10-06T03:01:42.275415+0000","last_fullsized":"2023-10-06T03:01:42.275415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:17.777678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285768+0000","last_change":"2023-10-06T03:01:42.280092+0000","last_active":"2023-10-06T03:01:42.285768+0000","last_peered":"2023-10-06T03:01:42.285768+0000","last_clean":"2023-10-06T03:01:42.285768+0000","last_became_active":"2023-10-06T03:01:42.279833+0000","last_became_peered":"2023-10-06T03:01:42.279833+0000","last_unstale":"2023-10-06T03:01:42.285768+0000","last_undegraded":"2023-10-06T03:01:42.285768+0000","last_fullsized":"2023-10-06T03:01:42.285768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.375608+0000","last_change":"2023-10-06T03:01:41.386282+0000","last_active":"2023-10-06T03:01:42.375608+0000","last_peered":"2023-10-06T03:01:42.375608+0000","last_clean":"2023-10-06T03:01:42.375608+0000","last_became_active":"2023-10-06T03:01:41.385955+0000","last_became_peered":"2023-10-06T03:01:41.385955+0000","last_unstale":"2023-10-06T03:01:42.375608+0000","last_undegraded":"2023-10-06T03:01:42.375608+0000","last_fullsized":"2023-10-06T03:01:42.375608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287940+0000","last_change":"2023-10-06T03:01:42.283470+0000","last_active":"2023-10-06T03:01:42.287940+0000","last_peered":"2023-10-06T03:01:42.287940+0000","last_clean":"2023-10-06T03:01:42.287940+0000","last_became_active":"2023-10-06T03:01:42.283220+0000","last_became_peered":"2023-10-06T03:01:42.283220+0000","last_unstale":"2023-10-06T03:01:42.287940+0000","last_undegraded":"2023-10-06T03:01:42.287940+0000","last_fullsized":"2023-10-06T03:01:42.287940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373872+0000","last_change":"2023-10-06T03:01:41.380049+0000","last_active":"2023-10-06T03:01:42.373872+0000","last_peered":"2023-10-06T03:01:42.373872+0000","last_clean":"2023-10-06T03:01:42.373872+0000","last_became_active":"2023-10-06T03:01:41.379732+0000","last_became_peered":"2023-10-06T03:01:41.379732+0000","last_unstale":"2023-10-06T03:01:42.373872+0000","last_undegraded":"2023-10-06T03:01:42.373872+0000","last_fullsized":"2023-10-06T03:01:42.373872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:03.057587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286048+0000","last_change":"2023-10-06T03:01:42.279660+0000","last_active":"2023-10-06T03:01:42.286048+0000","last_peered":"2023-10-06T03:01:42.286048+0000","last_clean":"2023-10-06T03:01:42.286048+0000","last_became_active":"2023-10-06T03:01:42.279394+0000","last_became_peered":"2023-10-06T03:01:42.279394+0000","last_unstale":"2023-10-06T03:01:42.286048+0000","last_undegraded":"2023-10-06T03:01:42.286048+0000","last_fullsized":"2023-10-06T03:01:42.286048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377939+0000","last_change":"2023-10-06T03:01:41.384922+0000","last_active":"2023-10-06T03:01:42.377939+0000","last_peered":"2023-10-06T03:01:42.377939+0000","last_clean":"2023-10-06T03:01:42.377939+0000","last_became_active":"2023-10-06T03:01:41.384669+0000","last_became_peered":"2023-10-06T03:01:41.384669+0000","last_unstale":"2023-10-06T03:01:42.377939+0000","last_undegraded":"2023-10-06T03:01:42.377939+0000","last_fullsized":"2023-10-06T03:01:42.377939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.334690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284069+0000","last_change":"2023-10-06T03:01:42.277487+0000","last_active":"2023-10-06T03:01:42.284069+0000","last_peered":"2023-10-06T03:01:42.284069+0000","last_clean":"2023-10-06T03:01:42.284069+0000","last_became_active":"2023-10-06T03:01:42.277135+0000","last_became_peered":"2023-10-06T03:01:42.277135+0000","last_unstale":"2023-10-06T03:01:42.284069+0000","last_undegraded":"2023-10-06T03:01:42.284069+0000","last_fullsized":"2023-10-06T03:01:42.284069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373790+0000","last_change":"2023-10-06T03:01:41.380292+0000","last_active":"2023-10-06T03:01:42.373790+0000","last_peered":"2023-10-06T03:01:42.373790+0000","last_clean":"2023-10-06T03:01:42.373790+0000","last_became_active":"2023-10-06T03:01:41.380074+0000","last_became_peered":"2023-10-06T03:01:41.380074+0000","last_unstale":"2023-10-06T03:01:42.373790+0000","last_undegraded":"2023-10-06T03:01:42.373790+0000","last_fullsized":"2023-10-06T03:01:42.373790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:27.982776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282615+0000","last_change":"2023-10-06T03:01:42.277855+0000","last_active":"2023-10-06T03:01:42.282615+0000","last_peered":"2023-10-06T03:01:42.282615+0000","last_clean":"2023-10-06T03:01:42.282615+0000","last_became_active":"2023-10-06T03:01:42.277539+0000","last_became_peered":"2023-10-06T03:01:42.277539+0000","last_unstale":"2023-10-06T03:01:42.282615+0000","last_undegraded":"2023-10-06T03:01:42.282615+0000","last_fullsized":"2023-10-06T03:01:42.282615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268326+0000","last_change":"2023-10-06T03:01:41.379182+0000","last_active":"2023-10-06T03:01:42.268326+0000","last_peered":"2023-10-06T03:01:42.268326+0000","last_clean":"2023-10-06T03:01:42.268326+0000","last_became_active":"2023-10-06T03:01:41.378857+0000","last_became_peered":"2023-10-06T03:01:41.378857+0000","last_unstale":"2023-10-06T03:01:42.268326+0000","last_undegraded":"2023-10-06T03:01:42.268326+0000","last_fullsized":"2023-10-06T03:01:42.268326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281816+0000","last_change":"2023-10-06T03:01:42.277028+0000","last_active":"2023-10-06T03:01:42.281816+0000","last_peered":"2023-10-06T03:01:42.281816+0000","last_clean":"2023-10-06T03:01:42.281816+0000","last_became_active":"2023-10-06T03:01:42.276777+0000","last_became_peered":"2023-10-06T03:01:42.276777+0000","last_unstale":"2023-10-06T03:01:42.281816+0000","last_undegraded":"2023-10-06T03:01:42.281816+0000","last_fullsized":"2023-10-06T03:01:42.281816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.371956+0000","last_change":"2023-10-06T03:01:41.383140+0000","last_active":"2023-10-06T03:01:42.371956+0000","last_peered":"2023-10-06T03:01:42.371956+0000","last_clean":"2023-10-06T03:01:42.371956+0000","last_became_active":"2023-10-06T03:01:41.382894+0000","last_became_peered":"2023-10-06T03:01:41.382894+0000","last_unstale":"2023-10-06T03:01:42.371956+0000","last_undegraded":"2023-10-06T03:01:42.371956+0000","last_fullsized":"2023-10-06T03:01:42.371956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284623+0000","last_change":"2023-10-06T03:01:42.280965+0000","last_active":"2023-10-06T03:01:42.284623+0000","last_peered":"2023-10-06T03:01:42.284623+0000","last_clean":"2023-10-06T03:01:42.284623+0000","last_became_active":"2023-10-06T03:01:42.278565+0000","last_became_peered":"2023-10-06T03:01:42.278565+0000","last_unstale":"2023-10-06T03:01:42.284623+0000","last_undegraded":"2023-10-06T03:01:42.284623+0000","last_fullsized":"2023-10-06T03:01:42.284623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274323+0000","last_change":"2023-10-06T03:01:41.385727+0000","last_active":"2023-10-06T03:01:42.274323+0000","last_peered":"2023-10-06T03:01:42.274323+0000","last_clean":"2023-10-06T03:01:42.274323+0000","last_became_active":"2023-10-06T03:01:41.385354+0000","last_became_peered":"2023-10-06T03:01:41.385354+0000","last_unstale":"2023-10-06T03:01:42.274323+0000","last_undegraded":"2023-10-06T03:01:42.274323+0000","last_fullsized":"2023-10-06T03:01:42.274323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284128+0000","last_change":"2023-10-06T03:01:42.278082+0000","last_active":"2023-10-06T03:01:42.284128+0000","last_peered":"2023-10-06T03:01:42.284128+0000","last_clean":"2023-10-06T03:01:42.284128+0000","last_became_active":"2023-10-06T03:01:42.277812+0000","last_became_peered":"2023-10-06T03:01:42.277812+0000","last_unstale":"2023-10-06T03:01:42.284128+0000","last_undegraded":"2023-10-06T03:01:42.284128+0000","last_fullsized":"2023-10-06T03:01:42.284128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270092+0000","last_change":"2023-10-06T03:01:41.381181+0000","last_active":"2023-10-06T03:01:42.270092+0000","last_peered":"2023-10-06T03:01:42.270092+0000","last_clean":"2023-10-06T03:01:42.270092+0000","last_became_active":"2023-10-06T03:01:41.378547+0000","last_became_peered":"2023-10-06T03:01:41.378547+0000","last_unstale":"2023-10-06T03:01:42.270092+0000","last_undegraded":"2023-10-06T03:01:42.270092+0000","last_fullsized":"2023-10-06T03:01:42.270092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284704+0000","last_change":"2023-10-06T03:01:42.281464+0000","last_active":"2023-10-06T03:01:42.284704+0000","last_peered":"2023-10-06T03:01:42.284704+0000","last_clean":"2023-10-06T03:01:42.284704+0000","last_became_active":"2023-10-06T03:01:42.281186+0000","last_became_peered":"2023-10-06T03:01:42.281186+0000","last_unstale":"2023-10-06T03:01:42.284704+0000","last_undegraded":"2023-10-06T03:01:42.284704+0000","last_fullsized":"2023-10-06T03:01:42.284704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.673116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268609+0000","last_change":"2023-10-06T03:01:41.378017+0000","last_active":"2023-10-06T03:01:42.268609+0000","last_peered":"2023-10-06T03:01:42.268609+0000","last_clean":"2023-10-06T03:01:42.268609+0000","last_became_active":"2023-10-06T03:01:41.377759+0000","last_became_peered":"2023-10-06T03:01:41.377759+0000","last_unstale":"2023-10-06T03:01:42.268609+0000","last_undegraded":"2023-10-06T03:01:42.268609+0000","last_fullsized":"2023-10-06T03:01:42.268609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287848+0000","last_change":"2023-10-06T03:01:42.278383+0000","last_active":"2023-10-06T03:01:42.287848+0000","last_peered":"2023-10-06T03:01:42.287848+0000","last_clean":"2023-10-06T03:01:42.287848+0000","last_became_active":"2023-10-06T03:01:42.277973+0000","last_became_peered":"2023-10-06T03:01:42.277973+0000","last_unstale":"2023-10-06T03:01:42.287848+0000","last_undegraded":"2023-10-06T03:01:42.287848+0000","last_fullsized":"2023-10-06T03:01:42.287848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271774+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:01:42.271774+0000","last_peered":"2023-10-06T03:01:42.271774+0000","last_clean":"2023-10-06T03:01:42.271774+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:01:42.271774+0000","last_undegraded":"2023-10-06T03:01:42.271774+0000","last_fullsized":"2023-10-06T03:01:42.271774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267761+0000","last_change":"2023-10-06T03:01:41.379310+0000","last_active":"2023-10-06T03:01:42.267761+0000","last_peered":"2023-10-06T03:01:42.267761+0000","last_clean":"2023-10-06T03:01:42.267761+0000","last_became_active":"2023-10-06T03:01:41.379017+0000","last_became_peered":"2023-10-06T03:01:41.379017+0000","last_unstale":"2023-10-06T03:01:42.267761+0000","last_undegraded":"2023-10-06T03:01:42.267761+0000","last_fullsized":"2023-10-06T03:01:42.267761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288159+0000","last_change":"2023-10-06T03:01:42.285640+0000","last_active":"2023-10-06T03:01:42.288159+0000","last_peered":"2023-10-06T03:01:42.288159+0000","last_clean":"2023-10-06T03:01:42.288159+0000","last_became_active":"2023-10-06T03:01:42.283071+0000","last_became_peered":"2023-10-06T03:01:42.283071+0000","last_unstale":"2023-10-06T03:01:42.288159+0000","last_undegraded":"2023-10-06T03:01:42.288159+0000","last_fullsized":"2023-10-06T03:01:42.288159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269088+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:01:42.269088+0000","last_peered":"2023-10-06T03:01:42.269088+0000","last_clean":"2023-10-06T03:01:42.269088+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:01:42.269088+0000","last_undegraded":"2023-10-06T03:01:42.269088+0000","last_fullsized":"2023-10-06T03:01:42.269088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268853+0000","last_change":"2023-10-06T03:01:41.382782+0000","last_active":"2023-10-06T03:01:42.268853+0000","last_peered":"2023-10-06T03:01:42.268853+0000","last_clean":"2023-10-06T03:01:42.268853+0000","last_became_active":"2023-10-06T03:01:41.382522+0000","last_became_peered":"2023-10-06T03:01:41.382522+0000","last_unstale":"2023-10-06T03:01:42.268853+0000","last_undegraded":"2023-10-06T03:01:42.268853+0000","last_fullsized":"2023-10-06T03:01:42.268853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288056+0000","last_change":"2023-10-06T03:01:42.282263+0000","last_active":"2023-10-06T03:01:42.288056+0000","last_peered":"2023-10-06T03:01:42.288056+0000","last_clean":"2023-10-06T03:01:42.288056+0000","last_became_active":"2023-10-06T03:01:42.282019+0000","last_became_peered":"2023-10-06T03:01:42.282019+0000","last_unstale":"2023-10-06T03:01:42.288056+0000","last_undegraded":"2023-10-06T03:01:42.288056+0000","last_fullsized":"2023-10-06T03:01:42.288056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.265449+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:01:42.265449+0000","last_peered":"2023-10-06T03:01:42.265449+0000","last_clean":"2023-10-06T03:01:42.265449+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:01:42.265449+0000","last_undegraded":"2023-10-06T03:01:42.265449+0000","last_fullsized":"2023-10-06T03:01:42.265449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268243+0000","last_change":"2023-10-06T03:01:41.378349+0000","last_active":"2023-10-06T03:01:42.268243+0000","last_peered":"2023-10-06T03:01:42.268243+0000","last_clean":"2023-10-06T03:01:42.268243+0000","last_became_active":"2023-10-06T03:01:41.378089+0000","last_became_peered":"2023-10-06T03:01:41.378089+0000","last_unstale":"2023-10-06T03:01:42.268243+0000","last_undegraded":"2023-10-06T03:01:42.268243+0000","last_fullsized":"2023-10-06T03:01:42.268243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288080+0000","last_change":"2023-10-06T03:01:42.283418+0000","last_active":"2023-10-06T03:01:42.288080+0000","last_peered":"2023-10-06T03:01:42.288080+0000","last_clean":"2023-10-06T03:01:42.288080+0000","last_became_active":"2023-10-06T03:01:42.283035+0000","last_became_peered":"2023-10-06T03:01:42.283035+0000","last_unstale":"2023-10-06T03:01:42.288080+0000","last_undegraded":"2023-10-06T03:01:42.288080+0000","last_fullsized":"2023-10-06T03:01:42.288080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:27.703859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.266739+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:01:42.266739+0000","last_peered":"2023-10-06T03:01:42.266739+0000","last_clean":"2023-10-06T03:01:42.266739+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:01:42.266739+0000","last_undegraded":"2023-10-06T03:01:42.266739+0000","last_fullsized":"2023-10-06T03:01:42.266739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274310+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:01:42.274310+0000","last_peered":"2023-10-06T03:01:42.274310+0000","last_clean":"2023-10-06T03:01:42.274310+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:01:42.274310+0000","last_undegraded":"2023-10-06T03:01:42.274310+0000","last_fullsized":"2023-10-06T03:01:42.274310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268619+0000","last_change":"2023-10-06T03:01:41.378526+0000","last_active":"2023-10-06T03:01:42.268619+0000","last_peered":"2023-10-06T03:01:42.268619+0000","last_clean":"2023-10-06T03:01:42.268619+0000","last_became_active":"2023-10-06T03:01:41.378245+0000","last_became_peered":"2023-10-06T03:01:41.378245+0000","last_unstale":"2023-10-06T03:01:42.268619+0000","last_undegraded":"2023-10-06T03:01:42.268619+0000","last_fullsized":"2023-10-06T03:01:42.268619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284302+0000","last_change":"2023-10-06T03:01:42.281327+0000","last_active":"2023-10-06T03:01:42.284302+0000","last_peered":"2023-10-06T03:01:42.284302+0000","last_clean":"2023-10-06T03:01:42.284302+0000","last_became_active":"2023-10-06T03:01:42.281028+0000","last_became_peered":"2023-10-06T03:01:42.281028+0000","last_unstale":"2023-10-06T03:01:42.284302+0000","last_undegraded":"2023-10-06T03:01:42.284302+0000","last_fullsized":"2023-10-06T03:01:42.284302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:28.986647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274776+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:01:42.274776+0000","last_peered":"2023-10-06T03:01:42.274776+0000","last_clean":"2023-10-06T03:01:42.274776+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:01:42.274776+0000","last_undegraded":"2023-10-06T03:01:42.274776+0000","last_fullsized":"2023-10-06T03:01:42.274776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268368+0000","last_change":"2023-10-06T03:01:41.377764+0000","last_active":"2023-10-06T03:01:42.268368+0000","last_peered":"2023-10-06T03:01:42.268368+0000","last_clean":"2023-10-06T03:01:42.268368+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.268368+0000","last_undegraded":"2023-10-06T03:01:42.268368+0000","last_fullsized":"2023-10-06T03:01:42.268368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285753+0000","last_change":"2023-10-06T03:01:42.279939+0000","last_active":"2023-10-06T03:01:42.285753+0000","last_peered":"2023-10-06T03:01:42.285753+0000","last_clean":"2023-10-06T03:01:42.285753+0000","last_became_active":"2023-10-06T03:01:42.279453+0000","last_became_peered":"2023-10-06T03:01:42.279453+0000","last_unstale":"2023-10-06T03:01:42.285753+0000","last_undegraded":"2023-10-06T03:01:42.285753+0000","last_fullsized":"2023-10-06T03:01:42.285753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.276154+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:01:42.276154+0000","last_peered":"2023-10-06T03:01:42.276154+0000","last_clean":"2023-10-06T03:01:42.276154+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:01:42.276154+0000","last_undegraded":"2023-10-06T03:01:42.276154+0000","last_fullsized":"2023-10-06T03:01:42.276154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271880+0000","last_change":"2023-10-06T03:01:41.378694+0000","last_active":"2023-10-06T03:01:42.271880+0000","last_peered":"2023-10-06T03:01:42.271880+0000","last_clean":"2023-10-06T03:01:42.271880+0000","last_became_active":"2023-10-06T03:01:41.378366+0000","last_became_peered":"2023-10-06T03:01:41.378366+0000","last_unstale":"2023-10-06T03:01:42.271880+0000","last_undegraded":"2023-10-06T03:01:42.271880+0000","last_fullsized":"2023-10-06T03:01:42.271880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285642+0000","last_change":"2023-10-06T03:01:42.278204+0000","last_active":"2023-10-06T03:01:42.285642+0000","last_peered":"2023-10-06T03:01:42.285642+0000","last_clean":"2023-10-06T03:01:42.285642+0000","last_became_active":"2023-10-06T03:01:42.277950+0000","last_became_peered":"2023-10-06T03:01:42.277950+0000","last_unstale":"2023-10-06T03:01:42.285642+0000","last_undegraded":"2023-10-06T03:01:42.285642+0000","last_fullsized":"2023-10-06T03:01:42.285642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378135+0000","last_change":"2023-10-06T03:01:41.378450+0000","last_active":"2023-10-06T03:01:42.378135+0000","last_peered":"2023-10-06T03:01:42.378135+0000","last_clean":"2023-10-06T03:01:42.378135+0000","last_became_active":"2023-10-06T03:01:41.378167+0000","last_became_peered":"2023-10-06T03:01:41.378167+0000","last_unstale":"2023-10-06T03:01:42.378135+0000","last_undegraded":"2023-10-06T03:01:42.378135+0000","last_fullsized":"2023-10-06T03:01:42.378135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269297+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:01:42.269297+0000","last_peered":"2023-10-06T03:01:42.269297+0000","last_clean":"2023-10-06T03:01:42.269297+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:01:42.269297+0000","last_undegraded":"2023-10-06T03:01:42.269297+0000","last_fullsized":"2023-10-06T03:01:42.269297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288032+0000","last_change":"2023-10-06T03:01:42.284704+0000","last_active":"2023-10-06T03:01:42.288032+0000","last_peered":"2023-10-06T03:01:42.288032+0000","last_clean":"2023-10-06T03:01:42.288032+0000","last_became_active":"2023-10-06T03:01:42.284380+0000","last_became_peered":"2023-10-06T03:01:42.284380+0000","last_unstale":"2023-10-06T03:01:42.288032+0000","last_undegraded":"2023-10-06T03:01:42.288032+0000","last_fullsized":"2023-10-06T03:01:42.288032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.273934+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:01:42.273934+0000","last_peered":"2023-10-06T03:01:42.273934+0000","last_clean":"2023-10-06T03:01:42.273934+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:01:42.273934+0000","last_undegraded":"2023-10-06T03:01:42.273934+0000","last_fullsized":"2023-10-06T03:01:42.273934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270423+0000","last_change":"2023-10-06T03:01:41.376173+0000","last_active":"2023-10-06T03:01:42.270423+0000","last_peered":"2023-10-06T03:01:42.270423+0000","last_clean":"2023-10-06T03:01:42.270423+0000","last_became_active":"2023-10-06T03:01:41.375877+0000","last_became_peered":"2023-10-06T03:01:41.375877+0000","last_unstale":"2023-10-06T03:01:42.270423+0000","last_undegraded":"2023-10-06T03:01:42.270423+0000","last_fullsized":"2023-10-06T03:01:42.270423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.277029+0000","last_change":"2023-10-06T03:01:42.271755+0000","last_active":"2023-10-06T03:01:42.277029+0000","last_peered":"2023-10-06T03:01:42.277029+0000","last_clean":"2023-10-06T03:01:42.277029+0000","last_became_active":"2023-10-06T03:01:42.271483+0000","last_became_peered":"2023-10-06T03:01:42.271483+0000","last_unstale":"2023-10-06T03:01:42.277029+0000","last_undegraded":"2023-10-06T03:01:42.277029+0000","last_fullsized":"2023-10-06T03:01:42.277029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373544+0000","last_change":"2023-10-06T03:01:41.381914+0000","last_active":"2023-10-06T03:01:42.373544+0000","last_peered":"2023-10-06T03:01:42.373544+0000","last_clean":"2023-10-06T03:01:42.373544+0000","last_became_active":"2023-10-06T03:01:41.381635+0000","last_became_peered":"2023-10-06T03:01:41.381635+0000","last_unstale":"2023-10-06T03:01:42.373544+0000","last_undegraded":"2023-10-06T03:01:42.373544+0000","last_fullsized":"2023-10-06T03:01:42.373544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280515+0000","last_change":"2023-10-06T03:01:42.277000+0000","last_active":"2023-10-06T03:01:42.280515+0000","last_peered":"2023-10-06T03:01:42.280515+0000","last_clean":"2023-10-06T03:01:42.280515+0000","last_became_active":"2023-10-06T03:01:42.276746+0000","last_became_peered":"2023-10-06T03:01:42.276746+0000","last_unstale":"2023-10-06T03:01:42.280515+0000","last_undegraded":"2023-10-06T03:01:42.280515+0000","last_fullsized":"2023-10-06T03:01:42.280515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378477+0000","last_change":"2023-10-06T03:01:41.377203+0000","last_active":"2023-10-06T03:01:42.378477+0000","last_peered":"2023-10-06T03:01:42.378477+0000","last_clean":"2023-10-06T03:01:42.378477+0000","last_became_active":"2023-10-06T03:01:41.376950+0000","last_became_peered":"2023-10-06T03:01:41.376950+0000","last_unstale":"2023-10-06T03:01:42.378477+0000","last_undegraded":"2023-10-06T03:01:42.378477+0000","last_fullsized":"2023-10-06T03:01:42.378477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284682+0000","last_change":"2023-10-06T03:01:42.282305+0000","last_active":"2023-10-06T03:01:42.284682+0000","last_peered":"2023-10-06T03:01:42.284682+0000","last_clean":"2023-10-06T03:01:42.284682+0000","last_became_active":"2023-10-06T03:01:42.282037+0000","last_became_peered":"2023-10-06T03:01:42.282037+0000","last_unstale":"2023-10-06T03:01:42.284682+0000","last_undegraded":"2023-10-06T03:01:42.284682+0000","last_fullsized":"2023-10-06T03:01:42.284682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:24.802638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378519+0000","last_change":"2023-10-06T03:01:41.377561+0000","last_active":"2023-10-06T03:01:42.378519+0000","last_peered":"2023-10-06T03:01:42.378519+0000","last_clean":"2023-10-06T03:01:42.378519+0000","last_became_active":"2023-10-06T03:01:41.377308+0000","last_became_peered":"2023-10-06T03:01:41.377308+0000","last_unstale":"2023-10-06T03:01:42.378519+0000","last_undegraded":"2023-10-06T03:01:42.378519+0000","last_fullsized":"2023-10-06T03:01:42.378519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284090+0000","last_change":"2023-10-06T03:01:42.277443+0000","last_active":"2023-10-06T03:01:42.284090+0000","last_peered":"2023-10-06T03:01:42.284090+0000","last_clean":"2023-10-06T03:01:42.284090+0000","last_became_active":"2023-10-06T03:01:42.277152+0000","last_became_peered":"2023-10-06T03:01:42.277152+0000","last_unstale":"2023-10-06T03:01:42.284090+0000","last_undegraded":"2023-10-06T03:01:42.284090+0000","last_fullsized":"2023-10-06T03:01:42.284090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373306+0000","last_change":"2023-10-06T03:01:41.379186+0000","last_active":"2023-10-06T03:01:42.373306+0000","last_peered":"2023-10-06T03:01:42.373306+0000","last_clean":"2023-10-06T03:01:42.373306+0000","last_became_active":"2023-10-06T03:01:41.378963+0000","last_became_peered":"2023-10-06T03:01:41.378963+0000","last_unstale":"2023-10-06T03:01:42.373306+0000","last_undegraded":"2023-10-06T03:01:42.373306+0000","last_fullsized":"2023-10-06T03:01:42.373306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280768+0000","last_change":"2023-10-06T03:01:42.276973+0000","last_active":"2023-10-06T03:01:42.280768+0000","last_peered":"2023-10-06T03:01:42.280768+0000","last_clean":"2023-10-06T03:01:42.280768+0000","last_became_active":"2023-10-06T03:01:42.276297+0000","last_became_peered":"2023-10-06T03:01:42.276297+0000","last_unstale":"2023-10-06T03:01:42.280768+0000","last_undegraded":"2023-10-06T03:01:42.280768+0000","last_fullsized":"2023-10-06T03:01:42.280768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373671+0000","last_change":"2023-10-06T03:01:41.380486+0000","last_active":"2023-10-06T03:01:42.373671+0000","last_peered":"2023-10-06T03:01:42.373671+0000","last_clean":"2023-10-06T03:01:42.373671+0000","last_became_active":"2023-10-06T03:01:41.380212+0000","last_became_peered":"2023-10-06T03:01:41.380212+0000","last_unstale":"2023-10-06T03:01:42.373671+0000","last_undegraded":"2023-10-06T03:01:42.373671+0000","last_fullsized":"2023-10-06T03:01:42.373671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284486+0000","last_change":"2023-10-06T03:01:42.280604+0000","last_active":"2023-10-06T03:01:42.284486+0000","last_peered":"2023-10-06T03:01:42.284486+0000","last_clean":"2023-10-06T03:01:42.284486+0000","last_became_active":"2023-10-06T03:01:42.280041+0000","last_became_peered":"2023-10-06T03:01:42.280041+0000","last_unstale":"2023-10-06T03:01:42.284486+0000","last_undegraded":"2023-10-06T03:01:42.284486+0000","last_fullsized":"2023-10-06T03:01:42.284486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:02.038765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283231+0000","last_change":"2023-10-06T03:01:42.277027+0000","last_active":"2023-10-06T03:01:42.283231+0000","last_peered":"2023-10-06T03:01:42.283231+0000","last_clean":"2023-10-06T03:01:42.283231+0000","last_became_active":"2023-10-06T03:01:42.276730+0000","last_became_peered":"2023-10-06T03:01:42.276730+0000","last_unstale":"2023-10-06T03:01:42.283231+0000","last_undegraded":"2023-10-06T03:01:42.283231+0000","last_fullsized":"2023-10-06T03:01:42.283231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:18.781548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280787+0000","last_change":"2023-10-06T03:01:42.277145+0000","last_active":"2023-10-06T03:01:42.280787+0000","last_peered":"2023-10-06T03:01:42.280787+0000","last_clean":"2023-10-06T03:01:42.280787+0000","last_became_active":"2023-10-06T03:01:42.276735+0000","last_became_peered":"2023-10-06T03:01:42.276735+0000","last_unstale":"2023-10-06T03:01:42.280787+0000","last_undegraded":"2023-10-06T03:01:42.280787+0000","last_fullsized":"2023-10-06T03:01:42.280787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284561+0000","last_change":"2023-10-06T03:01:42.281639+0000","last_active":"2023-10-06T03:01:42.284561+0000","last_peered":"2023-10-06T03:01:42.284561+0000","last_clean":"2023-10-06T03:01:42.284561+0000","last_became_active":"2023-10-06T03:01:42.278720+0000","last_became_peered":"2023-10-06T03:01:42.278720+0000","last_unstale":"2023-10-06T03:01:42.284561+0000","last_undegraded":"2023-10-06T03:01:42.284561+0000","last_fullsized":"2023-10-06T03:01:42.284561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:03.312309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288057+0000","last_change":"2023-10-06T03:01:42.283924+0000","last_active":"2023-10-06T03:01:42.288057+0000","last_peered":"2023-10-06T03:01:42.288057+0000","last_clean":"2023-10-06T03:01:42.288057+0000","last_became_active":"2023-10-06T03:01:42.283527+0000","last_became_peered":"2023-10-06T03:01:42.283527+0000","last_unstale":"2023-10-06T03:01:42.288057+0000","last_undegraded":"2023-10-06T03:01:42.288057+0000","last_fullsized":"2023-10-06T03:01:42.288057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288138+0000","last_change":"2023-10-06T03:01:42.284183+0000","last_active":"2023-10-06T03:01:42.288138+0000","last_peered":"2023-10-06T03:01:42.288138+0000","last_clean":"2023-10-06T03:01:42.288138+0000","last_became_active":"2023-10-06T03:01:42.283895+0000","last_became_peered":"2023-10-06T03:01:42.283895+0000","last_unstale":"2023-10-06T03:01:42.288138+0000","last_undegraded":"2023-10-06T03:01:42.288138+0000","last_fullsized":"2023-10-06T03:01:42.288138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":674798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":827392,"data_stored":673600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:01:46.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:01:46.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.464+0000 7f5e83b3a700 1 Processor -- start 2023-10-06T03:01:46.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.464+0000 7f5e83b3a700 1 -- start start 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e83b3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c159890 0x7f5e7c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e83b3a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5e7c159c60 0x7f5e7c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e83b3a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e7c15db10 0x7f5e7c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e83b3a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c15a160 con 0x7f5e7c15db10 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e83b3a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0d13a0 con 0x7f5e7c159c60 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e83b3a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c15e140 con 0x7f5e7c159890 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e820d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e7c15db10 0x7f5e7c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e820d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e7c15db10 0x7f5e7c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44306/0 (socket says 172.21.15.92:44306) 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.465+0000 7f5e820d7700 1 -- 172.21.15.92:0/3765060923 learned_addr learned my addr 172.21.15.92:0/3765060923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e818d6700 1 --2- 172.21.15.92:0/3765060923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c159890 0x7f5e7c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e810d5700 1 --2- 172.21.15.92:0/3765060923 >> v2:172.21.15.136:3300/0 conn(0x7f5e7c159c60 0x7f5e7c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e820d7700 1 -- 172.21.15.92:0/3765060923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c159890 msgr2=0x7f5e7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e820d7700 1 --2- 172.21.15.92:0/3765060923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c159890 0x7f5e7c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e820d7700 1 -- 172.21.15.92:0/3765060923 >> v2:172.21.15.136:3300/0 conn(0x7f5e7c159c60 msgr2=0x7f5e7c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e820d7700 1 --2- 172.21.15.92:0/3765060923 >> v2:172.21.15.136:3300/0 conn(0x7f5e7c159c60 0x7f5e7c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e820d7700 1 -- 172.21.15.92:0/3765060923 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c15a3d0 con 0x7f5e7c15db10 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e810d5700 1 --2- 172.21.15.92:0/3765060923 >> v2:172.21.15.136:3300/0 conn(0x7f5e7c159c60 0x7f5e7c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e820d7700 1 --2- 172.21.15.92:0/3765060923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c15db10 0x7f5e7c069560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0650f0 tx=0x7f5e74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229b78978faae1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e808d4700 1 -- 172.21.15.92:0/3765060923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400e070 con 0x7f5e7c15db10 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e808d4700 1 -- 172.21.15.92:0/3765060923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400ba50 con 0x7f5e7c15db10 2023-10-06T03:01:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e808d4700 1 -- 172.21.15.92:0/3765060923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e740092e0 con 0x7f5e7c15db10 2023-10-06T03:01:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e83b3a700 1 -- 172.21.15.92:0/3765060923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c15db10 msgr2=0x7f5e7c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/3765060923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c15db10 0x7f5e7c069560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0650f0 tx=0x7f5e74009840 comp rx=0 tx=0).stop 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.466+0000 7f5e83b3a700 1 -- 172.21.15.92:0/3765060923 shutdown_connections 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.467+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/3765060923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c159890 0x7f5e7c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.467+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/3765060923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c15db10 0x7f5e7c069560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.467+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/3765060923 >> v2:172.21.15.136:3300/0 conn(0x7f5e7c159c60 0x7f5e7c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.467+0000 7f5e83b3a700 1 -- 172.21.15.92:0/3765060923 >> 172.21.15.92:0/3765060923 conn(0x7f5e7c068e20 msgr2=0x7f5e7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.467+0000 7f5e83b3a700 1 -- 172.21.15.92:0/3765060923 shutdown_connections 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.467+0000 7f5e83b3a700 1 -- 172.21.15.92:0/3765060923 wait complete. 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 Processor -- start 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 -- start start 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e7c159890 0x7f5e7c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 0x7f5e7c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c15db10 0x7f5e7c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b4f70 con 0x7f5e7c159c60 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c155bb0 con 0x7f5e7c159890 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e83b3a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0b46d0 con 0x7f5e7c15db10 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e810d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 0x7f5e7c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.468+0000 7f5e810d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 0x7f5e7c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44308/0 (socket says 172.21.15.92:44308) 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 -- 172.21.15.92:0/2948675606 learned_addr learned my addr 172.21.15.92:0/2948675606 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e820d7700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c15db10 0x7f5e7c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e818d6700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e7c159890 0x7f5e7c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 -- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c15db10 msgr2=0x7f5e7c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c15db10 0x7f5e7c0b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 -- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e7c159890 msgr2=0x7f5e7c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e7c159890 0x7f5e7c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 -- 172.21.15.92:0/2948675606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74009ae0 con 0x7f5e7c159c60 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e810d5700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 0x7f5e7c0b1a20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00e8a0 tx=0x7f5e6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c013410 con 0x7f5e7c159c60 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c013570 con 0x7f5e7c159c60 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c013930 con 0x7f5e7c159c60 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c0b49b0 con 0x7f5e7c159c60 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.469+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0ce710 con 0x7f5e7c159c60 2023-10-06T03:01:46.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.470+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c0ce570 con 0x7f5e7c159c60 2023-10-06T03:01:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.470+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e6c013ca0 con 0x7f5e7c159c60 2023-10-06T03:01:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.471+0000 7f5e71ffb700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e60066f30 0x7f5e600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.471+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e6c084340 con 0x7f5e7c159c60 2023-10-06T03:01:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.471+0000 7f5e818d6700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e60066f30 0x7f5e600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.471+0000 7f5e818d6700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e60066f30 0x7f5e600693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006fd0 tx=0x7f5e78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.475+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e7c0ce570 con 0x7f5e7c159c60 2023-10-06T03:01:46.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.654+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e7c0ce570 con 0x7f5e60066f30 2023-10-06T03:01:46.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.662+0000 7f5e71ffb700 1 -- 172.21.15.92:0/2948675606 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286724 (secure 0 0 0) 0x7f5e7c0ce570 con 0x7f5e60066f30 2023-10-06T03:01:46.666 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:46.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e60066f30 msgr2=0x7f5e600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e60066f30 0x7f5e600693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e78006fd0 tx=0x7f5e78008040 comp rx=0 tx=0).stop 2023-10-06T03:01:46.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 msgr2=0x7f5e7c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 0x7f5e7c0b1a20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00e8a0 tx=0x7f5e6c008040 comp rx=0 tx=0).stop 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 shutdown_connections 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e60066f30 0x7f5e600693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e7c15db10 0x7f5e7c0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e7c159890 0x7f5e7c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 --2- 172.21.15.92:0/2948675606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e7c159c60 0x7f5e7c0b1a20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 >> 172.21.15.92:0/2948675606 conn(0x7f5e7c068e20 msgr2=0x7f5e7c15ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 shutdown_connections 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.665+0000 7f5e83b3a700 1 -- 172.21.15.92:0/2948675606 wait complete. 2023-10-06T03:01:46.669 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:01:46.678 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T03:01:45.183022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381380,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593340,"statfs":{"total":773094113280,"available":770655580160,"internally_reserved":0,"allocated":3149824,"data_stored":1879977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.825470"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280436+0000","last_change":"2023-10-06T03:01:42.278647+0000","last_active":"2023-10-06T03:01:42.280436+0000","last_peered":"2023-10-06T03:01:42.280436+0000","last_clean":"2023-10-06T03:01:42.280436+0000","last_became_active":"2023-10-06T03:01:42.278364+0000","last_became_peered":"2023-10-06T03:01:42.278364+0000","last_unstale":"2023-10-06T03:01:42.280436+0000","last_undegraded":"2023-10-06T03:01:42.280436+0000","last_fullsized":"2023-10-06T03:01:42.280436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:06.498462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282043+0000","last_change":"2023-10-06T03:01:42.277062+0000","last_active":"2023-10-06T03:01:42.282043+0000","last_peered":"2023-10-06T03:01:42.282043+0000","last_clean":"2023-10-06T03:01:42.282043+0000","last_became_active":"2023-10-06T03:01:42.276189+0000","last_became_peered":"2023-10-06T03:01:42.276189+0000","last_unstale":"2023-10-06T03:01:42.282043+0000","last_undegraded":"2023-10-06T03:01:42.282043+0000","last_fullsized":"2023-10-06T03:01:42.282043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284329+0000","last_change":"2023-10-06T03:01:42.280530+0000","last_active":"2023-10-06T03:01:42.284329+0000","last_peered":"2023-10-06T03:01:42.284329+0000","last_clean":"2023-10-06T03:01:42.284329+0000","last_became_active":"2023-10-06T03:01:42.280239+0000","last_became_peered":"2023-10-06T03:01:42.280239+0000","last_unstale":"2023-10-06T03:01:42.284329+0000","last_undegraded":"2023-10-06T03:01:42.284329+0000","last_fullsized":"2023-10-06T03:01:42.284329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284290+0000","last_change":"2023-10-06T03:01:42.279732+0000","last_active":"2023-10-06T03:01:42.284290+0000","last_peered":"2023-10-06T03:01:42.284290+0000","last_clean":"2023-10-06T03:01:42.284290+0000","last_became_active":"2023-10-06T03:01:42.279445+0000","last_became_peered":"2023-10-06T03:01:42.279445+0000","last_unstale":"2023-10-06T03:01:42.284290+0000","last_undegraded":"2023-10-06T03:01:42.284290+0000","last_fullsized":"2023-10-06T03:01:42.284290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:08.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285662+0000","last_change":"2023-10-06T03:01:42.279674+0000","last_active":"2023-10-06T03:01:42.285662+0000","last_peered":"2023-10-06T03:01:42.285662+0000","last_clean":"2023-10-06T03:01:42.285662+0000","last_became_active":"2023-10-06T03:01:42.279435+0000","last_became_peered":"2023-10-06T03:01:42.279435+0000","last_unstale":"2023-10-06T03:01:42.285662+0000","last_undegraded":"2023-10-06T03:01:42.285662+0000","last_fullsized":"2023-10-06T03:01:42.285662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284581+0000","last_change":"2023-10-06T03:01:42.282579+0000","last_active":"2023-10-06T03:01:42.284581+0000","last_peered":"2023-10-06T03:01:42.284581+0000","last_clean":"2023-10-06T03:01:42.284581+0000","last_became_active":"2023-10-06T03:01:42.282276+0000","last_became_peered":"2023-10-06T03:01:42.282276+0000","last_unstale":"2023-10-06T03:01:42.284581+0000","last_undegraded":"2023-10-06T03:01:42.284581+0000","last_fullsized":"2023-10-06T03:01:42.284581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286199+0000","last_change":"2023-10-06T03:01:42.282922+0000","last_active":"2023-10-06T03:01:42.286199+0000","last_peered":"2023-10-06T03:01:42.286199+0000","last_clean":"2023-10-06T03:01:42.286199+0000","last_became_active":"2023-10-06T03:01:42.282688+0000","last_became_peered":"2023-10-06T03:01:42.282688+0000","last_unstale":"2023-10-06T03:01:42.286199+0000","last_undegraded":"2023-10-06T03:01:42.286199+0000","last_fullsized":"2023-10-06T03:01:42.286199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285998+0000","last_change":"2023-10-06T03:01:42.279085+0000","last_active":"2023-10-06T03:01:42.285998+0000","last_peered":"2023-10-06T03:01:42.285998+0000","last_clean":"2023-10-06T03:01:42.285998+0000","last_became_active":"2023-10-06T03:01:42.278311+0000","last_became_peered":"2023-10-06T03:01:42.278311+0000","last_unstale":"2023-10-06T03:01:42.285998+0000","last_undegraded":"2023-10-06T03:01:42.285998+0000","last_fullsized":"2023-10-06T03:01:42.285998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285685+0000","last_change":"2023-10-06T03:01:42.280111+0000","last_active":"2023-10-06T03:01:42.285685+0000","last_peered":"2023-10-06T03:01:42.285685+0000","last_clean":"2023-10-06T03:01:42.285685+0000","last_became_active":"2023-10-06T03:01:42.279534+0000","last_became_peered":"2023-10-06T03:01:42.279534+0000","last_unstale":"2023-10-06T03:01:42.285685+0000","last_undegraded":"2023-10-06T03:01:42.285685+0000","last_fullsized":"2023-10-06T03:01:42.285685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286459+0000","last_change":"2023-10-06T03:01:42.281699+0000","last_active":"2023-10-06T03:01:42.286459+0000","last_peered":"2023-10-06T03:01:42.286459+0000","last_clean":"2023-10-06T03:01:42.286459+0000","last_became_active":"2023-10-06T03:01:42.279098+0000","last_became_peered":"2023-10-06T03:01:42.279098+0000","last_unstale":"2023-10-06T03:01:42.286459+0000","last_undegraded":"2023-10-06T03:01:42.286459+0000","last_fullsized":"2023-10-06T03:01:42.286459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283045+0000","last_change":"2023-10-06T03:01:42.277096+0000","last_active":"2023-10-06T03:01:42.283045+0000","last_peered":"2023-10-06T03:01:42.283045+0000","last_clean":"2023-10-06T03:01:42.283045+0000","last_became_active":"2023-10-06T03:01:42.276719+0000","last_became_peered":"2023-10-06T03:01:42.276719+0000","last_unstale":"2023-10-06T03:01:42.283045+0000","last_undegraded":"2023-10-06T03:01:42.283045+0000","last_fullsized":"2023-10-06T03:01:42.283045+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:13.143094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275038+0000","last_change":"2023-10-06T03:01:42.272031+0000","last_active":"2023-10-06T03:01:42.275038+0000","last_peered":"2023-10-06T03:01:42.275038+0000","last_clean":"2023-10-06T03:01:42.275038+0000","last_became_active":"2023-10-06T03:01:42.271796+0000","last_became_peered":"2023-10-06T03:01:42.271796+0000","last_unstale":"2023-10-06T03:01:42.275038+0000","last_undegraded":"2023-10-06T03:01:42.275038+0000","last_fullsized":"2023-10-06T03:01:42.275038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281109+0000","last_change":"2023-10-06T03:01:42.279147+0000","last_active":"2023-10-06T03:01:42.281109+0000","last_peered":"2023-10-06T03:01:42.281109+0000","last_clean":"2023-10-06T03:01:42.281109+0000","last_became_active":"2023-10-06T03:01:42.278911+0000","last_became_peered":"2023-10-06T03:01:42.278911+0000","last_unstale":"2023-10-06T03:01:42.281109+0000","last_undegraded":"2023-10-06T03:01:42.281109+0000","last_fullsized":"2023-10-06T03:01:42.281109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:14.170785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286021+0000","last_change":"2023-10-06T03:01:42.277711+0000","last_active":"2023-10-06T03:01:42.286021+0000","last_peered":"2023-10-06T03:01:42.286021+0000","last_clean":"2023-10-06T03:01:42.286021+0000","last_became_active":"2023-10-06T03:01:42.276996+0000","last_became_peered":"2023-10-06T03:01:42.276996+0000","last_unstale":"2023-10-06T03:01:42.286021+0000","last_undegraded":"2023-10-06T03:01:42.286021+0000","last_fullsized":"2023-10-06T03:01:42.286021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.289013+0000","last_change":"2023-10-06T03:01:42.288226+0000","last_active":"2023-10-06T03:01:42.289013+0000","last_peered":"2023-10-06T03:01:42.289013+0000","last_clean":"2023-10-06T03:01:42.289013+0000","last_became_active":"2023-10-06T03:01:42.287944+0000","last_became_peered":"2023-10-06T03:01:42.287944+0000","last_unstale":"2023-10-06T03:01:42.289013+0000","last_undegraded":"2023-10-06T03:01:42.289013+0000","last_fullsized":"2023-10-06T03:01:42.289013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284504+0000","last_change":"2023-10-06T03:01:42.281156+0000","last_active":"2023-10-06T03:01:42.284504+0000","last_peered":"2023-10-06T03:01:42.284504+0000","last_clean":"2023-10-06T03:01:42.284504+0000","last_became_active":"2023-10-06T03:01:42.280924+0000","last_became_peered":"2023-10-06T03:01:42.280924+0000","last_unstale":"2023-10-06T03:01:42.284504+0000","last_undegraded":"2023-10-06T03:01:42.284504+0000","last_fullsized":"2023-10-06T03:01:42.284504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285706+0000","last_change":"2023-10-06T03:01:42.280272+0000","last_active":"2023-10-06T03:01:42.285706+0000","last_peered":"2023-10-06T03:01:42.285706+0000","last_clean":"2023-10-06T03:01:42.285706+0000","last_became_active":"2023-10-06T03:01:42.279857+0000","last_became_peered":"2023-10-06T03:01:42.279857+0000","last_unstale":"2023-10-06T03:01:42.285706+0000","last_undegraded":"2023-10-06T03:01:42.285706+0000","last_fullsized":"2023-10-06T03:01:42.285706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284102+0000","last_change":"2023-10-06T03:01:42.278122+0000","last_active":"2023-10-06T03:01:42.284102+0000","last_peered":"2023-10-06T03:01:42.284102+0000","last_clean":"2023-10-06T03:01:42.284102+0000","last_became_active":"2023-10-06T03:01:42.277860+0000","last_became_peered":"2023-10-06T03:01:42.277860+0000","last_unstale":"2023-10-06T03:01:42.284102+0000","last_undegraded":"2023-10-06T03:01:42.284102+0000","last_fullsized":"2023-10-06T03:01:42.284102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:23.395145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283307+0000","last_change":"2023-10-06T03:01:42.281950+0000","last_active":"2023-10-06T03:01:42.283307+0000","last_peered":"2023-10-06T03:01:42.283307+0000","last_clean":"2023-10-06T03:01:42.283307+0000","last_became_active":"2023-10-06T03:01:42.281696+0000","last_became_peered":"2023-10-06T03:01:42.281696+0000","last_unstale":"2023-10-06T03:01:42.283307+0000","last_undegraded":"2023-10-06T03:01:42.283307+0000","last_fullsized":"2023-10-06T03:01:42.283307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:22.656089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284642+0000","last_change":"2023-10-06T03:01:42.281651+0000","last_active":"2023-10-06T03:01:42.284642+0000","last_peered":"2023-10-06T03:01:42.284642+0000","last_clean":"2023-10-06T03:01:42.284642+0000","last_became_active":"2023-10-06T03:01:42.278739+0000","last_became_peered":"2023-10-06T03:01:42.278739+0000","last_unstale":"2023-10-06T03:01:42.284642+0000","last_undegraded":"2023-10-06T03:01:42.284642+0000","last_fullsized":"2023-10-06T03:01:42.284642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:22.011159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286267+0000","last_change":"2023-10-06T03:01:42.283331+0000","last_active":"2023-10-06T03:01:42.286267+0000","last_peered":"2023-10-06T03:01:42.286267+0000","last_clean":"2023-10-06T03:01:42.286267+0000","last_became_active":"2023-10-06T03:01:42.283048+0000","last_became_peered":"2023-10-06T03:01:42.283048+0000","last_unstale":"2023-10-06T03:01:42.286267+0000","last_undegraded":"2023-10-06T03:01:42.286267+0000","last_fullsized":"2023-10-06T03:01:42.286267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285977+0000","last_change":"2023-10-06T03:01:42.282637+0000","last_active":"2023-10-06T03:01:42.285977+0000","last_peered":"2023-10-06T03:01:42.285977+0000","last_clean":"2023-10-06T03:01:42.285977+0000","last_became_active":"2023-10-06T03:01:42.282269+0000","last_became_peered":"2023-10-06T03:01:42.282269+0000","last_unstale":"2023-10-06T03:01:42.285977+0000","last_undegraded":"2023-10-06T03:01:42.285977+0000","last_fullsized":"2023-10-06T03:01:42.285977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281082+0000","last_change":"2023-10-06T03:01:42.279886+0000","last_active":"2023-10-06T03:01:42.281082+0000","last_peered":"2023-10-06T03:01:42.281082+0000","last_clean":"2023-10-06T03:01:42.281082+0000","last_became_active":"2023-10-06T03:01:42.279246+0000","last_became_peered":"2023-10-06T03:01:42.279246+0000","last_unstale":"2023-10-06T03:01:42.281082+0000","last_undegraded":"2023-10-06T03:01:42.281082+0000","last_fullsized":"2023-10-06T03:01:42.281082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286308+0000","last_change":"2023-10-06T03:01:42.283900+0000","last_active":"2023-10-06T03:01:42.286308+0000","last_peered":"2023-10-06T03:01:42.286308+0000","last_clean":"2023-10-06T03:01:42.286308+0000","last_became_active":"2023-10-06T03:01:42.283581+0000","last_became_peered":"2023-10-06T03:01:42.283581+0000","last_unstale":"2023-10-06T03:01:42.286308+0000","last_undegraded":"2023-10-06T03:01:42.286308+0000","last_fullsized":"2023-10-06T03:01:42.286308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287866+0000","last_change":"2023-10-06T03:01:42.277988+0000","last_active":"2023-10-06T03:01:42.287866+0000","last_peered":"2023-10-06T03:01:42.287866+0000","last_clean":"2023-10-06T03:01:42.287866+0000","last_became_active":"2023-10-06T03:01:42.277260+0000","last_became_peered":"2023-10-06T03:01:42.277260+0000","last_unstale":"2023-10-06T03:01:42.287866+0000","last_undegraded":"2023-10-06T03:01:42.287866+0000","last_fullsized":"2023-10-06T03:01:42.287866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287998+0000","last_change":"2023-10-06T03:01:42.280682+0000","last_active":"2023-10-06T03:01:42.287998+0000","last_peered":"2023-10-06T03:01:42.287998+0000","last_clean":"2023-10-06T03:01:42.287998+0000","last_became_active":"2023-10-06T03:01:42.280427+0000","last_became_peered":"2023-10-06T03:01:42.280427+0000","last_unstale":"2023-10-06T03:01:42.287998+0000","last_undegraded":"2023-10-06T03:01:42.287998+0000","last_fullsized":"2023-10-06T03:01:42.287998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:05.130474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286079+0000","last_change":"2023-10-06T03:01:42.280012+0000","last_active":"2023-10-06T03:01:42.286079+0000","last_peered":"2023-10-06T03:01:42.286079+0000","last_clean":"2023-10-06T03:01:42.286079+0000","last_became_active":"2023-10-06T03:01:42.279744+0000","last_became_peered":"2023-10-06T03:01:42.279744+0000","last_unstale":"2023-10-06T03:01:42.286079+0000","last_undegraded":"2023-10-06T03:01:42.286079+0000","last_fullsized":"2023-10-06T03:01:42.286079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373578+0000","last_change":"2023-10-06T03:01:41.380874+0000","last_active":"2023-10-06T03:01:42.373578+0000","last_peered":"2023-10-06T03:01:42.373578+0000","last_clean":"2023-10-06T03:01:42.373578+0000","last_became_active":"2023-10-06T03:01:41.379061+0000","last_became_peered":"2023-10-06T03:01:41.379061+0000","last_unstale":"2023-10-06T03:01:42.373578+0000","last_undegraded":"2023-10-06T03:01:42.373578+0000","last_fullsized":"2023-10-06T03:01:42.373578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280483+0000","last_change":"2023-10-06T03:01:42.276652+0000","last_active":"2023-10-06T03:01:42.280483+0000","last_peered":"2023-10-06T03:01:42.280483+0000","last_clean":"2023-10-06T03:01:42.280483+0000","last_became_active":"2023-10-06T03:01:42.276378+0000","last_became_peered":"2023-10-06T03:01:42.276378+0000","last_unstale":"2023-10-06T03:01:42.280483+0000","last_undegraded":"2023-10-06T03:01:42.280483+0000","last_fullsized":"2023-10-06T03:01:42.280483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270139+0000","last_change":"2023-10-06T03:01:41.379177+0000","last_active":"2023-10-06T03:01:42.270139+0000","last_peered":"2023-10-06T03:01:42.270139+0000","last_clean":"2023-10-06T03:01:42.270139+0000","last_became_active":"2023-10-06T03:01:41.378913+0000","last_became_peered":"2023-10-06T03:01:41.378913+0000","last_unstale":"2023-10-06T03:01:42.270139+0000","last_undegraded":"2023-10-06T03:01:42.270139+0000","last_fullsized":"2023-10-06T03:01:42.270139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286002+0000","last_change":"2023-10-06T03:01:42.284965+0000","last_active":"2023-10-06T03:01:42.286002+0000","last_peered":"2023-10-06T03:01:42.286002+0000","last_clean":"2023-10-06T03:01:42.286002+0000","last_became_active":"2023-10-06T03:01:42.284715+0000","last_became_peered":"2023-10-06T03:01:42.284715+0000","last_unstale":"2023-10-06T03:01:42.286002+0000","last_undegraded":"2023-10-06T03:01:42.286002+0000","last_fullsized":"2023-10-06T03:01:42.286002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268732+0000","last_change":"2023-10-06T03:01:41.383082+0000","last_active":"2023-10-06T03:01:42.268732+0000","last_peered":"2023-10-06T03:01:42.268732+0000","last_clean":"2023-10-06T03:01:42.268732+0000","last_became_active":"2023-10-06T03:01:41.382776+0000","last_became_peered":"2023-10-06T03:01:41.382776+0000","last_unstale":"2023-10-06T03:01:42.268732+0000","last_undegraded":"2023-10-06T03:01:42.268732+0000","last_fullsized":"2023-10-06T03:01:42.268732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285524+0000","last_change":"2023-10-06T03:01:42.283247+0000","last_active":"2023-10-06T03:01:42.285524+0000","last_peered":"2023-10-06T03:01:42.285524+0000","last_clean":"2023-10-06T03:01:42.285524+0000","last_became_active":"2023-10-06T03:01:42.282951+0000","last_became_peered":"2023-10-06T03:01:42.282951+0000","last_unstale":"2023-10-06T03:01:42.285524+0000","last_undegraded":"2023-10-06T03:01:42.285524+0000","last_fullsized":"2023-10-06T03:01:42.285524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269263+0000","last_change":"2023-10-06T03:01:41.381863+0000","last_active":"2023-10-06T03:01:42.269263+0000","last_peered":"2023-10-06T03:01:42.269263+0000","last_clean":"2023-10-06T03:01:42.269263+0000","last_became_active":"2023-10-06T03:01:41.381602+0000","last_became_peered":"2023-10-06T03:01:41.381602+0000","last_unstale":"2023-10-06T03:01:42.269263+0000","last_undegraded":"2023-10-06T03:01:42.269263+0000","last_fullsized":"2023-10-06T03:01:42.269263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280585+0000","last_change":"2023-10-06T03:01:42.276543+0000","last_active":"2023-10-06T03:01:42.280585+0000","last_peered":"2023-10-06T03:01:42.280585+0000","last_clean":"2023-10-06T03:01:42.280585+0000","last_became_active":"2023-10-06T03:01:42.276256+0000","last_became_peered":"2023-10-06T03:01:42.276256+0000","last_unstale":"2023-10-06T03:01:42.280585+0000","last_undegraded":"2023-10-06T03:01:42.280585+0000","last_fullsized":"2023-10-06T03:01:42.280585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267847+0000","last_change":"2023-10-06T03:01:41.378556+0000","last_active":"2023-10-06T03:01:42.267847+0000","last_peered":"2023-10-06T03:01:42.267847+0000","last_clean":"2023-10-06T03:01:42.267847+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.267847+0000","last_undegraded":"2023-10-06T03:01:42.267847+0000","last_fullsized":"2023-10-06T03:01:42.267847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:30.948926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287982+0000","last_change":"2023-10-06T03:01:42.283975+0000","last_active":"2023-10-06T03:01:42.287982+0000","last_peered":"2023-10-06T03:01:42.287982+0000","last_clean":"2023-10-06T03:01:42.287982+0000","last_became_active":"2023-10-06T03:01:42.283734+0000","last_became_peered":"2023-10-06T03:01:42.283734+0000","last_unstale":"2023-10-06T03:01:42.287982+0000","last_undegraded":"2023-10-06T03:01:42.287982+0000","last_fullsized":"2023-10-06T03:01:42.287982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.374949+0000","last_change":"2023-10-06T03:01:41.384246+0000","last_active":"2023-10-06T03:01:42.374949+0000","last_peered":"2023-10-06T03:01:42.374949+0000","last_clean":"2023-10-06T03:01:42.374949+0000","last_became_active":"2023-10-06T03:01:41.383977+0000","last_became_peered":"2023-10-06T03:01:41.383977+0000","last_unstale":"2023-10-06T03:01:42.374949+0000","last_undegraded":"2023-10-06T03:01:42.374949+0000","last_fullsized":"2023-10-06T03:01:42.374949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.372963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287980+0000","last_change":"2023-10-06T03:01:42.280871+0000","last_active":"2023-10-06T03:01:42.287980+0000","last_peered":"2023-10-06T03:01:42.287980+0000","last_clean":"2023-10-06T03:01:42.287980+0000","last_became_active":"2023-10-06T03:01:42.277563+0000","last_became_peered":"2023-10-06T03:01:42.277563+0000","last_unstale":"2023-10-06T03:01:42.287980+0000","last_undegraded":"2023-10-06T03:01:42.287980+0000","last_fullsized":"2023-10-06T03:01:42.287980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.379545+0000","last_change":"2023-10-06T03:01:41.381401+0000","last_active":"2023-10-06T03:01:42.379545+0000","last_peered":"2023-10-06T03:01:42.379545+0000","last_clean":"2023-10-06T03:01:42.379545+0000","last_became_active":"2023-10-06T03:01:41.379277+0000","last_became_peered":"2023-10-06T03:01:41.379277+0000","last_unstale":"2023-10-06T03:01:42.379545+0000","last_undegraded":"2023-10-06T03:01:42.379545+0000","last_fullsized":"2023-10-06T03:01:42.379545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284666+0000","last_change":"2023-10-06T03:01:42.281115+0000","last_active":"2023-10-06T03:01:42.284666+0000","last_peered":"2023-10-06T03:01:42.284666+0000","last_clean":"2023-10-06T03:01:42.284666+0000","last_became_active":"2023-10-06T03:01:42.278708+0000","last_became_peered":"2023-10-06T03:01:42.278708+0000","last_unstale":"2023-10-06T03:01:42.284666+0000","last_undegraded":"2023-10-06T03:01:42.284666+0000","last_fullsized":"2023-10-06T03:01:42.284666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.231633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377978+0000","last_change":"2023-10-06T03:01:41.384981+0000","last_active":"2023-10-06T03:01:42.377978+0000","last_peered":"2023-10-06T03:01:42.377978+0000","last_clean":"2023-10-06T03:01:42.377978+0000","last_became_active":"2023-10-06T03:01:41.384739+0000","last_became_peered":"2023-10-06T03:01:41.384739+0000","last_unstale":"2023-10-06T03:01:42.377978+0000","last_undegraded":"2023-10-06T03:01:42.377978+0000","last_fullsized":"2023-10-06T03:01:42.377978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280582+0000","last_change":"2023-10-06T03:01:42.276570+0000","last_active":"2023-10-06T03:01:42.280582+0000","last_peered":"2023-10-06T03:01:42.280582+0000","last_clean":"2023-10-06T03:01:42.280582+0000","last_became_active":"2023-10-06T03:01:42.276163+0000","last_became_peered":"2023-10-06T03:01:42.276163+0000","last_unstale":"2023-10-06T03:01:42.280582+0000","last_undegraded":"2023-10-06T03:01:42.280582+0000","last_fullsized":"2023-10-06T03:01:42.280582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.272049+0000","last_change":"2023-10-06T03:01:41.381162+0000","last_active":"2023-10-06T03:01:42.272049+0000","last_peered":"2023-10-06T03:01:42.272049+0000","last_clean":"2023-10-06T03:01:42.272049+0000","last_became_active":"2023-10-06T03:01:41.380885+0000","last_became_peered":"2023-10-06T03:01:41.380885+0000","last_unstale":"2023-10-06T03:01:42.272049+0000","last_undegraded":"2023-10-06T03:01:42.272049+0000","last_fullsized":"2023-10-06T03:01:42.272049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286056+0000","last_change":"2023-10-06T03:01:42.279843+0000","last_active":"2023-10-06T03:01:42.286056+0000","last_peered":"2023-10-06T03:01:42.286056+0000","last_clean":"2023-10-06T03:01:42.286056+0000","last_became_active":"2023-10-06T03:01:42.279409+0000","last_became_peered":"2023-10-06T03:01:42.279409+0000","last_unstale":"2023-10-06T03:01:42.286056+0000","last_undegraded":"2023-10-06T03:01:42.286056+0000","last_fullsized":"2023-10-06T03:01:42.286056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275415+0000","last_change":"2023-10-06T03:01:41.382403+0000","last_active":"2023-10-06T03:01:42.275415+0000","last_peered":"2023-10-06T03:01:42.275415+0000","last_clean":"2023-10-06T03:01:42.275415+0000","last_became_active":"2023-10-06T03:01:41.382119+0000","last_became_peered":"2023-10-06T03:01:41.382119+0000","last_unstale":"2023-10-06T03:01:42.275415+0000","last_undegraded":"2023-10-06T03:01:42.275415+0000","last_fullsized":"2023-10-06T03:01:42.275415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:17.777678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285768+0000","last_change":"2023-10-06T03:01:42.280092+0000","last_active":"2023-10-06T03:01:42.285768+0000","last_peered":"2023-10-06T03:01:42.285768+0000","last_clean":"2023-10-06T03:01:42.285768+0000","last_became_active":"2023-10-06T03:01:42.279833+0000","last_became_peered":"2023-10-06T03:01:42.279833+0000","last_unstale":"2023-10-06T03:01:42.285768+0000","last_undegraded":"2023-10-06T03:01:42.285768+0000","last_fullsized":"2023-10-06T03:01:42.285768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.375608+0000","last_change":"2023-10-06T03:01:41.386282+0000","last_active":"2023-10-06T03:01:42.375608+0000","last_peered":"2023-10-06T03:01:42.375608+0000","last_clean":"2023-10-06T03:01:42.375608+0000","last_became_active":"2023-10-06T03:01:41.385955+0000","last_became_peered":"2023-10-06T03:01:41.385955+0000","last_unstale":"2023-10-06T03:01:42.375608+0000","last_undegraded":"2023-10-06T03:01:42.375608+0000","last_fullsized":"2023-10-06T03:01:42.375608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287940+0000","last_change":"2023-10-06T03:01:42.283470+0000","last_active":"2023-10-06T03:01:42.287940+0000","last_peered":"2023-10-06T03:01:42.287940+0000","last_clean":"2023-10-06T03:01:42.287940+0000","last_became_active":"2023-10-06T03:01:42.283220+0000","last_became_peered":"2023-10-06T03:01:42.283220+0000","last_unstale":"2023-10-06T03:01:42.287940+0000","last_undegraded":"2023-10-06T03:01:42.287940+0000","last_fullsized":"2023-10-06T03:01:42.287940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373872+0000","last_change":"2023-10-06T03:01:41.380049+0000","last_active":"2023-10-06T03:01:42.373872+0000","last_peered":"2023-10-06T03:01:42.373872+0000","last_clean":"2023-10-06T03:01:42.373872+0000","last_became_active":"2023-10-06T03:01:41.379732+0000","last_became_peered":"2023-10-06T03:01:41.379732+0000","last_unstale":"2023-10-06T03:01:42.373872+0000","last_undegraded":"2023-10-06T03:01:42.373872+0000","last_fullsized":"2023-10-06T03:01:42.373872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:03.057587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286048+0000","last_change":"2023-10-06T03:01:42.279660+0000","last_active":"2023-10-06T03:01:42.286048+0000","last_peered":"2023-10-06T03:01:42.286048+0000","last_clean":"2023-10-06T03:01:42.286048+0000","last_became_active":"2023-10-06T03:01:42.279394+0000","last_became_peered":"2023-10-06T03:01:42.279394+0000","last_unstale":"2023-10-06T03:01:42.286048+0000","last_undegraded":"2023-10-06T03:01:42.286048+0000","last_fullsized":"2023-10-06T03:01:42.286048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377939+0000","last_change":"2023-10-06T03:01:41.384922+0000","last_active":"2023-10-06T03:01:42.377939+0000","last_peered":"2023-10-06T03:01:42.377939+0000","last_clean":"2023-10-06T03:01:42.377939+0000","last_became_active":"2023-10-06T03:01:41.384669+0000","last_became_peered":"2023-10-06T03:01:41.384669+0000","last_unstale":"2023-10-06T03:01:42.377939+0000","last_undegraded":"2023-10-06T03:01:42.377939+0000","last_fullsized":"2023-10-06T03:01:42.377939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.334690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284069+0000","last_change":"2023-10-06T03:01:42.277487+0000","last_active":"2023-10-06T03:01:42.284069+0000","last_peered":"2023-10-06T03:01:42.284069+0000","last_clean":"2023-10-06T03:01:42.284069+0000","last_became_active":"2023-10-06T03:01:42.277135+0000","last_became_peered":"2023-10-06T03:01:42.277135+0000","last_unstale":"2023-10-06T03:01:42.284069+0000","last_undegraded":"2023-10-06T03:01:42.284069+0000","last_fullsized":"2023-10-06T03:01:42.284069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373790+0000","last_change":"2023-10-06T03:01:41.380292+0000","last_active":"2023-10-06T03:01:42.373790+0000","last_peered":"2023-10-06T03:01:42.373790+0000","last_clean":"2023-10-06T03:01:42.373790+0000","last_became_active":"2023-10-06T03:01:41.380074+0000","last_became_peered":"2023-10-06T03:01:41.380074+0000","last_unstale":"2023-10-06T03:01:42.373790+0000","last_undegraded":"2023-10-06T03:01:42.373790+0000","last_fullsized":"2023-10-06T03:01:42.373790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:27.982776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282615+0000","last_change":"2023-10-06T03:01:42.277855+0000","last_active":"2023-10-06T03:01:42.282615+0000","last_peered":"2023-10-06T03:01:42.282615+0000","last_clean":"2023-10-06T03:01:42.282615+0000","last_became_active":"2023-10-06T03:01:42.277539+0000","last_became_peered":"2023-10-06T03:01:42.277539+0000","last_unstale":"2023-10-06T03:01:42.282615+0000","last_undegraded":"2023-10-06T03:01:42.282615+0000","last_fullsized":"2023-10-06T03:01:42.282615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268326+0000","last_change":"2023-10-06T03:01:41.379182+0000","last_active":"2023-10-06T03:01:42.268326+0000","last_peered":"2023-10-06T03:01:42.268326+0000","last_clean":"2023-10-06T03:01:42.268326+0000","last_became_active":"2023-10-06T03:01:41.378857+0000","last_became_peered":"2023-10-06T03:01:41.378857+0000","last_unstale":"2023-10-06T03:01:42.268326+0000","last_undegraded":"2023-10-06T03:01:42.268326+0000","last_fullsized":"2023-10-06T03:01:42.268326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281816+0000","last_change":"2023-10-06T03:01:42.277028+0000","last_active":"2023-10-06T03:01:42.281816+0000","last_peered":"2023-10-06T03:01:42.281816+0000","last_clean":"2023-10-06T03:01:42.281816+0000","last_became_active":"2023-10-06T03:01:42.276777+0000","last_became_peered":"2023-10-06T03:01:42.276777+0000","last_unstale":"2023-10-06T03:01:42.281816+0000","last_undegraded":"2023-10-06T03:01:42.281816+0000","last_fullsized":"2023-10-06T03:01:42.281816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.371956+0000","last_change":"2023-10-06T03:01:41.383140+0000","last_active":"2023-10-06T03:01:42.371956+0000","last_peered":"2023-10-06T03:01:42.371956+0000","last_clean":"2023-10-06T03:01:42.371956+0000","last_became_active":"2023-10-06T03:01:41.382894+0000","last_became_peered":"2023-10-06T03:01:41.382894+0000","last_unstale":"2023-10-06T03:01:42.371956+0000","last_undegraded":"2023-10-06T03:01:42.371956+0000","last_fullsized":"2023-10-06T03:01:42.371956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284623+0000","last_change":"2023-10-06T03:01:42.280965+0000","last_active":"2023-10-06T03:01:42.284623+0000","last_peered":"2023-10-06T03:01:42.284623+0000","last_clean":"2023-10-06T03:01:42.284623+0000","last_became_active":"2023-10-06T03:01:42.278565+0000","last_became_peered":"2023-10-06T03:01:42.278565+0000","last_unstale":"2023-10-06T03:01:42.284623+0000","last_undegraded":"2023-10-06T03:01:42.284623+0000","last_fullsized":"2023-10-06T03:01:42.284623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274323+0000","last_change":"2023-10-06T03:01:41.385727+0000","last_active":"2023-10-06T03:01:42.274323+0000","last_peered":"2023-10-06T03:01:42.274323+0000","last_clean":"2023-10-06T03:01:42.274323+0000","last_became_active":"2023-10-06T03:01:41.385354+0000","last_became_peered":"2023-10-06T03:01:41.385354+0000","last_unstale":"2023-10-06T03:01:42.274323+0000","last_undegraded":"2023-10-06T03:01:42.274323+0000","last_fullsized":"2023-10-06T03:01:42.274323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284128+0000","last_change":"2023-10-06T03:01:42.278082+0000","last_active":"2023-10-06T03:01:42.284128+0000","last_peered":"2023-10-06T03:01:42.284128+0000","last_clean":"2023-10-06T03:01:42.284128+0000","last_became_active":"2023-10-06T03:01:42.277812+0000","last_became_peered":"2023-10-06T03:01:42.277812+0000","last_unstale":"2023-10-06T03:01:42.284128+0000","last_undegraded":"2023-10-06T03:01:42.284128+0000","last_fullsized":"2023-10-06T03:01:42.284128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270092+0000","last_change":"2023-10-06T03:01:41.381181+0000","last_active":"2023-10-06T03:01:42.270092+0000","last_peered":"2023-10-06T03:01:42.270092+0000","last_clean":"2023-10-06T03:01:42.270092+0000","last_became_active":"2023-10-06T03:01:41.378547+0000","last_became_peered":"2023-10-06T03:01:41.378547+0000","last_unstale":"2023-10-06T03:01:42.270092+0000","last_undegraded":"2023-10-06T03:01:42.270092+0000","last_fullsized":"2023-10-06T03:01:42.270092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284704+0000","last_change":"2023-10-06T03:01:42.281464+0000","last_active":"2023-10-06T03:01:42.284704+0000","last_peered":"2023-10-06T03:01:42.284704+0000","last_clean":"2023-10-06T03:01:42.284704+0000","last_became_active":"2023-10-06T03:01:42.281186+0000","last_became_peered":"2023-10-06T03:01:42.281186+0000","last_unstale":"2023-10-06T03:01:42.284704+0000","last_undegraded":"2023-10-06T03:01:42.284704+0000","last_fullsized":"2023-10-06T03:01:42.284704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.673116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268609+0000","last_change":"2023-10-06T03:01:41.378017+0000","last_active":"2023-10-06T03:01:42.268609+0000","last_peered":"2023-10-06T03:01:42.268609+0000","last_clean":"2023-10-06T03:01:42.268609+0000","last_became_active":"2023-10-06T03:01:41.377759+0000","last_became_peered":"2023-10-06T03:01:41.377759+0000","last_unstale":"2023-10-06T03:01:42.268609+0000","last_undegraded":"2023-10-06T03:01:42.268609+0000","last_fullsized":"2023-10-06T03:01:42.268609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287848+0000","last_change":"2023-10-06T03:01:42.278383+0000","last_active":"2023-10-06T03:01:42.287848+0000","last_peered":"2023-10-06T03:01:42.287848+0000","last_clean":"2023-10-06T03:01:42.287848+0000","last_became_active":"2023-10-06T03:01:42.277973+0000","last_became_peered":"2023-10-06T03:01:42.277973+0000","last_unstale":"2023-10-06T03:01:42.287848+0000","last_undegraded":"2023-10-06T03:01:42.287848+0000","last_fullsized":"2023-10-06T03:01:42.287848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271774+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:01:42.271774+0000","last_peered":"2023-10-06T03:01:42.271774+0000","last_clean":"2023-10-06T03:01:42.271774+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:01:42.271774+0000","last_undegraded":"2023-10-06T03:01:42.271774+0000","last_fullsized":"2023-10-06T03:01:42.271774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267761+0000","last_change":"2023-10-06T03:01:41.379310+0000","last_active":"2023-10-06T03:01:42.267761+0000","last_peered":"2023-10-06T03:01:42.267761+0000","last_clean":"2023-10-06T03:01:42.267761+0000","last_became_active":"2023-10-06T03:01:41.379017+0000","last_became_peered":"2023-10-06T03:01:41.379017+0000","last_unstale":"2023-10-06T03:01:42.267761+0000","last_undegraded":"2023-10-06T03:01:42.267761+0000","last_fullsized":"2023-10-06T03:01:42.267761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288159+0000","last_change":"2023-10-06T03:01:42.285640+0000","last_active":"2023-10-06T03:01:42.288159+0000","last_peered":"2023-10-06T03:01:42.288159+0000","last_clean":"2023-10-06T03:01:42.288159+0000","last_became_active":"2023-10-06T03:01:42.283071+0000","last_became_peered":"2023-10-06T03:01:42.283071+0000","last_unstale":"2023-10-06T03:01:42.288159+0000","last_undegraded":"2023-10-06T03:01:42.288159+0000","last_fullsized":"2023-10-06T03:01:42.288159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269088+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:01:42.269088+0000","last_peered":"2023-10-06T03:01:42.269088+0000","last_clean":"2023-10-06T03:01:42.269088+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:01:42.269088+0000","last_undegraded":"2023-10-06T03:01:42.269088+0000","last_fullsized":"2023-10-06T03:01:42.269088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268853+0000","last_change":"2023-10-06T03:01:41.382782+0000","last_active":"2023-10-06T03:01:42.268853+0000","last_peered":"2023-10-06T03:01:42.268853+0000","last_clean":"2023-10-06T03:01:42.268853+0000","last_became_active":"2023-10-06T03:01:41.382522+0000","last_became_peered":"2023-10-06T03:01:41.382522+0000","last_unstale":"2023-10-06T03:01:42.268853+0000","last_undegraded":"2023-10-06T03:01:42.268853+0000","last_fullsized":"2023-10-06T03:01:42.268853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288056+0000","last_change":"2023-10-06T03:01:42.282263+0000","last_active":"2023-10-06T03:01:42.288056+0000","last_peered":"2023-10-06T03:01:42.288056+0000","last_clean":"2023-10-06T03:01:42.288056+0000","last_became_active":"2023-10-06T03:01:42.282019+0000","last_became_peered":"2023-10-06T03:01:42.282019+0000","last_unstale":"2023-10-06T03:01:42.288056+0000","last_undegraded":"2023-10-06T03:01:42.288056+0000","last_fullsized":"2023-10-06T03:01:42.288056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.265449+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:01:42.265449+0000","last_peered":"2023-10-06T03:01:42.265449+0000","last_clean":"2023-10-06T03:01:42.265449+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:01:42.265449+0000","last_undegraded":"2023-10-06T03:01:42.265449+0000","last_fullsized":"2023-10-06T03:01:42.265449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268243+0000","last_change":"2023-10-06T03:01:41.378349+0000","last_active":"2023-10-06T03:01:42.268243+0000","last_peered":"2023-10-06T03:01:42.268243+0000","last_clean":"2023-10-06T03:01:42.268243+0000","last_became_active":"2023-10-06T03:01:41.378089+0000","last_became_peered":"2023-10-06T03:01:41.378089+0000","last_unstale":"2023-10-06T03:01:42.268243+0000","last_undegraded":"2023-10-06T03:01:42.268243+0000","last_fullsized":"2023-10-06T03:01:42.268243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288080+0000","last_change":"2023-10-06T03:01:42.283418+0000","last_active":"2023-10-06T03:01:42.288080+0000","last_peered":"2023-10-06T03:01:42.288080+0000","last_clean":"2023-10-06T03:01:42.288080+0000","last_became_active":"2023-10-06T03:01:42.283035+0000","last_became_peered":"2023-10-06T03:01:42.283035+0000","last_unstale":"2023-10-06T03:01:42.288080+0000","last_undegraded":"2023-10-06T03:01:42.288080+0000","last_fullsized":"2023-10-06T03:01:42.288080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:27.703859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.266739+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:01:42.266739+0000","last_peered":"2023-10-06T03:01:42.266739+0000","last_clean":"2023-10-06T03:01:42.266739+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:01:42.266739+0000","last_undegraded":"2023-10-06T03:01:42.266739+0000","last_fullsized":"2023-10-06T03:01:42.266739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274310+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:01:42.274310+0000","last_peered":"2023-10-06T03:01:42.274310+0000","last_clean":"2023-10-06T03:01:42.274310+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:01:42.274310+0000","last_undegraded":"2023-10-06T03:01:42.274310+0000","last_fullsized":"2023-10-06T03:01:42.274310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268619+0000","last_change":"2023-10-06T03:01:41.378526+0000","last_active":"2023-10-06T03:01:42.268619+0000","last_peered":"2023-10-06T03:01:42.268619+0000","last_clean":"2023-10-06T03:01:42.268619+0000","last_became_active":"2023-10-06T03:01:41.378245+0000","last_became_peered":"2023-10-06T03:01:41.378245+0000","last_unstale":"2023-10-06T03:01:42.268619+0000","last_undegraded":"2023-10-06T03:01:42.268619+0000","last_fullsized":"2023-10-06T03:01:42.268619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284302+0000","last_change":"2023-10-06T03:01:42.281327+0000","last_active":"2023-10-06T03:01:42.284302+0000","last_peered":"2023-10-06T03:01:42.284302+0000","last_clean":"2023-10-06T03:01:42.284302+0000","last_became_active":"2023-10-06T03:01:42.281028+0000","last_became_peered":"2023-10-06T03:01:42.281028+0000","last_unstale":"2023-10-06T03:01:42.284302+0000","last_undegraded":"2023-10-06T03:01:42.284302+0000","last_fullsized":"2023-10-06T03:01:42.284302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:28.986647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274776+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:01:42.274776+0000","last_peered":"2023-10-06T03:01:42.274776+0000","last_clean":"2023-10-06T03:01:42.274776+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:01:42.274776+0000","last_undegraded":"2023-10-06T03:01:42.274776+0000","last_fullsized":"2023-10-06T03:01:42.274776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268368+0000","last_change":"2023-10-06T03:01:41.377764+0000","last_active":"2023-10-06T03:01:42.268368+0000","last_peered":"2023-10-06T03:01:42.268368+0000","last_clean":"2023-10-06T03:01:42.268368+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.268368+0000","last_undegraded":"2023-10-06T03:01:42.268368+0000","last_fullsized":"2023-10-06T03:01:42.268368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285753+0000","last_change":"2023-10-06T03:01:42.279939+0000","last_active":"2023-10-06T03:01:42.285753+0000","last_peered":"2023-10-06T03:01:42.285753+0000","last_clean":"2023-10-06T03:01:42.285753+0000","last_became_active":"2023-10-06T03:01:42.279453+0000","last_became_peered":"2023-10-06T03:01:42.279453+0000","last_unstale":"2023-10-06T03:01:42.285753+0000","last_undegraded":"2023-10-06T03:01:42.285753+0000","last_fullsized":"2023-10-06T03:01:42.285753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.276154+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:01:42.276154+0000","last_peered":"2023-10-06T03:01:42.276154+0000","last_clean":"2023-10-06T03:01:42.276154+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:01:42.276154+0000","last_undegraded":"2023-10-06T03:01:42.276154+0000","last_fullsized":"2023-10-06T03:01:42.276154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271880+0000","last_change":"2023-10-06T03:01:41.378694+0000","last_active":"2023-10-06T03:01:42.271880+0000","last_peered":"2023-10-06T03:01:42.271880+0000","last_clean":"2023-10-06T03:01:42.271880+0000","last_became_active":"2023-10-06T03:01:41.378366+0000","last_became_peered":"2023-10-06T03:01:41.378366+0000","last_unstale":"2023-10-06T03:01:42.271880+0000","last_undegraded":"2023-10-06T03:01:42.271880+0000","last_fullsized":"2023-10-06T03:01:42.271880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285642+0000","last_change":"2023-10-06T03:01:42.278204+0000","last_active":"2023-10-06T03:01:42.285642+0000","last_peered":"2023-10-06T03:01:42.285642+0000","last_clean":"2023-10-06T03:01:42.285642+0000","last_became_active":"2023-10-06T03:01:42.277950+0000","last_became_peered":"2023-10-06T03:01:42.277950+0000","last_unstale":"2023-10-06T03:01:42.285642+0000","last_undegraded":"2023-10-06T03:01:42.285642+0000","last_fullsized":"2023-10-06T03:01:42.285642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378135+0000","last_change":"2023-10-06T03:01:41.378450+0000","last_active":"2023-10-06T03:01:42.378135+0000","last_peered":"2023-10-06T03:01:42.378135+0000","last_clean":"2023-10-06T03:01:42.378135+0000","last_became_active":"2023-10-06T03:01:41.378167+0000","last_became_peered":"2023-10-06T03:01:41.378167+0000","last_unstale":"2023-10-06T03:01:42.378135+0000","last_undegraded":"2023-10-06T03:01:42.378135+0000","last_fullsized":"2023-10-06T03:01:42.378135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269297+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:01:42.269297+0000","last_peered":"2023-10-06T03:01:42.269297+0000","last_clean":"2023-10-06T03:01:42.269297+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:01:42.269297+0000","last_undegraded":"2023-10-06T03:01:42.269297+0000","last_fullsized":"2023-10-06T03:01:42.269297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288032+0000","last_change":"2023-10-06T03:01:42.284704+0000","last_active":"2023-10-06T03:01:42.288032+0000","last_peered":"2023-10-06T03:01:42.288032+0000","last_clean":"2023-10-06T03:01:42.288032+0000","last_became_active":"2023-10-06T03:01:42.284380+0000","last_became_peered":"2023-10-06T03:01:42.284380+0000","last_unstale":"2023-10-06T03:01:42.288032+0000","last_undegraded":"2023-10-06T03:01:42.288032+0000","last_fullsized":"2023-10-06T03:01:42.288032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.273934+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:01:42.273934+0000","last_peered":"2023-10-06T03:01:42.273934+0000","last_clean":"2023-10-06T03:01:42.273934+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:01:42.273934+0000","last_undegraded":"2023-10-06T03:01:42.273934+0000","last_fullsized":"2023-10-06T03:01:42.273934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270423+0000","last_change":"2023-10-06T03:01:41.376173+0000","last_active":"2023-10-06T03:01:42.270423+0000","last_peered":"2023-10-06T03:01:42.270423+0000","last_clean":"2023-10-06T03:01:42.270423+0000","last_became_active":"2023-10-06T03:01:41.375877+0000","last_became_peered":"2023-10-06T03:01:41.375877+0000","last_unstale":"2023-10-06T03:01:42.270423+0000","last_undegraded":"2023-10-06T03:01:42.270423+0000","last_fullsized":"2023-10-06T03:01:42.270423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.277029+0000","last_change":"2023-10-06T03:01:42.271755+0000","last_active":"2023-10-06T03:01:42.277029+0000","last_peered":"2023-10-06T03:01:42.277029+0000","last_clean":"2023-10-06T03:01:42.277029+0000","last_became_active":"2023-10-06T03:01:42.271483+0000","last_became_peered":"2023-10-06T03:01:42.271483+0000","last_unstale":"2023-10-06T03:01:42.277029+0000","last_undegraded":"2023-10-06T03:01:42.277029+0000","last_fullsized":"2023-10-06T03:01:42.277029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373544+0000","last_change":"2023-10-06T03:01:41.381914+0000","last_active":"2023-10-06T03:01:42.373544+0000","last_peered":"2023-10-06T03:01:42.373544+0000","last_clean":"2023-10-06T03:01:42.373544+0000","last_became_active":"2023-10-06T03:01:41.381635+0000","last_became_peered":"2023-10-06T03:01:41.381635+0000","last_unstale":"2023-10-06T03:01:42.373544+0000","last_undegraded":"2023-10-06T03:01:42.373544+0000","last_fullsized":"2023-10-06T03:01:42.373544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280515+0000","last_change":"2023-10-06T03:01:42.277000+0000","last_active":"2023-10-06T03:01:42.280515+0000","last_peered":"2023-10-06T03:01:42.280515+0000","last_clean":"2023-10-06T03:01:42.280515+0000","last_became_active":"2023-10-06T03:01:42.276746+0000","last_became_peered":"2023-10-06T03:01:42.276746+0000","last_unstale":"2023-10-06T03:01:42.280515+0000","last_undegraded":"2023-10-06T03:01:42.280515+0000","last_fullsized":"2023-10-06T03:01:42.280515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378477+0000","last_change":"2023-10-06T03:01:41.377203+0000","last_active":"2023-10-06T03:01:42.378477+0000","last_peered":"2023-10-06T03:01:42.378477+0000","last_clean":"2023-10-06T03:01:42.378477+0000","last_became_active":"2023-10-06T03:01:41.376950+0000","last_became_peered":"2023-10-06T03:01:41.376950+0000","last_unstale":"2023-10-06T03:01:42.378477+0000","last_undegraded":"2023-10-06T03:01:42.378477+0000","last_fullsized":"2023-10-06T03:01:42.378477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284682+0000","last_change":"2023-10-06T03:01:42.282305+0000","last_active":"2023-10-06T03:01:42.284682+0000","last_peered":"2023-10-06T03:01:42.284682+0000","last_clean":"2023-10-06T03:01:42.284682+0000","last_became_active":"2023-10-06T03:01:42.282037+0000","last_became_peered":"2023-10-06T03:01:42.282037+0000","last_unstale":"2023-10-06T03:01:42.284682+0000","last_undegraded":"2023-10-06T03:01:42.284682+0000","last_fullsized":"2023-10-06T03:01:42.284682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:24.802638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378519+0000","last_change":"2023-10-06T03:01:41.377561+0000","last_active":"2023-10-06T03:01:42.378519+0000","last_peered":"2023-10-06T03:01:42.378519+0000","last_clean":"2023-10-06T03:01:42.378519+0000","last_became_active":"2023-10-06T03:01:41.377308+0000","last_became_peered":"2023-10-06T03:01:41.377308+0000","last_unstale":"2023-10-06T03:01:42.378519+0000","last_undegraded":"2023-10-06T03:01:42.378519+0000","last_fullsized":"2023-10-06T03:01:42.378519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284090+0000","last_change":"2023-10-06T03:01:42.277443+0000","last_active":"2023-10-06T03:01:42.284090+0000","last_peered":"2023-10-06T03:01:42.284090+0000","last_clean":"2023-10-06T03:01:42.284090+0000","last_became_active":"2023-10-06T03:01:42.277152+0000","last_became_peered":"2023-10-06T03:01:42.277152+0000","last_unstale":"2023-10-06T03:01:42.284090+0000","last_undegraded":"2023-10-06T03:01:42.284090+0000","last_fullsized":"2023-10-06T03:01:42.284090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373306+0000","last_change":"2023-10-06T03:01:41.379186+0000","last_active":"2023-10-06T03:01:42.373306+0000","last_peered":"2023-10-06T03:01:42.373306+0000","last_clean":"2023-10-06T03:01:42.373306+0000","last_became_active":"2023-10-06T03:01:41.378963+0000","last_became_peered":"2023-10-06T03:01:41.378963+0000","last_unstale":"2023-10-06T03:01:42.373306+0000","last_undegraded":"2023-10-06T03:01:42.373306+0000","last_fullsized":"2023-10-06T03:01:42.373306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280768+0000","last_change":"2023-10-06T03:01:42.276973+0000","last_active":"2023-10-06T03:01:42.280768+0000","last_peered":"2023-10-06T03:01:42.280768+0000","last_clean":"2023-10-06T03:01:42.280768+0000","last_became_active":"2023-10-06T03:01:42.276297+0000","last_became_peered":"2023-10-06T03:01:42.276297+0000","last_unstale":"2023-10-06T03:01:42.280768+0000","last_undegraded":"2023-10-06T03:01:42.280768+0000","last_fullsized":"2023-10-06T03:01:42.280768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373671+0000","last_change":"2023-10-06T03:01:41.380486+0000","last_active":"2023-10-06T03:01:42.373671+0000","last_peered":"2023-10-06T03:01:42.373671+0000","last_clean":"2023-10-06T03:01:42.373671+0000","last_became_active":"2023-10-06T03:01:41.380212+0000","last_became_peered":"2023-10-06T03:01:41.380212+0000","last_unstale":"2023-10-06T03:01:42.373671+0000","last_undegraded":"2023-10-06T03:01:42.373671+0000","last_fullsized":"2023-10-06T03:01:42.373671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284486+0000","last_change":"2023-10-06T03:01:42.280604+0000","last_active":"2023-10-06T03:01:42.284486+0000","last_peered":"2023-10-06T03:01:42.284486+0000","last_clean":"2023-10-06T03:01:42.284486+0000","last_became_active":"2023-10-06T03:01:42.280041+0000","last_became_peered":"2023-10-06T03:01:42.280041+0000","last_unstale":"2023-10-06T03:01:42.284486+0000","last_undegraded":"2023-10-06T03:01:42.284486+0000","last_fullsized":"2023-10-06T03:01:42.284486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:02.038765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283231+0000","last_change":"2023-10-06T03:01:42.277027+0000","last_active":"2023-10-06T03:01:42.283231+0000","last_peered":"2023-10-06T03:01:42.283231+0000","last_clean":"2023-10-06T03:01:42.283231+0000","last_became_active":"2023-10-06T03:01:42.276730+0000","last_became_peered":"2023-10-06T03:01:42.276730+0000","last_unstale":"2023-10-06T03:01:42.283231+0000","last_undegraded":"2023-10-06T03:01:42.283231+0000","last_fullsized":"2023-10-06T03:01:42.283231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:18.781548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280787+0000","last_change":"2023-10-06T03:01:42.277145+0000","last_active":"2023-10-06T03:01:42.280787+0000","last_peered":"2023-10-06T03:01:42.280787+0000","last_clean":"2023-10-06T03:01:42.280787+0000","last_became_active":"2023-10-06T03:01:42.276735+0000","last_became_peered":"2023-10-06T03:01:42.276735+0000","last_unstale":"2023-10-06T03:01:42.280787+0000","last_undegraded":"2023-10-06T03:01:42.280787+0000","last_fullsized":"2023-10-06T03:01:42.280787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284561+0000","last_change":"2023-10-06T03:01:42.281639+0000","last_active":"2023-10-06T03:01:42.284561+0000","last_peered":"2023-10-06T03:01:42.284561+0000","last_clean":"2023-10-06T03:01:42.284561+0000","last_became_active":"2023-10-06T03:01:42.278720+0000","last_became_peered":"2023-10-06T03:01:42.278720+0000","last_unstale":"2023-10-06T03:01:42.284561+0000","last_undegraded":"2023-10-06T03:01:42.284561+0000","last_fullsized":"2023-10-06T03:01:42.284561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:03.312309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288057+0000","last_change":"2023-10-06T03:01:42.283924+0000","last_active":"2023-10-06T03:01:42.288057+0000","last_peered":"2023-10-06T03:01:42.288057+0000","last_clean":"2023-10-06T03:01:42.288057+0000","last_became_active":"2023-10-06T03:01:42.283527+0000","last_became_peered":"2023-10-06T03:01:42.283527+0000","last_unstale":"2023-10-06T03:01:42.288057+0000","last_undegraded":"2023-10-06T03:01:42.288057+0000","last_fullsized":"2023-10-06T03:01:42.288057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288138+0000","last_change":"2023-10-06T03:01:42.284183+0000","last_active":"2023-10-06T03:01:42.288138+0000","last_peered":"2023-10-06T03:01:42.288138+0000","last_clean":"2023-10-06T03:01:42.288138+0000","last_became_active":"2023-10-06T03:01:42.283895+0000","last_became_peered":"2023-10-06T03:01:42.283895+0000","last_unstale":"2023-10-06T03:01:42.288138+0000","last_undegraded":"2023-10-06T03:01:42.288138+0000","last_fullsized":"2023-10-06T03:01:42.288138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298120,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073720,"statfs":{"total":96636764160,"available":96331489280,"internally_reserved":0,"allocated":851968,"data_stored":674798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298096,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073744,"statfs":{"total":96636764160,"available":96331513856,"internally_reserved":0,"allocated":827392,"data_stored":673600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.222}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:01:46.683 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:01:46.684 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T03:01:46.684 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T03:01:46.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T03:01:46.684 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T03:01:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.830+0000 7ff08fbed700 1 Processor -- start 2023-10-06T03:01:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.830+0000 7ff08fbed700 1 -- start start 2023-10-06T03:01:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08fbed700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff088159c60 0x7ff08815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08fbed700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff08815db10 0x7ff088069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08fbed700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08815a160 con 0x7ff088159890 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08fbed700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08815df50 con 0x7ff088159c60 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08fbed700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0880d13a0 con 0x7ff08815db10 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08d188700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff088159c60 0x7ff08815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08d188700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff088159c60 0x7ff08815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44314/0 (socket says 172.21.15.92:44314) 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.831+0000 7ff08d188700 1 -- 172.21.15.92:0/3549228288 learned_addr learned my addr 172.21.15.92:0/3549228288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08e18a700 1 --2- 172.21.15.92:0/3549228288 >> v2:172.21.15.136:3300/0 conn(0x7ff08815db10 0x7ff088069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08d188700 1 -- 172.21.15.92:0/3549228288 >> v2:172.21.15.136:3300/0 conn(0x7ff08815db10 msgr2=0x7ff088069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08d188700 1 --2- 172.21.15.92:0/3549228288 >> v2:172.21.15.136:3300/0 conn(0x7ff08815db10 0x7ff088069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08d188700 1 -- 172.21.15.92:0/3549228288 >> v1:172.21.15.136:6789/0 conn(0x7ff088159890 legacy=0x7ff08815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08d188700 1 -- 172.21.15.92:0/3549228288 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08815e140 con 0x7ff088159c60 2023-10-06T03:01:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08d188700 1 --2- 172.21.15.92:0/3549228288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff088159c60 0x7ff08815d5d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff08806a890 tx=0x7ff078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b2b0842bfc521a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08c987700 1 -- 172.21.15.92:0/3549228288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800b860 con 0x7ff088159c60 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08c987700 1 -- 172.21.15.92:0/3549228288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800b9c0 con 0x7ff088159c60 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.832+0000 7ff08c987700 1 -- 172.21.15.92:0/3549228288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800bd60 con 0x7ff088159c60 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 -- 172.21.15.92:0/3549228288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff088159c60 msgr2=0x7ff08815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 --2- 172.21.15.92:0/3549228288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff088159c60 0x7ff08815d5d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff08806a890 tx=0x7ff078009250 comp rx=0 tx=0).stop 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 -- 172.21.15.92:0/3549228288 shutdown_connections 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 --2- 172.21.15.92:0/3549228288 >> v2:172.21.15.136:3300/0 conn(0x7ff08815db10 0x7ff088069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 --2- 172.21.15.92:0/3549228288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff088159c60 0x7ff08815d5d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 -- 172.21.15.92:0/3549228288 >> 172.21.15.92:0/3549228288 conn(0x7ff088068e20 msgr2=0x7ff0880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 -- 172.21.15.92:0/3549228288 shutdown_connections 2023-10-06T03:01:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.833+0000 7ff08fbed700 1 -- 172.21.15.92:0/3549228288 wait complete. 2023-10-06T03:01:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 Processor -- start 2023-10-06T03:01:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 -- start start 2023-10-06T03:01:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff088159890 0x7ff088140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff088159c60 0x7ff0880b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 0x7ff0880b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088141300 con 0x7ff08815db10 2023-10-06T03:01:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.834+0000 7ff08fbed700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880cd6d0 con 0x7ff088159890 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08fbed700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0880cd9b0 con 0x7ff088159c60 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 0x7ff0880b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 0x7ff0880b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44320/0 (socket says 172.21.15.92:44320) 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 -- 172.21.15.92:0/162231435 learned_addr learned my addr 172.21.15.92:0/162231435 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08d989700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff088159890 0x7ff088140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08d188700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff088159c60 0x7ff0880b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 -- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff088159c60 msgr2=0x7ff0880b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff088159c60 0x7ff0880b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 -- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff088159890 msgr2=0x7ff088140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff088159890 0x7ff088140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 -- 172.21.15.92:0/162231435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0780094f0 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08e18a700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 0x7ff0880b5250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff080007f20 tx=0x7ff080009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000eb90 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08000ed10 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0880cdc90 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff080014670 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.835+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0880ce250 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.836+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0880ce440 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.836+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0800149e0 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.837+0000 7ff07dffb700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff06c066f30 0x7ff06c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.837+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff080084c90 con 0x7ff08815db10 2023-10-06T03:01:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.837+0000 7ff08d989700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff06c066f30 0x7ff06c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.837+0000 7ff08d989700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff06c066f30 0x7ff06c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff084003a80 tx=0x7ff084007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:46.841+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff080051da0 con 0x7ff08815db10 2023-10-06T03:01:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.077+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff08804c3c0 con 0x7ff08815db10 2023-10-06T03:01:47.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.079+0000 7ff07dffb700 1 -- 172.21.15.92:0/162231435 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163698 (secure 0 0 0) 0x7ff0800553c0 con 0x7ff08815db10 2023-10-06T03:01:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:47.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff06c066f30 msgr2=0x7ff06c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff06c066f30 0x7ff06c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff084003a80 tx=0x7ff084007ec0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 msgr2=0x7ff0880b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 0x7ff0880b5250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff080007f20 tx=0x7ff080009860 comp rx=0 tx=0).stop 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 shutdown_connections 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff06c066f30 0x7ff06c0693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff08815db10 0x7ff0880b5250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff088159890 0x7ff088140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 --2- 172.21.15.92:0/162231435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff088159c60 0x7ff0880b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 >> 172.21.15.92:0/162231435 conn(0x7ff088068e20 msgr2=0x7ff088157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 shutdown_connections 2023-10-06T03:01:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.082+0000 7ff08fbed700 1 -- 172.21.15.92:0/162231435 wait complete. 2023-10-06T03:01:47.095 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":90454},{"type":"v1","addr":"172.21.15.92:6833","nonce":90454}]},"active_addr":"172.21.15.92:6833/90454","active_change":"2023-10-06T03:01:23.174447+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":965524267}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1095017404}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3384388277}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1755639507}]}]} 2023-10-06T03:01:47.100 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T03:01:47.100 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T03:01:47.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:47.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.213+0000 7fb63b59e700 1 Processor -- start 2023-10-06T03:01:47.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.213+0000 7fb63b59e700 1 -- start start 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c1597f0 0x7fb63c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb63c15d230 0x7fb63c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63b59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c15dbd0 con 0x7fb63c159420 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0b11d0 con 0x7fb63c15d230 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0b14b0 con 0x7fb63c1597f0 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb63c15d230 0x7fb63c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb63c15d230 0x7fb63c15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44324/0 (socket says 172.21.15.92:44324) 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 -- 172.21.15.92:0/1054835521 learned_addr learned my addr 172.21.15.92:0/1054835521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb639d9b700 1 --2- 172.21.15.92:0/1054835521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c1597f0 0x7fb63c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 -- 172.21.15.92:0/1054835521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c1597f0 msgr2=0x7fb63c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 --2- 172.21.15.92:0/1054835521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c1597f0 0x7fb63c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 -- 172.21.15.92:0/1054835521 >> v1:172.21.15.136:6789/0 conn(0x7fb63c159420 legacy=0x7fb63c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 -- 172.21.15.92:0/1054835521 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0b1790 con 0x7fb63c15d230 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63ad9d700 1 --2- 172.21.15.92:0/1054835521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c15d230 0x7fb63c15d690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0099b0 tx=0x7fb62c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d09eb5ed876ae7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.214+0000 7fb63959a700 1 -- 172.21.15.92:0/1054835521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00b860 con 0x7fb63c15d230 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63959a700 1 -- 172.21.15.92:0/1054835521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c00b9c0 con 0x7fb63c15d230 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63959a700 1 -- 172.21.15.92:0/1054835521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c00bd60 con 0x7fb63c15d230 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 -- 172.21.15.92:0/1054835521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c15d230 msgr2=0x7fb63c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 --2- 172.21.15.92:0/1054835521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c15d230 0x7fb63c15d690 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0099b0 tx=0x7fb62c009250 comp rx=0 tx=0).stop 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 -- 172.21.15.92:0/1054835521 shutdown_connections 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 --2- 172.21.15.92:0/1054835521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c15d230 0x7fb63c15d690 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 --2- 172.21.15.92:0/1054835521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c1597f0 0x7fb63c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 -- 172.21.15.92:0/1054835521 >> 172.21.15.92:0/1054835521 conn(0x7fb63c0d0660 msgr2=0x7fb63c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 -- 172.21.15.92:0/1054835521 shutdown_connections 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.215+0000 7fb63b59e700 1 -- 172.21.15.92:0/1054835521 wait complete. 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.216+0000 7fb63b59e700 1 Processor -- start 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.216+0000 7fb63b59e700 1 -- start start 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb63c159420 0x7fb63c16ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 0x7fb63c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c15d230 0x7fb63c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c16a6d0 con 0x7fb63c1597f0 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c157be0 con 0x7fb63c159420 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c157ec0 con 0x7fb63c15d230 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 0x7fb63c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 0x7fb63c157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44332/0 (socket says 172.21.15.92:44332) 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 -- 172.21.15.92:0/538573418 learned_addr learned my addr 172.21.15.92:0/538573418 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb63ad9d700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c15d230 0x7fb63c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 -- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c15d230 msgr2=0x7fb63c154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c15d230 0x7fb63c154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 -- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb63c159420 msgr2=0x7fb63c16ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb63c159420 0x7fb63c16ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 -- 172.21.15.92:0/538573418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0094f0 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.217+0000 7fb639d9b700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 0x7fb63c157330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c880 tx=0x7fb624008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.218+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011600 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.218+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6240117a0 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.218+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c16a080 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.218+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011b60 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.218+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c157870 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.218+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c16a520 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.219+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb624018360 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.219+0000 7fb6327fc700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb61c066f30 0x7fb61c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.219+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb624084510 con 0x7fb63c1597f0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.219+0000 7fb63a59c700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb61c066f30 0x7fb61c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.220+0000 7fb63a59c700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb61c066f30 0x7fb61c0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb63c155110 tx=0x7fb634007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.223+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb624051580 con 0x7fb63c1597f0 2023-10-06T03:01:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.409+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63c04c3c0 con 0x7fb63c1597f0 2023-10-06T03:01:47.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.410+0000 7fb6327fc700 1 -- 172.21.15.92:0/538573418 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fb624054ba0 con 0x7fb63c1597f0 2023-10-06T03:01:47.412 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:47.412 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb61c066f30 msgr2=0x7fb61c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb61c066f30 0x7fb61c0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb63c155110 tx=0x7fb634007ec0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 msgr2=0x7fb63c157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 0x7fb63c157330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c880 tx=0x7fb624008040 comp rx=0 tx=0).stop 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 shutdown_connections 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb61c066f30 0x7fb61c0693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb63c15d230 0x7fb63c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb63c159420 0x7fb63c16ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 --2- 172.21.15.92:0/538573418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb63c1597f0 0x7fb63c157330 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 >> 172.21.15.92:0/538573418 conn(0x7fb63c0d0660 msgr2=0x7fb63c155180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 shutdown_connections 2023-10-06T03:01:47.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.412+0000 7fb63b59e700 1 -- 172.21.15.92:0/538573418 wait complete. 2023-10-06T03:01:47.426 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T03:01:47.426 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:47.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.539+0000 7fda5a125700 1 Processor -- start 2023-10-06T03:01:47.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.539+0000 7fda5a125700 1 -- start start 2023-10-06T03:01:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda5a125700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fda540b2c80 0x7fda540cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda5a125700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540b1500 con 0x7fda54140cd0 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda5a125700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540cda20 con 0x7fda5406c0f0 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda5a125700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540cdd00 con 0x7fda540b2c80 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda59123700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fda54140cd0 0x7fda5406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34324/0 (socket says 172.21.15.92:34324) 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda59123700 1 -- 172.21.15.92:0/3376429082 learned_addr learned my addr 172.21.15.92:0/3376429082 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914887474 0 0) 0x7fda540b1500 con 0x7fda54140cd0 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda59924700 1 --2- 172.21.15.92:0/3376429082 >> v2:172.21.15.136:3300/0 conn(0x7fda540b2c80 0x7fda540cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda40003570 con 0x7fda54140cd0 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282655154 0 0) 0x7fda40003570 con 0x7fda54140cd0 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 >> v2:172.21.15.136:3300/0 conn(0x7fda540b2c80 msgr2=0x7fda540cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 --2- 172.21.15.92:0/3376429082 >> v2:172.21.15.136:3300/0 conn(0x7fda540b2c80 0x7fda540cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 >> v1:172.21.15.136:6789/0 conn(0x7fda5406c0f0 legacy=0x7fda540ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540cdfe0 con 0x7fda54140cd0 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fda50001470 con 0x7fda54140cd0 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 >> v1:172.21.15.92:6789/0 conn(0x7fda54140cd0 legacy=0x7fda5406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda40003a60 0x7fda40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40004810 0x7fda40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda400052e0 0x7fda4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda540cdd00 con 0x7fda40003a60 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4000c230 con 0x7fda40004810 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4000c3f0 con 0x7fda400052e0 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.540+0000 7fda59924700 1 --2- 172.21.15.92:0/3376429082 >> v2:172.21.15.136:3300/0 conn(0x7fda540b2c80 0x7fda540cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda40003a60 0x7fda40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda58922700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40004810 0x7fda40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59924700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda400052e0 0x7fda4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 -- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda400052e0 msgr2=0x7fda4000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda400052e0 0x7fda4000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 -- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40004810 msgr2=0x7fda40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40004810 0x7fda40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 -- 172.21.15.92:0/3376429082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda40003570 con 0x7fda40003a60 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59924700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda400052e0 0x7fda4000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda58922700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40004810 0x7fda40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59924700 1 -- 172.21.15.92:0/3376429082 reap_dead start 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda59123700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda40003a60 0x7fda40003e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda5000c3c0 tx=0x7fda50001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3526fd1750c8e91d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000cc90 con 0x7fda40003a60 2023-10-06T03:01:47.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5000cdf0 con 0x7fda40003a60 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.541+0000 7fda4bfff700 1 -- 172.21.15.92:0/3376429082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50005b70 con 0x7fda40003a60 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 -- 172.21.15.92:0/3376429082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda40003a60 msgr2=0x7fda40003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda40003a60 0x7fda40003e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda5000c3c0 tx=0x7fda50001e50 comp rx=0 tx=0).stop 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 -- 172.21.15.92:0/3376429082 shutdown_connections 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 --2- 172.21.15.92:0/3376429082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda40003a60 0x7fda40003e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 -- 172.21.15.92:0/3376429082 >> 172.21.15.92:0/3376429082 conn(0x7fda540ce430 msgr2=0x7fda540ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 -- 172.21.15.92:0/3376429082 shutdown_connections 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.542+0000 7fda5a125700 1 -- 172.21.15.92:0/3376429082 wait complete. 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.543+0000 7fda5a125700 1 Processor -- start 2023-10-06T03:01:47.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.543+0000 7fda5a125700 1 -- start start 2023-10-06T03:01:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40003a60 0x7fda5415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 0x7fda541387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda54138d00 0x7fda54158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54139180 con 0x7fda5415ad60 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54158570 con 0x7fda40003a60 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda54157c40 con 0x7fda54138d00 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 0x7fda541387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 0x7fda541387c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44346/0 (socket says 172.21.15.92:44346) 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 -- 172.21.15.92:0/1803152537 learned_addr learned my addr 172.21.15.92:0/1803152537 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda59924700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda54138d00 0x7fda54158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 -- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda54138d00 msgr2=0x7fda54158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda59123700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40003a60 0x7fda5415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda54138d00 0x7fda54158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 -- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40003a60 msgr2=0x7fda5415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40003a60 0x7fda5415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 -- 172.21.15.92:0/1803152537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54157f20 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda59924700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda54138d00 0x7fda54158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda59123700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40003a60 0x7fda5415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda58922700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 0x7fda541387c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e590 tx=0x7fda44009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44010520 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44010680 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44010a00 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54158200 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.544+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54158f40 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.545+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5404c3c0 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.546+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda44014480 con 0x7fda5415ad60 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.546+0000 7fda497fa700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fda38066f30 0x7fda380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.546+0000 7fda59123700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fda38066f30 0x7fda380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.546+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fda44084080 con 0x7fda5415ad60 2023-10-06T03:01:47.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.546+0000 7fda59123700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fda38066f30 0x7fda380693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda50001e90 tx=0x7fda50015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.550+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda4404d5a0 con 0x7fda5415ad60 2023-10-06T03:01:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.734+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda541586b0 con 0x7fda5415ad60 2023-10-06T03:01:47.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.735+0000 7fda497fa700 1 -- 172.21.15.92:0/1803152537 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fda44019090 con 0x7fda5415ad60 2023-10-06T03:01:47.737 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:47.737 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fda38066f30 msgr2=0x7fda380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fda38066f30 0x7fda380693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda50001e90 tx=0x7fda50015000 comp rx=0 tx=0).stop 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 msgr2=0x7fda541387c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 0x7fda541387c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda4400e590 tx=0x7fda44009040 comp rx=0 tx=0).stop 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 shutdown_connections 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fda38066f30 0x7fda380693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda54138d00 0x7fda54158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda40003a60 0x7fda5415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 --2- 172.21.15.92:0/1803152537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda5415ad60 0x7fda541387c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 >> 172.21.15.92:0/1803152537 conn(0x7fda540ce430 msgr2=0x7fda54156dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 shutdown_connections 2023-10-06T03:01:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.737+0000 7fda5a125700 1 -- 172.21.15.92:0/1803152537 wait complete. 2023-10-06T03:01:47.751 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T03:01:47.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T03:01:47.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T03:01:47.752 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T03:01:47.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T03:01:47.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T03:01:47.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T03:01:47.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T03:01:47.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.945+0000 7f43ec8f1700 1 Processor -- start 2023-10-06T03:01:47.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.945+0000 7f43ec8f1700 1 -- start start 2023-10-06T03:01:47.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43ec8f1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e80cba20 0x7f43e80ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43ec8f1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43e8156760 0x7f43e8068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43ec8f1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e8069450 con 0x7f43e8068aa0 2023-10-06T03:01:47.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43ec8f1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80aceb0 con 0x7f43e8156760 2023-10-06T03:01:47.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43ec8f1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80b2490 con 0x7f43e80cba20 2023-10-06T03:01:47.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43e6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43e8156760 0x7f43e8068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43e6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43e8156760 0x7f43e8068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44356/0 (socket says 172.21.15.92:44356) 2023-10-06T03:01:47.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43e6d9d700 1 -- 172.21.15.92:0/145570790 learned_addr learned my addr 172.21.15.92:0/145570790 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.947+0000 7f43e659c700 1 --2- 172.21.15.92:0/145570790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e80cba20 0x7f43e80ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e6d9d700 1 -- 172.21.15.92:0/145570790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e80cba20 msgr2=0x7f43e80ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e6d9d700 1 --2- 172.21.15.92:0/145570790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e80cba20 0x7f43e80ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e6d9d700 1 -- 172.21.15.92:0/145570790 >> v1:172.21.15.92:6789/0 conn(0x7f43e8068aa0 legacy=0x7f43e8156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e6d9d700 1 -- 172.21.15.92:0/145570790 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80ac4e0 con 0x7f43e8156760 2023-10-06T03:01:47.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e6d9d700 1 --2- 172.21.15.92:0/145570790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8156760 0x7f43e8068f10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43d80089b0 tx=0x7f43d80082d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce09edeb23a16ac3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e559a700 1 -- 172.21.15.92:0/145570790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8010070 con 0x7f43e8156760 2023-10-06T03:01:47.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7f3570451700 1 Processor -- start 2023-10-06T03:01:47.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e559a700 1 -- 172.21.15.92:0/145570790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d800d980 con 0x7f43e8156760 2023-10-06T03:01:47.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.948+0000 7f43e559a700 1 -- 172.21.15.92:0/145570790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d800dd00 con 0x7f43e8156760 2023-10-06T03:01:47.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7f43ec8f1700 1 -- 172.21.15.92:0/145570790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8156760 msgr2=0x7f43e8068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7f43ec8f1700 1 --2- 172.21.15.92:0/145570790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8156760 0x7f43e8068f10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43d80089b0 tx=0x7f43d80082d0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7fd9c18de700 1 Processor -- start 2023-10-06T03:01:47.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7fd9c18de700 1 -- start start 2023-10-06T03:01:47.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f43ec8f1700 1 -- 172.21.15.92:0/145570790 shutdown_connections 2023-10-06T03:01:47.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f43ec8f1700 1 --2- 172.21.15.92:0/145570790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e80cba20 0x7f43e80ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f43ec8f1700 1 --2- 172.21.15.92:0/145570790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8156760 0x7f43e8068f10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f43ec8f1700 1 -- 172.21.15.92:0/145570790 >> 172.21.15.92:0/145570790 conn(0x7f43e80cd000 msgr2=0x7f43e8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f46ce19c700 1 Processor -- start 2023-10-06T03:01:47.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f46ce19c700 1 -- start start 2023-10-06T03:01:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f43ec8f1700 1 -- 172.21.15.92:0/145570790 shutdown_connections 2023-10-06T03:01:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7fd9c18de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc166960 0x7fd9bc0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7fd9c18de700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9bc0abe00 0x7fd9bc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7fd9c18de700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc0ad400 con 0x7fd9bc068aa0 2023-10-06T03:01:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7fd9c18de700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc087e70 con 0x7fd9bc0abe00 2023-10-06T03:01:47.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7fd9c18de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0ac610 con 0x7fd9bc166960 2023-10-06T03:01:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f43ec8f1700 1 -- 172.21.15.92:0/145570790 wait complete. 2023-10-06T03:01:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.950+0000 7f61f1790700 1 Processor -- start 2023-10-06T03:01:47.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9bc0abe00 0x7fd9bc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9bc0abe00 0x7fd9bc0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44364/0 (socket says 172.21.15.92:44364) 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 -- 172.21.15.92:0/11652872 learned_addr learned my addr 172.21.15.92:0/11652872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 -- 172.21.15.92:0/11652872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc166960 msgr2=0x7fd9bc0ab8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 --2- 172.21.15.92:0/11652872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc166960 0x7fd9bc0ab8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 -- 172.21.15.92:0/11652872 >> v1:172.21.15.136:6789/0 conn(0x7fd9bc068aa0 legacy=0x7fd9bc068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 -- 172.21.15.92:0/11652872 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0ac8f0 con 0x7fd9bc0abe00 2023-10-06T03:01:47.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46ce19c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f46c81575f0 0x7f46c80abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46ce19c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46c81579c0 0x7f46c80cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46ce19c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80acfe0 con 0x7f46c8068aa0 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46ce19c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80cbcb0 con 0x7f46c81579c0 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46ce19c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c80ac3d0 con 0x7f46c81575f0 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46c6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46c81579c0 0x7f46c80cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46c6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f46c81579c0 0x7f46c80cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44380/0 (socket says 172.21.15.92:44380) 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46c6ffd700 1 -- 172.21.15.92:0/3312717835 learned_addr learned my addr 172.21.15.92:0/3312717835 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46c77fe700 1 --2- 172.21.15.92:0/3312717835 >> v2:172.21.15.136:3300/0 conn(0x7f46c81575f0 0x7f46c80abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7fd9bb7fe700 1 --2- 172.21.15.92:0/11652872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc0abe00 0x7fd9bc0ad890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009f50 tx=0x7fd9b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7584b026e11994 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46c6ffd700 1 -- 172.21.15.92:0/3312717835 >> v2:172.21.15.136:3300/0 conn(0x7f46c81575f0 msgr2=0x7f46c80abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f46c6ffd700 1 --2- 172.21.15.92:0/3312717835 >> v2:172.21.15.136:3300/0 conn(0x7f46c81575f0 0x7f46c80abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f46c6ffd700 1 -- 172.21.15.92:0/3312717835 >> v1:172.21.15.136:6789/0 conn(0x7f46c8068aa0 legacy=0x7f46c80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f46c6ffd700 1 -- 172.21.15.92:0/3312717835 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c80ac6b0 con 0x7f46c81579c0 2023-10-06T03:01:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f46c6ffd700 1 --2- 172.21.15.92:0/3312717835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 0x7f46c80cb770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46c8063d40 tx=0x7f46b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f16ed98385082f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f43ec8f1700 1 Processor -- start 2023-10-06T03:01:47.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f43ec8f1700 1 -- start start 2023-10-06T03:01:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f61f1790700 1 -- start start 2023-10-06T03:01:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f3570451700 1 -- start start 2023-10-06T03:01:47.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43ec8f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8068aa0 0x7f43e8142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43ec8f1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e80cba20 0x7f43e80aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.952+0000 7f3570451700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 0x7f3568068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f3570451700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3568068fe0 0x7f35680b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f3570451700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35680b1fe0 0x7f35681619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f3570451700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680aef20 con 0x7f35680b1fe0 2023-10-06T03:01:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f3570451700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356815f260 con 0x7f3568158440 2023-10-06T03:01:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f3570451700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3568069430 con 0x7f3568068fe0 2023-10-06T03:01:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f356e1ed700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 0x7f3568068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f356e1ed700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 0x7f3568068aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50608/0 (socket says 172.21.15.92:50608) 2023-10-06T03:01:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f356e1ed700 1 -- 172.21.15.92:0/640579639 learned_addr learned my addr 172.21.15.92:0/640579639 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43ec8f1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e8156760 0x7f43e80aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43ec8f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81422b0 con 0x7f43e8068aa0 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43ec8f1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8142c80 con 0x7f43e80cba20 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43ec8f1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81416a0 con 0x7f43e8156760 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8068aa0 0x7f43e8142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8068aa0 0x7f43e8142740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44382/0 (socket says 172.21.15.92:44382) 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 -- 172.21.15.92:0/381104710 learned_addr learned my addr 172.21.15.92:0/381104710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e6d9d700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e8156760 0x7f43e80aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7f2a24370700 1 Processor -- start 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.949+0000 7f2a24370700 1 -- start start 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a24370700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2a1c0875d0 0x7f2a1c0879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a24370700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2a1c0b1c50 0x7f2a1c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a24370700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c068aa0 0x7f2a1c068f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a24370700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1c069480 con 0x7f2a1c0875d0 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a24370700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1c0acdf0 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a24370700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0ac1e0 con 0x7f2a1c068aa0 2023-10-06T03:01:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a2190b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2a1c0b1c50 0x7f2a1c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a2190b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2a1c0b1c50 0x7f2a1c0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50600/0 (socket says 172.21.15.92:50600) 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.951+0000 7f2a2190b700 1 -- 172.21.15.92:0/2930758624 learned_addr learned my addr 172.21.15.92:0/2930758624 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2290d700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c068aa0 0x7f2a1c068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61f1790700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 0x7f61ec1683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61f1790700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec06afc0 con 0x7f61ec0adee0 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61f1790700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec06a3b0 con 0x7f61ec06db00 2023-10-06T03:01:47.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61f1790700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec06a1c0 con 0x7f61ec0b1690 2023-10-06T03:01:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 -- 172.21.15.92:0/381104710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e8156760 msgr2=0x7f43e80aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e8156760 0x7f43e80aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 -- 172.21.15.92:0/381104710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e80cba20 msgr2=0x7f43e80aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43e80cba20 0x7f43e80aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e659c700 1 -- 172.21.15.92:0/381104710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8008570 con 0x7f43e8068aa0 2023-10-06T03:01:47.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f43e6d9d700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43e8156760 0x7f43e80aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61eb7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 0x7f61ec1683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61eb7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 0x7f61ec1683f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59754/0 (socket says 172.21.15.92:59754) 2023-10-06T03:01:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f61eb7fe700 1 -- 172.21.15.92:0/954408770 learned_addr learned my addr 172.21.15.92:0/954408770 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f356d9ec700 1 --2- 172.21.15.92:0/640579639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3568068fe0 0x7f35680b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2190b700 1 -- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c068aa0 msgr2=0x7f2a1c068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2190b700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c068aa0 0x7f2a1c068f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2190b700 1 -- 172.21.15.92:0/2930758624 >> v2:172.21.15.92:3300/0 conn(0x7f2a1c0875d0 msgr2=0x7f2a1c0879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2190b700 1 --2- 172.21.15.92:0/2930758624 >> v2:172.21.15.92:3300/0 conn(0x7f2a1c0875d0 0x7f2a1c0879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2190b700 1 -- 172.21.15.92:0/2930758624 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c0ac4c0 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f2a2290d700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c068aa0 0x7f2a1c068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.953+0000 7f356e9ee700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.92:3300/0 conn(0x7f35680b1fe0 0x7f35681619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f43e659c700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8068aa0 0x7f43e8142740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00ec80 tx=0x7f43dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356d9ec700 1 -- 172.21.15.92:0/640579639 >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 msgr2=0x7f3568068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356d9ec700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 0x7f3568068aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356d9ec700 1 -- 172.21.15.92:0/640579639 >> v2:172.21.15.92:3300/0 conn(0x7f35680b1fe0 msgr2=0x7f35681619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356d9ec700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.92:3300/0 conn(0x7f35680b1fe0 0x7f35681619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f2a2190b700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c0ab8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0acd60 tx=0x7f2a0c00a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5caa22cb58fd96f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356d9ec700 1 -- 172.21.15.92:0/640579639 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680b2440 con 0x7f3568068fe0 2023-10-06T03:01:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356e1ed700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 0x7f3568068aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356e9ee700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.92:3300/0 conn(0x7f35680b1fe0 0x7f35681619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f46c67fc700 1 -- 172.21.15.92:0/3312717835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8009e70 con 0x7f46c81579c0 2023-10-06T03:01:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f46c67fc700 1 -- 172.21.15.92:0/3312717835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b800b860 con 0x7f46c81579c0 2023-10-06T03:01:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f46c67fc700 1 -- 172.21.15.92:0/3312717835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800bc00 con 0x7f46c81579c0 2023-10-06T03:01:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f2a2110a700 1 -- 172.21.15.92:0/2930758624 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00b750 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f2a2110a700 1 -- 172.21.15.92:0/2930758624 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c00b8b0 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f2a2110a700 1 -- 172.21.15.92:0/2930758624 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c00bc50 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f356d9ec700 1 --2- 172.21.15.92:0/640579639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3568068fe0 0x7f35680b1aa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f35680ad380 tx=0x7f35580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c945bbf5dd5a960e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f46ce19c700 1 -- 172.21.15.92:0/3312717835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 msgr2=0x7f46c80cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f46ce19c700 1 --2- 172.21.15.92:0/3312717835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 0x7f46c80cb770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f46c8063d40 tx=0x7f46b8009250 comp rx=0 tx=0).stop 2023-10-06T03:01:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f46ce19c700 1 -- 172.21.15.92:0/3312717835 shutdown_connections 2023-10-06T03:01:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f46ce19c700 1 --2- 172.21.15.92:0/3312717835 >> v2:172.21.15.136:3300/0 conn(0x7f46c81575f0 0x7f46c80abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f46ce19c700 1 --2- 172.21.15.92:0/3312717835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 0x7f46c80cb770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f46ce19c700 1 -- 172.21.15.92:0/3312717835 >> 172.21.15.92:0/3312717835 conn(0x7f46c80cd000 msgr2=0x7f46c8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00ca60 con 0x7f43e8068aa0 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc00cbc0 con 0x7f43e8068aa0 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc013470 con 0x7f43e8068aa0 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e8141980 con 0x7f43e8068aa0 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e8141f40 con 0x7f43e8068aa0 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7fd9b9ffb700 1 -- 172.21.15.92:0/11652872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400e070 con 0x7fd9bc0abe00 2023-10-06T03:01:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7fd9b9ffb700 1 -- 172.21.15.92:0/11652872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b400ba70 con 0x7fd9bc0abe00 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7fd9b9ffb700 1 -- 172.21.15.92:0/11652872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40092e0 con 0x7fd9bc0abe00 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7fd9c18de700 1 -- 172.21.15.92:0/11652872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc0abe00 msgr2=0x7fd9bc0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7fd9c18de700 1 --2- 172.21.15.92:0/11652872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc0abe00 0x7fd9bc0ad890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009f50 tx=0x7fd9b4009840 comp rx=0 tx=0).stop 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7fd9c18de700 1 -- 172.21.15.92:0/11652872 shutdown_connections 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7fd9c18de700 1 --2- 172.21.15.92:0/11652872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc0abe00 0x7fd9bc0ad890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7fd9c18de700 1 --2- 172.21.15.92:0/11652872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc166960 0x7fd9bc0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7fd9c18de700 1 -- 172.21.15.92:0/11652872 >> 172.21.15.92:0/11652872 conn(0x7fd9bc0cd000 msgr2=0x7fd9bc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f61eb7fe700 1 -- 172.21.15.92:0/954408770 >> v1:172.21.15.136:6789/0 conn(0x7f61ec06db00 legacy=0x7f61ec069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f61eb7fe700 1 -- 172.21.15.92:0/954408770 >> v1:172.21.15.92:6789/0 conn(0x7f61ec0adee0 legacy=0x7f61ec0ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f61eb7fe700 1 -- 172.21.15.92:0/954408770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec06a690 con 0x7f61ec0b1690 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.954+0000 7f61eb7fe700 1 --2- 172.21.15.92:0/954408770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 0x7f61ec1683f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009f50 tx=0x7f61e4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f1e435cdb503a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f61ea7fc700 1 -- 172.21.15.92:0/954408770 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400e070 con 0x7f61ec0b1690 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f61ea7fc700 1 -- 172.21.15.92:0/954408770 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e400ba70 con 0x7f61ec0b1690 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f61ea7fc700 1 -- 172.21.15.92:0/954408770 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40092e0 con 0x7f61ec0b1690 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f2a24370700 1 -- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c0b1c50 msgr2=0x7f2a1c0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.955+0000 7f2a24370700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c0ab8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0acd60 tx=0x7f2a0c00a330 comp rx=0 tx=0).stop 2023-10-06T03:01:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f356d1eb700 1 -- 172.21.15.92:0/640579639 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558011630 con 0x7f3568068fe0 2023-10-06T03:01:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f356d1eb700 1 -- 172.21.15.92:0/640579639 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558011790 con 0x7f3568068fe0 2023-10-06T03:01:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f356d1eb700 1 -- 172.21.15.92:0/640579639 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558011b30 con 0x7f3568068fe0 2023-10-06T03:01:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 -- 172.21.15.92:0/954408770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 msgr2=0x7f61ec1683f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 --2- 172.21.15.92:0/954408770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 0x7f61ec1683f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009f50 tx=0x7f61e4009840 comp rx=0 tx=0).stop 2023-10-06T03:01:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 -- 172.21.15.92:0/954408770 shutdown_connections 2023-10-06T03:01:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 --2- 172.21.15.92:0/954408770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0b1690 0x7f61ec1683f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 -- 172.21.15.92:0/954408770 >> 172.21.15.92:0/954408770 conn(0x7f61ec068390 msgr2=0x7f61ec068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43e804c3c0 con 0x7f43e8068aa0 2023-10-06T03:01:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 -- 172.21.15.92:0/954408770 shutdown_connections 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 -- 172.21.15.92:0/2930758624 shutdown_connections 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c068aa0 0x7f2a1c068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 --2- 172.21.15.92:0/2930758624 >> v2:172.21.15.92:3300/0 conn(0x7f2a1c0875d0 0x7f2a1c0879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 --2- 172.21.15.92:0/2930758624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c0ab8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 -- 172.21.15.92:0/2930758624 >> 172.21.15.92:0/2930758624 conn(0x7f2a1c0cd000 msgr2=0x7f2a1c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 -- 172.21.15.92:0/2930758624 shutdown_connections 2023-10-06T03:01:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f2a24370700 1 -- 172.21.15.92:0/2930758624 wait complete. 2023-10-06T03:01:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7fd9c18de700 1 -- 172.21.15.92:0/11652872 shutdown_connections 2023-10-06T03:01:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f61f1790700 1 -- 172.21.15.92:0/954408770 wait complete. 2023-10-06T03:01:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f46ce19c700 1 -- 172.21.15.92:0/3312717835 shutdown_connections 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.956+0000 7f3570451700 1 -- 172.21.15.92:0/640579639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3568068fe0 msgr2=0x7f35680b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 --2- 172.21.15.92:0/640579639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3568068fe0 0x7f35680b1aa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f35680ad380 tx=0x7f35580091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 -- 172.21.15.92:0/640579639 shutdown_connections 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.92:3300/0 conn(0x7f35680b1fe0 0x7f35681619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 --2- 172.21.15.92:0/640579639 >> v2:172.21.15.136:3300/0 conn(0x7f3568158440 0x7f3568068aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 --2- 172.21.15.92:0/640579639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3568068fe0 0x7f35680b1aa0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 -- 172.21.15.92:0/640579639 >> 172.21.15.92:0/640579639 conn(0x7f35680cd000 msgr2=0x7f3568068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:47.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7fe24516e700 1 Processor -- start 2023-10-06T03:01:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 -- 172.21.15.92:0/640579639 shutdown_connections 2023-10-06T03:01:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f3570451700 1 -- 172.21.15.92:0/640579639 wait complete. 2023-10-06T03:01:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43dc013820 con 0x7f43e8068aa0 2023-10-06T03:01:47.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43d67fc700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43c8067000 0x7f43c80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f2a24370700 1 Processor -- start 2023-10-06T03:01:47.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43dc0849f0 con 0x7f43e8068aa0 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43e5d9b700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43c8067000 0x7f43c80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43d67fc700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f43c806f8b0 0x7f43c8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 --> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43c8072370 con 0x7f43c806f8b0 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43dc089240 con 0x7f43e8068aa0 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43e6d9d700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f43c806f8b0 0x7f43c8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f3570451700 1 Processor -- start 2023-10-06T03:01:47.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fd9c18de700 1 -- 172.21.15.92:0/11652872 wait complete. 2023-10-06T03:01:47.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.957+0000 7fe24516e700 1 -- start start 2023-10-06T03:01:47.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fe24516e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe24006b830 0x7fe2400c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fe24516e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400c74b0 con 0x7fe2400cb440 2023-10-06T03:01:47.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fe24516e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24006bc00 con 0x7fe240162250 2023-10-06T03:01:47.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fe24516e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe240158a30 con 0x7fe24006b830 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fe23e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe2400cb440 0x7fe240161b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34346/0 (socket says 172.21.15.92:34346) 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7fe23e59c700 1 -- 172.21.15.92:0/858758201 learned_addr learned my addr 172.21.15.92:0/858758201 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f2a24370700 1 -- start start 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a24370700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c068aa0 0x7f2a1c086880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a24370700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c0875d0 0x7f2a1c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a24370700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a24370700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c167720 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a24370700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c168780 con 0x7f2a1c068aa0 2023-10-06T03:01:47.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a24370700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1679c0 con 0x7f2a1c0875d0 2023-10-06T03:01:47.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7fe23ed9d700 1 --2- 172.21.15.92:0/858758201 >> v2:172.21.15.136:3300/0 conn(0x7fe24006b830 0x7fe2400c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f3570451700 1 -- start start 2023-10-06T03:01:47.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f3570451700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3568068fe0 0x7f35681407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f3570451700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35680b1fe0 0x7f35681410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f3570451700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3568158440 0x7f35681418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:47.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f3570451700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568140310 con 0x7f3568158440 2023-10-06T03:01:47.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a2290d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a2290d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c15fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44402/0 (socket says 172.21.15.92:44402) 2023-10-06T03:01:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a2290d700 1 -- 172.21.15.92:0/2354879622 learned_addr learned my addr 172.21.15.92:0/2354879622 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f3570451700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3568138e20 con 0x7f3568068fe0 2023-10-06T03:01:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f3570451700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3568140ce0 con 0x7f35680b1fe0 2023-10-06T03:01:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f356e9ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3568158440 0x7f35681418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f356e9ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3568158440 0x7f35681418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44412/0 (socket says 172.21.15.92:44412) 2023-10-06T03:01:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f356e9ee700 1 -- 172.21.15.92:0/2124508525 learned_addr learned my addr 172.21.15.92:0/2124508525 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f2a2210c700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c068aa0 0x7f2a1c086880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43e5d9b700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43c8067000 0x7f43c80694b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43e80af880 tx=0x7f43e0007e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f43e6d9d700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f43c806f8b0 0x7f43c8071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== osd.1 v2:172.21.15.92:6800/90685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f43c8072370 con 0x7f43c806f8b0 2023-10-06T03:01:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3568068fe0 0x7f35681407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356d9ec700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35680b1fe0 0x7f35681410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308009754 0 0) 0x7fe24006bc00 con 0x7fe240162250 2023-10-06T03:01:47.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe228003570 con 0x7fe240162250 2023-10-06T03:01:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a2290d700 1 -- 172.21.15.92:0/2354879622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c0875d0 msgr2=0x7f2a1c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a2290d700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a1c0875d0 0x7f2a1c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a2290d700 1 -- 172.21.15.92:0/2354879622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c068aa0 msgr2=0x7f2a1c086880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a2290d700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a1c068aa0 0x7f2a1c086880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a2290d700 1 -- 172.21.15.92:0/2354879622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a140077d0 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137860265 0 0) 0x7fe2400c74b0 con 0x7fe2400cb440 2023-10-06T03:01:47.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe24006bc00 con 0x7fe2400cb440 2023-10-06T03:01:47.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 -- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35680b1fe0 msgr2=0x7f35681410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35680b1fe0 0x7f35681410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 -- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3568158440 msgr2=0x7f35681418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3568158440 0x7f35681418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 -- 172.21.15.92:0/2124508525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640094f0 con 0x7f3568068fe0 2023-10-06T03:01:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23ed9d700 1 -- 172.21.15.92:0/858758201 >> v1:172.21.15.136:6789/0 conn(0x7fe240162250 legacy=0x7fe2400685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23ed9d700 1 -- 172.21.15.92:0/858758201 >> v1:172.21.15.92:6789/0 conn(0x7fe2400cb440 legacy=0x7fe240161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1883951912 0 0) 0x7fe24006bc00 con 0x7fe2400cb440 2023-10-06T03:01:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23ed9d700 1 -- 172.21.15.92:0/858758201 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400c95e0 con 0x7fe24006b830 2023-10-06T03:01:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a2290d700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c15fa70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400f360 tx=0x7f2a14006500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f356e1ed700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3568068fe0 0x7f35681407a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3564006fd0 tx=0x7f3564009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14012450 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7fe23ed9d700 1 --2- 172.21.15.92:0/858758201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe24006b830 0x7fe2400c8f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2340098e0 tx=0x7fe2340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcddf268ab37a2de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:47.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.958+0000 7f61f1790700 1 Processor -- start 2023-10-06T03:01:47.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7fd9c18de700 1 Processor -- start 2023-10-06T03:01:47.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.960+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c00a590 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c167f80 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.959+0000 7f46ce19c700 1 -- 172.21.15.92:0/3312717835 wait complete. 2023-10-06T03:01:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f46ce19c700 1 Processor -- start 2023-10-06T03:01:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a140125b0 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14012970 con 0x7f2a1c0b1c50 2023-10-06T03:01:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7fd9c18de700 1 -- start start 2023-10-06T03:01:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400c6a0 con 0x7f3568068fe0 2023-10-06T03:01:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356400c800 con 0x7f3568068fe0 2023-10-06T03:01:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400cb80 con 0x7f3568068fe0 2023-10-06T03:01:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230001470 con 0x7fe24006b830 2023-10-06T03:01:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234011420 con 0x7fe24006b830 2023-10-06T03:01:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7fe23dd9b700 1 -- 172.21.15.92:0/858758201 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340117c0 con 0x7fe24006b830 2023-10-06T03:01:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3558009420 con 0x7f3568068fe0 2023-10-06T03:01:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356813fcc0 con 0x7f3568068fe0 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9c18de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9bc068aa0 0x7fd9bc0a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9c18de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc0abe00 0x7fd9bc1642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9c18de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc166960 0x7fd9bc0a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9c18de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0a7d00 con 0x7fd9bc166960 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9c18de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1674c0 con 0x7fd9bc068aa0 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9c18de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1677a0 con 0x7fd9bc0abe00 2023-10-06T03:01:48.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9baffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9bc068aa0 0x7fd9bc0a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9baffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9bc068aa0 0x7fd9bc0a77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50658/0 (socket says 172.21.15.92:50658) 2023-10-06T03:01:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7fd9baffd700 1 -- 172.21.15.92:0/744000359 learned_addr learned my addr 172.21.15.92:0/744000359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a04000fd0 con 0x7f2a1c0b1c50 2023-10-06T03:01:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f356813fb20 con 0x7f3568068fe0 2023-10-06T03:01:48.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a1401a3f0 con 0x7f2a1c0b1c50 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 -- 172.21.15.92:0/858758201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe24006b830 msgr2=0x7fe2400c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 --2- 172.21.15.92:0/858758201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe24006b830 0x7fe2400c8f50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2340098e0 tx=0x7fe2340091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.961+0000 7f61f1790700 1 -- start start 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f61f1790700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61ec06db00 0x7f61ec163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f61f1790700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0adee0 0x7f61ec0c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f61f1790700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61ec0b1690 0x7f61ec135020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f61f1790700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec160390 con 0x7f61ec0b1690 2023-10-06T03:01:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f61f1790700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0c25f0 con 0x7f61ec06db00 2023-10-06T03:01:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f61f1790700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0c28c0 con 0x7f61ec0adee0 2023-10-06T03:01:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 -- 172.21.15.92:0/858758201 shutdown_connections 2023-10-06T03:01:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 --2- 172.21.15.92:0/858758201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe24006b830 0x7fe2400c8f50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 -- 172.21.15.92:0/858758201 >> 172.21.15.92:0/858758201 conn(0x7fe2400c9870 msgr2=0x7fe2400c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.962+0000 7f46ce19c700 1 -- start start 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46ce19c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46c8068aa0 0x7f46c81465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46ce19c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46c81575f0 0x7f46c80af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46ce19c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 0x7f46c8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46ce19c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8144820 con 0x7f46c81579c0 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46ce19c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8158f80 con 0x7f46c8068aa0 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46ce19c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8146b30 con 0x7f46c81575f0 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 -- 172.21.15.92:0/858758201 shutdown_connections 2023-10-06T03:01:48.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7fe24516e700 1 -- 172.21.15.92:0/858758201 wait complete. 2023-10-06T03:01:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46c77fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46c8068aa0 0x7f46c81465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46c77fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46c8068aa0 0x7f46c81465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50684/0 (socket says 172.21.15.92:50684) 2023-10-06T03:01:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f46c77fe700 1 -- 172.21.15.92:0/4190384528 learned_addr learned my addr 172.21.15.92:0/4190384528 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f46c77fe700 1 -- 172.21.15.92:0/4190384528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46c81575f0 msgr2=0x7f46c80af2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f46c77fe700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46c81575f0 0x7f46c80af2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f46c77fe700 1 -- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 msgr2=0x7f46c8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f46c77fe700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46c81579c0 0x7f46c8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f46c77fe700 1 -- 172.21.15.92:0/4190384528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b80094f0 con 0x7f46c8068aa0 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f2a127fc700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2a00066f30 0x7f2a000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2a14088170 con 0x7f2a1c0b1c50 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f2a127fc700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f2a0006f760 0x7f2a00071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 --> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a00072220 con 0x7f2a0006f760 2023-10-06T03:01:48.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1408a7b0 con 0x7f2a1c0b1c50 2023-10-06T03:01:48.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f46c77fe700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46c8068aa0 0x7f46c81465f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ecb0 tx=0x7f46bc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3564013470 con 0x7f3568068fe0 2023-10-06T03:01:48.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f355e7fc700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f354c066f30 0x7f354c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f2a2190b700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f2a0006f760 0x7f2a00071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f2a2210c700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2a00066f30 0x7f2a000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3564084af0 con 0x7f3568068fe0 2023-10-06T03:01:48.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f355e7fc700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7f354c06f7e0 0x7f354c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 --> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f354c0722a0 con 0x7f354c06f7e0 2023-10-06T03:01:48.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3564084e90 con 0x7f3568068fe0 2023-10-06T03:01:48.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f2a2210c700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2a00066f30 0x7f2a000693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a180099b0 tx=0x7f2a18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f356d9ec700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f354c066f30 0x7f354c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00c970 con 0x7f46c8068aa0 2023-10-06T03:01:48.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc00cad0 con 0x7f46c8068aa0 2023-10-06T03:01:48.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00ce70 con 0x7f46c8068aa0 2023-10-06T03:01:48.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c813b7f0 con 0x7f46c8068aa0 2023-10-06T03:01:48.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c8159230 con 0x7f46c8068aa0 2023-10-06T03:01:48.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7fd9bb7fe700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc166960 0x7fd9bc0a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7fe24516e700 1 Processor -- start 2023-10-06T03:01:48.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.965+0000 7fe24516e700 1 -- start start 2023-10-06T03:01:48.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7f356e9ee700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7f354c06f7e0 0x7f354c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7f356d9ec700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f354c066f30 0x7f354c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35580093f0 tx=0x7f3558017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9baffd700 1 -- 172.21.15.92:0/744000359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc0abe00 msgr2=0x7fd9bc1642f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:01:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9baffd700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9bc0abe00 0x7fd9bc1642f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9baffd700 1 -- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc166960 msgr2=0x7fd9bc0a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9baffd700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9bc166960 0x7fd9bc0a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9baffd700 1 -- 172.21.15.92:0/744000359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4009ae0 con 0x7fd9bc068aa0 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61ec06db00 0x7f61ec163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61ec06db00 0x7f61ec163050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50668/0 (socket says 172.21.15.92:50668) 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 -- 172.21.15.92:0/775601235 learned_addr learned my addr 172.21.15.92:0/775601235 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 -- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0adee0 msgr2=0x7f61ec0c20b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61ec0adee0 0x7f61ec0c20b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 -- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61ec0b1690 msgr2=0x7f61ec135020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f61ec0b1690 0x7f61ec135020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.963+0000 7f61eaffd700 1 -- 172.21.15.92:0/775601235 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4009ae0 con 0x7f61ec06db00 2023-10-06T03:01:48.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f61eaffd700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61ec06db00 0x7f61ec163050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b600 tx=0x7f61d40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d400f510 con 0x7f61ec06db00 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d400f6b0 con 0x7f61ec06db00 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d400fa70 con 0x7f61ec06db00 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec0c2ba0 con 0x7f61ec06db00 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.964+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec1608c0 con 0x7f61ec06db00 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9baffd700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9bc068aa0 0x7fd9bc0a77c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c4e0 tx=0x7fd9b000c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7f356e9ee700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7f354c06f7e0 0x7f354c071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0011590 con 0x7fd9bc068aa0 2023-10-06T03:01:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0011710 con 0x7fd9bc068aa0 2023-10-06T03:01:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0011ad0 con 0x7fd9bc068aa0 2023-10-06T03:01:48.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc167a80 con 0x7fd9bc068aa0 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc164830 con 0x7fd9bc068aa0 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe24516e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24006b830 0x7fe24015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe24516e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2400cb440 0x7fe24016e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe24516e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe240162250 0x7fe24016e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe24516e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe240158eb0 con 0x7fe240162250 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe24516e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24016ee90 con 0x7fe2400cb440 2023-10-06T03:01:48.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe24516e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24016f170 con 0x7fe24006b830 2023-10-06T03:01:48.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe23ed9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24006b830 0x7fe24015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe23ed9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24006b830 0x7fe24015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59794/0 (socket says 172.21.15.92:59794) 2023-10-06T03:01:48.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7fe23ed9d700 1 -- 172.21.15.92:0/3383913669 learned_addr learned my addr 172.21.15.92:0/3383913669 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe240162250 0x7fe24016e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== osd.4 v2:172.21.15.136:6816/90892 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f354c0722a0 con 0x7f354c06f7e0 2023-10-06T03:01:48.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f2a2190b700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f2a0006f760 0x7f2a00071b70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 -- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24006b830 msgr2=0x7fe24015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe24006b830 0x7fe24015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 -- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2400cb440 msgr2=0x7fe24016e2d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:48.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2400cb440 0x7fe24016e2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 -- 172.21.15.92:0/3383913669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009420 con 0x7fe240162250 2023-10-06T03:01:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.966+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61d401a400 con 0x7f61ec06db00 2023-10-06T03:01:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f61e37fe700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f61cc066fb0 0x7f61cc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61d4084210 con 0x7f61ec06db00 2023-10-06T03:01:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9bc167630 con 0x7fd9bc068aa0 2023-10-06T03:01:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7fe23f59e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe240162250 0x7fe24016e810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c810 tx=0x7fe238002650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== osd.2 v2:172.21.15.92:6812/90689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2a00072220 con 0x7f2a0006f760 2023-10-06T03:01:48.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012630 con 0x7fe240162250 2023-10-06T03:01:48.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2380127b0 con 0x7fe240162250 2023-10-06T03:01:48.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24016f4a0 con 0x7fe240162250 2023-10-06T03:01:48.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238012b70 con 0x7fe240162250 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46bc013680 con 0x7f46c8068aa0 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f46c804c3c0 con 0x7f46c8068aa0 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.968+0000 7f46af7fe700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46b4067000 0x7f46b40694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f46bc0849a0 con 0x7f46c8068aa0 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7f46c6ffd700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46b4067000 0x7f46b40694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7f46af7fe700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7f46b406f8b0 0x7f46b4071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 --> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46b4072370 con 0x7f46b406f8b0 2023-10-06T03:01:48.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24016fa60 con 0x7fe240162250 2023-10-06T03:01:48.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46bc089240 con 0x7f46c8068aa0 2023-10-06T03:01:48.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7f46c6ffd700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46b4067000 0x7f46b40694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46c8063d40 tx=0x7f46b8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.969+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe23801a3a0 con 0x7fe240162250 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe226ffd700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe22c066f30 0x7fe22c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe23ed9d700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe22c066f30 0x7fe22c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe24004c3c0 con 0x7fe240162250 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe238055530 con 0x7fe240162250 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe23ed9d700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe22c066f30 0x7fe22c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2340093c0 tx=0x7fe234016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe226ffd700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7fe22c06f760 0x7fe22c071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 --> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe22c072220 con 0x7fe22c06f760 2023-10-06T03:01:48.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe238051170 con 0x7fe240162250 2023-10-06T03:01:48.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.971+0000 7f46c7fff700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7f46b406f8b0 0x7f46b4071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.971+0000 7fe23e59c700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7fe22c06f760 0x7fe22c071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.971+0000 7f46c7fff700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7f46b406f8b0 0x7f46b4071cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.971+0000 7fe23e59c700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7fe22c06f760 0x7fe22c071b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.971+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9b0001020 con 0x7fd9bc068aa0 2023-10-06T03:01:48.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.971+0000 7f36da58b700 1 Processor -- start 2023-10-06T03:01:48.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7f36da58b700 1 -- start start 2023-10-06T03:01:48.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== osd.0 v2:172.21.15.92:6808/90688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f46b4072370 con 0x7f46b406f8b0 2023-10-06T03:01:48.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== osd.6 v2:172.21.15.136:6808/90875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe22c072220 con 0x7fe22c06f760 2023-10-06T03:01:48.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7f36da58b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d41575f0 0x7f36d40abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7f36da58b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36d4068aa0 0x7f36d4068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7f36da58b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d40acfe0 con 0x7f36d41579c0 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36da58b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d4069420 con 0x7f36d4068aa0 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36da58b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d40ac3d0 con 0x7f36d41575f0 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36d4068aa0 0x7f36d4068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36d4068aa0 0x7f36d4068ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44458/0 (socket says 172.21.15.92:44458) 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 -- 172.21.15.92:0/2398843784 learned_addr learned my addr 172.21.15.92:0/2398843784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fd9a2ffd700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9a8066f30 0x7fd9a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd9b0083b50 con 0x7fd9bc068aa0 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fd9a2ffd700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7fd9a806f760 0x7fd9a8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 --> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9a8072220 con 0x7fd9a806f760 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fd9ba7fc700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9a8066f30 0x7fd9a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.972+0000 7fd9bb7fe700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7fd9a806f760 0x7fd9a8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7fd9bb7fe700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7fd9a806f760 0x7fd9a8071b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7fd9ba7fc700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9a8066f30 0x7fd9a80693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0cc4b0 tx=0x7fd9ac006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 -- 172.21.15.92:0/2398843784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d41575f0 msgr2=0x7f36d40abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 --2- 172.21.15.92:0/2398843784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d41575f0 0x7f36d40abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 -- 172.21.15.92:0/2398843784 >> v1:172.21.15.136:6789/0 conn(0x7f36d41579c0 legacy=0x7f36d40cb770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 -- 172.21.15.92:0/2398843784 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d40ac6b0 con 0x7f36d4068aa0 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d8b28700 1 --2- 172.21.15.92:0/2398843784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4068aa0 0x7f36d4068ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009f20 tx=0x7f36cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ca9d6fbb25a0a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d2ffd700 1 -- 172.21.15.92:0/2398843784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc00e070 con 0x7f36d4068aa0 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.973+0000 7f36d2ffd700 1 -- 172.21.15.92:0/2398843784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc00ba40 con 0x7f36d4068aa0 2023-10-06T03:01:48.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.974+0000 7f36d2ffd700 1 -- 172.21.15.92:0/2398843784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0092e0 con 0x7f36d4068aa0 2023-10-06T03:01:48.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.974+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== osd.3 v2:172.21.15.92:6810/90687 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd9a8072220 con 0x7fd9a806f760 2023-10-06T03:01:48.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.974+0000 7f36da58b700 1 -- 172.21.15.92:0/2398843784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4068aa0 msgr2=0x7f36d4068ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.974+0000 7f36da58b700 1 --2- 172.21.15.92:0/2398843784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4068aa0 0x7f36d4068ee0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009f20 tx=0x7f36cc009840 comp rx=0 tx=0).stop 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f61f1790700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f61d00015f0 0x7f61d0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 --> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61d0006b90 con 0x7f61d00015f0 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.967+0000 7f61e27fc700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f61cc066fb0 0x7f61cc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.970+0000 7f61e27fc700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f61cc066fb0 0x7f61cc069460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61dc001050 tx=0x7f61dc007ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.982+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b0050bc0 con 0x7fd9bc068aa0 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.978+0000 7f61eb7fe700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f61d00015f0 0x7f61d0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.981+0000 7f61eb7fe700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f61d00015f0 0x7f61d0003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.983+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== osd.7 v2:172.21.15.136:6824/90905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f61d0006b90 con 0x7f61d00015f0 2023-10-06T03:01:48.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.988+0000 7f36da58b700 1 -- 172.21.15.92:0/2398843784 shutdown_connections 2023-10-06T03:01:48.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.988+0000 7f36da58b700 1 --2- 172.21.15.92:0/2398843784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4068aa0 0x7f36d4068ee0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.988+0000 7f36da58b700 1 --2- 172.21.15.92:0/2398843784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d41575f0 0x7f36d40abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.988+0000 7f36da58b700 1 -- 172.21.15.92:0/2398843784 >> 172.21.15.92:0/2398843784 conn(0x7f36d40cd000 msgr2=0x7f36d4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.988+0000 7f36da58b700 1 -- 172.21.15.92:0/2398843784 shutdown_connections 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.988+0000 7f36da58b700 1 -- 172.21.15.92:0/2398843784 wait complete. 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.990+0000 7f36da58b700 1 Processor -- start 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.990+0000 7f36da58b700 1 -- start start 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.991+0000 7f36da58b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d4068aa0 0x7f36d4149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.991+0000 7f36da58b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d41575f0 0x7f36d4151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.991+0000 7f36da58b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36d41579c0 0x7f36d4158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.991+0000 7f36da58b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41519d0 con 0x7f36d41575f0 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.991+0000 7f36da58b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4150dc0 con 0x7f36d41579c0 2023-10-06T03:01:48.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.991+0000 7f36da58b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41510a0 con 0x7f36d4068aa0 2023-10-06T03:01:48.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.992+0000 7f36d37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d41575f0 0x7f36d4151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.992+0000 7f36d37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d41575f0 0x7f36d4151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44462/0 (socket says 172.21.15.92:44462) 2023-10-06T03:01:48.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.992+0000 7f36d37fe700 1 -- 172.21.15.92:0/2181060496 learned_addr learned my addr 172.21.15.92:0/2181060496 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.992+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 --> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f43e80678d0 con 0x7f43c806f8b0 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.992+0000 7f36d8b28700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36d41579c0 0x7f36d4158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.992+0000 7f36d3fff700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d4068aa0 0x7f36d4149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36d8b28700 1 -- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d4068aa0 msgr2=0x7f36d4149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36d8b28700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36d4068aa0 0x7f36d4149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36d8b28700 1 -- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d41575f0 msgr2=0x7f36d4151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36d8b28700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d41575f0 0x7f36d4151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36d8b28700 1 -- 172.21.15.92:0/2181060496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc009ae0 con 0x7f36d41579c0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36d8b28700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36d41579c0 0x7f36d4158bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36cc014000 tx=0x7f36cc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0092e0 con 0x7f36d41579c0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc012d00 con 0x7f36d41579c0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d4151380 con 0x7f36d41579c0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d4150a50 con 0x7f36d41579c0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.993+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc01e660 con 0x7f36d41579c0 2023-10-06T03:01:48.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.994+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 --> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f46c813ac40 con 0x7f46b406f8b0 2023-10-06T03:01:48.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.994+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 --> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe24006f270 con 0x7fe22c06f760 2023-10-06T03:01:48.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.994+0000 7fe226ffd700 1 -- 172.21.15.92:0/3383913669 <== osd.6 v2:172.21.15.136:6808/90875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe24006f270 con 0x7fe22c06f760 2023-10-06T03:01:48.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.994+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f36b0000fd0 con 0x7f36d41579c0 2023-10-06T03:01:48.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.994+0000 7f46af7fe700 1 -- 172.21.15.92:0/4190384528 <== osd.0 v2:172.21.15.92:6808/90688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f46c0001920 con 0x7f46b406f8b0 2023-10-06T03:01:48.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.995+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7f46b406f8b0 msgr2=0x7f46b4071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.995+0000 7f46ce19c700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7f46b406f8b0 0x7f46b4071cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.995+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36cc01ea50 con 0x7f36d41579c0 2023-10-06T03:01:48.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43d67fc700 1 -- 172.21.15.92:0/381104710 <== osd.1 v2:172.21.15.92:6800/90685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f43e80678d0 con 0x7f43c806f8b0 2023-10-06T03:01:48.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36c3fff700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36b8066f00 0x7f36b80693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f36cc0860b0 con 0x7f36d41579c0 2023-10-06T03:01:48.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46b4067000 msgr2=0x7f46b40694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f46ce19c700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46b4067000 0x7f46b40694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f46c8063d40 tx=0x7f46b8017000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46c8068aa0 msgr2=0x7f46c81465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f46ce19c700 1 --2- 172.21.15.92:0/4190384528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46c8068aa0 0x7f46c81465f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ecb0 tx=0x7f46bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f46c7fff700 1 -- 172.21.15.92:0/4190384528 reap_dead start 2023-10-06T03:01:48.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36d3fff700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36b8066f00 0x7f36b80693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36c3fff700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f36b806f7b0 0x7f36b8071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 --> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f36b8072270 con 0x7f36b806f7b0 2023-10-06T03:01:48.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36cc0886f0 con 0x7f36d41579c0 2023-10-06T03:01:48.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 --> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2a04002d30 con 0x7f2a0006f760 2023-10-06T03:01:48.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f43c806f8b0 msgr2=0x7f43c8071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43ec8f1700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f43c806f8b0 0x7f43c8071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 --> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f356804c3c0 con 0x7f354c06f7e0 2023-10-06T03:01:48.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36d37fe700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f36b806f7b0 0x7f36b8071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43c8067000 msgr2=0x7f43c80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43ec8f1700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43c8067000 0x7f43c80694b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43e80af880 tx=0x7f43e0007e20 comp rx=0 tx=0).stop 2023-10-06T03:01:48.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8068aa0 msgr2=0x7f43e8142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43ec8f1700 1 --2- 172.21.15.92:0/381104710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43e8068aa0 0x7f43e8142740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00ec80 tx=0x7f43dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f43e6d9d700 1 -- 172.21.15.92:0/381104710 reap_dead start 2023-10-06T03:01:48.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.996+0000 7f36d3fff700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36b8066f00 0x7f36b80693b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009bf0 tx=0x7f36c40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 shutdown_connections 2023-10-06T03:01:48.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 >> 172.21.15.92:0/4190384528 conn(0x7f46c80cd000 msgr2=0x7f46c8146e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 shutdown_connections 2023-10-06T03:01:48.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f355e7fc700 1 -- 172.21.15.92:0/2124508525 <== osd.4 v2:172.21.15.136:6816/90892 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f356804c3c0 con 0x7f354c06f7e0 2023-10-06T03:01:48.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f46ce19c700 1 -- 172.21.15.92:0/4190384528 wait complete. 2023-10-06T03:01:48.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f36d37fe700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f36b806f7b0 0x7f36b8071bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7fe22c06f760 msgr2=0x7fe22c071b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7fe22c06f760 0x7fe22c071b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7f354c06f7e0 msgr2=0x7f354c071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7f354c06f7e0 0x7f354c071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe22c066f30 msgr2=0x7fe22c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe22c066f30 0x7fe22c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2340093c0 tx=0x7fe234016000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f354c066f30 msgr2=0x7f354c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe240162250 msgr2=0x7fe24016e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 --2- 172.21.15.92:0/3383913669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe240162250 0x7fe24016e810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c810 tx=0x7fe238002650 comp rx=0 tx=0).stop 2023-10-06T03:01:48.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f354c066f30 0x7f354c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35580093f0 tx=0x7f3558017000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3568068fe0 msgr2=0x7f35681407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 --2- 172.21.15.92:0/2124508525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3568068fe0 0x7f35681407a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3564006fd0 tx=0x7f3564009250 comp rx=0 tx=0).stop 2023-10-06T03:01:48.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe23f59e700 1 -- 172.21.15.92:0/3383913669 reap_dead start 2023-10-06T03:01:48.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f356e9ee700 1 -- 172.21.15.92:0/2124508525 reap_dead start 2023-10-06T03:01:48.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 shutdown_connections 2023-10-06T03:01:48.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 >> 172.21.15.92:0/2124508525 conn(0x7f35680cd000 msgr2=0x7f3568068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 shutdown_connections 2023-10-06T03:01:48.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 >> 172.21.15.92:0/3383913669 conn(0x7fe2400c9870 msgr2=0x7fe240068d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 shutdown_connections 2023-10-06T03:01:48.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7fe24516e700 1 -- 172.21.15.92:0/3383913669 wait complete. 2023-10-06T03:01:48.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 shutdown_connections 2023-10-06T03:01:48.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f3570451700 1 -- 172.21.15.92:0/2124508525 wait complete. 2023-10-06T03:01:48.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.997+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== osd.5 v2:172.21.15.136:6800/90874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f36b8072270 con 0x7f36b806f7b0 2023-10-06T03:01:48.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.999+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 shutdown_connections 2023-10-06T03:01:48.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.999+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 >> 172.21.15.92:0/381104710 conn(0x7f43e80cd000 msgr2=0x7f43e80afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.999+0000 7f2a127fc700 1 -- 172.21.15.92:0/2354879622 <== osd.2 v2:172.21.15.92:6812/90689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2a04002d30 con 0x7f2a0006f760 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.999+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f2a0006f760 msgr2=0x7f2a00071b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:47.999+0000 7f2a24370700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f2a0006f760 0x7f2a00071b70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2a00066f30 msgr2=0x7f2a000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2a00066f30 0x7f2a000693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a180099b0 tx=0x7f2a18008040 comp rx=0 tx=0).stop 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a1c0b1c50 msgr2=0x7f2a1c15fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 --2- 172.21.15.92:0/2354879622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2a1c0b1c50 0x7f2a1c15fa70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400f360 tx=0x7f2a14006500 comp rx=0 tx=0).stop 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a2290d700 1 -- 172.21.15.92:0/2354879622 reap_dead start 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 shutdown_connections 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 >> 172.21.15.92:0/2354879622 conn(0x7f2a1c0cd000 msgr2=0x7f2a1c0ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 shutdown_connections 2023-10-06T03:01:48.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.000+0000 7f2a24370700 1 -- 172.21.15.92:0/2354879622 wait complete. 2023-10-06T03:01:48.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.003+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 shutdown_connections 2023-10-06T03:01:48.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.003+0000 7f43ec8f1700 1 -- 172.21.15.92:0/381104710 wait complete. 2023-10-06T03:01:48.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.006+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 --> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9bc04c3c0 con 0x7fd9a806f760 2023-10-06T03:01:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.006+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 --> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61d0005c20 con 0x7f61d00015f0 2023-10-06T03:01:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.007+0000 7fd9a2ffd700 1 -- 172.21.15.92:0/744000359 <== osd.3 v2:172.21.15.92:6810/90687 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd9bc04c3c0 con 0x7fd9a806f760 2023-10-06T03:01:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.007+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7fd9a806f760 msgr2=0x7fd9a8071b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.007+0000 7fd9c18de700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7fd9a806f760 0x7fd9a8071b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.057 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:48.057 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T03:01:48.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.011+0000 7f61e37fe700 1 -- 172.21.15.92:0/775601235 <== osd.7 v2:172.21.15.136:6824/90905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61d0005c20 con 0x7f61d00015f0 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.011+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f61d00015f0 msgr2=0x7f61d0003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.011+0000 7f61f1790700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f61d00015f0 0x7f61d0003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f61cc066fb0 msgr2=0x7f61cc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61f1790700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f61cc066fb0 0x7f61cc069460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f61dc001050 tx=0x7f61dc007ac0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61ec06db00 msgr2=0x7f61ec163050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61f1790700 1 --2- 172.21.15.92:0/775601235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61ec06db00 0x7f61ec163050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b600 tx=0x7f61d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61eb7fe700 1 -- 172.21.15.92:0/775601235 reap_dead start 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 shutdown_connections 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.020+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 >> 172.21.15.92:0/775601235 conn(0x7f61ec068390 msgr2=0x7f61ec0c9140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9a8066f30 msgr2=0x7fd9a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9c18de700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9a8066f30 0x7fd9a80693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0cc4b0 tx=0x7fd9ac006c60 comp rx=0 tx=0).stop 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9bc068aa0 msgr2=0x7fd9bc0a77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9c18de700 1 --2- 172.21.15.92:0/744000359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9bc068aa0 0x7fd9bc0a77c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c4e0 tx=0x7fd9b000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9bb7fe700 1 -- 172.21.15.92:0/744000359 reap_dead start 2023-10-06T03:01:48.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 shutdown_connections 2023-10-06T03:01:48.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.017+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 >> 172.21.15.92:0/744000359 conn(0x7fd9bc0cd000 msgr2=0x7fd9bc0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.018+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 shutdown_connections 2023-10-06T03:01:48.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.018+0000 7fd9c18de700 1 -- 172.21.15.92:0/744000359 wait complete. 2023-10-06T03:01:48.063 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:48.063 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T03:01:48.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.023+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 shutdown_connections 2023-10-06T03:01:48.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.023+0000 7f61f1790700 1 -- 172.21.15.92:0/775601235 wait complete. 2023-10-06T03:01:48.064 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:48.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T03:01:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.025+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 --> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36b0002d30 con 0x7f36b806f7b0 2023-10-06T03:01:48.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.025+0000 7f36c3fff700 1 -- 172.21.15.92:0/2181060496 <== osd.5 v2:172.21.15.136:6800/90874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f36b0002d30 con 0x7f36b806f7b0 2023-10-06T03:01:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f36b806f7b0 msgr2=0x7f36b8071bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f36b806f7b0 0x7f36b8071bc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36b8066f00 msgr2=0x7f36b80693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36b8066f00 0x7f36b80693b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009bf0 tx=0x7f36c40091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36d41579c0 msgr2=0x7f36d4158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 --2- 172.21.15.92:0/2181060496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36d41579c0 0x7f36d4158bb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36cc014000 tx=0x7f36cc017000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36d8b28700 1 -- 172.21.15.92:0/2181060496 reap_dead start 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 shutdown_connections 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 >> 172.21.15.92:0/2181060496 conn(0x7f36d40cd000 msgr2=0x7f36d4153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 shutdown_connections 2023-10-06T03:01:48.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.026+0000 7f36da58b700 1 -- 172.21.15.92:0/2181060496 wait complete. 2023-10-06T03:01:48.068 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:48.068 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T03:01:48.069 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T03:01:48.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T03:01:48.070 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-10-06T03:01:48.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T03:01:48.071 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-10-06T03:01:48.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T03:01:48.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.293+0000 7f17bc97b700 1 Processor -- start 2023-10-06T03:01:48.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.294+0000 7f17bc97b700 1 -- start start 2023-10-06T03:01:48.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.294+0000 7f866cd2d700 1 Processor -- start 2023-10-06T03:01:48.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.294+0000 7f866cd2d700 1 -- start start 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17bc97b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b80cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17bc97b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17b806ec80 0x7f17b80ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17bc97b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b80adcf0 con 0x7f17b806bbe0 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17bc97b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b806cbf0 con 0x7f17b806ec80 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17bc97b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80aceb0 con 0x7f17b8156950 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17b806ec80 0x7f17b80ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f17b806ec80 0x7f17b80ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44464/0 (socket says 172.21.15.92:44464) 2023-10-06T03:01:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 -- 172.21.15.92:0/2862682257 learned_addr learned my addr 172.21.15.92:0/2862682257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866cd2d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 0x7f866806a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866cd2d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f86680c83c0 0x7f86680cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866cd2d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866806bc10 con 0x7f866806ab70 2023-10-06T03:01:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b6d9d700 1 --2- 172.21.15.92:0/2862682257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b80cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866cd2d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866806b000 con 0x7f86680c83c0 2023-10-06T03:01:48.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866cd2d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866806b2e0 con 0x7f866815de50 2023-10-06T03:01:48.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 0x7f866806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 0x7f866806a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59830/0 (socket says 172.21.15.92:59830) 2023-10-06T03:01:48.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f866659c700 1 -- 172.21.15.92:0/2113569236 learned_addr learned my addr 172.21.15.92:0/2113569236 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 -- 172.21.15.92:0/2862682257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 msgr2=0x7f17b80cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 --2- 172.21.15.92:0/2862682257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b80cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 -- 172.21.15.92:0/2862682257 >> v1:172.21.15.92:6789/0 conn(0x7f17b806bbe0 legacy=0x7f17b806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b5d9b700 1 -- 172.21.15.92:0/2862682257 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763747486 0 0) 0x7f17b80adcf0 con 0x7f17b806bbe0 2023-10-06T03:01:48.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f17b759e700 1 -- 172.21.15.92:0/2862682257 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b80adfb0 con 0x7f17b806ec80 2023-10-06T03:01:48.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f8666d9d700 1 --2- 172.21.15.92:0/2113569236 >> v2:172.21.15.136:3300/0 conn(0x7f86680c83c0 0x7f86680cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f17b759e700 1 --2- 172.21.15.92:0/2862682257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806ec80 0x7f17b80ae440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17b000d1d0 tx=0x7f17b000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6215548c446fc152 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866659c700 1 -- 172.21.15.92:0/2113569236 >> v2:172.21.15.136:3300/0 conn(0x7f86680c83c0 msgr2=0x7f86680cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866659c700 1 --2- 172.21.15.92:0/2113569236 >> v2:172.21.15.136:3300/0 conn(0x7f86680c83c0 0x7f86680cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866659c700 1 -- 172.21.15.92:0/2113569236 >> v1:172.21.15.136:6789/0 conn(0x7f866806ab70 legacy=0x7f86680c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866659c700 1 -- 172.21.15.92:0/2113569236 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866806b5c0 con 0x7f866815de50 2023-10-06T03:01:48.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f17b5d9b700 1 -- 172.21.15.92:0/2862682257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0010070 con 0x7f17b806ec80 2023-10-06T03:01:48.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f17b5d9b700 1 -- 172.21.15.92:0/2862682257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000b0d0 con 0x7f17b806ec80 2023-10-06T03:01:48.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f17b5d9b700 1 -- 172.21.15.92:0/2862682257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000b490 con 0x7f17b806ec80 2023-10-06T03:01:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866659c700 1 --2- 172.21.15.92:0/2113569236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 0x7f866806a630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8658009990 tx=0x7f8658009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f681b5e221950851 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866559a700 1 -- 172.21.15.92:0/2113569236 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658009e80 con 0x7f866815de50 2023-10-06T03:01:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866559a700 1 -- 172.21.15.92:0/2113569236 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658005c10 con 0x7f866815de50 2023-10-06T03:01:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f866559a700 1 -- 172.21.15.92:0/2113569236 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865800b650 con 0x7f866815de50 2023-10-06T03:01:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f17bc97b700 1 -- 172.21.15.92:0/2862682257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806ec80 msgr2=0x7f17b80ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f17bc97b700 1 --2- 172.21.15.92:0/2862682257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806ec80 0x7f17b80ae440 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17b000d1d0 tx=0x7f17b000ba50 comp rx=0 tx=0).stop 2023-10-06T03:01:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f866cd2d700 1 -- 172.21.15.92:0/2113569236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 msgr2=0x7f866806a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2113569236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 0x7f866806a630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8658009990 tx=0x7f8658009220 comp rx=0 tx=0).stop 2023-10-06T03:01:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f17bc97b700 1 -- 172.21.15.92:0/2862682257 shutdown_connections 2023-10-06T03:01:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f17bc97b700 1 --2- 172.21.15.92:0/2862682257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b80cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f17bc97b700 1 --2- 172.21.15.92:0/2862682257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806ec80 0x7f17b80ae440 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f17bc97b700 1 -- 172.21.15.92:0/2862682257 >> 172.21.15.92:0/2862682257 conn(0x7f17b806b490 msgr2=0x7f17b80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f866cd2d700 1 -- 172.21.15.92:0/2113569236 shutdown_connections 2023-10-06T03:01:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2113569236 >> v2:172.21.15.136:3300/0 conn(0x7f86680c83c0 0x7f86680cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2113569236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866815de50 0x7f866806a630 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f866cd2d700 1 -- 172.21.15.92:0/2113569236 >> 172.21.15.92:0/2113569236 conn(0x7f86680cd670 msgr2=0x7f86680cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.295+0000 7f041f92c700 1 Processor -- start 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.296+0000 7f041f92c700 1 -- start start 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041f92c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04180875d0 0x7f04180879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041f92c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180b1c50 0x7f04180ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041f92c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04180acdf0 con 0x7f0418068aa0 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041f92c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04180ac1e0 con 0x7f04180875d0 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041f92c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04180ac4c0 con 0x7f04180b1c50 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041dec9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0418068aa0 0x7f041816a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34366/0 (socket says 172.21.15.92:34366) 2023-10-06T03:01:48.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041dec9700 1 -- 172.21.15.92:0/2735369056 learned_addr learned my addr 172.21.15.92:0/2735369056 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:48.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041d6c8700 1 --2- 172.21.15.92:0/2735369056 >> v2:172.21.15.92:3300/0 conn(0x7f04180875d0 0x7f04180879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f041cec7700 1 --2- 172.21.15.92:0/2735369056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180b1c50 0x7f04180ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f866cd2d700 1 -- 172.21.15.92:0/2113569236 shutdown_connections 2023-10-06T03:01:48.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.297+0000 7f040ffff700 1 -- 172.21.15.92:0/2735369056 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000856324 0 0) 0x7f04180acdf0 con 0x7f0418068aa0 2023-10-06T03:01:48.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f040ffff700 1 -- 172.21.15.92:0/2735369056 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0404003570 con 0x7f0418068aa0 2023-10-06T03:01:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f866cd2d700 1 -- 172.21.15.92:0/2113569236 wait complete. 2023-10-06T03:01:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f040ffff700 1 -- 172.21.15.92:0/2735369056 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518823623 0 0) 0x7f0404003570 con 0x7f0418068aa0 2023-10-06T03:01:48.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f041cec7700 1 -- 172.21.15.92:0/2735369056 >> v2:172.21.15.92:3300/0 conn(0x7f04180875d0 msgr2=0x7f04180879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f041cec7700 1 --2- 172.21.15.92:0/2735369056 >> v2:172.21.15.92:3300/0 conn(0x7f04180875d0 0x7f04180879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f041cec7700 1 -- 172.21.15.92:0/2735369056 >> v1:172.21.15.92:6789/0 conn(0x7f0418068aa0 legacy=0x7f041816a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f041cec7700 1 -- 172.21.15.92:0/2735369056 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04180ac7a0 con 0x7f04180b1c50 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f041d6c8700 1 --2- 172.21.15.92:0/2735369056 >> v2:172.21.15.92:3300/0 conn(0x7f04180875d0 0x7f04180879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f041cec7700 1 --2- 172.21.15.92:0/2735369056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180b1c50 0x7f04180ab8c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0414009e60 tx=0x7f0414009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf2dbc6daacc808 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7fbb113db700 1 Processor -- start 2023-10-06T03:01:48.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7fbb113db700 1 -- start start 2023-10-06T03:01:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f040ffff700 1 -- 172.21.15.92:0/2735369056 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100017f0 con 0x7f04180b1c50 2023-10-06T03:01:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f040ffff700 1 -- 172.21.15.92:0/2735369056 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0414010480 con 0x7f04180b1c50 2023-10-06T03:01:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7f040ffff700 1 -- 172.21.15.92:0/2735369056 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414010840 con 0x7f04180b1c50 2023-10-06T03:01:48.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb113db700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0691b0 0x7fbb0c0ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb113db700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0c0accf0 con 0x7fbb0c159fb0 2023-10-06T03:01:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb113db700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0c15a440 con 0x7fbb0c0b1f40 2023-10-06T03:01:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb113db700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c15a720 con 0x7fbb0c0691b0 2023-10-06T03:01:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7f866cd2d700 1 Processor -- start 2023-10-06T03:01:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7f041f92c700 1 -- 172.21.15.92:0/2735369056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180b1c50 msgr2=0x7f04180ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7f041f92c700 1 --2- 172.21.15.92:0/2735369056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180b1c50 0x7f04180ab8c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0414009e60 tx=0x7f0414009780 comp rx=0 tx=0).stop 2023-10-06T03:01:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7f866cd2d700 1 -- start start 2023-10-06T03:01:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb027fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbb0c159fb0 0x7fbb0c068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34378/0 (socket says 172.21.15.92:34378) 2023-10-06T03:01:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb027fc700 1 -- 172.21.15.92:0/3796625516 learned_addr learned my addr 172.21.15.92:0/3796625516 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:48.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fbb0affd700 1 --1- 172.21.15.92:0/3796625516 >> v1:172.21.15.136:6789/0 conn(0x7fbb0c0b1f40 0x7fbb0c0aabd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:34670/0 (socket says 172.21.15.92:34670) 2023-10-06T03:01:48.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7fe45b48d700 1 Processor -- start 2023-10-06T03:01:48.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.298+0000 7fe45b48d700 1 -- start start 2023-10-06T03:01:48.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe45b48d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe454161d90 0x7fe4540c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe45b48d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe454162160 0x7fe4540cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe45b48d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe45406b220 0x7fe45406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe45b48d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540c74b0 con 0x7fe45406b220 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe45b48d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540c89b0 con 0x7fe454162160 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe45b48d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540c80f0 con 0x7fe454161d90 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe459a2a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe45406b220 0x7fe45406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe459a2a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe45406b220 0x7fe45406b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44480/0 (socket says 172.21.15.92:44480) 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe459a2a700 1 -- 172.21.15.92:0/2914726647 learned_addr learned my addr 172.21.15.92:0/2914726647 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.299+0000 7fe459229700 1 --2- 172.21.15.92:0/2914726647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe454161d90 0x7fe4540c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe458a28700 1 --2- 172.21.15.92:0/2914726647 >> v2:172.21.15.136:3300/0 conn(0x7fe454162160 0x7fe4540cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe459229700 1 -- 172.21.15.92:0/2914726647 >> v2:172.21.15.136:3300/0 conn(0x7fe454162160 msgr2=0x7fe4540cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe459229700 1 --2- 172.21.15.92:0/2914726647 >> v2:172.21.15.136:3300/0 conn(0x7fe454162160 0x7fe4540cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe459229700 1 -- 172.21.15.92:0/2914726647 >> v2:172.21.15.92:3300/0 conn(0x7fe45406b220 msgr2=0x7fe45406b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe459229700 1 --2- 172.21.15.92:0/2914726647 >> v2:172.21.15.92:3300/0 conn(0x7fe45406b220 0x7fe45406b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe459229700 1 -- 172.21.15.92:0/2914726647 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45406bb80 con 0x7fe454161d90 2023-10-06T03:01:48.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7f17bc97b700 1 -- 172.21.15.92:0/2862682257 shutdown_connections 2023-10-06T03:01:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701258716 0 0) 0x7fbb0c0accf0 con 0x7fbb0c159fb0 2023-10-06T03:01:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaf8003570 con 0x7fbb0c159fb0 2023-10-06T03:01:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410970184 0 0) 0x7fbb0c15a440 con 0x7fbb0c0b1f40 2023-10-06T03:01:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb0c0accf0 con 0x7fbb0c0b1f40 2023-10-06T03:01:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7f17bc97b700 1 -- 172.21.15.92:0/2862682257 wait complete. 2023-10-06T03:01:48.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 518608317 0 0) 0x7fbaf8003570 con 0x7fbb0c159fb0 2023-10-06T03:01:48.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0691b0 msgr2=0x7fbb0c0ac7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:01:48.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0691b0 0x7fbb0c0ac7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 >> v1:172.21.15.136:6789/0 conn(0x7fbb0c0b1f40 legacy=0x7fbb0c0aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c15aa00 con 0x7fbb0c159fb0 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fbaf4001470 con 0x7fbb0c159fb0 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 >> v1:172.21.15.92:6789/0 conn(0x7fbb0c159fb0 legacy=0x7fbb0c068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf8004cb0 0x7fbaf8005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf80037b0 0x7fbaf8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbaf8004280 0x7fbaf800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc001570 con 0x7fbaf8004cb0 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c15a720 con 0x7fbaf80037b0 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf800c280 con 0x7fbaf8004280 2023-10-06T03:01:48.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fe459229700 1 --2- 172.21.15.92:0/2914726647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe454161d90 0x7fe4540c8f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe454064900 tx=0x7fe44c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=108f14d60105da7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb0affd700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf8004cb0 0x7fbaf8005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7f041f92c700 1 -- 172.21.15.92:0/2735369056 shutdown_connections 2023-10-06T03:01:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7f041f92c700 1 --2- 172.21.15.92:0/2735369056 >> v2:172.21.15.92:3300/0 conn(0x7f04180875d0 0x7f04180879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7f041f92c700 1 --2- 172.21.15.92:0/2735369056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180b1c50 0x7f04180ab8c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7f041f92c700 1 -- 172.21.15.92:0/2735369056 >> 172.21.15.92:0/2735369056 conn(0x7f04180cd000 msgr2=0x7f0418068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.300+0000 7fbb027fc700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf80037b0 0x7fbaf8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866cd2d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 0x7f8668177200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866cd2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86680c83c0 0x7f8668177c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866cd2d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f866815de50 0x7f8668157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866cd2d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8668177a60 con 0x7f86680c83c0 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866cd2d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86681557d0 con 0x7f866815de50 2023-10-06T03:01:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866cd2d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8668155ab0 con 0x7f866806ab70 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 0x7f8668177200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fbb0affd700 1 -- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbaf8004280 msgr2=0x7fbaf800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fbb0affd700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbaf8004280 0x7fbaf800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fbb0affd700 1 -- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf80037b0 msgr2=0x7fbaf8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fbb0affd700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf80037b0 0x7fbaf8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fbb0affd700 1 -- 172.21.15.92:0/3796625516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8003570 con 0x7fbaf8004cb0 2023-10-06T03:01:48.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f041f92c700 1 -- 172.21.15.92:0/2735369056 shutdown_connections 2023-10-06T03:01:48.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fe44bfff700 1 -- 172.21.15.92:0/2914726647 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00e070 con 0x7fe454161d90 2023-10-06T03:01:48.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fe44bfff700 1 -- 172.21.15.92:0/2914726647 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00bab0 con 0x7fe454161d90 2023-10-06T03:01:48.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 0x7f8668177200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59852/0 (socket says 172.21.15.92:59852) 2023-10-06T03:01:48.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 -- 172.21.15.92:0/2609644446 learned_addr learned my addr 172.21.15.92:0/2609644446 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fe44bfff700 1 -- 172.21.15.92:0/2914726647 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00be70 con 0x7fe454161d90 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f8665d9b700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86680c83c0 0x7f8668177c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 -- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f866815de50 msgr2=0x7f8668157a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f866815de50 0x7f8668157a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 -- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86680c83c0 msgr2=0x7f8668177c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86680c83c0 0x7f8668177c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f866659c700 1 -- 172.21.15.92:0/2609644446 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8658009480 con 0x7f866806ab70 2023-10-06T03:01:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f8665d9b700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86680c83c0 0x7f8668177c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7fbb0affd700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf8004cb0 0x7fbaf8005080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c5c0 tx=0x7fbafc001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85beed6c5cb139f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f866659c700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 0x7f8668177200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f865800b3c0 tx=0x7f8658016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 -- 172.21.15.92:0/2914726647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe454161d90 msgr2=0x7fe4540c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 --2- 172.21.15.92:0/2914726647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe454161d90 0x7fe4540c8f50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe454064900 tx=0x7fe44c009780 comp rx=0 tx=0).stop 2023-10-06T03:01:48.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00cd80 con 0x7fbaf8004cb0 2023-10-06T03:01:48.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbafc004bb0 con 0x7fbaf8004cb0 2023-10-06T03:01:48.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fbb0a7fc700 1 -- 172.21.15.92:0/3796625516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007150 con 0x7fbaf8004cb0 2023-10-06T03:01:48.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658011ae0 con 0x7f866806ab70 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f17bc97b700 1 Processor -- start 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 -- 172.21.15.92:0/2914726647 shutdown_connections 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 --2- 172.21.15.92:0/2914726647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe454161d90 0x7fe4540c8f50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 --2- 172.21.15.92:0/2914726647 >> v2:172.21.15.92:3300/0 conn(0x7fe45406b220 0x7fe45406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 --2- 172.21.15.92:0/2914726647 >> v2:172.21.15.136:3300/0 conn(0x7fe454162160 0x7fe4540cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7fe45b48d700 1 -- 172.21.15.92:0/2914726647 >> 172.21.15.92:0/2914726647 conn(0x7fe4540c9870 msgr2=0x7fe4540c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7fe45b48d700 1 -- 172.21.15.92:0/2914726647 shutdown_connections 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658020070 con 0x7f866806ab70 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658011e70 con 0x7f866806ab70 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8668155d90 con 0x7f866806ab70 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8668177740 con 0x7f866806ab70 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7fbb113db700 1 -- 172.21.15.92:0/3796625516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf8004cb0 msgr2=0x7fbaf8005080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7fbb113db700 1 --2- 172.21.15.92:0/3796625516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbaf8004cb0 0x7fbaf8005080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c5c0 tx=0x7fbafc001c90 comp rx=0 tx=0).stop 2023-10-06T03:01:48.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7fbb0b7fe700 1 -- 172.21.15.92:0/3796625516 reap_dead start 2023-10-06T03:01:48.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f17bc97b700 1 -- start start 2023-10-06T03:01:48.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.301+0000 7f041f92c700 1 -- 172.21.15.92:0/2735369056 wait complete. 2023-10-06T03:01:48.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f041f92c700 1 Processor -- start 2023-10-06T03:01:48.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f041f92c700 1 -- start start 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f43f5451700 1 Processor -- start 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.302+0000 7f43f5451700 1 -- start start 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f43f5451700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f43f00b1d30 0x7f43f015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f43f5451700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00691b0 0x7f43f01537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f43f5451700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f00ad530 con 0x7f43f015d600 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f43f5451700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f015b1d0 con 0x7f43f00b1d30 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f43f5451700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f015af60 con 0x7f43f00691b0 2023-10-06T03:01:48.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7f43ee7fc700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f43f015d600 0x7f43f0068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:34674/0 (socket says 172.21.15.92:34674) 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43ee7fc700 1 -- 172.21.15.92:0/2307487458 learned_addr learned my addr 172.21.15.92:0/2307487458 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/2307487458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00691b0 0x7f43f01537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43edffb700 1 -- 172.21.15.92:0/2307487458 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334685024 0 0) 0x7f43f00ad530 con 0x7f43f015d600 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43edffb700 1 -- 172.21.15.92:0/2307487458 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d8003570 con 0x7f43f015d600 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17bc97b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 0x7f17b80c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17bc97b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b806ec80 0x7f17b80c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17bc97b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b81765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17bc97b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80c6540 con 0x7f17b806bbe0 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17bc97b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80c6820 con 0x7f17b806ec80 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17bc97b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80c6b00 con 0x7f17b8156950 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f041f92c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0418068aa0 0x7f0418182540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f041f92c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 0x7f041817f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f041f92c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04180b1c50 0x7f04181603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f041f92c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0418182e00 con 0x7f04180b1c50 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f041f92c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04181830e0 con 0x7f0418068aa0 2023-10-06T03:01:48.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f041f92c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04181833c0 con 0x7f04180875d0 2023-10-06T03:01:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17b6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 0x7f17b80c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17b6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 0x7f17b80c4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44498/0 (socket says 172.21.15.92:44498) 2023-10-06T03:01:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f17b6d9d700 1 -- 172.21.15.92:0/3883531082 learned_addr learned my addr 172.21.15.92:0/3883531082 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 0x7f041817f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 0x7f041817f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59878/0 (socket says 172.21.15.92:59878) 2023-10-06T03:01:48.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 -- 172.21.15.92:0/1174939963 learned_addr learned my addr 172.21.15.92:0/1174939963 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8658022030 con 0x7f866806ab70 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f86567fc700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8648066f30 0x7f86480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8658086310 con 0x7f866806ab70 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43eeffd700 1 --2- 172.21.15.92:0/2307487458 >> v2:172.21.15.136:3300/0 conn(0x7f43f00b1d30 0x7f43f015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43ef7fe700 1 -- 172.21.15.92:0/2307487458 >> v2:172.21.15.136:3300/0 conn(0x7f43f00b1d30 msgr2=0x7f43f015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/2307487458 >> v2:172.21.15.136:3300/0 conn(0x7f43f00b1d30 0x7f43f015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43ef7fe700 1 -- 172.21.15.92:0/2307487458 >> v1:172.21.15.136:6789/0 conn(0x7f43f015d600 legacy=0x7f43f0068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7f43ef7fe700 1 -- 172.21.15.92:0/2307487458 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f015db00 con 0x7f43f00691b0 2023-10-06T03:01:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f8665d9b700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8648066f30 0x7f86480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.303+0000 7fe45b48d700 1 -- 172.21.15.92:0/2914726647 wait complete. 2023-10-06T03:01:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b659c700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b806ec80 0x7f17b80c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b6d9d700 1 -- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 msgr2=0x7f17b81765a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b6d9d700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b81765a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b6d9d700 1 -- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b806ec80 msgr2=0x7f17b80c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b6d9d700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b806ec80 0x7f17b80c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7fe45b48d700 1 Processor -- start 2023-10-06T03:01:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7fe45b48d700 1 -- start start 2023-10-06T03:01:48.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b6d9d700 1 -- 172.21.15.92:0/3883531082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b000bcf0 con 0x7f17b806bbe0 2023-10-06T03:01:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7fbb113db700 1 -- 172.21.15.92:0/3796625516 shutdown_connections 2023-10-06T03:01:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.304+0000 7fbb113db700 1 -- 172.21.15.92:0/3796625516 >> 172.21.15.92:0/3796625516 conn(0x7fbb0c0cd000 msgr2=0x7fbb0c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041dec9700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04180b1c50 0x7f04181603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041d6c8700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0418068aa0 0x7f0418182540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f8665d9b700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8648066f30 0x7f86480693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86680ccab0 tx=0x7f8660009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7fbb113db700 1 -- 172.21.15.92:0/3796625516 shutdown_connections 2023-10-06T03:01:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7fbb113db700 1 -- 172.21.15.92:0/3796625516 wait complete. 2023-10-06T03:01:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 -- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0418068aa0 msgr2=0x7f0418182540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0418068aa0 0x7f0418182540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 -- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04180b1c50 msgr2=0x7f04181603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04180b1c50 0x7f04181603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f041cec7700 1 -- 172.21.15.92:0/1174939963 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0408009420 con 0x7f04180875d0 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/2307487458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00691b0 0x7f43f01537f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009e60 tx=0x7f43e8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a51e0a721a21a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f43edffb700 1 -- 172.21.15.92:0/2307487458 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800e070 con 0x7f43f00691b0 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f43edffb700 1 -- 172.21.15.92:0/2307487458 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e800bb60 con 0x7f43f00691b0 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f43edffb700 1 -- 172.21.15.92:0/2307487458 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8012430 con 0x7f43f00691b0 2023-10-06T03:01:48.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.305+0000 7f17b6d9d700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 0x7f17b80c4cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00c880 tx=0x7f17ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f041cec7700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 0x7f041817f850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04140099f0 tx=0x7f0414019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400b7c0 con 0x7f04180875d0 2023-10-06T03:01:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041400b920 con 0x7f04180875d0 2023-10-06T03:01:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400bce0 con 0x7f04180875d0 2023-10-06T03:01:48.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0414009a20 con 0x7f04180875d0 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe45b48d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe45406b220 0x7fe4541417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe45b48d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe454161d90 0x7fe454147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe45b48d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe454162160 0x7fe454147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe45b48d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454141f20 con 0x7fe454161d90 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe45b48d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45414ae00 con 0x7fe454162160 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe45b48d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe454141cf0 con 0x7fe45406b220 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe459229700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe45406b220 0x7fe4541417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe459229700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe45406b220 0x7fe4541417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59882/0 (socket says 172.21.15.92:59882) 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe459229700 1 -- 172.21.15.92:0/3662905384 learned_addr learned my addr 172.21.15.92:0/3662905384 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866804c3c0 con 0x7f866806ab70 2023-10-06T03:01:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac011600 con 0x7f17b806bbe0 2023-10-06T03:01:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac011780 con 0x7f17b806bbe0 2023-10-06T03:01:48.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041817f330 con 0x7f04180875d0 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 -- 172.21.15.92:0/2307487458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00691b0 msgr2=0x7f43f01537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 --2- 172.21.15.92:0/2307487458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00691b0 0x7f43f01537f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009e60 tx=0x7f43e8009780 comp rx=0 tx=0).stop 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 -- 172.21.15.92:0/2307487458 shutdown_connections 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 --2- 172.21.15.92:0/2307487458 >> v2:172.21.15.136:3300/0 conn(0x7f43f00b1d30 0x7f43f015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 --2- 172.21.15.92:0/2307487458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00691b0 0x7f43f01537f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 -- 172.21.15.92:0/2307487458 >> 172.21.15.92:0/2307487458 conn(0x7f43f00cd000 msgr2=0x7f43f0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7fe459a2a700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe454162160 0x7fe454147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f43f5451700 1 -- 172.21.15.92:0/2307487458 shutdown_connections 2023-10-06T03:01:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fbb113db700 1 Processor -- start 2023-10-06T03:01:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fbb113db700 1 -- start start 2023-10-06T03:01:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fe459a2a700 1 -- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe45406b220 msgr2=0x7fe4541417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fe459a2a700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe45406b220 0x7fe4541417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fe459a2a700 1 -- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe454161d90 msgr2=0x7fe454147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fe459a2a700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe454161d90 0x7fe454147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7fe459a2a700 1 -- 172.21.15.92:0/3662905384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c009a20 con 0x7fe454162160 2023-10-06T03:01:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac011b40 con 0x7f17b806bbe0 2023-10-06T03:01:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b80c6de0 con 0x7f17b806bbe0 2023-10-06T03:01:48.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.306+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b80c6230 con 0x7f17b806bbe0 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb113db700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf8004cb0 0x7fbb0c140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb113db700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb0c140c00 0x7fbb0c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb113db700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 0x7fbb0c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb113db700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c0c9580 con 0x7fbb0c140c00 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb113db700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c0cb990 con 0x7fbaf8004cb0 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb113db700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c0cbc70 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fe459a2a700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe454162160 0x7fe454147ff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe444009fd0 tx=0x7fe444009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444013680 con 0x7fe454162160 2023-10-06T03:01:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb0b7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 0x7fbb0c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb0b7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 0x7fbb0c06ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59886/0 (socket says 172.21.15.92:59886) 2023-10-06T03:01:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fbb0b7fe700 1 -- 172.21.15.92:0/2585141703 learned_addr learned my addr 172.21.15.92:0/2585141703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0414020030 con 0x7f04180875d0 2023-10-06T03:01:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4440137e0 con 0x7fe454162160 2023-10-06T03:01:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444013b80 con 0x7fe454162160 2023-10-06T03:01:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.307+0000 7f43f5451700 1 -- 172.21.15.92:0/2307487458 wait complete. 2023-10-06T03:01:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7f43f5451700 1 Processor -- start 2023-10-06T03:01:48.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f040d7fa700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f03fc067080 0x7f03fc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0414085b10 con 0x7f04180875d0 2023-10-06T03:01:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4540c69c0 con 0x7fe454162160 2023-10-06T03:01:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f330676b700 1 Processor -- start 2023-10-06T03:01:48.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4540c6370 con 0x7fe454162160 2023-10-06T03:01:48.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17ac018360 con 0x7f17b806bbe0 2023-10-06T03:01:48.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f179effd700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f17a4067080 0x7f17a4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17ac0844c0 con 0x7f17b806bbe0 2023-10-06T03:01:48.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f17b659c700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f17a4067080 0x7f17a4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f330676b700 1 -- start start 2023-10-06T03:01:48.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b80c72e0 con 0x7f17b806bbe0 2023-10-06T03:01:48.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f80052b0 con 0x7f04180875d0 2023-10-06T03:01:48.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.308+0000 7f43f5451700 1 -- start start 2023-10-06T03:01:48.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43f5451700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43f00691b0 0x7f43f014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43f5451700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00b1d30 0x7f43f006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43f5451700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f015d600 0x7f43f013ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43f5451700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f014d210 con 0x7f43f015d600 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43f5451700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f014c340 con 0x7f43f00691b0 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43f5451700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f014cd10 con 0x7f43f00b1d30 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00b1d30 0x7f43f006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00b1d30 0x7f43f006eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59896/0 (socket says 172.21.15.92:59896) 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f015d600 0x7f43f013ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ee7fc700 1 -- 172.21.15.92:0/3451081349 learned_addr learned my addr 172.21.15.92:0/3451081349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f041d6c8700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f03fc067080 0x7f03fc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f17b659c700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f17a4067080 0x7f17a4069530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17b80ac8e0 tx=0x7f17a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43eeffd700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43f00691b0 0x7f43f014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 -- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00b1d30 msgr2=0x7f43f006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00b1d30 0x7f43f006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 -- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43f00691b0 msgr2=0x7f43f014c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43f00691b0 0x7f43f014c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 -- 172.21.15.92:0/3451081349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009a20 con 0x7f43f015d600 2023-10-06T03:01:48.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43eeffd700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43f00691b0 0x7f43f014c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:48.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f041d6c8700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f03fc067080 0x7f03fc069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0408009ea0 tx=0x7f04080091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.310+0000 7f43ef7fe700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f015d600 0x7f43f013ce10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43e8016000 tx=0x7f43e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fbb0b7fe700 1 -- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf8004cb0 msgr2=0x7fbb0c140610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:01:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fbb0b7fe700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf8004cb0 0x7fbb0c140610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fbb0b7fe700 1 -- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb0c140c00 msgr2=0x7fbb0c0c9a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:01:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fbb0b7fe700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb0c140c00 0x7fbb0c0c9a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fbb0b7fe700 1 -- 172.21.15.92:0/2585141703 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c0c8f30 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.309+0000 7fbb0b7fe700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 0x7fbb0c06ecc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c069580 tx=0x7fbb04002de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f330676b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f8034170 0x7f32f80e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f330676b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f8005f10 con 0x7f32f80ddfc0 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f330676b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f8053ba0 con 0x7f32f804f330 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f330676b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f8053e80 con 0x7f32f8034170 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f3305769700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f32f80ddfc0 0x7f32f8053350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34380/0 (socket says 172.21.15.92:34380) 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f3305769700 1 -- 172.21.15.92:0/1663605162 learned_addr learned my addr 172.21.15.92:0/1663605162 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:48.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f3305f6a700 1 --2- 172.21.15.92:0/1663605162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f8034170 0x7f32f80e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4540c6b50 con 0x7fe454162160 2023-10-06T03:01:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f3305f6a700 1 -- 172.21.15.92:0/1663605162 >> v1:172.21.15.136:6789/0 conn(0x7f32f804f330 legacy=0x7f32f8033a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f3305f6a700 1 -- 172.21.15.92:0/1663605162 >> v1:172.21.15.92:6789/0 conn(0x7f32f80ddfc0 legacy=0x7f32f8053350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.311+0000 7f3305f6a700 1 -- 172.21.15.92:0/1663605162 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8054160 con 0x7f32f8034170 2023-10-06T03:01:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.312+0000 7f3305f6a700 1 --2- 172.21.15.92:0/1663605162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f8034170 0x7f32f80e91d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009910 tx=0x7f32f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15adc4142f918351 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.312+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4440123d0 con 0x7fe454162160 2023-10-06T03:01:48.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.312+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8658053420 con 0x7f866806ab70 2023-10-06T03:01:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.312+0000 7fe4497fa700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438067000 0x7fe4380694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.312+0000 7fe459229700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438067000 0x7fe4380694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.312+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe4440846f0 con 0x7fe454162160 2023-10-06T03:01:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.313+0000 7fe459229700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438067000 0x7fe4380694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0099f0 tx=0x7fe44c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f962fdbb700 1 Processor -- start 2023-10-06T03:01:48.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f962fdbb700 1 -- start start 2023-10-06T03:01:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8027470 con 0x7f43f015d600 2023-10-06T03:01:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8027630 con 0x7f43f015d600 2023-10-06T03:01:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80279f0 con 0x7f43f015d600 2023-10-06T03:01:48.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17ac0515b0 con 0x7f17b806bbe0 2023-10-06T03:01:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f014ba50 con 0x7f43f015d600 2023-10-06T03:01:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0414052c00 con 0x7f04180875d0 2023-10-06T03:01:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.315+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f014b480 con 0x7f43f015d600 2023-10-06T03:01:48.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f014be10 con 0x7f43f015d600 2023-10-06T03:01:48.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe444055290 con 0x7fe454162160 2023-10-06T03:01:48.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f43e8024070 con 0x7f43f015d600 2023-10-06T03:01:48.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962fdbb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96280cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962fdbb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f962806bac0 0x7f962806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962fdbb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96280cc210 con 0x7f962806b350 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962fdbb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96280cc4f0 con 0x7f962806bac0 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962fdbb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96280cc7d0 con 0x7f962806aa40 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962db57700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96280cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962db57700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96280cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59912/0 (socket says 172.21.15.92:59912) 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7f962db57700 1 -- 172.21.15.92:0/836774440 learned_addr learned my addr 172.21.15.92:0/836774440 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f962db57700 1 -- 172.21.15.92:0/836774440 >> v2:172.21.15.136:3300/0 conn(0x7f962806bac0 msgr2=0x7f962806ae10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f962db57700 1 --2- 172.21.15.92:0/836774440 >> v2:172.21.15.136:3300/0 conn(0x7f962806bac0 0x7f962806ae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f962db57700 1 -- 172.21.15.92:0/836774440 >> v1:172.21.15.92:6789/0 conn(0x7f962806b350 legacy=0x7f96280c81d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:01:48.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f962db57700 1 -- 172.21.15.92:0/836774440 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280ccab0 con 0x7f962806aa40 2023-10-06T03:01:48.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f962db57700 1 --2- 172.21.15.92:0/836774440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96280cbc00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9624009990 tx=0x7f9624009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d64c2f82f2238f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.317+0000 7f962cb55700 1 -- 172.21.15.92:0/836774440 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624005c10 con 0x7f962806aa40 2023-10-06T03:01:48.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f962cb55700 1 -- 172.21.15.92:0/836774440 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624005d70 con 0x7f962806aa40 2023-10-06T03:01:48.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f962cb55700 1 -- 172.21.15.92:0/836774440 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400b840 con 0x7f962806aa40 2023-10-06T03:01:48.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f43d6ffd700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43e8086c40 con 0x7f43f015d600 2023-10-06T03:01:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f43eeffd700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f43eeffd700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006fd0 tx=0x7f43e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7f962fdbb700 1 -- 172.21.15.92:0/836774440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 msgr2=0x7f96280cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f962fdbb700 1 --2- 172.21.15.92:0/836774440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96280cbc00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9624009990 tx=0x7f9624009220 comp rx=0 tx=0).stop 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f32f7fff700 1 -- 172.21.15.92:0/1663605162 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000b860 con 0x7f32f8034170 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f32f7fff700 1 -- 172.21.15.92:0/1663605162 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f000b9c0 con 0x7f32f8034170 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f32f7fff700 1 -- 172.21.15.92:0/1663605162 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000bd60 con 0x7f32f8034170 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f330676b700 1 -- 172.21.15.92:0/1663605162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f8034170 msgr2=0x7f32f80e91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f330676b700 1 --2- 172.21.15.92:0/1663605162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f8034170 0x7f32f80e91d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009910 tx=0x7f32f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f962fdbb700 1 -- 172.21.15.92:0/836774440 shutdown_connections 2023-10-06T03:01:48.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f962fdbb700 1 --2- 172.21.15.92:0/836774440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96280cbc00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f962fdbb700 1 --2- 172.21.15.92:0/836774440 >> v2:172.21.15.136:3300/0 conn(0x7f962806bac0 0x7f962806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7f962fdbb700 1 -- 172.21.15.92:0/836774440 >> 172.21.15.92:0/836774440 conn(0x7f96280cd670 msgr2=0x7f96280cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f430 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0400f5b0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c0c9210 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c0cb6f0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.316+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f970 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.318+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c0c93a0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb040183c0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7fbb037fe700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbaec06f740 0x7fbaec071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb040847c0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.319+0000 7fbb0affd700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbaec06f740 0x7fbaec071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.320+0000 7fbb0affd700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbaec06f740 0x7fbaec071bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c3d0 tx=0x7fbafc015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.323+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e8053d50 con 0x7f43f015d600 2023-10-06T03:01:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.324+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb040518d0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.324+0000 7f962fdbb700 1 -- 172.21.15.92:0/836774440 shutdown_connections 2023-10-06T03:01:48.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.328+0000 7f962fdbb700 1 -- 172.21.15.92:0/836774440 wait complete. 2023-10-06T03:01:48.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.329+0000 7f330676b700 1 -- 172.21.15.92:0/1663605162 shutdown_connections 2023-10-06T03:01:48.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.329+0000 7f330676b700 1 --2- 172.21.15.92:0/1663605162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f8034170 0x7f32f80e91d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.329+0000 7f330676b700 1 -- 172.21.15.92:0/1663605162 >> 172.21.15.92:0/1663605162 conn(0x7f32f80516f0 msgr2=0x7f32f8051af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.329+0000 7f962fdbb700 1 Processor -- start 2023-10-06T03:01:48.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.329+0000 7f962fdbb700 1 -- start start 2023-10-06T03:01:48.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962fdbb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96281470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962fdbb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f962806b350 0x7f96280b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962fdbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f962806bac0 0x7f96280c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962fdbb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b22c0 con 0x7f962806bac0 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962fdbb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b18f0 con 0x7f962806b350 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962fdbb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96280c5610 con 0x7f962806aa40 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96281470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96281470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:59924/0 (socket says 172.21.15.92:59924) 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 -- 172.21.15.92:0/3355962656 learned_addr learned my addr 172.21.15.92:0/3355962656 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 -- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f962806b350 msgr2=0x7f96280b1d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f962806b350 0x7f96280b1d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 -- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f962806bac0 msgr2=0x7f96280c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f962806bac0 0x7f96280c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.331+0000 7f962db57700 1 -- 172.21.15.92:0/3355962656 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9624009480 con 0x7f962806aa40 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.332+0000 7f962db57700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96281470c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f962400b3c0 tx=0x7f9624015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.332+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624000d10 con 0x7f962806aa40 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.332+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624006730 con 0x7f962806aa40 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.332+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962401d430 con 0x7f962806aa40 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.333+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96280b0fc0 con 0x7f962806aa40 2023-10-06T03:01:48.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.333+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96280b1580 con 0x7f962806aa40 2023-10-06T03:01:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.335+0000 7f330676b700 1 -- 172.21.15.92:0/1663605162 shutdown_connections 2023-10-06T03:01:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.335+0000 7f330676b700 1 -- 172.21.15.92:0/1663605162 wait complete. 2023-10-06T03:01:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.335+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f962401d7e0 con 0x7f962806aa40 2023-10-06T03:01:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.335+0000 7f961dffb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96140670a0 0x7f9614069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.336+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96240864b0 con 0x7f962806aa40 2023-10-06T03:01:48.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.336+0000 7f962d356700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96140670a0 0x7f9614069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.336+0000 7f330676b700 1 Processor -- start 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.337+0000 7f330676b700 1 -- start start 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f330676b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 0x7f32f804e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f330676b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f804f330 0x7f32f8039d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f330676b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f80ddfc0 0x7f32f804bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f330676b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80cf720 con 0x7f32f804f330 2023-10-06T03:01:48.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f330676b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f804ed30 con 0x7f32f8034170 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f330676b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f80c7800 con 0x7f32f80ddfc0 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f3304f68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f804f330 0x7f32f8039d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f3305769700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 0x7f32f804e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f3305769700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 0x7f32f804e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50822/0 (socket says 172.21.15.92:50822) 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f3304f68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f804f330 0x7f32f8039d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44528/0 (socket says 172.21.15.92:44528) 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.338+0000 7f3305769700 1 -- 172.21.15.92:0/3994010222 learned_addr learned my addr 172.21.15.92:0/3994010222 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f3305769700 1 -- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f80ddfc0 msgr2=0x7f32f804bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f3305769700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f80ddfc0 0x7f32f804bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f3305769700 1 -- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f804f330 msgr2=0x7f32f8039d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f3305769700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f804f330 0x7f32f8039d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f3305769700 1 -- 172.21.15.92:0/3994010222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0009420 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f3305769700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 0x7f32f804e7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33000701a0 tx=0x7f3300066260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300068010 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330004c440 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330006a870 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f80cedf0 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.339+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f80cf3b0 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.341+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f330006aca0 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.341+0000 7f32f57fa700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32ec066ff0 0x7f32ec0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.341+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f33000fbc60 con 0x7f32f8034170 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.342+0000 7f3304f68700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32ec066ff0 0x7f32ec0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:48.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.342+0000 7f3304f68700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32ec066ff0 0x7f32ec0694a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32f80c7ed0 tx=0x7f32fc000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.344+0000 7f962d356700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96140670a0 0x7f9614069550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9628147600 tx=0x7f9618006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.346+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f8005380 con 0x7f32f8034170 2023-10-06T03:01:48.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.347+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96080052b0 con 0x7f962806aa40 2023-10-06T03:01:48.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.351+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3300095020 con 0x7f32f8034170 2023-10-06T03:01:48.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.357+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96240535e0 con 0x7f962806aa40 2023-10-06T03:01:48.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.677+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03f8005560 con 0x7f04180875d0 2023-10-06T03:01:48.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.680+0000 7f040d7fa700 1 -- 172.21.15.92:0/1174939963 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0414056220 con 0x7f04180875d0 2023-10-06T03:01:48.685 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-10-06T03:01:48.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.685+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f03fc067080 msgr2=0x7f03fc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.685+0000 7f041f92c700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f03fc067080 0x7f03fc069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0408009ea0 tx=0x7f04080091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.685+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 msgr2=0x7f041817f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.685+0000 7f041f92c700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 0x7f041817f850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04140099f0 tx=0x7f0414019000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 shutdown_connections 2023-10-06T03:01:48.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f03fc067080 0x7f03fc069530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04180b1c50 0x7f04181603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0418068aa0 0x7f0418182540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 --2- 172.21.15.92:0/1174939963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04180875d0 0x7f041817f850 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 >> 172.21.15.92:0/1174939963 conn(0x7f04180cd000 msgr2=0x7f0418068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.691+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 shutdown_connections 2023-10-06T03:01:48.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.692+0000 7f041f92c700 1 -- 172.21.15.92:0/1174939963 wait complete. 2023-10-06T03:01:48.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.697+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f43f014c480 con 0x7f43f015d600 2023-10-06T03:01:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.699+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f866806b8a0 con 0x7f866806ab70 2023-10-06T03:01:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.701+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbb0c0cbdb0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.701+0000 7f43d6ffd700 1 -- 172.21.15.92:0/3451081349 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f43e802c090 con 0x7f43f015d600 2023-10-06T03:01:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T03:01:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.701+0000 7f86567fc700 1 -- 172.21.15.92:0/2609644446 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8658056a40 con 0x7f866806ab70 2023-10-06T03:01:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.701+0000 7fbb037fe700 1 -- 172.21.15.92:0/2585141703 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb04054ef0 con 0x7fbb0c0cb0b0 2023-10-06T03:01:48.703 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-10-06T03:01:48.706 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.704+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17b804c3c0 con 0x7f17b806bbe0 2023-10-06T03:01:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7f179effd700 1 -- 172.21.15.92:0/3883531082 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17ac01e070 con 0x7f17b806bbe0 2023-10-06T03:01:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbaec06f740 msgr2=0x7fbaec071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbaec06f740 0x7fbaec071bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c3d0 tx=0x7fbafc015000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 msgr2=0x7fbb0c06ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 0x7fbb0c06ecc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c069580 tx=0x7fbb04002de0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 shutdown_connections 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbaec06f740 0x7fbaec071bf0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c3d0 tx=0x7fbafc015000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb0c0cb0b0 0x7fbb0c06ecc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbaf8004cb0 0x7fbb0c140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 --2- 172.21.15.92:0/2585141703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb0c140c00 0x7fbb0c0c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 >> 172.21.15.92:0/2585141703 conn(0x7fbb0c0cd000 msgr2=0x7fbb0c0a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.708 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-10-06T03:01:48.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8648066f30 msgr2=0x7f86480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8648066f30 0x7f86480693e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86680ccab0 tx=0x7f8660009040 comp rx=0 tx=0).stop 2023-10-06T03:01:48.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 msgr2=0x7f8668177200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 0x7f8668177200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f865800b3c0 tx=0x7f8658016000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 shutdown_connections 2023-10-06T03:01:48.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.705+0000 7fbb113db700 1 -- 172.21.15.92:0/2585141703 wait complete. 2023-10-06T03:01:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.706+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 shutdown_connections 2023-10-06T03:01:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.706+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8648066f30 0x7f86480693e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.706+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f866815de50 0x7f8668157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.706+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f866806ab70 0x7f8668177200 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.706+0000 7f866cd2d700 1 --2- 172.21.15.92:0/2609644446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86680c83c0 0x7f8668177c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.706+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 >> 172.21.15.92:0/2609644446 conn(0x7f86680cd670 msgr2=0x7f86681642a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.707+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 shutdown_connections 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.707+0000 7f866cd2d700 1 -- 172.21.15.92:0/2609644446 wait complete. 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.707+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43dc066f30 msgr2=0x7f43dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.707+0000 7f43f5451700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43dc066f30 0x7f43dc0693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43e0006fd0 tx=0x7f43e0008040 comp rx=0 tx=0).stop 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.707+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f015d600 msgr2=0x7f43f013ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.707+0000 7f43f5451700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f015d600 0x7f43f013ce10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f43e8016000 tx=0x7f43e8019000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.708+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f17a4067080 msgr2=0x7f17a4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.708+0000 7f17bc97b700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f17a4067080 0x7f17a4069530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17b80ac8e0 tx=0x7f17a8009040 comp rx=0 tx=0).stop 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.708+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 msgr2=0x7f17b80c4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.708+0000 7f17bc97b700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 0x7f17b80c4cd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00c880 tx=0x7f17ac008040 comp rx=0 tx=0).stop 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 shutdown_connections 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43dc066f30 0x7f43dc0693e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43f015d600 0x7f43f013ce10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43f00691b0 0x7f43f014c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 --2- 172.21.15.92:0/3451081349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43f00b1d30 0x7f43f006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 >> 172.21.15.92:0/3451081349 conn(0x7f43f00cd000 msgr2=0x7f43f014dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 shutdown_connections 2023-10-06T03:01:48.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.709+0000 7f43f5451700 1 -- 172.21.15.92:0/3451081349 wait complete. 2023-10-06T03:01:48.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-06T03:01:48.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 shutdown_connections 2023-10-06T03:01:48.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f17a4067080 0x7f17a4069530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17b806bbe0 0x7f17b80c4cd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17b8156950 0x7f17b81765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 --2- 172.21.15.92:0/3883531082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17b806ec80 0x7f17b80c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 >> 172.21.15.92:0/3883531082 conn(0x7f17b806b490 msgr2=0x7f17b80ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 shutdown_connections 2023-10-06T03:01:48.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.712+0000 7f17bc97b700 1 -- 172.21.15.92:0/3883531082 wait complete. 2023-10-06T03:01:48.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-06T03:01:48.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.732+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe45404c3c0 con 0x7fe454162160 2023-10-06T03:01:48.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-06T03:01:48.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-10-06T03:01:48.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.732+0000 7fe4497fa700 1 -- 172.21.15.92:0/3662905384 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe444054e20 con 0x7fe454162160 2023-10-06T03:01:48.736 INFO:teuthology.orchestra.run.smithi092.stdout:55834574853 2023-10-06T03:01:48.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-06T03:01:48.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438067000 msgr2=0x7fe4380694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438067000 0x7fe4380694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe44c0099f0 tx=0x7fe44c019000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe454162160 msgr2=0x7fe454147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe454162160 0x7fe454147ff0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe444009fd0 tx=0x7fe444009250 comp rx=0 tx=0).stop 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 shutdown_connections 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438067000 0x7fe4380694b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe454162160 0x7fe454147ff0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe45406b220 0x7fe4541417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 --2- 172.21.15.92:0/3662905384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe454161d90 0x7fe454147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 >> 172.21.15.92:0/3662905384 conn(0x7fe4540c9870 msgr2=0x7fe4540b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 shutdown_connections 2023-10-06T03:01:48.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.735+0000 7fe45b48d700 1 -- 172.21.15.92:0/3662905384 wait complete. 2023-10-06T03:01:48.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.736+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9608005830 con 0x7f962806aa40 2023-10-06T03:01:48.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.736+0000 7f961dffb700 1 -- 172.21.15.92:0/3355962656 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9624056c00 con 0x7f962806aa40 2023-10-06T03:01:48.739 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T03:01:48.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.738+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f32f8054440 con 0x7f32f8034170 2023-10-06T03:01:48.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.738+0000 7f32f57fa700 1 -- 172.21.15.92:0/3994010222 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f33000c8da0 con 0x7f32f8034170 2023-10-06T03:01:48.740 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96140670a0 msgr2=0x7f9614069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96140670a0 0x7f9614069550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9628147600 tx=0x7f9618006bc0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 msgr2=0x7f96281470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96281470c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f962400b3c0 tx=0x7f9624015000 comp rx=0 tx=0).stop 2023-10-06T03:01:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 shutdown_connections 2023-10-06T03:01:48.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96140670a0 0x7f9614069550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f962806bac0 0x7f96280c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f962806aa40 0x7f96281470c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 --2- 172.21.15.92:0/3355962656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f962806b350 0x7f96280b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 >> 172.21.15.92:0/3355962656 conn(0x7f96280cd670 msgr2=0x7f96280c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 shutdown_connections 2023-10-06T03:01:48.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.739+0000 7f962fdbb700 1 -- 172.21.15.92:0/3355962656 wait complete. 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32ec066ff0 msgr2=0x7f32ec0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32ec066ff0 0x7f32ec0694a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32f80c7ed0 tx=0x7f32fc000bc0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 msgr2=0x7f32f804e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 0x7f32f804e7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f33000701a0 tx=0x7f3300066260 comp rx=0 tx=0).stop 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 shutdown_connections 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32ec066ff0 0x7f32ec0694a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32f80ddfc0 0x7f32f804bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32f8034170 0x7f32f804e7f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 --2- 172.21.15.92:0/3994010222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32f804f330 0x7f32f8039d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:48.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 >> 172.21.15.92:0/3994010222 conn(0x7f32f80516f0 msgr2=0x7f32f8036950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:48.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 shutdown_connections 2023-10-06T03:01:48.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:48.742+0000 7f330676b700 1 -- 172.21.15.92:0/3994010222 wait complete. 2023-10-06T03:01:48.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-06T03:01:48.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2023-10-06T03:01:48.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-10-06T03:01:49.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T03:01:49.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T03:01:49.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T03:01:49.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T03:01:49.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T03:01:49.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T03:01:49.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T03:01:49.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.899+0000 7f1896f86700 1 Processor -- start 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.899+0000 7f1896f86700 1 -- start start 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.900+0000 7f1896f86700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900b1c50 0x7f18900879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1896f86700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1890068aa0 0x7f1890068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1896f86700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1890069450 con 0x7f18900875d0 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1896f86700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18900abfd0 con 0x7f1890068aa0 2023-10-06T03:01:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1896f86700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18900acdf0 con 0x7f18900b1c50 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1895523700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1890068aa0 0x7f1890068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1895523700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1890068aa0 0x7f1890068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34436/0 (socket says 172.21.15.92:34436) 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.901+0000 7f1895523700 1 -- 172.21.15.92:0/3484961299 learned_addr learned my addr 172.21.15.92:0/3484961299 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f188ffff700 1 --2- 172.21.15.92:0/3484961299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900b1c50 0x7f18900879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f1895523700 1 -- 172.21.15.92:0/3484961299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900b1c50 msgr2=0x7f18900879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f1895523700 1 --2- 172.21.15.92:0/3484961299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900b1c50 0x7f18900879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f1895523700 1 -- 172.21.15.92:0/3484961299 >> v1:172.21.15.92:6789/0 conn(0x7f18900875d0 legacy=0x7f18900ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f1895523700 1 -- 172.21.15.92:0/3484961299 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900ac1e0 con 0x7f1890068aa0 2023-10-06T03:01:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f1895523700 1 --2- 172.21.15.92:0/3484961299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890068f10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1888007750 tx=0x7f188800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecf3d03f4231d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.902+0000 7f188f7fe700 1 -- 172.21.15.92:0/3484961299 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800ce50 con 0x7f1890068aa0 2023-10-06T03:01:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.903+0000 7f188f7fe700 1 -- 172.21.15.92:0/3484961299 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188800c030 con 0x7f1890068aa0 2023-10-06T03:01:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.903+0000 7f188f7fe700 1 -- 172.21.15.92:0/3484961299 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800c3d0 con 0x7f1890068aa0 2023-10-06T03:01:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.903+0000 7f1896f86700 1 -- 172.21.15.92:0/3484961299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 msgr2=0x7f1890068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.903+0000 7f1896f86700 1 --2- 172.21.15.92:0/3484961299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890068f10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1888007750 tx=0x7f188800b490 comp rx=0 tx=0).stop 2023-10-06T03:01:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.904+0000 7f1896f86700 1 -- 172.21.15.92:0/3484961299 shutdown_connections 2023-10-06T03:01:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.904+0000 7f1896f86700 1 --2- 172.21.15.92:0/3484961299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890068f10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.904+0000 7f1896f86700 1 --2- 172.21.15.92:0/3484961299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900b1c50 0x7f18900879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.904+0000 7f1896f86700 1 -- 172.21.15.92:0/3484961299 >> 172.21.15.92:0/3484961299 conn(0x7f18900cd000 msgr2=0x7f1890068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.904+0000 7f1896f86700 1 -- 172.21.15.92:0/3484961299 shutdown_connections 2023-10-06T03:01:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.904+0000 7f1896f86700 1 -- 172.21.15.92:0/3484961299 wait complete. 2023-10-06T03:01:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.906+0000 7f1896f86700 1 Processor -- start 2023-10-06T03:01:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.906+0000 7f1896f86700 1 -- start start 2023-10-06T03:01:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1896f86700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890176910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1896f86700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900875d0 0x7f18900a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1896f86700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18900b1c50 0x7f18900a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1896f86700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18900a7240 con 0x7f1890068aa0 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1896f86700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1890177800 con 0x7f18900b1c50 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1896f86700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189017a0b0 con 0x7f18900875d0 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1894d22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890176910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1894d22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890176910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34448/0 (socket says 172.21.15.92:34448) 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.907+0000 7f1894d22700 1 -- 172.21.15.92:0/3938563811 learned_addr learned my addr 172.21.15.92:0/3938563811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.908+0000 7f1894d22700 1 -- 172.21.15.92:0/3938563811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900875d0 msgr2=0x7f18900a6d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.908+0000 7f1894d22700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900875d0 0x7f18900a6d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.908+0000 7f1894d22700 1 -- 172.21.15.92:0/3938563811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18900b1c50 msgr2=0x7f18900a9a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:49.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.908+0000 7f1894d22700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18900b1c50 0x7f18900a9a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.908+0000 7f1894d22700 1 -- 172.21.15.92:0/3938563811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1888006020 con 0x7f1890068aa0 2023-10-06T03:01:49.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.908+0000 7f1894d22700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890176910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f188400c590 tx=0x7f188400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.909+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884011590 con 0x7f1890068aa0 2023-10-06T03:01:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.909+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18840116f0 con 0x7f1890068aa0 2023-10-06T03:01:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.909+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884011ab0 con 0x7f1890068aa0 2023-10-06T03:01:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.909+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1890176f60 con 0x7f1890068aa0 2023-10-06T03:01:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.909+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1890177520 con 0x7f1890068aa0 2023-10-06T03:01:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.911+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f188401b3b0 con 0x7f1890068aa0 2023-10-06T03:01:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.912+0000 7f188cff9700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18780671e0 0x7f1878069690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.912+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1884083c90 con 0x7f1890068aa0 2023-10-06T03:01:49.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.913+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c0052e0 con 0x7f1890068aa0 2023-10-06T03:01:49.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.918+0000 7f188ffff700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18780671e0 0x7f1878069690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.919+0000 7f188ffff700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18780671e0 0x7f1878069690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18901643f0 tx=0x7f1880008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.922+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1884017080 con 0x7f1890068aa0 2023-10-06T03:01:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.937+0000 7fa629ccc700 1 Processor -- start 2023-10-06T03:01:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.937+0000 7fa629ccc700 1 -- start start 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa629ccc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240b1c50 0x7fa6240879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa629ccc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa624068aa0 0x7fa624068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa629ccc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa624069450 con 0x7fa6240875d0 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa629ccc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6240abfd0 con 0x7fa624068aa0 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa629ccc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240acdf0 con 0x7fa6240b1c50 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa623fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa624068aa0 0x7fa624068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa623fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa624068aa0 0x7fa624068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34460/0 (socket says 172.21.15.92:34460) 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa623fff700 1 -- 172.21.15.92:0/1208644679 learned_addr learned my addr 172.21.15.92:0/1208644679 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.939+0000 7fa622ffd700 1 --2- 172.21.15.92:0/1208644679 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240b1c50 0x7fa6240879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa623fff700 1 -- 172.21.15.92:0/1208644679 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240b1c50 msgr2=0x7fa6240879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa623fff700 1 --2- 172.21.15.92:0/1208644679 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240b1c50 0x7fa6240879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa623fff700 1 -- 172.21.15.92:0/1208644679 >> v1:172.21.15.136:6789/0 conn(0x7fa6240875d0 legacy=0x7fa6240ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa623fff700 1 -- 172.21.15.92:0/1208644679 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240ac1e0 con 0x7fa624068aa0 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.943+0000 7fa6227fc700 1 -- 172.21.15.92:0/1208644679 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971580692 0 0) 0x7fa624069450 con 0x7fa6240875d0 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa623fff700 1 --2- 172.21.15.92:0/1208644679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 0x7fa624068f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa6180099d0 tx=0x7fa618009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8b12da80dca3ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa6227fc700 1 -- 172.21.15.92:0/1208644679 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800e070 con 0x7fa624068aa0 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa6227fc700 1 -- 172.21.15.92:0/1208644679 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6180090d0 con 0x7fa624068aa0 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.944+0000 7fa6227fc700 1 -- 172.21.15.92:0/1208644679 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6180126e0 con 0x7fa624068aa0 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 -- 172.21.15.92:0/1208644679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 msgr2=0x7fa624068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 --2- 172.21.15.92:0/1208644679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 0x7fa624068f10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa6180099d0 tx=0x7fa618009730 comp rx=0 tx=0).stop 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 -- 172.21.15.92:0/1208644679 shutdown_connections 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 --2- 172.21.15.92:0/1208644679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 0x7fa624068f10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 --2- 172.21.15.92:0/1208644679 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240b1c50 0x7fa6240879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 -- 172.21.15.92:0/1208644679 >> 172.21.15.92:0/1208644679 conn(0x7fa6240cd000 msgr2=0x7fa624068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 -- 172.21.15.92:0/1208644679 shutdown_connections 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.945+0000 7fa629ccc700 1 -- 172.21.15.92:0/1208644679 wait complete. 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.947+0000 7fa629ccc700 1 Processor -- start 2023-10-06T03:01:49.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.947+0000 7fa629ccc700 1 -- start start 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa629ccc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 0x7fa624152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa629ccc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 0x7fa62416cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa629ccc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 0x7fa624151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa629ccc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62416ac00 con 0x7fa624068aa0 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa629ccc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241529a0 con 0x7fa6240b1c50 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa629ccc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62416a310 con 0x7fa6240875d0 2023-10-06T03:01:49.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa622ffd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 0x7fa62416cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa623fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 0x7fa624151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa622ffd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 0x7fa62416cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42628/0 (socket says 172.21.15.92:42628) 2023-10-06T03:01:49.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa622ffd700 1 -- 172.21.15.92:0/3305093919 learned_addr learned my addr 172.21.15.92:0/3305093919 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.948+0000 7fa623fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 0x7fa624151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:43266/0 (socket says 172.21.15.92:43266) 2023-10-06T03:01:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa623fff700 1 -- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 msgr2=0x7fa62416cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa623fff700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 0x7fa62416cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa623fff700 1 -- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 msgr2=0x7fa624152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa623fff700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 0x7fa624152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa623fff700 1 -- 172.21.15.92:0/3305093919 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa618007040 con 0x7fa6240b1c50 2023-10-06T03:01:49.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa622ffd700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 0x7fa62416cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa623fff700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 0x7fa624151c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa62406f950 tx=0x7fa61801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800e070 con 0x7fa6240b1c50 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61800b5c0 con 0x7fa6240b1c50 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62416a5b0 con 0x7fa6240b1c50 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62416d000 con 0x7fa6240b1c50 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.949+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61800b960 con 0x7fa6240b1c50 2023-10-06T03:01:49.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.951+0000 7f26ef33f700 1 Processor -- start 2023-10-06T03:01:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.951+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62404c3c0 con 0x7fa6240b1c50 2023-10-06T03:01:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.951+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa618021070 con 0x7fa6240b1c50 2023-10-06T03:01:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.951+0000 7fa60b7fe700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.952+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa618016030 con 0x7fa6240b1c50 2023-10-06T03:01:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.952+0000 7fa6237fe700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.952+0000 7fa6237fe700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa61c007fd0 tx=0x7fa61c007e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.952+0000 7f26ef33f700 1 -- start start 2023-10-06T03:01:49.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f26ef33f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f26e8156310 0x7f26e80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f26ef33f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26e81566e0 0x7f26e80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f26ef33f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e80ca300 0x7f26e80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f26ef33f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80ca860 con 0x7f26e81566e0 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f26ef33f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80cab40 con 0x7f26e8156310 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f26ef33f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80cae20 con 0x7f26e80ca300 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26e81566e0 0x7f26e80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f98e1657700 1 Processor -- start 2023-10-06T03:01:49.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.953+0000 7f98e1657700 1 -- start start 2023-10-06T03:01:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f26e81566e0 0x7f26e80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34474/0 (socket says 172.21.15.92:34474) 2023-10-06T03:01:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 -- 172.21.15.92:0/2970288780 learned_addr learned my addr 172.21.15.92:0/2970288780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 -- 172.21.15.92:0/2970288780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e80ca300 msgr2=0x7f26e80ae540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:01:49.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 --2- 172.21.15.92:0/2970288780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e80ca300 0x7f26e80ae540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 -- 172.21.15.92:0/2970288780 >> v2:172.21.15.136:3300/0 conn(0x7f26e8156310 msgr2=0x7f26e80c9dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 --2- 172.21.15.92:0/2970288780 >> v2:172.21.15.136:3300/0 conn(0x7f26e8156310 0x7f26e80c9dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 -- 172.21.15.92:0/2970288780 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80cb100 con 0x7f26e81566e0 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26edb3c700 1 --2- 172.21.15.92:0/2970288780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e81566e0 0x7f26e80ae000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cce20 tx=0x7f26e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2e925af85b9300 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26ed33b700 1 -- 172.21.15.92:0/2970288780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000e070 con 0x7f26e81566e0 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26ed33b700 1 -- 172.21.15.92:0/2970288780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e000ba50 con 0x7f26e81566e0 2023-10-06T03:01:49.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f26ed33b700 1 -- 172.21.15.92:0/2970288780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00092e0 con 0x7f26e81566e0 2023-10-06T03:01:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f98e1657700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98dc06d4f0 0x7f98dc0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f98e1657700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f98dc0cb060 0x7f98dc06a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f98e1657700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc06a190 con 0x7f98dc0872f0 2023-10-06T03:01:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f98e1657700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc1668f0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.954+0000 7f98e1657700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc069860 con 0x7f98dc0cb060 2023-10-06T03:01:49.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98dc06d4f0 0x7f98dc0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98dc06d4f0 0x7f98dc0cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34490/0 (socket says 172.21.15.92:34490) 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 -- 172.21.15.92:0/3581217075 learned_addr learned my addr 172.21.15.92:0/3581217075 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98e0e56700 1 --2- 172.21.15.92:0/3581217075 >> v2:172.21.15.136:3300/0 conn(0x7f98dc0cb060 0x7f98dc06a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 -- 172.21.15.92:0/3581217075 >> v2:172.21.15.136:3300/0 conn(0x7f98dc0cb060 msgr2=0x7f98dc06a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 --2- 172.21.15.92:0/3581217075 >> v2:172.21.15.136:3300/0 conn(0x7f98dc0cb060 0x7f98dc06a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 -- 172.21.15.92:0/3581217075 >> v1:172.21.15.136:6789/0 conn(0x7f98dc0872f0 legacy=0x7f98dc0876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 -- 172.21.15.92:0/3581217075 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc069b40 con 0x7f98dc06d4f0 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98db7fe700 1 --2- 172.21.15.92:0/3581217075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc0cab20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f98dc071340 tx=0x7f98d0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a40fd4ad867fbdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98daffd700 1 -- 172.21.15.92:0/3581217075 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000b860 con 0x7f98dc06d4f0 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98daffd700 1 -- 172.21.15.92:0/3581217075 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d000b9c0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f98daffd700 1 -- 172.21.15.92:0/3581217075 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d000bd60 con 0x7f98dc06d4f0 2023-10-06T03:01:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 -- 172.21.15.92:0/2970288780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e81566e0 msgr2=0x7f26e80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2970288780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e81566e0 0x7f26e80ae000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cce20 tx=0x7f26e0009840 comp rx=0 tx=0).stop 2023-10-06T03:01:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 -- 172.21.15.92:0/2970288780 shutdown_connections 2023-10-06T03:01:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2970288780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e80ca300 0x7f26e80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2970288780 >> v2:172.21.15.136:3300/0 conn(0x7f26e8156310 0x7f26e80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2970288780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e81566e0 0x7f26e80ae000 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cce20 tx=0x7f26e0009840 comp rx=0 tx=0).stop 2023-10-06T03:01:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.955+0000 7f26ef33f700 1 -- 172.21.15.92:0/2970288780 >> 172.21.15.92:0/2970288780 conn(0x7f26e80cc1f0 msgr2=0x7f26e80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.956+0000 7f98e1657700 1 -- 172.21.15.92:0/3581217075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 msgr2=0x7f98dc0cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.956+0000 7f98e1657700 1 --2- 172.21.15.92:0/3581217075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc0cab20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f98dc071340 tx=0x7f98d0009ae0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.956+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa618052b10 con 0x7fa6240b1c50 2023-10-06T03:01:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f26ef33f700 1 -- 172.21.15.92:0/2970288780 shutdown_connections 2023-10-06T03:01:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f98e1657700 1 -- 172.21.15.92:0/3581217075 shutdown_connections 2023-10-06T03:01:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f98e1657700 1 --2- 172.21.15.92:0/3581217075 >> v2:172.21.15.136:3300/0 conn(0x7f98dc0cb060 0x7f98dc06a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f98e1657700 1 --2- 172.21.15.92:0/3581217075 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc0cab20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f98e1657700 1 -- 172.21.15.92:0/3581217075 >> 172.21.15.92:0/3581217075 conn(0x7f98dc0cc1f0 msgr2=0x7f98dc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f26ef33f700 1 -- 172.21.15.92:0/2970288780 wait complete. 2023-10-06T03:01:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.958+0000 7f98e1657700 1 -- 172.21.15.92:0/3581217075 shutdown_connections 2023-10-06T03:01:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.959+0000 7f98e1657700 1 -- 172.21.15.92:0/3581217075 wait complete. 2023-10-06T03:01:49.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.960+0000 7f26ef33f700 1 Processor -- start 2023-10-06T03:01:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.960+0000 7fc7b7063700 1 Processor -- start 2023-10-06T03:01:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.960+0000 7fc7b7063700 1 -- start start 2023-10-06T03:01:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.960+0000 7f26ef33f700 1 -- start start 2023-10-06T03:01:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b7063700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc7b0158440 0x7fc7b0068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b7063700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b0068fe0 0x7fc7b00b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f98e1657700 1 Processor -- start 2023-10-06T03:01:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b7063700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b015f260 con 0x7fc7b00b1fe0 2023-10-06T03:01:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b7063700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b00aef20 con 0x7fc7b0158440 2023-10-06T03:01:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b7063700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0069430 con 0x7fc7b0068fe0 2023-10-06T03:01:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f98e1657700 1 -- start start 2023-10-06T03:01:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ef33f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 0x7f26e8154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ef33f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e8156310 0x7f26e8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ef33f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e8140fc0 0x7f26e81453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ef33f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8167b70 con 0x7f26e8156310 2023-10-06T03:01:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ef33f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8141440 con 0x7f26e80ca300 2023-10-06T03:01:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ef33f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8167e10 con 0x7f26e8140fc0 2023-10-06T03:01:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b5600700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fc7b00b1fe0 0x7fc7b01619d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:39082/0 (socket says 172.21.15.92:39082) 2023-10-06T03:01:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b5600700 1 -- 172.21.15.92:0/2022405663 learned_addr learned my addr 172.21.15.92:0/2022405663 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7f26ee33d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 0x7f26e8154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 0x7f26e8154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:43312/0 (socket says 172.21.15.92:43312) 2023-10-06T03:01:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 -- 172.21.15.92:0/2293810927 learned_addr learned my addr 172.21.15.92:0/2293810927 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.961+0000 7fc7b4dff700 1 --2- 172.21.15.92:0/2022405663 >> v2:172.21.15.136:3300/0 conn(0x7fc7b0158440 0x7fc7b0068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7affff700 1 --2- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b0068fe0 0x7fc7b00b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26eeb3e700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e8140fc0 0x7f26e81453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26edb3c700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e8156310 0x7f26e8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7b4dff700 1 -- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b0068fe0 msgr2=0x7fc7b00b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7b4dff700 1 --2- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b0068fe0 0x7fc7b00b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7b4dff700 1 -- 172.21.15.92:0/2022405663 >> v1:172.21.15.136:6789/0 conn(0x7fc7b00b1fe0 legacy=0x7fc7b01619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 -- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e8140fc0 msgr2=0x7f26e81453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e8140fc0 0x7f26e81453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 -- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e8156310 msgr2=0x7f26e8140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e8156310 0x7f26e8140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 -- 172.21.15.92:0/2293810927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0009ae0 con 0x7f26e80ca300 2023-10-06T03:01:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7b4dff700 1 -- 172.21.15.92:0/2022405663 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00b2440 con 0x7fc7b0158440 2023-10-06T03:01:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7b4dff700 1 --2- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b0068aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40098e0 tx=0x7fc7a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df82ab40796c403 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26ee33d700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 0x7f26e8154650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b5e0 tx=0x7f26d80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f98e1657700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc165cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f98e1657700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98dc0872f0 0x7f98dc0b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f98e1657700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98dc0cb060 0x7f98dc0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f98e1657700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0a6d70 con 0x7f98dc06d4f0 2023-10-06T03:01:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f98e1657700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0a6160 con 0x7f98dc0cb060 2023-10-06T03:01:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f98e1657700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0a6440 con 0x7f98dc0872f0 2023-10-06T03:01:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7af7fe700 1 -- 172.21.15.92:0/2022405663 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4011630 con 0x7fc7b0158440 2023-10-06T03:01:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7af7fe700 1 -- 172.21.15.92:0/2022405663 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4011790 con 0x7fc7b0158440 2023-10-06T03:01:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7fc7af7fe700 1 -- 172.21.15.92:0/2022405663 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4011b30 con 0x7fc7b0158440 2023-10-06T03:01:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc165cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc165cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34502/0 (socket says 172.21.15.92:34502) 2023-10-06T03:01:49.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 -- 172.21.15.92:0/3304791601 learned_addr learned my addr 172.21.15.92:0/3304791601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800f510 con 0x7f26e80ca300 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800f690 con 0x7f26e80ca300 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.962+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800fa50 con 0x7f26e80ca300 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e81680f0 con 0x7f26e80ca300 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e81686b0 con 0x7f26e80ca300 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 -- 172.21.15.92:0/3304791601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98dc0872f0 msgr2=0x7f98dc0b1210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98dc0872f0 0x7f98dc0b1210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 -- 172.21.15.92:0/3304791601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98dc0cb060 msgr2=0x7f98dc0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98dc0cb060 0x7f98dc0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 -- 172.21.15.92:0/3304791601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d0009d80 con 0x7f98dc06d4f0 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7fc7b7063700 1 -- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 msgr2=0x7fc7b0068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7fc7b7063700 1 --2- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b0068aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40098e0 tx=0x7fc7a40091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.966+0000 7fc7b7063700 1 -- 172.21.15.92:0/2022405663 shutdown_connections 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.966+0000 7fc7b7063700 1 --2- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b0068aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.966+0000 7fc7b7063700 1 --2- 172.21.15.92:0/2022405663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b0068fe0 0x7fc7b00b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.966+0000 7fc7b7063700 1 -- 172.21.15.92:0/2022405663 >> 172.21.15.92:0/2022405663 conn(0x7fc7b00cd000 msgr2=0x7fc7b0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.965+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26d801a400 con 0x7f26e80ca300 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.965+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e804c3c0 con 0x7f26e80ca300 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.965+0000 7f26de7fc700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26cc066f30 0x7f26cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.965+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26d8084030 con 0x7f26e80ca300 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.965+0000 7f26edb3c700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26cc066f30 0x7f26cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.966+0000 7f26edb3c700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26cc066f30 0x7f26cc0693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cce20 tx=0x7f26e0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.963+0000 7f98dbfff700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc165cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00c590 tx=0x7f98cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.964+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc011590 con 0x7f98dc06d4f0 2023-10-06T03:01:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.964+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc0116f0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.964+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc011ab0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.964+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc0a6720 con 0x7f98dc06d4f0 2023-10-06T03:01:49.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.964+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc1661f0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.965+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc0a7100 con 0x7f98dc06d4f0 2023-10-06T03:01:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.967+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98cc01b3b0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.967+0000 7f98cbfff700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f98c0066f00 0x7f98c00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.967+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f98cc083fd0 con 0x7f98dc06d4f0 2023-10-06T03:01:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.967+0000 7f98db7fe700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f98c0066f00 0x7f98c00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.967+0000 7f98db7fe700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f98c0066f00 0x7f98c00693b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98dc071340 tx=0x7f98d0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.969+0000 7f66b9290700 1 Processor -- start 2023-10-06T03:01:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.969+0000 7f66b9290700 1 -- start start 2023-10-06T03:01:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.969+0000 7f66b9290700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f66b40b2190 0x7f66b406a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7f66b9290700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b4157d50 con 0x7f66b41414b0 2023-10-06T03:01:49.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7f66b9290700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b4157fd0 con 0x7f66b41410e0 2023-10-06T03:01:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7f66b9290700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b41582b0 con 0x7f66b40b2190 2023-10-06T03:01:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7f66b37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f66b41414b0 0x7f66b406a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38960/0 (socket says 172.21.15.92:38960) 2023-10-06T03:01:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7f66b2ffd700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f66b41410e0 0x7f66b40b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:39094/0 (socket says 172.21.15.92:39094) 2023-10-06T03:01:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7f66b37fe700 1 -- 172.21.15.92:0/1653704754 learned_addr learned my addr 172.21.15.92:0/1653704754 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:49.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.970+0000 7fc7b7063700 1 -- 172.21.15.92:0/2022405663 shutdown_connections 2023-10-06T03:01:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7fc7b7063700 1 -- 172.21.15.92:0/2022405663 wait complete. 2023-10-06T03:01:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846690772 0 0) 0x7f66b4157d50 con 0x7f66b41414b0 2023-10-06T03:01:49.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98cc051160 con 0x7f98dc06d4f0 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f669c003570 con 0x7f66b41414b0 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3838227164 0 0) 0x7f669c003570 con 0x7f66b41414b0 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 >> v2:172.21.15.136:3300/0 conn(0x7f66b40b2190 msgr2=0x7f66b406a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 --2- 172.21.15.92:0/1653704754 >> v2:172.21.15.136:3300/0 conn(0x7f66b40b2190 0x7f66b406a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 >> v1:172.21.15.136:6789/0 conn(0x7f66b41410e0 legacy=0x7f66b40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b4158590 con 0x7f66b41414b0 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f66a8001470 con 0x7f66b41414b0 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 >> v1:172.21.15.92:6789/0 conn(0x7f66b41414b0 legacy=0x7f66b406a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f669c003ab0 0x7f669c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f669c004810 0x7f669c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f669c00fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b41582b0 con 0x7f669c0053c0 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0106b0 con 0x7f669c003ab0 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c010870 con 0x7f669c004810 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b37fe700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f669c00fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.971+0000 7f66b2ffd700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f669c003ab0 0x7f669c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 -- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f669c004810 msgr2=0x7f669c004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f669c004810 0x7f669c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 -- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f669c003ab0 msgr2=0x7f669c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f669c003ab0 0x7f669c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 -- 172.21.15.92:0/1653704754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c003570 con 0x7f669c0053c0 2023-10-06T03:01:49.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b2ffd700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f669c003ab0 0x7f669c003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 -- 172.21.15.92:0/1653704754 reap_dead start 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b37fe700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f669c00fe70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c390 tx=0x7f66a8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c22c931cf03d180 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a800cc60 con 0x7f669c0053c0 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a800cde0 con 0x7f669c0053c0 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7f66b1ffb700 1 -- 172.21.15.92:0/1653704754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8005b70 con 0x7f669c0053c0 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7fc7b7063700 1 Processor -- start 2023-10-06T03:01:49.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.972+0000 7fc7b7063700 1 -- start start 2023-10-06T03:01:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7f66b9290700 1 -- 172.21.15.92:0/1653704754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 msgr2=0x7f669c00fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7f66b9290700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f669c00fe70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66a800c390 tx=0x7f66a8001e70 comp rx=0 tx=0).stop 2023-10-06T03:01:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7f66b9290700 1 -- 172.21.15.92:0/1653704754 shutdown_connections 2023-10-06T03:01:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7f66b9290700 1 --2- 172.21.15.92:0/1653704754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f669c00fe70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7f66b9290700 1 -- 172.21.15.92:0/1653704754 >> 172.21.15.92:0/1653704754 conn(0x7f66b4068390 msgr2=0x7f66b4068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26d80511c0 con 0x7f26e80ca300 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b7063700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 0x7fc7b016f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b7063700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b00b1fe0 0x7fc7b01716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b7063700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b015c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b7063700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0171bf0 con 0x7fc7b0068fe0 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b7063700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0155ed0 con 0x7fc7b0158440 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b7063700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b006dfc0 con 0x7fc7b00b1fe0 2023-10-06T03:01:49.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.973+0000 7fc7b4dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 0x7fc7b016f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 0x7fc7b016f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34514/0 (socket says 172.21.15.92:34514) 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 -- 172.21.15.92:0/46600998 learned_addr learned my addr 172.21.15.92:0/46600998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7affff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b00b1fe0 0x7fc7b01716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 -- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b00b1fe0 msgr2=0x7fc7b01716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b5600700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b00b1fe0 0x7fc7b01716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 -- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 msgr2=0x7fc7b015c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b015c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 -- 172.21.15.92:0/46600998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009420 con 0x7fc7b0068fe0 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7affff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b00b1fe0 0x7fc7b01716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b5600700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:49.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.974+0000 7fc7b4dff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 0x7fc7b016f9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40093f0 tx=0x7fc7a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.975+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a401be60 con 0x7fc7b0068fe0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.975+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a4020070 con 0x7fc7b0068fe0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.975+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40276d0 con 0x7fc7b0068fe0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.975+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0155bf0 con 0x7fc7b0068fe0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.975+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b006dc50 con 0x7fc7b0068fe0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.976+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b004c3c0 con 0x7fc7b0068fe0 2023-10-06T03:01:49.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.977+0000 7fe23792c700 1 Processor -- start 2023-10-06T03:01:49.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.977+0000 7f66b9290700 1 -- 172.21.15.92:0/1653704754 shutdown_connections 2023-10-06T03:01:49.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.977+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7a4027a40 con 0x7fc7b0068fe0 2023-10-06T03:01:49.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.977+0000 7fc7acff9700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc794067000 0x7fc7940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.977+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc7a4022070 con 0x7fc7b0068fe0 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.977+0000 7fe23792c700 1 -- start start 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe23792c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe2300cba20 0x7fe2300ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe23792c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe230068aa0 0x7fe230068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe23792c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230156050 0x7fe230156490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe23792c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2300aceb0 con 0x7fe230068aa0 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe23792c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2300b2490 con 0x7fe2300cba20 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe23792c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300ac4e0 con 0x7fe230156050 2023-10-06T03:01:49.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fc7affff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc794067000 0x7fc7940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe235ec9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230156050 0x7fe230156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe235ec9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230156050 0x7fe230156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42676/0 (socket says 172.21.15.92:42676) 2023-10-06T03:01:49.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.978+0000 7fe235ec9700 1 -- 172.21.15.92:0/700962780 learned_addr learned my addr 172.21.15.92:0/700962780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:49.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 --2- 172.21.15.92:0/700962780 >> v2:172.21.15.92:3300/0 conn(0x7fe230068aa0 0x7fe230068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 -- 172.21.15.92:0/700962780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230156050 msgr2=0x7fe230156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 --2- 172.21.15.92:0/700962780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230156050 0x7fe230156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 -- 172.21.15.92:0/700962780 >> v2:172.21.15.136:3300/0 conn(0x7fe2300cba20 msgr2=0x7fe2300ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 --2- 172.21.15.92:0/700962780 >> v2:172.21.15.136:3300/0 conn(0x7fe2300cba20 0x7fe2300ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 -- 172.21.15.92:0/700962780 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230069430 con 0x7fe230068aa0 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fe234ec7700 1 --2- 172.21.15.92:0/700962780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230068aa0 0x7fe230068ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe23006d420 tx=0x7fe22000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c150f1344e6c5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.979+0000 7fc7affff700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc794067000 0x7fc7940694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7b016c3d0 tx=0x7fc79c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:49.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.983+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7a40537d0 con 0x7fc7b0068fe0 2023-10-06T03:01:49.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.987+0000 7f66b9290700 1 -- 172.21.15.92:0/1653704754 wait complete. 2023-10-06T03:01:49.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.989+0000 7f66b9290700 1 Processor -- start 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.992+0000 7f66b9290700 1 -- start start 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b9290700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f66b4139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b9290700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66b4069be0 0x7f66b406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b9290700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66b40c5500 0x7f66b40c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b9290700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b406a590 con 0x7f669c0053c0 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b9290700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40c8280 con 0x7f66b40c5500 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b9290700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40c76b0 con 0x7f66b4069be0 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f66b4139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f66b4139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34518/0 (socket says 172.21.15.92:34518) 2023-10-06T03:01:50.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b27fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66b4069be0 0x7f66b406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b2ffd700 1 -- 172.21.15.92:0/2041006082 learned_addr learned my addr 172.21.15.92:0/2041006082 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b37fe700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66b40c5500 0x7f66b40c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b27fc700 1 -- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66b40c5500 msgr2=0x7f66b40c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b27fc700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66b40c5500 0x7f66b40c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b27fc700 1 -- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 msgr2=0x7f66b4139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b27fc700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f66b4139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7fe227fff700 1 -- 172.21.15.92:0/700962780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000da30 con 0x7fe230068aa0 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7fe227fff700 1 -- 172.21.15.92:0/700962780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000db90 con 0x7fe230068aa0 2023-10-06T03:01:50.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7fe227fff700 1 -- 172.21.15.92:0/700962780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220017420 con 0x7fe230068aa0 2023-10-06T03:01:50.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b27fc700 1 -- 172.21.15.92:0/2041006082 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40c7950 con 0x7f66b4069be0 2023-10-06T03:01:50.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b37fe700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66b40c5500 0x7f66b40c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:50.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.994+0000 7f66b2ffd700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f66b4139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:50.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.995+0000 7fe23792c700 1 -- 172.21.15.92:0/700962780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230068aa0 msgr2=0x7fe230068ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.995+0000 7fe23792c700 1 --2- 172.21.15.92:0/700962780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230068aa0 0x7fe230068ef0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe23006d420 tx=0x7fe22000b430 comp rx=0 tx=0).stop 2023-10-06T03:01:50.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.995+0000 7f66b27fc700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66b4069be0 0x7f66b406a050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f66b40b2790 tx=0x7f66a40090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.995+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400be70 con 0x7f66b4069be0 2023-10-06T03:01:50.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.995+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a400f510 con 0x7f66b4069be0 2023-10-06T03:01:50.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.995+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400f8f0 con 0x7f66b4069be0 2023-10-06T03:01:50.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 -- 172.21.15.92:0/700962780 shutdown_connections 2023-10-06T03:01:50.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 --2- 172.21.15.92:0/700962780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230156050 0x7fe230156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 --2- 172.21.15.92:0/700962780 >> v2:172.21.15.136:3300/0 conn(0x7fe2300cba20 0x7fe2300ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 --2- 172.21.15.92:0/700962780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230068aa0 0x7fe230068ef0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 -- 172.21.15.92:0/700962780 >> 172.21.15.92:0/700962780 conn(0x7fe2300cd000 msgr2=0x7fe230068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40c7c30 con 0x7f66b4069be0 2023-10-06T03:01:50.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b413a530 con 0x7f66b4069be0 2023-10-06T03:01:50.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 -- 172.21.15.92:0/700962780 shutdown_connections 2023-10-06T03:01:50.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.996+0000 7fe23792c700 1 -- 172.21.15.92:0/700962780 wait complete. 2023-10-06T03:01:50.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.998+0000 7fe23792c700 1 Processor -- start 2023-10-06T03:01:50.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.998+0000 7fe23792c700 1 -- start start 2023-10-06T03:01:50.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.999+0000 7fe23792c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 0x7fe23014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.999+0000 7fe23792c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2300cba20 0x7fe2300ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.999+0000 7fe23792c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230156050 0x7fe2300a8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.999+0000 7fe23792c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230150e70 con 0x7fe230156050 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.999+0000 7fe23792c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301516d0 con 0x7fe2300cba20 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:49.999+0000 7fe23792c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23014d510 con 0x7fe230068aa0 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 0x7fe23014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 0x7fe23014cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42688/0 (socket says 172.21.15.92:42688) 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 -- 172.21.15.92:0/3591823115 learned_addr learned my addr 172.21.15.92:0/3591823115 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe235ec9700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230156050 0x7fe2300a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 -- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2300cba20 msgr2=0x7fe2300ca4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:01:50.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2300cba20 0x7fe2300ca4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 -- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230156050 msgr2=0x7fe2300a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230156050 0x7fe2300a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.000+0000 7fe2356c8700 1 -- 172.21.15.92:0/3591823115 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22000b6d0 con 0x7fe230068aa0 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.001+0000 7fe2356c8700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 0x7fe23014cfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b810 tx=0x7fe22c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b40c8410 con 0x7f66b4069be0 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f66a400fca0 con 0x7f66b4069be0 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7f669affd700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f66a0066f30 0x7f66a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.003+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f66a4083f40 con 0x7f66b4069be0 2023-10-06T03:01:50.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00f510 con 0x7fe230068aa0 2023-10-06T03:01:50.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c00f690 con 0x7fe230068aa0 2023-10-06T03:01:50.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00fa50 con 0x7fe230068aa0 2023-10-06T03:01:50.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2300cae40 con 0x7fe230068aa0 2023-10-06T03:01:50.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.002+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe230151360 con 0x7fe230068aa0 2023-10-06T03:01:50.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.004+0000 7f66b2ffd700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f66a0066f30 0x7f66a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.004+0000 7f66b2ffd700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f66a0066f30 0x7f66a00693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0072e0 tx=0x7f66ac00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.005+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2100052b0 con 0x7fe230068aa0 2023-10-06T03:01:50.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.007+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66a40510d0 con 0x7f66b4069be0 2023-10-06T03:01:50.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.013+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe22c024430 con 0x7fe230068aa0 2023-10-06T03:01:50.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.013+0000 7fe2257fa700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe214067080 0x7fe214069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.014+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe22c084d90 con 0x7fe230068aa0 2023-10-06T03:01:50.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.014+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe22c085220 con 0x7fe230068aa0 2023-10-06T03:01:50.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.014+0000 7fe234ec7700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe214067080 0x7fe214069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.015+0000 7fe234ec7700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe214067080 0x7fe214069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ab4f0 tx=0x7fe22001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.034+0000 7f24e5108700 1 Processor -- start 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.034+0000 7f24e5108700 1 -- start start 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.035+0000 7f24e5108700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00875d0 0x7f24e00879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.035+0000 7f24e5108700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e0068aa0 0x7f24e0068f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.035+0000 7f24e5108700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e00acdf0 con 0x7f24e00b1c50 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.035+0000 7f24e5108700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e00abfd0 con 0x7f24e0068aa0 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.035+0000 7f24e5108700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0069460 con 0x7f24e00875d0 2023-10-06T03:01:50.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.036+0000 7f24de59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24e00b1c50 0x7f24e00ab8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38962/0 (socket says 172.21.15.92:38962) 2023-10-06T03:01:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.036+0000 7f24de59c700 1 -- 172.21.15.92:0/2974511418 learned_addr learned my addr 172.21.15.92:0/2974511418 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.036+0000 7f24df59e700 1 --2- 172.21.15.92:0/2974511418 >> v2:172.21.15.92:3300/0 conn(0x7f24e0068aa0 0x7f24e0068f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223351962 0 0) 0x7f24e00acdf0 con 0x7f24e00b1c50 2023-10-06T03:01:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c8003550 con 0x7f24e00b1c50 2023-10-06T03:01:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3075870270 0 0) 0x7f24c8003550 con 0x7f24e00b1c50 2023-10-06T03:01:50.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00875d0 msgr2=0x7f24e00879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.036+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00875d0 0x7f24e00879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00875d0 0x7f24e00879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 >> v2:172.21.15.92:3300/0 conn(0x7f24e0068aa0 msgr2=0x7f24e0068f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/2974511418 >> v2:172.21.15.92:3300/0 conn(0x7f24e0068aa0 0x7f24e0068f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00ac1e0 con 0x7f24e00b1c50 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f24d80017f0 con 0x7f24e00b1c50 2023-10-06T03:01:50.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 >> v1:172.21.15.92:6789/0 conn(0x7f24e00b1c50 legacy=0x7f24e00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8003d20 0x7f24c8004130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24c8004790 0x7f24c800b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c800b6a0 0x7f24c800db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00abfd0 con 0x7f24c8003d20 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0069460 con 0x7f24c800b6a0 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.037+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c800e460 con 0x7f24c8004790 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.038+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00875d0 0x7f24e00879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:50.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.038+0000 7f24df59e700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c800b6a0 0x7f24c800db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.038+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8003d20 0x7f24c8004130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.038+0000 7f24de59c700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24c8004790 0x7f24c800b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ded9d700 1 -- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24c8004790 msgr2=0x7f24c800b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:50.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24c8004790 0x7f24c800b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ded9d700 1 -- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c800b6a0 msgr2=0x7f24c800db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24c800b6a0 0x7f24c800db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ded9d700 1 -- 172.21.15.92:0/2974511418 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00ac1e0 con 0x7f24c8003d20 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8003d20 0x7f24c8004130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f24e0070ec0 tx=0x7f24d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8f200f7cc7e710 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40113f0 con 0x7f24c8003d20 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d4011570 con 0x7f24c8003d20 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.039+0000 7f24ddd9b700 1 -- 172.21.15.92:0/2974511418 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4011930 con 0x7f24c8003d20 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.040+0000 7f24e5108700 1 -- 172.21.15.92:0/2974511418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8003d20 msgr2=0x7f24c8004130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.040+0000 7f24e5108700 1 --2- 172.21.15.92:0/2974511418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24c8003d20 0x7f24c8004130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f24e0070ec0 tx=0x7f24d4008040 comp rx=0 tx=0).stop 2023-10-06T03:01:50.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.046+0000 7f24df59e700 1 -- 172.21.15.92:0/2974511418 reap_dead start 2023-10-06T03:01:50.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.046+0000 7f24e5108700 1 -- 172.21.15.92:0/2974511418 shutdown_connections 2023-10-06T03:01:50.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.046+0000 7f24e5108700 1 -- 172.21.15.92:0/2974511418 >> 172.21.15.92:0/2974511418 conn(0x7f24e00cd000 msgr2=0x7f24e0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.046+0000 7f24e5108700 1 -- 172.21.15.92:0/2974511418 shutdown_connections 2023-10-06T03:01:50.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.046+0000 7f24e5108700 1 -- 172.21.15.92:0/2974511418 wait complete. 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.048+0000 7f24e5108700 1 Processor -- start 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.048+0000 7f24e5108700 1 -- start start 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.049+0000 7f24e5108700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00a5940 0x7f24e0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.049+0000 7f24e5108700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 0x7f24e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.049+0000 7f24e5108700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0167550 0x7f24e01679c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.049+0000 7f24e5108700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0152580 con 0x7f24e00a5d10 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.049+0000 7f24e5108700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e0167f00 con 0x7f24e0167550 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.049+0000 7f24e5108700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e01533a0 con 0x7f24e00a5940 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 0x7f24e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 0x7f24e00a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34550/0 (socket says 172.21.15.92:34550) 2023-10-06T03:01:50.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 -- 172.21.15.92:0/651613749 learned_addr learned my addr 172.21.15.92:0/651613749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 -- 172.21.15.92:0/651613749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00a5940 msgr2=0x7f24e0152040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00a5940 0x7f24e0152040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 -- 172.21.15.92:0/651613749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0167550 msgr2=0x7f24e01679c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0167550 0x7f24e01679c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 -- 172.21.15.92:0/651613749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0152790 con 0x7f24e00a5d10 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.050+0000 7f24de59c700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 0x7f24e00a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24d800b400 tx=0x7f24d8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.051+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8007380 con 0x7f24e00a5d10 2023-10-06T03:01:50.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.051+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d800d050 con 0x7f24e00a5d10 2023-10-06T03:01:50.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.051+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d80076f0 con 0x7f24e00a5d10 2023-10-06T03:01:50.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.051+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e0152a70 con 0x7f24e00a5d10 2023-10-06T03:01:50.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.051+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e0153030 con 0x7f24e00a5d10 2023-10-06T03:01:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.055+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24d80123d0 con 0x7f24e00a5d10 2023-10-06T03:01:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.055+0000 7f24c6ffd700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c806d030 0x7f24c806f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.055+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f24d8084ef0 con 0x7f24e00a5d10 2023-10-06T03:01:50.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.057+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc0052e0 con 0x7f24e00a5d10 2023-10-06T03:01:50.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.063+0000 7f24ded9d700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c806d030 0x7f24c806f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.063+0000 7f24ded9d700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c806d030 0x7f24c806f4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24d400a4b0 tx=0x7f24d4019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.066+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24d804c5a0 con 0x7f24e00a5d10 2023-10-06T03:01:50.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.316+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f187c005590 con 0x7f1890068aa0 2023-10-06T03:01:50.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.316+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa6240b0d00 con 0x7fa6240b1c50 2023-10-06T03:01:50.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.316+0000 7fa60b7fe700 1 -- 172.21.15.92:0/3305093919 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa618028090 con 0x7fa6240b1c50 2023-10-06T03:01:50.318 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-10-06T03:01:50.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.318+0000 7f188cff9700 1 -- 172.21.15.92:0/3938563811 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1884054810 con 0x7f1890068aa0 2023-10-06T03:01:50.321 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:50.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.321+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18780671e0 msgr2=0x7f1878069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.321+0000 7f1896f86700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18780671e0 0x7f1878069690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18901643f0 tx=0x7f1880008040 comp rx=0 tx=0).stop 2023-10-06T03:01:50.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.321+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 msgr2=0x7f1890176910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.321+0000 7f1896f86700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890176910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f188400c590 tx=0x7f188400c950 comp rx=0 tx=0).stop 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa610066f30 msgr2=0x7fa6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa610066f30 0x7fa6100693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa61c007fd0 tx=0x7fa61c007e30 comp rx=0 tx=0).stop 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 msgr2=0x7fa624151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 0x7fa624151c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa62406f950 tx=0x7fa61801c000 comp rx=0 tx=0).stop 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 shutdown_connections 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7f1896f86700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18780671e0 0x7f1878069690 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7f1896f86700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1890068aa0 0x7f1890176910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7f1896f86700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18900b1c50 0x7f18900a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7f1896f86700 1 --2- 172.21.15.92:0/3938563811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18900875d0 0x7f18900a6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 >> 172.21.15.92:0/3938563811 conn(0x7f18900cd000 msgr2=0x7f189006e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 shutdown_connections 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa610066f30 0x7fa6100693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa624068aa0 0x7fa624152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6240b1c50 0x7fa624151c30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 --2- 172.21.15.92:0/3305093919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6240875d0 0x7fa62416cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 >> 172.21.15.92:0/3305093919 conn(0x7fa6240cd000 msgr2=0x7fa624168d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 shutdown_connections 2023-10-06T03:01:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.322+0000 7fa629ccc700 1 -- 172.21.15.92:0/3305093919 wait complete. 2023-10-06T03:01:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.323+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 shutdown_connections 2023-10-06T03:01:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.323+0000 7f1896f86700 1 -- 172.21.15.92:0/3938563811 wait complete. 2023-10-06T03:01:50.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.346+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc7b00acf90 con 0x7fc7b0068fe0 2023-10-06T03:01:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.346+0000 7fc7acff9700 1 -- 172.21.15.92:0/46600998 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7a4056df0 con 0x7fc7b0068fe0 2023-10-06T03:01:50.348 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T03:01:50.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-06T03:01:50.350 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-06T03:01:50.350 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.349+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc794067000 msgr2=0x7fc7940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.349+0000 7fc7b7063700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc794067000 0x7fc7940694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7b016c3d0 tx=0x7fc79c008040 comp rx=0 tx=0).stop 2023-10-06T03:01:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.349+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 msgr2=0x7fc7b016f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.349+0000 7fc7b7063700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 0x7fc7b016f9b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40093f0 tx=0x7fc7a4016000 comp rx=0 tx=0).stop 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 shutdown_connections 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc794067000 0x7fc7940694b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc7b0068fe0 0x7fc7b016f9b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b0158440 0x7fc7b015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 --2- 172.21.15.92:0/46600998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b00b1fe0 0x7fc7b01716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 >> 172.21.15.92:0/46600998 conn(0x7fc7b00cd000 msgr2=0x7fc7b00adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 shutdown_connections 2023-10-06T03:01:50.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.350+0000 7fc7b7063700 1 -- 172.21.15.92:0/46600998 wait complete. 2023-10-06T03:01:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.353+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f98dc04c3c0 con 0x7f98dc06d4f0 2023-10-06T03:01:50.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.353+0000 7f98cbfff700 1 -- 172.21.15.92:0/3304791601 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98cc017090 con 0x7f98dc06d4f0 2023-10-06T03:01:50.355 INFO:teuthology.orchestra.run.smithi092.stdout:55834574856 2023-10-06T03:01:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.355+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f98c0066f00 msgr2=0x7f98c00693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.355+0000 7f98e1657700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f98c0066f00 0x7f98c00693b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98dc071340 tx=0x7f98d0017000 comp rx=0 tx=0).stop 2023-10-06T03:01:50.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.355+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 msgr2=0x7f98dc165cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.355+0000 7f98e1657700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc165cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00c590 tx=0x7f98cc00c950 comp rx=0 tx=0).stop 2023-10-06T03:01:50.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 shutdown_connections 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f98c0066f00 0x7f98c00693b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98dc06d4f0 0x7f98dc165cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98dc0cb060 0x7f98dc0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 --2- 172.21.15.92:0/3304791601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98dc0872f0 0x7f98dc0b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 >> 172.21.15.92:0/3304791601 conn(0x7f98dc0cc1f0 msgr2=0x7f98dc0cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 shutdown_connections 2023-10-06T03:01:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.359+0000 7f98e1657700 1 -- 172.21.15.92:0/3304791601 wait complete. 2023-10-06T03:01:50.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.360+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f26e80cd0b0 con 0x7f26e80ca300 2023-10-06T03:01:50.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.360+0000 7f26de7fc700 1 -- 172.21.15.92:0/2293810927 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26d80547e0 con 0x7f26e80ca300 2023-10-06T03:01:50.365 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:50.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T03:01:50.366 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.365+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26cc066f30 msgr2=0x7f26cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.365+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26cc066f30 0x7f26cc0693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f26e80cce20 tx=0x7f26e0018000 comp rx=0 tx=0).stop 2023-10-06T03:01:50.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.365+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 msgr2=0x7f26e8154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.365+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 0x7f26e8154650 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b5e0 tx=0x7f26d80090f0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 shutdown_connections 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26cc066f30 0x7f26cc0693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26e8140fc0 0x7f26e81453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26e80ca300 0x7f26e8154650 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 --2- 172.21.15.92:0/2293810927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26e8156310 0x7f26e8140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 >> 172.21.15.92:0/2293810927 conn(0x7f26e80cc1f0 msgr2=0x7f26e8069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 shutdown_connections 2023-10-06T03:01:50.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.366+0000 7f26ef33f700 1 -- 172.21.15.92:0/2293810927 wait complete. 2023-10-06T03:01:50.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-10-06T03:01:50.378 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-06T03:01:50.382 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.388+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe210005830 con 0x7fe230068aa0 2023-10-06T03:01:50.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.389+0000 7fe2257fa700 1 -- 172.21.15.92:0/3591823115 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe22c051fb0 con 0x7fe230068aa0 2023-10-06T03:01:50.391 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:50.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe214067080 msgr2=0x7fe214069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe214067080 0x7fe214069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ab4f0 tx=0x7fe22001a000 comp rx=0 tx=0).stop 2023-10-06T03:01:50.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 msgr2=0x7fe23014cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 0x7fe23014cfd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b810 tx=0x7fe22c009250 comp rx=0 tx=0).stop 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 shutdown_connections 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe214067080 0x7fe214069530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe230156050 0x7fe2300a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe230068aa0 0x7fe23014cfd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 --2- 172.21.15.92:0/3591823115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2300cba20 0x7fe2300ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 >> 172.21.15.92:0/3591823115 conn(0x7fe2300cd000 msgr2=0x7fe230153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 shutdown_connections 2023-10-06T03:01:50.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.391+0000 7fe23792c700 1 -- 172.21.15.92:0/3591823115 wait complete. 2023-10-06T03:01:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.401+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66b404c3c0 con 0x7f66b4069be0 2023-10-06T03:01:50.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.402+0000 7f669affd700 1 -- 172.21.15.92:0/2041006082 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66a40546f0 con 0x7f66b4069be0 2023-10-06T03:01:50.403 INFO:teuthology.orchestra.run.smithi092.stdout:55834574854 2023-10-06T03:01:50.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-06T03:01:50.405 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f66a0066f30 msgr2=0x7f66a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f66a0066f30 0x7f66a00693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0072e0 tx=0x7f66ac00c040 comp rx=0 tx=0).stop 2023-10-06T03:01:50.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66b4069be0 msgr2=0x7f66b406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66b4069be0 0x7f66b406a050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f66b40b2790 tx=0x7f66a40090f0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 shutdown_connections 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f66a0066f30 0x7f66a00693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f669c0053c0 0x7f66b4139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66b40c5500 0x7f66b40c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 --2- 172.21.15.92:0/2041006082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66b4069be0 0x7f66b406a050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 >> 172.21.15.92:0/2041006082 conn(0x7f66b4068390 msgr2=0x7f66b40cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 shutdown_connections 2023-10-06T03:01:50.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.404+0000 7f66b9290700 1 -- 172.21.15.92:0/2041006082 wait complete. 2023-10-06T03:01:50.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-06T03:01:50.418 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.430+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24cc005590 con 0x7f24e00a5d10 2023-10-06T03:01:50.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.431+0000 7f24c6ffd700 1 -- 172.21.15.92:0/651613749 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24d8018070 con 0x7f24e00a5d10 2023-10-06T03:01:50.433 INFO:teuthology.orchestra.run.smithi092.stdout:55834574855 2023-10-06T03:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c806d030 msgr2=0x7f24c806f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c806d030 0x7f24c806f4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f24d400a4b0 tx=0x7f24d4019000 comp rx=0 tx=0).stop 2023-10-06T03:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 msgr2=0x7f24e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 0x7f24e00a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f24d800b400 tx=0x7f24d8009040 comp rx=0 tx=0).stop 2023-10-06T03:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 shutdown_connections 2023-10-06T03:01:50.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c806d030 0x7f24c806f4e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0167550 0x7f24e01679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e00a5940 0x7f24e0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 --2- 172.21.15.92:0/651613749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e00a5d10 0x7f24e00a6180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 >> 172.21.15.92:0/651613749 conn(0x7f24e00cd000 msgr2=0x7f24e0165ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 shutdown_connections 2023-10-06T03:01:50.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.433+0000 7f24e5108700 1 -- 172.21.15.92:0/651613749 wait complete. 2023-10-06T03:01:50.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-06T03:01:50.447 DEBUG:teuthology.parallel:result is None 2023-10-06T03:01:50.447 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T03:01:50.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:01:50.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.598+0000 7effdb59e700 1 Processor -- start 2023-10-06T03:01:50.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.598+0000 7effdb59e700 1 -- start start 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdb59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effdc15d230 0x7effdc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdb59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc0b2980 con 0x7effdc159420 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdb59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc15db70 con 0x7effdc1597f0 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdb59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc0b11d0 con 0x7effdc15d230 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effdc15d230 0x7effdc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effda59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7effdc159420 0x7effdc0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38978/0 (socket says 172.21.15.92:38978) 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effda59c700 1 -- 172.21.15.92:0/2018404043 learned_addr learned my addr 172.21.15.92:0/2018404043 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdad9d700 1 -- 172.21.15.92:0/2018404043 >> v1:172.21.15.136:6789/0 conn(0x7effdc1597f0 legacy=0x7effdc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdad9d700 1 -- 172.21.15.92:0/2018404043 >> v1:172.21.15.92:6789/0 conn(0x7effdc159420 legacy=0x7effdc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdad9d700 1 -- 172.21.15.92:0/2018404043 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc0b14b0 con 0x7effdc15d230 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effd9d9b700 1 -- 172.21.15.92:0/2018404043 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175621489 0 0) 0x7effdc0b2980 con 0x7effdc159420 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effdad9d700 1 --2- 172.21.15.92:0/2018404043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc15d230 0x7effdc15d630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effd4009e60 tx=0x7effd4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f18284cd99d7bfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.599+0000 7effd9d9b700 1 -- 172.21.15.92:0/2018404043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400e070 con 0x7effdc15d230 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effd9d9b700 1 -- 172.21.15.92:0/2018404043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd400bbf0 con 0x7effdc15d230 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effd9d9b700 1 -- 172.21.15.92:0/2018404043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4012430 con 0x7effdc15d230 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 -- 172.21.15.92:0/2018404043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc15d230 msgr2=0x7effdc15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 --2- 172.21.15.92:0/2018404043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc15d230 0x7effdc15d630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effd4009e60 tx=0x7effd4009780 comp rx=0 tx=0).stop 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 -- 172.21.15.92:0/2018404043 shutdown_connections 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 --2- 172.21.15.92:0/2018404043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc15d230 0x7effdc15d630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 -- 172.21.15.92:0/2018404043 >> 172.21.15.92:0/2018404043 conn(0x7effdc0d0660 msgr2=0x7effdc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 -- 172.21.15.92:0/2018404043 shutdown_connections 2023-10-06T03:01:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.600+0000 7effdb59e700 1 -- 172.21.15.92:0/2018404043 wait complete. 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.601+0000 7effdb59e700 1 Processor -- start 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.601+0000 7effdb59e700 1 -- start start 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdb59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effdc159420 0x7effdc154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdb59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 0x7effdc159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdb59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effdc15d230 0x7effdc169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdb59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc14f840 con 0x7effdc1597f0 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdb59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15f1c0 con 0x7effdc15d230 2023-10-06T03:01:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdb59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15a1e0 con 0x7effdc159420 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 0x7effdc159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 0x7effdc159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34560/0 (socket says 172.21.15.92:34560) 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 -- 172.21.15.92:0/4091232821 learned_addr learned my addr 172.21.15.92:0/4091232821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effda59c700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effdc159420 0x7effdc154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effdad9d700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effdc15d230 0x7effdc169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 -- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effdc159420 msgr2=0x7effdc154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effdc159420 0x7effdc154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 -- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effdc15d230 msgr2=0x7effdc169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effdc15d230 0x7effdc169770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.602+0000 7effd1d9b700 1 -- 172.21.15.92:0/4091232821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd4009a20 con 0x7effdc1597f0 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.603+0000 7effd1d9b700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 0x7effdc159ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effc400b7e0 tx=0x7effc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.603+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400f510 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.603+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc400f670 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.603+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc155420 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.603+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc400fa10 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.603+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc15eb70 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.604+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc15f330 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.604+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effc401a400 con 0x7effdc1597f0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.604+0000 7effd2ffd700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7effbc066f30 0x7effbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.604+0000 7effda59c700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7effbc066f30 0x7effbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.604+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7effc4084140 con 0x7effdc1597f0 2023-10-06T03:01:50.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.605+0000 7effda59c700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7effbc066f30 0x7effbc0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effcc0011e0 tx=0x7effcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.609+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effc40512d0 con 0x7effdc1597f0 2023-10-06T03:01:50.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.786+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7effdc0b1790 con 0x7effbc066f30 2023-10-06T03:01:50.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.794+0000 7effd2ffd700 1 -- 172.21.15.92:0/4091232821 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286621 (secure 0 0 0) 0x7effdc0b1790 con 0x7effbc066f30 2023-10-06T03:01:50.799 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7effbc066f30 msgr2=0x7effbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7effbc066f30 0x7effbc0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effcc0011e0 tx=0x7effcc009040 comp rx=0 tx=0).stop 2023-10-06T03:01:50.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 msgr2=0x7effdc159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 0x7effdc159ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effc400b7e0 tx=0x7effc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 shutdown_connections 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7effbc066f30 0x7effbc0693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effdc15d230 0x7effdc169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effdc159420 0x7effdc154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 --2- 172.21.15.92:0/4091232821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effdc1597f0 0x7effdc159ca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 >> 172.21.15.92:0/4091232821 conn(0x7effdc0d0660 msgr2=0x7effdc0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 shutdown_connections 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.797+0000 7effdb59e700 1 -- 172.21.15.92:0/4091232821 wait complete. 2023-10-06T03:01:50.802 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:01:50.811 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T03:01:49.184153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.821177"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280436+0000","last_change":"2023-10-06T03:01:42.278647+0000","last_active":"2023-10-06T03:01:42.280436+0000","last_peered":"2023-10-06T03:01:42.280436+0000","last_clean":"2023-10-06T03:01:42.280436+0000","last_became_active":"2023-10-06T03:01:42.278364+0000","last_became_peered":"2023-10-06T03:01:42.278364+0000","last_unstale":"2023-10-06T03:01:42.280436+0000","last_undegraded":"2023-10-06T03:01:42.280436+0000","last_fullsized":"2023-10-06T03:01:42.280436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:06.498462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282043+0000","last_change":"2023-10-06T03:01:42.277062+0000","last_active":"2023-10-06T03:01:42.282043+0000","last_peered":"2023-10-06T03:01:42.282043+0000","last_clean":"2023-10-06T03:01:42.282043+0000","last_became_active":"2023-10-06T03:01:42.276189+0000","last_became_peered":"2023-10-06T03:01:42.276189+0000","last_unstale":"2023-10-06T03:01:42.282043+0000","last_undegraded":"2023-10-06T03:01:42.282043+0000","last_fullsized":"2023-10-06T03:01:42.282043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284329+0000","last_change":"2023-10-06T03:01:42.280530+0000","last_active":"2023-10-06T03:01:42.284329+0000","last_peered":"2023-10-06T03:01:42.284329+0000","last_clean":"2023-10-06T03:01:42.284329+0000","last_became_active":"2023-10-06T03:01:42.280239+0000","last_became_peered":"2023-10-06T03:01:42.280239+0000","last_unstale":"2023-10-06T03:01:42.284329+0000","last_undegraded":"2023-10-06T03:01:42.284329+0000","last_fullsized":"2023-10-06T03:01:42.284329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284290+0000","last_change":"2023-10-06T03:01:42.279732+0000","last_active":"2023-10-06T03:01:42.284290+0000","last_peered":"2023-10-06T03:01:42.284290+0000","last_clean":"2023-10-06T03:01:42.284290+0000","last_became_active":"2023-10-06T03:01:42.279445+0000","last_became_peered":"2023-10-06T03:01:42.279445+0000","last_unstale":"2023-10-06T03:01:42.284290+0000","last_undegraded":"2023-10-06T03:01:42.284290+0000","last_fullsized":"2023-10-06T03:01:42.284290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:08.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285662+0000","last_change":"2023-10-06T03:01:42.279674+0000","last_active":"2023-10-06T03:01:42.285662+0000","last_peered":"2023-10-06T03:01:42.285662+0000","last_clean":"2023-10-06T03:01:42.285662+0000","last_became_active":"2023-10-06T03:01:42.279435+0000","last_became_peered":"2023-10-06T03:01:42.279435+0000","last_unstale":"2023-10-06T03:01:42.285662+0000","last_undegraded":"2023-10-06T03:01:42.285662+0000","last_fullsized":"2023-10-06T03:01:42.285662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284581+0000","last_change":"2023-10-06T03:01:42.282579+0000","last_active":"2023-10-06T03:01:42.284581+0000","last_peered":"2023-10-06T03:01:42.284581+0000","last_clean":"2023-10-06T03:01:42.284581+0000","last_became_active":"2023-10-06T03:01:42.282276+0000","last_became_peered":"2023-10-06T03:01:42.282276+0000","last_unstale":"2023-10-06T03:01:42.284581+0000","last_undegraded":"2023-10-06T03:01:42.284581+0000","last_fullsized":"2023-10-06T03:01:42.284581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286199+0000","last_change":"2023-10-06T03:01:42.282922+0000","last_active":"2023-10-06T03:01:42.286199+0000","last_peered":"2023-10-06T03:01:42.286199+0000","last_clean":"2023-10-06T03:01:42.286199+0000","last_became_active":"2023-10-06T03:01:42.282688+0000","last_became_peered":"2023-10-06T03:01:42.282688+0000","last_unstale":"2023-10-06T03:01:42.286199+0000","last_undegraded":"2023-10-06T03:01:42.286199+0000","last_fullsized":"2023-10-06T03:01:42.286199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285998+0000","last_change":"2023-10-06T03:01:42.279085+0000","last_active":"2023-10-06T03:01:42.285998+0000","last_peered":"2023-10-06T03:01:42.285998+0000","last_clean":"2023-10-06T03:01:42.285998+0000","last_became_active":"2023-10-06T03:01:42.278311+0000","last_became_peered":"2023-10-06T03:01:42.278311+0000","last_unstale":"2023-10-06T03:01:42.285998+0000","last_undegraded":"2023-10-06T03:01:42.285998+0000","last_fullsized":"2023-10-06T03:01:42.285998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285685+0000","last_change":"2023-10-06T03:01:42.280111+0000","last_active":"2023-10-06T03:01:42.285685+0000","last_peered":"2023-10-06T03:01:42.285685+0000","last_clean":"2023-10-06T03:01:42.285685+0000","last_became_active":"2023-10-06T03:01:42.279534+0000","last_became_peered":"2023-10-06T03:01:42.279534+0000","last_unstale":"2023-10-06T03:01:42.285685+0000","last_undegraded":"2023-10-06T03:01:42.285685+0000","last_fullsized":"2023-10-06T03:01:42.285685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286459+0000","last_change":"2023-10-06T03:01:42.281699+0000","last_active":"2023-10-06T03:01:42.286459+0000","last_peered":"2023-10-06T03:01:42.286459+0000","last_clean":"2023-10-06T03:01:42.286459+0000","last_became_active":"2023-10-06T03:01:42.279098+0000","last_became_peered":"2023-10-06T03:01:42.279098+0000","last_unstale":"2023-10-06T03:01:42.286459+0000","last_undegraded":"2023-10-06T03:01:42.286459+0000","last_fullsized":"2023-10-06T03:01:42.286459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283045+0000","last_change":"2023-10-06T03:01:42.277096+0000","last_active":"2023-10-06T03:01:42.283045+0000","last_peered":"2023-10-06T03:01:42.283045+0000","last_clean":"2023-10-06T03:01:42.283045+0000","last_became_active":"2023-10-06T03:01:42.276719+0000","last_became_peered":"2023-10-06T03:01:42.276719+0000","last_unstale":"2023-10-06T03:01:42.283045+0000","last_undegraded":"2023-10-06T03:01:42.283045+0000","last_fullsized":"2023-10-06T03:01:42.283045+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:13.143094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275038+0000","last_change":"2023-10-06T03:01:42.272031+0000","last_active":"2023-10-06T03:01:42.275038+0000","last_peered":"2023-10-06T03:01:42.275038+0000","last_clean":"2023-10-06T03:01:42.275038+0000","last_became_active":"2023-10-06T03:01:42.271796+0000","last_became_peered":"2023-10-06T03:01:42.271796+0000","last_unstale":"2023-10-06T03:01:42.275038+0000","last_undegraded":"2023-10-06T03:01:42.275038+0000","last_fullsized":"2023-10-06T03:01:42.275038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281109+0000","last_change":"2023-10-06T03:01:42.279147+0000","last_active":"2023-10-06T03:01:42.281109+0000","last_peered":"2023-10-06T03:01:42.281109+0000","last_clean":"2023-10-06T03:01:42.281109+0000","last_became_active":"2023-10-06T03:01:42.278911+0000","last_became_peered":"2023-10-06T03:01:42.278911+0000","last_unstale":"2023-10-06T03:01:42.281109+0000","last_undegraded":"2023-10-06T03:01:42.281109+0000","last_fullsized":"2023-10-06T03:01:42.281109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:14.170785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286021+0000","last_change":"2023-10-06T03:01:42.277711+0000","last_active":"2023-10-06T03:01:42.286021+0000","last_peered":"2023-10-06T03:01:42.286021+0000","last_clean":"2023-10-06T03:01:42.286021+0000","last_became_active":"2023-10-06T03:01:42.276996+0000","last_became_peered":"2023-10-06T03:01:42.276996+0000","last_unstale":"2023-10-06T03:01:42.286021+0000","last_undegraded":"2023-10-06T03:01:42.286021+0000","last_fullsized":"2023-10-06T03:01:42.286021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.289013+0000","last_change":"2023-10-06T03:01:42.288226+0000","last_active":"2023-10-06T03:01:42.289013+0000","last_peered":"2023-10-06T03:01:42.289013+0000","last_clean":"2023-10-06T03:01:42.289013+0000","last_became_active":"2023-10-06T03:01:42.287944+0000","last_became_peered":"2023-10-06T03:01:42.287944+0000","last_unstale":"2023-10-06T03:01:42.289013+0000","last_undegraded":"2023-10-06T03:01:42.289013+0000","last_fullsized":"2023-10-06T03:01:42.289013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284504+0000","last_change":"2023-10-06T03:01:42.281156+0000","last_active":"2023-10-06T03:01:42.284504+0000","last_peered":"2023-10-06T03:01:42.284504+0000","last_clean":"2023-10-06T03:01:42.284504+0000","last_became_active":"2023-10-06T03:01:42.280924+0000","last_became_peered":"2023-10-06T03:01:42.280924+0000","last_unstale":"2023-10-06T03:01:42.284504+0000","last_undegraded":"2023-10-06T03:01:42.284504+0000","last_fullsized":"2023-10-06T03:01:42.284504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285706+0000","last_change":"2023-10-06T03:01:42.280272+0000","last_active":"2023-10-06T03:01:42.285706+0000","last_peered":"2023-10-06T03:01:42.285706+0000","last_clean":"2023-10-06T03:01:42.285706+0000","last_became_active":"2023-10-06T03:01:42.279857+0000","last_became_peered":"2023-10-06T03:01:42.279857+0000","last_unstale":"2023-10-06T03:01:42.285706+0000","last_undegraded":"2023-10-06T03:01:42.285706+0000","last_fullsized":"2023-10-06T03:01:42.285706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284102+0000","last_change":"2023-10-06T03:01:42.278122+0000","last_active":"2023-10-06T03:01:42.284102+0000","last_peered":"2023-10-06T03:01:42.284102+0000","last_clean":"2023-10-06T03:01:42.284102+0000","last_became_active":"2023-10-06T03:01:42.277860+0000","last_became_peered":"2023-10-06T03:01:42.277860+0000","last_unstale":"2023-10-06T03:01:42.284102+0000","last_undegraded":"2023-10-06T03:01:42.284102+0000","last_fullsized":"2023-10-06T03:01:42.284102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:23.395145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283307+0000","last_change":"2023-10-06T03:01:42.281950+0000","last_active":"2023-10-06T03:01:42.283307+0000","last_peered":"2023-10-06T03:01:42.283307+0000","last_clean":"2023-10-06T03:01:42.283307+0000","last_became_active":"2023-10-06T03:01:42.281696+0000","last_became_peered":"2023-10-06T03:01:42.281696+0000","last_unstale":"2023-10-06T03:01:42.283307+0000","last_undegraded":"2023-10-06T03:01:42.283307+0000","last_fullsized":"2023-10-06T03:01:42.283307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:22.656089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284642+0000","last_change":"2023-10-06T03:01:42.281651+0000","last_active":"2023-10-06T03:01:42.284642+0000","last_peered":"2023-10-06T03:01:42.284642+0000","last_clean":"2023-10-06T03:01:42.284642+0000","last_became_active":"2023-10-06T03:01:42.278739+0000","last_became_peered":"2023-10-06T03:01:42.278739+0000","last_unstale":"2023-10-06T03:01:42.284642+0000","last_undegraded":"2023-10-06T03:01:42.284642+0000","last_fullsized":"2023-10-06T03:01:42.284642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:22.011159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286267+0000","last_change":"2023-10-06T03:01:42.283331+0000","last_active":"2023-10-06T03:01:42.286267+0000","last_peered":"2023-10-06T03:01:42.286267+0000","last_clean":"2023-10-06T03:01:42.286267+0000","last_became_active":"2023-10-06T03:01:42.283048+0000","last_became_peered":"2023-10-06T03:01:42.283048+0000","last_unstale":"2023-10-06T03:01:42.286267+0000","last_undegraded":"2023-10-06T03:01:42.286267+0000","last_fullsized":"2023-10-06T03:01:42.286267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285977+0000","last_change":"2023-10-06T03:01:42.282637+0000","last_active":"2023-10-06T03:01:42.285977+0000","last_peered":"2023-10-06T03:01:42.285977+0000","last_clean":"2023-10-06T03:01:42.285977+0000","last_became_active":"2023-10-06T03:01:42.282269+0000","last_became_peered":"2023-10-06T03:01:42.282269+0000","last_unstale":"2023-10-06T03:01:42.285977+0000","last_undegraded":"2023-10-06T03:01:42.285977+0000","last_fullsized":"2023-10-06T03:01:42.285977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281082+0000","last_change":"2023-10-06T03:01:42.279886+0000","last_active":"2023-10-06T03:01:42.281082+0000","last_peered":"2023-10-06T03:01:42.281082+0000","last_clean":"2023-10-06T03:01:42.281082+0000","last_became_active":"2023-10-06T03:01:42.279246+0000","last_became_peered":"2023-10-06T03:01:42.279246+0000","last_unstale":"2023-10-06T03:01:42.281082+0000","last_undegraded":"2023-10-06T03:01:42.281082+0000","last_fullsized":"2023-10-06T03:01:42.281082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286308+0000","last_change":"2023-10-06T03:01:42.283900+0000","last_active":"2023-10-06T03:01:42.286308+0000","last_peered":"2023-10-06T03:01:42.286308+0000","last_clean":"2023-10-06T03:01:42.286308+0000","last_became_active":"2023-10-06T03:01:42.283581+0000","last_became_peered":"2023-10-06T03:01:42.283581+0000","last_unstale":"2023-10-06T03:01:42.286308+0000","last_undegraded":"2023-10-06T03:01:42.286308+0000","last_fullsized":"2023-10-06T03:01:42.286308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287866+0000","last_change":"2023-10-06T03:01:42.277988+0000","last_active":"2023-10-06T03:01:42.287866+0000","last_peered":"2023-10-06T03:01:42.287866+0000","last_clean":"2023-10-06T03:01:42.287866+0000","last_became_active":"2023-10-06T03:01:42.277260+0000","last_became_peered":"2023-10-06T03:01:42.277260+0000","last_unstale":"2023-10-06T03:01:42.287866+0000","last_undegraded":"2023-10-06T03:01:42.287866+0000","last_fullsized":"2023-10-06T03:01:42.287866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287998+0000","last_change":"2023-10-06T03:01:42.280682+0000","last_active":"2023-10-06T03:01:42.287998+0000","last_peered":"2023-10-06T03:01:42.287998+0000","last_clean":"2023-10-06T03:01:42.287998+0000","last_became_active":"2023-10-06T03:01:42.280427+0000","last_became_peered":"2023-10-06T03:01:42.280427+0000","last_unstale":"2023-10-06T03:01:42.287998+0000","last_undegraded":"2023-10-06T03:01:42.287998+0000","last_fullsized":"2023-10-06T03:01:42.287998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:05.130474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286079+0000","last_change":"2023-10-06T03:01:42.280012+0000","last_active":"2023-10-06T03:01:42.286079+0000","last_peered":"2023-10-06T03:01:42.286079+0000","last_clean":"2023-10-06T03:01:42.286079+0000","last_became_active":"2023-10-06T03:01:42.279744+0000","last_became_peered":"2023-10-06T03:01:42.279744+0000","last_unstale":"2023-10-06T03:01:42.286079+0000","last_undegraded":"2023-10-06T03:01:42.286079+0000","last_fullsized":"2023-10-06T03:01:42.286079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373578+0000","last_change":"2023-10-06T03:01:41.380874+0000","last_active":"2023-10-06T03:01:42.373578+0000","last_peered":"2023-10-06T03:01:42.373578+0000","last_clean":"2023-10-06T03:01:42.373578+0000","last_became_active":"2023-10-06T03:01:41.379061+0000","last_became_peered":"2023-10-06T03:01:41.379061+0000","last_unstale":"2023-10-06T03:01:42.373578+0000","last_undegraded":"2023-10-06T03:01:42.373578+0000","last_fullsized":"2023-10-06T03:01:42.373578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280483+0000","last_change":"2023-10-06T03:01:42.276652+0000","last_active":"2023-10-06T03:01:42.280483+0000","last_peered":"2023-10-06T03:01:42.280483+0000","last_clean":"2023-10-06T03:01:42.280483+0000","last_became_active":"2023-10-06T03:01:42.276378+0000","last_became_peered":"2023-10-06T03:01:42.276378+0000","last_unstale":"2023-10-06T03:01:42.280483+0000","last_undegraded":"2023-10-06T03:01:42.280483+0000","last_fullsized":"2023-10-06T03:01:42.280483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270139+0000","last_change":"2023-10-06T03:01:41.379177+0000","last_active":"2023-10-06T03:01:42.270139+0000","last_peered":"2023-10-06T03:01:42.270139+0000","last_clean":"2023-10-06T03:01:42.270139+0000","last_became_active":"2023-10-06T03:01:41.378913+0000","last_became_peered":"2023-10-06T03:01:41.378913+0000","last_unstale":"2023-10-06T03:01:42.270139+0000","last_undegraded":"2023-10-06T03:01:42.270139+0000","last_fullsized":"2023-10-06T03:01:42.270139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286002+0000","last_change":"2023-10-06T03:01:42.284965+0000","last_active":"2023-10-06T03:01:42.286002+0000","last_peered":"2023-10-06T03:01:42.286002+0000","last_clean":"2023-10-06T03:01:42.286002+0000","last_became_active":"2023-10-06T03:01:42.284715+0000","last_became_peered":"2023-10-06T03:01:42.284715+0000","last_unstale":"2023-10-06T03:01:42.286002+0000","last_undegraded":"2023-10-06T03:01:42.286002+0000","last_fullsized":"2023-10-06T03:01:42.286002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268732+0000","last_change":"2023-10-06T03:01:41.383082+0000","last_active":"2023-10-06T03:01:42.268732+0000","last_peered":"2023-10-06T03:01:42.268732+0000","last_clean":"2023-10-06T03:01:42.268732+0000","last_became_active":"2023-10-06T03:01:41.382776+0000","last_became_peered":"2023-10-06T03:01:41.382776+0000","last_unstale":"2023-10-06T03:01:42.268732+0000","last_undegraded":"2023-10-06T03:01:42.268732+0000","last_fullsized":"2023-10-06T03:01:42.268732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285524+0000","last_change":"2023-10-06T03:01:42.283247+0000","last_active":"2023-10-06T03:01:42.285524+0000","last_peered":"2023-10-06T03:01:42.285524+0000","last_clean":"2023-10-06T03:01:42.285524+0000","last_became_active":"2023-10-06T03:01:42.282951+0000","last_became_peered":"2023-10-06T03:01:42.282951+0000","last_unstale":"2023-10-06T03:01:42.285524+0000","last_undegraded":"2023-10-06T03:01:42.285524+0000","last_fullsized":"2023-10-06T03:01:42.285524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269263+0000","last_change":"2023-10-06T03:01:41.381863+0000","last_active":"2023-10-06T03:01:42.269263+0000","last_peered":"2023-10-06T03:01:42.269263+0000","last_clean":"2023-10-06T03:01:42.269263+0000","last_became_active":"2023-10-06T03:01:41.381602+0000","last_became_peered":"2023-10-06T03:01:41.381602+0000","last_unstale":"2023-10-06T03:01:42.269263+0000","last_undegraded":"2023-10-06T03:01:42.269263+0000","last_fullsized":"2023-10-06T03:01:42.269263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280585+0000","last_change":"2023-10-06T03:01:42.276543+0000","last_active":"2023-10-06T03:01:42.280585+0000","last_peered":"2023-10-06T03:01:42.280585+0000","last_clean":"2023-10-06T03:01:42.280585+0000","last_became_active":"2023-10-06T03:01:42.276256+0000","last_became_peered":"2023-10-06T03:01:42.276256+0000","last_unstale":"2023-10-06T03:01:42.280585+0000","last_undegraded":"2023-10-06T03:01:42.280585+0000","last_fullsized":"2023-10-06T03:01:42.280585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267847+0000","last_change":"2023-10-06T03:01:41.378556+0000","last_active":"2023-10-06T03:01:42.267847+0000","last_peered":"2023-10-06T03:01:42.267847+0000","last_clean":"2023-10-06T03:01:42.267847+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.267847+0000","last_undegraded":"2023-10-06T03:01:42.267847+0000","last_fullsized":"2023-10-06T03:01:42.267847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:30.948926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287982+0000","last_change":"2023-10-06T03:01:42.283975+0000","last_active":"2023-10-06T03:01:42.287982+0000","last_peered":"2023-10-06T03:01:42.287982+0000","last_clean":"2023-10-06T03:01:42.287982+0000","last_became_active":"2023-10-06T03:01:42.283734+0000","last_became_peered":"2023-10-06T03:01:42.283734+0000","last_unstale":"2023-10-06T03:01:42.287982+0000","last_undegraded":"2023-10-06T03:01:42.287982+0000","last_fullsized":"2023-10-06T03:01:42.287982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.374949+0000","last_change":"2023-10-06T03:01:41.384246+0000","last_active":"2023-10-06T03:01:42.374949+0000","last_peered":"2023-10-06T03:01:42.374949+0000","last_clean":"2023-10-06T03:01:42.374949+0000","last_became_active":"2023-10-06T03:01:41.383977+0000","last_became_peered":"2023-10-06T03:01:41.383977+0000","last_unstale":"2023-10-06T03:01:42.374949+0000","last_undegraded":"2023-10-06T03:01:42.374949+0000","last_fullsized":"2023-10-06T03:01:42.374949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.372963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287980+0000","last_change":"2023-10-06T03:01:42.280871+0000","last_active":"2023-10-06T03:01:42.287980+0000","last_peered":"2023-10-06T03:01:42.287980+0000","last_clean":"2023-10-06T03:01:42.287980+0000","last_became_active":"2023-10-06T03:01:42.277563+0000","last_became_peered":"2023-10-06T03:01:42.277563+0000","last_unstale":"2023-10-06T03:01:42.287980+0000","last_undegraded":"2023-10-06T03:01:42.287980+0000","last_fullsized":"2023-10-06T03:01:42.287980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.379545+0000","last_change":"2023-10-06T03:01:41.381401+0000","last_active":"2023-10-06T03:01:42.379545+0000","last_peered":"2023-10-06T03:01:42.379545+0000","last_clean":"2023-10-06T03:01:42.379545+0000","last_became_active":"2023-10-06T03:01:41.379277+0000","last_became_peered":"2023-10-06T03:01:41.379277+0000","last_unstale":"2023-10-06T03:01:42.379545+0000","last_undegraded":"2023-10-06T03:01:42.379545+0000","last_fullsized":"2023-10-06T03:01:42.379545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284666+0000","last_change":"2023-10-06T03:01:42.281115+0000","last_active":"2023-10-06T03:01:42.284666+0000","last_peered":"2023-10-06T03:01:42.284666+0000","last_clean":"2023-10-06T03:01:42.284666+0000","last_became_active":"2023-10-06T03:01:42.278708+0000","last_became_peered":"2023-10-06T03:01:42.278708+0000","last_unstale":"2023-10-06T03:01:42.284666+0000","last_undegraded":"2023-10-06T03:01:42.284666+0000","last_fullsized":"2023-10-06T03:01:42.284666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.231633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377978+0000","last_change":"2023-10-06T03:01:41.384981+0000","last_active":"2023-10-06T03:01:42.377978+0000","last_peered":"2023-10-06T03:01:42.377978+0000","last_clean":"2023-10-06T03:01:42.377978+0000","last_became_active":"2023-10-06T03:01:41.384739+0000","last_became_peered":"2023-10-06T03:01:41.384739+0000","last_unstale":"2023-10-06T03:01:42.377978+0000","last_undegraded":"2023-10-06T03:01:42.377978+0000","last_fullsized":"2023-10-06T03:01:42.377978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280582+0000","last_change":"2023-10-06T03:01:42.276570+0000","last_active":"2023-10-06T03:01:42.280582+0000","last_peered":"2023-10-06T03:01:42.280582+0000","last_clean":"2023-10-06T03:01:42.280582+0000","last_became_active":"2023-10-06T03:01:42.276163+0000","last_became_peered":"2023-10-06T03:01:42.276163+0000","last_unstale":"2023-10-06T03:01:42.280582+0000","last_undegraded":"2023-10-06T03:01:42.280582+0000","last_fullsized":"2023-10-06T03:01:42.280582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.272049+0000","last_change":"2023-10-06T03:01:41.381162+0000","last_active":"2023-10-06T03:01:42.272049+0000","last_peered":"2023-10-06T03:01:42.272049+0000","last_clean":"2023-10-06T03:01:42.272049+0000","last_became_active":"2023-10-06T03:01:41.380885+0000","last_became_peered":"2023-10-06T03:01:41.380885+0000","last_unstale":"2023-10-06T03:01:42.272049+0000","last_undegraded":"2023-10-06T03:01:42.272049+0000","last_fullsized":"2023-10-06T03:01:42.272049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286056+0000","last_change":"2023-10-06T03:01:42.279843+0000","last_active":"2023-10-06T03:01:42.286056+0000","last_peered":"2023-10-06T03:01:42.286056+0000","last_clean":"2023-10-06T03:01:42.286056+0000","last_became_active":"2023-10-06T03:01:42.279409+0000","last_became_peered":"2023-10-06T03:01:42.279409+0000","last_unstale":"2023-10-06T03:01:42.286056+0000","last_undegraded":"2023-10-06T03:01:42.286056+0000","last_fullsized":"2023-10-06T03:01:42.286056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275415+0000","last_change":"2023-10-06T03:01:41.382403+0000","last_active":"2023-10-06T03:01:42.275415+0000","last_peered":"2023-10-06T03:01:42.275415+0000","last_clean":"2023-10-06T03:01:42.275415+0000","last_became_active":"2023-10-06T03:01:41.382119+0000","last_became_peered":"2023-10-06T03:01:41.382119+0000","last_unstale":"2023-10-06T03:01:42.275415+0000","last_undegraded":"2023-10-06T03:01:42.275415+0000","last_fullsized":"2023-10-06T03:01:42.275415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:17.777678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285768+0000","last_change":"2023-10-06T03:01:42.280092+0000","last_active":"2023-10-06T03:01:42.285768+0000","last_peered":"2023-10-06T03:01:42.285768+0000","last_clean":"2023-10-06T03:01:42.285768+0000","last_became_active":"2023-10-06T03:01:42.279833+0000","last_became_peered":"2023-10-06T03:01:42.279833+0000","last_unstale":"2023-10-06T03:01:42.285768+0000","last_undegraded":"2023-10-06T03:01:42.285768+0000","last_fullsized":"2023-10-06T03:01:42.285768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.375608+0000","last_change":"2023-10-06T03:01:41.386282+0000","last_active":"2023-10-06T03:01:42.375608+0000","last_peered":"2023-10-06T03:01:42.375608+0000","last_clean":"2023-10-06T03:01:42.375608+0000","last_became_active":"2023-10-06T03:01:41.385955+0000","last_became_peered":"2023-10-06T03:01:41.385955+0000","last_unstale":"2023-10-06T03:01:42.375608+0000","last_undegraded":"2023-10-06T03:01:42.375608+0000","last_fullsized":"2023-10-06T03:01:42.375608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287940+0000","last_change":"2023-10-06T03:01:42.283470+0000","last_active":"2023-10-06T03:01:42.287940+0000","last_peered":"2023-10-06T03:01:42.287940+0000","last_clean":"2023-10-06T03:01:42.287940+0000","last_became_active":"2023-10-06T03:01:42.283220+0000","last_became_peered":"2023-10-06T03:01:42.283220+0000","last_unstale":"2023-10-06T03:01:42.287940+0000","last_undegraded":"2023-10-06T03:01:42.287940+0000","last_fullsized":"2023-10-06T03:01:42.287940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373872+0000","last_change":"2023-10-06T03:01:41.380049+0000","last_active":"2023-10-06T03:01:42.373872+0000","last_peered":"2023-10-06T03:01:42.373872+0000","last_clean":"2023-10-06T03:01:42.373872+0000","last_became_active":"2023-10-06T03:01:41.379732+0000","last_became_peered":"2023-10-06T03:01:41.379732+0000","last_unstale":"2023-10-06T03:01:42.373872+0000","last_undegraded":"2023-10-06T03:01:42.373872+0000","last_fullsized":"2023-10-06T03:01:42.373872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:03.057587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286048+0000","last_change":"2023-10-06T03:01:42.279660+0000","last_active":"2023-10-06T03:01:42.286048+0000","last_peered":"2023-10-06T03:01:42.286048+0000","last_clean":"2023-10-06T03:01:42.286048+0000","last_became_active":"2023-10-06T03:01:42.279394+0000","last_became_peered":"2023-10-06T03:01:42.279394+0000","last_unstale":"2023-10-06T03:01:42.286048+0000","last_undegraded":"2023-10-06T03:01:42.286048+0000","last_fullsized":"2023-10-06T03:01:42.286048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377939+0000","last_change":"2023-10-06T03:01:41.384922+0000","last_active":"2023-10-06T03:01:42.377939+0000","last_peered":"2023-10-06T03:01:42.377939+0000","last_clean":"2023-10-06T03:01:42.377939+0000","last_became_active":"2023-10-06T03:01:41.384669+0000","last_became_peered":"2023-10-06T03:01:41.384669+0000","last_unstale":"2023-10-06T03:01:42.377939+0000","last_undegraded":"2023-10-06T03:01:42.377939+0000","last_fullsized":"2023-10-06T03:01:42.377939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.334690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284069+0000","last_change":"2023-10-06T03:01:42.277487+0000","last_active":"2023-10-06T03:01:42.284069+0000","last_peered":"2023-10-06T03:01:42.284069+0000","last_clean":"2023-10-06T03:01:42.284069+0000","last_became_active":"2023-10-06T03:01:42.277135+0000","last_became_peered":"2023-10-06T03:01:42.277135+0000","last_unstale":"2023-10-06T03:01:42.284069+0000","last_undegraded":"2023-10-06T03:01:42.284069+0000","last_fullsized":"2023-10-06T03:01:42.284069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373790+0000","last_change":"2023-10-06T03:01:41.380292+0000","last_active":"2023-10-06T03:01:42.373790+0000","last_peered":"2023-10-06T03:01:42.373790+0000","last_clean":"2023-10-06T03:01:42.373790+0000","last_became_active":"2023-10-06T03:01:41.380074+0000","last_became_peered":"2023-10-06T03:01:41.380074+0000","last_unstale":"2023-10-06T03:01:42.373790+0000","last_undegraded":"2023-10-06T03:01:42.373790+0000","last_fullsized":"2023-10-06T03:01:42.373790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:27.982776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282615+0000","last_change":"2023-10-06T03:01:42.277855+0000","last_active":"2023-10-06T03:01:42.282615+0000","last_peered":"2023-10-06T03:01:42.282615+0000","last_clean":"2023-10-06T03:01:42.282615+0000","last_became_active":"2023-10-06T03:01:42.277539+0000","last_became_peered":"2023-10-06T03:01:42.277539+0000","last_unstale":"2023-10-06T03:01:42.282615+0000","last_undegraded":"2023-10-06T03:01:42.282615+0000","last_fullsized":"2023-10-06T03:01:42.282615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268326+0000","last_change":"2023-10-06T03:01:41.379182+0000","last_active":"2023-10-06T03:01:42.268326+0000","last_peered":"2023-10-06T03:01:42.268326+0000","last_clean":"2023-10-06T03:01:42.268326+0000","last_became_active":"2023-10-06T03:01:41.378857+0000","last_became_peered":"2023-10-06T03:01:41.378857+0000","last_unstale":"2023-10-06T03:01:42.268326+0000","last_undegraded":"2023-10-06T03:01:42.268326+0000","last_fullsized":"2023-10-06T03:01:42.268326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281816+0000","last_change":"2023-10-06T03:01:42.277028+0000","last_active":"2023-10-06T03:01:42.281816+0000","last_peered":"2023-10-06T03:01:42.281816+0000","last_clean":"2023-10-06T03:01:42.281816+0000","last_became_active":"2023-10-06T03:01:42.276777+0000","last_became_peered":"2023-10-06T03:01:42.276777+0000","last_unstale":"2023-10-06T03:01:42.281816+0000","last_undegraded":"2023-10-06T03:01:42.281816+0000","last_fullsized":"2023-10-06T03:01:42.281816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.371956+0000","last_change":"2023-10-06T03:01:41.383140+0000","last_active":"2023-10-06T03:01:42.371956+0000","last_peered":"2023-10-06T03:01:42.371956+0000","last_clean":"2023-10-06T03:01:42.371956+0000","last_became_active":"2023-10-06T03:01:41.382894+0000","last_became_peered":"2023-10-06T03:01:41.382894+0000","last_unstale":"2023-10-06T03:01:42.371956+0000","last_undegraded":"2023-10-06T03:01:42.371956+0000","last_fullsized":"2023-10-06T03:01:42.371956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284623+0000","last_change":"2023-10-06T03:01:42.280965+0000","last_active":"2023-10-06T03:01:42.284623+0000","last_peered":"2023-10-06T03:01:42.284623+0000","last_clean":"2023-10-06T03:01:42.284623+0000","last_became_active":"2023-10-06T03:01:42.278565+0000","last_became_peered":"2023-10-06T03:01:42.278565+0000","last_unstale":"2023-10-06T03:01:42.284623+0000","last_undegraded":"2023-10-06T03:01:42.284623+0000","last_fullsized":"2023-10-06T03:01:42.284623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274323+0000","last_change":"2023-10-06T03:01:41.385727+0000","last_active":"2023-10-06T03:01:42.274323+0000","last_peered":"2023-10-06T03:01:42.274323+0000","last_clean":"2023-10-06T03:01:42.274323+0000","last_became_active":"2023-10-06T03:01:41.385354+0000","last_became_peered":"2023-10-06T03:01:41.385354+0000","last_unstale":"2023-10-06T03:01:42.274323+0000","last_undegraded":"2023-10-06T03:01:42.274323+0000","last_fullsized":"2023-10-06T03:01:42.274323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284128+0000","last_change":"2023-10-06T03:01:42.278082+0000","last_active":"2023-10-06T03:01:42.284128+0000","last_peered":"2023-10-06T03:01:42.284128+0000","last_clean":"2023-10-06T03:01:42.284128+0000","last_became_active":"2023-10-06T03:01:42.277812+0000","last_became_peered":"2023-10-06T03:01:42.277812+0000","last_unstale":"2023-10-06T03:01:42.284128+0000","last_undegraded":"2023-10-06T03:01:42.284128+0000","last_fullsized":"2023-10-06T03:01:42.284128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270092+0000","last_change":"2023-10-06T03:01:41.381181+0000","last_active":"2023-10-06T03:01:42.270092+0000","last_peered":"2023-10-06T03:01:42.270092+0000","last_clean":"2023-10-06T03:01:42.270092+0000","last_became_active":"2023-10-06T03:01:41.378547+0000","last_became_peered":"2023-10-06T03:01:41.378547+0000","last_unstale":"2023-10-06T03:01:42.270092+0000","last_undegraded":"2023-10-06T03:01:42.270092+0000","last_fullsized":"2023-10-06T03:01:42.270092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284704+0000","last_change":"2023-10-06T03:01:42.281464+0000","last_active":"2023-10-06T03:01:42.284704+0000","last_peered":"2023-10-06T03:01:42.284704+0000","last_clean":"2023-10-06T03:01:42.284704+0000","last_became_active":"2023-10-06T03:01:42.281186+0000","last_became_peered":"2023-10-06T03:01:42.281186+0000","last_unstale":"2023-10-06T03:01:42.284704+0000","last_undegraded":"2023-10-06T03:01:42.284704+0000","last_fullsized":"2023-10-06T03:01:42.284704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.673116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268609+0000","last_change":"2023-10-06T03:01:41.378017+0000","last_active":"2023-10-06T03:01:42.268609+0000","last_peered":"2023-10-06T03:01:42.268609+0000","last_clean":"2023-10-06T03:01:42.268609+0000","last_became_active":"2023-10-06T03:01:41.377759+0000","last_became_peered":"2023-10-06T03:01:41.377759+0000","last_unstale":"2023-10-06T03:01:42.268609+0000","last_undegraded":"2023-10-06T03:01:42.268609+0000","last_fullsized":"2023-10-06T03:01:42.268609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287848+0000","last_change":"2023-10-06T03:01:42.278383+0000","last_active":"2023-10-06T03:01:42.287848+0000","last_peered":"2023-10-06T03:01:42.287848+0000","last_clean":"2023-10-06T03:01:42.287848+0000","last_became_active":"2023-10-06T03:01:42.277973+0000","last_became_peered":"2023-10-06T03:01:42.277973+0000","last_unstale":"2023-10-06T03:01:42.287848+0000","last_undegraded":"2023-10-06T03:01:42.287848+0000","last_fullsized":"2023-10-06T03:01:42.287848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271774+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:01:42.271774+0000","last_peered":"2023-10-06T03:01:42.271774+0000","last_clean":"2023-10-06T03:01:42.271774+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:01:42.271774+0000","last_undegraded":"2023-10-06T03:01:42.271774+0000","last_fullsized":"2023-10-06T03:01:42.271774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267761+0000","last_change":"2023-10-06T03:01:41.379310+0000","last_active":"2023-10-06T03:01:42.267761+0000","last_peered":"2023-10-06T03:01:42.267761+0000","last_clean":"2023-10-06T03:01:42.267761+0000","last_became_active":"2023-10-06T03:01:41.379017+0000","last_became_peered":"2023-10-06T03:01:41.379017+0000","last_unstale":"2023-10-06T03:01:42.267761+0000","last_undegraded":"2023-10-06T03:01:42.267761+0000","last_fullsized":"2023-10-06T03:01:42.267761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288159+0000","last_change":"2023-10-06T03:01:42.285640+0000","last_active":"2023-10-06T03:01:42.288159+0000","last_peered":"2023-10-06T03:01:42.288159+0000","last_clean":"2023-10-06T03:01:42.288159+0000","last_became_active":"2023-10-06T03:01:42.283071+0000","last_became_peered":"2023-10-06T03:01:42.283071+0000","last_unstale":"2023-10-06T03:01:42.288159+0000","last_undegraded":"2023-10-06T03:01:42.288159+0000","last_fullsized":"2023-10-06T03:01:42.288159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269088+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:01:42.269088+0000","last_peered":"2023-10-06T03:01:42.269088+0000","last_clean":"2023-10-06T03:01:42.269088+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:01:42.269088+0000","last_undegraded":"2023-10-06T03:01:42.269088+0000","last_fullsized":"2023-10-06T03:01:42.269088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268853+0000","last_change":"2023-10-06T03:01:41.382782+0000","last_active":"2023-10-06T03:01:42.268853+0000","last_peered":"2023-10-06T03:01:42.268853+0000","last_clean":"2023-10-06T03:01:42.268853+0000","last_became_active":"2023-10-06T03:01:41.382522+0000","last_became_peered":"2023-10-06T03:01:41.382522+0000","last_unstale":"2023-10-06T03:01:42.268853+0000","last_undegraded":"2023-10-06T03:01:42.268853+0000","last_fullsized":"2023-10-06T03:01:42.268853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288056+0000","last_change":"2023-10-06T03:01:42.282263+0000","last_active":"2023-10-06T03:01:42.288056+0000","last_peered":"2023-10-06T03:01:42.288056+0000","last_clean":"2023-10-06T03:01:42.288056+0000","last_became_active":"2023-10-06T03:01:42.282019+0000","last_became_peered":"2023-10-06T03:01:42.282019+0000","last_unstale":"2023-10-06T03:01:42.288056+0000","last_undegraded":"2023-10-06T03:01:42.288056+0000","last_fullsized":"2023-10-06T03:01:42.288056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.265449+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:01:42.265449+0000","last_peered":"2023-10-06T03:01:42.265449+0000","last_clean":"2023-10-06T03:01:42.265449+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:01:42.265449+0000","last_undegraded":"2023-10-06T03:01:42.265449+0000","last_fullsized":"2023-10-06T03:01:42.265449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268243+0000","last_change":"2023-10-06T03:01:41.378349+0000","last_active":"2023-10-06T03:01:42.268243+0000","last_peered":"2023-10-06T03:01:42.268243+0000","last_clean":"2023-10-06T03:01:42.268243+0000","last_became_active":"2023-10-06T03:01:41.378089+0000","last_became_peered":"2023-10-06T03:01:41.378089+0000","last_unstale":"2023-10-06T03:01:42.268243+0000","last_undegraded":"2023-10-06T03:01:42.268243+0000","last_fullsized":"2023-10-06T03:01:42.268243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288080+0000","last_change":"2023-10-06T03:01:42.283418+0000","last_active":"2023-10-06T03:01:42.288080+0000","last_peered":"2023-10-06T03:01:42.288080+0000","last_clean":"2023-10-06T03:01:42.288080+0000","last_became_active":"2023-10-06T03:01:42.283035+0000","last_became_peered":"2023-10-06T03:01:42.283035+0000","last_unstale":"2023-10-06T03:01:42.288080+0000","last_undegraded":"2023-10-06T03:01:42.288080+0000","last_fullsized":"2023-10-06T03:01:42.288080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:27.703859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.266739+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:01:42.266739+0000","last_peered":"2023-10-06T03:01:42.266739+0000","last_clean":"2023-10-06T03:01:42.266739+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:01:42.266739+0000","last_undegraded":"2023-10-06T03:01:42.266739+0000","last_fullsized":"2023-10-06T03:01:42.266739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274310+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:01:42.274310+0000","last_peered":"2023-10-06T03:01:42.274310+0000","last_clean":"2023-10-06T03:01:42.274310+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:01:42.274310+0000","last_undegraded":"2023-10-06T03:01:42.274310+0000","last_fullsized":"2023-10-06T03:01:42.274310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268619+0000","last_change":"2023-10-06T03:01:41.378526+0000","last_active":"2023-10-06T03:01:42.268619+0000","last_peered":"2023-10-06T03:01:42.268619+0000","last_clean":"2023-10-06T03:01:42.268619+0000","last_became_active":"2023-10-06T03:01:41.378245+0000","last_became_peered":"2023-10-06T03:01:41.378245+0000","last_unstale":"2023-10-06T03:01:42.268619+0000","last_undegraded":"2023-10-06T03:01:42.268619+0000","last_fullsized":"2023-10-06T03:01:42.268619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284302+0000","last_change":"2023-10-06T03:01:42.281327+0000","last_active":"2023-10-06T03:01:42.284302+0000","last_peered":"2023-10-06T03:01:42.284302+0000","last_clean":"2023-10-06T03:01:42.284302+0000","last_became_active":"2023-10-06T03:01:42.281028+0000","last_became_peered":"2023-10-06T03:01:42.281028+0000","last_unstale":"2023-10-06T03:01:42.284302+0000","last_undegraded":"2023-10-06T03:01:42.284302+0000","last_fullsized":"2023-10-06T03:01:42.284302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:28.986647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274776+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:01:42.274776+0000","last_peered":"2023-10-06T03:01:42.274776+0000","last_clean":"2023-10-06T03:01:42.274776+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:01:42.274776+0000","last_undegraded":"2023-10-06T03:01:42.274776+0000","last_fullsized":"2023-10-06T03:01:42.274776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268368+0000","last_change":"2023-10-06T03:01:41.377764+0000","last_active":"2023-10-06T03:01:42.268368+0000","last_peered":"2023-10-06T03:01:42.268368+0000","last_clean":"2023-10-06T03:01:42.268368+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.268368+0000","last_undegraded":"2023-10-06T03:01:42.268368+0000","last_fullsized":"2023-10-06T03:01:42.268368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285753+0000","last_change":"2023-10-06T03:01:42.279939+0000","last_active":"2023-10-06T03:01:42.285753+0000","last_peered":"2023-10-06T03:01:42.285753+0000","last_clean":"2023-10-06T03:01:42.285753+0000","last_became_active":"2023-10-06T03:01:42.279453+0000","last_became_peered":"2023-10-06T03:01:42.279453+0000","last_unstale":"2023-10-06T03:01:42.285753+0000","last_undegraded":"2023-10-06T03:01:42.285753+0000","last_fullsized":"2023-10-06T03:01:42.285753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.276154+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:01:42.276154+0000","last_peered":"2023-10-06T03:01:42.276154+0000","last_clean":"2023-10-06T03:01:42.276154+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:01:42.276154+0000","last_undegraded":"2023-10-06T03:01:42.276154+0000","last_fullsized":"2023-10-06T03:01:42.276154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271880+0000","last_change":"2023-10-06T03:01:41.378694+0000","last_active":"2023-10-06T03:01:42.271880+0000","last_peered":"2023-10-06T03:01:42.271880+0000","last_clean":"2023-10-06T03:01:42.271880+0000","last_became_active":"2023-10-06T03:01:41.378366+0000","last_became_peered":"2023-10-06T03:01:41.378366+0000","last_unstale":"2023-10-06T03:01:42.271880+0000","last_undegraded":"2023-10-06T03:01:42.271880+0000","last_fullsized":"2023-10-06T03:01:42.271880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285642+0000","last_change":"2023-10-06T03:01:42.278204+0000","last_active":"2023-10-06T03:01:42.285642+0000","last_peered":"2023-10-06T03:01:42.285642+0000","last_clean":"2023-10-06T03:01:42.285642+0000","last_became_active":"2023-10-06T03:01:42.277950+0000","last_became_peered":"2023-10-06T03:01:42.277950+0000","last_unstale":"2023-10-06T03:01:42.285642+0000","last_undegraded":"2023-10-06T03:01:42.285642+0000","last_fullsized":"2023-10-06T03:01:42.285642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378135+0000","last_change":"2023-10-06T03:01:41.378450+0000","last_active":"2023-10-06T03:01:42.378135+0000","last_peered":"2023-10-06T03:01:42.378135+0000","last_clean":"2023-10-06T03:01:42.378135+0000","last_became_active":"2023-10-06T03:01:41.378167+0000","last_became_peered":"2023-10-06T03:01:41.378167+0000","last_unstale":"2023-10-06T03:01:42.378135+0000","last_undegraded":"2023-10-06T03:01:42.378135+0000","last_fullsized":"2023-10-06T03:01:42.378135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269297+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:01:42.269297+0000","last_peered":"2023-10-06T03:01:42.269297+0000","last_clean":"2023-10-06T03:01:42.269297+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:01:42.269297+0000","last_undegraded":"2023-10-06T03:01:42.269297+0000","last_fullsized":"2023-10-06T03:01:42.269297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288032+0000","last_change":"2023-10-06T03:01:42.284704+0000","last_active":"2023-10-06T03:01:42.288032+0000","last_peered":"2023-10-06T03:01:42.288032+0000","last_clean":"2023-10-06T03:01:42.288032+0000","last_became_active":"2023-10-06T03:01:42.284380+0000","last_became_peered":"2023-10-06T03:01:42.284380+0000","last_unstale":"2023-10-06T03:01:42.288032+0000","last_undegraded":"2023-10-06T03:01:42.288032+0000","last_fullsized":"2023-10-06T03:01:42.288032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.273934+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:01:42.273934+0000","last_peered":"2023-10-06T03:01:42.273934+0000","last_clean":"2023-10-06T03:01:42.273934+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:01:42.273934+0000","last_undegraded":"2023-10-06T03:01:42.273934+0000","last_fullsized":"2023-10-06T03:01:42.273934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T03:01:43.460248+0000","last_change":"2023-10-06T03:01:41.376173+0000","last_active":"2023-10-06T03:01:43.460248+0000","last_peered":"2023-10-06T03:01:43.460248+0000","last_clean":"2023-10-06T03:01:43.460248+0000","last_became_active":"2023-10-06T03:01:41.375877+0000","last_became_peered":"2023-10-06T03:01:41.375877+0000","last_unstale":"2023-10-06T03:01:43.460248+0000","last_undegraded":"2023-10-06T03:01:43.460248+0000","last_fullsized":"2023-10-06T03:01:43.460248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.277029+0000","last_change":"2023-10-06T03:01:42.271755+0000","last_active":"2023-10-06T03:01:42.277029+0000","last_peered":"2023-10-06T03:01:42.277029+0000","last_clean":"2023-10-06T03:01:42.277029+0000","last_became_active":"2023-10-06T03:01:42.271483+0000","last_became_peered":"2023-10-06T03:01:42.271483+0000","last_unstale":"2023-10-06T03:01:42.277029+0000","last_undegraded":"2023-10-06T03:01:42.277029+0000","last_fullsized":"2023-10-06T03:01:42.277029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373544+0000","last_change":"2023-10-06T03:01:41.381914+0000","last_active":"2023-10-06T03:01:42.373544+0000","last_peered":"2023-10-06T03:01:42.373544+0000","last_clean":"2023-10-06T03:01:42.373544+0000","last_became_active":"2023-10-06T03:01:41.381635+0000","last_became_peered":"2023-10-06T03:01:41.381635+0000","last_unstale":"2023-10-06T03:01:42.373544+0000","last_undegraded":"2023-10-06T03:01:42.373544+0000","last_fullsized":"2023-10-06T03:01:42.373544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280515+0000","last_change":"2023-10-06T03:01:42.277000+0000","last_active":"2023-10-06T03:01:42.280515+0000","last_peered":"2023-10-06T03:01:42.280515+0000","last_clean":"2023-10-06T03:01:42.280515+0000","last_became_active":"2023-10-06T03:01:42.276746+0000","last_became_peered":"2023-10-06T03:01:42.276746+0000","last_unstale":"2023-10-06T03:01:42.280515+0000","last_undegraded":"2023-10-06T03:01:42.280515+0000","last_fullsized":"2023-10-06T03:01:42.280515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378477+0000","last_change":"2023-10-06T03:01:41.377203+0000","last_active":"2023-10-06T03:01:42.378477+0000","last_peered":"2023-10-06T03:01:42.378477+0000","last_clean":"2023-10-06T03:01:42.378477+0000","last_became_active":"2023-10-06T03:01:41.376950+0000","last_became_peered":"2023-10-06T03:01:41.376950+0000","last_unstale":"2023-10-06T03:01:42.378477+0000","last_undegraded":"2023-10-06T03:01:42.378477+0000","last_fullsized":"2023-10-06T03:01:42.378477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284682+0000","last_change":"2023-10-06T03:01:42.282305+0000","last_active":"2023-10-06T03:01:42.284682+0000","last_peered":"2023-10-06T03:01:42.284682+0000","last_clean":"2023-10-06T03:01:42.284682+0000","last_became_active":"2023-10-06T03:01:42.282037+0000","last_became_peered":"2023-10-06T03:01:42.282037+0000","last_unstale":"2023-10-06T03:01:42.284682+0000","last_undegraded":"2023-10-06T03:01:42.284682+0000","last_fullsized":"2023-10-06T03:01:42.284682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:24.802638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378519+0000","last_change":"2023-10-06T03:01:41.377561+0000","last_active":"2023-10-06T03:01:42.378519+0000","last_peered":"2023-10-06T03:01:42.378519+0000","last_clean":"2023-10-06T03:01:42.378519+0000","last_became_active":"2023-10-06T03:01:41.377308+0000","last_became_peered":"2023-10-06T03:01:41.377308+0000","last_unstale":"2023-10-06T03:01:42.378519+0000","last_undegraded":"2023-10-06T03:01:42.378519+0000","last_fullsized":"2023-10-06T03:01:42.378519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284090+0000","last_change":"2023-10-06T03:01:42.277443+0000","last_active":"2023-10-06T03:01:42.284090+0000","last_peered":"2023-10-06T03:01:42.284090+0000","last_clean":"2023-10-06T03:01:42.284090+0000","last_became_active":"2023-10-06T03:01:42.277152+0000","last_became_peered":"2023-10-06T03:01:42.277152+0000","last_unstale":"2023-10-06T03:01:42.284090+0000","last_undegraded":"2023-10-06T03:01:42.284090+0000","last_fullsized":"2023-10-06T03:01:42.284090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373306+0000","last_change":"2023-10-06T03:01:41.379186+0000","last_active":"2023-10-06T03:01:42.373306+0000","last_peered":"2023-10-06T03:01:42.373306+0000","last_clean":"2023-10-06T03:01:42.373306+0000","last_became_active":"2023-10-06T03:01:41.378963+0000","last_became_peered":"2023-10-06T03:01:41.378963+0000","last_unstale":"2023-10-06T03:01:42.373306+0000","last_undegraded":"2023-10-06T03:01:42.373306+0000","last_fullsized":"2023-10-06T03:01:42.373306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280768+0000","last_change":"2023-10-06T03:01:42.276973+0000","last_active":"2023-10-06T03:01:42.280768+0000","last_peered":"2023-10-06T03:01:42.280768+0000","last_clean":"2023-10-06T03:01:42.280768+0000","last_became_active":"2023-10-06T03:01:42.276297+0000","last_became_peered":"2023-10-06T03:01:42.276297+0000","last_unstale":"2023-10-06T03:01:42.280768+0000","last_undegraded":"2023-10-06T03:01:42.280768+0000","last_fullsized":"2023-10-06T03:01:42.280768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373671+0000","last_change":"2023-10-06T03:01:41.380486+0000","last_active":"2023-10-06T03:01:42.373671+0000","last_peered":"2023-10-06T03:01:42.373671+0000","last_clean":"2023-10-06T03:01:42.373671+0000","last_became_active":"2023-10-06T03:01:41.380212+0000","last_became_peered":"2023-10-06T03:01:41.380212+0000","last_unstale":"2023-10-06T03:01:42.373671+0000","last_undegraded":"2023-10-06T03:01:42.373671+0000","last_fullsized":"2023-10-06T03:01:42.373671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284486+0000","last_change":"2023-10-06T03:01:42.280604+0000","last_active":"2023-10-06T03:01:42.284486+0000","last_peered":"2023-10-06T03:01:42.284486+0000","last_clean":"2023-10-06T03:01:42.284486+0000","last_became_active":"2023-10-06T03:01:42.280041+0000","last_became_peered":"2023-10-06T03:01:42.280041+0000","last_unstale":"2023-10-06T03:01:42.284486+0000","last_undegraded":"2023-10-06T03:01:42.284486+0000","last_fullsized":"2023-10-06T03:01:42.284486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:02.038765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283231+0000","last_change":"2023-10-06T03:01:42.277027+0000","last_active":"2023-10-06T03:01:42.283231+0000","last_peered":"2023-10-06T03:01:42.283231+0000","last_clean":"2023-10-06T03:01:42.283231+0000","last_became_active":"2023-10-06T03:01:42.276730+0000","last_became_peered":"2023-10-06T03:01:42.276730+0000","last_unstale":"2023-10-06T03:01:42.283231+0000","last_undegraded":"2023-10-06T03:01:42.283231+0000","last_fullsized":"2023-10-06T03:01:42.283231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:18.781548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280787+0000","last_change":"2023-10-06T03:01:42.277145+0000","last_active":"2023-10-06T03:01:42.280787+0000","last_peered":"2023-10-06T03:01:42.280787+0000","last_clean":"2023-10-06T03:01:42.280787+0000","last_became_active":"2023-10-06T03:01:42.276735+0000","last_became_peered":"2023-10-06T03:01:42.276735+0000","last_unstale":"2023-10-06T03:01:42.280787+0000","last_undegraded":"2023-10-06T03:01:42.280787+0000","last_fullsized":"2023-10-06T03:01:42.280787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284561+0000","last_change":"2023-10-06T03:01:42.281639+0000","last_active":"2023-10-06T03:01:42.284561+0000","last_peered":"2023-10-06T03:01:42.284561+0000","last_clean":"2023-10-06T03:01:42.284561+0000","last_became_active":"2023-10-06T03:01:42.278720+0000","last_became_peered":"2023-10-06T03:01:42.278720+0000","last_unstale":"2023-10-06T03:01:42.284561+0000","last_undegraded":"2023-10-06T03:01:42.284561+0000","last_fullsized":"2023-10-06T03:01:42.284561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:03.312309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288057+0000","last_change":"2023-10-06T03:01:42.283924+0000","last_active":"2023-10-06T03:01:42.288057+0000","last_peered":"2023-10-06T03:01:42.288057+0000","last_clean":"2023-10-06T03:01:42.288057+0000","last_became_active":"2023-10-06T03:01:42.283527+0000","last_became_peered":"2023-10-06T03:01:42.283527+0000","last_unstale":"2023-10-06T03:01:42.288057+0000","last_undegraded":"2023-10-06T03:01:42.288057+0000","last_fullsized":"2023-10-06T03:01:42.288057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288138+0000","last_change":"2023-10-06T03:01:42.284183+0000","last_active":"2023-10-06T03:01:42.288138+0000","last_peered":"2023-10-06T03:01:42.288138+0000","last_clean":"2023-10-06T03:01:42.288138+0000","last_became_active":"2023-10-06T03:01:42.283895+0000","last_became_peered":"2023-10-06T03:01:42.283895+0000","last_unstale":"2023-10-06T03:01:42.288138+0000","last_undegraded":"2023-10-06T03:01:42.288138+0000","last_fullsized":"2023-10-06T03:01:42.288138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:01:50.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:01:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.965+0000 7faec7d52700 1 Processor -- start 2023-10-06T03:01:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.966+0000 7faec7d52700 1 -- start start 2023-10-06T03:01:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.966+0000 7faec7d52700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.966+0000 7faec7d52700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec015a160 con 0x7faec0069560 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.966+0000 7faec7d52700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec00d13a0 con 0x7faec0159c60 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.966+0000 7faec7d52700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec015e140 con 0x7faec0159890 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec62ef700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7faec0069560 0x7faec00ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38994/0 (socket says 172.21.15.92:38994) 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec62ef700 1 -- 172.21.15.92:0/2428585294 learned_addr learned my addr 172.21.15.92:0/2428585294 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec5aee700 1 --2- 172.21.15.92:0/2428585294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592245091 0 0) 0x7faec015a160 con 0x7faec0069560 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeac003570 con 0x7faec0069560 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429181191 0 0) 0x7faec00d13a0 con 0x7faec0159c60 2023-10-06T03:01:50.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec015a160 con 0x7faec0159c60 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec5aee700 1 -- 172.21.15.92:0/2428585294 >> v1:172.21.15.136:6789/0 conn(0x7faec0159c60 legacy=0x7faec015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec5aee700 1 -- 172.21.15.92:0/2428585294 >> v1:172.21.15.92:6789/0 conn(0x7faec0069560 legacy=0x7faec00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec5aee700 1 -- 172.21.15.92:0/2428585294 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec015a3d0 con 0x7faec0159890 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec5aee700 1 --2- 172.21.15.92:0/2428585294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec015abd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faebc009910 tx=0x7faebc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f73a9d087914c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc011630 con 0x7faec0159890 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc011790 con 0x7faec0159890 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.967+0000 7faec4aec700 1 -- 172.21.15.92:0/2428585294 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc011b30 con 0x7faec0159890 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 -- 172.21.15.92:0/2428585294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 msgr2=0x7faec015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 --2- 172.21.15.92:0/2428585294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec015abd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faebc009910 tx=0x7faebc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 -- 172.21.15.92:0/2428585294 shutdown_connections 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 --2- 172.21.15.92:0/2428585294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec015abd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 -- 172.21.15.92:0/2428585294 >> 172.21.15.92:0/2428585294 conn(0x7faec0068e20 msgr2=0x7faec00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 -- 172.21.15.92:0/2428585294 shutdown_connections 2023-10-06T03:01:50.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.968+0000 7faec7d52700 1 -- 172.21.15.92:0/2428585294 wait complete. 2023-10-06T03:01:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.969+0000 7faec7d52700 1 Processor -- start 2023-10-06T03:01:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.969+0000 7faec7d52700 1 -- start start 2023-10-06T03:01:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec7d52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 0x7faec014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec7d52700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec00b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec7d52700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faec0159c60 0x7faec015d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec7d52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b62e0 con 0x7faec0069560 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec7d52700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b56d0 con 0x7faec0159c60 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec7d52700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec00b59b0 con 0x7faec0159890 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 0x7faec014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 0x7faec014d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34566/0 (socket says 172.21.15.92:34566) 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 -- 172.21.15.92:0/2088460041 learned_addr learned my addr 172.21.15.92:0/2088460041 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec62ef700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faec0159c60 0x7faec015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec52ed700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec00b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 -- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 msgr2=0x7faec00b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec00b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 -- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faec0159c60 msgr2=0x7faec015d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:50.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faec0159c60 0x7faec015d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 -- 172.21.15.92:0/2088460041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc009420 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faec5aee700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 0x7faec014d400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faebc009f00 tx=0x7faebc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.970+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc01f070 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.971+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc000bc0 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.971+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec00b5c90 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.971+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc025420 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.971+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec00b53c0 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.972+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec004c3c0 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.972+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faebc023070 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.972+0000 7faeb5ffb700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faea4066f30 0x7faea40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.973+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faebc085b90 con 0x7faec0069560 2023-10-06T03:01:50.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.973+0000 7faec52ed700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faea4066f30 0x7faea40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:50.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.973+0000 7faec52ed700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faea4066f30 0x7faea40693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faec016d210 tx=0x7faeb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:50.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:50.977+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faebc052c00 con 0x7faec0069560 2023-10-06T03:01:51.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.155+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faec00b6420 con 0x7faea4066f30 2023-10-06T03:01:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.163+0000 7faeb5ffb700 1 -- 172.21.15.92:0/2088460041 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286621 (secure 0 0 0) 0x7faeb0013420 con 0x7faea4066f30 2023-10-06T03:01:51.170 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faea4066f30 msgr2=0x7faea40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faea4066f30 0x7faea40693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faec016d210 tx=0x7faeb0009040 comp rx=0 tx=0).stop 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 msgr2=0x7faec014d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 0x7faec014d400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faebc009f00 tx=0x7faebc017000 comp rx=0 tx=0).stop 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 shutdown_connections 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7faea4066f30 0x7faea40693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faec0069560 0x7faec014d400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faec0159c60 0x7faec015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 --2- 172.21.15.92:0/2088460041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faec0159890 0x7faec00b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 >> 172.21.15.92:0/2088460041 conn(0x7faec0068e20 msgr2=0x7faec0154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 shutdown_connections 2023-10-06T03:01:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.166+0000 7faec7d52700 1 -- 172.21.15.92:0/2088460041 wait complete. 2023-10-06T03:01:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:01:51.182 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T03:01:49.184153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.821177"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280436+0000","last_change":"2023-10-06T03:01:42.278647+0000","last_active":"2023-10-06T03:01:42.280436+0000","last_peered":"2023-10-06T03:01:42.280436+0000","last_clean":"2023-10-06T03:01:42.280436+0000","last_became_active":"2023-10-06T03:01:42.278364+0000","last_became_peered":"2023-10-06T03:01:42.278364+0000","last_unstale":"2023-10-06T03:01:42.280436+0000","last_undegraded":"2023-10-06T03:01:42.280436+0000","last_fullsized":"2023-10-06T03:01:42.280436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:06.498462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282043+0000","last_change":"2023-10-06T03:01:42.277062+0000","last_active":"2023-10-06T03:01:42.282043+0000","last_peered":"2023-10-06T03:01:42.282043+0000","last_clean":"2023-10-06T03:01:42.282043+0000","last_became_active":"2023-10-06T03:01:42.276189+0000","last_became_peered":"2023-10-06T03:01:42.276189+0000","last_unstale":"2023-10-06T03:01:42.282043+0000","last_undegraded":"2023-10-06T03:01:42.282043+0000","last_fullsized":"2023-10-06T03:01:42.282043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284329+0000","last_change":"2023-10-06T03:01:42.280530+0000","last_active":"2023-10-06T03:01:42.284329+0000","last_peered":"2023-10-06T03:01:42.284329+0000","last_clean":"2023-10-06T03:01:42.284329+0000","last_became_active":"2023-10-06T03:01:42.280239+0000","last_became_peered":"2023-10-06T03:01:42.280239+0000","last_unstale":"2023-10-06T03:01:42.284329+0000","last_undegraded":"2023-10-06T03:01:42.284329+0000","last_fullsized":"2023-10-06T03:01:42.284329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284290+0000","last_change":"2023-10-06T03:01:42.279732+0000","last_active":"2023-10-06T03:01:42.284290+0000","last_peered":"2023-10-06T03:01:42.284290+0000","last_clean":"2023-10-06T03:01:42.284290+0000","last_became_active":"2023-10-06T03:01:42.279445+0000","last_became_peered":"2023-10-06T03:01:42.279445+0000","last_unstale":"2023-10-06T03:01:42.284290+0000","last_undegraded":"2023-10-06T03:01:42.284290+0000","last_fullsized":"2023-10-06T03:01:42.284290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:08.338560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285662+0000","last_change":"2023-10-06T03:01:42.279674+0000","last_active":"2023-10-06T03:01:42.285662+0000","last_peered":"2023-10-06T03:01:42.285662+0000","last_clean":"2023-10-06T03:01:42.285662+0000","last_became_active":"2023-10-06T03:01:42.279435+0000","last_became_peered":"2023-10-06T03:01:42.279435+0000","last_unstale":"2023-10-06T03:01:42.285662+0000","last_undegraded":"2023-10-06T03:01:42.285662+0000","last_fullsized":"2023-10-06T03:01:42.285662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284581+0000","last_change":"2023-10-06T03:01:42.282579+0000","last_active":"2023-10-06T03:01:42.284581+0000","last_peered":"2023-10-06T03:01:42.284581+0000","last_clean":"2023-10-06T03:01:42.284581+0000","last_became_active":"2023-10-06T03:01:42.282276+0000","last_became_peered":"2023-10-06T03:01:42.282276+0000","last_unstale":"2023-10-06T03:01:42.284581+0000","last_undegraded":"2023-10-06T03:01:42.284581+0000","last_fullsized":"2023-10-06T03:01:42.284581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286199+0000","last_change":"2023-10-06T03:01:42.282922+0000","last_active":"2023-10-06T03:01:42.286199+0000","last_peered":"2023-10-06T03:01:42.286199+0000","last_clean":"2023-10-06T03:01:42.286199+0000","last_became_active":"2023-10-06T03:01:42.282688+0000","last_became_peered":"2023-10-06T03:01:42.282688+0000","last_unstale":"2023-10-06T03:01:42.286199+0000","last_undegraded":"2023-10-06T03:01:42.286199+0000","last_fullsized":"2023-10-06T03:01:42.286199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285998+0000","last_change":"2023-10-06T03:01:42.279085+0000","last_active":"2023-10-06T03:01:42.285998+0000","last_peered":"2023-10-06T03:01:42.285998+0000","last_clean":"2023-10-06T03:01:42.285998+0000","last_became_active":"2023-10-06T03:01:42.278311+0000","last_became_peered":"2023-10-06T03:01:42.278311+0000","last_unstale":"2023-10-06T03:01:42.285998+0000","last_undegraded":"2023-10-06T03:01:42.285998+0000","last_fullsized":"2023-10-06T03:01:42.285998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285685+0000","last_change":"2023-10-06T03:01:42.280111+0000","last_active":"2023-10-06T03:01:42.285685+0000","last_peered":"2023-10-06T03:01:42.285685+0000","last_clean":"2023-10-06T03:01:42.285685+0000","last_became_active":"2023-10-06T03:01:42.279534+0000","last_became_peered":"2023-10-06T03:01:42.279534+0000","last_unstale":"2023-10-06T03:01:42.285685+0000","last_undegraded":"2023-10-06T03:01:42.285685+0000","last_fullsized":"2023-10-06T03:01:42.285685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286459+0000","last_change":"2023-10-06T03:01:42.281699+0000","last_active":"2023-10-06T03:01:42.286459+0000","last_peered":"2023-10-06T03:01:42.286459+0000","last_clean":"2023-10-06T03:01:42.286459+0000","last_became_active":"2023-10-06T03:01:42.279098+0000","last_became_peered":"2023-10-06T03:01:42.279098+0000","last_unstale":"2023-10-06T03:01:42.286459+0000","last_undegraded":"2023-10-06T03:01:42.286459+0000","last_fullsized":"2023-10-06T03:01:42.286459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283045+0000","last_change":"2023-10-06T03:01:42.277096+0000","last_active":"2023-10-06T03:01:42.283045+0000","last_peered":"2023-10-06T03:01:42.283045+0000","last_clean":"2023-10-06T03:01:42.283045+0000","last_became_active":"2023-10-06T03:01:42.276719+0000","last_became_peered":"2023-10-06T03:01:42.276719+0000","last_unstale":"2023-10-06T03:01:42.283045+0000","last_undegraded":"2023-10-06T03:01:42.283045+0000","last_fullsized":"2023-10-06T03:01:42.283045+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:13.143094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275038+0000","last_change":"2023-10-06T03:01:42.272031+0000","last_active":"2023-10-06T03:01:42.275038+0000","last_peered":"2023-10-06T03:01:42.275038+0000","last_clean":"2023-10-06T03:01:42.275038+0000","last_became_active":"2023-10-06T03:01:42.271796+0000","last_became_peered":"2023-10-06T03:01:42.271796+0000","last_unstale":"2023-10-06T03:01:42.275038+0000","last_undegraded":"2023-10-06T03:01:42.275038+0000","last_fullsized":"2023-10-06T03:01:42.275038+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281109+0000","last_change":"2023-10-06T03:01:42.279147+0000","last_active":"2023-10-06T03:01:42.281109+0000","last_peered":"2023-10-06T03:01:42.281109+0000","last_clean":"2023-10-06T03:01:42.281109+0000","last_became_active":"2023-10-06T03:01:42.278911+0000","last_became_peered":"2023-10-06T03:01:42.278911+0000","last_unstale":"2023-10-06T03:01:42.281109+0000","last_undegraded":"2023-10-06T03:01:42.281109+0000","last_fullsized":"2023-10-06T03:01:42.281109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:14.170785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286021+0000","last_change":"2023-10-06T03:01:42.277711+0000","last_active":"2023-10-06T03:01:42.286021+0000","last_peered":"2023-10-06T03:01:42.286021+0000","last_clean":"2023-10-06T03:01:42.286021+0000","last_became_active":"2023-10-06T03:01:42.276996+0000","last_became_peered":"2023-10-06T03:01:42.276996+0000","last_unstale":"2023-10-06T03:01:42.286021+0000","last_undegraded":"2023-10-06T03:01:42.286021+0000","last_fullsized":"2023-10-06T03:01:42.286021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.289013+0000","last_change":"2023-10-06T03:01:42.288226+0000","last_active":"2023-10-06T03:01:42.289013+0000","last_peered":"2023-10-06T03:01:42.289013+0000","last_clean":"2023-10-06T03:01:42.289013+0000","last_became_active":"2023-10-06T03:01:42.287944+0000","last_became_peered":"2023-10-06T03:01:42.287944+0000","last_unstale":"2023-10-06T03:01:42.289013+0000","last_undegraded":"2023-10-06T03:01:42.289013+0000","last_fullsized":"2023-10-06T03:01:42.289013+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284504+0000","last_change":"2023-10-06T03:01:42.281156+0000","last_active":"2023-10-06T03:01:42.284504+0000","last_peered":"2023-10-06T03:01:42.284504+0000","last_clean":"2023-10-06T03:01:42.284504+0000","last_became_active":"2023-10-06T03:01:42.280924+0000","last_became_peered":"2023-10-06T03:01:42.280924+0000","last_unstale":"2023-10-06T03:01:42.284504+0000","last_undegraded":"2023-10-06T03:01:42.284504+0000","last_fullsized":"2023-10-06T03:01:42.284504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285706+0000","last_change":"2023-10-06T03:01:42.280272+0000","last_active":"2023-10-06T03:01:42.285706+0000","last_peered":"2023-10-06T03:01:42.285706+0000","last_clean":"2023-10-06T03:01:42.285706+0000","last_became_active":"2023-10-06T03:01:42.279857+0000","last_became_peered":"2023-10-06T03:01:42.279857+0000","last_unstale":"2023-10-06T03:01:42.285706+0000","last_undegraded":"2023-10-06T03:01:42.285706+0000","last_fullsized":"2023-10-06T03:01:42.285706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284102+0000","last_change":"2023-10-06T03:01:42.278122+0000","last_active":"2023-10-06T03:01:42.284102+0000","last_peered":"2023-10-06T03:01:42.284102+0000","last_clean":"2023-10-06T03:01:42.284102+0000","last_became_active":"2023-10-06T03:01:42.277860+0000","last_became_peered":"2023-10-06T03:01:42.277860+0000","last_unstale":"2023-10-06T03:01:42.284102+0000","last_undegraded":"2023-10-06T03:01:42.284102+0000","last_fullsized":"2023-10-06T03:01:42.284102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:53:23.395145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283307+0000","last_change":"2023-10-06T03:01:42.281950+0000","last_active":"2023-10-06T03:01:42.283307+0000","last_peered":"2023-10-06T03:01:42.283307+0000","last_clean":"2023-10-06T03:01:42.283307+0000","last_became_active":"2023-10-06T03:01:42.281696+0000","last_became_peered":"2023-10-06T03:01:42.281696+0000","last_unstale":"2023-10-06T03:01:42.283307+0000","last_undegraded":"2023-10-06T03:01:42.283307+0000","last_fullsized":"2023-10-06T03:01:42.283307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:22.656089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284642+0000","last_change":"2023-10-06T03:01:42.281651+0000","last_active":"2023-10-06T03:01:42.284642+0000","last_peered":"2023-10-06T03:01:42.284642+0000","last_clean":"2023-10-06T03:01:42.284642+0000","last_became_active":"2023-10-06T03:01:42.278739+0000","last_became_peered":"2023-10-06T03:01:42.278739+0000","last_unstale":"2023-10-06T03:01:42.284642+0000","last_undegraded":"2023-10-06T03:01:42.284642+0000","last_fullsized":"2023-10-06T03:01:42.284642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:22.011159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286267+0000","last_change":"2023-10-06T03:01:42.283331+0000","last_active":"2023-10-06T03:01:42.286267+0000","last_peered":"2023-10-06T03:01:42.286267+0000","last_clean":"2023-10-06T03:01:42.286267+0000","last_became_active":"2023-10-06T03:01:42.283048+0000","last_became_peered":"2023-10-06T03:01:42.283048+0000","last_unstale":"2023-10-06T03:01:42.286267+0000","last_undegraded":"2023-10-06T03:01:42.286267+0000","last_fullsized":"2023-10-06T03:01:42.286267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285977+0000","last_change":"2023-10-06T03:01:42.282637+0000","last_active":"2023-10-06T03:01:42.285977+0000","last_peered":"2023-10-06T03:01:42.285977+0000","last_clean":"2023-10-06T03:01:42.285977+0000","last_became_active":"2023-10-06T03:01:42.282269+0000","last_became_peered":"2023-10-06T03:01:42.282269+0000","last_unstale":"2023-10-06T03:01:42.285977+0000","last_undegraded":"2023-10-06T03:01:42.285977+0000","last_fullsized":"2023-10-06T03:01:42.285977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281082+0000","last_change":"2023-10-06T03:01:42.279886+0000","last_active":"2023-10-06T03:01:42.281082+0000","last_peered":"2023-10-06T03:01:42.281082+0000","last_clean":"2023-10-06T03:01:42.281082+0000","last_became_active":"2023-10-06T03:01:42.279246+0000","last_became_peered":"2023-10-06T03:01:42.279246+0000","last_unstale":"2023-10-06T03:01:42.281082+0000","last_undegraded":"2023-10-06T03:01:42.281082+0000","last_fullsized":"2023-10-06T03:01:42.281082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286308+0000","last_change":"2023-10-06T03:01:42.283900+0000","last_active":"2023-10-06T03:01:42.286308+0000","last_peered":"2023-10-06T03:01:42.286308+0000","last_clean":"2023-10-06T03:01:42.286308+0000","last_became_active":"2023-10-06T03:01:42.283581+0000","last_became_peered":"2023-10-06T03:01:42.283581+0000","last_unstale":"2023-10-06T03:01:42.286308+0000","last_undegraded":"2023-10-06T03:01:42.286308+0000","last_fullsized":"2023-10-06T03:01:42.286308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287866+0000","last_change":"2023-10-06T03:01:42.277988+0000","last_active":"2023-10-06T03:01:42.287866+0000","last_peered":"2023-10-06T03:01:42.287866+0000","last_clean":"2023-10-06T03:01:42.287866+0000","last_became_active":"2023-10-06T03:01:42.277260+0000","last_became_peered":"2023-10-06T03:01:42.277260+0000","last_unstale":"2023-10-06T03:01:42.287866+0000","last_undegraded":"2023-10-06T03:01:42.287866+0000","last_fullsized":"2023-10-06T03:01:42.287866+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287998+0000","last_change":"2023-10-06T03:01:42.280682+0000","last_active":"2023-10-06T03:01:42.287998+0000","last_peered":"2023-10-06T03:01:42.287998+0000","last_clean":"2023-10-06T03:01:42.287998+0000","last_became_active":"2023-10-06T03:01:42.280427+0000","last_became_peered":"2023-10-06T03:01:42.280427+0000","last_unstale":"2023-10-06T03:01:42.287998+0000","last_undegraded":"2023-10-06T03:01:42.287998+0000","last_fullsized":"2023-10-06T03:01:42.287998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:27:05.130474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286079+0000","last_change":"2023-10-06T03:01:42.280012+0000","last_active":"2023-10-06T03:01:42.286079+0000","last_peered":"2023-10-06T03:01:42.286079+0000","last_clean":"2023-10-06T03:01:42.286079+0000","last_became_active":"2023-10-06T03:01:42.279744+0000","last_became_peered":"2023-10-06T03:01:42.279744+0000","last_unstale":"2023-10-06T03:01:42.286079+0000","last_undegraded":"2023-10-06T03:01:42.286079+0000","last_fullsized":"2023-10-06T03:01:42.286079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373578+0000","last_change":"2023-10-06T03:01:41.380874+0000","last_active":"2023-10-06T03:01:42.373578+0000","last_peered":"2023-10-06T03:01:42.373578+0000","last_clean":"2023-10-06T03:01:42.373578+0000","last_became_active":"2023-10-06T03:01:41.379061+0000","last_became_peered":"2023-10-06T03:01:41.379061+0000","last_unstale":"2023-10-06T03:01:42.373578+0000","last_undegraded":"2023-10-06T03:01:42.373578+0000","last_fullsized":"2023-10-06T03:01:42.373578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280483+0000","last_change":"2023-10-06T03:01:42.276652+0000","last_active":"2023-10-06T03:01:42.280483+0000","last_peered":"2023-10-06T03:01:42.280483+0000","last_clean":"2023-10-06T03:01:42.280483+0000","last_became_active":"2023-10-06T03:01:42.276378+0000","last_became_peered":"2023-10-06T03:01:42.276378+0000","last_unstale":"2023-10-06T03:01:42.280483+0000","last_undegraded":"2023-10-06T03:01:42.280483+0000","last_fullsized":"2023-10-06T03:01:42.280483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270139+0000","last_change":"2023-10-06T03:01:41.379177+0000","last_active":"2023-10-06T03:01:42.270139+0000","last_peered":"2023-10-06T03:01:42.270139+0000","last_clean":"2023-10-06T03:01:42.270139+0000","last_became_active":"2023-10-06T03:01:41.378913+0000","last_became_peered":"2023-10-06T03:01:41.378913+0000","last_unstale":"2023-10-06T03:01:42.270139+0000","last_undegraded":"2023-10-06T03:01:42.270139+0000","last_fullsized":"2023-10-06T03:01:42.270139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286002+0000","last_change":"2023-10-06T03:01:42.284965+0000","last_active":"2023-10-06T03:01:42.286002+0000","last_peered":"2023-10-06T03:01:42.286002+0000","last_clean":"2023-10-06T03:01:42.286002+0000","last_became_active":"2023-10-06T03:01:42.284715+0000","last_became_peered":"2023-10-06T03:01:42.284715+0000","last_unstale":"2023-10-06T03:01:42.286002+0000","last_undegraded":"2023-10-06T03:01:42.286002+0000","last_fullsized":"2023-10-06T03:01:42.286002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268732+0000","last_change":"2023-10-06T03:01:41.383082+0000","last_active":"2023-10-06T03:01:42.268732+0000","last_peered":"2023-10-06T03:01:42.268732+0000","last_clean":"2023-10-06T03:01:42.268732+0000","last_became_active":"2023-10-06T03:01:41.382776+0000","last_became_peered":"2023-10-06T03:01:41.382776+0000","last_unstale":"2023-10-06T03:01:42.268732+0000","last_undegraded":"2023-10-06T03:01:42.268732+0000","last_fullsized":"2023-10-06T03:01:42.268732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285524+0000","last_change":"2023-10-06T03:01:42.283247+0000","last_active":"2023-10-06T03:01:42.285524+0000","last_peered":"2023-10-06T03:01:42.285524+0000","last_clean":"2023-10-06T03:01:42.285524+0000","last_became_active":"2023-10-06T03:01:42.282951+0000","last_became_peered":"2023-10-06T03:01:42.282951+0000","last_unstale":"2023-10-06T03:01:42.285524+0000","last_undegraded":"2023-10-06T03:01:42.285524+0000","last_fullsized":"2023-10-06T03:01:42.285524+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269263+0000","last_change":"2023-10-06T03:01:41.381863+0000","last_active":"2023-10-06T03:01:42.269263+0000","last_peered":"2023-10-06T03:01:42.269263+0000","last_clean":"2023-10-06T03:01:42.269263+0000","last_became_active":"2023-10-06T03:01:41.381602+0000","last_became_peered":"2023-10-06T03:01:41.381602+0000","last_unstale":"2023-10-06T03:01:42.269263+0000","last_undegraded":"2023-10-06T03:01:42.269263+0000","last_fullsized":"2023-10-06T03:01:42.269263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280585+0000","last_change":"2023-10-06T03:01:42.276543+0000","last_active":"2023-10-06T03:01:42.280585+0000","last_peered":"2023-10-06T03:01:42.280585+0000","last_clean":"2023-10-06T03:01:42.280585+0000","last_became_active":"2023-10-06T03:01:42.276256+0000","last_became_peered":"2023-10-06T03:01:42.276256+0000","last_unstale":"2023-10-06T03:01:42.280585+0000","last_undegraded":"2023-10-06T03:01:42.280585+0000","last_fullsized":"2023-10-06T03:01:42.280585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267847+0000","last_change":"2023-10-06T03:01:41.378556+0000","last_active":"2023-10-06T03:01:42.267847+0000","last_peered":"2023-10-06T03:01:42.267847+0000","last_clean":"2023-10-06T03:01:42.267847+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.267847+0000","last_undegraded":"2023-10-06T03:01:42.267847+0000","last_fullsized":"2023-10-06T03:01:42.267847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:30.948926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287982+0000","last_change":"2023-10-06T03:01:42.283975+0000","last_active":"2023-10-06T03:01:42.287982+0000","last_peered":"2023-10-06T03:01:42.287982+0000","last_clean":"2023-10-06T03:01:42.287982+0000","last_became_active":"2023-10-06T03:01:42.283734+0000","last_became_peered":"2023-10-06T03:01:42.283734+0000","last_unstale":"2023-10-06T03:01:42.287982+0000","last_undegraded":"2023-10-06T03:01:42.287982+0000","last_fullsized":"2023-10-06T03:01:42.287982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.374949+0000","last_change":"2023-10-06T03:01:41.384246+0000","last_active":"2023-10-06T03:01:42.374949+0000","last_peered":"2023-10-06T03:01:42.374949+0000","last_clean":"2023-10-06T03:01:42.374949+0000","last_became_active":"2023-10-06T03:01:41.383977+0000","last_became_peered":"2023-10-06T03:01:41.383977+0000","last_unstale":"2023-10-06T03:01:42.374949+0000","last_undegraded":"2023-10-06T03:01:42.374949+0000","last_fullsized":"2023-10-06T03:01:42.374949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.372963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287980+0000","last_change":"2023-10-06T03:01:42.280871+0000","last_active":"2023-10-06T03:01:42.287980+0000","last_peered":"2023-10-06T03:01:42.287980+0000","last_clean":"2023-10-06T03:01:42.287980+0000","last_became_active":"2023-10-06T03:01:42.277563+0000","last_became_peered":"2023-10-06T03:01:42.277563+0000","last_unstale":"2023-10-06T03:01:42.287980+0000","last_undegraded":"2023-10-06T03:01:42.287980+0000","last_fullsized":"2023-10-06T03:01:42.287980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.379545+0000","last_change":"2023-10-06T03:01:41.381401+0000","last_active":"2023-10-06T03:01:42.379545+0000","last_peered":"2023-10-06T03:01:42.379545+0000","last_clean":"2023-10-06T03:01:42.379545+0000","last_became_active":"2023-10-06T03:01:41.379277+0000","last_became_peered":"2023-10-06T03:01:41.379277+0000","last_unstale":"2023-10-06T03:01:42.379545+0000","last_undegraded":"2023-10-06T03:01:42.379545+0000","last_fullsized":"2023-10-06T03:01:42.379545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284666+0000","last_change":"2023-10-06T03:01:42.281115+0000","last_active":"2023-10-06T03:01:42.284666+0000","last_peered":"2023-10-06T03:01:42.284666+0000","last_clean":"2023-10-06T03:01:42.284666+0000","last_became_active":"2023-10-06T03:01:42.278708+0000","last_became_peered":"2023-10-06T03:01:42.278708+0000","last_unstale":"2023-10-06T03:01:42.284666+0000","last_undegraded":"2023-10-06T03:01:42.284666+0000","last_fullsized":"2023-10-06T03:01:42.284666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:45.231633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377978+0000","last_change":"2023-10-06T03:01:41.384981+0000","last_active":"2023-10-06T03:01:42.377978+0000","last_peered":"2023-10-06T03:01:42.377978+0000","last_clean":"2023-10-06T03:01:42.377978+0000","last_became_active":"2023-10-06T03:01:41.384739+0000","last_became_peered":"2023-10-06T03:01:41.384739+0000","last_unstale":"2023-10-06T03:01:42.377978+0000","last_undegraded":"2023-10-06T03:01:42.377978+0000","last_fullsized":"2023-10-06T03:01:42.377978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280582+0000","last_change":"2023-10-06T03:01:42.276570+0000","last_active":"2023-10-06T03:01:42.280582+0000","last_peered":"2023-10-06T03:01:42.280582+0000","last_clean":"2023-10-06T03:01:42.280582+0000","last_became_active":"2023-10-06T03:01:42.276163+0000","last_became_peered":"2023-10-06T03:01:42.276163+0000","last_unstale":"2023-10-06T03:01:42.280582+0000","last_undegraded":"2023-10-06T03:01:42.280582+0000","last_fullsized":"2023-10-06T03:01:42.280582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.272049+0000","last_change":"2023-10-06T03:01:41.381162+0000","last_active":"2023-10-06T03:01:42.272049+0000","last_peered":"2023-10-06T03:01:42.272049+0000","last_clean":"2023-10-06T03:01:42.272049+0000","last_became_active":"2023-10-06T03:01:41.380885+0000","last_became_peered":"2023-10-06T03:01:41.380885+0000","last_unstale":"2023-10-06T03:01:42.272049+0000","last_undegraded":"2023-10-06T03:01:42.272049+0000","last_fullsized":"2023-10-06T03:01:42.272049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286056+0000","last_change":"2023-10-06T03:01:42.279843+0000","last_active":"2023-10-06T03:01:42.286056+0000","last_peered":"2023-10-06T03:01:42.286056+0000","last_clean":"2023-10-06T03:01:42.286056+0000","last_became_active":"2023-10-06T03:01:42.279409+0000","last_became_peered":"2023-10-06T03:01:42.279409+0000","last_unstale":"2023-10-06T03:01:42.286056+0000","last_undegraded":"2023-10-06T03:01:42.286056+0000","last_fullsized":"2023-10-06T03:01:42.286056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.275415+0000","last_change":"2023-10-06T03:01:41.382403+0000","last_active":"2023-10-06T03:01:42.275415+0000","last_peered":"2023-10-06T03:01:42.275415+0000","last_clean":"2023-10-06T03:01:42.275415+0000","last_became_active":"2023-10-06T03:01:41.382119+0000","last_became_peered":"2023-10-06T03:01:41.382119+0000","last_unstale":"2023-10-06T03:01:42.275415+0000","last_undegraded":"2023-10-06T03:01:42.275415+0000","last_fullsized":"2023-10-06T03:01:42.275415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:17.777678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285768+0000","last_change":"2023-10-06T03:01:42.280092+0000","last_active":"2023-10-06T03:01:42.285768+0000","last_peered":"2023-10-06T03:01:42.285768+0000","last_clean":"2023-10-06T03:01:42.285768+0000","last_became_active":"2023-10-06T03:01:42.279833+0000","last_became_peered":"2023-10-06T03:01:42.279833+0000","last_unstale":"2023-10-06T03:01:42.285768+0000","last_undegraded":"2023-10-06T03:01:42.285768+0000","last_fullsized":"2023-10-06T03:01:42.285768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.375608+0000","last_change":"2023-10-06T03:01:41.386282+0000","last_active":"2023-10-06T03:01:42.375608+0000","last_peered":"2023-10-06T03:01:42.375608+0000","last_clean":"2023-10-06T03:01:42.375608+0000","last_became_active":"2023-10-06T03:01:41.385955+0000","last_became_peered":"2023-10-06T03:01:41.385955+0000","last_unstale":"2023-10-06T03:01:42.375608+0000","last_undegraded":"2023-10-06T03:01:42.375608+0000","last_fullsized":"2023-10-06T03:01:42.375608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:41.588556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287940+0000","last_change":"2023-10-06T03:01:42.283470+0000","last_active":"2023-10-06T03:01:42.287940+0000","last_peered":"2023-10-06T03:01:42.287940+0000","last_clean":"2023-10-06T03:01:42.287940+0000","last_became_active":"2023-10-06T03:01:42.283220+0000","last_became_peered":"2023-10-06T03:01:42.283220+0000","last_unstale":"2023-10-06T03:01:42.287940+0000","last_undegraded":"2023-10-06T03:01:42.287940+0000","last_fullsized":"2023-10-06T03:01:42.287940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:49.819662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373872+0000","last_change":"2023-10-06T03:01:41.380049+0000","last_active":"2023-10-06T03:01:42.373872+0000","last_peered":"2023-10-06T03:01:42.373872+0000","last_clean":"2023-10-06T03:01:42.373872+0000","last_became_active":"2023-10-06T03:01:41.379732+0000","last_became_peered":"2023-10-06T03:01:41.379732+0000","last_unstale":"2023-10-06T03:01:42.373872+0000","last_undegraded":"2023-10-06T03:01:42.373872+0000","last_fullsized":"2023-10-06T03:01:42.373872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:03.057587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.286048+0000","last_change":"2023-10-06T03:01:42.279660+0000","last_active":"2023-10-06T03:01:42.286048+0000","last_peered":"2023-10-06T03:01:42.286048+0000","last_clean":"2023-10-06T03:01:42.286048+0000","last_became_active":"2023-10-06T03:01:42.279394+0000","last_became_peered":"2023-10-06T03:01:42.279394+0000","last_unstale":"2023-10-06T03:01:42.286048+0000","last_undegraded":"2023-10-06T03:01:42.286048+0000","last_fullsized":"2023-10-06T03:01:42.286048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.377939+0000","last_change":"2023-10-06T03:01:41.384922+0000","last_active":"2023-10-06T03:01:42.377939+0000","last_peered":"2023-10-06T03:01:42.377939+0000","last_clean":"2023-10-06T03:01:42.377939+0000","last_became_active":"2023-10-06T03:01:41.384669+0000","last_became_peered":"2023-10-06T03:01:41.384669+0000","last_unstale":"2023-10-06T03:01:42.377939+0000","last_undegraded":"2023-10-06T03:01:42.377939+0000","last_fullsized":"2023-10-06T03:01:42.377939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:07.334690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284069+0000","last_change":"2023-10-06T03:01:42.277487+0000","last_active":"2023-10-06T03:01:42.284069+0000","last_peered":"2023-10-06T03:01:42.284069+0000","last_clean":"2023-10-06T03:01:42.284069+0000","last_became_active":"2023-10-06T03:01:42.277135+0000","last_became_peered":"2023-10-06T03:01:42.277135+0000","last_unstale":"2023-10-06T03:01:42.284069+0000","last_undegraded":"2023-10-06T03:01:42.284069+0000","last_fullsized":"2023-10-06T03:01:42.284069+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:13.688192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373790+0000","last_change":"2023-10-06T03:01:41.380292+0000","last_active":"2023-10-06T03:01:42.373790+0000","last_peered":"2023-10-06T03:01:42.373790+0000","last_clean":"2023-10-06T03:01:42.373790+0000","last_became_active":"2023-10-06T03:01:41.380074+0000","last_became_peered":"2023-10-06T03:01:41.380074+0000","last_unstale":"2023-10-06T03:01:42.373790+0000","last_undegraded":"2023-10-06T03:01:42.373790+0000","last_fullsized":"2023-10-06T03:01:42.373790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:27.982776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.282615+0000","last_change":"2023-10-06T03:01:42.277855+0000","last_active":"2023-10-06T03:01:42.282615+0000","last_peered":"2023-10-06T03:01:42.282615+0000","last_clean":"2023-10-06T03:01:42.282615+0000","last_became_active":"2023-10-06T03:01:42.277539+0000","last_became_peered":"2023-10-06T03:01:42.277539+0000","last_unstale":"2023-10-06T03:01:42.282615+0000","last_undegraded":"2023-10-06T03:01:42.282615+0000","last_fullsized":"2023-10-06T03:01:42.282615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:04.061458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268326+0000","last_change":"2023-10-06T03:01:41.379182+0000","last_active":"2023-10-06T03:01:42.268326+0000","last_peered":"2023-10-06T03:01:42.268326+0000","last_clean":"2023-10-06T03:01:42.268326+0000","last_became_active":"2023-10-06T03:01:41.378857+0000","last_became_peered":"2023-10-06T03:01:41.378857+0000","last_unstale":"2023-10-06T03:01:42.268326+0000","last_undegraded":"2023-10-06T03:01:42.268326+0000","last_fullsized":"2023-10-06T03:01:42.268326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.281816+0000","last_change":"2023-10-06T03:01:42.277028+0000","last_active":"2023-10-06T03:01:42.281816+0000","last_peered":"2023-10-06T03:01:42.281816+0000","last_clean":"2023-10-06T03:01:42.281816+0000","last_became_active":"2023-10-06T03:01:42.276777+0000","last_became_peered":"2023-10-06T03:01:42.276777+0000","last_unstale":"2023-10-06T03:01:42.281816+0000","last_undegraded":"2023-10-06T03:01:42.281816+0000","last_fullsized":"2023-10-06T03:01:42.281816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:05.812625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.371956+0000","last_change":"2023-10-06T03:01:41.383140+0000","last_active":"2023-10-06T03:01:42.371956+0000","last_peered":"2023-10-06T03:01:42.371956+0000","last_clean":"2023-10-06T03:01:42.371956+0000","last_became_active":"2023-10-06T03:01:41.382894+0000","last_became_peered":"2023-10-06T03:01:41.382894+0000","last_unstale":"2023-10-06T03:01:42.371956+0000","last_undegraded":"2023-10-06T03:01:42.371956+0000","last_fullsized":"2023-10-06T03:01:42.371956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284623+0000","last_change":"2023-10-06T03:01:42.280965+0000","last_active":"2023-10-06T03:01:42.284623+0000","last_peered":"2023-10-06T03:01:42.284623+0000","last_clean":"2023-10-06T03:01:42.284623+0000","last_became_active":"2023-10-06T03:01:42.278565+0000","last_became_peered":"2023-10-06T03:01:42.278565+0000","last_unstale":"2023-10-06T03:01:42.284623+0000","last_undegraded":"2023-10-06T03:01:42.284623+0000","last_fullsized":"2023-10-06T03:01:42.284623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:51.765176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274323+0000","last_change":"2023-10-06T03:01:41.385727+0000","last_active":"2023-10-06T03:01:42.274323+0000","last_peered":"2023-10-06T03:01:42.274323+0000","last_clean":"2023-10-06T03:01:42.274323+0000","last_became_active":"2023-10-06T03:01:41.385354+0000","last_became_peered":"2023-10-06T03:01:41.385354+0000","last_unstale":"2023-10-06T03:01:42.274323+0000","last_undegraded":"2023-10-06T03:01:42.274323+0000","last_fullsized":"2023-10-06T03:01:42.274323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284128+0000","last_change":"2023-10-06T03:01:42.278082+0000","last_active":"2023-10-06T03:01:42.284128+0000","last_peered":"2023-10-06T03:01:42.284128+0000","last_clean":"2023-10-06T03:01:42.284128+0000","last_became_active":"2023-10-06T03:01:42.277812+0000","last_became_peered":"2023-10-06T03:01:42.277812+0000","last_unstale":"2023-10-06T03:01:42.284128+0000","last_undegraded":"2023-10-06T03:01:42.284128+0000","last_fullsized":"2023-10-06T03:01:42.284128+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:32.435628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.270092+0000","last_change":"2023-10-06T03:01:41.381181+0000","last_active":"2023-10-06T03:01:42.270092+0000","last_peered":"2023-10-06T03:01:42.270092+0000","last_clean":"2023-10-06T03:01:42.270092+0000","last_became_active":"2023-10-06T03:01:41.378547+0000","last_became_peered":"2023-10-06T03:01:41.378547+0000","last_unstale":"2023-10-06T03:01:42.270092+0000","last_undegraded":"2023-10-06T03:01:42.270092+0000","last_fullsized":"2023-10-06T03:01:42.270092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284704+0000","last_change":"2023-10-06T03:01:42.281464+0000","last_active":"2023-10-06T03:01:42.284704+0000","last_peered":"2023-10-06T03:01:42.284704+0000","last_clean":"2023-10-06T03:01:42.284704+0000","last_became_active":"2023-10-06T03:01:42.281186+0000","last_became_peered":"2023-10-06T03:01:42.281186+0000","last_unstale":"2023-10-06T03:01:42.284704+0000","last_undegraded":"2023-10-06T03:01:42.284704+0000","last_fullsized":"2023-10-06T03:01:42.284704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:33.673116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268609+0000","last_change":"2023-10-06T03:01:41.378017+0000","last_active":"2023-10-06T03:01:42.268609+0000","last_peered":"2023-10-06T03:01:42.268609+0000","last_clean":"2023-10-06T03:01:42.268609+0000","last_became_active":"2023-10-06T03:01:41.377759+0000","last_became_peered":"2023-10-06T03:01:41.377759+0000","last_unstale":"2023-10-06T03:01:42.268609+0000","last_undegraded":"2023-10-06T03:01:42.268609+0000","last_fullsized":"2023-10-06T03:01:42.268609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.287848+0000","last_change":"2023-10-06T03:01:42.278383+0000","last_active":"2023-10-06T03:01:42.287848+0000","last_peered":"2023-10-06T03:01:42.287848+0000","last_clean":"2023-10-06T03:01:42.287848+0000","last_became_active":"2023-10-06T03:01:42.277973+0000","last_became_peered":"2023-10-06T03:01:42.277973+0000","last_unstale":"2023-10-06T03:01:42.287848+0000","last_undegraded":"2023-10-06T03:01:42.287848+0000","last_fullsized":"2023-10-06T03:01:42.287848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271774+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:01:42.271774+0000","last_peered":"2023-10-06T03:01:42.271774+0000","last_clean":"2023-10-06T03:01:42.271774+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:01:42.271774+0000","last_undegraded":"2023-10-06T03:01:42.271774+0000","last_fullsized":"2023-10-06T03:01:42.271774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.267761+0000","last_change":"2023-10-06T03:01:41.379310+0000","last_active":"2023-10-06T03:01:42.267761+0000","last_peered":"2023-10-06T03:01:42.267761+0000","last_clean":"2023-10-06T03:01:42.267761+0000","last_became_active":"2023-10-06T03:01:41.379017+0000","last_became_peered":"2023-10-06T03:01:41.379017+0000","last_unstale":"2023-10-06T03:01:42.267761+0000","last_undegraded":"2023-10-06T03:01:42.267761+0000","last_fullsized":"2023-10-06T03:01:42.267761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288159+0000","last_change":"2023-10-06T03:01:42.285640+0000","last_active":"2023-10-06T03:01:42.288159+0000","last_peered":"2023-10-06T03:01:42.288159+0000","last_clean":"2023-10-06T03:01:42.288159+0000","last_became_active":"2023-10-06T03:01:42.283071+0000","last_became_peered":"2023-10-06T03:01:42.283071+0000","last_unstale":"2023-10-06T03:01:42.288159+0000","last_undegraded":"2023-10-06T03:01:42.288159+0000","last_fullsized":"2023-10-06T03:01:42.288159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:41.523347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269088+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:01:42.269088+0000","last_peered":"2023-10-06T03:01:42.269088+0000","last_clean":"2023-10-06T03:01:42.269088+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:01:42.269088+0000","last_undegraded":"2023-10-06T03:01:42.269088+0000","last_fullsized":"2023-10-06T03:01:42.269088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268853+0000","last_change":"2023-10-06T03:01:41.382782+0000","last_active":"2023-10-06T03:01:42.268853+0000","last_peered":"2023-10-06T03:01:42.268853+0000","last_clean":"2023-10-06T03:01:42.268853+0000","last_became_active":"2023-10-06T03:01:41.382522+0000","last_became_peered":"2023-10-06T03:01:41.382522+0000","last_unstale":"2023-10-06T03:01:42.268853+0000","last_undegraded":"2023-10-06T03:01:42.268853+0000","last_fullsized":"2023-10-06T03:01:42.268853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288056+0000","last_change":"2023-10-06T03:01:42.282263+0000","last_active":"2023-10-06T03:01:42.288056+0000","last_peered":"2023-10-06T03:01:42.288056+0000","last_clean":"2023-10-06T03:01:42.288056+0000","last_became_active":"2023-10-06T03:01:42.282019+0000","last_became_peered":"2023-10-06T03:01:42.282019+0000","last_unstale":"2023-10-06T03:01:42.288056+0000","last_undegraded":"2023-10-06T03:01:42.288056+0000","last_fullsized":"2023-10-06T03:01:42.288056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.265449+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:01:42.265449+0000","last_peered":"2023-10-06T03:01:42.265449+0000","last_clean":"2023-10-06T03:01:42.265449+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:01:42.265449+0000","last_undegraded":"2023-10-06T03:01:42.265449+0000","last_fullsized":"2023-10-06T03:01:42.265449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268243+0000","last_change":"2023-10-06T03:01:41.378349+0000","last_active":"2023-10-06T03:01:42.268243+0000","last_peered":"2023-10-06T03:01:42.268243+0000","last_clean":"2023-10-06T03:01:42.268243+0000","last_became_active":"2023-10-06T03:01:41.378089+0000","last_became_peered":"2023-10-06T03:01:41.378089+0000","last_unstale":"2023-10-06T03:01:42.268243+0000","last_undegraded":"2023-10-06T03:01:42.268243+0000","last_fullsized":"2023-10-06T03:01:42.268243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:17.130594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288080+0000","last_change":"2023-10-06T03:01:42.283418+0000","last_active":"2023-10-06T03:01:42.288080+0000","last_peered":"2023-10-06T03:01:42.288080+0000","last_clean":"2023-10-06T03:01:42.288080+0000","last_became_active":"2023-10-06T03:01:42.283035+0000","last_became_peered":"2023-10-06T03:01:42.283035+0000","last_unstale":"2023-10-06T03:01:42.288080+0000","last_undegraded":"2023-10-06T03:01:42.288080+0000","last_fullsized":"2023-10-06T03:01:42.288080+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:27.703859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.266739+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:01:42.266739+0000","last_peered":"2023-10-06T03:01:42.266739+0000","last_clean":"2023-10-06T03:01:42.266739+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:01:42.266739+0000","last_undegraded":"2023-10-06T03:01:42.266739+0000","last_fullsized":"2023-10-06T03:01:42.266739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274310+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:01:42.274310+0000","last_peered":"2023-10-06T03:01:42.274310+0000","last_clean":"2023-10-06T03:01:42.274310+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:01:42.274310+0000","last_undegraded":"2023-10-06T03:01:42.274310+0000","last_fullsized":"2023-10-06T03:01:42.274310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268619+0000","last_change":"2023-10-06T03:01:41.378526+0000","last_active":"2023-10-06T03:01:42.268619+0000","last_peered":"2023-10-06T03:01:42.268619+0000","last_clean":"2023-10-06T03:01:42.268619+0000","last_became_active":"2023-10-06T03:01:41.378245+0000","last_became_peered":"2023-10-06T03:01:41.378245+0000","last_unstale":"2023-10-06T03:01:42.268619+0000","last_undegraded":"2023-10-06T03:01:42.268619+0000","last_fullsized":"2023-10-06T03:01:42.268619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284302+0000","last_change":"2023-10-06T03:01:42.281327+0000","last_active":"2023-10-06T03:01:42.284302+0000","last_peered":"2023-10-06T03:01:42.284302+0000","last_clean":"2023-10-06T03:01:42.284302+0000","last_became_active":"2023-10-06T03:01:42.281028+0000","last_became_peered":"2023-10-06T03:01:42.281028+0000","last_unstale":"2023-10-06T03:01:42.284302+0000","last_undegraded":"2023-10-06T03:01:42.284302+0000","last_fullsized":"2023-10-06T03:01:42.284302+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:28.986647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.274776+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:01:42.274776+0000","last_peered":"2023-10-06T03:01:42.274776+0000","last_clean":"2023-10-06T03:01:42.274776+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:01:42.274776+0000","last_undegraded":"2023-10-06T03:01:42.274776+0000","last_fullsized":"2023-10-06T03:01:42.274776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.268368+0000","last_change":"2023-10-06T03:01:41.377764+0000","last_active":"2023-10-06T03:01:42.268368+0000","last_peered":"2023-10-06T03:01:42.268368+0000","last_clean":"2023-10-06T03:01:42.268368+0000","last_became_active":"2023-10-06T03:01:41.377501+0000","last_became_peered":"2023-10-06T03:01:41.377501+0000","last_unstale":"2023-10-06T03:01:42.268368+0000","last_undegraded":"2023-10-06T03:01:42.268368+0000","last_fullsized":"2023-10-06T03:01:42.268368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285753+0000","last_change":"2023-10-06T03:01:42.279939+0000","last_active":"2023-10-06T03:01:42.285753+0000","last_peered":"2023-10-06T03:01:42.285753+0000","last_clean":"2023-10-06T03:01:42.285753+0000","last_became_active":"2023-10-06T03:01:42.279453+0000","last_became_peered":"2023-10-06T03:01:42.279453+0000","last_unstale":"2023-10-06T03:01:42.285753+0000","last_undegraded":"2023-10-06T03:01:42.285753+0000","last_fullsized":"2023-10-06T03:01:42.285753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:09.963570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.276154+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:01:42.276154+0000","last_peered":"2023-10-06T03:01:42.276154+0000","last_clean":"2023-10-06T03:01:42.276154+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:01:42.276154+0000","last_undegraded":"2023-10-06T03:01:42.276154+0000","last_fullsized":"2023-10-06T03:01:42.276154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.271880+0000","last_change":"2023-10-06T03:01:41.378694+0000","last_active":"2023-10-06T03:01:42.271880+0000","last_peered":"2023-10-06T03:01:42.271880+0000","last_clean":"2023-10-06T03:01:42.271880+0000","last_became_active":"2023-10-06T03:01:41.378366+0000","last_became_peered":"2023-10-06T03:01:41.378366+0000","last_unstale":"2023-10-06T03:01:42.271880+0000","last_undegraded":"2023-10-06T03:01:42.271880+0000","last_fullsized":"2023-10-06T03:01:42.271880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:31.480590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.285642+0000","last_change":"2023-10-06T03:01:42.278204+0000","last_active":"2023-10-06T03:01:42.285642+0000","last_peered":"2023-10-06T03:01:42.285642+0000","last_clean":"2023-10-06T03:01:42.285642+0000","last_became_active":"2023-10-06T03:01:42.277950+0000","last_became_peered":"2023-10-06T03:01:42.277950+0000","last_unstale":"2023-10-06T03:01:42.285642+0000","last_undegraded":"2023-10-06T03:01:42.285642+0000","last_fullsized":"2023-10-06T03:01:42.285642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378135+0000","last_change":"2023-10-06T03:01:41.378450+0000","last_active":"2023-10-06T03:01:42.378135+0000","last_peered":"2023-10-06T03:01:42.378135+0000","last_clean":"2023-10-06T03:01:42.378135+0000","last_became_active":"2023-10-06T03:01:41.378167+0000","last_became_peered":"2023-10-06T03:01:41.378167+0000","last_unstale":"2023-10-06T03:01:42.378135+0000","last_undegraded":"2023-10-06T03:01:42.378135+0000","last_fullsized":"2023-10-06T03:01:42.378135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:18:52.632777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.269297+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:01:42.269297+0000","last_peered":"2023-10-06T03:01:42.269297+0000","last_clean":"2023-10-06T03:01:42.269297+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:01:42.269297+0000","last_undegraded":"2023-10-06T03:01:42.269297+0000","last_fullsized":"2023-10-06T03:01:42.269297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288032+0000","last_change":"2023-10-06T03:01:42.284704+0000","last_active":"2023-10-06T03:01:42.288032+0000","last_peered":"2023-10-06T03:01:42.288032+0000","last_clean":"2023-10-06T03:01:42.288032+0000","last_became_active":"2023-10-06T03:01:42.284380+0000","last_became_peered":"2023-10-06T03:01:42.284380+0000","last_unstale":"2023-10-06T03:01:42.288032+0000","last_undegraded":"2023-10-06T03:01:42.288032+0000","last_fullsized":"2023-10-06T03:01:42.288032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:28:30.532475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.273934+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:01:42.273934+0000","last_peered":"2023-10-06T03:01:42.273934+0000","last_clean":"2023-10-06T03:01:42.273934+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:01:42.273934+0000","last_undegraded":"2023-10-06T03:01:42.273934+0000","last_fullsized":"2023-10-06T03:01:42.273934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T03:01:43.460248+0000","last_change":"2023-10-06T03:01:41.376173+0000","last_active":"2023-10-06T03:01:43.460248+0000","last_peered":"2023-10-06T03:01:43.460248+0000","last_clean":"2023-10-06T03:01:43.460248+0000","last_became_active":"2023-10-06T03:01:41.375877+0000","last_became_peered":"2023-10-06T03:01:41.375877+0000","last_unstale":"2023-10-06T03:01:43.460248+0000","last_undegraded":"2023-10-06T03:01:43.460248+0000","last_fullsized":"2023-10-06T03:01:43.460248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:30.099017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.277029+0000","last_change":"2023-10-06T03:01:42.271755+0000","last_active":"2023-10-06T03:01:42.277029+0000","last_peered":"2023-10-06T03:01:42.277029+0000","last_clean":"2023-10-06T03:01:42.277029+0000","last_became_active":"2023-10-06T03:01:42.271483+0000","last_became_peered":"2023-10-06T03:01:42.271483+0000","last_unstale":"2023-10-06T03:01:42.277029+0000","last_undegraded":"2023-10-06T03:01:42.277029+0000","last_fullsized":"2023-10-06T03:01:42.277029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:39:12.906259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373544+0000","last_change":"2023-10-06T03:01:41.381914+0000","last_active":"2023-10-06T03:01:42.373544+0000","last_peered":"2023-10-06T03:01:42.373544+0000","last_clean":"2023-10-06T03:01:42.373544+0000","last_became_active":"2023-10-06T03:01:41.381635+0000","last_became_peered":"2023-10-06T03:01:41.381635+0000","last_unstale":"2023-10-06T03:01:42.373544+0000","last_undegraded":"2023-10-06T03:01:42.373544+0000","last_fullsized":"2023-10-06T03:01:42.373544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280515+0000","last_change":"2023-10-06T03:01:42.277000+0000","last_active":"2023-10-06T03:01:42.280515+0000","last_peered":"2023-10-06T03:01:42.280515+0000","last_clean":"2023-10-06T03:01:42.280515+0000","last_became_active":"2023-10-06T03:01:42.276746+0000","last_became_peered":"2023-10-06T03:01:42.276746+0000","last_unstale":"2023-10-06T03:01:42.280515+0000","last_undegraded":"2023-10-06T03:01:42.280515+0000","last_fullsized":"2023-10-06T03:01:42.280515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:31.952796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378477+0000","last_change":"2023-10-06T03:01:41.377203+0000","last_active":"2023-10-06T03:01:42.378477+0000","last_peered":"2023-10-06T03:01:42.378477+0000","last_clean":"2023-10-06T03:01:42.378477+0000","last_became_active":"2023-10-06T03:01:41.376950+0000","last_became_peered":"2023-10-06T03:01:41.376950+0000","last_unstale":"2023-10-06T03:01:42.378477+0000","last_undegraded":"2023-10-06T03:01:42.378477+0000","last_fullsized":"2023-10-06T03:01:42.378477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:29.246739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284682+0000","last_change":"2023-10-06T03:01:42.282305+0000","last_active":"2023-10-06T03:01:42.284682+0000","last_peered":"2023-10-06T03:01:42.284682+0000","last_clean":"2023-10-06T03:01:42.284682+0000","last_became_active":"2023-10-06T03:01:42.282037+0000","last_became_peered":"2023-10-06T03:01:42.282037+0000","last_unstale":"2023-10-06T03:01:42.284682+0000","last_undegraded":"2023-10-06T03:01:42.284682+0000","last_fullsized":"2023-10-06T03:01:42.284682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:24.802638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.378519+0000","last_change":"2023-10-06T03:01:41.377561+0000","last_active":"2023-10-06T03:01:42.378519+0000","last_peered":"2023-10-06T03:01:42.378519+0000","last_clean":"2023-10-06T03:01:42.378519+0000","last_became_active":"2023-10-06T03:01:41.377308+0000","last_became_peered":"2023-10-06T03:01:41.377308+0000","last_unstale":"2023-10-06T03:01:42.378519+0000","last_undegraded":"2023-10-06T03:01:42.378519+0000","last_fullsized":"2023-10-06T03:01:42.378519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:21.652218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284090+0000","last_change":"2023-10-06T03:01:42.277443+0000","last_active":"2023-10-06T03:01:42.284090+0000","last_peered":"2023-10-06T03:01:42.284090+0000","last_clean":"2023-10-06T03:01:42.284090+0000","last_became_active":"2023-10-06T03:01:42.277152+0000","last_became_peered":"2023-10-06T03:01:42.277152+0000","last_unstale":"2023-10-06T03:01:42.284090+0000","last_undegraded":"2023-10-06T03:01:42.284090+0000","last_fullsized":"2023-10-06T03:01:42.284090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373306+0000","last_change":"2023-10-06T03:01:41.379186+0000","last_active":"2023-10-06T03:01:42.373306+0000","last_peered":"2023-10-06T03:01:42.373306+0000","last_clean":"2023-10-06T03:01:42.373306+0000","last_became_active":"2023-10-06T03:01:41.378963+0000","last_became_peered":"2023-10-06T03:01:41.378963+0000","last_unstale":"2023-10-06T03:01:42.373306+0000","last_undegraded":"2023-10-06T03:01:42.373306+0000","last_fullsized":"2023-10-06T03:01:42.373306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:06:05.494591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280768+0000","last_change":"2023-10-06T03:01:42.276973+0000","last_active":"2023-10-06T03:01:42.280768+0000","last_peered":"2023-10-06T03:01:42.280768+0000","last_clean":"2023-10-06T03:01:42.280768+0000","last_became_active":"2023-10-06T03:01:42.276297+0000","last_became_peered":"2023-10-06T03:01:42.276297+0000","last_unstale":"2023-10-06T03:01:42.280768+0000","last_undegraded":"2023-10-06T03:01:42.280768+0000","last_fullsized":"2023-10-06T03:01:42.280768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:13.690329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.373671+0000","last_change":"2023-10-06T03:01:41.380486+0000","last_active":"2023-10-06T03:01:42.373671+0000","last_peered":"2023-10-06T03:01:42.373671+0000","last_clean":"2023-10-06T03:01:42.373671+0000","last_became_active":"2023-10-06T03:01:41.380212+0000","last_became_peered":"2023-10-06T03:01:41.380212+0000","last_unstale":"2023-10-06T03:01:42.373671+0000","last_undegraded":"2023-10-06T03:01:42.373671+0000","last_fullsized":"2023-10-06T03:01:42.373671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:40.351068+0000","last_clean_scrub_stamp":"2023-10-06T03:01:40.351068+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:13.166914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284486+0000","last_change":"2023-10-06T03:01:42.280604+0000","last_active":"2023-10-06T03:01:42.284486+0000","last_peered":"2023-10-06T03:01:42.284486+0000","last_clean":"2023-10-06T03:01:42.284486+0000","last_became_active":"2023-10-06T03:01:42.280041+0000","last_became_peered":"2023-10-06T03:01:42.280041+0000","last_unstale":"2023-10-06T03:01:42.284486+0000","last_undegraded":"2023-10-06T03:01:42.284486+0000","last_fullsized":"2023-10-06T03:01:42.284486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:02.038765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.283231+0000","last_change":"2023-10-06T03:01:42.277027+0000","last_active":"2023-10-06T03:01:42.283231+0000","last_peered":"2023-10-06T03:01:42.283231+0000","last_clean":"2023-10-06T03:01:42.283231+0000","last_became_active":"2023-10-06T03:01:42.276730+0000","last_became_peered":"2023-10-06T03:01:42.276730+0000","last_unstale":"2023-10-06T03:01:42.283231+0000","last_undegraded":"2023-10-06T03:01:42.283231+0000","last_fullsized":"2023-10-06T03:01:42.283231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:18.781548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.280787+0000","last_change":"2023-10-06T03:01:42.277145+0000","last_active":"2023-10-06T03:01:42.280787+0000","last_peered":"2023-10-06T03:01:42.280787+0000","last_clean":"2023-10-06T03:01:42.280787+0000","last_became_active":"2023-10-06T03:01:42.276735+0000","last_became_peered":"2023-10-06T03:01:42.276735+0000","last_unstale":"2023-10-06T03:01:42.280787+0000","last_undegraded":"2023-10-06T03:01:42.280787+0000","last_fullsized":"2023-10-06T03:01:42.280787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:32.949943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.284561+0000","last_change":"2023-10-06T03:01:42.281639+0000","last_active":"2023-10-06T03:01:42.284561+0000","last_peered":"2023-10-06T03:01:42.284561+0000","last_clean":"2023-10-06T03:01:42.284561+0000","last_became_active":"2023-10-06T03:01:42.278720+0000","last_became_peered":"2023-10-06T03:01:42.278720+0000","last_unstale":"2023-10-06T03:01:42.284561+0000","last_undegraded":"2023-10-06T03:01:42.284561+0000","last_fullsized":"2023-10-06T03:01:42.284561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:03.312309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288057+0000","last_change":"2023-10-06T03:01:42.283924+0000","last_active":"2023-10-06T03:01:42.288057+0000","last_peered":"2023-10-06T03:01:42.288057+0000","last_clean":"2023-10-06T03:01:42.288057+0000","last_became_active":"2023-10-06T03:01:42.283527+0000","last_became_peered":"2023-10-06T03:01:42.283527+0000","last_unstale":"2023-10-06T03:01:42.288057+0000","last_undegraded":"2023-10-06T03:01:42.288057+0000","last_fullsized":"2023-10-06T03:01:42.288057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:38.731382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T03:01:42.288138+0000","last_change":"2023-10-06T03:01:42.284183+0000","last_active":"2023-10-06T03:01:42.288138+0000","last_peered":"2023-10-06T03:01:42.288138+0000","last_clean":"2023-10-06T03:01:42.288138+0000","last_became_active":"2023-10-06T03:01:42.283895+0000","last_became_peered":"2023-10-06T03:01:42.283895+0000","last_unstale":"2023-10-06T03:01:42.288138+0000","last_undegraded":"2023-10-06T03:01:42.288138+0000","last_fullsized":"2023-10-06T03:01:42.288138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:41.354939+0000","last_clean_scrub_stamp":"2023-10-06T03:01:41.354939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:23.829909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:01:51.187 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T03:01:51.188 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T03:01:51.188 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T03:01:51.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T03:01:51.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.334+0000 7f2316206700 1 Processor -- start 2023-10-06T03:01:51.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.334+0000 7f2316206700 1 -- start start 2023-10-06T03:01:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f2316206700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2310159ec0 0x7f23100b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f2316206700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23100b53d0 0x7f23100cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f2316206700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23100b5b00 con 0x7f23100cce40 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f2316206700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23100b5850 con 0x7f23100b53d0 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f2316206700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23100b5da0 con 0x7f2310159ec0 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23100b53d0 0x7f23100cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f23100b53d0 0x7f23100cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34582/0 (socket says 172.21.15.92:34582) 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 -- 172.21.15.92:0/2817077233 learned_addr learned my addr 172.21.15.92:0/2817077233 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230f7fe700 1 --2- 172.21.15.92:0/2817077233 >> v2:172.21.15.136:3300/0 conn(0x7f2310159ec0 0x7f23100b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 -- 172.21.15.92:0/2817077233 >> v2:172.21.15.136:3300/0 conn(0x7f2310159ec0 msgr2=0x7f23100b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 --2- 172.21.15.92:0/2817077233 >> v2:172.21.15.136:3300/0 conn(0x7f2310159ec0 0x7f23100b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 -- 172.21.15.92:0/2817077233 >> v1:172.21.15.136:6789/0 conn(0x7f23100cce40 legacy=0x7f23100cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.335+0000 7f230effd700 1 -- 172.21.15.92:0/2817077233 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23100b6080 con 0x7f23100b53d0 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f230effd700 1 --2- 172.21.15.92:0/2817077233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100b53d0 0x7f23100cc900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2310089120 tx=0x7f2304009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211b72061d48acfe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:51.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f230e7fc700 1 -- 172.21.15.92:0/2817077233 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400b860 con 0x7f23100b53d0 2023-10-06T03:01:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f230e7fc700 1 -- 172.21.15.92:0/2817077233 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400b9c0 con 0x7f23100b53d0 2023-10-06T03:01:51.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f230e7fc700 1 -- 172.21.15.92:0/2817077233 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400bd60 con 0x7f23100b53d0 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 -- 172.21.15.92:0/2817077233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100b53d0 msgr2=0x7f23100cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 --2- 172.21.15.92:0/2817077233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100b53d0 0x7f23100cc900 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2310089120 tx=0x7f2304009250 comp rx=0 tx=0).stop 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 -- 172.21.15.92:0/2817077233 shutdown_connections 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 --2- 172.21.15.92:0/2817077233 >> v2:172.21.15.136:3300/0 conn(0x7f2310159ec0 0x7f23100b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 --2- 172.21.15.92:0/2817077233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100b53d0 0x7f23100cc900 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 -- 172.21.15.92:0/2817077233 >> 172.21.15.92:0/2817077233 conn(0x7f23100999f0 msgr2=0x7f23100b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 -- 172.21.15.92:0/2817077233 shutdown_connections 2023-10-06T03:01:51.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.336+0000 7f2316206700 1 -- 172.21.15.92:0/2817077233 wait complete. 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.337+0000 7f2316206700 1 Processor -- start 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.337+0000 7f2316206700 1 -- start start 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f2316206700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23100b53d0 0x7f2310154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f2316206700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 0x7f2310152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f2316206700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2310159ec0 0x7f2310152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f2316206700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231006a5a0 con 0x7f23100cce40 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f2316206700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101542a0 con 0x7f2310159ec0 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f2316206700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2310154c70 con 0x7f23100b53d0 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f230effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 0x7f2310152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f230effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 0x7f2310152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34590/0 (socket says 172.21.15.92:34590) 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f230effd700 1 -- 172.21.15.92:0/476728116 learned_addr learned my addr 172.21.15.92:0/476728116 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:51.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f230f7fe700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23100b53d0 0x7f2310154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.338+0000 7f230ffff700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2310159ec0 0x7f2310152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f230effd700 1 -- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23100b53d0 msgr2=0x7f2310154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f230effd700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23100b53d0 0x7f2310154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f230effd700 1 -- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2310159ec0 msgr2=0x7f2310152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f230effd700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2310159ec0 0x7f2310152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f230effd700 1 -- 172.21.15.92:0/476728116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23040094f0 con 0x7f23100cce40 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f230effd700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 0x7f2310152a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2310089390 tx=0x7f2304017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304024440 con 0x7f23100cce40 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23040245a0 con 0x7f23100cce40 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2310153690 con 0x7f23100cce40 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304024960 con 0x7f23100cce40 2023-10-06T03:01:51.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.339+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2310153c50 con 0x7f23100cce40 2023-10-06T03:01:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.340+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23100647c0 con 0x7f23100cce40 2023-10-06T03:01:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.340+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2304024d50 con 0x7f23100cce40 2023-10-06T03:01:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.340+0000 7f22f77fe700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22fc066f30 0x7f22fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.341+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f23040861b0 con 0x7f23100cce40 2023-10-06T03:01:51.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.341+0000 7f230f7fe700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22fc066f30 0x7f22fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.341+0000 7f230f7fe700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22fc066f30 0x7f22fc0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2300009fd0 tx=0x7f23000098a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:51.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.345+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23100647c0 con 0x7f23100cce40 2023-10-06T03:01:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.569+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f23100647c0 con 0x7f23100cce40 2023-10-06T03:01:51.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.569+0000 7f22f77fe700 1 -- 172.21.15.92:0/476728116 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f23100647c0 con 0x7f23100cce40 2023-10-06T03:01:51.571 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:51.571 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T03:01:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22fc066f30 msgr2=0x7f22fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22fc066f30 0x7f22fc0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2300009fd0 tx=0x7f23000098a0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 msgr2=0x7f2310152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 0x7f2310152a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2310089390 tx=0x7f2304017000 comp rx=0 tx=0).stop 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 shutdown_connections 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22fc066f30 0x7f22fc0693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2310159ec0 0x7f2310152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23100b53d0 0x7f2310154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 --2- 172.21.15.92:0/476728116 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23100cce40 0x7f2310152a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 >> 172.21.15.92:0/476728116 conn(0x7f23100999f0 msgr2=0x7f23100c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 shutdown_connections 2023-10-06T03:01:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.571+0000 7f2316206700 1 -- 172.21.15.92:0/476728116 wait complete. 2023-10-06T03:01:51.585 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T03:01:51.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:51.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.697+0000 7fbc2c014700 1 Processor -- start 2023-10-06T03:01:51.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc2c014700 1 -- start start 2023-10-06T03:01:51.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc2c014700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbc2406c0a0 0x7fbc240caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc2c014700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 0x7fbc2415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc2c014700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2415d7e0 con 0x7fbc2406bcd0 2023-10-06T03:01:51.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc2c014700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2415dac0 con 0x7fbc2406c0a0 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc2c014700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2415aa30 con 0x7fbc2415df50 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc29db0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbc2406bcd0 0x7fbc2415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39008/0 (socket says 172.21.15.92:39008) 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.698+0000 7fbc29db0700 1 -- 172.21.15.92:0/1919258042 learned_addr learned my addr 172.21.15.92:0/1919258042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187057179 0 0) 0x7fbc2415d7e0 con 0x7fbc2406bcd0 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc2a5b1700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 0x7fbc2415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 --2- 172.21.15.92:0/1919258042 >> v2:172.21.15.136:3300/0 conn(0x7fbc2406c0a0 0x7fbc240caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc10003570 con 0x7fbc2406bcd0 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2100575087 0 0) 0x7fbc10003570 con 0x7fbc2406bcd0 2023-10-06T03:01:51.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 msgr2=0x7fbc2415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 0x7fbc2415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 >> v2:172.21.15.136:3300/0 conn(0x7fbc2406c0a0 msgr2=0x7fbc240caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 --2- 172.21.15.92:0/1919258042 >> v2:172.21.15.136:3300/0 conn(0x7fbc2406c0a0 0x7fbc240caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2415ad10 con 0x7fbc2406bcd0 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fbc20001470 con 0x7fbc2406bcd0 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 --2- 172.21.15.92:0/1919258042 >> v2:172.21.15.136:3300/0 conn(0x7fbc2406c0a0 0x7fbc240caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 >> v1:172.21.15.92:6789/0 conn(0x7fbc2406bcd0 legacy=0x7fbc2415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 0x7fbc10003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10004d50 0x7fbc100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2415dac0 con 0x7fbc10003fe0 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2415aa30 con 0x7fbc10004d50 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1000bbf0 con 0x7fbc2415df50 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc29db0700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 0x7fbc10003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc2a5b1700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10004d50 0x7fbc100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 -- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 msgr2=0x7fbc10003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc2415df50 0x7fbc10003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 -- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10004d50 msgr2=0x7fbc100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10004d50 0x7fbc100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 -- 172.21.15.92:0/1919258042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc10003570 con 0x7fbc10003fe0 2023-10-06T03:01:51.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc2a5b1700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10004d50 0x7fbc100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.699+0000 7fbc295af700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc14006fd0 tx=0x7fbc14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d216e38bf111d275 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400c070 con 0x7fbc10003fe0 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1400e5e0 con 0x7fbc10003fe0 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc28dae700 1 -- 172.21.15.92:0/1919258042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14009510 con 0x7fbc10003fe0 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2c014700 1 -- 172.21.15.92:0/1919258042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10003fe0 msgr2=0x7fbc10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2c014700 1 --2- 172.21.15.92:0/1919258042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc10003fe0 0x7fbc10004810 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc14006fd0 tx=0x7fbc14008040 comp rx=0 tx=0).stop 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2a5b1700 1 -- 172.21.15.92:0/1919258042 reap_dead start 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2c014700 1 -- 172.21.15.92:0/1919258042 shutdown_connections 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2c014700 1 -- 172.21.15.92:0/1919258042 >> 172.21.15.92:0/1919258042 conn(0x7fbc240ce600 msgr2=0x7fbc240cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2c014700 1 -- 172.21.15.92:0/1919258042 shutdown_connections 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.700+0000 7fbc2c014700 1 -- 172.21.15.92:0/1919258042 wait complete. 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.701+0000 7fbc2c014700 1 Processor -- start 2023-10-06T03:01:51.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.701+0000 7fbc2c014700 1 -- start start 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2c014700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10003fe0 0x7fbc240b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2c014700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 0x7fbc24169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2c014700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc240b2940 0x7fbc240c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2c014700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c8160 con 0x7fbc240b24d0 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2c014700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2416a980 con 0x7fbc10003fe0 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2c014700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2416ac60 con 0x7fbc240b2940 2023-10-06T03:01:51.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 0x7fbc24169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 0x7fbc24169ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34596/0 (socket says 172.21.15.92:34596) 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 -- 172.21.15.92:0/3483957433 learned_addr learned my addr 172.21.15.92:0/3483957433 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2a5b1700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc240b2940 0x7fbc240c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc29db0700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10003fe0 0x7fbc240b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 -- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc240b2940 msgr2=0x7fbc240c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc240b2940 0x7fbc240c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 -- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10003fe0 msgr2=0x7fbc240b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10003fe0 0x7fbc240b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 -- 172.21.15.92:0/3483957433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240c8400 con 0x7fbc240b24d0 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc29db0700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10003fe0 0x7fbc240b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:51.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc2a5b1700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc240b2940 0x7fbc240c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.702+0000 7fbc295af700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 0x7fbc24169ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400a010 tx=0x7fbc1401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.703+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14009510 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.703+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14013c90 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.703+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1401c670 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.703+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc240c86e0 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.703+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc240c5c60 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.704+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc240c8a50 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.704+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc1401caa0 con 0x7fbc240b24d0 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.704+0000 7fbc19ffb700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:51.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.705+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbc14085e60 con 0x7fbc240b24d0 2023-10-06T03:01:51.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.705+0000 7fbc29db0700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:51.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.705+0000 7fbc29db0700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc08066f30 0x7fbc080693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000e10 tx=0x7fbc20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:51.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.709+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc14052f70 con 0x7fbc240b24d0 2023-10-06T03:01:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.893+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc2404c3c0 con 0x7fbc240b24d0 2023-10-06T03:01:51.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.894+0000 7fbc19ffb700 1 -- 172.21.15.92:0/3483957433 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fbc14056590 con 0x7fbc240b24d0 2023-10-06T03:01:51.896 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:51.896 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc08066f30 msgr2=0x7fbc080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc08066f30 0x7fbc080693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc20000e10 tx=0x7fbc20009040 comp rx=0 tx=0).stop 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 msgr2=0x7fbc24169ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 0x7fbc24169ec0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400a010 tx=0x7fbc1401a000 comp rx=0 tx=0).stop 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 shutdown_connections 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc240b2940 0x7fbc240c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc10003fe0 0x7fbc240b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 --2- 172.21.15.92:0/3483957433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc240b24d0 0x7fbc24169ec0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 >> 172.21.15.92:0/3483957433 conn(0x7fbc240ce600 msgr2=0x7fbc2415c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 shutdown_connections 2023-10-06T03:01:51.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:51.896+0000 7fbc2c014700 1 -- 172.21.15.92:0/3483957433 wait complete. 2023-10-06T03:01:51.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:01:51.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:01:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.025+0000 7fb81f735700 1 Processor -- start 2023-10-06T03:01:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.025+0000 7fb81f735700 1 -- start start 2023-10-06T03:01:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81f735700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb818159890 0x7fb81815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81f735700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 0x7fb8180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81f735700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb818069ee0 con 0x7fb818159c60 2023-10-06T03:01:52.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81f735700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81815a160 con 0x7fb818159890 2023-10-06T03:01:52.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81f735700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180d13a0 con 0x7fb818069560 2023-10-06T03:01:52.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81dcd2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 0x7fb8180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81d4d1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb818159890 0x7fb81815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81dcd2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 0x7fb8180699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42790/0 (socket says 172.21.15.92:42790) 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81ccd0700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb818159c60 0x7fb81815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:39132/0 (socket says 172.21.15.92:39132) 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.026+0000 7fb81dcd2700 1 -- 172.21.15.92:0/1127332925 learned_addr learned my addr 172.21.15.92:0/1127332925 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81dcd2700 1 -- 172.21.15.92:0/1127332925 >> v2:172.21.15.136:3300/0 conn(0x7fb818159890 msgr2=0x7fb81815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81dcd2700 1 --2- 172.21.15.92:0/1127332925 >> v2:172.21.15.136:3300/0 conn(0x7fb818159890 0x7fb81815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81dcd2700 1 -- 172.21.15.92:0/1127332925 >> v1:172.21.15.136:6789/0 conn(0x7fb818159c60 legacy=0x7fb81815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81dcd2700 1 -- 172.21.15.92:0/1127332925 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81815e140 con 0x7fb818069560 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81dcd2700 1 --2- 172.21.15.92:0/1127332925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 0x7fb8180699a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb8140099b0 tx=0x7fb814009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8c238476b07754 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb80ffff700 1 -- 172.21.15.92:0/1127332925 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814009ea0 con 0x7fb818069560 2023-10-06T03:01:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb80ffff700 1 -- 172.21.15.92:0/1127332925 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81400b860 con 0x7fb818069560 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb80ffff700 1 -- 172.21.15.92:0/1127332925 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81400bc00 con 0x7fb818069560 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81f735700 1 -- 172.21.15.92:0/1127332925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 msgr2=0x7fb8180699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81f735700 1 --2- 172.21.15.92:0/1127332925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 0x7fb8180699a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb8140099b0 tx=0x7fb814009250 comp rx=0 tx=0).stop 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81f735700 1 -- 172.21.15.92:0/1127332925 shutdown_connections 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81f735700 1 --2- 172.21.15.92:0/1127332925 >> v2:172.21.15.136:3300/0 conn(0x7fb818159890 0x7fb81815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81f735700 1 --2- 172.21.15.92:0/1127332925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818069560 0x7fb8180699a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.027+0000 7fb81f735700 1 -- 172.21.15.92:0/1127332925 >> 172.21.15.92:0/1127332925 conn(0x7fb818068e20 msgr2=0x7fb8180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.028+0000 7fb81f735700 1 -- 172.21.15.92:0/1127332925 shutdown_connections 2023-10-06T03:01:52.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.028+0000 7fb81f735700 1 -- 172.21.15.92:0/1127332925 wait complete. 2023-10-06T03:01:52.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 Processor -- start 2023-10-06T03:01:52.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 -- start start 2023-10-06T03:01:52.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 0x7fb81815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818159890 0x7fb818140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb818159c60 0x7fb8180b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180cdb20 con 0x7fb818069560 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180cde00 con 0x7fb818159c60 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.029+0000 7fb81f735700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180ce0e0 con 0x7fb818159890 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 0x7fb81815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 0x7fb81815cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34612/0 (socket says 172.21.15.92:34612) 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 -- 172.21.15.92:0/1608203835 learned_addr learned my addr 172.21.15.92:0/1608203835 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81ccd0700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818159890 0x7fb818140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81dcd2700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb818159c60 0x7fb8180b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 -- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818159890 msgr2=0x7fb818140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818159890 0x7fb818140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 -- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb818159c60 msgr2=0x7fb8180b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb818159c60 0x7fb8180b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 -- 172.21.15.92:0/1608203835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8140094f0 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81d4d1700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 0x7fb81815cf60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb810009ef0 tx=0x7fb81000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81000cb70 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000f040 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8180ce3c0 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810014480 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.030+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81815d4a0 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.031+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8180ce860 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.032+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8100147f0 con 0x7fb818069560 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.032+0000 7fb80d7fa700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb7fc066f30 0x7fb7fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.032+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb810085290 con 0x7fb818069560 2023-10-06T03:01:52.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.032+0000 7fb81ccd0700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb7fc066f30 0x7fb7fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.032+0000 7fb81ccd0700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb7fc066f30 0x7fb7fc0693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8180b2750 tx=0x7fb808007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.036+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8100523a0 con 0x7fb818069560 2023-10-06T03:01:52.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.218+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb818064ed0 con 0x7fb818069560 2023-10-06T03:01:52.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.219+0000 7fb80d7fa700 1 -- 172.21.15.92:0/1608203835 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8100559c0 con 0x7fb818069560 2023-10-06T03:01:52.221 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:01:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb7fc066f30 msgr2=0x7fb7fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb7fc066f30 0x7fb7fc0693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb8180b2750 tx=0x7fb808007eb0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 msgr2=0x7fb81815cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 0x7fb81815cf60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb810009ef0 tx=0x7fb81000b040 comp rx=0 tx=0).stop 2023-10-06T03:01:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 shutdown_connections 2023-10-06T03:01:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb7fc066f30 0x7fb7fc0693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb818069560 0x7fb81815cf60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb818159c60 0x7fb8180b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 --2- 172.21.15.92:0/1608203835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb818159890 0x7fb818140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 >> 172.21.15.92:0/1608203835 conn(0x7fb818068e20 msgr2=0x7fb818152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 shutdown_connections 2023-10-06T03:01:52.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.221+0000 7fb81f735700 1 -- 172.21.15.92:0/1608203835 wait complete. 2023-10-06T03:01:52.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:01:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.350+0000 7f7c4a13a700 1 Processor -- start 2023-10-06T03:01:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c4a13a700 1 -- start start 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c4a13a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 0x7f7c4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c4a13a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4415a160 con 0x7f7c4415db10 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c4a13a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440d13a0 con 0x7f7c44159890 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c4a13a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4415e140 con 0x7f7c44159c60 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c43fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7c4415db10 0x7f7c44069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39018/0 (socket says 172.21.15.92:39018) 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.351+0000 7f7c43fff700 1 -- 172.21.15.92:0/534272932 learned_addr learned my addr 172.21.15.92:0/534272932 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 0x7f7c4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051886296 0 0) 0x7f7c4415a160 con 0x7f7c4415db10 2023-10-06T03:01:52.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c2c003570 con 0x7f7c4415db10 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742735297 0 0) 0x7f7c2c003570 con 0x7f7c4415db10 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 msgr2=0x7f7c4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 0x7f7c4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 >> v1:172.21.15.136:6789/0 conn(0x7f7c44159890 legacy=0x7f7c4415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4415a3d0 con 0x7f7c4415db10 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7c3c0017f0 con 0x7f7c4415db10 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 >> v1:172.21.15.92:6789/0 conn(0x7f7c4415db10 legacy=0x7f7c44069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 0x7f7c2c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c2c004d50 0x7f7c2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4415e140 con 0x7f7c2c004010 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c00bbf0 con 0x7f7c2c004d50 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c00bdb0 con 0x7f7c44159c60 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c437fe700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 0x7f7c2c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c43fff700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c2c004d50 0x7f7c2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 -- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 msgr2=0x7f7c2c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c44159c60 0x7f7c2c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 -- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c2c004d50 msgr2=0x7f7c2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c2c004d50 0x7f7c2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.352+0000 7f7c42ffd700 1 -- 172.21.15.92:0/534272932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c003570 con 0x7f7c2c004010 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c2c004810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006f00 tx=0x7f7c380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2719d4c9855b4a27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800e6e0 con 0x7f7c2c004010 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3800e840 con 0x7f7c2c004010 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c427fc700 1 -- 172.21.15.92:0/534272932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800ec00 con 0x7f7c2c004010 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c4a13a700 1 -- 172.21.15.92:0/534272932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 msgr2=0x7f7c2c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/534272932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c2c004810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006f00 tx=0x7f7c380091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c43fff700 1 -- 172.21.15.92:0/534272932 reap_dead start 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c4a13a700 1 -- 172.21.15.92:0/534272932 shutdown_connections 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c4a13a700 1 -- 172.21.15.92:0/534272932 >> 172.21.15.92:0/534272932 conn(0x7f7c44068e20 msgr2=0x7f7c440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c4a13a700 1 -- 172.21.15.92:0/534272932 shutdown_connections 2023-10-06T03:01:52.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.353+0000 7f7c4a13a700 1 -- 172.21.15.92:0/534272932 wait complete. 2023-10-06T03:01:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.354+0000 7f7c4a13a700 1 Processor -- start 2023-10-06T03:01:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.354+0000 7f7c4a13a700 1 -- start start 2023-10-06T03:01:52.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c4a13a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c4416d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c4a13a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4416e040 0x7f7c440b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c4a13a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c440b61c0 0x7f7c4414c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c4a13a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4415baf0 con 0x7f7c2c004010 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c4a13a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440ceca0 con 0x7f7c4416e040 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c4a13a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440cef80 con 0x7f7c440b61c0 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c4416d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c437fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c4416d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34634/0 (socket says 172.21.15.92:34634) 2023-10-06T03:01:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c437fe700 1 -- 172.21.15.92:0/30561238 learned_addr learned my addr 172.21.15.92:0/30561238 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4416e040 0x7f7c440b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.355+0000 7f7c43fff700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c440b61c0 0x7f7c4414c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c437fe700 1 -- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c440b61c0 msgr2=0x7f7c4414c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c437fe700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c440b61c0 0x7f7c4414c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c437fe700 1 -- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4416e040 msgr2=0x7f7c440b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c437fe700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4416e040 0x7f7c440b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c437fe700 1 -- 172.21.15.92:0/30561238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440cf260 con 0x7f7c2c004010 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c437fe700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c4416d670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7c34002920 tx=0x7f7c34007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400f670 con 0x7f7c2c004010 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3400f7d0 con 0x7f7c2c004010 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c440cf540 con 0x7f7c2c004010 2023-10-06T03:01:52.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400fb90 con 0x7f7c2c004010 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.356+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4414c630 con 0x7f7c2c004010 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.357+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4404c3c0 con 0x7f7c2c004010 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.357+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c3401c400 con 0x7f7c2c004010 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.358+0000 7f7c2b7fe700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c30066fb0 0x7f7c30069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.358+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c30066fb0 0x7f7c30069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.358+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7c34084ac0 con 0x7f7c2c004010 2023-10-06T03:01:52.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.358+0000 7f7c42ffd700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c30066fb0 0x7f7c30069460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7c380091e0 tx=0x7f7c3801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.362+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c34051be0 con 0x7f7c2c004010 2023-10-06T03:01:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.545+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7c4406a5b0 con 0x7f7c2c004010 2023-10-06T03:01:52.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.545+0000 7f7c2b7fe700 1 -- 172.21.15.92:0/30561238 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7c34055200 con 0x7f7c2c004010 2023-10-06T03:01:52.547 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c30066fb0 msgr2=0x7f7c30069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c30066fb0 0x7f7c30069460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7c380091e0 tx=0x7f7c3801a000 comp rx=0 tx=0).stop 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 msgr2=0x7f7c4416d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c4416d670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7c34002920 tx=0x7f7c34007d30 comp rx=0 tx=0).stop 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 shutdown_connections 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c30066fb0 0x7f7c30069460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c2c004010 0x7f7c4416d670 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c440b61c0 0x7f7c4414c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 --2- 172.21.15.92:0/30561238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c4416e040 0x7f7c440b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 >> 172.21.15.92:0/30561238 conn(0x7f7c44068e20 msgr2=0x7f7c440c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 shutdown_connections 2023-10-06T03:01:52.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.548+0000 7f7c4a13a700 1 -- 172.21.15.92:0/30561238 wait complete. 2023-10-06T03:01:52.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:01:52.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.677+0000 7f55c87c7700 1 Processor -- start 2023-10-06T03:01:52.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.677+0000 7f55c87c7700 1 -- start start 2023-10-06T03:01:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c87c7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55c0159890 0x7f55c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c87c7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c015a160 con 0x7f55c0159c60 2023-10-06T03:01:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c87c7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00d13a0 con 0x7f55c0069560 2023-10-06T03:01:52.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c87c7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c015e140 con 0x7f55c0159890 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c6563700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55c0159890 0x7f55c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c5d62700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f55c0159c60 0x7f55c015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39030/0 (socket says 172.21.15.92:39030) 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c5d62700 1 -- 172.21.15.92:0/1069090750 learned_addr learned my addr 172.21.15.92:0/1069090750 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c6563700 1 -- 172.21.15.92:0/1069090750 >> v1:172.21.15.136:6789/0 conn(0x7f55c0069560 legacy=0x7f55c00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c6563700 1 -- 172.21.15.92:0/1069090750 >> v1:172.21.15.92:6789/0 conn(0x7f55c0159c60 legacy=0x7f55c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c5561700 1 -- 172.21.15.92:0/1069090750 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179044243 0 0) 0x7f55c015a160 con 0x7f55c0159c60 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.678+0000 7f55c6563700 1 -- 172.21.15.92:0/1069090750 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015a3d0 con 0x7f55c0159890 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c6563700 1 --2- 172.21.15.92:0/1069090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0159890 0x7f55c015abd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009e60 tx=0x7f55bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720fc75f8b973d59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c5561700 1 -- 172.21.15.92:0/1069090750 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00e070 con 0x7f55c0159890 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c5561700 1 -- 172.21.15.92:0/1069090750 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc00bbf0 con 0x7f55c0159890 2023-10-06T03:01:52.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c5561700 1 -- 172.21.15.92:0/1069090750 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc012430 con 0x7f55c0159890 2023-10-06T03:01:52.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 -- 172.21.15.92:0/1069090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0159890 msgr2=0x7f55c015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 --2- 172.21.15.92:0/1069090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0159890 0x7f55c015abd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009e60 tx=0x7f55bc009780 comp rx=0 tx=0).stop 2023-10-06T03:01:52.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 -- 172.21.15.92:0/1069090750 shutdown_connections 2023-10-06T03:01:52.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 --2- 172.21.15.92:0/1069090750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0159890 0x7f55c015abd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 -- 172.21.15.92:0/1069090750 >> 172.21.15.92:0/1069090750 conn(0x7f55c0068e20 msgr2=0x7f55c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 -- 172.21.15.92:0/1069090750 shutdown_connections 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.679+0000 7f55c87c7700 1 -- 172.21.15.92:0/1069090750 wait complete. 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.680+0000 7f55c87c7700 1 Processor -- start 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.680+0000 7f55c87c7700 1 -- start start 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c87c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 0x7f55c00ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c87c7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0159890 0x7f55c00b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c87c7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55c0159c60 0x7f55c00c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c87c7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00c81c0 con 0x7f55c0069560 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c87c7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00c7ee0 con 0x7f55c0159890 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c87c7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c00b36c0 con 0x7f55c0159c60 2023-10-06T03:01:52.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c6563700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 0x7f55c00ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c6563700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 0x7f55c00ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34646/0 (socket says 172.21.15.92:34646) 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c6563700 1 -- 172.21.15.92:0/2738900405 learned_addr learned my addr 172.21.15.92:0/2738900405 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c6d64700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55c0159c60 0x7f55c00c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c5d62700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0159890 0x7f55c00b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.681+0000 7f55c6563700 1 -- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55c0159c60 msgr2=0x7f55c00c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c6563700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55c0159c60 0x7f55c00c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c6563700 1 -- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0159890 msgr2=0x7f55c00b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c6563700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0159890 0x7f55c00b5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c6563700 1 -- 172.21.15.92:0/2738900405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc009a20 con 0x7f55c0069560 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c5d62700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0159890 0x7f55c00b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c6563700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 0x7f55c00ce730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b6c0 tx=0x7f55bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc027470 con 0x7f55c0069560 2023-10-06T03:01:52.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc0275f0 con 0x7f55c0069560 2023-10-06T03:01:52.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0279b0 con 0x7f55c0069560 2023-10-06T03:01:52.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c00b39a0 con 0x7f55c0069560 2023-10-06T03:01:52.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.682+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c00c7b20 con 0x7f55c0069560 2023-10-06T03:01:52.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.683+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c004c3c0 con 0x7f55c0069560 2023-10-06T03:01:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.683+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55bc024070 con 0x7f55c0069560 2023-10-06T03:01:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.683+0000 7f55b67fc700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55a4066f30 0x7f55a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.684+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55bc0872e0 con 0x7f55c0069560 2023-10-06T03:01:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.684+0000 7f55c5d62700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55a4066f30 0x7f55a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:52.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.684+0000 7f55c5d62700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55a4066f30 0x7f55a40693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55c0154380 tx=0x7f55b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.688+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55bc0543f0 con 0x7f55c0069560 2023-10-06T03:01:52.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.870+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55c0159030 con 0x7f55c0069560 2023-10-06T03:01:52.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.871+0000 7f55b67fc700 1 -- 172.21.15.92:0/2738900405 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f55bc057a10 con 0x7f55c0069560 2023-10-06T03:01:52.873 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:01:52.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55a4066f30 msgr2=0x7f55a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55a4066f30 0x7f55a40693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55c0154380 tx=0x7f55b0009040 comp rx=0 tx=0).stop 2023-10-06T03:01:52.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 msgr2=0x7f55c00ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:52.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 0x7f55c00ce730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00b6c0 tx=0x7f55bc018000 comp rx=0 tx=0).stop 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 shutdown_connections 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55a4066f30 0x7f55a40693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55c0069560 0x7f55c00ce730 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55c0159c60 0x7f55c00c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 --2- 172.21.15.92:0/2738900405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0159890 0x7f55c00b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 >> 172.21.15.92:0/2738900405 conn(0x7f55c0068e20 msgr2=0x7f55c0152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 shutdown_connections 2023-10-06T03:01:52.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:52.873+0000 7f55c87c7700 1 -- 172.21.15.92:0/2738900405 wait complete. 2023-10-06T03:01:52.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:01:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.002+0000 7ff962635700 1 Processor -- start 2023-10-06T03:01:53.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.002+0000 7ff962635700 1 -- start start 2023-10-06T03:01:53.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff962635700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff95c159890 0x7ff95c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff962635700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff95c159c60 0x7ff95c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff962635700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95c15a160 con 0x7ff95c15db10 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff962635700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c15df50 con 0x7ff95c159890 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff962635700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c0d13a0 con 0x7ff95c159c60 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff95bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff95c159890 0x7ff95c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff95bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff95c159890 0x7ff95c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34660/0 (socket says 172.21.15.92:34660) 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.003+0000 7ff95bfff700 1 -- 172.21.15.92:0/4228814862 learned_addr learned my addr 172.21.15.92:0/4228814862 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95b7fe700 1 --2- 172.21.15.92:0/4228814862 >> v2:172.21.15.136:3300/0 conn(0x7ff95c159c60 0x7ff95c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95bfff700 1 -- 172.21.15.92:0/4228814862 >> v2:172.21.15.136:3300/0 conn(0x7ff95c159c60 msgr2=0x7ff95c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:53.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95bfff700 1 --2- 172.21.15.92:0/4228814862 >> v2:172.21.15.136:3300/0 conn(0x7ff95c159c60 0x7ff95c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95bfff700 1 -- 172.21.15.92:0/4228814862 >> v1:172.21.15.136:6789/0 conn(0x7ff95c15db10 legacy=0x7ff95c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95bfff700 1 -- 172.21.15.92:0/4228814862 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c15e140 con 0x7ff95c159890 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95b7fe700 1 --2- 172.21.15.92:0/4228814862 >> v2:172.21.15.136:3300/0 conn(0x7ff95c159c60 0x7ff95c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95bfff700 1 --2- 172.21.15.92:0/4228814862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c159890 0x7ff95c15abd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0b67f0 tx=0x7ff94c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f8662d16173a41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95affd700 1 -- 172.21.15.92:0/4228814862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00b820 con 0x7ff95c159890 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95affd700 1 -- 172.21.15.92:0/4228814862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c00b980 con 0x7ff95c159890 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff95affd700 1 -- 172.21.15.92:0/4228814862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00bd20 con 0x7ff95c159890 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff962635700 1 -- 172.21.15.92:0/4228814862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c159890 msgr2=0x7ff95c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff962635700 1 --2- 172.21.15.92:0/4228814862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c159890 0x7ff95c15abd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0b67f0 tx=0x7ff94c009250 comp rx=0 tx=0).stop 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff962635700 1 -- 172.21.15.92:0/4228814862 shutdown_connections 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff962635700 1 --2- 172.21.15.92:0/4228814862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c159890 0x7ff95c15abd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff962635700 1 --2- 172.21.15.92:0/4228814862 >> v2:172.21.15.136:3300/0 conn(0x7ff95c159c60 0x7ff95c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.004+0000 7ff962635700 1 -- 172.21.15.92:0/4228814862 >> 172.21.15.92:0/4228814862 conn(0x7ff95c068e20 msgr2=0x7ff95c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.005+0000 7ff962635700 1 -- 172.21.15.92:0/4228814862 shutdown_connections 2023-10-06T03:01:53.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.005+0000 7ff962635700 1 -- 172.21.15.92:0/4228814862 wait complete. 2023-10-06T03:01:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 Processor -- start 2023-10-06T03:01:53.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 -- start start 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff95c159890 0x7ff95c0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95c159c60 0x7ff95c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 0x7ff95c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c1584d0 con 0x7ff95c15db10 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c16d650 con 0x7ff95c159c60 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.006+0000 7ff962635700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c16d930 con 0x7ff95c159890 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 0x7ff95c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 0x7ff95c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34664/0 (socket says 172.21.15.92:34664) 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 -- 172.21.15.92:0/606092224 learned_addr learned my addr 172.21.15.92:0/606092224 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff95b7fe700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95c159c60 0x7ff95c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff95bfff700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff95c159890 0x7ff95c0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 -- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff95c159890 msgr2=0x7ff95c0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff95c159890 0x7ff95c0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 -- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95c159c60 msgr2=0x7ff95c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95c159c60 0x7ff95c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 -- 172.21.15.92:0/606092224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c0094f0 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff960bd2700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 0x7ff95c0c8810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff95400bc30 tx=0x7ff954009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400c840 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95400c9e0 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c158700 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400cda0 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.007+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c0c8d50 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.008+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c16dae0 con 0x7ff95c15db10 2023-10-06T03:01:53.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.009+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff954014620 con 0x7ff95c15db10 2023-10-06T03:01:53.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.009+0000 7ff943fff700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff948066f30 0x7ff9480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.009+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff954084630 con 0x7ff95c15db10 2023-10-06T03:01:53.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.009+0000 7ff95bfff700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff948066f30 0x7ff9480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.009+0000 7ff95bfff700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff948066f30 0x7ff9480693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0b67f0 tx=0x7ff94c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.013+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff95404c5a0 con 0x7ff95c15db10 2023-10-06T03:01:53.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.194+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff95c04c3c0 con 0x7ff95c15db10 2023-10-06T03:01:53.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.195+0000 7ff943fff700 1 -- 172.21.15.92:0/606092224 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff95c16dae0 con 0x7ff95c15db10 2023-10-06T03:01:53.197 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:01:53.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.196+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff948066f30 msgr2=0x7ff9480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.196+0000 7ff962635700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff948066f30 0x7ff9480693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0b67f0 tx=0x7ff94c016000 comp rx=0 tx=0).stop 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.196+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 msgr2=0x7ff95c0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.196+0000 7ff962635700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 0x7ff95c0c8810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff95400bc30 tx=0x7ff954009820 comp rx=0 tx=0).stop 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.196+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 shutdown_connections 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff948066f30 0x7ff9480693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff95c15db10 0x7ff95c0c8810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff95c159890 0x7ff95c0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 --2- 172.21.15.92:0/606092224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95c159c60 0x7ff95c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 >> 172.21.15.92:0/606092224 conn(0x7ff95c068e20 msgr2=0x7ff95c153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:53.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 shutdown_connections 2023-10-06T03:01:53.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.197+0000 7ff962635700 1 -- 172.21.15.92:0/606092224 wait complete. 2023-10-06T03:01:53.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:01:53.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.323+0000 7f7f58c2d700 1 Processor -- start 2023-10-06T03:01:53.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.323+0000 7f7f58c2d700 1 -- start start 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f58c2d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7f54159c60 0x7f7f5415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f58c2d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5415a160 con 0x7f7f54159890 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f58c2d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540d13a0 con 0x7f7f5415db10 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f58c2d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5415df50 con 0x7f7f54159c60 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f5259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f54159890 0x7f7f5415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39032/0 (socket says 172.21.15.92:39032) 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f5259c700 1 -- 172.21.15.92:0/311163522 learned_addr learned my addr 172.21.15.92:0/311163522 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f51d9b700 1 --2- 172.21.15.92:0/311163522 >> v2:172.21.15.136:3300/0 conn(0x7f7f54159c60 0x7f7f5415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f5159a700 1 -- 172.21.15.92:0/311163522 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639525950 0 0) 0x7f7f5415a160 con 0x7f7f54159890 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f5159a700 1 -- 172.21.15.92:0/311163522 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3c003570 con 0x7f7f54159890 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f51d9b700 1 -- 172.21.15.92:0/311163522 >> v1:172.21.15.136:6789/0 conn(0x7f7f5415db10 legacy=0x7f7f54069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f51d9b700 1 -- 172.21.15.92:0/311163522 >> v1:172.21.15.92:6789/0 conn(0x7f7f54159890 legacy=0x7f7f5415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f51d9b700 1 -- 172.21.15.92:0/311163522 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5415e140 con 0x7f7f54159c60 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f51d9b700 1 --2- 172.21.15.92:0/311163522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159c60 0x7f7f5415d5d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009e60 tx=0x7f7f48009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=500bf510c1dde29f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.324+0000 7f7f5159a700 1 -- 172.21.15.92:0/311163522 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800e070 con 0x7f7f54159c60 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f5159a700 1 -- 172.21.15.92:0/311163522 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4800bb60 con 0x7f7f54159c60 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f5159a700 1 -- 172.21.15.92:0/311163522 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48012430 con 0x7f7f54159c60 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 -- 172.21.15.92:0/311163522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159c60 msgr2=0x7f7f5415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/311163522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159c60 0x7f7f5415d5d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009e60 tx=0x7f7f48009780 comp rx=0 tx=0).stop 2023-10-06T03:01:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 -- 172.21.15.92:0/311163522 shutdown_connections 2023-10-06T03:01:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/311163522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159c60 0x7f7f5415d5d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 -- 172.21.15.92:0/311163522 >> 172.21.15.92:0/311163522 conn(0x7f7f54068e20 msgr2=0x7f7f540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 -- 172.21.15.92:0/311163522 shutdown_connections 2023-10-06T03:01:53.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.325+0000 7f7f58c2d700 1 -- 172.21.15.92:0/311163522 wait complete. 2023-10-06T03:01:53.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.326+0000 7f7f58c2d700 1 Processor -- start 2023-10-06T03:01:53.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.326+0000 7f7f58c2d700 1 -- start start 2023-10-06T03:01:53.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f58c2d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159890 0x7f7f541450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f58c2d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 0x7f7f540b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f58c2d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f5415db10 0x7f7f540b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f58c2d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540cf700 con 0x7f7f54159c60 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f58c2d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540ceaf0 con 0x7f7f54159890 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f58c2d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540cedd0 con 0x7f7f5415db10 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 0x7f7f540b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 0x7f7f540b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34678/0 (socket says 172.21.15.92:34678) 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 -- 172.21.15.92:0/3308297331 learned_addr learned my addr 172.21.15.92:0/3308297331 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f5259c700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159890 0x7f7f541450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f52d9d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f5415db10 0x7f7f540b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 -- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f5415db10 msgr2=0x7f7f540b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f5415db10 0x7f7f540b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 -- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159890 msgr2=0x7f7f541450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159890 0x7f7f541450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 -- 172.21.15.92:0/3308297331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f48009a20 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f52d9d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f5415db10 0x7f7f540b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.327+0000 7f7f51d9b700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 0x7f7f540b1590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f54070e20 tx=0x7f7f48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.328+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48012d60 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.328+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f48027470 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.328+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f540cf0b0 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.328+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48027830 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.328+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f540cfb90 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.329+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f540cf8b0 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.329+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f48024070 con 0x7f7f54159c60 2023-10-06T03:01:53.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.329+0000 7f7f427fc700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.329+0000 7f7f5259c700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.329+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f48086d60 con 0x7f7f54159c60 2023-10-06T03:01:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.330+0000 7f7f5259c700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f34066f30 0x7f7f340693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0058d0 tx=0x7f7f4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.334+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f540cf8b0 con 0x7f7f54159c60 2023-10-06T03:01:53.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.563+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f5404c3c0 con 0x7f7f54159c60 2023-10-06T03:01:53.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.563+0000 7f7f427fc700 1 -- 172.21.15.92:0/3308297331 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f7f48053e70 con 0x7f7f54159c60 2023-10-06T03:01:53.565 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:53.565 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:01:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f34066f30 msgr2=0x7f7f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f34066f30 0x7f7f340693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0058d0 tx=0x7f7f4c009040 comp rx=0 tx=0).stop 2023-10-06T03:01:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 msgr2=0x7f7f540b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 0x7f7f540b1590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f54070e20 tx=0x7f7f48019000 comp rx=0 tx=0).stop 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 shutdown_connections 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f34066f30 0x7f7f340693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f5415db10 0x7f7f540b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54159890 0x7f7f541450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 --2- 172.21.15.92:0/3308297331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54159c60 0x7f7f540b1590 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 >> 172.21.15.92:0/3308297331 conn(0x7f7f54068e20 msgr2=0x7f7f5415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 shutdown_connections 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.565+0000 7f7f58c2d700 1 -- 172.21.15.92:0/3308297331 wait complete. 2023-10-06T03:01:53.568 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:01:53.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.692+0000 7fcefcd9c700 1 Processor -- start 2023-10-06T03:01:53.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.692+0000 7fcefcd9c700 1 -- start start 2023-10-06T03:01:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcefcd9c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 0x7fcef80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcefcd9c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef815d7e0 con 0x7fcef806bcd0 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcefcd9c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef815dac0 con 0x7fcef815df50 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcefcd9c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef815aa30 con 0x7fcef806c0a0 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcef659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcef806bcd0 0x7fcef815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39036/0 (socket says 172.21.15.92:39036) 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcef659c700 1 -- 172.21.15.92:0/3471667793 learned_addr learned my addr 172.21.15.92:0/3471667793 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661495960 0 0) 0x7fcef815d7e0 con 0x7fcef806bcd0 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee4003570 con 0x7fcef806bcd0 2023-10-06T03:01:53.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.693+0000 7fceedd9b700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 0x7fcef80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185722419 0 0) 0x7fcee4003570 con 0x7fcef806bcd0 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 msgr2=0x7fcef80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 0x7fcef80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 >> v1:172.21.15.136:6789/0 conn(0x7fcef815df50 legacy=0x7fcef80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef815ad10 con 0x7fcef806bcd0 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fcee8001470 con 0x7fcef806bcd0 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 >> v1:172.21.15.92:6789/0 conn(0x7fcef806bcd0 legacy=0x7fcef815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcee4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 0x7fcee4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4004da0 0x7fcee40051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef815aa30 con 0x7fcee4003a90 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee400bcc0 con 0x7fcee4004da0 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee400be80 con 0x7fcef806c0a0 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcee4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fceedd9b700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 0x7fcee4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef6d9d700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4004da0 0x7fcee40051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 -- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 msgr2=0x7fcee4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef806c0a0 0x7fcee4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 -- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4004da0 msgr2=0x7fcee40051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:53.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee4004da0 0x7fcee40051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 -- 172.21.15.92:0/3471667793 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee4003570 con 0x7fcee4003a90 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef659c700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcee4003ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcee8001ed0 tx=0x7fcef80ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5691151243b9ee3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800cc30 con 0x7fcee4003a90 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee800cd90 con 0x7fcee4003a90 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.694+0000 7fcef5d9b700 1 -- 172.21.15.92:0/3471667793 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8005b70 con 0x7fcee4003a90 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3471667793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 msgr2=0x7fcee4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3471667793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcee4003ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcee8001ed0 tx=0x7fcef80ce4d0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcef6d9d700 1 -- 172.21.15.92:0/3471667793 reap_dead start 2023-10-06T03:01:53.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3471667793 shutdown_connections 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3471667793 >> 172.21.15.92:0/3471667793 conn(0x7fcef80ce600 msgr2=0x7fcef80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3471667793 shutdown_connections 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.695+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3471667793 wait complete. 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.696+0000 7fcefcd9c700 1 Processor -- start 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.696+0000 7fcefcd9c700 1 -- start start 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcefcd9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcef8169b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcefcd9c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef80c7da0 0x7fcef80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcefcd9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcef80c45b0 0x7fcef80c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcefcd9c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80c4f90 con 0x7fcee4003a90 2023-10-06T03:01:53.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcefcd9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80b1e00 con 0x7fcef80c45b0 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcefcd9c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80c88d0 con 0x7fcef80c7da0 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcef8169b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcef8169b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34690/0 (socket says 172.21.15.92:34690) 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 -- 172.21.15.92:0/3710490319 learned_addr learned my addr 172.21.15.92:0/3710490319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fceedd9b700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef80c7da0 0x7fcef80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef6d9d700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcef80c45b0 0x7fcef80c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 -- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef80c7da0 msgr2=0x7fcef80c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef80c7da0 0x7fcef80c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 -- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcef80c45b0 msgr2=0x7fcef80c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcef80c45b0 0x7fcef80c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 -- 172.21.15.92:0/3710490319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80b14d0 con 0x7fcee4003a90 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fceedd9b700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef80c7da0 0x7fcef80c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.697+0000 7fcef659c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcef8169b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcee400bc90 tx=0x7fcee8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.698+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80070d0 con 0x7fcee4003a90 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.698+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee8007250 con 0x7fcee4003a90 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.698+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8007610 con 0x7fcee4003a90 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.698+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef80b17b0 con 0x7fcee4003a90 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.698+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef816a050 con 0x7fcee4003a90 2023-10-06T03:01:53.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.699+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef80c4f90 con 0x7fcee4003a90 2023-10-06T03:01:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.699+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcee8019030 con 0x7fcee4003a90 2023-10-06T03:01:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.699+0000 7fceeeffd700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fced8066f30 0x7fced80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.699+0000 7fceedd9b700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fced8066f30 0x7fced80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.699+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcee80859a0 con 0x7fcee4003a90 2023-10-06T03:01:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.700+0000 7fceedd9b700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fced8066f30 0x7fced80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcee0006dc0 tx=0x7fcee0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:53.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.703+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcee8052ab0 con 0x7fcee4003a90 2023-10-06T03:01:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.889+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcef804c3c0 con 0x7fcee4003a90 2023-10-06T03:01:53.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.890+0000 7fceeeffd700 1 -- 172.21.15.92:0/3710490319 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fcee80560d0 con 0x7fcee4003a90 2023-10-06T03:01:53.892 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:53.892 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fced8066f30 msgr2=0x7fced80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fced8066f30 0x7fced80693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcee0006dc0 tx=0x7fcee0008040 comp rx=0 tx=0).stop 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 msgr2=0x7fcef8169b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcef8169b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcee400bc90 tx=0x7fcee8015000 comp rx=0 tx=0).stop 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 shutdown_connections 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fced8066f30 0x7fced80693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee4003a90 0x7fcef8169b10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcef80c45b0 0x7fcef80c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 --2- 172.21.15.92:0/3710490319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcef80c7da0 0x7fcef80c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 >> 172.21.15.92:0/3710490319 conn(0x7fcef80ce600 msgr2=0x7fcef8155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 shutdown_connections 2023-10-06T03:01:53.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:53.892+0000 7fcefcd9c700 1 -- 172.21.15.92:0/3710490319 wait complete. 2023-10-06T03:01:53.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:01:53.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:01:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.021+0000 7f371f144700 1 Processor -- start 2023-10-06T03:01:54.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.021+0000 7f371f144700 1 -- start start 2023-10-06T03:01:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371f144700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157680 0x7f37180cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371f144700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37180cd240 con 0x7f3718157a50 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371f144700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37180c9a20 con 0x7f37180b1450 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371f144700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37180cc700 con 0x7f3718157680 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371d941700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3718157a50 0x7f37180c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39044/0 (socket says 172.21.15.92:39044) 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371d941700 1 -- 172.21.15.92:0/558857376 learned_addr learned my addr 172.21.15.92:0/558857376 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371e142700 1 --2- 172.21.15.92:0/558857376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157680 0x7f37180cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269162897 0 0) 0x7f37180cd240 con 0x7f3718157a50 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.022+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3704003570 con 0x7f3718157a50 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382955258 0 0) 0x7f37180c9a20 con 0x7f37180b1450 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37180cd240 con 0x7f37180b1450 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 273169129 0 0) 0x7f3704003570 con 0x7f3718157a50 2023-10-06T03:01:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371e142700 1 -- 172.21.15.92:0/558857376 >> v1:172.21.15.136:6789/0 conn(0x7f37180b1450 legacy=0x7f37180cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371e142700 1 -- 172.21.15.92:0/558857376 >> v1:172.21.15.92:6789/0 conn(0x7f3718157a50 legacy=0x7f37180c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371e142700 1 -- 172.21.15.92:0/558857376 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37180cc910 con 0x7f3718157680 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371e142700 1 --2- 172.21.15.92:0/558857376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157680 0x7f37180cae80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3714009910 tx=0x7f37140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8de1bc6d86e4b77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708001470 con 0x7f3718157680 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3714011420 con 0x7f3718157680 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.023+0000 7f371d140700 1 -- 172.21.15.92:0/558857376 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37140117a0 con 0x7f3718157680 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 -- 172.21.15.92:0/558857376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157680 msgr2=0x7f37180cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 --2- 172.21.15.92:0/558857376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157680 0x7f37180cae80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3714009910 tx=0x7f37140091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 -- 172.21.15.92:0/558857376 shutdown_connections 2023-10-06T03:01:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 --2- 172.21.15.92:0/558857376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157680 0x7f37180cae80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 -- 172.21.15.92:0/558857376 >> 172.21.15.92:0/558857376 conn(0x7f371806baa0 msgr2=0x7f371806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 -- 172.21.15.92:0/558857376 shutdown_connections 2023-10-06T03:01:54.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.024+0000 7f371f144700 1 -- 172.21.15.92:0/558857376 wait complete. 2023-10-06T03:01:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 Processor -- start 2023-10-06T03:01:54.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 -- start start 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 0x7f37181511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3718157680 0x7f37180c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157a50 0x7f37180c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3718151dc0 con 0x7f37180b1450 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37180c7130 con 0x7f3718157680 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.025+0000 7f371f144700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37181685d0 con 0x7f3718157a50 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 0x7f37181511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 0x7f37181511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34698/0 (socket says 172.21.15.92:34698) 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 -- 172.21.15.92:0/1832024605 learned_addr learned my addr 172.21.15.92:0/1832024605 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:54.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e943700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157a50 0x7f37180c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371d941700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3718157680 0x7f37180c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 -- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157a50 msgr2=0x7f37180c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157a50 0x7f37180c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 -- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3718157680 msgr2=0x7f37180c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3718157680 0x7f37180c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 -- 172.21.15.92:0/1832024605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3714009420 con 0x7f37180b1450 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371e142700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 0x7f37181511e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37140091e0 tx=0x7f3714016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371401caf0 con 0x7f37180b1450 2023-10-06T03:01:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371401cc50 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37181679e0 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371401a450 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.026+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3718167f80 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.027+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371806eba0 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.027+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f371401a800 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.028+0000 7f370e7fc700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36fc066f30 0x7f36fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.028+0000 7f371d941700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36fc066f30 0x7f36fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.028+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3714085ef0 con 0x7f37180b1450 2023-10-06T03:01:54.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.028+0000 7f371d941700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36fc066f30 0x7f36fc0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f37180c63d0 tx=0x7f3708000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.032+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3714053000 con 0x7f37180b1450 2023-10-06T03:01:54.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.217+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37180ccbf0 con 0x7f37180b1450 2023-10-06T03:01:54.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.218+0000 7f370e7fc700 1 -- 172.21.15.92:0/1832024605 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3714056620 con 0x7f37180b1450 2023-10-06T03:01:54.220 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36fc066f30 msgr2=0x7f36fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36fc066f30 0x7f36fc0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f37180c63d0 tx=0x7f3708000bc0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 msgr2=0x7f37181511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 0x7f37181511e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37140091e0 tx=0x7f3714016000 comp rx=0 tx=0).stop 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 shutdown_connections 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36fc066f30 0x7f36fc0693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37180b1450 0x7f37181511e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3718157a50 0x7f37180c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 --2- 172.21.15.92:0/1832024605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3718157680 0x7f37180c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 >> 172.21.15.92:0/1832024605 conn(0x7f371806baa0 msgr2=0x7f37180ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 shutdown_connections 2023-10-06T03:01:54.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.220+0000 7f371f144700 1 -- 172.21.15.92:0/1832024605 wait complete. 2023-10-06T03:01:54.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:01:54.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.349+0000 7f48c359e700 1 Processor -- start 2023-10-06T03:01:54.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.349+0000 7f48c359e700 1 -- start start 2023-10-06T03:01:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c40b2080 con 0x7f48c40b33b0 2023-10-06T03:01:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c359e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c40cfb00 con 0x7f48c40b4e70 2023-10-06T03:01:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c40b5c40 con 0x7f48c40b16d0 2023-10-06T03:01:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c2d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f48c40b33b0 0x7f48c40b3830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39060/0 (socket says 172.21.15.92:39060) 2023-10-06T03:01:54.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c2d9d700 1 -- 172.21.15.92:0/3352385083 learned_addr learned my addr 172.21.15.92:0/3352385083 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48b9d9b700 1 --2- 172.21.15.92:0/3352385083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291075846 0 0) 0x7f48c40b2080 con 0x7f48c40b33b0 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b0003570 con 0x7f48c40b33b0 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279709778 0 0) 0x7f48c40cfb00 con 0x7f48c40b4e70 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c40b2080 con 0x7f48c40b4e70 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48b9d9b700 1 -- 172.21.15.92:0/3352385083 >> v1:172.21.15.136:6789/0 conn(0x7f48c40b4e70 legacy=0x7f48c40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48b9d9b700 1 -- 172.21.15.92:0/3352385083 >> v1:172.21.15.92:6789/0 conn(0x7f48c40b33b0 legacy=0x7f48c40b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.350+0000 7f48b9d9b700 1 -- 172.21.15.92:0/3352385083 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c40b5f20 con 0x7f48c40b16d0 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48b9d9b700 1 --2- 172.21.15.92:0/3352385083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c40b1b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009910 tx=0x7f48ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e54c674fe0f748d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011630 con 0x7f48c40b16d0 2023-10-06T03:01:54.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac011790 con 0x7f48c40b16d0 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c1d9b700 1 -- 172.21.15.92:0/3352385083 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011b10 con 0x7f48c40b16d0 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 -- 172.21.15.92:0/3352385083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 msgr2=0x7f48c40b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 --2- 172.21.15.92:0/3352385083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c40b1b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009910 tx=0x7f48ac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 -- 172.21.15.92:0/3352385083 shutdown_connections 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 --2- 172.21.15.92:0/3352385083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c40b1b40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 -- 172.21.15.92:0/3352385083 >> 172.21.15.92:0/3352385083 conn(0x7f48c40d10c0 msgr2=0x7f48c40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 -- 172.21.15.92:0/3352385083 shutdown_connections 2023-10-06T03:01:54.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.351+0000 7f48c359e700 1 -- 172.21.15.92:0/3352385083 wait complete. 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.352+0000 7f48c359e700 1 Processor -- start 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 -- start start 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c416a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 0x7f48c40afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c40b4e70 0x7f48c40b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c4167b50 con 0x7f48c40b33b0 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c416d9a0 con 0x7f48c40b4e70 2023-10-06T03:01:54.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48c359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c416dc80 con 0x7f48c40b16d0 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48b9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 0x7f48c40afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48b9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 0x7f48c40afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34708/0 (socket says 172.21.15.92:34708) 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.353+0000 7f48b9d9b700 1 -- 172.21.15.92:0/967182969 learned_addr learned my addr 172.21.15.92:0/967182969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48c259c700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c416a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48c2d9d700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c40b4e70 0x7f48c40b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48b9d9b700 1 -- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 msgr2=0x7f48c416a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48b9d9b700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c416a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48b9d9b700 1 -- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c40b4e70 msgr2=0x7f48c40b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48b9d9b700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c40b4e70 0x7f48c40b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48b9d9b700 1 -- 172.21.15.92:0/967182969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac009420 con 0x7f48c40b33b0 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48c2d9d700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c40b4e70 0x7f48c40b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:54.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48b9d9b700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 0x7f48c40afef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f48c4070dd0 tx=0x7f48ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011900 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac011a60 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c416df60 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac025640 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.354+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c40a7110 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.355+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c4167cc0 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.355+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48ac025a70 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.356+0000 7f48baffd700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f48a4066f30 0x7f48a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.356+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f48ac085d20 con 0x7f48c40b33b0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.356+0000 7f48c259c700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f48a4066f30 0x7f48a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.356+0000 7f48c259c700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f48a4066f30 0x7f48a40693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f48b4007fd0 tx=0x7f48b4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.360+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48c4167cc0 con 0x7f48c40b33b0 2023-10-06T03:01:54.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.544+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48c4167cc0 con 0x7f48c40b33b0 2023-10-06T03:01:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.544+0000 7f48baffd700 1 -- 172.21.15.92:0/967182969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48c4167cc0 con 0x7f48c40b33b0 2023-10-06T03:01:54.546 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:01:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f48a4066f30 msgr2=0x7f48a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f48a4066f30 0x7f48a40693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f48b4007fd0 tx=0x7f48b4007d30 comp rx=0 tx=0).stop 2023-10-06T03:01:54.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 msgr2=0x7f48c40afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 0x7f48c40afef0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f48c4070dd0 tx=0x7f48ac017000 comp rx=0 tx=0).stop 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 shutdown_connections 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f48a4066f30 0x7f48a40693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f48c40b4e70 0x7f48c40b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48c40b16d0 0x7f48c416a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 --2- 172.21.15.92:0/967182969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48c40b33b0 0x7f48c40afef0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 >> 172.21.15.92:0/967182969 conn(0x7f48c40d10c0 msgr2=0x7f48c40cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 shutdown_connections 2023-10-06T03:01:54.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.546+0000 7f48c359e700 1 -- 172.21.15.92:0/967182969 wait complete. 2023-10-06T03:01:54.560 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:01:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.675+0000 7fa3bafce700 1 Processor -- start 2023-10-06T03:01:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.675+0000 7fa3bafce700 1 -- start start 2023-10-06T03:01:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.676+0000 7fa3bafce700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa3b4159890 0x7fa3b415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.676+0000 7fa3bafce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.676+0000 7fa3bafce700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b4069ee0 con 0x7fa3b4159c60 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.676+0000 7fa3bafce700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b415a160 con 0x7fa3b4159890 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.676+0000 7fa3bafce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40d13a0 con 0x7fa3b4069560 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b956b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa3b4159890 0x7fa3b415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3abfff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fa3b4159c60 0x7fa3b415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:39202/0 (socket says 172.21.15.92:39202) 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa3b4159890 0x7fa3b415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:43556/0 (socket says 172.21.15.92:43556) 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b956b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42964/0 (socket says 172.21.15.92:42964) 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3abfff700 1 -- 172.21.15.92:0/3154297052 learned_addr learned my addr 172.21.15.92:0/3154297052 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 -- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 msgr2=0x7fa3b40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 --2- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 -- 172.21.15.92:0/3154297052 >> v1:172.21.15.136:6789/0 conn(0x7fa3b4159c60 legacy=0x7fa3b415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 -- 172.21.15.92:0/3154297052 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b415e140 con 0x7fa3b4159890 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b956b700 1 --2- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3b8d6a700 1 --2- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159890 0x7fa3b415abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009e60 tx=0x7fa3b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94bb305647daa09c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3ab7fe700 1 -- 172.21.15.92:0/3154297052 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000e070 con 0x7fa3b4159890 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3ab7fe700 1 -- 172.21.15.92:0/3154297052 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000bb60 con 0x7fa3b4159890 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.677+0000 7fa3ab7fe700 1 -- 172.21.15.92:0/3154297052 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0012430 con 0x7fa3b4159890 2023-10-06T03:01:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 -- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159890 msgr2=0x7fa3b415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 --2- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159890 0x7fa3b415abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009e60 tx=0x7fa3b0009780 comp rx=0 tx=0).stop 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 -- 172.21.15.92:0/3154297052 shutdown_connections 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 --2- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159890 0x7fa3b415abd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 --2- 172.21.15.92:0/3154297052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 -- 172.21.15.92:0/3154297052 >> 172.21.15.92:0/3154297052 conn(0x7fa3b4068e20 msgr2=0x7fa3b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 -- 172.21.15.92:0/3154297052 shutdown_connections 2023-10-06T03:01:54.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.678+0000 7fa3bafce700 1 -- 172.21.15.92:0/3154297052 wait complete. 2023-10-06T03:01:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.679+0000 7fa3bafce700 1 Processor -- start 2023-10-06T03:01:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.679+0000 7fa3bafce700 1 -- start start 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 0x7fa3b4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159c60 0x7fa3b4158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b4158c50 con 0x7fa3b4159890 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b41589b0 con 0x7fa3b4159c60 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b40b40d0 con 0x7fa3b4069560 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 0x7fa3b4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 0x7fa3b4153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34720/0 (socket says 172.21.15.92:34720) 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 -- 172.21.15.92:0/1097256492 learned_addr learned my addr 172.21.15.92:0/1097256492 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3b956b700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159c60 0x7fa3b4158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3b8d6a700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 -- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 msgr2=0x7fa3b40c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 -- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159c60 msgr2=0x7fa3b4158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159c60 0x7fa3b4158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 -- 172.21.15.92:0/1097256492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac009420 con 0x7fa3b4159890 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3abfff700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 0x7fa3b4153c40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400c7c0 tx=0x7fa3a4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4005b30 con 0x7fa3b4159890 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a4005cb0 con 0x7fa3b4159890 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4004a80 con 0x7fa3b4159890 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b0009a20 con 0x7fa3b4159890 2023-10-06T03:01:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.680+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b40b4690 con 0x7fa3b4159890 2023-10-06T03:01:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.681+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b404c3c0 con 0x7fa3b4159890 2023-10-06T03:01:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.682+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3a40070f0 con 0x7fa3b4159890 2023-10-06T03:01:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.682+0000 7fa3a8ff9700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3a0066fb0 0x7fa3a0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.682+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa3a4083bb0 con 0x7fa3b4159890 2023-10-06T03:01:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.682+0000 7fa3b8d6a700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3a0066fb0 0x7fa3a0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.683+0000 7fa3b8d6a700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3a0066fb0 0x7fa3a0069460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00097c0 tx=0x7fa3b0019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:54.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.686+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3a4050ca0 con 0x7fa3b4159890 2023-10-06T03:01:54.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.866+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3b406a5b0 con 0x7fa3b4159890 2023-10-06T03:01:54.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.867+0000 7fa3a8ff9700 1 -- 172.21.15.92:0/1097256492 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3a4018020 con 0x7fa3b4159890 2023-10-06T03:01:54.868 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3a0066fb0 msgr2=0x7fa3a0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3a0066fb0 0x7fa3a0069460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00097c0 tx=0x7fa3b0019000 comp rx=0 tx=0).stop 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 msgr2=0x7fa3b4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 0x7fa3b4153c40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400c7c0 tx=0x7fa3a4007f70 comp rx=0 tx=0).stop 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 shutdown_connections 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3a0066fb0 0x7fa3a0069460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3b4159c60 0x7fa3b4158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3b4069560 0x7fa3b40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 --2- 172.21.15.92:0/1097256492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3b4159890 0x7fa3b4153c40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:54.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 >> 172.21.15.92:0/1097256492 conn(0x7fa3b4068e20 msgr2=0x7fa3b40d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:54.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 shutdown_connections 2023-10-06T03:01:54.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.869+0000 7fa3bafce700 1 -- 172.21.15.92:0/1097256492 wait complete. 2023-10-06T03:01:54.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:01:54.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.998+0000 7fefeab66700 1 Processor -- start 2023-10-06T03:01:54.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.998+0000 7fefeab66700 1 -- start start 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefeab66700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fefe40b16d0 0x7fefe40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefeab66700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefe40b33b0 0x7fefe40b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefeab66700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe40b2080 con 0x7fefe40b4e70 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefeab66700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe40cfb00 con 0x7fefe40b33b0 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefeab66700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe40b5c40 con 0x7fefe40b16d0 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefe40b33b0 0x7fefe40b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fefe40b33b0 0x7fefe40b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34736/0 (socket says 172.21.15.92:34736) 2023-10-06T03:01:55.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 -- 172.21.15.92:0/3455974079 learned_addr learned my addr 172.21.15.92:0/3455974079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefe9363700 1 --2- 172.21.15.92:0/3455974079 >> v2:172.21.15.136:3300/0 conn(0x7fefe40b16d0 0x7fefe40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 -- 172.21.15.92:0/3455974079 >> v2:172.21.15.136:3300/0 conn(0x7fefe40b16d0 msgr2=0x7fefe40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 --2- 172.21.15.92:0/3455974079 >> v2:172.21.15.136:3300/0 conn(0x7fefe40b16d0 0x7fefe40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 -- 172.21.15.92:0/3455974079 >> v1:172.21.15.136:6789/0 conn(0x7fefe40b4e70 legacy=0x7fefe40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 -- 172.21.15.92:0/3455974079 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe40b5f20 con 0x7fefe40b33b0 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefea365700 1 --2- 172.21.15.92:0/3455974079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b33b0 0x7fefe40b37d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefe4089830 tx=0x7fefdc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d661086a1f5e1907 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:54.999+0000 7fefe8b62700 1 -- 172.21.15.92:0/3455974079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc009e70 con 0x7fefe40b33b0 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefe8b62700 1 -- 172.21.15.92:0/3455974079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc00b860 con 0x7fefe40b33b0 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefe8b62700 1 -- 172.21.15.92:0/3455974079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00bc20 con 0x7fefe40b33b0 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 -- 172.21.15.92:0/3455974079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b33b0 msgr2=0x7fefe40b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 --2- 172.21.15.92:0/3455974079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b33b0 0x7fefe40b37d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefe4089830 tx=0x7fefdc009250 comp rx=0 tx=0).stop 2023-10-06T03:01:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 -- 172.21.15.92:0/3455974079 shutdown_connections 2023-10-06T03:01:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 --2- 172.21.15.92:0/3455974079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b33b0 0x7fefe40b37d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 --2- 172.21.15.92:0/3455974079 >> v2:172.21.15.136:3300/0 conn(0x7fefe40b16d0 0x7fefe40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 -- 172.21.15.92:0/3455974079 >> 172.21.15.92:0/3455974079 conn(0x7fefe40d10c0 msgr2=0x7fefe40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 -- 172.21.15.92:0/3455974079 shutdown_connections 2023-10-06T03:01:55.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.000+0000 7fefeab66700 1 -- 172.21.15.92:0/3455974079 wait complete. 2023-10-06T03:01:55.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.001+0000 7fefeab66700 1 Processor -- start 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.001+0000 7fefeab66700 1 -- start start 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefeab66700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 0x7fefe40c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefeab66700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefe40b33b0 0x7fefe416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefeab66700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe40b4e70 0x7fefe40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefeab66700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40cbae0 con 0x7fefe40b16d0 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefeab66700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe416a9c0 con 0x7fefe40b4e70 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefeab66700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe40cb270 con 0x7fefe40b33b0 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 0x7fefe40c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 0x7fefe40c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34744/0 (socket says 172.21.15.92:34744) 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 -- 172.21.15.92:0/3088203745 learned_addr learned my addr 172.21.15.92:0/3088203745 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9363700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefe40b33b0 0x7fefe416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefea365700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe40b4e70 0x7fefe40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 -- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefe40b33b0 msgr2=0x7fefe416a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefe40b33b0 0x7fefe416a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 -- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe40b4e70 msgr2=0x7fefe40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe40b4e70 0x7fefe40cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 -- 172.21.15.92:0/3088203745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0094f0 con 0x7fefe40b16d0 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefea365700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe40b4e70 0x7fefe40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.002+0000 7fefe9b64700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 0x7fefe40c8f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefe000cb10 tx=0x7fefe00017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.003+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe00020c0 con 0x7fefe40b16d0 2023-10-06T03:01:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.003+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000a7e0 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.003+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000ab80 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.003+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe416b090 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.003+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe416b650 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.004+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe4088990 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.004+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fefe0003030 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.004+0000 7fefd9ffb700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefc8066f30 0x7fefc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.004+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fefe00846e0 con 0x7fefe40b16d0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.004+0000 7fefe9363700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefc8066f30 0x7fefc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.005+0000 7fefe9363700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefc8066f30 0x7fefc80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fefe41580c0 tx=0x7fefd40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.008+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefe00517f0 con 0x7fefe40b16d0 2023-10-06T03:01:55.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.190+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fefe404bb80 con 0x7fefe40b16d0 2023-10-06T03:01:55.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.191+0000 7fefd9ffb700 1 -- 172.21.15.92:0/3088203745 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fefe0054e10 con 0x7fefe40b16d0 2023-10-06T03:01:55.193 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:01:55.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefc8066f30 msgr2=0x7fefc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefc8066f30 0x7fefc80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fefe41580c0 tx=0x7fefd40091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 msgr2=0x7fefe40c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 0x7fefe40c8f10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fefe000cb10 tx=0x7fefe00017b0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 shutdown_connections 2023-10-06T03:01:55.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefc8066f30 0x7fefc80693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefe40b16d0 0x7fefe40c8f10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefe40b4e70 0x7fefe40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 --2- 172.21.15.92:0/3088203745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefe40b33b0 0x7fefe416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 >> 172.21.15.92:0/3088203745 conn(0x7fefe40d10c0 msgr2=0x7fefe415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 shutdown_connections 2023-10-06T03:01:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.193+0000 7fefeab66700 1 -- 172.21.15.92:0/3088203745 wait complete. 2023-10-06T03:01:55.207 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:01:55.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.318+0000 7f8d55121700 1 Processor -- start 2023-10-06T03:01:55.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.318+0000 7f8d55121700 1 -- start start 2023-10-06T03:01:55.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d55121700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8d5006bcd0 0x7f8d500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d55121700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 0x7f8d5015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d55121700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5015d7e0 con 0x7f8d5006c0a0 2023-10-06T03:01:55.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d55121700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5015dac0 con 0x7f8d5006bcd0 2023-10-06T03:01:55.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d55121700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5015aa30 con 0x7f8d5015df50 2023-10-06T03:01:55.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d4e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d5006c0a0 0x7f8d5015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39070/0 (socket says 172.21.15.92:39070) 2023-10-06T03:01:55.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d4e59c700 1 -- 172.21.15.92:0/3404300568 learned_addr learned my addr 172.21.15.92:0/3404300568 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:55.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d4ed9d700 1 --2- 172.21.15.92:0/3404300568 >> v2:172.21.15.136:3300/0 conn(0x7f8d5006bcd0 0x7f8d500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d4f59e700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 0x7f8d5015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.319+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898741435 0 0) 0x7f8d5015d7e0 con 0x7f8d5006c0a0 2023-10-06T03:01:55.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d38003570 con 0x7f8d5006c0a0 2023-10-06T03:01:55.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540725062 0 0) 0x7f8d38003570 con 0x7f8d5006c0a0 2023-10-06T03:01:55.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 msgr2=0x7f8d5015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 0x7f8d5015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 >> v2:172.21.15.136:3300/0 conn(0x7f8d5006bcd0 msgr2=0x7f8d500caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 --2- 172.21.15.92:0/3404300568 >> v2:172.21.15.136:3300/0 conn(0x7f8d5006bcd0 0x7f8d500caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5015ad10 con 0x7f8d5006c0a0 2023-10-06T03:01:55.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8d44001470 con 0x7f8d5006c0a0 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 >> v1:172.21.15.92:6789/0 conn(0x7f8d5006c0a0 legacy=0x7f8d5015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 0x7f8d38003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38003fe0 0x7f8d38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d38004d50 0x7f8d380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5015dac0 con 0x7f8d38003fe0 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5015aa30 con 0x7f8d38004d50 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3800bb40 con 0x7f8d5015df50 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38003fe0 0x7f8d38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4ed9d700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 0x7f8d38003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4f59e700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d38004d50 0x7f8d380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 -- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 msgr2=0x7f8d38003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d5015df50 0x7f8d38003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 -- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d38004d50 msgr2=0x7f8d380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d38004d50 0x7f8d380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 -- 172.21.15.92:0/3404300568 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d38003570 con 0x7f8d38003fe0 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.320+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38003fe0 0x7f8d38004810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400c390 tx=0x7f8d44001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75a3354085a91ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400cc60 con 0x7f8d38003fe0 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4400cde0 con 0x7f8d38003fe0 2023-10-06T03:01:55.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d4dd9b700 1 -- 172.21.15.92:0/3404300568 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44005b70 con 0x7f8d38003fe0 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d55121700 1 -- 172.21.15.92:0/3404300568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38003fe0 msgr2=0x7f8d38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d55121700 1 --2- 172.21.15.92:0/3404300568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d38003fe0 0x7f8d38004810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400c390 tx=0x7f8d44001e90 comp rx=0 tx=0).stop 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d4f59e700 1 -- 172.21.15.92:0/3404300568 reap_dead start 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d55121700 1 -- 172.21.15.92:0/3404300568 shutdown_connections 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d55121700 1 -- 172.21.15.92:0/3404300568 >> 172.21.15.92:0/3404300568 conn(0x7f8d500ce600 msgr2=0x7f8d500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d55121700 1 -- 172.21.15.92:0/3404300568 shutdown_connections 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.321+0000 7f8d55121700 1 -- 172.21.15.92:0/3404300568 wait complete. 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.322+0000 7f8d55121700 1 Processor -- start 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.322+0000 7f8d55121700 1 -- start start 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d55121700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d38003fe0 0x7f8d50154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d55121700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 0x7f8d500c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d55121700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d50157eb0 0x7f8d50158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d55121700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500c75c0 con 0x7f8d500c56b0 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d55121700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500c6060 con 0x7f8d50157eb0 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d55121700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50158c10 con 0x7f8d38003fe0 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 0x7f8d500c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 0x7f8d500c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34760/0 (socket says 172.21.15.92:34760) 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 -- 172.21.15.92:0/158052482 learned_addr learned my addr 172.21.15.92:0/158052482 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4f59e700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d50157eb0 0x7f8d50158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4ed9d700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d38003fe0 0x7f8d50154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 -- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d38003fe0 msgr2=0x7f8d50154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d38003fe0 0x7f8d50154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 -- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d50157eb0 msgr2=0x7f8d50158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d50157eb0 0x7f8d50158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 -- 172.21.15.92:0/158052482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50158ef0 con 0x7f8d500c56b0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4ed9d700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d38003fe0 0x7f8d50154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4f59e700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d50157eb0 0x7f8d50158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.323+0000 7f8d4e59c700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 0x7f8d500c5b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ba90 tx=0x7f8d44015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.324+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400cc60 con 0x7f8d500c56b0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.324+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440072e0 con 0x7f8d500c56b0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.324+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d501591d0 con 0x7f8d500c56b0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.324+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44007680 con 0x7f8d500c56b0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.324+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d50154a50 con 0x7f8d500c56b0 2023-10-06T03:01:55.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.325+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d500c7750 con 0x7f8d500c56b0 2023-10-06T03:01:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.325+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d44019030 con 0x7f8d500c56b0 2023-10-06T03:01:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.325+0000 7f8d36ffd700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d3c066f30 0x7f8d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.326+0000 7f8d4ed9d700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d3c066f30 0x7f8d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.326+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8d440857d0 con 0x7f8d500c56b0 2023-10-06T03:01:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.326+0000 7f8d4ed9d700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d3c066f30 0x7f8d3c0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8d400099f0 tx=0x7f8d4000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.330+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d500c7750 con 0x7f8d500c56b0 2023-10-06T03:01:55.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.557+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d500c7750 con 0x7f8d500c56b0 2023-10-06T03:01:55.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.558+0000 7f8d36ffd700 1 -- 172.21.15.92:0/158052482 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f8d500c7750 con 0x7f8d500c56b0 2023-10-06T03:01:55.559 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:55.559 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:01:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d3c066f30 msgr2=0x7f8d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d3c066f30 0x7f8d3c0693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8d400099f0 tx=0x7f8d4000b040 comp rx=0 tx=0).stop 2023-10-06T03:01:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 msgr2=0x7f8d500c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 0x7f8d500c5b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ba90 tx=0x7f8d44015000 comp rx=0 tx=0).stop 2023-10-06T03:01:55.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 shutdown_connections 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d3c066f30 0x7f8d3c0693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d50157eb0 0x7f8d50158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d38003fe0 0x7f8d50154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 --2- 172.21.15.92:0/158052482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d500c56b0 0x7f8d500c5b20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.559+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 >> 172.21.15.92:0/158052482 conn(0x7f8d500ce600 msgr2=0x7f8d5015c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.560+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 shutdown_connections 2023-10-06T03:01:55.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.560+0000 7f8d55121700 1 -- 172.21.15.92:0/158052482 wait complete. 2023-10-06T03:01:55.563 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:01:55.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:01:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.686+0000 7f153139e700 1 Processor -- start 2023-10-06T03:01:55.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.686+0000 7f153139e700 1 -- start start 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f153139e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f152c15d230 0x7f152c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f153139e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c0b2980 con 0x7f152c1597f0 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f153139e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c15db70 con 0x7f152c159420 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f153139e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c0b11d0 con 0x7f152c15d230 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f1530b9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f152c15d230 0x7f152c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f152b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f152c1597f0 0x7f152c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39076/0 (socket says 172.21.15.92:39076) 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f152b7fe700 1 -- 172.21.15.92:0/469127638 learned_addr learned my addr 172.21.15.92:0/469127638 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:55.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f1530b9d700 1 -- 172.21.15.92:0/469127638 >> v1:172.21.15.136:6789/0 conn(0x7f152c159420 legacy=0x7f152c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f1530b9d700 1 -- 172.21.15.92:0/469127638 >> v1:172.21.15.92:6789/0 conn(0x7f152c1597f0 legacy=0x7f152c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f1530b9d700 1 -- 172.21.15.92:0/469127638 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0b14b0 con 0x7f152c15d230 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.687+0000 7f152affd700 1 -- 172.21.15.92:0/469127638 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293741789 0 0) 0x7f152c0b2980 con 0x7f152c1597f0 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f1530b9d700 1 --2- 172.21.15.92:0/469127638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c15d630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1524009e60 tx=0x7f1524009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9103e36ceb57d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f152affd700 1 -- 172.21.15.92:0/469127638 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152400e070 con 0x7f152c15d230 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f152affd700 1 -- 172.21.15.92:0/469127638 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152400bc30 con 0x7f152c15d230 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f152affd700 1 -- 172.21.15.92:0/469127638 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524012430 con 0x7f152c15d230 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 -- 172.21.15.92:0/469127638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 msgr2=0x7f152c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 --2- 172.21.15.92:0/469127638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c15d630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1524009e60 tx=0x7f1524009780 comp rx=0 tx=0).stop 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 -- 172.21.15.92:0/469127638 shutdown_connections 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 --2- 172.21.15.92:0/469127638 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c15d630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 -- 172.21.15.92:0/469127638 >> 172.21.15.92:0/469127638 conn(0x7f152c0d0660 msgr2=0x7f152c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:55.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 -- 172.21.15.92:0/469127638 shutdown_connections 2023-10-06T03:01:55.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.688+0000 7f153139e700 1 -- 172.21.15.92:0/469127638 wait complete. 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.689+0000 7f153139e700 1 Processor -- start 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.689+0000 7f153139e700 1 -- start start 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f153139e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f152c159420 0x7f152c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f153139e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f152c1597f0 0x7f152c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f153139e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f153139e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c15b600 con 0x7f152c15d230 2023-10-06T03:01:55.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f153139e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c15a9f0 con 0x7f152c159420 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f153139e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c15acd0 con 0x7f152c1597f0 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f1530b9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f1530b9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34786/0 (socket says 172.21.15.92:34786) 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f1530b9d700 1 -- 172.21.15.92:0/1133770401 learned_addr learned my addr 172.21.15.92:0/1133770401 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f152b7fe700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f152c1597f0 0x7f152c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f152bfff700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f152c159420 0x7f152c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f1530b9d700 1 -- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f152c1597f0 msgr2=0x7f152c15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f1530b9d700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f152c1597f0 0x7f152c15c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.690+0000 7f1530b9d700 1 -- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f152c159420 msgr2=0x7f152c0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f1530b9d700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f152c159420 0x7f152c0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f1530b9d700 1 -- 172.21.15.92:0/1133770401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009a20 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f152b7fe700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f152c1597f0 0x7f152c15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f152bfff700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f152c159420 0x7f152c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f1530b9d700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c155ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f152c089340 tx=0x7f1524019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152401b630 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152401b7d0 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c15afb0 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152401bb90 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.691+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c15ba90 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.692+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c04c3c0 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.692+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f152401c400 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.692+0000 7f1513fff700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1518066f30 0x7f15180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.692+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1524086590 con 0x7f152c15d230 2023-10-06T03:01:55.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.692+0000 7f152bfff700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1518066f30 0x7f15180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:55.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.693+0000 7f152bfff700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1518066f30 0x7f15180693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f151c001840 tx=0x7f151c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:55.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.697+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1524053750 con 0x7f152c15d230 2023-10-06T03:01:55.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.883+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f152c15b740 con 0x7f152c15d230 2023-10-06T03:01:55.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.884+0000 7f1513fff700 1 -- 172.21.15.92:0/1133770401 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f1524056d70 con 0x7f152c15d230 2023-10-06T03:01:55.885 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:55.886 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1518066f30 msgr2=0x7f15180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1518066f30 0x7f15180693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f151c001840 tx=0x7f151c009040 comp rx=0 tx=0).stop 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 msgr2=0x7f152c155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c155ce0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f152c089340 tx=0x7f1524019000 comp rx=0 tx=0).stop 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 shutdown_connections 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1518066f30 0x7f15180693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f152c15d230 0x7f152c155ce0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f152c159420 0x7f152c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 --2- 172.21.15.92:0/1133770401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f152c1597f0 0x7f152c15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 >> 172.21.15.92:0/1133770401 conn(0x7f152c0d0660 msgr2=0x7f152c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 shutdown_connections 2023-10-06T03:01:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:55.886+0000 7f153139e700 1 -- 172.21.15.92:0/1133770401 wait complete. 2023-10-06T03:01:55.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:01:56.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.012+0000 7fa3e31ef700 1 Processor -- start 2023-10-06T03:01:56.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.012+0000 7fa3e31ef700 1 -- start start 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3e31ef700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa3dc06bcd0 0x7fa3dc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3e31ef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc15d7e0 con 0x7fa3dc06c0a0 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3e31ef700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc15dac0 con 0x7fa3dc15df50 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3e31ef700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc15aa30 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3dbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa3dc06c0a0 0x7fa3dc15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39084/0 (socket says 172.21.15.92:39084) 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3dbfff700 1 -- 172.21.15.92:0/2647644150 learned_addr learned my addr 172.21.15.92:0/2647644150 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3e0f8b700 1 --2- 172.21.15.92:0/2647644150 >> v2:172.21.15.136:3300/0 conn(0x7fa3dc06bcd0 0x7fa3dc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3db7fe700 1 -- 172.21.15.92:0/2647644150 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891956509 0 0) 0x7fa3dc15d7e0 con 0x7fa3dc06c0a0 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.013+0000 7fa3db7fe700 1 -- 172.21.15.92:0/2647644150 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c4003570 con 0x7fa3dc06c0a0 2023-10-06T03:01:56.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/2647644150 >> v1:172.21.15.136:6789/0 conn(0x7fa3dc15df50 legacy=0x7fa3dc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/2647644150 >> v1:172.21.15.92:6789/0 conn(0x7fa3dc06c0a0 legacy=0x7fa3dc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3db7fe700 1 -- 172.21.15.92:0/2647644150 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449460797 0 0) 0x7fa3c4003570 con 0x7fa3dc06c0a0 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/2647644150 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc15ad10 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3e0f8b700 1 --2- 172.21.15.92:0/2647644150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0caa90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009e60 tx=0x7fa3cc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98835f5ccfc88fde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3db7fe700 1 -- 172.21.15.92:0/2647644150 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00e070 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3db7fe700 1 -- 172.21.15.92:0/2647644150 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc00bb60 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3db7fe700 1 -- 172.21.15.92:0/2647644150 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc012430 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3e31ef700 1 -- 172.21.15.92:0/2647644150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc06bcd0 msgr2=0x7fa3dc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.014+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/2647644150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0caa90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009e60 tx=0x7fa3cc009780 comp rx=0 tx=0).stop 2023-10-06T03:01:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.015+0000 7fa3e31ef700 1 -- 172.21.15.92:0/2647644150 shutdown_connections 2023-10-06T03:01:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.015+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/2647644150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0caa90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.015+0000 7fa3e31ef700 1 -- 172.21.15.92:0/2647644150 >> 172.21.15.92:0/2647644150 conn(0x7fa3dc0ce600 msgr2=0x7fa3dc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.015+0000 7fa3e31ef700 1 -- 172.21.15.92:0/2647644150 shutdown_connections 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.015+0000 7fa3e31ef700 1 -- 172.21.15.92:0/2647644150 wait complete. 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.016+0000 7fa3e31ef700 1 Processor -- start 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.016+0000 7fa3e31ef700 1 -- start start 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3dc06c0a0 0x7fa3dc169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc15df50 0x7fa3dc157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b3cc0 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b2dd0 con 0x7fa3dc15df50 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0b30b0 con 0x7fa3dc06c0a0 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34788/0 (socket says 172.21.15.92:34788) 2023-10-06T03:01:56.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/1566176214 learned_addr learned my addr 172.21.15.92:0/1566176214 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3dc06c0a0 msgr2=0x7fa3dc169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3dc06c0a0 0x7fa3dc169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc15df50 msgr2=0x7fa3dc157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e178c700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc15df50 0x7fa3dc157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc15df50 0x7fa3dc157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 -- 172.21.15.92:0/1566176214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009a20 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e0f8b700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0b1d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b6c0 tx=0x7fa3cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc012de0 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0263f0 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc0b3390 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0267b0 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.017+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc0b3950 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.018+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc0c9990 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.018+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa3cc026ba0 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.019+0000 7fa3d8ff9700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3c8066f30 0x7fa3c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.019+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa3cc086520 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.019+0000 7fa3dbfff700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3c8066f30 0x7fa3c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.019+0000 7fa3dbfff700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3c8066f30 0x7fa3c80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00011e0 tx=0x7fa3d0007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.023+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3cc053630 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.250+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3dc04c3c0 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.250+0000 7fa3d8ff9700 1 -- 172.21.15.92:0/1566176214 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fa3cc056c50 con 0x7fa3dc06bcd0 2023-10-06T03:01:56.252 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:01:56.252 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:01:56.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3c8066f30 msgr2=0x7fa3c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3c8066f30 0x7fa3c80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00011e0 tx=0x7fa3d0007990 comp rx=0 tx=0).stop 2023-10-06T03:01:56.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 msgr2=0x7fa3dc0b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0b1d60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b6c0 tx=0x7fa3cc019000 comp rx=0 tx=0).stop 2023-10-06T03:01:56.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 shutdown_connections 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa3c8066f30 0x7fa3c80693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3dc06bcd0 0x7fa3dc0b1d60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3dc15df50 0x7fa3dc157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 --2- 172.21.15.92:0/1566176214 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3dc06c0a0 0x7fa3dc169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 >> 172.21.15.92:0/1566176214 conn(0x7fa3dc0ce600 msgr2=0x7fa3dc158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 shutdown_connections 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.252+0000 7fa3e31ef700 1 -- 172.21.15.92:0/1566176214 wait complete. 2023-10-06T03:01:56.255 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:01:56.668 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:01:41.757251+0000', 'modified': '2023-10-06T03:01:45.275637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4306': {'gid': 4306, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.136:6835/3764157697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6833', 'nonce': 3764157697}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 3764157697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:01:56.668 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:01:56.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T03:01:56.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.783+0000 7fb83a210700 1 Processor -- start 2023-10-06T03:01:56.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.783+0000 7fb83a210700 1 -- start start 2023-10-06T03:01:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb83a210700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb834159890 0x7fb83415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb83a210700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb83415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb83a210700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83415a160 con 0x7fb83415db10 2023-10-06T03:01:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb83a210700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340d13a0 con 0x7fb834159890 2023-10-06T03:01:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb83a210700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83415e140 con 0x7fb834159c60 2023-10-06T03:01:56.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb834159890 0x7fb83415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb833fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb83415db10 0x7fb834069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39096/0 (socket says 172.21.15.92:39096) 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb834159890 0x7fb83415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34796/0 (socket says 172.21.15.92:34796) 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb833fff700 1 -- 172.21.15.92:0/1873137174 learned_addr learned my addr 172.21.15.92:0/1873137174 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb832ffd700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb83415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 -- 172.21.15.92:0/1873137174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 msgr2=0x7fb83415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb83415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 -- 172.21.15.92:0/1873137174 >> v1:172.21.15.92:6789/0 conn(0x7fb83415db10 legacy=0x7fb834069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 -- 172.21.15.92:0/1873137174 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83415a3d0 con 0x7fb834159890 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8327fc700 1 -- 172.21.15.92:0/1873137174 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891146129 0 0) 0x7fb83415a160 con 0x7fb83415db10 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb832ffd700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb83415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:56.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8337fe700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb83415abd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb824009e60 tx=0x7fb824009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f2d977057e93eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.784+0000 7fb8327fc700 1 -- 172.21.15.92:0/1873137174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400e070 con 0x7fb834159890 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb8327fc700 1 -- 172.21.15.92:0/1873137174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82400bc30 con 0x7fb834159890 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb8327fc700 1 -- 172.21.15.92:0/1873137174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824012430 con 0x7fb834159890 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 -- 172.21.15.92:0/1873137174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 msgr2=0x7fb83415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb83415abd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb824009e60 tx=0x7fb824009780 comp rx=0 tx=0).stop 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 -- 172.21.15.92:0/1873137174 shutdown_connections 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb83415abd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 --2- 172.21.15.92:0/1873137174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb83415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 -- 172.21.15.92:0/1873137174 >> 172.21.15.92:0/1873137174 conn(0x7fb834068e20 msgr2=0x7fb8340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 -- 172.21.15.92:0/1873137174 shutdown_connections 2023-10-06T03:01:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.785+0000 7fb83a210700 1 -- 172.21.15.92:0/1873137174 wait complete. 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.786+0000 7fb83a210700 1 Processor -- start 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.786+0000 7fb83a210700 1 -- start start 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb83a210700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb8340b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb83a210700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb8340b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb83a210700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb83415db10 0x7fb8340b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb83a210700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340c90e0 con 0x7fb834159890 2023-10-06T03:01:56.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb83a210700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340c84d0 con 0x7fb83415db10 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb83a210700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340c87b0 con 0x7fb834159c60 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb8340b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb8340b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34804/0 (socket says 172.21.15.92:34804) 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 -- 172.21.15.92:0/2546162761 learned_addr learned my addr 172.21.15.92:0/2546162761 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb832ffd700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb8340b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 -- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 msgr2=0x7fb8340b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb8340b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 -- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb83415db10 msgr2=0x7fb8340b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb833fff700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb83415db10 0x7fb8340b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb83415db10 0x7fb8340b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 -- 172.21.15.92:0/2546162761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb824009a20 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.787+0000 7fb8337fe700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb8340b5ec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb824013000 tx=0x7fb824019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.788+0000 7fb81b7fe700 1 -- 172.21.15.92:0/2546162761 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824027470 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.788+0000 7fb81b7fe700 1 -- 172.21.15.92:0/2546162761 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8240275f0 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.788+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8340c8a90 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.788+0000 7fb81b7fe700 1 -- 172.21.15.92:0/2546162761 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8240279b0 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.788+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8340b6400 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.789+0000 7fb81b7fe700 1 -- 172.21.15.92:0/2546162761 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb824024070 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.789+0000 7fb81b7fe700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb820066f30 0x7fb8200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.789+0000 7fb81b7fe700 1 -- 172.21.15.92:0/2546162761 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb824087130 con 0x7fb834159890 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.789+0000 7fb832ffd700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb820066f30 0x7fb8200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.790+0000 7fb832ffd700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb820066f30 0x7fb8200693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb83416cc90 tx=0x7fb828008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.791+0000 7fb843508b80 1 Processor -- start 2023-10-06T03:01:56.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.791+0000 7fb843508b80 1 -- start start 2023-10-06T03:01:56.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f128c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c6cc0 0x557f129156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb843508b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 0x557f12916050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb843508b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1281ca70 con 0x557f12915c00 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f1281a270 con 0x557f128c5840 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f12816060 con 0x557f128c6cc0 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb833fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 0x557f12916050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb833fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 0x557f12916050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34820/0 (socket says 172.21.15.92:34820) 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb833fff700 1 -- 172.21.15.92:0/2815619849 learned_addr learned my addr 172.21.15.92:0/2815619849 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb8337fe700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f128c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.792+0000 7fb833fff700 1 -- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c6cc0 msgr2=0x557f129156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb832ffd700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c6cc0 0x557f129156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb833fff700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c6cc0 0x557f129156c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb833fff700 1 -- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 msgr2=0x557f128c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb833fff700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f128c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb833fff700 1 -- 172.21.15.92:0/2815619849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f126b0270 con 0x557f12915c00 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb8337fe700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f128c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb832ffd700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c6cc0 0x557f129156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb833fff700 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 0x557f12916050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0016f0 tx=0x7fb82c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba690ce9a555f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb818ff9700 1 -- 172.21.15.92:0/2815619849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00b070 con 0x557f12915c00 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb818ff9700 1 -- 172.21.15.92:0/2815619849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c005ad0 con 0x557f12915c00 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb818ff9700 1 -- 172.21.15.92:0/2815619849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c005e50 con 0x557f12915c00 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb843508b80 1 -- 172.21.15.92:0/2815619849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 msgr2=0x557f12916050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.793+0000 7fb843508b80 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 0x557f12916050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0016f0 tx=0x7fb82c009040 comp rx=0 tx=0).stop 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 -- 172.21.15.92:0/2815619849 shutdown_connections 2023-10-06T03:01:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f12915c00 0x557f12916050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f128c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 --2- 172.21.15.92:0/2815619849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c6cc0 0x557f129156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 -- 172.21.15.92:0/2815619849 >> 172.21.15.92:0/2815619849 conn(0x557f128c7160 msgr2=0x557f128c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 -- 172.21.15.92:0/2815619849 shutdown_connections 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 -- 172.21.15.92:0/2815619849 wait complete. 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 -- start start 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.794+0000 7fb843508b80 1 Processor -- start 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12908f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12908190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f129086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f127c1890 con 0x557f128c6cc0 2023-10-06T03:01:56.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f127dcea0 con 0x557f128c5840 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f12698e30 con 0x557f12915c00 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb843508b80 10 client.0 _ll_register_callbacks cb 0x557f12888e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12908190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12908190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34830/0 (socket says 172.21.15.92:34830) 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 -- 172.21.15.92:0/3540650516 learned_addr learned my addr 172.21.15.92:0/3540650516 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb8337fe700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12908f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb833fff700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f129086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 -- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 msgr2=0x557f129086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f129086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 -- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 msgr2=0x557f12908f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12908f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 -- 172.21.15.92:0/3540650516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c0012a0 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb832ffd700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12908190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb83415a090 tx=0x7fb828016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e5e2aeb5742350 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.795+0000 7fb811ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb811ffb700 1 -- 172.21.15.92:0/3540650516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800b050 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb811ffb700 1 -- 172.21.15.92:0/3540650516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb828013600 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f128ba940 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb811ffb700 1 -- 172.21.15.92:0/3540650516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8280139d0 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 10 client.4686 fetch_fsmap learned FSMap version 7 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x557f12652fe0 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb811ffb700 1 -- 172.21.15.92:0/3540650516 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557f128ba940 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb811ffb700 1 -- 172.21.15.92:0/3540650516 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb828013b30 con 0x557f128c6cc0 2023-10-06T03:01:56.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 10 client.4686 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 10 client.4686 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 20 client.4686 populate_metadata read hostname 'smithi092' 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f0970 0x557f12929690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 4 client.4686 mds_command: new command op to 4306 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 --> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f12897870 con 0x557f128f0970 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.796+0000 7fb8337fe700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f0970 0x557f12929690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.797+0000 7fb811ffb700 10 client.4686 ms_handle_connect on v2:172.21.15.136:6833/3764157697 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.797+0000 7fb8337fe700 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f0970 0x557f12929690 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6124e13df5d790 server_cookie=3f3deb8d77704392 in_seq=0 out_seq=0 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.797+0000 7fb811ffb700 1 -- 172.21.15.92:0/3540650516 <== mds.0 v2:172.21.15.136:6833/3764157697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557f12897870 con 0x557f128f0970 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.797+0000 7fb811ffb700 10 client.4686 handle_command_reply: tid=0 2023-10-06T03:01:56.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.797+0000 7fb843508b80 1 client.4686 shutdown 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 msgr2=0x557f12908190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12908190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb83415a090 tx=0x7fb828016040 comp rx=0 tx=0).stop 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 shutdown_connections 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f0970 0x557f12929690 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f129086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb811ffb700 0 client.4686 ms_handle_reset on v2:172.21.15.136:6833/3764157697 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12908f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 --2- 172.21.15.92:0/3540650516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12908190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 >> 172.21.15.92:0/3540650516 conn(0x557f128c7160 msgr2=0x557f128c61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 shutdown_connections 2023-10-06T03:01:56.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 1 -- 172.21.15.92:0/3540650516 wait complete. 2023-10-06T03:01:56.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.798+0000 7fb843508b80 20 client.4686 trim_cache size 0 max 16384 2023-10-06T03:01:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 Processor -- start 2023-10-06T03:01:56.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 -- start start 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12915460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12906450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f12906990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f128f2710 con 0x557f128c6cc0 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f12904ea0 con 0x557f128c5840 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f129058b0 con 0x557f12915c00 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb832ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12906450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb832ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12906450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34840/0 (socket says 172.21.15.92:34840) 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.815+0000 7fb832ffd700 1 -- 172.21.15.92:0/1787791940 learned_addr learned my addr 172.21.15.92:0/1787791940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb833fff700 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f12906990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb8337fe700 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12915460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb832ffd700 1 -- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 msgr2=0x557f12906990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb832ffd700 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f12906990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb832ffd700 1 -- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 msgr2=0x557f12915460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb832ffd700 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12915460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb832ffd700 1 -- 172.21.15.92:0/1787791940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f127e4780 con 0x557f128c6cc0 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb832ffd700 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12906450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8280083f0 tx=0x7fb828018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bc97d9a038da5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb8197fa700 1 -- 172.21.15.92:0/1787791940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800b070 con 0x557f128c6cc0 2023-10-06T03:01:56.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb8197fa700 1 -- 172.21.15.92:0/1787791940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82801a050 con 0x557f128c6cc0 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb8197fa700 1 -- 172.21.15.92:0/1787791940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828013be0 con 0x557f128c6cc0 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 -- 172.21.15.92:0/1787791940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 msgr2=0x557f12906450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12906450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8280083f0 tx=0x7fb828018000 comp rx=0 tx=0).stop 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 -- 172.21.15.92:0/1787791940 shutdown_connections 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f12915c00 0x557f12906990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f128c5840 0x557f12915460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 --2- 172.21.15.92:0/1787791940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12906450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 -- 172.21.15.92:0/1787791940 >> 172.21.15.92:0/1787791940 conn(0x557f128c7160 msgr2=0x557f1292d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 -- 172.21.15.92:0/1787791940 shutdown_connections 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.816+0000 7fb843508b80 1 -- 172.21.15.92:0/1787791940 wait complete. 2023-10-06T03:01:56.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.817+0000 7fb843508b80 1 -- start start 2023-10-06T03:01:56.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.817+0000 7fb843508b80 1 Processor -- start 2023-10-06T03:01:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c5840 0x557f12975540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12975a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f12915c00 0x557f128f5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f128e9940 con 0x557f128c6cc0 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f128f16e0 con 0x557f12915c00 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f128f4ba0 con 0x557f128c5840 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 10 client.0 _ll_register_callbacks cb 0x557f12888e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12975a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12975a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34854/0 (socket says 172.21.15.92:34854) 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 -- 172.21.15.92:0/3333208719 learned_addr learned my addr 172.21.15.92:0/3333208719 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb8337fe700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c5840 0x557f12975540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb833fff700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f12915c00 0x557f128f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 -- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c5840 msgr2=0x557f12975540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c5840 0x557f12975540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 -- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f12915c00 msgr2=0x557f128f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f12915c00 0x557f128f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 -- 172.21.15.92:0/3333208719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f12652fe0 con 0x557f128c6cc0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb832ffd700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12975a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb828000be0 tx=0x7fb828031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d0ec873f70d386 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb811ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb811ffb700 1 -- 172.21.15.92:0/3333208719 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82800b040 con 0x557f128c6cc0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb811ffb700 1 -- 172.21.15.92:0/3333208719 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb828044c20 con 0x557f128c6cc0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb82c005ad0 con 0x557f128c6cc0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb811ffb700 1 -- 172.21.15.92:0/3333208719 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb828013cb0 con 0x557f128c6cc0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.818+0000 7fb843508b80 10 client.4692 fetch_fsmap learned FSMap version 7 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x557f127cb470 con 0x557f128c6cc0 2023-10-06T03:01:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb811ffb700 1 -- 172.21.15.92:0/3333208719 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c005ad0 con 0x557f128c6cc0 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb811ffb700 1 -- 172.21.15.92:0/3333208719 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb828041070 con 0x557f128c6cc0 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 10 client.4692 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 10 client.4692 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 20 client.4692 populate_metadata read hostname 'smithi092' 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f5570 0x557f12971a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 4 client.4692 mds_command: new command op to 4306 tid=0[{"prefix": "status"}] 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 --> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f127b8e00 con 0x557f128f5570 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb8337fe700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f5570 0x557f12971a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb811ffb700 10 client.4692 ms_handle_connect on v2:172.21.15.136:6833/3764157697 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb8337fe700 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f5570 0x557f12971a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39baa75f33c5d1f server_cookie=f8671a5b904e41e2 in_seq=0 out_seq=0 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb811ffb700 1 -- 172.21.15.92:0/3333208719 <== mds.0 v2:172.21.15.136:6833/3764157697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x557f127b8e00 con 0x557f128f5570 2023-10-06T03:01:56.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb811ffb700 10 client.4692 handle_command_reply: tid=0 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.819+0000 7fb843508b80 1 client.4692 shutdown 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 msgr2=0x557f12975a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12975a80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb828000be0 tx=0x7fb828031000 comp rx=0 tx=0).stop 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 shutdown_connections 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:6833/3764157697,v1:172.21.15.136:6835/3764157697] conn(0x557f128f5570 0x557f12971a90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557f12915c00 0x557f128f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557f128c5840 0x557f12975540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb811ffb700 0 client.4692 ms_handle_reset on v2:172.21.15.136:6833/3764157697 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 --2- 172.21.15.92:0/3333208719 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f128c6cc0 0x557f12975a80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 >> 172.21.15.92:0/3333208719 conn(0x557f128c7160 msgr2=0x557f12909860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 shutdown_connections 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 1 -- 172.21.15.92:0/3333208719 wait complete. 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb843508b80 20 client.4692 trim_cache size 0 max 16384 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4306, 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:01:56.830 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:01:56.831 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 14.457311848, 2023-10-06T03:01:56.831 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 7, 2023-10-06T03:01:56.831 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 23, 2023-10-06T03:01:56.831 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 23, 2023-10-06T03:01:56.831 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 19.464823996 2023-10-06T03:01:56.831 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb820066f30 msgr2=0x7fb8200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb820066f30 0x7fb8200693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb83416cc90 tx=0x7fb828008040 comp rx=0 tx=0).stop 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 msgr2=0x7fb8340b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb8340b5ec0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb824013000 tx=0x7fb824019000 comp rx=0 tx=0).stop 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 shutdown_connections 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb820066f30 0x7fb8200693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb834159890 0x7fb8340b5ec0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb83415db10 0x7fb8340b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 --2- 172.21.15.92:0/2546162761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb834159c60 0x7fb8340b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 >> 172.21.15.92:0/2546162761 conn(0x7fb834068e20 msgr2=0x7fb8340b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 shutdown_connections 2023-10-06T03:01:56.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:56.820+0000 7fb83a210700 1 -- 172.21.15.92:0/2546162761 wait complete. 2023-10-06T03:01:56.834 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T03:01:56.855 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T03:01:56.855 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T03:01:56.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:56.855 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:01:56.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:56.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T03:01:56.937 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-10-06T03:01:56.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:01:56.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:56.939 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:01:56.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:56.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T03:01:57.059 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-10-06T03:01:57.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:01:57.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.060 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:01:57.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.079 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link delete ceph-brx 2023-10-06T03:01:57.151 INFO:teuthology.orchestra.run.smithi136.stderr:Cannot find device "ceph-brx" 2023-10-06T03:01:57.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:01:57.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.152 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:01:57.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.209 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link delete ceph-brx 2023-10-06T03:01:57.276 INFO:teuthology.orchestra.run.smithi136.stderr:Cannot find device "ceph-brx" 2023-10-06T03:01:57.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:01:57.277 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-06T03:01:57.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:01:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.392+0000 7fb32ca9b700 1 Processor -- start 2023-10-06T03:01:57.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.392+0000 7fb32ca9b700 1 -- start start 2023-10-06T03:01:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb32ca9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb328157940 0x7fb32806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb32ca9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb32815baa0 0x7fb32815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb32ca9b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32806c750 con 0x7fb32806c1e0 2023-10-06T03:01:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb32ca9b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32815c460 con 0x7fb32815baa0 2023-10-06T03:01:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb32ca9b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32806ca30 con 0x7fb328157940 2023-10-06T03:01:57.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb31dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb32806c1e0 0x7fb3280cb6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39106/0 (socket says 172.21.15.92:39106) 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.393+0000 7fb31dd9b700 1 -- 172.21.15.92:0/3677299459 learned_addr learned my addr 172.21.15.92:0/3677299459 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb32659c700 1 --2- 172.21.15.92:0/3677299459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb328157940 0x7fb32806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb326d9d700 1 --2- 172.21.15.92:0/3677299459 >> v2:172.21.15.136:3300/0 conn(0x7fb32815baa0 0x7fb32815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb325d9b700 1 -- 172.21.15.92:0/3677299459 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223898793 0 0) 0x7fb32806c750 con 0x7fb32806c1e0 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb325d9b700 1 -- 172.21.15.92:0/3677299459 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb314003570 con 0x7fb32806c1e0 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb32659c700 1 -- 172.21.15.92:0/3677299459 >> v2:172.21.15.136:3300/0 conn(0x7fb32815baa0 msgr2=0x7fb32815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb32659c700 1 --2- 172.21.15.92:0/3677299459 >> v2:172.21.15.136:3300/0 conn(0x7fb32815baa0 0x7fb32815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb32659c700 1 -- 172.21.15.92:0/3677299459 >> v1:172.21.15.92:6789/0 conn(0x7fb32806c1e0 legacy=0x7fb3280cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb325d9b700 1 -- 172.21.15.92:0/3677299459 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4211517555 0 0) 0x7fb314003570 con 0x7fb32806c1e0 2023-10-06T03:01:57.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb32659c700 1 -- 172.21.15.92:0/3677299459 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32806cd10 con 0x7fb328157940 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb32659c700 1 --2- 172.21.15.92:0/3677299459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb328157940 0x7fb32806f5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3180098e0 tx=0x7fb3180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc5244c1292c480 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb325d9b700 1 -- 172.21.15.92:0/3677299459 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310001470 con 0x7fb328157940 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb325d9b700 1 -- 172.21.15.92:0/3677299459 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318011420 con 0x7fb328157940 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.394+0000 7fb325d9b700 1 -- 172.21.15.92:0/3677299459 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3180117c0 con 0x7fb328157940 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 -- 172.21.15.92:0/3677299459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb328157940 msgr2=0x7fb32806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/3677299459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb328157940 0x7fb32806f5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb3180098e0 tx=0x7fb3180091a0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 -- 172.21.15.92:0/3677299459 shutdown_connections 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/3677299459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb328157940 0x7fb32806f5a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/3677299459 >> v2:172.21.15.136:3300/0 conn(0x7fb32815baa0 0x7fb32815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 -- 172.21.15.92:0/3677299459 >> 172.21.15.92:0/3677299459 conn(0x7fb3280cd480 msgr2=0x7fb3280cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:57.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 -- 172.21.15.92:0/3677299459 shutdown_connections 2023-10-06T03:01:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.395+0000 7fb32ca9b700 1 -- 172.21.15.92:0/3677299459 wait complete. 2023-10-06T03:01:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.396+0000 7fb32ca9b700 1 Processor -- start 2023-10-06T03:01:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.396+0000 7fb32ca9b700 1 -- start start 2023-10-06T03:01:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32ca9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb32806c1e0 0x7fb3280c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32ca9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 0x7fb328159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32ca9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb32815baa0 0x7fb32815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:57.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32ca9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280c2350 con 0x7fb328157940 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32ca9b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280c25f0 con 0x7fb32806c1e0 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32ca9b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280c28d0 con 0x7fb32815baa0 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 0x7fb328159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 0x7fb328159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34864/0 (socket says 172.21.15.92:34864) 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 -- 172.21.15.92:0/2532578531 learned_addr learned my addr 172.21.15.92:0/2532578531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb326d9d700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb32815baa0 0x7fb32815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32659c700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb32806c1e0 0x7fb3280c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 -- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb32815baa0 msgr2=0x7fb32815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb32815baa0 0x7fb32815a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 -- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb32806c1e0 msgr2=0x7fb3280c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb32806c1e0 0x7fb3280c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 -- 172.21.15.92:0/2532578531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb318009420 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb32659c700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb32806c1e0 0x7fb3280c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.397+0000 7fb31dd9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 0x7fb328159e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb31000cb90 tx=0x7fb310001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.398+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310001d40 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.398+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb310007360 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.398+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3280c2bb0 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.398+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310004830 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.398+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3280c4250 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.398+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3280c30d0 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.399+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb310004ba0 con 0x7fb328157940 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.399+0000 7fb31effd700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb308066f30 0x7fb3080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:57.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.399+0000 7fb32659c700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb308066f30 0x7fb3080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.399+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb310083e90 con 0x7fb328157940 2023-10-06T03:01:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.400+0000 7fb32659c700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb308066f30 0x7fb3080693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3180093f0 tx=0x7fb318016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:57.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.404+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3280c30d0 con 0x7fb328157940 2023-10-06T03:01:57.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.631+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb3280c30d0 con 0x7fb328157940 2023-10-06T03:01:57.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.631+0000 7fb31effd700 1 -- 172.21.15.92:0/2532578531 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb3280c30d0 con 0x7fb328157940 2023-10-06T03:01:57.633 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:01:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb308066f30 msgr2=0x7fb3080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb308066f30 0x7fb3080693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3180093f0 tx=0x7fb318016000 comp rx=0 tx=0).stop 2023-10-06T03:01:57.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 msgr2=0x7fb328159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 0x7fb328159e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb31000cb90 tx=0x7fb310001ab0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 shutdown_connections 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb308066f30 0x7fb3080693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb32815baa0 0x7fb32815a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb32806c1e0 0x7fb3280c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 --2- 172.21.15.92:0/2532578531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb328157940 0x7fb328159e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 >> 172.21.15.92:0/2532578531 conn(0x7fb3280cd480 msgr2=0x7fb3281512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 shutdown_connections 2023-10-06T03:01:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:57.633+0000 7fb32ca9b700 1 -- 172.21.15.92:0/2532578531 wait complete. 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:01:57.647 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:01:57.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.648 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:01:57.662 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:01:57.662 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:01:57.663 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42488sec preferred_lft 42488sec 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:57.664 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T03:01:57.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add name ceph-brx type bridge 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr flush dev ceph-brx 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set ceph-brx up 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T03:01:57.665 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:57.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.818 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:01:57.859 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:01:57.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.861 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:01:57.935 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:01:57.935 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:01:57.935 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:01:57.935 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:01:57.936 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:01:57.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:57.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:57.936 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:57.936 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:01:57.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:01:57.937 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:01:57.937 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:58.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:58.204 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:01:58.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:58.218 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-10-06T03:01:58.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:58.273 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:58.273 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:58.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:01:58.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T03:01:58.273 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:58.363 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T03:01:58.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:58.363 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:58.363 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:58.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T03:01:58.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T03:01:58.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T03:01:58.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T03:01:58.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T03:01:58.364 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:58.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:58.560 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:58.560 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:58.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:01:58.561 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T03:01:58.561 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:58.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:01:58.652 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:01:58.667 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:01:58.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:01:58.667 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:01:58.721 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:01:58.722 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:01:58.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:01:58.830 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:01:58.830 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:01:58.936 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:01:58.937 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:01:58.961 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:01:58.962 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:01:58.962 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:01:58.962 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:01:58.962 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:01:58.964 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:01:58.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:58.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:01:58.991 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-06T03:01:58.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:01:59.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.143+0000 7f3ea51f7700 1 Processor -- start 2023-10-06T03:01:59.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.143+0000 7f3ea51f7700 1 -- start start 2023-10-06T03:01:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea51f7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea0159420 0x7f3ea00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea51f7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ea015d230 0x7f3ea015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea51f7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea015dbd0 con 0x7f3ea01597f0 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea51f7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00b2980 con 0x7f3ea015d230 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea51f7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00b11d0 con 0x7f3ea0159420 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ea015d230 0x7f3ea015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3ea015d230 0x7f3ea015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34874/0 (socket says 172.21.15.92:34874) 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 -- 172.21.15.92:0/2411313779 learned_addr learned my addr 172.21.15.92:0/2411313779 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3e9ffff700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea0159420 0x7f3ea00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 -- 172.21.15.92:0/2411313779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea0159420 msgr2=0x7f3ea00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea0159420 0x7f3ea00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 -- 172.21.15.92:0/2411313779 >> v1:172.21.15.136:6789/0 conn(0x7f3ea01597f0 legacy=0x7f3ea00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:01:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.144+0000 7f3ea49f6700 1 -- 172.21.15.92:0/2411313779 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00b14b0 con 0x7f3ea015d230 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3e9ffff700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea0159420 0x7f3ea00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea49f6700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea015d690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009f20 tx=0x7f3e98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb98ef59ff8b97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3e9effd700 1 -- 172.21.15.92:0/2411313779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800e070 con 0x7f3ea015d230 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3e9effd700 1 -- 172.21.15.92:0/2411313779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800ba40 con 0x7f3ea015d230 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3e9effd700 1 -- 172.21.15.92:0/2411313779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980092e0 con 0x7f3ea015d230 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 -- 172.21.15.92:0/2411313779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 msgr2=0x7f3ea015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea015d690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009f20 tx=0x7f3e98009840 comp rx=0 tx=0).stop 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 -- 172.21.15.92:0/2411313779 shutdown_connections 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea015d690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/2411313779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea0159420 0x7f3ea00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 -- 172.21.15.92:0/2411313779 >> 172.21.15.92:0/2411313779 conn(0x7f3ea00d0660 msgr2=0x7f3ea00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 -- 172.21.15.92:0/2411313779 shutdown_connections 2023-10-06T03:01:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.145+0000 7f3ea51f7700 1 -- 172.21.15.92:0/2411313779 wait complete. 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.147+0000 7f3ea51f7700 1 Processor -- start 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.147+0000 7f3ea51f7700 1 -- start start 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.147+0000 7f3ea51f7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea0159420 0x7f3ea0169670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.147+0000 7f3ea51f7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea01597f0 0x7f3ea00c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.147+0000 7f3ea51f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea00c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea51f7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00c7a20 con 0x7f3ea015d230 2023-10-06T03:01:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea51f7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00c7ea0 con 0x7f3ea0159420 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea51f7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00c7590 con 0x7f3ea01597f0 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea00c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea00c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34878/0 (socket says 172.21.15.92:34878) 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 -- 172.21.15.92:0/3447529225 learned_addr learned my addr 172.21.15.92:0/3447529225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3e9f7fe700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea01597f0 0x7f3ea00c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3e9ffff700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea0159420 0x7f3ea0169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 -- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea01597f0 msgr2=0x7f3ea00c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea01597f0 0x7f3ea00c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 -- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea0159420 msgr2=0x7f3ea0169670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea0159420 0x7f3ea0169670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 -- 172.21.15.92:0/3447529225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98009ae0 con 0x7f3ea015d230 2023-10-06T03:01:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea49f6700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea00c62f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0089340 tx=0x7f3e98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980032f0 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e98012d00 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00c6980 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9801d670 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.148+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00c6f40 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.149+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea004c3c0 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.149+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e9801da60 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.150+0000 7f3e87fff700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e8c066f30 0x7f3e8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.150+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3e98085d50 con 0x7f3ea015d230 2023-10-06T03:01:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.150+0000 7f3e9ffff700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e8c066f30 0x7f3e8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:01:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.150+0000 7f3e9ffff700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e8c066f30 0x7f3e8c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000adc0 tx=0x7f3e90006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:01:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.154+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e98089030 con 0x7f3ea015d230 2023-10-06T03:01:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.382+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ea015dbd0 con 0x7f3ea015d230 2023-10-06T03:01:59.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.382+0000 7f3e87fff700 1 -- 172.21.15.92:0/3447529225 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ea015dbd0 con 0x7f3ea015d230 2023-10-06T03:01:59.384 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:01:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e8c066f30 msgr2=0x7f3e8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:59.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e8c066f30 0x7f3e8c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000adc0 tx=0x7f3e90006f90 comp rx=0 tx=0).stop 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 msgr2=0x7f3ea00c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea00c62f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0089340 tx=0x7f3e98017000 comp rx=0 tx=0).stop 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 shutdown_connections 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e8c066f30 0x7f3e8c0693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ea015d230 0x7f3ea00c62f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea0159420 0x7f3ea0169670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 --2- 172.21.15.92:0/3447529225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea01597f0 0x7f3ea00c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.384+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 >> 172.21.15.92:0/3447529225 conn(0x7f3ea00d0660 msgr2=0x7f3ea0155180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.385+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 shutdown_connections 2023-10-06T03:01:59.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:01:59.385+0000 7f3ea51f7700 1 -- 172.21.15.92:0/3447529225 wait complete. 2023-10-06T03:01:59.398 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:01:59.398 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:01:59.398 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:01:59.398 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T03:01:59.399 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T03:01:59.399 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:01:59.399 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:01:59.399 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:01:59.399 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T03:01:59.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.399 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.414 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42487sec preferred_lft 42487sec 2023-10-06T03:01:59.415 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link tentative 2023-10-06T03:01:59.416 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.417 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:01:59.478 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:01:59.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.479 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:01:59.535 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:01:59.536 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:01:59.536 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:01:59.536 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:01:59.536 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:01:59.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.536 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:59.536 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:59.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:01:59.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:01:59.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:01:59.537 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:59.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.635 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:01:59.649 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:01:59.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.650 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list-id 2023-10-06T03:01:59.704 INFO:teuthology.orchestra.run.smithi092.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T03:01:59.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.704 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:59.704 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:59.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:01:59.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T03:01:59.704 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:01:59.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T03:01:59.826 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:01:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:01:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:01:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:01:59.826 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.827 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:01:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether be:32:84:65:69:bc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:01:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:01:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::bc32:84ff:fe65:69bc/64 scope link tentative 2023-10-06T03:01:59.827 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.909 INFO:teuthology.orchestra.run.smithi092.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:01:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether be:32:84:65:69:bc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:01:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:01:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::bc32:84ff:fe65:69bc/64 scope link tentative 2023-10-06T03:01:59.910 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:01:59.926 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T03:01:59.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T03:01:59.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T03:01:59.927 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:02:00.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:00.186 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:02:00.186 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:02:00.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.1 up 2023-10-06T03:02:00.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T03:02:00.186 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:02:00.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:00.521 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T03:02:00.536 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T03:02:00.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:00.537 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T03:02:00.590 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-06T03:02:00.726 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:02:00.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:00.767 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:00.767 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:00.785 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 2@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:01.245 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 2@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:01.247 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:02:01.248 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:02:01.248 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:02:01.248 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:02:01.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:01.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T03:02:01.312 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-06T03:02:01.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:02:01.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.667+0000 7fca481f5700 1 Processor -- start 2023-10-06T03:02:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.667+0000 7fca481f5700 1 -- start start 2023-10-06T03:02:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca481f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca40156fb0 0x7fca4006a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca481f5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca4006ab20 0x7fca400cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca481f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca400cbf40 con 0x7fca400cbae0 2023-10-06T03:02:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca481f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4006af70 con 0x7fca40156fb0 2023-10-06T03:02:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca481f5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca40069d30 con 0x7fca4006ab20 2023-10-06T03:02:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca45f91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca40156fb0 0x7fca4006a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca46792700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fca400cbae0 0x7fca40159bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56290/0 (socket says 172.21.15.92:56290) 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.668+0000 7fca46792700 1 -- 172.21.15.92:0/4045980801 learned_addr learned my addr 172.21.15.92:0/4045980801 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca45790700 1 --2- 172.21.15.92:0/4045980801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca4006ab20 0x7fca400cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca45f91700 1 -- 172.21.15.92:0/4045980801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca4006ab20 msgr2=0x7fca400cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca45f91700 1 --2- 172.21.15.92:0/4045980801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca4006ab20 0x7fca400cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca45f91700 1 -- 172.21.15.92:0/4045980801 >> v1:172.21.15.92:6789/0 conn(0x7fca400cbae0 legacy=0x7fca40159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca44f8f700 1 -- 172.21.15.92:0/4045980801 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533537475 0 0) 0x7fca38001490 con 0x7fca400cbae0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca45f91700 1 -- 172.21.15.92:0/4045980801 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca400c8110 con 0x7fca40156fb0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca45f91700 1 --2- 172.21.15.92:0/4045980801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40156fb0 0x7fca4006a5e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00aa90 tx=0x7fca3c00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb116416796b4d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca44f8f700 1 -- 172.21.15.92:0/4045980801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00ca40 con 0x7fca40156fb0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca44f8f700 1 -- 172.21.15.92:0/4045980801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00cba0 con 0x7fca40156fb0 2023-10-06T03:02:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.669+0000 7fca44f8f700 1 -- 172.21.15.92:0/4045980801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c018400 con 0x7fca40156fb0 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 -- 172.21.15.92:0/4045980801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40156fb0 msgr2=0x7fca4006a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 --2- 172.21.15.92:0/4045980801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40156fb0 0x7fca4006a5e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00aa90 tx=0x7fca3c00a330 comp rx=0 tx=0).stop 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 -- 172.21.15.92:0/4045980801 shutdown_connections 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 --2- 172.21.15.92:0/4045980801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca40156fb0 0x7fca4006a5e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 --2- 172.21.15.92:0/4045980801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca4006ab20 0x7fca400cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 -- 172.21.15.92:0/4045980801 >> 172.21.15.92:0/4045980801 conn(0x7fca400c9870 msgr2=0x7fca400c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 -- 172.21.15.92:0/4045980801 shutdown_connections 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.670+0000 7fca481f5700 1 -- 172.21.15.92:0/4045980801 wait complete. 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.671+0000 7fca481f5700 1 Processor -- start 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.671+0000 7fca481f5700 1 -- start start 2023-10-06T03:02:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 0x7fca400aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca400cbae0 0x7fca400c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca40156fb0 0x7fca4014d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40150ee0 con 0x7fca4006ab20 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca400c3f50 con 0x7fca40156fb0 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca400c4230 con 0x7fca400cbae0 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 0x7fca400aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 0x7fca400aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45286/0 (socket says 172.21.15.92:45286) 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 -- 172.21.15.92:0/956245180 learned_addr learned my addr 172.21.15.92:0/956245180 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca46792700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca40156fb0 0x7fca4014d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45790700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca400cbae0 0x7fca400c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 -- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca400cbae0 msgr2=0x7fca400c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca400cbae0 0x7fca400c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 -- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca40156fb0 msgr2=0x7fca4014d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca40156fb0 0x7fca4014d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 -- 172.21.15.92:0/956245180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c00a5d0 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45790700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca400cbae0 0x7fca400c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca45f91700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 0x7fca400aefd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00a5a0 tx=0x7fca3c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00cee0 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c010600 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca400c4510 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.672+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0109a0 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.673+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca400af510 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.673+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca400c4b40 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.674+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca3c010d50 con 0x7fca4006ab20 2023-10-06T03:02:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.674+0000 7fca35ffb700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca24066f30 0x7fca240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.674+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca3c085cb0 con 0x7fca4006ab20 2023-10-06T03:02:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.674+0000 7fca45790700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca24066f30 0x7fca240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.675+0000 7fca45790700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca24066f30 0x7fca240693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca4016d5f0 tx=0x7fca30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.678+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca3c052dc0 con 0x7fca4006ab20 2023-10-06T03:02:01.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.902+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca4004c3c0 con 0x7fca4006ab20 2023-10-06T03:02:01.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.903+0000 7fca35ffb700 1 -- 172.21.15.92:0/956245180 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fca3c026070 con 0x7fca4006ab20 2023-10-06T03:02:01.905 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca24066f30 msgr2=0x7fca240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca24066f30 0x7fca240693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca4016d5f0 tx=0x7fca30008040 comp rx=0 tx=0).stop 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 msgr2=0x7fca400aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 0x7fca400aefd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00a5a0 tx=0x7fca3c01a000 comp rx=0 tx=0).stop 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 shutdown_connections 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca24066f30 0x7fca240693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca4006ab20 0x7fca400aefd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca40156fb0 0x7fca4014d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 --2- 172.21.15.92:0/956245180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca400cbae0 0x7fca400c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 >> 172.21.15.92:0/956245180 conn(0x7fca400c9870 msgr2=0x7fca400c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:01.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 shutdown_connections 2023-10-06T03:02:01.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:01.905+0000 7fca481f5700 1 -- 172.21.15.92:0/956245180 wait complete. 2023-10-06T03:02:01.918 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:02:01.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:02:01.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:02:01.919 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:02:01.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:01.919 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:02:01.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:02:01.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:02:01.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:02:01.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:01.919 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:01.938 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:02.009 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42485sec preferred_lft 42485sec 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:02.010 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T03:02:02.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.010 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:02.010 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:02.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link add name ceph-brx type bridge 2023-10-06T03:02:02.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip addr flush dev ceph-brx 2023-10-06T03:02:02.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set ceph-brx up 2023-10-06T03:02:02.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T03:02:02.011 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:02.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.183 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:02:02.209 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:02:02.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.328 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:02:02.414 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:02:02.475 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:02:02.476 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:02:02.476 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:02:02.476 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:02:02.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.476 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:02.476 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:02.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:02:02.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:02:02.477 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:02:02.477 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:02.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.791 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:02:02.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.806 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list-id 2023-10-06T03:02:02.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.859 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:02.859 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:02.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:02:02.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T03:02:02.860 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:02.948 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T03:02:02.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T03:02:02.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T03:02:02.949 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:03.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:03.176 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:03.177 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:03.177 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:02:03.177 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T03:02:03.177 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:03.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:03.269 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:03.283 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:02:03.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:03.283 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:03.337 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:02:03.337 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:02:03.338 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:03.424 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:03.424 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:03.507 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:03.507 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:03.508 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:03.508 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:03.508 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:03.508 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:03.508 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:03.508 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:03.529 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:03.530 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:02:03.530 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:02:03.530 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:02:03.531 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:02:03.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:03.532 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:03.556 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-06T03:02:03.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:02:03.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.678+0000 7fd7234e0700 1 Processor -- start 2023-10-06T03:02:03.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.678+0000 7fd7234e0700 1 -- start start 2023-10-06T03:02:03.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd7234e0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd71c155c10 0x7fd71c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd7234e0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd71c0ac6d0 0x7fd71c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd7234e0700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c0acac0 con 0x7fd71c0ac230 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd7234e0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0ad930 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd7234e0700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c0acd20 con 0x7fd71c155c10 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd721a7d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd71c0ac6d0 0x7fd71c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd721a7d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd71c0ac6d0 0x7fd71c0b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45298/0 (socket says 172.21.15.92:45298) 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.679+0000 7fd721a7d700 1 -- 172.21.15.92:0/1358903472 learned_addr learned my addr 172.21.15.92:0/1358903472 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd72127c700 1 --2- 172.21.15.92:0/1358903472 >> v2:172.21.15.136:3300/0 conn(0x7fd71c155c10 0x7fd71c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd721a7d700 1 -- 172.21.15.92:0/1358903472 >> v2:172.21.15.136:3300/0 conn(0x7fd71c155c10 msgr2=0x7fd71c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd721a7d700 1 --2- 172.21.15.92:0/1358903472 >> v2:172.21.15.136:3300/0 conn(0x7fd71c155c10 0x7fd71c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd721a7d700 1 -- 172.21.15.92:0/1358903472 >> v1:172.21.15.136:6789/0 conn(0x7fd71c0ac230 legacy=0x7fd71c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd721a7d700 1 -- 172.21.15.92:0/1358903472 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0ad000 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd72127c700 1 --2- 172.21.15.92:0/1358903472 >> v2:172.21.15.136:3300/0 conn(0x7fd71c155c10 0x7fd71c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd721a7d700 1 --2- 172.21.15.92:0/1358903472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0b0300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd71c063d40 tx=0x7fd71400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1ff5ba87ff39b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd71bfff700 1 -- 172.21.15.92:0/1358903472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400dc10 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd71bfff700 1 -- 172.21.15.92:0/1358903472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400dd70 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.680+0000 7fd71bfff700 1 -- 172.21.15.92:0/1358903472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140106c0 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 -- 172.21.15.92:0/1358903472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 msgr2=0x7fd71c0b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 --2- 172.21.15.92:0/1358903472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0b0300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd71c063d40 tx=0x7fd71400b4f0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 -- 172.21.15.92:0/1358903472 shutdown_connections 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 --2- 172.21.15.92:0/1358903472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0b0300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 --2- 172.21.15.92:0/1358903472 >> v2:172.21.15.136:3300/0 conn(0x7fd71c155c10 0x7fd71c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 -- 172.21.15.92:0/1358903472 >> 172.21.15.92:0/1358903472 conn(0x7fd71c06a320 msgr2=0x7fd71c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:03.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 -- 172.21.15.92:0/1358903472 shutdown_connections 2023-10-06T03:02:03.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.681+0000 7fd7234e0700 1 -- 172.21.15.92:0/1358903472 wait complete. 2023-10-06T03:02:03.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.682+0000 7fd7234e0700 1 Processor -- start 2023-10-06T03:02:03.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.682+0000 7fd7234e0700 1 -- start start 2023-10-06T03:02:03.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd7234e0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd71c0ac230 0x7fd71c152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:03.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd7234e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd7234e0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd71c155c10 0x7fd71c0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd7234e0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c1657e0 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd7234e0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c165a80 con 0x7fd71c155c10 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd7234e0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0aa430 con 0x7fd71c0ac230 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0a9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45308/0 (socket says 172.21.15.92:45308) 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 -- 172.21.15.92:0/2703418603 learned_addr learned my addr 172.21.15.92:0/2703418603 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd721a7d700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd71c155c10 0x7fd71c0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd72127c700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd71c0ac230 0x7fd71c152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 -- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd71c0ac230 msgr2=0x7fd71c152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd71c0ac230 0x7fd71c152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 -- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd71c155c10 msgr2=0x7fd71c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd71c155c10 0x7fd71c0cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 -- 172.21.15.92:0/2703418603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71400b790 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd72127c700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd71c0ac230 0x7fd71c152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.683+0000 7fd720a7b700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0a9a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c5d0 tx=0x7fd710007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.684+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710011590 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.684+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd710011710 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.684+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c0aa710 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.684+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710011ad0 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.684+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c152580 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.684+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c06bb10 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.685+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd71001c460 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.685+0000 7fd7197fa700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd708066f30 0x7fd7080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.685+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd710084490 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.685+0000 7fd72127c700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd708066f30 0x7fd7080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.686+0000 7fd72127c700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd708066f30 0x7fd7080693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00d4a0 tx=0x7fd70c0075d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:03.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.690+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7100515a0 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.916+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd71c165bf0 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.916+0000 7fd7197fa700 1 -- 172.21.15.92:0/2703418603 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd710054bc0 con 0x7fd71c0ac6d0 2023-10-06T03:02:03.918 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:02:03.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.918+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd708066f30 msgr2=0x7fd7080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.918+0000 7fd7234e0700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd708066f30 0x7fd7080693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00d4a0 tx=0x7fd70c0075d0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.918+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 msgr2=0x7fd71c0a9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.918+0000 7fd7234e0700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0a9a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd71000c5d0 tx=0x7fd710007f60 comp rx=0 tx=0).stop 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 shutdown_connections 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd708066f30 0x7fd7080693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd71c155c10 0x7fd71c0cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd71c0ac230 0x7fd71c152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 --2- 172.21.15.92:0/2703418603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd71c0ac6d0 0x7fd71c0a9a50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 >> 172.21.15.92:0/2703418603 conn(0x7fd71c06a320 msgr2=0x7fd71c0ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 shutdown_connections 2023-10-06T03:02:03.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:03.919+0000 7fd7234e0700 1 -- 172.21.15.92:0/2703418603 wait complete. 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:02:03.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:02:03.933 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:02:03.933 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T03:02:03.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:03.933 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:02:03.948 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:02:03.948 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:03.949 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42483sec preferred_lft 42483sec 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:03.950 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:02:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:03.951 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:02:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:02:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link tentative 2023-10-06T03:02:03.951 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:03.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:03.951 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:02:04.011 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:02:04.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.012 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:02:04.068 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:02:04.068 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:02:04.068 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:02:04.069 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:02:04.069 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:02:04.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.069 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:04.069 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:04.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:02:04.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:02:04.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:02:04.069 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:04.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.167 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:02:04.181 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:02:04.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.182 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list-id 2023-10-06T03:02:04.237 INFO:teuthology.orchestra.run.smithi136.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T03:02:04.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.237 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:04.238 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:04.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:02:04.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T03:02:04.238 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:04.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.326 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T03:02:04.360 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:02:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:02:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:02:04.360 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.361 INFO:teuthology.orchestra.run.smithi136.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:02:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 06:09:3d:23:9d:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:02:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:02:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::409:3dff:fe23:9deb/64 scope link 2023-10-06T03:02:04.361 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.367 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T03:02:04.441 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:04.441 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 06:09:3d:23:9d:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.442 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::409:3dff:fe23:9deb/64 scope link 2023-10-06T03:02:04.443 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:04.458 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T03:02:04.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T03:02:04.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T03:02:04.460 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:04.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.699 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:04.699 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:04.699 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.1 up 2023-10-06T03:02:04.699 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T03:02:04.700 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:04.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:04.779 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T03:02:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T03:02:04.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:04.793 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T03:02:04.847 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-06T03:02:04.848 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:02:04.848 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:04.927 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:04.927 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:04.945 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 3@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:04.946 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:04.967 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:04.967 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:04.967 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:04.967 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 3@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:04.967 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:02:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:02:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:02:04.969 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:02:04.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:04.969 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T03:02:05.034 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T03:02:05.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:05.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.167+0000 7fec99c7a700 1 Processor -- start 2023-10-06T03:02:05.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.167+0000 7fec99c7a700 1 -- start start 2023-10-06T03:02:05.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec99c7a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159890 0x7fec9415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec99c7a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec94159c60 0x7fec9415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec99c7a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9415a160 con 0x7fec9415db10 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec99c7a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec940d13a0 con 0x7fec94159c60 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec99c7a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9415e140 con 0x7fec94159890 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec92ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec94159c60 0x7fec9415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec92ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec94159c60 0x7fec9415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45322/0 (socket says 172.21.15.92:45322) 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec92ffd700 1 -- 172.21.15.92:0/1592763795 learned_addr learned my addr 172.21.15.92:0/1592763795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.168+0000 7fec937fe700 1 --2- 172.21.15.92:0/1592763795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159890 0x7fec9415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec92ffd700 1 -- 172.21.15.92:0/1592763795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159890 msgr2=0x7fec9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec92ffd700 1 --2- 172.21.15.92:0/1592763795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159890 0x7fec9415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec92ffd700 1 -- 172.21.15.92:0/1592763795 >> v1:172.21.15.136:6789/0 conn(0x7fec9415db10 legacy=0x7fec94069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec92ffd700 1 -- 172.21.15.92:0/1592763795 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9415a3d0 con 0x7fec94159c60 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec92ffd700 1 --2- 172.21.15.92:0/1592763795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159c60 0x7fec9415d5d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0099b0 tx=0x7fec7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb28f00cace5c92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec927fc700 1 -- 172.21.15.92:0/1592763795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c009ea0 con 0x7fec94159c60 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec927fc700 1 -- 172.21.15.92:0/1592763795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c00b860 con 0x7fec94159c60 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.169+0000 7fec927fc700 1 -- 172.21.15.92:0/1592763795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00bc00 con 0x7fec94159c60 2023-10-06T03:02:05.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 -- 172.21.15.92:0/1592763795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159c60 msgr2=0x7fec9415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 --2- 172.21.15.92:0/1592763795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159c60 0x7fec9415d5d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0099b0 tx=0x7fec7c009250 comp rx=0 tx=0).stop 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 -- 172.21.15.92:0/1592763795 shutdown_connections 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 --2- 172.21.15.92:0/1592763795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159890 0x7fec9415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 --2- 172.21.15.92:0/1592763795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159c60 0x7fec9415d5d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 -- 172.21.15.92:0/1592763795 >> 172.21.15.92:0/1592763795 conn(0x7fec94068e20 msgr2=0x7fec940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 -- 172.21.15.92:0/1592763795 shutdown_connections 2023-10-06T03:02:05.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.170+0000 7fec99c7a700 1 -- 172.21.15.92:0/1592763795 wait complete. 2023-10-06T03:02:05.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.171+0000 7fec99c7a700 1 Processor -- start 2023-10-06T03:02:05.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.171+0000 7fec99c7a700 1 -- start start 2023-10-06T03:02:05.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 0x7fec940b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159c60 0x7fec9416d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec9415db10 0x7fec9416d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940cfa80 con 0x7fec94159890 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940cee70 con 0x7fec9415db10 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec940cf150 con 0x7fec94159c60 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 0x7fec940b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 0x7fec940b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45330/0 (socket says 172.21.15.92:45330) 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 -- 172.21.15.92:0/3684371571 learned_addr learned my addr 172.21.15.92:0/3684371571 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec92ffd700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159c60 0x7fec9416d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec93fff700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec9415db10 0x7fec9416d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 -- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159c60 msgr2=0x7fec9416d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159c60 0x7fec9416d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 -- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec9415db10 msgr2=0x7fec9416d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec9415db10 0x7fec9416d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 -- 172.21.15.92:0/3684371571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0094f0 con 0x7fec94159890 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec937fe700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 0x7fec940b38c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec84009ef0 tx=0x7fec8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400cb70 con 0x7fec94159890 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8400f040 con 0x7fec94159890 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec940cf430 con 0x7fec94159890 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84014480 con 0x7fec94159890 2023-10-06T03:02:05.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.172+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec940b3e00 con 0x7fec94159890 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.173+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9404c3c0 con 0x7fec94159890 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.174+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec840147f0 con 0x7fec94159890 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.174+0000 7fec8b7fe700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec74066f30 0x7fec740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.174+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec84084fc0 con 0x7fec94159890 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.174+0000 7fec92ffd700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec74066f30 0x7fec740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.174+0000 7fec92ffd700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec74066f30 0x7fec740693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0099b0 tx=0x7fec7c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.178+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec840520d0 con 0x7fec94159890 2023-10-06T03:02:05.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.364+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec9406a5b0 con 0x7fec94159890 2023-10-06T03:02:05.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.364+0000 7fec8b7fe700 1 -- 172.21.15.92:0/3684371571 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fec840556f0 con 0x7fec94159890 2023-10-06T03:02:05.366 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:05.366 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:05.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec74066f30 msgr2=0x7fec740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec74066f30 0x7fec740693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0099b0 tx=0x7fec7c017000 comp rx=0 tx=0).stop 2023-10-06T03:02:05.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 msgr2=0x7fec940b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 0x7fec940b38c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec84009ef0 tx=0x7fec8400b040 comp rx=0 tx=0).stop 2023-10-06T03:02:05.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 shutdown_connections 2023-10-06T03:02:05.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec74066f30 0x7fec740693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec94159890 0x7fec940b38c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec9415db10 0x7fec9416d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.366+0000 7fec99c7a700 1 --2- 172.21.15.92:0/3684371571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec94159c60 0x7fec9416d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.367+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 >> 172.21.15.92:0/3684371571 conn(0x7fec94068e20 msgr2=0x7fec941546f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.367+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 shutdown_connections 2023-10-06T03:02:05.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.367+0000 7fec99c7a700 1 -- 172.21.15.92:0/3684371571 wait complete. 2023-10-06T03:02:05.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:02:05.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:05.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.496+0000 7fb3873b5700 1 Processor -- start 2023-10-06T03:02:05.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.497+0000 7fb3873b5700 1 -- start start 2023-10-06T03:02:05.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.497+0000 7fb3873b5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb380141500 0x7fb3800c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.497+0000 7fb3873b5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38015c2a0 con 0x7fb38006c0a0 2023-10-06T03:02:05.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.497+0000 7fb3873b5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38015b690 con 0x7fb38006bcd0 2023-10-06T03:02:05.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.497+0000 7fb3873b5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38015b970 con 0x7fb380141500 2023-10-06T03:02:05.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb385952700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb38006c0a0 0x7fb3800c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56294/0 (socket says 172.21.15.92:56294) 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb385952700 1 -- 172.21.15.92:0/1281962207 learned_addr learned my addr 172.21.15.92:0/1281962207 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb384950700 1 --2- 172.21.15.92:0/1281962207 >> v2:172.21.15.136:3300/0 conn(0x7fb380141500 0x7fb3800c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb377fff700 1 -- 172.21.15.92:0/1281962207 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016753225 0 0) 0x7fb38015c2a0 con 0x7fb38006c0a0 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb377fff700 1 -- 172.21.15.92:0/1281962207 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb36c003570 con 0x7fb38006c0a0 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb384950700 1 -- 172.21.15.92:0/1281962207 >> v1:172.21.15.136:6789/0 conn(0x7fb38006bcd0 legacy=0x7fb380140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb377fff700 1 -- 172.21.15.92:0/1281962207 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3852361612 0 0) 0x7fb36c003570 con 0x7fb38006c0a0 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb384950700 1 -- 172.21.15.92:0/1281962207 >> v1:172.21.15.92:6789/0 conn(0x7fb38006c0a0 legacy=0x7fb3800c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb384950700 1 -- 172.21.15.92:0/1281962207 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38015bc50 con 0x7fb380141500 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb384950700 1 --2- 172.21.15.92:0/1281962207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb380141500 0x7fb3800c8e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb3700098e0 tx=0x7fb3700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00239f1c44efbcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb377fff700 1 -- 172.21.15.92:0/1281962207 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378001470 con 0x7fb380141500 2023-10-06T03:02:05.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb377fff700 1 -- 172.21.15.92:0/1281962207 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370011420 con 0x7fb380141500 2023-10-06T03:02:05.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.498+0000 7fb377fff700 1 -- 172.21.15.92:0/1281962207 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700117a0 con 0x7fb380141500 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 -- 172.21.15.92:0/1281962207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb380141500 msgr2=0x7fb3800c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 --2- 172.21.15.92:0/1281962207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb380141500 0x7fb3800c8e00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb3700098e0 tx=0x7fb3700091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 -- 172.21.15.92:0/1281962207 shutdown_connections 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 --2- 172.21.15.92:0/1281962207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb380141500 0x7fb3800c8e00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 -- 172.21.15.92:0/1281962207 >> 172.21.15.92:0/1281962207 conn(0x7fb3800ce600 msgr2=0x7fb3800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 -- 172.21.15.92:0/1281962207 shutdown_connections 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.499+0000 7fb3873b5700 1 -- 172.21.15.92:0/1281962207 wait complete. 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.500+0000 7fb3873b5700 1 Processor -- start 2023-10-06T03:02:05.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.500+0000 7fb3873b5700 1 -- start start 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb3873b5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb38006bcd0 0x7fb380157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb3873b5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb38006c0a0 0x7fb3800c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb3873b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 0x7fb3800b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb3873b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800c5e20 con 0x7fb380141500 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb3873b5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb380154ba0 con 0x7fb38006c0a0 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb3873b5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3800c5bf0 con 0x7fb38006bcd0 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 0x7fb3800b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 0x7fb3800b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45338/0 (socket says 172.21.15.92:45338) 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 -- 172.21.15.92:0/3922080897 learned_addr learned my addr 172.21.15.92:0/3922080897 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb384950700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb38006c0a0 0x7fb3800c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385151700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb38006bcd0 0x7fb380157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 -- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb38006bcd0 msgr2=0x7fb380157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb38006bcd0 0x7fb380157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 -- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb38006c0a0 msgr2=0x7fb3800c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb38006c0a0 0x7fb3800c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385952700 1 -- 172.21.15.92:0/3922080897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb370009420 con 0x7fb380141500 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb384950700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb38006c0a0 0x7fb3800c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.501+0000 7fb385151700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb38006bcd0 0x7fb380157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.502+0000 7fb385952700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 0x7fb3800b3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c7c0 tx=0x7fb378001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.502+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378007360 con 0x7fb380141500 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.502+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3780074e0 con 0x7fb380141500 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.502+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3800b1590 con 0x7fb380141500 2023-10-06T03:02:05.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.502+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378004830 con 0x7fb380141500 2023-10-06T03:02:05.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.502+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3800b2b40 con 0x7fb380141500 2023-10-06T03:02:05.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.503+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38004c3c0 con 0x7fb380141500 2023-10-06T03:02:05.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.503+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb378004c60 con 0x7fb380141500 2023-10-06T03:02:05.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.503+0000 7fb3757fa700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb364066f60 0x7fb364069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.503+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb378083fb0 con 0x7fb380141500 2023-10-06T03:02:05.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.503+0000 7fb385151700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb364066f60 0x7fb364069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.504+0000 7fb385151700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb364066f60 0x7fb364069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb37c007fd0 tx=0x7fb37c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.508+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb378051020 con 0x7fb380141500 2023-10-06T03:02:05.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.692+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb38015bf30 con 0x7fb380141500 2023-10-06T03:02:05.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.692+0000 7fb3757fa700 1 -- 172.21.15.92:0/3922080897 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb378018020 con 0x7fb380141500 2023-10-06T03:02:05.693 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:05.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb364066f60 msgr2=0x7fb364069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb364066f60 0x7fb364069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb37c007fd0 tx=0x7fb37c009040 comp rx=0 tx=0).stop 2023-10-06T03:02:05.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 msgr2=0x7fb3800b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 0x7fb3800b3b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c7c0 tx=0x7fb378001a20 comp rx=0 tx=0).stop 2023-10-06T03:02:05.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 shutdown_connections 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb364066f60 0x7fb364069410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb380141500 0x7fb3800b3b30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb38006bcd0 0x7fb380157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 --2- 172.21.15.92:0/3922080897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb38006c0a0 0x7fb3800c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 >> 172.21.15.92:0/3922080897 conn(0x7fb3800ce600 msgr2=0x7fb3800cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 shutdown_connections 2023-10-06T03:02:05.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.694+0000 7fb3873b5700 1 -- 172.21.15.92:0/3922080897 wait complete. 2023-10-06T03:02:05.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:05.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.822+0000 7fbeff59e700 1 Processor -- start 2023-10-06T03:02:05.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.822+0000 7fbeff59e700 1 -- start start 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbeff59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf000b16d0 0x7fbf000b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbeff59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000b2080 con 0x7fbf000b33b0 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbeff59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000cfb00 con 0x7fbf000b4e70 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbeff59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000b5c40 con 0x7fbf000b16d0 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbf000b33b0 0x7fbf000b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56298/0 (socket says 172.21.15.92:56298) 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefed9d700 1 -- 172.21.15.92:0/1910963128 learned_addr learned my addr 172.21.15.92:0/1910963128 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefdd9b700 1 --2- 172.21.15.92:0/1910963128 >> v2:172.21.15.136:3300/0 conn(0x7fbf000b16d0 0x7fbf000b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435400935 0 0) 0x7fbf000b2080 con 0x7fbf000b33b0 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee8003570 con 0x7fbf000b33b0 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426496078 0 0) 0x7fbf000cfb00 con 0x7fbf000b4e70 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf000b2080 con 0x7fbf000b4e70 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefdd9b700 1 -- 172.21.15.92:0/1910963128 >> v1:172.21.15.136:6789/0 conn(0x7fbf000b4e70 legacy=0x7fbf000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefdd9b700 1 -- 172.21.15.92:0/1910963128 >> v1:172.21.15.92:6789/0 conn(0x7fbf000b33b0 legacy=0x7fbf000b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.823+0000 7fbefdd9b700 1 -- 172.21.15.92:0/1910963128 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000b5f20 con 0x7fbf000b16d0 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbefdd9b700 1 --2- 172.21.15.92:0/1910963128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b16d0 0x7fbf000b1b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009e60 tx=0x7fbef4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d477d01bba23e686 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef400e070 con 0x7fbf000b16d0 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef400bab0 con 0x7fbf000b16d0 2023-10-06T03:02:05.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbefd59a700 1 -- 172.21.15.92:0/1910963128 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef400be30 con 0x7fbf000b16d0 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 -- 172.21.15.92:0/1910963128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b16d0 msgr2=0x7fbf000b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1910963128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b16d0 0x7fbf000b1b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009e60 tx=0x7fbef4009780 comp rx=0 tx=0).stop 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 -- 172.21.15.92:0/1910963128 shutdown_connections 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1910963128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b16d0 0x7fbf000b1b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 -- 172.21.15.92:0/1910963128 >> 172.21.15.92:0/1910963128 conn(0x7fbf000d10c0 msgr2=0x7fbf000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 -- 172.21.15.92:0/1910963128 shutdown_connections 2023-10-06T03:02:05.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.824+0000 7fbeff59e700 1 -- 172.21.15.92:0/1910963128 wait complete. 2023-10-06T03:02:05.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.825+0000 7fbeff59e700 1 Processor -- start 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.825+0000 7fbeff59e700 1 -- start start 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbeff59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 0x7fbf000cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbeff59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b33b0 0x7fbf0015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbeff59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf000b4e70 0x7fbf0015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbeff59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf001583c0 con 0x7fbf000b16d0 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbeff59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000cc070 con 0x7fbf000b33b0 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbeff59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000cc350 con 0x7fbf000b4e70 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbefe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 0x7fbf000cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbefe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 0x7fbf000cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45342/0 (socket says 172.21.15.92:45342) 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbefe59c700 1 -- 172.21.15.92:0/1029107033 learned_addr learned my addr 172.21.15.92:0/1029107033 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbefdd9b700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b33b0 0x7fbf0015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.826+0000 7fbefed9d700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf000b4e70 0x7fbf0015fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefe59c700 1 -- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf000b4e70 msgr2=0x7fbf0015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefe59c700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf000b4e70 0x7fbf0015fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefe59c700 1 -- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b33b0 msgr2=0x7fbf0015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefe59c700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b33b0 0x7fbf0015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefe59c700 1 -- 172.21.15.92:0/1029107033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4009a20 con 0x7fbf000b16d0 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefed9d700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf000b4e70 0x7fbf0015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbefe59c700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 0x7fbf000cb9b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbef800c580 tx=0x7fbef8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8010510 con 0x7fbf000b16d0 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef8010670 con 0x7fbf000b16d0 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf000cc630 con 0x7fbf000b16d0 2023-10-06T03:02:05.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8010a30 con 0x7fbf000b16d0 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.827+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0015add0 con 0x7fbf000b16d0 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.828+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0015aff0 con 0x7fbf000b16d0 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.828+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbef801b400 con 0x7fbf000b16d0 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.829+0000 7fbeee7fc700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbee0066f30 0x7fbee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.829+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbef80843b0 con 0x7fbf000b16d0 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.829+0000 7fbefdd9b700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbee0066f30 0x7fbee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:05.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.829+0000 7fbefdd9b700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbee0066f30 0x7fbee00693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbef40099f0 tx=0x7fbef4019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:05.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:05.833+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbef8051420 con 0x7fbf000b16d0 2023-10-06T03:02:06.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.013+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbf0004bb80 con 0x7fbf000b16d0 2023-10-06T03:02:06.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.014+0000 7fbeee7fc700 1 -- 172.21.15.92:0/1029107033 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbef8054a40 con 0x7fbf000b16d0 2023-10-06T03:02:06.015 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbee0066f30 msgr2=0x7fbee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbee0066f30 0x7fbee00693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbef40099f0 tx=0x7fbef4019000 comp rx=0 tx=0).stop 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 msgr2=0x7fbf000cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 0x7fbf000cb9b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbef800c580 tx=0x7fbef8007e10 comp rx=0 tx=0).stop 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 shutdown_connections 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbee0066f30 0x7fbee00693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf000b16d0 0x7fbf000cb9b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf000b4e70 0x7fbf0015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 --2- 172.21.15.92:0/1029107033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf000b33b0 0x7fbf0015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 >> 172.21.15.92:0/1029107033 conn(0x7fbf000d10c0 msgr2=0x7fbf000c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 shutdown_connections 2023-10-06T03:02:06.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.016+0000 7fbeff59e700 1 -- 172.21.15.92:0/1029107033 wait complete. 2023-10-06T03:02:06.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:02:06.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.145+0000 7f7a8d68c700 1 Processor -- start 2023-10-06T03:02:06.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.145+0000 7f7a8d68c700 1 -- start start 2023-10-06T03:02:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8d68c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7a881579c0 0x7f7a880c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8d68c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a8806b300 0x7f7a8806b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8d68c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a880c9310 con 0x7f7a88068390 2023-10-06T03:02:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8d68c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a88157d90 con 0x7f7a881579c0 2023-10-06T03:02:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8d68c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880c8700 con 0x7f7a8806b300 2023-10-06T03:02:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8ce8b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7a88068390 0x7f7a880b00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56314/0 (socket says 172.21.15.92:56314) 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a8ce8b700 1 -- 172.21.15.92:0/2673569069 learned_addr learned my addr 172.21.15.92:0/2673569069 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a877fe700 1 --2- 172.21.15.92:0/2673569069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a8806b300 0x7f7a8806b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a87fff700 1 --2- 172.21.15.92:0/2673569069 >> v2:172.21.15.136:3300/0 conn(0x7f7a881579c0 0x7f7a880c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a86ffd700 1 -- 172.21.15.92:0/2673569069 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107060469 0 0) 0x7f7a880c9310 con 0x7f7a88068390 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a86ffd700 1 -- 172.21.15.92:0/2673569069 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a70003570 con 0x7f7a88068390 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a86ffd700 1 -- 172.21.15.92:0/2673569069 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4145502405 0 0) 0x7f7a70003570 con 0x7f7a88068390 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a877fe700 1 -- 172.21.15.92:0/2673569069 >> v2:172.21.15.136:3300/0 conn(0x7f7a881579c0 msgr2=0x7f7a880c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a877fe700 1 --2- 172.21.15.92:0/2673569069 >> v2:172.21.15.136:3300/0 conn(0x7f7a881579c0 0x7f7a880c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a877fe700 1 -- 172.21.15.92:0/2673569069 >> v1:172.21.15.92:6789/0 conn(0x7f7a88068390 legacy=0x7f7a880b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a877fe700 1 -- 172.21.15.92:0/2673569069 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a880c89e0 con 0x7f7a8806b300 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.146+0000 7f7a87fff700 1 --2- 172.21.15.92:0/2673569069 >> v2:172.21.15.136:3300/0 conn(0x7f7a881579c0 0x7f7a880c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:06.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a877fe700 1 --2- 172.21.15.92:0/2673569069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a8806b300 0x7f7a8806b770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098d0 tx=0x7f7a7c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53345d57d8e85f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a86ffd700 1 -- 172.21.15.92:0/2673569069 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800017f0 con 0x7f7a8806b300 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a86ffd700 1 -- 172.21.15.92:0/2673569069 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c011420 con 0x7f7a8806b300 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a86ffd700 1 -- 172.21.15.92:0/2673569069 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0117e0 con 0x7f7a8806b300 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 -- 172.21.15.92:0/2673569069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a8806b300 msgr2=0x7f7a8806b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/2673569069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a8806b300 0x7f7a8806b770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098d0 tx=0x7f7a7c009120 comp rx=0 tx=0).stop 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 -- 172.21.15.92:0/2673569069 shutdown_connections 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/2673569069 >> v2:172.21.15.136:3300/0 conn(0x7f7a881579c0 0x7f7a880c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/2673569069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a8806b300 0x7f7a8806b770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 -- 172.21.15.92:0/2673569069 >> 172.21.15.92:0/2673569069 conn(0x7f7a880cddc0 msgr2=0x7f7a880c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 -- 172.21.15.92:0/2673569069 shutdown_connections 2023-10-06T03:02:06.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.147+0000 7f7a8d68c700 1 -- 172.21.15.92:0/2673569069 wait complete. 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.148+0000 7f7a8d68c700 1 Processor -- start 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.148+0000 7f7a8d68c700 1 -- start start 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8d68c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a88068390 0x7f7a88143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8d68c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 0x7f7a880c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8d68c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a881579c0 0x7f7a8806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8d68c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88157f50 con 0x7f7a8806b300 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8d68c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881581c0 con 0x7f7a88068390 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8d68c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880c4d80 con 0x7f7a881579c0 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 0x7f7a880c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 0x7f7a880c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45344/0 (socket says 172.21.15.92:45344) 2023-10-06T03:02:06.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a877fe700 1 -- 172.21.15.92:0/3179911066 learned_addr learned my addr 172.21.15.92:0/3179911066 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a87fff700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a88068390 0x7f7a88143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.149+0000 7f7a8ce8b700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a881579c0 0x7f7a8806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a877fe700 1 -- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a881579c0 msgr2=0x7f7a8806e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a877fe700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a881579c0 0x7f7a8806e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a877fe700 1 -- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a88068390 msgr2=0x7f7a88143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a877fe700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a88068390 0x7f7a88143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a877fe700 1 -- 172.21.15.92:0/3179911066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78009420 con 0x7f7a8806b300 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a877fe700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 0x7f7a880c43a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a880c8840 tx=0x7f7a7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01cc50 con 0x7f7a8806b300 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c01cdd0 con 0x7f7a8806b300 2023-10-06T03:02:06.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c0093c0 con 0x7f7a8806b300 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01a6a0 con 0x7f7a8806b300 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.150+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880c5340 con 0x7f7a8806b300 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.151+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8806d560 con 0x7f7a8806b300 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.151+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a7c01aa90 con 0x7f7a8806b300 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.151+0000 7f7a6ffff700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a74066f30 0x7f7a740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.152+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7a7c085f10 con 0x7f7a8806b300 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.152+0000 7f7a87fff700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a74066f30 0x7f7a740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.152+0000 7f7a87fff700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a74066f30 0x7f7a740693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a780093f0 tx=0x7f7a78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.156+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a7c053020 con 0x7f7a8806b300 2023-10-06T03:02:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.341+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a88158300 con 0x7f7a8806b300 2023-10-06T03:02:06.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.341+0000 7f7a6ffff700 1 -- 172.21.15.92:0/3179911066 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a7c056640 con 0x7f7a8806b300 2023-10-06T03:02:06.342 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:02:06.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a74066f30 msgr2=0x7f7a740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a74066f30 0x7f7a740693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7a780093f0 tx=0x7f7a78008040 comp rx=0 tx=0).stop 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 msgr2=0x7f7a880c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 0x7f7a880c43a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a880c8840 tx=0x7f7a7c016000 comp rx=0 tx=0).stop 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 shutdown_connections 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a74066f30 0x7f7a740693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a881579c0 0x7f7a8806e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a88068390 0x7f7a88143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 --2- 172.21.15.92:0/3179911066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a8806b300 0x7f7a880c43a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 >> 172.21.15.92:0/3179911066 conn(0x7f7a880cddc0 msgr2=0x7f7a880704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 shutdown_connections 2023-10-06T03:02:06.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.343+0000 7f7a8d68c700 1 -- 172.21.15.92:0/3179911066 wait complete. 2023-10-06T03:02:06.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:02:06.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.473+0000 7fd488c97700 1 Processor -- start 2023-10-06T03:02:06.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.473+0000 7fd488c97700 1 -- start start 2023-10-06T03:02:06.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd488c97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4840b4e70 0x7fd4840ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd488c97700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b1b40 0x7fd48415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd488c97700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4840cfb00 con 0x7fd4840b16d0 2023-10-06T03:02:06.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd488c97700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4840b5c40 con 0x7fd4840b4e70 2023-10-06T03:02:06.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd488c97700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840b5f20 con 0x7fd4840b1b40 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4840b4e70 0x7fd4840ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4840b4e70 0x7fd4840ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45354/0 (socket says 172.21.15.92:45354) 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 -- 172.21.15.92:0/2840726166 learned_addr learned my addr 172.21.15.92:0/2840726166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 -- 172.21.15.92:0/2840726166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b1b40 msgr2=0x7fd48415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd483fff700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b1b40 0x7fd48415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b1b40 0x7fd48415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 -- 172.21.15.92:0/2840726166 >> v1:172.21.15.136:6789/0 conn(0x7fd4840b16d0 legacy=0x7fd48415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd4837fe700 1 -- 172.21.15.92:0/2840726166 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840b6200 con 0x7fd4840b4e70 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.474+0000 7fd483fff700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b1b40 0x7fd48415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd4837fe700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd4840ccff0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd4740099b0 tx=0x7fd474009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e202fb7c5b90dfcf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd4827fc700 1 -- 172.21.15.92:0/2840726166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47400b820 con 0x7fd4840b4e70 2023-10-06T03:02:06.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd4827fc700 1 -- 172.21.15.92:0/2840726166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47400b980 con 0x7fd4840b4e70 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd4827fc700 1 -- 172.21.15.92:0/2840726166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47400bd20 con 0x7fd4840b4e70 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 -- 172.21.15.92:0/2840726166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 msgr2=0x7fd4840ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd4840ccff0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd4740099b0 tx=0x7fd474009250 comp rx=0 tx=0).stop 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 -- 172.21.15.92:0/2840726166 shutdown_connections 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd4840ccff0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 --2- 172.21.15.92:0/2840726166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b1b40 0x7fd48415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 -- 172.21.15.92:0/2840726166 >> 172.21.15.92:0/2840726166 conn(0x7fd4840d10c0 msgr2=0x7fd4840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 -- 172.21.15.92:0/2840726166 shutdown_connections 2023-10-06T03:02:06.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.475+0000 7fd488c97700 1 -- 172.21.15.92:0/2840726166 wait complete. 2023-10-06T03:02:06.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 Processor -- start 2023-10-06T03:02:06.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 -- start start 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b16d0 0x7fd4840c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4840b1b40 0x7fd4840c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd484169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48416aa90 con 0x7fd4840b4e70 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd484157c30 con 0x7fd4840b1b40 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.477+0000 7fd488c97700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840cc1a0 con 0x7fd4840b16d0 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd484169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd484169880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45358/0 (socket says 172.21.15.92:45358) 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 -- 172.21.15.92:0/2995008494 learned_addr learned my addr 172.21.15.92:0/2995008494 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd4837fe700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b16d0 0x7fd4840c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd482ffd700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4840b1b40 0x7fd4840c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 -- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b16d0 msgr2=0x7fd4840c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b16d0 0x7fd4840c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 -- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4840b1b40 msgr2=0x7fd4840c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4840b1b40 0x7fd4840c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 -- 172.21.15.92:0/2995008494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740094f0 con 0x7fd4840b4e70 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd483fff700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd484169880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00a380 tx=0x7fd47c007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c015440 con 0x7fd4840b4e70 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c0155a0 con 0x7fd4840b4e70 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48416b010 con 0x7fd4840b4e70 2023-10-06T03:02:06.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c015960 con 0x7fd4840b4e70 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.478+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4840c8010 con 0x7fd4840b4e70 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.479+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484088990 con 0x7fd4840b4e70 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.480+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd47c00d440 con 0x7fd4840b4e70 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.480+0000 7fd46b7fe700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470066f30 0x7fd4700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.480+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd47c085170 con 0x7fd4840b4e70 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.480+0000 7fd4837fe700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470066f30 0x7fd4700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.480+0000 7fd4837fe700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470066f30 0x7fd4700693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd4740099b0 tx=0x7fd474016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.484+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd47c052280 con 0x7fd4840b4e70 2023-10-06T03:02:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.666+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4840cf4b0 con 0x7fd4840b4e70 2023-10-06T03:02:06.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.667+0000 7fd46b7fe700 1 -- 172.21.15.92:0/2995008494 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd47c0558a0 con 0x7fd4840b4e70 2023-10-06T03:02:06.668 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470066f30 msgr2=0x7fd4700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470066f30 0x7fd4700693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd4740099b0 tx=0x7fd474016000 comp rx=0 tx=0).stop 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 msgr2=0x7fd484169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd484169880 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00a380 tx=0x7fd47c007900 comp rx=0 tx=0).stop 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 shutdown_connections 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470066f30 0x7fd4700693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4840b4e70 0x7fd484169880 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4840b16d0 0x7fd4840c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 --2- 172.21.15.92:0/2995008494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4840b1b40 0x7fd4840c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 >> 172.21.15.92:0/2995008494 conn(0x7fd4840d10c0 msgr2=0x7fd484159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 shutdown_connections 2023-10-06T03:02:06.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.669+0000 7fd488c97700 1 -- 172.21.15.92:0/2995008494 wait complete. 2023-10-06T03:02:06.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:06.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.795+0000 7f89ed315700 1 Processor -- start 2023-10-06T03:02:06.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.795+0000 7f89ed315700 1 -- start start 2023-10-06T03:02:06.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.796+0000 7f89ed315700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f89e806bcd0 0x7f89e80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.796+0000 7f89ed315700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e815d7e0 con 0x7f89e806c0a0 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.796+0000 7f89ed315700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e815dac0 con 0x7f89e815df50 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.796+0000 7f89ed315700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e815aa30 con 0x7f89e806bcd0 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.796+0000 7f89e67fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f89e806c0a0 0x7f89e815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56324/0 (socket says 172.21.15.92:56324) 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.796+0000 7f89e67fc700 1 -- 172.21.15.92:0/1474881789 learned_addr learned my addr 172.21.15.92:0/1474881789 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e6ffd700 1 --2- 172.21.15.92:0/1474881789 >> v2:172.21.15.136:3300/0 conn(0x7f89e806bcd0 0x7f89e80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195145709 0 0) 0x7f89e815d7e0 con 0x7f89e806c0a0 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d0003570 con 0x7f89e806c0a0 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884452526 0 0) 0x7f89d0003570 con 0x7f89e806c0a0 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 >> v2:172.21.15.136:3300/0 conn(0x7f89e806bcd0 msgr2=0x7f89e80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:06.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 --2- 172.21.15.92:0/1474881789 >> v2:172.21.15.136:3300/0 conn(0x7f89e806bcd0 0x7f89e80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 >> v1:172.21.15.136:6789/0 conn(0x7f89e815df50 legacy=0x7f89e80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e815ad10 con 0x7f89e806c0a0 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f89dc001470 con 0x7f89e806c0a0 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 >> v1:172.21.15.92:6789/0 conn(0x7f89e806c0a0 legacy=0x7f89e815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0003a60 0x7f89d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d0004810 0x7f89d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89d00052e0 0x7f89d000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e815aa30 con 0x7f89d0004810 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d000c2a0 con 0x7f89d0003a60 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d000c460 con 0x7f89d00052e0 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d0004810 0x7f89d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e6ffd700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0003a60 0x7f89d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e77fe700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89d00052e0 0x7f89d000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 -- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89d00052e0 msgr2=0x7f89d000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89d00052e0 0x7f89d000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 -- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0003a60 msgr2=0x7f89d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0003a60 0x7f89d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 -- 172.21.15.92:0/1474881789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0003570 con 0x7f89d0004810 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e77fe700 1 -- 172.21.15.92:0/1474881789 reap_dead start 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.797+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d0004810 0x7f89d0004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c390 tx=0x7f89e815da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1684bd0c6f5760 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00cc60 con 0x7f89d0004810 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00cdc0 con 0x7f89d0004810 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89e5ffb700 1 -- 172.21.15.92:0/1474881789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc005b70 con 0x7f89d0004810 2023-10-06T03:02:06.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 -- 172.21.15.92:0/1474881789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d0004810 msgr2=0x7f89d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d0004810 0x7f89d0004c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c390 tx=0x7f89e815da30 comp rx=0 tx=0).stop 2023-10-06T03:02:06.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 -- 172.21.15.92:0/1474881789 shutdown_connections 2023-10-06T03:02:06.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 --2- 172.21.15.92:0/1474881789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89d0004810 0x7f89d0004c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 -- 172.21.15.92:0/1474881789 >> 172.21.15.92:0/1474881789 conn(0x7f89e80ce600 msgr2=0x7f89e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 -- 172.21.15.92:0/1474881789 shutdown_connections 2023-10-06T03:02:06.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.798+0000 7f89ed315700 1 -- 172.21.15.92:0/1474881789 wait complete. 2023-10-06T03:02:06.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.799+0000 7f89ed315700 1 Processor -- start 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.799+0000 7f89ed315700 1 -- start start 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89ed315700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0004810 0x7f89e80c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89ed315700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 0x7f89e80b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89ed315700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89e8143180 0x7f89e80b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89ed315700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c59e0 con 0x7f89e80c9700 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89ed315700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c8d30 con 0x7f89d0004810 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89ed315700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c8120 con 0x7f89e8143180 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89e67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 0x7f89e80b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89e67fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 0x7f89e80b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45388/0 (socket says 172.21.15.92:45388) 2023-10-06T03:02:06.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89e67fc700 1 -- 172.21.15.92:0/1714674439 learned_addr learned my addr 172.21.15.92:0/1714674439 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89e6ffd700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0004810 0x7f89e80c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.800+0000 7f89e77fe700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89e8143180 0x7f89e80b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e67fc700 1 -- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89e8143180 msgr2=0x7f89e80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89e8143180 0x7f89e80b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e67fc700 1 -- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0004810 msgr2=0x7f89e80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0004810 0x7f89e80c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e67fc700 1 -- 172.21.15.92:0/1714674439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80c8400 con 0x7f89e80c9700 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e6ffd700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0004810 0x7f89e80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89e67fc700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 0x7f89e80b19a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89d000c270 tx=0x7f89dc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc022070 con 0x7f89e80c9700 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc0070d0 con 0x7f89e80c9700 2023-10-06T03:02:06.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc007470 con 0x7f89e80c9700 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e80c86e0 con 0x7f89e80c9700 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.801+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e80c7440 con 0x7f89e80c9700 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.802+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e80c8ee0 con 0x7f89e80c9700 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.802+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89dc01e430 con 0x7f89e80c9700 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.803+0000 7f89ceffd700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f89d4066f30 0x7f89d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.803+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f89dc085b20 con 0x7f89e80c9700 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.803+0000 7f89e6ffd700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f89d4066f30 0x7f89d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:06.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.803+0000 7f89e6ffd700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f89d4066f30 0x7f89d40693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f89d80094d0 tx=0x7f89d800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:06.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.807+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89e80c8ee0 con 0x7f89e80c9700 2023-10-06T03:02:06.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.990+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e80c8ea0 con 0x7f89e80c9700 2023-10-06T03:02:06.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.991+0000 7f89ceffd700 1 -- 172.21.15.92:0/1714674439 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f89dc017020 con 0x7f89e80c9700 2023-10-06T03:02:06.992 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:06.992 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:06.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f89d4066f30 msgr2=0x7f89d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f89d4066f30 0x7f89d40693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f89d80094d0 tx=0x7f89d800b040 comp rx=0 tx=0).stop 2023-10-06T03:02:06.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 msgr2=0x7f89e80b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 0x7f89e80b19a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89d000c270 tx=0x7f89dc015000 comp rx=0 tx=0).stop 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 shutdown_connections 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f89d4066f30 0x7f89d40693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f89e8143180 0x7f89e80b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89d0004810 0x7f89e80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 --2- 172.21.15.92:0/1714674439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89e80c9700 0x7f89e80b19a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 >> 172.21.15.92:0/1714674439 conn(0x7f89e80ce600 msgr2=0x7f89e8155150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 shutdown_connections 2023-10-06T03:02:06.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:06.993+0000 7f89ed315700 1 -- 172.21.15.92:0/1714674439 wait complete. 2023-10-06T03:02:07.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:02:07.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:07.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.123+0000 7f5fda8aa700 1 Processor -- start 2023-10-06T03:02:07.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.123+0000 7f5fda8aa700 1 -- start start 2023-10-06T03:02:07.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda8aa700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fd40b4e70 0x7f5fd40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda8aa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b1b40 0x7f5fd415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda8aa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd40cfb00 con 0x7f5fd40b16d0 2023-10-06T03:02:07.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda8aa700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd40b5c40 con 0x7f5fd40b4e70 2023-10-06T03:02:07.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda8aa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40b5f20 con 0x7f5fd40b1b40 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fd90a7700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5fd40b16d0 0x7f5fd415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56330/0 (socket says 172.21.15.92:56330) 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fd90a7700 1 -- 172.21.15.92:0/2175338992 learned_addr learned my addr 172.21.15.92:0/2175338992 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda0a9700 1 --2- 172.21.15.92:0/2175338992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b1b40 0x7f5fd415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fd98a8700 1 --2- 172.21.15.92:0/2175338992 >> v2:172.21.15.136:3300/0 conn(0x7f5fd40b4e70 0x7f5fd40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fd88a6700 1 -- 172.21.15.92:0/2175338992 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863523531 0 0) 0x7f5fd40cfb00 con 0x7f5fd40b16d0 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fd88a6700 1 -- 172.21.15.92:0/2175338992 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc0003570 con 0x7f5fd40b16d0 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda0a9700 1 -- 172.21.15.92:0/2175338992 >> v2:172.21.15.136:3300/0 conn(0x7f5fd40b4e70 msgr2=0x7f5fd40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda0a9700 1 --2- 172.21.15.92:0/2175338992 >> v2:172.21.15.136:3300/0 conn(0x7f5fd40b4e70 0x7f5fd40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda0a9700 1 -- 172.21.15.92:0/2175338992 >> v1:172.21.15.92:6789/0 conn(0x7f5fd40b16d0 legacy=0x7f5fd415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda0a9700 1 -- 172.21.15.92:0/2175338992 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40b6200 con 0x7f5fd40b1b40 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fd98a8700 1 --2- 172.21.15.92:0/2175338992 >> v2:172.21.15.136:3300/0 conn(0x7f5fd40b4e70 0x7f5fd40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.124+0000 7f5fda0a9700 1 --2- 172.21.15.92:0/2175338992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b1b40 0x7f5fd415d950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009e60 tx=0x7f5fcc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691cc9c59d9417a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fd88a6700 1 -- 172.21.15.92:0/2175338992 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00e070 con 0x7f5fd40b1b40 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fd88a6700 1 -- 172.21.15.92:0/2175338992 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc00bab0 con 0x7f5fd40b1b40 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fd88a6700 1 -- 172.21.15.92:0/2175338992 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00be70 con 0x7f5fd40b1b40 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 -- 172.21.15.92:0/2175338992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b1b40 msgr2=0x7f5fd415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/2175338992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b1b40 0x7f5fd415d950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009e60 tx=0x7f5fcc009780 comp rx=0 tx=0).stop 2023-10-06T03:02:07.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 -- 172.21.15.92:0/2175338992 shutdown_connections 2023-10-06T03:02:07.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/2175338992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b1b40 0x7f5fd415d950 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/2175338992 >> v2:172.21.15.136:3300/0 conn(0x7f5fd40b4e70 0x7f5fd40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 -- 172.21.15.92:0/2175338992 >> 172.21.15.92:0/2175338992 conn(0x7f5fd40d10c0 msgr2=0x7f5fd40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:07.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 -- 172.21.15.92:0/2175338992 shutdown_connections 2023-10-06T03:02:07.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.125+0000 7f5fda8aa700 1 -- 172.21.15.92:0/2175338992 wait complete. 2023-10-06T03:02:07.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.126+0000 7f5fda8aa700 1 Processor -- start 2023-10-06T03:02:07.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.126+0000 7f5fda8aa700 1 -- start start 2023-10-06T03:02:07.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda8aa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b16d0 0x7f5fd4068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda8aa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd40b1b40 0x7f5fd40688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda8aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 0x7f5fd416de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda8aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd416ee20 con 0x7f5fd40b4e70 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda8aa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd416f7a0 con 0x7f5fd40b1b40 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda8aa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd416eb40 con 0x7f5fd40b16d0 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 0x7f5fd416de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 0x7f5fd416de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45396/0 (socket says 172.21.15.92:45396) 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 -- 172.21.15.92:0/3483047250 learned_addr learned my addr 172.21.15.92:0/3483047250 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fd98a8700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b16d0 0x7f5fd4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fd90a7700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd40b1b40 0x7f5fd40688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 -- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b16d0 msgr2=0x7f5fd4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b16d0 0x7f5fd4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 -- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd40b1b40 msgr2=0x7f5fd40688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd40b1b40 0x7f5fd40688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fda0a9700 1 -- 172.21.15.92:0/3483047250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0009420 con 0x7f5fd40b4e70 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.127+0000 7f5fd98a8700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b16d0 0x7f5fd4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fd90a7700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd40b1b40 0x7f5fd40688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fda0a9700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 0x7f5fd416de10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40b6060 tx=0x7f5fcc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc027470 con 0x7f5fd40b4e70 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc0275f0 con 0x7f5fd40b4e70 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc009a20 con 0x7f5fd40b4e70 2023-10-06T03:02:07.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0279b0 con 0x7f5fd40b4e70 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.128+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd416f430 con 0x7f5fd40b4e70 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.129+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd40ccbe0 con 0x7f5fd40b4e70 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.129+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fcc024070 con 0x7f5fd40b4e70 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.130+0000 7f5fc9ffb700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fb8066f30 0x7f5fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.130+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5fcc086fe0 con 0x7f5fd40b4e70 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.130+0000 7f5fd98a8700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fb8066f30 0x7f5fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.130+0000 7f5fd98a8700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fb8066f30 0x7f5fb80693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00093f0 tx=0x7f5fd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.134+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fcc054050 con 0x7f5fd40b4e70 2023-10-06T03:02:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.317+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fd404bb80 con 0x7f5fd40b4e70 2023-10-06T03:02:07.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.317+0000 7f5fc9ffb700 1 -- 172.21.15.92:0/3483047250 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fcc057670 con 0x7f5fd40b4e70 2023-10-06T03:02:07.318 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fb8066f30 msgr2=0x7f5fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fb8066f30 0x7f5fb80693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00093f0 tx=0x7f5fd0008040 comp rx=0 tx=0).stop 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 msgr2=0x7f5fd416de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 0x7f5fd416de10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40b6060 tx=0x7f5fcc019000 comp rx=0 tx=0).stop 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 shutdown_connections 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fb8066f30 0x7f5fb80693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fd40b4e70 0x7f5fd416de10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd40b16d0 0x7f5fd4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 --2- 172.21.15.92:0/3483047250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd40b1b40 0x7f5fd40688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 >> 172.21.15.92:0/3483047250 conn(0x7f5fd40d10c0 msgr2=0x7f5fd40b54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 shutdown_connections 2023-10-06T03:02:07.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.319+0000 7f5fda8aa700 1 -- 172.21.15.92:0/3483047250 wait complete. 2023-10-06T03:02:07.331 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.448+0000 7fc204172700 1 Processor -- start 2023-10-06T03:02:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.448+0000 7fc204172700 1 -- start start 2023-10-06T03:02:07.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc204172700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc1fc159890 0x7fc1fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc204172700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 0x7fc1fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc204172700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc15a160 con 0x7fc1fc15db10 2023-10-06T03:02:07.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc204172700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d13a0 con 0x7fc1fc159890 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc204172700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc15e140 con 0x7fc1fc159c60 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc20270f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc1fc15db10 0x7fc1fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56332/0 (socket says 172.21.15.92:56332) 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc20270f700 1 -- 172.21.15.92:0/4129423857 learned_addr learned my addr 172.21.15.92:0/4129423857 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc201f0e700 1 --2- 172.21.15.92:0/4129423857 >> v2:172.21.15.136:3300/0 conn(0x7fc1fc159890 0x7fc1fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083430110 0 0) 0x7fc1fc15a160 con 0x7fc1fc15db10 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.449+0000 7fc20170d700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 0x7fc1fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e8003570 con 0x7fc1fc15db10 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109513584 0 0) 0x7fc1e8003570 con 0x7fc1fc15db10 2023-10-06T03:02:07.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 msgr2=0x7fc1fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 0x7fc1fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 >> v2:172.21.15.136:3300/0 conn(0x7fc1fc159890 msgr2=0x7fc1fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 --2- 172.21.15.92:0/4129423857 >> v2:172.21.15.136:3300/0 conn(0x7fc1fc159890 0x7fc1fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc15a3d0 con 0x7fc1fc15db10 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc1f4001470 con 0x7fc1fc15db10 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 >> v1:172.21.15.92:6789/0 conn(0x7fc1fc15db10 legacy=0x7fc1fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 0x7fc1e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1e8004d50 0x7fc1e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d13a0 con 0x7fc1e8003a60 2023-10-06T03:02:07.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc15e140 con 0x7fc1e8004d50 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800ba60 con 0x7fc1fc159c60 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc20170d700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 0x7fc1e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc20270f700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1e8004d50 0x7fc1e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 -- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 msgr2=0x7fc1e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1fc159c60 0x7fc1e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 -- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1e8004d50 msgr2=0x7fc1e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1e8004d50 0x7fc1e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 -- 172.21.15.92:0/4129423857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8003570 con 0x7fc1e8003a60 2023-10-06T03:02:07.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc201f0e700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800e210 tx=0x7fc1f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb80f587d112561 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800ea60 con 0x7fc1e8003a60 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f800f040 con 0x7fc1e8003a60 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.450+0000 7fc200f0c700 1 -- 172.21.15.92:0/4129423857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800ee10 con 0x7fc1e8003a60 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc204172700 1 -- 172.21.15.92:0/4129423857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1e8003a60 msgr2=0x7fc1e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc204172700 1 --2- 172.21.15.92:0/4129423857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1e8003a60 0x7fc1e8003e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800e210 tx=0x7fc1f800b040 comp rx=0 tx=0).stop 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc20270f700 1 -- 172.21.15.92:0/4129423857 reap_dead start 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc204172700 1 -- 172.21.15.92:0/4129423857 shutdown_connections 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc204172700 1 -- 172.21.15.92:0/4129423857 >> 172.21.15.92:0/4129423857 conn(0x7fc1fc068e20 msgr2=0x7fc1fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc204172700 1 -- 172.21.15.92:0/4129423857 shutdown_connections 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.451+0000 7fc204172700 1 -- 172.21.15.92:0/4129423857 wait complete. 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.452+0000 7fc204172700 1 Processor -- start 2023-10-06T03:02:07.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.452+0000 7fc204172700 1 -- start start 2023-10-06T03:02:07.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc204172700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc204172700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1fc0ce130 0x7fc1fc16cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc204172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 0x7fc1fc0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc204172700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc16d920 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc204172700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b4840 con 0x7fc1fc0ce130 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc204172700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b3f50 con 0x7fc1e8003a60 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 0x7fc1fc0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 0x7fc1fc0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45414/0 (socket says 172.21.15.92:45414) 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 -- 172.21.15.92:0/3301444541 learned_addr learned my addr 172.21.15.92:0/3301444541 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20170d700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1fc0ce130 0x7fc1fc16cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc201f0e700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 -- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1e8003a60 msgr2=0x7fc1fc0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 -- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1fc0ce130 msgr2=0x7fc1fc16cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1fc0ce130 0x7fc1fc16cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 -- 172.21.15.92:0/3301444541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0ce5a0 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20170d700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1fc0ce130 0x7fc1fc16cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.453+0000 7fc20270f700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 0x7fc1fc0ce7e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000fd0 tx=0x7fc1f4001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.454+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400ee70 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.454+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f400c7b0 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.454+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc0b62c0 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.454+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400cb70 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.454+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc0b44d0 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.455+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc0b4690 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.455+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1f401d460 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.456+0000 7fc1f1ffb700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e0066f30 0x7fc1e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.456+0000 7fc201f0e700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e0066f30 0x7fc1e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.456+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1f4084580 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.456+0000 7fc201f0e700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e0066f30 0x7fc1e00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0cd280 tx=0x7fc1f801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.460+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1f40515f0 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.646+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc1fc0b49b0 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.647+0000 7fc1f1ffb700 1 -- 172.21.15.92:0/3301444541 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc1f4054c10 con 0x7fc1fc16d4b0 2023-10-06T03:02:07.648 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e0066f30 msgr2=0x7fc1e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e0066f30 0x7fc1e00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0cd280 tx=0x7fc1f801a000 comp rx=0 tx=0).stop 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 msgr2=0x7fc1fc0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 0x7fc1fc0ce7e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4000fd0 tx=0x7fc1f4001510 comp rx=0 tx=0).stop 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 shutdown_connections 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e0066f30 0x7fc1e00693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1fc16d4b0 0x7fc1fc0ce7e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1e8003a60 0x7fc1fc0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 --2- 172.21.15.92:0/3301444541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1fc0ce130 0x7fc1fc16cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 >> 172.21.15.92:0/3301444541 conn(0x7fc1fc068e20 msgr2=0x7fc1fc15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 shutdown_connections 2023-10-06T03:02:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.650+0000 7fc204172700 1 -- 172.21.15.92:0/3301444541 wait complete. 2023-10-06T03:02:07.663 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:02:07.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.779+0000 7f6c58931700 1 Processor -- start 2023-10-06T03:02:07.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.779+0000 7f6c58931700 1 -- start start 2023-10-06T03:02:07.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c58931700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c540ccc00 0x7f6c54069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c58931700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c540c9f30 con 0x7f6c54159a60 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c58931700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c540ca470 con 0x7f6c5406a020 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c58931700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5406dfc0 con 0x7f6c540ccc00 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c52ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c540ccc00 0x7f6c54069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c537fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c54159a60 0x7f6c540c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56338/0 (socket says 172.21.15.92:56338) 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.780+0000 7f6c537fe700 1 -- 172.21.15.92:0/673726943 learned_addr learned my addr 172.21.15.92:0/673726943 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c52ffd700 1 -- 172.21.15.92:0/673726943 >> v1:172.21.15.136:6789/0 conn(0x7f6c5406a020 legacy=0x7f6c540cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c52ffd700 1 -- 172.21.15.92:0/673726943 >> v1:172.21.15.92:6789/0 conn(0x7f6c54159a60 legacy=0x7f6c540c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c52ffd700 1 -- 172.21.15.92:0/673726943 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540ca750 con 0x7f6c540ccc00 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c527fc700 1 -- 172.21.15.92:0/673726943 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473420751 0 0) 0x7f6c540c9f30 con 0x7f6c54159a60 2023-10-06T03:02:07.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c52ffd700 1 --2- 172.21.15.92:0/673726943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540ccc00 0x7f6c54069ae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c480098e0 tx=0x7f6c480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac6f9adef4a9031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c527fc700 1 -- 172.21.15.92:0/673726943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48009e80 con 0x7f6c540ccc00 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c527fc700 1 -- 172.21.15.92:0/673726943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4800b8b0 con 0x7f6c540ccc00 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c527fc700 1 -- 172.21.15.92:0/673726943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800bc30 con 0x7f6c540ccc00 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 -- 172.21.15.92:0/673726943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540ccc00 msgr2=0x7f6c54069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 --2- 172.21.15.92:0/673726943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540ccc00 0x7f6c54069ae0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c480098e0 tx=0x7f6c480091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 -- 172.21.15.92:0/673726943 shutdown_connections 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 --2- 172.21.15.92:0/673726943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c540ccc00 0x7f6c54069ae0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 -- 172.21.15.92:0/673726943 >> 172.21.15.92:0/673726943 conn(0x7f6c5406baa0 msgr2=0x7f6c5406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 -- 172.21.15.92:0/673726943 shutdown_connections 2023-10-06T03:02:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.781+0000 7f6c58931700 1 -- 172.21.15.92:0/673726943 wait complete. 2023-10-06T03:02:07.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 Processor -- start 2023-10-06T03:02:07.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 -- start start 2023-10-06T03:02:07.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 0x7f6c540c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c540ccc00 0x7f6c54152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c54159a60 0x7f6c540c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540c5e40 con 0x7f6c5406a020 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540b0f70 con 0x7f6c54159a60 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c58931700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540b25e0 con 0x7f6c540ccc00 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 0x7f6c540c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c537fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 0x7f6c540c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45424/0 (socket says 172.21.15.92:45424) 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.783+0000 7f6c537fe700 1 -- 172.21.15.92:0/651456034 learned_addr learned my addr 172.21.15.92:0/651456034 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c537fe700 1 -- 172.21.15.92:0/651456034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c540ccc00 msgr2=0x7f6c54152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c537fe700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c540ccc00 0x7f6c54152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c537fe700 1 -- 172.21.15.92:0/651456034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c54159a60 msgr2=0x7f6c540c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c537fe700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c54159a60 0x7f6c540c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c537fe700 1 -- 172.21.15.92:0/651456034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009420 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c537fe700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 0x7f6c540c5740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400cc70 tx=0x7f6c44001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400ce80 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4400ab70 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c540b28c0 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440070f0 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.784+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c540b2e80 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.785+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c540b30a0 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.785+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c440074e0 con 0x7f6c5406a020 2023-10-06T03:02:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.786+0000 7f6c3b7fe700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c40066f30 0x7f6c400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:07.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.786+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6c44084b30 con 0x7f6c5406a020 2023-10-06T03:02:07.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.786+0000 7f6c52ffd700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c40066f30 0x7f6c400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:07.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.786+0000 7f6c52ffd700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c40066f30 0x7f6c400693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c540cc3e0 tx=0x7f6c48016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:07.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.790+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c44051cc0 con 0x7f6c5406a020 2023-10-06T03:02:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.970+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c540caa30 con 0x7f6c5406a020 2023-10-06T03:02:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.970+0000 7f6c3b7fe700 1 -- 172.21.15.92:0/651456034 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c440552e0 con 0x7f6c5406a020 2023-10-06T03:02:07.971 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:02:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c40066f30 msgr2=0x7f6c400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c40066f30 0x7f6c400693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6c540cc3e0 tx=0x7f6c48016000 comp rx=0 tx=0).stop 2023-10-06T03:02:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 msgr2=0x7f6c540c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 0x7f6c540c5740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400cc70 tx=0x7f6c44001a30 comp rx=0 tx=0).stop 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 shutdown_connections 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c40066f30 0x7f6c400693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c5406a020 0x7f6c540c5740 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c54159a60 0x7f6c540c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 --2- 172.21.15.92:0/651456034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c540ccc00 0x7f6c54152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 >> 172.21.15.92:0/651456034 conn(0x7f6c5406baa0 msgr2=0x7f6c54159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 shutdown_connections 2023-10-06T03:02:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:07.972+0000 7f6c58931700 1 -- 172.21.15.92:0/651456034 wait complete. 2023-10-06T03:02:07.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:02:08.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.101+0000 7f68ed9d7700 1 Processor -- start 2023-10-06T03:02:08.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.101+0000 7f68ed9d7700 1 -- start start 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68ed9d7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806bcd0 0x7f68e8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68ed9d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68e806c0a0 0x7f68e8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68ed9d7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f68e80c8e00 0x7f68e80c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68ed9d7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e815c2a0 con 0x7f68e806c0a0 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68ed9d7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e815b690 con 0x7f68e80c8e00 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68ed9d7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e815b970 con 0x7f68e806bcd0 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68e806c0a0 0x7f68e8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68e806c0a0 0x7f68e8141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45432/0 (socket says 172.21.15.92:45432) 2023-10-06T03:02:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 -- 172.21.15.92:0/885961187 learned_addr learned my addr 172.21.15.92:0/885961187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 -- 172.21.15.92:0/885961187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806bcd0 msgr2=0x7f68e8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e6ffd700 1 --2- 172.21.15.92:0/885961187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806bcd0 0x7f68e8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 --2- 172.21.15.92:0/885961187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806bcd0 0x7f68e8140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 -- 172.21.15.92:0/885961187 >> v2:172.21.15.136:3300/0 conn(0x7f68e80c8e00 msgr2=0x7f68e80c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e77fe700 1 --2- 172.21.15.92:0/885961187 >> v2:172.21.15.136:3300/0 conn(0x7f68e80c8e00 0x7f68e80c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.102+0000 7f68e67fc700 1 --2- 172.21.15.92:0/885961187 >> v2:172.21.15.136:3300/0 conn(0x7f68e80c8e00 0x7f68e80c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68e67fc700 1 -- 172.21.15.92:0/885961187 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e815bc50 con 0x7f68e806c0a0 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68e67fc700 1 --2- 172.21.15.92:0/885961187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e806c0a0 0x7f68e8141330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f68e815b530 tx=0x7f68d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9fa9f0102050bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68e5ffb700 1 -- 172.21.15.92:0/885961187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000b860 con 0x7f68e806c0a0 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68e5ffb700 1 -- 172.21.15.92:0/885961187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d000b9c0 con 0x7f68e806c0a0 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68e5ffb700 1 -- 172.21.15.92:0/885961187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000bd60 con 0x7f68e806c0a0 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 -- 172.21.15.92:0/885961187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e806c0a0 msgr2=0x7f68e8141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/885961187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e806c0a0 0x7f68e8141330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f68e815b530 tx=0x7f68d0009250 comp rx=0 tx=0).stop 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 -- 172.21.15.92:0/885961187 shutdown_connections 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/885961187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806bcd0 0x7f68e8140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/885961187 >> v2:172.21.15.136:3300/0 conn(0x7f68e80c8e00 0x7f68e80c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/885961187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e806c0a0 0x7f68e8141330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 -- 172.21.15.92:0/885961187 >> 172.21.15.92:0/885961187 conn(0x7f68e80ce600 msgr2=0x7f68e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 -- 172.21.15.92:0/885961187 shutdown_connections 2023-10-06T03:02:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.103+0000 7f68ed9d7700 1 -- 172.21.15.92:0/885961187 wait complete. 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 Processor -- start 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 -- start start 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e806bcd0 0x7f68e80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806c0a0 0x7f68e8069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 0x7f68e80afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80c5c50 con 0x7f68e80c8e00 2023-10-06T03:02:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e806b300 con 0x7f68e806bcd0 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68ed9d7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e806a390 con 0x7f68e806c0a0 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 0x7f68e80afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68e77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 0x7f68e80afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45436/0 (socket says 172.21.15.92:45436) 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.105+0000 7f68e77fe700 1 -- 172.21.15.92:0/3911317735 learned_addr learned my addr 172.21.15.92:0/3911317735 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e6ffd700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e806bcd0 0x7f68e80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e67fc700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806c0a0 0x7f68e8069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e77fe700 1 -- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806c0a0 msgr2=0x7f68e8069e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e77fe700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806c0a0 0x7f68e8069e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e77fe700 1 -- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e806bcd0 msgr2=0x7f68e80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e77fe700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e806bcd0 0x7f68e80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e77fe700 1 -- 172.21.15.92:0/3911317735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00094f0 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68e77fe700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 0x7f68e80afce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c8b0 tx=0x7f68e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0011600 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e0011760 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e80c56b0 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0011b00 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.106+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e806af90 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.107+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e806b4b0 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.107+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68e0018360 con 0x7f68e80c8e00 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.108+0000 7f68deffd700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c8066f30 0x7f68c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.108+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f68e0084240 con 0x7f68e80c8e00 2023-10-06T03:02:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.108+0000 7f68e6ffd700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c8066f30 0x7f68c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.108+0000 7f68e6ffd700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c8066f30 0x7f68c80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68d800f610 tx=0x7f68d80077f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.112+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68e806b4b0 con 0x7f68e80c8e00 2023-10-06T03:02:08.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.291+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68e806b4b0 con 0x7f68e80c8e00 2023-10-06T03:02:08.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.292+0000 7f68deffd700 1 -- 172.21.15.92:0/3911317735 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68e806b4b0 con 0x7f68e80c8e00 2023-10-06T03:02:08.293 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:02:08.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c8066f30 msgr2=0x7f68c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c8066f30 0x7f68c80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68d800f610 tx=0x7f68d80077f0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 msgr2=0x7f68e80afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 0x7f68e80afce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c8b0 tx=0x7f68e0008040 comp rx=0 tx=0).stop 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 shutdown_connections 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c8066f30 0x7f68c80693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e80c8e00 0x7f68e80afce0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e806bcd0 0x7f68e80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 --2- 172.21.15.92:0/3911317735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e806c0a0 0x7f68e8069e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 >> 172.21.15.92:0/3911317735 conn(0x7f68e80ce600 msgr2=0x7f68e80c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 shutdown_connections 2023-10-06T03:02:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.294+0000 7f68ed9d7700 1 -- 172.21.15.92:0/3911317735 wait complete. 2023-10-06T03:02:08.306 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:08.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.419+0000 7fa791cb2700 1 Processor -- start 2023-10-06T03:02:08.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.419+0000 7fa791cb2700 1 -- start start 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa791cb2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa78c159c60 0x7fa78c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa791cb2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 0x7fa78c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa791cb2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c15a160 con 0x7fa78c159890 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa791cb2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c15df50 con 0x7fa78c159c60 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa791cb2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0d13a0 con 0x7fa78c15db10 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa78b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa78c159890 0x7fa78c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56350/0 (socket says 172.21.15.92:56350) 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa78b7fe700 1 -- 172.21.15.92:0/378755462 learned_addr learned my addr 172.21.15.92:0/378755462 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:08.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa78bfff700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 0x7fa78c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa78affd700 1 --2- 172.21.15.92:0/378755462 >> v2:172.21.15.136:3300/0 conn(0x7fa78c159c60 0x7fa78c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816545699 0 0) 0x7fa78c15a160 con 0x7fa78c159890 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.420+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa778003570 con 0x7fa78c159890 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438823788 0 0) 0x7fa778003570 con 0x7fa78c159890 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 msgr2=0x7fa78c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 0x7fa78c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 >> v2:172.21.15.136:3300/0 conn(0x7fa78c159c60 msgr2=0x7fa78c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 --2- 172.21.15.92:0/378755462 >> v2:172.21.15.136:3300/0 conn(0x7fa78c159c60 0x7fa78c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c15e140 con 0x7fa78c159890 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa7840017f0 con 0x7fa78c159890 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 >> v1:172.21.15.92:6789/0 conn(0x7fa78c159890 legacy=0x7fa78c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 0x7fa778003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa778003fe0 0x7fa778004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa778004d50 0x7fa7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0d13a0 con 0x7fa778004d50 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77800ba60 con 0x7fa778003fe0 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77800bbd0 con 0x7fa78c15db10 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa778004d50 0x7fa7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78b7fe700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 0x7fa778003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78affd700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa778003fe0 0x7fa778004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 -- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 msgr2=0x7fa778003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c15db10 0x7fa778003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 -- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa778003fe0 msgr2=0x7fa778004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa778003fe0 0x7fa778004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 -- 172.21.15.92:0/378755462 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778003570 con 0x7fa778004d50 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78affd700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa778003fe0 0x7fa778004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78bfff700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa778004d50 0x7fa7780051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009c00 tx=0x7fa77c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e18ff629e23e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00e6b0 con 0x7fa778004d50 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00e810 con 0x7fa778004d50 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.421+0000 7fa78a7fc700 1 -- 172.21.15.92:0/378755462 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00ebd0 con 0x7fa778004d50 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa791cb2700 1 -- 172.21.15.92:0/378755462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa778004d50 msgr2=0x7fa7780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa791cb2700 1 --2- 172.21.15.92:0/378755462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa778004d50 0x7fa7780051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009c00 tx=0x7fa77c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa78bfff700 1 -- 172.21.15.92:0/378755462 reap_dead start 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa791cb2700 1 -- 172.21.15.92:0/378755462 shutdown_connections 2023-10-06T03:02:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa791cb2700 1 -- 172.21.15.92:0/378755462 >> 172.21.15.92:0/378755462 conn(0x7fa78c068e20 msgr2=0x7fa78c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:08.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa791cb2700 1 -- 172.21.15.92:0/378755462 shutdown_connections 2023-10-06T03:02:08.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.422+0000 7fa791cb2700 1 -- 172.21.15.92:0/378755462 wait complete. 2023-10-06T03:02:08.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.423+0000 7fa791cb2700 1 Processor -- start 2023-10-06T03:02:08.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.423+0000 7fa791cb2700 1 -- start start 2023-10-06T03:02:08.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa791cb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 0x7fa78c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa791cb2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa78c0cf2b0 0x7fa78c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa791cb2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c0cf720 0x7fa78c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa791cb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1519e0 con 0x7fa78c154740 2023-10-06T03:02:08.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa791cb2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0cee20 con 0x7fa78c0cf2b0 2023-10-06T03:02:08.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa791cb2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0cdca0 con 0x7fa78c0cf720 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 0x7fa78c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 0x7fa78c154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45458/0 (socket says 172.21.15.92:45458) 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 -- 172.21.15.92:0/2308049887 learned_addr learned my addr 172.21.15.92:0/2308049887 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78affd700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa78c0cf2b0 0x7fa78c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78bfff700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c0cf720 0x7fa78c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 -- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c0cf720 msgr2=0x7fa78c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c0cf720 0x7fa78c0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 -- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa78c0cf2b0 msgr2=0x7fa78c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa78c0cf2b0 0x7fa78c0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.424+0000 7fa78b7fe700 1 -- 172.21.15.92:0/2308049887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0cdf80 con 0x7fa78c154740 2023-10-06T03:02:08.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.425+0000 7fa78b7fe700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 0x7fa78c154b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa784007c90 tx=0x7fa784002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.425+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784013600 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.425+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784013760 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.425+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c0ce7d0 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.425+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784013b00 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.425+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c140e10 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.426+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c0cc540 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.426+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa78401c400 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.426+0000 7fa7837fe700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa76c066f30 0x7fa76c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.427+0000 7fa78affd700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa76c066f30 0x7fa76c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.427+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7840858d0 con 0x7fa78c154740 2023-10-06T03:02:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.427+0000 7fa78affd700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa76c066f30 0x7fa76c0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa774006c80 tx=0x7fa774008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.431+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7840529e0 con 0x7fa78c154740 2023-10-06T03:02:08.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.616+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa78c0cf100 con 0x7fa78c154740 2023-10-06T03:02:08.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.616+0000 7fa7837fe700 1 -- 172.21.15.92:0/2308049887 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fa784017090 con 0x7fa78c154740 2023-10-06T03:02:08.618 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:08.618 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:08.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa76c066f30 msgr2=0x7fa76c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa76c066f30 0x7fa76c0693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa774006c80 tx=0x7fa774008040 comp rx=0 tx=0).stop 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 msgr2=0x7fa78c154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 0x7fa78c154b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa784007c90 tx=0x7fa784002210 comp rx=0 tx=0).stop 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 shutdown_connections 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa76c066f30 0x7fa76c0693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa78c154740 0x7fa78c154b10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa78c0cf720 0x7fa78c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 --2- 172.21.15.92:0/2308049887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa78c0cf2b0 0x7fa78c0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.618+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 >> 172.21.15.92:0/2308049887 conn(0x7fa78c068e20 msgr2=0x7fa78c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.619+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 shutdown_connections 2023-10-06T03:02:08.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.619+0000 7fa791cb2700 1 -- 172.21.15.92:0/2308049887 wait complete. 2023-10-06T03:02:08.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:02:08.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:08.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.749+0000 7fa85f2d5700 1 Processor -- start 2023-10-06T03:02:08.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.749+0000 7fa85f2d5700 1 -- start start 2023-10-06T03:02:08.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85f2d5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8580b4e70 0x7fa8580ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85f2d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa8580b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85f2d5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8580b2080 con 0x7fa85815d240 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85f2d5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8580cfb00 con 0x7fa8580b4e70 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85f2d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580b5c40 con 0x7fa8580b16d0 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85ead4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa85815d240 0x7fa85815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56358/0 (socket says 172.21.15.92:56358) 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85ead4700 1 -- 172.21.15.92:0/802974695 learned_addr learned my addr 172.21.15.92:0/802974695 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85e2d3700 1 --2- 172.21.15.92:0/802974695 >> v2:172.21.15.136:3300/0 conn(0x7fa8580b4e70 0x7fa8580ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85dad2700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa8580b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251262374 0 0) 0x7fa8580b2080 con 0x7fa85815d240 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.750+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa840003570 con 0x7fa85815d240 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 236398629 0 0) 0x7fa840003570 con 0x7fa85815d240 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 msgr2=0x7fa8580b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa8580b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 >> v2:172.21.15.136:3300/0 conn(0x7fa8580b4e70 msgr2=0x7fa8580ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 --2- 172.21.15.92:0/802974695 >> v2:172.21.15.136:3300/0 conn(0x7fa8580b4e70 0x7fa8580ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580b5f20 con 0x7fa85815d240 2023-10-06T03:02:08.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85e2d3700 1 --2- 172.21.15.92:0/802974695 >> v2:172.21.15.136:3300/0 conn(0x7fa8580b4e70 0x7fa8580ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa8480017f0 con 0x7fa85815d240 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 >> v1:172.21.15.92:6789/0 conn(0x7fa85815d240 legacy=0x7fa85815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840003a60 0x7fa840003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa840004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840004d50 0x7fa8400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580cfb00 con 0x7fa840004d50 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580b5c40 con 0x7fa840003a60 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000ba60 con 0x7fa8580b16d0 2023-10-06T03:02:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840004d50 0x7fa8400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85e2d3700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840003a60 0x7fa840003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85dad2700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa840004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 -- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 msgr2=0x7fa840004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa840004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 -- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840003a60 msgr2=0x7fa840003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840003a60 0x7fa840003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 -- 172.21.15.92:0/802974695 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840003570 con 0x7fa840004d50 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85dad2700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580b16d0 0x7fa840004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85ead4700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840004d50 0x7fa8400051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa84800a930 tx=0x7fa848002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e13fd7dcc80171c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.751+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480070d0 con 0x7fa840004d50 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848007230 con 0x7fa840004d50 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85d2d1700 1 -- 172.21.15.92:0/802974695 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480075f0 con 0x7fa840004d50 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85f2d5700 1 -- 172.21.15.92:0/802974695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840004d50 msgr2=0x7fa8400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/802974695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa840004d50 0x7fa8400051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa84800a930 tx=0x7fa848002180 comp rx=0 tx=0).stop 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85ead4700 1 -- 172.21.15.92:0/802974695 reap_dead start 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85f2d5700 1 -- 172.21.15.92:0/802974695 shutdown_connections 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85f2d5700 1 -- 172.21.15.92:0/802974695 >> 172.21.15.92:0/802974695 conn(0x7fa8580d10c0 msgr2=0x7fa8580affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85f2d5700 1 -- 172.21.15.92:0/802974695 shutdown_connections 2023-10-06T03:02:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.752+0000 7fa85f2d5700 1 -- 172.21.15.92:0/802974695 wait complete. 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.753+0000 7fa85f2d5700 1 Processor -- start 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.753+0000 7fa85f2d5700 1 -- start start 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85f2d5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840004d50 0x7fa8581556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85f2d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 0x7fa8581571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85f2d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580c5ec0 0x7fa8580c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85f2d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580c8dc0 con 0x7fa858156d50 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85f2d5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581576e0 con 0x7fa840004d50 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85f2d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580c6c20 con 0x7fa8580c5ec0 2023-10-06T03:02:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 0x7fa8581571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 0x7fa8581571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45468/0 (socket says 172.21.15.92:45468) 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 -- 172.21.15.92:0/2205265429 learned_addr learned my addr 172.21.15.92:0/2205265429 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85e2d3700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840004d50 0x7fa8581556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85ead4700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580c5ec0 0x7fa8580c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 -- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580c5ec0 msgr2=0x7fa8580c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580c5ec0 0x7fa8580c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 -- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840004d50 msgr2=0x7fa8581556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840004d50 0x7fa8581556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 -- 172.21.15.92:0/2205265429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580c6f00 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85ead4700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580c5ec0 0x7fa8580c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.754+0000 7fa85dad2700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 0x7fa8581571a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011960 tx=0x7fa84c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.755+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c011ea0 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.755+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c00f8e0 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.755+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8580c71e0 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.755+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00fc60 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.755+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8580c77a0 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.756+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8580c79c0 con 0x7fa858156d50 2023-10-06T03:02:08.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.756+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa84c015460 con 0x7fa858156d50 2023-10-06T03:02:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.756+0000 7fa8567fc700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa844066f60 0x7fa844069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.756+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa84c084ec0 con 0x7fa858156d50 2023-10-06T03:02:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.756+0000 7fa85e2d3700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa844066f60 0x7fa844069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.757+0000 7fa85e2d3700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa844066f60 0x7fa844069410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa858157a40 tx=0x7fa850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.760+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa84c051fd0 con 0x7fa858156d50 2023-10-06T03:02:08.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.941+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8580cf4b0 con 0x7fa858156d50 2023-10-06T03:02:08.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.942+0000 7fa8567fc700 1 -- 172.21.15.92:0/2205265429 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa84c0555f0 con 0x7fa858156d50 2023-10-06T03:02:08.942 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa844066f60 msgr2=0x7fa844069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa844066f60 0x7fa844069410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa858157a40 tx=0x7fa850008040 comp rx=0 tx=0).stop 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 msgr2=0x7fa8581571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 0x7fa8581571a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa84c011960 tx=0x7fa84c008040 comp rx=0 tx=0).stop 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 shutdown_connections 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa844066f60 0x7fa844069410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8580c5ec0 0x7fa8580c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa840004d50 0x7fa8581556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 --2- 172.21.15.92:0/2205265429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa858156d50 0x7fa8581571a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:08.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 >> 172.21.15.92:0/2205265429 conn(0x7fa8580d10c0 msgr2=0x7fa8580b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:08.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 shutdown_connections 2023-10-06T03:02:08.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:08.944+0000 7fa85f2d5700 1 -- 172.21.15.92:0/2205265429 wait complete. 2023-10-06T03:02:08.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:09.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.073+0000 7fe7af317700 1 Processor -- start 2023-10-06T03:02:09.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.073+0000 7fe7af317700 1 -- start start 2023-10-06T03:02:09.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7af317700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe7a8069560 0x7fe7a80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7af317700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a8069ee0 con 0x7fe7a8159c60 2023-10-06T03:02:09.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7af317700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a815a160 con 0x7fe7a8159890 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7af317700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80d13a0 con 0x7fe7a8069560 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ac8b2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7a8159c60 0x7fe7a815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56370/0 (socket says 172.21.15.92:56370) 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ac8b2700 1 -- 172.21.15.92:0/2305471796 learned_addr learned my addr 172.21.15.92:0/2305471796 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ad8b4700 1 --2- 172.21.15.92:0/2305471796 >> v2:172.21.15.136:3300/0 conn(0x7fe7a8069560 0x7fe7a80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe79ffff700 1 -- 172.21.15.92:0/2305471796 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065166207 0 0) 0x7fe7a8069ee0 con 0x7fe7a8159c60 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe79ffff700 1 -- 172.21.15.92:0/2305471796 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe794003570 con 0x7fe7a8159c60 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe79ffff700 1 -- 172.21.15.92:0/2305471796 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2880970468 0 0) 0x7fe794003570 con 0x7fe7a8159c60 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ad8b4700 1 -- 172.21.15.92:0/2305471796 >> v1:172.21.15.136:6789/0 conn(0x7fe7a8159890 legacy=0x7fe7a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ad8b4700 1 -- 172.21.15.92:0/2305471796 >> v1:172.21.15.92:6789/0 conn(0x7fe7a8159c60 legacy=0x7fe7a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ad8b4700 1 -- 172.21.15.92:0/2305471796 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a815e140 con 0x7fe7a8069560 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.074+0000 7fe7ad8b4700 1 --2- 172.21.15.92:0/2305471796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8069560 0x7fe7a80699a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009e60 tx=0x7fe7a0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa00486f494bdc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe79ffff700 1 -- 172.21.15.92:0/2305471796 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000e070 con 0x7fe7a8069560 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe79ffff700 1 -- 172.21.15.92:0/2305471796 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a000b850 con 0x7fe7a8069560 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe79ffff700 1 -- 172.21.15.92:0/2305471796 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798001470 con 0x7fe7a8069560 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 -- 172.21.15.92:0/2305471796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8069560 msgr2=0x7fe7a80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 --2- 172.21.15.92:0/2305471796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8069560 0x7fe7a80699a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009e60 tx=0x7fe7a0009780 comp rx=0 tx=0).stop 2023-10-06T03:02:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 -- 172.21.15.92:0/2305471796 shutdown_connections 2023-10-06T03:02:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 --2- 172.21.15.92:0/2305471796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8069560 0x7fe7a80699a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 -- 172.21.15.92:0/2305471796 >> 172.21.15.92:0/2305471796 conn(0x7fe7a8068e20 msgr2=0x7fe7a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 -- 172.21.15.92:0/2305471796 shutdown_connections 2023-10-06T03:02:09.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.075+0000 7fe7af317700 1 -- 172.21.15.92:0/2305471796 wait complete. 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.076+0000 7fe7af317700 1 Processor -- start 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.076+0000 7fe7af317700 1 -- start start 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7af317700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 0x7fe7a8148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7af317700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8159890 0x7fe7a80b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7af317700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a8159c60 0x7fe7a815f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7af317700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80ce9d0 con 0x7fe7a8069560 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7af317700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80ce3f0 con 0x7fe7a8159890 2023-10-06T03:02:09.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7af317700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b48e0 con 0x7fe7a8159c60 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7ad0b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 0x7fe7a8148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7ad0b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 0x7fe7a8148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45480/0 (socket says 172.21.15.92:45480) 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.077+0000 7fe7ad0b3700 1 -- 172.21.15.92:0/3638932491 learned_addr learned my addr 172.21.15.92:0/3638932491 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ac8b2700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8159890 0x7fe7a80b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad8b4700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a8159c60 0x7fe7a815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad0b3700 1 -- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a8159c60 msgr2=0x7fe7a815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad0b3700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a8159c60 0x7fe7a815f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad0b3700 1 -- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8159890 msgr2=0x7fe7a80b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad0b3700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8159890 0x7fe7a80b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad0b3700 1 -- 172.21.15.92:0/3638932491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0009a20 con 0x7fe7a8069560 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad8b4700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a8159c60 0x7fe7a815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7ad0b3700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 0x7fe7a8148290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400c5c0 tx=0x7fe7a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4011590 con 0x7fe7a8069560 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a4011710 con 0x7fe7a8069560 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a80ce660 con 0x7fe7a8069560 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4011ad0 con 0x7fe7a8069560 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.078+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80b4590 con 0x7fe7a8069560 2023-10-06T03:02:09.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.079+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a804c3c0 con 0x7fe7a8069560 2023-10-06T03:02:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.079+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7a401b3b0 con 0x7fe7a8069560 2023-10-06T03:02:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.080+0000 7fe79d7fa700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe78c066f30 0x7fe78c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.080+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe7a4083b80 con 0x7fe7a8069560 2023-10-06T03:02:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.080+0000 7fe7ac8b2700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe78c066f30 0x7fe78c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.080+0000 7fe7ac8b2700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe78c066f30 0x7fe78c0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8157280 tx=0x7fe798000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.084+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7a4050c90 con 0x7fe7a8069560 2023-10-06T03:02:09.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.266+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe7a80d0d40 con 0x7fe7a8069560 2023-10-06T03:02:09.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.266+0000 7fe79d7fa700 1 -- 172.21.15.92:0/3638932491 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe7a4017070 con 0x7fe7a8069560 2023-10-06T03:02:09.267 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.268+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe78c066f30 msgr2=0x7fe78c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.268+0000 7fe7af317700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe78c066f30 0x7fe78c0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8157280 tx=0x7fe798000bc0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.268+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 msgr2=0x7fe7a8148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.268+0000 7fe7af317700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 0x7fe7a8148290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400c5c0 tx=0x7fe7a400c8d0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 shutdown_connections 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe78c066f30 0x7fe78c0693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8069560 0x7fe7a8148290 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a8159c60 0x7fe7a815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 --2- 172.21.15.92:0/3638932491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a8159890 0x7fe7a80b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 >> 172.21.15.92:0/3638932491 conn(0x7fe7a8068e20 msgr2=0x7fe7a815d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 shutdown_connections 2023-10-06T03:02:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.269+0000 7fe7af317700 1 -- 172.21.15.92:0/3638932491 wait complete. 2023-10-06T03:02:09.281 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:02:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.398+0000 7fcda14e7700 1 Processor -- start 2023-10-06T03:02:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.398+0000 7fcda14e7700 1 -- start start 2023-10-06T03:02:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcda14e7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd9c06a9c0 0x7fcd9c06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcda14e7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c0ac450 con 0x7fcd9c06b160 2023-10-06T03:02:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcda14e7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c069d00 con 0x7fcd9c0637d0 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcda14e7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c0693d0 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcd9b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd9c06a9c0 0x7fcd9c06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcd9b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd9c06a9c0 0x7fcd9c06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50876/0 (socket says 172.21.15.92:50876) 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcd9b7fe700 1 -- 172.21.15.92:0/3221058330 learned_addr learned my addr 172.21.15.92:0/3221058330 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.399+0000 7fcd9affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcd9c06b160 0x7fcd9c068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46580/0 (socket says 172.21.15.92:46580) 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9b7fe700 1 -- 172.21.15.92:0/3221058330 >> v1:172.21.15.136:6789/0 conn(0x7fcd9c0637d0 legacy=0x7fcd9c06a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9b7fe700 1 -- 172.21.15.92:0/3221058330 >> v1:172.21.15.92:6789/0 conn(0x7fcd9c06b160 legacy=0x7fcd9c068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9b7fe700 1 -- 172.21.15.92:0/3221058330 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c068830 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9a7fc700 1 -- 172.21.15.92:0/3221058330 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795128150 0 0) 0x7fcd9c0ac450 con 0x7fcd9c06b160 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9b7fe700 1 --2- 172.21.15.92:0/3221058330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c06b530 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009f20 tx=0x7fcd94009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ae091ee0fad951 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9a7fc700 1 -- 172.21.15.92:0/3221058330 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9400e070 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9a7fc700 1 -- 172.21.15.92:0/3221058330 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9400ba40 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcd9a7fc700 1 -- 172.21.15.92:0/3221058330 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940092e0 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcda14e7700 1 -- 172.21.15.92:0/3221058330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 msgr2=0x7fcd9c06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.400+0000 7fcda14e7700 1 --2- 172.21.15.92:0/3221058330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c06b530 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009f20 tx=0x7fcd94009840 comp rx=0 tx=0).stop 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.401+0000 7fcda14e7700 1 -- 172.21.15.92:0/3221058330 shutdown_connections 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.401+0000 7fcda14e7700 1 --2- 172.21.15.92:0/3221058330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c06b530 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.401+0000 7fcda14e7700 1 -- 172.21.15.92:0/3221058330 >> 172.21.15.92:0/3221058330 conn(0x7fcd9c06ded0 msgr2=0x7fcd9c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.401+0000 7fcda14e7700 1 -- 172.21.15.92:0/3221058330 shutdown_connections 2023-10-06T03:02:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.401+0000 7fcda14e7700 1 -- 172.21.15.92:0/3221058330 wait complete. 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.402+0000 7fcda14e7700 1 Processor -- start 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.402+0000 7fcda14e7700 1 -- start start 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.402+0000 7fcda14e7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd9c0637d0 0x7fcd9c0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd9c06b160 0x7fcd9c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0b2900 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c145bb0 con 0x7fcd9c0637d0 2023-10-06T03:02:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1557c0 con 0x7fcd9c06b160 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50884/0 (socket says 172.21.15.92:50884) 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 -- 172.21.15.92:0/4255946867 learned_addr learned my addr 172.21.15.92:0/4255946867 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd9b7fe700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd9c06b160 0x7fcd9c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd9affd700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd9c0637d0 0x7fcd9c0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 -- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd9c06b160 msgr2=0x7fcd9c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd9c06b160 0x7fcd9c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 -- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd9c0637d0 msgr2=0x7fcd9c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd9c0637d0 0x7fcd9c0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 -- 172.21.15.92:0/4255946867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94009ae0 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd927fc700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c1594a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b5e0 tx=0x7fcd84009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400f510 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8400f670 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c155aa0 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400fa30 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.403+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c156060 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.404+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c04c3c0 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.405+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd8401a400 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.405+0000 7fcd937fe700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd7c066f30 0x7fcd7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.405+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcd84084080 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.405+0000 7fcd9affd700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd7c066f30 0x7fcd7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.405+0000 7fcd9affd700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd7c066f30 0x7fcd7c0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c007fd0 tx=0x7fcd8c007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.409+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd84051210 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.592+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd9c0635b0 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.593+0000 7fcd937fe700 1 -- 172.21.15.92:0/4255946867 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd84058e60 con 0x7fcd9c06a9c0 2023-10-06T03:02:09.594 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd7c066f30 msgr2=0x7fcd7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd7c066f30 0x7fcd7c0693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c007fd0 tx=0x7fcd8c007d50 comp rx=0 tx=0).stop 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 msgr2=0x7fcd9c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c1594a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400b5e0 tx=0x7fcd84009120 comp rx=0 tx=0).stop 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 shutdown_connections 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd7c066f30 0x7fcd7c0693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd9c06b160 0x7fcd9c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd9c0637d0 0x7fcd9c0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 --2- 172.21.15.92:0/4255946867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd9c06a9c0 0x7fcd9c1594a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 >> 172.21.15.92:0/4255946867 conn(0x7fcd9c06ded0 msgr2=0x7fcd9c0acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 shutdown_connections 2023-10-06T03:02:09.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.595+0000 7fcda14e7700 1 -- 172.21.15.92:0/4255946867 wait complete. 2023-10-06T03:02:09.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:02:09.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.723+0000 7f45c19e5700 1 Processor -- start 2023-10-06T03:02:09.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.723+0000 7f45c19e5700 1 -- start start 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45c19e5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0aca80 0x7f45bc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45c19e5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45bc0c83c0 0x7f45bc0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45c19e5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc06bdb0 con 0x7f45bc069d60 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45c19e5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0cb370 con 0x7f45bc0c83c0 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45c19e5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc159f60 con 0x7f45bc0aca80 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45bb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45bc0c83c0 0x7f45bc0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45ba7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f45bc069d60 0x7f45bc0c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46592/0 (socket says 172.21.15.92:46592) 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45ba7fc700 1 -- 172.21.15.92:0/1269832292 learned_addr learned my addr 172.21.15.92:0/1269832292 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.724+0000 7f45baffd700 1 --2- 172.21.15.92:0/1269832292 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0aca80 0x7f45bc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45bb7fe700 1 -- 172.21.15.92:0/1269832292 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0aca80 msgr2=0x7f45bc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45bb7fe700 1 --2- 172.21.15.92:0/1269832292 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0aca80 0x7f45bc06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45bb7fe700 1 -- 172.21.15.92:0/1269832292 >> v1:172.21.15.92:6789/0 conn(0x7f45bc069d60 legacy=0x7f45bc0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45b9ffb700 1 -- 172.21.15.92:0/1269832292 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869304452 0 0) 0x7f45bc06bdb0 con 0x7f45bc069d60 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45bb7fe700 1 -- 172.21.15.92:0/1269832292 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc06b510 con 0x7f45bc0c83c0 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45bb7fe700 1 --2- 172.21.15.92:0/1269832292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0c83c0 0x7f45bc0cb800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45b40098e0 tx=0x7f45b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f6e89234633b7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45b9ffb700 1 -- 172.21.15.92:0/1269832292 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400b640 con 0x7f45bc0c83c0 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45b9ffb700 1 -- 172.21.15.92:0/1269832292 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400b7a0 con 0x7f45bc0c83c0 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45b9ffb700 1 -- 172.21.15.92:0/1269832292 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400bb40 con 0x7f45bc0c83c0 2023-10-06T03:02:09.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45c19e5700 1 -- 172.21.15.92:0/1269832292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0c83c0 msgr2=0x7f45bc0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.725+0000 7f45c19e5700 1 --2- 172.21.15.92:0/1269832292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0c83c0 0x7f45bc0cb800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45b40098e0 tx=0x7f45b40091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.726+0000 7f45c19e5700 1 -- 172.21.15.92:0/1269832292 shutdown_connections 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.726+0000 7f45c19e5700 1 --2- 172.21.15.92:0/1269832292 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0aca80 0x7f45bc06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.726+0000 7f45c19e5700 1 --2- 172.21.15.92:0/1269832292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0c83c0 0x7f45bc0cb800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.726+0000 7f45c19e5700 1 -- 172.21.15.92:0/1269832292 >> 172.21.15.92:0/1269832292 conn(0x7f45bc0cd670 msgr2=0x7f45bc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.726+0000 7f45c19e5700 1 -- 172.21.15.92:0/1269832292 shutdown_connections 2023-10-06T03:02:09.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.726+0000 7f45c19e5700 1 -- 172.21.15.92:0/1269832292 wait complete. 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.727+0000 7f45c19e5700 1 Processor -- start 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.727+0000 7f45c19e5700 1 -- start start 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45bc069d60 0x7f45bc0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 0x7f45bc0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0c83c0 0x7f45bc0c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c8230 con 0x7f45bc0aca80 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0b2b50 con 0x7f45bc069d60 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c7620 con 0x7f45bc0c83c0 2023-10-06T03:02:09.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 0x7f45bc0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 0x7f45bc0c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50908/0 (socket says 172.21.15.92:50908) 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 -- 172.21.15.92:0/2807723909 learned_addr learned my addr 172.21.15.92:0/2807723909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45bb7fe700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0c83c0 0x7f45bc0c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45baffd700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45bc069d60 0x7f45bc0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 -- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0c83c0 msgr2=0x7f45bc0c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0c83c0 0x7f45bc0c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 -- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45bc069d60 msgr2=0x7f45bc0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45bc069d60 0x7f45bc0b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 -- 172.21.15.92:0/2807723909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009420 con 0x7f45bc0aca80 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45baffd700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45bc069d60 0x7f45bc0b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:09.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45bb7fe700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0c83c0 0x7f45bc0c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45ba7fc700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 0x7f45bc0c4720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c810 tx=0x7f45b0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0005c00 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b0005d60 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0007320 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc0c7900 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.728+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc0c7ec0 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.729+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc0cbae0 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.730+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45b001b400 con 0x7f45bc0aca80 2023-10-06T03:02:09.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.730+0000 7f45a2ffd700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45a8066f30 0x7f45a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.730+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45b0084170 con 0x7f45bc0aca80 2023-10-06T03:02:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.730+0000 7f45baffd700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45a8066f30 0x7f45a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.731+0000 7f45baffd700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45a8066f30 0x7f45a80693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0b3460 tx=0x7f45ac00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:09.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.734+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45bc0cbae0 con 0x7f45bc0aca80 2023-10-06T03:02:09.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.916+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45bc0cbae0 con 0x7f45bc0aca80 2023-10-06T03:02:09.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.916+0000 7f45a2ffd700 1 -- 172.21.15.92:0/2807723909 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45bc0cbae0 con 0x7f45bc0aca80 2023-10-06T03:02:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:02:09.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45a8066f30 msgr2=0x7f45a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45a8066f30 0x7f45a80693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0b3460 tx=0x7f45ac00b040 comp rx=0 tx=0).stop 2023-10-06T03:02:09.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 msgr2=0x7f45bc0c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:09.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 0x7f45bc0c4720 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c810 tx=0x7f45b0007db0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 shutdown_connections 2023-10-06T03:02:09.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45a8066f30 0x7f45a80693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45bc0c83c0 0x7f45bc0c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45bc069d60 0x7f45bc0b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 --2- 172.21.15.92:0/2807723909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45bc0aca80 0x7f45bc0c4720 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 >> 172.21.15.92:0/2807723909 conn(0x7f45bc0cd670 msgr2=0x7f45bc159070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 shutdown_connections 2023-10-06T03:02:09.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:09.918+0000 7f45c19e5700 1 -- 172.21.15.92:0/2807723909 wait complete. 2023-10-06T03:02:09.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.044+0000 7f6223cb2700 1 Processor -- start 2023-10-06T03:02:10.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.044+0000 7f6223cb2700 1 -- start start 2023-10-06T03:02:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6223cb2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 0x7f621c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6223cb2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f621c159c60 0x7f621c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6223cb2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c15a160 con 0x7f621c15db10 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6223cb2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621c0d13a0 con 0x7f621c159c60 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6223cb2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c15e140 con 0x7f621c159890 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f622224f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f621c15db10 0x7f621c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46594/0 (socket says 172.21.15.92:46594) 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f622224f700 1 -- 172.21.15.92:0/3249118357 learned_addr learned my addr 172.21.15.92:0/3249118357 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f622124d700 1 --2- 172.21.15.92:0/3249118357 >> v2:172.21.15.136:3300/0 conn(0x7f621c159c60 0x7f621c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901628089 0 0) 0x7f621c15a160 con 0x7f621c15db10 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6221a4e700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 0x7f621c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6208003570 con 0x7f621c15db10 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3670540065 0 0) 0x7f6208003570 con 0x7f621c15db10 2023-10-06T03:02:10.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 msgr2=0x7f621c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 0x7f621c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 >> v2:172.21.15.136:3300/0 conn(0x7f621c159c60 msgr2=0x7f621c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 --2- 172.21.15.92:0/3249118357 >> v2:172.21.15.136:3300/0 conn(0x7f621c159c60 0x7f621c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c15a3d0 con 0x7f621c15db10 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.045+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f6218001470 con 0x7f621c15db10 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 >> v1:172.21.15.92:6789/0 conn(0x7f621c15db10 legacy=0x7f621c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6208003a60 0x7f6208003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 0x7f6208004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6208004d50 0x7f62080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0d13a0 con 0x7f6208004d50 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c15e140 con 0x7f6208003a60 2023-10-06T03:02:10.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620800ba60 con 0x7f621c159890 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6208004d50 0x7f62080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6221a4e700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6208003a60 0x7f6208003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622124d700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 0x7f6208004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 -- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 msgr2=0x7f6208004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c159890 0x7f6208004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 -- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6208003a60 msgr2=0x7f6208003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6208003a60 0x7f6208003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 -- 172.21.15.92:0/3249118357 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6208003570 con 0x7f6208004d50 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f622224f700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6208004d50 0x7f62080051c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f621800c3c0 tx=0x7f6218001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda119f200dbc9c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800cc30 con 0x7f6208004d50 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621800cdb0 con 0x7f6208004d50 2023-10-06T03:02:10.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.046+0000 7f6220a4c700 1 -- 172.21.15.92:0/3249118357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218005b70 con 0x7f6208004d50 2023-10-06T03:02:10.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f6223cb2700 1 -- 172.21.15.92:0/3249118357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6208004d50 msgr2=0x7f62080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3249118357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6208004d50 0x7f62080051c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f621800c3c0 tx=0x7f6218001e90 comp rx=0 tx=0).stop 2023-10-06T03:02:10.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f622224f700 1 -- 172.21.15.92:0/3249118357 reap_dead start 2023-10-06T03:02:10.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f6223cb2700 1 -- 172.21.15.92:0/3249118357 shutdown_connections 2023-10-06T03:02:10.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f6223cb2700 1 -- 172.21.15.92:0/3249118357 >> 172.21.15.92:0/3249118357 conn(0x7f621c068e20 msgr2=0x7f621c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:10.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f6223cb2700 1 -- 172.21.15.92:0/3249118357 shutdown_connections 2023-10-06T03:02:10.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.047+0000 7f6223cb2700 1 -- 172.21.15.92:0/3249118357 wait complete. 2023-10-06T03:02:10.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.048+0000 7f6223cb2700 1 Processor -- start 2023-10-06T03:02:10.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.048+0000 7f6223cb2700 1 -- start start 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 0x7f621c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c0b3f30 0x7f621c0b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c0b1110 0x7f621c0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0cf740 con 0x7f621c14c610 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0cef30 con 0x7f621c0b1110 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0b3aa0 con 0x7f621c0b3f30 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 0x7f621c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 0x7f621c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50918/0 (socket says 172.21.15.92:50918) 2023-10-06T03:02:10.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 -- 172.21.15.92:0/3557740811 learned_addr learned my addr 172.21.15.92:0/3557740811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f622124d700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c0b3f30 0x7f621c0b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 -- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c0b3f30 msgr2=0x7f621c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c0b3f30 0x7f621c0b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 -- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c0b1110 msgr2=0x7f621c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f622224f700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c0b1110 0x7f621c0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c0b1110 0x7f621c0b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 -- 172.21.15.92:0/3557740811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c14c9e0 con 0x7f621c14c610 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f622124d700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c0b3f30 0x7f621c0b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6221a4e700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 0x7f621c0b5c60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6214011f00 tx=0x7f621400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400e040 con 0x7f621c14c610 2023-10-06T03:02:10.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621400f040 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c0cf180 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.049+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214015430 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.050+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c0b3730 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.050+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c0b3590 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.051+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6214015860 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.051+0000 7f6211ffb700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6200066f30 0x7f62000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.051+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6214085390 con 0x7f621c14c610 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.051+0000 7f622124d700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6200066f30 0x7f62000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.052+0000 7f622124d700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6200066f30 0x7f62000693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f621c0b2fa0 tx=0x7f620c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.055+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f621c0b3590 con 0x7f621c14c610 2023-10-06T03:02:10.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.240+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f621c0b3590 con 0x7f621c14c610 2023-10-06T03:02:10.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.241+0000 7f6211ffb700 1 -- 172.21.15.92:0/3557740811 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f621c0b3590 con 0x7f621c14c610 2023-10-06T03:02:10.242 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:10.242 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:10.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6200066f30 msgr2=0x7f62000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6200066f30 0x7f62000693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f621c0b2fa0 tx=0x7f620c008040 comp rx=0 tx=0).stop 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 msgr2=0x7f621c0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 0x7f621c0b5c60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6214011f00 tx=0x7f621400b040 comp rx=0 tx=0).stop 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 shutdown_connections 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6200066f30 0x7f62000693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c14c610 0x7f621c0b5c60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c0b1110 0x7f621c0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 --2- 172.21.15.92:0/3557740811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c0b3f30 0x7f621c0b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 >> 172.21.15.92:0/3557740811 conn(0x7f621c068e20 msgr2=0x7f621c14d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:10.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 shutdown_connections 2023-10-06T03:02:10.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.243+0000 7f6223cb2700 1 -- 172.21.15.92:0/3557740811 wait complete. 2023-10-06T03:02:10.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:02:10.256 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:10.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.373+0000 7f308c50c700 1 Processor -- start 2023-10-06T03:02:10.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.373+0000 7f308c50c700 1 -- start start 2023-10-06T03:02:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308c50c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3084069560 0x7f30840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308c50c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3084069ee0 con 0x7f3084159890 2023-10-06T03:02:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308c50c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308415a160 con 0x7f3084159c60 2023-10-06T03:02:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308c50c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30840d13a0 con 0x7f3084069560 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308a2a8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3084159890 0x7f308415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46596/0 (socket says 172.21.15.92:46596) 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308aaa9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3084069560 0x7f30840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308a2a8700 1 -- 172.21.15.92:0/928221246 learned_addr learned my addr 172.21.15.92:0/928221246 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f30892a6700 1 -- 172.21.15.92:0/928221246 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068713981 0 0) 0x7f3084069ee0 con 0x7f3084159890 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f30892a6700 1 -- 172.21.15.92:0/928221246 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3070003570 con 0x7f3084159890 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308aaa9700 1 -- 172.21.15.92:0/928221246 >> v1:172.21.15.136:6789/0 conn(0x7f3084159c60 legacy=0x7f308415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308aaa9700 1 -- 172.21.15.92:0/928221246 >> v1:172.21.15.92:6789/0 conn(0x7f3084159890 legacy=0x7f308415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308aaa9700 1 -- 172.21.15.92:0/928221246 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308415e140 con 0x7f3084069560 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.374+0000 7f308aaa9700 1 --2- 172.21.15.92:0/928221246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f30840699a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f307c009e60 tx=0x7f307c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2713848048ae7a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f30892a6700 1 -- 172.21.15.92:0/928221246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c00e070 con 0x7f3084069560 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f30892a6700 1 -- 172.21.15.92:0/928221246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307c00bb20 con 0x7f3084069560 2023-10-06T03:02:10.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f30892a6700 1 -- 172.21.15.92:0/928221246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c012430 con 0x7f3084069560 2023-10-06T03:02:10.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 -- 172.21.15.92:0/928221246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 msgr2=0x7f30840699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 --2- 172.21.15.92:0/928221246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f30840699a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f307c009e60 tx=0x7f307c009780 comp rx=0 tx=0).stop 2023-10-06T03:02:10.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 -- 172.21.15.92:0/928221246 shutdown_connections 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 --2- 172.21.15.92:0/928221246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f30840699a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 -- 172.21.15.92:0/928221246 >> 172.21.15.92:0/928221246 conn(0x7f3084068e20 msgr2=0x7f30840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 -- 172.21.15.92:0/928221246 shutdown_connections 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.375+0000 7f308c50c700 1 -- 172.21.15.92:0/928221246 wait complete. 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.376+0000 7f308c50c700 1 Processor -- start 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.376+0000 7f308c50c700 1 -- start start 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308c50c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f3084154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308c50c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3084159890 0x7f3084157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308c50c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084159c60 0x7f3084157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308c50c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084169560 con 0x7f3084069560 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308c50c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308416a370 con 0x7f3084159c60 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308c50c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3084169760 con 0x7f3084159890 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f3084154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f3084154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50926/0 (socket says 172.21.15.92:50926) 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 -- 172.21.15.92:0/2737752489 learned_addr learned my addr 172.21.15.92:0/2737752489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308aaa9700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084159c60 0x7f3084157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f3089aa7700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3084159890 0x7f3084157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 -- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3084159890 msgr2=0x7f3084157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3084159890 0x7f3084157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 -- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084159c60 msgr2=0x7f3084157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084159c60 0x7f3084157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f308a2a8700 1 -- 172.21.15.92:0/2737752489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c009a20 con 0x7f3084069560 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.377+0000 7f3089aa7700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3084159890 0x7f3084157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.378+0000 7f308a2a8700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f3084154230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f308000c7c0 tx=0x7f3080007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.378+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080005b30 con 0x7f3084069560 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.378+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080005c90 con 0x7f3084069560 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.378+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080007340 con 0x7f3084069560 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.378+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084169a40 con 0x7f3084069560 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.378+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308416a000 con 0x7f3084069560 2023-10-06T03:02:10.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.379+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308404c3c0 con 0x7f3084069560 2023-10-06T03:02:10.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.379+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f308001b400 con 0x7f3084069560 2023-10-06T03:02:10.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.380+0000 7f307a7fc700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068066f30 0x7f30680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.380+0000 7f3089aa7700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068066f30 0x7f30680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.380+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f30800841a0 con 0x7f3084069560 2023-10-06T03:02:10.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.380+0000 7f3089aa7700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068066f30 0x7f30680693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3074006fd0 tx=0x7f3074008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.384+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3080051330 con 0x7f3084069560 2023-10-06T03:02:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.567+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f308406a5b0 con 0x7f3084069560 2023-10-06T03:02:10.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.568+0000 7f307a7fc700 1 -- 172.21.15.92:0/2737752489 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3080054950 con 0x7f3084069560 2023-10-06T03:02:10.569 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068066f30 msgr2=0x7f30680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068066f30 0x7f30680693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3074006fd0 tx=0x7f3074008040 comp rx=0 tx=0).stop 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 msgr2=0x7f3084154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f3084154230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f308000c7c0 tx=0x7f3080007e00 comp rx=0 tx=0).stop 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 shutdown_connections 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068066f30 0x7f30680693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3084069560 0x7f3084154230 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084159c60 0x7f3084157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 --2- 172.21.15.92:0/2737752489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3084159890 0x7f3084157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 >> 172.21.15.92:0/2737752489 conn(0x7f3084068e20 msgr2=0x7f3084069930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:10.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 shutdown_connections 2023-10-06T03:02:10.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.570+0000 7f308c50c700 1 -- 172.21.15.92:0/2737752489 wait complete. 2023-10-06T03:02:10.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.699+0000 7fa6ca57f700 1 Processor -- start 2023-10-06T03:02:10.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.699+0000 7fa6ca57f700 1 -- start start 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6ca57f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa6c406b520 0x7fa6c40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6ca57f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c40cc210 con 0x7fa6c406aa40 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6ca57f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c40cc4f0 con 0x7fa6c40c91e0 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6ca57f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c40cc7d0 con 0x7fa6c406b520 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa6c406aa40 0x7fa6c406bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46602/0 (socket says 172.21.15.92:46602) 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c3fff700 1 -- 172.21.15.92:0/2557169174 learned_addr learned my addr 172.21.15.92:0/2557169174 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:10.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966713929 0 0) 0x7fa6c40cc210 con 0x7fa6c406aa40 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ac003570 con 0x7fa6c406aa40 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c8b1c700 1 --2- 172.21.15.92:0/2557169174 >> v2:172.21.15.136:3300/0 conn(0x7fa6c406b520 0x7fa6c40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1745395661 0 0) 0x7fa6ac003570 con 0x7fa6c406aa40 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 >> v2:172.21.15.136:3300/0 conn(0x7fa6c406b520 msgr2=0x7fa6c40cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 --2- 172.21.15.92:0/2557169174 >> v2:172.21.15.136:3300/0 conn(0x7fa6c406b520 0x7fa6c40cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 >> v1:172.21.15.136:6789/0 conn(0x7fa6c40c91e0 legacy=0x7fa6c406ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40ccab0 con 0x7fa6c406aa40 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa6b40015f0 con 0x7fa6c406aa40 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 >> v1:172.21.15.92:6789/0 conn(0x7fa6c406aa40 legacy=0x7fa6c406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ac003a60 0x7fa6ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6ac004810 0x7fa6ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40cc7d0 con 0x7fa6ac0051c0 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00c0d0 con 0x7fa6ac003a60 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.700+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac00c290 con 0x7fa6ac004810 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c3fff700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ac003a60 0x7fa6ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c37fe700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6ac004810 0x7fa6ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 -- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6ac004810 msgr2=0x7fa6ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6ac004810 0x7fa6ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 -- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ac003a60 msgr2=0x7fa6ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6ac003a60 0x7fa6ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 -- 172.21.15.92:0/2557169174 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac003570 con 0x7fa6ac0051c0 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 -- 172.21.15.92:0/2557169174 reap_dead start 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c8b1c700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6ac00ba60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cc140 tx=0x7fa6bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4373d89b47553997 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00ea90 con 0x7fa6ac0051c0 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00f040 con 0x7fa6ac0051c0 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.701+0000 7fa6c2ffd700 1 -- 172.21.15.92:0/2557169174 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00ee60 con 0x7fa6ac0051c0 2023-10-06T03:02:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 -- 172.21.15.92:0/2557169174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 msgr2=0x7fa6ac00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6ac00ba60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40cc140 tx=0x7fa6bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:02:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 -- 172.21.15.92:0/2557169174 shutdown_connections 2023-10-06T03:02:10.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/2557169174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6ac00ba60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 -- 172.21.15.92:0/2557169174 >> 172.21.15.92:0/2557169174 conn(0x7fa6c40cd670 msgr2=0x7fa6c40cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 -- 172.21.15.92:0/2557169174 shutdown_connections 2023-10-06T03:02:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.702+0000 7fa6ca57f700 1 -- 172.21.15.92:0/2557169174 wait complete. 2023-10-06T03:02:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.703+0000 7fa6ca57f700 1 Processor -- start 2023-10-06T03:02:10.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.703+0000 7fa6ca57f700 1 -- start start 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6c4150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6c40b0dd0 0x7fa6c40b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6c40b3040 0x7fa6c4157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40b1290 con 0x7fa6ac0051c0 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40b1570 con 0x7fa6c40b3040 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40b1850 con 0x7fa6c40b0dd0 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6c4150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6c4150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50944/0 (socket says 172.21.15.92:50944) 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 -- 172.21.15.92:0/249680065 learned_addr learned my addr 172.21.15.92:0/249680065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c37fe700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6c40b0dd0 0x7fa6c40b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c8b1c700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6c40b3040 0x7fa6c4157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 -- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6c40b0dd0 msgr2=0x7fa6c40b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6c40b0dd0 0x7fa6c40b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 -- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6c40b3040 msgr2=0x7fa6c4157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6c40b3040 0x7fa6c4157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 -- 172.21.15.92:0/249680065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40b1b30 con 0x7fa6ac0051c0 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c37fe700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6c40b0dd0 0x7fa6c40b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6c3fff700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6c4150810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400f9d0 tx=0x7fa6b4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40072e0 con 0x7fa6ac0051c0 2023-10-06T03:02:10.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b4007440 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c4150d50 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40057f0 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.704+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c41584a0 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.705+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c404c3c0 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.706+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6b4005bc0 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.706+0000 7fa6abfff700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa6b0066f30 0x7fa6b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.706+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa6b4084960 con 0x7fa6ac0051c0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.706+0000 7fa6c37fe700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa6b0066f30 0x7fa6b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.707+0000 7fa6c37fe700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa6b0066f30 0x7fa6b00693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8007e40 tx=0x7fa6b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:10.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.710+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6b4051a70 con 0x7fa6ac0051c0 2023-10-06T03:02:10.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.893+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6c40c4d90 con 0x7fa6ac0051c0 2023-10-06T03:02:10.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.894+0000 7fa6abfff700 1 -- 172.21.15.92:0/249680065 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa6b4055090 con 0x7fa6ac0051c0 2023-10-06T03:02:10.895 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa6b0066f30 msgr2=0x7fa6b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa6b0066f30 0x7fa6b00693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8007e40 tx=0x7fa6b8009040 comp rx=0 tx=0).stop 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 msgr2=0x7fa6c4150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6c4150810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400f9d0 tx=0x7fa6b4009040 comp rx=0 tx=0).stop 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 shutdown_connections 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa6b0066f30 0x7fa6b00693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6ac0051c0 0x7fa6c4150810 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6c40b3040 0x7fa6c4157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 --2- 172.21.15.92:0/249680065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6c40b0dd0 0x7fa6c40b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:10.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 >> 172.21.15.92:0/249680065 conn(0x7fa6c40cd670 msgr2=0x7fa6c406ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:10.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 shutdown_connections 2023-10-06T03:02:10.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:10.896+0000 7fa6ca57f700 1 -- 172.21.15.92:0/249680065 wait complete. 2023-10-06T03:02:10.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:02:11.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.026+0000 7f0e41f30700 1 Processor -- start 2023-10-06T03:02:11.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.026+0000 7f0e41f30700 1 -- start start 2023-10-06T03:02:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e41f30700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c159890 0x7f0e3c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e41f30700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0e3c159c60 0x7f0e3c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e41f30700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e3c15db10 0x7f0e3c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e41f30700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c15a160 con 0x7f0e3c15db10 2023-10-06T03:02:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e41f30700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c0d13a0 con 0x7f0e3c159c60 2023-10-06T03:02:11.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e41f30700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c15e140 con 0x7f0e3c159890 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e3c15db10 0x7f0e3c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0e3c15db10 0x7f0e3c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50954/0 (socket says 172.21.15.92:50954) 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 -- 172.21.15.92:0/1044615254 learned_addr learned my addr 172.21.15.92:0/1044615254 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3b7fe700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c159890 0x7f0e3c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3affd700 1 --2- 172.21.15.92:0/1044615254 >> v2:172.21.15.136:3300/0 conn(0x7f0e3c159c60 0x7f0e3c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 -- 172.21.15.92:0/1044615254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c159890 msgr2=0x7f0e3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c159890 0x7f0e3c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 -- 172.21.15.92:0/1044615254 >> v2:172.21.15.136:3300/0 conn(0x7f0e3c159c60 msgr2=0x7f0e3c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 --2- 172.21.15.92:0/1044615254 >> v2:172.21.15.136:3300/0 conn(0x7f0e3c159c60 0x7f0e3c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 -- 172.21.15.92:0/1044615254 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c15a3d0 con 0x7f0e3c15db10 2023-10-06T03:02:11.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3b7fe700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c159890 0x7f0e3c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3bfff700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c15db10 0x7f0e3c069560 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0650f0 tx=0x7f0e2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1dc8b2b45217cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3a7fc700 1 -- 172.21.15.92:0/1044615254 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00b860 con 0x7f0e3c15db10 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3a7fc700 1 -- 172.21.15.92:0/1044615254 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00b9c0 con 0x7f0e3c15db10 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.027+0000 7f0e3a7fc700 1 -- 172.21.15.92:0/1044615254 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00bd60 con 0x7f0e3c15db10 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 -- 172.21.15.92:0/1044615254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c15db10 msgr2=0x7f0e3c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c15db10 0x7f0e3c069560 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0650f0 tx=0x7f0e2c009250 comp rx=0 tx=0).stop 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 -- 172.21.15.92:0/1044615254 shutdown_connections 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c159890 0x7f0e3c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 --2- 172.21.15.92:0/1044615254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c15db10 0x7f0e3c069560 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 --2- 172.21.15.92:0/1044615254 >> v2:172.21.15.136:3300/0 conn(0x7f0e3c159c60 0x7f0e3c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 -- 172.21.15.92:0/1044615254 >> 172.21.15.92:0/1044615254 conn(0x7f0e3c068e20 msgr2=0x7f0e3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:11.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 -- 172.21.15.92:0/1044615254 shutdown_connections 2023-10-06T03:02:11.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.028+0000 7f0e41f30700 1 -- 172.21.15.92:0/1044615254 wait complete. 2023-10-06T03:02:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.029+0000 7f0e41f30700 1 Processor -- start 2023-10-06T03:02:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.029+0000 7f0e41f30700 1 -- start start 2023-10-06T03:02:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e41f30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 0x7f0e3c16a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e41f30700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c159c60 0x7f0e3c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e41f30700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c15db10 0x7f0e3c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e41f30700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c16abe0 con 0x7f0e3c159890 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e41f30700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0cfb80 con 0x7f0e3c159c60 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e41f30700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0cdb40 con 0x7f0e3c15db10 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 0x7f0e3c16a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 0x7f0e3c16a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50966/0 (socket says 172.21.15.92:50966) 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 -- 172.21.15.92:0/3796898361 learned_addr learned my addr 172.21.15.92:0/3796898361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3bfff700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c15db10 0x7f0e3c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3affd700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c159c60 0x7f0e3c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 -- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c15db10 msgr2=0x7f0e3c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c15db10 0x7f0e3c1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 -- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c159c60 msgr2=0x7f0e3c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c159c60 0x7f0e3c0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3b7fe700 1 -- 172.21.15.92:0/3796898361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0094f0 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.030+0000 7f0e3bfff700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c15db10 0x7f0e3c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.031+0000 7f0e3b7fe700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 0x7f0e3c16a6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400d1d0 tx=0x7f0e34009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.031+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400de70 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.031+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3400f040 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.031+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c0cf250 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.031+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34015670 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.031+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c0cf810 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.032+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c0d15a0 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.032+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e34015a60 con 0x7f0e3c159890 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.032+0000 7f0e337fe700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0e1c066f30 0x7f0e1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.033+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e34084b80 con 0x7f0e3c159890 2023-10-06T03:02:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.033+0000 7f0e3affd700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0e1c066f30 0x7f0e1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.033+0000 7f0e3affd700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0e1c066f30 0x7f0e1c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c153d60 tx=0x7f0e24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.037+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e3404e5a0 con 0x7f0e3c159890 2023-10-06T03:02:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.218+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e3c0cfe60 con 0x7f0e3c159890 2023-10-06T03:02:11.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.218+0000 7f0e337fe700 1 -- 172.21.15.92:0/3796898361 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e3401a020 con 0x7f0e3c159890 2023-10-06T03:02:11.219 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:02:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0e1c066f30 msgr2=0x7f0e1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0e1c066f30 0x7f0e1c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c153d60 tx=0x7f0e24008040 comp rx=0 tx=0).stop 2023-10-06T03:02:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 msgr2=0x7f0e3c16a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 0x7f0e3c16a6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400d1d0 tx=0x7f0e34009a90 comp rx=0 tx=0).stop 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 shutdown_connections 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0e1c066f30 0x7f0e1c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e3c159890 0x7f0e3c16a6a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e3c15db10 0x7f0e3c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 --2- 172.21.15.92:0/3796898361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e3c159c60 0x7f0e3c0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 >> 172.21.15.92:0/3796898361 conn(0x7f0e3c068e20 msgr2=0x7f0e3c14d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 shutdown_connections 2023-10-06T03:02:11.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.220+0000 7f0e41f30700 1 -- 172.21.15.92:0/3796898361 wait complete. 2023-10-06T03:02:11.233 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:02:11.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.349+0000 7f38a4676700 1 Processor -- start 2023-10-06T03:02:11.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.349+0000 7f38a4676700 1 -- start start 2023-10-06T03:02:11.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.350+0000 7f38a4676700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f389c159c60 0x7f389c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.350+0000 7f38a4676700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c15a160 con 0x7f389c159890 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.350+0000 7f38a4676700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389c0d13a0 con 0x7f389c15db10 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.350+0000 7f38a4676700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389c15df50 con 0x7f389c159c60 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.350+0000 7f38a2412700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f389c159890 0x7f389c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46614/0 (socket says 172.21.15.92:46614) 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.350+0000 7f38a2412700 1 -- 172.21.15.92:0/4073636241 learned_addr learned my addr 172.21.15.92:0/4073636241 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071476159 0 0) 0x7f389c15a160 con 0x7f389c159890 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3888003570 con 0x7f389c159890 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1c11700 1 --2- 172.21.15.92:0/4073636241 >> v2:172.21.15.136:3300/0 conn(0x7f389c159c60 0x7f389c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129256321 0 0) 0x7f3888003570 con 0x7f389c159890 2023-10-06T03:02:11.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 >> v2:172.21.15.136:3300/0 conn(0x7f389c159c60 msgr2=0x7f389c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 --2- 172.21.15.92:0/4073636241 >> v2:172.21.15.136:3300/0 conn(0x7f389c159c60 0x7f389c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 >> v1:172.21.15.136:6789/0 conn(0x7f389c15db10 legacy=0x7f389c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c15e140 con 0x7f389c159890 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f38980017f0 con 0x7f389c159890 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 >> v1:172.21.15.92:6789/0 conn(0x7f389c159890 legacy=0x7f389c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888003a60 0x7f3888003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888004810 0x7f3888004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38880051c0 0x7f388800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388800c0a0 con 0x7f3888003a60 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388800c210 con 0x7f3888004810 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388800c3d0 con 0x7f38880051c0 2023-10-06T03:02:11.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2412700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888003a60 0x7f3888003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a1c11700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888004810 0x7f3888004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2c13700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38880051c0 0x7f388800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2412700 1 -- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38880051c0 msgr2=0x7f388800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2412700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38880051c0 0x7f388800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2412700 1 -- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888004810 msgr2=0x7f3888004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2412700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888004810 0x7f3888004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2c13700 1 -- 172.21.15.92:0/4073636241 reap_dead start 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.351+0000 7f38a2412700 1 -- 172.21.15.92:0/4073636241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888003570 con 0x7f3888003a60 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a2412700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888003a60 0x7f3888003e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f389800a930 tx=0x7f38980021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d1be30ecf67369 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38980070d0 con 0x7f3888003a60 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898007230 con 0x7f3888003a60 2023-10-06T03:02:11.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a1410700 1 -- 172.21.15.92:0/4073636241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38980075d0 con 0x7f3888003a60 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 -- 172.21.15.92:0/4073636241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888003a60 msgr2=0x7f3888003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888003a60 0x7f3888003e70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f389800a930 tx=0x7f38980021d0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 -- 172.21.15.92:0/4073636241 shutdown_connections 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 --2- 172.21.15.92:0/4073636241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3888003a60 0x7f3888003e70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 -- 172.21.15.92:0/4073636241 >> 172.21.15.92:0/4073636241 conn(0x7f389c068e20 msgr2=0x7f389c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 -- 172.21.15.92:0/4073636241 shutdown_connections 2023-10-06T03:02:11.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.352+0000 7f38a4676700 1 -- 172.21.15.92:0/4073636241 wait complete. 2023-10-06T03:02:11.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.353+0000 7f38a4676700 1 Processor -- start 2023-10-06T03:02:11.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.353+0000 7f38a4676700 1 -- start start 2023-10-06T03:02:11.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a4676700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888003a60 0x7f389c151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a4676700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f389c0ca330 0x7f389c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a4676700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 0x7f389c0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a4676700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0c8250 con 0x7f389c0c78a0 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a4676700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0b4f00 con 0x7f3888003a60 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a4676700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389c0b4330 con 0x7f389c0ca330 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a2c13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 0x7f389c0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a2c13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 0x7f389c0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50994/0 (socket says 172.21.15.92:50994) 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a2c13700 1 -- 172.21.15.92:0/208159402 learned_addr learned my addr 172.21.15.92:0/208159402 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a1c11700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f389c0ca330 0x7f389c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.354+0000 7f38a2412700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888003a60 0x7f389c151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a2c13700 1 -- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f389c0ca330 msgr2=0x7f389c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a2c13700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f389c0ca330 0x7f389c0ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a2c13700 1 -- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888003a60 msgr2=0x7f389c151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a2c13700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888003a60 0x7f389c151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a2c13700 1 -- 172.21.15.92:0/208159402 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0b45d0 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a2c13700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 0x7f389c0c7d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f389c06a300 tx=0x7f3894007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011590 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38940116f0 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c0b48b0 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011ab0 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.355+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389c0b3e90 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.356+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389c04c3c0 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.356+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f389401d460 con 0x7f389c0c78a0 2023-10-06T03:02:11.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.357+0000 7f38927fc700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.357+0000 7f38a2412700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.357+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f38940845d0 con 0x7f389c0c78a0 2023-10-06T03:02:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.357+0000 7f38a2412700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3898002250 tx=0x7f3898013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.361+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38940516e0 con 0x7f389c0c78a0 2023-10-06T03:02:11.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.543+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f389c06a5b0 con 0x7f389c0c78a0 2023-10-06T03:02:11.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.544+0000 7f38927fc700 1 -- 172.21.15.92:0/208159402 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3894054d00 con 0x7f389c0c78a0 2023-10-06T03:02:11.545 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:02:11.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3880066f30 msgr2=0x7f38800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3880066f30 0x7f38800693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3898002250 tx=0x7f3898013000 comp rx=0 tx=0).stop 2023-10-06T03:02:11.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 msgr2=0x7f389c0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 0x7f389c0c7d10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f389c06a300 tx=0x7f3894007d30 comp rx=0 tx=0).stop 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 shutdown_connections 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3880066f30 0x7f38800693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f389c0c78a0 0x7f389c0c7d10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3888003a60 0x7f389c151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 --2- 172.21.15.92:0/208159402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f389c0ca330 0x7f389c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 >> 172.21.15.92:0/208159402 conn(0x7f389c068e20 msgr2=0x7f389c152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 shutdown_connections 2023-10-06T03:02:11.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.546+0000 7f38a4676700 1 -- 172.21.15.92:0/208159402 wait complete. 2023-10-06T03:02:11.618 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-10-06T03:02:11.618 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:02:11.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-06T03:02:11.650 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:02:11.664 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:02:11.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.736+0000 7ff5af6c4700 1 Processor -- start 2023-10-06T03:02:11.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.737+0000 7ff5af6c4700 1 -- start start 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.737+0000 7ff5af6c4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff5a80c91e0 0x7ff5a80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.737+0000 7ff5af6c4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a806b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.737+0000 7ff5af6c4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80cc210 con 0x7ff5a806aa40 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.737+0000 7ff5af6c4700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80cc4f0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.737+0000 7ff5af6c4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80cc7d0 con 0x7ff5a806ae10 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a806b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5acc5f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff5a80c91e0 0x7ff5a80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a806b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:54374/0 (socket says 172.21.15.92:54374) 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5ad460700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7ff5a806aa40 0x7ff5a806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:38928/0 (socket says 172.21.15.92:38928) 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 -- 172.21.15.92:0/2461435865 learned_addr learned my addr 172.21.15.92:0/2461435865 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 -- 172.21.15.92:0/2461435865 >> v2:172.21.15.136:3300/0 conn(0x7ff5a80c91e0 msgr2=0x7ff5a80cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 --2- 172.21.15.92:0/2461435865 >> v2:172.21.15.136:3300/0 conn(0x7ff5a80c91e0 0x7ff5a80cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 -- 172.21.15.92:0/2461435865 >> v1:172.21.15.136:6789/0 conn(0x7ff5a806aa40 legacy=0x7ff5a806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 -- 172.21.15.92:0/2461435865 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80ccab0 con 0x7ff5a806ae10 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff5adc61700 1 --2- 172.21.15.92:0/2461435865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a806b2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009f50 tx=0x7ff5a0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d13d10d8bb5fe1f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff59ffff700 1 -- 172.21.15.92:0/2461435865 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000e070 con 0x7ff5a806ae10 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff59ffff700 1 -- 172.21.15.92:0/2461435865 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a000b9c0 con 0x7ff5a806ae10 2023-10-06T03:02:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.738+0000 7ff59ffff700 1 -- 172.21.15.92:0/2461435865 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a00092e0 con 0x7ff5a806ae10 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 -- 172.21.15.92:0/2461435865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 msgr2=0x7ff5a806b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/2461435865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a806b2b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009f50 tx=0x7ff5a0009840 comp rx=0 tx=0).stop 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 -- 172.21.15.92:0/2461435865 shutdown_connections 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/2461435865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a806b2b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/2461435865 >> v2:172.21.15.136:3300/0 conn(0x7ff5a80c91e0 0x7ff5a80cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 -- 172.21.15.92:0/2461435865 >> 172.21.15.92:0/2461435865 conn(0x7ff5a80cd670 msgr2=0x7ff5a80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 -- 172.21.15.92:0/2461435865 shutdown_connections 2023-10-06T03:02:11.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.739+0000 7ff5af6c4700 1 -- 172.21.15.92:0/2461435865 wait complete. 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.740+0000 7ff5af6c4700 1 Processor -- start 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.740+0000 7ff5af6c4700 1 -- start start 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5af6c4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a806aa40 0x7ff5a80c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5af6c4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a816ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5af6c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 0x7ff5a816c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5af6c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a816ce90 con 0x7ff5a80c91e0 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5af6c4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b1d90 con 0x7ff5a806aa40 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5af6c4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a816a2e0 con 0x7ff5a806ae10 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5adc61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 0x7ff5a816c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5ad460700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a806aa40 0x7ff5a80c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5adc61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 0x7ff5a816c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51006/0 (socket says 172.21.15.92:51006) 2023-10-06T03:02:11.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5adc61700 1 -- 172.21.15.92:0/3321079291 learned_addr learned my addr 172.21.15.92:0/3321079291 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.741+0000 7ff5acc5f700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a816ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5adc61700 1 -- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 msgr2=0x7ff5a816ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5adc61700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a816ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5adc61700 1 -- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a806aa40 msgr2=0x7ff5a80c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5adc61700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a806aa40 0x7ff5a80c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5adc61700 1 -- 172.21.15.92:0/3321079291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009ae0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5ad460700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a806aa40 0x7ff5a80c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5adc61700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 0x7ff5a816c950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009880 tx=0x7ff5a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0008ba0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a0008d00 con 0x7ff5a80c91e0 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a816a5c0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0026690 con 0x7ff5a80c91e0 2023-10-06T03:02:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.742+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a80b1fe0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.743+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a804c3c0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.743+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5a0003040 con 0x7ff5a80c91e0 2023-10-06T03:02:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.744+0000 7ff59d7fa700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff58c066f30 0x7ff58c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.744+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5a0085fe0 con 0x7ff5a80c91e0 2023-10-06T03:02:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.744+0000 7ff5ad460700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff58c066f30 0x7ff58c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.744+0000 7ff5ad460700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff58c066f30 0x7ff58c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4007fd0 tx=0x7ff5a4007f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:11.748+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5a00531a0 con 0x7ff5a80c91e0 2023-10-06T03:02:12.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.001+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7ff5a80c7cb0 con 0x7ff5a80c91e0 2023-10-06T03:02:12.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.511+0000 7ff59d7fa700 1 -- 172.21.15.92:0/3321079291 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v50) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff5a00567c0 con 0x7ff5a80c91e0 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff58c066f30 msgr2=0x7ff58c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff58c066f30 0x7ff58c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4007fd0 tx=0x7ff5a4007f00 comp rx=0 tx=0).stop 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 msgr2=0x7ff5a816c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 0x7ff5a816c950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009880 tx=0x7ff5a0018000 comp rx=0 tx=0).stop 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 shutdown_connections 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff58c066f30 0x7ff58c0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5a80c91e0 0x7ff5a816c950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5a806aa40 0x7ff5a80c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 --2- 172.21.15.92:0/3321079291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5a806ae10 0x7ff5a816ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 >> 172.21.15.92:0/3321079291 conn(0x7ff5a80cd670 msgr2=0x7ff5a815b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:12.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 shutdown_connections 2023-10-06T03:02:12.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.514+0000 7ff5af6c4700 1 -- 172.21.15.92:0/3321079291 wait complete. 2023-10-06T03:02:12.528 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:02:12.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.688+0000 7fcb26d06700 1 Processor -- start 2023-10-06T03:02:12.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.688+0000 7fcb26d06700 1 -- start start 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.688+0000 7fcb26d06700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb20159890 0x7fcb2015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb26d06700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159c60 0x7fcb2015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb26d06700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcb2015db10 0x7fcb20069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb26d06700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2015a160 con 0x7fcb20159890 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb26d06700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb200d13a0 con 0x7fcb2015db10 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb26d06700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2015e140 con 0x7fcb20159c60 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb20159890 0x7fcb2015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb20159890 0x7fcb2015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51020/0 (socket says 172.21.15.92:51020) 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 -- 172.21.15.92:0/3251517322 learned_addr learned my addr 172.21.15.92:0/3251517322 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:12.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb17fff700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159c60 0x7fcb2015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb252a3700 1 --2- 172.21.15.92:0/3251517322 >> v2:172.21.15.136:3300/0 conn(0x7fcb2015db10 0x7fcb20069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 -- 172.21.15.92:0/3251517322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159c60 msgr2=0x7fcb2015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159c60 0x7fcb2015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 -- 172.21.15.92:0/3251517322 >> v2:172.21.15.136:3300/0 conn(0x7fcb2015db10 msgr2=0x7fcb20069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/3251517322 >> v2:172.21.15.136:3300/0 conn(0x7fcb2015db10 0x7fcb20069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 -- 172.21.15.92:0/3251517322 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2015a3d0 con 0x7fcb20159890 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb252a3700 1 --2- 172.21.15.92:0/3251517322 >> v2:172.21.15.136:3300/0 conn(0x7fcb2015db10 0x7fcb20069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb17fff700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159c60 0x7fcb2015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb20159890 0x7fcb2015abd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb20089b50 tx=0x7fcb1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde84da9c4e74121 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb177fe700 1 -- 172.21.15.92:0/3251517322 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c00e070 con 0x7fcb20159890 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb177fe700 1 -- 172.21.15.92:0/3251517322 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c00ba50 con 0x7fcb20159890 2023-10-06T03:02:12.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.689+0000 7fcb177fe700 1 -- 172.21.15.92:0/3251517322 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0092e0 con 0x7fcb20159890 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 -- 172.21.15.92:0/3251517322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb20159890 msgr2=0x7fcb2015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb20159890 0x7fcb2015abd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb20089b50 tx=0x7fcb1c009840 comp rx=0 tx=0).stop 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 -- 172.21.15.92:0/3251517322 shutdown_connections 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 --2- 172.21.15.92:0/3251517322 >> v2:172.21.15.136:3300/0 conn(0x7fcb2015db10 0x7fcb20069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb20159890 0x7fcb2015abd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 --2- 172.21.15.92:0/3251517322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159c60 0x7fcb2015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 -- 172.21.15.92:0/3251517322 >> 172.21.15.92:0/3251517322 conn(0x7fcb20068e20 msgr2=0x7fcb200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 -- 172.21.15.92:0/3251517322 shutdown_connections 2023-10-06T03:02:12.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.690+0000 7fcb26d06700 1 -- 172.21.15.92:0/3251517322 wait complete. 2023-10-06T03:02:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.691+0000 7fcb26d06700 1 Processor -- start 2023-10-06T03:02:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.691+0000 7fcb26d06700 1 -- start start 2023-10-06T03:02:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb26d06700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159890 0x7fcb2016ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb26d06700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb20159c60 0x7fcb2016cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb26d06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 0x7fcb200c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb26d06700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20157c50 con 0x7fcb2015db10 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb26d06700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20152c00 con 0x7fcb20159c60 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb26d06700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb200b5580 con 0x7fcb20159890 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 0x7fcb200c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 0x7fcb200c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51024/0 (socket says 172.21.15.92:51024) 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 -- 172.21.15.92:0/1482022090 learned_addr learned my addr 172.21.15.92:0/1482022090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb17fff700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb20159c60 0x7fcb2016cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159890 0x7fcb2016ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 -- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159890 msgr2=0x7fcb2016ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159890 0x7fcb2016ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 -- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb20159c60 msgr2=0x7fcb2016cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb20159c60 0x7fcb2016cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 -- 172.21.15.92:0/1482022090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c009ae0 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb17fff700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb20159c60 0x7fcb2016cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159890 0x7fcb2016ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.692+0000 7fcb252a3700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 0x7fcb200c9f50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800c850 tx=0x7fcb18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.693+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18011600 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.693+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb18011760 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.693+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb200b5820 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.693+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18011b20 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.693+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb200b5de0 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.694+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2004c3c0 con 0x7fcb2015db10 2023-10-06T03:02:12.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.694+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb18018350 con 0x7fcb2015db10 2023-10-06T03:02:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.694+0000 7fcb14ff9700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb0c066f30 0x7fcb0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.694+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcb180845b0 con 0x7fcb2015db10 2023-10-06T03:02:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.694+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb0c066f30 0x7fcb0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.695+0000 7fcb24aa2700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb0c066f30 0x7fcb0c0693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb200d0f40 tx=0x7fcb1c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:12.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.698+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb18055150 con 0x7fcb2015db10 2023-10-06T03:02:12.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.880+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcb200657f0 con 0x7fcb2015db10 2023-10-06T03:02:12.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.884+0000 7fcb14ff9700 1 -- 172.21.15.92:0/1482022090 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb18054ce0 con 0x7fcb2015db10 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb0c066f30 msgr2=0x7fcb0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb0c066f30 0x7fcb0c0693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcb200d0f40 tx=0x7fcb1c018000 comp rx=0 tx=0).stop 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 msgr2=0x7fcb200c9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 0x7fcb200c9f50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800c850 tx=0x7fcb18008040 comp rx=0 tx=0).stop 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 shutdown_connections 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb0c066f30 0x7fcb0c0693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb2015db10 0x7fcb200c9f50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb20159890 0x7fcb2016ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 --2- 172.21.15.92:0/1482022090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb20159c60 0x7fcb2016cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:12.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 >> 172.21.15.92:0/1482022090 conn(0x7fcb20068e20 msgr2=0x7fcb200c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:12.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 shutdown_connections 2023-10-06T03:02:12.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:12.887+0000 7fcb26d06700 1 -- 172.21.15.92:0/1482022090 wait complete. 2023-10-06T03:02:12.899 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:02:12.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:02:12.934 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:02:12.976 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:02:12.976 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:02:12.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:02:12.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T03:02:13.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:02:13.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:13.011 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:02:13.066 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:02:13.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:02:13.066 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T03:02:13.107 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:02:13.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:13.107 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:02:13.164 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T03:02:13.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:02:13.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T03:02:13.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:02:13.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:13.199 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T03:02:13.254 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T03:02:13.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:02:13.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T03:02:13.293 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:02:13.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:13.293 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T03:02:13.349 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:02:13.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:13.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.464+0000 7f321972e700 1 Processor -- start 2023-10-06T03:02:13.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.464+0000 7f321972e700 1 -- start start 2023-10-06T03:02:13.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f321972e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3214156680 0x7f321406c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f321972e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3214156a50 0x7f32140c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f321972e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f32140c7e70 0x7f32140af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f321972e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321406be50 con 0x7f3214156680 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f321972e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140c88d0 con 0x7f32140c7e70 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f321972e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321406c3f0 con 0x7f3214156a50 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3214156680 0x7f321406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3214156680 0x7f321406c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51026/0 (socket says 172.21.15.92:51026) 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 -- 172.21.15.92:0/2909719025 learned_addr learned my addr 172.21.15.92:0/2909719025 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f32127fc700 1 --2- 172.21.15.92:0/2909719025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3214156a50 0x7f32140c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 -- 172.21.15.92:0/2909719025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3214156a50 msgr2=0x7f32140c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f32137fe700 1 --2- 172.21.15.92:0/2909719025 >> v2:172.21.15.136:3300/0 conn(0x7f32140c7e70 0x7f32140af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 --2- 172.21.15.92:0/2909719025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3214156a50 0x7f32140c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 -- 172.21.15.92:0/2909719025 >> v2:172.21.15.136:3300/0 conn(0x7f32140c7e70 msgr2=0x7f32140af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.465+0000 7f3212ffd700 1 --2- 172.21.15.92:0/2909719025 >> v2:172.21.15.136:3300/0 conn(0x7f32140c7e70 0x7f32140af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f3212ffd700 1 -- 172.21.15.92:0/2909719025 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32140c8310 con 0x7f3214156680 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f3212ffd700 1 --2- 172.21.15.92:0/2909719025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f321406c880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32140afe30 tx=0x7f3204009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195fcf7e5ce011bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f3211ffb700 1 -- 172.21.15.92:0/2909719025 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320400b860 con 0x7f3214156680 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f3211ffb700 1 -- 172.21.15.92:0/2909719025 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320400b9c0 con 0x7f3214156680 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f3211ffb700 1 -- 172.21.15.92:0/2909719025 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320400bd60 con 0x7f3214156680 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 -- 172.21.15.92:0/2909719025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 msgr2=0x7f321406c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 --2- 172.21.15.92:0/2909719025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f321406c880 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32140afe30 tx=0x7f3204009250 comp rx=0 tx=0).stop 2023-10-06T03:02:13.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 -- 172.21.15.92:0/2909719025 shutdown_connections 2023-10-06T03:02:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 --2- 172.21.15.92:0/2909719025 >> v2:172.21.15.136:3300/0 conn(0x7f32140c7e70 0x7f32140af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 --2- 172.21.15.92:0/2909719025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f321406c880 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 --2- 172.21.15.92:0/2909719025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3214156a50 0x7f32140c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 -- 172.21.15.92:0/2909719025 >> 172.21.15.92:0/2909719025 conn(0x7f32140cdd80 msgr2=0x7f32140c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 -- 172.21.15.92:0/2909719025 shutdown_connections 2023-10-06T03:02:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.466+0000 7f321972e700 1 -- 172.21.15.92:0/2909719025 wait complete. 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 Processor -- start 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 -- start start 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32140c7e70 0x7f32140b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f3214143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3214156a50 0x7f32140c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321414f580 con 0x7f3214156680 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214167fa0 con 0x7f3214156a50 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f321972e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321416bad0 con 0x7f32140c7e70 2023-10-06T03:02:13.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f32127fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f3214143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f32127fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f3214143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51042/0 (socket says 172.21.15.92:51042) 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.468+0000 7f32127fc700 1 -- 172.21.15.92:0/1822163526 learned_addr learned my addr 172.21.15.92:0/1822163526 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f3212ffd700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32140c7e70 0x7f32140b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32137fe700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3214156a50 0x7f32140c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32127fc700 1 -- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32140c7e70 msgr2=0x7f32140b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32127fc700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32140c7e70 0x7f32140b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32127fc700 1 -- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3214156a50 msgr2=0x7f32140c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32127fc700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3214156a50 0x7f32140c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32127fc700 1 -- 172.21.15.92:0/1822163526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040094f0 con 0x7f3214156680 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32137fe700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3214156a50 0x7f32140c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:13.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f32127fc700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f3214143080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f320800c880 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208011600 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208011760 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208011b00 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321416bdb0 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.469+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321416c370 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.470+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32141681c0 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.470+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32080223a0 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.471+0000 7f31faffd700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3200066f30 0x7f32000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.471+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3208084750 con 0x7f3214156680 2023-10-06T03:02:13.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.471+0000 7f3212ffd700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3200066f30 0x7f32000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.471+0000 7f3212ffd700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3200066f30 0x7f32000693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32140afe30 tx=0x7f3204017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:13.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.475+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32141681c0 con 0x7f3214156680 2023-10-06T03:02:13.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.702+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32141681c0 con 0x7f3214156680 2023-10-06T03:02:13.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.702+0000 7f31faffd700 1 -- 172.21.15.92:0/1822163526 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f32141681c0 con 0x7f3214156680 2023-10-06T03:02:13.703 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:13.703 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:02:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3200066f30 msgr2=0x7f32000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3200066f30 0x7f32000693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32140afe30 tx=0x7f3204017000 comp rx=0 tx=0).stop 2023-10-06T03:02:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 msgr2=0x7f3214143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f3214143080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f320800c880 tx=0x7f3208008040 comp rx=0 tx=0).stop 2023-10-06T03:02:13.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 shutdown_connections 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3200066f30 0x7f32000693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3214156a50 0x7f32140c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32140c7e70 0x7f32140b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 --2- 172.21.15.92:0/1822163526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3214156680 0x7f3214143080 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 >> 172.21.15.92:0/1822163526 conn(0x7f32140cdd80 msgr2=0x7f3214151020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 shutdown_connections 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.704+0000 7f321972e700 1 -- 172.21.15.92:0/1822163526 wait complete. 2023-10-06T03:02:13.706 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:02:13.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.830+0000 7f5516b73700 1 Processor -- start 2023-10-06T03:02:13.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.830+0000 7f5516b73700 1 -- start start 2023-10-06T03:02:13.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5516b73700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5510159890 0x7f551015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5516b73700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5510069560 0x7f55100699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5516b73700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510069ee0 con 0x7f5510159c60 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5516b73700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551015a160 con 0x7f5510069560 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5516b73700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55100d13a0 con 0x7f5510159890 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5515110700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5510069560 0x7f55100699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5515110700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5510069560 0x7f55100699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51052/0 (socket says 172.21.15.92:51052) 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f5515110700 1 -- 172.21.15.92:0/3617542869 learned_addr learned my addr 172.21.15.92:0/3617542869 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:13.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.831+0000 7f551490f700 1 --2- 172.21.15.92:0/3617542869 >> v2:172.21.15.136:3300/0 conn(0x7f5510159890 0x7f551015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5515110700 1 -- 172.21.15.92:0/3617542869 >> v2:172.21.15.136:3300/0 conn(0x7f5510159890 msgr2=0x7f551015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5515110700 1 --2- 172.21.15.92:0/3617542869 >> v2:172.21.15.136:3300/0 conn(0x7f5510159890 0x7f551015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5515110700 1 -- 172.21.15.92:0/3617542869 >> v1:172.21.15.136:6789/0 conn(0x7f5510159c60 legacy=0x7f551015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5515110700 1 -- 172.21.15.92:0/3617542869 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551015e140 con 0x7f5510069560 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5515110700 1 --2- 172.21.15.92:0/3617542869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510069560 0x7f55100699a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f55100d1090 tx=0x7f5508009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c59f274d913ac35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f55077fe700 1 -- 172.21.15.92:0/3617542869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550800e070 con 0x7f5510069560 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f55077fe700 1 -- 172.21.15.92:0/3617542869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550800ba10 con 0x7f5510069560 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f55077fe700 1 -- 172.21.15.92:0/3617542869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55080092e0 con 0x7f5510069560 2023-10-06T03:02:13.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 -- 172.21.15.92:0/3617542869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510069560 msgr2=0x7f55100699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 --2- 172.21.15.92:0/3617542869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510069560 0x7f55100699a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f55100d1090 tx=0x7f5508009840 comp rx=0 tx=0).stop 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 -- 172.21.15.92:0/3617542869 shutdown_connections 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 --2- 172.21.15.92:0/3617542869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510069560 0x7f55100699a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 --2- 172.21.15.92:0/3617542869 >> v2:172.21.15.136:3300/0 conn(0x7f5510159890 0x7f551015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 -- 172.21.15.92:0/3617542869 >> 172.21.15.92:0/3617542869 conn(0x7f5510068e20 msgr2=0x7f55100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 -- 172.21.15.92:0/3617542869 shutdown_connections 2023-10-06T03:02:13.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.832+0000 7f5516b73700 1 -- 172.21.15.92:0/3617542869 wait complete. 2023-10-06T03:02:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.833+0000 7f5516b73700 1 Processor -- start 2023-10-06T03:02:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 -- start start 2023-10-06T03:02:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5510069560 0x7f551016a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5510159890 0x7f551014d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 0x7f55101522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551015d420 con 0x7f5510159c60 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551015b9b0 con 0x7f5510159890 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5516b73700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551015bc90 con 0x7f5510069560 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5515110700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 0x7f55101522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5515110700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 0x7f55101522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51068/0 (socket says 172.21.15.92:51068) 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.834+0000 7f5515110700 1 -- 172.21.15.92:0/2185764723 learned_addr learned my addr 172.21.15.92:0/2185764723 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5507fff700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5510159890 0x7f551014d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f551490f700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5510069560 0x7f551016a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5515110700 1 -- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5510069560 msgr2=0x7f551016a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5515110700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5510069560 0x7f551016a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5515110700 1 -- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5510159890 msgr2=0x7f551014d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5515110700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5510159890 0x7f551014d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5515110700 1 -- 172.21.15.92:0/2185764723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5508009ae0 con 0x7f5510159c60 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5515110700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 0x7f55101522a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5510070e20 tx=0x7f5508017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55080092e0 con 0x7f5510159c60 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508012d00 con 0x7f5510159c60 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551015bf70 con 0x7f5510159c60 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550801d650 con 0x7f5510159c60 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.835+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551015c530 con 0x7f5510159c60 2023-10-06T03:02:13.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.836+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551004c3c0 con 0x7f5510159c60 2023-10-06T03:02:13.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.837+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f550801da00 con 0x7f5510159c60 2023-10-06T03:02:13.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.837+0000 7f5504ff9700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f54fc066f30 0x7f54fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:13.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.837+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55080863d0 con 0x7f5510159c60 2023-10-06T03:02:13.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.837+0000 7f551490f700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f54fc066f30 0x7f54fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:13.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.838+0000 7f551490f700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f54fc066f30 0x7f54fc0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f550c00acb0 tx=0x7f550c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:13.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:13.841+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55080534c0 con 0x7f5510159c60 2023-10-06T03:02:14.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.026+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f551006a5b0 con 0x7f5510159c60 2023-10-06T03:02:14.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.027+0000 7f5504ff9700 1 -- 172.21.15.92:0/2185764723 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f550801dcb0 con 0x7f5510159c60 2023-10-06T03:02:14.028 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:14.028 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f54fc066f30 msgr2=0x7f54fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f54fc066f30 0x7f54fc0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f550c00acb0 tx=0x7f550c008040 comp rx=0 tx=0).stop 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 msgr2=0x7f55101522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 0x7f55101522a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5510070e20 tx=0x7f5508017000 comp rx=0 tx=0).stop 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 shutdown_connections 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f54fc066f30 0x7f54fc0693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5510159c60 0x7f55101522a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5510069560 0x7f551016a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 --2- 172.21.15.92:0/2185764723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5510159890 0x7f551014d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 >> 172.21.15.92:0/2185764723 conn(0x7f5510068e20 msgr2=0x7f55100b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 shutdown_connections 2023-10-06T03:02:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.029+0000 7f5516b73700 1 -- 172.21.15.92:0/2185764723 wait complete. 2023-10-06T03:02:14.042 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:01:40.259324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:01:40.709153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:02:14.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.159+0000 7f3d8eda7700 1 Processor -- start 2023-10-06T03:02:14.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.159+0000 7f3d8eda7700 1 -- start start 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8eda7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d88155eb0 0x7f3d8806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8eda7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d880c70b0 0x7f3d88069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8eda7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d88069cb0 0x7f3d880ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8eda7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8806a130 con 0x7f3d880c70b0 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8eda7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8806eb90 con 0x7f3d88069cb0 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8eda7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8806ee70 con 0x7f3d88155eb0 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d7ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d880c70b0 0x7f3d88069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d7ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d880c70b0 0x7f3d88069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51070/0 (socket says 172.21.15.92:51070) 2023-10-06T03:02:14.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d7ffff700 1 -- 172.21.15.92:0/4229496731 learned_addr learned my addr 172.21.15.92:0/4229496731 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8d344700 1 --2- 172.21.15.92:0/4229496731 >> v2:172.21.15.136:3300/0 conn(0x7f3d88069cb0 0x7f3d880ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.160+0000 7f3d8cb43700 1 --2- 172.21.15.92:0/4229496731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d88155eb0 0x7f3d8806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7ffff700 1 -- 172.21.15.92:0/4229496731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d88155eb0 msgr2=0x7f3d8806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7ffff700 1 --2- 172.21.15.92:0/4229496731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d88155eb0 0x7f3d8806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7ffff700 1 -- 172.21.15.92:0/4229496731 >> v2:172.21.15.136:3300/0 conn(0x7f3d88069cb0 msgr2=0x7f3d880ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7ffff700 1 --2- 172.21.15.92:0/4229496731 >> v2:172.21.15.136:3300/0 conn(0x7f3d88069cb0 0x7f3d880ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7ffff700 1 -- 172.21.15.92:0/4229496731 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8806f150 con 0x7f3d880c70b0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7ffff700 1 --2- 172.21.15.92:0/4229496731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d880c70b0 0x7f3d88069770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d880c7740 tx=0x7f3d7800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499fbee586ddf0f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7f7fe700 1 -- 172.21.15.92:0/4229496731 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800db40 con 0x7f3d880c70b0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7f7fe700 1 -- 172.21.15.92:0/4229496731 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800dca0 con 0x7f3d880c70b0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d7f7fe700 1 -- 172.21.15.92:0/4229496731 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780196b0 con 0x7f3d880c70b0 2023-10-06T03:02:14.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d8eda7700 1 -- 172.21.15.92:0/4229496731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d880c70b0 msgr2=0x7f3d88069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/4229496731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d880c70b0 0x7f3d88069770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d880c7740 tx=0x7f3d7800b4f0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d8eda7700 1 -- 172.21.15.92:0/4229496731 shutdown_connections 2023-10-06T03:02:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/4229496731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d88155eb0 0x7f3d8806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/4229496731 >> v2:172.21.15.136:3300/0 conn(0x7f3d88069cb0 0x7f3d880ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.161+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/4229496731 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d880c70b0 0x7f3d88069770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.162+0000 7f3d8eda7700 1 -- 172.21.15.92:0/4229496731 >> 172.21.15.92:0/4229496731 conn(0x7f3d88068e10 msgr2=0x7f3d880cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.162+0000 7f3d8eda7700 1 -- 172.21.15.92:0/4229496731 shutdown_connections 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.162+0000 7f3d8eda7700 1 -- 172.21.15.92:0/4229496731 wait complete. 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 Processor -- start 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 -- start start 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d88069cb0 0x7f3d880c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d880c70b0 0x7f3d880c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 0x7f3d880c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880c4c10 con 0x7f3d88155eb0 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880c5e30 con 0x7f3d88069cb0 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.163+0000 7f3d8eda7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880c6110 con 0x7f3d880c70b0 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 0x7f3d880c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 0x7f3d880c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51076/0 (socket says 172.21.15.92:51076) 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 -- 172.21.15.92:0/3996161112 learned_addr learned my addr 172.21.15.92:0/3996161112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8cb43700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d88069cb0 0x7f3d880c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d7ffff700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d880c70b0 0x7f3d880c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 -- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d880c70b0 msgr2=0x7f3d880c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d880c70b0 0x7f3d880c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 -- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d88069cb0 msgr2=0x7f3d880c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d88069cb0 0x7f3d880c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 -- 172.21.15.92:0/3996161112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7800b790 con 0x7f3d88155eb0 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8d344700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 0x7f3d880c5800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000ea50 tx=0x7f3d800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000ee90 con 0x7f3d88155eb0 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8000c850 con 0x7f3d88155eb0 2023-10-06T03:02:14.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000cc10 con 0x7f3d88155eb0 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d880c63f0 con 0x7f3d88155eb0 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.164+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d880c69b0 con 0x7f3d88155eb0 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.165+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8804c3c0 con 0x7f3d88155eb0 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.166+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d80013470 con 0x7f3d88155eb0 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.166+0000 7f3d7cff9700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d74066f30 0x7f3d740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.166+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d80084850 con 0x7f3d88155eb0 2023-10-06T03:02:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.166+0000 7f3d8cb43700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d74066f30 0x7f3d740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.166+0000 7f3d8cb43700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d74066f30 0x7f3d740693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009fd0 tx=0x7f3d8400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.170+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d80051960 con 0x7f3d88155eb0 2023-10-06T03:02:14.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.357+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d8806f430 con 0x7f3d88155eb0 2023-10-06T03:02:14.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.357+0000 7f3d7cff9700 1 -- 172.21.15.92:0/3996161112 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d80054f80 con 0x7f3d88155eb0 2023-10-06T03:02:14.358 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.359+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d74066f30 msgr2=0x7f3d740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.359+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d74066f30 0x7f3d740693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009fd0 tx=0x7f3d8400b040 comp rx=0 tx=0).stop 2023-10-06T03:02:14.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.359+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 msgr2=0x7f3d880c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.359+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 0x7f3d880c5800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000ea50 tx=0x7f3d800091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.359+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 shutdown_connections 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.359+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d74066f30 0x7f3d740693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.360+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d88155eb0 0x7f3d880c5800 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.360+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d88069cb0 0x7f3d880c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.360+0000 7f3d8eda7700 1 --2- 172.21.15.92:0/3996161112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d880c70b0 0x7f3d880c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.360+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 >> 172.21.15.92:0/3996161112 conn(0x7f3d88068e10 msgr2=0x7f3d880af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.360+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 shutdown_connections 2023-10-06T03:02:14.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.360+0000 7f3d8eda7700 1 -- 172.21.15.92:0/3996161112 wait complete. 2023-10-06T03:02:14.372 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:14.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.489+0000 7f3812c54700 1 Processor -- start 2023-10-06T03:02:14.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.489+0000 7f3812c54700 1 -- start start 2023-10-06T03:02:14.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3812c54700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f380c06f060 0x7f380c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3812c54700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c06ebd0 con 0x7f380c0cb1a0 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3812c54700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c15aa70 con 0x7f380c06f4b0 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3812c54700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380c15b2e0 con 0x7f380c06f060 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3811c52700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f380c0cb1a0 0x7f380c15a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46630/0 (socket says 172.21.15.92:46630) 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3811c52700 1 -- 172.21.15.92:0/3409647238 learned_addr learned my addr 172.21.15.92:0/3409647238 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768659724 0 0) 0x7f380c06ebd0 con 0x7f380c0cb1a0 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f8003570 con 0x7f380c0cb1a0 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3811451700 1 --2- 172.21.15.92:0/3409647238 >> v2:172.21.15.136:3300/0 conn(0x7f380c06f060 0x7f380c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.490+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2124365931 0 0) 0x7f37f8003570 con 0x7f380c0cb1a0 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 >> v2:172.21.15.136:3300/0 conn(0x7f380c06f060 msgr2=0x7f380c0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 --2- 172.21.15.92:0/3409647238 >> v2:172.21.15.136:3300/0 conn(0x7f380c06f060 0x7f380c0ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 >> v1:172.21.15.136:6789/0 conn(0x7f380c06f4b0 legacy=0x7f380c0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c15ad20 con 0x7f380c0cb1a0 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f38080017f0 con 0x7f380c0cb1a0 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 >> v1:172.21.15.92:6789/0 conn(0x7f380c0cb1a0 legacy=0x7f380c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f37f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37f8004810 0x7f37f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37f8005230 0x7f37f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c15b2e0 con 0x7f37f8003a60 2023-10-06T03:02:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f800c160 con 0x7f37f8005230 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f800c320 con 0x7f37f8004810 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f37f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811451700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37f8004810 0x7f37f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3812453700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37f8005230 0x7f37f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 -- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37f8004810 msgr2=0x7f37f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37f8004810 0x7f37f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 -- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37f8005230 msgr2=0x7f37f800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37f8005230 0x7f37f800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 -- 172.21.15.92:0/3409647238 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8003570 con 0x7f37f8003a60 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3812453700 1 -- 172.21.15.92:0/3409647238 reap_dead start 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811451700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37f8004810 0x7f37f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3811c52700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f37f8003e50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f380800a940 tx=0x7f380c0c8be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af35ac419cac1058 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080070d0 con 0x7f37f8003a60 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3808007230 con 0x7f37f8003a60 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.491+0000 7f3810c50700 1 -- 172.21.15.92:0/3409647238 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080075f0 con 0x7f37f8003a60 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 -- 172.21.15.92:0/3409647238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 msgr2=0x7f37f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f37f8003e50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f380800a940 tx=0x7f380c0c8be0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 -- 172.21.15.92:0/3409647238 shutdown_connections 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 --2- 172.21.15.92:0/3409647238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f37f8003e50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 -- 172.21.15.92:0/3409647238 >> 172.21.15.92:0/3409647238 conn(0x7f380c0cc1e0 msgr2=0x7f380c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 -- 172.21.15.92:0/3409647238 shutdown_connections 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.492+0000 7f3812c54700 1 -- 172.21.15.92:0/3409647238 wait complete. 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.493+0000 7f3812c54700 1 Processor -- start 2023-10-06T03:02:14.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.493+0000 7f3812c54700 1 -- start start 2023-10-06T03:02:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f380c14a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380c0c5700 0x7f380c0c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380c14d930 0x7f380c14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c157e40 con 0x7f37f8003a60 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c14e2e0 con 0x7f380c0c5700 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c1672c0 con 0x7f380c14d930 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f380c14a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f380c14a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51084/0 (socket says 172.21.15.92:51084) 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 -- 172.21.15.92:0/3329274330 learned_addr learned my addr 172.21.15.92:0/3329274330 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 -- 172.21.15.92:0/3329274330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380c14d930 msgr2=0x7f380c14dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380c14d930 0x7f380c14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 -- 172.21.15.92:0/3329274330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380c0c5700 msgr2=0x7f380c0c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380c0c5700 0x7f380c0c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 -- 172.21.15.92:0/3329274330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c14adb0 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3811c52700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f380c14a870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c130 tx=0x7f3808015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808022070 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38080017b0 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c0c5be0 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.494+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808001b90 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.495+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c0c61a0 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.495+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380c158690 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.496+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f380801d030 con 0x7f37f8003a60 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.496+0000 7f3801ffb700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37f0066f30 0x7f37f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.496+0000 7f3811451700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37f0066f30 0x7f37f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.497+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3808088140 con 0x7f37f8003a60 2023-10-06T03:02:14.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.497+0000 7f3811451700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37f0066f30 0x7f37f00693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37fc006dc0 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.501+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3808055250 con 0x7f37f8003a60 2023-10-06T03:02:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.680+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f380c06c4b0 con 0x7f37f8003a60 2023-10-06T03:02:14.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.680+0000 7f3801ffb700 1 -- 172.21.15.92:0/3329274330 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38080075b0 con 0x7f37f8003a60 2023-10-06T03:02:14.681 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37f0066f30 msgr2=0x7f37f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37f0066f30 0x7f37f00693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37fc006dc0 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 msgr2=0x7f380c14a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f380c14a870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c130 tx=0x7f3808015000 comp rx=0 tx=0).stop 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 shutdown_connections 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37f0066f30 0x7f37f00693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37f8003a60 0x7f380c14a870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380c14d930 0x7f380c14dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 --2- 172.21.15.92:0/3329274330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380c0c5700 0x7f380c0c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 >> 172.21.15.92:0/3329274330 conn(0x7f380c0cc1e0 msgr2=0x7f380c15a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:14.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 shutdown_connections 2023-10-06T03:02:14.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.683+0000 7f3812c54700 1 -- 172.21.15.92:0/3329274330 wait complete. 2023-10-06T03:02:14.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:02:14.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.811+0000 7fc17cdd1700 1 Processor -- start 2023-10-06T03:02:14.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.811+0000 7fc17cdd1700 1 -- start start 2023-10-06T03:02:14.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc17cdd1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc178069560 0x7fc1780699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc17cdd1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc178069ee0 con 0x7fc178159890 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc17cdd1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17815a160 con 0x7fc178159c60 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc17cdd1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780d13a0 con 0x7fc178069560 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc17659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc178159890 0x7fc17815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46636/0 (socket says 172.21.15.92:46636) 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc17659c700 1 -- 172.21.15.92:0/3809139111 learned_addr learned my addr 172.21.15.92:0/3809139111 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc176d9d700 1 --2- 172.21.15.92:0/3809139111 >> v2:172.21.15.136:3300/0 conn(0x7fc178069560 0x7fc1780699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060398701 0 0) 0x7fc178069ee0 con 0x7fc178159890 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.812+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc164003570 con 0x7fc178159890 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2870200867 0 0) 0x7fc164003570 con 0x7fc178159890 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 >> v2:172.21.15.136:3300/0 conn(0x7fc178069560 msgr2=0x7fc1780699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 --2- 172.21.15.92:0/3809139111 >> v2:172.21.15.136:3300/0 conn(0x7fc178069560 0x7fc1780699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 >> v1:172.21.15.136:6789/0 conn(0x7fc178159c60 legacy=0x7fc17815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17815e140 con 0x7fc178159890 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc168001470 con 0x7fc178159890 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 >> v1:172.21.15.92:6789/0 conn(0x7fc178159890 legacy=0x7fc17815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164003a60 0x7fc164003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc164004810 0x7fc164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1640051c0 0x7fc16400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780d13a0 con 0x7fc164003a60 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16400c0d0 con 0x7fc1640051c0 2023-10-06T03:02:14.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16400c290 con 0x7fc164004810 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164003a60 0x7fc164003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc16dd9b700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc164004810 0x7fc164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc176d9d700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1640051c0 0x7fc16400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 -- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc164004810 msgr2=0x7fc164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc164004810 0x7fc164004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 -- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1640051c0 msgr2=0x7fc16400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1640051c0 0x7fc16400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 -- 172.21.15.92:0/3809139111 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164003570 con 0x7fc164003a60 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc176d9d700 1 -- 172.21.15.92:0/3809139111 reap_dead start 2023-10-06T03:02:14.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc17659c700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164003a60 0x7fc164003e70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c3c0 tx=0x7fc168001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6b97ddb1d2ea1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.813+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16800cc10 con 0x7fc164003a60 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16800cd70 con 0x7fc164003a60 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc175d9b700 1 -- 172.21.15.92:0/3809139111 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc168005b50 con 0x7fc164003a60 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 -- 172.21.15.92:0/3809139111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164003a60 msgr2=0x7fc164003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164003a60 0x7fc164003e70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c3c0 tx=0x7fc168001e50 comp rx=0 tx=0).stop 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 -- 172.21.15.92:0/3809139111 shutdown_connections 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/3809139111 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc164003a60 0x7fc164003e70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 -- 172.21.15.92:0/3809139111 >> 172.21.15.92:0/3809139111 conn(0x7fc178068e20 msgr2=0x7fc1780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 -- 172.21.15.92:0/3809139111 shutdown_connections 2023-10-06T03:02:14.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.814+0000 7fc17cdd1700 1 -- 172.21.15.92:0/3809139111 wait complete. 2023-10-06T03:02:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.815+0000 7fc17cdd1700 1 Processor -- start 2023-10-06T03:02:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.815+0000 7fc17cdd1700 1 -- start start 2023-10-06T03:02:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17cdd1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc164003a60 0x7fc178169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17cdd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 0x7fc17815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17cdd1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17815d540 0x7fc17815f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17cdd1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780cd960 con 0x7fc1780cf840 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17cdd1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780cd0c0 con 0x7fc164003a60 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17cdd1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780cfdb0 con 0x7fc17815d540 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 0x7fc17815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 0x7fc17815eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51098/0 (socket says 172.21.15.92:51098) 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 -- 172.21.15.92:0/4168692887 learned_addr learned my addr 172.21.15.92:0/4168692887 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc17659c700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc164003a60 0x7fc178169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc176d9d700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17815d540 0x7fc17815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 -- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17815d540 msgr2=0x7fc17815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17815d540 0x7fc17815f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 -- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc164003a60 msgr2=0x7fc178169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc164003a60 0x7fc178169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.816+0000 7fc16dd9b700 1 -- 172.21.15.92:0/4168692887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780cd310 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.817+0000 7fc16dd9b700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 0x7fc17815eb00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc160002980 tx=0x7fc160007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.817+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000f7f0 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.817+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000f950 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.817+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1780cd5f0 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.817+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000fcf0 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.817+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17815bce0 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.818+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17804c3c0 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.818+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc160013600 con 0x7fc1780cf840 2023-10-06T03:02:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.818+0000 7fc16effd700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc158066f30 0x7fc1580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.819+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc160084b10 con 0x7fc1780cf840 2023-10-06T03:02:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.819+0000 7fc17659c700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc158066f30 0x7fc1580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.819+0000 7fc17659c700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc158066f30 0x7fc1580693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc168007cd0 tx=0x7fc168015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:14.823+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc160051c20 con 0x7fc1780cf840 2023-10-06T03:02:15.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.006+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1780cd780 con 0x7fc1780cf840 2023-10-06T03:02:15.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.007+0000 7fc16effd700 1 -- 172.21.15.92:0/4168692887 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc160055240 con 0x7fc1780cf840 2023-10-06T03:02:15.007 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:02:15.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.008+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc158066f30 msgr2=0x7fc1580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc158066f30 0x7fc1580693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc168007cd0 tx=0x7fc168015000 comp rx=0 tx=0).stop 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 msgr2=0x7fc17815eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 0x7fc17815eb00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc160002980 tx=0x7fc160007d30 comp rx=0 tx=0).stop 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 shutdown_connections 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc158066f30 0x7fc1580693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17815d540 0x7fc17815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc164003a60 0x7fc178169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 --2- 172.21.15.92:0/4168692887 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1780cf840 0x7fc17815eb00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 >> 172.21.15.92:0/4168692887 conn(0x7fc178068e20 msgr2=0x7fc17814d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:15.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 shutdown_connections 2023-10-06T03:02:15.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.009+0000 7fc17cdd1700 1 -- 172.21.15.92:0/4168692887 wait complete. 2023-10-06T03:02:15.021 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:02:15.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.137+0000 7fe2ae96a700 1 Processor -- start 2023-10-06T03:02:15.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.137+0000 7fe2ae96a700 1 -- start start 2023-10-06T03:02:15.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.138+0000 7fe2ae96a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe2a806bcd0 0x7fe2a80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.138+0000 7fe2ae96a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 0x7fe2a815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.138+0000 7fe2ae96a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a815d7e0 con 0x7fe2a806c0a0 2023-10-06T03:02:15.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.138+0000 7fe2ae96a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a815dac0 con 0x7fe2a806bcd0 2023-10-06T03:02:15.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.138+0000 7fe2ae96a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a815aa30 con 0x7fe2a815df50 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 0x7fe2a815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 0x7fe2a815e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:54488/0 (socket says 172.21.15.92:54488) 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2a7fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe2a806bcd0 0x7fe2a80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 -- 172.21.15.92:0/2336423569 learned_addr learned my addr 172.21.15.92:0/2336423569 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 -- 172.21.15.92:0/2336423569 >> v2:172.21.15.136:3300/0 conn(0x7fe2a806bcd0 msgr2=0x7fe2a80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 --2- 172.21.15.92:0/2336423569 >> v2:172.21.15.136:3300/0 conn(0x7fe2a806bcd0 0x7fe2a80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 -- 172.21.15.92:0/2336423569 >> v1:172.21.15.136:6789/0 conn(0x7fe2a806c0a0 legacy=0x7fe2a815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 -- 172.21.15.92:0/2336423569 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a815ad10 con 0x7fe2a815df50 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2acf07700 1 --2- 172.21.15.92:0/2336423569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 0x7fe2a815e3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009f20 tx=0x7fe2a0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7741016c3c86b16f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2a6ffd700 1 -- 172.21.15.92:0/2336423569 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000e070 con 0x7fe2a815df50 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2a6ffd700 1 -- 172.21.15.92:0/2336423569 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a000b9d0 con 0x7fe2a815df50 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.139+0000 7fe2a6ffd700 1 -- 172.21.15.92:0/2336423569 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a00092e0 con 0x7fe2a815df50 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 -- 172.21.15.92:0/2336423569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 msgr2=0x7fe2a815e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/2336423569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 0x7fe2a815e3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009f20 tx=0x7fe2a0009840 comp rx=0 tx=0).stop 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 -- 172.21.15.92:0/2336423569 shutdown_connections 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/2336423569 >> v2:172.21.15.136:3300/0 conn(0x7fe2a806bcd0 0x7fe2a80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/2336423569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a815df50 0x7fe2a815e3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 -- 172.21.15.92:0/2336423569 >> 172.21.15.92:0/2336423569 conn(0x7fe2a80ce600 msgr2=0x7fe2a80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:15.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 -- 172.21.15.92:0/2336423569 shutdown_connections 2023-10-06T03:02:15.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.140+0000 7fe2ae96a700 1 -- 172.21.15.92:0/2336423569 wait complete. 2023-10-06T03:02:15.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.141+0000 7fe2ae96a700 1 Processor -- start 2023-10-06T03:02:15.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.141+0000 7fe2ae96a700 1 -- start start 2023-10-06T03:02:15.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2ae96a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2a806bcd0 0x7fe2a8169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2ae96a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a806c0a0 0x7fe2a8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2ae96a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 0x7fe2a80c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2ae96a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80c56b0 con 0x7fe2a815df50 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2ae96a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80c5990 con 0x7fe2a806bcd0 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2ae96a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a816a6c0 con 0x7fe2a806c0a0 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 0x7fe2a80c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 0x7fe2a80c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51108/0 (socket says 172.21.15.92:51108) 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 -- 172.21.15.92:0/621271687 learned_addr learned my addr 172.21.15.92:0/621271687 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2a7fff700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2a806bcd0 0x7fe2a8169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2a77fe700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a806c0a0 0x7fe2a8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 -- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a806c0a0 msgr2=0x7fe2a8154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a806c0a0 0x7fe2a8154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 -- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2a806bcd0 msgr2=0x7fe2a8169e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2a806bcd0 0x7fe2a8169e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.142+0000 7fe2acf07700 1 -- 172.21.15.92:0/621271687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a0009ae0 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.143+0000 7fe2acf07700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 0x7fe2a80c5e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0007710 tx=0x7fe2a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.143+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0007b40 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.143+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a0007cc0 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.143+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a816a9a0 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.143+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a001d670 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.143+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a816a360 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.144+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a804c3c0 con 0x7fe2a815df50 2023-10-06T03:02:15.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.144+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2a0003050 con 0x7fe2a815df50 2023-10-06T03:02:15.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.144+0000 7fe28ffff700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294066fb0 0x7fe294069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.145+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe2a00862e0 con 0x7fe2a815df50 2023-10-06T03:02:15.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.145+0000 7fe2a7fff700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294066fb0 0x7fe294069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.145+0000 7fe2a7fff700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294066fb0 0x7fe294069460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe298009d10 tx=0x7fe2980091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.149+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2a0053350 con 0x7fe2a815df50 2023-10-06T03:02:15.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.330+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2a815aff0 con 0x7fe2a815df50 2023-10-06T03:02:15.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.330+0000 7fe28ffff700 1 -- 172.21.15.92:0/621271687 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2a0056970 con 0x7fe2a815df50 2023-10-06T03:02:15.331 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:02:15.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294066fb0 msgr2=0x7fe294069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294066fb0 0x7fe294069460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe298009d10 tx=0x7fe2980091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 msgr2=0x7fe2a80c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 0x7fe2a80c5e20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0007710 tx=0x7fe2a0018000 comp rx=0 tx=0).stop 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 shutdown_connections 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294066fb0 0x7fe294069460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2a815df50 0x7fe2a80c5e20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2a806bcd0 0x7fe2a8169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 --2- 172.21.15.92:0/621271687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2a806c0a0 0x7fe2a8154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 >> 172.21.15.92:0/621271687 conn(0x7fe2a80ce600 msgr2=0x7fe2a80c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:15.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 shutdown_connections 2023-10-06T03:02:15.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.332+0000 7fe2ae96a700 1 -- 172.21.15.92:0/621271687 wait complete. 2023-10-06T03:02:15.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:15.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.459+0000 7f7fea372700 1 Processor -- start 2023-10-06T03:02:15.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.459+0000 7f7fea372700 1 -- start start 2023-10-06T03:02:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fea372700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 0x7f7fe415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fea372700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7fe415db10 0x7f7fe4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fea372700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe415a160 con 0x7f7fe4159890 2023-10-06T03:02:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fea372700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe40d13a0 con 0x7f7fe415db10 2023-10-06T03:02:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fea372700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe415e140 con 0x7f7fe4159c60 2023-10-06T03:02:15.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 0x7f7fe415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fe890f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7fe415db10 0x7f7fe4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fe37fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 0x7f7fe415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:54506/0 (socket says 172.21.15.92:54506) 2023-10-06T03:02:15.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.460+0000 7f7fe37fe700 1 -- 172.21.15.92:0/1216651728 learned_addr learned my addr 172.21.15.92:0/1216651728 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:15.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe37fe700 1 -- 172.21.15.92:0/1216651728 >> v2:172.21.15.136:3300/0 conn(0x7f7fe415db10 msgr2=0x7f7fe4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:15.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe37fe700 1 --2- 172.21.15.92:0/1216651728 >> v2:172.21.15.136:3300/0 conn(0x7f7fe415db10 0x7f7fe4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe37fe700 1 -- 172.21.15.92:0/1216651728 >> v1:172.21.15.136:6789/0 conn(0x7f7fe4159890 legacy=0x7f7fe415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe37fe700 1 -- 172.21.15.92:0/1216651728 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe415a3d0 con 0x7f7fe4159c60 2023-10-06T03:02:15.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe37fe700 1 --2- 172.21.15.92:0/1216651728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 0x7f7fe415d5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80099b0 tx=0x7f7fd8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f9e8bb0c5515cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe2ffd700 1 -- 172.21.15.92:0/1216651728 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8009ea0 con 0x7f7fe4159c60 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe2ffd700 1 -- 172.21.15.92:0/1216651728 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd800b860 con 0x7f7fe4159c60 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.461+0000 7f7fe2ffd700 1 -- 172.21.15.92:0/1216651728 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800bc00 con 0x7f7fe4159c60 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 -- 172.21.15.92:0/1216651728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 msgr2=0x7f7fe415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 --2- 172.21.15.92:0/1216651728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 0x7f7fe415d5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80099b0 tx=0x7f7fd8009250 comp rx=0 tx=0).stop 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 -- 172.21.15.92:0/1216651728 shutdown_connections 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 --2- 172.21.15.92:0/1216651728 >> v2:172.21.15.136:3300/0 conn(0x7f7fe415db10 0x7f7fe4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 --2- 172.21.15.92:0/1216651728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159c60 0x7f7fe415d5d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 -- 172.21.15.92:0/1216651728 >> 172.21.15.92:0/1216651728 conn(0x7f7fe4068e20 msgr2=0x7f7fe40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 -- 172.21.15.92:0/1216651728 shutdown_connections 2023-10-06T03:02:15.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.462+0000 7f7fea372700 1 -- 172.21.15.92:0/1216651728 wait complete. 2023-10-06T03:02:15.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.463+0000 7f7fea372700 1 Processor -- start 2023-10-06T03:02:15.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.463+0000 7f7fea372700 1 -- start start 2023-10-06T03:02:15.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159890 0x7f7fe4138060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe4159c60 0x7f7fe416d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 0x7f7fe415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe414f1a0 con 0x7f7fe415db10 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe416dec0 con 0x7f7fe4159c60 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe416e1a0 con 0x7f7fe4159890 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 0x7f7fe415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 0x7f7fe415bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51124/0 (socket says 172.21.15.92:51124) 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 -- 172.21.15.92:0/4292543559 learned_addr learned my addr 172.21.15.92:0/4292543559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe3fff700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159890 0x7f7fe4138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe37fe700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe4159c60 0x7f7fe416d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 -- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159890 msgr2=0x7f7fe4138060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159890 0x7f7fe4138060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 -- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe4159c60 msgr2=0x7f7fe416d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe4159c60 0x7f7fe416d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 -- 172.21.15.92:0/4292543559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd80094f0 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fe890f700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 0x7f7fe415bac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009df0 tx=0x7f7fdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00caa0 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc00f040 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe414ee30 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc00ce70 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.464+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe414ea70 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.465+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe414ec60 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.466+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fdc014690 con 0x7f7fe415db10 2023-10-06T03:02:15.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.466+0000 7f7fcbfff700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fd0066f30 0x7f7fd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.466+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7fdc084ea0 con 0x7f7fe415db10 2023-10-06T03:02:15.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.466+0000 7f7fe3fff700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fd0066f30 0x7f7fd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.466+0000 7f7fe3fff700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fd0066f30 0x7f7fd00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40039f0 tx=0x7f7fd4007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.470+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fdc055a40 con 0x7f7fe415db10 2023-10-06T03:02:15.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.700+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fe404c3c0 con 0x7f7fe415db10 2023-10-06T03:02:15.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.701+0000 7f7fcbfff700 1 -- 172.21.15.92:0/4292543559 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f7fdc0555d0 con 0x7f7fe415db10 2023-10-06T03:02:15.702 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:15.702 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:02:15.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fd0066f30 msgr2=0x7f7fd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fd0066f30 0x7f7fd00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40039f0 tx=0x7f7fd4007ec0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 msgr2=0x7f7fe415bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 0x7f7fe415bac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009df0 tx=0x7f7fdc00b040 comp rx=0 tx=0).stop 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 shutdown_connections 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fd0066f30 0x7f7fd00693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fe415db10 0x7f7fe415bac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fe4159890 0x7f7fe4138060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 --2- 172.21.15.92:0/4292543559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fe4159c60 0x7f7fe416d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 >> 172.21.15.92:0/4292543559 conn(0x7f7fe4068e20 msgr2=0x7f7fe415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 shutdown_connections 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.703+0000 7f7fea372700 1 -- 172.21.15.92:0/4292543559 wait complete. 2023-10-06T03:02:15.705 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:02:15.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:15.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.830+0000 7efc7eb12700 1 Processor -- start 2023-10-06T03:02:15.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.830+0000 7efc7eb12700 1 -- start start 2023-10-06T03:02:15.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7eb12700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc78069560 0x7efc780699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7eb12700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc78069ee0 con 0x7efc78159c60 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7eb12700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc7815a160 con 0x7efc78159890 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7eb12700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc780d13a0 con 0x7efc78069560 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7d0af700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc78069560 0x7efc780699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc6ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efc78159c60 0x7efc7815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46652/0 (socket says 172.21.15.92:46652) 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc6ffff700 1 -- 172.21.15.92:0/1523719044 learned_addr learned my addr 172.21.15.92:0/1523719044 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7d0af700 1 -- 172.21.15.92:0/1523719044 >> v1:172.21.15.136:6789/0 conn(0x7efc78159890 legacy=0x7efc7815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7d0af700 1 -- 172.21.15.92:0/1523719044 >> v1:172.21.15.92:6789/0 conn(0x7efc78159c60 legacy=0x7efc7815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc6f7fe700 1 -- 172.21.15.92:0/1523719044 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081778408 0 0) 0x7efc78069ee0 con 0x7efc78159c60 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7d0af700 1 -- 172.21.15.92:0/1523719044 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7815e140 con 0x7efc78069560 2023-10-06T03:02:15.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc7d0af700 1 --2- 172.21.15.92:0/1523719044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc780699a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efc700098e0 tx=0x7efc700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5c67503998bc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc6f7fe700 1 -- 172.21.15.92:0/1523719044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70009e80 con 0x7efc78069560 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc6f7fe700 1 -- 172.21.15.92:0/1523719044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7000b820 con 0x7efc78069560 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.831+0000 7efc6f7fe700 1 -- 172.21.15.92:0/1523719044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000bba0 con 0x7efc78069560 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 -- 172.21.15.92:0/1523719044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 msgr2=0x7efc780699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1523719044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc780699a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efc700098e0 tx=0x7efc700091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 -- 172.21.15.92:0/1523719044 shutdown_connections 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1523719044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc780699a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 -- 172.21.15.92:0/1523719044 >> 172.21.15.92:0/1523719044 conn(0x7efc78068e20 msgr2=0x7efc780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 -- 172.21.15.92:0/1523719044 shutdown_connections 2023-10-06T03:02:15.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.832+0000 7efc7eb12700 1 -- 172.21.15.92:0/1523719044 wait complete. 2023-10-06T03:02:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.833+0000 7efc7eb12700 1 Processor -- start 2023-10-06T03:02:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.833+0000 7efc7eb12700 1 -- start start 2023-10-06T03:02:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc7814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc78159890 0x7efc78153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc78159c60 0x7efc78153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7815c4e0 con 0x7efc78069560 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780ce170 con 0x7efc78159890 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc780ce450 con 0x7efc78159c60 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc7814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc7814d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51132/0 (socket says 172.21.15.92:51132) 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 -- 172.21.15.92:0/1332029968 learned_addr learned my addr 172.21.15.92:0/1332029968 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7d0af700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc78159c60 0x7efc78153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc6ffff700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc78159890 0x7efc78153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 -- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc78159c60 msgr2=0x7efc78153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc78159c60 0x7efc78153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 -- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc78159890 msgr2=0x7efc78153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc78159890 0x7efc78153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 -- 172.21.15.92:0/1332029968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70009420 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7c8ae700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc7814d510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c950 tx=0x7efc74002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7400a7e0 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7400a960 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7400ad20 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.834+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc780ce730 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.835+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc780cd5c0 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.836+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7804c3c0 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.836+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc74012630 con 0x7efc78069560 2023-10-06T03:02:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.836+0000 7efc6cff9700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc64066f30 0x7efc640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.836+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efc74084a70 con 0x7efc78069560 2023-10-06T03:02:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.836+0000 7efc6ffff700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc64066f30 0x7efc640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:15.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.837+0000 7efc6ffff700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc64066f30 0x7efc640693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc7814da50 tx=0x7efc68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:15.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:15.840+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc74051c30 con 0x7efc78069560 2023-10-06T03:02:16.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.026+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc7815c2b0 con 0x7efc78069560 2023-10-06T03:02:16.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.027+0000 7efc6cff9700 1 -- 172.21.15.92:0/1332029968 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7efc7406be60 con 0x7efc78069560 2023-10-06T03:02:16.028 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:16.028 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.029+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc64066f30 msgr2=0x7efc640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.029+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc64066f30 0x7efc640693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc7814da50 tx=0x7efc68009040 comp rx=0 tx=0).stop 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 msgr2=0x7efc7814d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc7814d510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c950 tx=0x7efc74002760 comp rx=0 tx=0).stop 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 shutdown_connections 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc64066f30 0x7efc640693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc78069560 0x7efc7814d510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc78159c60 0x7efc78153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 --2- 172.21.15.92:0/1332029968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc78159890 0x7efc78153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 >> 172.21.15.92:0/1332029968 conn(0x7efc78068e20 msgr2=0x7efc78157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:16.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 shutdown_connections 2023-10-06T03:02:16.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.030+0000 7efc7eb12700 1 -- 172.21.15.92:0/1332029968 wait complete. 2023-10-06T03:02:16.043 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:02:16.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.156+0000 7f97d5b7b700 1 Processor -- start 2023-10-06T03:02:16.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.156+0000 7f97d5b7b700 1 -- start start 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97d5b7b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97d0159c60 0x7f97d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97d5b7b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015a160 con 0x7f97d0159890 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97d5b7b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d00d13a0 con 0x7f97d015db10 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97d5b7b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d015df50 con 0x7f97d0159c60 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97cf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f97d0159890 0x7f97d015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46662/0 (socket says 172.21.15.92:46662) 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97cf7fe700 1 -- 172.21.15.92:0/3163968999 learned_addr learned my addr 172.21.15.92:0/3163968999 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:16.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668586318 0 0) 0x7f97d015a160 con 0x7f97d0159890 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ceffd700 1 --2- 172.21.15.92:0/3163968999 >> v2:172.21.15.136:3300/0 conn(0x7f97d0159c60 0x7f97d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b8003570 con 0x7f97d0159890 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711077263 0 0) 0x7f97b8003570 con 0x7f97d0159890 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 >> v2:172.21.15.136:3300/0 conn(0x7f97d0159c60 msgr2=0x7f97d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 --2- 172.21.15.92:0/3163968999 >> v2:172.21.15.136:3300/0 conn(0x7f97d0159c60 0x7f97d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 >> v1:172.21.15.136:6789/0 conn(0x7f97d015db10 legacy=0x7f97d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d015e140 con 0x7f97d0159890 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f97c00017f0 con 0x7f97d0159890 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 >> v1:172.21.15.92:6789/0 conn(0x7f97d0159890 legacy=0x7f97d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.157+0000 7f97ce7fc700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97b8003a60 0x7f97b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b8004810 0x7f97b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b80051c0 0x7f97b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b800c0a0 con 0x7f97b8003a60 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b800c210 con 0x7f97b80051c0 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b800c3d0 con 0x7f97b8004810 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97b8003a60 0x7f97b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ceffd700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b8004810 0x7f97b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cffff700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b80051c0 0x7f97b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 -- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b8004810 msgr2=0x7f97b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b8004810 0x7f97b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 -- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b80051c0 msgr2=0x7f97b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b80051c0 0x7f97b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cffff700 1 -- 172.21.15.92:0/3163968999 reap_dead start 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 -- 172.21.15.92:0/3163968999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8003570 con 0x7f97b8003a60 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ceffd700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b8004810 0x7f97b8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97b8003a60 0x7f97b8003e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f97c000a930 tx=0x7f97c00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93005f73aef61437 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00070d0 con 0x7f97b8003a60 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0007230 con 0x7f97b8003a60 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.158+0000 7f97ce7fc700 1 -- 172.21.15.92:0/3163968999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00075d0 con 0x7f97b8003a60 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 -- 172.21.15.92:0/3163968999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97b8003a60 msgr2=0x7f97b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97b8003a60 0x7f97b8003e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f97c000a930 tx=0x7f97c00021d0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 -- 172.21.15.92:0/3163968999 shutdown_connections 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/3163968999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97b8003a60 0x7f97b8003e70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 -- 172.21.15.92:0/3163968999 >> 172.21.15.92:0/3163968999 conn(0x7f97d0068e20 msgr2=0x7f97d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:16.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 -- 172.21.15.92:0/3163968999 shutdown_connections 2023-10-06T03:02:16.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.159+0000 7f97d5b7b700 1 -- 172.21.15.92:0/3163968999 wait complete. 2023-10-06T03:02:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.160+0000 7f97d5b7b700 1 Processor -- start 2023-10-06T03:02:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.160+0000 7f97d5b7b700 1 -- start start 2023-10-06T03:02:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b8003a60 0x7f97d00c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d016a010 0x7f97d00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 0x7f97d0147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01478c0 con 0x7f97d0146f00 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00b4c50 con 0x7f97b8003a60 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d016a820 con 0x7f97d016a010 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 0x7f97d0147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 0x7f97d0147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51138/0 (socket says 172.21.15.92:51138) 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 -- 172.21.15.92:0/4001766724 learned_addr learned my addr 172.21.15.92:0/4001766724 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97ceffd700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d016a010 0x7f97d00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b8003a60 0x7f97d00c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 -- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d016a010 msgr2=0x7f97d00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d016a010 0x7f97d00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 -- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b8003a60 msgr2=0x7f97d00c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b8003a60 0x7f97d00c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 -- 172.21.15.92:0/4001766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d016ab00 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97cffff700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 0x7f97d0147380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f97d00d03b0 tx=0x7f97c8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c8006820 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c80069a0 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d016ade0 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c8006d60 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.161+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d00cee70 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.162+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d016b020 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.163+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97c8007240 con 0x7f97d0146f00 2023-10-06T03:02:16.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.163+0000 7f97b77fe700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97bc066f30 0x7f97bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.163+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f97c8083970 con 0x7f97d0146f00 2023-10-06T03:02:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.163+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97bc066f30 0x7f97bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.164+0000 7f97cf7fe700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97bc066f30 0x7f97bc0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97c000afd0 tx=0x7f97c0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.167+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97d016b020 con 0x7f97d0146f00 2023-10-06T03:02:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.392+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97d016b020 con 0x7f97d0146f00 2023-10-06T03:02:16.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.393+0000 7f97b77fe700 1 -- 172.21.15.92:0/4001766724 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f97d016b020 con 0x7f97d0146f00 2023-10-06T03:02:16.393 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:16.394 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97bc066f30 msgr2=0x7f97bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97bc066f30 0x7f97bc0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f97c000afd0 tx=0x7f97c0016000 comp rx=0 tx=0).stop 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 msgr2=0x7f97d0147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 0x7f97d0147380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f97d00d03b0 tx=0x7f97c8009040 comp rx=0 tx=0).stop 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 shutdown_connections 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97bc066f30 0x7f97bc0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d0146f00 0x7f97d0147380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b8003a60 0x7f97d00c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 --2- 172.21.15.92:0/4001766724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d016a010 0x7f97d00ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 >> 172.21.15.92:0/4001766724 conn(0x7f97d0068e20 msgr2=0x7f97d00c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 shutdown_connections 2023-10-06T03:02:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.395+0000 7f97d5b7b700 1 -- 172.21.15.92:0/4001766724 wait complete. 2023-10-06T03:02:16.397 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:02:16.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:16.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 Processor -- start 2023-10-06T03:02:16.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 -- start start 2023-10-06T03:02:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4580c6b30 0x7fe458157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe4580ae840 0x7fe4580aec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4580af1c0 con 0x7fe458157380 2023-10-06T03:02:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45806ea60 con 0x7fe4580c6b30 2023-10-06T03:02:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.522+0000 7fe45d43a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45806ed40 con 0x7fe4580ae840 2023-10-06T03:02:16.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe4577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4580c6b30 0x7fe458157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe457fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe458157380 0x7fe45815bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46676/0 (socket says 172.21.15.92:46676) 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe457fff700 1 -- 172.21.15.92:0/3218776991 learned_addr learned my addr 172.21.15.92:0/3218776991 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe45cc39700 1 --2- 172.21.15.92:0/3218776991 >> v2:172.21.15.136:3300/0 conn(0x7fe4580ae840 0x7fe4580aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe456ffd700 1 -- 172.21.15.92:0/3218776991 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779081851 0 0) 0x7fe4580af1c0 con 0x7fe458157380 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe456ffd700 1 -- 172.21.15.92:0/3218776991 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe444003570 con 0x7fe458157380 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe4577fe700 1 -- 172.21.15.92:0/3218776991 >> v2:172.21.15.136:3300/0 conn(0x7fe4580ae840 msgr2=0x7fe4580aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe4577fe700 1 --2- 172.21.15.92:0/3218776991 >> v2:172.21.15.136:3300/0 conn(0x7fe4580ae840 0x7fe4580aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe4577fe700 1 -- 172.21.15.92:0/3218776991 >> v1:172.21.15.92:6789/0 conn(0x7fe458157380 legacy=0x7fe45815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe4577fe700 1 -- 172.21.15.92:0/3218776991 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580afe60 con 0x7fe4580c6b30 2023-10-06T03:02:16.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe4577fe700 1 --2- 172.21.15.92:0/3218776991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4580c6b30 0x7fe458157750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4580cc8e0 tx=0x7fe4480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed59d41493819106 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe456ffd700 1 -- 172.21.15.92:0/3218776991 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448011630 con 0x7fe4580c6b30 2023-10-06T03:02:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe456ffd700 1 -- 172.21.15.92:0/3218776991 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480117b0 con 0x7fe4580c6b30 2023-10-06T03:02:16.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.523+0000 7fe456ffd700 1 -- 172.21.15.92:0/3218776991 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448011b70 con 0x7fe4580c6b30 2023-10-06T03:02:16.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 -- 172.21.15.92:0/3218776991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4580c6b30 msgr2=0x7fe458157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 --2- 172.21.15.92:0/3218776991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4580c6b30 0x7fe458157750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe4580cc8e0 tx=0x7fe4480091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 -- 172.21.15.92:0/3218776991 shutdown_connections 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 --2- 172.21.15.92:0/3218776991 >> v2:172.21.15.136:3300/0 conn(0x7fe4580ae840 0x7fe4580aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 --2- 172.21.15.92:0/3218776991 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4580c6b30 0x7fe458157750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 -- 172.21.15.92:0/3218776991 >> 172.21.15.92:0/3218776991 conn(0x7fe45806ddb0 msgr2=0x7fe45806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 -- 172.21.15.92:0/3218776991 shutdown_connections 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.524+0000 7fe45d43a700 1 -- 172.21.15.92:0/3218776991 wait complete. 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.525+0000 7fe45d43a700 1 Processor -- start 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.525+0000 7fe45d43a700 1 -- start start 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4580ae840 0x7fe45806be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4580c6b30 0x7fe458069b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 0x7fe45815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45814f1e0 con 0x7fe458157380 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458158620 con 0x7fe4580c6b30 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4581588e0 con 0x7fe4580ae840 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 0x7fe45815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 0x7fe45815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51160/0 (socket says 172.21.15.92:51160) 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 -- 172.21.15.92:0/52473196 learned_addr learned my addr 172.21.15.92:0/52473196 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe457fff700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4580ae840 0x7fe45806be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe4577fe700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4580c6b30 0x7fe458069b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 -- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4580ae840 msgr2=0x7fe45806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4580ae840 0x7fe45806be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 -- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4580c6b30 msgr2=0x7fe458069b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4580c6b30 0x7fe458069b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 -- 172.21.15.92:0/52473196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448009420 con 0x7fe458157380 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe457fff700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4580ae840 0x7fe45806be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45cc39700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 0x7fe45815ab30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe450009e90 tx=0x7fe45000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000cc20 con 0x7fe458157380 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45000f040 con 0x7fe458157380 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe458158bc0 con 0x7fe458157380 2023-10-06T03:02:16.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.526+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450014480 con 0x7fe458157380 2023-10-06T03:02:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.527+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe458159180 con 0x7fe458157380 2023-10-06T03:02:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.527+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458159370 con 0x7fe458157380 2023-10-06T03:02:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.528+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe450014830 con 0x7fe458157380 2023-10-06T03:02:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.528+0000 7fe44ffff700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438066f30 0x7fe4380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.528+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe450085100 con 0x7fe458157380 2023-10-06T03:02:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.528+0000 7fe457fff700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438066f30 0x7fe4380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.529+0000 7fe457fff700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438066f30 0x7fe4380693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4400058d0 tx=0x7fe440009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.532+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe450055ca0 con 0x7fe458157380 2023-10-06T03:02:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.760+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe45804c3c0 con 0x7fe458157380 2023-10-06T03:02:16.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.760+0000 7fe44ffff700 1 -- 172.21.15.92:0/52473196 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fe450059e60 con 0x7fe458157380 2023-10-06T03:02:16.761 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:16.762 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1293977248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1293977248},{"type":"v1","addr":"172.21.15.136:6834","nonce":1293977248}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1223570781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1223570781},{"type":"v1","addr":"172.21.15.92:6836","nonce":1223570781}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/622287121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":622287121},{"type":"v1","addr":"172.21.15.92:6837","nonce":622287121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:01:41.757251+0000","modified":"2023-10-06T03:01:45.275637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4306":{"gid":4306,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/3764157697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3764157697},{"type":"v1","addr":"172.21.15.136:6835","nonce":3764157697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:02:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438066f30 msgr2=0x7fe4380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438066f30 0x7fe4380693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4400058d0 tx=0x7fe440009040 comp rx=0 tx=0).stop 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 msgr2=0x7fe45815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 0x7fe45815ab30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe450009e90 tx=0x7fe45000b040 comp rx=0 tx=0).stop 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 shutdown_connections 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe438066f30 0x7fe4380693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe458157380 0x7fe45815ab30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4580ae840 0x7fe45806be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 --2- 172.21.15.92:0/52473196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4580c6b30 0x7fe458069b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.762+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 >> 172.21.15.92:0/52473196 conn(0x7fe45806ddb0 msgr2=0x7fe4580c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.763+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 shutdown_connections 2023-10-06T03:02:16.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.763+0000 7fe45d43a700 1 -- 172.21.15.92:0/52473196 wait complete. 2023-10-06T03:02:16.765 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 7 2023-10-06T03:02:16.775 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.889+0000 7ffab2e3b700 1 Processor -- start 2023-10-06T03:02:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.889+0000 7ffab2e3b700 1 -- start start 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffab2e3b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffaac069560 0x7ffaac0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffab2e3b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac069ee0 con 0x7ffaac159c60 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffab2e3b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac15a160 con 0x7ffaac159890 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffab2e3b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0d13a0 con 0x7ffaac069560 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaabfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffaac159c60 0x7ffaac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46680/0 (socket says 172.21.15.92:46680) 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaabfff700 1 -- 172.21.15.92:0/2832311210 learned_addr learned my addr 172.21.15.92:0/2832311210 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:16.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511543244 0 0) 0x7ffaac069ee0 con 0x7ffaac159c60 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffab13d8700 1 --2- 172.21.15.92:0/2832311210 >> v2:172.21.15.136:3300/0 conn(0x7ffaac069560 0x7ffaac0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa98003570 con 0x7ffaac159c60 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 945205016 0 0) 0x7ffa98003570 con 0x7ffaac159c60 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 >> v2:172.21.15.136:3300/0 conn(0x7ffaac069560 msgr2=0x7ffaac0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 --2- 172.21.15.92:0/2832311210 >> v2:172.21.15.136:3300/0 conn(0x7ffaac069560 0x7ffaac0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 >> v1:172.21.15.136:6789/0 conn(0x7ffaac159890 legacy=0x7ffaac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac15e140 con 0x7ffaac159c60 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7ffa94001470 con 0x7ffaac159c60 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 >> v1:172.21.15.92:6789/0 conn(0x7ffaac159c60 legacy=0x7ffaac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa98003a60 0x7ffa98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa98004810 0x7ffa98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa980051c0 0x7ffa9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0d13a0 con 0x7ffa980051c0 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9800c0d0 con 0x7ffa98004810 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9800c290 con 0x7ffa98003a60 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.890+0000 7ffab13d8700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa980051c0 0x7ffa9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffaabfff700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa98004810 0x7ffa98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab0bd7700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa98003a60 0x7ffa98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 -- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa98003a60 msgr2=0x7ffa98003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa98003a60 0x7ffa98003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 -- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa98004810 msgr2=0x7ffa98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa98004810 0x7ffa98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 -- 172.21.15.92:0/2832311210 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa98003570 con 0x7ffa980051c0 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab0bd7700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa98003a60 0x7ffa98003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 -- 172.21.15.92:0/2832311210 reap_dead start 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffab13d8700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa980051c0 0x7ffa9800ba60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009c00 tx=0x7ffaa40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f52705ebd70a90 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa400e710 con 0x7ffa980051c0 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa400e890 con 0x7ffa980051c0 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.891+0000 7ffaab7fe700 1 -- 172.21.15.92:0/2832311210 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa400ec50 con 0x7ffa980051c0 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2832311210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa980051c0 msgr2=0x7ffa9800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa980051c0 0x7ffa9800ba60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009c00 tx=0x7ffaa40091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2832311210 shutdown_connections 2023-10-06T03:02:16.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2832311210 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa980051c0 0x7ffa9800ba60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2832311210 >> 172.21.15.92:0/2832311210 conn(0x7ffaac068e20 msgr2=0x7ffaac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:16.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2832311210 shutdown_connections 2023-10-06T03:02:16.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.892+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2832311210 wait complete. 2023-10-06T03:02:16.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.893+0000 7ffab2e3b700 1 Processor -- start 2023-10-06T03:02:16.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.893+0000 7ffab2e3b700 1 -- start start 2023-10-06T03:02:16.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa980051c0 0x7ffaac156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 0x7ffaac158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac151320 0x7ffaac1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac152c10 con 0x7ffaac157f90 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac152090 con 0x7ffaac151320 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac152370 con 0x7ffa980051c0 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 0x7ffaac158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 0x7ffaac158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51170/0 (socket says 172.21.15.92:51170) 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 -- 172.21.15.92:0/2264681685 learned_addr learned my addr 172.21.15.92:0/2264681685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab0bd7700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa980051c0 0x7ffaac156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab13d8700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac151320 0x7ffaac1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 -- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa980051c0 msgr2=0x7ffaac156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa980051c0 0x7ffaac156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 -- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac151320 msgr2=0x7ffaac1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac151320 0x7ffaac1517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 -- 172.21.15.92:0/2264681685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac152650 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaabfff700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 0x7ffaac158400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa94007370 tx=0x7ffa94001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94018030 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9400f740 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac152930 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400fb00 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.894+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac151ce0 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.895+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac04c3c0 con 0x7ffaac157f90 2023-10-06T03:02:16.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.896+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa9401b070 con 0x7ffaac157f90 2023-10-06T03:02:16.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.896+0000 7ffaa8ff9700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa8c066fb0 0x7ffa8c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:16.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.896+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa94019070 con 0x7ffaac157f90 2023-10-06T03:02:16.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.896+0000 7ffab0bd7700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa8c066fb0 0x7ffa8c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:16.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.897+0000 7ffab0bd7700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa8c066fb0 0x7ffa8c069460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c007fd0 tx=0x7ffa9c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:16.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:16.900+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa94051600 con 0x7ffaac157f90 2023-10-06T03:02:17.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.084+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaac06a5b0 con 0x7ffaac157f90 2023-10-06T03:02:17.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.085+0000 7ffaa8ff9700 1 -- 172.21.15.92:0/2264681685 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16866 (secure 0 0 0) 0x7ffa94054c20 con 0x7ffaac157f90 2023-10-06T03:02:17.086 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:17.086 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:01:43.264121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:01:40.259324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:01:40.709153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:17.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa8c066fb0 msgr2=0x7ffa8c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa8c066fb0 0x7ffa8c069460 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c007fd0 tx=0x7ffa9c00c040 comp rx=0 tx=0).stop 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 msgr2=0x7ffaac158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 0x7ffaac158400 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa94007370 tx=0x7ffa94001e50 comp rx=0 tx=0).stop 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 shutdown_connections 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa8c066fb0 0x7ffa8c069460 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaac151320 0x7ffaac1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa980051c0 0x7ffaac156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 --2- 172.21.15.92:0/2264681685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaac157f90 0x7ffaac158400 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 >> 172.21.15.92:0/2264681685 conn(0x7ffaac068e20 msgr2=0x7ffaac1553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:17.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 shutdown_connections 2023-10-06T03:02:17.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.087+0000 7ffab2e3b700 1 -- 172.21.15.92:0/2264681685 wait complete. 2023-10-06T03:02:17.100 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:02:17.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.215+0000 7f691759e700 1 Processor -- start 2023-10-06T03:02:17.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.215+0000 7f691759e700 1 -- start start 2023-10-06T03:02:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f69180b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691759e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f691815d230 0x7f691815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691759e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691815dbd0 con 0x7f69181597f0 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691759e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69180b2980 con 0x7f691815d230 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69180b11d0 con 0x7f6918159420 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f69180b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f691815d230 0x7f691815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f691815d230 0x7f691815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:46884/0 (socket says 172.21.15.92:46884) 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f69180b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:54582/0 (socket says 172.21.15.92:54582) 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 -- 172.21.15.92:0/2020649753 learned_addr learned my addr 172.21.15.92:0/2020649753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6915d9b700 1 --1- 172.21.15.92:0/2020649753 >> v1:172.21.15.136:6789/0 conn(0x7f69181597f0 0x7f69180b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:39014/0 (socket says 172.21.15.92:39014) 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 -- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 msgr2=0x7f69180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 --2- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f69180b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 -- 172.21.15.92:0/2020649753 >> v1:172.21.15.136:6789/0 conn(0x7f69181597f0 legacy=0x7f69180b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f6916d9d700 1 -- 172.21.15.92:0/2020649753 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69180b14b0 con 0x7f691815d230 2023-10-06T03:02:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.216+0000 7f691659c700 1 --2- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f69180b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f6916d9d700 1 --2- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f691815d690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6910009f20 tx=0x7f6910009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7efce6139ed88849 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691559a700 1 -- 172.21.15.92:0/2020649753 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000e070 con 0x7f691815d230 2023-10-06T03:02:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691559a700 1 -- 172.21.15.92:0/2020649753 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691000b9d0 con 0x7f691815d230 2023-10-06T03:02:17.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691559a700 1 -- 172.21.15.92:0/2020649753 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100092e0 con 0x7f691815d230 2023-10-06T03:02:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 -- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 msgr2=0x7f691815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 --2- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f691815d690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6910009f20 tx=0x7f6910009840 comp rx=0 tx=0).stop 2023-10-06T03:02:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 -- 172.21.15.92:0/2020649753 shutdown_connections 2023-10-06T03:02:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 --2- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f691815d690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 --2- 172.21.15.92:0/2020649753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f69180b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 -- 172.21.15.92:0/2020649753 >> 172.21.15.92:0/2020649753 conn(0x7f69180d0660 msgr2=0x7f69180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 -- 172.21.15.92:0/2020649753 shutdown_connections 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.217+0000 7f691759e700 1 -- 172.21.15.92:0/2020649753 wait complete. 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.218+0000 7f691759e700 1 Processor -- start 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.218+0000 7f691759e700 1 -- start start 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f6918159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 0x7f69180b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f6918151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691815a380 con 0x7f69181597f0 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691815a660 con 0x7f691815d230 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691815a940 con 0x7f6918159420 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f6915d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 0x7f69180b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f6915d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 0x7f69180b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51174/0 (socket says 172.21.15.92:51174) 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f6915d9b700 1 -- 172.21.15.92:0/317116047 learned_addr learned my addr 172.21.15.92:0/317116047 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f6916d9d700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f6918151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.219+0000 7f691659c700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f6918159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f6915d9b700 1 -- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 msgr2=0x7f6918159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f6915d9b700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f6918159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f6915d9b700 1 -- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 msgr2=0x7f6918151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f6915d9b700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f6918151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f6915d9b700 1 -- 172.21.15.92:0/317116047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908009420 con 0x7f69181597f0 2023-10-06T03:02:17.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f6915d9b700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 0x7f69180b5580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c7c0 tx=0x7f690c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c005a70 con 0x7f69181597f0 2023-10-06T03:02:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c005bd0 con 0x7f69181597f0 2023-10-06T03:02:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6910009ae0 con 0x7f69181597f0 2023-10-06T03:02:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c0070d0 con 0x7f69181597f0 2023-10-06T03:02:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.220+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691815af00 con 0x7f69181597f0 2023-10-06T03:02:17.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.221+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691815b120 con 0x7f69181597f0 2023-10-06T03:02:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.221+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f690c007480 con 0x7f69181597f0 2023-10-06T03:02:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.222+0000 7f69067fc700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68f8066f30 0x7f68f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.222+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f690c083930 con 0x7f69181597f0 2023-10-06T03:02:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.222+0000 7f691659c700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68f8066f30 0x7f68f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.222+0000 7f691659c700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68f8066f30 0x7f68f80693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6908009f40 tx=0x7f69080091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:17.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.226+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f690c050a40 con 0x7f69181597f0 2023-10-06T03:02:17.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.453+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f691815dbd0 con 0x7f69181597f0 2023-10-06T03:02:17.454 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:02:17.453+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:02:17.459 INFO:tasks.ceph.mds.d.smithi136.stderr: -1853> 2023-10-06T03:01:43.356+0000 7fabfec2c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:02:17.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.459+0000 7f69067fc700 1 -- 172.21.15.92:0/317116047 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f691815dbd0 con 0x7f69181597f0 2023-10-06T03:02:17.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68f8066f30 msgr2=0x7f68f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68f8066f30 0x7f68f80693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6908009f40 tx=0x7f69080091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 msgr2=0x7f69180b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 0x7f69180b5580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c7c0 tx=0x7f690c007f70 comp rx=0 tx=0).stop 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 shutdown_connections 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68f8066f30 0x7f68f80693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f691815d230 0x7f6918151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6918159420 0x7f6918159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 --2- 172.21.15.92:0/317116047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69181597f0 0x7f69180b5580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.461+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 >> 172.21.15.92:0/317116047 conn(0x7f69180d0660 msgr2=0x7f6918068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.462+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 shutdown_connections 2023-10-06T03:02:17.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.462+0000 7f691759e700 1 -- 172.21.15.92:0/317116047 wait complete. 2023-10-06T03:02:17.464 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:02:17.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:02:17.485 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:02:17.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.593+0000 7f6cb96e9700 1 Processor -- start 2023-10-06T03:02:17.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.593+0000 7f6cb96e9700 1 -- start start 2023-10-06T03:02:17.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb96e9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb415d990 0x7f6cb4164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb96e9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb4150b50 con 0x7f6cb40cc9d0 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb96e9700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb40cede0 con 0x7f6cb40cc600 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb96e9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40b2260 con 0x7f6cb415d990 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6cb40cc9d0 0x7f6cb415d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46696/0 (socket says 172.21.15.92:46696) 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb37fe700 1 -- 172.21.15.92:0/1454415302 learned_addr learned my addr 172.21.15.92:0/1454415302 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb8ee8700 1 --2- 172.21.15.92:0/1454415302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb415d990 0x7f6cb4164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060559230 0 0) 0x7f6cb4150b50 con 0x7f6cb40cc9d0 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c9c003570 con 0x7f6cb40cc9d0 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840667458 0 0) 0x7f6cb40cede0 con 0x7f6cb40cc600 2023-10-06T03:02:17.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.594+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb4150b50 con 0x7f6cb40cc600 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/1454415302 >> v1:172.21.15.136:6789/0 conn(0x7f6cb40cc600 legacy=0x7f6cb40b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/1454415302 >> v1:172.21.15.92:6789/0 conn(0x7f6cb40cc9d0 legacy=0x7f6cb415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/1454415302 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40b2540 con 0x7f6cb415d990 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb8ee8700 1 --2- 172.21.15.92:0/1454415302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb415d990 0x7f6cb4164620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009e90 tx=0x7f6cac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e35145144bea93b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00e070 con 0x7f6cb415d990 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00bb80 con 0x7f6cb415d990 2023-10-06T03:02:17.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb2ffd700 1 -- 172.21.15.92:0/1454415302 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac012430 con 0x7f6cb415d990 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb96e9700 1 -- 172.21.15.92:0/1454415302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb415d990 msgr2=0x7f6cb4164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.595+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/1454415302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb415d990 0x7f6cb4164620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009e90 tx=0x7f6cac009780 comp rx=0 tx=0).stop 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.596+0000 7f6cb96e9700 1 -- 172.21.15.92:0/1454415302 shutdown_connections 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.596+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/1454415302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb415d990 0x7f6cb4164620 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.596+0000 7f6cb96e9700 1 -- 172.21.15.92:0/1454415302 >> 172.21.15.92:0/1454415302 conn(0x7f6cb40d0630 msgr2=0x7f6cb40d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.596+0000 7f6cb96e9700 1 -- 172.21.15.92:0/1454415302 shutdown_connections 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.596+0000 7f6cb96e9700 1 -- 172.21.15.92:0/1454415302 wait complete. 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.597+0000 7f6cb96e9700 1 Processor -- start 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.597+0000 7f6cb96e9700 1 -- start start 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb40cc600 0x7f6cb414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb40cc9d0 0x7f6cb4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 0x7f6cb40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb414f350 con 0x7f6cb415d990 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb414e890 con 0x7f6cb40cc600 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb416a3a0 con 0x7f6cb40cc9d0 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 0x7f6cb40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 0x7f6cb40b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51180/0 (socket says 172.21.15.92:51180) 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/728994736 learned_addr learned my addr 172.21.15.92:0/728994736 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb37fe700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb40cc9d0 0x7f6cb4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb3fff700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb40cc600 0x7f6cb414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb40cc9d0 msgr2=0x7f6cb4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb40cc9d0 0x7f6cb4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb40cc600 msgr2=0x7f6cb414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb40cc600 0x7f6cb414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 -- 172.21.15.92:0/728994736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac009a20 con 0x7f6cb415d990 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb8ee8700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 0x7f6cb40b4db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6cac013000 tx=0x7f6cac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac027470 con 0x7f6cb415d990 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac0275f0 con 0x7f6cb415d990 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb4169a90 con 0x7f6cb415d990 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0279b0 con 0x7f6cb415d990 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.598+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb4169d50 con 0x7f6cb415d990 2023-10-06T03:02:17.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.599+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb40889d0 con 0x7f6cb415d990 2023-10-06T03:02:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.600+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cac024070 con 0x7f6cb415d990 2023-10-06T03:02:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.600+0000 7f6c9bfff700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ca0066f30 0x7f6ca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.600+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f6cac087130 con 0x7f6cb415d990 2023-10-06T03:02:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.600+0000 7f6cb3fff700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ca0066f30 0x7f6ca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.600+0000 7f6cb3fff700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ca0066f30 0x7f6ca00693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8001120 tx=0x7f6ca8007d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:17.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.604+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cac0541f0 con 0x7f6cb415d990 2023-10-06T03:02:17.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:17.830+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6cb40b2820 con 0x7f6cb415d990 2023-10-06T03:02:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.472+0000 7f6c9bfff700 1 -- 172.21.15.92:0/728994736 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6cac057810 con 0x7f6cb415d990 2023-10-06T03:02:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ca0066f30 msgr2=0x7f6ca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ca0066f30 0x7f6ca00693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8001120 tx=0x7f6ca8007d00 comp rx=0 tx=0).stop 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 msgr2=0x7f6cb40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 0x7f6cb40b4db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6cac013000 tx=0x7f6cac019000 comp rx=0 tx=0).stop 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 shutdown_connections 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ca0066f30 0x7f6ca00693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6cb415d990 0x7f6cb40b4db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6cb40cc600 0x7f6cb414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 --2- 172.21.15.92:0/728994736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6cb40cc9d0 0x7f6cb4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 >> 172.21.15.92:0/728994736 conn(0x7f6cb40d0630 msgr2=0x7f6cb40afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:18.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 shutdown_connections 2023-10-06T03:02:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.475+0000 7f6cb96e9700 1 -- 172.21.15.92:0/728994736 wait complete. 2023-10-06T03:02:18.488 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:02:18.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.609+0000 7f4f4499d700 1 Processor -- start 2023-10-06T03:02:18.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.609+0000 7f4f4499d700 1 -- start start 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f4499d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4f400b03c0 0x7f4f400cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f4499d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f401591e0 0x7f4f40159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f4499d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f400cfa00 con 0x7f4f400b0790 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f4499d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f400cb580 con 0x7f4f401591e0 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f4499d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f400cf160 con 0x7f4f400b03c0 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f401591e0 0x7f4f40159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:18.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f401591e0 0x7f4f40159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51184/0 (socket says 172.21.15.92:51184) 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 -- 172.21.15.92:0/3141235583 learned_addr learned my addr 172.21.15.92:0/3141235583 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3f7fe700 1 --2- 172.21.15.92:0/3141235583 >> v2:172.21.15.136:3300/0 conn(0x7f4f400b03c0 0x7f4f400cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 -- 172.21.15.92:0/3141235583 >> v2:172.21.15.136:3300/0 conn(0x7f4f400b03c0 msgr2=0x7f4f400cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 --2- 172.21.15.92:0/3141235583 >> v2:172.21.15.136:3300/0 conn(0x7f4f400b03c0 0x7f4f400cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 -- 172.21.15.92:0/3141235583 >> v1:172.21.15.136:6789/0 conn(0x7f4f400b0790 legacy=0x7f4f400b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3ffff700 1 -- 172.21.15.92:0/3141235583 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f400cf440 con 0x7f4f401591e0 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.610+0000 7f4f3f7fe700 1 --2- 172.21.15.92:0/3141235583 >> v2:172.21.15.136:3300/0 conn(0x7f4f400b03c0 0x7f4f400cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f3ffff700 1 --2- 172.21.15.92:0/3141235583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f401591e0 0x7f4f40159600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009f20 tx=0x7f4f38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942d4d9403cd8c51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f3e7fc700 1 -- 172.21.15.92:0/3141235583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800e070 con 0x7f4f401591e0 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f3e7fc700 1 -- 172.21.15.92:0/3141235583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3800ba00 con 0x7f4f401591e0 2023-10-06T03:02:18.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f3e7fc700 1 -- 172.21.15.92:0/3141235583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380092e0 con 0x7f4f401591e0 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 -- 172.21.15.92:0/3141235583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f401591e0 msgr2=0x7f4f40159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3141235583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f401591e0 0x7f4f40159600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009f20 tx=0x7f4f38009840 comp rx=0 tx=0).stop 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 -- 172.21.15.92:0/3141235583 shutdown_connections 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3141235583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f401591e0 0x7f4f40159600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3141235583 >> v2:172.21.15.136:3300/0 conn(0x7f4f400b03c0 0x7f4f400cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 -- 172.21.15.92:0/3141235583 >> 172.21.15.92:0/3141235583 conn(0x7f4f40063230 msgr2=0x7f4f40063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 -- 172.21.15.92:0/3141235583 shutdown_connections 2023-10-06T03:02:18.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.611+0000 7f4f4499d700 1 -- 172.21.15.92:0/3141235583 wait complete. 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.612+0000 7f4f4499d700 1 Processor -- start 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.612+0000 7f4f4499d700 1 -- start start 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f4499d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 0x7f4f400c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f4499d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f400b0790 0x7f4f40168880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f4499d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f401591e0 0x7f4f40168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f4499d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f40169300 con 0x7f4f400b03c0 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f4499d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400c8150 con 0x7f4f400b0790 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f4499d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f400c8430 con 0x7f4f401591e0 2023-10-06T03:02:18.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 0x7f4f400c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 0x7f4f400c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51192/0 (socket says 172.21.15.92:51192) 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.613+0000 7f4f3f7fe700 1 -- 172.21.15.92:0/3160149954 learned_addr learned my addr 172.21.15.92:0/3160149954 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3effd700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f400b0790 0x7f4f40168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3ffff700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f401591e0 0x7f4f40168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3f7fe700 1 -- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f401591e0 msgr2=0x7f4f40168dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3f7fe700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f401591e0 0x7f4f40168dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3f7fe700 1 -- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f400b0790 msgr2=0x7f4f40168880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3f7fe700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f400b0790 0x7f4f40168880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3f7fe700 1 -- 172.21.15.92:0/3160149954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009ae0 con 0x7f4f400b03c0 2023-10-06T03:02:18.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3effd700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f400b0790 0x7f4f40168880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3ffff700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f401591e0 0x7f4f40168dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f3f7fe700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 0x7f4f400c64c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c850 tx=0x7f4f30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30011600 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f30011760 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f400c8710 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30011b00 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.614+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f400c8cd0 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.615+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4004c3c0 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.615+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f300073b0 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.616+0000 7f4f277fe700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.616+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f4f30084670 con 0x7f4f400b03c0 2023-10-06T03:02:18.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.616+0000 7f4f3effd700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:18.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.616+0000 7f4f3effd700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f400c3bf0 tx=0x7f4f34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:18.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.620+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f300551c0 con 0x7f4f400b03c0 2023-10-06T03:02:18.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:18.803+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4f400cad50 con 0x7f4f400b03c0 2023-10-06T03:02:19.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.535+0000 7f4f277fe700 1 -- 172.21.15.92:0/3160149954 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4f30054d50 con 0x7f4f400b03c0 2023-10-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.538+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f2c066f30 msgr2=0x7f4f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.538+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f2c066f30 0x7f4f2c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f400c3bf0 tx=0x7f4f34009040 comp rx=0 tx=0).stop 2023-10-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.538+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 msgr2=0x7f4f400c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.538+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 0x7f4f400c64c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c850 tx=0x7f4f30008040 comp rx=0 tx=0).stop 2023-10-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 shutdown_connections 2023-10-06T03:02:19.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f2c066f30 0x7f4f2c0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f400b03c0 0x7f4f400c64c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f401591e0 0x7f4f40168dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 --2- 172.21.15.92:0/3160149954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f400b0790 0x7f4f40168880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 >> 172.21.15.92:0/3160149954 conn(0x7f4f40063230 msgr2=0x7f4f400d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 shutdown_connections 2023-10-06T03:02:19.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.539+0000 7f4f4499d700 1 -- 172.21.15.92:0/3160149954 wait complete. 2023-10-06T03:02:19.541 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:02:19.553 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.674+0000 7fb1a994c700 1 Processor -- start 2023-10-06T03:02:19.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.674+0000 7fb1a994c700 1 -- start start 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a994c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b0790 0x7fb1a40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a994c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb1a41591e0 0x7fb1a4159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a994c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40cfa00 con 0x7fb1a40b03c0 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a994c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a40cb580 con 0x7fb1a41591e0 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a994c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40cf160 con 0x7fb1a40b0790 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a914b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb1a41591e0 0x7fb1a4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb19bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b0790 0x7fb1a40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:19.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a894a700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb1a40b03c0 0x7fb1a40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:40726/0 (socket says 172.21.15.92:40726) 2023-10-06T03:02:19.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.675+0000 7fb1a894a700 1 -- 172.21.15.92:0/1253328940 learned_addr learned my addr 172.21.15.92:0/1253328940 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:19.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb1a914b700 1 -- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b0790 msgr2=0x7fb1a40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:19.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb1a914b700 1 --2- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b0790 0x7fb1a40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb1a914b700 1 -- 172.21.15.92:0/1253328940 >> v1:172.21.15.136:6789/0 conn(0x7fb1a40b03c0 legacy=0x7fb1a40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:19.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb1a914b700 1 -- 172.21.15.92:0/1253328940 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40cf440 con 0x7fb1a41591e0 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb19bfff700 1 --2- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b0790 0x7fb1a40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb1a914b700 1 --2- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a4159620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009e60 tx=0x7fb19c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb71f3c1feaf898a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb19b7fe700 1 -- 172.21.15.92:0/1253328940 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00e070 con 0x7fb1a41591e0 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb19b7fe700 1 -- 172.21.15.92:0/1253328940 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00bb60 con 0x7fb1a41591e0 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.676+0000 7fb19b7fe700 1 -- 172.21.15.92:0/1253328940 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c012430 con 0x7fb1a41591e0 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 -- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 msgr2=0x7fb1a4159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 --2- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a4159620 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009e60 tx=0x7fb19c009780 comp rx=0 tx=0).stop 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 -- 172.21.15.92:0/1253328940 shutdown_connections 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 --2- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a4159620 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 --2- 172.21.15.92:0/1253328940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b0790 0x7fb1a40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 -- 172.21.15.92:0/1253328940 >> 172.21.15.92:0/1253328940 conn(0x7fb1a4063230 msgr2=0x7fb1a4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:19.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 -- 172.21.15.92:0/1253328940 shutdown_connections 2023-10-06T03:02:19.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.677+0000 7fb1a994c700 1 -- 172.21.15.92:0/1253328940 wait complete. 2023-10-06T03:02:19.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.678+0000 7fb1a994c700 1 Processor -- start 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.678+0000 7fb1a994c700 1 -- start start 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a994c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b03c0 0x7fb1a4169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a994c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 0x7fb1a415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a994c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a414c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a994c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a415d810 con 0x7fb1a40b0790 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a994c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4169f00 con 0x7fb1a41591e0 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a994c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a416a1e0 con 0x7fb1a40b03c0 2023-10-06T03:02:19.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 0x7fb1a415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 0x7fb1a415d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53534/0 (socket says 172.21.15.92:53534) 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 -- 172.21.15.92:0/576999242 learned_addr learned my addr 172.21.15.92:0/576999242 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a894a700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b03c0 0x7fb1a4169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb1a914b700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a414c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 -- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b03c0 msgr2=0x7fb1a4169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b03c0 0x7fb1a4169870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 -- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 msgr2=0x7fb1a414c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a414c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 -- 172.21.15.92:0/576999242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb194009420 con 0x7fb1a40b0790 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb19bfff700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 0x7fb1a415d2d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb19400ec80 tx=0x7fb1940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:19.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.679+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400c920 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.680+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19400ca80 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.680+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c009a20 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.680+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19400ce00 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.680+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4159680 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.680+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a41598a0 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.681+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb194013680 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.681+0000 7fb198ff9700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb190066f30 0x7fb1900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.681+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fb194084910 con 0x7fb1a40b0790 2023-10-06T03:02:19.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.681+0000 7fb1a894a700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb190066f30 0x7fb1900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:19.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.682+0000 7fb1a894a700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb190066f30 0x7fb1900693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40b4430 tx=0x7fb1a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:19.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.684+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb194051b80 con 0x7fb1a40b0790 2023-10-06T03:02:19.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:19.868+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1a416a3a0 con 0x7fb1a40b0790 2023-10-06T03:02:20.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.539+0000 7fb198ff9700 1 -- 172.21.15.92:0/576999242 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb1940551a0 con 0x7fb1a40b0790 2023-10-06T03:02:20.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb190066f30 msgr2=0x7fb1900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb190066f30 0x7fb1900693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40b4430 tx=0x7fb1a0009040 comp rx=0 tx=0).stop 2023-10-06T03:02:20.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 msgr2=0x7fb1a415d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 0x7fb1a415d2d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb19400ec80 tx=0x7fb1940091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 shutdown_connections 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb190066f30 0x7fb1900693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1a41591e0 0x7fb1a414c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1a40b03c0 0x7fb1a4169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 --2- 172.21.15.92:0/576999242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1a40b0790 0x7fb1a415d2d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.542+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 >> 172.21.15.92:0/576999242 conn(0x7fb1a4063230 msgr2=0x7fb1a4158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.543+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 shutdown_connections 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.543+0000 7fb1a994c700 1 -- 172.21.15.92:0/576999242 wait complete. 2023-10-06T03:02:20.545 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:02:20.558 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:02:20.558 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:02:20.559 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:02:20.559 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:20.559 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:02:20.559 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:02:20.559 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:02:20.559 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:20.560 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:02:20.560 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:02:20.560 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:02:20.560 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:20.560 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:02:20.560 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:02:20.560 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:02:20.561 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:20.567 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:02:20.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:02:20.569 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:02:20.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:02:20.571 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:02:20.571 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:02:20.573 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:02:20.573 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:02:20.610 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:02:20.611 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:20.611 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:02:20.612 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:20.614 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:02:20.614 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:20.615 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:02:20.616 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:20.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:02:20.683 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:02:20.684 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:02:20.696 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:02:20.696 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:02:20.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.735+0000 7f7f4b59e700 1 Processor -- start 2023-10-06T03:02:20.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.735+0000 7f7f4b59e700 1 -- start start 2023-10-06T03:02:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4b59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7f4c15d230 0x7f7f4c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c0b2980 con 0x7f7f4c1597f0 2023-10-06T03:02:20.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4b59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c15db70 con 0x7f7f4c159420 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4b59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4c0b11d0 con 0x7f7f4c15d230 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f49d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f4c1597f0 0x7f7f4c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35486/0 (socket says 172.21.15.92:35486) 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f49d9b700 1 -- 172.21.15.92:0/1901787113 learned_addr learned my addr 172.21.15.92:0/1901787113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4ad9d700 1 --2- 172.21.15.92:0/1901787113 >> v2:172.21.15.136:3300/0 conn(0x7f7f4c15d230 0x7f7f4c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636768063 0 0) 0x7f7f4c0b2980 con 0x7f7f4c1597f0 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f34003570 con 0x7f7f4c1597f0 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176094282 0 0) 0x7f7f4c15db70 con 0x7f7f4c159420 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f4c0b2980 con 0x7f7f4c159420 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4ad9d700 1 -- 172.21.15.92:0/1901787113 >> v1:172.21.15.136:6789/0 conn(0x7f7f4c159420 legacy=0x7f7f4c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4ad9d700 1 -- 172.21.15.92:0/1901787113 >> v1:172.21.15.92:6789/0 conn(0x7f7f4c1597f0 legacy=0x7f7f4c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.736+0000 7f7f4ad9d700 1 -- 172.21.15.92:0/1901787113 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c0b14b0 con 0x7f7f4c15d230 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4ad9d700 1 --2- 172.21.15.92:0/1901787113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c15d230 0x7f7f4c15d630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009e60 tx=0x7f7f44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fc13fcc9182ba7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400e070 con 0x7f7f4c15d230 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4400bab0 con 0x7f7f4c15d230 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4959a700 1 -- 172.21.15.92:0/1901787113 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400be70 con 0x7f7f4c15d230 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 -- 172.21.15.92:0/1901787113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c15d230 msgr2=0x7f7f4c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/1901787113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c15d230 0x7f7f4c15d630 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009e60 tx=0x7f7f44009780 comp rx=0 tx=0).stop 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 -- 172.21.15.92:0/1901787113 shutdown_connections 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/1901787113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c15d230 0x7f7f4c15d630 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 -- 172.21.15.92:0/1901787113 >> 172.21.15.92:0/1901787113 conn(0x7f7f4c0d0660 msgr2=0x7f7f4c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 -- 172.21.15.92:0/1901787113 shutdown_connections 2023-10-06T03:02:20.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.737+0000 7f7f4b59e700 1 -- 172.21.15.92:0/1901787113 wait complete. 2023-10-06T03:02:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 Processor -- start 2023-10-06T03:02:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 -- start start 2023-10-06T03:02:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 0x7f7f4c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:20.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c1597f0 0x7f7f4c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f4c15d230 0x7f7f4c0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1568c0 con 0x7f7f4c159420 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c16d6d0 con 0x7f7f4c1597f0 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c16cac0 con 0x7f7f4c15d230 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 0x7f7f4c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 0x7f7f4c156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53568/0 (socket says 172.21.15.92:53568) 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.739+0000 7f7f4a59c700 1 -- 172.21.15.92:0/3669349145 learned_addr learned my addr 172.21.15.92:0/3669349145 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f49d9b700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c1597f0 0x7f7f4c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4ad9d700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f4c15d230 0x7f7f4c0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4a59c700 1 -- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f4c15d230 msgr2=0x7f7f4c0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4a59c700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f4c15d230 0x7f7f4c0b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4a59c700 1 -- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c1597f0 msgr2=0x7f7f4c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4a59c700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c1597f0 0x7f7f4c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4a59c700 1 -- 172.21.15.92:0/3669349145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f44009a20 con 0x7f7f4c159420 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4ad9d700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f4c15d230 0x7f7f4c0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4a59c700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 0x7f7f4c156380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c790 tx=0x7f7f3c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c010510 con 0x7f7f4c159420 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c010670 con 0x7f7f4c159420 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c16cda0 con 0x7f7f4c159420 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c010a30 con 0x7f7f4c159420 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.740+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c16d360 con 0x7f7f4c159420 2023-10-06T03:02:20.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.741+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c04c3c0 con 0x7f7f4c159420 2023-10-06T03:02:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.741+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f3c01b3b0 con 0x7f7f4c159420 2023-10-06T03:02:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.742+0000 7f7f3a7fc700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f2c066f30 0x7f7f2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.742+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7f3c083cb0 con 0x7f7f4c159420 2023-10-06T03:02:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.742+0000 7f7f49d9b700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f2c066f30 0x7f7f2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.742+0000 7f7f49d9b700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f2c066f30 0x7f7f2c0693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c157c60 tx=0x7f7f40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:20.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.746+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f3c0510c0 con 0x7f7f4c159420 2023-10-06T03:02:20.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:20.943+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7f4c0b1790 con 0x7f7f4c159420 2023-10-06T03:02:21.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.555+0000 7f7f3a7fc700 1 -- 172.21.15.92:0/3669349145 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7f3c017020 con 0x7f7f4c159420 2023-10-06T03:02:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f2c066f30 msgr2=0x7f7f2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f2c066f30 0x7f7f2c0693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c157c60 tx=0x7f7f40009040 comp rx=0 tx=0).stop 2023-10-06T03:02:21.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 msgr2=0x7f7f4c156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 0x7f7f4c156380 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00c790 tx=0x7f7f3c007e10 comp rx=0 tx=0).stop 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 shutdown_connections 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f2c066f30 0x7f7f2c0693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f4c159420 0x7f7f4c156380 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f4c15d230 0x7f7f4c0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 --2- 172.21.15.92:0/3669349145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f4c1597f0 0x7f7f4c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 >> 172.21.15.92:0/3669349145 conn(0x7f7f4c0d0660 msgr2=0x7f7f4c1570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 shutdown_connections 2023-10-06T03:02:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.557+0000 7f7f4b59e700 1 -- 172.21.15.92:0/3669349145 wait complete. 2023-10-06T03:02:21.561 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:02:21.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:02:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.684+0000 7ff90b7d3700 1 Processor -- start 2023-10-06T03:02:21.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90b7d3700 1 -- start start 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90b7d3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff904159c60 0x7ff90415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90b7d3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff90415db10 0x7ff904069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90b7d3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90415a160 con 0x7ff904159890 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90b7d3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90415df50 con 0x7ff90415db10 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90b7d3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9040d13a0 con 0x7ff904159c60 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff909d70700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff90415db10 0x7ff904069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90956f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff904159890 0x7ff90415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35500/0 (socket says 172.21.15.92:35500) 2023-10-06T03:02:21.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.685+0000 7ff90956f700 1 -- 172.21.15.92:0/2622189757 learned_addr learned my addr 172.21.15.92:0/2622189757 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff908d6e700 1 --2- 172.21.15.92:0/2622189757 >> v2:172.21.15.136:3300/0 conn(0x7ff904159c60 0x7ff90415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff8fbfff700 1 -- 172.21.15.92:0/2622189757 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412731121 0 0) 0x7ff90415a160 con 0x7ff904159890 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff8fbfff700 1 -- 172.21.15.92:0/2622189757 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f0003570 con 0x7ff904159890 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff909d70700 1 -- 172.21.15.92:0/2622189757 >> v2:172.21.15.136:3300/0 conn(0x7ff904159c60 msgr2=0x7ff90415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff909d70700 1 --2- 172.21.15.92:0/2622189757 >> v2:172.21.15.136:3300/0 conn(0x7ff904159c60 0x7ff90415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff909d70700 1 -- 172.21.15.92:0/2622189757 >> v1:172.21.15.92:6789/0 conn(0x7ff904159890 legacy=0x7ff90415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff909d70700 1 -- 172.21.15.92:0/2622189757 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90415e140 con 0x7ff90415db10 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff909d70700 1 --2- 172.21.15.92:0/2622189757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff904069560 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9040b67f0 tx=0x7ff8fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657cc455ae8a318f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff8fbfff700 1 -- 172.21.15.92:0/2622189757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00e070 con 0x7ff90415db10 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff8fbfff700 1 -- 172.21.15.92:0/2622189757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00bb60 con 0x7ff90415db10 2023-10-06T03:02:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.686+0000 7ff8fbfff700 1 -- 172.21.15.92:0/2622189757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc012430 con 0x7ff90415db10 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 -- 172.21.15.92:0/2622189757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 msgr2=0x7ff904069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/2622189757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff904069560 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9040b67f0 tx=0x7ff8fc009780 comp rx=0 tx=0).stop 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 -- 172.21.15.92:0/2622189757 shutdown_connections 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/2622189757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff904069560 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/2622189757 >> v2:172.21.15.136:3300/0 conn(0x7ff904159c60 0x7ff90415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 -- 172.21.15.92:0/2622189757 >> 172.21.15.92:0/2622189757 conn(0x7ff904068e20 msgr2=0x7ff9040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 -- 172.21.15.92:0/2622189757 shutdown_connections 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.687+0000 7ff90b7d3700 1 -- 172.21.15.92:0/2622189757 wait complete. 2023-10-06T03:02:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.688+0000 7ff90b7d3700 1 Processor -- start 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.688+0000 7ff90b7d3700 1 -- start start 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff904159890 0x7ff9040b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff904159c60 0x7ff9040c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff9040c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90416cdc0 con 0x7ff90415db10 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9040cd430 con 0x7ff904159c60 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9040cd710 con 0x7ff904159890 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff9040c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff9040c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53574/0 (socket says 172.21.15.92:53574) 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 -- 172.21.15.92:0/3071565889 learned_addr learned my addr 172.21.15.92:0/3071565889 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:21.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff908d6e700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff904159c60 0x7ff9040c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90956f700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff904159890 0x7ff9040b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 -- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff904159890 msgr2=0x7ff9040b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff904159890 0x7ff9040b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 -- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff904159c60 msgr2=0x7ff9040c6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff904159c60 0x7ff9040c6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 -- 172.21.15.92:0/3071565889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc009a20 con 0x7ff90415db10 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff908d6e700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff904159c60 0x7ff9040c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90956f700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff904159890 0x7ff9040b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff909d70700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff9040c7090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009fd0 tx=0x7ff8fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc012e00 con 0x7ff90415db10 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc01d630 con 0x7ff90415db10 2023-10-06T03:02:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9040cd9f0 con 0x7ff90415db10 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc01d9f0 con 0x7ff90415db10 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.689+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9040cd0c0 con 0x7ff90415db10 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.690+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90416cf70 con 0x7ff90415db10 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.691+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8fc024070 con 0x7ff90415db10 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.691+0000 7ff8f97fa700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e8066f30 0x7ff8e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.691+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff8fc086b60 con 0x7ff90415db10 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.691+0000 7ff90956f700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e8066f30 0x7ff8e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.691+0000 7ff90956f700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e8066f30 0x7ff8e80693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9000058d0 tx=0x7ff900009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.695+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff90416cf70 con 0x7ff90415db10 2023-10-06T03:02:21.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.952+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff90404c3c0 con 0x7ff90415db10 2023-10-06T03:02:21.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.953+0000 7ff8f97fa700 1 -- 172.21.15.92:0/3071565889 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff8fc053fc0 con 0x7ff90415db10 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:21.956 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.958 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:02:21.961 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:21.966 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mds", 2023-10-06T03:02:21.968 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB9eB9lC28DJxAA7TpByj0PYGAp3UzM+jdLOw==", 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB9eB9lZXkDJxAAR5kL+obuEUOuFyUJ763McA==", 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-osd", 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB9eB9lg4IDJxAA5KXFu/3sJIrSapWmtKBf1w==", 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB9eB9ltokDJxAAJ6dziSxNAAd9CjcUiZ+raw==", 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T03:02:21.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB9eB9lKpEDJxAAbFZS5D7Beti7A8aoz/sEnw==", 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB9eB9l3pgDJxAA0Lhuisydgr1C3dcGviuovw==", 2023-10-06T03:02:21.971 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:21.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:02:21.973 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e8066f30 msgr2=0x7ff8e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e8066f30 0x7ff8e80693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9000058d0 tx=0x7ff900009040 comp rx=0 tx=0).stop 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 msgr2=0x7ff9040c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff9040c7090 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009fd0 tx=0x7ff8fc019000 comp rx=0 tx=0).stop 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 shutdown_connections 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e8066f30 0x7ff8e80693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff90415db10 0x7ff9040c7090 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff904159890 0x7ff9040b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 --2- 172.21.15.92:0/3071565889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff904159c60 0x7ff9040c6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:21.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 >> 172.21.15.92:0/3071565889 conn(0x7ff904068e20 msgr2=0x7ff904149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:21.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 shutdown_connections 2023-10-06T03:02:21.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:21.955+0000 7ff90b7d3700 1 -- 172.21.15.92:0/3071565889 wait complete. 2023-10-06T03:02:21.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T03:02:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.089+0000 7fbc3835b700 1 Processor -- start 2023-10-06T03:02:22.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.089+0000 7fbc3835b700 1 -- start start 2023-10-06T03:02:22.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc3835b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc300699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc3835b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc30069ee0 con 0x7fbc30159c60 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc3835b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3015a160 con 0x7fbc30159890 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc3835b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300d13a0 con 0x7fbc30069560 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc358f6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbc30159c60 0x7fbc3015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35516/0 (socket says 172.21.15.92:35516) 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc358f6700 1 -- 172.21.15.92:0/2270599872 learned_addr learned my addr 172.21.15.92:0/2270599872 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc368f8700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc300699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458805681 0 0) 0x7fbc30069ee0 con 0x7fbc30159c60 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc1c003570 con 0x7fbc30159c60 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 955410134 0 0) 0x7fbc1c003570 con 0x7fbc30159c60 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 msgr2=0x7fbc300699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc300699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 >> v1:172.21.15.136:6789/0 conn(0x7fbc30159890 legacy=0x7fbc3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3015e140 con 0x7fbc30159c60 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fbc20001470 con 0x7fbc30159c60 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 >> v1:172.21.15.92:6789/0 conn(0x7fbc30159c60 legacy=0x7fbc3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.090+0000 7fbc350f5700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc1c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc1c003fd0 0x7fbc1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc1c004d50 0x7fbc1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300d13a0 con 0x7fbc1c004d50 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c00bb40 con 0x7fbc1c003fd0 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c00bd00 con 0x7fbc30069560 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc1c004d50 0x7fbc1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc358f6700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc1c003fd0 0x7fbc1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc360f7700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc1c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 -- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 msgr2=0x7fbc1c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc1c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 -- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc1c003fd0 msgr2=0x7fbc1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc1c003fd0 0x7fbc1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 -- 172.21.15.92:0/2270599872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c003570 con 0x7fbc1c004d50 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc360f7700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc30069560 0x7fbc1c003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc368f8700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc1c004d50 0x7fbc1c0051c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e210 tx=0x7fbc2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d66061974d2e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800eac0 con 0x7fbc1c004d50 2023-10-06T03:02:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2800f040 con 0x7fbc1c004d50 2023-10-06T03:02:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.091+0000 7fbc350f5700 1 -- 172.21.15.92:0/2270599872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800ee70 con 0x7fbc1c004d50 2023-10-06T03:02:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc3835b700 1 -- 172.21.15.92:0/2270599872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc1c004d50 msgr2=0x7fbc1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc3835b700 1 --2- 172.21.15.92:0/2270599872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc1c004d50 0x7fbc1c0051c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800e210 tx=0x7fbc2800b040 comp rx=0 tx=0).stop 2023-10-06T03:02:22.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc368f8700 1 -- 172.21.15.92:0/2270599872 reap_dead start 2023-10-06T03:02:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc3835b700 1 -- 172.21.15.92:0/2270599872 shutdown_connections 2023-10-06T03:02:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc3835b700 1 -- 172.21.15.92:0/2270599872 >> 172.21.15.92:0/2270599872 conn(0x7fbc30068e20 msgr2=0x7fbc300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc3835b700 1 -- 172.21.15.92:0/2270599872 shutdown_connections 2023-10-06T03:02:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.092+0000 7fbc3835b700 1 -- 172.21.15.92:0/2270599872 wait complete. 2023-10-06T03:02:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.093+0000 7fbc3835b700 1 Processor -- start 2023-10-06T03:02:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.093+0000 7fbc3835b700 1 -- start start 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc1c004d50 0x7fbc300ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 0x7fbc300cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc30153a30 0x7fbc300cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300ce840 con 0x7fbc301535c0 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300cf680 con 0x7fbc30153a30 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300cea70 con 0x7fbc1c004d50 2023-10-06T03:02:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 0x7fbc300cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 0x7fbc300cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53588/0 (socket says 172.21.15.92:53588) 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 -- 172.21.15.92:0/1602436570 learned_addr learned my addr 172.21.15.92:0/1602436570 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc368f8700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc30153a30 0x7fbc300cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc360f7700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc1c004d50 0x7fbc300ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 -- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc1c004d50 msgr2=0x7fbc300ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc1c004d50 0x7fbc300ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 -- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc30153a30 msgr2=0x7fbc300cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc30153a30 0x7fbc300cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 -- 172.21.15.92:0/1602436570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300ced50 con 0x7fbc301535c0 2023-10-06T03:02:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc358f6700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 0x7fbc300cd0c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbc200073c0 tx=0x7fbc20001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20018030 con 0x7fbc301535c0 2023-10-06T03:02:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2000f940 con 0x7fbc301535c0 2023-10-06T03:02:22.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc300cf030 con 0x7fbc301535c0 2023-10-06T03:02:22.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000fce0 con 0x7fbc301535c0 2023-10-06T03:02:22.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.094+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc300cfb10 con 0x7fbc301535c0 2023-10-06T03:02:22.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.095+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc300cfd30 con 0x7fbc301535c0 2023-10-06T03:02:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.096+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc2001b070 con 0x7fbc301535c0 2023-10-06T03:02:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.096+0000 7fbc267fc700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.096+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbc20019070 con 0x7fbc301535c0 2023-10-06T03:02:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.096+0000 7fbc360f7700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.096+0000 7fbc360f7700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc14066f30 0x7fbc140693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc30158eb0 tx=0x7fbc2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.100+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc200516f0 con 0x7fbc301535c0 2023-10-06T03:02:22.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.358+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fbc3004c3c0 con 0x7fbc301535c0 2023-10-06T03:02:22.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.362+0000 7fbc267fc700 1 -- 172.21.15.92:0/1602436570 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbc20054d10 con 0x7fbc301535c0 2023-10-06T03:02:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc14066f30 msgr2=0x7fbc140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc14066f30 0x7fbc140693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc30158eb0 tx=0x7fbc2c009040 comp rx=0 tx=0).stop 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 msgr2=0x7fbc300cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 0x7fbc300cd0c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbc200073c0 tx=0x7fbc20001e70 comp rx=0 tx=0).stop 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 shutdown_connections 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc14066f30 0x7fbc140693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc30153a30 0x7fbc300cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc1c004d50 0x7fbc300ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 --2- 172.21.15.92:0/1602436570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc301535c0 0x7fbc300cd0c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 >> 172.21.15.92:0/1602436570 conn(0x7fbc30068e20 msgr2=0x7fbc3015cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 shutdown_connections 2023-10-06T03:02:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.364+0000 7fbc3835b700 1 -- 172.21.15.92:0/1602436570 wait complete. 2023-10-06T03:02:22.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T03:02:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.493+0000 7f9c07178700 1 Processor -- start 2023-10-06T03:02:22.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.493+0000 7f9c07178700 1 -- start start 2023-10-06T03:02:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.494+0000 7f9c07178700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9c0006b830 0x7f9c000c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.494+0000 7f9c07178700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c00162250 0x7f9c000685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.494+0000 7f9c07178700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c000c74b0 con 0x7f9c000cb440 2023-10-06T03:02:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.494+0000 7f9c07178700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0006bc00 con 0x7f9c0006b830 2023-10-06T03:02:22.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.494+0000 7f9c07178700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c00158a30 con 0x7f9c00162250 2023-10-06T03:02:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c04f14700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9c0006b830 0x7f9c000c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c05715700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c00162250 0x7f9c000685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9bf7fff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f9c000cb440 0x7f9c00161b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:40772/0 (socket says 172.21.15.92:40772) 2023-10-06T03:02:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9bf7fff700 1 -- 172.21.15.92:0/564111463 learned_addr learned my addr 172.21.15.92:0/564111463 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9bf77fe700 1 -- 172.21.15.92:0/564111463 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846497468 0 0) 0x7f9c000c74b0 con 0x7f9c000cb440 2023-10-06T03:02:22.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c05715700 1 -- 172.21.15.92:0/564111463 >> v2:172.21.15.136:3300/0 conn(0x7f9c0006b830 msgr2=0x7f9c000c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c05715700 1 --2- 172.21.15.92:0/564111463 >> v2:172.21.15.136:3300/0 conn(0x7f9c0006b830 0x7f9c000c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c05715700 1 -- 172.21.15.92:0/564111463 >> v1:172.21.15.136:6789/0 conn(0x7f9c000cb440 legacy=0x7f9c00161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c05715700 1 -- 172.21.15.92:0/564111463 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000c95e0 con 0x7f9c00162250 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.495+0000 7f9c05715700 1 --2- 172.21.15.92:0/564111463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c00162250 0x7f9c000685d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80098e0 tx=0x7f9bf80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21511e2e2416a8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9bf77fe700 1 -- 172.21.15.92:0/564111463 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800b6d0 con 0x7f9c00162250 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9bf77fe700 1 -- 172.21.15.92:0/564111463 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800b830 con 0x7f9c00162250 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9bf77fe700 1 -- 172.21.15.92:0/564111463 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800bbd0 con 0x7f9c00162250 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 -- 172.21.15.92:0/564111463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c00162250 msgr2=0x7f9c000685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 --2- 172.21.15.92:0/564111463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c00162250 0x7f9c000685d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80098e0 tx=0x7f9bf80091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 -- 172.21.15.92:0/564111463 shutdown_connections 2023-10-06T03:02:22.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 --2- 172.21.15.92:0/564111463 >> v2:172.21.15.136:3300/0 conn(0x7f9c0006b830 0x7f9c000c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 --2- 172.21.15.92:0/564111463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c00162250 0x7f9c000685d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 -- 172.21.15.92:0/564111463 >> 172.21.15.92:0/564111463 conn(0x7f9c000c9870 msgr2=0x7f9c000c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:22.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 -- 172.21.15.92:0/564111463 shutdown_connections 2023-10-06T03:02:22.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.496+0000 7f9c07178700 1 -- 172.21.15.92:0/564111463 wait complete. 2023-10-06T03:02:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.497+0000 7f9c07178700 1 Processor -- start 2023-10-06T03:02:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.497+0000 7f9c07178700 1 -- start start 2023-10-06T03:02:22.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c07178700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c0006b830 0x7f9c000c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c07178700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c000cb440 0x7f9c000c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c07178700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 0x7f9c001610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c07178700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000c62d0 con 0x7f9c00162250 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c07178700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000c6ca0 con 0x7f9c0006b830 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c07178700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c00161620 con 0x7f9c000cb440 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c05715700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 0x7f9c001610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c05715700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 0x7f9c001610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53594/0 (socket says 172.21.15.92:53594) 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c05715700 1 -- 172.21.15.92:0/473293874 learned_addr learned my addr 172.21.15.92:0/473293874 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9bf7fff700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c000cb440 0x7f9c000c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c04f14700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c0006b830 0x7f9c000c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c05715700 1 -- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c000cb440 msgr2=0x7f9c000c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.498+0000 7f9c05715700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c000cb440 0x7f9c000c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9c05715700 1 -- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c0006b830 msgr2=0x7f9c000c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9c05715700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c0006b830 0x7f9c000c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9c05715700 1 -- 172.21.15.92:0/473293874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8009420 con 0x7f9c00162250 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9c05715700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 0x7f9c001610e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8013000 tx=0x7f9bf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8025420 con 0x7f9c00162250 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf80255c0 con 0x7f9c00162250 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c000c56c0 con 0x7f9c00162250 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8025980 con 0x7f9c00162250 2023-10-06T03:02:22.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.499+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c000c5c80 con 0x7f9c00162250 2023-10-06T03:02:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.500+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0004c3c0 con 0x7f9c00162250 2023-10-06T03:02:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.500+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bf8023070 con 0x7f9c00162250 2023-10-06T03:02:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.500+0000 7f9bf4ff9700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9bec066f30 0x7f9bec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.501+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9bf8085ac0 con 0x7f9c00162250 2023-10-06T03:02:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.501+0000 7f9c04f14700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9bec066f30 0x7f9bec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.501+0000 7f9c04f14700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9bec066f30 0x7f9bec0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c000ca190 tx=0x7f9bfc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.505+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bf8052fd0 con 0x7f9c00162250 2023-10-06T03:02:22.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.765+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9c000cab70 con 0x7f9c00162250 2023-10-06T03:02:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.770+0000 7f9bf4ff9700 1 -- 172.21.15.92:0/473293874 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9bf80565f0 con 0x7f9c00162250 2023-10-06T03:02:22.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9bec066f30 msgr2=0x7f9bec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9bec066f30 0x7f9bec0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c000ca190 tx=0x7f9bfc00b040 comp rx=0 tx=0).stop 2023-10-06T03:02:22.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 msgr2=0x7f9c001610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 0x7f9c001610e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8013000 tx=0x7f9bf8017000 comp rx=0 tx=0).stop 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 shutdown_connections 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9bec066f30 0x7f9bec0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c00162250 0x7f9c001610e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c0006b830 0x7f9c000c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 --2- 172.21.15.92:0/473293874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c000cb440 0x7f9c000c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 >> 172.21.15.92:0/473293874 conn(0x7f9c000c9870 msgr2=0x7f9c0006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 shutdown_connections 2023-10-06T03:02:22.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.772+0000 7f9c07178700 1 -- 172.21.15.92:0/473293874 wait complete. 2023-10-06T03:02:22.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T03:02:22.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.899+0000 7f9bad6ab700 1 Processor -- start 2023-10-06T03:02:22.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.899+0000 7f9bad6ab700 1 -- start start 2023-10-06T03:02:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9bad6ab700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9ba8159c60 0x7f9ba815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9bad6ab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba815a160 con 0x7f9ba815db10 2023-10-06T03:02:22.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9bad6ab700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba80d13a0 con 0x7f9ba8159890 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9bad6ab700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba815df50 con 0x7f9ba8159c60 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ba815db10 0x7f9ba8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35524/0 (socket says 172.21.15.92:35524) 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba77fe700 1 -- 172.21.15.92:0/1148419085 learned_addr learned my addr 172.21.15.92:0/1148419085 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba67fc700 1 --2- 172.21.15.92:0/1148419085 >> v2:172.21.15.136:3300/0 conn(0x7f9ba8159c60 0x7f9ba815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047975399 0 0) 0x7f9ba815a160 con 0x7f9ba815db10 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b90003570 con 0x7f9ba815db10 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210341849 0 0) 0x7f9ba80d13a0 con 0x7f9ba8159890 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ba815a160 con 0x7f9ba8159890 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793853488 0 0) 0x7f9b90003570 con 0x7f9ba815db10 2023-10-06T03:02:22.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 >> v2:172.21.15.136:3300/0 conn(0x7f9ba8159c60 msgr2=0x7f9ba815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 --2- 172.21.15.92:0/1148419085 >> v2:172.21.15.136:3300/0 conn(0x7f9ba8159c60 0x7f9ba815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 >> v1:172.21.15.136:6789/0 conn(0x7f9ba8159890 legacy=0x7f9ba815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba815e140 con 0x7f9ba815db10 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f9ba00017f0 con 0x7f9ba815db10 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 >> v1:172.21.15.92:6789/0 conn(0x7f9ba815db10 legacy=0x7f9ba8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90003c40 0x7f9b90004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b90004810 0x7f9b90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b90005210 0x7f9b9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000c130 con 0x7f9b90005210 2023-10-06T03:02:22.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000c270 con 0x7f9b90004810 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.900+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9000c400 con 0x7f9b90003c40 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b90005210 0x7f9b9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba67fc700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b90004810 0x7f9b90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba6ffd700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90003c40 0x7f9b90004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 -- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90003c40 msgr2=0x7f9b90004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90003c40 0x7f9b90004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 -- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b90004810 msgr2=0x7f9b90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b90004810 0x7f9b90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 -- 172.21.15.92:0/1148419085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90003570 con 0x7f9b90005210 2023-10-06T03:02:22.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 -- 172.21.15.92:0/1148419085 reap_dead start 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b90005210 0x7f9b9000ba60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000a850 tx=0x7f9ba00020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de95410b377124a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00070f0 con 0x7f9b90005210 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba0007270 con 0x7f9b90005210 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.901+0000 7f9ba5ffb700 1 -- 172.21.15.92:0/1148419085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0007630 con 0x7f9b90005210 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 -- 172.21.15.92:0/1148419085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b90005210 msgr2=0x7f9b9000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b90005210 0x7f9b9000ba60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000a850 tx=0x7f9ba00020c0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 -- 172.21.15.92:0/1148419085 shutdown_connections 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/1148419085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b90005210 0x7f9b9000ba60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 -- 172.21.15.92:0/1148419085 >> 172.21.15.92:0/1148419085 conn(0x7f9ba8068e20 msgr2=0x7f9ba80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 -- 172.21.15.92:0/1148419085 shutdown_connections 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.902+0000 7f9bad6ab700 1 -- 172.21.15.92:0/1148419085 wait complete. 2023-10-06T03:02:22.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.903+0000 7f9bad6ab700 1 Processor -- start 2023-10-06T03:02:22.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.903+0000 7f9bad6ab700 1 -- start start 2023-10-06T03:02:22.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.903+0000 7f9bad6ab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90005210 0x7f9ba814f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ba80cd0c0 0x7f9ba80cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 0x7f9ba8150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80b4060 con 0x7f9ba814feb0 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba8150840 con 0x7f9ba80cd0c0 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80cfec0 con 0x7f9b90005210 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 0x7f9ba8150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 0x7f9ba8150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53606/0 (socket says 172.21.15.92:53606) 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 -- 172.21.15.92:0/339035579 learned_addr learned my addr 172.21.15.92:0/339035579 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba6ffd700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90005210 0x7f9ba814f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba67fc700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ba80cd0c0 0x7f9ba80cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 -- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90005210 msgr2=0x7f9ba814f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90005210 0x7f9ba814f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 -- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ba80cd0c0 msgr2=0x7f9ba80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ba80cd0c0 0x7f9ba80cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 -- 172.21.15.92:0/339035579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80cdc30 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9ba77fe700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 0x7f9ba8150300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0000f80 tx=0x7f9ba0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00075f0 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba001e430 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba80b3730 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba001e7d0 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.904+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba80b3cf0 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.905+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba804c3c0 con 0x7f9ba814feb0 2023-10-06T03:02:22.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.906+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ba001ebc0 con 0x7f9ba814feb0 2023-10-06T03:02:22.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.906+0000 7f9b8effd700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9b94066f30 0x7f9b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:22.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.906+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9ba0086870 con 0x7f9ba814feb0 2023-10-06T03:02:22.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.906+0000 7f9ba6ffd700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9b94066f30 0x7f9b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:22.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.906+0000 7f9ba6ffd700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9b94066f30 0x7f9b940693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9b980010e0 tx=0x7f9b98007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:22.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:22.910+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ba0053cd0 con 0x7f9ba814feb0 2023-10-06T03:02:23.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.173+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9ba81530f0 con 0x7f9ba814feb0 2023-10-06T03:02:23.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.179+0000 7f9b8effd700 1 -- 172.21.15.92:0/339035579 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9ba00572f0 con 0x7f9ba814feb0 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9b94066f30 msgr2=0x7f9b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9b94066f30 0x7f9b940693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9b980010e0 tx=0x7f9b98007d30 comp rx=0 tx=0).stop 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 msgr2=0x7f9ba8150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 0x7f9ba8150300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0000f80 tx=0x7f9ba0016000 comp rx=0 tx=0).stop 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 shutdown_connections 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9b94066f30 0x7f9b940693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ba814feb0 0x7f9ba8150300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b90005210 0x7f9ba814f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 --2- 172.21.15.92:0/339035579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ba80cd0c0 0x7f9ba80cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 >> 172.21.15.92:0/339035579 conn(0x7f9ba8068e20 msgr2=0x7f9ba8069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:23.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 shutdown_connections 2023-10-06T03:02:23.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.181+0000 7f9bad6ab700 1 -- 172.21.15.92:0/339035579 wait complete. 2023-10-06T03:02:23.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T03:02:23.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 Processor -- start 2023-10-06T03:02:23.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 -- start start 2023-10-06T03:02:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ec006bcd0 0x7f0ec0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec00c9510 0x7f0ec0141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec015c2a0 con 0x7f0ec006c0a0 2023-10-06T03:02:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec015b690 con 0x7f0ec006bcd0 2023-10-06T03:02:23.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.309+0000 7f0ec577e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec015b970 con 0x7f0ec00c9510 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ec006bcd0 0x7f0ec0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0ec006bcd0 0x7f0ec0140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53614/0 (socket says 172.21.15.92:53614) 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 -- 172.21.15.92:0/2368845923 learned_addr learned my addr 172.21.15.92:0/2368845923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebf7fe700 1 --2- 172.21.15.92:0/2368845923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec00c9510 0x7f0ec0141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 -- 172.21.15.92:0/2368845923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec00c9510 msgr2=0x7f0ec0141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 --2- 172.21.15.92:0/2368845923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec00c9510 0x7f0ec0141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 -- 172.21.15.92:0/2368845923 >> v1:172.21.15.136:6789/0 conn(0x7f0ec006c0a0 legacy=0x7f0ec00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 -- 172.21.15.92:0/2368845923 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec015bc50 con 0x7f0ec006bcd0 2023-10-06T03:02:23.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebeffd700 1 --2- 172.21.15.92:0/2368845923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0140df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00099b0 tx=0x7f0eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4559db568737f83d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebe7fc700 1 -- 172.21.15.92:0/2368845923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0009ea0 con 0x7f0ec006bcd0 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebe7fc700 1 -- 172.21.15.92:0/2368845923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb000b860 con 0x7f0ec006bcd0 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.310+0000 7f0ebe7fc700 1 -- 172.21.15.92:0/2368845923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000bc00 con 0x7f0ec006bcd0 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 -- 172.21.15.92:0/2368845923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec006bcd0 msgr2=0x7f0ec0140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 --2- 172.21.15.92:0/2368845923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0140df0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00099b0 tx=0x7f0eb0009250 comp rx=0 tx=0).stop 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 -- 172.21.15.92:0/2368845923 shutdown_connections 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 --2- 172.21.15.92:0/2368845923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0140df0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 --2- 172.21.15.92:0/2368845923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec00c9510 0x7f0ec0141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 -- 172.21.15.92:0/2368845923 >> 172.21.15.92:0/2368845923 conn(0x7f0ec00ce600 msgr2=0x7f0ec00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 -- 172.21.15.92:0/2368845923 shutdown_connections 2023-10-06T03:02:23.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.311+0000 7f0ec577e700 1 -- 172.21.15.92:0/2368845923 wait complete. 2023-10-06T03:02:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.312+0000 7f0ec577e700 1 Processor -- start 2023-10-06T03:02:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.312+0000 7f0ec577e700 1 -- start start 2023-10-06T03:02:23.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006c0a0 0x7f0ec013c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 0x7f0ec015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec016a280 con 0x7f0ec00c9510 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00697d0 con 0x7f0ec006bcd0 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0069ab0 con 0x7f0ec006c0a0 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 0x7f0ec015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 0x7f0ec015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53618/0 (socket says 172.21.15.92:53618) 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 -- 172.21.15.92:0/789404744 learned_addr learned my addr 172.21.15.92:0/789404744 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebeffd700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0eb67fc700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006c0a0 0x7f0ec013c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 -- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006c0a0 msgr2=0x7f0ec013c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006c0a0 0x7f0ec013c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 -- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec006bcd0 msgr2=0x7f0ec0169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0169d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 -- 172.21.15.92:0/789404744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb00094f0 con 0x7f0ec00c9510 2023-10-06T03:02:23.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebeffd700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0eb67fc700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006c0a0 0x7f0ec013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ebf7fe700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 0x7f0ec015d070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009df0 tx=0x7f0eb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800cb50 con 0x7f0ec00c9510 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb800f040 con 0x7f0ec00c9510 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec0069d90 con 0x7f0ec00c9510 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8014480 con 0x7f0ec00c9510 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.313+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec013ca50 con 0x7f0ec00c9510 2023-10-06T03:02:23.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.314+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec004c3c0 con 0x7f0ec00c9510 2023-10-06T03:02:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.315+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0eb8014830 con 0x7f0ec00c9510 2023-10-06T03:02:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.315+0000 7f0eb77fe700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ea0066f30 0x7f0ea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.315+0000 7f0ebeffd700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ea0066f30 0x7f0ea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.315+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0eb8085040 con 0x7f0ec00c9510 2023-10-06T03:02:23.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.316+0000 7f0ebeffd700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ea0066f30 0x7f0ea00693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00099b0 tx=0x7f0eb0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:23.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.319+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eb80524a0 con 0x7f0ec00c9510 2023-10-06T03:02:23.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.576+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f0ec015b3d0 con 0x7f0ec00c9510 2023-10-06T03:02:23.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.580+0000 7f0eb77fe700 1 -- 172.21.15.92:0/789404744 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0eb8055ac0 con 0x7f0ec00c9510 2023-10-06T03:02:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ea0066f30 msgr2=0x7f0ea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ea0066f30 0x7f0ea00693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00099b0 tx=0x7f0eb0017000 comp rx=0 tx=0).stop 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 msgr2=0x7f0ec015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 0x7f0ec015d070 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009df0 tx=0x7f0eb800b040 comp rx=0 tx=0).stop 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 shutdown_connections 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ea0066f30 0x7f0ea00693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0ec00c9510 0x7f0ec015d070 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec006bcd0 0x7f0ec0169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 --2- 172.21.15.92:0/789404744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006c0a0 0x7f0ec013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 >> 172.21.15.92:0/789404744 conn(0x7f0ec00ce600 msgr2=0x7f0ec00ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 shutdown_connections 2023-10-06T03:02:23.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.582+0000 7f0ec577e700 1 -- 172.21.15.92:0/789404744 wait complete. 2023-10-06T03:02:23.596 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T03:02:23.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.711+0000 7fb36295d700 1 Processor -- start 2023-10-06T03:02:23.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.711+0000 7fb36295d700 1 -- start start 2023-10-06T03:02:23.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb36295d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb35c159890 0x7fb35c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb36295d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb36295d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c15a160 con 0x7fb35c15db10 2023-10-06T03:02:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb36295d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c0d13a0 con 0x7fb35c159890 2023-10-06T03:02:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb36295d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c15e140 con 0x7fb35c159c60 2023-10-06T03:02:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb35c159890 0x7fb35c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb360efa700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb35c15db10 0x7fb35c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35534/0 (socket says 172.21.15.92:35534) 2023-10-06T03:02:23.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb360efa700 1 -- 172.21.15.92:0/2279725134 learned_addr learned my addr 172.21.15.92:0/2279725134 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:23.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35bfff700 1 -- 172.21.15.92:0/2279725134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 msgr2=0x7fb35c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:23.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35b7fe700 1 --2- 172.21.15.92:0/2279725134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35bfff700 1 --2- 172.21.15.92:0/2279725134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35affd700 1 -- 172.21.15.92:0/2279725134 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221390539 0 0) 0x7fb35c15a160 con 0x7fb35c15db10 2023-10-06T03:02:23.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35bfff700 1 -- 172.21.15.92:0/2279725134 >> v1:172.21.15.92:6789/0 conn(0x7fb35c15db10 legacy=0x7fb35c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35bfff700 1 -- 172.21.15.92:0/2279725134 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c15a3d0 con 0x7fb35c159890 2023-10-06T03:02:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.712+0000 7fb35bfff700 1 --2- 172.21.15.92:0/2279725134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c159890 0x7fb35c15abd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0098e0 tx=0x7fb34c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223bfa5496704057 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb35affd700 1 -- 172.21.15.92:0/2279725134 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c009e80 con 0x7fb35c159890 2023-10-06T03:02:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb35affd700 1 -- 172.21.15.92:0/2279725134 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c00b860 con 0x7fb35c159890 2023-10-06T03:02:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb35affd700 1 -- 172.21.15.92:0/2279725134 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c00bbe0 con 0x7fb35c159890 2023-10-06T03:02:23.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 -- 172.21.15.92:0/2279725134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c159890 msgr2=0x7fb35c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 --2- 172.21.15.92:0/2279725134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c159890 0x7fb35c15abd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0098e0 tx=0x7fb34c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 -- 172.21.15.92:0/2279725134 shutdown_connections 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 --2- 172.21.15.92:0/2279725134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c159890 0x7fb35c15abd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 --2- 172.21.15.92:0/2279725134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 -- 172.21.15.92:0/2279725134 >> 172.21.15.92:0/2279725134 conn(0x7fb35c068e20 msgr2=0x7fb35c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 -- 172.21.15.92:0/2279725134 shutdown_connections 2023-10-06T03:02:23.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.713+0000 7fb36295d700 1 -- 172.21.15.92:0/2279725134 wait complete. 2023-10-06T03:02:23.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.714+0000 7fb36295d700 1 Processor -- start 2023-10-06T03:02:23.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.714+0000 7fb36295d700 1 -- start start 2023-10-06T03:02:23.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb36295d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb35c159890 0x7fb35c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb36295d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c0c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb36295d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 0x7fb35c0c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb36295d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c14ce60 con 0x7fb35c15db10 2023-10-06T03:02:23.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb36295d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0c6fc0 con 0x7fb35c159890 2023-10-06T03:02:23.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb36295d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0cda90 con 0x7fb35c159c60 2023-10-06T03:02:23.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb360efa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 0x7fb35c0c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb360efa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 0x7fb35c0c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53638/0 (socket says 172.21.15.92:53638) 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb360efa700 1 -- 172.21.15.92:0/4039362312 learned_addr learned my addr 172.21.15.92:0/4039362312 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb35bfff700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb35c159890 0x7fb35c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.715+0000 7fb35b7fe700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c0c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb360efa700 1 -- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 msgr2=0x7fb35c0c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb360efa700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c0c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb360efa700 1 -- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb35c159890 msgr2=0x7fb35c1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb360efa700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb35c159890 0x7fb35c1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb360efa700 1 -- 172.21.15.92:0/4039362312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c009420 con 0x7fb35c15db10 2023-10-06T03:02:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb360efa700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 0x7fb35c0c6990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb35400cbf0 tx=0x7fb3540019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35400a9c0 con 0x7fb35c15db10 2023-10-06T03:02:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35400ab20 con 0x7fb35c15db10 2023-10-06T03:02:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c0cdf20 con 0x7fb35c15db10 2023-10-06T03:02:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3540070f0 con 0x7fb35c15db10 2023-10-06T03:02:23.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.716+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c0cd440 con 0x7fb35c15db10 2023-10-06T03:02:23.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.717+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c0cdc00 con 0x7fb35c15db10 2023-10-06T03:02:23.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.717+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb354007520 con 0x7fb35c15db10 2023-10-06T03:02:23.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.718+0000 7fb343fff700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb348066f30 0x7fb3480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:23.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.718+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb354084720 con 0x7fb35c15db10 2023-10-06T03:02:23.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.718+0000 7fb35bfff700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb348066f30 0x7fb3480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:23.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.718+0000 7fb35bfff700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb348066f30 0x7fb3480693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0091e0 tx=0x7fb34c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:23.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.722+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb354051c00 con 0x7fb35c15db10 2023-10-06T03:02:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.981+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb35c04c3c0 con 0x7fb35c15db10 2023-10-06T03:02:23.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.987+0000 7fb343fff700 1 -- 172.21.15.92:0/4039362312 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb354055220 con 0x7fb35c15db10 2023-10-06T03:02:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb348066f30 msgr2=0x7fb3480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb348066f30 0x7fb3480693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0091e0 tx=0x7fb34c017000 comp rx=0 tx=0).stop 2023-10-06T03:02:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 msgr2=0x7fb35c0c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 0x7fb35c0c6990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb35400cbf0 tx=0x7fb3540019b0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 shutdown_connections 2023-10-06T03:02:23.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb348066f30 0x7fb3480693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb35c15db10 0x7fb35c0c6990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb35c159890 0x7fb35c1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 --2- 172.21.15.92:0/4039362312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb35c159c60 0x7fb35c0c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 >> 172.21.15.92:0/4039362312 conn(0x7fb35c068e20 msgr2=0x7fb35c14d680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 shutdown_connections 2023-10-06T03:02:23.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:23.989+0000 7fb36295d700 1 -- 172.21.15.92:0/4039362312 wait complete. 2023-10-06T03:02:24.003 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T03:02:24.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.115+0000 7f1cdae9d700 1 Processor -- start 2023-10-06T03:02:24.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.115+0000 7f1cdae9d700 1 -- start start 2023-10-06T03:02:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cdae9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cd406bcd0 0x7f1cd4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cdae9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd40c9510 0x7f1cd4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cdae9d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd415c2a0 con 0x7f1cd406c0a0 2023-10-06T03:02:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cdae9d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd415b690 con 0x7f1cd406bcd0 2023-10-06T03:02:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cdae9d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd415b970 con 0x7f1cd40c9510 2023-10-06T03:02:24.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cd8c39700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cd406bcd0 0x7f1cd4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cd8c39700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1cd406bcd0 0x7f1cd4140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53648/0 (socket says 172.21.15.92:53648) 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cd8c39700 1 -- 172.21.15.92:0/1533737334 learned_addr learned my addr 172.21.15.92:0/1533737334 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.116+0000 7f1cd943a700 1 --2- 172.21.15.92:0/1533737334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd40c9510 0x7f1cd4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cd8c39700 1 -- 172.21.15.92:0/1533737334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd40c9510 msgr2=0x7f1cd4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cd8c39700 1 --2- 172.21.15.92:0/1533737334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd40c9510 0x7f1cd4141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cd8c39700 1 -- 172.21.15.92:0/1533737334 >> v1:172.21.15.136:6789/0 conn(0x7f1cd406c0a0 legacy=0x7f1cd40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cd8c39700 1 -- 172.21.15.92:0/1533737334 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd415bc50 con 0x7f1cd406bcd0 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cd8c39700 1 --2- 172.21.15.92:0/1533737334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4140df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009f20 tx=0x7f1cd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc77e01f126353f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1ccb7fe700 1 -- 172.21.15.92:0/1533737334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000e070 con 0x7f1cd406bcd0 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1ccb7fe700 1 -- 172.21.15.92:0/1533737334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000ba80 con 0x7f1cd406bcd0 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1ccb7fe700 1 -- 172.21.15.92:0/1533737334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00092e0 con 0x7f1cd406bcd0 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1533737334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd406bcd0 msgr2=0x7f1cd4140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1533737334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4140df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009f20 tx=0x7f1cd0009840 comp rx=0 tx=0).stop 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1533737334 shutdown_connections 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1533737334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4140df0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1533737334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd40c9510 0x7f1cd4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1533737334 >> 172.21.15.92:0/1533737334 conn(0x7f1cd40ce600 msgr2=0x7f1cd40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1533737334 shutdown_connections 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.117+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1533737334 wait complete. 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.118+0000 7f1cdae9d700 1 Processor -- start 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 -- start start 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd406c0a0 0x7f1cd40c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 0x7f1cd4156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40c6420 con 0x7f1cd40c9510 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd416be30 con 0x7f1cd406bcd0 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cdae9d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd416c110 con 0x7f1cd406c0a0 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.119+0000 7f1cd943a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 0x7f1cd4156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 0x7f1cd4156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53658/0 (socket says 172.21.15.92:53658) 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 -- 172.21.15.92:0/1854498318 learned_addr learned my addr 172.21.15.92:0/1854498318 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:24.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd8c39700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1ccbfff700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd406c0a0 0x7f1cd40c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 -- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd406c0a0 msgr2=0x7f1cd40c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd406c0a0 0x7f1cd40c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 -- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd406bcd0 msgr2=0x7f1cd4157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 -- 172.21.15.92:0/1854498318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009ae0 con 0x7f1cd40c9510 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cd943a700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 0x7f1cd4156420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00d860 tx=0x7f1ccc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0125f0 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc012790 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd416c3f0 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc012b50 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.120+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd4156960 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.121+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd40b2650 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.121+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ccc013480 con 0x7f1cd40c9510 2023-10-06T03:02:24.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.122+0000 7f1cc8ff9700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1cc0066f30 0x7f1cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.122+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1ccc084610 con 0x7f1cd40c9510 2023-10-06T03:02:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.122+0000 7f1cd8c39700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1cc0066f30 0x7f1cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.122+0000 7f1cd8c39700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1cc0066f30 0x7f1cc00693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009880 tx=0x7f1cd0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.126+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cd40b2650 con 0x7f1cd40c9510 2023-10-06T03:02:24.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.384+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f1cd40b2650 con 0x7f1cd40c9510 2023-10-06T03:02:24.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.389+0000 7f1cc8ff9700 1 -- 172.21.15.92:0/1854498318 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1cd40b2650 con 0x7f1cd40c9510 2023-10-06T03:02:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1cc0066f30 msgr2=0x7f1cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1cc0066f30 0x7f1cc00693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0009880 tx=0x7f1cd0018000 comp rx=0 tx=0).stop 2023-10-06T03:02:24.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 msgr2=0x7f1cd4156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 0x7f1cd4156420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00d860 tx=0x7f1ccc008040 comp rx=0 tx=0).stop 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 shutdown_connections 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1cc0066f30 0x7f1cc00693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1cd40c9510 0x7f1cd4156420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cd406bcd0 0x7f1cd4157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 --2- 172.21.15.92:0/1854498318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cd406c0a0 0x7f1cd40c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 >> 172.21.15.92:0/1854498318 conn(0x7f1cd40ce600 msgr2=0x7f1cd40c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 shutdown_connections 2023-10-06T03:02:24.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.391+0000 7f1cdae9d700 1 -- 172.21.15.92:0/1854498318 wait complete. 2023-10-06T03:02:24.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:24.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.517+0000 7fca76184700 1 Processor -- start 2023-10-06T03:02:24.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.517+0000 7fca76184700 1 -- start start 2023-10-06T03:02:24.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca76184700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca700ccc40 0x7fca70155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca76184700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca76184700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca700ad930 con 0x7fca70156150 2023-10-06T03:02:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca76184700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca700acd20 con 0x7fca700ccc40 2023-10-06T03:02:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca76184700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca700ad000 con 0x7fca700cd010 2023-10-06T03:02:24.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca6f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca700ccc40 0x7fca70155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca6f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca700ccc40 0x7fca70155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53674/0 (socket says 172.21.15.92:53674) 2023-10-06T03:02:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.518+0000 7fca6f7fe700 1 -- 172.21.15.92:0/2127986118 learned_addr learned my addr 172.21.15.92:0/2127986118 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6f7fe700 1 -- 172.21.15.92:0/2127986118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 msgr2=0x7fca700b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6ffff700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6f7fe700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6f7fe700 1 -- 172.21.15.92:0/2127986118 >> v1:172.21.15.136:6789/0 conn(0x7fca70156150 legacy=0x7fca700ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6f7fe700 1 -- 172.21.15.92:0/2127986118 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700ad2e0 con 0x7fca700ccc40 2023-10-06T03:02:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6ffff700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6f7fe700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca700ccc40 0x7fca70155c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d1a0 tx=0x7fca6000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65ee56226d332c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6e7fc700 1 -- 172.21.15.92:0/2127986118 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60010070 con 0x7fca700ccc40 2023-10-06T03:02:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6e7fc700 1 -- 172.21.15.92:0/2127986118 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6000de80 con 0x7fca700ccc40 2023-10-06T03:02:24.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.519+0000 7fca6e7fc700 1 -- 172.21.15.92:0/2127986118 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6000b320 con 0x7fca700ccc40 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 -- 172.21.15.92:0/2127986118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca700ccc40 msgr2=0x7fca70155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca700ccc40 0x7fca70155c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fca6000d1a0 tx=0x7fca6000ba50 comp rx=0 tx=0).stop 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 -- 172.21.15.92:0/2127986118 shutdown_connections 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca700ccc40 0x7fca70155c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 --2- 172.21.15.92:0/2127986118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 -- 172.21.15.92:0/2127986118 >> 172.21.15.92:0/2127986118 conn(0x7fca7006a320 msgr2=0x7fca700cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:24.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 -- 172.21.15.92:0/2127986118 shutdown_connections 2023-10-06T03:02:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.520+0000 7fca76184700 1 -- 172.21.15.92:0/2127986118 wait complete. 2023-10-06T03:02:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.521+0000 7fca76184700 1 Processor -- start 2023-10-06T03:02:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.521+0000 7fca76184700 1 -- start start 2023-10-06T03:02:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca700ccc40 0x7fca7014b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 0x7fca70165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca700a54c0 con 0x7fca70156150 2023-10-06T03:02:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7014c600 con 0x7fca700ccc40 2023-10-06T03:02:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7014d460 con 0x7fca700cd010 2023-10-06T03:02:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 0x7fca70165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 0x7fca70165440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53682/0 (socket says 172.21.15.92:53682) 2023-10-06T03:02:24.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 -- 172.21.15.92:0/13506500 learned_addr learned my addr 172.21.15.92:0/13506500 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6f7fe700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca700ccc40 0x7fca7014b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6effd700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 -- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 msgr2=0x7fca700c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 -- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca700ccc40 msgr2=0x7fca7014b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca700ccc40 0x7fca7014b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 -- 172.21.15.92:0/13506500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6000bcf0 con 0x7fca70156150 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca6ffff700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 0x7fca70165440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca6800c880 tx=0x7fca68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011600 con 0x7fca70156150 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68011760 con 0x7fca70156150 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68011b20 con 0x7fca70156150 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7014f1b0 con 0x7fca70156150 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.522+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7014ce10 con 0x7fca70156150 2023-10-06T03:02:24.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.523+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7014c8e0 con 0x7fca70156150 2023-10-06T03:02:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.524+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca680193b0 con 0x7fca70156150 2023-10-06T03:02:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.524+0000 7fca577fe700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca5c066f30 0x7fca5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.524+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fca68083db0 con 0x7fca70156150 2023-10-06T03:02:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.524+0000 7fca6f7fe700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca5c066f30 0x7fca5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.524+0000 7fca6f7fe700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca5c066f30 0x7fca5c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6000ba90 tx=0x7fca60019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.528+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca68054ca0 con 0x7fca70156150 2023-10-06T03:02:24.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.715+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca7004c3c0 con 0x7fca70156150 2023-10-06T03:02:24.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.716+0000 7fca577fe700 1 -- 172.21.15.92:0/13506500 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13650 (secure 0 0 0) 0x7fca68058e60 con 0x7fca70156150 2023-10-06T03:02:24.718 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:24.718 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:21.565231+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca5c066f30 msgr2=0x7fca5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca5c066f30 0x7fca5c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca6000ba90 tx=0x7fca60019000 comp rx=0 tx=0).stop 2023-10-06T03:02:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 msgr2=0x7fca70165440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 0x7fca70165440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca6800c880 tx=0x7fca68008040 comp rx=0 tx=0).stop 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 shutdown_connections 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca5c066f30 0x7fca5c0693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca70156150 0x7fca70165440 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca700ccc40 0x7fca7014b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 --2- 172.21.15.92:0/13506500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca700cd010 0x7fca700c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 >> 172.21.15.92:0/13506500 conn(0x7fca7006a320 msgr2=0x7fca700b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 shutdown_connections 2023-10-06T03:02:24.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.718+0000 7fca76184700 1 -- 172.21.15.92:0/13506500 wait complete. 2023-10-06T03:02:24.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:02:24.732 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:24.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.848+0000 7f2f485c3700 1 Processor -- start 2023-10-06T03:02:24.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.848+0000 7f2f485c3700 1 -- start start 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f485c3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f40159c60 0x7f2f4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f485c3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f4015db10 0x7f2f40069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f485c3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4015a160 con 0x7f2f40159890 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f485c3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4015df50 con 0x7f2f40159c60 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f485c3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400d13a0 con 0x7f2f4015db10 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f40159c60 0x7f2f4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f40159c60 0x7f2f4015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53692/0 (socket says 172.21.15.92:53692) 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 -- 172.21.15.92:0/3519964239 learned_addr learned my addr 172.21.15.92:0/3519964239 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:24.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f46b60700 1 --2- 172.21.15.92:0/3519964239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f4015db10 0x7f2f40069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 -- 172.21.15.92:0/3519964239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f4015db10 msgr2=0x7f2f40069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 --2- 172.21.15.92:0/3519964239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f4015db10 0x7f2f40069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 -- 172.21.15.92:0/3519964239 >> v1:172.21.15.136:6789/0 conn(0x7f2f40159890 legacy=0x7f2f4015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 -- 172.21.15.92:0/3519964239 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4015e140 con 0x7f2f40159c60 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f45b5e700 1 --2- 172.21.15.92:0/3519964239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f40159c60 0x7f2f4015d5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f300099b0 tx=0x7f2f30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1318aa89a465fd7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f4535d700 1 -- 172.21.15.92:0/3519964239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000b820 con 0x7f2f40159c60 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f4535d700 1 -- 172.21.15.92:0/3519964239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3000b980 con 0x7f2f40159c60 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.849+0000 7f2f4535d700 1 -- 172.21.15.92:0/3519964239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000bd20 con 0x7f2f40159c60 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 -- 172.21.15.92:0/3519964239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f40159c60 msgr2=0x7f2f4015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 --2- 172.21.15.92:0/3519964239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f40159c60 0x7f2f4015d5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2f300099b0 tx=0x7f2f30009250 comp rx=0 tx=0).stop 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 -- 172.21.15.92:0/3519964239 shutdown_connections 2023-10-06T03:02:24.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 --2- 172.21.15.92:0/3519964239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f4015db10 0x7f2f40069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 --2- 172.21.15.92:0/3519964239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f40159c60 0x7f2f4015d5d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 -- 172.21.15.92:0/3519964239 >> 172.21.15.92:0/3519964239 conn(0x7f2f40068e20 msgr2=0x7f2f400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 -- 172.21.15.92:0/3519964239 shutdown_connections 2023-10-06T03:02:24.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.850+0000 7f2f485c3700 1 -- 172.21.15.92:0/3519964239 wait complete. 2023-10-06T03:02:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.851+0000 7f2f485c3700 1 Processor -- start 2023-10-06T03:02:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.851+0000 7f2f485c3700 1 -- start start 2023-10-06T03:02:24.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f485c3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f40159890 0x7f2f400b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f485c3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f40159c60 0x7f2f400b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f485c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 0x7f2f40140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f485c3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40169df0 con 0x7f2f4015db10 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f485c3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400b1440 con 0x7f2f40159c60 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f485c3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400b1650 con 0x7f2f40159890 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 0x7f2f40140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 0x7f2f40140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53708/0 (socket says 172.21.15.92:53708) 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 -- 172.21.15.92:0/4036726739 learned_addr learned my addr 172.21.15.92:0/4036726739 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:24.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f4635f700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f40159890 0x7f2f400b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f45b5e700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f40159c60 0x7f2f400b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 -- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f40159890 msgr2=0x7f2f400b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f40159890 0x7f2f400b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 -- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f40159c60 msgr2=0x7f2f400b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f40159c60 0x7f2f400b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 -- 172.21.15.92:0/4036726739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300094f0 con 0x7f2f4015db10 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f45b5e700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f40159c60 0x7f2f400b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f4635f700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f40159890 0x7f2f400b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f46b60700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 0x7f2f40140b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f38007f20 tx=0x7f2f38009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800eca0 con 0x7f2f4015db10 2023-10-06T03:02:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.852+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3800ee20 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.853+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f400b1930 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.853+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38014670 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.853+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f400b1ef0 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.853+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f400cd3b0 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.854+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f380149e0 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.854+0000 7f2f367fc700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f24066f30 0x7f2f240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.854+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2f38084a70 con 0x7f2f4015db10 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.854+0000 7f2f4635f700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f24066f30 0x7f2f240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:24.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.855+0000 7f2f4635f700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f24066f30 0x7f2f240693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c002880 tx=0x7f2f3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:24.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:24.858+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f400cd3b0 con 0x7f2f4015db10 2023-10-06T03:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.040+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f400cd3b0 con 0x7f2f4015db10 2023-10-06T03:02:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.041+0000 7f2f367fc700 1 -- 172.21.15.92:0/4036726739 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f400cd3b0 con 0x7f2f4015db10 2023-10-06T03:02:25.043 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f24066f30 msgr2=0x7f2f240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f24066f30 0x7f2f240693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c002880 tx=0x7f2f3c009040 comp rx=0 tx=0).stop 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 msgr2=0x7f2f40140b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 0x7f2f40140b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f38007f20 tx=0x7f2f38009860 comp rx=0 tx=0).stop 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 shutdown_connections 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f24066f30 0x7f2f240693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f4015db10 0x7f2f40140b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f40159890 0x7f2f400b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 --2- 172.21.15.92:0/4036726739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f40159c60 0x7f2f400b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 >> 172.21.15.92:0/4036726739 conn(0x7f2f40068e20 msgr2=0x7f2f40153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:25.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 shutdown_connections 2023-10-06T03:02:25.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.043+0000 7f2f485c3700 1 -- 172.21.15.92:0/4036726739 wait complete. 2023-10-06T03:02:25.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:25.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.171+0000 7fa045b13700 1 Processor -- start 2023-10-06T03:02:25.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 -- start start 2023-10-06T03:02:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006bcd0 0x7fa040140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa04006c0a0 0x7fa040141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa0400c8e00 0x7fa0400c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04015c2a0 con 0x7fa04006c0a0 2023-10-06T03:02:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04015b690 con 0x7fa0400c8e00 2023-10-06T03:02:25.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa045b13700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04015b970 con 0x7fa04006bcd0 2023-10-06T03:02:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.172+0000 7fa03effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa04006c0a0 0x7fa040141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa04006c0a0 0x7fa040141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53716/0 (socket says 172.21.15.92:53716) 2023-10-06T03:02:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 -- 172.21.15.92:0/771891881 learned_addr learned my addr 172.21.15.92:0/771891881 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03f7fe700 1 --2- 172.21.15.92:0/771891881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006bcd0 0x7fa040140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03ffff700 1 --2- 172.21.15.92:0/771891881 >> v2:172.21.15.136:3300/0 conn(0x7fa0400c8e00 0x7fa0400c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 -- 172.21.15.92:0/771891881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006bcd0 msgr2=0x7fa040140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 --2- 172.21.15.92:0/771891881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006bcd0 0x7fa040140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 -- 172.21.15.92:0/771891881 >> v2:172.21.15.136:3300/0 conn(0x7fa0400c8e00 msgr2=0x7fa0400c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 --2- 172.21.15.92:0/771891881 >> v2:172.21.15.136:3300/0 conn(0x7fa0400c8e00 0x7fa0400c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 -- 172.21.15.92:0/771891881 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04015bc50 con 0x7fa04006c0a0 2023-10-06T03:02:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03effd700 1 --2- 172.21.15.92:0/771891881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006c0a0 0x7fa040141330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa04015b530 tx=0x7fa034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc1ca53908e8af1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03e7fc700 1 -- 172.21.15.92:0/771891881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03400b860 con 0x7fa04006c0a0 2023-10-06T03:02:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03e7fc700 1 -- 172.21.15.92:0/771891881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03400b9c0 con 0x7fa04006c0a0 2023-10-06T03:02:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.173+0000 7fa03e7fc700 1 -- 172.21.15.92:0/771891881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03400bd60 con 0x7fa04006c0a0 2023-10-06T03:02:25.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 -- 172.21.15.92:0/771891881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006c0a0 msgr2=0x7fa040141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 --2- 172.21.15.92:0/771891881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006c0a0 0x7fa040141330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa04015b530 tx=0x7fa034009250 comp rx=0 tx=0).stop 2023-10-06T03:02:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 -- 172.21.15.92:0/771891881 shutdown_connections 2023-10-06T03:02:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 --2- 172.21.15.92:0/771891881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006bcd0 0x7fa040140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 --2- 172.21.15.92:0/771891881 >> v2:172.21.15.136:3300/0 conn(0x7fa0400c8e00 0x7fa0400c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 --2- 172.21.15.92:0/771891881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006c0a0 0x7fa040141330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 -- 172.21.15.92:0/771891881 >> 172.21.15.92:0/771891881 conn(0x7fa0400ce600 msgr2=0x7fa0400cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 -- 172.21.15.92:0/771891881 shutdown_connections 2023-10-06T03:02:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.174+0000 7fa045b13700 1 -- 172.21.15.92:0/771891881 wait complete. 2023-10-06T03:02:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.175+0000 7fa045b13700 1 Processor -- start 2023-10-06T03:02:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.175+0000 7fa045b13700 1 -- start start 2023-10-06T03:02:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 0x7fa04013ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006c0a0 0x7fa04013e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0400c8e00 0x7fa0400b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400b3820 con 0x7fa04006bcd0 2023-10-06T03:02:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400b1e70 con 0x7fa0400c8e00 2023-10-06T03:02:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0400b3580 con 0x7fa04006c0a0 2023-10-06T03:02:25.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 0x7fa04013ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 0x7fa04013ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53720/0 (socket says 172.21.15.92:53720) 2023-10-06T03:02:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 -- 172.21.15.92:0/4051618803 learned_addr learned my addr 172.21.15.92:0/4051618803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03effd700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006c0a0 0x7fa04013e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03ffff700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0400c8e00 0x7fa0400b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 -- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006c0a0 msgr2=0x7fa04013e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006c0a0 0x7fa04013e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 -- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0400c8e00 msgr2=0x7fa0400b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0400c8e00 0x7fa0400b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 -- 172.21.15.92:0/4051618803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340094f0 con 0x7fa04006bcd0 2023-10-06T03:02:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03effd700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006c0a0 0x7fa04013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:25.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa03f7fe700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 0x7fa04013ec10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e190 tx=0x7fa030009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03000edd0 con 0x7fa04006bcd0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa030014420 con 0x7fa04006bcd0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0400b1540 con 0x7fa04006bcd0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0300147e0 con 0x7fa04006bcd0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.176+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0400b1b00 con 0x7fa04006bcd0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.177+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04004c3c0 con 0x7fa04006bcd0 2023-10-06T03:02:25.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.178+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa030014b50 con 0x7fa04006bcd0 2023-10-06T03:02:25.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.178+0000 7fa0277fe700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa02c066f30 0x7fa02c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.178+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa030084c00 con 0x7fa04006bcd0 2023-10-06T03:02:25.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.178+0000 7fa03effd700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa02c066f30 0x7fa02c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.178+0000 7fa03effd700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa02c066f30 0x7fa02c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa04015b530 tx=0x7fa034017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.182+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa03008b080 con 0x7fa04006bcd0 2023-10-06T03:02:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.367+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa04006c9d0 con 0x7fa04006bcd0 2023-10-06T03:02:25.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.367+0000 7fa0277fe700 1 -- 172.21.15.92:0/4051618803 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa030052060 con 0x7fa04006bcd0 2023-10-06T03:02:25.369 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa02c066f30 msgr2=0x7fa02c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa02c066f30 0x7fa02c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa04015b530 tx=0x7fa034017000 comp rx=0 tx=0).stop 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 msgr2=0x7fa04013ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 0x7fa04013ec10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa03000e190 tx=0x7fa030009900 comp rx=0 tx=0).stop 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 shutdown_connections 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa02c066f30 0x7fa02c0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa04006bcd0 0x7fa04013ec10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0400c8e00 0x7fa0400b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 --2- 172.21.15.92:0/4051618803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa04006c0a0 0x7fa04013e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 >> 172.21.15.92:0/4051618803 conn(0x7fa0400ce600 msgr2=0x7fa0400c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.369+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 shutdown_connections 2023-10-06T03:02:25.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.370+0000 7fa045b13700 1 -- 172.21.15.92:0/4051618803 wait complete. 2023-10-06T03:02:25.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:02:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.496+0000 7fbc5638d700 1 Processor -- start 2023-10-06T03:02:25.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.496+0000 7fbc5638d700 1 -- start start 2023-10-06T03:02:25.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc5638d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc50159c60 0x7fbc5015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc5638d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbc5015db10 0x7fbc50069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc5638d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5015a160 con 0x7fbc50159890 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc5638d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5015df50 con 0x7fbc50159c60 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc5638d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500d13a0 con 0x7fbc5015db10 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc50159c60 0x7fbc5015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc4ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbc50159890 0x7fbc5015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35540/0 (socket says 172.21.15.92:35540) 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc4ffff700 1 -- 172.21.15.92:0/3497242479 learned_addr learned my addr 172.21.15.92:0/3497242479 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.497+0000 7fbc5492a700 1 --2- 172.21.15.92:0/3497242479 >> v2:172.21.15.136:3300/0 conn(0x7fbc5015db10 0x7fbc50069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4f7fe700 1 -- 172.21.15.92:0/3497242479 >> v2:172.21.15.136:3300/0 conn(0x7fbc5015db10 msgr2=0x7fbc50069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4f7fe700 1 --2- 172.21.15.92:0/3497242479 >> v2:172.21.15.136:3300/0 conn(0x7fbc5015db10 0x7fbc50069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4f7fe700 1 -- 172.21.15.92:0/3497242479 >> v1:172.21.15.92:6789/0 conn(0x7fbc50159890 legacy=0x7fbc5015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4f7fe700 1 -- 172.21.15.92:0/3497242479 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5015e140 con 0x7fbc50159c60 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4effd700 1 -- 172.21.15.92:0/3497242479 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410506592 0 0) 0x7fbc5015a160 con 0x7fbc50159890 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4f7fe700 1 --2- 172.21.15.92:0/3497242479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50159c60 0x7fbc5015d5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbc5006a890 tx=0x7fbc440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a7e11f58ba94c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4effd700 1 -- 172.21.15.92:0/3497242479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400b6b0 con 0x7fbc50159c60 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4effd700 1 -- 172.21.15.92:0/3497242479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400b810 con 0x7fbc50159c60 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc4effd700 1 -- 172.21.15.92:0/3497242479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400bbb0 con 0x7fbc50159c60 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 -- 172.21.15.92:0/3497242479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50159c60 msgr2=0x7fbc5015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 --2- 172.21.15.92:0/3497242479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50159c60 0x7fbc5015d5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbc5006a890 tx=0x7fbc440091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 -- 172.21.15.92:0/3497242479 shutdown_connections 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 --2- 172.21.15.92:0/3497242479 >> v2:172.21.15.136:3300/0 conn(0x7fbc5015db10 0x7fbc50069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 --2- 172.21.15.92:0/3497242479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc50159c60 0x7fbc5015d5d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 -- 172.21.15.92:0/3497242479 >> 172.21.15.92:0/3497242479 conn(0x7fbc50068e20 msgr2=0x7fbc500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 -- 172.21.15.92:0/3497242479 shutdown_connections 2023-10-06T03:02:25.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.498+0000 7fbc5638d700 1 -- 172.21.15.92:0/3497242479 wait complete. 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.499+0000 7fbc5638d700 1 Processor -- start 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 -- start start 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc50159890 0x7fbc5014b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc50159c60 0x7fbc500cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 0x7fbc50151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500b1fb0 con 0x7fbc5015db10 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500c5ff0 con 0x7fbc50159890 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.500+0000 7fbc5638d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500c56c0 con 0x7fbc50159c60 2023-10-06T03:02:25.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 0x7fbc50151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 0x7fbc50151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53738/0 (socket says 172.21.15.92:53738) 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 -- 172.21.15.92:0/2253282807 learned_addr learned my addr 172.21.15.92:0/2253282807 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc4f7fe700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc50159c60 0x7fbc500cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc4ffff700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc50159890 0x7fbc5014b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 -- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc50159c60 msgr2=0x7fbc500cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc50159c60 0x7fbc500cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 -- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc50159890 msgr2=0x7fbc5014b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc50159890 0x7fbc5014b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 -- 172.21.15.92:0/2253282807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44009420 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5492a700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 0x7fbc50151d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007f20 tx=0x7fbc48009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800eb90 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4800ed10 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc500c59a0 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48014670 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.501+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5014bdd0 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.502+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc500c6160 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.503+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc480149e0 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.503+0000 7fbc37fff700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc3c066f30 0x7fbc3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.503+0000 7fbc4ffff700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc3c066f30 0x7fbc3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.503+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbc48084940 con 0x7fbc5015db10 2023-10-06T03:02:25.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.503+0000 7fbc4ffff700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc3c066f30 0x7fbc3c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc40007fd0 tx=0x7fbc40007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.507+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc48051da0 con 0x7fbc5015db10 2023-10-06T03:02:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.693+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbc5004c3c0 con 0x7fbc5015db10 2023-10-06T03:02:25.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.693+0000 7fbc37fff700 1 -- 172.21.15.92:0/2253282807 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+126 (secure 0 0 0) 0x7fbc480553c0 con 0x7fbc5015db10 2023-10-06T03:02:25.695 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:25.695 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:25.696 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:02:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc3c066f30 msgr2=0x7fbc3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc3c066f30 0x7fbc3c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc40007fd0 tx=0x7fbc40007d30 comp rx=0 tx=0).stop 2023-10-06T03:02:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 msgr2=0x7fbc50151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 0x7fbc50151d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007f20 tx=0x7fbc48009860 comp rx=0 tx=0).stop 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 shutdown_connections 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc3c066f30 0x7fbc3c0693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc5015db10 0x7fbc50151d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc50159890 0x7fbc5014b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 --2- 172.21.15.92:0/2253282807 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc50159c60 0x7fbc500cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 >> 172.21.15.92:0/2253282807 conn(0x7fbc50068e20 msgr2=0x7fbc501573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 shutdown_connections 2023-10-06T03:02:25.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.695+0000 7fbc5638d700 1 -- 172.21.15.92:0/2253282807 wait complete. 2023-10-06T03:02:25.709 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:02:25.709 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:02:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.826+0000 7ff618dbf700 1 Processor -- start 2023-10-06T03:02:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.826+0000 7ff618dbf700 1 -- start start 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff618dbf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6140cd910 0x7ff6140ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff618dbf700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff6140b55a0 0x7ff6140ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff618dbf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6140b5b00 con 0x7ff614159ec0 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff618dbf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6140b5da0 con 0x7ff6140cd910 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff618dbf700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6140b6080 con 0x7ff6140b55a0 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff611d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff6140cd910 0x7ff6140ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff61259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff614159ec0 0x7ff6140b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35544/0 (socket says 172.21.15.92:35544) 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff61259c700 1 -- 172.21.15.92:0/3815283629 learned_addr learned my addr 172.21.15.92:0/3815283629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:25.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff612d9d700 1 --2- 172.21.15.92:0/3815283629 >> v2:172.21.15.136:3300/0 conn(0x7ff6140b55a0 0x7ff6140ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff611d9b700 1 -- 172.21.15.92:0/3815283629 >> v2:172.21.15.136:3300/0 conn(0x7ff6140b55a0 msgr2=0x7ff6140ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff611d9b700 1 --2- 172.21.15.92:0/3815283629 >> v2:172.21.15.136:3300/0 conn(0x7ff6140b55a0 0x7ff6140ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff611d9b700 1 -- 172.21.15.92:0/3815283629 >> v1:172.21.15.92:6789/0 conn(0x7ff614159ec0 legacy=0x7ff6140b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff611d9b700 1 -- 172.21.15.92:0/3815283629 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6140cc100 con 0x7ff6140cd910 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff61159a700 1 -- 172.21.15.92:0/3815283629 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112056416 0 0) 0x7ff6140b5b00 con 0x7ff614159ec0 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff611d9b700 1 --2- 172.21.15.92:0/3815283629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140cd910 0x7ff6140ce340 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098e0 tx=0x7ff6080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e5e05a56433ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff61159a700 1 -- 172.21.15.92:0/3815283629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608009e80 con 0x7ff6140cd910 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff61159a700 1 -- 172.21.15.92:0/3815283629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800b820 con 0x7ff6140cd910 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.827+0000 7ff61159a700 1 -- 172.21.15.92:0/3815283629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800bba0 con 0x7ff6140cd910 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 -- 172.21.15.92:0/3815283629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140cd910 msgr2=0x7ff6140ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 --2- 172.21.15.92:0/3815283629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140cd910 0x7ff6140ce340 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098e0 tx=0x7ff6080091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 -- 172.21.15.92:0/3815283629 shutdown_connections 2023-10-06T03:02:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 --2- 172.21.15.92:0/3815283629 >> v2:172.21.15.136:3300/0 conn(0x7ff6140b55a0 0x7ff6140ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 --2- 172.21.15.92:0/3815283629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140cd910 0x7ff6140ce340 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 -- 172.21.15.92:0/3815283629 >> 172.21.15.92:0/3815283629 conn(0x7ff6140999f0 msgr2=0x7ff6140b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 -- 172.21.15.92:0/3815283629 shutdown_connections 2023-10-06T03:02:25.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.828+0000 7ff618dbf700 1 -- 172.21.15.92:0/3815283629 wait complete. 2023-10-06T03:02:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.829+0000 7ff618dbf700 1 Processor -- start 2023-10-06T03:02:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.829+0000 7ff618dbf700 1 -- start start 2023-10-06T03:02:25.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff618dbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 0x7ff614144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff618dbf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff614159ec0 0x7ff614145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff618dbf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614069360 0x7ff614069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff618dbf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff614146680 con 0x7ff6140b55a0 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff618dbf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff614145a70 con 0x7ff614159ec0 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff618dbf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff614145d50 con 0x7ff614069360 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 0x7ff614144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 0x7ff614144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53758/0 (socket says 172.21.15.92:53758) 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 -- 172.21.15.92:0/1002323737 learned_addr learned my addr 172.21.15.92:0/1002323737 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 -- 172.21.15.92:0/1002323737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614069360 msgr2=0x7ff614069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614069360 0x7ff614069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 -- 172.21.15.92:0/1002323737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff614159ec0 msgr2=0x7ff614145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff614159ec0 0x7ff614145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 -- 172.21.15.92:0/1002323737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608009420 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff61259c700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 0x7ff614144f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff60400c790 tx=0x7ff604007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.830+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040072e0 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.831+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604007460 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.831+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040057f0 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.831+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff614146030 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.831+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6141572f0 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.831+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff614063730 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.832+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff604005be0 con 0x7ff6140b55a0 2023-10-06T03:02:25.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.832+0000 7ff6027fc700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4066f30 0x7ff5f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.832+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff604083780 con 0x7ff6140b55a0 2023-10-06T03:02:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.832+0000 7ff611d9b700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4066f30 0x7ff5f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:25.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.833+0000 7ff611d9b700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4066f30 0x7ff5f40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6140c75d0 tx=0x7ff608017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:25.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:25.836+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff60404b5a0 con 0x7ff6140b55a0 2023-10-06T03:02:26.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.017+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff6140cbd80 con 0x7ff6140b55a0 2023-10-06T03:02:26.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.567+0000 7ff6027fc700 1 -- 172.21.15.92:0/1002323737 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff6040170b0 con 0x7ff6140b55a0 2023-10-06T03:02:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4066f30 msgr2=0x7ff5f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4066f30 0x7ff5f40693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6140c75d0 tx=0x7ff608017000 comp rx=0 tx=0).stop 2023-10-06T03:02:26.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 msgr2=0x7ff614144f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 0x7ff614144f00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff60400c790 tx=0x7ff604007c80 comp rx=0 tx=0).stop 2023-10-06T03:02:26.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 shutdown_connections 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4066f30 0x7ff5f40693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff6140b55a0 0x7ff614144f00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614069360 0x7ff614069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 --2- 172.21.15.92:0/1002323737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff614159ec0 0x7ff614145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 >> 172.21.15.92:0/1002323737 conn(0x7ff6140999f0 msgr2=0x7ff6140c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 shutdown_connections 2023-10-06T03:02:26.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.569+0000 7ff618dbf700 1 -- 172.21.15.92:0/1002323737 wait complete. 2023-10-06T03:02:26.575 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:02:26.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:02:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.706+0000 7f8abdaa1700 1 Processor -- start 2023-10-06T03:02:26.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.706+0000 7f8abdaa1700 1 -- start start 2023-10-06T03:02:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8abdaa1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ab8159950 0x7f8ab80cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8abdaa1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80ce380 con 0x7f8ab80af320 2023-10-06T03:02:26.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8abdaa1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80cec20 con 0x7f8ab8069c70 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8abdaa1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab80afa90 con 0x7f8ab8159950 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ab8159950 0x7f8ab80cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab6ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8ab80af320 0x7f8ab8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35546/0 (socket says 172.21.15.92:35546) 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab6ffd700 1 -- 172.21.15.92:0/4135628369 learned_addr learned my addr 172.21.15.92:0/4135628369 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab77fe700 1 -- 172.21.15.92:0/4135628369 >> v1:172.21.15.136:6789/0 conn(0x7f8ab8069c70 legacy=0x7f8ab80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab77fe700 1 -- 172.21.15.92:0/4135628369 >> v1:172.21.15.92:6789/0 conn(0x7f8ab80af320 legacy=0x7f8ab8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:26.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab77fe700 1 -- 172.21.15.92:0/4135628369 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80ce5d0 con 0x7f8ab8159950 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.707+0000 7f8ab67fc700 1 -- 172.21.15.92:0/4135628369 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054379787 0 0) 0x7f8ab80ce380 con 0x7f8ab80af320 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8ab77fe700 1 --2- 172.21.15.92:0/4135628369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab8159950 0x7f8ab80cbb90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009e60 tx=0x7f8aa8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73d15d599d748f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8ab67fc700 1 -- 172.21.15.92:0/4135628369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800e070 con 0x7f8ab8159950 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8ab67fc700 1 -- 172.21.15.92:0/4135628369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa800bbf0 con 0x7f8ab8159950 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8ab67fc700 1 -- 172.21.15.92:0/4135628369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8012430 con 0x7f8ab8159950 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 -- 172.21.15.92:0/4135628369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab8159950 msgr2=0x7f8ab80cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/4135628369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab8159950 0x7f8ab80cbb90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009e60 tx=0x7f8aa8009780 comp rx=0 tx=0).stop 2023-10-06T03:02:26.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 -- 172.21.15.92:0/4135628369 shutdown_connections 2023-10-06T03:02:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/4135628369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab8159950 0x7f8ab80cbb90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 -- 172.21.15.92:0/4135628369 >> 172.21.15.92:0/4135628369 conn(0x7f8ab8068e00 msgr2=0x7f8ab80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 -- 172.21.15.92:0/4135628369 shutdown_connections 2023-10-06T03:02:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.708+0000 7f8abdaa1700 1 -- 172.21.15.92:0/4135628369 wait complete. 2023-10-06T03:02:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.709+0000 7f8abdaa1700 1 Processor -- start 2023-10-06T03:02:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.709+0000 7f8abdaa1700 1 -- start start 2023-10-06T03:02:26.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8abdaa1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ab8069c70 0x7f8ab80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8abdaa1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 0x7f8ab80c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8abdaa1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ab8159950 0x7f8ab815ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8abdaa1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b4200 con 0x7f8ab80af320 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8abdaa1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b35f0 con 0x7f8ab8159950 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8abdaa1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b38d0 con 0x7f8ab8069c70 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 0x7f8ab80c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 0x7f8ab80c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53778/0 (socket says 172.21.15.92:53778) 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 -- 172.21.15.92:0/2695275734 learned_addr learned my addr 172.21.15.92:0/2695275734 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:26.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab77fe700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ab8069c70 0x7f8ab80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab7fff700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ab8159950 0x7f8ab815ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 -- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ab8069c70 msgr2=0x7f8ab80ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ab8069c70 0x7f8ab80ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 -- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ab8159950 msgr2=0x7f8ab815ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ab8159950 0x7f8ab815ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.710+0000 7f8ab6ffd700 1 -- 172.21.15.92:0/2695275734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8009a20 con 0x7f8ab80af320 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.711+0000 7f8ab6ffd700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 0x7f8ab80c8520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c7e0 tx=0x7f8aac007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.711+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac005b30 con 0x7f8ab80af320 2023-10-06T03:02:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.711+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005c90 con 0x7f8ab80af320 2023-10-06T03:02:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.711+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab80b3bb0 con 0x7f8ab80af320 2023-10-06T03:02:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.711+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007300 con 0x7f8ab80af320 2023-10-06T03:02:26.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.711+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab80b6160 con 0x7f8ab80af320 2023-10-06T03:02:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.712+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab80b43b0 con 0x7f8ab80af320 2023-10-06T03:02:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.712+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8aac01b400 con 0x7f8ab80af320 2023-10-06T03:02:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.713+0000 7f8a9f7fe700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8aa4066f30 0x7f8aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.713+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f8aac083f60 con 0x7f8ab80af320 2023-10-06T03:02:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.713+0000 7f8ab77fe700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8aa4066f30 0x7f8aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:26.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.713+0000 7f8ab77fe700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8aa4066f30 0x7f8aa40693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80099f0 tx=0x7f8aa8018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:26.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.717+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ab80b43b0 con 0x7f8ab80af320 2023-10-06T03:02:26.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:26.899+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8ab80718b0 con 0x7f8ab80af320 2023-10-06T03:02:27.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.576+0000 7f8a9f7fe700 1 -- 172.21.15.92:0/2695275734 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8aac054d60 con 0x7f8ab80af320 2023-10-06T03:02:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.580+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8aa4066f30 msgr2=0x7f8aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.580+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8aa4066f30 0x7f8aa40693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80099f0 tx=0x7f8aa8018000 comp rx=0 tx=0).stop 2023-10-06T03:02:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.580+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 msgr2=0x7f8ab80c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.580+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 0x7f8ab80c8520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c7e0 tx=0x7f8aac007df0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 shutdown_connections 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8aa4066f30 0x7f8aa40693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ab8159950 0x7f8ab815ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ab8069c70 0x7f8ab80ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 --2- 172.21.15.92:0/2695275734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ab80af320 0x7f8ab80c8520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 >> 172.21.15.92:0/2695275734 conn(0x7f8ab8068e00 msgr2=0x7f8ab80c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 shutdown_connections 2023-10-06T03:02:27.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.581+0000 7f8abdaa1700 1 -- 172.21.15.92:0/2695275734 wait complete. 2023-10-06T03:02:27.588 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:02:27.604 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:02:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.721+0000 7f40c8577700 1 Processor -- start 2023-10-06T03:02:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.722+0000 7f40c8577700 1 -- start start 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.722+0000 7f40c8577700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40c0159890 0x7f40c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.722+0000 7f40c8577700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c0159c60 0x7f40c015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.722+0000 7f40c8577700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c015a160 con 0x7f40c015db10 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c8577700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00d13a0 con 0x7f40c0159890 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c8577700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015e140 con 0x7f40c0159c60 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40c0159890 0x7f40c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:27.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40c0159890 0x7f40c015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53788/0 (socket says 172.21.15.92:53788) 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 -- 172.21.15.92:0/2412497509 learned_addr learned my addr 172.21.15.92:0/2412497509 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c5b12700 1 --2- 172.21.15.92:0/2412497509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c0159c60 0x7f40c015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 -- 172.21.15.92:0/2412497509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c0159c60 msgr2=0x7f40c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 --2- 172.21.15.92:0/2412497509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c0159c60 0x7f40c015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 -- 172.21.15.92:0/2412497509 >> v1:172.21.15.136:6789/0 conn(0x7f40c015db10 legacy=0x7f40c0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 -- 172.21.15.92:0/2412497509 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c015a3d0 con 0x7f40c0159890 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c6313700 1 --2- 172.21.15.92:0/2412497509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159890 0x7f40c015abd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0099b0 tx=0x7f40bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e08dc60576ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c5311700 1 -- 172.21.15.92:0/2412497509 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00b860 con 0x7f40c0159890 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c5311700 1 -- 172.21.15.92:0/2412497509 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00b9c0 con 0x7f40c0159890 2023-10-06T03:02:27.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.723+0000 7f40c5311700 1 -- 172.21.15.92:0/2412497509 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00bd60 con 0x7f40c0159890 2023-10-06T03:02:27.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 -- 172.21.15.92:0/2412497509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159890 msgr2=0x7f40c015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:27.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 --2- 172.21.15.92:0/2412497509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159890 0x7f40c015abd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0099b0 tx=0x7f40bc009250 comp rx=0 tx=0).stop 2023-10-06T03:02:27.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 -- 172.21.15.92:0/2412497509 shutdown_connections 2023-10-06T03:02:27.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 --2- 172.21.15.92:0/2412497509 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159890 0x7f40c015abd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 --2- 172.21.15.92:0/2412497509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c0159c60 0x7f40c015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 -- 172.21.15.92:0/2412497509 >> 172.21.15.92:0/2412497509 conn(0x7f40c0068e20 msgr2=0x7f40c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 -- 172.21.15.92:0/2412497509 shutdown_connections 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.724+0000 7f40c8577700 1 -- 172.21.15.92:0/2412497509 wait complete. 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.725+0000 7f40c8577700 1 Processor -- start 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.725+0000 7f40c8577700 1 -- start start 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40c0159890 0x7f40c014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 0x7f40c014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c015db10 0x7f40c00b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00b39a0 con 0x7f40c0159c60 2023-10-06T03:02:27.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00ce340 con 0x7f40c0159890 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00b5d90 con 0x7f40c015db10 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 0x7f40c014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 0x7f40c014f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53796/0 (socket says 172.21.15.92:53796) 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 -- 172.21.15.92:0/4159818489 learned_addr learned my addr 172.21.15.92:0/4159818489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c6313700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40c0159890 0x7f40c014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 -- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c015db10 msgr2=0x7f40c00b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c015db10 0x7f40c00b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 -- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40c0159890 msgr2=0x7f40c014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40c0159890 0x7f40c014c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 -- 172.21.15.92:0/4159818489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0094f0 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c5b12700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 0x7f40c014f300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40b000d860 tx=0x7f40b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00125f0 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0012750 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c00b6070 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0012ad0 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.726+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c00b3630 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.727+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c00b3b50 con 0x7f40c0159c60 2023-10-06T03:02:27.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.728+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40b001e460 con 0x7f40c0159c60 2023-10-06T03:02:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.728+0000 7f40b67fc700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40a4066f30 0x7f40a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.728+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f40b0084cf0 con 0x7f40c0159c60 2023-10-06T03:02:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.728+0000 7f40c6313700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40a4066f30 0x7f40a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:27.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.729+0000 7f40c6313700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40a4066f30 0x7f40a40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0099b0 tx=0x7f40bc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:27.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.733+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40c00b3b50 con 0x7f40c0159c60 2023-10-06T03:02:27.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:27.960+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f40c004c3c0 con 0x7f40c0159c60 2023-10-06T03:02:28.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.586+0000 7f40b67fc700 1 -- 172.21.15.92:0/4159818489 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f40b0089070 con 0x7f40c0159c60 2023-10-06T03:02:28.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.591+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40a4066f30 msgr2=0x7f40a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.591+0000 7f40c8577700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40a4066f30 0x7f40a40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0099b0 tx=0x7f40bc017000 comp rx=0 tx=0).stop 2023-10-06T03:02:28.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.591+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 msgr2=0x7f40c014f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.591+0000 7f40c8577700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 0x7f40c014f300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40b000d860 tx=0x7f40b0008040 comp rx=0 tx=0).stop 2023-10-06T03:02:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 shutdown_connections 2023-10-06T03:02:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40a4066f30 0x7f40a40693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40c015db10 0x7f40c00b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40c0159890 0x7f40c014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 --2- 172.21.15.92:0/4159818489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40c0159c60 0x7f40c014f300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 >> 172.21.15.92:0/4159818489 conn(0x7f40c0068e20 msgr2=0x7f40c00b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:28.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.595+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 shutdown_connections 2023-10-06T03:02:28.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.596+0000 7f40c8577700 1 -- 172.21.15.92:0/4159818489 wait complete. 2023-10-06T03:02:28.600 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T03:02:28.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:28.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.737+0000 7f49dee9c700 1 Processor -- start 2023-10-06T03:02:28.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dee9c700 1 -- start start 2023-10-06T03:02:28.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dee9c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f49d80c91e0 0x7f49d80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dee9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f49d806ae10 0x7f49d806b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dee9c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d80cc210 con 0x7f49d806aa40 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dee9c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80cc4f0 con 0x7f49d806ae10 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dee9c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80cc7d0 con 0x7f49d80c91e0 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dcc38700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f49d806aa40 0x7f49d806bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35552/0 (socket says 172.21.15.92:35552) 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dcc38700 1 -- 172.21.15.92:0/2870826061 learned_addr learned my addr 172.21.15.92:0/2870826061 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.738+0000 7f49dd439700 1 --2- 172.21.15.92:0/2870826061 >> v2:172.21.15.92:3300/0 conn(0x7f49d806ae10 0x7f49d806b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49d7fff700 1 --2- 172.21.15.92:0/2870826061 >> v2:172.21.15.136:3300/0 conn(0x7f49d80c91e0 0x7f49d80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49d77fe700 1 -- 172.21.15.92:0/2870826061 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453439489 0 0) 0x7f49d80cc210 con 0x7f49d806aa40 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49d77fe700 1 -- 172.21.15.92:0/2870826061 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49c4003570 con 0x7f49d806aa40 2023-10-06T03:02:28.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49dd439700 1 -- 172.21.15.92:0/2870826061 >> v2:172.21.15.136:3300/0 conn(0x7f49d80c91e0 msgr2=0x7f49d80cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49dd439700 1 --2- 172.21.15.92:0/2870826061 >> v2:172.21.15.136:3300/0 conn(0x7f49d80c91e0 0x7f49d80cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49dd439700 1 -- 172.21.15.92:0/2870826061 >> v1:172.21.15.92:6789/0 conn(0x7f49d806aa40 legacy=0x7f49d806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49dd439700 1 -- 172.21.15.92:0/2870826061 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d80ccab0 con 0x7f49d806ae10 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49dd439700 1 --2- 172.21.15.92:0/2870826061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806ae10 0x7f49d806b270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098e0 tx=0x7f49d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e708aa31379c0448 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49d77fe700 1 -- 172.21.15.92:0/2870826061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0011630 con 0x7f49d806ae10 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49d77fe700 1 -- 172.21.15.92:0/2870826061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0011790 con 0x7f49d806ae10 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.739+0000 7f49d77fe700 1 -- 172.21.15.92:0/2870826061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0011b30 con 0x7f49d806ae10 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 -- 172.21.15.92:0/2870826061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806ae10 msgr2=0x7f49d806b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 --2- 172.21.15.92:0/2870826061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806ae10 0x7f49d806b270 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098e0 tx=0x7f49d00091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 -- 172.21.15.92:0/2870826061 shutdown_connections 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 --2- 172.21.15.92:0/2870826061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806ae10 0x7f49d806b270 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 --2- 172.21.15.92:0/2870826061 >> v2:172.21.15.136:3300/0 conn(0x7f49d80c91e0 0x7f49d80cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 -- 172.21.15.92:0/2870826061 >> 172.21.15.92:0/2870826061 conn(0x7f49d80cd670 msgr2=0x7f49d80cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:28.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 -- 172.21.15.92:0/2870826061 shutdown_connections 2023-10-06T03:02:28.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.740+0000 7f49dee9c700 1 -- 172.21.15.92:0/2870826061 wait complete. 2023-10-06T03:02:28.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.741+0000 7f49dee9c700 1 Processor -- start 2023-10-06T03:02:28.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.741+0000 7f49dee9c700 1 -- start start 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 0x7f49d80b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49d806ae10 0x7f49d806b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d80c91e0 0x7f49d80b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b0df0 con 0x7f49d806aa40 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b10d0 con 0x7f49d80c91e0 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80b13b0 con 0x7f49d806ae10 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 0x7f49d80b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 0x7f49d80b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53818/0 (socket says 172.21.15.92:53818) 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 -- 172.21.15.92:0/1118811384 learned_addr learned my addr 172.21.15.92:0/1118811384 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:28.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49d7fff700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49d806ae10 0x7f49d806b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dd439700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d80c91e0 0x7f49d80b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 -- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49d806ae10 msgr2=0x7f49d806b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49d806ae10 0x7f49d806b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 -- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d80c91e0 msgr2=0x7f49d80b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d80c91e0 0x7f49d80b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 -- 172.21.15.92:0/1118811384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009420 con 0x7f49d806aa40 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49d7fff700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49d806ae10 0x7f49d806b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dd439700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d80c91e0 0x7f49d80b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dcc38700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 0x7f49d80b1e90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f49c800cb10 tx=0x7f49c80027e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8005d80 con 0x7f49d806aa40 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c80070f0 con 0x7f49d806aa40 2023-10-06T03:02:28.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d80b1690 con 0x7f49d806aa40 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8007470 con 0x7f49d806aa40 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.742+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d80b23d0 con 0x7f49d806aa40 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.743+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d80b1210 con 0x7f49d806aa40 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.744+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49c80078a0 con 0x7f49d806aa40 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.744+0000 7f49d4ff9700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f49b8066f30 0x7f49b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.744+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f49c8084570 con 0x7f49d806aa40 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.744+0000 7f49d7fff700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f49b8066f30 0x7f49b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.744+0000 7f49d7fff700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f49b8066f30 0x7f49b80693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49d80c7270 tx=0x7f49c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:28.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.748+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49d80b1210 con 0x7f49d806aa40 2023-10-06T03:02:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.933+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49d80b1210 con 0x7f49d806aa40 2023-10-06T03:02:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.934+0000 7f49d4ff9700 1 -- 172.21.15.92:0/1118811384 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f49d80b1210 con 0x7f49d806aa40 2023-10-06T03:02:28.936 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:28.936 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:27.962780+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f49b8066f30 msgr2=0x7f49b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f49b8066f30 0x7f49b80693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f49d80c7270 tx=0x7f49c0008040 comp rx=0 tx=0).stop 2023-10-06T03:02:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 msgr2=0x7f49d80b1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 0x7f49d80b1e90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f49c800cb10 tx=0x7f49c80027e0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 shutdown_connections 2023-10-06T03:02:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f49b8066f30 0x7f49b80693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49d806aa40 0x7f49d80b1e90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49d80c91e0 0x7f49d80b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 --2- 172.21.15.92:0/1118811384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49d806ae10 0x7f49d806b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:28.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 >> 172.21.15.92:0/1118811384 conn(0x7f49d80cd670 msgr2=0x7f49d815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:28.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 shutdown_connections 2023-10-06T03:02:28.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:28.936+0000 7f49dee9c700 1 -- 172.21.15.92:0/1118811384 wait complete. 2023-10-06T03:02:28.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:29.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.063+0000 7f663ce7a700 1 Processor -- start 2023-10-06T03:02:29.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.063+0000 7f663ce7a700 1 -- start start 2023-10-06T03:02:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663ce7a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6638069560 0x7f66380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663ce7a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6638069ee0 con 0x7f6638159c60 2023-10-06T03:02:29.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663ce7a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663815a160 con 0x7f6638159890 2023-10-06T03:02:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663ce7a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66380d13a0 con 0x7f6638069560 2023-10-06T03:02:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6636d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6638069560 0x7f66380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6635d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6638159c60 0x7f663815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35556/0 (socket says 172.21.15.92:35556) 2023-10-06T03:02:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6635d9b700 1 -- 172.21.15.92:0/3036543738 learned_addr learned my addr 172.21.15.92:0/3036543738 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6636d9d700 1 -- 172.21.15.92:0/3036543738 >> v1:172.21.15.136:6789/0 conn(0x7f6638159890 legacy=0x7f663815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:29.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6636d9d700 1 -- 172.21.15.92:0/3036543738 >> v1:172.21.15.92:6789/0 conn(0x7f6638159c60 legacy=0x7f663815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6636d9d700 1 -- 172.21.15.92:0/3036543738 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663815e140 con 0x7f6638069560 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663559a700 1 -- 172.21.15.92:0/3036543738 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796856941 0 0) 0x7f6638069ee0 con 0x7f6638159c60 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f6636d9d700 1 --2- 172.21.15.92:0/3036543738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638069560 0x7f66380699a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66280098e0 tx=0x7f66280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfd094dc6ea188f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663559a700 1 -- 172.21.15.92:0/3036543738 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628009e80 con 0x7f6638069560 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663559a700 1 -- 172.21.15.92:0/3036543738 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662800b820 con 0x7f6638069560 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.064+0000 7f663559a700 1 -- 172.21.15.92:0/3036543738 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662800bba0 con 0x7f6638069560 2023-10-06T03:02:29.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 -- 172.21.15.92:0/3036543738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638069560 msgr2=0x7f66380699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 --2- 172.21.15.92:0/3036543738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638069560 0x7f66380699a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66280098e0 tx=0x7f66280091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 -- 172.21.15.92:0/3036543738 shutdown_connections 2023-10-06T03:02:29.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 --2- 172.21.15.92:0/3036543738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638069560 0x7f66380699a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 -- 172.21.15.92:0/3036543738 >> 172.21.15.92:0/3036543738 conn(0x7f6638068e20 msgr2=0x7f66380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:29.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 -- 172.21.15.92:0/3036543738 shutdown_connections 2023-10-06T03:02:29.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.065+0000 7f663ce7a700 1 -- 172.21.15.92:0/3036543738 wait complete. 2023-10-06T03:02:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.066+0000 7f663ce7a700 1 Processor -- start 2023-10-06T03:02:29.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.066+0000 7f663ce7a700 1 -- start start 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663ce7a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6638069560 0x7f663814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663ce7a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6638159890 0x7f6638153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663ce7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 0x7f6638153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663ce7a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663815c4e0 con 0x7f6638159c60 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663ce7a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66380ce170 con 0x7f6638159890 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663ce7a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66380ce450 con 0x7f6638069560 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 0x7f6638153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 0x7f6638153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:53840/0 (socket says 172.21.15.92:53840) 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 -- 172.21.15.92:0/2575137882 learned_addr learned my addr 172.21.15.92:0/2575137882 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663659c700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6638069560 0x7f663814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6635d9b700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6638159890 0x7f6638153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 -- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6638069560 msgr2=0x7f663814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6638069560 0x7f663814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 -- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6638159890 msgr2=0x7f6638153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6638159890 0x7f6638153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 -- 172.21.15.92:0/2575137882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628009420 con 0x7f6638159c60 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f663659c700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6638069560 0x7f663814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6635d9b700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6638159890 0x7f6638153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.067+0000 7f6636d9d700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 0x7f6638153d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f66280030c0 tx=0x7f6628016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.068+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662801b680 con 0x7f6638159c60 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.068+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662801b7e0 con 0x7f6638159c60 2023-10-06T03:02:29.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.068+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66380ce730 con 0x7f6638159c60 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.068+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662801bba0 con 0x7f6638159c60 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.068+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66380cd5c0 con 0x7f6638159c60 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.068+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663815c650 con 0x7f6638159c60 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.069+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f662801c450 con 0x7f6638159c60 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.069+0000 7f662e7fc700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6618066f30 0x7f66180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.069+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6628086670 con 0x7f6638159c60 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.069+0000 7f663659c700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6618066f30 0x7f66180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.070+0000 7f663659c700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6618066f30 0x7f66180693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6630007fd0 tx=0x7f6630009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.073+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6628053780 con 0x7f6638159c60 2023-10-06T03:02:29.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.257+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f663804c3c0 con 0x7f6638159c60 2023-10-06T03:02:29.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.257+0000 7f662e7fc700 1 -- 172.21.15.92:0/2575137882 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f6628056da0 con 0x7f6638159c60 2023-10-06T03:02:29.259 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:29.260 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":33,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:27.962780+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:29.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.259+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6618066f30 msgr2=0x7f66180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.259+0000 7f663ce7a700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6618066f30 0x7f66180693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6630007fd0 tx=0x7f6630009040 comp rx=0 tx=0).stop 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.259+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 msgr2=0x7f6638153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.259+0000 7f663ce7a700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 0x7f6638153d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f66280030c0 tx=0x7f6628016000 comp rx=0 tx=0).stop 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.259+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 shutdown_connections 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.259+0000 7f663ce7a700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6618066f30 0x7f66180693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.260+0000 7f663ce7a700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6638159c60 0x7f6638153d70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.260+0000 7f663ce7a700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6638069560 0x7f663814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.260+0000 7f663ce7a700 1 --2- 172.21.15.92:0/2575137882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6638159890 0x7f6638153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.260+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 >> 172.21.15.92:0/2575137882 conn(0x7f6638068e20 msgr2=0x7f6638157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:29.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.260+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 shutdown_connections 2023-10-06T03:02:29.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.260+0000 7f663ce7a700 1 -- 172.21.15.92:0/2575137882 wait complete. 2023-10-06T03:02:29.273 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:02:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.390+0000 7fabe5a89700 1 Processor -- start 2023-10-06T03:02:29.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.390+0000 7fabe5a89700 1 -- start start 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.390+0000 7fabe5a89700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 0x7fabe015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabe5a89700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe015a160 con 0x7fabe0159c60 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabe5a89700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe00d13a0 con 0x7fabe0069560 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabe5a89700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015e140 con 0x7fabe0159890 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabdeffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fabe0159c60 0x7fabe015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36082/0 (socket says 172.21.15.92:36082) 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabdeffd700 1 -- 172.21.15.92:0/972502066 learned_addr learned my addr 172.21.15.92:0/972502066 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682602048 0 0) 0x7fabe015a160 con 0x7fabe0159c60 2023-10-06T03:02:29.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 0x7fabe015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc8003570 con 0x7fabe0159c60 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 384673801 0 0) 0x7fabc8003570 con 0x7fabe0159c60 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 msgr2=0x7fabe015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 0x7fabe015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 >> v1:172.21.15.136:6789/0 conn(0x7fabe0069560 legacy=0x7fabe00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe015a3d0 con 0x7fabe0159c60 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fabd4001470 con 0x7fabe0159c60 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 >> v1:172.21.15.92:6789/0 conn(0x7fabe0159c60 legacy=0x7fabe015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc8003a90 0x7fabc8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc8004810 0x7fabc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 0x7fabc80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015e140 con 0x7fabc8003a90 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc800bb60 con 0x7fabc8004810 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc800bd20 con 0x7fabe0159890 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.391+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc8003a90 0x7fabc8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdeffd700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc8004810 0x7fabc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdffff700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 0x7fabc80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdf7fe700 1 -- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 msgr2=0x7fabc80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabe0159890 0x7fabc80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdf7fe700 1 -- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc8004810 msgr2=0x7fabc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc8004810 0x7fabc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdf7fe700 1 -- 172.21.15.92:0/972502066 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8003570 con 0x7fabc8003a90 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc8003a90 0x7fabc8003ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fabd000e210 tx=0x7fabd000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62c1ac23a740bc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000eac0 con 0x7fabc8003a90 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000f040 con 0x7fabc8003a90 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabde7fc700 1 -- 172.21.15.92:0/972502066 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000ee70 con 0x7fabc8003a90 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabe5a89700 1 -- 172.21.15.92:0/972502066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc8003a90 msgr2=0x7fabc8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabe5a89700 1 --2- 172.21.15.92:0/972502066 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabc8003a90 0x7fabc8003ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fabd000e210 tx=0x7fabd000b040 comp rx=0 tx=0).stop 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabdffff700 1 -- 172.21.15.92:0/972502066 reap_dead start 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabe5a89700 1 -- 172.21.15.92:0/972502066 shutdown_connections 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabe5a89700 1 -- 172.21.15.92:0/972502066 >> 172.21.15.92:0/972502066 conn(0x7fabe0068e20 msgr2=0x7fabe00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabe5a89700 1 -- 172.21.15.92:0/972502066 shutdown_connections 2023-10-06T03:02:29.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.392+0000 7fabe5a89700 1 -- 172.21.15.92:0/972502066 wait complete. 2023-10-06T03:02:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 Processor -- start 2023-10-06T03:02:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 -- start start 2023-10-06T03:02:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc8003a90 0x7fabe00ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 0x7fabe00c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabe016d4e0 0x7fabe00ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00c7db0 con 0x7fabe00b3a50 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00ca700 con 0x7fabe016d4e0 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabe5a89700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe016de50 con 0x7fabc8003a90 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.394+0000 7fabdeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 0x7fabe00c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 0x7fabe00c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34166/0 (socket says 172.21.15.92:34166) 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 -- 172.21.15.92:0/1344385361 learned_addr learned my addr 172.21.15.92:0/1344385361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc8003a90 0x7fabe00ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdffff700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabe016d4e0 0x7fabe00ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 -- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc8003a90 msgr2=0x7fabe00ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc8003a90 0x7fabe00ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 -- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabe016d4e0 msgr2=0x7fabe00ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabe016d4e0 0x7fabe00ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:29.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 -- 172.21.15.92:0/1344385361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe016e130 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabdeffd700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 0x7fabe00c8240 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fabd400ebf0 tx=0x7fabd4000d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4005e70 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd400c7e0 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400cba0 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe00c7a40 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.395+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe016d980 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.396+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe00c7f60 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.396+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabd401d460 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.397+0000 7fabc77fe700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fabcc066f60 0x7fabcc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.397+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fabd4084540 con 0x7fabe00b3a50 2023-10-06T03:02:29.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.397+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fabcc066f60 0x7fabcc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.397+0000 7fabdf7fe700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fabcc066f60 0x7fabcc069410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fabd000afd0 tx=0x7fabd001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:29.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.401+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabe00c7f60 con 0x7fabe00b3a50 2023-10-06T03:02:29.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:29.632+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fabe00c7880 con 0x7fabe00b3a50 2023-10-06T03:02:30.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.599+0000 7fabc77fe700 1 -- 172.21.15.92:0/1344385361 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fabd4051650 con 0x7fabe00b3a50 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fabcc066f60 msgr2=0x7fabcc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fabcc066f60 0x7fabcc069410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fabd000afd0 tx=0x7fabd001a000 comp rx=0 tx=0).stop 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 msgr2=0x7fabe00c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 0x7fabe00c8240 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fabd400ebf0 tx=0x7fabd4000d30 comp rx=0 tx=0).stop 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 shutdown_connections 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fabcc066f60 0x7fabcc069410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabe016d4e0 0x7fabe00ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc8003a90 0x7fabe00ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 --2- 172.21.15.92:0/1344385361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fabe00b3a50 0x7fabe00c8240 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 >> 172.21.15.92:0/1344385361 conn(0x7fabe0068e20 msgr2=0x7fabe01540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 shutdown_connections 2023-10-06T03:02:30.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.602+0000 7fabe5a89700 1 -- 172.21.15.92:0/1344385361 wait complete. 2023-10-06T03:02:30.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:02:30.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.738+0000 7f56f5c88700 1 Processor -- start 2023-10-06T03:02:30.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.738+0000 7f56f5c88700 1 -- start start 2023-10-06T03:02:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56f5c88700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 0x7f56f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56f5c88700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f56f0159c60 0x7f56f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56f5c88700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f015a160 con 0x7f56f015db10 2023-10-06T03:02:30.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56f5c88700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00d13a0 con 0x7f56f0159c60 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56f5c88700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015e140 con 0x7f56f0159890 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56effff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f56f015db10 0x7f56f0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36098/0 (socket says 172.21.15.92:36098) 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.739+0000 7f56effff700 1 -- 172.21.15.92:0/4121365757 learned_addr learned my addr 172.21.15.92:0/4121365757 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537386503 0 0) 0x7f56f015a160 con 0x7f56f015db10 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56dc003570 con 0x7f56f015db10 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56ef7fe700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 0x7f56f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 --2- 172.21.15.92:0/4121365757 >> v2:172.21.15.136:3300/0 conn(0x7f56f0159c60 0x7f56f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1682497654 0 0) 0x7f56dc003570 con 0x7f56f015db10 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 msgr2=0x7f56f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 0x7f56f015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 >> v2:172.21.15.136:3300/0 conn(0x7f56f0159c60 msgr2=0x7f56f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 --2- 172.21.15.92:0/4121365757 >> v2:172.21.15.136:3300/0 conn(0x7f56f0159c60 0x7f56f015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f015a3d0 con 0x7f56f015db10 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f56e80017f0 con 0x7f56f015db10 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 >> v1:172.21.15.92:6789/0 conn(0x7f56f015db10 legacy=0x7f56f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 0x7f56dc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc004d50 0x7f56dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00d13a0 con 0x7f56dc003fe0 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015e140 con 0x7f56dc004d50 2023-10-06T03:02:30.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc00ba90 con 0x7f56f0159890 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56ef7fe700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 0x7f56dc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56effff700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc004d50 0x7f56dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 -- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 msgr2=0x7f56dc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f0159890 0x7f56dc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 -- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc004d50 msgr2=0x7f56dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc004d50 0x7f56dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56e7fff700 1 -- 172.21.15.92:0/4121365757 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc003570 con 0x7f56dc003fe0 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.740+0000 7f56effff700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc004d50 0x7f56dc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56e7fff700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56dc004810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56f015a090 tx=0x7f56d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa716bc94f26859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:30.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800e6b0 con 0x7f56dc003fe0 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d800e810 con 0x7f56dc003fe0 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56eeffd700 1 -- 172.21.15.92:0/4121365757 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800ebd0 con 0x7f56dc003fe0 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56f5c88700 1 -- 172.21.15.92:0/4121365757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 msgr2=0x7f56dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56f5c88700 1 --2- 172.21.15.92:0/4121365757 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56dc004810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f56f015a090 tx=0x7f56d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56effff700 1 -- 172.21.15.92:0/4121365757 reap_dead start 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56f5c88700 1 -- 172.21.15.92:0/4121365757 shutdown_connections 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56f5c88700 1 -- 172.21.15.92:0/4121365757 >> 172.21.15.92:0/4121365757 conn(0x7f56f0068e20 msgr2=0x7f56f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56f5c88700 1 -- 172.21.15.92:0/4121365757 shutdown_connections 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.741+0000 7f56f5c88700 1 -- 172.21.15.92:0/4121365757 wait complete. 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.742+0000 7f56f5c88700 1 Processor -- start 2023-10-06T03:02:30.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 -- start start 2023-10-06T03:02:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56f016a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f00b3fa0 0x7f56f00cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f00b4410 0x7f56f00cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cecf0 con 0x7f56dc003fe0 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cde00 con 0x7f56f00b4410 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56f5c88700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00ce0e0 con 0x7f56f00b3fa0 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56f016a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56f016a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34178/0 (socket says 172.21.15.92:34178) 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56ef7fe700 1 -- 172.21.15.92:0/1850194420 learned_addr learned my addr 172.21.15.92:0/1850194420 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56e7fff700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f00b3fa0 0x7f56f00cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.743+0000 7f56ef7fe700 1 -- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f00b3fa0 msgr2=0x7f56f00cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56ef7fe700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f00b3fa0 0x7f56f00cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56ef7fe700 1 -- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f00b4410 msgr2=0x7f56f00cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56ef7fe700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f00b4410 0x7f56f00cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56ef7fe700 1 -- 172.21.15.92:0/1850194420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00ce3c0 con 0x7f56dc003fe0 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56ef7fe700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56f016a310 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f56e0011d40 tx=0x7f56e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000fac0 con 0x7f56dc003fe0 2023-10-06T03:02:30.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000fc20 con 0x7f56dc003fe0 2023-10-06T03:02:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f00ce6a0 con 0x7f56dc003fe0 2023-10-06T03:02:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0014440 con 0x7f56dc003fe0 2023-10-06T03:02:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.744+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f016d590 con 0x7f56dc003fe0 2023-10-06T03:02:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.745+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f00cefd0 con 0x7f56dc003fe0 2023-10-06T03:02:30.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.745+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f56e0014870 con 0x7f56dc003fe0 2023-10-06T03:02:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.745+0000 7f56e77fe700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56d0066f30 0x7f56d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.746+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f56e0018030 con 0x7f56dc003fe0 2023-10-06T03:02:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.746+0000 7f56e7fff700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56d0066f30 0x7f56d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.746+0000 7f56e7fff700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56d0066f30 0x7f56d00693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f56f00b3970 tx=0x7f56d801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:30.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.750+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56e0052280 con 0x7f56dc003fe0 2023-10-06T03:02:30.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:30.973+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f56f004c3c0 con 0x7f56dc003fe0 2023-10-06T03:02:31.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.604+0000 7f56e77fe700 1 -- 172.21.15.92:0/1850194420 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f56e00558a0 con 0x7f56dc003fe0 2023-10-06T03:02:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56d0066f30 msgr2=0x7f56d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56d0066f30 0x7f56d00693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f56f00b3970 tx=0x7f56d801a000 comp rx=0 tx=0).stop 2023-10-06T03:02:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 msgr2=0x7f56f016a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56f016a310 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f56e0011d40 tx=0x7f56e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 shutdown_connections 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56d0066f30 0x7f56d00693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56dc003fe0 0x7f56f016a310 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f00b4410 0x7f56f00cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 --2- 172.21.15.92:0/1850194420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f00b3fa0 0x7f56f00cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 >> 172.21.15.92:0/1850194420 conn(0x7f56f0068e20 msgr2=0x7f56f0158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 shutdown_connections 2023-10-06T03:02:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.606+0000 7f56f5c88700 1 -- 172.21.15.92:0/1850194420 wait complete. 2023-10-06T03:02:31.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.737+0000 7f460834e700 1 Processor -- start 2023-10-06T03:02:31.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.737+0000 7f460834e700 1 -- start start 2023-10-06T03:02:31.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f460834e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4600159c60 0x7f460015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:31.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f460834e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:31.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f460834e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460015a160 con 0x7f4600159890 2023-10-06T03:02:31.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f460834e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460015df50 con 0x7f4600159c60 2023-10-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f460834e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46000d13a0 con 0x7f460015db10 2023-10-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f46058e9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4600159c60 0x7f460015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f46068eb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f46060ea700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f4600159890 0x7f460015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:42348/0 (socket says 172.21.15.92:42348) 2023-10-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.738+0000 7f46060ea700 1 -- 172.21.15.92:0/2943996893 learned_addr learned my addr 172.21.15.92:0/2943996893 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:31.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46058e9700 1 -- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 msgr2=0x7f4600069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:31.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46058e9700 1 --2- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46058e9700 1 -- 172.21.15.92:0/2943996893 >> v1:172.21.15.136:6789/0 conn(0x7f4600159890 legacy=0x7f460015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46058e9700 1 -- 172.21.15.92:0/2943996893 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460015e140 con 0x7f4600159c60 2023-10-06T03:02:31.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46068eb700 1 --2- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:31.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46058e9700 1 --2- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f460015d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45f00098e0 tx=0x7f45f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d832b460710aa57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:31.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46050e8700 1 -- 172.21.15.92:0/2943996893 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000b640 con 0x7f4600159c60 2023-10-06T03:02:31.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46050e8700 1 -- 172.21.15.92:0/2943996893 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f000b7a0 con 0x7f4600159c60 2023-10-06T03:02:31.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.739+0000 7f46050e8700 1 -- 172.21.15.92:0/2943996893 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000bb40 con 0x7f4600159c60 2023-10-06T03:02:31.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 -- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 msgr2=0x7f460015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 --2- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f460015d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45f00098e0 tx=0x7f45f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 -- 172.21.15.92:0/2943996893 shutdown_connections 2023-10-06T03:02:31.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 --2- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 --2- 172.21.15.92:0/2943996893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f460015d5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 -- 172.21.15.92:0/2943996893 >> 172.21.15.92:0/2943996893 conn(0x7f4600068e20 msgr2=0x7f46000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:31.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 -- 172.21.15.92:0/2943996893 shutdown_connections 2023-10-06T03:02:31.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.740+0000 7f460834e700 1 -- 172.21.15.92:0/2943996893 wait complete. 2023-10-06T03:02:31.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.741+0000 7f460834e700 1 Processor -- start 2023-10-06T03:02:31.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.741+0000 7f460834e700 1 -- start start 2023-10-06T03:02:31.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 0x7f4600142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:31.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f4600152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:31.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:31.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600142870 con 0x7f4600159890 2023-10-06T03:02:31.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000b3a10 con 0x7f4600159c60 2023-10-06T03:02:31.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46000cff30 con 0x7f460015db10 2023-10-06T03:02:31.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 0x7f4600142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:31.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 0x7f4600142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34184/0 (socket says 172.21.15.92:34184) 2023-10-06T03:02:31.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 -- 172.21.15.92:0/1212862724 learned_addr learned my addr 172.21.15.92:0/1212862724 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:31.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46068eb700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:31.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46058e9700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f4600152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:31.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 -- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 msgr2=0x7f4600152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 -- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 msgr2=0x7f4600152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f4600152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 -- 172.21.15.92:0/1212862724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8009a20 con 0x7f4600159890 2023-10-06T03:02:31.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f46060ea700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 0x7f4600142330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c7f0 tx=0x7f45fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:31.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc005b30 con 0x7f4600159890 2023-10-06T03:02:31.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc005cb0 con 0x7f4600159890 2023-10-06T03:02:31.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f0009420 con 0x7f4600159890 2023-10-06T03:02:31.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc004a80 con 0x7f4600159890 2023-10-06T03:02:31.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.742+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46000cfc50 con 0x7f4600159890 2023-10-06T03:02:31.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.743+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460004c3c0 con 0x7f4600159890 2023-10-06T03:02:31.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.744+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45fc0070f0 con 0x7f4600159890 2023-10-06T03:02:31.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.744+0000 7f45f67fc700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45e4066f30 0x7f45e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:31.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.744+0000 7f46058e9700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45e4066f30 0x7f45e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:31.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.744+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45fc083b20 con 0x7f4600159890 2023-10-06T03:02:31.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.745+0000 7f46058e9700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45e4066f30 0x7f45e40693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45f00093f0 tx=0x7f45f0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:31.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.748+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45fc050c30 con 0x7f4600159890 2023-10-06T03:02:31.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.971+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46000cbca0 con 0x7f4600159890 2023-10-06T03:02:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.972+0000 7f45f67fc700 1 -- 172.21.15.92:0/1212862724 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f45fc018070 con 0x7f4600159890 2023-10-06T03:02:31.974 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:31.974 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4965,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/971383855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":971383855},{"type":"v1","addr":"172.21.15.92:6836","nonce":971383855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:27.963102+0000","modified":"2023-10-06T03:02:31.599832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2171492021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2171492021},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171492021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:02:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45e4066f30 msgr2=0x7f45e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45e4066f30 0x7f45e40693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45f00093f0 tx=0x7f45f0017000 comp rx=0 tx=0).stop 2023-10-06T03:02:31.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 msgr2=0x7f4600142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 0x7f4600142330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f45fc00c7f0 tx=0x7f45fc007f70 comp rx=0 tx=0).stop 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 shutdown_connections 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f45e4066f30 0x7f45e40693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4600159890 0x7f4600142330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f460015db10 0x7f4600152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 --2- 172.21.15.92:0/1212862724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4600159c60 0x7f4600152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 >> 172.21.15.92:0/1212862724 conn(0x7f4600068e20 msgr2=0x7f4600158510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 shutdown_connections 2023-10-06T03:02:31.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:31.974+0000 7f460834e700 1 -- 172.21.15.92:0/1212862724 wait complete. 2023-10-06T03:02:31.978 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2023-10-06T03:02:31.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.101+0000 7f391b59e700 1 Processor -- start 2023-10-06T03:02:32.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.101+0000 7f391b59e700 1 -- start start 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c159420 0x7f391c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f391c15d230 0x7f391c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391c15dbd0 con 0x7f391c1597f0 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c0b2980 con 0x7f391c15d230 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c0b11d0 con 0x7f391c159420 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f391c15d230 0x7f391c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f3911d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f391c1597f0 0x7f391c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36104/0 (socket says 172.21.15.92:36104) 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f3911d9b700 1 -- 172.21.15.92:0/1577559986 learned_addr learned my addr 172.21.15.92:0/1577559986 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391ad9d700 1 -- 172.21.15.92:0/1577559986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c159420 msgr2=0x7f391c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391a59c700 1 --2- 172.21.15.92:0/1577559986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c159420 0x7f391c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f3919d9b700 1 -- 172.21.15.92:0/1577559986 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130256687 0 0) 0x7f3904001110 con 0x7f391c1597f0 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391ad9d700 1 --2- 172.21.15.92:0/1577559986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c159420 0x7f391c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391ad9d700 1 -- 172.21.15.92:0/1577559986 >> v1:172.21.15.92:6789/0 conn(0x7f391c1597f0 legacy=0x7f391c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391ad9d700 1 -- 172.21.15.92:0/1577559986 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c0b14b0 con 0x7f391c15d230 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.102+0000 7f391ad9d700 1 --2- 172.21.15.92:0/1577559986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c15d230 0x7f391c15d690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3914009e60 tx=0x7f3914009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c9cd9cb08e3564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f3919d9b700 1 -- 172.21.15.92:0/1577559986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391400e070 con 0x7f391c15d230 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f3919d9b700 1 -- 172.21.15.92:0/1577559986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391400bc30 con 0x7f391c15d230 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f3919d9b700 1 -- 172.21.15.92:0/1577559986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914012430 con 0x7f391c15d230 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 -- 172.21.15.92:0/1577559986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c15d230 msgr2=0x7f391c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 --2- 172.21.15.92:0/1577559986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c15d230 0x7f391c15d690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3914009e60 tx=0x7f3914009780 comp rx=0 tx=0).stop 2023-10-06T03:02:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 -- 172.21.15.92:0/1577559986 shutdown_connections 2023-10-06T03:02:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 --2- 172.21.15.92:0/1577559986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c159420 0x7f391c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 --2- 172.21.15.92:0/1577559986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c15d230 0x7f391c15d690 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 -- 172.21.15.92:0/1577559986 >> 172.21.15.92:0/1577559986 conn(0x7f391c0d0660 msgr2=0x7f391c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 -- 172.21.15.92:0/1577559986 shutdown_connections 2023-10-06T03:02:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.103+0000 7f391b59e700 1 -- 172.21.15.92:0/1577559986 wait complete. 2023-10-06T03:02:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.104+0000 7f391b59e700 1 Processor -- start 2023-10-06T03:02:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.104+0000 7f391b59e700 1 -- start start 2023-10-06T03:02:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 0x7f391c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c1597f0 0x7f391c0b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c15d230 0x7f391c0b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c15a910 con 0x7f391c159420 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c159d00 con 0x7f391c1597f0 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c159fe0 con 0x7f391c15d230 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 0x7f391c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 0x7f391c15c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34214/0 (socket says 172.21.15.92:34214) 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 -- 172.21.15.92:0/2034673940 learned_addr learned my addr 172.21.15.92:0/2034673940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 -- 172.21.15.92:0/2034673940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c15d230 msgr2=0x7f391c0b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c15d230 0x7f391c0b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 -- 172.21.15.92:0/2034673940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c1597f0 msgr2=0x7f391c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c1597f0 0x7f391c0b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.105+0000 7f391a59c700 1 -- 172.21.15.92:0/2034673940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914009a20 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.106+0000 7f391a59c700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 0x7f391c15c3a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f390c00c7d0 tx=0x7f390c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.106+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c011600 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.106+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c011760 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.106+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c011b00 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.106+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c15a2c0 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.106+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c15ada0 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.107+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c15abf0 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.107+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f390c0193f0 con 0x7f391c159420 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.108+0000 7f3912ffd700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38fc066f30 0x7f38fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.108+0000 7f3911d9b700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38fc066f30 0x7f38fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.108+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f390c084b40 con 0x7f391c159420 2023-10-06T03:02:32.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.108+0000 7f3911d9b700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38fc066f30 0x7f38fc0693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f391c155cf0 tx=0x7f3904001270 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.112+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f390c051c50 con 0x7f391c159420 2023-10-06T03:02:32.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.295+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f391c04c3c0 con 0x7f391c159420 2023-10-06T03:02:32.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.296+0000 7f3912ffd700 1 -- 172.21.15.92:0/2034673940 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f390c055270 con 0x7f391c159420 2023-10-06T03:02:32.298 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:32.298 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":34,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:29.584322+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38fc066f30 msgr2=0x7f38fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38fc066f30 0x7f38fc0693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f391c155cf0 tx=0x7f3904001270 comp rx=0 tx=0).stop 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 msgr2=0x7f391c15c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 0x7f391c15c3a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f390c00c7d0 tx=0x7f390c008040 comp rx=0 tx=0).stop 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 shutdown_connections 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38fc066f30 0x7f38fc0693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f391c159420 0x7f391c15c3a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f391c15d230 0x7f391c0b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 --2- 172.21.15.92:0/2034673940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f391c1597f0 0x7f391c0b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 >> 172.21.15.92:0/2034673940 conn(0x7f391c0d0660 msgr2=0x7f391c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:32.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 shutdown_connections 2023-10-06T03:02:32.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.298+0000 7f391b59e700 1 -- 172.21.15.92:0/2034673940 wait complete. 2023-10-06T03:02:32.312 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:02:32.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:02:32.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.424+0000 7f438781b700 1 Processor -- start 2023-10-06T03:02:32.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.424+0000 7f438781b700 1 -- start start 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f438781b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4380159890 0x7f438015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f438781b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4380159c60 0x7f438015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f438781b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438015a160 con 0x7f438015db10 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f438781b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438015df50 con 0x7f4380159c60 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f438781b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43800d13a0 con 0x7f4380159890 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4380159c60 0x7f438015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4380159c60 0x7f438015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34216/0 (socket says 172.21.15.92:34216) 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 -- 172.21.15.92:0/2599066477 learned_addr learned my addr 172.21.15.92:0/2599066477 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 -- 172.21.15.92:0/2599066477 >> v2:172.21.15.136:3300/0 conn(0x7f4380159890 msgr2=0x7f438015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:32.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 --2- 172.21.15.92:0/2599066477 >> v2:172.21.15.136:3300/0 conn(0x7f4380159890 0x7f438015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 -- 172.21.15.92:0/2599066477 >> v1:172.21.15.136:6789/0 conn(0x7f438015db10 legacy=0x7f4380069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 -- 172.21.15.92:0/2599066477 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438015e140 con 0x7f4380159c60 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4384db6700 1 --2- 172.21.15.92:0/2599066477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380159c60 0x7f438015d5d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f43800d1090 tx=0x7f4370009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9541b269c38a029 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4377fff700 1 -- 172.21.15.92:0/2599066477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000b820 con 0x7f4380159c60 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4377fff700 1 -- 172.21.15.92:0/2599066477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437000b980 con 0x7f4380159c60 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.425+0000 7f4377fff700 1 -- 172.21.15.92:0/2599066477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437000bd20 con 0x7f4380159c60 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 -- 172.21.15.92:0/2599066477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380159c60 msgr2=0x7f438015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 --2- 172.21.15.92:0/2599066477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380159c60 0x7f438015d5d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f43800d1090 tx=0x7f4370009250 comp rx=0 tx=0).stop 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 -- 172.21.15.92:0/2599066477 shutdown_connections 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 --2- 172.21.15.92:0/2599066477 >> v2:172.21.15.136:3300/0 conn(0x7f4380159890 0x7f438015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 --2- 172.21.15.92:0/2599066477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4380159c60 0x7f438015d5d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 -- 172.21.15.92:0/2599066477 >> 172.21.15.92:0/2599066477 conn(0x7f4380068e20 msgr2=0x7f43800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:32.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 -- 172.21.15.92:0/2599066477 shutdown_connections 2023-10-06T03:02:32.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.426+0000 7f438781b700 1 -- 172.21.15.92:0/2599066477 wait complete. 2023-10-06T03:02:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.427+0000 7f438781b700 1 Processor -- start 2023-10-06T03:02:32.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.427+0000 7f438781b700 1 -- start start 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f438781b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4380159890 0x7f43800b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f438781b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4380159c60 0x7f438014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f438781b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 0x7f4380152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f438781b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43800b3ad0 con 0x7f438015db10 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f438781b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43800b1710 con 0x7f4380159890 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f438781b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4380169d80 con 0x7f4380159c60 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 0x7f4380152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 0x7f4380152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34232/0 (socket says 172.21.15.92:34232) 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 -- 172.21.15.92:0/914430939 learned_addr learned my addr 172.21.15.92:0/914430939 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f43855b7700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4380159890 0x7f43800b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 -- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4380159c60 msgr2=0x7f438014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4380159c60 0x7f438014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 -- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4380159890 msgr2=0x7f43800b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4380159890 0x7f43800b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 -- 172.21.15.92:0/914430939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700094f0 con 0x7f438015db10 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.428+0000 7f4385db8700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 0x7f4380152420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f437800bbb0 tx=0x7f4378009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.429+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43780092e0 con 0x7f438015db10 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.429+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437800c7a0 con 0x7f438015db10 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.429+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438014d190 con 0x7f438015db10 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.429+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437800cb40 con 0x7f438015db10 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.429+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43800b0de0 con 0x7f438015db10 2023-10-06T03:02:32.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.430+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43800b18a0 con 0x7f438015db10 2023-10-06T03:02:32.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.430+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4378012070 con 0x7f438015db10 2023-10-06T03:02:32.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.431+0000 7f43757fa700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4364066f30 0x7f43640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.431+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f43780842b0 con 0x7f438015db10 2023-10-06T03:02:32.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.431+0000 7f43855b7700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4364066f30 0x7f43640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.431+0000 7f43855b7700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4364066f30 0x7f43640693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f437c006fd0 tx=0x7f437c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.435+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43780513c0 con 0x7f438015db10 2023-10-06T03:02:32.560 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T03:02:32.558+0000 7fb7fde46700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:02:32.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.615+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f438004c3c0 con 0x7f4364066f30 2023-10-06T03:02:32.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.623+0000 7f43757fa700 1 -- 172.21.15.92:0/914430939 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289696 (secure 0 0 0) 0x7f438004c3c0 con 0x7f4364066f30 2023-10-06T03:02:32.629 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.626+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4364066f30 msgr2=0x7f43640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.626+0000 7f438781b700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4364066f30 0x7f43640693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f437c006fd0 tx=0x7f437c008040 comp rx=0 tx=0).stop 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 msgr2=0x7f4380152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 0x7f4380152420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f437800bbb0 tx=0x7f4378009840 comp rx=0 tx=0).stop 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 shutdown_connections 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4364066f30 0x7f43640693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f438015db10 0x7f4380152420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4380159890 0x7f43800b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 --2- 172.21.15.92:0/914430939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4380159c60 0x7f438014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 >> 172.21.15.92:0/914430939 conn(0x7f4380068e20 msgr2=0x7f43801584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:32.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 shutdown_connections 2023-10-06T03:02:32.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.627+0000 7f438781b700 1 -- 172.21.15.92:0/914430939 wait complete. 2023-10-06T03:02:32.632 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:02:32.640 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-10-06T03:02:31.192392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382388,"kb_used_data":4084,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592332,"statfs":{"total":773094113280,"available":770654547968,"internally_reserved":0,"allocated":4182016,"data_stored":2359812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.625133"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591767+0000","last_change":"2023-10-06T03:02:28.591767+0000","last_active":"2023-10-06T03:02:28.591767+0000","last_peered":"2023-10-06T03:02:28.591767+0000","last_clean":"2023-10-06T03:02:28.591767+0000","last_became_active":"2023-10-06T03:02:28.591426+0000","last_became_peered":"2023-10-06T03:02:28.591426+0000","last_unstale":"2023-10-06T03:02:28.591767+0000","last_undegraded":"2023-10-06T03:02:28.591767+0000","last_fullsized":"2023-10-06T03:02:28.591767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:12.373681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591435+0000","last_change":"2023-10-06T03:02:28.591435+0000","last_active":"2023-10-06T03:02:28.591435+0000","last_peered":"2023-10-06T03:02:28.591435+0000","last_clean":"2023-10-06T03:02:28.591435+0000","last_became_active":"2023-10-06T03:02:28.590854+0000","last_became_peered":"2023-10-06T03:02:28.590854+0000","last_unstale":"2023-10-06T03:02:28.591435+0000","last_undegraded":"2023-10-06T03:02:28.591435+0000","last_fullsized":"2023-10-06T03:02:28.591435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:09.926644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.593370+0000","last_change":"2023-10-06T03:02:28.593370+0000","last_active":"2023-10-06T03:02:28.593370+0000","last_peered":"2023-10-06T03:02:28.593370+0000","last_clean":"2023-10-06T03:02:28.593370+0000","last_became_active":"2023-10-06T03:02:28.593066+0000","last_became_peered":"2023-10-06T03:02:28.593066+0000","last_unstale":"2023-10-06T03:02:28.593370+0000","last_undegraded":"2023-10-06T03:02:28.593370+0000","last_fullsized":"2023-10-06T03:02:28.593370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:23.624727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601442+0000","last_change":"2023-10-06T03:02:28.601442+0000","last_active":"2023-10-06T03:02:28.601442+0000","last_peered":"2023-10-06T03:02:28.601442+0000","last_clean":"2023-10-06T03:02:28.601442+0000","last_became_active":"2023-10-06T03:02:28.596354+0000","last_became_peered":"2023-10-06T03:02:28.596354+0000","last_unstale":"2023-10-06T03:02:28.601442+0000","last_undegraded":"2023-10-06T03:02:28.601442+0000","last_fullsized":"2023-10-06T03:02:28.601442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:41.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.586706+0000","last_change":"2023-10-06T03:02:28.586706+0000","last_active":"2023-10-06T03:02:28.586706+0000","last_peered":"2023-10-06T03:02:28.586706+0000","last_clean":"2023-10-06T03:02:28.586706+0000","last_became_active":"2023-10-06T03:02:28.586437+0000","last_became_peered":"2023-10-06T03:02:28.586437+0000","last_unstale":"2023-10-06T03:02:28.586706+0000","last_undegraded":"2023-10-06T03:02:28.586706+0000","last_fullsized":"2023-10-06T03:02:28.586706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:25.627957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.590489+0000","last_change":"2023-10-06T03:02:28.590489+0000","last_active":"2023-10-06T03:02:28.590489+0000","last_peered":"2023-10-06T03:02:28.590489+0000","last_clean":"2023-10-06T03:02:28.590489+0000","last_became_active":"2023-10-06T03:02:28.590204+0000","last_became_peered":"2023-10-06T03:02:28.590204+0000","last_unstale":"2023-10-06T03:02:28.590489+0000","last_undegraded":"2023-10-06T03:02:28.590489+0000","last_fullsized":"2023-10-06T03:02:28.590489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:25.619653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.595506+0000","last_change":"2023-10-06T03:02:28.595506+0000","last_active":"2023-10-06T03:02:28.595506+0000","last_peered":"2023-10-06T03:02:28.595506+0000","last_clean":"2023-10-06T03:02:28.595506+0000","last_became_active":"2023-10-06T03:02:28.594548+0000","last_became_peered":"2023-10-06T03:02:28.594548+0000","last_unstale":"2023-10-06T03:02:28.595506+0000","last_undegraded":"2023-10-06T03:02:28.595506+0000","last_fullsized":"2023-10-06T03:02:28.595506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:37.604138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594907+0000","last_change":"2023-10-06T03:02:28.594907+0000","last_active":"2023-10-06T03:02:28.594907+0000","last_peered":"2023-10-06T03:02:28.594907+0000","last_clean":"2023-10-06T03:02:28.594907+0000","last_became_active":"2023-10-06T03:02:28.594656+0000","last_became_peered":"2023-10-06T03:02:28.594656+0000","last_unstale":"2023-10-06T03:02:28.594907+0000","last_undegraded":"2023-10-06T03:02:28.594907+0000","last_fullsized":"2023-10-06T03:02:28.594907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:29.064735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594132+0000","last_change":"2023-10-06T03:02:28.594132+0000","last_active":"2023-10-06T03:02:28.594132+0000","last_peered":"2023-10-06T03:02:28.594132+0000","last_clean":"2023-10-06T03:02:28.594132+0000","last_became_active":"2023-10-06T03:02:28.593565+0000","last_became_peered":"2023-10-06T03:02:28.593565+0000","last_unstale":"2023-10-06T03:02:28.594132+0000","last_undegraded":"2023-10-06T03:02:28.594132+0000","last_fullsized":"2023-10-06T03:02:28.594132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:33.260525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596708+0000","last_change":"2023-10-06T03:02:28.596708+0000","last_active":"2023-10-06T03:02:28.596708+0000","last_peered":"2023-10-06T03:02:28.596708+0000","last_clean":"2023-10-06T03:02:28.596708+0000","last_became_active":"2023-10-06T03:02:28.596409+0000","last_became_peered":"2023-10-06T03:02:28.596409+0000","last_unstale":"2023-10-06T03:02:28.596708+0000","last_undegraded":"2023-10-06T03:02:28.596708+0000","last_fullsized":"2023-10-06T03:02:28.596708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:29.064735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.593879+0000","last_change":"2023-10-06T03:02:28.593879+0000","last_active":"2023-10-06T03:02:28.593879+0000","last_peered":"2023-10-06T03:02:28.593879+0000","last_clean":"2023-10-06T03:02:28.593879+0000","last_became_active":"2023-10-06T03:02:28.593598+0000","last_became_peered":"2023-10-06T03:02:28.593598+0000","last_unstale":"2023-10-06T03:02:28.593879+0000","last_undegraded":"2023-10-06T03:02:28.593879+0000","last_fullsized":"2023-10-06T03:02:28.593879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:46.533418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.588264+0000","last_change":"2023-10-06T03:02:28.588264+0000","last_active":"2023-10-06T03:02:28.588264+0000","last_peered":"2023-10-06T03:02:28.588264+0000","last_clean":"2023-10-06T03:02:28.588264+0000","last_became_active":"2023-10-06T03:02:28.587712+0000","last_became_peered":"2023-10-06T03:02:28.587712+0000","last_unstale":"2023-10-06T03:02:28.588264+0000","last_undegraded":"2023-10-06T03:02:28.588264+0000","last_fullsized":"2023-10-06T03:02:28.588264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:37.604138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596805+0000","last_change":"2023-10-06T03:02:28.596805+0000","last_active":"2023-10-06T03:02:28.596805+0000","last_peered":"2023-10-06T03:02:28.596805+0000","last_clean":"2023-10-06T03:02:28.596805+0000","last_became_active":"2023-10-06T03:02:28.596533+0000","last_became_peered":"2023-10-06T03:02:28.596533+0000","last_unstale":"2023-10-06T03:02:28.596805+0000","last_undegraded":"2023-10-06T03:02:28.596805+0000","last_fullsized":"2023-10-06T03:02:28.596805+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:12.373681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604298+0000","last_change":"2023-10-06T03:02:28.604298+0000","last_active":"2023-10-06T03:02:28.604298+0000","last_peered":"2023-10-06T03:02:28.604298+0000","last_clean":"2023-10-06T03:02:28.604298+0000","last_became_active":"2023-10-06T03:02:28.604027+0000","last_became_peered":"2023-10-06T03:02:28.604027+0000","last_unstale":"2023-10-06T03:02:28.604298+0000","last_undegraded":"2023-10-06T03:02:28.604298+0000","last_fullsized":"2023-10-06T03:02:28.604298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:03.102138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594059+0000","last_change":"2023-10-06T03:02:28.594059+0000","last_active":"2023-10-06T03:02:28.594059+0000","last_peered":"2023-10-06T03:02:28.594059+0000","last_clean":"2023-10-06T03:02:28.594059+0000","last_became_active":"2023-10-06T03:02:28.593791+0000","last_became_peered":"2023-10-06T03:02:28.593791+0000","last_unstale":"2023-10-06T03:02:28.594059+0000","last_undegraded":"2023-10-06T03:02:28.594059+0000","last_fullsized":"2023-10-06T03:02:28.594059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:15.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596582+0000","last_change":"2023-10-06T03:02:28.596582+0000","last_active":"2023-10-06T03:02:28.596582+0000","last_peered":"2023-10-06T03:02:28.596582+0000","last_clean":"2023-10-06T03:02:28.596582+0000","last_became_active":"2023-10-06T03:02:28.596314+0000","last_became_peered":"2023-10-06T03:02:28.596314+0000","last_unstale":"2023-10-06T03:02:28.596582+0000","last_undegraded":"2023-10-06T03:02:28.596582+0000","last_fullsized":"2023-10-06T03:02:28.596582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:14.295301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601095+0000","last_change":"2023-10-06T03:02:28.601095+0000","last_active":"2023-10-06T03:02:28.601095+0000","last_peered":"2023-10-06T03:02:28.601095+0000","last_clean":"2023-10-06T03:02:28.601095+0000","last_became_active":"2023-10-06T03:02:28.600831+0000","last_became_peered":"2023-10-06T03:02:28.600831+0000","last_unstale":"2023-10-06T03:02:28.601095+0000","last_undegraded":"2023-10-06T03:02:28.601095+0000","last_fullsized":"2023-10-06T03:02:28.601095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:46.533418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601776+0000","last_change":"2023-10-06T03:02:28.601776+0000","last_active":"2023-10-06T03:02:28.601776+0000","last_peered":"2023-10-06T03:02:28.601776+0000","last_clean":"2023-10-06T03:02:28.601776+0000","last_became_active":"2023-10-06T03:02:28.601482+0000","last_became_peered":"2023-10-06T03:02:28.601482+0000","last_unstale":"2023-10-06T03:02:28.601776+0000","last_undegraded":"2023-10-06T03:02:28.601776+0000","last_fullsized":"2023-10-06T03:02:28.601776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:46.265185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.588842+0000","last_change":"2023-10-06T03:02:28.588842+0000","last_active":"2023-10-06T03:02:28.588842+0000","last_peered":"2023-10-06T03:02:28.588842+0000","last_clean":"2023-10-06T03:02:28.588842+0000","last_became_active":"2023-10-06T03:02:28.588581+0000","last_became_peered":"2023-10-06T03:02:28.588581+0000","last_unstale":"2023-10-06T03:02:28.588842+0000","last_undegraded":"2023-10-06T03:02:28.588842+0000","last_fullsized":"2023-10-06T03:02:28.588842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:54.674740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602198+0000","last_change":"2023-10-06T03:02:28.602198+0000","last_active":"2023-10-06T03:02:28.602198+0000","last_peered":"2023-10-06T03:02:28.602198+0000","last_clean":"2023-10-06T03:02:28.602198+0000","last_became_active":"2023-10-06T03:02:28.601919+0000","last_became_peered":"2023-10-06T03:02:28.601919+0000","last_unstale":"2023-10-06T03:02:28.602198+0000","last_undegraded":"2023-10-06T03:02:28.602198+0000","last_fullsized":"2023-10-06T03:02:28.602198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:25.627957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.592211+0000","last_change":"2023-10-06T03:02:28.592211+0000","last_active":"2023-10-06T03:02:28.592211+0000","last_peered":"2023-10-06T03:02:28.592211+0000","last_clean":"2023-10-06T03:02:28.592211+0000","last_became_active":"2023-10-06T03:02:28.591936+0000","last_became_peered":"2023-10-06T03:02:28.591936+0000","last_unstale":"2023-10-06T03:02:28.592211+0000","last_undegraded":"2023-10-06T03:02:28.592211+0000","last_fullsized":"2023-10-06T03:02:28.592211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:15.928007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587905+0000","last_change":"2023-10-06T03:02:28.587905+0000","last_active":"2023-10-06T03:02:28.587905+0000","last_peered":"2023-10-06T03:02:28.587905+0000","last_clean":"2023-10-06T03:02:28.587905+0000","last_became_active":"2023-10-06T03:02:28.587624+0000","last_became_peered":"2023-10-06T03:02:28.587624+0000","last_unstale":"2023-10-06T03:02:28.587905+0000","last_undegraded":"2023-10-06T03:02:28.587905+0000","last_fullsized":"2023-10-06T03:02:28.587905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:23.557486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.605750+0000","last_change":"2023-10-06T03:02:28.605750+0000","last_active":"2023-10-06T03:02:28.605750+0000","last_peered":"2023-10-06T03:02:28.605750+0000","last_clean":"2023-10-06T03:02:28.605750+0000","last_became_active":"2023-10-06T03:02:28.605494+0000","last_became_peered":"2023-10-06T03:02:28.605494+0000","last_unstale":"2023-10-06T03:02:28.605750+0000","last_undegraded":"2023-10-06T03:02:28.605750+0000","last_fullsized":"2023-10-06T03:02:28.605750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:15.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601376+0000","last_change":"2023-10-06T03:02:28.601376+0000","last_active":"2023-10-06T03:02:28.601376+0000","last_peered":"2023-10-06T03:02:28.601376+0000","last_clean":"2023-10-06T03:02:28.601376+0000","last_became_active":"2023-10-06T03:02:28.601089+0000","last_became_peered":"2023-10-06T03:02:28.601089+0000","last_unstale":"2023-10-06T03:02:28.601376+0000","last_undegraded":"2023-10-06T03:02:28.601376+0000","last_fullsized":"2023-10-06T03:02:28.601376+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:00.837178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604987+0000","last_change":"2023-10-06T03:02:28.604987+0000","last_active":"2023-10-06T03:02:28.604987+0000","last_peered":"2023-10-06T03:02:28.604987+0000","last_clean":"2023-10-06T03:02:28.604987+0000","last_became_active":"2023-10-06T03:02:28.601264+0000","last_became_peered":"2023-10-06T03:02:28.601264+0000","last_unstale":"2023-10-06T03:02:28.604987+0000","last_undegraded":"2023-10-06T03:02:28.604987+0000","last_fullsized":"2023-10-06T03:02:28.604987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:41.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.595998+0000","last_change":"2023-10-06T03:02:28.595998+0000","last_active":"2023-10-06T03:02:28.595998+0000","last_peered":"2023-10-06T03:02:28.595998+0000","last_clean":"2023-10-06T03:02:28.595998+0000","last_became_active":"2023-10-06T03:02:28.595734+0000","last_became_peered":"2023-10-06T03:02:28.595734+0000","last_unstale":"2023-10-06T03:02:28.595998+0000","last_undegraded":"2023-10-06T03:02:28.595998+0000","last_fullsized":"2023-10-06T03:02:28.595998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.130409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604991+0000","last_change":"2023-10-06T03:02:28.604991+0000","last_active":"2023-10-06T03:02:28.604991+0000","last_peered":"2023-10-06T03:02:28.604991+0000","last_clean":"2023-10-06T03:02:28.604991+0000","last_became_active":"2023-10-06T03:02:28.604732+0000","last_became_peered":"2023-10-06T03:02:28.604732+0000","last_unstale":"2023-10-06T03:02:28.604991+0000","last_undegraded":"2023-10-06T03:02:28.604991+0000","last_fullsized":"2023-10-06T03:02:28.604991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:02.045058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594870+0000","last_change":"2023-10-06T03:02:28.594870+0000","last_active":"2023-10-06T03:02:28.594870+0000","last_peered":"2023-10-06T03:02:28.594870+0000","last_clean":"2023-10-06T03:02:28.594870+0000","last_became_active":"2023-10-06T03:02:28.594610+0000","last_became_peered":"2023-10-06T03:02:28.594610+0000","last_unstale":"2023-10-06T03:02:28.594870+0000","last_undegraded":"2023-10-06T03:02:28.594870+0000","last_fullsized":"2023-10-06T03:02:28.594870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:44.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602829+0000","last_change":"2023-10-06T03:02:28.602829+0000","last_active":"2023-10-06T03:02:28.602829+0000","last_peered":"2023-10-06T03:02:28.602829+0000","last_clean":"2023-10-06T03:02:28.602829+0000","last_became_active":"2023-10-06T03:02:28.602535+0000","last_became_peered":"2023-10-06T03:02:28.602535+0000","last_unstale":"2023-10-06T03:02:28.602829+0000","last_undegraded":"2023-10-06T03:02:28.602829+0000","last_fullsized":"2023-10-06T03:02:28.602829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:51.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604509+0000","last_change":"2023-10-06T03:02:28.604509+0000","last_active":"2023-10-06T03:02:28.604509+0000","last_peered":"2023-10-06T03:02:28.604509+0000","last_clean":"2023-10-06T03:02:28.604509+0000","last_became_active":"2023-10-06T03:02:28.604203+0000","last_became_peered":"2023-10-06T03:02:28.604203+0000","last_unstale":"2023-10-06T03:02:28.604509+0000","last_undegraded":"2023-10-06T03:02:28.604509+0000","last_fullsized":"2023-10-06T03:02:28.604509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:00.837178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.606031+0000","last_change":"2023-10-06T03:02:28.606031+0000","last_active":"2023-10-06T03:02:28.606031+0000","last_peered":"2023-10-06T03:02:28.606031+0000","last_clean":"2023-10-06T03:02:28.606031+0000","last_became_active":"2023-10-06T03:02:28.605162+0000","last_became_peered":"2023-10-06T03:02:28.605162+0000","last_unstale":"2023-10-06T03:02:28.606031+0000","last_undegraded":"2023-10-06T03:02:28.606031+0000","last_fullsized":"2023-10-06T03:02:28.606031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:37.604138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.590034+0000","last_change":"2023-10-06T03:02:28.590034+0000","last_active":"2023-10-06T03:02:28.590034+0000","last_peered":"2023-10-06T03:02:28.590034+0000","last_clean":"2023-10-06T03:02:28.590034+0000","last_became_active":"2023-10-06T03:02:28.589817+0000","last_became_peered":"2023-10-06T03:02:28.589817+0000","last_unstale":"2023-10-06T03:02:28.590034+0000","last_undegraded":"2023-10-06T03:02:28.590034+0000","last_fullsized":"2023-10-06T03:02:28.590034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:34.626012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594017+0000","last_change":"2023-10-06T03:02:28.594017+0000","last_active":"2023-10-06T03:02:28.594017+0000","last_peered":"2023-10-06T03:02:28.594017+0000","last_clean":"2023-10-06T03:02:28.594017+0000","last_became_active":"2023-10-06T03:02:28.593737+0000","last_became_peered":"2023-10-06T03:02:28.593737+0000","last_unstale":"2023-10-06T03:02:28.594017+0000","last_undegraded":"2023-10-06T03:02:28.594017+0000","last_fullsized":"2023-10-06T03:02:28.594017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:33.086786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.608404+0000","last_change":"2023-10-06T03:02:27.590039+0000","last_active":"2023-10-06T03:02:28.608404+0000","last_peered":"2023-10-06T03:02:28.608404+0000","last_clean":"2023-10-06T03:02:28.608404+0000","last_became_active":"2023-10-06T03:02:27.589818+0000","last_became_peered":"2023-10-06T03:02:27.589818+0000","last_unstale":"2023-10-06T03:02:28.608404+0000","last_undegraded":"2023-10-06T03:02:28.608404+0000","last_fullsized":"2023-10-06T03:02:28.608404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:40.995528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597759+0000","last_change":"2023-10-06T03:02:28.597759+0000","last_active":"2023-10-06T03:02:28.597759+0000","last_peered":"2023-10-06T03:02:28.597759+0000","last_clean":"2023-10-06T03:02:28.597759+0000","last_became_active":"2023-10-06T03:02:28.597363+0000","last_became_peered":"2023-10-06T03:02:28.597363+0000","last_unstale":"2023-10-06T03:02:28.597759+0000","last_undegraded":"2023-10-06T03:02:28.597759+0000","last_fullsized":"2023-10-06T03:02:28.597759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:23.171200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.578158+0000","last_change":"2023-10-06T03:02:27.591875+0000","last_active":"2023-10-06T03:02:28.578158+0000","last_peered":"2023-10-06T03:02:28.578158+0000","last_clean":"2023-10-06T03:02:28.578158+0000","last_became_active":"2023-10-06T03:02:27.591625+0000","last_became_peered":"2023-10-06T03:02:27.591625+0000","last_unstale":"2023-10-06T03:02:28.578158+0000","last_undegraded":"2023-10-06T03:02:28.578158+0000","last_fullsized":"2023-10-06T03:02:28.578158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:40.497752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602070+0000","last_change":"2023-10-06T03:02:28.602070+0000","last_active":"2023-10-06T03:02:28.602070+0000","last_peered":"2023-10-06T03:02:28.602070+0000","last_clean":"2023-10-06T03:02:28.602070+0000","last_became_active":"2023-10-06T03:02:28.601590+0000","last_became_peered":"2023-10-06T03:02:28.601590+0000","last_unstale":"2023-10-06T03:02:28.602070+0000","last_undegraded":"2023-10-06T03:02:28.602070+0000","last_fullsized":"2023-10-06T03:02:28.602070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:45.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604382+0000","last_change":"2023-10-06T03:02:27.592012+0000","last_active":"2023-10-06T03:02:28.604382+0000","last_peered":"2023-10-06T03:02:28.604382+0000","last_clean":"2023-10-06T03:02:28.604382+0000","last_became_active":"2023-10-06T03:02:27.591313+0000","last_became_peered":"2023-10-06T03:02:27.591313+0000","last_unstale":"2023-10-06T03:02:28.604382+0000","last_undegraded":"2023-10-06T03:02:28.604382+0000","last_fullsized":"2023-10-06T03:02:28.604382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:59.832424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.603963+0000","last_change":"2023-10-06T03:02:28.603963+0000","last_active":"2023-10-06T03:02:28.603963+0000","last_peered":"2023-10-06T03:02:28.603963+0000","last_clean":"2023-10-06T03:02:28.603963+0000","last_became_active":"2023-10-06T03:02:28.603708+0000","last_became_peered":"2023-10-06T03:02:28.603708+0000","last_unstale":"2023-10-06T03:02:28.603963+0000","last_undegraded":"2023-10-06T03:02:28.603963+0000","last_fullsized":"2023-10-06T03:02:28.603963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.624464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.610894+0000","last_change":"2023-10-06T03:02:27.583485+0000","last_active":"2023-10-06T03:02:28.610894+0000","last_peered":"2023-10-06T03:02:28.610894+0000","last_clean":"2023-10-06T03:02:28.610894+0000","last_became_active":"2023-10-06T03:02:27.580481+0000","last_became_peered":"2023-10-06T03:02:27.580481+0000","last_unstale":"2023-10-06T03:02:28.610894+0000","last_undegraded":"2023-10-06T03:02:28.610894+0000","last_fullsized":"2023-10-06T03:02:28.610894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:32.082032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.590185+0000","last_change":"2023-10-06T03:02:28.590185+0000","last_active":"2023-10-06T03:02:28.590185+0000","last_peered":"2023-10-06T03:02:28.590185+0000","last_clean":"2023-10-06T03:02:28.590185+0000","last_became_active":"2023-10-06T03:02:28.589928+0000","last_became_peered":"2023-10-06T03:02:28.589928+0000","last_unstale":"2023-10-06T03:02:28.590185+0000","last_undegraded":"2023-10-06T03:02:28.590185+0000","last_fullsized":"2023-10-06T03:02:28.590185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:15.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587157+0000","last_change":"2023-10-06T03:02:27.588523+0000","last_active":"2023-10-06T03:02:28.587157+0000","last_peered":"2023-10-06T03:02:28.587157+0000","last_clean":"2023-10-06T03:02:28.587157+0000","last_became_active":"2023-10-06T03:02:27.588249+0000","last_became_peered":"2023-10-06T03:02:27.588249+0000","last_unstale":"2023-10-06T03:02:28.587157+0000","last_undegraded":"2023-10-06T03:02:28.587157+0000","last_fullsized":"2023-10-06T03:02:28.587157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:01.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591431+0000","last_change":"2023-10-06T03:02:28.591431+0000","last_active":"2023-10-06T03:02:28.591431+0000","last_peered":"2023-10-06T03:02:28.591431+0000","last_clean":"2023-10-06T03:02:28.591431+0000","last_became_active":"2023-10-06T03:02:28.591090+0000","last_became_peered":"2023-10-06T03:02:28.591090+0000","last_unstale":"2023-10-06T03:02:28.591431+0000","last_undegraded":"2023-10-06T03:02:28.591431+0000","last_fullsized":"2023-10-06T03:02:28.591431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:11.561752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.585742+0000","last_change":"2023-10-06T03:02:27.591685+0000","last_active":"2023-10-06T03:02:28.585742+0000","last_peered":"2023-10-06T03:02:28.585742+0000","last_clean":"2023-10-06T03:02:28.585742+0000","last_became_active":"2023-10-06T03:02:27.591429+0000","last_became_peered":"2023-10-06T03:02:27.591429+0000","last_unstale":"2023-10-06T03:02:28.585742+0000","last_undegraded":"2023-10-06T03:02:28.585742+0000","last_fullsized":"2023-10-06T03:02:28.585742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:09.871577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597043+0000","last_change":"2023-10-06T03:02:28.597043+0000","last_active":"2023-10-06T03:02:28.597043+0000","last_peered":"2023-10-06T03:02:28.597043+0000","last_clean":"2023-10-06T03:02:28.597043+0000","last_became_active":"2023-10-06T03:02:28.593037+0000","last_became_peered":"2023-10-06T03:02:28.593037+0000","last_unstale":"2023-10-06T03:02:28.597043+0000","last_undegraded":"2023-10-06T03:02:28.597043+0000","last_fullsized":"2023-10-06T03:02:28.597043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:09.622324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.596479+0000","last_change":"2023-10-06T03:02:27.592599+0000","last_active":"2023-10-06T03:02:27.596479+0000","last_peered":"2023-10-06T03:02:27.596479+0000","last_clean":"2023-10-06T03:02:27.596479+0000","last_became_active":"2023-10-06T03:02:27.592342+0000","last_became_peered":"2023-10-06T03:02:27.592342+0000","last_unstale":"2023-10-06T03:02:27.596479+0000","last_undegraded":"2023-10-06T03:02:27.596479+0000","last_fullsized":"2023-10-06T03:02:27.596479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:45.260431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596951+0000","last_change":"2023-10-06T03:02:28.596951+0000","last_active":"2023-10-06T03:02:28.596951+0000","last_peered":"2023-10-06T03:02:28.596951+0000","last_clean":"2023-10-06T03:02:28.596951+0000","last_became_active":"2023-10-06T03:02:28.596689+0000","last_became_peered":"2023-10-06T03:02:28.596689+0000","last_unstale":"2023-10-06T03:02:28.596951+0000","last_undegraded":"2023-10-06T03:02:28.596951+0000","last_fullsized":"2023-10-06T03:02:28.596951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:47.335522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.594099+0000","last_change":"2023-10-06T03:02:27.591313+0000","last_active":"2023-10-06T03:02:27.594099+0000","last_peered":"2023-10-06T03:02:27.594099+0000","last_clean":"2023-10-06T03:02:27.594099+0000","last_became_active":"2023-10-06T03:02:27.591066+0000","last_became_peered":"2023-10-06T03:02:27.591066+0000","last_unstale":"2023-10-06T03:02:27.594099+0000","last_undegraded":"2023-10-06T03:02:27.594099+0000","last_fullsized":"2023-10-06T03:02:27.594099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:36.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601572+0000","last_change":"2023-10-06T03:02:28.601572+0000","last_active":"2023-10-06T03:02:28.601572+0000","last_peered":"2023-10-06T03:02:28.601572+0000","last_clean":"2023-10-06T03:02:28.601572+0000","last_became_active":"2023-10-06T03:02:28.596311+0000","last_became_peered":"2023-10-06T03:02:28.596311+0000","last_unstale":"2023-10-06T03:02:28.601572+0000","last_undegraded":"2023-10-06T03:02:28.601572+0000","last_fullsized":"2023-10-06T03:02:28.601572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:02.045058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595192+0000","last_change":"2023-10-06T03:02:27.592032+0000","last_active":"2023-10-06T03:02:27.595192+0000","last_peered":"2023-10-06T03:02:27.595192+0000","last_clean":"2023-10-06T03:02:27.595192+0000","last_became_active":"2023-10-06T03:02:27.591715+0000","last_became_peered":"2023-10-06T03:02:27.591715+0000","last_unstale":"2023-10-06T03:02:27.595192+0000","last_undegraded":"2023-10-06T03:02:27.595192+0000","last_fullsized":"2023-10-06T03:02:27.595192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:24.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602262+0000","last_change":"2023-10-06T03:02:28.602262+0000","last_active":"2023-10-06T03:02:28.602262+0000","last_peered":"2023-10-06T03:02:28.602262+0000","last_clean":"2023-10-06T03:02:28.602262+0000","last_became_active":"2023-10-06T03:02:28.601932+0000","last_became_peered":"2023-10-06T03:02:28.601932+0000","last_unstale":"2023-10-06T03:02:28.602262+0000","last_undegraded":"2023-10-06T03:02:28.602262+0000","last_fullsized":"2023-10-06T03:02:28.602262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:23.624727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.593610+0000","last_change":"2023-10-06T03:02:27.591529+0000","last_active":"2023-10-06T03:02:27.593610+0000","last_peered":"2023-10-06T03:02:27.593610+0000","last_clean":"2023-10-06T03:02:27.593610+0000","last_became_active":"2023-10-06T03:02:27.591256+0000","last_became_peered":"2023-10-06T03:02:27.591256+0000","last_unstale":"2023-10-06T03:02:27.593610+0000","last_undegraded":"2023-10-06T03:02:27.593610+0000","last_fullsized":"2023-10-06T03:02:27.593610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:40.497752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596626+0000","last_change":"2023-10-06T03:02:28.596626+0000","last_active":"2023-10-06T03:02:28.596626+0000","last_peered":"2023-10-06T03:02:28.596626+0000","last_clean":"2023-10-06T03:02:28.596626+0000","last_became_active":"2023-10-06T03:02:28.596310+0000","last_became_peered":"2023-10-06T03:02:28.596310+0000","last_unstale":"2023-10-06T03:02:28.596626+0000","last_undegraded":"2023-10-06T03:02:28.596626+0000","last_fullsized":"2023-10-06T03:02:28.596626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:23.171200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.607136+0000","last_change":"2023-10-06T03:02:27.593977+0000","last_active":"2023-10-06T03:02:28.607136+0000","last_peered":"2023-10-06T03:02:28.607136+0000","last_clean":"2023-10-06T03:02:28.607136+0000","last_became_active":"2023-10-06T03:02:27.593704+0000","last_became_peered":"2023-10-06T03:02:27.593704+0000","last_unstale":"2023-10-06T03:02:28.607136+0000","last_undegraded":"2023-10-06T03:02:28.607136+0000","last_fullsized":"2023-10-06T03:02:28.607136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:09.138335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.589064+0000","last_change":"2023-10-06T03:02:28.589064+0000","last_active":"2023-10-06T03:02:28.589064+0000","last_peered":"2023-10-06T03:02:28.589064+0000","last_clean":"2023-10-06T03:02:28.589064+0000","last_became_active":"2023-10-06T03:02:28.588732+0000","last_became_peered":"2023-10-06T03:02:28.588732+0000","last_unstale":"2023-10-06T03:02:28.589064+0000","last_undegraded":"2023-10-06T03:02:28.589064+0000","last_fullsized":"2023-10-06T03:02:28.589064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:12.750850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.608245+0000","last_change":"2023-10-06T03:02:27.589760+0000","last_active":"2023-10-06T03:02:28.608245+0000","last_peered":"2023-10-06T03:02:28.608245+0000","last_clean":"2023-10-06T03:02:28.608245+0000","last_became_active":"2023-10-06T03:02:27.589398+0000","last_became_peered":"2023-10-06T03:02:27.589398+0000","last_unstale":"2023-10-06T03:02:28.608245+0000","last_undegraded":"2023-10-06T03:02:28.608245+0000","last_fullsized":"2023-10-06T03:02:28.608245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:36.507008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591761+0000","last_change":"2023-10-06T03:02:28.591761+0000","last_active":"2023-10-06T03:02:28.591761+0000","last_peered":"2023-10-06T03:02:28.591761+0000","last_clean":"2023-10-06T03:02:28.591761+0000","last_became_active":"2023-10-06T03:02:28.591485+0000","last_became_peered":"2023-10-06T03:02:28.591485+0000","last_unstale":"2023-10-06T03:02:28.591761+0000","last_undegraded":"2023-10-06T03:02:28.591761+0000","last_fullsized":"2023-10-06T03:02:28.591761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:14.295301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604146+0000","last_change":"2023-10-06T03:02:27.587620+0000","last_active":"2023-10-06T03:02:28.604146+0000","last_peered":"2023-10-06T03:02:28.604146+0000","last_clean":"2023-10-06T03:02:28.604146+0000","last_became_active":"2023-10-06T03:02:27.587387+0000","last_became_peered":"2023-10-06T03:02:27.587387+0000","last_unstale":"2023-10-06T03:02:28.604146+0000","last_undegraded":"2023-10-06T03:02:28.604146+0000","last_fullsized":"2023-10-06T03:02:28.604146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:00.684065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587279+0000","last_change":"2023-10-06T03:02:28.587279+0000","last_active":"2023-10-06T03:02:28.587279+0000","last_peered":"2023-10-06T03:02:28.587279+0000","last_clean":"2023-10-06T03:02:28.587279+0000","last_became_active":"2023-10-06T03:02:28.587042+0000","last_became_peered":"2023-10-06T03:02:28.587042+0000","last_unstale":"2023-10-06T03:02:28.587279+0000","last_undegraded":"2023-10-06T03:02:28.587279+0000","last_fullsized":"2023-10-06T03:02:28.587279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:11.561752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.596536+0000","last_change":"2023-10-06T03:02:27.593855+0000","last_active":"2023-10-06T03:02:27.596536+0000","last_peered":"2023-10-06T03:02:27.596536+0000","last_clean":"2023-10-06T03:02:27.596536+0000","last_became_active":"2023-10-06T03:02:27.593595+0000","last_became_peered":"2023-10-06T03:02:27.593595+0000","last_unstale":"2023-10-06T03:02:27.596536+0000","last_undegraded":"2023-10-06T03:02:27.596536+0000","last_fullsized":"2023-10-06T03:02:27.596536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:01.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594846+0000","last_change":"2023-10-06T03:02:28.594846+0000","last_active":"2023-10-06T03:02:28.594846+0000","last_peered":"2023-10-06T03:02:28.594846+0000","last_clean":"2023-10-06T03:02:28.594846+0000","last_became_active":"2023-10-06T03:02:28.594605+0000","last_became_peered":"2023-10-06T03:02:28.594605+0000","last_unstale":"2023-10-06T03:02:28.594846+0000","last_undegraded":"2023-10-06T03:02:28.594846+0000","last_fullsized":"2023-10-06T03:02:28.594846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.130409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597311+0000","last_change":"2023-10-06T03:02:27.590192+0000","last_active":"2023-10-06T03:02:27.597311+0000","last_peered":"2023-10-06T03:02:27.597311+0000","last_clean":"2023-10-06T03:02:27.597311+0000","last_became_active":"2023-10-06T03:02:27.589914+0000","last_became_peered":"2023-10-06T03:02:27.589914+0000","last_unstale":"2023-10-06T03:02:27.597311+0000","last_undegraded":"2023-10-06T03:02:27.597311+0000","last_fullsized":"2023-10-06T03:02:27.597311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:25.006902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.592307+0000","last_change":"2023-10-06T03:02:28.592307+0000","last_active":"2023-10-06T03:02:28.592307+0000","last_peered":"2023-10-06T03:02:28.592307+0000","last_clean":"2023-10-06T03:02:28.592307+0000","last_became_active":"2023-10-06T03:02:28.592092+0000","last_became_peered":"2023-10-06T03:02:28.592092+0000","last_unstale":"2023-10-06T03:02:28.592307+0000","last_undegraded":"2023-10-06T03:02:28.592307+0000","last_fullsized":"2023-10-06T03:02:28.592307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:45.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.613240+0000","last_change":"2023-10-06T03:02:27.591740+0000","last_active":"2023-10-06T03:02:28.613240+0000","last_peered":"2023-10-06T03:02:28.613240+0000","last_clean":"2023-10-06T03:02:28.613240+0000","last_became_active":"2023-10-06T03:02:27.591457+0000","last_became_peered":"2023-10-06T03:02:27.591457+0000","last_unstale":"2023-10-06T03:02:28.613240+0000","last_undegraded":"2023-10-06T03:02:28.613240+0000","last_fullsized":"2023-10-06T03:02:28.613240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:50.788840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602597+0000","last_change":"2023-10-06T03:02:28.602597+0000","last_active":"2023-10-06T03:02:28.602597+0000","last_peered":"2023-10-06T03:02:28.602597+0000","last_clean":"2023-10-06T03:02:28.602597+0000","last_became_active":"2023-10-06T03:02:28.602322+0000","last_became_peered":"2023-10-06T03:02:28.602322+0000","last_unstale":"2023-10-06T03:02:28.602597+0000","last_undegraded":"2023-10-06T03:02:28.602597+0000","last_fullsized":"2023-10-06T03:02:28.602597+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:09.622324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587301+0000","last_change":"2023-10-06T03:02:27.589171+0000","last_active":"2023-10-06T03:02:28.587301+0000","last_peered":"2023-10-06T03:02:28.587301+0000","last_clean":"2023-10-06T03:02:28.587301+0000","last_became_active":"2023-10-06T03:02:27.588877+0000","last_became_peered":"2023-10-06T03:02:27.588877+0000","last_unstale":"2023-10-06T03:02:28.587301+0000","last_undegraded":"2023-10-06T03:02:28.587301+0000","last_fullsized":"2023-10-06T03:02:28.587301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:59.832424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591518+0000","last_change":"2023-10-06T03:02:28.591518+0000","last_active":"2023-10-06T03:02:28.591518+0000","last_peered":"2023-10-06T03:02:28.591518+0000","last_clean":"2023-10-06T03:02:28.591518+0000","last_became_active":"2023-10-06T03:02:28.591138+0000","last_became_peered":"2023-10-06T03:02:28.591138+0000","last_unstale":"2023-10-06T03:02:28.591518+0000","last_undegraded":"2023-10-06T03:02:28.591518+0000","last_fullsized":"2023-10-06T03:02:28.591518+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:09.926644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.610797+0000","last_change":"2023-10-06T03:02:27.588184+0000","last_active":"2023-10-06T03:02:28.610797+0000","last_peered":"2023-10-06T03:02:28.610797+0000","last_clean":"2023-10-06T03:02:28.610797+0000","last_became_active":"2023-10-06T03:02:27.587911+0000","last_became_peered":"2023-10-06T03:02:27.587911+0000","last_unstale":"2023-10-06T03:02:28.610797+0000","last_undegraded":"2023-10-06T03:02:28.610797+0000","last_fullsized":"2023-10-06T03:02:28.610797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:22.619973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587681+0000","last_change":"2023-10-06T03:02:28.587681+0000","last_active":"2023-10-06T03:02:28.587681+0000","last_peered":"2023-10-06T03:02:28.587681+0000","last_clean":"2023-10-06T03:02:28.587681+0000","last_became_active":"2023-10-06T03:02:28.587463+0000","last_became_peered":"2023-10-06T03:02:28.587463+0000","last_unstale":"2023-10-06T03:02:28.587681+0000","last_undegraded":"2023-10-06T03:02:28.587681+0000","last_fullsized":"2023-10-06T03:02:28.587681+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:09:45.677843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.582259+0000","last_change":"2023-10-06T03:02:27.588070+0000","last_active":"2023-10-06T03:02:28.582259+0000","last_peered":"2023-10-06T03:02:28.582259+0000","last_clean":"2023-10-06T03:02:28.582259+0000","last_became_active":"2023-10-06T03:02:27.587798+0000","last_became_peered":"2023-10-06T03:02:27.587798+0000","last_unstale":"2023-10-06T03:02:28.582259+0000","last_undegraded":"2023-10-06T03:02:28.582259+0000","last_fullsized":"2023-10-06T03:02:28.582259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:40.995528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594828+0000","last_change":"2023-10-06T03:02:28.594828+0000","last_active":"2023-10-06T03:02:28.594828+0000","last_peered":"2023-10-06T03:02:28.594828+0000","last_clean":"2023-10-06T03:02:28.594828+0000","last_became_active":"2023-10-06T03:02:28.594564+0000","last_became_peered":"2023-10-06T03:02:28.594564+0000","last_unstale":"2023-10-06T03:02:28.594828+0000","last_undegraded":"2023-10-06T03:02:28.594828+0000","last_fullsized":"2023-10-06T03:02:28.594828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:46.265185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.594415+0000","last_change":"2023-10-06T03:02:27.591268+0000","last_active":"2023-10-06T03:02:27.594415+0000","last_peered":"2023-10-06T03:02:27.594415+0000","last_clean":"2023-10-06T03:02:27.594415+0000","last_became_active":"2023-10-06T03:02:27.590327+0000","last_became_peered":"2023-10-06T03:02:27.590327+0000","last_unstale":"2023-10-06T03:02:27.594415+0000","last_undegraded":"2023-10-06T03:02:27.594415+0000","last_fullsized":"2023-10-06T03:02:27.594415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:40.497752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.606514+0000","last_change":"2023-10-06T03:02:28.606514+0000","last_active":"2023-10-06T03:02:28.606514+0000","last_peered":"2023-10-06T03:02:28.606514+0000","last_clean":"2023-10-06T03:02:28.606514+0000","last_became_active":"2023-10-06T03:02:28.606223+0000","last_became_peered":"2023-10-06T03:02:28.606223+0000","last_unstale":"2023-10-06T03:02:28.606514+0000","last_undegraded":"2023-10-06T03:02:28.606514+0000","last_fullsized":"2023-10-06T03:02:28.606514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:33.260525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.609898+0000","last_change":"2023-10-06T03:02:27.592808+0000","last_active":"2023-10-06T03:02:28.609898+0000","last_peered":"2023-10-06T03:02:28.609898+0000","last_clean":"2023-10-06T03:02:28.609898+0000","last_became_active":"2023-10-06T03:02:27.592554+0000","last_became_peered":"2023-10-06T03:02:27.592554+0000","last_unstale":"2023-10-06T03:02:28.609898+0000","last_undegraded":"2023-10-06T03:02:28.609898+0000","last_fullsized":"2023-10-06T03:02:28.609898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.926792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597803+0000","last_change":"2023-10-06T03:02:28.597803+0000","last_active":"2023-10-06T03:02:28.597803+0000","last_peered":"2023-10-06T03:02:28.597803+0000","last_clean":"2023-10-06T03:02:28.597803+0000","last_became_active":"2023-10-06T03:02:28.597456+0000","last_became_peered":"2023-10-06T03:02:28.597456+0000","last_unstale":"2023-10-06T03:02:28.597803+0000","last_undegraded":"2023-10-06T03:02:28.597803+0000","last_fullsized":"2023-10-06T03:02:28.597803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.130409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.589924+0000","last_change":"2023-10-06T03:02:27.588373+0000","last_active":"2023-10-06T03:02:27.589924+0000","last_peered":"2023-10-06T03:02:27.589924+0000","last_clean":"2023-10-06T03:02:27.589924+0000","last_became_active":"2023-10-06T03:02:27.588224+0000","last_became_peered":"2023-10-06T03:02:27.588224+0000","last_unstale":"2023-10-06T03:02:27.589924+0000","last_undegraded":"2023-10-06T03:02:27.589924+0000","last_fullsized":"2023-10-06T03:02:27.589924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:30.283231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602806+0000","last_change":"2023-10-06T03:02:28.602806+0000","last_active":"2023-10-06T03:02:28.602806+0000","last_peered":"2023-10-06T03:02:28.602806+0000","last_clean":"2023-10-06T03:02:28.602806+0000","last_became_active":"2023-10-06T03:02:28.602552+0000","last_became_peered":"2023-10-06T03:02:28.602552+0000","last_unstale":"2023-10-06T03:02:28.602806+0000","last_undegraded":"2023-10-06T03:02:28.602806+0000","last_fullsized":"2023-10-06T03:02:28.602806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:47.335522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597462+0000","last_change":"2023-10-06T03:02:27.591604+0000","last_active":"2023-10-06T03:02:27.597462+0000","last_peered":"2023-10-06T03:02:27.597462+0000","last_clean":"2023-10-06T03:02:27.597462+0000","last_became_active":"2023-10-06T03:02:27.591294+0000","last_became_peered":"2023-10-06T03:02:27.591294+0000","last_unstale":"2023-10-06T03:02:27.597462+0000","last_undegraded":"2023-10-06T03:02:27.597462+0000","last_fullsized":"2023-10-06T03:02:27.597462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:45.260431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597147+0000","last_change":"2023-10-06T03:02:28.597147+0000","last_active":"2023-10-06T03:02:28.597147+0000","last_peered":"2023-10-06T03:02:28.597147+0000","last_clean":"2023-10-06T03:02:28.597147+0000","last_became_active":"2023-10-06T03:02:28.596892+0000","last_became_peered":"2023-10-06T03:02:28.596892+0000","last_unstale":"2023-10-06T03:02:28.597147+0000","last_undegraded":"2023-10-06T03:02:28.597147+0000","last_fullsized":"2023-10-06T03:02:28.597147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:15.928007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595137+0000","last_change":"2023-10-06T03:02:27.590785+0000","last_active":"2023-10-06T03:02:27.595137+0000","last_peered":"2023-10-06T03:02:27.595137+0000","last_clean":"2023-10-06T03:02:27.595137+0000","last_became_active":"2023-10-06T03:02:27.590532+0000","last_became_peered":"2023-10-06T03:02:27.590532+0000","last_unstale":"2023-10-06T03:02:27.595137+0000","last_undegraded":"2023-10-06T03:02:27.595137+0000","last_fullsized":"2023-10-06T03:02:27.595137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:25.006902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594742+0000","last_change":"2023-10-06T03:02:28.594742+0000","last_active":"2023-10-06T03:02:28.594742+0000","last_peered":"2023-10-06T03:02:28.594742+0000","last_clean":"2023-10-06T03:02:28.594742+0000","last_became_active":"2023-10-06T03:02:28.594488+0000","last_became_peered":"2023-10-06T03:02:28.594488+0000","last_unstale":"2023-10-06T03:02:28.594742+0000","last_undegraded":"2023-10-06T03:02:28.594742+0000","last_fullsized":"2023-10-06T03:02:28.594742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:33.086786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595397+0000","last_change":"2023-10-06T03:02:27.591843+0000","last_active":"2023-10-06T03:02:27.595397+0000","last_peered":"2023-10-06T03:02:27.595397+0000","last_clean":"2023-10-06T03:02:27.595397+0000","last_became_active":"2023-10-06T03:02:27.591574+0000","last_became_peered":"2023-10-06T03:02:27.591574+0000","last_unstale":"2023-10-06T03:02:27.595397+0000","last_undegraded":"2023-10-06T03:02:27.595397+0000","last_fullsized":"2023-10-06T03:02:27.595397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:53.669986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.584001+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:02:27.584001+0000","last_peered":"2023-10-06T03:02:27.584001+0000","last_clean":"2023-10-06T03:02:27.584001+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:02:27.584001+0000","last_undegraded":"2023-10-06T03:02:27.584001+0000","last_fullsized":"2023-10-06T03:02:27.584001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591639+0000","last_change":"2023-10-06T03:02:28.591639+0000","last_active":"2023-10-06T03:02:28.591639+0000","last_peered":"2023-10-06T03:02:28.591639+0000","last_clean":"2023-10-06T03:02:28.591639+0000","last_became_active":"2023-10-06T03:02:28.591164+0000","last_became_peered":"2023-10-06T03:02:28.591164+0000","last_unstale":"2023-10-06T03:02:28.591639+0000","last_undegraded":"2023-10-06T03:02:28.591639+0000","last_fullsized":"2023-10-06T03:02:28.591639+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:44.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595503+0000","last_change":"2023-10-06T03:02:27.593767+0000","last_active":"2023-10-06T03:02:27.595503+0000","last_peered":"2023-10-06T03:02:27.595503+0000","last_clean":"2023-10-06T03:02:27.595503+0000","last_became_active":"2023-10-06T03:02:27.593470+0000","last_became_peered":"2023-10-06T03:02:27.593470+0000","last_unstale":"2023-10-06T03:02:27.595503+0000","last_undegraded":"2023-10-06T03:02:27.595503+0000","last_fullsized":"2023-10-06T03:02:27.595503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:01.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.577457+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:02:27.577457+0000","last_peered":"2023-10-06T03:02:27.577457+0000","last_clean":"2023-10-06T03:02:27.577457+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:02:27.577457+0000","last_undegraded":"2023-10-06T03:02:27.577457+0000","last_fullsized":"2023-10-06T03:02:27.577457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.603062+0000","last_change":"2023-10-06T03:02:28.603062+0000","last_active":"2023-10-06T03:02:28.603062+0000","last_peered":"2023-10-06T03:02:28.603062+0000","last_clean":"2023-10-06T03:02:28.603062+0000","last_became_active":"2023-10-06T03:02:28.602213+0000","last_became_peered":"2023-10-06T03:02:28.602213+0000","last_unstale":"2023-10-06T03:02:28.603062+0000","last_undegraded":"2023-10-06T03:02:28.603062+0000","last_fullsized":"2023-10-06T03:02:28.603062+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:23.171200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595567+0000","last_change":"2023-10-06T03:02:27.594470+0000","last_active":"2023-10-06T03:02:27.595567+0000","last_peered":"2023-10-06T03:02:27.595567+0000","last_clean":"2023-10-06T03:02:27.595567+0000","last_became_active":"2023-10-06T03:02:27.594209+0000","last_became_peered":"2023-10-06T03:02:27.594209+0000","last_unstale":"2023-10-06T03:02:27.595567+0000","last_undegraded":"2023-10-06T03:02:27.595567+0000","last_fullsized":"2023-10-06T03:02:27.595567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.587055+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:02:27.587055+0000","last_peered":"2023-10-06T03:02:27.587055+0000","last_clean":"2023-10-06T03:02:27.587055+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:02:27.587055+0000","last_undegraded":"2023-10-06T03:02:27.587055+0000","last_fullsized":"2023-10-06T03:02:27.587055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594979+0000","last_change":"2023-10-06T03:02:28.594979+0000","last_active":"2023-10-06T03:02:28.594979+0000","last_peered":"2023-10-06T03:02:28.594979+0000","last_clean":"2023-10-06T03:02:28.594979+0000","last_became_active":"2023-10-06T03:02:28.594719+0000","last_became_peered":"2023-10-06T03:02:28.594719+0000","last_unstale":"2023-10-06T03:02:28.594979+0000","last_undegraded":"2023-10-06T03:02:28.594979+0000","last_fullsized":"2023-10-06T03:02:28.594979+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:14.295301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597515+0000","last_change":"2023-10-06T03:02:27.592728+0000","last_active":"2023-10-06T03:02:27.597515+0000","last_peered":"2023-10-06T03:02:27.597515+0000","last_clean":"2023-10-06T03:02:27.597515+0000","last_became_active":"2023-10-06T03:02:27.592217+0000","last_became_peered":"2023-10-06T03:02:27.592217+0000","last_unstale":"2023-10-06T03:02:27.597515+0000","last_undegraded":"2023-10-06T03:02:27.597515+0000","last_fullsized":"2023-10-06T03:02:27.597515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:24.623203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.584442+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:02:27.584442+0000","last_peered":"2023-10-06T03:02:27.584442+0000","last_clean":"2023-10-06T03:02:27.584442+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:02:27.584442+0000","last_undegraded":"2023-10-06T03:02:27.584442+0000","last_fullsized":"2023-10-06T03:02:27.584442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":174,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.585281+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:02:27.585281+0000","last_peered":"2023-10-06T03:02:27.585281+0000","last_clean":"2023-10-06T03:02:27.585281+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:02:27.585281+0000","last_undegraded":"2023-10-06T03:02:27.585281+0000","last_fullsized":"2023-10-06T03:02:27.585281+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597033+0000","last_change":"2023-10-06T03:02:28.597033+0000","last_active":"2023-10-06T03:02:28.597033+0000","last_peered":"2023-10-06T03:02:28.597033+0000","last_clean":"2023-10-06T03:02:28.597033+0000","last_became_active":"2023-10-06T03:02:28.596795+0000","last_became_peered":"2023-10-06T03:02:28.596795+0000","last_unstale":"2023-10-06T03:02:28.597033+0000","last_undegraded":"2023-10-06T03:02:28.597033+0000","last_fullsized":"2023-10-06T03:02:28.597033+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:33.260525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.606938+0000","last_change":"2023-10-06T03:02:27.590423+0000","last_active":"2023-10-06T03:02:28.606938+0000","last_peered":"2023-10-06T03:02:28.606938+0000","last_clean":"2023-10-06T03:02:28.606938+0000","last_became_active":"2023-10-06T03:02:27.590019+0000","last_became_peered":"2023-10-06T03:02:27.590019+0000","last_unstale":"2023-10-06T03:02:28.606938+0000","last_undegraded":"2023-10-06T03:02:28.606938+0000","last_fullsized":"2023-10-06T03:02:28.606938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:53.669986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.587678+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:02:27.587678+0000","last_peered":"2023-10-06T03:02:27.587678+0000","last_clean":"2023-10-06T03:02:27.587678+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:02:27.587678+0000","last_undegraded":"2023-10-06T03:02:27.587678+0000","last_fullsized":"2023-10-06T03:02:27.587678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.598646+0000","last_change":"2023-10-06T03:02:28.598646+0000","last_active":"2023-10-06T03:02:28.598646+0000","last_peered":"2023-10-06T03:02:28.598646+0000","last_clean":"2023-10-06T03:02:28.598646+0000","last_became_active":"2023-10-06T03:02:28.598385+0000","last_became_peered":"2023-10-06T03:02:28.598385+0000","last_unstale":"2023-10-06T03:02:28.598646+0000","last_undegraded":"2023-10-06T03:02:28.598646+0000","last_fullsized":"2023-10-06T03:02:28.598646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:39.357965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597304+0000","last_change":"2023-10-06T03:02:27.591146+0000","last_active":"2023-10-06T03:02:27.597304+0000","last_peered":"2023-10-06T03:02:27.597304+0000","last_clean":"2023-10-06T03:02:27.597304+0000","last_became_active":"2023-10-06T03:02:27.590890+0000","last_became_peered":"2023-10-06T03:02:27.590890+0000","last_unstale":"2023-10-06T03:02:27.597304+0000","last_undegraded":"2023-10-06T03:02:27.597304+0000","last_fullsized":"2023-10-06T03:02:27.597304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.585884+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:02:27.585884+0000","last_peered":"2023-10-06T03:02:27.585884+0000","last_clean":"2023-10-06T03:02:27.585884+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:02:27.585884+0000","last_undegraded":"2023-10-06T03:02:27.585884+0000","last_fullsized":"2023-10-06T03:02:27.585884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594697+0000","last_change":"2023-10-06T03:02:28.594697+0000","last_active":"2023-10-06T03:02:28.594697+0000","last_peered":"2023-10-06T03:02:28.594697+0000","last_clean":"2023-10-06T03:02:28.594697+0000","last_became_active":"2023-10-06T03:02:28.594453+0000","last_became_peered":"2023-10-06T03:02:28.594453+0000","last_unstale":"2023-10-06T03:02:28.594697+0000","last_undegraded":"2023-10-06T03:02:28.594697+0000","last_fullsized":"2023-10-06T03:02:28.594697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.624464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.585427+0000","last_change":"2023-10-06T03:02:27.579590+0000","last_active":"2023-10-06T03:02:27.585427+0000","last_peered":"2023-10-06T03:02:27.585427+0000","last_clean":"2023-10-06T03:02:27.585427+0000","last_became_active":"2023-10-06T03:02:27.579305+0000","last_became_peered":"2023-10-06T03:02:27.579305+0000","last_unstale":"2023-10-06T03:02:27.585427+0000","last_undegraded":"2023-10-06T03:02:27.585427+0000","last_fullsized":"2023-10-06T03:02:27.585427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:14.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.589201+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:02:27.589201+0000","last_peered":"2023-10-06T03:02:27.589201+0000","last_clean":"2023-10-06T03:02:27.589201+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:02:27.589201+0000","last_undegraded":"2023-10-06T03:02:27.589201+0000","last_fullsized":"2023-10-06T03:02:27.589201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.598694+0000","last_change":"2023-10-06T03:02:28.598694+0000","last_active":"2023-10-06T03:02:28.598694+0000","last_peered":"2023-10-06T03:02:28.598694+0000","last_clean":"2023-10-06T03:02:28.598694+0000","last_became_active":"2023-10-06T03:02:28.598387+0000","last_became_peered":"2023-10-06T03:02:28.598387+0000","last_unstale":"2023-10-06T03:02:28.598694+0000","last_undegraded":"2023-10-06T03:02:28.598694+0000","last_fullsized":"2023-10-06T03:02:28.598694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:25.619653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.594876+0000","last_change":"2023-10-06T03:02:27.590715+0000","last_active":"2023-10-06T03:02:27.594876+0000","last_peered":"2023-10-06T03:02:27.594876+0000","last_clean":"2023-10-06T03:02:27.594876+0000","last_became_active":"2023-10-06T03:02:27.590436+0000","last_became_peered":"2023-10-06T03:02:27.590436+0000","last_unstale":"2023-10-06T03:02:27.594876+0000","last_undegraded":"2023-10-06T03:02:27.594876+0000","last_fullsized":"2023-10-06T03:02:27.594876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:28.160477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.588609+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:02:27.588609+0000","last_peered":"2023-10-06T03:02:27.588609+0000","last_clean":"2023-10-06T03:02:27.588609+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:02:27.588609+0000","last_undegraded":"2023-10-06T03:02:27.588609+0000","last_fullsized":"2023-10-06T03:02:27.588609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":983040,"data_stored":741603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":142513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":142513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":740031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":151199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":142513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.308},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.36799999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.51000000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.435},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":149741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:02:32.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:02:32.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.793+0000 7f5431915700 1 Processor -- start 2023-10-06T03:02:32.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.793+0000 7f5431915700 1 -- start start 2023-10-06T03:02:32.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.794+0000 7f5431915700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f542c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.794+0000 7f5431915700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c15d7e0 con 0x7f542c15df50 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.794+0000 7f5431915700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c15dac0 con 0x7f542c06bcd0 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.794+0000 7f5431915700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15aa30 con 0x7f542c06c0a0 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.794+0000 7f542b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f542c15df50 0x7f542c0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36108/0 (socket says 172.21.15.92:36108) 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.794+0000 7f542b7fe700 1 -- 172.21.15.92:0/298725790 learned_addr learned my addr 172.21.15.92:0/298725790 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700823453 0 0) 0x7f542c15d7e0 con 0x7f542c15df50 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5418003570 con 0x7f542c15df50 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f542c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4125448111 0 0) 0x7f5418003570 con 0x7f542c15df50 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 msgr2=0x7f542c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f542c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 >> v1:172.21.15.136:6789/0 conn(0x7f542c06bcd0 legacy=0x7f542c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c15ad10 con 0x7f542c15df50 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f54240017f0 con 0x7f542c15df50 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f542c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 >> v1:172.21.15.92:6789/0 conn(0x7f542c15df50 legacy=0x7f542c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418003a90 0x7f5418003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5418004810 0x7f5418004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f541800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15aa30 con 0x7f5418004810 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418005500 con 0x7f5418003a90 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180056c0 con 0x7f542c06c0a0 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5418004810 0x7f5418004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542affd700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418003a90 0x7f5418003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f542b7fe700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f541800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 -- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 msgr2=0x7f541800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c06c0a0 0x7f541800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 -- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418003a90 msgr2=0x7f5418003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418003a90 0x7f5418003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 -- 172.21.15.92:0/298725790 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418003570 con 0x7f5418004810 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.795+0000 7f54227fc700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5418004810 0x7f5418004c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5414009ba0 tx=0x7f5414008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43b75e066846866 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400c070 con 0x7f5418004810 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541400e5e0 con 0x7f5418004810 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f542a7fc700 1 -- 172.21.15.92:0/298725790 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414009510 con 0x7f5418004810 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f5431915700 1 -- 172.21.15.92:0/298725790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5418004810 msgr2=0x7f5418004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f5431915700 1 --2- 172.21.15.92:0/298725790 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5418004810 0x7f5418004c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5414009ba0 tx=0x7f5414008040 comp rx=0 tx=0).stop 2023-10-06T03:02:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f542b7fe700 1 -- 172.21.15.92:0/298725790 reap_dead start 2023-10-06T03:02:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f5431915700 1 -- 172.21.15.92:0/298725790 shutdown_connections 2023-10-06T03:02:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f5431915700 1 -- 172.21.15.92:0/298725790 >> 172.21.15.92:0/298725790 conn(0x7f542c0ce600 msgr2=0x7f542c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f5431915700 1 -- 172.21.15.92:0/298725790 shutdown_connections 2023-10-06T03:02:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.796+0000 7f5431915700 1 -- 172.21.15.92:0/298725790 wait complete. 2023-10-06T03:02:32.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.797+0000 7f5431915700 1 Processor -- start 2023-10-06T03:02:32.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.797+0000 7f5431915700 1 -- start start 2023-10-06T03:02:32.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f5431915700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418004810 0x7f542c16a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f5431915700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c0c8410 0x7f542c143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f5431915700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 0x7f542c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f5431915700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c159b60 con 0x7f542c15b230 2023-10-06T03:02:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f5431915700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c15bbc0 con 0x7f5418004810 2023-10-06T03:02:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f5431915700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c16aed0 con 0x7f542c0c8410 2023-10-06T03:02:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 0x7f542c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 0x7f542c15b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34254/0 (socket says 172.21.15.92:34254) 2023-10-06T03:02:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 -- 172.21.15.92:0/2244787246 learned_addr learned my addr 172.21.15.92:0/2244787246 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542affd700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418004810 0x7f542c16a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 -- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c0c8410 msgr2=0x7f542c143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c0c8410 0x7f542c143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 -- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418004810 msgr2=0x7f542c16a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418004810 0x7f542c16a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.798+0000 7f542b7fe700 1 -- 172.21.15.92:0/2244787246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c158f50 con 0x7f542c15b230 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.799+0000 7f542b7fe700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 0x7f542c15b680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f542c157db0 tx=0x7f542c0ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.799+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400ebc0 con 0x7f542c15b230 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.799+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400ed20 con 0x7f542c15b230 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.799+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424013670 con 0x7f542c15b230 2023-10-06T03:02:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.799+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c159230 con 0x7f542c15b230 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.799+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c1597f0 con 0x7f542c15b230 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.800+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c04c3c0 con 0x7f542c15b230 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.800+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54240139e0 con 0x7f542c15b230 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.800+0000 7f54237fe700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f540c066f30 0x7f540c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.801+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5424084b70 con 0x7f542c15b230 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.801+0000 7f542affd700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f540c066f30 0x7f540c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:32.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.801+0000 7f542affd700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f540c066f30 0x7f540c0693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f541c000f60 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:32.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.805+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5424051be0 con 0x7f542c15b230 2023-10-06T03:02:32.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.984+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f542c15aff0 con 0x7f540c066f30 2023-10-06T03:02:32.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.992+0000 7f54237fe700 1 -- 172.21.15.92:0/2244787246 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289696 (secure 0 0 0) 0x7f542c15aff0 con 0x7f540c066f30 2023-10-06T03:02:32.997 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f540c066f30 msgr2=0x7f540c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f540c066f30 0x7f540c0693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f541c000f60 tx=0x7f541c009040 comp rx=0 tx=0).stop 2023-10-06T03:02:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 msgr2=0x7f542c15b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 0x7f542c15b680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f542c157db0 tx=0x7f542c0ce4d0 comp rx=0 tx=0).stop 2023-10-06T03:02:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 shutdown_connections 2023-10-06T03:02:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f540c066f30 0x7f540c0693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f542c15b230 0x7f542c15b680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5418004810 0x7f542c16a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 --2- 172.21.15.92:0/2244787246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542c0c8410 0x7f542c143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 >> 172.21.15.92:0/2244787246 conn(0x7f542c0ce600 msgr2=0x7f542c0cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 shutdown_connections 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:32.995+0000 7f5431915700 1 -- 172.21.15.92:0/2244787246 wait complete. 2023-10-06T03:02:33.000 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:02:33.008 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-10-06T03:02:31.192392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382388,"kb_used_data":4084,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592332,"statfs":{"total":773094113280,"available":770654547968,"internally_reserved":0,"allocated":4182016,"data_stored":2359812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.625133"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591767+0000","last_change":"2023-10-06T03:02:28.591767+0000","last_active":"2023-10-06T03:02:28.591767+0000","last_peered":"2023-10-06T03:02:28.591767+0000","last_clean":"2023-10-06T03:02:28.591767+0000","last_became_active":"2023-10-06T03:02:28.591426+0000","last_became_peered":"2023-10-06T03:02:28.591426+0000","last_unstale":"2023-10-06T03:02:28.591767+0000","last_undegraded":"2023-10-06T03:02:28.591767+0000","last_fullsized":"2023-10-06T03:02:28.591767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:12.373681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591435+0000","last_change":"2023-10-06T03:02:28.591435+0000","last_active":"2023-10-06T03:02:28.591435+0000","last_peered":"2023-10-06T03:02:28.591435+0000","last_clean":"2023-10-06T03:02:28.591435+0000","last_became_active":"2023-10-06T03:02:28.590854+0000","last_became_peered":"2023-10-06T03:02:28.590854+0000","last_unstale":"2023-10-06T03:02:28.591435+0000","last_undegraded":"2023-10-06T03:02:28.591435+0000","last_fullsized":"2023-10-06T03:02:28.591435+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:09.926644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.593370+0000","last_change":"2023-10-06T03:02:28.593370+0000","last_active":"2023-10-06T03:02:28.593370+0000","last_peered":"2023-10-06T03:02:28.593370+0000","last_clean":"2023-10-06T03:02:28.593370+0000","last_became_active":"2023-10-06T03:02:28.593066+0000","last_became_peered":"2023-10-06T03:02:28.593066+0000","last_unstale":"2023-10-06T03:02:28.593370+0000","last_undegraded":"2023-10-06T03:02:28.593370+0000","last_fullsized":"2023-10-06T03:02:28.593370+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:23.624727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601442+0000","last_change":"2023-10-06T03:02:28.601442+0000","last_active":"2023-10-06T03:02:28.601442+0000","last_peered":"2023-10-06T03:02:28.601442+0000","last_clean":"2023-10-06T03:02:28.601442+0000","last_became_active":"2023-10-06T03:02:28.596354+0000","last_became_peered":"2023-10-06T03:02:28.596354+0000","last_unstale":"2023-10-06T03:02:28.601442+0000","last_undegraded":"2023-10-06T03:02:28.601442+0000","last_fullsized":"2023-10-06T03:02:28.601442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:41.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.586706+0000","last_change":"2023-10-06T03:02:28.586706+0000","last_active":"2023-10-06T03:02:28.586706+0000","last_peered":"2023-10-06T03:02:28.586706+0000","last_clean":"2023-10-06T03:02:28.586706+0000","last_became_active":"2023-10-06T03:02:28.586437+0000","last_became_peered":"2023-10-06T03:02:28.586437+0000","last_unstale":"2023-10-06T03:02:28.586706+0000","last_undegraded":"2023-10-06T03:02:28.586706+0000","last_fullsized":"2023-10-06T03:02:28.586706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:25.627957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.590489+0000","last_change":"2023-10-06T03:02:28.590489+0000","last_active":"2023-10-06T03:02:28.590489+0000","last_peered":"2023-10-06T03:02:28.590489+0000","last_clean":"2023-10-06T03:02:28.590489+0000","last_became_active":"2023-10-06T03:02:28.590204+0000","last_became_peered":"2023-10-06T03:02:28.590204+0000","last_unstale":"2023-10-06T03:02:28.590489+0000","last_undegraded":"2023-10-06T03:02:28.590489+0000","last_fullsized":"2023-10-06T03:02:28.590489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:25.619653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.595506+0000","last_change":"2023-10-06T03:02:28.595506+0000","last_active":"2023-10-06T03:02:28.595506+0000","last_peered":"2023-10-06T03:02:28.595506+0000","last_clean":"2023-10-06T03:02:28.595506+0000","last_became_active":"2023-10-06T03:02:28.594548+0000","last_became_peered":"2023-10-06T03:02:28.594548+0000","last_unstale":"2023-10-06T03:02:28.595506+0000","last_undegraded":"2023-10-06T03:02:28.595506+0000","last_fullsized":"2023-10-06T03:02:28.595506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:37.604138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594907+0000","last_change":"2023-10-06T03:02:28.594907+0000","last_active":"2023-10-06T03:02:28.594907+0000","last_peered":"2023-10-06T03:02:28.594907+0000","last_clean":"2023-10-06T03:02:28.594907+0000","last_became_active":"2023-10-06T03:02:28.594656+0000","last_became_peered":"2023-10-06T03:02:28.594656+0000","last_unstale":"2023-10-06T03:02:28.594907+0000","last_undegraded":"2023-10-06T03:02:28.594907+0000","last_fullsized":"2023-10-06T03:02:28.594907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:29.064735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594132+0000","last_change":"2023-10-06T03:02:28.594132+0000","last_active":"2023-10-06T03:02:28.594132+0000","last_peered":"2023-10-06T03:02:28.594132+0000","last_clean":"2023-10-06T03:02:28.594132+0000","last_became_active":"2023-10-06T03:02:28.593565+0000","last_became_peered":"2023-10-06T03:02:28.593565+0000","last_unstale":"2023-10-06T03:02:28.594132+0000","last_undegraded":"2023-10-06T03:02:28.594132+0000","last_fullsized":"2023-10-06T03:02:28.594132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:33.260525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596708+0000","last_change":"2023-10-06T03:02:28.596708+0000","last_active":"2023-10-06T03:02:28.596708+0000","last_peered":"2023-10-06T03:02:28.596708+0000","last_clean":"2023-10-06T03:02:28.596708+0000","last_became_active":"2023-10-06T03:02:28.596409+0000","last_became_peered":"2023-10-06T03:02:28.596409+0000","last_unstale":"2023-10-06T03:02:28.596708+0000","last_undegraded":"2023-10-06T03:02:28.596708+0000","last_fullsized":"2023-10-06T03:02:28.596708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:29.064735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.593879+0000","last_change":"2023-10-06T03:02:28.593879+0000","last_active":"2023-10-06T03:02:28.593879+0000","last_peered":"2023-10-06T03:02:28.593879+0000","last_clean":"2023-10-06T03:02:28.593879+0000","last_became_active":"2023-10-06T03:02:28.593598+0000","last_became_peered":"2023-10-06T03:02:28.593598+0000","last_unstale":"2023-10-06T03:02:28.593879+0000","last_undegraded":"2023-10-06T03:02:28.593879+0000","last_fullsized":"2023-10-06T03:02:28.593879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:46.533418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.588264+0000","last_change":"2023-10-06T03:02:28.588264+0000","last_active":"2023-10-06T03:02:28.588264+0000","last_peered":"2023-10-06T03:02:28.588264+0000","last_clean":"2023-10-06T03:02:28.588264+0000","last_became_active":"2023-10-06T03:02:28.587712+0000","last_became_peered":"2023-10-06T03:02:28.587712+0000","last_unstale":"2023-10-06T03:02:28.588264+0000","last_undegraded":"2023-10-06T03:02:28.588264+0000","last_fullsized":"2023-10-06T03:02:28.588264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:37.604138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596805+0000","last_change":"2023-10-06T03:02:28.596805+0000","last_active":"2023-10-06T03:02:28.596805+0000","last_peered":"2023-10-06T03:02:28.596805+0000","last_clean":"2023-10-06T03:02:28.596805+0000","last_became_active":"2023-10-06T03:02:28.596533+0000","last_became_peered":"2023-10-06T03:02:28.596533+0000","last_unstale":"2023-10-06T03:02:28.596805+0000","last_undegraded":"2023-10-06T03:02:28.596805+0000","last_fullsized":"2023-10-06T03:02:28.596805+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:12.373681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604298+0000","last_change":"2023-10-06T03:02:28.604298+0000","last_active":"2023-10-06T03:02:28.604298+0000","last_peered":"2023-10-06T03:02:28.604298+0000","last_clean":"2023-10-06T03:02:28.604298+0000","last_became_active":"2023-10-06T03:02:28.604027+0000","last_became_peered":"2023-10-06T03:02:28.604027+0000","last_unstale":"2023-10-06T03:02:28.604298+0000","last_undegraded":"2023-10-06T03:02:28.604298+0000","last_fullsized":"2023-10-06T03:02:28.604298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:03.102138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594059+0000","last_change":"2023-10-06T03:02:28.594059+0000","last_active":"2023-10-06T03:02:28.594059+0000","last_peered":"2023-10-06T03:02:28.594059+0000","last_clean":"2023-10-06T03:02:28.594059+0000","last_became_active":"2023-10-06T03:02:28.593791+0000","last_became_peered":"2023-10-06T03:02:28.593791+0000","last_unstale":"2023-10-06T03:02:28.594059+0000","last_undegraded":"2023-10-06T03:02:28.594059+0000","last_fullsized":"2023-10-06T03:02:28.594059+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:15.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596582+0000","last_change":"2023-10-06T03:02:28.596582+0000","last_active":"2023-10-06T03:02:28.596582+0000","last_peered":"2023-10-06T03:02:28.596582+0000","last_clean":"2023-10-06T03:02:28.596582+0000","last_became_active":"2023-10-06T03:02:28.596314+0000","last_became_peered":"2023-10-06T03:02:28.596314+0000","last_unstale":"2023-10-06T03:02:28.596582+0000","last_undegraded":"2023-10-06T03:02:28.596582+0000","last_fullsized":"2023-10-06T03:02:28.596582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:14.295301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601095+0000","last_change":"2023-10-06T03:02:28.601095+0000","last_active":"2023-10-06T03:02:28.601095+0000","last_peered":"2023-10-06T03:02:28.601095+0000","last_clean":"2023-10-06T03:02:28.601095+0000","last_became_active":"2023-10-06T03:02:28.600831+0000","last_became_peered":"2023-10-06T03:02:28.600831+0000","last_unstale":"2023-10-06T03:02:28.601095+0000","last_undegraded":"2023-10-06T03:02:28.601095+0000","last_fullsized":"2023-10-06T03:02:28.601095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:46.533418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601776+0000","last_change":"2023-10-06T03:02:28.601776+0000","last_active":"2023-10-06T03:02:28.601776+0000","last_peered":"2023-10-06T03:02:28.601776+0000","last_clean":"2023-10-06T03:02:28.601776+0000","last_became_active":"2023-10-06T03:02:28.601482+0000","last_became_peered":"2023-10-06T03:02:28.601482+0000","last_unstale":"2023-10-06T03:02:28.601776+0000","last_undegraded":"2023-10-06T03:02:28.601776+0000","last_fullsized":"2023-10-06T03:02:28.601776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:46.265185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.588842+0000","last_change":"2023-10-06T03:02:28.588842+0000","last_active":"2023-10-06T03:02:28.588842+0000","last_peered":"2023-10-06T03:02:28.588842+0000","last_clean":"2023-10-06T03:02:28.588842+0000","last_became_active":"2023-10-06T03:02:28.588581+0000","last_became_peered":"2023-10-06T03:02:28.588581+0000","last_unstale":"2023-10-06T03:02:28.588842+0000","last_undegraded":"2023-10-06T03:02:28.588842+0000","last_fullsized":"2023-10-06T03:02:28.588842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:54.674740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602198+0000","last_change":"2023-10-06T03:02:28.602198+0000","last_active":"2023-10-06T03:02:28.602198+0000","last_peered":"2023-10-06T03:02:28.602198+0000","last_clean":"2023-10-06T03:02:28.602198+0000","last_became_active":"2023-10-06T03:02:28.601919+0000","last_became_peered":"2023-10-06T03:02:28.601919+0000","last_unstale":"2023-10-06T03:02:28.602198+0000","last_undegraded":"2023-10-06T03:02:28.602198+0000","last_fullsized":"2023-10-06T03:02:28.602198+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:25.627957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.592211+0000","last_change":"2023-10-06T03:02:28.592211+0000","last_active":"2023-10-06T03:02:28.592211+0000","last_peered":"2023-10-06T03:02:28.592211+0000","last_clean":"2023-10-06T03:02:28.592211+0000","last_became_active":"2023-10-06T03:02:28.591936+0000","last_became_peered":"2023-10-06T03:02:28.591936+0000","last_unstale":"2023-10-06T03:02:28.592211+0000","last_undegraded":"2023-10-06T03:02:28.592211+0000","last_fullsized":"2023-10-06T03:02:28.592211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:15.928007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587905+0000","last_change":"2023-10-06T03:02:28.587905+0000","last_active":"2023-10-06T03:02:28.587905+0000","last_peered":"2023-10-06T03:02:28.587905+0000","last_clean":"2023-10-06T03:02:28.587905+0000","last_became_active":"2023-10-06T03:02:28.587624+0000","last_became_peered":"2023-10-06T03:02:28.587624+0000","last_unstale":"2023-10-06T03:02:28.587905+0000","last_undegraded":"2023-10-06T03:02:28.587905+0000","last_fullsized":"2023-10-06T03:02:28.587905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:23.557486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.605750+0000","last_change":"2023-10-06T03:02:28.605750+0000","last_active":"2023-10-06T03:02:28.605750+0000","last_peered":"2023-10-06T03:02:28.605750+0000","last_clean":"2023-10-06T03:02:28.605750+0000","last_became_active":"2023-10-06T03:02:28.605494+0000","last_became_peered":"2023-10-06T03:02:28.605494+0000","last_unstale":"2023-10-06T03:02:28.605750+0000","last_undegraded":"2023-10-06T03:02:28.605750+0000","last_fullsized":"2023-10-06T03:02:28.605750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:15.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601376+0000","last_change":"2023-10-06T03:02:28.601376+0000","last_active":"2023-10-06T03:02:28.601376+0000","last_peered":"2023-10-06T03:02:28.601376+0000","last_clean":"2023-10-06T03:02:28.601376+0000","last_became_active":"2023-10-06T03:02:28.601089+0000","last_became_peered":"2023-10-06T03:02:28.601089+0000","last_unstale":"2023-10-06T03:02:28.601376+0000","last_undegraded":"2023-10-06T03:02:28.601376+0000","last_fullsized":"2023-10-06T03:02:28.601376+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:00.837178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604987+0000","last_change":"2023-10-06T03:02:28.604987+0000","last_active":"2023-10-06T03:02:28.604987+0000","last_peered":"2023-10-06T03:02:28.604987+0000","last_clean":"2023-10-06T03:02:28.604987+0000","last_became_active":"2023-10-06T03:02:28.601264+0000","last_became_peered":"2023-10-06T03:02:28.601264+0000","last_unstale":"2023-10-06T03:02:28.604987+0000","last_undegraded":"2023-10-06T03:02:28.604987+0000","last_fullsized":"2023-10-06T03:02:28.604987+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:41.502506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.595998+0000","last_change":"2023-10-06T03:02:28.595998+0000","last_active":"2023-10-06T03:02:28.595998+0000","last_peered":"2023-10-06T03:02:28.595998+0000","last_clean":"2023-10-06T03:02:28.595998+0000","last_became_active":"2023-10-06T03:02:28.595734+0000","last_became_peered":"2023-10-06T03:02:28.595734+0000","last_unstale":"2023-10-06T03:02:28.595998+0000","last_undegraded":"2023-10-06T03:02:28.595998+0000","last_fullsized":"2023-10-06T03:02:28.595998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.130409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604991+0000","last_change":"2023-10-06T03:02:28.604991+0000","last_active":"2023-10-06T03:02:28.604991+0000","last_peered":"2023-10-06T03:02:28.604991+0000","last_clean":"2023-10-06T03:02:28.604991+0000","last_became_active":"2023-10-06T03:02:28.604732+0000","last_became_peered":"2023-10-06T03:02:28.604732+0000","last_unstale":"2023-10-06T03:02:28.604991+0000","last_undegraded":"2023-10-06T03:02:28.604991+0000","last_fullsized":"2023-10-06T03:02:28.604991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:02.045058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594870+0000","last_change":"2023-10-06T03:02:28.594870+0000","last_active":"2023-10-06T03:02:28.594870+0000","last_peered":"2023-10-06T03:02:28.594870+0000","last_clean":"2023-10-06T03:02:28.594870+0000","last_became_active":"2023-10-06T03:02:28.594610+0000","last_became_peered":"2023-10-06T03:02:28.594610+0000","last_unstale":"2023-10-06T03:02:28.594870+0000","last_undegraded":"2023-10-06T03:02:28.594870+0000","last_fullsized":"2023-10-06T03:02:28.594870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:44.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602829+0000","last_change":"2023-10-06T03:02:28.602829+0000","last_active":"2023-10-06T03:02:28.602829+0000","last_peered":"2023-10-06T03:02:28.602829+0000","last_clean":"2023-10-06T03:02:28.602829+0000","last_became_active":"2023-10-06T03:02:28.602535+0000","last_became_peered":"2023-10-06T03:02:28.602535+0000","last_unstale":"2023-10-06T03:02:28.602829+0000","last_undegraded":"2023-10-06T03:02:28.602829+0000","last_fullsized":"2023-10-06T03:02:28.602829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:51.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604509+0000","last_change":"2023-10-06T03:02:28.604509+0000","last_active":"2023-10-06T03:02:28.604509+0000","last_peered":"2023-10-06T03:02:28.604509+0000","last_clean":"2023-10-06T03:02:28.604509+0000","last_became_active":"2023-10-06T03:02:28.604203+0000","last_became_peered":"2023-10-06T03:02:28.604203+0000","last_unstale":"2023-10-06T03:02:28.604509+0000","last_undegraded":"2023-10-06T03:02:28.604509+0000","last_fullsized":"2023-10-06T03:02:28.604509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:00.837178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.606031+0000","last_change":"2023-10-06T03:02:28.606031+0000","last_active":"2023-10-06T03:02:28.606031+0000","last_peered":"2023-10-06T03:02:28.606031+0000","last_clean":"2023-10-06T03:02:28.606031+0000","last_became_active":"2023-10-06T03:02:28.605162+0000","last_became_peered":"2023-10-06T03:02:28.605162+0000","last_unstale":"2023-10-06T03:02:28.606031+0000","last_undegraded":"2023-10-06T03:02:28.606031+0000","last_fullsized":"2023-10-06T03:02:28.606031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:37.604138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.590034+0000","last_change":"2023-10-06T03:02:28.590034+0000","last_active":"2023-10-06T03:02:28.590034+0000","last_peered":"2023-10-06T03:02:28.590034+0000","last_clean":"2023-10-06T03:02:28.590034+0000","last_became_active":"2023-10-06T03:02:28.589817+0000","last_became_peered":"2023-10-06T03:02:28.589817+0000","last_unstale":"2023-10-06T03:02:28.590034+0000","last_undegraded":"2023-10-06T03:02:28.590034+0000","last_fullsized":"2023-10-06T03:02:28.590034+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:34.626012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594017+0000","last_change":"2023-10-06T03:02:28.594017+0000","last_active":"2023-10-06T03:02:28.594017+0000","last_peered":"2023-10-06T03:02:28.594017+0000","last_clean":"2023-10-06T03:02:28.594017+0000","last_became_active":"2023-10-06T03:02:28.593737+0000","last_became_peered":"2023-10-06T03:02:28.593737+0000","last_unstale":"2023-10-06T03:02:28.594017+0000","last_undegraded":"2023-10-06T03:02:28.594017+0000","last_fullsized":"2023-10-06T03:02:28.594017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:33.086786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.608404+0000","last_change":"2023-10-06T03:02:27.590039+0000","last_active":"2023-10-06T03:02:28.608404+0000","last_peered":"2023-10-06T03:02:28.608404+0000","last_clean":"2023-10-06T03:02:28.608404+0000","last_became_active":"2023-10-06T03:02:27.589818+0000","last_became_peered":"2023-10-06T03:02:27.589818+0000","last_unstale":"2023-10-06T03:02:28.608404+0000","last_undegraded":"2023-10-06T03:02:28.608404+0000","last_fullsized":"2023-10-06T03:02:28.608404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:40.995528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597759+0000","last_change":"2023-10-06T03:02:28.597759+0000","last_active":"2023-10-06T03:02:28.597759+0000","last_peered":"2023-10-06T03:02:28.597759+0000","last_clean":"2023-10-06T03:02:28.597759+0000","last_became_active":"2023-10-06T03:02:28.597363+0000","last_became_peered":"2023-10-06T03:02:28.597363+0000","last_unstale":"2023-10-06T03:02:28.597759+0000","last_undegraded":"2023-10-06T03:02:28.597759+0000","last_fullsized":"2023-10-06T03:02:28.597759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:23.171200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.578158+0000","last_change":"2023-10-06T03:02:27.591875+0000","last_active":"2023-10-06T03:02:28.578158+0000","last_peered":"2023-10-06T03:02:28.578158+0000","last_clean":"2023-10-06T03:02:28.578158+0000","last_became_active":"2023-10-06T03:02:27.591625+0000","last_became_peered":"2023-10-06T03:02:27.591625+0000","last_unstale":"2023-10-06T03:02:28.578158+0000","last_undegraded":"2023-10-06T03:02:28.578158+0000","last_fullsized":"2023-10-06T03:02:28.578158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:40.497752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602070+0000","last_change":"2023-10-06T03:02:28.602070+0000","last_active":"2023-10-06T03:02:28.602070+0000","last_peered":"2023-10-06T03:02:28.602070+0000","last_clean":"2023-10-06T03:02:28.602070+0000","last_became_active":"2023-10-06T03:02:28.601590+0000","last_became_peered":"2023-10-06T03:02:28.601590+0000","last_unstale":"2023-10-06T03:02:28.602070+0000","last_undegraded":"2023-10-06T03:02:28.602070+0000","last_fullsized":"2023-10-06T03:02:28.602070+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:45.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604382+0000","last_change":"2023-10-06T03:02:27.592012+0000","last_active":"2023-10-06T03:02:28.604382+0000","last_peered":"2023-10-06T03:02:28.604382+0000","last_clean":"2023-10-06T03:02:28.604382+0000","last_became_active":"2023-10-06T03:02:27.591313+0000","last_became_peered":"2023-10-06T03:02:27.591313+0000","last_unstale":"2023-10-06T03:02:28.604382+0000","last_undegraded":"2023-10-06T03:02:28.604382+0000","last_fullsized":"2023-10-06T03:02:28.604382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:59.832424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.603963+0000","last_change":"2023-10-06T03:02:28.603963+0000","last_active":"2023-10-06T03:02:28.603963+0000","last_peered":"2023-10-06T03:02:28.603963+0000","last_clean":"2023-10-06T03:02:28.603963+0000","last_became_active":"2023-10-06T03:02:28.603708+0000","last_became_peered":"2023-10-06T03:02:28.603708+0000","last_unstale":"2023-10-06T03:02:28.603963+0000","last_undegraded":"2023-10-06T03:02:28.603963+0000","last_fullsized":"2023-10-06T03:02:28.603963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.624464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.610894+0000","last_change":"2023-10-06T03:02:27.583485+0000","last_active":"2023-10-06T03:02:28.610894+0000","last_peered":"2023-10-06T03:02:28.610894+0000","last_clean":"2023-10-06T03:02:28.610894+0000","last_became_active":"2023-10-06T03:02:27.580481+0000","last_became_peered":"2023-10-06T03:02:27.580481+0000","last_unstale":"2023-10-06T03:02:28.610894+0000","last_undegraded":"2023-10-06T03:02:28.610894+0000","last_fullsized":"2023-10-06T03:02:28.610894+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:32.082032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.590185+0000","last_change":"2023-10-06T03:02:28.590185+0000","last_active":"2023-10-06T03:02:28.590185+0000","last_peered":"2023-10-06T03:02:28.590185+0000","last_clean":"2023-10-06T03:02:28.590185+0000","last_became_active":"2023-10-06T03:02:28.589928+0000","last_became_peered":"2023-10-06T03:02:28.589928+0000","last_unstale":"2023-10-06T03:02:28.590185+0000","last_undegraded":"2023-10-06T03:02:28.590185+0000","last_fullsized":"2023-10-06T03:02:28.590185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:15.851009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587157+0000","last_change":"2023-10-06T03:02:27.588523+0000","last_active":"2023-10-06T03:02:28.587157+0000","last_peered":"2023-10-06T03:02:28.587157+0000","last_clean":"2023-10-06T03:02:28.587157+0000","last_became_active":"2023-10-06T03:02:27.588249+0000","last_became_peered":"2023-10-06T03:02:27.588249+0000","last_unstale":"2023-10-06T03:02:28.587157+0000","last_undegraded":"2023-10-06T03:02:28.587157+0000","last_fullsized":"2023-10-06T03:02:28.587157+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:01.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591431+0000","last_change":"2023-10-06T03:02:28.591431+0000","last_active":"2023-10-06T03:02:28.591431+0000","last_peered":"2023-10-06T03:02:28.591431+0000","last_clean":"2023-10-06T03:02:28.591431+0000","last_became_active":"2023-10-06T03:02:28.591090+0000","last_became_peered":"2023-10-06T03:02:28.591090+0000","last_unstale":"2023-10-06T03:02:28.591431+0000","last_undegraded":"2023-10-06T03:02:28.591431+0000","last_fullsized":"2023-10-06T03:02:28.591431+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:11.561752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.585742+0000","last_change":"2023-10-06T03:02:27.591685+0000","last_active":"2023-10-06T03:02:28.585742+0000","last_peered":"2023-10-06T03:02:28.585742+0000","last_clean":"2023-10-06T03:02:28.585742+0000","last_became_active":"2023-10-06T03:02:27.591429+0000","last_became_peered":"2023-10-06T03:02:27.591429+0000","last_unstale":"2023-10-06T03:02:28.585742+0000","last_undegraded":"2023-10-06T03:02:28.585742+0000","last_fullsized":"2023-10-06T03:02:28.585742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:09.871577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597043+0000","last_change":"2023-10-06T03:02:28.597043+0000","last_active":"2023-10-06T03:02:28.597043+0000","last_peered":"2023-10-06T03:02:28.597043+0000","last_clean":"2023-10-06T03:02:28.597043+0000","last_became_active":"2023-10-06T03:02:28.593037+0000","last_became_peered":"2023-10-06T03:02:28.593037+0000","last_unstale":"2023-10-06T03:02:28.597043+0000","last_undegraded":"2023-10-06T03:02:28.597043+0000","last_fullsized":"2023-10-06T03:02:28.597043+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:09.622324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.596479+0000","last_change":"2023-10-06T03:02:27.592599+0000","last_active":"2023-10-06T03:02:27.596479+0000","last_peered":"2023-10-06T03:02:27.596479+0000","last_clean":"2023-10-06T03:02:27.596479+0000","last_became_active":"2023-10-06T03:02:27.592342+0000","last_became_peered":"2023-10-06T03:02:27.592342+0000","last_unstale":"2023-10-06T03:02:27.596479+0000","last_undegraded":"2023-10-06T03:02:27.596479+0000","last_fullsized":"2023-10-06T03:02:27.596479+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:45.260431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596951+0000","last_change":"2023-10-06T03:02:28.596951+0000","last_active":"2023-10-06T03:02:28.596951+0000","last_peered":"2023-10-06T03:02:28.596951+0000","last_clean":"2023-10-06T03:02:28.596951+0000","last_became_active":"2023-10-06T03:02:28.596689+0000","last_became_peered":"2023-10-06T03:02:28.596689+0000","last_unstale":"2023-10-06T03:02:28.596951+0000","last_undegraded":"2023-10-06T03:02:28.596951+0000","last_fullsized":"2023-10-06T03:02:28.596951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:47.335522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.594099+0000","last_change":"2023-10-06T03:02:27.591313+0000","last_active":"2023-10-06T03:02:27.594099+0000","last_peered":"2023-10-06T03:02:27.594099+0000","last_clean":"2023-10-06T03:02:27.594099+0000","last_became_active":"2023-10-06T03:02:27.591066+0000","last_became_peered":"2023-10-06T03:02:27.591066+0000","last_unstale":"2023-10-06T03:02:27.594099+0000","last_undegraded":"2023-10-06T03:02:27.594099+0000","last_fullsized":"2023-10-06T03:02:27.594099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:36.599384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.601572+0000","last_change":"2023-10-06T03:02:28.601572+0000","last_active":"2023-10-06T03:02:28.601572+0000","last_peered":"2023-10-06T03:02:28.601572+0000","last_clean":"2023-10-06T03:02:28.601572+0000","last_became_active":"2023-10-06T03:02:28.596311+0000","last_became_peered":"2023-10-06T03:02:28.596311+0000","last_unstale":"2023-10-06T03:02:28.601572+0000","last_undegraded":"2023-10-06T03:02:28.601572+0000","last_fullsized":"2023-10-06T03:02:28.601572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:02.045058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595192+0000","last_change":"2023-10-06T03:02:27.592032+0000","last_active":"2023-10-06T03:02:27.595192+0000","last_peered":"2023-10-06T03:02:27.595192+0000","last_clean":"2023-10-06T03:02:27.595192+0000","last_became_active":"2023-10-06T03:02:27.591715+0000","last_became_peered":"2023-10-06T03:02:27.591715+0000","last_unstale":"2023-10-06T03:02:27.595192+0000","last_undegraded":"2023-10-06T03:02:27.595192+0000","last_fullsized":"2023-10-06T03:02:27.595192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:24.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602262+0000","last_change":"2023-10-06T03:02:28.602262+0000","last_active":"2023-10-06T03:02:28.602262+0000","last_peered":"2023-10-06T03:02:28.602262+0000","last_clean":"2023-10-06T03:02:28.602262+0000","last_became_active":"2023-10-06T03:02:28.601932+0000","last_became_peered":"2023-10-06T03:02:28.601932+0000","last_unstale":"2023-10-06T03:02:28.602262+0000","last_undegraded":"2023-10-06T03:02:28.602262+0000","last_fullsized":"2023-10-06T03:02:28.602262+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:23.624727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.593610+0000","last_change":"2023-10-06T03:02:27.591529+0000","last_active":"2023-10-06T03:02:27.593610+0000","last_peered":"2023-10-06T03:02:27.593610+0000","last_clean":"2023-10-06T03:02:27.593610+0000","last_became_active":"2023-10-06T03:02:27.591256+0000","last_became_peered":"2023-10-06T03:02:27.591256+0000","last_unstale":"2023-10-06T03:02:27.593610+0000","last_undegraded":"2023-10-06T03:02:27.593610+0000","last_fullsized":"2023-10-06T03:02:27.593610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:40.497752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.596626+0000","last_change":"2023-10-06T03:02:28.596626+0000","last_active":"2023-10-06T03:02:28.596626+0000","last_peered":"2023-10-06T03:02:28.596626+0000","last_clean":"2023-10-06T03:02:28.596626+0000","last_became_active":"2023-10-06T03:02:28.596310+0000","last_became_peered":"2023-10-06T03:02:28.596310+0000","last_unstale":"2023-10-06T03:02:28.596626+0000","last_undegraded":"2023-10-06T03:02:28.596626+0000","last_fullsized":"2023-10-06T03:02:28.596626+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:23.171200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.607136+0000","last_change":"2023-10-06T03:02:27.593977+0000","last_active":"2023-10-06T03:02:28.607136+0000","last_peered":"2023-10-06T03:02:28.607136+0000","last_clean":"2023-10-06T03:02:28.607136+0000","last_became_active":"2023-10-06T03:02:27.593704+0000","last_became_peered":"2023-10-06T03:02:27.593704+0000","last_unstale":"2023-10-06T03:02:28.607136+0000","last_undegraded":"2023-10-06T03:02:28.607136+0000","last_fullsized":"2023-10-06T03:02:28.607136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:09.138335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.589064+0000","last_change":"2023-10-06T03:02:28.589064+0000","last_active":"2023-10-06T03:02:28.589064+0000","last_peered":"2023-10-06T03:02:28.589064+0000","last_clean":"2023-10-06T03:02:28.589064+0000","last_became_active":"2023-10-06T03:02:28.588732+0000","last_became_peered":"2023-10-06T03:02:28.588732+0000","last_unstale":"2023-10-06T03:02:28.589064+0000","last_undegraded":"2023-10-06T03:02:28.589064+0000","last_fullsized":"2023-10-06T03:02:28.589064+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:12.750850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.608245+0000","last_change":"2023-10-06T03:02:27.589760+0000","last_active":"2023-10-06T03:02:28.608245+0000","last_peered":"2023-10-06T03:02:28.608245+0000","last_clean":"2023-10-06T03:02:28.608245+0000","last_became_active":"2023-10-06T03:02:27.589398+0000","last_became_peered":"2023-10-06T03:02:27.589398+0000","last_unstale":"2023-10-06T03:02:28.608245+0000","last_undegraded":"2023-10-06T03:02:28.608245+0000","last_fullsized":"2023-10-06T03:02:28.608245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:36.507008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591761+0000","last_change":"2023-10-06T03:02:28.591761+0000","last_active":"2023-10-06T03:02:28.591761+0000","last_peered":"2023-10-06T03:02:28.591761+0000","last_clean":"2023-10-06T03:02:28.591761+0000","last_became_active":"2023-10-06T03:02:28.591485+0000","last_became_peered":"2023-10-06T03:02:28.591485+0000","last_unstale":"2023-10-06T03:02:28.591761+0000","last_undegraded":"2023-10-06T03:02:28.591761+0000","last_fullsized":"2023-10-06T03:02:28.591761+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:14.295301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.604146+0000","last_change":"2023-10-06T03:02:27.587620+0000","last_active":"2023-10-06T03:02:28.604146+0000","last_peered":"2023-10-06T03:02:28.604146+0000","last_clean":"2023-10-06T03:02:28.604146+0000","last_became_active":"2023-10-06T03:02:27.587387+0000","last_became_peered":"2023-10-06T03:02:27.587387+0000","last_unstale":"2023-10-06T03:02:28.604146+0000","last_undegraded":"2023-10-06T03:02:28.604146+0000","last_fullsized":"2023-10-06T03:02:28.604146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:00.684065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587279+0000","last_change":"2023-10-06T03:02:28.587279+0000","last_active":"2023-10-06T03:02:28.587279+0000","last_peered":"2023-10-06T03:02:28.587279+0000","last_clean":"2023-10-06T03:02:28.587279+0000","last_became_active":"2023-10-06T03:02:28.587042+0000","last_became_peered":"2023-10-06T03:02:28.587042+0000","last_unstale":"2023-10-06T03:02:28.587279+0000","last_undegraded":"2023-10-06T03:02:28.587279+0000","last_fullsized":"2023-10-06T03:02:28.587279+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:11.561752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.596536+0000","last_change":"2023-10-06T03:02:27.593855+0000","last_active":"2023-10-06T03:02:27.596536+0000","last_peered":"2023-10-06T03:02:27.596536+0000","last_clean":"2023-10-06T03:02:27.596536+0000","last_became_active":"2023-10-06T03:02:27.593595+0000","last_became_peered":"2023-10-06T03:02:27.593595+0000","last_unstale":"2023-10-06T03:02:27.596536+0000","last_undegraded":"2023-10-06T03:02:27.596536+0000","last_fullsized":"2023-10-06T03:02:27.596536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:01.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594846+0000","last_change":"2023-10-06T03:02:28.594846+0000","last_active":"2023-10-06T03:02:28.594846+0000","last_peered":"2023-10-06T03:02:28.594846+0000","last_clean":"2023-10-06T03:02:28.594846+0000","last_became_active":"2023-10-06T03:02:28.594605+0000","last_became_peered":"2023-10-06T03:02:28.594605+0000","last_unstale":"2023-10-06T03:02:28.594846+0000","last_undegraded":"2023-10-06T03:02:28.594846+0000","last_fullsized":"2023-10-06T03:02:28.594846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.130409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597311+0000","last_change":"2023-10-06T03:02:27.590192+0000","last_active":"2023-10-06T03:02:27.597311+0000","last_peered":"2023-10-06T03:02:27.597311+0000","last_clean":"2023-10-06T03:02:27.597311+0000","last_became_active":"2023-10-06T03:02:27.589914+0000","last_became_peered":"2023-10-06T03:02:27.589914+0000","last_unstale":"2023-10-06T03:02:27.597311+0000","last_undegraded":"2023-10-06T03:02:27.597311+0000","last_fullsized":"2023-10-06T03:02:27.597311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:25.006902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.592307+0000","last_change":"2023-10-06T03:02:28.592307+0000","last_active":"2023-10-06T03:02:28.592307+0000","last_peered":"2023-10-06T03:02:28.592307+0000","last_clean":"2023-10-06T03:02:28.592307+0000","last_became_active":"2023-10-06T03:02:28.592092+0000","last_became_peered":"2023-10-06T03:02:28.592092+0000","last_unstale":"2023-10-06T03:02:28.592307+0000","last_undegraded":"2023-10-06T03:02:28.592307+0000","last_fullsized":"2023-10-06T03:02:28.592307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:45.643495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.613240+0000","last_change":"2023-10-06T03:02:27.591740+0000","last_active":"2023-10-06T03:02:28.613240+0000","last_peered":"2023-10-06T03:02:28.613240+0000","last_clean":"2023-10-06T03:02:28.613240+0000","last_became_active":"2023-10-06T03:02:27.591457+0000","last_became_peered":"2023-10-06T03:02:27.591457+0000","last_unstale":"2023-10-06T03:02:28.613240+0000","last_undegraded":"2023-10-06T03:02:28.613240+0000","last_fullsized":"2023-10-06T03:02:28.613240+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:50.788840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602597+0000","last_change":"2023-10-06T03:02:28.602597+0000","last_active":"2023-10-06T03:02:28.602597+0000","last_peered":"2023-10-06T03:02:28.602597+0000","last_clean":"2023-10-06T03:02:28.602597+0000","last_became_active":"2023-10-06T03:02:28.602322+0000","last_became_peered":"2023-10-06T03:02:28.602322+0000","last_unstale":"2023-10-06T03:02:28.602597+0000","last_undegraded":"2023-10-06T03:02:28.602597+0000","last_fullsized":"2023-10-06T03:02:28.602597+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:09.622324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587301+0000","last_change":"2023-10-06T03:02:27.589171+0000","last_active":"2023-10-06T03:02:28.587301+0000","last_peered":"2023-10-06T03:02:28.587301+0000","last_clean":"2023-10-06T03:02:28.587301+0000","last_became_active":"2023-10-06T03:02:27.588877+0000","last_became_peered":"2023-10-06T03:02:27.588877+0000","last_unstale":"2023-10-06T03:02:28.587301+0000","last_undegraded":"2023-10-06T03:02:28.587301+0000","last_fullsized":"2023-10-06T03:02:28.587301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:59.832424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591518+0000","last_change":"2023-10-06T03:02:28.591518+0000","last_active":"2023-10-06T03:02:28.591518+0000","last_peered":"2023-10-06T03:02:28.591518+0000","last_clean":"2023-10-06T03:02:28.591518+0000","last_became_active":"2023-10-06T03:02:28.591138+0000","last_became_peered":"2023-10-06T03:02:28.591138+0000","last_unstale":"2023-10-06T03:02:28.591518+0000","last_undegraded":"2023-10-06T03:02:28.591518+0000","last_fullsized":"2023-10-06T03:02:28.591518+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:09.926644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.610797+0000","last_change":"2023-10-06T03:02:27.588184+0000","last_active":"2023-10-06T03:02:28.610797+0000","last_peered":"2023-10-06T03:02:28.610797+0000","last_clean":"2023-10-06T03:02:28.610797+0000","last_became_active":"2023-10-06T03:02:27.587911+0000","last_became_peered":"2023-10-06T03:02:27.587911+0000","last_unstale":"2023-10-06T03:02:28.610797+0000","last_undegraded":"2023-10-06T03:02:28.610797+0000","last_fullsized":"2023-10-06T03:02:28.610797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:22.619973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.587681+0000","last_change":"2023-10-06T03:02:28.587681+0000","last_active":"2023-10-06T03:02:28.587681+0000","last_peered":"2023-10-06T03:02:28.587681+0000","last_clean":"2023-10-06T03:02:28.587681+0000","last_became_active":"2023-10-06T03:02:28.587463+0000","last_became_peered":"2023-10-06T03:02:28.587463+0000","last_unstale":"2023-10-06T03:02:28.587681+0000","last_undegraded":"2023-10-06T03:02:28.587681+0000","last_fullsized":"2023-10-06T03:02:28.587681+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:09:45.677843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.582259+0000","last_change":"2023-10-06T03:02:27.588070+0000","last_active":"2023-10-06T03:02:28.582259+0000","last_peered":"2023-10-06T03:02:28.582259+0000","last_clean":"2023-10-06T03:02:28.582259+0000","last_became_active":"2023-10-06T03:02:27.587798+0000","last_became_peered":"2023-10-06T03:02:27.587798+0000","last_unstale":"2023-10-06T03:02:28.582259+0000","last_undegraded":"2023-10-06T03:02:28.582259+0000","last_fullsized":"2023-10-06T03:02:28.582259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:40.995528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594828+0000","last_change":"2023-10-06T03:02:28.594828+0000","last_active":"2023-10-06T03:02:28.594828+0000","last_peered":"2023-10-06T03:02:28.594828+0000","last_clean":"2023-10-06T03:02:28.594828+0000","last_became_active":"2023-10-06T03:02:28.594564+0000","last_became_peered":"2023-10-06T03:02:28.594564+0000","last_unstale":"2023-10-06T03:02:28.594828+0000","last_undegraded":"2023-10-06T03:02:28.594828+0000","last_fullsized":"2023-10-06T03:02:28.594828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:46.265185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.594415+0000","last_change":"2023-10-06T03:02:27.591268+0000","last_active":"2023-10-06T03:02:27.594415+0000","last_peered":"2023-10-06T03:02:27.594415+0000","last_clean":"2023-10-06T03:02:27.594415+0000","last_became_active":"2023-10-06T03:02:27.590327+0000","last_became_peered":"2023-10-06T03:02:27.590327+0000","last_unstale":"2023-10-06T03:02:27.594415+0000","last_undegraded":"2023-10-06T03:02:27.594415+0000","last_fullsized":"2023-10-06T03:02:27.594415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:40.497752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.606514+0000","last_change":"2023-10-06T03:02:28.606514+0000","last_active":"2023-10-06T03:02:28.606514+0000","last_peered":"2023-10-06T03:02:28.606514+0000","last_clean":"2023-10-06T03:02:28.606514+0000","last_became_active":"2023-10-06T03:02:28.606223+0000","last_became_peered":"2023-10-06T03:02:28.606223+0000","last_unstale":"2023-10-06T03:02:28.606514+0000","last_undegraded":"2023-10-06T03:02:28.606514+0000","last_fullsized":"2023-10-06T03:02:28.606514+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:33.260525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.609898+0000","last_change":"2023-10-06T03:02:27.592808+0000","last_active":"2023-10-06T03:02:28.609898+0000","last_peered":"2023-10-06T03:02:28.609898+0000","last_clean":"2023-10-06T03:02:28.609898+0000","last_became_active":"2023-10-06T03:02:27.592554+0000","last_became_peered":"2023-10-06T03:02:27.592554+0000","last_unstale":"2023-10-06T03:02:28.609898+0000","last_undegraded":"2023-10-06T03:02:28.609898+0000","last_fullsized":"2023-10-06T03:02:28.609898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.926792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597803+0000","last_change":"2023-10-06T03:02:28.597803+0000","last_active":"2023-10-06T03:02:28.597803+0000","last_peered":"2023-10-06T03:02:28.597803+0000","last_clean":"2023-10-06T03:02:28.597803+0000","last_became_active":"2023-10-06T03:02:28.597456+0000","last_became_peered":"2023-10-06T03:02:28.597456+0000","last_unstale":"2023-10-06T03:02:28.597803+0000","last_undegraded":"2023-10-06T03:02:28.597803+0000","last_fullsized":"2023-10-06T03:02:28.597803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.130409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.589924+0000","last_change":"2023-10-06T03:02:27.588373+0000","last_active":"2023-10-06T03:02:27.589924+0000","last_peered":"2023-10-06T03:02:27.589924+0000","last_clean":"2023-10-06T03:02:27.589924+0000","last_became_active":"2023-10-06T03:02:27.588224+0000","last_became_peered":"2023-10-06T03:02:27.588224+0000","last_unstale":"2023-10-06T03:02:27.589924+0000","last_undegraded":"2023-10-06T03:02:27.589924+0000","last_fullsized":"2023-10-06T03:02:27.589924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:30.283231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.602806+0000","last_change":"2023-10-06T03:02:28.602806+0000","last_active":"2023-10-06T03:02:28.602806+0000","last_peered":"2023-10-06T03:02:28.602806+0000","last_clean":"2023-10-06T03:02:28.602806+0000","last_became_active":"2023-10-06T03:02:28.602552+0000","last_became_peered":"2023-10-06T03:02:28.602552+0000","last_unstale":"2023-10-06T03:02:28.602806+0000","last_undegraded":"2023-10-06T03:02:28.602806+0000","last_fullsized":"2023-10-06T03:02:28.602806+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:33:47.335522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597462+0000","last_change":"2023-10-06T03:02:27.591604+0000","last_active":"2023-10-06T03:02:27.597462+0000","last_peered":"2023-10-06T03:02:27.597462+0000","last_clean":"2023-10-06T03:02:27.597462+0000","last_became_active":"2023-10-06T03:02:27.591294+0000","last_became_peered":"2023-10-06T03:02:27.591294+0000","last_unstale":"2023-10-06T03:02:27.597462+0000","last_undegraded":"2023-10-06T03:02:27.597462+0000","last_fullsized":"2023-10-06T03:02:27.597462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:45.260431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597147+0000","last_change":"2023-10-06T03:02:28.597147+0000","last_active":"2023-10-06T03:02:28.597147+0000","last_peered":"2023-10-06T03:02:28.597147+0000","last_clean":"2023-10-06T03:02:28.597147+0000","last_became_active":"2023-10-06T03:02:28.596892+0000","last_became_peered":"2023-10-06T03:02:28.596892+0000","last_unstale":"2023-10-06T03:02:28.597147+0000","last_undegraded":"2023-10-06T03:02:28.597147+0000","last_fullsized":"2023-10-06T03:02:28.597147+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:15.928007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595137+0000","last_change":"2023-10-06T03:02:27.590785+0000","last_active":"2023-10-06T03:02:27.595137+0000","last_peered":"2023-10-06T03:02:27.595137+0000","last_clean":"2023-10-06T03:02:27.595137+0000","last_became_active":"2023-10-06T03:02:27.590532+0000","last_became_peered":"2023-10-06T03:02:27.590532+0000","last_unstale":"2023-10-06T03:02:27.595137+0000","last_undegraded":"2023-10-06T03:02:27.595137+0000","last_fullsized":"2023-10-06T03:02:27.595137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:25.006902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594742+0000","last_change":"2023-10-06T03:02:28.594742+0000","last_active":"2023-10-06T03:02:28.594742+0000","last_peered":"2023-10-06T03:02:28.594742+0000","last_clean":"2023-10-06T03:02:28.594742+0000","last_became_active":"2023-10-06T03:02:28.594488+0000","last_became_peered":"2023-10-06T03:02:28.594488+0000","last_unstale":"2023-10-06T03:02:28.594742+0000","last_undegraded":"2023-10-06T03:02:28.594742+0000","last_fullsized":"2023-10-06T03:02:28.594742+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:33.086786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595397+0000","last_change":"2023-10-06T03:02:27.591843+0000","last_active":"2023-10-06T03:02:27.595397+0000","last_peered":"2023-10-06T03:02:27.595397+0000","last_clean":"2023-10-06T03:02:27.595397+0000","last_became_active":"2023-10-06T03:02:27.591574+0000","last_became_peered":"2023-10-06T03:02:27.591574+0000","last_unstale":"2023-10-06T03:02:27.595397+0000","last_undegraded":"2023-10-06T03:02:27.595397+0000","last_fullsized":"2023-10-06T03:02:27.595397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:53.669986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.584001+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:02:27.584001+0000","last_peered":"2023-10-06T03:02:27.584001+0000","last_clean":"2023-10-06T03:02:27.584001+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:02:27.584001+0000","last_undegraded":"2023-10-06T03:02:27.584001+0000","last_fullsized":"2023-10-06T03:02:27.584001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.591639+0000","last_change":"2023-10-06T03:02:28.591639+0000","last_active":"2023-10-06T03:02:28.591639+0000","last_peered":"2023-10-06T03:02:28.591639+0000","last_clean":"2023-10-06T03:02:28.591639+0000","last_became_active":"2023-10-06T03:02:28.591164+0000","last_became_peered":"2023-10-06T03:02:28.591164+0000","last_unstale":"2023-10-06T03:02:28.591639+0000","last_undegraded":"2023-10-06T03:02:28.591639+0000","last_fullsized":"2023-10-06T03:02:28.591639+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:44.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595503+0000","last_change":"2023-10-06T03:02:27.593767+0000","last_active":"2023-10-06T03:02:27.595503+0000","last_peered":"2023-10-06T03:02:27.595503+0000","last_clean":"2023-10-06T03:02:27.595503+0000","last_became_active":"2023-10-06T03:02:27.593470+0000","last_became_peered":"2023-10-06T03:02:27.593470+0000","last_unstale":"2023-10-06T03:02:27.595503+0000","last_undegraded":"2023-10-06T03:02:27.595503+0000","last_fullsized":"2023-10-06T03:02:27.595503+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:01.040304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.577457+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:02:27.577457+0000","last_peered":"2023-10-06T03:02:27.577457+0000","last_clean":"2023-10-06T03:02:27.577457+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:02:27.577457+0000","last_undegraded":"2023-10-06T03:02:27.577457+0000","last_fullsized":"2023-10-06T03:02:27.577457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.603062+0000","last_change":"2023-10-06T03:02:28.603062+0000","last_active":"2023-10-06T03:02:28.603062+0000","last_peered":"2023-10-06T03:02:28.603062+0000","last_clean":"2023-10-06T03:02:28.603062+0000","last_became_active":"2023-10-06T03:02:28.602213+0000","last_became_peered":"2023-10-06T03:02:28.602213+0000","last_unstale":"2023-10-06T03:02:28.603062+0000","last_undegraded":"2023-10-06T03:02:28.603062+0000","last_fullsized":"2023-10-06T03:02:28.603062+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:23.171200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.595567+0000","last_change":"2023-10-06T03:02:27.594470+0000","last_active":"2023-10-06T03:02:27.595567+0000","last_peered":"2023-10-06T03:02:27.595567+0000","last_clean":"2023-10-06T03:02:27.595567+0000","last_became_active":"2023-10-06T03:02:27.594209+0000","last_became_peered":"2023-10-06T03:02:27.594209+0000","last_unstale":"2023-10-06T03:02:27.595567+0000","last_undegraded":"2023-10-06T03:02:27.595567+0000","last_fullsized":"2023-10-06T03:02:27.595567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.587055+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:02:27.587055+0000","last_peered":"2023-10-06T03:02:27.587055+0000","last_clean":"2023-10-06T03:02:27.587055+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:02:27.587055+0000","last_undegraded":"2023-10-06T03:02:27.587055+0000","last_fullsized":"2023-10-06T03:02:27.587055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594979+0000","last_change":"2023-10-06T03:02:28.594979+0000","last_active":"2023-10-06T03:02:28.594979+0000","last_peered":"2023-10-06T03:02:28.594979+0000","last_clean":"2023-10-06T03:02:28.594979+0000","last_became_active":"2023-10-06T03:02:28.594719+0000","last_became_peered":"2023-10-06T03:02:28.594719+0000","last_unstale":"2023-10-06T03:02:28.594979+0000","last_undegraded":"2023-10-06T03:02:28.594979+0000","last_fullsized":"2023-10-06T03:02:28.594979+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:14.295301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597515+0000","last_change":"2023-10-06T03:02:27.592728+0000","last_active":"2023-10-06T03:02:27.597515+0000","last_peered":"2023-10-06T03:02:27.597515+0000","last_clean":"2023-10-06T03:02:27.597515+0000","last_became_active":"2023-10-06T03:02:27.592217+0000","last_became_peered":"2023-10-06T03:02:27.592217+0000","last_unstale":"2023-10-06T03:02:27.597515+0000","last_undegraded":"2023-10-06T03:02:27.597515+0000","last_fullsized":"2023-10-06T03:02:27.597515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:24.623203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.584442+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:02:27.584442+0000","last_peered":"2023-10-06T03:02:27.584442+0000","last_clean":"2023-10-06T03:02:27.584442+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:02:27.584442+0000","last_undegraded":"2023-10-06T03:02:27.584442+0000","last_fullsized":"2023-10-06T03:02:27.584442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":174,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.585281+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:02:27.585281+0000","last_peered":"2023-10-06T03:02:27.585281+0000","last_clean":"2023-10-06T03:02:27.585281+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:02:27.585281+0000","last_undegraded":"2023-10-06T03:02:27.585281+0000","last_fullsized":"2023-10-06T03:02:27.585281+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.597033+0000","last_change":"2023-10-06T03:02:28.597033+0000","last_active":"2023-10-06T03:02:28.597033+0000","last_peered":"2023-10-06T03:02:28.597033+0000","last_clean":"2023-10-06T03:02:28.597033+0000","last_became_active":"2023-10-06T03:02:28.596795+0000","last_became_peered":"2023-10-06T03:02:28.596795+0000","last_unstale":"2023-10-06T03:02:28.597033+0000","last_undegraded":"2023-10-06T03:02:28.597033+0000","last_fullsized":"2023-10-06T03:02:28.597033+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:33.260525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.606938+0000","last_change":"2023-10-06T03:02:27.590423+0000","last_active":"2023-10-06T03:02:28.606938+0000","last_peered":"2023-10-06T03:02:28.606938+0000","last_clean":"2023-10-06T03:02:28.606938+0000","last_became_active":"2023-10-06T03:02:27.590019+0000","last_became_peered":"2023-10-06T03:02:27.590019+0000","last_unstale":"2023-10-06T03:02:28.606938+0000","last_undegraded":"2023-10-06T03:02:28.606938+0000","last_fullsized":"2023-10-06T03:02:28.606938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:53.669986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.587678+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:02:27.587678+0000","last_peered":"2023-10-06T03:02:27.587678+0000","last_clean":"2023-10-06T03:02:27.587678+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:02:27.587678+0000","last_undegraded":"2023-10-06T03:02:27.587678+0000","last_fullsized":"2023-10-06T03:02:27.587678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.598646+0000","last_change":"2023-10-06T03:02:28.598646+0000","last_active":"2023-10-06T03:02:28.598646+0000","last_peered":"2023-10-06T03:02:28.598646+0000","last_clean":"2023-10-06T03:02:28.598646+0000","last_became_active":"2023-10-06T03:02:28.598385+0000","last_became_peered":"2023-10-06T03:02:28.598385+0000","last_unstale":"2023-10-06T03:02:28.598646+0000","last_undegraded":"2023-10-06T03:02:28.598646+0000","last_fullsized":"2023-10-06T03:02:28.598646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:39.357965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.597304+0000","last_change":"2023-10-06T03:02:27.591146+0000","last_active":"2023-10-06T03:02:27.597304+0000","last_peered":"2023-10-06T03:02:27.597304+0000","last_clean":"2023-10-06T03:02:27.597304+0000","last_became_active":"2023-10-06T03:02:27.590890+0000","last_became_peered":"2023-10-06T03:02:27.590890+0000","last_unstale":"2023-10-06T03:02:27.597304+0000","last_undegraded":"2023-10-06T03:02:27.597304+0000","last_fullsized":"2023-10-06T03:02:27.597304+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.024669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.585884+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:02:27.585884+0000","last_peered":"2023-10-06T03:02:27.585884+0000","last_clean":"2023-10-06T03:02:27.585884+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:02:27.585884+0000","last_undegraded":"2023-10-06T03:02:27.585884+0000","last_fullsized":"2023-10-06T03:02:27.585884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.594697+0000","last_change":"2023-10-06T03:02:28.594697+0000","last_active":"2023-10-06T03:02:28.594697+0000","last_peered":"2023-10-06T03:02:28.594697+0000","last_clean":"2023-10-06T03:02:28.594697+0000","last_became_active":"2023-10-06T03:02:28.594453+0000","last_became_peered":"2023-10-06T03:02:28.594453+0000","last_unstale":"2023-10-06T03:02:28.594697+0000","last_undegraded":"2023-10-06T03:02:28.594697+0000","last_fullsized":"2023-10-06T03:02:28.594697+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:28.624464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.585427+0000","last_change":"2023-10-06T03:02:27.579590+0000","last_active":"2023-10-06T03:02:27.585427+0000","last_peered":"2023-10-06T03:02:27.585427+0000","last_clean":"2023-10-06T03:02:27.585427+0000","last_became_active":"2023-10-06T03:02:27.579305+0000","last_became_peered":"2023-10-06T03:02:27.579305+0000","last_unstale":"2023-10-06T03:02:27.585427+0000","last_undegraded":"2023-10-06T03:02:27.585427+0000","last_fullsized":"2023-10-06T03:02:27.585427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:14.846255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.589201+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:02:27.589201+0000","last_peered":"2023-10-06T03:02:27.589201+0000","last_clean":"2023-10-06T03:02:27.589201+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:02:27.589201+0000","last_undegraded":"2023-10-06T03:02:27.589201+0000","last_fullsized":"2023-10-06T03:02:27.589201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T03:02:28.598694+0000","last_change":"2023-10-06T03:02:28.598694+0000","last_active":"2023-10-06T03:02:28.598694+0000","last_peered":"2023-10-06T03:02:28.598694+0000","last_clean":"2023-10-06T03:02:28.598694+0000","last_became_active":"2023-10-06T03:02:28.598387+0000","last_became_peered":"2023-10-06T03:02:28.598387+0000","last_unstale":"2023-10-06T03:02:28.598694+0000","last_undegraded":"2023-10-06T03:02:28.598694+0000","last_fullsized":"2023-10-06T03:02:28.598694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:27.567364+0000","last_clean_scrub_stamp":"2023-10-06T03:02:27.567364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:25.619653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.594876+0000","last_change":"2023-10-06T03:02:27.590715+0000","last_active":"2023-10-06T03:02:27.594876+0000","last_peered":"2023-10-06T03:02:27.594876+0000","last_clean":"2023-10-06T03:02:27.594876+0000","last_became_active":"2023-10-06T03:02:27.590436+0000","last_became_peered":"2023-10-06T03:02:27.590436+0000","last_unstale":"2023-10-06T03:02:27.594876+0000","last_undegraded":"2023-10-06T03:02:27.594876+0000","last_fullsized":"2023-10-06T03:02:27.594876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:02:26.562610+0000","last_clean_scrub_stamp":"2023-10-06T03:02:26.562610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:28.160477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T03:02:27.588609+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:02:27.588609+0000","last_peered":"2023-10-06T03:02:27.588609+0000","last_clean":"2023-10-06T03:02:27.588609+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:02:27.588609+0000","last_undegraded":"2023-10-06T03:02:27.588609+0000","last_fullsized":"2023-10-06T03:02:27.588609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":983040,"data_stored":741603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":142513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":142513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":740031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":151199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":142513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.308},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.36799999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.51000000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.435},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.498}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":149741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:02:33.014 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:02:33.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:02:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.168+0000 7f4f72b74700 1 Processor -- start 2023-10-06T03:02:33.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.168+0000 7f4f72b74700 1 -- start start 2023-10-06T03:02:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f72b74700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f72b74700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f6c069c70 0x7f4f6c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f72b74700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c0ce380 con 0x7f4f6c0af320 2023-10-06T03:02:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f72b74700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c0cec20 con 0x7f4f6c069c70 2023-10-06T03:02:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f72b74700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0afa90 con 0x7f4f6c159950 2023-10-06T03:02:33.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f6c069c70 0x7f4f6c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f6c069c70 0x7f4f6c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34256/0 (socket says 172.21.15.92:34256) 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 -- 172.21.15.92:0/4254692242 learned_addr learned my addr 172.21.15.92:0/4254692242 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f70910700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 -- 172.21.15.92:0/4254692242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 msgr2=0x7f4f6c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 -- 172.21.15.92:0/4254692242 >> v1:172.21.15.136:6789/0 conn(0x7f4f6c0af320 legacy=0x7f4f6c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 -- 172.21.15.92:0/4254692242 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0ce5d0 con 0x7f4f6c069c70 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f70910700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.169+0000 7f4f71111700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c069c70 0x7f4f6c0cf0b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0099b0 tx=0x7f4f5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941a37b73a196501 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/4254692242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00b820 con 0x7f4f6c069c70 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/4254692242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c00b980 con 0x7f4f6c069c70 2023-10-06T03:02:33.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f6b7fe700 1 -- 172.21.15.92:0/4254692242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00bd20 con 0x7f4f6c069c70 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 -- 172.21.15.92:0/4254692242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c069c70 msgr2=0x7f4f6c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c069c70 0x7f4f6c0cf0b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c0099b0 tx=0x7f4f5c009250 comp rx=0 tx=0).stop 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 -- 172.21.15.92:0/4254692242 shutdown_connections 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c069c70 0x7f4f6c0cf0b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 --2- 172.21.15.92:0/4254692242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 -- 172.21.15.92:0/4254692242 >> 172.21.15.92:0/4254692242 conn(0x7f4f6c068e00 msgr2=0x7f4f6c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 -- 172.21.15.92:0/4254692242 shutdown_connections 2023-10-06T03:02:33.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.170+0000 7f4f72b74700 1 -- 172.21.15.92:0/4254692242 wait complete. 2023-10-06T03:02:33.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.171+0000 7f4f72b74700 1 Processor -- start 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.171+0000 7f4f72b74700 1 -- start start 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f72b74700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f6c069c70 0x7f4f6c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f72b74700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 0x7f4f6c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f72b74700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f72b74700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1551e0 con 0x7f4f6c0af320 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f72b74700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0c9770 con 0x7f4f6c069c70 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f72b74700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0b5e90 con 0x7f4f6c159950 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 0x7f4f6c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 0x7f4f6c151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34266/0 (socket says 172.21.15.92:34266) 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 -- 172.21.15.92:0/2196958347 learned_addr learned my addr 172.21.15.92:0/2196958347 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:33.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 -- 172.21.15.92:0/2196958347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 msgr2=0x7f4f6c0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 -- 172.21.15.92:0/2196958347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f6c069c70 msgr2=0x7f4f6c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.172+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f6c069c70 0x7f4f6c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f6bfff700 1 -- 172.21.15.92:0/2196958347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0094f0 con 0x7f4f6c0af320 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f6bfff700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 0x7f4f6c151020 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c5d0 tx=0x7f4f60007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60011590 con 0x7f4f6c0af320 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f60011710 con 0x7f4f6c0af320 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c0b6150 con 0x7f4f6c0af320 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60011ad0 con 0x7f4f6c0af320 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.173+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c0b38d0 con 0x7f4f6c0af320 2023-10-06T03:02:33.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.174+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c0718b0 con 0x7f4f6c0af320 2023-10-06T03:02:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.174+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f6001c460 con 0x7f4f6c0af320 2023-10-06T03:02:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.175+0000 7f4f68ff9700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f58066f30 0x7f4f580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.175+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4f60084490 con 0x7f4f6c0af320 2023-10-06T03:02:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.175+0000 7f4f70910700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f58066f30 0x7f4f580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.175+0000 7f4f70910700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f58066f30 0x7f4f580693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f64007f40 tx=0x7f4f64007430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:33.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.179+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f600515a0 con 0x7f4f6c0af320 2023-10-06T03:02:33.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.439+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4f6c0ce8b0 con 0x7f4f6c0af320 2023-10-06T03:02:33.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.443+0000 7f4f68ff9700 1 -- 172.21.15.92:0/2196958347 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4f60054bc0 con 0x7f4f6c0af320 2023-10-06T03:02:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.445+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f58066f30 msgr2=0x7f4f580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f58066f30 0x7f4f580693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f64007f40 tx=0x7f4f64007430 comp rx=0 tx=0).stop 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 msgr2=0x7f4f6c151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 0x7f4f6c151020 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000c5d0 tx=0x7f4f60007f60 comp rx=0 tx=0).stop 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 shutdown_connections 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f58066f30 0x7f4f580693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f6c159950 0x7f4f6c0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f6c069c70 0x7f4f6c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 --2- 172.21.15.92:0/2196958347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f6c0af320 0x7f4f6c151020 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 >> 172.21.15.92:0/2196958347 conn(0x7f4f6c068e00 msgr2=0x7f4f6c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 shutdown_connections 2023-10-06T03:02:33.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.446+0000 7f4f72b74700 1 -- 172.21.15.92:0/2196958347 wait complete. 2023-10-06T03:02:33.449 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:02:33.459 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:33.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.571+0000 7f858f1c8700 1 Processor -- start 2023-10-06T03:02:33.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858f1c8700 1 -- start start 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858f1c8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8588159890 0x7f858815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858f1c8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8588069560 0x7f85880699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858f1c8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8588069ee0 con 0x7f8588159c60 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858f1c8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858815a160 con 0x7f8588159890 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858f1c8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880d13a0 con 0x7f8588069560 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f858cf64700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8588159890 0x7f858815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.572+0000 7f857ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8588159c60 0x7f858815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36118/0 (socket says 172.21.15.92:36118) 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f857ffff700 1 -- 172.21.15.92:0/2744671987 learned_addr learned my addr 172.21.15.92:0/2744671987 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:33.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f858d765700 1 --2- 172.21.15.92:0/2744671987 >> v2:172.21.15.136:3300/0 conn(0x7f8588069560 0x7f85880699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f858cf64700 1 -- 172.21.15.92:0/2744671987 >> v2:172.21.15.136:3300/0 conn(0x7f8588069560 msgr2=0x7f85880699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f858cf64700 1 --2- 172.21.15.92:0/2744671987 >> v2:172.21.15.136:3300/0 conn(0x7f8588069560 0x7f85880699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f858cf64700 1 -- 172.21.15.92:0/2744671987 >> v1:172.21.15.92:6789/0 conn(0x7f8588159c60 legacy=0x7f858815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f858cf64700 1 -- 172.21.15.92:0/2744671987 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858815e140 con 0x7f8588159890 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f857f7fe700 1 -- 172.21.15.92:0/2744671987 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913758390 0 0) 0x7f8588069ee0 con 0x7f8588159c60 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f858cf64700 1 --2- 172.21.15.92:0/2744671987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588159890 0x7f858815abd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f858806a890 tx=0x7f8584009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920222ce4c701c81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f857f7fe700 1 -- 172.21.15.92:0/2744671987 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858400e070 con 0x7f8588159890 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f857f7fe700 1 -- 172.21.15.92:0/2744671987 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858400bc30 con 0x7f8588159890 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.573+0000 7f857f7fe700 1 -- 172.21.15.92:0/2744671987 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584012430 con 0x7f8588159890 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 -- 172.21.15.92:0/2744671987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588159890 msgr2=0x7f858815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 --2- 172.21.15.92:0/2744671987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588159890 0x7f858815abd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f858806a890 tx=0x7f8584009780 comp rx=0 tx=0).stop 2023-10-06T03:02:33.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 -- 172.21.15.92:0/2744671987 shutdown_connections 2023-10-06T03:02:33.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 --2- 172.21.15.92:0/2744671987 >> v2:172.21.15.136:3300/0 conn(0x7f8588069560 0x7f85880699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 --2- 172.21.15.92:0/2744671987 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588159890 0x7f858815abd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 -- 172.21.15.92:0/2744671987 >> 172.21.15.92:0/2744671987 conn(0x7f8588068e20 msgr2=0x7f85880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:33.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 -- 172.21.15.92:0/2744671987 shutdown_connections 2023-10-06T03:02:33.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.574+0000 7f858f1c8700 1 -- 172.21.15.92:0/2744671987 wait complete. 2023-10-06T03:02:33.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.575+0000 7f858f1c8700 1 Processor -- start 2023-10-06T03:02:33.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.575+0000 7f858f1c8700 1 -- start start 2023-10-06T03:02:33.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 0x7f8588156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8588159890 0x7f85880b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8588159c60 0x7f8588153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b3fd0 con 0x7f8588069560 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b36a0 con 0x7f8588159890 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b3980 con 0x7f8588159c60 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 0x7f8588156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 0x7f8588156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34284/0 (socket says 172.21.15.92:34284) 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 -- 172.21.15.92:0/3843998068 learned_addr learned my addr 172.21.15.92:0/3843998068 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f857ffff700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8588159890 0x7f85880b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 -- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8588159c60 msgr2=0x7f8588153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8588159c60 0x7f8588153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 -- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8588159890 msgr2=0x7f85880b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8588159890 0x7f85880b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 -- 172.21.15.92:0/3843998068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584009a20 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858cf64700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 0x7f8588156430 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85840099f0 tx=0x7f8584019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584027470 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584027610 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85880b3c60 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85840279d0 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.576+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8588156970 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.577+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858804c3c0 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.578+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8584024070 con 0x7f8588069560 2023-10-06T03:02:33.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.578+0000 7f857cff9700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8574066f30 0x7f85740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:33.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.578+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8584086d40 con 0x7f8588069560 2023-10-06T03:02:33.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.578+0000 7f857ffff700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8574066f30 0x7f85740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:33.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.579+0000 7f857ffff700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8574066f30 0x7f85740693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8578001750 tx=0x7f8578009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:33.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.583+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8584053e50 con 0x7f8588069560 2023-10-06T03:02:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.813+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f858806a5b0 con 0x7f8588069560 2023-10-06T03:02:33.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.813+0000 7f857cff9700 1 -- 172.21.15.92:0/3843998068 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f858402c020 con 0x7f8588069560 2023-10-06T03:02:33.815 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:33.815 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4965,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/971383855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":971383855},{"type":"v1","addr":"172.21.15.92:6836","nonce":971383855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:27.963102+0000","modified":"2023-10-06T03:02:31.599832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2171492021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2171492021},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171492021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:02:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8574066f30 msgr2=0x7f85740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8574066f30 0x7f85740693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8578001750 tx=0x7f8578009040 comp rx=0 tx=0).stop 2023-10-06T03:02:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 msgr2=0x7f8588156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:33.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 0x7f8588156430 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f85840099f0 tx=0x7f8584019000 comp rx=0 tx=0).stop 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 shutdown_connections 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8574066f30 0x7f85740693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8588069560 0x7f8588156430 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8588159c60 0x7f8588153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 --2- 172.21.15.92:0/3843998068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8588159890 0x7f85880b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 >> 172.21.15.92:0/3843998068 conn(0x7f8588068e20 msgr2=0x7f8588155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 shutdown_connections 2023-10-06T03:02:33.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:33.815+0000 7f858f1c8700 1 -- 172.21.15.92:0/3843998068 wait complete. 2023-10-06T03:02:33.819 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 16 2023-10-06T03:02:34.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:02:27.963102+0000', 'modified': '2023-10-06T03:02:31.599832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4968}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4968': {'gid': 4968, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2171492021', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 2171492021}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2171492021}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:02:34.231 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:02:34.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:02:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.347+0000 7fec69238700 1 Processor -- start 2023-10-06T03:02:34.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.347+0000 7fec69238700 1 -- start start 2023-10-06T03:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec69238700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fec64068920 0x7fec6406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec69238700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6406ec70 con 0x7fec6406f640 2023-10-06T03:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec69238700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec640b2e70 con 0x7fec64068550 2023-10-06T03:02:34.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec69238700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64068d90 con 0x7fec64068920 2023-10-06T03:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec6359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fec6406f640 0x7fec64159510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36134/0 (socket says 172.21.15.92:36134) 2023-10-06T03:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec6359e700 1 -- 172.21.15.92:0/555070869 learned_addr learned my addr 172.21.15.92:0/555070869 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118245281 0 0) 0x7fec6406ec70 con 0x7fec6406f640 2023-10-06T03:02:34.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec6259c700 1 --2- 172.21.15.92:0/555070869 >> v2:172.21.15.136:3300/0 conn(0x7fec64068920 0x7fec6406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.348+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec4c003570 con 0x7fec6406f640 2023-10-06T03:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545622851 0 0) 0x7fec4c003570 con 0x7fec6406f640 2023-10-06T03:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 >> v2:172.21.15.136:3300/0 conn(0x7fec64068920 msgr2=0x7fec6406f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 --2- 172.21.15.92:0/555070869 >> v2:172.21.15.136:3300/0 conn(0x7fec64068920 0x7fec6406f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 >> v1:172.21.15.136:6789/0 conn(0x7fec64068550 legacy=0x7fec6415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640b3150 con 0x7fec6406f640 2023-10-06T03:02:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fec5c0017f0 con 0x7fec6406f640 2023-10-06T03:02:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 >> v1:172.21.15.92:6789/0 conn(0x7fec6406f640 legacy=0x7fec64159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec4c003a60 0x7fec4c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec4c0051c0 0x7fec4c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64068d90 con 0x7fec4c004810 2023-10-06T03:02:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c00c160 con 0x7fec4c003a60 2023-10-06T03:02:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c00c320 con 0x7fec4c0051c0 2023-10-06T03:02:34.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec62d9d700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec4c003a60 0x7fec4c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6359e700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec4c0051c0 0x7fec4c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 -- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec4c0051c0 msgr2=0x7fec4c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec4c0051c0 0x7fec4c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 -- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec4c003a60 msgr2=0x7fec4c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec4c003a60 0x7fec4c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6359e700 1 -- 172.21.15.92:0/555070869 reap_dead start 2023-10-06T03:02:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 -- 172.21.15.92:0/555070869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c003570 con 0x7fec4c004810 2023-10-06T03:02:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec62d9d700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec4c003a60 0x7fec4c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:34.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec6259c700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec4c004c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fec58009c00 tx=0x7fec580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d8ed244a0fa504 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800e710 con 0x7fec4c004810 2023-10-06T03:02:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5800e870 con 0x7fec4c004810 2023-10-06T03:02:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.349+0000 7fec61d9b700 1 -- 172.21.15.92:0/555070869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800ec30 con 0x7fec4c004810 2023-10-06T03:02:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 -- 172.21.15.92:0/555070869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 msgr2=0x7fec4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec4c004c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fec58009c00 tx=0x7fec580091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 -- 172.21.15.92:0/555070869 shutdown_connections 2023-10-06T03:02:34.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 --2- 172.21.15.92:0/555070869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec4c004c80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 -- 172.21.15.92:0/555070869 >> 172.21.15.92:0/555070869 conn(0x7fec640cd480 msgr2=0x7fec640cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 -- 172.21.15.92:0/555070869 shutdown_connections 2023-10-06T03:02:34.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.350+0000 7fec69238700 1 -- 172.21.15.92:0/555070869 wait complete. 2023-10-06T03:02:34.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.351+0000 7fec69238700 1 Processor -- start 2023-10-06T03:02:34.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.351+0000 7fec69238700 1 -- start start 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec69238700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec64140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec69238700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec640aa7e0 0x7fec640aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec69238700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec640ab190 0x7fec6416d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec69238700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64141490 con 0x7fec4c004810 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec69238700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec641598a0 con 0x7fec640aa7e0 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec69238700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec640a9e50 con 0x7fec640ab190 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec64140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec64140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34306/0 (socket says 172.21.15.92:34306) 2023-10-06T03:02:34.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 -- 172.21.15.92:0/2696743101 learned_addr learned my addr 172.21.15.92:0/2696743101 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec6259c700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec640aa7e0 0x7fec640aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec6359e700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec640ab190 0x7fec6416d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 -- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec640ab190 msgr2=0x7fec6416d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec640ab190 0x7fec6416d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 -- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec640aa7e0 msgr2=0x7fec640aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec640aa7e0 0x7fec640aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 -- 172.21.15.92:0/2696743101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec640ab600 con 0x7fec4c004810 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.352+0000 7fec62d9d700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec64140f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec54000c00 tx=0x7fec54009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.353+0000 7fec4affd700 1 -- 172.21.15.92:0/2696743101 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400eea0 con 0x7fec4c004810 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.353+0000 7fec4affd700 1 -- 172.21.15.92:0/2696743101 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec540114f0 con 0x7fec4c004810 2023-10-06T03:02:34.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.353+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec640acba0 con 0x7fec4c004810 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.353+0000 7fec4affd700 1 -- 172.21.15.92:0/2696743101 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54011890 con 0x7fec4c004810 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.353+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641595c0 con 0x7fec4c004810 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.354+0000 7fec4affd700 1 -- 172.21.15.92:0/2696743101 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec54011c00 con 0x7fec4c004810 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.354+0000 7fec4affd700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec50066f30 0x7fec500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.355+0000 7fec4affd700 1 -- 172.21.15.92:0/2696743101 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec54083da0 con 0x7fec4c004810 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.355+0000 7fec6259c700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec50066f30 0x7fec500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.355+0000 7fec6259c700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec50066f30 0x7fec500693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec580091e0 tx=0x7fec5801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.356+0000 7fec72530b80 1 Processor -- start 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.356+0000 7fec72530b80 1 -- start start 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec72530b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 0x56052a78dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78ecc0 0x56052a7dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a7ddc00 0x56052a7de050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec72530b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a6e4a70 con 0x56052a78d840 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a6e2270 con 0x56052a78ecc0 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x56052a6de060 con 0x56052a7ddc00 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 0x56052a78dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec62d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 0x56052a78dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34314/0 (socket says 172.21.15.92:34314) 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec62d9d700 1 -- 172.21.15.92:0/4141521211 learned_addr learned my addr 172.21.15.92:0/4141521211 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec6259c700 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78ecc0 0x56052a7dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.357+0000 7fec6359e700 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a7ddc00 0x56052a7de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec62d9d700 1 -- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a7ddc00 msgr2=0x56052a7de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec62d9d700 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a7ddc00 0x56052a7de050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec62d9d700 1 -- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78ecc0 msgr2=0x56052a7dd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec62d9d700 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78ecc0 0x56052a7dd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec62d9d700 1 -- 172.21.15.92:0/4141521211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56052a578270 con 0x56052a78d840 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec6259c700 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78ecc0 0x56052a7dd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec62d9d700 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 0x56052a78dc10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec540546c0 tx=0x7fec540820c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f816778538fc7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec43fff700 1 -- 172.21.15.92:0/4141521211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec540549f0 con 0x56052a78d840 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec43fff700 1 -- 172.21.15.92:0/4141521211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54057dc0 con 0x56052a78d840 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec43fff700 1 -- 172.21.15.92:0/4141521211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5405bb10 con 0x56052a78d840 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec72530b80 1 -- 172.21.15.92:0/4141521211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 msgr2=0x56052a78dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.358+0000 7fec72530b80 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 0x56052a78dc10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec540546c0 tx=0x7fec540820c0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 -- 172.21.15.92:0/4141521211 shutdown_connections 2023-10-06T03:02:34.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78d840 0x56052a78dc10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a7ddc00 0x56052a7de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 --2- 172.21.15.92:0/4141521211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78ecc0 0x56052a7dd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 -- 172.21.15.92:0/4141521211 >> 172.21.15.92:0/4141521211 conn(0x56052a78f160 msgr2=0x56052a78f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 -- 172.21.15.92:0/4141521211 shutdown_connections 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 -- 172.21.15.92:0/4141521211 wait complete. 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 -- start start 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.359+0000 7fec72530b80 1 Processor -- start 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a7cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a689890 con 0x56052a7ddc00 2023-10-06T03:02:34.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a6a4ea0 con 0x56052a78d840 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x56052a560e30 con 0x56052a78ecc0 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec72530b80 10 client.0 _ll_register_callbacks cb 0x56052a750e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec6359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a7cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec6359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a7cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34316/0 (socket says 172.21.15.92:34316) 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec6359e700 1 -- 172.21.15.92:0/3002521537 learned_addr learned my addr 172.21.15.92:0/3002521537 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec62d9d700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.360+0000 7fec6259c700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec6359e700 1 -- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 msgr2=0x56052a7cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec6359e700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec6359e700 1 -- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 msgr2=0x56052a7ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec6359e700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec6359e700 1 -- 172.21.15.92:0/3002521537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641595c0 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec6359e700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a7cbf50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fec6415d590 tx=0x7fec5c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986a0af0d242e48c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec417fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec417fa700 1 -- 172.21.15.92:0/3002521537 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012c70 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec417fa700 1 -- 172.21.15.92:0/3002521537 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec5c012dd0 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56052a782940 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec417fa700 1 -- 172.21.15.92:0/3002521537 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec5c015650 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 10 client.5160 fetch_fsmap learned FSMap version 16 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56052a51afe0 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec417fa700 1 -- 172.21.15.92:0/3002521537 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56052a782940 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec417fa700 1 -- 172.21.15.92:0/3002521537 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fec5c0157b0 con 0x56052a7ddc00 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 10 client.5160 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:02:34.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 10 client.5160 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 20 client.5160 populate_metadata read hostname 'smithi092' 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7b99c0 0x56052a7b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 4 client.5160 mds_command: new command op to 4968 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.361+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 --> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56052a75f870 con 0x56052a7b99c0 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec62d9d700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7b99c0 0x56052a7b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec417fa700 10 client.5160 ms_handle_connect on v2:172.21.15.92:6835/2171492021 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec62d9d700 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7b99c0 0x56052a7b9d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fcf4197a0b1324 server_cookie=832a16ce73d89547 in_seq=0 out_seq=0 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec417fa700 1 -- 172.21.15.92:0/3002521537 <== mds.0 v2:172.21.15.92:6835/2171492021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56052a75f870 con 0x56052a7b99c0 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec417fa700 10 client.5160 handle_command_reply: tid=0 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec72530b80 1 client.5160 shutdown 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 msgr2=0x56052a7cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec72530b80 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a7cbf50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fec6415d590 tx=0x7fec5c009040 comp rx=0 tx=0).stop 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.362+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 shutdown_connections 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7b99c0 0x56052a7b9d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a7cbf50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec417fa700 0 client.5160 ms_handle_reset on v2:172.21.15.92:6835/2171492021 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 --2- 172.21.15.92:0/3002521537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 >> 172.21.15.92:0/3002521537 conn(0x56052a78f160 msgr2=0x56052a7b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 shutdown_connections 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 1 -- 172.21.15.92:0/3002521537 wait complete. 2023-10-06T03:02:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.363+0000 7fec72530b80 20 client.5160 trim_cache size 0 max 16384 2023-10-06T03:02:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.379+0000 7fec72530b80 1 Processor -- start 2023-10-06T03:02:34.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.379+0000 7fec72530b80 1 -- start start 2023-10-06T03:02:34.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec72530b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a790ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec72530b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a755e00 con 0x56052a7ddc00 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a76d2e0 con 0x56052a78d840 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x56052a78dd10 con 0x56052a78ecc0 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a790ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a790ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34330/0 (socket says 172.21.15.92:34330) 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 -- 172.21.15.92:0/60841545 learned_addr learned my addr 172.21.15.92:0/60841545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec62d9d700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6259c700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 -- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 msgr2=0x56052a7bd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7bd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 -- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 msgr2=0x56052a7d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.380+0000 7fec6359e700 1 -- 172.21.15.92:0/60841545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56052a6ac780 con 0x56052a7ddc00 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec62d9d700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec6259c700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec6359e700 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a790ca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec5c015fd0 tx=0x7fec5c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41471fe2c3e16887 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec48ff9700 1 -- 172.21.15.92:0/60841545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c001e60 con 0x56052a7ddc00 2023-10-06T03:02:34.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec48ff9700 1 -- 172.21.15.92:0/60841545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c015cc0 con 0x56052a7ddc00 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec48ff9700 1 -- 172.21.15.92:0/60841545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c001a50 con 0x56052a7ddc00 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 -- 172.21.15.92:0/60841545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 msgr2=0x56052a790ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a790ca0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec5c015fd0 tx=0x7fec5c01b000 comp rx=0 tx=0).stop 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 -- 172.21.15.92:0/60841545 shutdown_connections 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a7ddc00 0x56052a790ca0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a78d840 0x56052a7d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 --2- 172.21.15.92:0/60841545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78ecc0 0x56052a7bd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 -- 172.21.15.92:0/60841545 >> 172.21.15.92:0/60841545 conn(0x56052a78f160 msgr2=0x56052a7ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 -- 172.21.15.92:0/60841545 shutdown_connections 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.381+0000 7fec72530b80 1 -- 172.21.15.92:0/60841545 wait complete. 2023-10-06T03:02:34.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.382+0000 7fec72530b80 1 -- start start 2023-10-06T03:02:34.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.382+0000 7fec72530b80 1 Processor -- start 2023-10-06T03:02:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78d840 0x56052a839ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 0x56052a837170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a7ddc00 0x56052a7dc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a7d3980 con 0x56052a78ecc0 2023-10-06T03:02:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x56052a7cc3f0 con 0x56052a7ddc00 2023-10-06T03:02:34.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x56052a83a330 con 0x56052a78d840 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec72530b80 10 client.0 _ll_register_callbacks cb 0x56052a750e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 0x56052a837170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 0x56052a837170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34332/0 (socket says 172.21.15.92:34332) 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 -- 172.21.15.92:0/2984356799 learned_addr learned my addr 172.21.15.92:0/2984356799 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6359e700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a7ddc00 0x56052a7dc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec62d9d700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78d840 0x56052a839ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 -- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78d840 msgr2=0x56052a839ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78d840 0x56052a839ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 -- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a7ddc00 msgr2=0x56052a7dc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a7ddc00 0x56052a7dc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 -- 172.21.15.92:0/2984356799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56052a51afe0 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec6259c700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 0x56052a837170 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c130 tx=0x7fec5801f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed45f6f5e39d0aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.383+0000 7fec417fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec417fa700 1 -- 172.21.15.92:0/2984356799 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58013dc0 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec417fa700 1 -- 172.21.15.92:0/2984356799 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fec58037480 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec54057dc0 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec417fa700 1 -- 172.21.15.92:0/2984356799 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec58005a00 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 10 client.5166 fetch_fsmap learned FSMap version 16 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56052a693470 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec417fa700 1 -- 172.21.15.92:0/2984356799 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54057dc0 con 0x56052a78ecc0 2023-10-06T03:02:34.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec417fa700 1 -- 172.21.15.92:0/2984356799 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fec58005b60 con 0x56052a78ecc0 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 10 client.5166 fetch_fsmap finished waiting for FSMap version 16 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 10 client.5166 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 20 client.5166 populate_metadata read hostname 'smithi092' 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7ec930 0x56052a7ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 4 client.5166 mds_command: new command op to 4968 tid=0[{"prefix": "status"}] 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 --> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56052a680e00 con 0x56052a7ec930 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec62d9d700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7ec930 0x56052a7ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec417fa700 10 client.5166 ms_handle_connect on v2:172.21.15.92:6835/2171492021 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.384+0000 7fec62d9d700 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7ec930 0x56052a7ecd00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69452deed4b7e0e server_cookie=293c386abc477be1 in_seq=0 out_seq=0 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec417fa700 1 -- 172.21.15.92:0/2984356799 <== mds.0 v2:172.21.15.92:6835/2171492021 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x56052a680e00 con 0x56052a7ec930 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec417fa700 10 client.5166 handle_command_reply: tid=0 2023-10-06T03:02:34.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 client.5166 shutdown 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 msgr2=0x56052a837170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 0x56052a837170 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c130 tx=0x7fec5801f050 comp rx=0 tx=0).stop 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 shutdown_connections 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:6835/2171492021,v1:172.21.15.92:6837/2171492021] conn(0x56052a7ec930 0x56052a7ecd00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56052a7ddc00 0x56052a7dc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x56052a78d840 0x56052a839ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec417fa700 0 client.5166 ms_handle_reset on v2:172.21.15.92:6835/2171492021 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 --2- 172.21.15.92:0/2984356799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56052a78ecc0 0x56052a837170 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 >> 172.21.15.92:0/2984356799 conn(0x56052a78f160 msgr2=0x56052a7b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 shutdown_connections 2023-10-06T03:02:34.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 1 -- 172.21.15.92:0/2984356799 wait complete. 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec72530b80 20 client.5166 trim_cache size 0 max 16384 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 4968, 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:02:34.395 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.797981117, 2023-10-06T03:02:34.396 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 16, 2023-10-06T03:02:34.396 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 34, 2023-10-06T03:02:34.396 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 34, 2023-10-06T03:02:34.396 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 13.701361516 2023-10-06T03:02:34.396 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec50066f30 msgr2=0x7fec500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec50066f30 0x7fec500693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fec580091e0 tx=0x7fec5801a000 comp rx=0 tx=0).stop 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 msgr2=0x7fec64140f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec64140f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fec54000c00 tx=0x7fec54009040 comp rx=0 tx=0).stop 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 shutdown_connections 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fec50066f30 0x7fec500693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec4c004810 0x7fec64140f50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec640ab190 0x7fec6416d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 --2- 172.21.15.92:0/2696743101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec640aa7e0 0x7fec640aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.385+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 >> 172.21.15.92:0/2696743101 conn(0x7fec640cd480 msgr2=0x7fec64155bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.386+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 shutdown_connections 2023-10-06T03:02:34.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.386+0000 7fec69238700 1 -- 172.21.15.92:0/2696743101 wait complete. 2023-10-06T03:02:34.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:02:34.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.511+0000 7f8da48e3700 1 Processor -- start 2023-10-06T03:02:34.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.511+0000 7f8da48e3700 1 -- start start 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8da48e3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8da00aca80 0x7f8da006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8da48e3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8da48e3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da006bdb0 con 0x7f8da00c81f0 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8da48e3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00c8670 con 0x7f8da00aca80 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8da48e3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00cb370 con 0x7f8da0069d60 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8d95d9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8d9e59c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8da00aca80 0x7f8da006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8d9ed9d700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f8da00c81f0 0x7f8da0068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:42398/0 (socket says 172.21.15.92:42398) 2023-10-06T03:02:34.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.512+0000 7f8d9ed9d700 1 -- 172.21.15.92:0/862639698 learned_addr learned my addr 172.21.15.92:0/862639698 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9e59c700 1 -- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 msgr2=0x7f8da00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9e59c700 1 --2- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da00c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9e59c700 1 -- 172.21.15.92:0/862639698 >> v1:172.21.15.136:6789/0 conn(0x7f8da00c81f0 legacy=0x7f8da0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9e59c700 1 -- 172.21.15.92:0/862639698 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0159f60 con 0x7f8da00aca80 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d95d9b700 1 --2- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da00c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9e59c700 1 --2- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00aca80 0x7f8da006aba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098d0 tx=0x7f8d90009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe6dd227be85bb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9dd9b700 1 -- 172.21.15.92:0/862639698 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90005c10 con 0x7f8da00aca80 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9dd9b700 1 -- 172.21.15.92:0/862639698 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90005d70 con 0x7f8da00aca80 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8d9dd9b700 1 -- 172.21.15.92:0/862639698 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90006280 con 0x7f8da00aca80 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8da48e3700 1 -- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00aca80 msgr2=0x7f8da006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.513+0000 7f8da48e3700 1 --2- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00aca80 0x7f8da006aba0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098d0 tx=0x7f8d90009120 comp rx=0 tx=0).stop 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.514+0000 7f8da48e3700 1 -- 172.21.15.92:0/862639698 shutdown_connections 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.514+0000 7f8da48e3700 1 --2- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00aca80 0x7f8da006aba0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.514+0000 7f8da48e3700 1 --2- 172.21.15.92:0/862639698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da00c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.514+0000 7f8da48e3700 1 -- 172.21.15.92:0/862639698 >> 172.21.15.92:0/862639698 conn(0x7f8da00cd670 msgr2=0x7f8da00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.514+0000 7f8da48e3700 1 -- 172.21.15.92:0/862639698 shutdown_connections 2023-10-06T03:02:34.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.514+0000 7f8da48e3700 1 -- 172.21.15.92:0/862639698 wait complete. 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.515+0000 7f8da48e3700 1 Processor -- start 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.515+0000 7f8da48e3700 1 -- start start 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da016ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 0x7f8da01505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00c81f0 0x7f8da015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da015cff0 con 0x7f8da00aca80 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da015b250 con 0x7f8da00c81f0 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da015a250 con 0x7f8da0069d60 2023-10-06T03:02:34.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 0x7f8da01505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 0x7f8da01505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34338/0 (socket says 172.21.15.92:34338) 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 -- 172.21.15.92:0/2849252614 learned_addr learned my addr 172.21.15.92:0/2849252614 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d9e59c700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da016ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d9ed9d700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00c81f0 0x7f8da015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 -- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 msgr2=0x7f8da016ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da016ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 -- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00c81f0 msgr2=0x7f8da015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00c81f0 0x7f8da015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 -- 172.21.15.92:0/2849252614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d88009420 con 0x7f8da00aca80 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d95d9b700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 0x7f8da01505f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800ecb0 tx=0x7f8d880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800c920 con 0x7f8da00aca80 2023-10-06T03:02:34.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8800ca80 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d900093c0 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800ce20 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.516+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da015af70 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.517+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da006e380 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.518+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d88013680 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.518+0000 7f8d96ffd700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d80066f30 0x7f8d800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.518+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8d880847c0 con 0x7f8da00aca80 2023-10-06T03:02:34.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.518+0000 7f8d9e59c700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d80066f30 0x7f8d800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:34.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.519+0000 7f8d9e59c700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d80066f30 0x7f8d800693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8da016d300 tx=0x7f8d90016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:34.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.522+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d880518d0 con 0x7f8da00aca80 2023-10-06T03:02:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.750+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8da004c3c0 con 0x7f8da00aca80 2023-10-06T03:02:34.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.750+0000 7f8d96ffd700 1 -- 172.21.15.92:0/2849252614 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8d88054ef0 con 0x7f8da00aca80 2023-10-06T03:02:34.754 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:02:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d80066f30 msgr2=0x7f8d800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d80066f30 0x7f8d800693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8da016d300 tx=0x7f8d90016000 comp rx=0 tx=0).stop 2023-10-06T03:02:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 msgr2=0x7f8da01505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 0x7f8da01505f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800ecb0 tx=0x7f8d880091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 shutdown_connections 2023-10-06T03:02:34.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d80066f30 0x7f8d800693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8da00c81f0 0x7f8da015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8da0069d60 0x7f8da016ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 --2- 172.21.15.92:0/2849252614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8da00aca80 0x7f8da01505f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 >> 172.21.15.92:0/2849252614 conn(0x7f8da00cd670 msgr2=0x7f8da01526f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 shutdown_connections 2023-10-06T03:02:34.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:34.753+0000 7f8da48e3700 1 -- 172.21.15.92:0/2849252614 wait complete. 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:02:34.767 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:02:34.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:34.768 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.783 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:02:34.784 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42451sec preferred_lft 42451sec 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:02:34.785 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:02:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.786 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:02:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:02:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:02:34.786 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:34.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:34.786 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:02:34.846 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:02:34.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:34.848 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:02:34.903 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:02:34.903 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:02:34.903 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:02:34.903 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:02:34.904 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:02:34.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:34.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:02:34.904 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:02:34.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:02:34.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:02:34.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:02:34.905 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:02:35.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.004 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:02:35.018 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:02:35.018 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:02:35.019 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:02:35.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:02:35.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:35.083 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:02:35.137 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:02:35.137 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:35.137 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:02:35.191 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:02:35.192 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:02:35.192 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:35.271 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:35.271 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:35.290 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:02:35.313 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:02:35.314 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:02:35.315 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:02:35.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:02:35.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:02:35.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.532+0000 7f1943fff700 1 Processor -- start 2023-10-06T03:02:35.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.532+0000 7f1943fff700 1 -- start start 2023-10-06T03:02:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f1943fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1944159420 0x7f19440b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f1943fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19441597f0 0x7f19440b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:35.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f1943fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19440b11d0 con 0x7f194415d230 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f1943fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19440b14b0 con 0x7f19441597f0 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f1943fff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19440b1790 con 0x7f1944159420 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f193a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f19441597f0 0x7f19440b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f19437fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f194415d230 0x7f194415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36146/0 (socket says 172.21.15.92:36146) 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f19437fe700 1 -- 172.21.15.92:0/4210750930 learned_addr learned my addr 172.21.15.92:0/4210750930 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f1942ffd700 1 --2- 172.21.15.92:0/4210750930 >> v2:172.21.15.136:3300/0 conn(0x7f1944159420 0x7f19440b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f193a7fc700 1 -- 172.21.15.92:0/4210750930 >> v2:172.21.15.136:3300/0 conn(0x7f1944159420 msgr2=0x7f19440b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.533+0000 7f193a7fc700 1 --2- 172.21.15.92:0/4210750930 >> v2:172.21.15.136:3300/0 conn(0x7f1944159420 0x7f19440b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f193a7fc700 1 -- 172.21.15.92:0/4210750930 >> v1:172.21.15.92:6789/0 conn(0x7f194415d230 legacy=0x7f194415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f19427fc700 1 -- 172.21.15.92:0/4210750930 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965641097 0 0) 0x7f19440b11d0 con 0x7f194415d230 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f193a7fc700 1 -- 172.21.15.92:0/4210750930 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19440b1a70 con 0x7f19441597f0 2023-10-06T03:02:35.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f193a7fc700 1 --2- 172.21.15.92:0/4210750930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19441597f0 0x7f19440b2270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19440b2f20 tx=0x7f192c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a25a0f2907c721 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f19427fc700 1 -- 172.21.15.92:0/4210750930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00b6b0 con 0x7f19441597f0 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f19427fc700 1 -- 172.21.15.92:0/4210750930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00b810 con 0x7f19441597f0 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f19427fc700 1 -- 172.21.15.92:0/4210750930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00bbb0 con 0x7f19441597f0 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 -- 172.21.15.92:0/4210750930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19441597f0 msgr2=0x7f19440b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 --2- 172.21.15.92:0/4210750930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19441597f0 0x7f19440b2270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f19440b2f20 tx=0x7f192c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 -- 172.21.15.92:0/4210750930 shutdown_connections 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 --2- 172.21.15.92:0/4210750930 >> v2:172.21.15.136:3300/0 conn(0x7f1944159420 0x7f19440b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 --2- 172.21.15.92:0/4210750930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19441597f0 0x7f19440b2270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 -- 172.21.15.92:0/4210750930 >> 172.21.15.92:0/4210750930 conn(0x7f19440d0660 msgr2=0x7f19440d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 -- 172.21.15.92:0/4210750930 shutdown_connections 2023-10-06T03:02:35.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.534+0000 7f1943fff700 1 -- 172.21.15.92:0/4210750930 wait complete. 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 Processor -- start 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 -- start start 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 0x7f194415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19441597f0 0x7f194414dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194415d230 0x7f1944159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194414e280 con 0x7f1944159420 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194415a220 con 0x7f194415d230 2023-10-06T03:02:35.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1943fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19440b5130 con 0x7f19441597f0 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 0x7f194415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 0x7f194415b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34366/0 (socket says 172.21.15.92:34366) 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.536+0000 7f1942ffd700 1 -- 172.21.15.92:0/120260489 learned_addr learned my addr 172.21.15.92:0/120260489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f19437fe700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194415d230 0x7f1944159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f193a7fc700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19441597f0 0x7f194414dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1942ffd700 1 -- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19441597f0 msgr2=0x7f194414dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1942ffd700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19441597f0 0x7f194414dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1942ffd700 1 -- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194415d230 msgr2=0x7f1944159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1942ffd700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194415d230 0x7f1944159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1942ffd700 1 -- 172.21.15.92:0/120260489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c009420 con 0x7f1944159420 2023-10-06T03:02:35.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f19437fe700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194415d230 0x7f1944159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1942ffd700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 0x7f194415b960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f193400d890 tx=0x7f1934008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19340125f0 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1934012750 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934012ad0 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19440b5410 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.537+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19440b59d0 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.538+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194404c3c0 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.538+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f193401e460 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.539+0000 7f193b7fe700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1924066f30 0x7f19240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.539+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1934084cb0 con 0x7f1944159420 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.539+0000 7f193a7fc700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1924066f30 0x7f19240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:35.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.539+0000 7f193a7fc700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1924066f30 0x7f19240693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1944159c70 tx=0x7f192c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.543+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1934051dc0 con 0x7f1944159420 2023-10-06T03:02:35.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.773+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f19440b1d50 con 0x7f1944159420 2023-10-06T03:02:35.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.773+0000 7f193b7fe700 1 -- 172.21.15.92:0/120260489 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f19340553e0 con 0x7f1944159420 2023-10-06T03:02:35.775 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:02:35.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1924066f30 msgr2=0x7f19240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:35.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1924066f30 0x7f19240693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1944159c70 tx=0x7f192c017000 comp rx=0 tx=0).stop 2023-10-06T03:02:35.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 msgr2=0x7f194415b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:35.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 0x7f194415b960 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f193400d890 tx=0x7f1934008040 comp rx=0 tx=0).stop 2023-10-06T03:02:35.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 shutdown_connections 2023-10-06T03:02:35.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1924066f30 0x7f19240693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1944159420 0x7f194415b960 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194415d230 0x7f1944159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 --2- 172.21.15.92:0/120260489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19441597f0 0x7f194414dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:35.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 >> 172.21.15.92:0/120260489 conn(0x7f19440d0660 msgr2=0x7f1944156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:35.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 shutdown_connections 2023-10-06T03:02:35.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:35.775+0000 7f1943fff700 1 -- 172.21.15.92:0/120260489 wait complete. 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:02:35.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:02:35.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.790 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:02:35.808 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:02:35.808 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:02:35.808 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:02:35.808 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:02:35.809 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42451sec preferred_lft 42451sec 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.810 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:02:35.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.811 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:02:35.875 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:02:35.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.877 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:02:35.935 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:02:35.935 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:02:35.935 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:02:35.935 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:02:35.935 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:02:35.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:35.936 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:02:35.936 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:02:35.936 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:02:35.936 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:02:35.936 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:02:35.936 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:02:36.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:36.041 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:02:36.056 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:02:36.056 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:02:36.056 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:02:36.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:36.057 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:02:36.122 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:36.122 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:36.177 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:02:36.177 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:02:36.178 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:36.231 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:02:36.232 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:02:36.232 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:36.311 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:36.311 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:36.329 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:36.329 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:36.330 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:36.330 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:36.330 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:36.330 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:36.330 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:36.330 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:02:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:02:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:02:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:02:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:02:36.351 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:02:36.352 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:02:36.352 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:02:36.352 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:02:36.352 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:02:36.352 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:02:36.353 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:02:36.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:36.354 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:36.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:02:36.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.536+0000 7f1891a5c700 1 Processor -- start 2023-10-06T03:02:36.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.536+0000 7f1891a5c700 1 -- start start 2023-10-06T03:02:36.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1891a5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 0x7f188c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1891a5c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 0x7f188c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1891a5c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0cfa00 con 0x7f188c1591e0 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1891a5c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c0cb580 con 0x7f188c0b0790 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1891a5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0b2de0 con 0x7f188c0b03c0 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1890a5a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 0x7f188c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f188bfff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 0x7f188c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f189125b700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f188c1591e0 0x7f188c159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:42414/0 (socket says 172.21.15.92:42414) 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f1890a5a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 0x7f188c0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:53502/0 (socket says 172.21.15.92:53502) 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f188bfff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 0x7f188c0b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:57130/0 (socket says 172.21.15.92:57130) 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.537+0000 7f189125b700 1 -- 172.21.15.92:0/135750833 learned_addr learned my addr 172.21.15.92:0/135750833 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:36.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1890a5a700 1 -- 172.21.15.92:0/135750833 >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 msgr2=0x7f188c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1890a5a700 1 --2- 172.21.15.92:0/135750833 >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 0x7f188c0b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1890a5a700 1 -- 172.21.15.92:0/135750833 >> v1:172.21.15.136:6789/0 conn(0x7f188c1591e0 legacy=0x7f188c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1890a5a700 1 -- 172.21.15.92:0/135750833 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0cf160 con 0x7f188c0b03c0 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f188bfff700 1 --2- 172.21.15.92:0/135750833 >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 0x7f188c0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1890a5a700 1 --2- 172.21.15.92:0/135750833 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 0x7f188c0cea70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18800098e0 tx=0x7f18800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45518a3f435180e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f188b7fe700 1 -- 172.21.15.92:0/135750833 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000b640 con 0x7f188c0b03c0 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f188b7fe700 1 -- 172.21.15.92:0/135750833 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188000b7a0 con 0x7f188c0b03c0 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f188b7fe700 1 -- 172.21.15.92:0/135750833 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000bb40 con 0x7f188c0b03c0 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1891a5c700 1 -- 172.21.15.92:0/135750833 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 msgr2=0x7f188c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:36.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1891a5c700 1 --2- 172.21.15.92:0/135750833 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 0x7f188c0cea70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18800098e0 tx=0x7f18800091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:36.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1891a5c700 1 -- 172.21.15.92:0/135750833 shutdown_connections 2023-10-06T03:02:36.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1891a5c700 1 --2- 172.21.15.92:0/135750833 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c0b03c0 0x7f188c0cea70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:36.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1891a5c700 1 --2- 172.21.15.92:0/135750833 >> v2:172.21.15.136:3300/0 conn(0x7f188c0b0790 0x7f188c0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:36.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.538+0000 7f1891a5c700 1 -- 172.21.15.92:0/135750833 >> 172.21.15.92:0/135750833 conn(0x7f188c063230 msgr2=0x7f188c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:36.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.539+0000 7f1891a5c700 1 -- 172.21.15.92:0/135750833 shutdown_connections 2023-10-06T03:02:36.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.539+0000 7f1891a5c700 1 -- 172.21.15.92:0/135750833 wait complete. 2023-10-06T03:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.540+0000 7f1891a5c700 1 Processor -- start 2023-10-06T03:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.540+0000 7f1891a5c700 1 -- start start 2023-10-06T03:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.540+0000 7f1891a5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 0x7f188c155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f188c0b0790 0x7f188c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c1591e0 0x7f188c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c49b0 con 0x7f188c0b03c0 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c4c90 con 0x7f188c0b0790 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c4f70 con 0x7f188c1591e0 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1890a5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 0x7f188c155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f188bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f188c0b0790 0x7f188c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1890a5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 0x7f188c155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34376/0 (socket says 172.21.15.92:34376) 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1890a5a700 1 -- 172.21.15.92:0/1309910383 learned_addr learned my addr 172.21.15.92:0/1309910383 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c1591e0 0x7f188c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 -- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f188c0b0790 msgr2=0x7f188c0c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:36.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f188c0b0790 0x7f188c0c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 -- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 msgr2=0x7f188c155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 0x7f188c155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 -- 172.21.15.92:0/1309910383 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878009420 con 0x7f188c1591e0 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1890a5a700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 0x7f188c155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f188bfff700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f188c0b0790 0x7f188c0c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f189125b700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c1591e0 0x7f188c157610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f188400cb90 tx=0x7f18840019a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400a7e0 con 0x7f188c1591e0 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188400a980 con 0x7f188c1591e0 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400ad40 con 0x7f188c1591e0 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1880009420 con 0x7f188c1591e0 2023-10-06T03:02:36.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.541+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0c5e30 con 0x7f188c1591e0 2023-10-06T03:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.542+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c0c75a0 con 0x7f188c1591e0 2023-10-06T03:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.543+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1884007300 con 0x7f188c1591e0 2023-10-06T03:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.543+0000 7f1888ff9700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1870066f60 0x7f1870069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.543+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1884084930 con 0x7f188c1591e0 2023-10-06T03:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.543+0000 7f1890a5a700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1870066f60 0x7f1870069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:36.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.543+0000 7f1890a5a700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1870066f60 0x7f1870069410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1880003080 tx=0x7f1880017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:36.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.548+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f188c0c75a0 con 0x7f188c1591e0 2023-10-06T03:02:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:36.781+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f188c0c75a0 con 0x7f188c1591e0 2023-10-06T03:02:37.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.630+0000 7f1888ff9700 1 -- 172.21.15.92:0/1309910383 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7f188c0c75a0 con 0x7f188c1591e0 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1870066f60 msgr2=0x7f1870069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1870066f60 0x7f1870069410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1880003080 tx=0x7f1880017000 comp rx=0 tx=0).stop 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c1591e0 msgr2=0x7f188c157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c1591e0 0x7f188c157610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f188400cb90 tx=0x7f18840019a0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 shutdown_connections 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1870066f60 0x7f1870069410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f188c0b03c0 0x7f188c155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f188c1591e0 0x7f188c157610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 --2- 172.21.15.92:0/1309910383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f188c0b0790 0x7f188c0c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 >> 172.21.15.92:0/1309910383 conn(0x7f188c063230 msgr2=0x7f188c1587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:37.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 shutdown_connections 2023-10-06T03:02:37.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.633+0000 7f1891a5c700 1 -- 172.21.15.92:0/1309910383 wait complete. 2023-10-06T03:02:37.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:02:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.763+0000 7f4f6f2fc700 1 Processor -- start 2023-10-06T03:02:37.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.763+0000 7f4f6f2fc700 1 -- start start 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6f2fc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4f68159890 0x7f4f6815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6f2fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f68069560 0x7f4f680699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6f2fc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f68069ee0 con 0x7f4f68159c60 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6f2fc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6815a160 con 0x7f4f68069560 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6f2fc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f680d13a0 con 0x7f4f68159890 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f68069560 0x7f4f680699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4f68069560 0x7f4f680699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34390/0 (socket says 172.21.15.92:34390) 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 -- 172.21.15.92:0/3818296593 learned_addr learned my addr 172.21.15.92:0/3818296593 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:37.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d098700 1 --2- 172.21.15.92:0/3818296593 >> v2:172.21.15.136:3300/0 conn(0x7f4f68159890 0x7f4f6815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 -- 172.21.15.92:0/3818296593 >> v2:172.21.15.136:3300/0 conn(0x7f4f68159890 msgr2=0x7f4f6815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 --2- 172.21.15.92:0/3818296593 >> v2:172.21.15.136:3300/0 conn(0x7f4f68159890 0x7f4f6815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 -- 172.21.15.92:0/3818296593 >> v1:172.21.15.136:6789/0 conn(0x7f4f68159c60 legacy=0x7f4f6815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 -- 172.21.15.92:0/3818296593 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6815e140 con 0x7f4f68069560 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d098700 1 --2- 172.21.15.92:0/3818296593 >> v2:172.21.15.136:3300/0 conn(0x7f4f68159890 0x7f4f6815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f6d899700 1 --2- 172.21.15.92:0/3818296593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68069560 0x7f4f680699a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4f680d1090 tx=0x7f4f60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d0ec5821570b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f5ffff700 1 -- 172.21.15.92:0/3818296593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6000e070 con 0x7f4f68069560 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f5ffff700 1 -- 172.21.15.92:0/3818296593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6000ba10 con 0x7f4f68069560 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.764+0000 7f4f5ffff700 1 -- 172.21.15.92:0/3818296593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f600092e0 con 0x7f4f68069560 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/3818296593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68069560 msgr2=0x7f4f680699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/3818296593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68069560 0x7f4f680699a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4f680d1090 tx=0x7f4f60009840 comp rx=0 tx=0).stop 2023-10-06T03:02:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/3818296593 shutdown_connections 2023-10-06T03:02:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/3818296593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68069560 0x7f4f680699a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/3818296593 >> v2:172.21.15.136:3300/0 conn(0x7f4f68159890 0x7f4f6815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/3818296593 >> 172.21.15.92:0/3818296593 conn(0x7f4f68068e20 msgr2=0x7f4f680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/3818296593 shutdown_connections 2023-10-06T03:02:37.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.765+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/3818296593 wait complete. 2023-10-06T03:02:37.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.766+0000 7f4f6f2fc700 1 Processor -- start 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.766+0000 7f4f6f2fc700 1 -- start start 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6f2fc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f68069560 0x7f4f68152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6f2fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f68159890 0x7f4f680b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6f2fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 0x7f4f680cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6f2fc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680b1840 con 0x7f4f68159c60 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6f2fc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68152a90 con 0x7f4f68069560 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6f2fc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680b2210 con 0x7f4f68159890 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 0x7f4f680cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 0x7f4f680cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34404/0 (socket says 172.21.15.92:34404) 2023-10-06T03:02:37.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 -- 172.21.15.92:0/168156794 learned_addr learned my addr 172.21.15.92:0/168156794 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d098700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f68069560 0x7f4f68152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6c897700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f68159890 0x7f4f680b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 -- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f68159890 msgr2=0x7f4f680b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f68159890 0x7f4f680b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 -- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f68069560 msgr2=0x7f4f68152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f68069560 0x7f4f68152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.767+0000 7f4f6d899700 1 -- 172.21.15.92:0/168156794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f60009ae0 con 0x7f4f68159c60 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.768+0000 7f4f6d899700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 0x7f4f680cd9e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009fd0 tx=0x7f4f60017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.768+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f60012e90 con 0x7f4f68159c60 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.768+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f600032f0 con 0x7f4f68159c60 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.768+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f680b0c30 con 0x7f4f68159c60 2023-10-06T03:02:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.768+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6001d630 con 0x7f4f68159c60 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.768+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f680b11f0 con 0x7f4f68159c60 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.769+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f68089580 con 0x7f4f68159c60 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.769+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f6001da60 con 0x7f4f68159c60 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.769+0000 7f4f5d7fa700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f4c066f30 0x7f4f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.770+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4f60085fb0 con 0x7f4f68159c60 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.770+0000 7f4f6d098700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f4c066f30 0x7f4f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.770+0000 7f4f6d098700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f4c066f30 0x7f4f4c0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400ac00 tx=0x7f4f64006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:37.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.774+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f600b75a0 con 0x7f4f68159c60 2023-10-06T03:02:37.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.982+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f6804c3c0 con 0x7f4f4c066f30 2023-10-06T03:02:37.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.983+0000 7f4f5d7fa700 1 -- 172.21.15.92:0/168156794 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4f6804c3c0 con 0x7f4f4c066f30 2023-10-06T03:02:37.985 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:02:37.985 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:37.985 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:02:37.985 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:37.985 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:02:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f4c066f30 msgr2=0x7f4f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f4c066f30 0x7f4f4c0693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400ac00 tx=0x7f4f64006f90 comp rx=0 tx=0).stop 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 msgr2=0x7f4f680cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 0x7f4f680cd9e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009fd0 tx=0x7f4f60017000 comp rx=0 tx=0).stop 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 shutdown_connections 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f4c066f30 0x7f4f4c0693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f68159c60 0x7f4f680cd9e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f68069560 0x7f4f68152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 --2- 172.21.15.92:0/168156794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f68159890 0x7f4f680b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 >> 172.21.15.92:0/168156794 conn(0x7f4f68068e20 msgr2=0x7f4f68155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 shutdown_connections 2023-10-06T03:02:37.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:37.985+0000 7f4f6f2fc700 1 -- 172.21.15.92:0/168156794 wait complete. 2023-10-06T03:02:37.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:02:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.114+0000 7fe8013d8700 1 Processor -- start 2023-10-06T03:02:38.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.114+0000 7fe8013d8700 1 -- start start 2023-10-06T03:02:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe8013d8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe7fc159970 0x7fe7fc1647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe8013d8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc06cff0 con 0x7fe7fc068920 2023-10-06T03:02:38.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe8013d8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc06bf50 con 0x7fe7fc068550 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe8013d8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc068d90 con 0x7fe7fc159970 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe7fb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7fc068920 0x7fe7fc159260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36158/0 (socket says 172.21.15.92:36158) 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe7fb7fe700 1 -- 172.21.15.92:0/2105432542 learned_addr learned my addr 172.21.15.92:0/2105432542 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe800bd7700 1 --2- 172.21.15.92:0/2105432542 >> v2:172.21.15.136:3300/0 conn(0x7fe7fc159970 0x7fe7fc1647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe7faffd700 1 -- 172.21.15.92:0/2105432542 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355515460 0 0) 0x7fe7fc06cff0 con 0x7fe7fc068920 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.115+0000 7fe7faffd700 1 -- 172.21.15.92:0/2105432542 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e4003570 con 0x7fe7fc068920 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe7faffd700 1 -- 172.21.15.92:0/2105432542 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520224404 0 0) 0x7fe7e4003570 con 0x7fe7fc068920 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe800bd7700 1 -- 172.21.15.92:0/2105432542 >> v1:172.21.15.136:6789/0 conn(0x7fe7fc068550 legacy=0x7fe7fc0b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe800bd7700 1 -- 172.21.15.92:0/2105432542 >> v1:172.21.15.92:6789/0 conn(0x7fe7fc068920 legacy=0x7fe7fc159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe800bd7700 1 -- 172.21.15.92:0/2105432542 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc06c230 con 0x7fe7fc159970 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe800bd7700 1 --2- 172.21.15.92:0/2105432542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc1647f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009e60 tx=0x7fe7f4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d39134a8782a77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe7faffd700 1 -- 172.21.15.92:0/2105432542 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400e070 con 0x7fe7fc159970 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe7faffd700 1 -- 172.21.15.92:0/2105432542 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400b850 con 0x7fe7fc159970 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe7faffd700 1 -- 172.21.15.92:0/2105432542 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0001470 con 0x7fe7fc159970 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe8013d8700 1 -- 172.21.15.92:0/2105432542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 msgr2=0x7fe7fc1647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.116+0000 7fe8013d8700 1 --2- 172.21.15.92:0/2105432542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc1647f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009e60 tx=0x7fe7f4009780 comp rx=0 tx=0).stop 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.117+0000 7fe8013d8700 1 -- 172.21.15.92:0/2105432542 shutdown_connections 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.117+0000 7fe8013d8700 1 --2- 172.21.15.92:0/2105432542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc1647f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.117+0000 7fe8013d8700 1 -- 172.21.15.92:0/2105432542 >> 172.21.15.92:0/2105432542 conn(0x7fe7fc06a940 msgr2=0x7fe7fc06ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.117+0000 7fe8013d8700 1 -- 172.21.15.92:0/2105432542 shutdown_connections 2023-10-06T03:02:38.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.117+0000 7fe8013d8700 1 -- 172.21.15.92:0/2105432542 wait complete. 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.118+0000 7fe8013d8700 1 Processor -- start 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.118+0000 7fe8013d8700 1 -- start start 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.118+0000 7fe8013d8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7fc068550 0x7fe7fc14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.118+0000 7fe8013d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 0x7fe7fc151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe8013d8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe8013d8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0add60 con 0x7fe7fc068920 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe8013d8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0aed20 con 0x7fe7fc159970 2023-10-06T03:02:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe8013d8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc0ae110 con 0x7fe7fc068550 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 0x7fe7fc151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 0x7fe7fc151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34410/0 (socket says 172.21.15.92:34410) 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 -- 172.21.15.92:0/1507028019 learned_addr learned my addr 172.21.15.92:0/1507028019 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe800bd7700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fbfff700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7fc068550 0x7fe7fc14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 -- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7fc068550 msgr2=0x7fe7fc14b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7fc068550 0x7fe7fc14b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 -- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 msgr2=0x7fe7fc152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 -- 172.21.15.92:0/1507028019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4009a20 con 0x7fe7fc068920 2023-10-06T03:02:38.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe800bd7700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7fb7fe700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 0x7fe7fc151c00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000cb90 tx=0x7fe7f0001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0001d10 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f0007360 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc0ae3f0 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0004830 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.119+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc0ae9b0 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.120+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc068ed0 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.121+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7f0004ba0 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.121+0000 7fe7e3fff700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7e8066f30 0x7fe7e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.121+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe7f0083b20 con 0x7fe7fc068920 2023-10-06T03:02:38.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.121+0000 7fe7fbfff700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7e8066f30 0x7fe7e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.121+0000 7fe7fbfff700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7e8066f30 0x7fe7e80693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0037c0 tx=0x7fe7ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:38.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.125+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7f0050c30 con 0x7fe7fc068920 2023-10-06T03:02:38.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.306+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe7fc0aee60 con 0x7fe7fc068920 2023-10-06T03:02:38.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.307+0000 7fe7e3fff700 1 -- 172.21.15.92:0/1507028019 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7f0018020 con 0x7fe7fc068920 2023-10-06T03:02:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7e8066f30 msgr2=0x7fe7e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7e8066f30 0x7fe7e80693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0037c0 tx=0x7fe7ec009040 comp rx=0 tx=0).stop 2023-10-06T03:02:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 msgr2=0x7fe7fc151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 0x7fe7fc151c00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000cb90 tx=0x7fe7f0001aa0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 shutdown_connections 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7e8066f30 0x7fe7e80693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7fc159970 0x7fe7fc152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7fc068550 0x7fe7fc14b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 --2- 172.21.15.92:0/1507028019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7fc068920 0x7fe7fc151c00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 >> 172.21.15.92:0/1507028019 conn(0x7fe7fc06a940 msgr2=0x7fe7fc0a9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 shutdown_connections 2023-10-06T03:02:38.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.309+0000 7fe8013d8700 1 -- 172.21.15.92:0/1507028019 wait complete. 2023-10-06T03:02:38.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000092295 2023-10-06T03:02:38.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.436+0000 7fd6ea4c3700 1 Processor -- start 2023-10-06T03:02:38.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.437+0000 7fd6ea4c3700 1 -- start start 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.437+0000 7fd6ea4c3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.437+0000 7fd6ea4c3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6e406c0a0 0x7fd6e415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.437+0000 7fd6ea4c3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e415d7e0 con 0x7fd6e415df50 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.437+0000 7fd6ea4c3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e415dac0 con 0x7fd6e406c0a0 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.437+0000 7fd6ea4c3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e415aa30 con 0x7fd6e406bcd0 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e3fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e37fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6e406c0a0 0x7fd6e415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e8a60700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd6e415df50 0x7fd6e40c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:42444/0 (socket says 172.21.15.92:42444) 2023-10-06T03:02:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e8a60700 1 -- 172.21.15.92:0/2676450869 learned_addr learned my addr 172.21.15.92:0/2676450869 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e3fff700 1 -- 172.21.15.92:0/2676450869 >> v2:172.21.15.136:3300/0 conn(0x7fd6e406c0a0 msgr2=0x7fd6e415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e3fff700 1 --2- 172.21.15.92:0/2676450869 >> v2:172.21.15.136:3300/0 conn(0x7fd6e406c0a0 0x7fd6e415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e3fff700 1 -- 172.21.15.92:0/2676450869 >> v1:172.21.15.136:6789/0 conn(0x7fd6e415df50 legacy=0x7fd6e40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e3fff700 1 -- 172.21.15.92:0/2676450869 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e415ad10 con 0x7fd6e406bcd0 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e3fff700 1 --2- 172.21.15.92:0/2676450869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e40caa90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40098e0 tx=0x7fd6d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322fca94767b4334 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e2ffd700 1 -- 172.21.15.92:0/2676450869 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400b640 con 0x7fd6e406bcd0 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e2ffd700 1 -- 172.21.15.92:0/2676450869 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d400b7a0 con 0x7fd6e406bcd0 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.438+0000 7fd6e2ffd700 1 -- 172.21.15.92:0/2676450869 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d400bb40 con 0x7fd6e406bcd0 2023-10-06T03:02:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/2676450869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 msgr2=0x7fd6e40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/2676450869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e40caa90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40098e0 tx=0x7fd6d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/2676450869 shutdown_connections 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/2676450869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e40caa90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/2676450869 >> v2:172.21.15.136:3300/0 conn(0x7fd6e406c0a0 0x7fd6e415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/2676450869 >> 172.21.15.92:0/2676450869 conn(0x7fd6e40ce600 msgr2=0x7fd6e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/2676450869 shutdown_connections 2023-10-06T03:02:38.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.439+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/2676450869 wait complete. 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.440+0000 7fd6ea4c3700 1 Processor -- start 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.440+0000 7fd6ea4c3700 1 -- start start 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6ea4c3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e414a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6ea4c3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6e406c0a0 0x7fd6e406cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6ea4c3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 0x7fd6e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6ea4c3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40c7c50 con 0x7fd6e415df50 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6ea4c3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40c5d30 con 0x7fd6e406c0a0 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6ea4c3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40c5750 con 0x7fd6e406bcd0 2023-10-06T03:02:38.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 0x7fd6e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 0x7fd6e406d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34426/0 (socket says 172.21.15.92:34426) 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 -- 172.21.15.92:0/3503506099 learned_addr learned my addr 172.21.15.92:0/3503506099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e3fff700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e414a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e37fe700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6e406c0a0 0x7fd6e406cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 -- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 msgr2=0x7fd6e414a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e414a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 -- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6e406c0a0 msgr2=0x7fd6e406cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6e406c0a0 0x7fd6e406cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.441+0000 7fd6e8a60700 1 -- 172.21.15.92:0/3503506099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4009420 con 0x7fd6e415df50 2023-10-06T03:02:38.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.442+0000 7fd6e8a60700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 0x7fd6e406d4e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00cc70 tx=0x7fd6dc001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.442+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00a850 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.442+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00a9b0 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.442+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e40c60c0 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.442+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00ad70 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.442+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e40c59c0 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.443+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e404c3c0 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.443+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6dc007300 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.443+0000 7fd6cbfff700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd6d0066f30 0x7fd6d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.444+0000 7fd6e3fff700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd6d0066f30 0x7fd6d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.444+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd6dc084980 con 0x7fd6e415df50 2023-10-06T03:02:38.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.444+0000 7fd6e3fff700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd6d0066f30 0x7fd6d00693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40093f0 tx=0x7fd6d4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:38.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.448+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6dc051a90 con 0x7fd6e415df50 2023-10-06T03:02:38.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:38.656+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000092295", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6e406c9d0 con 0x7fd6d0066f30 2023-10-06T03:02:41.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.648+0000 7fd6cbfff700 1 -- 172.21.15.92:0/3503506099 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fd6e406c9d0 con 0x7fd6d0066f30 2023-10-06T03:02:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.653+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd6d0066f30 msgr2=0x7fd6d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.653+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd6d0066f30 0x7fd6d00693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40093f0 tx=0x7fd6d4017000 comp rx=0 tx=0).stop 2023-10-06T03:02:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.653+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 msgr2=0x7fd6e406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:41.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.653+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 0x7fd6e406d4e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00cc70 tx=0x7fd6dc001a50 comp rx=0 tx=0).stop 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 shutdown_connections 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd6d0066f30 0x7fd6d00693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6e415df50 0x7fd6e406d4e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6e406bcd0 0x7fd6e414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 --2- 172.21.15.92:0/3503506099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6e406c0a0 0x7fd6e406cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 >> 172.21.15.92:0/3503506099 conn(0x7fd6e40ce600 msgr2=0x7fd6e40c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:41.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 shutdown_connections 2023-10-06T03:02:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.654+0000 7fd6ea4c3700 1 -- 172.21.15.92:0/3503506099 wait complete. 2023-10-06T03:02:41.656 INFO:teuthology.orchestra.run.smithi092.stderr:Volume created successfully (no MDS daemons created) 2023-10-06T03:02:41.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:02:41.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:02:41.728 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:02:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.803+0000 7f2d2340c700 1 Processor -- start 2023-10-06T03:02:41.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.803+0000 7f2d2340c700 1 -- start start 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d2340c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d1c159420 0x7f2d1c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d2340c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d1c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d2340c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1c0b11d0 con 0x7f2d1c15d230 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d2340c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1c0b14b0 con 0x7f2d1c159420 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d2340c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0b1790 con 0x7f2d1c1597f0 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d22c0b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d1c15d230 0x7f2d1c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50726/0 (socket says 172.21.15.92:50726) 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d22c0b700 1 -- 172.21.15.92:0/1079701213 learned_addr learned my addr 172.21.15.92:0/1079701213 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:41.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d21c09700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d1c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d2240a700 1 --2- 172.21.15.92:0/1079701213 >> v2:172.21.15.136:3300/0 conn(0x7f2d1c159420 0x7f2d1c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039700872 0 0) 0x7f2d1c0b11d0 con 0x7f2d1c15d230 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d08003570 con 0x7f2d1c15d230 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 590810902 0 0) 0x7f2d08003570 con 0x7f2d1c15d230 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 msgr2=0x7f2d1c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.804+0000 7f2d21408700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d1c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 >> v2:172.21.15.136:3300/0 conn(0x7f2d1c159420 msgr2=0x7f2d1c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 --2- 172.21.15.92:0/1079701213 >> v2:172.21.15.136:3300/0 conn(0x7f2d1c159420 0x7f2d1c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c0b1a70 con 0x7f2d1c15d230 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2d140017f0 con 0x7f2d1c15d230 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 >> v1:172.21.15.92:6789/0 conn(0x7f2d1c15d230 legacy=0x7f2d1c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 0x7f2d08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08004810 0x7f2d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0b14b0 con 0x7f2d08004810 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c0b1790 con 0x7f2d08003a60 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0800ba60 con 0x7f2d1c1597f0 2023-10-06T03:02:41.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08004810 0x7f2d08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d2240a700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 0x7f2d08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d22c0b700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 -- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 msgr2=0x7f2d080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 -- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 msgr2=0x7f2d08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 0x7f2d08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 -- 172.21.15.92:0/1079701213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08003570 con 0x7f2d08004810 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d22c0b700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c1597f0 0x7f2d080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21c09700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08004810 0x7f2d08004c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009c30 tx=0x7f2d0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3794a3af2402b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00e710 con 0x7f2d08004810 2023-10-06T03:02:41.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c00e870 con 0x7f2d08004810 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.805+0000 7f2d21408700 1 -- 172.21.15.92:0/1079701213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00ec30 con 0x7f2d08004810 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d2340c700 1 -- 172.21.15.92:0/1079701213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08004810 msgr2=0x7f2d08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d2340c700 1 --2- 172.21.15.92:0/1079701213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d08004810 0x7f2d08004c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009c30 tx=0x7f2d0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d22c0b700 1 -- 172.21.15.92:0/1079701213 reap_dead start 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d2340c700 1 -- 172.21.15.92:0/1079701213 shutdown_connections 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d2340c700 1 -- 172.21.15.92:0/1079701213 >> 172.21.15.92:0/1079701213 conn(0x7f2d1c0d0660 msgr2=0x7f2d1c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d2340c700 1 -- 172.21.15.92:0/1079701213 shutdown_connections 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.806+0000 7f2d2340c700 1 -- 172.21.15.92:0/1079701213 wait complete. 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.807+0000 7f2d2340c700 1 Processor -- start 2023-10-06T03:02:41.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.807+0000 7f2d2340c700 1 -- start start 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08004810 0x7f2d1c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c0b5890 0x7f2d1c0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 0x7f2d1c15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c169850 con 0x7f2d1c15be60 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c15cbd0 con 0x7f2d08004810 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1c16a7f0 con 0x7f2d1c0b5890 2023-10-06T03:02:41.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 0x7f2d1c15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 0x7f2d1c15c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52478/0 (socket says 172.21.15.92:52478) 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 -- 172.21.15.92:0/2935996651 learned_addr learned my addr 172.21.15.92:0/2935996651 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d21c09700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c0b5890 0x7f2d1c0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2240a700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08004810 0x7f2d1c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 -- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c0b5890 msgr2=0x7f2d1c0b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c0b5890 0x7f2d1c0b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 -- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08004810 msgr2=0x7f2d1c14fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08004810 0x7f2d1c14fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 -- 172.21.15.92:0/2935996651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c0b6220 con 0x7f2d1c15be60 2023-10-06T03:02:41.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d21c09700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c0b5890 0x7f2d1c0b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2240a700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08004810 0x7f2d1c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d22c0b700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 0x7f2d1c15c2e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001350 tx=0x7f2d14002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ec30 con 0x7f2d1c15be60 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1400ed90 con 0x7f2d1c15be60 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1c169a30 con 0x7f2d1c15be60 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14013670 con 0x7f2d1c15be60 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.808+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1c16a070 con 0x7f2d1c15be60 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.809+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1c04c3c0 con 0x7f2d1c15be60 2023-10-06T03:02:41.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.810+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d14013aa0 con 0x7f2d1c15be60 2023-10-06T03:02:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.810+0000 7f2d127fc700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d00066f30 0x7f2d000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.810+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6956+0+0 (secure 0 0 0) 0x7f2d140850c0 con 0x7f2d1c15be60 2023-10-06T03:02:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.810+0000 7f2d2240a700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d00066f30 0x7f2d000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.811+0000 7f2d2240a700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d00066f30 0x7f2d000693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1541c0 tx=0x7f2d18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:41.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:41.814+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d14051e50 con 0x7f2d1c15be60 2023-10-06T03:02:42.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.022+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d1c0b1d50 con 0x7f2d00066f30 2023-10-06T03:02:42.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.024+0000 7f2d127fc700 1 -- 172.21.15.92:0/2935996651 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f2d1c0b1d50 con 0x7f2d00066f30 2023-10-06T03:02:42.024 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:02:42.024 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:42.025 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:02:42.025 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:42.025 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:42.025 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volume_0000000000092295" 2023-10-06T03:02:42.025 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:42.025 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:02:42.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d00066f30 msgr2=0x7f2d000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d00066f30 0x7f2d000693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c1541c0 tx=0x7f2d18008040 comp rx=0 tx=0).stop 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 msgr2=0x7f2d1c15c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 0x7f2d1c15c2e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001350 tx=0x7f2d14002180 comp rx=0 tx=0).stop 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 shutdown_connections 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d00066f30 0x7f2d000693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1c15be60 0x7f2d1c15c2e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08004810 0x7f2d1c14fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 --2- 172.21.15.92:0/2935996651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d1c0b5890 0x7f2d1c0b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 >> 172.21.15.92:0/2935996651 conn(0x7f2d1c0d0660 msgr2=0x7f2d1c069840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:42.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 shutdown_connections 2023-10-06T03:02:42.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.026+0000 7f2d2340c700 1 -- 172.21.15.92:0/2935996651 wait complete. 2023-10-06T03:02:42.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2023-10-06T03:02:42.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.194+0000 7f585ca5a700 1 Processor -- start 2023-10-06T03:02:42.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.194+0000 7f585ca5a700 1 -- start start 2023-10-06T03:02:42.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585ca5a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5858159890 0x7f585815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585ca5a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5858069560 0x7f58580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585ca5a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5858069ee0 con 0x7f5858159c60 2023-10-06T03:02:42.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585ca5a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585815a160 con 0x7f5858159890 2023-10-06T03:02:42.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585ca5a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58580d13a0 con 0x7f5858069560 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5858159890 0x7f585815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5858159890 0x7f585815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52480/0 (socket says 172.21.15.92:52480) 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f585659c700 1 -- 172.21.15.92:0/3592512563 learned_addr learned my addr 172.21.15.92:0/3592512563 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.195+0000 7f5856d9d700 1 --2- 172.21.15.92:0/3592512563 >> v2:172.21.15.136:3300/0 conn(0x7f5858069560 0x7f58580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585659c700 1 -- 172.21.15.92:0/3592512563 >> v2:172.21.15.136:3300/0 conn(0x7f5858069560 msgr2=0x7f58580699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585659c700 1 --2- 172.21.15.92:0/3592512563 >> v2:172.21.15.136:3300/0 conn(0x7f5858069560 0x7f58580699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585659c700 1 -- 172.21.15.92:0/3592512563 >> v1:172.21.15.136:6789/0 conn(0x7f5858159c60 legacy=0x7f585815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585659c700 1 -- 172.21.15.92:0/3592512563 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585815e140 con 0x7f5858159890 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585659c700 1 --2- 172.21.15.92:0/3592512563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159890 0x7f585815abd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f585806a890 tx=0x7f5848009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11e9fdc3eab82a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f5855d9b700 1 -- 172.21.15.92:0/3592512563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848009e70 con 0x7f5858159890 2023-10-06T03:02:42.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f5855d9b700 1 -- 172.21.15.92:0/3592512563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800b860 con 0x7f5858159890 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f5855d9b700 1 -- 172.21.15.92:0/3592512563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800bc00 con 0x7f5858159890 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 -- 172.21.15.92:0/3592512563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159890 msgr2=0x7f585815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3592512563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159890 0x7f585815abd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f585806a890 tx=0x7f5848009250 comp rx=0 tx=0).stop 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 -- 172.21.15.92:0/3592512563 shutdown_connections 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3592512563 >> v2:172.21.15.136:3300/0 conn(0x7f5858069560 0x7f58580699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3592512563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159890 0x7f585815abd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 -- 172.21.15.92:0/3592512563 >> 172.21.15.92:0/3592512563 conn(0x7f5858068e20 msgr2=0x7f58580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 -- 172.21.15.92:0/3592512563 shutdown_connections 2023-10-06T03:02:42.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.196+0000 7f585ca5a700 1 -- 172.21.15.92:0/3592512563 wait complete. 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 Processor -- start 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 -- start start 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5858069560 0x7f58580c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5858159890 0x7f58580c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 0x7f58580b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580c9660 con 0x7f5858159c60 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580c8b40 con 0x7f5858159890 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.198+0000 7f585ca5a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580c7f30 con 0x7f5858069560 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 0x7f58580b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 0x7f58580b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52488/0 (socket says 172.21.15.92:52488) 2023-10-06T03:02:42.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 -- 172.21.15.92:0/3506622434 learned_addr learned my addr 172.21.15.92:0/3506622434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f585659c700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5858069560 0x7f58580c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f584dd9b700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5858159890 0x7f58580c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 -- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5858069560 msgr2=0x7f58580c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5858069560 0x7f58580c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 -- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5858159890 msgr2=0x7f58580c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5858159890 0x7f58580c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 -- 172.21.15.92:0/3506622434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480094f0 con 0x7f5858159c60 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f5856d9d700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 0x7f58580b5e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5850009e90 tx=0x7f585000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585000cb70 con 0x7f5858159c60 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585000f040 con 0x7f5858159c60 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850014480 con 0x7f5858159c60 2023-10-06T03:02:42.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58580c8210 con 0x7f5858159c60 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.199+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58580c87d0 con 0x7f5858159c60 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.200+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585804c3c0 con 0x7f5858159c60 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.200+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5850014830 con 0x7f5858159c60 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.201+0000 7f584effd700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5838066f30 0x7f58380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.201+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6956+0+0 (secure 0 0 0) 0x7f5850085620 con 0x7f5858159c60 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.201+0000 7f585659c700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5838066f30 0x7f58380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.201+0000 7f585659c700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5838066f30 0x7f58380693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f585806a890 tx=0x7f5848017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.205+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58500523b0 con 0x7f5858159c60 2023-10-06T03:02:42.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.384+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f585806a5b0 con 0x7f5858159c60 2023-10-06T03:02:42.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.385+0000 7f584effd700 1 -- 172.21.15.92:0/3506622434 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v38) v1 ==== 97+0+9556 (secure 0 0 0) 0x7f58500559d0 con 0x7f5858159c60 2023-10-06T03:02:42.386 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:42.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5838066f30 msgr2=0x7f58380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5838066f30 0x7f58380693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f585806a890 tx=0x7f5848017000 comp rx=0 tx=0).stop 2023-10-06T03:02:42.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 msgr2=0x7f58580b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 0x7f58580b5e10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5850009e90 tx=0x7f585000b040 comp rx=0 tx=0).stop 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 shutdown_connections 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5838066f30 0x7f58380693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5858159c60 0x7f58580b5e10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5858069560 0x7f58580c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 --2- 172.21.15.92:0/3506622434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5858159890 0x7f58580c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 >> 172.21.15.92:0/3506622434 conn(0x7f5858068e20 msgr2=0x7f5858153430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 shutdown_connections 2023-10-06T03:02:42.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.387+0000 7f585ca5a700 1 -- 172.21.15.92:0/3506622434 wait complete. 2023-10-06T03:02:42.400 INFO:teuthology.orchestra.run.smithi092.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool_id":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000092295.meta","create_time":"2023-10-06T03:02:38.675885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000092295"}}},{"pool_id":8,"pool_name":"cephfs.volume_0000000000092295.data","create_time":"2023-10-06T03:02:39.663118+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000092295"}}}] 2023-10-06T03:02:42.400 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000092295 --format=json 2023-10-06T03:02:42.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.516+0000 7fdd1a053700 1 Processor -- start 2023-10-06T03:02:42.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.516+0000 7fdd1a053700 1 -- start start 2023-10-06T03:02:42.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd1a053700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdd1406bcd0 0x7fdd14140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd1a053700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 0x7fdd14141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd1a053700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1415c2a0 con 0x7fdd1406c0a0 2023-10-06T03:02:42.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd1a053700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1415b690 con 0x7fdd1406bcd0 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd1a053700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1415b970 con 0x7fdd140c9510 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd12ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdd1406c0a0 0x7fdd140c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50728/0 (socket says 172.21.15.92:50728) 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd12ffd700 1 -- 172.21.15.92:0/860454663 learned_addr learned my addr 172.21.15.92:0/860454663 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd137fe700 1 --2- 172.21.15.92:0/860454663 >> v2:172.21.15.136:3300/0 conn(0x7fdd1406bcd0 0x7fdd14140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd13fff700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 0x7fdd14141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988265241 0 0) 0x7fdd1415c2a0 con 0x7fdd1406c0a0 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.517+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcfc003570 con 0x7fdd1406c0a0 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1553115595 0 0) 0x7fdcfc003570 con 0x7fdd1406c0a0 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 msgr2=0x7fdd14141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 0x7fdd14141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 >> v2:172.21.15.136:3300/0 conn(0x7fdd1406bcd0 msgr2=0x7fdd14140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 --2- 172.21.15.92:0/860454663 >> v2:172.21.15.136:3300/0 conn(0x7fdd1406bcd0 0x7fdd14140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1415bc50 con 0x7fdd1406c0a0 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fdd08001470 con 0x7fdd1406c0a0 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 >> v1:172.21.15.92:6789/0 conn(0x7fdd1406c0a0 legacy=0x7fdd140c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcfc003a60 0x7fdcfc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 0x7fdcfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc004d50 0x7fdcfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1415b690 con 0x7fdcfc004d50 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1415b970 con 0x7fdcfc003a60 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc00bb90 con 0x7fdd140c9510 2023-10-06T03:02:42.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc004d50 0x7fdcfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd137fe700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcfc003a60 0x7fdcfc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd12ffd700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 0x7fdcfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 -- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 msgr2=0x7fdcfc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd140c9510 0x7fdcfc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 -- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcfc003a60 msgr2=0x7fdcfc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdcfc003a60 0x7fdcfc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 -- 172.21.15.92:0/860454663 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc003570 con 0x7fdcfc004d50 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd13fff700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc004d50 0x7fdcfc0051c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007440 tx=0x7fdd0c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34db5dd388a01b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c009a40 con 0x7fdcfc004d50 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.518+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c007cf0 con 0x7fdcfc004d50 2023-10-06T03:02:42.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd127fc700 1 -- 172.21.15.92:0/860454663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c014630 con 0x7fdcfc004d50 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd1a053700 1 -- 172.21.15.92:0/860454663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc004d50 msgr2=0x7fdcfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd1a053700 1 --2- 172.21.15.92:0/860454663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdcfc004d50 0x7fdcfc0051c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007440 tx=0x7fdd0c009710 comp rx=0 tx=0).stop 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd13fff700 1 -- 172.21.15.92:0/860454663 reap_dead start 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd1a053700 1 -- 172.21.15.92:0/860454663 shutdown_connections 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd1a053700 1 -- 172.21.15.92:0/860454663 >> 172.21.15.92:0/860454663 conn(0x7fdd140ce600 msgr2=0x7fdd140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd1a053700 1 -- 172.21.15.92:0/860454663 shutdown_connections 2023-10-06T03:02:42.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.519+0000 7fdd1a053700 1 -- 172.21.15.92:0/860454663 wait complete. 2023-10-06T03:02:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.520+0000 7fdd1a053700 1 Processor -- start 2023-10-06T03:02:42.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.520+0000 7fdd1a053700 1 -- start start 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd1a053700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1413f500 0x7fdd140c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd1a053700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd141539f0 0x7fdd140c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd1a053700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 0x7fdd140ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd1a053700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cd130 con 0x7fdd140cc770 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd1a053700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140c6d30 con 0x7fdd141539f0 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd1a053700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1413f8d0 con 0x7fdd1413f500 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 0x7fdd140ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 0x7fdd140ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52498/0 (socket says 172.21.15.92:52498) 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 -- 172.21.15.92:0/571633814 learned_addr learned my addr 172.21.15.92:0/571633814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:42.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd12ffd700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd141539f0 0x7fdd140c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd137fe700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1413f500 0x7fdd140c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 -- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1413f500 msgr2=0x7fdd140c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1413f500 0x7fdd140c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 -- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd141539f0 msgr2=0x7fdd140c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd141539f0 0x7fdd140c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.521+0000 7fdd13fff700 1 -- 172.21.15.92:0/571633814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140c6120 con 0x7fdd140cc770 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdd12ffd700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd141539f0 0x7fdd140c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdd137fe700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1413f500 0x7fdd140c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdd13fff700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 0x7fdd140ccbf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009fd0 tx=0x7fdd0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c009a40 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c007e60 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd140c6400 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c013e10 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.522+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd140c69c0 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.523+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1404c3c0 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.523+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd0c003040 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.524+0000 7fdcfb7fe700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd00066f30 0x7fdd000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.524+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6956+0+0 (secure 0 0 0) 0x7fdd0c086ae0 con 0x7fdd140cc770 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.524+0000 7fdd137fe700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd00066f30 0x7fdd000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.524+0000 7fdd137fe700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd00066f30 0x7fdd000693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400b0b0 tx=0x7fdd0400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.528+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd0c053870 con 0x7fdd140cc770 2023-10-06T03:02:42.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.757+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000092295", "format": "json"} v 0) v1 -- 0x7fdd140c6e70 con 0x7fdd140cc770 2023-10-06T03:02:42.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.758+0000 7fdcfb7fe700 1 -- 172.21.15.92:0/571633814 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000092295", "format": "json"}]=0 v19) v1 ==== 110+0+1841 (secure 0 0 0) 0x7fdd0c056e90 con 0x7fdd140cc770 2023-10-06T03:02:42.759 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:42.759 INFO:teuthology.orchestra.run.smithi092.stdout:{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:40.649048+0000","modified":"2023-10-06T03:02:42.466684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4965},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4965":{"gid":4965,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.92:6836/971383855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":971383855},{"type":"v1","addr":"172.21.15.92:6836","nonce":971383855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"volume_0000000000092295","balancer":"","standby_count_wanted":1},"id":3} 2023-10-06T03:02:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd00066f30 msgr2=0x7fdd000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd00066f30 0x7fdd000693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400b0b0 tx=0x7fdd0400b040 comp rx=0 tx=0).stop 2023-10-06T03:02:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 msgr2=0x7fdd140ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 0x7fdd140ccbf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009fd0 tx=0x7fdd0c018000 comp rx=0 tx=0).stop 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 shutdown_connections 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd00066f30 0x7fdd000693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd140cc770 0x7fdd140ccbf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1413f500 0x7fdd140c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 --2- 172.21.15.92:0/571633814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd141539f0 0x7fdd140c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 >> 172.21.15.92:0/571633814 conn(0x7fdd140ce600 msgr2=0x7fdd140c4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 shutdown_connections 2023-10-06T03:02:42.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.760+0000 7fdd1a053700 1 -- 172.21.15.92:0/571633814 wait complete. 2023-10-06T03:02:42.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000092295 --yes-i-really-mean-it 2023-10-06T03:02:42.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.888+0000 7fd2a398f700 1 Processor -- start 2023-10-06T03:02:42.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.889+0000 7fd2a398f700 1 -- start start 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.889+0000 7fd2a398f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 0x7fd29c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a398f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c15a160 con 0x7fd29c069560 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a398f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c0d13a0 con 0x7fd29c159c60 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a398f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c15e140 con 0x7fd29c159890 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a1f2c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd29c069560 0x7fd29c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50740/0 (socket says 172.21.15.92:50740) 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a1f2c700 1 -- 172.21.15.92:0/2031937583 learned_addr learned my addr 172.21.15.92:0/2031937583 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a172b700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 0x7fd29c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490116795 0 0) 0x7fd29c15a160 con 0x7fd29c069560 2023-10-06T03:02:42.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd288003570 con 0x7fd29c069560 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510879927 0 0) 0x7fd288003570 con 0x7fd29c069560 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 msgr2=0x7fd29c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 0x7fd29c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 >> v1:172.21.15.136:6789/0 conn(0x7fd29c159c60 legacy=0x7fd29c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c15a3d0 con 0x7fd29c069560 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd2940017f0 con 0x7fd29c069560 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 >> v1:172.21.15.92:6789/0 conn(0x7fd29c069560 legacy=0x7fd29c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288003a90 0x7fd288003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 0x7fd288004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd288004d50 0x7fd2880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c15e140 con 0x7fd288004d50 2023-10-06T03:02:42.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28800bb70 con 0x7fd288003a90 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28800bd30 con 0x7fd29c159890 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a1f2c700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd288004d50 0x7fd2880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a172b700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288003a90 0x7fd288003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.890+0000 7fd2a0f2a700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 0x7fd288004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 -- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 msgr2=0x7fd288004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c159890 0x7fd288004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 -- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288003a90 msgr2=0x7fd288003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288003a90 0x7fd288003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 -- 172.21.15.92:0/2031937583 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288003570 con 0x7fd288004d50 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd288004d50 0x7fd2880051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd29400a930 tx=0x7fd2940021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf523d56632ea22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2940070d0 con 0x7fd288004d50 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd294007230 con 0x7fd288004d50 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd293fff700 1 -- 172.21.15.92:0/2031937583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2940075d0 con 0x7fd288004d50 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a398f700 1 -- 172.21.15.92:0/2031937583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd288004d50 msgr2=0x7fd2880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a398f700 1 --2- 172.21.15.92:0/2031937583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd288004d50 0x7fd2880051c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd29400a930 tx=0x7fd2940021d0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.891+0000 7fd2a1f2c700 1 -- 172.21.15.92:0/2031937583 reap_dead start 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.892+0000 7fd2a398f700 1 -- 172.21.15.92:0/2031937583 shutdown_connections 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.892+0000 7fd2a398f700 1 -- 172.21.15.92:0/2031937583 >> 172.21.15.92:0/2031937583 conn(0x7fd29c068e20 msgr2=0x7fd29c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.892+0000 7fd2a398f700 1 -- 172.21.15.92:0/2031937583 shutdown_connections 2023-10-06T03:02:42.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.892+0000 7fd2a398f700 1 -- 172.21.15.92:0/2031937583 wait complete. 2023-10-06T03:02:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.893+0000 7fd2a398f700 1 Processor -- start 2023-10-06T03:02:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.893+0000 7fd2a398f700 1 -- start start 2023-10-06T03:02:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288004d50 0x7fd29c1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 0x7fd29c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c15ca40 0x7fd29c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c15d420 con 0x7fd29c0b47b0 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c0b4320 con 0x7fd288004d50 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c149b10 con 0x7fd29c15ca40 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 0x7fd29c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 0x7fd29c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52516/0 (socket says 172.21.15.92:52516) 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 -- 172.21.15.92:0/3327053775 learned_addr learned my addr 172.21.15.92:0/3327053775 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a172b700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288004d50 0x7fd29c1495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a1f2c700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c15ca40 0x7fd29c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 -- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c15ca40 msgr2=0x7fd29c15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c15ca40 0x7fd29c15cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 -- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288004d50 msgr2=0x7fd29c1495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288004d50 0x7fd29c1495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 -- 172.21.15.92:0/3327053775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c0b3710 con 0x7fd29c0b47b0 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a1f2c700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c15ca40 0x7fd29c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a0f2a700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 0x7fd29c0cda80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd28c002950 tx=0x7fd28c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c006820 con 0x7fd29c0b47b0 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c006980 con 0x7fd29c0b47b0 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c0b39f0 con 0x7fd29c0b47b0 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c006d40 con 0x7fd29c0b47b0 2023-10-06T03:02:42.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.894+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c0b3fb0 con 0x7fd29c0b47b0 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.895+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c0b44b0 con 0x7fd29c0b47b0 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.896+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd28c007240 con 0x7fd29c0b47b0 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.896+0000 7fd2917fa700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd280066f30 0x7fd2800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.896+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6956+0+0 (secure 0 0 0) 0x7fd28c084ad0 con 0x7fd29c0b47b0 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.896+0000 7fd2a172b700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd280066f30 0x7fd2800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.896+0000 7fd2a172b700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd280066f30 0x7fd2800693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd29c16ce10 tx=0x7fd2980091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:42.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:42.900+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd28c051860 con 0x7fd29c0b47b0 2023-10-06T03:02:43.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:43.112+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000092295", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fd29c04c3c0 con 0x7fd280066f30 2023-10-06T03:02:43.116 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:02:43.115+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:02:43.139 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:02:46.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.132+0000 7fd2917fa700 1 -- 172.21.15.92:0/3327053775 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7fd29c04c3c0 con 0x7fd280066f30 2023-10-06T03:02:46.133 INFO:teuthology.orchestra.run.smithi092.stdout:metadata pool: cephfs.volume_0000000000092295.meta data pool: ['cephfs.volume_0000000000092295.data'] removed 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd280066f30 msgr2=0x7fd2800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd280066f30 0x7fd2800693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd29c16ce10 tx=0x7fd2980091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 msgr2=0x7fd29c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 0x7fd29c0cda80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd28c002950 tx=0x7fd28c007d30 comp rx=0 tx=0).stop 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 shutdown_connections 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd280066f30 0x7fd2800693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd29c15ca40 0x7fd29c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd288004d50 0x7fd29c1495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 --2- 172.21.15.92:0/3327053775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd29c0b47b0 0x7fd29c0cda80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 >> 172.21.15.92:0/3327053775 conn(0x7fd29c068e20 msgr2=0x7fd29c154870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 shutdown_connections 2023-10-06T03:02:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.135+0000 7fd2a398f700 1 -- 172.21.15.92:0/3327053775 wait complete. 2023-10-06T03:02:46.150 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:02:46.150 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:02:46.197 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:02:46.242 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:02:46.242 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:02:46.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:02:46.242 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:02:46.288 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:02:46.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:46.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:02:46.304 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:02:46.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:02:46.304 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:02:46.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:02:46.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:02:46.347 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:02:46.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:46.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.519+0000 7f3d7f027700 1 Processor -- start 2023-10-06T03:02:46.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.519+0000 7f3d7f027700 1 -- start start 2023-10-06T03:02:46.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d7f027700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d7806bcd0 0x7f3d78140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d7f027700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d7806c0a0 0x7f3d78141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d7f027700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d780c8e00 0x7f3d780c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d7f027700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7815c2a0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d7f027700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7815b690 con 0x7f3d780c8e00 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d7f027700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7815b970 con 0x7f3d7806bcd0 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d6ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d7806c0a0 0x7f3d78141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d6ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d7806c0a0 0x7f3d78141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52538/0 (socket says 172.21.15.92:52538) 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.520+0000 7f3d6ffff700 1 -- 172.21.15.92:0/3757931204 learned_addr learned my addr 172.21.15.92:0/3757931204 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7cdc3700 1 --2- 172.21.15.92:0/3757931204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d7806bcd0 0x7f3d78140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7d5c4700 1 --2- 172.21.15.92:0/3757931204 >> v2:172.21.15.136:3300/0 conn(0x7f3d780c8e00 0x7f3d780c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6ffff700 1 -- 172.21.15.92:0/3757931204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d7806bcd0 msgr2=0x7f3d78140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6ffff700 1 --2- 172.21.15.92:0/3757931204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d7806bcd0 0x7f3d78140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6ffff700 1 -- 172.21.15.92:0/3757931204 >> v2:172.21.15.136:3300/0 conn(0x7f3d780c8e00 msgr2=0x7f3d780c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6ffff700 1 --2- 172.21.15.92:0/3757931204 >> v2:172.21.15.136:3300/0 conn(0x7f3d780c8e00 0x7f3d780c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6ffff700 1 -- 172.21.15.92:0/3757931204 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7815bc50 con 0x7f3d7806c0a0 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6ffff700 1 --2- 172.21.15.92:0/3757931204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d78141330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d7815b530 tx=0x7f3d68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ffe642ecd0a9ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6f7fe700 1 -- 172.21.15.92:0/3757931204 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800b860 con 0x7f3d7806c0a0 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6f7fe700 1 -- 172.21.15.92:0/3757931204 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6800b9c0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d6f7fe700 1 -- 172.21.15.92:0/3757931204 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800bd60 con 0x7f3d7806c0a0 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 -- 172.21.15.92:0/3757931204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 msgr2=0x7f3d78141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 --2- 172.21.15.92:0/3757931204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d78141330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d7815b530 tx=0x7f3d68009250 comp rx=0 tx=0).stop 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 -- 172.21.15.92:0/3757931204 shutdown_connections 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 --2- 172.21.15.92:0/3757931204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d7806bcd0 0x7f3d78140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 --2- 172.21.15.92:0/3757931204 >> v2:172.21.15.136:3300/0 conn(0x7f3d780c8e00 0x7f3d780c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 --2- 172.21.15.92:0/3757931204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d78141330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 -- 172.21.15.92:0/3757931204 >> 172.21.15.92:0/3757931204 conn(0x7f3d780ce600 msgr2=0x7f3d780cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:46.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 -- 172.21.15.92:0/3757931204 shutdown_connections 2023-10-06T03:02:46.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.521+0000 7f3d7f027700 1 -- 172.21.15.92:0/3757931204 wait complete. 2023-10-06T03:02:46.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.522+0000 7f3d7f027700 1 Processor -- start 2023-10-06T03:02:46.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.523+0000 7f3d7f027700 1 -- start start 2023-10-06T03:02:46.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7806bcd0 0x7f3d7815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d780c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d780c8e00 0x7f3d780c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780c61a0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b1440 con 0x7f3d7806bcd0 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d780b1720 con 0x7f3d780c8e00 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d780c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d780c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52546/0 (socket says 172.21.15.92:52546) 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 -- 172.21.15.92:0/1924042028 learned_addr learned my addr 172.21.15.92:0/1924042028 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7cdc3700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7806bcd0 0x7f3d7815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7d5c4700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d780c8e00 0x7f3d780c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 -- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d780c8e00 msgr2=0x7f3d780c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d780c8e00 0x7f3d780c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 -- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7806bcd0 msgr2=0x7f3d7815a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7806bcd0 0x7f3d7815a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 -- 172.21.15.92:0/1924042028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d680094f0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6ffff700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d780c5720 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d78069220 tx=0x7f3d68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68011dc0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d68024440 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d780b1a00 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d68024800 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.524+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d780b1fc0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.525+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7804c3c0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.526+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d6801a420 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.526+0000 7f3d6cff9700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.526+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f3d680864f0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.526+0000 7f3d7cdc3700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.526+0000 7f3d7cdc3700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d64066f30 0x7f3d640693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009fd0 tx=0x7f3d74009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:46.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.530+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d680535b0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.758+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7806c9d0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.759+0000 7f3d6cff9700 1 -- 172.21.15.92:0/1924042028 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f3d68056bd0 con 0x7f3d7806c0a0 2023-10-06T03:02:46.760 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:46.760 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1384819232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1384819232},{"type":"v1","addr":"172.21.15.92:6836","nonce":1384819232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:27.963102+0000","modified":"2023-10-06T03:02:31.599832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2171492021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2171492021},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171492021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:02:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.760+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d64066f30 msgr2=0x7f3d640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.760+0000 7f3d7f027700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d64066f30 0x7f3d640693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009fd0 tx=0x7f3d74009880 comp rx=0 tx=0).stop 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 msgr2=0x7f3d780c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d780c5720 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d78069220 tx=0x7f3d68017000 comp rx=0 tx=0).stop 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 shutdown_connections 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d64066f30 0x7f3d640693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d780c8e00 0x7f3d780c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7806bcd0 0x7f3d7815a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 --2- 172.21.15.92:0/1924042028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d7806c0a0 0x7f3d780c5720 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 >> 172.21.15.92:0/1924042028 conn(0x7f3d780ce600 msgr2=0x7f3d780cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 shutdown_connections 2023-10-06T03:02:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.761+0000 7f3d7f027700 1 -- 172.21.15.92:0/1924042028 wait complete. 2023-10-06T03:02:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2023-10-06T03:02:46.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:46.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.885+0000 7f2d2adab700 1 Processor -- start 2023-10-06T03:02:46.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.885+0000 7f2d2adab700 1 -- start start 2023-10-06T03:02:46.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d2adab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240cbf30 0x7f2d240ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d2adab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d240ccf90 0x7f2d240b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d2adab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d240cc300 con 0x7f2d2415a270 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d2adab700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d24156bf0 con 0x7f2d240ccf90 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d2adab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240ae950 con 0x7f2d240cbf30 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d1bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2d240ccf90 0x7f2d240b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.886+0000 7f2d29348700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d2415a270 0x7f2d240c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50754/0 (socket says 172.21.15.92:50754) 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d29348700 1 -- 172.21.15.92:0/3081187614 learned_addr learned my addr 172.21.15.92:0/3081187614 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d28b47700 1 --2- 172.21.15.92:0/3081187614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240cbf30 0x7f2d240ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1bfff700 1 -- 172.21.15.92:0/3081187614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240cbf30 msgr2=0x7f2d240ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1bfff700 1 --2- 172.21.15.92:0/3081187614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240cbf30 0x7f2d240ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1bfff700 1 -- 172.21.15.92:0/3081187614 >> v1:172.21.15.92:6789/0 conn(0x7f2d2415a270 legacy=0x7f2d240c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1bfff700 1 -- 172.21.15.92:0/3081187614 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d240ad370 con 0x7f2d240ccf90 2023-10-06T03:02:46.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1b7fe700 1 -- 172.21.15.92:0/3081187614 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803125130 0 0) 0x7f2d240cc300 con 0x7f2d2415a270 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1bfff700 1 --2- 172.21.15.92:0/3081187614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240ccf90 0x7f2d240b0d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d13917aeb1a5e08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1b7fe700 1 -- 172.21.15.92:0/3081187614 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400b820 con 0x7f2d240ccf90 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1b7fe700 1 -- 172.21.15.92:0/3081187614 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1400b980 con 0x7f2d240ccf90 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.887+0000 7f2d1b7fe700 1 -- 172.21.15.92:0/3081187614 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400bd20 con 0x7f2d240ccf90 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 -- 172.21.15.92:0/3081187614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240ccf90 msgr2=0x7f2d240b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 --2- 172.21.15.92:0/3081187614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240ccf90 0x7f2d240b0d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009910 tx=0x7f2d140091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 -- 172.21.15.92:0/3081187614 shutdown_connections 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 --2- 172.21.15.92:0/3081187614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240cbf30 0x7f2d240ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 --2- 172.21.15.92:0/3081187614 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240ccf90 0x7f2d240b0d10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 -- 172.21.15.92:0/3081187614 >> 172.21.15.92:0/3081187614 conn(0x7f2d2406a320 msgr2=0x7f2d240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:46.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 -- 172.21.15.92:0/3081187614 shutdown_connections 2023-10-06T03:02:46.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.888+0000 7f2d2adab700 1 -- 172.21.15.92:0/3081187614 wait complete. 2023-10-06T03:02:46.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.889+0000 7f2d2adab700 1 Processor -- start 2023-10-06T03:02:46.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.889+0000 7f2d2adab700 1 -- start start 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 0x7f2d240c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240ccf90 0x7f2d241594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2415a270 0x7f2d241490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240c7d40 con 0x7f2d240cbf30 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240c7130 con 0x7f2d2415a270 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d240c7410 con 0x7f2d240ccf90 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 0x7f2d240c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 0x7f2d240c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52562/0 (socket says 172.21.15.92:52562) 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 -- 172.21.15.92:0/1712391408 learned_addr learned my addr 172.21.15.92:0/1712391408 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d1bfff700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240ccf90 0x7f2d241594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d29348700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2415a270 0x7f2d241490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 -- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240ccf90 msgr2=0x7f2d241594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240ccf90 0x7f2d241594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 -- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2415a270 msgr2=0x7f2d241490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2415a270 0x7f2d241490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 -- 172.21.15.92:0/1712391408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d14009420 con 0x7f2d240cbf30 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d1bfff700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240ccf90 0x7f2d241594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d28b47700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 0x7f2d240c5ac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e190 tx=0x7f2d20009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000ee30 con 0x7f2d240cbf30 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20014420 con 0x7f2d240cbf30 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d240c76f0 con 0x7f2d240cbf30 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200147e0 con 0x7f2d240cbf30 2023-10-06T03:02:46.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.890+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d240c6000 con 0x7f2d240cbf30 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.891+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2404c3c0 con 0x7f2d240cbf30 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.892+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d20014b90 con 0x7f2d240cbf30 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.892+0000 7f2d18ff9700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d10066f30 0x7f2d100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.892+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f2d20084f30 con 0x7f2d240cbf30 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.892+0000 7f2d1bfff700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d10066f30 0x7f2d100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.892+0000 7f2d1bfff700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d10066f30 0x7f2d100693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009860 tx=0x7f2d14017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:46.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:46.896+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d20051ff0 con 0x7f2d240cbf30 2023-10-06T03:02:47.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.081+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d240c8020 con 0x7f2d240cbf30 2023-10-06T03:02:47.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.082+0000 7f2d18ff9700 1 -- 172.21.15.92:0/1712391408 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f2d20009a60 con 0x7f2d240cbf30 2023-10-06T03:02:47.083 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:47.083 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:46.124159+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/971383855":"2023-10-07T03:02:43.116026+0000","172.21.15.92:6834/971383855":"2023-10-07T03:02:43.116026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d10066f30 msgr2=0x7f2d100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d10066f30 0x7f2d100693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009860 tx=0x7f2d14017000 comp rx=0 tx=0).stop 2023-10-06T03:02:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 msgr2=0x7f2d240c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 0x7f2d240c5ac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000e190 tx=0x7f2d20009920 comp rx=0 tx=0).stop 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 shutdown_connections 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d10066f30 0x7f2d100693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d240cbf30 0x7f2d240c5ac0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2415a270 0x7f2d241490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 --2- 172.21.15.92:0/1712391408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d240ccf90 0x7f2d241594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 >> 172.21.15.92:0/1712391408 conn(0x7f2d2406a320 msgr2=0x7f2d241551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 shutdown_connections 2023-10-06T03:02:47.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.084+0000 7f2d2adab700 1 -- 172.21.15.92:0/1712391408 wait complete. 2023-10-06T03:02:47.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:02:26.035987+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:02:26.917916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:02:47.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:47.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.214+0000 7f574ce53700 1 Processor -- start 2023-10-06T03:02:47.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.214+0000 7f574ce53700 1 -- start start 2023-10-06T03:02:47.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f574ce53700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 0x7f57480caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f574ce53700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f57480c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f574ce53700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57480af490 con 0x7f57480c93a0 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f574ce53700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57480af770 con 0x7f57480c9770 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f574ce53700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57480afa50 con 0x7f57480cb400 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5747fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f57480c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f573effd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 0x7f57480caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f573effd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 0x7f57480caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:51264/0 (socket says 172.21.15.92:51264) 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f57477fe700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f57480c93a0 0x7f574806c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:54128/0 (socket says 172.21.15.92:54128) 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5747fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f57480c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:39790/0 (socket says 172.21.15.92:39790) 2023-10-06T03:02:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f573effd700 1 -- 172.21.15.92:0/2269115583 learned_addr learned my addr 172.21.15.92:0/2269115583 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5746ffd700 1 -- 172.21.15.92:0/2269115583 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048104091 0 0) 0x7f57480af490 con 0x7f57480c93a0 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5746ffd700 1 -- 172.21.15.92:0/2269115583 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5734003570 con 0x7f57480c93a0 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5747fff700 1 -- 172.21.15.92:0/2269115583 >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 msgr2=0x7f57480caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5747fff700 1 --2- 172.21.15.92:0/2269115583 >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 0x7f57480caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5747fff700 1 -- 172.21.15.92:0/2269115583 >> v1:172.21.15.136:6789/0 conn(0x7f57480c93a0 legacy=0x7f574806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f5747fff700 1 -- 172.21.15.92:0/2269115583 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57480c7ab0 con 0x7f57480cb400 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.215+0000 7f573effd700 1 --2- 172.21.15.92:0/2269115583 >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 0x7f57480caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f5747fff700 1 --2- 172.21.15.92:0/2269115583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f57480c7fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5740009e60 tx=0x7f5740009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ef8de862e0cc16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f5746ffd700 1 -- 172.21.15.92:0/2269115583 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000e070 con 0x7f57480cb400 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f5746ffd700 1 -- 172.21.15.92:0/2269115583 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574000bab0 con 0x7f57480cb400 2023-10-06T03:02:47.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f5746ffd700 1 -- 172.21.15.92:0/2269115583 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000be70 con 0x7f57480cb400 2023-10-06T03:02:47.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 -- 172.21.15.92:0/2269115583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 msgr2=0x7f57480c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 --2- 172.21.15.92:0/2269115583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f57480c7fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5740009e60 tx=0x7f5740009780 comp rx=0 tx=0).stop 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 -- 172.21.15.92:0/2269115583 shutdown_connections 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 --2- 172.21.15.92:0/2269115583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f57480c7fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 --2- 172.21.15.92:0/2269115583 >> v2:172.21.15.136:3300/0 conn(0x7f57480c9770 0x7f57480caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 -- 172.21.15.92:0/2269115583 >> 172.21.15.92:0/2269115583 conn(0x7f57480cddc0 msgr2=0x7f57480c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 -- 172.21.15.92:0/2269115583 shutdown_connections 2023-10-06T03:02:47.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.216+0000 7f574ce53700 1 -- 172.21.15.92:0/2269115583 wait complete. 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.217+0000 7f574ce53700 1 Processor -- start 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 -- start start 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57480c93a0 0x7f57480c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 0x7f57480c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f5748151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480c5860 con 0x7f57480c9770 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57480c7120 con 0x7f57480c93a0 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f574ce53700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57480c4c50 con 0x7f57480cb400 2023-10-06T03:02:47.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f573effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 0x7f57480c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f573effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 0x7f57480c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52570/0 (socket says 172.21.15.92:52570) 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.218+0000 7f573effd700 1 -- 172.21.15.92:0/1458731996 learned_addr learned my addr 172.21.15.92:0/1458731996 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f5747fff700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f5748151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f57477fe700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57480c93a0 0x7f57480c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573effd700 1 -- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 msgr2=0x7f5748151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573effd700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f5748151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573effd700 1 -- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57480c93a0 msgr2=0x7f57480c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573effd700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57480c93a0 0x7f57480c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573effd700 1 -- 172.21.15.92:0/1458731996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5730009480 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573effd700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 0x7f57480c2a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f573000ec80 tx=0x7f5730009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730005a00 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730005b60 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5740009a20 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000c850 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.219+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57480c5210 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.220+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574806d560 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.220+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f573000cc00 con 0x7f57480c9770 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.221+0000 7f573ffff700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5728066f30 0x7f57280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.221+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5730084c40 con 0x7f57480c9770 2023-10-06T03:02:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.221+0000 7f57477fe700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5728066f30 0x7f57280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.221+0000 7f57477fe700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5728066f30 0x7f57280693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5738004f70 tx=0x7f5738007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.225+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5730051d00 con 0x7f57480c9770 2023-10-06T03:02:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.409+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f574806b500 con 0x7f57480c9770 2023-10-06T03:02:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.410+0000 7f573ffff700 1 -- 172.21.15.92:0/1458731996 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5730020020 con 0x7f57480c9770 2023-10-06T03:02:47.411 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5728066f30 msgr2=0x7f57280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5728066f30 0x7f57280693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5738004f70 tx=0x7f5738007d40 comp rx=0 tx=0).stop 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 msgr2=0x7f57480c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 0x7f57480c2a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f573000ec80 tx=0x7f5730009220 comp rx=0 tx=0).stop 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 shutdown_connections 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5728066f30 0x7f57280693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57480cb400 0x7f5748151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57480c93a0 0x7f57480c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 --2- 172.21.15.92:0/1458731996 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57480c9770 0x7f57480c2a10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 >> 172.21.15.92:0/1458731996 conn(0x7f57480cddc0 msgr2=0x7f57481548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 shutdown_connections 2023-10-06T03:02:47.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.412+0000 7f574ce53700 1 -- 172.21.15.92:0/1458731996 wait complete. 2023-10-06T03:02:47.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.542+0000 7f229359e700 1 Processor -- start 2023-10-06T03:02:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.542+0000 7f229359e700 1 -- start start 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b16d0 0x7f22940b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229359e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f22940b33b0 0x7f22940b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229359e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22940b2080 con 0x7f22940b4e70 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229359e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22940cfb00 con 0x7f22940b33b0 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22940b5c40 con 0x7f22940b16d0 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f2292d9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f22940b33b0 0x7f22940b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f228bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b16d0 0x7f22940b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229259c700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f22940b4e70 0x7f22940ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:54144/0 (socket says 172.21.15.92:54144) 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f2292d9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f22940b33b0 0x7f22940b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:51284/0 (socket says 172.21.15.92:51284) 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.543+0000 7f229259c700 1 -- 172.21.15.92:0/2200295211 learned_addr learned my addr 172.21.15.92:0/2200295211 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f2291d9b700 1 -- 172.21.15.92:0/2200295211 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314623021 0 0) 0x7f22940b2080 con 0x7f22940b4e70 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f2291d9b700 1 -- 172.21.15.92:0/2200295211 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2280003570 con 0x7f22940b4e70 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f228bfff700 1 -- 172.21.15.92:0/2200295211 >> v2:172.21.15.136:3300/0 conn(0x7f22940b33b0 msgr2=0x7f22940b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f228bfff700 1 --2- 172.21.15.92:0/2200295211 >> v2:172.21.15.136:3300/0 conn(0x7f22940b33b0 0x7f22940b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f228bfff700 1 -- 172.21.15.92:0/2200295211 >> v1:172.21.15.136:6789/0 conn(0x7f22940b4e70 legacy=0x7f22940ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f228bfff700 1 -- 172.21.15.92:0/2200295211 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940b5f20 con 0x7f22940b16d0 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f228bfff700 1 --2- 172.21.15.92:0/2200295211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b16d0 0x7f22940b1b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f227c0098e0 tx=0x7f227c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a046ff80a9474e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f2291d9b700 1 -- 172.21.15.92:0/2200295211 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c011630 con 0x7f22940b16d0 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f2291d9b700 1 -- 172.21.15.92:0/2200295211 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c011790 con 0x7f22940b16d0 2023-10-06T03:02:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.544+0000 7f2291d9b700 1 -- 172.21.15.92:0/2200295211 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c011b30 con 0x7f22940b16d0 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 -- 172.21.15.92:0/2200295211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b16d0 msgr2=0x7f22940b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 --2- 172.21.15.92:0/2200295211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b16d0 0x7f22940b1b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f227c0098e0 tx=0x7f227c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 -- 172.21.15.92:0/2200295211 shutdown_connections 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 --2- 172.21.15.92:0/2200295211 >> v2:172.21.15.136:3300/0 conn(0x7f22940b33b0 0x7f22940b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 --2- 172.21.15.92:0/2200295211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b16d0 0x7f22940b1b40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 -- 172.21.15.92:0/2200295211 >> 172.21.15.92:0/2200295211 conn(0x7f22940d10c0 msgr2=0x7f22940affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 -- 172.21.15.92:0/2200295211 shutdown_connections 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.545+0000 7f229359e700 1 -- 172.21.15.92:0/2200295211 wait complete. 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 Processor -- start 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 -- start start 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22940b16d0 0x7f2294158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b33b0 0x7f2294169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 0x7f229415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22941586d0 con 0x7f22940b4e70 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22940cab10 con 0x7f22940b16d0 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.546+0000 7f229359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22940cadf0 con 0x7f22940b33b0 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 0x7f229415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 0x7f229415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52584/0 (socket says 172.21.15.92:52584) 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 -- 172.21.15.92:0/1117088941 learned_addr learned my addr 172.21.15.92:0/1117088941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f228bfff700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b33b0 0x7f2294169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f229259c700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22940b16d0 0x7f2294158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 -- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b33b0 msgr2=0x7f2294169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b33b0 0x7f2294169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 -- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22940b16d0 msgr2=0x7f2294158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22940b16d0 0x7f2294158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 -- 172.21.15.92:0/1117088941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009420 con 0x7f22940b4e70 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f2292d9d700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 0x7f229415d4c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f228c009e90 tx=0x7f228c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c00cb70 con 0x7f22940b4e70 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c00f040 con 0x7f22940b4e70 2023-10-06T03:02:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22940cb0d0 con 0x7f22940b4e70 2023-10-06T03:02:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c014480 con 0x7f22940b4e70 2023-10-06T03:02:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.547+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22940cb690 con 0x7f22940b4e70 2023-10-06T03:02:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.548+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22940cb880 con 0x7f22940b4e70 2023-10-06T03:02:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.549+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f228c014830 con 0x7f22940b4e70 2023-10-06T03:02:47.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.549+0000 7f228a7fc700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2274066f30 0x7f22740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.549+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f228c085160 con 0x7f22940b4e70 2023-10-06T03:02:47.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.549+0000 7f229259c700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2274066f30 0x7f22740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.549+0000 7f229259c700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2274066f30 0x7f22740693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2284007fd0 tx=0x7f2284009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.553+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f228c052220 con 0x7f22940b4e70 2023-10-06T03:02:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.736+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f229404bb80 con 0x7f22940b4e70 2023-10-06T03:02:47.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.736+0000 7f228a7fc700 1 -- 172.21.15.92:0/1117088941 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f228c055840 con 0x7f22940b4e70 2023-10-06T03:02:47.737 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2274066f30 msgr2=0x7f22740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2274066f30 0x7f22740693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2284007fd0 tx=0x7f2284009040 comp rx=0 tx=0).stop 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 msgr2=0x7f229415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 0x7f229415d4c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f228c009e90 tx=0x7f228c00b040 comp rx=0 tx=0).stop 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 shutdown_connections 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2274066f30 0x7f22740693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f22940b4e70 0x7f229415d4c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22940b16d0 0x7f2294158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 --2- 172.21.15.92:0/1117088941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22940b33b0 0x7f2294169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 >> 172.21.15.92:0/1117088941 conn(0x7f22940d10c0 msgr2=0x7f2294069bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 shutdown_connections 2023-10-06T03:02:47.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.738+0000 7f229359e700 1 -- 172.21.15.92:0/1117088941 wait complete. 2023-10-06T03:02:47.750 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:02:47.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 Processor -- start 2023-10-06T03:02:47.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 -- start start 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc9d80af250 0x7fc9d8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9d80688d0 0x7fc9d8157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d806b030 con 0x7fc9d8157960 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d80ac760 con 0x7fc9d80688d0 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9ddcbf700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d80af830 con 0x7fc9d80af250 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.867+0000 7fc9d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9d80688d0 0x7fc9d8157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc9d80688d0 0x7fc9d8157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52596/0 (socket says 172.21.15.92:52596) 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 -- 172.21.15.92:0/2200402002 learned_addr learned my addr 172.21.15.92:0/2200402002 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d6ffd700 1 --2- 172.21.15.92:0/2200402002 >> v2:172.21.15.136:3300/0 conn(0x7fc9d80af250 0x7fc9d8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 -- 172.21.15.92:0/2200402002 >> v2:172.21.15.136:3300/0 conn(0x7fc9d80af250 msgr2=0x7fc9d8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 --2- 172.21.15.92:0/2200402002 >> v2:172.21.15.136:3300/0 conn(0x7fc9d80af250 0x7fc9d8068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 -- 172.21.15.92:0/2200402002 >> v1:172.21.15.136:6789/0 conn(0x7fc9d8157960 legacy=0x7fc9d806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 -- 172.21.15.92:0/2200402002 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d806bbe0 con 0x7fc9d80688d0 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d6ffd700 1 --2- 172.21.15.92:0/2200402002 >> v2:172.21.15.136:3300/0 conn(0x7fc9d80af250 0x7fc9d8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d7fff700 1 --2- 172.21.15.92:0/2200402002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80688d0 0x7fc9d8157d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9d806fe40 tx=0x7fc9c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b42fc67289cdb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d67fc700 1 -- 172.21.15.92:0/2200402002 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800b860 con 0x7fc9d80688d0 2023-10-06T03:02:47.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d67fc700 1 -- 172.21.15.92:0/2200402002 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c800b9c0 con 0x7fc9d80688d0 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.868+0000 7fc9d67fc700 1 -- 172.21.15.92:0/2200402002 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800bd60 con 0x7fc9d80688d0 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/2200402002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80688d0 msgr2=0x7fc9d8157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/2200402002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80688d0 0x7fc9d8157d30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9d806fe40 tx=0x7fc9c8009250 comp rx=0 tx=0).stop 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/2200402002 shutdown_connections 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/2200402002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80688d0 0x7fc9d8157d30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/2200402002 >> v2:172.21.15.136:3300/0 conn(0x7fc9d80af250 0x7fc9d8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/2200402002 >> 172.21.15.92:0/2200402002 conn(0x7fc9d80c9870 msgr2=0x7fc9d80c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/2200402002 shutdown_connections 2023-10-06T03:02:47.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.869+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/2200402002 wait complete. 2023-10-06T03:02:47.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.870+0000 7fc9ddcbf700 1 Processor -- start 2023-10-06T03:02:47.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.870+0000 7fc9ddcbf700 1 -- start start 2023-10-06T03:02:47.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9d80688d0 0x7fc9d814d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 0x7fc9d80ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d8157960 0x7fc9d81552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d815bc90 con 0x7fc9d80af250 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80acde0 con 0x7fc9d8157960 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d815b3a0 con 0x7fc9d80688d0 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 0x7fc9d80ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 0x7fc9d80ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52606/0 (socket says 172.21.15.92:52606) 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 -- 172.21.15.92:0/3960252408 learned_addr learned my addr 172.21.15.92:0/3960252408 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 -- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9d80688d0 msgr2=0x7fc9d814d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d77fe700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9d80688d0 0x7fc9d814d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9d80688d0 0x7fc9d814d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 -- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d8157960 msgr2=0x7fc9d81552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d8157960 0x7fc9d81552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:47.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 -- 172.21.15.92:0/3960252408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80094f0 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9d6ffd700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 0x7fc9d80ac8a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e900 tx=0x7fc9c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000eea0 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000c5e0 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d8155800 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000c980 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.871+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d815b920 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.872+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d815be40 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.873+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9c000cd30 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.873+0000 7fc9cf7fe700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc9b8066f30 0x7fc9b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.873+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc9c0084530 con 0x7fc9d80af250 2023-10-06T03:02:47.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.873+0000 7fc9d77fe700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc9b8066f30 0x7fc9b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:47.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.873+0000 7fc9d77fe700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc9b8066f30 0x7fc9b80693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007fd0 tx=0x7fc9d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:47.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:47.877+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9d815be40 con 0x7fc9d80af250 2023-10-06T03:02:48.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.057+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc9d815be40 con 0x7fc9d80af250 2023-10-06T03:02:48.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.057+0000 7fc9cf7fe700 1 -- 172.21.15.92:0/3960252408 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc9d815be40 con 0x7fc9d80af250 2023-10-06T03:02:48.058 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:02:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc9b8066f30 msgr2=0x7fc9b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc9b8066f30 0x7fc9b80693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007fd0 tx=0x7fc9d0009040 comp rx=0 tx=0).stop 2023-10-06T03:02:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 msgr2=0x7fc9d80ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 0x7fc9d80ac8a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e900 tx=0x7fc9c0008040 comp rx=0 tx=0).stop 2023-10-06T03:02:48.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 shutdown_connections 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc9b8066f30 0x7fc9b80693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9d8157960 0x7fc9d81552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9d80688d0 0x7fc9d814d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 --2- 172.21.15.92:0/3960252408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc9d80af250 0x7fc9d80ac8a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 >> 172.21.15.92:0/3960252408 conn(0x7fc9d80c9870 msgr2=0x7fc9d814fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 shutdown_connections 2023-10-06T03:02:48.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.059+0000 7fc9ddcbf700 1 -- 172.21.15.92:0/3960252408 wait complete. 2023-10-06T03:02:48.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:02:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.189+0000 7f83eec3c700 1 Processor -- start 2023-10-06T03:02:48.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.189+0000 7f83eec3c700 1 -- start start 2023-10-06T03:02:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83eec3c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e806bcd0 0x7f83e80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83eec3c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83e806c0a0 0x7f83e815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83eec3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83e815df50 0x7f83e815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83eec3c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e815d7e0 con 0x7f83e815df50 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83eec3c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e815dac0 con 0x7f83e806c0a0 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83eec3c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e815aa30 con 0x7f83e806bcd0 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83ed1d9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83e815df50 0x7f83e815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83ed1d9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83e815df50 0x7f83e815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52622/0 (socket says 172.21.15.92:52622) 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.190+0000 7f83ed1d9700 1 -- 172.21.15.92:0/3871400361 learned_addr learned my addr 172.21.15.92:0/3871400361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ec9d8700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e806bcd0 0x7f83e80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83dffff700 1 --2- 172.21.15.92:0/3871400361 >> v2:172.21.15.136:3300/0 conn(0x7f83e806c0a0 0x7f83e815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ed1d9700 1 -- 172.21.15.92:0/3871400361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e806bcd0 msgr2=0x7f83e80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ed1d9700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e806bcd0 0x7f83e80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ed1d9700 1 -- 172.21.15.92:0/3871400361 >> v2:172.21.15.136:3300/0 conn(0x7f83e806c0a0 msgr2=0x7f83e815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ed1d9700 1 --2- 172.21.15.92:0/3871400361 >> v2:172.21.15.136:3300/0 conn(0x7f83e806c0a0 0x7f83e815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ed1d9700 1 -- 172.21.15.92:0/3871400361 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e815ad10 con 0x7f83e815df50 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ec9d8700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e806bcd0 0x7f83e80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83ed1d9700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e815df50 0x7f83e815e3b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f83e80c5660 tx=0x7f83e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a8abffaac839c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83df7fe700 1 -- 172.21.15.92:0/3871400361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000b860 con 0x7f83e815df50 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83df7fe700 1 -- 172.21.15.92:0/3871400361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e000b9c0 con 0x7f83e815df50 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.191+0000 7f83df7fe700 1 -- 172.21.15.92:0/3871400361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000bd80 con 0x7f83e815df50 2023-10-06T03:02:48.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 -- 172.21.15.92:0/3871400361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e815df50 msgr2=0x7f83e815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e815df50 0x7f83e815e3b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f83e80c5660 tx=0x7f83e0009250 comp rx=0 tx=0).stop 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 -- 172.21.15.92:0/3871400361 shutdown_connections 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e806bcd0 0x7f83e80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 --2- 172.21.15.92:0/3871400361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e815df50 0x7f83e815e3b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 --2- 172.21.15.92:0/3871400361 >> v2:172.21.15.136:3300/0 conn(0x7f83e806c0a0 0x7f83e815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 -- 172.21.15.92:0/3871400361 >> 172.21.15.92:0/3871400361 conn(0x7f83e80ce600 msgr2=0x7f83e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 -- 172.21.15.92:0/3871400361 shutdown_connections 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.192+0000 7f83eec3c700 1 -- 172.21.15.92:0/3871400361 wait complete. 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.193+0000 7f83eec3c700 1 Processor -- start 2023-10-06T03:02:48.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.193+0000 7f83eec3c700 1 -- start start 2023-10-06T03:02:48.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 0x7f83e8159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83e806c0a0 0x7f83e816da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e815df50 0x7f83e8153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80c7e00 con 0x7f83e806bcd0 2023-10-06T03:02:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80b2c90 con 0x7f83e806c0a0 2023-10-06T03:02:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80b2640 con 0x7f83e815df50 2023-10-06T03:02:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 0x7f83e8159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 0x7f83e8159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52638/0 (socket says 172.21.15.92:52638) 2023-10-06T03:02:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 -- 172.21.15.92:0/111903414 learned_addr learned my addr 172.21.15.92:0/111903414 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:48.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83dffff700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83e806c0a0 0x7f83e816da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 -- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e815df50 msgr2=0x7f83e8153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e815df50 0x7f83e8153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 -- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83e806c0a0 msgr2=0x7f83e816da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83e806c0a0 0x7f83e816da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 -- 172.21.15.92:0/111903414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00094f0 con 0x7f83e806bcd0 2023-10-06T03:02:48.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83ec9d8700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 0x7f83e8159f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d180 tx=0x7f83e40099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400de20 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400f040 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e80c7560 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4015670 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.194+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e80c7b20 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.195+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e80c97a0 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.196+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83e4015a60 con 0x7f83e806bcd0 2023-10-06T03:02:48.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.196+0000 7f83dcff9700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d4066f30 0x7f83d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.196+0000 7f83dffff700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d4066f30 0x7f83d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.196+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f83e4084b10 con 0x7f83e806bcd0 2023-10-06T03:02:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.197+0000 7f83dffff700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d4066f30 0x7f83d40693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b0ef0 tx=0x7f83d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.200+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83e404e5a0 con 0x7f83e806bcd0 2023-10-06T03:02:48.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.383+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83e804c3c0 con 0x7f83e806bcd0 2023-10-06T03:02:48.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.383+0000 7f83dcff9700 1 -- 172.21.15.92:0/111903414 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83e401a0a0 con 0x7f83e806bcd0 2023-10-06T03:02:48.384 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:02:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d4066f30 msgr2=0x7f83d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d4066f30 0x7f83d40693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b0ef0 tx=0x7f83d8008040 comp rx=0 tx=0).stop 2023-10-06T03:02:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 msgr2=0x7f83e8159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 0x7f83e8159f20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d180 tx=0x7f83e40099f0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 shutdown_connections 2023-10-06T03:02:48.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d4066f30 0x7f83d40693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83e806bcd0 0x7f83e8159f20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83e815df50 0x7f83e8153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 --2- 172.21.15.92:0/111903414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83e806c0a0 0x7f83e816da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 >> 172.21.15.92:0/111903414 conn(0x7f83e80ce600 msgr2=0x7f83e8154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 shutdown_connections 2023-10-06T03:02:48.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.385+0000 7f83eec3c700 1 -- 172.21.15.92:0/111903414 wait complete. 2023-10-06T03:02:48.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:48.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.512+0000 7f93bb20a700 1 Processor -- start 2023-10-06T03:02:48.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.512+0000 7f93bb20a700 1 -- start start 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93bb20a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40cbf30 0x7f93b40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93bb20a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93b40ccf90 0x7f93b40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93bb20a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b40cc300 con 0x7f93b415a270 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93bb20a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b4156bf0 con 0x7f93b40ccf90 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93bb20a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40ae950 con 0x7f93b40cbf30 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93abfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93b40ccf90 0x7f93b40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93abfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93b40ccf90 0x7f93b40b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52646/0 (socket says 172.21.15.92:52646) 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93abfff700 1 -- 172.21.15.92:0/972102222 learned_addr learned my addr 172.21.15.92:0/972102222 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.513+0000 7f93b8fa6700 1 --2- 172.21.15.92:0/972102222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40cbf30 0x7f93b40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93abfff700 1 -- 172.21.15.92:0/972102222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40cbf30 msgr2=0x7f93b40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93abfff700 1 --2- 172.21.15.92:0/972102222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40cbf30 0x7f93b40ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93abfff700 1 -- 172.21.15.92:0/972102222 >> v1:172.21.15.136:6789/0 conn(0x7f93b415a270 legacy=0x7f93b40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93abfff700 1 -- 172.21.15.92:0/972102222 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40ad370 con 0x7f93b40ccf90 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93abfff700 1 --2- 172.21.15.92:0/972102222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40ccf90 0x7f93b40b0d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93a40099e0 tx=0x7f93a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618bc6a7b910cf38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93ab7fe700 1 -- 172.21.15.92:0/972102222 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400b820 con 0x7f93b40ccf90 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93ab7fe700 1 -- 172.21.15.92:0/972102222 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a400b980 con 0x7f93b40ccf90 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93ab7fe700 1 -- 172.21.15.92:0/972102222 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a400bd20 con 0x7f93b40ccf90 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93bb20a700 1 -- 172.21.15.92:0/972102222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40ccf90 msgr2=0x7f93b40b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93bb20a700 1 --2- 172.21.15.92:0/972102222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40ccf90 0x7f93b40b0d10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93a40099e0 tx=0x7f93a4009250 comp rx=0 tx=0).stop 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93bb20a700 1 -- 172.21.15.92:0/972102222 shutdown_connections 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93bb20a700 1 --2- 172.21.15.92:0/972102222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40cbf30 0x7f93b40ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93bb20a700 1 --2- 172.21.15.92:0/972102222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40ccf90 0x7f93b40b0d10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.514+0000 7f93bb20a700 1 -- 172.21.15.92:0/972102222 >> 172.21.15.92:0/972102222 conn(0x7f93b406a320 msgr2=0x7f93b40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:48.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.515+0000 7f93bb20a700 1 -- 172.21.15.92:0/972102222 shutdown_connections 2023-10-06T03:02:48.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.515+0000 7f93bb20a700 1 -- 172.21.15.92:0/972102222 wait complete. 2023-10-06T03:02:48.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.516+0000 7f93bb20a700 1 Processor -- start 2023-10-06T03:02:48.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.516+0000 7f93bb20a700 1 -- start start 2023-10-06T03:02:48.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 0x7f93b4148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40ccf90 0x7f93b40a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b415a270 0x7f93b40aa270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40a75c0 con 0x7f93b40cbf30 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40a78a0 con 0x7f93b415a270 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b40a7b80 con 0x7f93b40ccf90 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 0x7f93b4148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 0x7f93b4148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52648/0 (socket says 172.21.15.92:52648) 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 -- 172.21.15.92:0/2468762396 learned_addr learned my addr 172.21.15.92:0/2468762396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93abfff700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40ccf90 0x7f93b40a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b97a7700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b415a270 0x7f93b40aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 -- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40ccf90 msgr2=0x7f93b40a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40ccf90 0x7f93b40a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 -- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b415a270 msgr2=0x7f93b40aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b415a270 0x7f93b40aa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 -- 172.21.15.92:0/2468762396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a40094f0 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b97a7700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b415a270 0x7f93b40aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93b8fa6700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 0x7f93b4148f90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009e30 tx=0x7f93b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000c850 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b000c9b0 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b40a7e60 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000cd50 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.517+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b41494d0 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.518+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b40a8300 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.519+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f93b0011660 con 0x7f93b40cbf30 2023-10-06T03:02:48.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.519+0000 7f93a8ff9700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a0066f30 0x7f93a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.519+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f93b0086600 con 0x7f93b40cbf30 2023-10-06T03:02:48.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.519+0000 7f93abfff700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a0066f30 0x7f93a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.520+0000 7f93abfff700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a0066f30 0x7f93a00693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f93a40099e0 tx=0x7f93a4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.523+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b00536c0 con 0x7f93b40cbf30 2023-10-06T03:02:48.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.749+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93b40a84b0 con 0x7f93b40cbf30 2023-10-06T03:02:48.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.749+0000 7f93a8ff9700 1 -- 172.21.15.92:0/2468762396 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f93b0056ce0 con 0x7f93b40cbf30 2023-10-06T03:02:48.750 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:48.750 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1384819232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1384819232},{"type":"v1","addr":"172.21.15.92:6836","nonce":1384819232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:27.963102+0000","modified":"2023-10-06T03:02:31.599832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2171492021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2171492021},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171492021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:02:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a0066f30 msgr2=0x7f93a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a0066f30 0x7f93a00693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f93a40099e0 tx=0x7f93a4017000 comp rx=0 tx=0).stop 2023-10-06T03:02:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 msgr2=0x7f93b4148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 0x7f93b4148f90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009e30 tx=0x7f93b00091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 shutdown_connections 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a0066f30 0x7f93a00693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93b40cbf30 0x7f93b4148f90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b415a270 0x7f93b40aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 --2- 172.21.15.92:0/2468762396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b40ccf90 0x7f93b40a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 >> 172.21.15.92:0/2468762396 conn(0x7f93b406a320 msgr2=0x7f93b406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 shutdown_connections 2023-10-06T03:02:48.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.751+0000 7f93bb20a700 1 -- 172.21.15.92:0/2468762396 wait complete. 2023-10-06T03:02:48.754 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2023-10-06T03:02:48.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:48.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.877+0000 7f65b8bee700 1 Processor -- start 2023-10-06T03:02:48.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.877+0000 7f65b8bee700 1 -- start start 2023-10-06T03:02:48.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b8bee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65b4159c60 0x7f65b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b8bee700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f65b415db10 0x7f65b4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b8bee700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b415a160 con 0x7f65b4159890 2023-10-06T03:02:48.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b8bee700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b415df50 con 0x7f65b4159c60 2023-10-06T03:02:48.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b8bee700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b40d13a0 con 0x7f65b415db10 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f65b4159890 0x7f65b415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50760/0 (socket says 172.21.15.92:50760) 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b259c700 1 -- 172.21.15.92:0/3061891525 learned_addr learned my addr 172.21.15.92:0/3061891525 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.878+0000 7f65b1d9b700 1 --2- 172.21.15.92:0/3061891525 >> v2:172.21.15.92:3300/0 conn(0x7f65b4159c60 0x7f65b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b159a700 1 -- 172.21.15.92:0/3061891525 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920119104 0 0) 0x7f65b415a160 con 0x7f65b4159890 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b159a700 1 -- 172.21.15.92:0/3061891525 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65a0003570 con 0x7f65b4159890 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b2d9d700 1 --2- 172.21.15.92:0/3061891525 >> v2:172.21.15.136:3300/0 conn(0x7f65b415db10 0x7f65b4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b1d9b700 1 -- 172.21.15.92:0/3061891525 >> v2:172.21.15.136:3300/0 conn(0x7f65b415db10 msgr2=0x7f65b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b1d9b700 1 --2- 172.21.15.92:0/3061891525 >> v2:172.21.15.136:3300/0 conn(0x7f65b415db10 0x7f65b4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b1d9b700 1 -- 172.21.15.92:0/3061891525 >> v1:172.21.15.92:6789/0 conn(0x7f65b4159890 legacy=0x7f65b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b1d9b700 1 -- 172.21.15.92:0/3061891525 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b415e140 con 0x7f65b4159c60 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b1d9b700 1 --2- 172.21.15.92:0/3061891525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b415d5d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65b406a890 tx=0x7f659c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb7c3506a13151e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b159a700 1 -- 172.21.15.92:0/3061891525 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c011630 con 0x7f65b4159c60 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b159a700 1 -- 172.21.15.92:0/3061891525 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c011790 con 0x7f65b4159c60 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b159a700 1 -- 172.21.15.92:0/3061891525 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c011b30 con 0x7f65b4159c60 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b8bee700 1 -- 172.21.15.92:0/3061891525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 msgr2=0x7f65b415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.879+0000 7f65b8bee700 1 --2- 172.21.15.92:0/3061891525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b415d5d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65b406a890 tx=0x7f659c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.880+0000 7f65b8bee700 1 -- 172.21.15.92:0/3061891525 shutdown_connections 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.880+0000 7f65b8bee700 1 --2- 172.21.15.92:0/3061891525 >> v2:172.21.15.136:3300/0 conn(0x7f65b415db10 0x7f65b4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.880+0000 7f65b8bee700 1 --2- 172.21.15.92:0/3061891525 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b415d5d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.880+0000 7f65b8bee700 1 -- 172.21.15.92:0/3061891525 >> 172.21.15.92:0/3061891525 conn(0x7f65b4068e20 msgr2=0x7f65b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.880+0000 7f65b8bee700 1 -- 172.21.15.92:0/3061891525 shutdown_connections 2023-10-06T03:02:48.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.880+0000 7f65b8bee700 1 -- 172.21.15.92:0/3061891525 wait complete. 2023-10-06T03:02:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 Processor -- start 2023-10-06T03:02:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 -- start start 2023-10-06T03:02:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65b4159890 0x7f65b40cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b4147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65b415db10 0x7f65b4169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40b0540 con 0x7f65b4159c60 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40b5a60 con 0x7f65b4159890 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b8bee700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b40b5d00 con 0x7f65b415db10 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b4147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b4147150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52660/0 (socket says 172.21.15.92:52660) 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.881+0000 7f65b1d9b700 1 -- 172.21.15.92:0/82625107 learned_addr learned my addr 172.21.15.92:0/82625107 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b259c700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65b4159890 0x7f65b40cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b2d9d700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65b415db10 0x7f65b4169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b1d9b700 1 -- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65b415db10 msgr2=0x7f65b4169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b1d9b700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65b415db10 0x7f65b4169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b1d9b700 1 -- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65b4159890 msgr2=0x7f65b40cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b1d9b700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65b4159890 0x7f65b40cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b1d9b700 1 -- 172.21.15.92:0/82625107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009420 con 0x7f65b4159c60 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b2d9d700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65b415db10 0x7f65b4169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b1d9b700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b4147150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f65b4070e20 tx=0x7f659c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c011b10 con 0x7f65b4159c60 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0253f0 con 0x7f65b4159c60 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c025790 con 0x7f65b4159c60 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b40b5fe0 con 0x7f65b4159c60 2023-10-06T03:02:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.882+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b40cfe50 con 0x7f65b4159c60 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.883+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b40b06b0 con 0x7f65b4159c60 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.883+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f659c025b80 con 0x7f65b4159c60 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.884+0000 7f65aa7fc700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6594066f30 0x7f65940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.884+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f659c085f70 con 0x7f65b4159c60 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.884+0000 7f65b259c700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6594066f30 0x7f65940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.884+0000 7f65b259c700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6594066f30 0x7f65940693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005c50 tx=0x7f65a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:48.888+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f659c0b75a0 con 0x7f65b4159c60 2023-10-06T03:02:49.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.073+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65b404c3c0 con 0x7f65b4159c60 2023-10-06T03:02:49.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.074+0000 7f65aa7fc700 1 -- 172.21.15.92:0/82625107 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f659c053030 con 0x7f65b4159c60 2023-10-06T03:02:49.075 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:49.075 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:46.124159+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/971383855":"2023-10-07T03:02:43.116026+0000","172.21.15.92:6834/971383855":"2023-10-07T03:02:43.116026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6594066f30 msgr2=0x7f65940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6594066f30 0x7f65940693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65a4005c50 tx=0x7f65a4009040 comp rx=0 tx=0).stop 2023-10-06T03:02:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 msgr2=0x7f65b4147150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b4147150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f65b4070e20 tx=0x7f659c017000 comp rx=0 tx=0).stop 2023-10-06T03:02:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 shutdown_connections 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6594066f30 0x7f65940693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65b415db10 0x7f65b4169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65b4159890 0x7f65b40cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 --2- 172.21.15.92:0/82625107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65b4159c60 0x7f65b4147150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 >> 172.21.15.92:0/82625107 conn(0x7f65b4068e20 msgr2=0x7f65b4153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 shutdown_connections 2023-10-06T03:02:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.076+0000 7f65b8bee700 1 -- 172.21.15.92:0/82625107 wait complete. 2023-10-06T03:02:49.089 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:02:49.089 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:49.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.204+0000 7fc3d3d77700 1 Processor -- start 2023-10-06T03:02:49.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.204+0000 7fc3d3d77700 1 -- start start 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d3d77700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d3d77700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc15a160 con 0x7fc3cc069560 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d3d77700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc0d13a0 con 0x7fc3cc159c60 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d3d77700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc15e140 con 0x7fc3cc159890 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d2314700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc3cc069560 0x7fc3cc0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50766/0 (socket says 172.21.15.92:50766) 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d2314700 1 -- 172.21.15.92:0/1774778046 learned_addr learned my addr 172.21.15.92:0/1774778046 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:49.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d1b13700 1 --2- 172.21.15.92:0/1774778046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265057781 0 0) 0x7fc3cc15a160 con 0x7fc3cc069560 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b8003570 con 0x7fc3cc069560 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090472984 0 0) 0x7fc3cc0d13a0 con 0x7fc3cc159c60 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3cc15a160 con 0x7fc3cc159c60 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1774778046 >> v1:172.21.15.136:6789/0 conn(0x7fc3cc159c60 legacy=0x7fc3cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1774778046 >> v1:172.21.15.92:6789/0 conn(0x7fc3cc069560 legacy=0x7fc3cc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681942928 0 0) 0x7fc3b8003570 con 0x7fc3cc069560 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1774778046 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc15a3d0 con 0x7fc3cc159890 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.205+0000 7fc3d1b13700 1 --2- 172.21.15.92:0/1774778046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc15abd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009910 tx=0x7fc3c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0134f48a747751 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c40017f0 con 0x7fc3cc159890 2023-10-06T03:02:49.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8011420 con 0x7fc3cc159890 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d0b11700 1 -- 172.21.15.92:0/1774778046 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80117c0 con 0x7fc3cc159890 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1774778046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 msgr2=0x7fc3cc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1774778046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc15abd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009910 tx=0x7fc3c80091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1774778046 shutdown_connections 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1774778046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc15abd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1774778046 >> 172.21.15.92:0/1774778046 conn(0x7fc3cc068e20 msgr2=0x7fc3cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1774778046 shutdown_connections 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.206+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1774778046 wait complete. 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.207+0000 7fc3d3d77700 1 Processor -- start 2023-10-06T03:02:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.207+0000 7fc3d3d77700 1 -- start start 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d3d77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 0x7fc3cc0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d3d77700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d3d77700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3cc159c60 0x7fc3cc155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d3d77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc158350 con 0x7fc3cc069560 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d3d77700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc157e80 con 0x7fc3cc159c60 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d3d77700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc158b20 con 0x7fc3cc159890 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d1b13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 0x7fc3cc0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d1b13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 0x7fc3cc0cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52672/0 (socket says 172.21.15.92:52672) 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.208+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1120241961 learned_addr learned my addr 172.21.15.92:0/1120241961 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:49.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1312700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d2314700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3cc159c60 0x7fc3cc155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 msgr2=0x7fc3cc157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1b13700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3cc159c60 msgr2=0x7fc3cc155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1b13700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3cc159c60 0x7fc3cc155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1b13700 1 -- 172.21.15.92:0/1120241961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8009420 con 0x7fc3cc069560 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1312700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d2314700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3cc159c60 0x7fc3cc155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d1b13700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 0x7fc3cc0cf6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80030c0 tx=0x7fc3c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c801cc50 con 0x7fc3cc069560 2023-10-06T03:02:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c801cdb0 con 0x7fc3cc069560 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc0cd260 con 0x7fc3cc069560 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c801a6a0 con 0x7fc3cc069560 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.209+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc158840 con 0x7fc3cc069560 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.210+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc0cd0c0 con 0x7fc3cc069560 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.211+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3c801aa90 con 0x7fc3cc069560 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.211+0000 7fc3c1ffb700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc3b0066f30 0x7fc3b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.211+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc3c80865f0 con 0x7fc3cc069560 2023-10-06T03:02:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.211+0000 7fc3d1312700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc3b0066f30 0x7fc3b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.212+0000 7fc3d1312700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc3b0066f30 0x7fc3b00693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc007fd0 tx=0x7fc3bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.215+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3cc0cd0c0 con 0x7fc3cc069560 2023-10-06T03:02:49.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.442+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3cc0cd0c0 con 0x7fc3cc069560 2023-10-06T03:02:49.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.443+0000 7fc3c1ffb700 1 -- 172.21.15.92:0/1120241961 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fc3cc0cd0c0 con 0x7fc3cc069560 2023-10-06T03:02:49.444 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:49.444 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1384819232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1384819232},{"type":"v1","addr":"172.21.15.92:6836","nonce":1384819232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:27.963102+0000","modified":"2023-10-06T03:02:31.599832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2171492021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2171492021},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171492021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:02:49.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc3b0066f30 msgr2=0x7fc3b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc3b0066f30 0x7fc3b00693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc007fd0 tx=0x7fc3bc009040 comp rx=0 tx=0).stop 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 msgr2=0x7fc3cc0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 0x7fc3cc0cf6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80030c0 tx=0x7fc3c8016000 comp rx=0 tx=0).stop 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 shutdown_connections 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc3b0066f30 0x7fc3b00693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3cc069560 0x7fc3cc0cf6d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3cc159c60 0x7fc3cc155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 --2- 172.21.15.92:0/1120241961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3cc159890 0x7fc3cc157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 >> 172.21.15.92:0/1120241961 conn(0x7fc3cc068e20 msgr2=0x7fc3cc0b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:49.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 shutdown_connections 2023-10-06T03:02:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.445+0000 7fc3d3d77700 1 -- 172.21.15.92:0/1120241961 wait complete. 2023-10-06T03:02:49.448 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2023-10-06T03:02:49.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.572+0000 7fde1d976700 1 Processor -- start 2023-10-06T03:02:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.572+0000 7fde1d976700 1 -- start start 2023-10-06T03:02:49.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde1d976700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde1806c0a0 0x7fde180caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde1d976700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fde1815df50 0x7fde1815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde1d976700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1815d7e0 con 0x7fde1806bcd0 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde1d976700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1815dac0 con 0x7fde1806c0a0 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde1d976700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1815aa30 con 0x7fde1815df50 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde0e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde1806c0a0 0x7fde180caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde0e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde1806c0a0 0x7fde180caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55972/0 (socket says 172.21.15.92:55972) 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.573+0000 7fde0e7fc700 1 -- 172.21.15.92:0/4076066621 learned_addr learned my addr 172.21.15.92:0/4076066621 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde177fe700 1 --2- 172.21.15.92:0/4076066621 >> v2:172.21.15.136:3300/0 conn(0x7fde1815df50 0x7fde1815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde0e7fc700 1 -- 172.21.15.92:0/4076066621 >> v2:172.21.15.136:3300/0 conn(0x7fde1815df50 msgr2=0x7fde1815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde0e7fc700 1 --2- 172.21.15.92:0/4076066621 >> v2:172.21.15.136:3300/0 conn(0x7fde1815df50 0x7fde1815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde0e7fc700 1 -- 172.21.15.92:0/4076066621 >> v1:172.21.15.136:6789/0 conn(0x7fde1806bcd0 legacy=0x7fde1815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:49.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde0e7fc700 1 -- 172.21.15.92:0/4076066621 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1815ad10 con 0x7fde1806c0a0 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde0e7fc700 1 --2- 172.21.15.92:0/4076066621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1806c0a0 0x7fde180caa90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fde1815d720 tx=0x7fde00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64a1e4b9ef02c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde167fc700 1 -- 172.21.15.92:0/4076066621 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000b820 con 0x7fde1806c0a0 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde167fc700 1 -- 172.21.15.92:0/4076066621 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0000b980 con 0x7fde1806c0a0 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde167fc700 1 -- 172.21.15.92:0/4076066621 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000bd40 con 0x7fde1806c0a0 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 -- 172.21.15.92:0/4076066621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1806c0a0 msgr2=0x7fde180caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 --2- 172.21.15.92:0/4076066621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1806c0a0 0x7fde180caa90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fde1815d720 tx=0x7fde00009250 comp rx=0 tx=0).stop 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 -- 172.21.15.92:0/4076066621 shutdown_connections 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 --2- 172.21.15.92:0/4076066621 >> v2:172.21.15.136:3300/0 conn(0x7fde1815df50 0x7fde1815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 --2- 172.21.15.92:0/4076066621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1806c0a0 0x7fde180caa90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 -- 172.21.15.92:0/4076066621 >> 172.21.15.92:0/4076066621 conn(0x7fde180ce600 msgr2=0x7fde180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 -- 172.21.15.92:0/4076066621 shutdown_connections 2023-10-06T03:02:49.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.574+0000 7fde1d976700 1 -- 172.21.15.92:0/4076066621 wait complete. 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 Processor -- start 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 -- start start 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde1806bcd0 0x7fde18143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde1806c0a0 0x7fde1815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 0x7fde1815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180c60e0 con 0x7fde1815df50 2023-10-06T03:02:49.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180b2310 con 0x7fde1806bcd0 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde1d976700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde180c8640 con 0x7fde1806c0a0 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 0x7fde1815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 0x7fde1815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55984/0 (socket says 172.21.15.92:55984) 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.576+0000 7fde177fe700 1 -- 172.21.15.92:0/2490840811 learned_addr learned my addr 172.21.15.92:0/2490840811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde177fe700 1 -- 172.21.15.92:0/2490840811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde1806c0a0 msgr2=0x7fde1815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde177fe700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde1806c0a0 0x7fde1815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde177fe700 1 -- 172.21.15.92:0/2490840811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde1806bcd0 msgr2=0x7fde18143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde177fe700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde1806bcd0 0x7fde18143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde177fe700 1 -- 172.21.15.92:0/2490840811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000094f0 con 0x7fde1815df50 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde177fe700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 0x7fde1815cd30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde1000cdb0 tx=0x7fde100077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1000aa80 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1000abe0 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde180c8920 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10017430 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.577+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde180b2890 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.578+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde180b2a80 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.578+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde10003020 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.579+0000 7fde0f7fe700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddf8066f30 0x7fddf80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.579+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fde10013070 con 0x7fde1815df50 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.579+0000 7fde16ffd700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddf8066f30 0x7fddf80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.579+0000 7fde16ffd700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddf8066f30 0x7fddf80693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde08009740 tx=0x7fde08006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.583+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde10051920 con 0x7fde1815df50 2023-10-06T03:02:49.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.810+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde1804c3c0 con 0x7fde1815df50 2023-10-06T03:02:49.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.810+0000 7fde0f7fe700 1 -- 172.21.15.92:0/2490840811 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fde10054f40 con 0x7fde1815df50 2023-10-06T03:02:49.811 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:49.811 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1384819232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1384819232},{"type":"v1","addr":"172.21.15.92:6836","nonce":1384819232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:02:27.963102+0000","modified":"2023-10-06T03:02:31.599832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2171492021","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2171492021},{"type":"v1","addr":"172.21.15.92:6837","nonce":2171492021}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:02:49.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddf8066f30 msgr2=0x7fddf80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddf8066f30 0x7fddf80693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde08009740 tx=0x7fde08006c60 comp rx=0 tx=0).stop 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 msgr2=0x7fde1815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 0x7fde1815cd30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde1000cdb0 tx=0x7fde100077d0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 shutdown_connections 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddf8066f30 0x7fddf80693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde1815df50 0x7fde1815cd30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde1806bcd0 0x7fde18143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 --2- 172.21.15.92:0/2490840811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde1806c0a0 0x7fde1815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 >> 172.21.15.92:0/2490840811 conn(0x7fde180ce600 msgr2=0x7fde18069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 shutdown_connections 2023-10-06T03:02:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.813+0000 7fde1d976700 1 -- 172.21.15.92:0/2490840811 wait complete. 2023-10-06T03:02:49.815 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 21 2023-10-06T03:02:49.825 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.938+0000 7f78fb04f700 1 Processor -- start 2023-10-06T03:02:49.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.938+0000 7f78fb04f700 1 -- start start 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.939+0000 7f78fb04f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f78f4159890 0x7f78f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.939+0000 7f78fb04f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 0x7f78f40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.939+0000 7f78fb04f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f4069ee0 con 0x7f78f4159c60 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.939+0000 7f78fb04f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f415a160 con 0x7f78f4159890 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.939+0000 7f78fb04f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40d13a0 con 0x7f78f4069560 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.939+0000 7f78f95ec700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 0x7f78f40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f8deb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f78f4159890 0x7f78f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 0x7f78f40699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:60952/0 (socket says 172.21.15.92:60952) 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 -- 172.21.15.92:0/3456770551 learned_addr learned my addr 172.21.15.92:0/3456770551 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f3fff700 1 --1- 172.21.15.92:0/3456770551 >> v1:172.21.15.136:6789/0 conn(0x7f78f4159c60 0x7f78f415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:47346/0 (socket says 172.21.15.92:47346) 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 -- 172.21.15.92:0/3456770551 >> v2:172.21.15.136:3300/0 conn(0x7f78f4159890 msgr2=0x7f78f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 --2- 172.21.15.92:0/3456770551 >> v2:172.21.15.136:3300/0 conn(0x7f78f4159890 0x7f78f415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 -- 172.21.15.92:0/3456770551 >> v1:172.21.15.136:6789/0 conn(0x7f78f4159c60 legacy=0x7f78f415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 -- 172.21.15.92:0/3456770551 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f415e140 con 0x7f78f4069560 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f95ec700 1 --2- 172.21.15.92:0/3456770551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 0x7f78f40699a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0099b0 tx=0x7f78ec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02312f83cdd226 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f37fe700 1 -- 172.21.15.92:0/3456770551 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec009ea0 con 0x7f78f4069560 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f37fe700 1 -- 172.21.15.92:0/3456770551 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00b860 con 0x7f78f4069560 2023-10-06T03:02:49.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.940+0000 7f78f37fe700 1 -- 172.21.15.92:0/3456770551 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00bc00 con 0x7f78f4069560 2023-10-06T03:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 -- 172.21.15.92:0/3456770551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 msgr2=0x7f78f40699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 --2- 172.21.15.92:0/3456770551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 0x7f78f40699a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0099b0 tx=0x7f78ec009250 comp rx=0 tx=0).stop 2023-10-06T03:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 -- 172.21.15.92:0/3456770551 shutdown_connections 2023-10-06T03:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 --2- 172.21.15.92:0/3456770551 >> v2:172.21.15.136:3300/0 conn(0x7f78f4159890 0x7f78f415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 --2- 172.21.15.92:0/3456770551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4069560 0x7f78f40699a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 -- 172.21.15.92:0/3456770551 >> 172.21.15.92:0/3456770551 conn(0x7f78f4068e20 msgr2=0x7f78f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 -- 172.21.15.92:0/3456770551 shutdown_connections 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.941+0000 7f78fb04f700 1 -- 172.21.15.92:0/3456770551 wait complete. 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.942+0000 7f78fb04f700 1 Processor -- start 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.942+0000 7f78fb04f700 1 -- start start 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78fb04f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 0x7f78f40c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78fb04f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4159890 0x7f78f4152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78fb04f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f4159c60 0x7f78f416cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78fb04f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4151fa0 con 0x7f78f4069560 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78fb04f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4151390 con 0x7f78f4159c60 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78fb04f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4151670 con 0x7f78f4159890 2023-10-06T03:02:49.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 0x7f78f40c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 0x7f78f40c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55994/0 (socket says 172.21.15.92:55994) 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 -- 172.21.15.92:0/926525163 learned_addr learned my addr 172.21.15.92:0/926525163 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f3fff700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4159890 0x7f78f4152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f95ec700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f4159c60 0x7f78f416cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 -- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4159890 msgr2=0x7f78f4152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4159890 0x7f78f4152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 -- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f4159c60 msgr2=0x7f78f416cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f4159c60 0x7f78f416cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 -- 172.21.15.92:0/926525163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec0094f0 con 0x7f78f4069560 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f95ec700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f4159c60 0x7f78f416cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.943+0000 7f78f8deb700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 0x7f78f40c68f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009ef0 tx=0x7f78e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.944+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e400cc20 con 0x7f78f4069560 2023-10-06T03:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.944+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e400f040 con 0x7f78f4069560 2023-10-06T03:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.944+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f4151950 con 0x7f78f4069560 2023-10-06T03:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.944+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4014480 con 0x7f78f4069560 2023-10-06T03:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.944+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f40c6e30 con 0x7f78f4069560 2023-10-06T03:02:49.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.944+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f404c3c0 con 0x7f78f4069560 2023-10-06T03:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.945+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f78e40147f0 con 0x7f78f4069560 2023-10-06T03:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.945+0000 7f78f0ff9700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78e0066f30 0x7f78e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.945+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f78e40851a0 con 0x7f78f4069560 2023-10-06T03:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.945+0000 7f78f3fff700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78e0066f30 0x7f78e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:49.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.946+0000 7f78f3fff700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78e0066f30 0x7f78e00693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78f40cdcf0 tx=0x7f78e8007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:49.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:49.950+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78e4052260 con 0x7f78f4069560 2023-10-06T03:02:50.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.133+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78f4152280 con 0x7f78f4069560 2023-10-06T03:02:50.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.134+0000 7f78f0ff9700 1 -- 172.21.15.92:0/926525163 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f78e4055880 con 0x7f78f4069560 2023-10-06T03:02:50.135 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:50.135 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":42,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:46.124159+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:26.035987+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:02:26.917916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6836/971383855":"2023-10-07T03:02:43.116026+0000","172.21.15.92:6834/971383855":"2023-10-07T03:02:43.116026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78e0066f30 msgr2=0x7f78e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78e0066f30 0x7f78e00693e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78f40cdcf0 tx=0x7f78e8007eb0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 msgr2=0x7f78f40c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 0x7f78f40c68f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f78e4009ef0 tx=0x7f78e400b040 comp rx=0 tx=0).stop 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 shutdown_connections 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78e0066f30 0x7f78e00693e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f4069560 0x7f78f40c68f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f4159c60 0x7f78f416cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 --2- 172.21.15.92:0/926525163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f4159890 0x7f78f4152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.135+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 >> 172.21.15.92:0/926525163 conn(0x7f78f4068e20 msgr2=0x7f78f4157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.136+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 shutdown_connections 2023-10-06T03:02:50.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.136+0000 7f78fb04f700 1 -- 172.21.15.92:0/926525163 wait complete. 2023-10-06T03:02:50.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:02:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.263+0000 7fc539e15700 1 Processor -- start 2023-10-06T03:02:50.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.263+0000 7fc539e15700 1 -- start start 2023-10-06T03:02:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc539e15700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc534159890 0x7fc53415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc539e15700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534069560 0x7fc5340699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc539e15700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc534069ee0 con 0x7fc534159c60 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc539e15700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53415a160 con 0x7fc534159890 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc539e15700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340d13a0 con 0x7fc534069560 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc5337fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc534159890 0x7fc53415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc532ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc534159c60 0x7fc53415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42838/0 (socket says 172.21.15.92:42838) 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc532ffd700 1 -- 172.21.15.92:0/3602906096 learned_addr learned my addr 172.21.15.92:0/3602906096 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc533fff700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534069560 0x7fc5340699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc5337fe700 1 -- 172.21.15.92:0/3602906096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534069560 msgr2=0x7fc5340699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc5337fe700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534069560 0x7fc5340699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc5337fe700 1 -- 172.21.15.92:0/3602906096 >> v1:172.21.15.92:6789/0 conn(0x7fc534159c60 legacy=0x7fc53415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc5337fe700 1 -- 172.21.15.92:0/3602906096 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53415e140 con 0x7fc534159890 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc5327fc700 1 -- 172.21.15.92:0/3602906096 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907741719 0 0) 0x7fc534069ee0 con 0x7fc534159c60 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.264+0000 7fc533fff700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534069560 0x7fc5340699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:50.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc5337fe700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534159890 0x7fc53415abd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc524009e60 tx=0x7fc524009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4436062b5cc715fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc5327fc700 1 -- 172.21.15.92:0/3602906096 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400e070 con 0x7fc534159890 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc5327fc700 1 -- 172.21.15.92:0/3602906096 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400bc30 con 0x7fc534159890 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc5327fc700 1 -- 172.21.15.92:0/3602906096 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524012430 con 0x7fc534159890 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 -- 172.21.15.92:0/3602906096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534159890 msgr2=0x7fc53415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534159890 0x7fc53415abd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc524009e60 tx=0x7fc524009780 comp rx=0 tx=0).stop 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 -- 172.21.15.92:0/3602906096 shutdown_connections 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534069560 0x7fc5340699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 --2- 172.21.15.92:0/3602906096 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534159890 0x7fc53415abd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 -- 172.21.15.92:0/3602906096 >> 172.21.15.92:0/3602906096 conn(0x7fc534068e20 msgr2=0x7fc5340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 -- 172.21.15.92:0/3602906096 shutdown_connections 2023-10-06T03:02:50.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.265+0000 7fc539e15700 1 -- 172.21.15.92:0/3602906096 wait complete. 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 Processor -- start 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 -- start start 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 0x7fc534154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534159890 0x7fc5340c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc534159c60 0x7fc5340cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53414d020 con 0x7fc534069560 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340c9000 con 0x7fc534159c60 2023-10-06T03:02:50.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc539e15700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340cd9a0 con 0x7fc534159890 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc5337fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 0x7fc534154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc5337fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 0x7fc534154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56006/0 (socket says 172.21.15.92:56006) 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.267+0000 7fc5337fe700 1 -- 172.21.15.92:0/623446290 learned_addr learned my addr 172.21.15.92:0/623446290 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc5337fe700 1 -- 172.21.15.92:0/623446290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534159890 msgr2=0x7fc5340c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc5337fe700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534159890 0x7fc5340c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc5337fe700 1 -- 172.21.15.92:0/623446290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc534159c60 msgr2=0x7fc5340cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc5337fe700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc534159c60 0x7fc5340cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc5337fe700 1 -- 172.21.15.92:0/623446290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524009a20 con 0x7fc534069560 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc5337fe700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 0x7fc534154770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b6c0 tx=0x7fc524019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524012e00 con 0x7fc534069560 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524027470 con 0x7fc534069560 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5340c9210 con 0x7fc534069560 2023-10-06T03:02:50.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524027830 con 0x7fc534069560 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.268+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5340c97d0 con 0x7fc534069560 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.269+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5340cdae0 con 0x7fc534069560 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.269+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc524024070 con 0x7fc534069560 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.270+0000 7fc51b7fe700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc520066f30 0x7fc5200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.270+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc524086da0 con 0x7fc534069560 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.270+0000 7fc532ffd700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc520066f30 0x7fc5200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.270+0000 7fc532ffd700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc520066f30 0x7fc5200693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5340c7420 tx=0x7fc5280079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:50.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.274+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc524053e60 con 0x7fc534069560 2023-10-06T03:02:50.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.500+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc53404c3c0 con 0x7fc534069560 2023-10-06T03:02:50.501 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:02:50.500+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:02:50.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.504+0000 7fc51b7fe700 1 -- 172.21.15.92:0/623446290 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc524057480 con 0x7fc534069560 2023-10-06T03:02:50.506 INFO:tasks.ceph.mds.c.smithi092.stderr: -1123> 2023-10-06T03:02:32.558+0000 7fb7fde46700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:02:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc520066f30 msgr2=0x7fc5200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc520066f30 0x7fc5200693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5340c7420 tx=0x7fc5280079d0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 msgr2=0x7fc534154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 0x7fc534154770 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc52400b6c0 tx=0x7fc524019000 comp rx=0 tx=0).stop 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 shutdown_connections 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc520066f30 0x7fc5200693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc534069560 0x7fc534154770 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc534159c60 0x7fc5340cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 --2- 172.21.15.92:0/623446290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc534159890 0x7fc5340c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 >> 172.21.15.92:0/623446290 conn(0x7fc534068e20 msgr2=0x7fc5341587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 shutdown_connections 2023-10-06T03:02:50.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.506+0000 7fc539e15700 1 -- 172.21.15.92:0/623446290 wait complete. 2023-10-06T03:02:50.509 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:02:50.520 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:02:50.527 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:02:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.636+0000 7ff58de96700 1 Processor -- start 2023-10-06T03:02:50.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.636+0000 7ff58de96700 1 -- start start 2023-10-06T03:02:50.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58de96700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff5880c93a0 0x7ff58806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58de96700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5880c9770 0x7ff5880caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58de96700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff5880c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58de96700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5880af490 con 0x7ff5880c9770 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58de96700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5880af770 con 0x7ff5880c93a0 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58de96700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880afa50 con 0x7ff5880cb400 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff587fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5880c9770 0x7ff5880caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff587fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5880c9770 0x7ff5880caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56042/0 (socket says 172.21.15.92:56042) 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff587fff700 1 -- 172.21.15.92:0/2602644346 learned_addr learned my addr 172.21.15.92:0/2602644346 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:50.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58ce94700 1 --2- 172.21.15.92:0/2602644346 >> v2:172.21.15.136:3300/0 conn(0x7ff5880c93a0 0x7ff58806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.637+0000 7ff58d695700 1 --2- 172.21.15.92:0/2602644346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff5880c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff587fff700 1 -- 172.21.15.92:0/2602644346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 msgr2=0x7ff5880c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff587fff700 1 --2- 172.21.15.92:0/2602644346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff5880c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff587fff700 1 -- 172.21.15.92:0/2602644346 >> v2:172.21.15.136:3300/0 conn(0x7ff5880c93a0 msgr2=0x7ff58806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff587fff700 1 --2- 172.21.15.92:0/2602644346 >> v2:172.21.15.136:3300/0 conn(0x7ff5880c93a0 0x7ff58806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff587fff700 1 -- 172.21.15.92:0/2602644346 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880c7ab0 con 0x7ff5880c9770 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff587fff700 1 --2- 172.21.15.92:0/2602644346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c9770 0x7ff5880caec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff58806e2f0 tx=0x7ff570009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4386cb3c7ef800da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff5877fe700 1 -- 172.21.15.92:0/2602644346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570005c10 con 0x7ff5880c9770 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff5877fe700 1 -- 172.21.15.92:0/2602644346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570005d70 con 0x7ff5880c9770 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff5877fe700 1 -- 172.21.15.92:0/2602644346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570006260 con 0x7ff5880c9770 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 -- 172.21.15.92:0/2602644346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c9770 msgr2=0x7ff5880caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 --2- 172.21.15.92:0/2602644346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c9770 0x7ff5880caec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff58806e2f0 tx=0x7ff570009220 comp rx=0 tx=0).stop 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 -- 172.21.15.92:0/2602644346 shutdown_connections 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 --2- 172.21.15.92:0/2602644346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff5880c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 --2- 172.21.15.92:0/2602644346 >> v2:172.21.15.136:3300/0 conn(0x7ff5880c93a0 0x7ff58806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 --2- 172.21.15.92:0/2602644346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c9770 0x7ff5880caec0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 -- 172.21.15.92:0/2602644346 >> 172.21.15.92:0/2602644346 conn(0x7ff5880cddc0 msgr2=0x7ff5880c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 -- 172.21.15.92:0/2602644346 shutdown_connections 2023-10-06T03:02:50.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.638+0000 7ff58de96700 1 -- 172.21.15.92:0/2602644346 wait complete. 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.639+0000 7ff58de96700 1 Processor -- start 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 -- start start 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 0x7ff5880b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5880c9770 0x7ff5880a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff588146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b0de0 con 0x7ff5880c93a0 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b10c0 con 0x7ff5880c9770 2023-10-06T03:02:50.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58de96700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5880b13a0 con 0x7ff5880cb400 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.640+0000 7ff58ce94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 0x7ff5880b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 0x7ff5880b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56048/0 (socket says 172.21.15.92:56048) 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 -- 172.21.15.92:0/2758081340 learned_addr learned my addr 172.21.15.92:0/2758081340 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58d695700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff588146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff587fff700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5880c9770 0x7ff5880a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 -- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 msgr2=0x7ff588146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff588146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 -- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5880c9770 msgr2=0x7ff5880a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5880c9770 0x7ff5880a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 -- 172.21.15.92:0/2758081340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff570009480 con 0x7ff5880c93a0 2023-10-06T03:02:50.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff587fff700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5880c9770 0x7ff5880a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58ce94700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 0x7ff5880b2ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ef0 tx=0x7ff57800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800cc20 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57800f040 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5880b1680 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578014480 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.641+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5880a5a00 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.642+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5880b1b20 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.643+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5780147f0 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.643+0000 7ff584ff9700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff568066f30 0x7ff5680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.643+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6230+0+0 (secure 0 0 0) 0x7ff578085130 con 0x7ff5880c93a0 2023-10-06T03:02:50.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.643+0000 7ff587fff700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff568066f30 0x7ff5680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.643+0000 7ff587fff700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff568066f30 0x7ff5680693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff570009260 tx=0x7ff570016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:50.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.647+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff578052190 con 0x7ff5880c93a0 2023-10-06T03:02:50.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:50.875+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5880b1cd0 con 0x7ff5880c93a0 2023-10-06T03:02:51.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.505+0000 7ff584ff9700 1 -- 172.21.15.92:0/2758081340 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5780557b0 con 0x7ff5880c93a0 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff568066f30 msgr2=0x7ff5680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff568066f30 0x7ff5680693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff570009260 tx=0x7ff570016000 comp rx=0 tx=0).stop 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 msgr2=0x7ff5880b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 0x7ff5880b2ca0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ef0 tx=0x7ff57800b040 comp rx=0 tx=0).stop 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 shutdown_connections 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff568066f30 0x7ff5680693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5880c93a0 0x7ff5880b2ca0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5880cb400 0x7ff588146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 --2- 172.21.15.92:0/2758081340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5880c9770 0x7ff5880a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 >> 172.21.15.92:0/2758081340 conn(0x7ff5880cddc0 msgr2=0x7ff588153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:51.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 shutdown_connections 2023-10-06T03:02:51.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.508+0000 7ff58de96700 1 -- 172.21.15.92:0/2758081340 wait complete. 2023-10-06T03:02:51.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:02:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.683+0000 7f11ef58a700 1 Processor -- start 2023-10-06T03:02:51.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ef58a700 1 -- start start 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ef58a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11e8159ec0 0x7f11e80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ef58a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f11e80cf350 0x7f11e80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ef58a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e80b5b00 con 0x7f11e80b53d0 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ef58a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e80b5850 con 0x7f11e8159ec0 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ef58a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e80b5da0 con 0x7f11e80cf350 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.684+0000 7f11ed326700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11e8159ec0 0x7f11e80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:51.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11e8159ec0 0x7f11e80b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56054/0 (socket says 172.21.15.92:56054) 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 -- 172.21.15.92:0/2267142634 learned_addr learned my addr 172.21.15.92:0/2267142634 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 -- 172.21.15.92:0/2267142634 >> v2:172.21.15.136:3300/0 conn(0x7f11e80cf350 msgr2=0x7f11e80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11edb27700 1 --2- 172.21.15.92:0/2267142634 >> v2:172.21.15.136:3300/0 conn(0x7f11e80cf350 0x7f11e80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 --2- 172.21.15.92:0/2267142634 >> v2:172.21.15.136:3300/0 conn(0x7f11e80cf350 0x7f11e80cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 -- 172.21.15.92:0/2267142634 >> v1:172.21.15.136:6789/0 conn(0x7f11e80b53d0 legacy=0x7f11e80cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 -- 172.21.15.92:0/2267142634 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e80b6080 con 0x7f11e8159ec0 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11edb27700 1 --2- 172.21.15.92:0/2267142634 >> v2:172.21.15.136:3300/0 conn(0x7f11e80cf350 0x7f11e80cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11ed326700 1 --2- 172.21.15.92:0/2267142634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8159ec0 0x7f11e80b4e90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11e80cb4a0 tx=0x7f11e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d769ba145ecbb4e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11dffff700 1 -- 172.21.15.92:0/2267142634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400e070 con 0x7f11e8159ec0 2023-10-06T03:02:51.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11dffff700 1 -- 172.21.15.92:0/2267142634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e400ba90 con 0x7f11e8159ec0 2023-10-06T03:02:51.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.685+0000 7f11dffff700 1 -- 172.21.15.92:0/2267142634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40092e0 con 0x7f11e8159ec0 2023-10-06T03:02:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 -- 172.21.15.92:0/2267142634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8159ec0 msgr2=0x7f11e80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2267142634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8159ec0 0x7f11e80b4e90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11e80cb4a0 tx=0x7f11e4009840 comp rx=0 tx=0).stop 2023-10-06T03:02:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 -- 172.21.15.92:0/2267142634 shutdown_connections 2023-10-06T03:02:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2267142634 >> v2:172.21.15.136:3300/0 conn(0x7f11e80cf350 0x7f11e80cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2267142634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e8159ec0 0x7f11e80b4e90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 -- 172.21.15.92:0/2267142634 >> 172.21.15.92:0/2267142634 conn(0x7f11e80999f0 msgr2=0x7f11e80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 -- 172.21.15.92:0/2267142634 shutdown_connections 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.686+0000 7f11ef58a700 1 -- 172.21.15.92:0/2267142634 wait complete. 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.687+0000 7f11ef58a700 1 Processor -- start 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.687+0000 7f11ef58a700 1 -- start start 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11e80b53d0 0x7f11e8069010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 0x7f11e8069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11e8159ec0 0x7f11e8169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8069a90 con 0x7f11e80cf350 2023-10-06T03:02:51.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8069d20 con 0x7f11e8159ec0 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e806a000 con 0x7f11e80b53d0 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 0x7f11e8069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 0x7f11e8069550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56060/0 (socket says 172.21.15.92:56060) 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 -- 172.21.15.92:0/2440462779 learned_addr learned my addr 172.21.15.92:0/2440462779 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11edb27700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11e8159ec0 0x7f11e8169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ed326700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11e80b53d0 0x7f11e8069010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:51.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 -- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11e80b53d0 msgr2=0x7f11e8069010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11e80b53d0 0x7f11e8069010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 -- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11e8159ec0 msgr2=0x7f11e8169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11e8159ec0 0x7f11e8169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 -- 172.21.15.92:0/2440462779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4009ae0 con 0x7f11e80cf350 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ecb25700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 0x7f11e8069550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b5e0 tx=0x7f11d8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800f510 con 0x7f11e80cf350 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d800f670 con 0x7f11e80cf350 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e816adf0 con 0x7f11e80cf350 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800fa30 con 0x7f11e80cf350 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.688+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e816b3b0 con 0x7f11e80cf350 2023-10-06T03:02:51.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.689+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e80647c0 con 0x7f11e80cf350 2023-10-06T03:02:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.690+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11d801a400 con 0x7f11e80cf350 2023-10-06T03:02:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.690+0000 7f11dd7fa700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11cc066fb0 0x7f11cc069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.690+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f11d8084310 con 0x7f11e80cf350 2023-10-06T03:02:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.690+0000 7f11ed326700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11cc066fb0 0x7f11cc069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.691+0000 7f11ed326700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11cc066fb0 0x7f11cc069460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009fd0 tx=0x7f11e4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.694+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11e80647c0 con 0x7f11e80cf350 2023-10-06T03:02:51.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:51.878+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11e80647c0 con 0x7f11e80cf350 2023-10-06T03:02:52.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.506+0000 7f11dd7fa700 1 -- 172.21.15.92:0/2440462779 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11e80647c0 con 0x7f11e80cf350 2023-10-06T03:02:52.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11cc066fb0 msgr2=0x7f11cc069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:52.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11cc066fb0 0x7f11cc069460 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009fd0 tx=0x7f11e4017000 comp rx=0 tx=0).stop 2023-10-06T03:02:52.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 msgr2=0x7f11e8069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:52.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 0x7f11e8069550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b5e0 tx=0x7f11d8009120 comp rx=0 tx=0).stop 2023-10-06T03:02:52.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 shutdown_connections 2023-10-06T03:02:52.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11cc066fb0 0x7f11cc069460 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11e8159ec0 0x7f11e8169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11e80b53d0 0x7f11e8069010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 --2- 172.21.15.92:0/2440462779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11e80cf350 0x7f11e8069550 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 >> 172.21.15.92:0/2440462779 conn(0x7f11e80999f0 msgr2=0x7f11e80c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:52.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 shutdown_connections 2023-10-06T03:02:52.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.509+0000 7f11ef58a700 1 -- 172.21.15.92:0/2440462779 wait complete. 2023-10-06T03:02:52.512 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:02:52.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:02:52.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.648+0000 7fde4ce18700 1 Processor -- start 2023-10-06T03:02:52.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.648+0000 7fde4ce18700 1 -- start start 2023-10-06T03:02:52.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4ce18700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fde48159ec0 0x7fde480b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4ce18700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 0x7fde480cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4ce18700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde480b5b00 con 0x7fde480cce40 2023-10-06T03:02:52.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4ce18700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde480b5850 con 0x7fde48159ec0 2023-10-06T03:02:52.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4ce18700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde480b5da0 con 0x7fde480b53d0 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde46d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fde480cce40 0x7fde480cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42862/0 (socket says 172.21.15.92:42862) 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde46d9d700 1 -- 172.21.15.92:0/1818287401 learned_addr learned my addr 172.21.15.92:0/1818287401 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4659c700 1 --2- 172.21.15.92:0/1818287401 >> v2:172.21.15.136:3300/0 conn(0x7fde48159ec0 0x7fde480b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde45d9b700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 0x7fde480cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932038563 0 0) 0x7fde480b5b00 con 0x7fde480cce40 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde30003570 con 0x7fde480cce40 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3648242738 0 0) 0x7fde30003570 con 0x7fde480cce40 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 msgr2=0x7fde480cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 0x7fde480cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 >> v2:172.21.15.136:3300/0 conn(0x7fde48159ec0 msgr2=0x7fde480b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:52.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 --2- 172.21.15.92:0/1818287401 >> v2:172.21.15.136:3300/0 conn(0x7fde48159ec0 0x7fde480b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.649+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480b6080 con 0x7fde480cce40 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fde400017f0 con 0x7fde480cce40 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4659c700 1 --2- 172.21.15.92:0/1818287401 >> v2:172.21.15.136:3300/0 conn(0x7fde48159ec0 0x7fde480b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 >> v1:172.21.15.92:6789/0 conn(0x7fde480cce40 legacy=0x7fde480cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 0x7fde30003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde30003fe0 0x7fde30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde30004d50 0x7fde300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480b5850 con 0x7fde30004d50 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde480b5da0 con 0x7fde30003fe0 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000ba90 con 0x7fde480b53d0 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde30004d50 0x7fde300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde45d9b700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde30003fe0 0x7fde30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4659c700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 0x7fde30003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 -- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 msgr2=0x7fde30003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde480b53d0 0x7fde30003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 -- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde30003fe0 msgr2=0x7fde30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde30003fe0 0x7fde30004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 -- 172.21.15.92:0/1818287401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30003570 con 0x7fde30004d50 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde46d9d700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde30004d50 0x7fde300051c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde4000a970 tx=0x7fde40002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a01448a4a7222de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40012440 con 0x7fde30004d50 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde400125c0 con 0x7fde30004d50 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.650+0000 7fde4559a700 1 -- 172.21.15.92:0/1818287401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40012980 con 0x7fde30004d50 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde4ce18700 1 -- 172.21.15.92:0/1818287401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde30004d50 msgr2=0x7fde300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde4ce18700 1 --2- 172.21.15.92:0/1818287401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde30004d50 0x7fde300051c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde4000a970 tx=0x7fde40002210 comp rx=0 tx=0).stop 2023-10-06T03:02:52.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde46d9d700 1 -- 172.21.15.92:0/1818287401 reap_dead start 2023-10-06T03:02:52.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde4ce18700 1 -- 172.21.15.92:0/1818287401 shutdown_connections 2023-10-06T03:02:52.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde4ce18700 1 -- 172.21.15.92:0/1818287401 >> 172.21.15.92:0/1818287401 conn(0x7fde480999f0 msgr2=0x7fde480b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:52.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde4ce18700 1 -- 172.21.15.92:0/1818287401 shutdown_connections 2023-10-06T03:02:52.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.651+0000 7fde4ce18700 1 -- 172.21.15.92:0/1818287401 wait complete. 2023-10-06T03:02:52.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.652+0000 7fde4ce18700 1 Processor -- start 2023-10-06T03:02:52.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.652+0000 7fde4ce18700 1 -- start start 2023-10-06T03:02:52.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde30004d50 0x7fde481472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde48068390 0x7fde4806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 0x7fde480c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4806a940 con 0x7fde480c64e0 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4806b0f0 con 0x7fde48068390 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde481698a0 con 0x7fde30004d50 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 0x7fde480c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 0x7fde480c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56078/0 (socket says 172.21.15.92:56078) 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 -- 172.21.15.92:0/4218868941 learned_addr learned my addr 172.21.15.92:0/4218868941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde45d9b700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde48068390 0x7fde4806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4659c700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde30004d50 0x7fde481472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 -- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde30004d50 msgr2=0x7fde481472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:52.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde30004d50 0x7fde481472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 -- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde48068390 msgr2=0x7fde4806abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde48068390 0x7fde4806abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 -- 172.21.15.92:0/4218868941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde480c6e50 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde46d9d700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 0x7fde480c6910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde40012af0 tx=0x7fde40015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40023070 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40012440 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde48068850 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde400017b0 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.653+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde48068e10 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.654+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde48063c90 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.655+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde40019070 con 0x7fde480c64e0 2023-10-06T03:02:52.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.655+0000 7fde367fc700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde28066f20 0x7fde280693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:52.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.655+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fde40085a80 con 0x7fde480c64e0 2023-10-06T03:02:52.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.655+0000 7fde4659c700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde28066f20 0x7fde280693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:52.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.655+0000 7fde4659c700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde28066f20 0x7fde280693d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fde38006ea0 tx=0x7fde38009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:52.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.659+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde40052ca0 con 0x7fde480c64e0 2023-10-06T03:02:52.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:52.842+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde48169a10 con 0x7fde480c64e0 2023-10-06T03:02:53.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.519+0000 7fde367fc700 1 -- 172.21.15.92:0/4218868941 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fde40017020 con 0x7fde480c64e0 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde28066f20 msgr2=0x7fde280693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde28066f20 0x7fde280693d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fde38006ea0 tx=0x7fde38009120 comp rx=0 tx=0).stop 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 msgr2=0x7fde480c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 0x7fde480c6910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde40012af0 tx=0x7fde40015000 comp rx=0 tx=0).stop 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 shutdown_connections 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde28066f20 0x7fde280693d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde480c64e0 0x7fde480c6910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde30004d50 0x7fde481472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 --2- 172.21.15.92:0/4218868941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde48068390 0x7fde4806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 >> 172.21.15.92:0/4218868941 conn(0x7fde480999f0 msgr2=0x7fde480ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:53.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 shutdown_connections 2023-10-06T03:02:53.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.522+0000 7fde4ce18700 1 -- 172.21.15.92:0/4218868941 wait complete. 2023-10-06T03:02:53.524 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:02:53.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:02:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.653+0000 7ff4a554b700 1 Processor -- start 2023-10-06T03:02:53.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.653+0000 7ff4a554b700 1 -- start start 2023-10-06T03:02:53.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4a554b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4a0156760 0x7ff4a006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4a554b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a006ee20 con 0x7ff4a0156390 2023-10-06T03:02:53.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4a554b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a00aceb0 con 0x7ff4a006e9c0 2023-10-06T03:02:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4a554b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a00ad460 con 0x7ff4a0156760 2023-10-06T03:02:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff49effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff4a0156390 0x7ff4a00ac7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42874/0 (socket says 172.21.15.92:42874) 2023-10-06T03:02:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff49effd700 1 -- 172.21.15.92:0/1047044622 learned_addr learned my addr 172.21.15.92:0/1047044622 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4967fc700 1 --2- 172.21.15.92:0/1047044622 >> v2:172.21.15.92:3300/0 conn(0x7ff4a0156760 0x7ff4a006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff49e7fc700 1 -- 172.21.15.92:0/1047044622 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990055851 0 0) 0x7ff490001110 con 0x7ff4a0156390 2023-10-06T03:02:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff49e7fc700 1 -- 172.21.15.92:0/1047044622 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c003570 con 0x7ff4a0156390 2023-10-06T03:02:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4967fc700 1 -- 172.21.15.92:0/1047044622 >> v1:172.21.15.136:6789/0 conn(0x7ff4a006e9c0 legacy=0x7ff4a00c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4967fc700 1 -- 172.21.15.92:0/1047044622 >> v1:172.21.15.92:6789/0 conn(0x7ff4a0156390 legacy=0x7ff4a00ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.654+0000 7ff4967fc700 1 -- 172.21.15.92:0/1047044622 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a00ad180 con 0x7ff4a0156760 2023-10-06T03:02:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4967fc700 1 --2- 172.21.15.92:0/1047044622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0156760 0x7ff4a006e480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098e0 tx=0x7ff4880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc34084aa74be6fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:53.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff49e7fc700 1 -- 172.21.15.92:0/1047044622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488011630 con 0x7ff4a0156760 2023-10-06T03:02:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff49e7fc700 1 -- 172.21.15.92:0/1047044622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff488011790 con 0x7ff4a0156760 2023-10-06T03:02:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff49e7fc700 1 -- 172.21.15.92:0/1047044622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488011b30 con 0x7ff4a0156760 2023-10-06T03:02:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 -- 172.21.15.92:0/1047044622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0156760 msgr2=0x7ff4a006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 --2- 172.21.15.92:0/1047044622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0156760 0x7ff4a006e480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098e0 tx=0x7ff4880091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 -- 172.21.15.92:0/1047044622 shutdown_connections 2023-10-06T03:02:53.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 --2- 172.21.15.92:0/1047044622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a0156760 0x7ff4a006e480 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 -- 172.21.15.92:0/1047044622 >> 172.21.15.92:0/1047044622 conn(0x7ff4a00c9fb0 msgr2=0x7ff4a006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:53.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 -- 172.21.15.92:0/1047044622 shutdown_connections 2023-10-06T03:02:53.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.655+0000 7ff4a554b700 1 -- 172.21.15.92:0/1047044622 wait complete. 2023-10-06T03:02:53.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 Processor -- start 2023-10-06T03:02:53.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 -- start start 2023-10-06T03:02:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 0x7ff4a00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4a0156390 0x7ff4a006ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a0156760 0x7ff4a00690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a006c220 con 0x7ff4a006e9c0 2023-10-06T03:02:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a00c3a80 con 0x7ff4a0156760 2023-10-06T03:02:53.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff4a554b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a006bfc0 con 0x7ff4a0156390 2023-10-06T03:02:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff49effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 0x7ff4a00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff49effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 0x7ff4a00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56098/0 (socket says 172.21.15.92:56098) 2023-10-06T03:02:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.657+0000 7ff49effd700 1 -- 172.21.15.92:0/3060711478 learned_addr learned my addr 172.21.15.92:0/3060711478 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff4967fc700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4a0156390 0x7ff4a006ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49f7fe700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a0156760 0x7ff4a00690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49effd700 1 -- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4a0156390 msgr2=0x7ff4a006ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49effd700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4a0156390 0x7ff4a006ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49effd700 1 -- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a0156760 msgr2=0x7ff4a00690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49effd700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a0156760 0x7ff4a00690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49effd700 1 -- 172.21.15.92:0/3060711478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009420 con 0x7ff4a006e9c0 2023-10-06T03:02:53.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff49effd700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 0x7ff4a00b1b40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c7f0 tx=0x7ff490007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490005b30 con 0x7ff4a006e9c0 2023-10-06T03:02:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490005c90 con 0x7ff4a006e9c0 2023-10-06T03:02:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a0068430 con 0x7ff4a006e9c0 2023-10-06T03:02:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490007340 con 0x7ff4a006e9c0 2023-10-06T03:02:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.658+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a00689f0 con 0x7ff4a006e9c0 2023-10-06T03:02:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.659+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a006cfd0 con 0x7ff4a006e9c0 2023-10-06T03:02:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.659+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff49001b400 con 0x7ff4a006e9c0 2023-10-06T03:02:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.660+0000 7ff4977fe700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff480066f30 0x7ff4800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.660+0000 7ff4967fc700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff480066f30 0x7ff4800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.660+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5378+0+0 (secure 0 0 0) 0x7ff490083d30 con 0x7ff4a006e9c0 2023-10-06T03:02:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.660+0000 7ff4967fc700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff480066f30 0x7ff4800693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098e0 tx=0x7ff488016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.664+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff490051160 con 0x7ff4a006e9c0 2023-10-06T03:02:53.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.852+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff4a006ee20 con 0x7ff4a006e9c0 2023-10-06T03:02:53.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.855+0000 7ff4977fe700 1 -- 172.21.15.92:0/3060711478 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4a006ee20 con 0x7ff4a006e9c0 2023-10-06T03:02:53.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff480066f30 msgr2=0x7ff4800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff480066f30 0x7ff4800693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098e0 tx=0x7ff488016000 comp rx=0 tx=0).stop 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 msgr2=0x7ff4a00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 0x7ff4a00b1b40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c7f0 tx=0x7ff490007c70 comp rx=0 tx=0).stop 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 shutdown_connections 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff480066f30 0x7ff4800693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4a006e9c0 0x7ff4a00b1b40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a0156760 0x7ff4a00690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 --2- 172.21.15.92:0/3060711478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4a0156390 0x7ff4a006ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 >> 172.21.15.92:0/3060711478 conn(0x7ff4a00c9fb0 msgr2=0x7ff4a006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 shutdown_connections 2023-10-06T03:02:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.857+0000 7ff4a554b700 1 -- 172.21.15.92:0/3060711478 wait complete. 2023-10-06T03:02:53.870 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-06T03:02:53.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.983+0000 7fac29500700 1 Processor -- start 2023-10-06T03:02:53.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac29500700 1 -- start start 2023-10-06T03:02:53.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac29500700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac29500700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac29500700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2415a160 con 0x7fac24159890 2023-10-06T03:02:53.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac29500700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2415df50 con 0x7fac24159c60 2023-10-06T03:02:53.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac29500700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240d13a0 con 0x7fac2415db10 2023-10-06T03:02:53.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.984+0000 7fac1a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56102/0 (socket says 172.21.15.92:56102) 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 -- 172.21.15.92:0/10051926 learned_addr learned my addr 172.21.15.92:0/10051926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 -- 172.21.15.92:0/10051926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 msgr2=0x7fac24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac237fe700 1 --2- 172.21.15.92:0/10051926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 --2- 172.21.15.92:0/10051926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac24069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 -- 172.21.15.92:0/10051926 >> v1:172.21.15.136:6789/0 conn(0x7fac24159890 legacy=0x7fac2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:53.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 -- 172.21.15.92:0/10051926 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2415e140 con 0x7fac24159c60 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac1a7fc700 1 --2- 172.21.15.92:0/10051926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac2415d5d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0099b0 tx=0x7fac0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec3af38a73c49b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac227fc700 1 -- 172.21.15.92:0/10051926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00b820 con 0x7fac24159c60 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac227fc700 1 -- 172.21.15.92:0/10051926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0c00b980 con 0x7fac24159c60 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.985+0000 7fac227fc700 1 -- 172.21.15.92:0/10051926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c00bd20 con 0x7fac24159c60 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 -- 172.21.15.92:0/10051926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 msgr2=0x7fac2415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 --2- 172.21.15.92:0/10051926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac2415d5d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0099b0 tx=0x7fac0c009250 comp rx=0 tx=0).stop 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 -- 172.21.15.92:0/10051926 shutdown_connections 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 --2- 172.21.15.92:0/10051926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac24069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 --2- 172.21.15.92:0/10051926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 -- 172.21.15.92:0/10051926 >> 172.21.15.92:0/10051926 conn(0x7fac24068e20 msgr2=0x7fac240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 -- 172.21.15.92:0/10051926 shutdown_connections 2023-10-06T03:02:53.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.986+0000 7fac29500700 1 -- 172.21.15.92:0/10051926 wait complete. 2023-10-06T03:02:53.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.987+0000 7fac29500700 1 Processor -- start 2023-10-06T03:02:53.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.987+0000 7fac29500700 1 -- start start 2023-10-06T03:02:53.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac2415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac24159c60 0x7fac2416a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 0x7fac2414cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2415d530 con 0x7fac2415db10 2023-10-06T03:02:53.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2415cbe0 con 0x7fac24159890 2023-10-06T03:02:53.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2415b670 con 0x7fac24159c60 2023-10-06T03:02:53.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 0x7fac2414cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 0x7fac2414cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56112/0 (socket says 172.21.15.92:56112) 2023-10-06T03:02:53.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 -- 172.21.15.92:0/2588654372 learned_addr learned my addr 172.21.15.92:0/2588654372 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:53.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac2415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac1a7fc700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac24159c60 0x7fac2416a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 -- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac24159c60 msgr2=0x7fac2416a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac24159c60 0x7fac2416a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 -- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 msgr2=0x7fac2415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac2415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 -- 172.21.15.92:0/2588654372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c0094f0 con 0x7fac2415db10 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac2415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:53.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac237fe700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 0x7fac2414cf20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00cdb0 tx=0x7fac1c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00aa80 con 0x7fac2415db10 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00abe0 con 0x7fac2415db10 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2415b950 con 0x7fac2415db10 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c017430 con 0x7fac2415db10 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.988+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2414d460 con 0x7fac2415db10 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.989+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2415b4d0 con 0x7fac2415db10 2023-10-06T03:02:53.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.990+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac1c003020 con 0x7fac2415db10 2023-10-06T03:02:53.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.990+0000 7fac1b7fe700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac04066f30 0x7fac040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:53.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.990+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fac1c013070 con 0x7fac2415db10 2023-10-06T03:02:53.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.990+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac04066f30 0x7fac040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:53.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.991+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac04066f30 0x7fac040693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac14002880 tx=0x7fac14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:53.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:53.994+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac2415b4d0 con 0x7fac2415db10 2023-10-06T03:02:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.255+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fac2415b4d0 con 0x7fac2415db10 2023-10-06T03:02:54.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.517+0000 7fac1b7fe700 1 -- 172.21.15.92:0/2588654372 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v91) v1 ==== 135+0+0 (secure 0 0 0) 0x7fac2415b4d0 con 0x7fac2415db10 2023-10-06T03:02:54.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac04066f30 msgr2=0x7fac040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac04066f30 0x7fac040693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac14002880 tx=0x7fac14009040 comp rx=0 tx=0).stop 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 msgr2=0x7fac2414cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 0x7fac2414cf20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00cdb0 tx=0x7fac1c0077d0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 shutdown_connections 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac04066f30 0x7fac040693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac2415db10 0x7fac2414cf20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac2415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 --2- 172.21.15.92:0/2588654372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac24159c60 0x7fac2416a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 >> 172.21.15.92:0/2588654372 conn(0x7fac24068e20 msgr2=0x7fac24158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:54.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 shutdown_connections 2023-10-06T03:02:54.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.520+0000 7fac29500700 1 -- 172.21.15.92:0/2588654372 wait complete. 2023-10-06T03:02:54.532 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2023-10-06T03:02:54.533 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2023-10-06T03:02:54.533 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:02:54.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-06T03:02:54.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.646+0000 7f9f370ff700 1 Processor -- start 2023-10-06T03:02:54.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.646+0000 7f9f370ff700 1 -- start start 2023-10-06T03:02:54.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f370ff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9f30159c60 0x7f9f3015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:54.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f370ff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f3015db10 0x7f9f30069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f370ff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3015a160 con 0x7f9f30159890 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f370ff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3015df50 con 0x7f9f3015db10 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f370ff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f300d13a0 con 0x7f9f30159c60 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f3569c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f3015db10 0x7f9f30069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f34e9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9f30159890 0x7f9f3015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42884/0 (socket says 172.21.15.92:42884) 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.647+0000 7f9f34e9b700 1 -- 172.21.15.92:0/515762992 learned_addr learned my addr 172.21.15.92:0/515762992 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f27fff700 1 --2- 172.21.15.92:0/515762992 >> v2:172.21.15.136:3300/0 conn(0x7f9f30159c60 0x7f9f3015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f3569c700 1 -- 172.21.15.92:0/515762992 >> v2:172.21.15.136:3300/0 conn(0x7f9f30159c60 msgr2=0x7f9f3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f3569c700 1 --2- 172.21.15.92:0/515762992 >> v2:172.21.15.136:3300/0 conn(0x7f9f30159c60 0x7f9f3015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f3569c700 1 -- 172.21.15.92:0/515762992 >> v1:172.21.15.92:6789/0 conn(0x7f9f30159890 legacy=0x7f9f3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:54.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f3569c700 1 -- 172.21.15.92:0/515762992 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3015e140 con 0x7f9f3015db10 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f277fe700 1 -- 172.21.15.92:0/515762992 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348168260 0 0) 0x7f9f3015a160 con 0x7f9f30159890 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f3569c700 1 --2- 172.21.15.92:0/515762992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f30069560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f300b67f0 tx=0x7f9f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47750287081f1feb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f277fe700 1 -- 172.21.15.92:0/515762992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800b6b0 con 0x7f9f3015db10 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f277fe700 1 -- 172.21.15.92:0/515762992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2800b810 con 0x7f9f3015db10 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f277fe700 1 -- 172.21.15.92:0/515762992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800bbb0 con 0x7f9f3015db10 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f370ff700 1 -- 172.21.15.92:0/515762992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 msgr2=0x7f9f30069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.648+0000 7f9f370ff700 1 --2- 172.21.15.92:0/515762992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f30069560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f300b67f0 tx=0x7f9f280091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.649+0000 7f9f370ff700 1 -- 172.21.15.92:0/515762992 shutdown_connections 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.649+0000 7f9f370ff700 1 --2- 172.21.15.92:0/515762992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f30069560 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.649+0000 7f9f370ff700 1 --2- 172.21.15.92:0/515762992 >> v2:172.21.15.136:3300/0 conn(0x7f9f30159c60 0x7f9f3015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.649+0000 7f9f370ff700 1 -- 172.21.15.92:0/515762992 >> 172.21.15.92:0/515762992 conn(0x7f9f30068e20 msgr2=0x7f9f300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:54.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.649+0000 7f9f370ff700 1 -- 172.21.15.92:0/515762992 shutdown_connections 2023-10-06T03:02:54.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.649+0000 7f9f370ff700 1 -- 172.21.15.92:0/515762992 wait complete. 2023-10-06T03:02:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.650+0000 7f9f370ff700 1 Processor -- start 2023-10-06T03:02:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.650+0000 7f9f370ff700 1 -- start start 2023-10-06T03:02:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f30159890 0x7f9f300b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f30159c60 0x7f9f300b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f300b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:54.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b1d80 con 0x7f9f3015db10 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300cfa20 con 0x7f9f30159890 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300b1170 con 0x7f9f30159c60 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f300b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f300b06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56142/0 (socket says 172.21.15.92:56142) 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 -- 172.21.15.92:0/1983341081 learned_addr learned my addr 172.21.15.92:0/1983341081 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f34e9b700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f30159890 0x7f9f300b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f27fff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f30159c60 0x7f9f300b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 -- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f30159c60 msgr2=0x7f9f300b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f30159c60 0x7f9f300b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 -- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f30159890 msgr2=0x7f9f300b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:54.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f30159890 0x7f9f300b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 -- 172.21.15.92:0/1983341081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28009420 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f3569c700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f300b06d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f30070e20 tx=0x7f9f28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28011420 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f28011580 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f300b1450 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28011940 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.651+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300b1a10 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.652+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f300b1ef0 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.653+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f28011cb0 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.653+0000 7f9f24ff9700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f1c066f30 0x7f9f1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.653+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f9f28085d70 con 0x7f9f3015db10 2023-10-06T03:02:54.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.653+0000 7f9f34e9b700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f1c066f30 0x7f9f1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:54.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.654+0000 7f9f34e9b700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f1c066f30 0x7f9f1c0693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c002250 tx=0x7f9f2c001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:54.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.657+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f28053120 con 0x7f9f3015db10 2023-10-06T03:02:54.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:54.914+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f9f3004c3c0 con 0x7f9f3015db10 2023-10-06T03:02:55.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.523+0000 7f9f24ff9700 1 -- 172.21.15.92:0/1983341081 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v92) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9f28056740 con 0x7f9f3015db10 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f1c066f30 msgr2=0x7f9f1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f1c066f30 0x7f9f1c0693e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c002250 tx=0x7f9f2c001a20 comp rx=0 tx=0).stop 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 msgr2=0x7f9f300b06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f300b06d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f30070e20 tx=0x7f9f28017000 comp rx=0 tx=0).stop 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 shutdown_connections 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f1c066f30 0x7f9f1c0693e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f3015db10 0x7f9f300b06d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f30159890 0x7f9f300b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 --2- 172.21.15.92:0/1983341081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f30159c60 0x7f9f300b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 >> 172.21.15.92:0/1983341081 conn(0x7f9f30068e20 msgr2=0x7f9f301553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:55.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 shutdown_connections 2023-10-06T03:02:55.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.525+0000 7f9f370ff700 1 -- 172.21.15.92:0/1983341081 wait complete. 2023-10-06T03:02:55.539 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:02:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.656+0000 7f832a104700 1 Processor -- start 2023-10-06T03:02:55.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.656+0000 7f832a104700 1 -- start start 2023-10-06T03:02:55.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f832a104700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83240ccc00 0x7f8324069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f832a104700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832406a020 0x7f83240cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f832a104700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240c9f30 con 0x7f8324159a60 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f832a104700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832406dfc0 con 0x7f83240ccc00 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f832a104700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240ca470 con 0x7f832406a020 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f8328901700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83240ccc00 0x7f8324069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f8329903700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832406a020 0x7f83240cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f8329102700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f8324159a60 0x7f83240c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:47388/0 (socket says 172.21.15.92:47388) 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.657+0000 7f8329102700 1 -- 172.21.15.92:0/359197614 learned_addr learned my addr 172.21.15.92:0/359197614 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f8328901700 1 -- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832406a020 msgr2=0x7f83240cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:55.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f8328901700 1 --2- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832406a020 0x7f83240cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f8328901700 1 -- 172.21.15.92:0/359197614 >> v1:172.21.15.136:6789/0 conn(0x7f8324159a60 legacy=0x7f83240c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f8328901700 1 -- 172.21.15.92:0/359197614 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240ca750 con 0x7f83240ccc00 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f8329903700 1 --2- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832406a020 0x7f83240cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f8328901700 1 --2- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83240ccc00 0x7f8324069ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83140098e0 tx=0x7f83140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42470056fdb3e8bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f831bfff700 1 -- 172.21.15.92:0/359197614 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400b6d0 con 0x7f83240ccc00 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f831bfff700 1 -- 172.21.15.92:0/359197614 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831400b830 con 0x7f83240ccc00 2023-10-06T03:02:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f831bfff700 1 -- 172.21.15.92:0/359197614 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400bbd0 con 0x7f83240ccc00 2023-10-06T03:02:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f832a104700 1 -- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83240ccc00 msgr2=0x7f8324069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f832a104700 1 --2- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83240ccc00 0x7f8324069ae0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f83140098e0 tx=0x7f83140091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f832a104700 1 -- 172.21.15.92:0/359197614 shutdown_connections 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.658+0000 7f832a104700 1 --2- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832406a020 0x7f83240cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.659+0000 7f832a104700 1 --2- 172.21.15.92:0/359197614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83240ccc00 0x7f8324069ae0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.659+0000 7f832a104700 1 -- 172.21.15.92:0/359197614 >> 172.21.15.92:0/359197614 conn(0x7f832406baa0 msgr2=0x7f832406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.659+0000 7f832a104700 1 -- 172.21.15.92:0/359197614 shutdown_connections 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.659+0000 7f832a104700 1 -- 172.21.15.92:0/359197614 wait complete. 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 Processor -- start 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 -- start start 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f832406a020 0x7f83240c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 0x7f83240c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8324159a60 0x7f8324155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240aa9e0 con 0x7f83240ccc00 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240b2ee0 con 0x7f832406a020 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.660+0000 7f832a104700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240c6b50 con 0x7f8324159a60 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 0x7f83240c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 0x7f83240c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56158/0 (socket says 172.21.15.92:56158) 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 -- 172.21.15.92:0/3897464972 learned_addr learned my addr 172.21.15.92:0/3897464972 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8329903700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8324159a60 0x7f8324155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8329102700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f832406a020 0x7f83240c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 -- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8324159a60 msgr2=0x7f8324155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8324159a60 0x7f8324155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 -- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f832406a020 msgr2=0x7f83240c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f832406a020 0x7f83240c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 -- 172.21.15.92:0/3897464972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320009420 con 0x7f83240ccc00 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f8328901700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 0x7f83240c6610 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f832406cdc0 tx=0x7f8314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831401bca0 con 0x7f83240ccc00 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831401be00 con 0x7f83240ccc00 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8314009420 con 0x7f83240ccc00 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314011670 con 0x7f83240ccc00 2023-10-06T03:02:55.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.661+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83240aa420 con 0x7f83240ccc00 2023-10-06T03:02:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.662+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240b3050 con 0x7f83240ccc00 2023-10-06T03:02:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.663+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8314011a20 con 0x7f83240ccc00 2023-10-06T03:02:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.663+0000 7f83197fa700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308066f30 0x7f83080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.663+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f8314085860 con 0x7f83240ccc00 2023-10-06T03:02:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.663+0000 7f8329102700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308066f30 0x7f83080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:55.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.663+0000 7f8329102700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308066f30 0x7f83080693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f831c00f380 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:55.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.667+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83240b3050 con 0x7f83240ccc00 2023-10-06T03:02:55.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.849+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f83240b3050 con 0x7f83240ccc00 2023-10-06T03:02:55.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.855+0000 7f83197fa700 1 -- 172.21.15.92:0/3897464972 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83240b3050 con 0x7f83240ccc00 2023-10-06T03:02:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308066f30 msgr2=0x7f83080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308066f30 0x7f83080693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f831c00f380 tx=0x7f831c009040 comp rx=0 tx=0).stop 2023-10-06T03:02:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 msgr2=0x7f83240c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:55.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 0x7f83240c6610 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f832406cdc0 tx=0x7f8314017000 comp rx=0 tx=0).stop 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 shutdown_connections 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308066f30 0x7f83080693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8324159a60 0x7f8324155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f832406a020 0x7f83240c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 --2- 172.21.15.92:0/3897464972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83240ccc00 0x7f83240c6610 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 >> 172.21.15.92:0/3897464972 conn(0x7f832406baa0 msgr2=0x7f83241570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 shutdown_connections 2023-10-06T03:02:55.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:55.857+0000 7f832a104700 1 -- 172.21.15.92:0/3897464972 wait complete. 2023-10-06T03:02:55.871 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:02:55.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:02:55.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:02:55.952 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:02:55.952 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:02:55.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:02:56.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.064+0000 7f7a33f6e700 1 Processor -- start 2023-10-06T03:02:56.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.064+0000 7f7a33f6e700 1 -- start start 2023-10-06T03:02:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a33f6e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7a2c069560 0x7f7a2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a33f6e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2c069ee0 con 0x7f7a2c159890 2023-10-06T03:02:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a33f6e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2c15a160 con 0x7f7a2c159c60 2023-10-06T03:02:56.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a33f6e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a2c0d13a0 con 0x7f7a2c069560 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a31d0a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7a2c159890 0x7f7a2c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42898/0 (socket says 172.21.15.92:42898) 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a31d0a700 1 -- 172.21.15.92:0/3513624649 learned_addr learned my addr 172.21.15.92:0/3513624649 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a3250b700 1 --2- 172.21.15.92:0/3513624649 >> v2:172.21.15.136:3300/0 conn(0x7f7a2c069560 0x7f7a2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.065+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183913571 0 0) 0x7f7a2c069ee0 con 0x7f7a2c159890 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a14003570 con 0x7f7a2c159890 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321021976 0 0) 0x7f7a2c15a160 con 0x7f7a2c159c60 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a2c069ee0 con 0x7f7a2c159c60 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4079166499 0 0) 0x7f7a14003570 con 0x7f7a2c159890 2023-10-06T03:02:56.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 >> v2:172.21.15.136:3300/0 conn(0x7f7a2c069560 msgr2=0x7f7a2c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 --2- 172.21.15.92:0/3513624649 >> v2:172.21.15.136:3300/0 conn(0x7f7a2c069560 0x7f7a2c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 >> v1:172.21.15.136:6789/0 conn(0x7f7a2c159c60 legacy=0x7f7a2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c15e140 con 0x7f7a2c159890 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7a1c0017f0 con 0x7f7a2c159890 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 >> v1:172.21.15.92:6789/0 conn(0x7f7a2c159890 legacy=0x7f7a2c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14003bf0 0x7f7a14003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a14004810 0x7f7a14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a140051c0 0x7f7a1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c0d13a0 con 0x7f7a14004810 2023-10-06T03:02:56.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1400c0b0 con 0x7f7a14003bf0 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1400c270 con 0x7f7a140051c0 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a14004810 0x7f7a14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31d0a700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14003bf0 0x7f7a14003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a3250b700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a140051c0 0x7f7a1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 -- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a140051c0 msgr2=0x7f7a1400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a140051c0 0x7f7a1400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 -- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14003bf0 msgr2=0x7f7a14003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14003bf0 0x7f7a14003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 -- 172.21.15.92:0/3513624649 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a14003570 con 0x7f7a14004810 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a3250b700 1 -- 172.21.15.92:0/3513624649 reap_dead start 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.066+0000 7f7a31509700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a14004810 0x7f7a14004c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7a20001a10 tx=0x7f7a20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d733ef28c8688a16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000c590 con 0x7f7a14004810 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2000c6f0 con 0x7f7a14004810 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a30d08700 1 -- 172.21.15.92:0/3513624649 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000ca90 con 0x7f7a14004810 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3513624649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a14004810 msgr2=0x7f7a14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a14004810 0x7f7a14004c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7a20001a10 tx=0x7f7a20007d30 comp rx=0 tx=0).stop 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3513624649 shutdown_connections 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3513624649 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a14004810 0x7f7a14004c80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3513624649 >> 172.21.15.92:0/3513624649 conn(0x7f7a2c068e20 msgr2=0x7f7a2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:56.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3513624649 shutdown_connections 2023-10-06T03:02:56.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.067+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3513624649 wait complete. 2023-10-06T03:02:56.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.068+0000 7f7a33f6e700 1 Processor -- start 2023-10-06T03:02:56.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.068+0000 7f7a33f6e700 1 -- start start 2023-10-06T03:02:56.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a33f6e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14004810 0x7f7a2c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a33f6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 0x7f7a2c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a33f6e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a2c0b5df0 0x7f7a2c0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a33f6e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c15d700 con 0x7f7a2c15bc40 2023-10-06T03:02:56.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a33f6e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c1533a0 con 0x7f7a14004810 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a33f6e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c15cd30 con 0x7f7a2c0b5df0 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 0x7f7a2c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 0x7f7a2c152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56172/0 (socket says 172.21.15.92:56172) 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 -- 172.21.15.92:0/3631004602 learned_addr learned my addr 172.21.15.92:0/3631004602 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31d0a700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14004810 0x7f7a2c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a3250b700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a2c0b5df0 0x7f7a2c0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 -- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a2c0b5df0 msgr2=0x7f7a2c0b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a2c0b5df0 0x7f7a2c0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 -- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14004810 msgr2=0x7f7a2c15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14004810 0x7f7a2c15d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 -- 172.21.15.92:0/3631004602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c15c120 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31d0a700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14004810 0x7f7a2c15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.069+0000 7f7a31509700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 0x7f7a2c152e60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c3c0 tx=0x7f7a20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.070+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20001360 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.070+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2000c860 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.070+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c15c400 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.070+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20006820 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.070+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c15c9c0 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.071+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c0b3120 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.071+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a20019070 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.072+0000 7f7a29ffb700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a18066f30 0x7f7a180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.072+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f7a20085b10 con 0x7f7a2c15bc40 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.072+0000 7f7a31d0a700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a18066f30 0x7f7a180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.072+0000 7f7a31d0a700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a18066f30 0x7f7a180693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c001190 tx=0x7f7a1c006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:56.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.076+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a20052ec0 con 0x7f7a2c15bc40 2023-10-06T03:02:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.298+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a2c15cec0 con 0x7f7a2c15bc40 2023-10-06T03:02:56.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.299+0000 7f7a29ffb700 1 -- 172.21.15.92:0/3631004602 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f7a20022de0 con 0x7f7a2c15bc40 2023-10-06T03:02:56.300 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:56.301 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":23,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2641828522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2641828522},{"type":"v1","addr":"172.21.15.136:6834","nonce":2641828522}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4649,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2131764246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2131764246},{"type":"v1","addr":"172.21.15.136:6835","nonce":2131764246}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1384819232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1384819232},{"type":"v1","addr":"172.21.15.92:6836","nonce":1384819232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4220014825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4220014825},{"type":"v1","addr":"172.21.15.92:6837","nonce":4220014825}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[]} 2023-10-06T03:02:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a18066f30 msgr2=0x7f7a180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a18066f30 0x7f7a180693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c001190 tx=0x7f7a1c006040 comp rx=0 tx=0).stop 2023-10-06T03:02:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 msgr2=0x7f7a2c152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 0x7f7a2c152e60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c3c0 tx=0x7f7a20015000 comp rx=0 tx=0).stop 2023-10-06T03:02:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 shutdown_connections 2023-10-06T03:02:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a18066f30 0x7f7a180693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a2c0b5df0 0x7f7a2c0b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a14004810 0x7f7a2c15d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 --2- 172.21.15.92:0/3631004602 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a2c15bc40 0x7f7a2c152e60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 >> 172.21.15.92:0/3631004602 conn(0x7f7a2c068e20 msgr2=0x7f7a2c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 shutdown_connections 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.301+0000 7f7a33f6e700 1 -- 172.21.15.92:0/3631004602 wait complete. 2023-10-06T03:02:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 23 2023-10-06T03:02:56.314 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:02:56.315 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:02:56.315 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:02:56.315 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:56.315 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:02:56.315 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:02:56.316 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:02:56.316 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:56.316 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:02:56.316 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:02:56.316 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:02:56.316 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:56.316 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:02:56.317 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:02:56.317 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:02:56.317 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:02:56.324 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:02:56.324 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:02:56.325 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:02:56.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:02:56.327 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:02:56.327 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:02:56.328 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:02:56.328 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:02:56.369 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:02:56.369 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:02:56.370 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:56.370 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:56.370 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:02:56.371 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:02:56.371 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:56.371 DEBUG:teuthology.parallel:result is None 2023-10-06T03:02:56.371 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:02:56.441 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:02:56.442 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:02:56.449 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:02:56.450 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:02:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.491+0000 7fe8e9622700 1 Processor -- start 2023-10-06T03:02:56.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.491+0000 7fe8e9622700 1 -- start start 2023-10-06T03:02:56.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e9622700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe8e40caa90 0x7fe8e40c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e9622700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e415d7e0 con 0x7fe8e406bcd0 2023-10-06T03:02:56.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e9622700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e415dac0 con 0x7fe8e406c0a0 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e9622700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e40caef0 con 0x7fe8e40caa90 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe8e406bcd0 0x7fe8e415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42914/0 (socket says 172.21.15.92:42914) 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e2ffd700 1 -- 172.21.15.92:0/190002069 learned_addr learned my addr 172.21.15.92:0/190002069 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.492+0000 7fe8e37fe700 1 --2- 172.21.15.92:0/190002069 >> v2:172.21.15.136:3300/0 conn(0x7fe8e40caa90 0x7fe8e40c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386003287 0 0) 0x7fe8e415d7e0 con 0x7fe8e406bcd0 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc003570 con 0x7fe8e406bcd0 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7888324 0 0) 0x7fe8e415dac0 con 0x7fe8e406c0a0 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e415d7e0 con 0x7fe8e406c0a0 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e37fe700 1 -- 172.21.15.92:0/190002069 >> v1:172.21.15.136:6789/0 conn(0x7fe8e406c0a0 legacy=0x7fe8e415df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e37fe700 1 -- 172.21.15.92:0/190002069 >> v1:172.21.15.92:6789/0 conn(0x7fe8e406bcd0 legacy=0x7fe8e415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e37fe700 1 -- 172.21.15.92:0/190002069 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e415aa30 con 0x7fe8e40caa90 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e37fe700 1 --2- 172.21.15.92:0/190002069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e40caa90 0x7fe8e40c8c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009e60 tx=0x7fe8dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16d07369437bc70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00e070 con 0x7fe8e40caa90 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc00bab0 con 0x7fe8e40caa90 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.493+0000 7fe8e1ffb700 1 -- 172.21.15.92:0/190002069 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00be50 con 0x7fe8e40caa90 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 -- 172.21.15.92:0/190002069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e40caa90 msgr2=0x7fe8e40c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 --2- 172.21.15.92:0/190002069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e40caa90 0x7fe8e40c8c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009e60 tx=0x7fe8dc009780 comp rx=0 tx=0).stop 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 -- 172.21.15.92:0/190002069 shutdown_connections 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 --2- 172.21.15.92:0/190002069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e40caa90 0x7fe8e40c8c40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 -- 172.21.15.92:0/190002069 >> 172.21.15.92:0/190002069 conn(0x7fe8e40ce600 msgr2=0x7fe8e40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 -- 172.21.15.92:0/190002069 shutdown_connections 2023-10-06T03:02:56.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.494+0000 7fe8e9622700 1 -- 172.21.15.92:0/190002069 wait complete. 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.495+0000 7fe8e9622700 1 Processor -- start 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.495+0000 7fe8e9622700 1 -- start start 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8e406bcd0 0x7fe8e406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e406c0a0 0x7fe8e416d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 0x7fe8e416d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4155da0 con 0x7fe8e40caa90 2023-10-06T03:02:56.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e416e130 con 0x7fe8e406c0a0 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e416de50 con 0x7fe8e406bcd0 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 0x7fe8e416d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 0x7fe8e416d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56202/0 (socket says 172.21.15.92:56202) 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 -- 172.21.15.92:0/2642238330 learned_addr learned my addr 172.21.15.92:0/2642238330 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e27fc700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e406c0a0 0x7fe8e416d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e2ffd700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8e406bcd0 0x7fe8e406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 -- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8e406bcd0 msgr2=0x7fe8e406a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8e406bcd0 0x7fe8e406a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 -- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e406c0a0 msgr2=0x7fe8e416d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e406c0a0 0x7fe8e416d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 -- 172.21.15.92:0/2642238330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc009a20 con 0x7fe8e40caa90 2023-10-06T03:02:56.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e27fc700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e406c0a0 0x7fe8e416d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e37fe700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 0x7fe8e416d780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4069220 tx=0x7fe8dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc012d00 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc029070 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc025aa0 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e4169de0 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.496+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e416a3a0 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.497+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e40cf510 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.498+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8dc030480 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.498+0000 7fe8caffd700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.498+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fe8dc02b070 con 0x7fe8e40caa90 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.498+0000 7fe8e2ffd700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:56.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.499+0000 7fe8e2ffd700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe8d0066f30 0x7fe8d00693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007fd0 tx=0x7fe8d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:56.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.502+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8dc05bab0 con 0x7fe8e40caa90 2023-10-06T03:02:56.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-06T03:02:56.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T03:02:56.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-06T03:02:56.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:56.702+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe8e416a5c0 con 0x7fe8e40caa90 2023-10-06T03:02:57.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.537+0000 7fe8caffd700 1 -- 172.21.15.92:0/2642238330 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8dc012e60 con 0x7fe8e40caa90 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe8d0066f30 msgr2=0x7fe8d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe8d0066f30 0x7fe8d00693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007fd0 tx=0x7fe8d4009040 comp rx=0 tx=0).stop 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 msgr2=0x7fe8e416d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 0x7fe8e416d780 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4069220 tx=0x7fe8dc019000 comp rx=0 tx=0).stop 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 shutdown_connections 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe8e40caa90 0x7fe8e416d780 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8e406bcd0 0x7fe8e406a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 --2- 172.21.15.92:0/2642238330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8e406c0a0 0x7fe8e416d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 >> 172.21.15.92:0/2642238330 conn(0x7fe8e40ce600 msgr2=0x7fe8e40c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:57.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 shutdown_connections 2023-10-06T03:02:57.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.540+0000 7fe8e9622700 1 -- 172.21.15.92:0/2642238330 wait complete. 2023-10-06T03:02:57.543 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:02:57.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:02:57.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.672+0000 7f299ad28700 1 Processor -- start 2023-10-06T03:02:57.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.672+0000 7f299ad28700 1 -- start start 2023-10-06T03:02:57.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f299ad28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2994159890 0x7f299415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f299ad28700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f29940699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f299ad28700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2994069ee0 con 0x7f2994159c60 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f299ad28700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299415a160 con 0x7f2994159890 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f299ad28700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940d13a0 con 0x7f2994069560 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f2998ac4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2994159890 0x7f299415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f2998ac4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2994159890 0x7f299415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56212/0 (socket says 172.21.15.92:56212) 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f2998ac4700 1 -- 172.21.15.92:0/1152728844 learned_addr learned my addr 172.21.15.92:0/1152728844 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f29992c5700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f29940699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f2998ac4700 1 -- 172.21.15.92:0/1152728844 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 msgr2=0x7f29940699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f2998ac4700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f29940699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.673+0000 7f2998ac4700 1 -- 172.21.15.92:0/1152728844 >> v1:172.21.15.136:6789/0 conn(0x7f2994159c60 legacy=0x7f299415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f2998ac4700 1 -- 172.21.15.92:0/1152728844 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299415e140 con 0x7f2994159890 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f29992c5700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f29940699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f2998ac4700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f299415abd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29880099b0 tx=0x7f2988009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49ba4d5ba5d336 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f29937fe700 1 -- 172.21.15.92:0/1152728844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800b820 con 0x7f2994159890 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f29937fe700 1 -- 172.21.15.92:0/1152728844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298800b980 con 0x7f2994159890 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f29937fe700 1 -- 172.21.15.92:0/1152728844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800bd20 con 0x7f2994159890 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 -- 172.21.15.92:0/1152728844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 msgr2=0x7f299415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f299415abd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29880099b0 tx=0x7f2988009250 comp rx=0 tx=0).stop 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 -- 172.21.15.92:0/1152728844 shutdown_connections 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f299415abd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 --2- 172.21.15.92:0/1152728844 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f29940699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 -- 172.21.15.92:0/1152728844 >> 172.21.15.92:0/1152728844 conn(0x7f2994068e20 msgr2=0x7f29940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 -- 172.21.15.92:0/1152728844 shutdown_connections 2023-10-06T03:02:57.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.674+0000 7f299ad28700 1 -- 172.21.15.92:0/1152728844 wait complete. 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.675+0000 7f299ad28700 1 Processor -- start 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 -- start start 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f299416a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f29940ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2994159c60 0x7f2994148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940cf2a0 con 0x7f2994159890 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940cdb00 con 0x7f2994159c60 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f299ad28700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940cf4d0 con 0x7f2994069560 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f2993fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f29940ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f2993fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f29940ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56214/0 (socket says 172.21.15.92:56214) 2023-10-06T03:02:57.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.676+0000 7f2993fff700 1 -- 172.21.15.92:0/2953930347 learned_addr learned my addr 172.21.15.92:0/2953930347 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f29992c5700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2994159c60 0x7f2994148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2998ac4700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f299416a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2993fff700 1 -- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 msgr2=0x7f299416a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2993fff700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f299416a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2993fff700 1 -- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2994159c60 msgr2=0x7f2994148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2993fff700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2994159c60 0x7f2994148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2993fff700 1 -- 172.21.15.92:0/2953930347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880094f0 con 0x7f2994159890 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2993fff700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f29940ced60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f298000c5d0 tx=0x7f2980007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980011590 con 0x7f2994159890 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2980011710 con 0x7f2994159890 2023-10-06T03:02:57.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29940cd1d0 con 0x7f2994159890 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980011ad0 con 0x7f2994159890 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.677+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29940cd790 con 0x7f2994159890 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.678+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29940cdc70 con 0x7f2994159890 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.678+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f298001c460 con 0x7f2994159890 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.679+0000 7f2990ff9700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2978066f30 0x7f29780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.679+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f29800840f0 con 0x7f2994159890 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.679+0000 7f2998ac4700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2978066f30 0x7f29780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.679+0000 7f2998ac4700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2978066f30 0x7f29780693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29880099b0 tx=0x7f2988016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:57.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.683+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2980051550 con 0x7f2994159890 2023-10-06T03:02:57.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.940+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f299404c3c0 con 0x7f2994159890 2023-10-06T03:02:57.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.941+0000 7f2990ff9700 1 -- 172.21.15.92:0/2953930347 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2980054b70 con 0x7f2994159890 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.943 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:57.944 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:57.945 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:57.946 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.947 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:02:57.948 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.949 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.950 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:02:57.951 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:02:57.952 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.953 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:02:57.954 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:02:57.955 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:02:57.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:02:57.957 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:02:57.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:57.959 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:02:57.959 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:02:57.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.942+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2978066f30 msgr2=0x7f29780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.942+0000 7f299ad28700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2978066f30 0x7f29780693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f29880099b0 tx=0x7f2988016000 comp rx=0 tx=0).stop 2023-10-06T03:02:57.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.942+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 msgr2=0x7f29940ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:57.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.942+0000 7f299ad28700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f29940ced60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f298000c5d0 tx=0x7f2980007f60 comp rx=0 tx=0).stop 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 shutdown_connections 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2978066f30 0x7f29780693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2994159c60 0x7f2994148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2994069560 0x7f299416a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 --2- 172.21.15.92:0/2953930347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2994159890 0x7f29940ced60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 >> 172.21.15.92:0/2953930347 conn(0x7f2994068e20 msgr2=0x7f299415b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 shutdown_connections 2023-10-06T03:02:57.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:57.943+0000 7f299ad28700 1 -- 172.21.15.92:0/2953930347 wait complete. 2023-10-06T03:02:57.962 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:02:58.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 Processor -- start 2023-10-06T03:02:58.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 -- start start 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2630159c60 0x7f263015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f263015db10 0x7f2630069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263015a160 con 0x7f2630159890 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263015df50 con 0x7f2630159c60 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.074+0000 7f263655e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300d13a0 con 0x7f263015db10 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2630159c60 0x7f263015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2630159c60 0x7f263015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56230/0 (socket says 172.21.15.92:56230) 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 -- 172.21.15.92:0/1357465722 learned_addr learned my addr 172.21.15.92:0/1357465722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f2634afb700 1 --2- 172.21.15.92:0/1357465722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f263015db10 0x7f2630069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 -- 172.21.15.92:0/1357465722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f263015db10 msgr2=0x7f2630069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 --2- 172.21.15.92:0/1357465722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f263015db10 0x7f2630069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 -- 172.21.15.92:0/1357465722 >> v1:172.21.15.136:6789/0 conn(0x7f2630159890 legacy=0x7f263015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 -- 172.21.15.92:0/1357465722 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263015e140 con 0x7f2630159c60 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262f7fe700 1 --2- 172.21.15.92:0/1357465722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2630159c60 0x7f263015d5d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26240099b0 tx=0x7f2624009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edda279de35913c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262effd700 1 -- 172.21.15.92:0/1357465722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400b820 con 0x7f2630159c60 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262effd700 1 -- 172.21.15.92:0/1357465722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262400b980 con 0x7f2630159c60 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.075+0000 7f262effd700 1 -- 172.21.15.92:0/1357465722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400bd20 con 0x7f2630159c60 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 -- 172.21.15.92:0/1357465722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2630159c60 msgr2=0x7f263015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 --2- 172.21.15.92:0/1357465722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2630159c60 0x7f263015d5d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26240099b0 tx=0x7f2624009250 comp rx=0 tx=0).stop 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 -- 172.21.15.92:0/1357465722 shutdown_connections 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 --2- 172.21.15.92:0/1357465722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f263015db10 0x7f2630069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 --2- 172.21.15.92:0/1357465722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2630159c60 0x7f263015d5d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 -- 172.21.15.92:0/1357465722 >> 172.21.15.92:0/1357465722 conn(0x7f2630068e20 msgr2=0x7f26300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:58.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 -- 172.21.15.92:0/1357465722 shutdown_connections 2023-10-06T03:02:58.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.076+0000 7f263655e700 1 -- 172.21.15.92:0/1357465722 wait complete. 2023-10-06T03:02:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.077+0000 7f263655e700 1 Processor -- start 2023-10-06T03:02:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.077+0000 7f263655e700 1 -- start start 2023-10-06T03:02:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f263655e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2630159890 0x7f26300b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f263655e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2630159c60 0x7f26300cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f263655e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 0x7f2630153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f263655e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300ce140 con 0x7f263015db10 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f263655e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300cd770 con 0x7f2630159c60 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f263655e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300cd160 con 0x7f2630159890 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 0x7f2630153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 0x7f2630153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56238/0 (socket says 172.21.15.92:56238) 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 -- 172.21.15.92:0/4190355103 learned_addr learned my addr 172.21.15.92:0/4190355103 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 -- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2630159890 msgr2=0x7f26300b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f262f7fe700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2630159c60 0x7f26300cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f262ffff700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2630159890 0x7f26300b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2630159890 0x7f26300b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 -- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2630159c60 msgr2=0x7f26300cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2630159c60 0x7f26300cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 -- 172.21.15.92:0/4190355103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240094f0 con 0x7f263015db10 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2634afb700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 0x7f2630153980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2628007f20 tx=0x7f2628009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.078+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262800eca0 con 0x7f263015db10 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.079+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262800ee20 con 0x7f263015db10 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.079+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628014670 con 0x7f263015db10 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.079+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26300cfbc0 con 0x7f263015db10 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.079+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26300cfea0 con 0x7f263015db10 2023-10-06T03:02:58.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.079+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26300cd8e0 con 0x7f263015db10 2023-10-06T03:02:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.080+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26280149e0 con 0x7f263015db10 2023-10-06T03:02:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.080+0000 7f2617fff700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.080+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2628084a50 con 0x7f263015db10 2023-10-06T03:02:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.080+0000 7f262ffff700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.081+0000 7f262ffff700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f261c066f30 0x7f261c0693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26200027e0 tx=0x7f2620009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.084+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2628051eb0 con 0x7f263015db10 2023-10-06T03:02:58.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.269+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f263004c3c0 con 0x7f263015db10 2023-10-06T03:02:58.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.270+0000 7f2617fff700 1 -- 172.21.15.92:0/4190355103 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13650 (secure 0 0 0) 0x7f26280554d0 con 0x7f263015db10 2023-10-06T03:02:58.272 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:58.272 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":50,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:02:57.535596+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:02:58.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f261c066f30 msgr2=0x7f261c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f261c066f30 0x7f261c0693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26200027e0 tx=0x7f2620009040 comp rx=0 tx=0).stop 2023-10-06T03:02:58.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 msgr2=0x7f2630153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 0x7f2630153980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2628007f20 tx=0x7f2628009860 comp rx=0 tx=0).stop 2023-10-06T03:02:58.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 shutdown_connections 2023-10-06T03:02:58.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f261c066f30 0x7f261c0693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f263015db10 0x7f2630153980 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2630159890 0x7f26300b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 --2- 172.21.15.92:0/4190355103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2630159c60 0x7f26300cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 >> 172.21.15.92:0/4190355103 conn(0x7f2630068e20 msgr2=0x7f2630158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 shutdown_connections 2023-10-06T03:02:58.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.272+0000 7f263655e700 1 -- 172.21.15.92:0/4190355103 wait complete. 2023-10-06T03:02:58.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:02:58.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:02:58.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.402+0000 7fa2f224b700 1 Processor -- start 2023-10-06T03:02:58.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.402+0000 7fa2f224b700 1 -- start start 2023-10-06T03:02:58.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2f224b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2ec069560 0x7fa2ec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2f224b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec069ee0 con 0x7fa2ec159890 2023-10-06T03:02:58.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2f224b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec15a160 con 0x7fa2ec159c60 2023-10-06T03:02:58.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2f224b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0d13a0 con 0x7fa2ec069560 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2ebfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2ec069560 0x7fa2ec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2eb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa2ec159890 0x7fa2ec15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42928/0 (socket says 172.21.15.92:42928) 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.403+0000 7fa2eb7fe700 1 -- 172.21.15.92:0/2814691511 learned_addr learned my addr 172.21.15.92:0/2814691511 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ebfff700 1 -- 172.21.15.92:0/2814691511 >> v1:172.21.15.136:6789/0 conn(0x7fa2ec159c60 legacy=0x7fa2ec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ebfff700 1 -- 172.21.15.92:0/2814691511 >> v1:172.21.15.92:6789/0 conn(0x7fa2ec159890 legacy=0x7fa2ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ebfff700 1 -- 172.21.15.92:0/2814691511 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec15e140 con 0x7fa2ec069560 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ea7fc700 1 -- 172.21.15.92:0/2814691511 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203869707 0 0) 0x7fa2ec069ee0 con 0x7fa2ec159890 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ebfff700 1 --2- 172.21.15.92:0/2814691511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec0699a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40098e0 tx=0x7fa2e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1174186f74376ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ea7fc700 1 -- 172.21.15.92:0/2814691511 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4009e80 con 0x7fa2ec069560 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ea7fc700 1 -- 172.21.15.92:0/2814691511 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e400b860 con 0x7fa2ec069560 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2ea7fc700 1 -- 172.21.15.92:0/2814691511 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400bbe0 con 0x7fa2ec069560 2023-10-06T03:02:58.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2f224b700 1 -- 172.21.15.92:0/2814691511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 msgr2=0x7fa2ec0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2f224b700 1 --2- 172.21.15.92:0/2814691511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec0699a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40098e0 tx=0x7fa2e40091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2f224b700 1 -- 172.21.15.92:0/2814691511 shutdown_connections 2023-10-06T03:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2f224b700 1 --2- 172.21.15.92:0/2814691511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec0699a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.404+0000 7fa2f224b700 1 -- 172.21.15.92:0/2814691511 >> 172.21.15.92:0/2814691511 conn(0x7fa2ec068e20 msgr2=0x7fa2ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.405+0000 7fa2f224b700 1 -- 172.21.15.92:0/2814691511 shutdown_connections 2023-10-06T03:02:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.405+0000 7fa2f224b700 1 -- 172.21.15.92:0/2814691511 wait complete. 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 Processor -- start 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 -- start start 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2ec159890 0x7fa2ec0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2ec159c60 0x7fa2ec0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec149880 con 0x7fa2ec069560 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0cd0c0 con 0x7fa2ec159c60 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.406+0000 7fa2f224b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0c8320 con 0x7fa2ec159890 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56254/0 (socket says 172.21.15.92:56254) 2023-10-06T03:02:58.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 -- 172.21.15.92:0/1106271871 learned_addr learned my addr 172.21.15.92:0/1106271871 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2ebfff700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2ec159c60 0x7fa2ec0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eaffd700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2ec159890 0x7fa2ec0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 -- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2ec159890 msgr2=0x7fa2ec0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2ec159890 0x7fa2ec0b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 -- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2ec159c60 msgr2=0x7fa2ec0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2ec159c60 0x7fa2ec0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 -- 172.21.15.92:0/1106271871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4009420 con 0x7fa2ec069560 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eaffd700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2ec159890 0x7fa2ec0b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2eb7fe700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec149340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ccc0 tx=0x7fa2dc001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc00ab10 con 0x7fa2ec069560 2023-10-06T03:02:58.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2dc00ac70 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec0c8600 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0070f0 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.407+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec0cd310 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.408+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec0cd530 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.408+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2dc007540 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.409+0000 7fa2d37fe700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2d8066f30 0x7fa2d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.409+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa2dc084790 con 0x7fa2ec069560 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.409+0000 7fa2eaffd700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2d8066f30 0x7fa2d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.409+0000 7fa2eaffd700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2d8066f30 0x7fa2d80693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0b0330 tx=0x7fa2e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.413+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2dc051c70 con 0x7fa2ec069560 2023-10-06T03:02:58.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.597+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2ec04c3c0 con 0x7fa2ec069560 2023-10-06T03:02:58.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.597+0000 7fa2d37fe700 1 -- 172.21.15.92:0/1106271871 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2dc055290 con 0x7fa2ec069560 2023-10-06T03:02:58.599 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:02:58.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2d8066f30 msgr2=0x7fa2d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2d8066f30 0x7fa2d80693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0b0330 tx=0x7fa2e0009040 comp rx=0 tx=0).stop 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 msgr2=0x7fa2ec149340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec149340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ccc0 tx=0x7fa2dc001a80 comp rx=0 tx=0).stop 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 shutdown_connections 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2d8066f30 0x7fa2d80693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2ec069560 0x7fa2ec149340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2ec159c60 0x7fa2ec0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.599+0000 7fa2f224b700 1 --2- 172.21.15.92:0/1106271871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2ec159890 0x7fa2ec0b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.600+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 >> 172.21.15.92:0/1106271871 conn(0x7fa2ec068e20 msgr2=0x7fa2ec1553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.600+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 shutdown_connections 2023-10-06T03:02:58.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.600+0000 7fa2f224b700 1 -- 172.21.15.92:0/1106271871 wait complete. 2023-10-06T03:02:58.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:02:58.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.731+0000 7f6f3d0f2700 1 Processor -- start 2023-10-06T03:02:58.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.731+0000 7f6f3d0f2700 1 -- start start 2023-10-06T03:02:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f3d0f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f380b16d0 0x7f6f380b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f3d0f2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f380b2080 con 0x7f6f380b4e70 2023-10-06T03:02:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f3d0f2700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f380cfb00 con 0x7f6f380b33b0 2023-10-06T03:02:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f3d0f2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f380b5c40 con 0x7f6f380b16d0 2023-10-06T03:02:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f37fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6f380b4e70 0x7f6f380ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42932/0 (socket says 172.21.15.92:42932) 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f37fff700 1 -- 172.21.15.92:0/3943214113 learned_addr learned my addr 172.21.15.92:0/3943214113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f377fe700 1 --2- 172.21.15.92:0/3943214113 >> v2:172.21.15.92:3300/0 conn(0x7f6f380b16d0 0x7f6f380b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f36ffd700 1 -- 172.21.15.92:0/3943214113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155531322 0 0) 0x7f6f380b2080 con 0x7f6f380b4e70 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f36ffd700 1 -- 172.21.15.92:0/3943214113 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f20003570 con 0x7f6f380b4e70 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f377fe700 1 -- 172.21.15.92:0/3943214113 >> v1:172.21.15.136:6789/0 conn(0x7f6f380b33b0 legacy=0x7f6f380b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f377fe700 1 -- 172.21.15.92:0/3943214113 >> v1:172.21.15.92:6789/0 conn(0x7f6f380b4e70 legacy=0x7f6f380ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f377fe700 1 -- 172.21.15.92:0/3943214113 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f380b5f20 con 0x7f6f380b16d0 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f377fe700 1 --2- 172.21.15.92:0/3943214113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f380b1b40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0098e0 tx=0x7f6f2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc281e948b423654 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f36ffd700 1 -- 172.21.15.92:0/3943214113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c011630 con 0x7f6f380b16d0 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f36ffd700 1 -- 172.21.15.92:0/3943214113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c011790 con 0x7f6f380b16d0 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.732+0000 7f6f36ffd700 1 -- 172.21.15.92:0/3943214113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c011b30 con 0x7f6f380b16d0 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/3943214113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 msgr2=0x7f6f380b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/3943214113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f380b1b40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0098e0 tx=0x7f6f2c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/3943214113 shutdown_connections 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/3943214113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f380b1b40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/3943214113 >> 172.21.15.92:0/3943214113 conn(0x7f6f380d10c0 msgr2=0x7f6f380affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/3943214113 shutdown_connections 2023-10-06T03:02:58.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.733+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/3943214113 wait complete. 2023-10-06T03:02:58.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.734+0000 7f6f3d0f2700 1 Processor -- start 2023-10-06T03:02:58.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.734+0000 7f6f3d0f2700 1 -- start start 2023-10-06T03:02:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3d0f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f3816a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3d0f2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f380b33b0 0x7f6f380cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3d0f2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f380b4e70 0x7f6f38157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3d0f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f380a7e50 con 0x7f6f380b16d0 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3d0f2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3816a930 con 0x7f6f380b4e70 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3d0f2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f380ccba0 con 0x7f6f380b33b0 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f3816a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f3816a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56280/0 (socket says 172.21.15.92:56280) 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 -- 172.21.15.92:0/676341122 learned_addr learned my addr 172.21.15.92:0/676341122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 -- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f380b33b0 msgr2=0x7f6f380cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f380b33b0 0x7f6f380cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 -- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f380b4e70 msgr2=0x7f6f38157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3c8f1700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f380b4e70 0x7f6f38157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f380b4e70 0x7f6f38157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f37fff700 1 -- 172.21.15.92:0/676341122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c009420 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.735+0000 7f6f3c8f1700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f380b4e70 0x7f6f38157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.736+0000 7f6f37fff700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f3816a3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c590 tx=0x7f6f28007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.736+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28005b30 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.736+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f28005cb0 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.736+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f380c9b30 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.736+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28007320 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.736+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f380cc130 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.737+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f380cc4b0 con 0x7f6f380b16d0 2023-10-06T03:02:58.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.737+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f2801b3b0 con 0x7f6f380b16d0 2023-10-06T03:02:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.737+0000 7f6f1ffff700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f24066f30 0x7f6f240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.738+0000 7f6f377fe700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f24066f30 0x7f6f240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.738+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6f28083910 con 0x7f6f380b16d0 2023-10-06T03:02:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.738+0000 7f6f377fe700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f24066f30 0x7f6f240693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f38158490 tx=0x7f6f2c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:58.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.742+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f28050d70 con 0x7f6f380b16d0 2023-10-06T03:02:58.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.922+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f3804bb80 con 0x7f6f380b16d0 2023-10-06T03:02:58.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.922+0000 7f6f1ffff700 1 -- 172.21.15.92:0/676341122 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f28017020 con 0x7f6f380b16d0 2023-10-06T03:02:58.924 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:02:58.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f24066f30 msgr2=0x7f6f240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f24066f30 0x7f6f240693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6f38158490 tx=0x7f6f2c016000 comp rx=0 tx=0).stop 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 msgr2=0x7f6f3816a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f3816a3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c590 tx=0x7f6f28007e00 comp rx=0 tx=0).stop 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 shutdown_connections 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f24066f30 0x7f6f240693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f380b16d0 0x7f6f3816a3f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f380b4e70 0x7f6f38157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 --2- 172.21.15.92:0/676341122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f380b33b0 0x7f6f380cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.924+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 >> 172.21.15.92:0/676341122 conn(0x7f6f380d10c0 msgr2=0x7f6f38069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.925+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 shutdown_connections 2023-10-06T03:02:58.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:58.925+0000 7f6f3d0f2700 1 -- 172.21.15.92:0/676341122 wait complete. 2023-10-06T03:02:58.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:02:59.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.054+0000 7f34a567f700 1 Processor -- start 2023-10-06T03:02:59.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.054+0000 7f34a567f700 1 -- start start 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.055+0000 7f34a567f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f34a00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.055+0000 7f34a567f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a00cc300 con 0x7f34a00b1420 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.055+0000 7f34a567f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a00ad370 con 0x7f34a00cbf30 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.055+0000 7f34a567f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0156bf0 con 0x7f34a00ccf90 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.055+0000 7f349e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f34a00b1420 0x7f34a00c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:42948/0 (socket says 172.21.15.92:42948) 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.055+0000 7f349e7fc700 1 -- 172.21.15.92:0/2703281980 learned_addr learned my addr 172.21.15.92:0/2703281980 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f34a00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337153781 0 0) 0x7f34a00cc300 con 0x7f34a00b1420 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3488003570 con 0x7f34a00b1420 2023-10-06T03:02:59.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611495512 0 0) 0x7f3488003570 con 0x7f34a00b1420 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 msgr2=0x7f34a00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f34a00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 >> v1:172.21.15.136:6789/0 conn(0x7f34a00cbf30 legacy=0x7f34a00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00ad650 con 0x7f34a00b1420 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f3494001470 con 0x7f34a00b1420 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 >> v1:172.21.15.92:6789/0 conn(0x7f34a00b1420 legacy=0x7f34a00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f3488003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488004010 0x7f3488004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488004d50 0x7f34880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0156bf0 con 0x7f3488004d50 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348800bbf0 con 0x7f3488004010 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348800bdb0 con 0x7f34a00ccf90 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488004d50 0x7f34880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349effd700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f3488003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349e7fc700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488004010 0x7f3488004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 -- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 msgr2=0x7f3488003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f3488003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 -- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488004010 msgr2=0x7f3488004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488004010 0x7f3488004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 -- 172.21.15.92:0/2703281980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488003570 con 0x7f3488004d50 2023-10-06T03:02:59.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349effd700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a00ccf90 0x7f3488003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349e7fc700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488004010 0x7f3488004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.056+0000 7f349f7fe700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488004d50 0x7f34880051c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3490006f00 tx=0x7f34900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a89b7d5e51faf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000e710 con 0x7f3488004d50 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000e870 con 0x7f3488004d50 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f349dffb700 1 -- 172.21.15.92:0/2703281980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000ec30 con 0x7f3488004d50 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f34a567f700 1 -- 172.21.15.92:0/2703281980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488004d50 msgr2=0x7f34880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f34a567f700 1 --2- 172.21.15.92:0/2703281980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488004d50 0x7f34880051c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3490006f00 tx=0x7f34900091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f349f7fe700 1 -- 172.21.15.92:0/2703281980 reap_dead start 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f34a567f700 1 -- 172.21.15.92:0/2703281980 shutdown_connections 2023-10-06T03:02:59.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f34a567f700 1 -- 172.21.15.92:0/2703281980 >> 172.21.15.92:0/2703281980 conn(0x7f34a006a320 msgr2=0x7f34a00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f34a567f700 1 -- 172.21.15.92:0/2703281980 shutdown_connections 2023-10-06T03:02:59.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.057+0000 7f34a567f700 1 -- 172.21.15.92:0/2703281980 wait complete. 2023-10-06T03:02:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.058+0000 7f34a567f700 1 Processor -- start 2023-10-06T03:02:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 -- start start 2023-10-06T03:02:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 0x7f34a00abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a0145320 0x7f34a015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34a00ac2a0 0x7f34a015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a00a80e0 con 0x7f34a0144f50 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a00a7510 con 0x7f34a00ac2a0 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f34a567f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a00a77b0 con 0x7f34a0145320 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f349effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 0x7f34a00abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f349effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 0x7f34a00abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56300/0 (socket says 172.21.15.92:56300) 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.059+0000 7f349effd700 1 -- 172.21.15.92:0/413798872 learned_addr learned my addr 172.21.15.92:0/413798872 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f349effd700 1 -- 172.21.15.92:0/413798872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a0145320 msgr2=0x7f34a015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f349effd700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a0145320 0x7f34a015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f349effd700 1 -- 172.21.15.92:0/413798872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34a00ac2a0 msgr2=0x7f34a015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f349effd700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34a00ac2a0 0x7f34a015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f349effd700 1 -- 172.21.15.92:0/413798872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00a7a90 con 0x7f34a0144f50 2023-10-06T03:02:59.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f349effd700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 0x7f34a00abd60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3498002230 tx=0x7f3498009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498018070 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3498010040 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a00a7d70 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3498006dd0 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.060+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a00a54c0 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.061+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a004c3c0 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.061+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3498003030 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.062+0000 7f3486ffd700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f348c066f30 0x7f348c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.062+0000 7f349e7fc700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f348c066f30 0x7f348c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.062+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3498083d90 con 0x7f34a0144f50 2023-10-06T03:02:59.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.062+0000 7f349e7fc700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f348c066f30 0x7f348c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34a0145770 tx=0x7f3494009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:59.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.066+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34980511f0 con 0x7f34a0144f50 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.252+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f34a00ad930 con 0x7f34a0144f50 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.252+0000 7f3486ffd700 1 -- 172.21.15.92:0/413798872 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3498054810 con 0x7f34a0144f50 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:02:59.254 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:02:59.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:02:59.255 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:02:59.255 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:02:59.255 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:02:59.255 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:02:59.255 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f348c066f30 msgr2=0x7f348c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f348c066f30 0x7f348c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34a0145770 tx=0x7f3494009040 comp rx=0 tx=0).stop 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 msgr2=0x7f34a00abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 0x7f34a00abd60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3498002230 tx=0x7f3498009040 comp rx=0 tx=0).stop 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 shutdown_connections 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f348c066f30 0x7f348c0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34a0144f50 0x7f34a00abd60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34a00ac2a0 0x7f34a015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 --2- 172.21.15.92:0/413798872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34a0145320 0x7f34a015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 >> 172.21.15.92:0/413798872 conn(0x7f34a006a320 msgr2=0x7f34a006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 shutdown_connections 2023-10-06T03:02:59.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.254+0000 7f34a567f700 1 -- 172.21.15.92:0/413798872 wait complete. 2023-10-06T03:02:59.267 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:02:59.267 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:02:59.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.384+0000 7fb337659700 1 Processor -- start 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.384+0000 7fb337659700 1 -- start start 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb337659700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb330159ec0 0x7fb3300b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb337659700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3300cf350 0x7fb3300cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb337659700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300b5b00 con 0x7fb3300b53d0 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb337659700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300b5850 con 0x7fb3300cf350 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb337659700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300b5da0 con 0x7fb330159ec0 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb335bf6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3300cf350 0x7fb3300cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb334bf4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb3300b53d0 0x7fb3300cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34302/0 (socket says 172.21.15.92:34302) 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb334bf4700 1 -- 172.21.15.92:0/285379629 learned_addr learned my addr 172.21.15.92:0/285379629 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:02:59.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.385+0000 7fb335bf6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb3300cf350 0x7fb3300cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40080/0 (socket says 172.21.15.92:40080) 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb3353f5700 1 --2- 172.21.15.92:0/285379629 >> v2:172.21.15.136:3300/0 conn(0x7fb330159ec0 0x7fb3300b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb335bf6700 1 -- 172.21.15.92:0/285379629 >> v2:172.21.15.136:3300/0 conn(0x7fb330159ec0 msgr2=0x7fb3300b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb335bf6700 1 --2- 172.21.15.92:0/285379629 >> v2:172.21.15.136:3300/0 conn(0x7fb330159ec0 0x7fb3300b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb335bf6700 1 -- 172.21.15.92:0/285379629 >> v1:172.21.15.92:6789/0 conn(0x7fb3300b53d0 legacy=0x7fb3300cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb327fff700 1 -- 172.21.15.92:0/285379629 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988703781 0 0) 0x7fb3300b5b00 con 0x7fb3300b53d0 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb335bf6700 1 -- 172.21.15.92:0/285379629 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300b6080 con 0x7fb3300cf350 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb3353f5700 1 --2- 172.21.15.92:0/285379629 >> v2:172.21.15.136:3300/0 conn(0x7fb330159ec0 0x7fb3300b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb335bf6700 1 --2- 172.21.15.92:0/285379629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3300cf350 0x7fb3300cd010 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb330089120 tx=0x7fb3280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc046203d79b6b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb327fff700 1 -- 172.21.15.92:0/285379629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800b6b0 con 0x7fb3300cf350 2023-10-06T03:02:59.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb327fff700 1 -- 172.21.15.92:0/285379629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32800b810 con 0x7fb3300cf350 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb327fff700 1 -- 172.21.15.92:0/285379629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800bbb0 con 0x7fb3300cf350 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb337659700 1 -- 172.21.15.92:0/285379629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3300cf350 msgr2=0x7fb3300cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.386+0000 7fb337659700 1 --2- 172.21.15.92:0/285379629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3300cf350 0x7fb3300cd010 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb330089120 tx=0x7fb3280091a0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.387+0000 7fb337659700 1 -- 172.21.15.92:0/285379629 shutdown_connections 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.387+0000 7fb337659700 1 --2- 172.21.15.92:0/285379629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3300cf350 0x7fb3300cd010 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.387+0000 7fb337659700 1 --2- 172.21.15.92:0/285379629 >> v2:172.21.15.136:3300/0 conn(0x7fb330159ec0 0x7fb3300b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.387+0000 7fb337659700 1 -- 172.21.15.92:0/285379629 >> 172.21.15.92:0/285379629 conn(0x7fb3300999f0 msgr2=0x7fb3300b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.387+0000 7fb337659700 1 -- 172.21.15.92:0/285379629 shutdown_connections 2023-10-06T03:02:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.387+0000 7fb337659700 1 -- 172.21.15.92:0/285379629 wait complete. 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.388+0000 7fb337659700 1 Processor -- start 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.388+0000 7fb337659700 1 -- start start 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3300b53d0 0x7fb33009dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3300cf350 0x7fb33016b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 0x7fb3300c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330159050 con 0x7fb330159ec0 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330156650 con 0x7fb3300b53d0 2023-10-06T03:02:59.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb330158d70 con 0x7fb3300cf350 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 0x7fb3300c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 0x7fb3300c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40086/0 (socket says 172.21.15.92:40086) 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 -- 172.21.15.92:0/492217848 learned_addr learned my addr 172.21.15.92:0/492217848 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb3353f5700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3300b53d0 0x7fb33009dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 -- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3300cf350 msgr2=0x7fb33016b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3300cf350 0x7fb33016b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 -- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3300b53d0 msgr2=0x7fb33009dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3300b53d0 0x7fb33009dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 -- 172.21.15.92:0/492217848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009420 con 0x7fb330159ec0 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb335bf6700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 0x7fb3300c6c40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb32800b0e0 tx=0x7fb328017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328025420 con 0x7fb330159ec0 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328025580 con 0x7fb330159ec0 2023-10-06T03:02:59.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb330156000 con 0x7fb330159ec0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328025920 con 0x7fb330159ec0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.389+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33009e190 con 0x7fb330159ec0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.390+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330063c90 con 0x7fb330159ec0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.391+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb328023070 con 0x7fb330159ec0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.391+0000 7fb3257fa700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb314066f30 0x7fb3140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.391+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb328085ac0 con 0x7fb330159ec0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.391+0000 7fb3353f5700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb314066f30 0x7fb3140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:02:59.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.391+0000 7fb3353f5700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb314066f30 0x7fb3140693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b530 tx=0x7fb32c0075f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:02:59.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.395+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb328052f20 con 0x7fb330159ec0 2023-10-06T03:02:59.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:02:59.582+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb3300718b0 con 0x7fb330159ec0 2023-10-06T03:03:00.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.551+0000 7fb3257fa700 1 -- 172.21.15.92:0/492217848 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb328056540 con 0x7fb330159ec0 2023-10-06T03:03:00.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb314066f30 msgr2=0x7fb3140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:00.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb314066f30 0x7fb3140693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b530 tx=0x7fb32c0075f0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 msgr2=0x7fb3300c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:00.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 0x7fb3300c6c40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb32800b0e0 tx=0x7fb328017000 comp rx=0 tx=0).stop 2023-10-06T03:03:00.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 shutdown_connections 2023-10-06T03:03:00.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb314066f30 0x7fb3140693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb330159ec0 0x7fb3300c6c40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3300b53d0 0x7fb33009dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 --2- 172.21.15.92:0/492217848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3300cf350 0x7fb33016b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 >> 172.21.15.92:0/492217848 conn(0x7fb3300999f0 msgr2=0x7fb3300c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:00.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.556+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 shutdown_connections 2023-10-06T03:03:00.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.557+0000 7fb337659700 1 -- 172.21.15.92:0/492217848 wait complete. 2023-10-06T03:03:00.560 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:03:00.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:03:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.694+0000 7f2495740700 1 Processor -- start 2023-10-06T03:03:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.694+0000 7f2495740700 1 -- start start 2023-10-06T03:03:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f2495740700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24900cdcf0 0x7f24900c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f2495740700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24900c8340 0x7f249015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f2495740700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249015abd0 con 0x7f2490159cd0 2023-10-06T03:03:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f2495740700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24900c8ac0 con 0x7f24900c8340 2023-10-06T03:03:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f2495740700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249015a0a0 con 0x7f24900cdcf0 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f248effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2490159cd0 0x7f24900cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34316/0 (socket says 172.21.15.92:34316) 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f248effd700 1 -- 172.21.15.92:0/3362789088 learned_addr learned my addr 172.21.15.92:0/3362789088 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f248f7fe700 1 --2- 172.21.15.92:0/3362789088 >> v2:172.21.15.92:3300/0 conn(0x7f24900c8340 0x7f249015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.695+0000 7f24867fc700 1 --2- 172.21.15.92:0/3362789088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24900cdcf0 0x7f24900c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248e7fc700 1 -- 172.21.15.92:0/3362789088 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669626123 0 0) 0x7f249015abd0 con 0x7f2490159cd0 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248e7fc700 1 -- 172.21.15.92:0/3362789088 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f247c003570 con 0x7f2490159cd0 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248f7fe700 1 -- 172.21.15.92:0/3362789088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24900cdcf0 msgr2=0x7f24900c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248f7fe700 1 --2- 172.21.15.92:0/3362789088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24900cdcf0 0x7f24900c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248f7fe700 1 -- 172.21.15.92:0/3362789088 >> v1:172.21.15.92:6789/0 conn(0x7f2490159cd0 legacy=0x7f24900cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:00.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248f7fe700 1 -- 172.21.15.92:0/3362789088 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249015b060 con 0x7f24900c8340 2023-10-06T03:03:00.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248f7fe700 1 --2- 172.21.15.92:0/3362789088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f249015d680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2488009e60 tx=0x7f2488009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ee31eab86827c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:00.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248e7fc700 1 -- 172.21.15.92:0/3362789088 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800e070 con 0x7f24900c8340 2023-10-06T03:03:00.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248e7fc700 1 -- 172.21.15.92:0/3362789088 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800bab0 con 0x7f24900c8340 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.696+0000 7f248e7fc700 1 -- 172.21.15.92:0/3362789088 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800be70 con 0x7f24900c8340 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 -- 172.21.15.92:0/3362789088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 msgr2=0x7f249015d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 --2- 172.21.15.92:0/3362789088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f249015d680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2488009e60 tx=0x7f2488009780 comp rx=0 tx=0).stop 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 -- 172.21.15.92:0/3362789088 shutdown_connections 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 --2- 172.21.15.92:0/3362789088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f249015d680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 --2- 172.21.15.92:0/3362789088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24900cdcf0 0x7f24900c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 -- 172.21.15.92:0/3362789088 >> 172.21.15.92:0/3362789088 conn(0x7f24900694a0 msgr2=0x7f24900698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 -- 172.21.15.92:0/3362789088 shutdown_connections 2023-10-06T03:03:00.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.697+0000 7f2495740700 1 -- 172.21.15.92:0/3362789088 wait complete. 2023-10-06T03:03:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.698+0000 7f2495740700 1 Processor -- start 2023-10-06T03:03:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.698+0000 7f2495740700 1 -- start start 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f2490156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24900cdcf0 0x7f24900ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2490159cd0 0x7f24900ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900c7580 con 0x7f24900c8340 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24901574f0 con 0x7f24900cdcf0 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900b1570 con 0x7f2490159cd0 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f2490156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f2490156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40106/0 (socket says 172.21.15.92:40106) 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 -- 172.21.15.92:0/4247941951 learned_addr learned my addr 172.21.15.92:0/4247941951 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f24867fc700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24900cdcf0 0x7f24900ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248f7fe700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2490159cd0 0x7f24900ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 -- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2490159cd0 msgr2=0x7f24900ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2490159cd0 0x7f24900ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 -- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24900cdcf0 msgr2=0x7f24900ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24900cdcf0 0x7f24900ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 -- 172.21.15.92:0/4247941951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2488009a20 con 0x7f24900c8340 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f24867fc700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24900cdcf0 0x7f24900ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f248effd700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f2490156fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f248000c590 tx=0x7f2480007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:00.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480005b30 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480005cb0 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24900ac1b0 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480007320 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.699+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24900b0f20 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.700+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900718b0 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.701+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f248001b3b0 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.701+0000 7f24877fe700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.701+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f2480083a70 con 0x7f24900c8340 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.701+0000 7f24867fc700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:00.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.701+0000 7f24867fc700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2470066f30 0x7f24700693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f249016d380 tx=0x7f2478008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:00.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.705+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2480050d60 con 0x7f24900c8340 2023-10-06T03:03:00.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:00.886+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f249015a2a0 con 0x7f24900c8340 2023-10-06T03:03:01.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.559+0000 7f24877fe700 1 -- 172.21.15.92:0/4247941951 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2480017020 con 0x7f24900c8340 2023-10-06T03:03:01.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.563+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2470066f30 msgr2=0x7f24700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:01.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.563+0000 7f2495740700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2470066f30 0x7f24700693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f249016d380 tx=0x7f2478008040 comp rx=0 tx=0).stop 2023-10-06T03:03:01.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.563+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 msgr2=0x7f2490156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:01.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.563+0000 7f2495740700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f2490156fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f248000c590 tx=0x7f2480007e00 comp rx=0 tx=0).stop 2023-10-06T03:03:01.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.566+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 shutdown_connections 2023-10-06T03:03:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.566+0000 7f2495740700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.566+0000 7f2495740700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24900c8340 0x7f2490156fb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.566+0000 7f2495740700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2490159cd0 0x7f24900ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.566+0000 7f2495740700 1 --2- 172.21.15.92:0/4247941951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24900cdcf0 0x7f24900ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.566+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 >> 172.21.15.92:0/4247941951 conn(0x7f24900694a0 msgr2=0x7f2490158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:01.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.567+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 shutdown_connections 2023-10-06T03:03:01.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.568+0000 7f2495740700 1 -- 172.21.15.92:0/4247941951 wait complete. 2023-10-06T03:03:01.574 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:03:01.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:03:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.701+0000 7f4c7b4af700 1 Processor -- start 2023-10-06T03:03:01.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.701+0000 7f4c7b4af700 1 -- start start 2023-10-06T03:03:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c7b4af700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c740caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c7b4af700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c7415df50 0x7f4c7415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:01.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c7b4af700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7415d7e0 con 0x7f4c7406c0a0 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c7b4af700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7415dac0 con 0x7f4c7415df50 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c7b4af700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415aa30 con 0x7f4c7406bcd0 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c79a4c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c7415df50 0x7f4c7415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c79a4c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4c7415df50 0x7f4c7415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40116/0 (socket says 172.21.15.92:40116) 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.702+0000 7f4c79a4c700 1 -- 172.21.15.92:0/816576544 learned_addr learned my addr 172.21.15.92:0/816576544 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7924b700 1 --2- 172.21.15.92:0/816576544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c740caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c79a4c700 1 -- 172.21.15.92:0/816576544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 msgr2=0x7f4c740caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c79a4c700 1 --2- 172.21.15.92:0/816576544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c740caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c79a4c700 1 -- 172.21.15.92:0/816576544 >> v1:172.21.15.136:6789/0 conn(0x7f4c7406c0a0 legacy=0x7f4c7415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c79a4c700 1 -- 172.21.15.92:0/816576544 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7415ad10 con 0x7f4c7415df50 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c79a4c700 1 --2- 172.21.15.92:0/816576544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c7415e3b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0099b0 tx=0x7f4c6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa02f61728cba33a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:01.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c6bfff700 1 -- 172.21.15.92:0/816576544 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00b860 con 0x7f4c7415df50 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c6bfff700 1 -- 172.21.15.92:0/816576544 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00b9c0 con 0x7f4c7415df50 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c6bfff700 1 -- 172.21.15.92:0/816576544 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00bd80 con 0x7f4c7415df50 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7b4af700 1 -- 172.21.15.92:0/816576544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 msgr2=0x7f4c7415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/816576544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c7415e3b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0099b0 tx=0x7f4c6c009250 comp rx=0 tx=0).stop 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7b4af700 1 -- 172.21.15.92:0/816576544 shutdown_connections 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/816576544 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c7415e3b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/816576544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c740caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.703+0000 7f4c7b4af700 1 -- 172.21.15.92:0/816576544 >> 172.21.15.92:0/816576544 conn(0x7f4c740ce600 msgr2=0x7f4c740cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:01.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.704+0000 7f4c7b4af700 1 -- 172.21.15.92:0/816576544 shutdown_connections 2023-10-06T03:03:01.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.704+0000 7f4c7b4af700 1 -- 172.21.15.92:0/816576544 wait complete. 2023-10-06T03:03:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.705+0000 7f4c7b4af700 1 Processor -- start 2023-10-06T03:03:01.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.705+0000 7f4c7b4af700 1 -- start start 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c741546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7406c0a0 0x7f4c74069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c74069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740b1de0 con 0x7f4c7415df50 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415a040 con 0x7f4c7406c0a0 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c74159430 con 0x7f4c7406bcd0 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c74069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7924b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c741546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c74069f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40118/0 (socket says 172.21.15.92:40118) 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 -- 172.21.15.92:0/895333545 learned_addr learned my addr 172.21.15.92:0/895333545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c78a4a700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7406c0a0 0x7f4c74069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:01.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 -- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 msgr2=0x7f4c741546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c741546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 -- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7406c0a0 msgr2=0x7f4c74069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7406c0a0 0x7f4c74069a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 -- 172.21.15.92:0/895333545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0094f0 con 0x7f4c7415df50 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c79a4c700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c74069f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b3c0 tx=0x7f4c6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01f070 con 0x7f4c7415df50 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c011e90 con 0x7f4c7415df50 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c74159710 con 0x7f4c7415df50 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00b860 con 0x7f4c7415df50 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.706+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c74159cd0 con 0x7f4c7415df50 2023-10-06T03:03:01.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.707+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7404c3c0 con 0x7f4c7415df50 2023-10-06T03:03:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.708+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c6c01aab0 con 0x7f4c7415df50 2023-10-06T03:03:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.708+0000 7f4c697fa700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4c58066f30 0x7f4c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.708+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f4c6c085c20 con 0x7f4c7415df50 2023-10-06T03:03:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.708+0000 7f4c7924b700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4c58066f30 0x7f4c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.709+0000 7f4c7924b700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4c58066f30 0x7f4c580693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c700075a0 tx=0x7f4c700097e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:01.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.713+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c6c0568a0 con 0x7f4c7415df50 2023-10-06T03:03:01.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:01.939+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c7415aff0 con 0x7f4c7415df50 2023-10-06T03:03:02.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.558+0000 7f4c697fa700 1 -- 172.21.15.92:0/895333545 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v25) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4c6c0150c0 con 0x7f4c7415df50 2023-10-06T03:03:02.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.561+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4c58066f30 msgr2=0x7f4c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.561+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4c58066f30 0x7f4c580693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c700075a0 tx=0x7f4c700097e0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.561+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 msgr2=0x7f4c74069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.561+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c74069f80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b3c0 tx=0x7f4c6c017000 comp rx=0 tx=0).stop 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 shutdown_connections 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4c58066f30 0x7f4c580693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c7415df50 0x7f4c74069f80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c7406bcd0 0x7f4c741546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 --2- 172.21.15.92:0/895333545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7406c0a0 0x7f4c74069a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 >> 172.21.15.92:0/895333545 conn(0x7f4c740ce600 msgr2=0x7f4c74154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 shutdown_connections 2023-10-06T03:03:02.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.563+0000 7f4c7b4af700 1 -- 172.21.15.92:0/895333545 wait complete. 2023-10-06T03:03:02.566 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T03:03:02.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.692+0000 7f787d92b700 1 Processor -- start 2023-10-06T03:03:02.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.692+0000 7f787d92b700 1 -- start start 2023-10-06T03:03:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.692+0000 7f787d92b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7878159890 0x7f787815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.692+0000 7f787d92b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787815a160 con 0x7f7878159c60 2023-10-06T03:03:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.692+0000 7f787d92b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780d13a0 con 0x7f7878069560 2023-10-06T03:03:02.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.692+0000 7f787d92b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787815e140 con 0x7f7878159890 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7878159890 0x7f787815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7878159890 0x7f787815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40128/0 (socket says 172.21.15.92:40128) 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 -- 172.21.15.92:0/3472713924 learned_addr learned my addr 172.21.15.92:0/3472713924 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f786e7fc700 1 --1- 172.21.15.92:0/3472713924 >> v1:172.21.15.92:6789/0 conn(0x7f7878159c60 0x7f787815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34332/0 (socket says 172.21.15.92:34332) 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 -- 172.21.15.92:0/3472713924 >> v1:172.21.15.136:6789/0 conn(0x7f7878069560 legacy=0x7f78780ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 -- 172.21.15.92:0/3472713924 >> v1:172.21.15.92:6789/0 conn(0x7f7878159c60 legacy=0x7f787815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 -- 172.21.15.92:0/3472713924 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787815a3d0 con 0x7f7878159890 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f7876ffd700 1 --2- 172.21.15.92:0/3472713924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f787815abd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f78680099b0 tx=0x7f7868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8186d799714feda1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f78767fc700 1 -- 172.21.15.92:0/3472713924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786800b820 con 0x7f7878159890 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f78767fc700 1 -- 172.21.15.92:0/3472713924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786800b980 con 0x7f7878159890 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.693+0000 7f78767fc700 1 -- 172.21.15.92:0/3472713924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786800bd20 con 0x7f7878159890 2023-10-06T03:03:02.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 -- 172.21.15.92:0/3472713924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 msgr2=0x7f787815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 --2- 172.21.15.92:0/3472713924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f787815abd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f78680099b0 tx=0x7f7868009250 comp rx=0 tx=0).stop 2023-10-06T03:03:02.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 -- 172.21.15.92:0/3472713924 shutdown_connections 2023-10-06T03:03:02.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 --2- 172.21.15.92:0/3472713924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f787815abd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 -- 172.21.15.92:0/3472713924 >> 172.21.15.92:0/3472713924 conn(0x7f7878068e20 msgr2=0x7f78780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:02.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 -- 172.21.15.92:0/3472713924 shutdown_connections 2023-10-06T03:03:02.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.694+0000 7f787d92b700 1 -- 172.21.15.92:0/3472713924 wait complete. 2023-10-06T03:03:02.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.695+0000 7f787d92b700 1 Processor -- start 2023-10-06T03:03:02.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.695+0000 7f787d92b700 1 -- start start 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7878069560 0x7f78780c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f7878149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7878159c60 0x7f787815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878158050 con 0x7f7878159890 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780b3c80 con 0x7f7878069560 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c8730 con 0x7f7878159c60 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f7878149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f7878149310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40144/0 (socket says 172.21.15.92:40144) 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 -- 172.21.15.92:0/4083582331 learned_addr learned my addr 172.21.15.92:0/4083582331 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f7876ffd700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7878069560 0x7f78780c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f78777fe700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7878159c60 0x7f787815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:02.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 -- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7878159c60 msgr2=0x7f787815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7878159c60 0x7f787815cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 -- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7878069560 msgr2=0x7f78780c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7878069560 0x7f78780c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 -- 172.21.15.92:0/4083582331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680094f0 con 0x7f7878159890 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f7876ffd700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7878069560 0x7f78780c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786e7fc700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f7878149310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f786000c7c0 tx=0x7f7860007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860005a70 con 0x7f7878159890 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860005bf0 con 0x7f7878159890 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7878157710 con 0x7f7878159890 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600070f0 con 0x7f7878159890 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.696+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780b3630 con 0x7f7878159890 2023-10-06T03:03:02.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.697+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78780b3e30 con 0x7f7878159890 2023-10-06T03:03:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.698+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7860007460 con 0x7f7878159890 2023-10-06T03:03:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.698+0000 7f786f7fe700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7858066f30 0x7f78580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.698+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f78600840c0 con 0x7f7878159890 2023-10-06T03:03:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.698+0000 7f7876ffd700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7858066f30 0x7f78580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:02.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.699+0000 7f7876ffd700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7858066f30 0x7f78580693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f78680099b0 tx=0x7f7868016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:02.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.702+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78780b3e30 con 0x7f7878159890 2023-10-06T03:03:02.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.887+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78780b3df0 con 0x7f7878159890 2023-10-06T03:03:02.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.888+0000 7f786f7fe700 1 -- 172.21.15.92:0/4083582331 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16881 (secure 0 0 0) 0x7f7860051250 con 0x7f7878159890 2023-10-06T03:03:02.890 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:02.890 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":53,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:01.940829+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:59.601586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:00.905218+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:02.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7858066f30 msgr2=0x7f78580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7858066f30 0x7f78580693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f78680099b0 tx=0x7f7868016000 comp rx=0 tx=0).stop 2023-10-06T03:03:02.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 msgr2=0x7f7878149310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:02.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f7878149310 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f786000c7c0 tx=0x7f7860007f70 comp rx=0 tx=0).stop 2023-10-06T03:03:02.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 shutdown_connections 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7858066f30 0x7f78580693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7878159c60 0x7f787815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7878069560 0x7f78780c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 --2- 172.21.15.92:0/4083582331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7878159890 0x7f7878149310 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 >> 172.21.15.92:0/4083582331 conn(0x7f7878068e20 msgr2=0x7f78780cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 shutdown_connections 2023-10-06T03:03:02.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:02.890+0000 7f787d92b700 1 -- 172.21.15.92:0/4083582331 wait complete. 2023-10-06T03:03:02.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:03.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.016+0000 7f9948afe700 1 Processor -- start 2023-10-06T03:03:03.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.016+0000 7f9948afe700 1 -- start start 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9948afe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9944159c60 0x7f994415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9948afe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f9944069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9948afe700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994415a160 con 0x7f9944159890 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9948afe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994415df50 con 0x7f9944159c60 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9948afe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99440d13a0 con 0x7f994415db10 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9944159c60 0x7f994415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9944159c60 0x7f994415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40150/0 (socket says 172.21.15.92:40150) 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 -- 172.21.15.92:0/2265328249 learned_addr learned my addr 172.21.15.92:0/2265328249 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:03.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9942d9d700 1 --2- 172.21.15.92:0/2265328249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f9944069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 -- 172.21.15.92:0/2265328249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 msgr2=0x7f9944069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 --2- 172.21.15.92:0/2265328249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f9944069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 -- 172.21.15.92:0/2265328249 >> v1:172.21.15.136:6789/0 conn(0x7f9944159890 legacy=0x7f994415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 -- 172.21.15.92:0/2265328249 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994415e140 con 0x7f9944159c60 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.017+0000 7f9941d9b700 1 --2- 172.21.15.92:0/2265328249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159c60 0x7f994415d5d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f99380099b0 tx=0x7f9938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2be1e2ed483e48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f994159a700 1 -- 172.21.15.92:0/2265328249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800b820 con 0x7f9944159c60 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f994159a700 1 -- 172.21.15.92:0/2265328249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993800b980 con 0x7f9944159c60 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f994159a700 1 -- 172.21.15.92:0/2265328249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800bd20 con 0x7f9944159c60 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 -- 172.21.15.92:0/2265328249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159c60 msgr2=0x7f994415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 --2- 172.21.15.92:0/2265328249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159c60 0x7f994415d5d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f99380099b0 tx=0x7f9938009250 comp rx=0 tx=0).stop 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 -- 172.21.15.92:0/2265328249 shutdown_connections 2023-10-06T03:03:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 --2- 172.21.15.92:0/2265328249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f9944069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 --2- 172.21.15.92:0/2265328249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159c60 0x7f994415d5d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 -- 172.21.15.92:0/2265328249 >> 172.21.15.92:0/2265328249 conn(0x7f9944068e20 msgr2=0x7f99440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:03.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 -- 172.21.15.92:0/2265328249 shutdown_connections 2023-10-06T03:03:03.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.018+0000 7f9948afe700 1 -- 172.21.15.92:0/2265328249 wait complete. 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.019+0000 7f9948afe700 1 Processor -- start 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.019+0000 7f9948afe700 1 -- start start 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9948afe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 0x7f994415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9948afe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9944159c60 0x7f994416a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9948afe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f994414cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9948afe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994415d530 con 0x7f9944159890 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9948afe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994415cbe0 con 0x7f9944159c60 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9948afe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994415b670 con 0x7f994415db10 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 0x7f994415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 0x7f994415c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40164/0 (socket says 172.21.15.92:40164) 2023-10-06T03:03:03.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 -- 172.21.15.92:0/2569119499 learned_addr learned my addr 172.21.15.92:0/2569119499 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9941d9b700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9944159c60 0x7f994416a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9942d9d700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f994414cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 -- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 msgr2=0x7f994414cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f994414cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 -- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9944159c60 msgr2=0x7f994416a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9944159c60 0x7f994416a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f994259c700 1 -- 172.21.15.92:0/2569119499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380094f0 con 0x7f9944159890 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.020+0000 7f9941d9b700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9944159c60 0x7f994416a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.021+0000 7f994259c700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 0x7f994415c600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f993400c5d0 tx=0x7f9934007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.021+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934011590 con 0x7f9944159890 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.021+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9934011710 con 0x7f9944159890 2023-10-06T03:03:03.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.021+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994415b950 con 0x7f9944159890 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.021+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934011ad0 con 0x7f9944159890 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.021+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994414d460 con 0x7f9944159890 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.022+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994415b4d0 con 0x7f9944159890 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.022+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f993401c460 con 0x7f9944159890 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.023+0000 7f99327fc700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9924066f30 0x7f99240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.023+0000 7f9941d9b700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9924066f30 0x7f99240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.023+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9934084490 con 0x7f9944159890 2023-10-06T03:03:03.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.023+0000 7f9941d9b700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9924066f30 0x7f99240693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99380099b0 tx=0x7f9938016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:03.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.027+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f994415b4d0 con 0x7f9944159890 2023-10-06T03:03:03.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.210+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f994415b4d0 con 0x7f9944159890 2023-10-06T03:03:03.212 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.210+0000 7f99327fc700 1 -- 172.21.15.92:0/2569119499 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16881 (secure 0 0 0) 0x7f994415b4d0 con 0x7f9944159890 2023-10-06T03:03:03.214 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:03.214 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":53,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:01.940829+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:59.601586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:00.905218+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9924066f30 msgr2=0x7f99240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9924066f30 0x7f99240693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99380099b0 tx=0x7f9938016000 comp rx=0 tx=0).stop 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 msgr2=0x7f994415c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 0x7f994415c600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f993400c5d0 tx=0x7f9934007f60 comp rx=0 tx=0).stop 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 shutdown_connections 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9924066f30 0x7f99240693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9944159890 0x7f994415c600 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f994415db10 0x7f994414cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 --2- 172.21.15.92:0/2569119499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9944159c60 0x7f994416a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 >> 172.21.15.92:0/2569119499 conn(0x7f9944068e20 msgr2=0x7f9944158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 shutdown_connections 2023-10-06T03:03:03.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.213+0000 7f9948afe700 1 -- 172.21.15.92:0/2569119499 wait complete. 2023-10-06T03:03:03.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:03:03.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-10-06T03:03:03.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T03:03:03.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-06T03:03:03.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.342+0000 7f11f46ac700 1 Processor -- start 2023-10-06T03:03:03.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.342+0000 7f11f46ac700 1 -- start start 2023-10-06T03:03:03.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f46ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11ec159890 0x7f11ec15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f46ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f46ac700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f11ec15db10 0x7f11ec069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f46ac700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec15a160 con 0x7f11ec159890 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f46ac700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec0d13a0 con 0x7f11ec15db10 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f46ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec15e140 con 0x7f11ec159c60 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f2448700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11ec159890 0x7f11ec15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f2448700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11ec159890 0x7f11ec15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40180/0 (socket says 172.21.15.92:40180) 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.343+0000 7f11f2448700 1 -- 172.21.15.92:0/254624390 learned_addr learned my addr 172.21.15.92:0/254624390 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f2448700 1 -- 172.21.15.92:0/254624390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 msgr2=0x7f11ec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f2448700 1 --2- 172.21.15.92:0/254624390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f2448700 1 -- 172.21.15.92:0/254624390 >> v2:172.21.15.136:3300/0 conn(0x7f11ec15db10 msgr2=0x7f11ec069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f2448700 1 --2- 172.21.15.92:0/254624390 >> v2:172.21.15.136:3300/0 conn(0x7f11ec15db10 0x7f11ec069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f2448700 1 -- 172.21.15.92:0/254624390 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec15a3d0 con 0x7f11ec159890 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f2448700 1 --2- 172.21.15.92:0/254624390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec159890 0x7f11ec15abd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f11ec089b50 tx=0x7f11e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd17d977dd394f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f1446700 1 -- 172.21.15.92:0/254624390 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800e070 con 0x7f11ec159890 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f1446700 1 -- 172.21.15.92:0/254624390 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800ba50 con 0x7f11ec159890 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f1446700 1 -- 172.21.15.92:0/254624390 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80092e0 con 0x7f11ec159890 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 -- 172.21.15.92:0/254624390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec159890 msgr2=0x7f11ec15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 --2- 172.21.15.92:0/254624390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec159890 0x7f11ec15abd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f11ec089b50 tx=0x7f11e8009840 comp rx=0 tx=0).stop 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 -- 172.21.15.92:0/254624390 shutdown_connections 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 --2- 172.21.15.92:0/254624390 >> v2:172.21.15.136:3300/0 conn(0x7f11ec15db10 0x7f11ec069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 --2- 172.21.15.92:0/254624390 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec159890 0x7f11ec15abd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 --2- 172.21.15.92:0/254624390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.344+0000 7f11f46ac700 1 -- 172.21.15.92:0/254624390 >> 172.21.15.92:0/254624390 conn(0x7f11ec068e20 msgr2=0x7f11ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:03.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.345+0000 7f11f46ac700 1 -- 172.21.15.92:0/254624390 shutdown_connections 2023-10-06T03:03:03.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.345+0000 7f11f46ac700 1 -- 172.21.15.92:0/254624390 wait complete. 2023-10-06T03:03:03.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 Processor -- start 2023-10-06T03:03:03.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 -- start start 2023-10-06T03:03:03.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec159890 0x7f11ec0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec14d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 0x7f11ec153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec14d660 con 0x7f11ec15db10 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b47b0 con 0x7f11ec159890 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f46ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec16dae0 con 0x7f11ec159c60 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f2c49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 0x7f11ec153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f2c49700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 0x7f11ec153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40188/0 (socket says 172.21.15.92:40188) 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.346+0000 7f11f2c49700 1 -- 172.21.15.92:0/1575078661 learned_addr learned my addr 172.21.15.92:0/1575078661 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f1c47700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec14d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2448700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec159890 0x7f11ec0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2c49700 1 -- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 msgr2=0x7f11ec14d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2c49700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec14d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2c49700 1 -- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec159890 msgr2=0x7f11ec0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2c49700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec159890 0x7f11ec0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2c49700 1 -- 172.21.15.92:0/1575078661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009ae0 con 0x7f11ec15db10 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2448700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec159890 0x7f11ec0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f1c47700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f2c49700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 0x7f11ec153850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b510 tx=0x7f11e40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400f510 con 0x7f11ec15db10 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e400f670 con 0x7f11ec15db10 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec16ddc0 con 0x7f11ec15db10 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e400fa30 con 0x7f11ec15db10 2023-10-06T03:03:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.347+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec16e380 con 0x7f11ec15db10 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.348+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec0b4960 con 0x7f11ec15db10 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.348+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11e401a400 con 0x7f11ec15db10 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.349+0000 7f11e27fc700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d0066f30 0x7f11d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.349+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f11e40840f0 con 0x7f11ec15db10 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.349+0000 7f11f2448700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d0066f30 0x7f11d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.349+0000 7f11f2448700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d0066f30 0x7f11d00693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009880 tx=0x7f11e8018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:03.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.353+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11ec0b4960 con 0x7f11ec15db10 2023-10-06T03:03:03.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:03.582+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f11ec0b4920 con 0x7f11ec15db10 2023-10-06T03:03:04.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.568+0000 7f11e27fc700 1 -- 172.21.15.92:0/1575078661 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f11e4051330 con 0x7f11ec15db10 2023-10-06T03:03:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d0066f30 msgr2=0x7f11d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d0066f30 0x7f11d00693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009880 tx=0x7f11e8018000 comp rx=0 tx=0).stop 2023-10-06T03:03:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 msgr2=0x7f11ec153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 0x7f11ec153850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f11e400b510 tx=0x7f11e40090f0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 shutdown_connections 2023-10-06T03:03:04.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d0066f30 0x7f11d00693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec15db10 0x7f11ec153850 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11ec159890 0x7f11ec0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 --2- 172.21.15.92:0/1575078661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec159c60 0x7f11ec14d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 >> 172.21.15.92:0/1575078661 conn(0x7f11ec068e20 msgr2=0x7f11ec1585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 shutdown_connections 2023-10-06T03:03:04.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.571+0000 7f11f46ac700 1 -- 172.21.15.92:0/1575078661 wait complete. 2023-10-06T03:03:04.587 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:03:04.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.706+0000 7f724c5d6700 1 Processor -- start 2023-10-06T03:03:04.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.706+0000 7f724c5d6700 1 -- start start 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724c5d6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f724415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724c5d6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7244159c60 0x7f724415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724c5d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724415a160 con 0x7f724415db10 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724c5d6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72440d13a0 con 0x7f7244159c60 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724c5d6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724415e140 con 0x7f7244159890 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249b71700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7244159c60 0x7f724415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:04.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724ab73700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f724415db10 0x7f7244069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34336/0 (socket says 172.21.15.92:34336) 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724ab73700 1 -- 172.21.15.92:0/521354898 learned_addr learned my addr 172.21.15.92:0/521354898 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f724a372700 1 --2- 172.21.15.92:0/521354898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f724415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249370700 1 -- 172.21.15.92:0/521354898 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641312071 0 0) 0x7f724415a160 con 0x7f724415db10 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249370700 1 -- 172.21.15.92:0/521354898 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7230003570 con 0x7f724415db10 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249b71700 1 -- 172.21.15.92:0/521354898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 msgr2=0x7f724415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249b71700 1 --2- 172.21.15.92:0/521354898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f724415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249b71700 1 -- 172.21.15.92:0/521354898 >> v1:172.21.15.92:6789/0 conn(0x7f724415db10 legacy=0x7f7244069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249b71700 1 -- 172.21.15.92:0/521354898 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724415a3d0 con 0x7f7244159c60 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.707+0000 7f7249b71700 1 --2- 172.21.15.92:0/521354898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7244159c60 0x7f724415d5d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f72340098e0 tx=0x7f72340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce6f91a0c3db568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f7249370700 1 -- 172.21.15.92:0/521354898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234011630 con 0x7f7244159c60 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f7249370700 1 -- 172.21.15.92:0/521354898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234011790 con 0x7f7244159c60 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f7249370700 1 -- 172.21.15.92:0/521354898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234011b30 con 0x7f7244159c60 2023-10-06T03:03:04.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 -- 172.21.15.92:0/521354898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7244159c60 msgr2=0x7f724415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 --2- 172.21.15.92:0/521354898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7244159c60 0x7f724415d5d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f72340098e0 tx=0x7f72340091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 -- 172.21.15.92:0/521354898 shutdown_connections 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 --2- 172.21.15.92:0/521354898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f724415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 --2- 172.21.15.92:0/521354898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7244159c60 0x7f724415d5d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 -- 172.21.15.92:0/521354898 >> 172.21.15.92:0/521354898 conn(0x7f7244068e20 msgr2=0x7f72440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 -- 172.21.15.92:0/521354898 shutdown_connections 2023-10-06T03:03:04.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.708+0000 7f724c5d6700 1 -- 172.21.15.92:0/521354898 wait complete. 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.709+0000 7f724c5d6700 1 Processor -- start 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.709+0000 7f724c5d6700 1 -- start start 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724c5d6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f7244143df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724c5d6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7244159c60 0x7f72440c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724c5d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 0x7f72440c62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724c5d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440c8490 con 0x7f724415db10 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724c5d6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b3fa0 con 0x7f7244159c60 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724c5d6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72440b4930 con 0x7f7244159890 2023-10-06T03:03:04.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 0x7f72440c62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 0x7f72440c62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40200/0 (socket says 172.21.15.92:40200) 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 -- 172.21.15.92:0/1843355933 learned_addr learned my addr 172.21.15.92:0/1843355933 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724a372700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f7244143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f7249b71700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7244159c60 0x7f72440c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 -- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 msgr2=0x7f7244143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f7244143df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 -- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7244159c60 msgr2=0x7f72440c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7244159c60 0x7f72440c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 -- 172.21.15.92:0/1843355933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7234009420 con 0x7f724415db10 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f7249b71700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7244159c60 0x7f72440c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.710+0000 7f724ab73700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 0x7f72440c62d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f723c001460 tx=0x7f723c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.711+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00a950 con 0x7f724415db10 2023-10-06T03:03:04.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.711+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c00aab0 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.711+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72440ca560 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.711+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00ae30 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.711+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72440b45c0 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.712+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72440ca960 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.712+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f723c007300 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.712+0000 7f723a7fc700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7228066f30 0x7f72280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.712+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f723c084b30 con 0x7f724415db10 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.712+0000 7f724a372700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7228066f30 0x7f72280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:04.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.713+0000 7f724a372700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7228066f30 0x7f72280693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72400093f0 tx=0x7f7240008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:04.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.717+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f723c051c30 con 0x7f724415db10 2023-10-06T03:03:04.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:04.944+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f72440b4aa0 con 0x7f724415db10 2023-10-06T03:03:05.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.576+0000 7f723a7fc700 1 -- 172.21.15.92:0/1843355933 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7f723c055250 con 0x7f724415db10 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7228066f30 msgr2=0x7f72280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7228066f30 0x7f72280693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f72400093f0 tx=0x7f7240008040 comp rx=0 tx=0).stop 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 msgr2=0x7f72440c62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 0x7f72440c62d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f723c001460 tx=0x7f723c001a40 comp rx=0 tx=0).stop 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 shutdown_connections 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7228066f30 0x7f72280693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f724415db10 0x7f72440c62d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7244159890 0x7f7244143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 --2- 172.21.15.92:0/1843355933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7244159c60 0x7f72440c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 >> 172.21.15.92:0/1843355933 conn(0x7f7244068e20 msgr2=0x7f724415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 shutdown_connections 2023-10-06T03:03:05.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.579+0000 7f724c5d6700 1 -- 172.21.15.92:0/1843355933 wait complete. 2023-10-06T03:03:05.594 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:05.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 Processor -- start 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 -- start start 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7a81597f0 0x7fe7a80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe7a815d230 0x7fe7a815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a815db70 con 0x7fe7a8159420 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80b11d0 con 0x7fe7a81597f0 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.710+0000 7fe7acdf8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80b14b0 con 0x7fe7a815d230 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7a81597f0 0x7fe7a80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7a8159420 0x7fe7a80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34342/0 (socket says 172.21.15.92:34342) 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a77fe700 1 -- 172.21.15.92:0/1791196479 learned_addr learned my addr 172.21.15.92:0/1791196479 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:05.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a7fff700 1 --2- 172.21.15.92:0/1791196479 >> v2:172.21.15.136:3300/0 conn(0x7fe7a815d230 0x7fe7a815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a6ffd700 1 -- 172.21.15.92:0/1791196479 >> v2:172.21.15.136:3300/0 conn(0x7fe7a815d230 msgr2=0x7fe7a815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/1791196479 >> v2:172.21.15.136:3300/0 conn(0x7fe7a815d230 0x7fe7a815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a6ffd700 1 -- 172.21.15.92:0/1791196479 >> v1:172.21.15.92:6789/0 conn(0x7fe7a8159420 legacy=0x7fe7a80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a6ffd700 1 -- 172.21.15.92:0/1791196479 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80b1790 con 0x7fe7a81597f0 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1791196479 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064209908 0 0) 0x7fe7a815db70 con 0x7fe7a8159420 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a7fff700 1 --2- 172.21.15.92:0/1791196479 >> v2:172.21.15.136:3300/0 conn(0x7fe7a815d230 0x7fe7a815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/1791196479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a81597f0 0x7fe7a80b2270 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe7a815d170 tx=0x7fe79c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e25a8913c5b06e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1791196479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00b6b0 con 0x7fe7a81597f0 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1791196479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c00b810 con 0x7fe7a81597f0 2023-10-06T03:03:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.711+0000 7fe7a67fc700 1 -- 172.21.15.92:0/1791196479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00bbb0 con 0x7fe7a81597f0 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1791196479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a81597f0 msgr2=0x7fe7a80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1791196479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a81597f0 0x7fe7a80b2270 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe7a815d170 tx=0x7fe79c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1791196479 shutdown_connections 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1791196479 >> v2:172.21.15.136:3300/0 conn(0x7fe7a815d230 0x7fe7a815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1791196479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a81597f0 0x7fe7a80b2270 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1791196479 >> 172.21.15.92:0/1791196479 conn(0x7fe7a80d0660 msgr2=0x7fe7a80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1791196479 shutdown_connections 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.712+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1791196479 wait complete. 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.713+0000 7fe7acdf8700 1 Processor -- start 2023-10-06T03:03:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.713+0000 7fe7acdf8700 1 -- start start 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7acdf8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 0x7fe7a8155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7acdf8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a81597f0 0x7fe7a80ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7acdf8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a815d230 0x7fe7a80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7acdf8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80cad10 con 0x7fe7a8159420 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7acdf8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80caff0 con 0x7fe7a81597f0 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7acdf8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80cb2d0 con 0x7fe7a815d230 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 0x7fe7a8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 0x7fe7a8155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40220/0 (socket says 172.21.15.92:40220) 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 -- 172.21.15.92:0/1532659319 learned_addr learned my addr 172.21.15.92:0/1532659319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:05.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 -- 172.21.15.92:0/1532659319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a815d230 msgr2=0x7fe7a80ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a815d230 0x7fe7a80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 -- 172.21.15.92:0/1532659319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a81597f0 msgr2=0x7fe7a80ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a81597f0 0x7fe7a80ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 -- 172.21.15.92:0/1532659319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c009420 con 0x7fe7a8159420 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe7a77fe700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 0x7fe7a8155830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c7e0 tx=0x7fe798007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.714+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798005b30 con 0x7fe7a8159420 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.715+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe798005c90 con 0x7fe7a8159420 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.715+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a80cb5b0 con 0x7fe7a8159420 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.715+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798007300 con 0x7fe7a8159420 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.715+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80c5a90 con 0x7fe7a8159420 2023-10-06T03:03:05.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.715+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a80cbad0 con 0x7fe7a8159420 2023-10-06T03:03:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.716+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe79801b400 con 0x7fe7a8159420 2023-10-06T03:03:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.716+0000 7fe78f7fe700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe794066f30 0x7fe7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.716+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe7980841a0 con 0x7fe7a8159420 2023-10-06T03:03:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.716+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe794066f30 0x7fe7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.717+0000 7fe7a6ffd700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe794066f30 0x7fe7940693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8159c70 tx=0x7fe79c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:05.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.721+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe798051330 con 0x7fe7a8159420 2023-10-06T03:03:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.944+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7a804c3c0 con 0x7fe7a8159420 2023-10-06T03:03:05.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.945+0000 7fe78f7fe700 1 -- 172.21.15.92:0/1532659319 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fe798054950 con 0x7fe7a8159420 2023-10-06T03:03:05.947 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:05.947 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:01.941144+0000","modified":"2023-10-06T03:03:05.571224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3249599548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3249599548},{"type":"v1","addr":"172.21.15.92:6837","nonce":3249599548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe794066f30 msgr2=0x7fe7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe794066f30 0x7fe7940693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8159c70 tx=0x7fe79c017000 comp rx=0 tx=0).stop 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 msgr2=0x7fe7a8155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 0x7fe7a8155830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c7e0 tx=0x7fe798007df0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 shutdown_connections 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe794066f30 0x7fe7940693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7a8159420 0x7fe7a8155830 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7a815d230 0x7fe7a80ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 --2- 172.21.15.92:0/1532659319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7a81597f0 0x7fe7a80ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:05.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 >> 172.21.15.92:0/1532659319 conn(0x7fe7a80d0660 msgr2=0x7fe7a8155f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:05.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 shutdown_connections 2023-10-06T03:03:05.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:05.947+0000 7fe7acdf8700 1 -- 172.21.15.92:0/1532659319 wait complete. 2023-10-06T03:03:05.950 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2023-10-06T03:03:05.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:06.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.073+0000 7f22640d6700 1 Processor -- start 2023-10-06T03:03:06.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.073+0000 7f22640d6700 1 -- start start 2023-10-06T03:03:06.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f22640d6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f225c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f22640d6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f225c15db10 0x7f225c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f22640d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c15a160 con 0x7f225c159890 2023-10-06T03:03:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f22640d6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225c0d13a0 con 0x7f225c15db10 2023-10-06T03:03:06.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f22640d6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c15e140 con 0x7f225c159c60 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f2261e72700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f225c159890 0x7f225c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34356/0 (socket says 172.21.15.92:34356) 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.074+0000 7f2261e72700 1 -- 172.21.15.92:0/32627313 learned_addr learned my addr 172.21.15.92:0/32627313 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2262673700 1 --2- 172.21.15.92:0/32627313 >> v2:172.21.15.136:3300/0 conn(0x7f225c15db10 0x7f225c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261671700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f225c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4169645094 0 0) 0x7f225c15a160 con 0x7f225c159890 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2248003570 con 0x7f225c159890 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830765855 0 0) 0x7f2248003570 con 0x7f225c159890 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 msgr2=0x7f225c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f225c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 >> v2:172.21.15.136:3300/0 conn(0x7f225c15db10 msgr2=0x7f225c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:06.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 --2- 172.21.15.92:0/32627313 >> v2:172.21.15.136:3300/0 conn(0x7f225c15db10 0x7f225c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c15a3d0 con 0x7f225c159890 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2258001470 con 0x7f225c159890 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 >> v1:172.21.15.92:6789/0 conn(0x7f225c159890 legacy=0x7f225c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f2248003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f22480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c0d13a0 con 0x7f2248003a60 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c15e140 con 0x7f2248004810 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224800ba60 con 0x7f225c159c60 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261e72700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f2248003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261671700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2262673700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f22480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261e72700 1 -- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 msgr2=0x7f22480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261e72700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f22480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261e72700 1 -- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2248004810 msgr2=0x7f2248004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261e72700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261e72700 1 -- 172.21.15.92:0/32627313 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2248003570 con 0x7f2248003a60 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2262673700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c159c60 0x7f22480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.075+0000 7f2261671700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2248004810 0x7f2248004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f2261e72700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f2248003e70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f225800c3f0 tx=0x7f2258001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aa1d161c14527 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800cc80 con 0x7f2248003a60 2023-10-06T03:03:06.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800ce00 con 0x7f2248003a60 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f2260e70700 1 -- 172.21.15.92:0/32627313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258005b70 con 0x7f2248003a60 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f22640d6700 1 -- 172.21.15.92:0/32627313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 msgr2=0x7f2248003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f22640d6700 1 --2- 172.21.15.92:0/32627313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f2248003e70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f225800c3f0 tx=0x7f2258001e90 comp rx=0 tx=0).stop 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f2262673700 1 -- 172.21.15.92:0/32627313 reap_dead start 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f22640d6700 1 -- 172.21.15.92:0/32627313 shutdown_connections 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f22640d6700 1 -- 172.21.15.92:0/32627313 >> 172.21.15.92:0/32627313 conn(0x7f225c068e20 msgr2=0x7f225c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f22640d6700 1 -- 172.21.15.92:0/32627313 shutdown_connections 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.076+0000 7f22640d6700 1 -- 172.21.15.92:0/32627313 wait complete. 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.077+0000 7f22640d6700 1 Processor -- start 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.077+0000 7f22640d6700 1 -- start start 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f22640d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f225c15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f22640d6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f225c169cd0 0x7f225c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f22640d6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c153a70 0x7f225c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f22640d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c0b3750 con 0x7f2248003a60 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f22640d6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c1549a0 con 0x7f225c169cd0 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f22640d6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c16ac60 con 0x7f225c153a70 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f225c15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f225c15d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40234/0 (socket says 172.21.15.92:40234) 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 -- 172.21.15.92:0/1280153388 learned_addr learned my addr 172.21.15.92:0/1280153388 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 -- 172.21.15.92:0/1280153388 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c153a70 msgr2=0x7f225c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c153a70 0x7f225c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 -- 172.21.15.92:0/1280153388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f225c169cd0 msgr2=0x7f225c0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f225c169cd0 0x7f225c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.078+0000 7f2261e72700 1 -- 172.21.15.92:0/1280153388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c16a120 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.079+0000 7f2261e72700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f225c15d350 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f22480046d0 tx=0x7f2258015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.079+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800cc80 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.079+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2258001bb0 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.079+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c1543f0 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.079+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258007550 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.079+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c16a6a0 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.080+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225c04c3c0 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.080+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2258019030 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.080+0000 7f2251ffb700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.081+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2258085cb0 con 0x7f2248003a60 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.081+0000 7f2261671700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.081+0000 7f2261671700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f225c15a090 tx=0x7f224c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.085+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2258052dc0 con 0x7f2248003a60 2023-10-06T03:03:06.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.268+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f225c06a5b0 con 0x7f2248003a60 2023-10-06T03:03:06.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.269+0000 7f2251ffb700 1 -- 172.21.15.92:0/1280153388 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16868 (secure 0 0 0) 0x7f22580563e0 con 0x7f2248003a60 2023-10-06T03:03:06.271 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:06.271 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":54,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:03.471780+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:59.601586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:00.905218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:06.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2240066f30 msgr2=0x7f22400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2240066f30 0x7f22400693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f225c15a090 tx=0x7f224c008040 comp rx=0 tx=0).stop 2023-10-06T03:03:06.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 msgr2=0x7f225c15d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f225c15d350 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f22480046d0 tx=0x7f2258015000 comp rx=0 tx=0).stop 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 shutdown_connections 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2240066f30 0x7f22400693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2248003a60 0x7f225c15d350 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f225c153a70 0x7f225c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 --2- 172.21.15.92:0/1280153388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f225c169cd0 0x7f225c0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 >> 172.21.15.92:0/1280153388 conn(0x7f225c068e20 msgr2=0x7f225c1588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 shutdown_connections 2023-10-06T03:03:06.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.271+0000 7f22640d6700 1 -- 172.21.15.92:0/1280153388 wait complete. 2023-10-06T03:03:06.287 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:03:06.287 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:06.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.400+0000 7fbe3f07d700 1 Processor -- start 2023-10-06T03:03:06.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.400+0000 7fbe3f07d700 1 -- start start 2023-10-06T03:03:06.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3f07d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbe38069bb0 0x7fbe381645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3f07d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380cfb30 con 0x7fbe38159c10 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3f07d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380cbc40 con 0x7fbe38159840 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3f07d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380cbf10 con 0x7fbe38069bb0 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3d61a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbe38069bb0 0x7fbe381645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe2ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbe38159c10 0x7fbe380694a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34360/0 (socket says 172.21.15.92:34360) 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe2ffff700 1 -- 172.21.15.92:0/552874935 learned_addr learned my addr 172.21.15.92:0/552874935 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3d61a700 1 -- 172.21.15.92:0/552874935 >> v1:172.21.15.136:6789/0 conn(0x7fbe38159840 legacy=0x7fbe380cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3d61a700 1 -- 172.21.15.92:0/552874935 >> v1:172.21.15.92:6789/0 conn(0x7fbe38159c10 legacy=0x7fbe380694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3d61a700 1 -- 172.21.15.92:0/552874935 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380cf4e0 con 0x7fbe38069bb0 2023-10-06T03:03:06.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe2f7fe700 1 -- 172.21.15.92:0/552874935 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118547508 0 0) 0x7fbe380cfb30 con 0x7fbe38159c10 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe3d61a700 1 --2- 172.21.15.92:0/552874935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe381645f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009e90 tx=0x7fbe30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7786bb1afeb2a9a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe2f7fe700 1 -- 172.21.15.92:0/552874935 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000e070 con 0x7fbe38069bb0 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe2f7fe700 1 -- 172.21.15.92:0/552874935 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3000bbf0 con 0x7fbe38069bb0 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.401+0000 7fbe2f7fe700 1 -- 172.21.15.92:0/552874935 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30012430 con 0x7fbe38069bb0 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 -- 172.21.15.92:0/552874935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 msgr2=0x7fbe381645f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/552874935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe381645f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009e90 tx=0x7fbe30009780 comp rx=0 tx=0).stop 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 -- 172.21.15.92:0/552874935 shutdown_connections 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/552874935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe381645f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 -- 172.21.15.92:0/552874935 >> 172.21.15.92:0/552874935 conn(0x7fbe38068e10 msgr2=0x7fbe380d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 -- 172.21.15.92:0/552874935 shutdown_connections 2023-10-06T03:03:06.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.402+0000 7fbe3f07d700 1 -- 172.21.15.92:0/552874935 wait complete. 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.403+0000 7fbe3f07d700 1 Processor -- start 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.403+0000 7fbe3f07d700 1 -- start start 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe380b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe38159840 0x7fbe380c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe38159c10 0x7fbe380c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380c8e20 con 0x7fbe38069bb0 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380b6150 con 0x7fbe38159c10 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3816a090 con 0x7fbe38159840 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe380b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe380b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40250/0 (socket says 172.21.15.92:40250) 2023-10-06T03:03:06.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 -- 172.21.15.92:0/3926477454 learned_addr learned my addr 172.21.15.92:0/3926477454 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3d61a700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe38159c10 0x7fbe380c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe2ffff700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe38159840 0x7fbe380c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 -- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe38159840 msgr2=0x7fbe380c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe38159840 0x7fbe380c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 -- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe38159c10 msgr2=0x7fbe380c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe38159c10 0x7fbe380c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 -- 172.21.15.92:0/3926477454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30009a20 con 0x7fbe38069bb0 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3ce19700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe380b5c10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c590 tx=0x7fbe3400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34011590 con 0x7fbe38069bb0 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe340116f0 con 0x7fbe38069bb0 2023-10-06T03:03:06.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3816a370 con 0x7fbe38069bb0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.404+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34011ab0 con 0x7fbe38069bb0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.405+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3816a930 con 0x7fbe38069bb0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.406+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe38064550 con 0x7fbe38069bb0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.406+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe3401b3b0 con 0x7fbe38069bb0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.406+0000 7fbe2cff9700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe24066f30 0x7fbe240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.406+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbe34083c20 con 0x7fbe38069bb0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.406+0000 7fbe2ffff700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe24066f30 0x7fbe240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.406+0000 7fbe2ffff700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe24066f30 0x7fbe240693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbe3806a760 tx=0x7fbe28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.410+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe34050d30 con 0x7fbe38069bb0 2023-10-06T03:03:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.590+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe380b3cd0 con 0x7fbe24066f30 2023-10-06T03:03:06.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.599+0000 7fbe2cff9700 1 -- 172.21.15.92:0/3926477454 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311670 (secure 0 0 0) 0x7fbe380b3cd0 con 0x7fbe24066f30 2023-10-06T03:03:06.604 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe24066f30 msgr2=0x7fbe240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe24066f30 0x7fbe240693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbe3806a760 tx=0x7fbe28009040 comp rx=0 tx=0).stop 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 msgr2=0x7fbe380b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe380b5c10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400c590 tx=0x7fbe3400c950 comp rx=0 tx=0).stop 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 shutdown_connections 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe24066f30 0x7fbe240693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe38069bb0 0x7fbe380b5c10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe38159c10 0x7fbe380c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 --2- 172.21.15.92:0/3926477454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe38159840 0x7fbe380c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 >> 172.21.15.92:0/3926477454 conn(0x7fbe38068e10 msgr2=0x7fbe381577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 shutdown_connections 2023-10-06T03:03:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.602+0000 7fbe3f07d700 1 -- 172.21.15.92:0/3926477454 wait complete. 2023-10-06T03:03:06.608 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:06.616 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-10-06T03:03:05.199117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384524,"kb_used_data":6220,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590196,"statfs":{"total":773094113280,"available":770652360704,"internally_reserved":0,"allocated":6369280,"data_stored":3459759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.647253"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488602+0000","last_change":"2023-10-06T03:03:02.486434+0000","last_active":"2023-10-06T03:03:02.488602+0000","last_peered":"2023-10-06T03:03:02.488602+0000","last_clean":"2023-10-06T03:03:02.488602+0000","last_became_active":"2023-10-06T03:03:02.486237+0000","last_became_peered":"2023-10-06T03:03:02.486237+0000","last_unstale":"2023-10-06T03:03:02.488602+0000","last_undegraded":"2023-10-06T03:03:02.488602+0000","last_fullsized":"2023-10-06T03:03:02.488602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:03.646525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493317+0000","last_change":"2023-10-06T03:03:02.488901+0000","last_active":"2023-10-06T03:03:02.493317+0000","last_peered":"2023-10-06T03:03:02.493317+0000","last_clean":"2023-10-06T03:03:02.493317+0000","last_became_active":"2023-10-06T03:03:02.488603+0000","last_became_peered":"2023-10-06T03:03:02.488603+0000","last_unstale":"2023-10-06T03:03:02.493317+0000","last_undegraded":"2023-10-06T03:03:02.493317+0000","last_fullsized":"2023-10-06T03:03:02.493317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:52.093262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489952+0000","last_change":"2023-10-06T03:03:02.489952+0000","last_active":"2023-10-06T03:03:02.489952+0000","last_peered":"2023-10-06T03:03:02.489952+0000","last_clean":"2023-10-06T03:03:02.489952+0000","last_became_active":"2023-10-06T03:03:02.489744+0000","last_became_peered":"2023-10-06T03:03:02.489744+0000","last_unstale":"2023-10-06T03:03:02.489952+0000","last_undegraded":"2023-10-06T03:03:02.489952+0000","last_fullsized":"2023-10-06T03:03:02.489952+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:56.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493126+0000","last_change":"2023-10-06T03:03:02.489884+0000","last_active":"2023-10-06T03:03:02.493126+0000","last_peered":"2023-10-06T03:03:02.493126+0000","last_clean":"2023-10-06T03:03:02.493126+0000","last_became_active":"2023-10-06T03:03:02.489161+0000","last_became_peered":"2023-10-06T03:03:02.489161+0000","last_unstale":"2023-10-06T03:03:02.493126+0000","last_undegraded":"2023-10-06T03:03:02.493126+0000","last_fullsized":"2023-10-06T03:03:02.493126+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.494997+0000","last_change":"2023-10-06T03:03:02.494997+0000","last_active":"2023-10-06T03:03:02.494997+0000","last_peered":"2023-10-06T03:03:02.494997+0000","last_clean":"2023-10-06T03:03:02.494997+0000","last_became_active":"2023-10-06T03:03:02.494733+0000","last_became_peered":"2023-10-06T03:03:02.494733+0000","last_unstale":"2023-10-06T03:03:02.494997+0000","last_undegraded":"2023-10-06T03:03:02.494997+0000","last_fullsized":"2023-10-06T03:03:02.494997+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:03.016992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.487430+0000","last_change":"2023-10-06T03:03:02.487430+0000","last_active":"2023-10-06T03:03:02.487430+0000","last_peered":"2023-10-06T03:03:02.487430+0000","last_clean":"2023-10-06T03:03:02.487430+0000","last_became_active":"2023-10-06T03:03:02.487162+0000","last_became_peered":"2023-10-06T03:03:02.487162+0000","last_unstale":"2023-10-06T03:03:02.487430+0000","last_undegraded":"2023-10-06T03:03:02.487430+0000","last_fullsized":"2023-10-06T03:03:02.487430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.483351+0000","last_change":"2023-10-06T03:03:02.483351+0000","last_active":"2023-10-06T03:03:02.483351+0000","last_peered":"2023-10-06T03:03:02.483351+0000","last_clean":"2023-10-06T03:03:02.483351+0000","last_became_active":"2023-10-06T03:03:02.483119+0000","last_became_peered":"2023-10-06T03:03:02.483119+0000","last_unstale":"2023-10-06T03:03:02.483351+0000","last_undegraded":"2023-10-06T03:03:02.483351+0000","last_fullsized":"2023-10-06T03:03:02.483351+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:47.758547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489241+0000","last_change":"2023-10-06T03:03:02.483123+0000","last_active":"2023-10-06T03:03:02.489241+0000","last_peered":"2023-10-06T03:03:02.489241+0000","last_clean":"2023-10-06T03:03:02.489241+0000","last_became_active":"2023-10-06T03:03:02.482849+0000","last_became_peered":"2023-10-06T03:03:02.482849+0000","last_unstale":"2023-10-06T03:03:02.489241+0000","last_undegraded":"2023-10-06T03:03:02.489241+0000","last_fullsized":"2023-10-06T03:03:02.489241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493975+0000","last_change":"2023-10-06T03:03:02.493975+0000","last_active":"2023-10-06T03:03:02.493975+0000","last_peered":"2023-10-06T03:03:02.493975+0000","last_clean":"2023-10-06T03:03:02.493975+0000","last_became_active":"2023-10-06T03:03:02.493731+0000","last_became_peered":"2023-10-06T03:03:02.493731+0000","last_unstale":"2023-10-06T03:03:02.493975+0000","last_undegraded":"2023-10-06T03:03:02.493975+0000","last_fullsized":"2023-10-06T03:03:02.493975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:12.848469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489735+0000","last_change":"2023-10-06T03:03:02.489735+0000","last_active":"2023-10-06T03:03:02.489735+0000","last_peered":"2023-10-06T03:03:02.489735+0000","last_clean":"2023-10-06T03:03:02.489735+0000","last_became_active":"2023-10-06T03:03:02.489497+0000","last_became_peered":"2023-10-06T03:03:02.489497+0000","last_unstale":"2023-10-06T03:03:02.489735+0000","last_undegraded":"2023-10-06T03:03:02.489735+0000","last_fullsized":"2023-10-06T03:03:02.489735+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:24.544569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491784+0000","last_change":"2023-10-06T03:03:02.482440+0000","last_active":"2023-10-06T03:03:02.491784+0000","last_peered":"2023-10-06T03:03:02.491784+0000","last_clean":"2023-10-06T03:03:02.491784+0000","last_became_active":"2023-10-06T03:03:02.482164+0000","last_became_peered":"2023-10-06T03:03:02.482164+0000","last_unstale":"2023-10-06T03:03:02.491784+0000","last_undegraded":"2023-10-06T03:03:02.491784+0000","last_fullsized":"2023-10-06T03:03:02.491784+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489098+0000","last_change":"2023-10-06T03:03:02.483350+0000","last_active":"2023-10-06T03:03:02.489098+0000","last_peered":"2023-10-06T03:03:02.489098+0000","last_clean":"2023-10-06T03:03:02.489098+0000","last_became_active":"2023-10-06T03:03:02.483097+0000","last_became_peered":"2023-10-06T03:03:02.483097+0000","last_unstale":"2023-10-06T03:03:02.489098+0000","last_undegraded":"2023-10-06T03:03:02.489098+0000","last_fullsized":"2023-10-06T03:03:02.489098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.485334+0000","last_change":"2023-10-06T03:03:02.481663+0000","last_active":"2023-10-06T03:03:02.485334+0000","last_peered":"2023-10-06T03:03:02.485334+0000","last_clean":"2023-10-06T03:03:02.485334+0000","last_became_active":"2023-10-06T03:03:02.481384+0000","last_became_peered":"2023-10-06T03:03:02.481384+0000","last_unstale":"2023-10-06T03:03:02.485334+0000","last_undegraded":"2023-10-06T03:03:02.485334+0000","last_fullsized":"2023-10-06T03:03:02.485334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:12.848469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490013+0000","last_change":"2023-10-06T03:03:02.490013+0000","last_active":"2023-10-06T03:03:02.490013+0000","last_peered":"2023-10-06T03:03:02.490013+0000","last_clean":"2023-10-06T03:03:02.490013+0000","last_became_active":"2023-10-06T03:03:02.489762+0000","last_became_peered":"2023-10-06T03:03:02.489762+0000","last_unstale":"2023-10-06T03:03:02.490013+0000","last_undegraded":"2023-10-06T03:03:02.490013+0000","last_fullsized":"2023-10-06T03:03:02.490013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:52.154816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491288+0000","last_change":"2023-10-06T03:03:02.491288+0000","last_active":"2023-10-06T03:03:02.491288+0000","last_peered":"2023-10-06T03:03:02.491288+0000","last_clean":"2023-10-06T03:03:02.491288+0000","last_became_active":"2023-10-06T03:03:02.491043+0000","last_became_peered":"2023-10-06T03:03:02.491043+0000","last_unstale":"2023-10-06T03:03:02.491288+0000","last_undegraded":"2023-10-06T03:03:02.491288+0000","last_fullsized":"2023-10-06T03:03:02.491288+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:52.366760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.485254+0000","last_change":"2023-10-06T03:03:02.485254+0000","last_active":"2023-10-06T03:03:02.485254+0000","last_peered":"2023-10-06T03:03:02.485254+0000","last_clean":"2023-10-06T03:03:02.485254+0000","last_became_active":"2023-10-06T03:03:02.484988+0000","last_became_peered":"2023-10-06T03:03:02.484988+0000","last_unstale":"2023-10-06T03:03:02.485254+0000","last_undegraded":"2023-10-06T03:03:02.485254+0000","last_fullsized":"2023-10-06T03:03:02.485254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:56.243568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488579+0000","last_change":"2023-10-06T03:03:02.482533+0000","last_active":"2023-10-06T03:03:02.488579+0000","last_peered":"2023-10-06T03:03:02.488579+0000","last_clean":"2023-10-06T03:03:02.488579+0000","last_became_active":"2023-10-06T03:03:02.482275+0000","last_became_peered":"2023-10-06T03:03:02.482275+0000","last_unstale":"2023-10-06T03:03:02.488579+0000","last_undegraded":"2023-10-06T03:03:02.488579+0000","last_fullsized":"2023-10-06T03:03:02.488579+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:33.015231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493650+0000","last_change":"2023-10-06T03:03:02.489682+0000","last_active":"2023-10-06T03:03:02.493650+0000","last_peered":"2023-10-06T03:03:02.493650+0000","last_clean":"2023-10-06T03:03:02.493650+0000","last_became_active":"2023-10-06T03:03:02.489248+0000","last_became_peered":"2023-10-06T03:03:02.489248+0000","last_unstale":"2023-10-06T03:03:02.493650+0000","last_undegraded":"2023-10-06T03:03:02.493650+0000","last_fullsized":"2023-10-06T03:03:02.493650+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490604+0000","last_change":"2023-10-06T03:03:02.490604+0000","last_active":"2023-10-06T03:03:02.490604+0000","last_peered":"2023-10-06T03:03:02.490604+0000","last_clean":"2023-10-06T03:03:02.490604+0000","last_became_active":"2023-10-06T03:03:02.490212+0000","last_became_peered":"2023-10-06T03:03:02.490212+0000","last_unstale":"2023-10-06T03:03:02.490604+0000","last_undegraded":"2023-10-06T03:03:02.490604+0000","last_fullsized":"2023-10-06T03:03:02.490604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:26.513756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493437+0000","last_change":"2023-10-06T03:03:02.493437+0000","last_active":"2023-10-06T03:03:02.493437+0000","last_peered":"2023-10-06T03:03:02.493437+0000","last_clean":"2023-10-06T03:03:02.493437+0000","last_became_active":"2023-10-06T03:03:02.493110+0000","last_became_peered":"2023-10-06T03:03:02.493110+0000","last_unstale":"2023-10-06T03:03:02.493437+0000","last_undegraded":"2023-10-06T03:03:02.493437+0000","last_fullsized":"2023-10-06T03:03:02.493437+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:00.390049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.494228+0000","last_change":"2023-10-06T03:03:02.488883+0000","last_active":"2023-10-06T03:03:02.494228+0000","last_peered":"2023-10-06T03:03:02.494228+0000","last_clean":"2023-10-06T03:03:02.494228+0000","last_became_active":"2023-10-06T03:03:02.488621+0000","last_became_peered":"2023-10-06T03:03:02.488621+0000","last_unstale":"2023-10-06T03:03:02.494228+0000","last_undegraded":"2023-10-06T03:03:02.494228+0000","last_fullsized":"2023-10-06T03:03:02.494228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:42:59.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.484406+0000","last_change":"2023-10-06T03:03:02.484406+0000","last_active":"2023-10-06T03:03:02.484406+0000","last_peered":"2023-10-06T03:03:02.484406+0000","last_clean":"2023-10-06T03:03:02.484406+0000","last_became_active":"2023-10-06T03:03:02.484143+0000","last_became_peered":"2023-10-06T03:03:02.484143+0000","last_unstale":"2023-10-06T03:03:02.484406+0000","last_undegraded":"2023-10-06T03:03:02.484406+0000","last_fullsized":"2023-10-06T03:03:02.484406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:42:59.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.484150+0000","last_change":"2023-10-06T03:03:02.484150+0000","last_active":"2023-10-06T03:03:02.484150+0000","last_peered":"2023-10-06T03:03:02.484150+0000","last_clean":"2023-10-06T03:03:02.484150+0000","last_became_active":"2023-10-06T03:03:02.483672+0000","last_became_peered":"2023-10-06T03:03:02.483672+0000","last_unstale":"2023-10-06T03:03:02.484150+0000","last_undegraded":"2023-10-06T03:03:02.484150+0000","last_fullsized":"2023-10-06T03:03:02.484150+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:39.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490670+0000","last_change":"2023-10-06T03:03:02.490670+0000","last_active":"2023-10-06T03:03:02.490670+0000","last_peered":"2023-10-06T03:03:02.490670+0000","last_clean":"2023-10-06T03:03:02.490670+0000","last_became_active":"2023-10-06T03:03:02.490454+0000","last_became_peered":"2023-10-06T03:03:02.490454+0000","last_unstale":"2023-10-06T03:03:02.490670+0000","last_undegraded":"2023-10-06T03:03:02.490670+0000","last_fullsized":"2023-10-06T03:03:02.490670+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:30.255226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491342+0000","last_change":"2023-10-06T03:03:02.485795+0000","last_active":"2023-10-06T03:03:02.491342+0000","last_peered":"2023-10-06T03:03:02.491342+0000","last_clean":"2023-10-06T03:03:02.491342+0000","last_became_active":"2023-10-06T03:03:02.485544+0000","last_became_peered":"2023-10-06T03:03:02.485544+0000","last_unstale":"2023-10-06T03:03:02.491342+0000","last_undegraded":"2023-10-06T03:03:02.491342+0000","last_fullsized":"2023-10-06T03:03:02.491342+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:17.907864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489196+0000","last_change":"2023-10-06T03:03:02.483874+0000","last_active":"2023-10-06T03:03:02.489196+0000","last_peered":"2023-10-06T03:03:02.489196+0000","last_clean":"2023-10-06T03:03:02.489196+0000","last_became_active":"2023-10-06T03:03:02.483594+0000","last_became_peered":"2023-10-06T03:03:02.483594+0000","last_unstale":"2023-10-06T03:03:02.489196+0000","last_undegraded":"2023-10-06T03:03:02.489196+0000","last_fullsized":"2023-10-06T03:03:02.489196+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:00.390049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491763+0000","last_change":"2023-10-06T03:03:02.481629+0000","last_active":"2023-10-06T03:03:02.491763+0000","last_peered":"2023-10-06T03:03:02.491763+0000","last_clean":"2023-10-06T03:03:02.491763+0000","last_became_active":"2023-10-06T03:03:02.481336+0000","last_became_peered":"2023-10-06T03:03:02.481336+0000","last_unstale":"2023-10-06T03:03:02.491763+0000","last_undegraded":"2023-10-06T03:03:02.491763+0000","last_fullsized":"2023-10-06T03:03:02.491763+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:52.366760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.498674+0000","last_change":"2023-10-06T03:03:02.494100+0000","last_active":"2023-10-06T03:03:02.498674+0000","last_peered":"2023-10-06T03:03:02.498674+0000","last_clean":"2023-10-06T03:03:02.498674+0000","last_became_active":"2023-10-06T03:03:02.493834+0000","last_became_peered":"2023-10-06T03:03:02.493834+0000","last_unstale":"2023-10-06T03:03:02.498674+0000","last_undegraded":"2023-10-06T03:03:02.498674+0000","last_fullsized":"2023-10-06T03:03:02.498674+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.335097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493756+0000","last_change":"2023-10-06T03:03:02.493756+0000","last_active":"2023-10-06T03:03:02.493756+0000","last_peered":"2023-10-06T03:03:02.493756+0000","last_clean":"2023-10-06T03:03:02.493756+0000","last_became_active":"2023-10-06T03:03:02.493470+0000","last_became_peered":"2023-10-06T03:03:02.493470+0000","last_unstale":"2023-10-06T03:03:02.493756+0000","last_undegraded":"2023-10-06T03:03:02.493756+0000","last_fullsized":"2023-10-06T03:03:02.493756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491126+0000","last_change":"2023-10-06T03:03:02.485135+0000","last_active":"2023-10-06T03:03:02.491126+0000","last_peered":"2023-10-06T03:03:02.491126+0000","last_clean":"2023-10-06T03:03:02.491126+0000","last_became_active":"2023-10-06T03:03:02.484889+0000","last_became_peered":"2023-10-06T03:03:02.484889+0000","last_unstale":"2023-10-06T03:03:02.491126+0000","last_undegraded":"2023-10-06T03:03:02.491126+0000","last_fullsized":"2023-10-06T03:03:02.491126+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490343+0000","last_change":"2023-10-06T03:03:02.481522+0000","last_active":"2023-10-06T03:03:02.490343+0000","last_peered":"2023-10-06T03:03:02.490343+0000","last_clean":"2023-10-06T03:03:02.490343+0000","last_became_active":"2023-10-06T03:03:02.481241+0000","last_became_peered":"2023-10-06T03:03:02.481241+0000","last_unstale":"2023-10-06T03:03:02.490343+0000","last_undegraded":"2023-10-06T03:03:02.490343+0000","last_fullsized":"2023-10-06T03:03:02.490343+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:24.544569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.483156+0000","last_change":"2023-10-06T03:03:02.483156+0000","last_active":"2023-10-06T03:03:02.483156+0000","last_peered":"2023-10-06T03:03:02.483156+0000","last_clean":"2023-10-06T03:03:02.483156+0000","last_became_active":"2023-10-06T03:03:02.482272+0000","last_became_peered":"2023-10-06T03:03:02.482272+0000","last_unstale":"2023-10-06T03:03:02.483156+0000","last_undegraded":"2023-10-06T03:03:02.483156+0000","last_fullsized":"2023-10-06T03:03:02.483156+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493071+0000","last_change":"2023-10-06T03:03:02.493071+0000","last_active":"2023-10-06T03:03:02.493071+0000","last_peered":"2023-10-06T03:03:02.493071+0000","last_clean":"2023-10-06T03:03:02.493071+0000","last_became_active":"2023-10-06T03:03:02.492045+0000","last_became_peered":"2023-10-06T03:03:02.492045+0000","last_unstale":"2023-10-06T03:03:02.493071+0000","last_undegraded":"2023-10-06T03:03:02.493071+0000","last_fullsized":"2023-10-06T03:03:02.493071+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:36.131250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.569258+0000","last_change":"2023-10-06T03:03:01.575644+0000","last_active":"2023-10-06T03:03:02.569258+0000","last_peered":"2023-10-06T03:03:02.569258+0000","last_clean":"2023-10-06T03:03:02.569258+0000","last_became_active":"2023-10-06T03:03:01.575394+0000","last_became_peered":"2023-10-06T03:03:01.575394+0000","last_unstale":"2023-10-06T03:03:02.569258+0000","last_undegraded":"2023-10-06T03:03:02.569258+0000","last_fullsized":"2023-10-06T03:03:02.569258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:19.351394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.497004+0000","last_change":"2023-10-06T03:03:02.492893+0000","last_active":"2023-10-06T03:03:02.497004+0000","last_peered":"2023-10-06T03:03:02.497004+0000","last_clean":"2023-10-06T03:03:02.497004+0000","last_became_active":"2023-10-06T03:03:02.492613+0000","last_became_peered":"2023-10-06T03:03:02.492613+0000","last_unstale":"2023-10-06T03:03:02.497004+0000","last_undegraded":"2023-10-06T03:03:02.497004+0000","last_fullsized":"2023-10-06T03:03:02.497004+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:17.907864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.480574+0000","last_change":"2023-10-06T03:03:01.575648+0000","last_active":"2023-10-06T03:03:02.480574+0000","last_peered":"2023-10-06T03:03:02.480574+0000","last_clean":"2023-10-06T03:03:02.480574+0000","last_became_active":"2023-10-06T03:03:01.575391+0000","last_became_peered":"2023-10-06T03:03:01.575391+0000","last_unstale":"2023-10-06T03:03:02.480574+0000","last_undegraded":"2023-10-06T03:03:02.480574+0000","last_fullsized":"2023-10-06T03:03:02.480574+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:51.086706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488062+0000","last_change":"2023-10-06T03:03:02.488062+0000","last_active":"2023-10-06T03:03:02.488062+0000","last_peered":"2023-10-06T03:03:02.488062+0000","last_clean":"2023-10-06T03:03:02.488062+0000","last_became_active":"2023-10-06T03:03:02.487742+0000","last_became_peered":"2023-10-06T03:03:02.487742+0000","last_unstale":"2023-10-06T03:03:02.488062+0000","last_undegraded":"2023-10-06T03:03:02.488062+0000","last_fullsized":"2023-10-06T03:03:02.488062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:52.366760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566856+0000","last_change":"2023-10-06T03:03:01.573058+0000","last_active":"2023-10-06T03:03:02.566856+0000","last_peered":"2023-10-06T03:03:02.566856+0000","last_clean":"2023-10-06T03:03:02.566856+0000","last_became_active":"2023-10-06T03:03:01.572850+0000","last_became_peered":"2023-10-06T03:03:01.572850+0000","last_unstale":"2023-10-06T03:03:02.566856+0000","last_undegraded":"2023-10-06T03:03:02.566856+0000","last_fullsized":"2023-10-06T03:03:02.566856+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:32.880340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490066+0000","last_change":"2023-10-06T03:03:02.490066+0000","last_active":"2023-10-06T03:03:02.490066+0000","last_peered":"2023-10-06T03:03:02.490066+0000","last_clean":"2023-10-06T03:03:02.490066+0000","last_became_active":"2023-10-06T03:03:02.489470+0000","last_became_peered":"2023-10-06T03:03:02.489470+0000","last_unstale":"2023-10-06T03:03:02.490066+0000","last_undegraded":"2023-10-06T03:03:02.490066+0000","last_fullsized":"2023-10-06T03:03:02.490066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566154+0000","last_change":"2023-10-06T03:03:01.571202+0000","last_active":"2023-10-06T03:03:02.566154+0000","last_peered":"2023-10-06T03:03:02.566154+0000","last_clean":"2023-10-06T03:03:02.566154+0000","last_became_active":"2023-10-06T03:03:01.570929+0000","last_became_peered":"2023-10-06T03:03:01.570929+0000","last_unstale":"2023-10-06T03:03:02.566154+0000","last_undegraded":"2023-10-06T03:03:02.566154+0000","last_fullsized":"2023-10-06T03:03:02.566154+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:17.749222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481285+0000","last_change":"2023-10-06T03:03:02.481285+0000","last_active":"2023-10-06T03:03:02.481285+0000","last_peered":"2023-10-06T03:03:02.481285+0000","last_clean":"2023-10-06T03:03:02.481285+0000","last_became_active":"2023-10-06T03:03:02.480994+0000","last_became_peered":"2023-10-06T03:03:02.480994+0000","last_unstale":"2023-10-06T03:03:02.481285+0000","last_undegraded":"2023-10-06T03:03:02.481285+0000","last_fullsized":"2023-10-06T03:03:02.481285+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:12.223774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.477999+0000","last_change":"2023-10-06T03:03:01.571719+0000","last_active":"2023-10-06T03:03:02.477999+0000","last_peered":"2023-10-06T03:03:02.477999+0000","last_clean":"2023-10-06T03:03:02.477999+0000","last_became_active":"2023-10-06T03:03:01.571502+0000","last_became_peered":"2023-10-06T03:03:01.571502+0000","last_unstale":"2023-10-06T03:03:02.477999+0000","last_undegraded":"2023-10-06T03:03:02.477999+0000","last_fullsized":"2023-10-06T03:03:02.477999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:23.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488687+0000","last_change":"2023-10-06T03:03:02.483457+0000","last_active":"2023-10-06T03:03:02.488687+0000","last_peered":"2023-10-06T03:03:02.488687+0000","last_clean":"2023-10-06T03:03:02.488687+0000","last_became_active":"2023-10-06T03:03:02.483120+0000","last_became_peered":"2023-10-06T03:03:02.483120+0000","last_unstale":"2023-10-06T03:03:02.488687+0000","last_undegraded":"2023-10-06T03:03:02.488687+0000","last_fullsized":"2023-10-06T03:03:02.488687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.336698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.478088+0000","last_change":"2023-10-06T03:03:01.570124+0000","last_active":"2023-10-06T03:03:02.478088+0000","last_peered":"2023-10-06T03:03:02.478088+0000","last_clean":"2023-10-06T03:03:02.478088+0000","last_became_active":"2023-10-06T03:03:01.569860+0000","last_became_peered":"2023-10-06T03:03:01.569860+0000","last_unstale":"2023-10-06T03:03:02.478088+0000","last_undegraded":"2023-10-06T03:03:02.478088+0000","last_fullsized":"2023-10-06T03:03:02.478088+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:29.248670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493543+0000","last_change":"2023-10-06T03:03:02.493543+0000","last_active":"2023-10-06T03:03:02.493543+0000","last_peered":"2023-10-06T03:03:02.493543+0000","last_clean":"2023-10-06T03:03:02.493543+0000","last_became_active":"2023-10-06T03:03:02.493116+0000","last_became_peered":"2023-10-06T03:03:02.493116+0000","last_unstale":"2023-10-06T03:03:02.493543+0000","last_undegraded":"2023-10-06T03:03:02.493543+0000","last_fullsized":"2023-10-06T03:03:02.493543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:46.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577884+0000","last_change":"2023-10-06T03:03:01.573857+0000","last_active":"2023-10-06T03:03:01.577884+0000","last_peered":"2023-10-06T03:03:01.577884+0000","last_clean":"2023-10-06T03:03:01.577884+0000","last_became_active":"2023-10-06T03:03:01.573460+0000","last_became_peered":"2023-10-06T03:03:01.573460+0000","last_unstale":"2023-10-06T03:03:01.577884+0000","last_undegraded":"2023-10-06T03:03:01.577884+0000","last_fullsized":"2023-10-06T03:03:01.577884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:51.148260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493894+0000","last_change":"2023-10-06T03:03:02.493894+0000","last_active":"2023-10-06T03:03:02.493894+0000","last_peered":"2023-10-06T03:03:02.493894+0000","last_clean":"2023-10-06T03:03:02.493894+0000","last_became_active":"2023-10-06T03:03:02.493623+0000","last_became_peered":"2023-10-06T03:03:02.493623+0000","last_unstale":"2023-10-06T03:03:02.493894+0000","last_undegraded":"2023-10-06T03:03:02.493894+0000","last_fullsized":"2023-10-06T03:03:02.493894+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:08.428638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.579202+0000","last_change":"2023-10-06T03:03:01.575666+0000","last_active":"2023-10-06T03:03:01.579202+0000","last_peered":"2023-10-06T03:03:01.579202+0000","last_clean":"2023-10-06T03:03:01.579202+0000","last_became_active":"2023-10-06T03:03:01.575409+0000","last_became_peered":"2023-10-06T03:03:01.575409+0000","last_unstale":"2023-10-06T03:03:01.579202+0000","last_undegraded":"2023-10-06T03:03:01.579202+0000","last_fullsized":"2023-10-06T03:03:01.579202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:04.862470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488348+0000","last_change":"2023-10-06T03:03:02.481938+0000","last_active":"2023-10-06T03:03:02.488348+0000","last_peered":"2023-10-06T03:03:02.488348+0000","last_clean":"2023-10-06T03:03:02.488348+0000","last_became_active":"2023-10-06T03:03:02.481651+0000","last_became_peered":"2023-10-06T03:03:02.481651+0000","last_unstale":"2023-10-06T03:03:02.488348+0000","last_undegraded":"2023-10-06T03:03:02.488348+0000","last_fullsized":"2023-10-06T03:03:02.488348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:46.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.581688+0000","last_change":"2023-10-06T03:03:01.578081+0000","last_active":"2023-10-06T03:03:01.581688+0000","last_peered":"2023-10-06T03:03:01.581688+0000","last_clean":"2023-10-06T03:03:01.581688+0000","last_became_active":"2023-10-06T03:03:01.577868+0000","last_became_peered":"2023-10-06T03:03:01.577868+0000","last_unstale":"2023-10-06T03:03:01.581688+0000","last_undegraded":"2023-10-06T03:03:01.581688+0000","last_fullsized":"2023-10-06T03:03:01.581688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:02.010436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.484712+0000","last_change":"2023-10-06T03:03:02.484712+0000","last_active":"2023-10-06T03:03:02.484712+0000","last_peered":"2023-10-06T03:03:02.484712+0000","last_clean":"2023-10-06T03:03:02.484712+0000","last_became_active":"2023-10-06T03:03:02.484395+0000","last_became_peered":"2023-10-06T03:03:02.484395+0000","last_unstale":"2023-10-06T03:03:02.484712+0000","last_undegraded":"2023-10-06T03:03:02.484712+0000","last_fullsized":"2023-10-06T03:03:02.484712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.846528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.476264+0000","last_change":"2023-10-06T03:03:01.565506+0000","last_active":"2023-10-06T03:03:02.476264+0000","last_peered":"2023-10-06T03:03:02.476264+0000","last_clean":"2023-10-06T03:03:02.476264+0000","last_became_active":"2023-10-06T03:03:01.565190+0000","last_became_peered":"2023-10-06T03:03:01.565190+0000","last_unstale":"2023-10-06T03:03:02.476264+0000","last_undegraded":"2023-10-06T03:03:02.476264+0000","last_fullsized":"2023-10-06T03:03:02.476264+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.008675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493349+0000","last_change":"2023-10-06T03:03:02.493349+0000","last_active":"2023-10-06T03:03:02.493349+0000","last_peered":"2023-10-06T03:03:02.493349+0000","last_clean":"2023-10-06T03:03:02.493349+0000","last_became_active":"2023-10-06T03:03:02.493075+0000","last_became_peered":"2023-10-06T03:03:02.493075+0000","last_unstale":"2023-10-06T03:03:02.493349+0000","last_undegraded":"2023-10-06T03:03:02.493349+0000","last_fullsized":"2023-10-06T03:03:02.493349+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:28.188295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"53'3","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566145+0000","last_change":"2023-10-06T03:03:01.578277+0000","last_active":"2023-10-06T03:03:02.566145+0000","last_peered":"2023-10-06T03:03:02.566145+0000","last_clean":"2023-10-06T03:03:02.566145+0000","last_became_active":"2023-10-06T03:03:01.578022+0000","last_became_peered":"2023-10-06T03:03:01.578022+0000","last_unstale":"2023-10-06T03:03:02.566145+0000","last_undegraded":"2023-10-06T03:03:02.566145+0000","last_fullsized":"2023-10-06T03:03:02.566145+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:16.514444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492976+0000","last_change":"2023-10-06T03:03:02.487161+0000","last_active":"2023-10-06T03:03:02.492976+0000","last_peered":"2023-10-06T03:03:02.492976+0000","last_clean":"2023-10-06T03:03:02.492976+0000","last_became_active":"2023-10-06T03:03:02.486864+0000","last_became_peered":"2023-10-06T03:03:02.486864+0000","last_unstale":"2023-10-06T03:03:02.492976+0000","last_undegraded":"2023-10-06T03:03:02.492976+0000","last_fullsized":"2023-10-06T03:03:02.492976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:33.650190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.569596+0000","last_change":"2023-10-06T03:03:01.566835+0000","last_active":"2023-10-06T03:03:02.569596+0000","last_peered":"2023-10-06T03:03:02.569596+0000","last_clean":"2023-10-06T03:03:02.569596+0000","last_became_active":"2023-10-06T03:03:01.566351+0000","last_became_peered":"2023-10-06T03:03:01.566351+0000","last_unstale":"2023-10-06T03:03:02.569596+0000","last_undegraded":"2023-10-06T03:03:02.569596+0000","last_fullsized":"2023-10-06T03:03:02.569596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:24.009268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492953+0000","last_change":"2023-10-06T03:03:02.488237+0000","last_active":"2023-10-06T03:03:02.492953+0000","last_peered":"2023-10-06T03:03:02.492953+0000","last_clean":"2023-10-06T03:03:02.492953+0000","last_became_active":"2023-10-06T03:03:02.487979+0000","last_became_peered":"2023-10-06T03:03:02.487979+0000","last_unstale":"2023-10-06T03:03:02.492953+0000","last_undegraded":"2023-10-06T03:03:02.492953+0000","last_fullsized":"2023-10-06T03:03:02.492953+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:12.848469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.568912+0000","last_change":"2023-10-06T03:03:01.577347+0000","last_active":"2023-10-06T03:03:02.568912+0000","last_peered":"2023-10-06T03:03:02.568912+0000","last_clean":"2023-10-06T03:03:02.568912+0000","last_became_active":"2023-10-06T03:03:01.577073+0000","last_became_peered":"2023-10-06T03:03:01.577073+0000","last_unstale":"2023-10-06T03:03:02.568912+0000","last_undegraded":"2023-10-06T03:03:02.568912+0000","last_fullsized":"2023-10-06T03:03:02.568912+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:11.208832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491956+0000","last_change":"2023-10-06T03:03:02.488394+0000","last_active":"2023-10-06T03:03:02.491956+0000","last_peered":"2023-10-06T03:03:02.491956+0000","last_clean":"2023-10-06T03:03:02.491956+0000","last_became_active":"2023-10-06T03:03:02.488139+0000","last_became_peered":"2023-10-06T03:03:02.488139+0000","last_unstale":"2023-10-06T03:03:02.491956+0000","last_undegraded":"2023-10-06T03:03:02.491956+0000","last_fullsized":"2023-10-06T03:03:02.491956+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:39.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.478929+0000","last_change":"2023-10-06T03:03:01.571632+0000","last_active":"2023-10-06T03:03:02.478929+0000","last_peered":"2023-10-06T03:03:02.478929+0000","last_clean":"2023-10-06T03:03:02.478929+0000","last_became_active":"2023-10-06T03:03:01.571226+0000","last_became_peered":"2023-10-06T03:03:01.571226+0000","last_unstale":"2023-10-06T03:03:02.478929+0000","last_undegraded":"2023-10-06T03:03:02.478929+0000","last_fullsized":"2023-10-06T03:03:02.478929+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:36.328541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492135+0000","last_change":"2023-10-06T03:03:02.492135+0000","last_active":"2023-10-06T03:03:02.492135+0000","last_peered":"2023-10-06T03:03:02.492135+0000","last_clean":"2023-10-06T03:03:02.492135+0000","last_became_active":"2023-10-06T03:03:02.491581+0000","last_became_peered":"2023-10-06T03:03:02.491581+0000","last_unstale":"2023-10-06T03:03:02.492135+0000","last_undegraded":"2023-10-06T03:03:02.492135+0000","last_fullsized":"2023-10-06T03:03:02.492135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:06.481908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.581761+0000","last_change":"2023-10-06T03:03:01.578534+0000","last_active":"2023-10-06T03:03:01.581761+0000","last_peered":"2023-10-06T03:03:01.581761+0000","last_clean":"2023-10-06T03:03:01.581761+0000","last_became_active":"2023-10-06T03:03:01.578115+0000","last_became_peered":"2023-10-06T03:03:01.578115+0000","last_unstale":"2023-10-06T03:03:01.581761+0000","last_undegraded":"2023-10-06T03:03:01.581761+0000","last_fullsized":"2023-10-06T03:03:01.581761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:45.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490025+0000","last_change":"2023-10-06T03:03:02.486595+0000","last_active":"2023-10-06T03:03:02.490025+0000","last_peered":"2023-10-06T03:03:02.490025+0000","last_clean":"2023-10-06T03:03:02.490025+0000","last_became_active":"2023-10-06T03:03:02.486354+0000","last_became_peered":"2023-10-06T03:03:02.486354+0000","last_unstale":"2023-10-06T03:03:02.490025+0000","last_undegraded":"2023-10-06T03:03:02.490025+0000","last_fullsized":"2023-10-06T03:03:02.490025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:17.521000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'8","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.569693+0000","last_change":"2023-10-06T03:03:01.570862+0000","last_active":"2023-10-06T03:03:02.569693+0000","last_peered":"2023-10-06T03:03:02.569693+0000","last_clean":"2023-10-06T03:03:02.569693+0000","last_became_active":"2023-10-06T03:03:01.570568+0000","last_became_peered":"2023-10-06T03:03:01.570568+0000","last_unstale":"2023-10-06T03:03:02.569693+0000","last_undegraded":"2023-10-06T03:03:02.569693+0000","last_fullsized":"2023-10-06T03:03:02.569693+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:11.208832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.483470+0000","last_change":"2023-10-06T03:03:02.483470+0000","last_active":"2023-10-06T03:03:02.483470+0000","last_peered":"2023-10-06T03:03:02.483470+0000","last_clean":"2023-10-06T03:03:02.483470+0000","last_became_active":"2023-10-06T03:03:02.483212+0000","last_became_peered":"2023-10-06T03:03:02.483212+0000","last_unstale":"2023-10-06T03:03:02.483470+0000","last_undegraded":"2023-10-06T03:03:02.483470+0000","last_fullsized":"2023-10-06T03:03:02.483470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'2","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.568872+0000","last_change":"2023-10-06T03:03:01.574899+0000","last_active":"2023-10-06T03:03:02.568872+0000","last_peered":"2023-10-06T03:03:02.568872+0000","last_clean":"2023-10-06T03:03:02.568872+0000","last_became_active":"2023-10-06T03:03:01.574653+0000","last_became_peered":"2023-10-06T03:03:01.574653+0000","last_unstale":"2023-10-06T03:03:02.568872+0000","last_undegraded":"2023-10-06T03:03:02.568872+0000","last_fullsized":"2023-10-06T03:03:02.568872+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.008675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488144+0000","last_change":"2023-10-06T03:03:02.488144+0000","last_active":"2023-10-06T03:03:02.488144+0000","last_peered":"2023-10-06T03:03:02.488144+0000","last_clean":"2023-10-06T03:03:02.488144+0000","last_became_active":"2023-10-06T03:03:02.487900+0000","last_became_peered":"2023-10-06T03:03:02.487900+0000","last_unstale":"2023-10-06T03:03:02.488144+0000","last_undegraded":"2023-10-06T03:03:02.488144+0000","last_fullsized":"2023-10-06T03:03:02.488144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:52:03.868865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.477916+0000","last_change":"2023-10-06T03:03:01.570067+0000","last_active":"2023-10-06T03:03:02.477916+0000","last_peered":"2023-10-06T03:03:02.477916+0000","last_clean":"2023-10-06T03:03:02.477916+0000","last_became_active":"2023-10-06T03:03:01.569816+0000","last_became_peered":"2023-10-06T03:03:01.569816+0000","last_unstale":"2023-10-06T03:03:02.477916+0000","last_undegraded":"2023-10-06T03:03:02.477916+0000","last_fullsized":"2023-10-06T03:03:02.477916+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:11.208832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.494196+0000","last_change":"2023-10-06T03:03:02.489046+0000","last_active":"2023-10-06T03:03:02.494196+0000","last_peered":"2023-10-06T03:03:02.494196+0000","last_clean":"2023-10-06T03:03:02.494196+0000","last_became_active":"2023-10-06T03:03:02.488825+0000","last_became_peered":"2023-10-06T03:03:02.488825+0000","last_unstale":"2023-10-06T03:03:02.494196+0000","last_undegraded":"2023-10-06T03:03:02.494196+0000","last_fullsized":"2023-10-06T03:03:02.494196+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:08.881932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577847+0000","last_change":"2023-10-06T03:03:01.573133+0000","last_active":"2023-10-06T03:03:01.577847+0000","last_peered":"2023-10-06T03:03:01.577847+0000","last_clean":"2023-10-06T03:03:01.577847+0000","last_became_active":"2023-10-06T03:03:01.572897+0000","last_became_peered":"2023-10-06T03:03:01.572897+0000","last_unstale":"2023-10-06T03:03:01.577847+0000","last_undegraded":"2023-10-06T03:03:01.577847+0000","last_fullsized":"2023-10-06T03:03:01.577847+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:58.537763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491754+0000","last_change":"2023-10-06T03:03:02.491754+0000","last_active":"2023-10-06T03:03:02.491754+0000","last_peered":"2023-10-06T03:03:02.491754+0000","last_clean":"2023-10-06T03:03:02.491754+0000","last_became_active":"2023-10-06T03:03:02.491434+0000","last_became_peered":"2023-10-06T03:03:02.491434+0000","last_unstale":"2023-10-06T03:03:02.491754+0000","last_undegraded":"2023-10-06T03:03:02.491754+0000","last_fullsized":"2023-10-06T03:03:02.491754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:59.255542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.474176+0000","last_change":"2023-10-06T03:03:01.567735+0000","last_active":"2023-10-06T03:03:02.474176+0000","last_peered":"2023-10-06T03:03:02.474176+0000","last_clean":"2023-10-06T03:03:02.474176+0000","last_became_active":"2023-10-06T03:03:01.567440+0000","last_became_peered":"2023-10-06T03:03:01.567440+0000","last_unstale":"2023-10-06T03:03:02.474176+0000","last_undegraded":"2023-10-06T03:03:02.474176+0000","last_fullsized":"2023-10-06T03:03:02.474176+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:51.437003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490922+0000","last_change":"2023-10-06T03:03:02.490922+0000","last_active":"2023-10-06T03:03:02.490922+0000","last_peered":"2023-10-06T03:03:02.490922+0000","last_clean":"2023-10-06T03:03:02.490922+0000","last_became_active":"2023-10-06T03:03:02.490614+0000","last_became_peered":"2023-10-06T03:03:02.490614+0000","last_unstale":"2023-10-06T03:03:02.490922+0000","last_undegraded":"2023-10-06T03:03:02.490922+0000","last_fullsized":"2023-10-06T03:03:02.490922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:42.752266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.581773+0000","last_change":"2023-10-06T03:03:01.578668+0000","last_active":"2023-10-06T03:03:01.581773+0000","last_peered":"2023-10-06T03:03:01.581773+0000","last_clean":"2023-10-06T03:03:01.581773+0000","last_became_active":"2023-10-06T03:03:01.578364+0000","last_became_peered":"2023-10-06T03:03:01.578364+0000","last_unstale":"2023-10-06T03:03:01.581773+0000","last_undegraded":"2023-10-06T03:03:01.581773+0000","last_fullsized":"2023-10-06T03:03:01.581773+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.330142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492878+0000","last_change":"2023-10-06T03:03:02.485962+0000","last_active":"2023-10-06T03:03:02.492878+0000","last_peered":"2023-10-06T03:03:02.492878+0000","last_clean":"2023-10-06T03:03:02.492878+0000","last_became_active":"2023-10-06T03:03:02.485683+0000","last_became_peered":"2023-10-06T03:03:02.485683+0000","last_unstale":"2023-10-06T03:03:02.492878+0000","last_undegraded":"2023-10-06T03:03:02.492878+0000","last_fullsized":"2023-10-06T03:03:02.492878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:52.154816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.578038+0000","last_change":"2023-10-06T03:03:01.572643+0000","last_active":"2023-10-06T03:03:01.578038+0000","last_peered":"2023-10-06T03:03:01.578038+0000","last_clean":"2023-10-06T03:03:01.578038+0000","last_became_active":"2023-10-06T03:03:01.572373+0000","last_became_peered":"2023-10-06T03:03:01.572373+0000","last_unstale":"2023-10-06T03:03:01.578038+0000","last_undegraded":"2023-10-06T03:03:01.578038+0000","last_fullsized":"2023-10-06T03:03:01.578038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:24.721089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488556+0000","last_change":"2023-10-06T03:03:02.482115+0000","last_active":"2023-10-06T03:03:02.488556+0000","last_peered":"2023-10-06T03:03:02.488556+0000","last_clean":"2023-10-06T03:03:02.488556+0000","last_became_active":"2023-10-06T03:03:02.481822+0000","last_became_peered":"2023-10-06T03:03:02.481822+0000","last_unstale":"2023-10-06T03:03:02.488556+0000","last_undegraded":"2023-10-06T03:03:02.488556+0000","last_fullsized":"2023-10-06T03:03:02.488556+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:12.215388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577866+0000","last_change":"2023-10-06T03:03:01.573682+0000","last_active":"2023-10-06T03:03:01.577866+0000","last_peered":"2023-10-06T03:03:01.577866+0000","last_clean":"2023-10-06T03:03:01.577866+0000","last_became_active":"2023-10-06T03:03:01.573390+0000","last_became_peered":"2023-10-06T03:03:01.573390+0000","last_unstale":"2023-10-06T03:03:01.577866+0000","last_undegraded":"2023-10-06T03:03:01.577866+0000","last_fullsized":"2023-10-06T03:03:01.577866+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:32.643634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481218+0000","last_change":"2023-10-06T03:03:02.481218+0000","last_active":"2023-10-06T03:03:02.481218+0000","last_peered":"2023-10-06T03:03:02.481218+0000","last_clean":"2023-10-06T03:03:02.481218+0000","last_became_active":"2023-10-06T03:03:02.480916+0000","last_became_peered":"2023-10-06T03:03:02.480916+0000","last_unstale":"2023-10-06T03:03:02.481218+0000","last_undegraded":"2023-10-06T03:03:02.481218+0000","last_fullsized":"2023-10-06T03:03:02.481218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:59.858987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.473425+0000","last_change":"2023-10-06T03:03:01.575712+0000","last_active":"2023-10-06T03:03:02.473425+0000","last_peered":"2023-10-06T03:03:02.473425+0000","last_clean":"2023-10-06T03:03:02.473425+0000","last_became_active":"2023-10-06T03:03:01.575488+0000","last_became_peered":"2023-10-06T03:03:01.575488+0000","last_unstale":"2023-10-06T03:03:02.473425+0000","last_undegraded":"2023-10-06T03:03:02.473425+0000","last_fullsized":"2023-10-06T03:03:02.473425+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:48.973130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491547+0000","last_change":"2023-10-06T03:03:02.491547+0000","last_active":"2023-10-06T03:03:02.491547+0000","last_peered":"2023-10-06T03:03:02.491547+0000","last_clean":"2023-10-06T03:03:02.491547+0000","last_became_active":"2023-10-06T03:03:02.491261+0000","last_became_peered":"2023-10-06T03:03:02.491261+0000","last_unstale":"2023-10-06T03:03:02.491547+0000","last_undegraded":"2023-10-06T03:03:02.491547+0000","last_fullsized":"2023-10-06T03:03:02.491547+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:49.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577890+0000","last_change":"2023-10-06T03:03:01.571046+0000","last_active":"2023-10-06T03:03:01.577890+0000","last_peered":"2023-10-06T03:03:01.577890+0000","last_clean":"2023-10-06T03:03:01.577890+0000","last_became_active":"2023-10-06T03:03:01.570634+0000","last_became_peered":"2023-10-06T03:03:01.570634+0000","last_unstale":"2023-10-06T03:03:01.577890+0000","last_undegraded":"2023-10-06T03:03:01.577890+0000","last_fullsized":"2023-10-06T03:03:01.577890+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:55.922520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.574782+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:01.574782+0000","last_peered":"2023-10-06T03:03:01.574782+0000","last_clean":"2023-10-06T03:03:01.574782+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:01.574782+0000","last_undegraded":"2023-10-06T03:03:01.574782+0000","last_fullsized":"2023-10-06T03:03:01.574782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.496979+0000","last_change":"2023-10-06T03:03:02.492743+0000","last_active":"2023-10-06T03:03:02.496979+0000","last_peered":"2023-10-06T03:03:02.496979+0000","last_clean":"2023-10-06T03:03:02.496979+0000","last_became_active":"2023-10-06T03:03:02.492474+0000","last_became_peered":"2023-10-06T03:03:02.492474+0000","last_unstale":"2023-10-06T03:03:02.496979+0000","last_undegraded":"2023-10-06T03:03:02.496979+0000","last_fullsized":"2023-10-06T03:03:02.496979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:08.881932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481606+0000","last_change":"2023-10-06T03:03:01.578439+0000","last_active":"2023-10-06T03:03:02.481606+0000","last_peered":"2023-10-06T03:03:02.481606+0000","last_clean":"2023-10-06T03:03:02.481606+0000","last_became_active":"2023-10-06T03:03:01.578099+0000","last_became_peered":"2023-10-06T03:03:01.578099+0000","last_unstale":"2023-10-06T03:03:02.481606+0000","last_undegraded":"2023-10-06T03:03:02.481606+0000","last_fullsized":"2023-10-06T03:03:02.481606+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:39.896166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.474290+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:02.474290+0000","last_peered":"2023-10-06T03:03:02.474290+0000","last_clean":"2023-10-06T03:03:02.474290+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:02.474290+0000","last_undegraded":"2023-10-06T03:03:02.474290+0000","last_fullsized":"2023-10-06T03:03:02.474290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.495619+0000","last_change":"2023-10-06T03:03:02.495619+0000","last_active":"2023-10-06T03:03:02.495619+0000","last_peered":"2023-10-06T03:03:02.495619+0000","last_clean":"2023-10-06T03:03:02.495619+0000","last_became_active":"2023-10-06T03:03:02.494843+0000","last_became_peered":"2023-10-06T03:03:02.494843+0000","last_unstale":"2023-10-06T03:03:02.495619+0000","last_undegraded":"2023-10-06T03:03:02.495619+0000","last_fullsized":"2023-10-06T03:03:02.495619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:17.521000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.568864+0000","last_change":"2023-10-06T03:03:01.579531+0000","last_active":"2023-10-06T03:03:02.568864+0000","last_peered":"2023-10-06T03:03:02.568864+0000","last_clean":"2023-10-06T03:03:02.568864+0000","last_became_active":"2023-10-06T03:03:01.579281+0000","last_became_peered":"2023-10-06T03:03:01.579281+0000","last_unstale":"2023-10-06T03:03:02.568864+0000","last_undegraded":"2023-10-06T03:03:02.568864+0000","last_fullsized":"2023-10-06T03:03:02.568864+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:35.124694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.570742+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:01.570742+0000","last_peered":"2023-10-06T03:03:01.570742+0000","last_clean":"2023-10-06T03:03:01.570742+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:01.570742+0000","last_undegraded":"2023-10-06T03:03:01.570742+0000","last_fullsized":"2023-10-06T03:03:01.570742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481237+0000","last_change":"2023-10-06T03:03:02.481237+0000","last_active":"2023-10-06T03:03:02.481237+0000","last_peered":"2023-10-06T03:03:02.481237+0000","last_clean":"2023-10-06T03:03:02.481237+0000","last_became_active":"2023-10-06T03:03:02.481024+0000","last_became_peered":"2023-10-06T03:03:02.481024+0000","last_unstale":"2023-10-06T03:03:02.481237+0000","last_undegraded":"2023-10-06T03:03:02.481237+0000","last_fullsized":"2023-10-06T03:03:02.481237+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:24.544569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.567255+0000","last_change":"2023-10-06T03:03:01.575675+0000","last_active":"2023-10-06T03:03:02.567255+0000","last_peered":"2023-10-06T03:03:02.567255+0000","last_clean":"2023-10-06T03:03:02.567255+0000","last_became_active":"2023-10-06T03:03:01.575451+0000","last_became_peered":"2023-10-06T03:03:01.575451+0000","last_unstale":"2023-10-06T03:03:02.567255+0000","last_undegraded":"2023-10-06T03:03:02.567255+0000","last_fullsized":"2023-10-06T03:03:02.567255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.008675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.569043+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:01.569043+0000","last_peered":"2023-10-06T03:03:01.569043+0000","last_clean":"2023-10-06T03:03:01.569043+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:01.569043+0000","last_undegraded":"2023-10-06T03:03:01.569043+0000","last_fullsized":"2023-10-06T03:03:01.569043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491152+0000","last_change":"2023-10-06T03:03:02.491152+0000","last_active":"2023-10-06T03:03:02.491152+0000","last_peered":"2023-10-06T03:03:02.491152+0000","last_clean":"2023-10-06T03:03:02.491152+0000","last_became_active":"2023-10-06T03:03:02.490883+0000","last_became_peered":"2023-10-06T03:03:02.490883+0000","last_unstale":"2023-10-06T03:03:02.491152+0000","last_undegraded":"2023-10-06T03:03:02.491152+0000","last_fullsized":"2023-10-06T03:03:02.491152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:40.902722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577916+0000","last_change":"2023-10-06T03:03:01.570001+0000","last_active":"2023-10-06T03:03:01.577916+0000","last_peered":"2023-10-06T03:03:01.577916+0000","last_clean":"2023-10-06T03:03:01.577916+0000","last_became_active":"2023-10-06T03:03:01.569714+0000","last_became_peered":"2023-10-06T03:03:01.569714+0000","last_unstale":"2023-10-06T03:03:01.577916+0000","last_undegraded":"2023-10-06T03:03:01.577916+0000","last_fullsized":"2023-10-06T03:03:01.577916+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:51.360204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.569129+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:01.569129+0000","last_peered":"2023-10-06T03:03:01.569129+0000","last_clean":"2023-10-06T03:03:01.569129+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:01.569129+0000","last_undegraded":"2023-10-06T03:03:01.569129+0000","last_fullsized":"2023-10-06T03:03:01.569129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.477045+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:02.477045+0000","last_peered":"2023-10-06T03:03:02.477045+0000","last_clean":"2023-10-06T03:03:02.477045+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:02.477045+0000","last_undegraded":"2023-10-06T03:03:02.477045+0000","last_fullsized":"2023-10-06T03:03:02.477045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489236+0000","last_change":"2023-10-06T03:03:02.483349+0000","last_active":"2023-10-06T03:03:02.489236+0000","last_peered":"2023-10-06T03:03:02.489236+0000","last_clean":"2023-10-06T03:03:02.489236+0000","last_became_active":"2023-10-06T03:03:02.483137+0000","last_became_peered":"2023-10-06T03:03:02.483137+0000","last_unstale":"2023-10-06T03:03:02.489236+0000","last_undegraded":"2023-10-06T03:03:02.489236+0000","last_fullsized":"2023-10-06T03:03:02.489236+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.846528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.478959+0000","last_change":"2023-10-06T03:03:01.571488+0000","last_active":"2023-10-06T03:03:02.478959+0000","last_peered":"2023-10-06T03:03:02.478959+0000","last_clean":"2023-10-06T03:03:02.478959+0000","last_became_active":"2023-10-06T03:03:01.570904+0000","last_became_peered":"2023-10-06T03:03:01.570904+0000","last_unstale":"2023-10-06T03:03:02.478959+0000","last_undegraded":"2023-10-06T03:03:02.478959+0000","last_fullsized":"2023-10-06T03:03:02.478959+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:02.639969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.566181+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:01.566181+0000","last_peered":"2023-10-06T03:03:01.566181+0000","last_clean":"2023-10-06T03:03:01.566181+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:01.566181+0000","last_undegraded":"2023-10-06T03:03:01.566181+0000","last_fullsized":"2023-10-06T03:03:01.566181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491934+0000","last_change":"2023-10-06T03:03:02.491934+0000","last_active":"2023-10-06T03:03:02.491934+0000","last_peered":"2023-10-06T03:03:02.491934+0000","last_clean":"2023-10-06T03:03:02.491934+0000","last_became_active":"2023-10-06T03:03:02.491231+0000","last_became_peered":"2023-10-06T03:03:02.491231+0000","last_unstale":"2023-10-06T03:03:02.491934+0000","last_undegraded":"2023-10-06T03:03:02.491934+0000","last_fullsized":"2023-10-06T03:03:02.491934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:49.090760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.580689+0000","last_change":"2023-10-06T03:03:01.575645+0000","last_active":"2023-10-06T03:03:01.580689+0000","last_peered":"2023-10-06T03:03:01.580689+0000","last_clean":"2023-10-06T03:03:01.580689+0000","last_became_active":"2023-10-06T03:03:01.575381+0000","last_became_peered":"2023-10-06T03:03:01.575381+0000","last_unstale":"2023-10-06T03:03:01.580689+0000","last_undegraded":"2023-10-06T03:03:01.580689+0000","last_fullsized":"2023-10-06T03:03:01.580689+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:36.328541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.479055+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:02.479055+0000","last_peered":"2023-10-06T03:03:02.479055+0000","last_clean":"2023-10-06T03:03:02.479055+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:02.479055+0000","last_undegraded":"2023-10-06T03:03:02.479055+0000","last_fullsized":"2023-10-06T03:03:02.479055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493856+0000","last_change":"2023-10-06T03:03:02.493856+0000","last_active":"2023-10-06T03:03:02.493856+0000","last_peered":"2023-10-06T03:03:02.493856+0000","last_clean":"2023-10-06T03:03:02.493856+0000","last_became_active":"2023-10-06T03:03:02.493611+0000","last_became_peered":"2023-10-06T03:03:02.493611+0000","last_unstale":"2023-10-06T03:03:02.493856+0000","last_undegraded":"2023-10-06T03:03:02.493856+0000","last_fullsized":"2023-10-06T03:03:02.493856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"53'1","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566099+0000","last_change":"2023-10-06T03:03:01.571117+0000","last_active":"2023-10-06T03:03:02.566099+0000","last_peered":"2023-10-06T03:03:02.566099+0000","last_clean":"2023-10-06T03:03:02.566099+0000","last_became_active":"2023-10-06T03:03:01.570838+0000","last_became_peered":"2023-10-06T03:03:01.570838+0000","last_unstale":"2023-10-06T03:03:02.566099+0000","last_undegraded":"2023-10-06T03:03:02.566099+0000","last_fullsized":"2023-10-06T03:03:02.566099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:51.086706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.473527+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:02.473527+0000","last_peered":"2023-10-06T03:03:02.473527+0000","last_clean":"2023-10-06T03:03:02.473527+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:02.473527+0000","last_undegraded":"2023-10-06T03:03:02.473527+0000","last_fullsized":"2023-10-06T03:03:02.473527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298504,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073336,"statfs":{"total":96636764160,"available":96331096064,"internally_reserved":0,"allocated":1245184,"data_stored":872927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.63},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297912,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073928,"statfs":{"total":96636764160,"available":96331702272,"internally_reserved":0,"allocated":638976,"data_stored":281845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297948,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073892,"statfs":{"total":96636764160,"available":96331665408,"internally_reserved":0,"allocated":675840,"data_stored":290724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.253},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.26100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298488,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073352,"statfs":{"total":96636764160,"available":96331112448,"internally_reserved":0,"allocated":1228800,"data_stored":872179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.622}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":281811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.312}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.628}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":281811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":5.5220000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297932,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073908,"statfs":{"total":96636764160,"available":96331681792,"internally_reserved":0,"allocated":659456,"data_stored":289988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.62},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.44}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297924,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073916,"statfs":{"total":96636764160,"available":96331689984,"internally_reserved":0,"allocated":651264,"data_stored":288474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.3869999999999996,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:06.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:06.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.769+0000 7efec3596700 1 Processor -- start 2023-10-06T03:03:06.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.769+0000 7efec3596700 1 -- start start 2023-10-06T03:03:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec3596700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efebc159890 0x7efebc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec3596700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efebc069560 0x7efebc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec3596700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efebc069ee0 con 0x7efebc159c60 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec3596700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efebc15a160 con 0x7efebc159890 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec3596700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efebc0d13a0 con 0x7efebc069560 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec1332700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efebc159890 0x7efebc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec1332700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efebc159890 0x7efebc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40254/0 (socket says 172.21.15.92:40254) 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.770+0000 7efec1332700 1 -- 172.21.15.92:0/2556407442 learned_addr learned my addr 172.21.15.92:0/2556407442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1b33700 1 --2- 172.21.15.92:0/2556407442 >> v2:172.21.15.136:3300/0 conn(0x7efebc069560 0x7efebc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1332700 1 -- 172.21.15.92:0/2556407442 >> v2:172.21.15.136:3300/0 conn(0x7efebc069560 msgr2=0x7efebc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1332700 1 --2- 172.21.15.92:0/2556407442 >> v2:172.21.15.136:3300/0 conn(0x7efebc069560 0x7efebc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1332700 1 -- 172.21.15.92:0/2556407442 >> v1:172.21.15.136:6789/0 conn(0x7efebc159c60 legacy=0x7efebc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1332700 1 -- 172.21.15.92:0/2556407442 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc15e140 con 0x7efebc159890 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1b33700 1 --2- 172.21.15.92:0/2556407442 >> v2:172.21.15.136:3300/0 conn(0x7efebc069560 0x7efebc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec1332700 1 --2- 172.21.15.92:0/2556407442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159890 0x7efebc15abd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efebc06a890 tx=0x7efeb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f1345d59886985 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efeb3fff700 1 -- 172.21.15.92:0/2556407442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb800e070 con 0x7efebc159890 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efeb3fff700 1 -- 172.21.15.92:0/2556407442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb800ba10 con 0x7efebc159890 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efeb3fff700 1 -- 172.21.15.92:0/2556407442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb80092e0 con 0x7efebc159890 2023-10-06T03:03:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec3596700 1 -- 172.21.15.92:0/2556407442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159890 msgr2=0x7efebc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec3596700 1 --2- 172.21.15.92:0/2556407442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159890 0x7efebc15abd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efebc06a890 tx=0x7efeb8009840 comp rx=0 tx=0).stop 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec3596700 1 -- 172.21.15.92:0/2556407442 shutdown_connections 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec3596700 1 --2- 172.21.15.92:0/2556407442 >> v2:172.21.15.136:3300/0 conn(0x7efebc069560 0x7efebc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec3596700 1 --2- 172.21.15.92:0/2556407442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159890 0x7efebc15abd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.771+0000 7efec3596700 1 -- 172.21.15.92:0/2556407442 >> 172.21.15.92:0/2556407442 conn(0x7efebc068e20 msgr2=0x7efebc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.772+0000 7efec3596700 1 -- 172.21.15.92:0/2556407442 shutdown_connections 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.772+0000 7efec3596700 1 -- 172.21.15.92:0/2556407442 wait complete. 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.773+0000 7efec3596700 1 Processor -- start 2023-10-06T03:03:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.773+0000 7efec3596700 1 -- start start 2023-10-06T03:03:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.773+0000 7efec3596700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efebc069560 0x7efebc0b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.773+0000 7efec3596700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efebc159890 0x7efebc0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.773+0000 7efec3596700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 0x7efebc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec3596700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc0cec00 con 0x7efebc159c60 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec3596700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc0ce770 con 0x7efebc069560 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec3596700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efebc0cdb60 con 0x7efebc159890 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 0x7efebc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 0x7efebc0b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40262/0 (socket says 172.21.15.92:40262) 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 -- 172.21.15.92:0/3451810995 learned_addr learned my addr 172.21.15.92:0/3451810995 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1332700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efebc069560 0x7efebc0b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec0b31700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efebc159890 0x7efebc0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 -- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efebc159890 msgr2=0x7efebc0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efebc159890 0x7efebc0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 -- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efebc069560 msgr2=0x7efebc0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efebc069560 0x7efebc0b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 -- 172.21.15.92:0/3451810995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb8009ae0 con 0x7efebc159c60 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec0b31700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efebc159890 0x7efebc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec1b33700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 0x7efebc0b4dd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c880 tx=0x7efeb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4011600 con 0x7efebc159c60 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb4011760 con 0x7efebc159c60 2023-10-06T03:03:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4011b20 con 0x7efebc159c60 2023-10-06T03:03:06.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc0cde40 con 0x7efebc159c60 2023-10-06T03:03:06.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.774+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc0ce400 con 0x7efebc159c60 2023-10-06T03:03:06.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.775+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc0ce920 con 0x7efebc159c60 2023-10-06T03:03:06.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.776+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efeb40193b0 con 0x7efebc159c60 2023-10-06T03:03:06.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.776+0000 7efeb17fa700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efea0066f30 0x7efea00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:06.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.776+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efeb4084720 con 0x7efebc159c60 2023-10-06T03:03:06.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.776+0000 7efec1332700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efea0066f30 0x7efea00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:06.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.776+0000 7efec1332700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efea0066f30 0x7efea00693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009880 tx=0x7efeb8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:06.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.780+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efebc0ce920 con 0x7efebc159c60 2023-10-06T03:03:06.862 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:03:06.860+0000 7fa852b77700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:03:06.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.958+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efebc0ce8e0 con 0x7efea0066f30 2023-10-06T03:03:06.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.966+0000 7efeb17fa700 1 -- 172.21.15.92:0/3451810995 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311670 (secure 0 0 0) 0x7efebc0ce8e0 con 0x7efea0066f30 2023-10-06T03:03:06.972 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:06.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efea0066f30 msgr2=0x7efea00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efea0066f30 0x7efea00693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009880 tx=0x7efeb8017000 comp rx=0 tx=0).stop 2023-10-06T03:03:06.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 msgr2=0x7efebc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:06.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 0x7efebc0b4dd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c880 tx=0x7efeb4008040 comp rx=0 tx=0).stop 2023-10-06T03:03:06.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 shutdown_connections 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efea0066f30 0x7efea00693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efebc159c60 0x7efebc0b4dd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efebc069560 0x7efebc0b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 --2- 172.21.15.92:0/3451810995 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efebc159890 0x7efebc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 >> 172.21.15.92:0/3451810995 conn(0x7efebc068e20 msgr2=0x7efebc0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 shutdown_connections 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:06.970+0000 7efec3596700 1 -- 172.21.15.92:0/3451810995 wait complete. 2023-10-06T03:03:06.975 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:06.983 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-10-06T03:03:05.199117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2384524,"kb_used_data":6220,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752590196,"statfs":{"total":773094113280,"available":770652360704,"internally_reserved":0,"allocated":6369280,"data_stored":3459759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.647253"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488602+0000","last_change":"2023-10-06T03:03:02.486434+0000","last_active":"2023-10-06T03:03:02.488602+0000","last_peered":"2023-10-06T03:03:02.488602+0000","last_clean":"2023-10-06T03:03:02.488602+0000","last_became_active":"2023-10-06T03:03:02.486237+0000","last_became_peered":"2023-10-06T03:03:02.486237+0000","last_unstale":"2023-10-06T03:03:02.488602+0000","last_undegraded":"2023-10-06T03:03:02.488602+0000","last_fullsized":"2023-10-06T03:03:02.488602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:03.646525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493317+0000","last_change":"2023-10-06T03:03:02.488901+0000","last_active":"2023-10-06T03:03:02.493317+0000","last_peered":"2023-10-06T03:03:02.493317+0000","last_clean":"2023-10-06T03:03:02.493317+0000","last_became_active":"2023-10-06T03:03:02.488603+0000","last_became_peered":"2023-10-06T03:03:02.488603+0000","last_unstale":"2023-10-06T03:03:02.493317+0000","last_undegraded":"2023-10-06T03:03:02.493317+0000","last_fullsized":"2023-10-06T03:03:02.493317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:52.093262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489952+0000","last_change":"2023-10-06T03:03:02.489952+0000","last_active":"2023-10-06T03:03:02.489952+0000","last_peered":"2023-10-06T03:03:02.489952+0000","last_clean":"2023-10-06T03:03:02.489952+0000","last_became_active":"2023-10-06T03:03:02.489744+0000","last_became_peered":"2023-10-06T03:03:02.489744+0000","last_unstale":"2023-10-06T03:03:02.489952+0000","last_undegraded":"2023-10-06T03:03:02.489952+0000","last_fullsized":"2023-10-06T03:03:02.489952+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:56.929076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493126+0000","last_change":"2023-10-06T03:03:02.489884+0000","last_active":"2023-10-06T03:03:02.493126+0000","last_peered":"2023-10-06T03:03:02.493126+0000","last_clean":"2023-10-06T03:03:02.493126+0000","last_became_active":"2023-10-06T03:03:02.489161+0000","last_became_peered":"2023-10-06T03:03:02.489161+0000","last_unstale":"2023-10-06T03:03:02.493126+0000","last_undegraded":"2023-10-06T03:03:02.493126+0000","last_fullsized":"2023-10-06T03:03:02.493126+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.494997+0000","last_change":"2023-10-06T03:03:02.494997+0000","last_active":"2023-10-06T03:03:02.494997+0000","last_peered":"2023-10-06T03:03:02.494997+0000","last_clean":"2023-10-06T03:03:02.494997+0000","last_became_active":"2023-10-06T03:03:02.494733+0000","last_became_peered":"2023-10-06T03:03:02.494733+0000","last_unstale":"2023-10-06T03:03:02.494997+0000","last_undegraded":"2023-10-06T03:03:02.494997+0000","last_fullsized":"2023-10-06T03:03:02.494997+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:03.016992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.487430+0000","last_change":"2023-10-06T03:03:02.487430+0000","last_active":"2023-10-06T03:03:02.487430+0000","last_peered":"2023-10-06T03:03:02.487430+0000","last_clean":"2023-10-06T03:03:02.487430+0000","last_became_active":"2023-10-06T03:03:02.487162+0000","last_became_peered":"2023-10-06T03:03:02.487162+0000","last_unstale":"2023-10-06T03:03:02.487430+0000","last_undegraded":"2023-10-06T03:03:02.487430+0000","last_fullsized":"2023-10-06T03:03:02.487430+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.483351+0000","last_change":"2023-10-06T03:03:02.483351+0000","last_active":"2023-10-06T03:03:02.483351+0000","last_peered":"2023-10-06T03:03:02.483351+0000","last_clean":"2023-10-06T03:03:02.483351+0000","last_became_active":"2023-10-06T03:03:02.483119+0000","last_became_peered":"2023-10-06T03:03:02.483119+0000","last_unstale":"2023-10-06T03:03:02.483351+0000","last_undegraded":"2023-10-06T03:03:02.483351+0000","last_fullsized":"2023-10-06T03:03:02.483351+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:47.758547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489241+0000","last_change":"2023-10-06T03:03:02.483123+0000","last_active":"2023-10-06T03:03:02.489241+0000","last_peered":"2023-10-06T03:03:02.489241+0000","last_clean":"2023-10-06T03:03:02.489241+0000","last_became_active":"2023-10-06T03:03:02.482849+0000","last_became_peered":"2023-10-06T03:03:02.482849+0000","last_unstale":"2023-10-06T03:03:02.489241+0000","last_undegraded":"2023-10-06T03:03:02.489241+0000","last_fullsized":"2023-10-06T03:03:02.489241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493975+0000","last_change":"2023-10-06T03:03:02.493975+0000","last_active":"2023-10-06T03:03:02.493975+0000","last_peered":"2023-10-06T03:03:02.493975+0000","last_clean":"2023-10-06T03:03:02.493975+0000","last_became_active":"2023-10-06T03:03:02.493731+0000","last_became_peered":"2023-10-06T03:03:02.493731+0000","last_unstale":"2023-10-06T03:03:02.493975+0000","last_undegraded":"2023-10-06T03:03:02.493975+0000","last_fullsized":"2023-10-06T03:03:02.493975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:12.848469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489735+0000","last_change":"2023-10-06T03:03:02.489735+0000","last_active":"2023-10-06T03:03:02.489735+0000","last_peered":"2023-10-06T03:03:02.489735+0000","last_clean":"2023-10-06T03:03:02.489735+0000","last_became_active":"2023-10-06T03:03:02.489497+0000","last_became_peered":"2023-10-06T03:03:02.489497+0000","last_unstale":"2023-10-06T03:03:02.489735+0000","last_undegraded":"2023-10-06T03:03:02.489735+0000","last_fullsized":"2023-10-06T03:03:02.489735+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:24.544569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491784+0000","last_change":"2023-10-06T03:03:02.482440+0000","last_active":"2023-10-06T03:03:02.491784+0000","last_peered":"2023-10-06T03:03:02.491784+0000","last_clean":"2023-10-06T03:03:02.491784+0000","last_became_active":"2023-10-06T03:03:02.482164+0000","last_became_peered":"2023-10-06T03:03:02.482164+0000","last_unstale":"2023-10-06T03:03:02.491784+0000","last_undegraded":"2023-10-06T03:03:02.491784+0000","last_fullsized":"2023-10-06T03:03:02.491784+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489098+0000","last_change":"2023-10-06T03:03:02.483350+0000","last_active":"2023-10-06T03:03:02.489098+0000","last_peered":"2023-10-06T03:03:02.489098+0000","last_clean":"2023-10-06T03:03:02.489098+0000","last_became_active":"2023-10-06T03:03:02.483097+0000","last_became_peered":"2023-10-06T03:03:02.483097+0000","last_unstale":"2023-10-06T03:03:02.489098+0000","last_undegraded":"2023-10-06T03:03:02.489098+0000","last_fullsized":"2023-10-06T03:03:02.489098+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.485334+0000","last_change":"2023-10-06T03:03:02.481663+0000","last_active":"2023-10-06T03:03:02.485334+0000","last_peered":"2023-10-06T03:03:02.485334+0000","last_clean":"2023-10-06T03:03:02.485334+0000","last_became_active":"2023-10-06T03:03:02.481384+0000","last_became_peered":"2023-10-06T03:03:02.481384+0000","last_unstale":"2023-10-06T03:03:02.485334+0000","last_undegraded":"2023-10-06T03:03:02.485334+0000","last_fullsized":"2023-10-06T03:03:02.485334+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:12.848469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490013+0000","last_change":"2023-10-06T03:03:02.490013+0000","last_active":"2023-10-06T03:03:02.490013+0000","last_peered":"2023-10-06T03:03:02.490013+0000","last_clean":"2023-10-06T03:03:02.490013+0000","last_became_active":"2023-10-06T03:03:02.489762+0000","last_became_peered":"2023-10-06T03:03:02.489762+0000","last_unstale":"2023-10-06T03:03:02.490013+0000","last_undegraded":"2023-10-06T03:03:02.490013+0000","last_fullsized":"2023-10-06T03:03:02.490013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:52.154816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491288+0000","last_change":"2023-10-06T03:03:02.491288+0000","last_active":"2023-10-06T03:03:02.491288+0000","last_peered":"2023-10-06T03:03:02.491288+0000","last_clean":"2023-10-06T03:03:02.491288+0000","last_became_active":"2023-10-06T03:03:02.491043+0000","last_became_peered":"2023-10-06T03:03:02.491043+0000","last_unstale":"2023-10-06T03:03:02.491288+0000","last_undegraded":"2023-10-06T03:03:02.491288+0000","last_fullsized":"2023-10-06T03:03:02.491288+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:52.366760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.485254+0000","last_change":"2023-10-06T03:03:02.485254+0000","last_active":"2023-10-06T03:03:02.485254+0000","last_peered":"2023-10-06T03:03:02.485254+0000","last_clean":"2023-10-06T03:03:02.485254+0000","last_became_active":"2023-10-06T03:03:02.484988+0000","last_became_peered":"2023-10-06T03:03:02.484988+0000","last_unstale":"2023-10-06T03:03:02.485254+0000","last_undegraded":"2023-10-06T03:03:02.485254+0000","last_fullsized":"2023-10-06T03:03:02.485254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:56.243568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488579+0000","last_change":"2023-10-06T03:03:02.482533+0000","last_active":"2023-10-06T03:03:02.488579+0000","last_peered":"2023-10-06T03:03:02.488579+0000","last_clean":"2023-10-06T03:03:02.488579+0000","last_became_active":"2023-10-06T03:03:02.482275+0000","last_became_peered":"2023-10-06T03:03:02.482275+0000","last_unstale":"2023-10-06T03:03:02.488579+0000","last_undegraded":"2023-10-06T03:03:02.488579+0000","last_fullsized":"2023-10-06T03:03:02.488579+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:33.015231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493650+0000","last_change":"2023-10-06T03:03:02.489682+0000","last_active":"2023-10-06T03:03:02.493650+0000","last_peered":"2023-10-06T03:03:02.493650+0000","last_clean":"2023-10-06T03:03:02.493650+0000","last_became_active":"2023-10-06T03:03:02.489248+0000","last_became_peered":"2023-10-06T03:03:02.489248+0000","last_unstale":"2023-10-06T03:03:02.493650+0000","last_undegraded":"2023-10-06T03:03:02.493650+0000","last_fullsized":"2023-10-06T03:03:02.493650+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490604+0000","last_change":"2023-10-06T03:03:02.490604+0000","last_active":"2023-10-06T03:03:02.490604+0000","last_peered":"2023-10-06T03:03:02.490604+0000","last_clean":"2023-10-06T03:03:02.490604+0000","last_became_active":"2023-10-06T03:03:02.490212+0000","last_became_peered":"2023-10-06T03:03:02.490212+0000","last_unstale":"2023-10-06T03:03:02.490604+0000","last_undegraded":"2023-10-06T03:03:02.490604+0000","last_fullsized":"2023-10-06T03:03:02.490604+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:26.513756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493437+0000","last_change":"2023-10-06T03:03:02.493437+0000","last_active":"2023-10-06T03:03:02.493437+0000","last_peered":"2023-10-06T03:03:02.493437+0000","last_clean":"2023-10-06T03:03:02.493437+0000","last_became_active":"2023-10-06T03:03:02.493110+0000","last_became_peered":"2023-10-06T03:03:02.493110+0000","last_unstale":"2023-10-06T03:03:02.493437+0000","last_undegraded":"2023-10-06T03:03:02.493437+0000","last_fullsized":"2023-10-06T03:03:02.493437+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:00.390049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.494228+0000","last_change":"2023-10-06T03:03:02.488883+0000","last_active":"2023-10-06T03:03:02.494228+0000","last_peered":"2023-10-06T03:03:02.494228+0000","last_clean":"2023-10-06T03:03:02.494228+0000","last_became_active":"2023-10-06T03:03:02.488621+0000","last_became_peered":"2023-10-06T03:03:02.488621+0000","last_unstale":"2023-10-06T03:03:02.494228+0000","last_undegraded":"2023-10-06T03:03:02.494228+0000","last_fullsized":"2023-10-06T03:03:02.494228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:42:59.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.484406+0000","last_change":"2023-10-06T03:03:02.484406+0000","last_active":"2023-10-06T03:03:02.484406+0000","last_peered":"2023-10-06T03:03:02.484406+0000","last_clean":"2023-10-06T03:03:02.484406+0000","last_became_active":"2023-10-06T03:03:02.484143+0000","last_became_peered":"2023-10-06T03:03:02.484143+0000","last_unstale":"2023-10-06T03:03:02.484406+0000","last_undegraded":"2023-10-06T03:03:02.484406+0000","last_fullsized":"2023-10-06T03:03:02.484406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:42:59.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.484150+0000","last_change":"2023-10-06T03:03:02.484150+0000","last_active":"2023-10-06T03:03:02.484150+0000","last_peered":"2023-10-06T03:03:02.484150+0000","last_clean":"2023-10-06T03:03:02.484150+0000","last_became_active":"2023-10-06T03:03:02.483672+0000","last_became_peered":"2023-10-06T03:03:02.483672+0000","last_unstale":"2023-10-06T03:03:02.484150+0000","last_undegraded":"2023-10-06T03:03:02.484150+0000","last_fullsized":"2023-10-06T03:03:02.484150+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:39.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490670+0000","last_change":"2023-10-06T03:03:02.490670+0000","last_active":"2023-10-06T03:03:02.490670+0000","last_peered":"2023-10-06T03:03:02.490670+0000","last_clean":"2023-10-06T03:03:02.490670+0000","last_became_active":"2023-10-06T03:03:02.490454+0000","last_became_peered":"2023-10-06T03:03:02.490454+0000","last_unstale":"2023-10-06T03:03:02.490670+0000","last_undegraded":"2023-10-06T03:03:02.490670+0000","last_fullsized":"2023-10-06T03:03:02.490670+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:30.255226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491342+0000","last_change":"2023-10-06T03:03:02.485795+0000","last_active":"2023-10-06T03:03:02.491342+0000","last_peered":"2023-10-06T03:03:02.491342+0000","last_clean":"2023-10-06T03:03:02.491342+0000","last_became_active":"2023-10-06T03:03:02.485544+0000","last_became_peered":"2023-10-06T03:03:02.485544+0000","last_unstale":"2023-10-06T03:03:02.491342+0000","last_undegraded":"2023-10-06T03:03:02.491342+0000","last_fullsized":"2023-10-06T03:03:02.491342+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:17.907864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489196+0000","last_change":"2023-10-06T03:03:02.483874+0000","last_active":"2023-10-06T03:03:02.489196+0000","last_peered":"2023-10-06T03:03:02.489196+0000","last_clean":"2023-10-06T03:03:02.489196+0000","last_became_active":"2023-10-06T03:03:02.483594+0000","last_became_peered":"2023-10-06T03:03:02.483594+0000","last_unstale":"2023-10-06T03:03:02.489196+0000","last_undegraded":"2023-10-06T03:03:02.489196+0000","last_fullsized":"2023-10-06T03:03:02.489196+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:00.390049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491763+0000","last_change":"2023-10-06T03:03:02.481629+0000","last_active":"2023-10-06T03:03:02.491763+0000","last_peered":"2023-10-06T03:03:02.491763+0000","last_clean":"2023-10-06T03:03:02.491763+0000","last_became_active":"2023-10-06T03:03:02.481336+0000","last_became_peered":"2023-10-06T03:03:02.481336+0000","last_unstale":"2023-10-06T03:03:02.491763+0000","last_undegraded":"2023-10-06T03:03:02.491763+0000","last_fullsized":"2023-10-06T03:03:02.491763+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:52.366760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.498674+0000","last_change":"2023-10-06T03:03:02.494100+0000","last_active":"2023-10-06T03:03:02.498674+0000","last_peered":"2023-10-06T03:03:02.498674+0000","last_clean":"2023-10-06T03:03:02.498674+0000","last_became_active":"2023-10-06T03:03:02.493834+0000","last_became_peered":"2023-10-06T03:03:02.493834+0000","last_unstale":"2023-10-06T03:03:02.498674+0000","last_undegraded":"2023-10-06T03:03:02.498674+0000","last_fullsized":"2023-10-06T03:03:02.498674+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.335097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493756+0000","last_change":"2023-10-06T03:03:02.493756+0000","last_active":"2023-10-06T03:03:02.493756+0000","last_peered":"2023-10-06T03:03:02.493756+0000","last_clean":"2023-10-06T03:03:02.493756+0000","last_became_active":"2023-10-06T03:03:02.493470+0000","last_became_peered":"2023-10-06T03:03:02.493470+0000","last_unstale":"2023-10-06T03:03:02.493756+0000","last_undegraded":"2023-10-06T03:03:02.493756+0000","last_fullsized":"2023-10-06T03:03:02.493756+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491126+0000","last_change":"2023-10-06T03:03:02.485135+0000","last_active":"2023-10-06T03:03:02.491126+0000","last_peered":"2023-10-06T03:03:02.491126+0000","last_clean":"2023-10-06T03:03:02.491126+0000","last_became_active":"2023-10-06T03:03:02.484889+0000","last_became_peered":"2023-10-06T03:03:02.484889+0000","last_unstale":"2023-10-06T03:03:02.491126+0000","last_undegraded":"2023-10-06T03:03:02.491126+0000","last_fullsized":"2023-10-06T03:03:02.491126+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:12.975116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490343+0000","last_change":"2023-10-06T03:03:02.481522+0000","last_active":"2023-10-06T03:03:02.490343+0000","last_peered":"2023-10-06T03:03:02.490343+0000","last_clean":"2023-10-06T03:03:02.490343+0000","last_became_active":"2023-10-06T03:03:02.481241+0000","last_became_peered":"2023-10-06T03:03:02.481241+0000","last_unstale":"2023-10-06T03:03:02.490343+0000","last_undegraded":"2023-10-06T03:03:02.490343+0000","last_fullsized":"2023-10-06T03:03:02.490343+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:24.544569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.483156+0000","last_change":"2023-10-06T03:03:02.483156+0000","last_active":"2023-10-06T03:03:02.483156+0000","last_peered":"2023-10-06T03:03:02.483156+0000","last_clean":"2023-10-06T03:03:02.483156+0000","last_became_active":"2023-10-06T03:03:02.482272+0000","last_became_peered":"2023-10-06T03:03:02.482272+0000","last_unstale":"2023-10-06T03:03:02.483156+0000","last_undegraded":"2023-10-06T03:03:02.483156+0000","last_fullsized":"2023-10-06T03:03:02.483156+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493071+0000","last_change":"2023-10-06T03:03:02.493071+0000","last_active":"2023-10-06T03:03:02.493071+0000","last_peered":"2023-10-06T03:03:02.493071+0000","last_clean":"2023-10-06T03:03:02.493071+0000","last_became_active":"2023-10-06T03:03:02.492045+0000","last_became_peered":"2023-10-06T03:03:02.492045+0000","last_unstale":"2023-10-06T03:03:02.493071+0000","last_undegraded":"2023-10-06T03:03:02.493071+0000","last_fullsized":"2023-10-06T03:03:02.493071+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:36.131250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"53'4","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.569258+0000","last_change":"2023-10-06T03:03:01.575644+0000","last_active":"2023-10-06T03:03:02.569258+0000","last_peered":"2023-10-06T03:03:02.569258+0000","last_clean":"2023-10-06T03:03:02.569258+0000","last_became_active":"2023-10-06T03:03:01.575394+0000","last_became_peered":"2023-10-06T03:03:01.575394+0000","last_unstale":"2023-10-06T03:03:02.569258+0000","last_undegraded":"2023-10-06T03:03:02.569258+0000","last_fullsized":"2023-10-06T03:03:02.569258+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:19.351394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.497004+0000","last_change":"2023-10-06T03:03:02.492893+0000","last_active":"2023-10-06T03:03:02.497004+0000","last_peered":"2023-10-06T03:03:02.497004+0000","last_clean":"2023-10-06T03:03:02.497004+0000","last_became_active":"2023-10-06T03:03:02.492613+0000","last_became_peered":"2023-10-06T03:03:02.492613+0000","last_unstale":"2023-10-06T03:03:02.497004+0000","last_undegraded":"2023-10-06T03:03:02.497004+0000","last_fullsized":"2023-10-06T03:03:02.497004+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:17.907864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.480574+0000","last_change":"2023-10-06T03:03:01.575648+0000","last_active":"2023-10-06T03:03:02.480574+0000","last_peered":"2023-10-06T03:03:02.480574+0000","last_clean":"2023-10-06T03:03:02.480574+0000","last_became_active":"2023-10-06T03:03:01.575391+0000","last_became_peered":"2023-10-06T03:03:01.575391+0000","last_unstale":"2023-10-06T03:03:02.480574+0000","last_undegraded":"2023-10-06T03:03:02.480574+0000","last_fullsized":"2023-10-06T03:03:02.480574+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:51.086706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488062+0000","last_change":"2023-10-06T03:03:02.488062+0000","last_active":"2023-10-06T03:03:02.488062+0000","last_peered":"2023-10-06T03:03:02.488062+0000","last_clean":"2023-10-06T03:03:02.488062+0000","last_became_active":"2023-10-06T03:03:02.487742+0000","last_became_peered":"2023-10-06T03:03:02.487742+0000","last_unstale":"2023-10-06T03:03:02.488062+0000","last_undegraded":"2023-10-06T03:03:02.488062+0000","last_fullsized":"2023-10-06T03:03:02.488062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:52.366760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566856+0000","last_change":"2023-10-06T03:03:01.573058+0000","last_active":"2023-10-06T03:03:02.566856+0000","last_peered":"2023-10-06T03:03:02.566856+0000","last_clean":"2023-10-06T03:03:02.566856+0000","last_became_active":"2023-10-06T03:03:01.572850+0000","last_became_peered":"2023-10-06T03:03:01.572850+0000","last_unstale":"2023-10-06T03:03:02.566856+0000","last_undegraded":"2023-10-06T03:03:02.566856+0000","last_fullsized":"2023-10-06T03:03:02.566856+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:32.880340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490066+0000","last_change":"2023-10-06T03:03:02.490066+0000","last_active":"2023-10-06T03:03:02.490066+0000","last_peered":"2023-10-06T03:03:02.490066+0000","last_clean":"2023-10-06T03:03:02.490066+0000","last_became_active":"2023-10-06T03:03:02.489470+0000","last_became_peered":"2023-10-06T03:03:02.489470+0000","last_unstale":"2023-10-06T03:03:02.490066+0000","last_undegraded":"2023-10-06T03:03:02.490066+0000","last_fullsized":"2023-10-06T03:03:02.490066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:15.621480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566154+0000","last_change":"2023-10-06T03:03:01.571202+0000","last_active":"2023-10-06T03:03:02.566154+0000","last_peered":"2023-10-06T03:03:02.566154+0000","last_clean":"2023-10-06T03:03:02.566154+0000","last_became_active":"2023-10-06T03:03:01.570929+0000","last_became_peered":"2023-10-06T03:03:01.570929+0000","last_unstale":"2023-10-06T03:03:02.566154+0000","last_undegraded":"2023-10-06T03:03:02.566154+0000","last_fullsized":"2023-10-06T03:03:02.566154+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:17.749222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481285+0000","last_change":"2023-10-06T03:03:02.481285+0000","last_active":"2023-10-06T03:03:02.481285+0000","last_peered":"2023-10-06T03:03:02.481285+0000","last_clean":"2023-10-06T03:03:02.481285+0000","last_became_active":"2023-10-06T03:03:02.480994+0000","last_became_peered":"2023-10-06T03:03:02.480994+0000","last_unstale":"2023-10-06T03:03:02.481285+0000","last_undegraded":"2023-10-06T03:03:02.481285+0000","last_fullsized":"2023-10-06T03:03:02.481285+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:19:12.223774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.477999+0000","last_change":"2023-10-06T03:03:01.571719+0000","last_active":"2023-10-06T03:03:02.477999+0000","last_peered":"2023-10-06T03:03:02.477999+0000","last_clean":"2023-10-06T03:03:02.477999+0000","last_became_active":"2023-10-06T03:03:01.571502+0000","last_became_peered":"2023-10-06T03:03:01.571502+0000","last_unstale":"2023-10-06T03:03:02.477999+0000","last_undegraded":"2023-10-06T03:03:02.477999+0000","last_fullsized":"2023-10-06T03:03:02.477999+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:23.538013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488687+0000","last_change":"2023-10-06T03:03:02.483457+0000","last_active":"2023-10-06T03:03:02.488687+0000","last_peered":"2023-10-06T03:03:02.488687+0000","last_clean":"2023-10-06T03:03:02.488687+0000","last_became_active":"2023-10-06T03:03:02.483120+0000","last_became_peered":"2023-10-06T03:03:02.483120+0000","last_unstale":"2023-10-06T03:03:02.488687+0000","last_undegraded":"2023-10-06T03:03:02.488687+0000","last_fullsized":"2023-10-06T03:03:02.488687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:05.336698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.478088+0000","last_change":"2023-10-06T03:03:01.570124+0000","last_active":"2023-10-06T03:03:02.478088+0000","last_peered":"2023-10-06T03:03:02.478088+0000","last_clean":"2023-10-06T03:03:02.478088+0000","last_became_active":"2023-10-06T03:03:01.569860+0000","last_became_peered":"2023-10-06T03:03:01.569860+0000","last_unstale":"2023-10-06T03:03:02.478088+0000","last_undegraded":"2023-10-06T03:03:02.478088+0000","last_fullsized":"2023-10-06T03:03:02.478088+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:29.248670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493543+0000","last_change":"2023-10-06T03:03:02.493543+0000","last_active":"2023-10-06T03:03:02.493543+0000","last_peered":"2023-10-06T03:03:02.493543+0000","last_clean":"2023-10-06T03:03:02.493543+0000","last_became_active":"2023-10-06T03:03:02.493116+0000","last_became_peered":"2023-10-06T03:03:02.493116+0000","last_unstale":"2023-10-06T03:03:02.493543+0000","last_undegraded":"2023-10-06T03:03:02.493543+0000","last_fullsized":"2023-10-06T03:03:02.493543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:46.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577884+0000","last_change":"2023-10-06T03:03:01.573857+0000","last_active":"2023-10-06T03:03:01.577884+0000","last_peered":"2023-10-06T03:03:01.577884+0000","last_clean":"2023-10-06T03:03:01.577884+0000","last_became_active":"2023-10-06T03:03:01.573460+0000","last_became_peered":"2023-10-06T03:03:01.573460+0000","last_unstale":"2023-10-06T03:03:01.577884+0000","last_undegraded":"2023-10-06T03:03:01.577884+0000","last_fullsized":"2023-10-06T03:03:01.577884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:51.148260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493894+0000","last_change":"2023-10-06T03:03:02.493894+0000","last_active":"2023-10-06T03:03:02.493894+0000","last_peered":"2023-10-06T03:03:02.493894+0000","last_clean":"2023-10-06T03:03:02.493894+0000","last_became_active":"2023-10-06T03:03:02.493623+0000","last_became_peered":"2023-10-06T03:03:02.493623+0000","last_unstale":"2023-10-06T03:03:02.493894+0000","last_undegraded":"2023-10-06T03:03:02.493894+0000","last_fullsized":"2023-10-06T03:03:02.493894+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:08.428638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.579202+0000","last_change":"2023-10-06T03:03:01.575666+0000","last_active":"2023-10-06T03:03:01.579202+0000","last_peered":"2023-10-06T03:03:01.579202+0000","last_clean":"2023-10-06T03:03:01.579202+0000","last_became_active":"2023-10-06T03:03:01.575409+0000","last_became_peered":"2023-10-06T03:03:01.575409+0000","last_unstale":"2023-10-06T03:03:01.579202+0000","last_undegraded":"2023-10-06T03:03:01.579202+0000","last_fullsized":"2023-10-06T03:03:01.579202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:04.862470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488348+0000","last_change":"2023-10-06T03:03:02.481938+0000","last_active":"2023-10-06T03:03:02.488348+0000","last_peered":"2023-10-06T03:03:02.488348+0000","last_clean":"2023-10-06T03:03:02.488348+0000","last_became_active":"2023-10-06T03:03:02.481651+0000","last_became_peered":"2023-10-06T03:03:02.481651+0000","last_unstale":"2023-10-06T03:03:02.488348+0000","last_undegraded":"2023-10-06T03:03:02.488348+0000","last_fullsized":"2023-10-06T03:03:02.488348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:46.805863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.581688+0000","last_change":"2023-10-06T03:03:01.578081+0000","last_active":"2023-10-06T03:03:01.581688+0000","last_peered":"2023-10-06T03:03:01.581688+0000","last_clean":"2023-10-06T03:03:01.581688+0000","last_became_active":"2023-10-06T03:03:01.577868+0000","last_became_peered":"2023-10-06T03:03:01.577868+0000","last_unstale":"2023-10-06T03:03:01.581688+0000","last_undegraded":"2023-10-06T03:03:01.581688+0000","last_fullsized":"2023-10-06T03:03:01.581688+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:02.010436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.484712+0000","last_change":"2023-10-06T03:03:02.484712+0000","last_active":"2023-10-06T03:03:02.484712+0000","last_peered":"2023-10-06T03:03:02.484712+0000","last_clean":"2023-10-06T03:03:02.484712+0000","last_became_active":"2023-10-06T03:03:02.484395+0000","last_became_peered":"2023-10-06T03:03:02.484395+0000","last_unstale":"2023-10-06T03:03:02.484712+0000","last_undegraded":"2023-10-06T03:03:02.484712+0000","last_fullsized":"2023-10-06T03:03:02.484712+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.846528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.476264+0000","last_change":"2023-10-06T03:03:01.565506+0000","last_active":"2023-10-06T03:03:02.476264+0000","last_peered":"2023-10-06T03:03:02.476264+0000","last_clean":"2023-10-06T03:03:02.476264+0000","last_became_active":"2023-10-06T03:03:01.565190+0000","last_became_peered":"2023-10-06T03:03:01.565190+0000","last_unstale":"2023-10-06T03:03:02.476264+0000","last_undegraded":"2023-10-06T03:03:02.476264+0000","last_fullsized":"2023-10-06T03:03:02.476264+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.008675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493349+0000","last_change":"2023-10-06T03:03:02.493349+0000","last_active":"2023-10-06T03:03:02.493349+0000","last_peered":"2023-10-06T03:03:02.493349+0000","last_clean":"2023-10-06T03:03:02.493349+0000","last_became_active":"2023-10-06T03:03:02.493075+0000","last_became_peered":"2023-10-06T03:03:02.493075+0000","last_unstale":"2023-10-06T03:03:02.493349+0000","last_undegraded":"2023-10-06T03:03:02.493349+0000","last_fullsized":"2023-10-06T03:03:02.493349+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:28.188295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"53'3","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566145+0000","last_change":"2023-10-06T03:03:01.578277+0000","last_active":"2023-10-06T03:03:02.566145+0000","last_peered":"2023-10-06T03:03:02.566145+0000","last_clean":"2023-10-06T03:03:02.566145+0000","last_became_active":"2023-10-06T03:03:01.578022+0000","last_became_peered":"2023-10-06T03:03:01.578022+0000","last_unstale":"2023-10-06T03:03:02.566145+0000","last_undegraded":"2023-10-06T03:03:02.566145+0000","last_fullsized":"2023-10-06T03:03:02.566145+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:16.514444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492976+0000","last_change":"2023-10-06T03:03:02.487161+0000","last_active":"2023-10-06T03:03:02.492976+0000","last_peered":"2023-10-06T03:03:02.492976+0000","last_clean":"2023-10-06T03:03:02.492976+0000","last_became_active":"2023-10-06T03:03:02.486864+0000","last_became_peered":"2023-10-06T03:03:02.486864+0000","last_unstale":"2023-10-06T03:03:02.492976+0000","last_undegraded":"2023-10-06T03:03:02.492976+0000","last_fullsized":"2023-10-06T03:03:02.492976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:33.650190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"53'6","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.569596+0000","last_change":"2023-10-06T03:03:01.566835+0000","last_active":"2023-10-06T03:03:02.569596+0000","last_peered":"2023-10-06T03:03:02.569596+0000","last_clean":"2023-10-06T03:03:02.569596+0000","last_became_active":"2023-10-06T03:03:01.566351+0000","last_became_peered":"2023-10-06T03:03:01.566351+0000","last_unstale":"2023-10-06T03:03:02.569596+0000","last_undegraded":"2023-10-06T03:03:02.569596+0000","last_fullsized":"2023-10-06T03:03:02.569596+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:24.009268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492953+0000","last_change":"2023-10-06T03:03:02.488237+0000","last_active":"2023-10-06T03:03:02.492953+0000","last_peered":"2023-10-06T03:03:02.492953+0000","last_clean":"2023-10-06T03:03:02.492953+0000","last_became_active":"2023-10-06T03:03:02.487979+0000","last_became_peered":"2023-10-06T03:03:02.487979+0000","last_unstale":"2023-10-06T03:03:02.492953+0000","last_undegraded":"2023-10-06T03:03:02.492953+0000","last_fullsized":"2023-10-06T03:03:02.492953+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:12.848469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.568912+0000","last_change":"2023-10-06T03:03:01.577347+0000","last_active":"2023-10-06T03:03:02.568912+0000","last_peered":"2023-10-06T03:03:02.568912+0000","last_clean":"2023-10-06T03:03:02.568912+0000","last_became_active":"2023-10-06T03:03:01.577073+0000","last_became_peered":"2023-10-06T03:03:01.577073+0000","last_unstale":"2023-10-06T03:03:02.568912+0000","last_undegraded":"2023-10-06T03:03:02.568912+0000","last_fullsized":"2023-10-06T03:03:02.568912+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:11.208832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491956+0000","last_change":"2023-10-06T03:03:02.488394+0000","last_active":"2023-10-06T03:03:02.491956+0000","last_peered":"2023-10-06T03:03:02.491956+0000","last_clean":"2023-10-06T03:03:02.491956+0000","last_became_active":"2023-10-06T03:03:02.488139+0000","last_became_peered":"2023-10-06T03:03:02.488139+0000","last_unstale":"2023-10-06T03:03:02.491956+0000","last_undegraded":"2023-10-06T03:03:02.491956+0000","last_fullsized":"2023-10-06T03:03:02.491956+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:39.743590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.478929+0000","last_change":"2023-10-06T03:03:01.571632+0000","last_active":"2023-10-06T03:03:02.478929+0000","last_peered":"2023-10-06T03:03:02.478929+0000","last_clean":"2023-10-06T03:03:02.478929+0000","last_became_active":"2023-10-06T03:03:01.571226+0000","last_became_peered":"2023-10-06T03:03:01.571226+0000","last_unstale":"2023-10-06T03:03:02.478929+0000","last_undegraded":"2023-10-06T03:03:02.478929+0000","last_fullsized":"2023-10-06T03:03:02.478929+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:36.328541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492135+0000","last_change":"2023-10-06T03:03:02.492135+0000","last_active":"2023-10-06T03:03:02.492135+0000","last_peered":"2023-10-06T03:03:02.492135+0000","last_clean":"2023-10-06T03:03:02.492135+0000","last_became_active":"2023-10-06T03:03:02.491581+0000","last_became_peered":"2023-10-06T03:03:02.491581+0000","last_unstale":"2023-10-06T03:03:02.492135+0000","last_undegraded":"2023-10-06T03:03:02.492135+0000","last_fullsized":"2023-10-06T03:03:02.492135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:06.481908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.581761+0000","last_change":"2023-10-06T03:03:01.578534+0000","last_active":"2023-10-06T03:03:01.581761+0000","last_peered":"2023-10-06T03:03:01.581761+0000","last_clean":"2023-10-06T03:03:01.581761+0000","last_became_active":"2023-10-06T03:03:01.578115+0000","last_became_peered":"2023-10-06T03:03:01.578115+0000","last_unstale":"2023-10-06T03:03:01.581761+0000","last_undegraded":"2023-10-06T03:03:01.581761+0000","last_fullsized":"2023-10-06T03:03:01.581761+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:45.799307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490025+0000","last_change":"2023-10-06T03:03:02.486595+0000","last_active":"2023-10-06T03:03:02.490025+0000","last_peered":"2023-10-06T03:03:02.490025+0000","last_clean":"2023-10-06T03:03:02.490025+0000","last_became_active":"2023-10-06T03:03:02.486354+0000","last_became_peered":"2023-10-06T03:03:02.486354+0000","last_unstale":"2023-10-06T03:03:02.490025+0000","last_undegraded":"2023-10-06T03:03:02.490025+0000","last_fullsized":"2023-10-06T03:03:02.490025+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:17.521000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"53'8","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.569693+0000","last_change":"2023-10-06T03:03:01.570862+0000","last_active":"2023-10-06T03:03:02.569693+0000","last_peered":"2023-10-06T03:03:02.569693+0000","last_clean":"2023-10-06T03:03:02.569693+0000","last_became_active":"2023-10-06T03:03:01.570568+0000","last_became_peered":"2023-10-06T03:03:01.570568+0000","last_unstale":"2023-10-06T03:03:02.569693+0000","last_undegraded":"2023-10-06T03:03:02.569693+0000","last_fullsized":"2023-10-06T03:03:02.569693+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:11.208832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.483470+0000","last_change":"2023-10-06T03:03:02.483470+0000","last_active":"2023-10-06T03:03:02.483470+0000","last_peered":"2023-10-06T03:03:02.483470+0000","last_clean":"2023-10-06T03:03:02.483470+0000","last_became_active":"2023-10-06T03:03:02.483212+0000","last_became_peered":"2023-10-06T03:03:02.483212+0000","last_unstale":"2023-10-06T03:03:02.483470+0000","last_undegraded":"2023-10-06T03:03:02.483470+0000","last_fullsized":"2023-10-06T03:03:02.483470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"53'2","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.568872+0000","last_change":"2023-10-06T03:03:01.574899+0000","last_active":"2023-10-06T03:03:02.568872+0000","last_peered":"2023-10-06T03:03:02.568872+0000","last_clean":"2023-10-06T03:03:02.568872+0000","last_became_active":"2023-10-06T03:03:01.574653+0000","last_became_peered":"2023-10-06T03:03:01.574653+0000","last_unstale":"2023-10-06T03:03:02.568872+0000","last_undegraded":"2023-10-06T03:03:02.568872+0000","last_fullsized":"2023-10-06T03:03:02.568872+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.008675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488144+0000","last_change":"2023-10-06T03:03:02.488144+0000","last_active":"2023-10-06T03:03:02.488144+0000","last_peered":"2023-10-06T03:03:02.488144+0000","last_clean":"2023-10-06T03:03:02.488144+0000","last_became_active":"2023-10-06T03:03:02.487900+0000","last_became_peered":"2023-10-06T03:03:02.487900+0000","last_unstale":"2023-10-06T03:03:02.488144+0000","last_undegraded":"2023-10-06T03:03:02.488144+0000","last_fullsized":"2023-10-06T03:03:02.488144+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:52:03.868865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.477916+0000","last_change":"2023-10-06T03:03:01.570067+0000","last_active":"2023-10-06T03:03:02.477916+0000","last_peered":"2023-10-06T03:03:02.477916+0000","last_clean":"2023-10-06T03:03:02.477916+0000","last_became_active":"2023-10-06T03:03:01.569816+0000","last_became_peered":"2023-10-06T03:03:01.569816+0000","last_unstale":"2023-10-06T03:03:02.477916+0000","last_undegraded":"2023-10-06T03:03:02.477916+0000","last_fullsized":"2023-10-06T03:03:02.477916+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:11.208832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.494196+0000","last_change":"2023-10-06T03:03:02.489046+0000","last_active":"2023-10-06T03:03:02.494196+0000","last_peered":"2023-10-06T03:03:02.494196+0000","last_clean":"2023-10-06T03:03:02.494196+0000","last_became_active":"2023-10-06T03:03:02.488825+0000","last_became_peered":"2023-10-06T03:03:02.488825+0000","last_unstale":"2023-10-06T03:03:02.494196+0000","last_undegraded":"2023-10-06T03:03:02.494196+0000","last_fullsized":"2023-10-06T03:03:02.494196+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:08.881932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577847+0000","last_change":"2023-10-06T03:03:01.573133+0000","last_active":"2023-10-06T03:03:01.577847+0000","last_peered":"2023-10-06T03:03:01.577847+0000","last_clean":"2023-10-06T03:03:01.577847+0000","last_became_active":"2023-10-06T03:03:01.572897+0000","last_became_peered":"2023-10-06T03:03:01.572897+0000","last_unstale":"2023-10-06T03:03:01.577847+0000","last_undegraded":"2023-10-06T03:03:01.577847+0000","last_fullsized":"2023-10-06T03:03:01.577847+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:58.537763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491754+0000","last_change":"2023-10-06T03:03:02.491754+0000","last_active":"2023-10-06T03:03:02.491754+0000","last_peered":"2023-10-06T03:03:02.491754+0000","last_clean":"2023-10-06T03:03:02.491754+0000","last_became_active":"2023-10-06T03:03:02.491434+0000","last_became_peered":"2023-10-06T03:03:02.491434+0000","last_unstale":"2023-10-06T03:03:02.491754+0000","last_undegraded":"2023-10-06T03:03:02.491754+0000","last_fullsized":"2023-10-06T03:03:02.491754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:59.255542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.474176+0000","last_change":"2023-10-06T03:03:01.567735+0000","last_active":"2023-10-06T03:03:02.474176+0000","last_peered":"2023-10-06T03:03:02.474176+0000","last_clean":"2023-10-06T03:03:02.474176+0000","last_became_active":"2023-10-06T03:03:01.567440+0000","last_became_peered":"2023-10-06T03:03:01.567440+0000","last_unstale":"2023-10-06T03:03:02.474176+0000","last_undegraded":"2023-10-06T03:03:02.474176+0000","last_fullsized":"2023-10-06T03:03:02.474176+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:51.437003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.490922+0000","last_change":"2023-10-06T03:03:02.490922+0000","last_active":"2023-10-06T03:03:02.490922+0000","last_peered":"2023-10-06T03:03:02.490922+0000","last_clean":"2023-10-06T03:03:02.490922+0000","last_became_active":"2023-10-06T03:03:02.490614+0000","last_became_peered":"2023-10-06T03:03:02.490614+0000","last_unstale":"2023-10-06T03:03:02.490922+0000","last_undegraded":"2023-10-06T03:03:02.490922+0000","last_fullsized":"2023-10-06T03:03:02.490922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:42.752266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.581773+0000","last_change":"2023-10-06T03:03:01.578668+0000","last_active":"2023-10-06T03:03:01.581773+0000","last_peered":"2023-10-06T03:03:01.581773+0000","last_clean":"2023-10-06T03:03:01.581773+0000","last_became_active":"2023-10-06T03:03:01.578364+0000","last_became_peered":"2023-10-06T03:03:01.578364+0000","last_unstale":"2023-10-06T03:03:01.581773+0000","last_undegraded":"2023-10-06T03:03:01.581773+0000","last_fullsized":"2023-10-06T03:03:01.581773+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:04.330142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.492878+0000","last_change":"2023-10-06T03:03:02.485962+0000","last_active":"2023-10-06T03:03:02.492878+0000","last_peered":"2023-10-06T03:03:02.492878+0000","last_clean":"2023-10-06T03:03:02.492878+0000","last_became_active":"2023-10-06T03:03:02.485683+0000","last_became_peered":"2023-10-06T03:03:02.485683+0000","last_unstale":"2023-10-06T03:03:02.492878+0000","last_undegraded":"2023-10-06T03:03:02.492878+0000","last_fullsized":"2023-10-06T03:03:02.492878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:52.154816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.578038+0000","last_change":"2023-10-06T03:03:01.572643+0000","last_active":"2023-10-06T03:03:01.578038+0000","last_peered":"2023-10-06T03:03:01.578038+0000","last_clean":"2023-10-06T03:03:01.578038+0000","last_became_active":"2023-10-06T03:03:01.572373+0000","last_became_peered":"2023-10-06T03:03:01.572373+0000","last_unstale":"2023-10-06T03:03:01.578038+0000","last_undegraded":"2023-10-06T03:03:01.578038+0000","last_fullsized":"2023-10-06T03:03:01.578038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:24.721089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.488556+0000","last_change":"2023-10-06T03:03:02.482115+0000","last_active":"2023-10-06T03:03:02.488556+0000","last_peered":"2023-10-06T03:03:02.488556+0000","last_clean":"2023-10-06T03:03:02.488556+0000","last_became_active":"2023-10-06T03:03:02.481822+0000","last_became_peered":"2023-10-06T03:03:02.481822+0000","last_unstale":"2023-10-06T03:03:02.488556+0000","last_undegraded":"2023-10-06T03:03:02.488556+0000","last_fullsized":"2023-10-06T03:03:02.488556+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:12:12.215388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577866+0000","last_change":"2023-10-06T03:03:01.573682+0000","last_active":"2023-10-06T03:03:01.577866+0000","last_peered":"2023-10-06T03:03:01.577866+0000","last_clean":"2023-10-06T03:03:01.577866+0000","last_became_active":"2023-10-06T03:03:01.573390+0000","last_became_peered":"2023-10-06T03:03:01.573390+0000","last_unstale":"2023-10-06T03:03:01.577866+0000","last_undegraded":"2023-10-06T03:03:01.577866+0000","last_fullsized":"2023-10-06T03:03:01.577866+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:32.643634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481218+0000","last_change":"2023-10-06T03:03:02.481218+0000","last_active":"2023-10-06T03:03:02.481218+0000","last_peered":"2023-10-06T03:03:02.481218+0000","last_clean":"2023-10-06T03:03:02.481218+0000","last_became_active":"2023-10-06T03:03:02.480916+0000","last_became_peered":"2023-10-06T03:03:02.480916+0000","last_unstale":"2023-10-06T03:03:02.481218+0000","last_undegraded":"2023-10-06T03:03:02.481218+0000","last_fullsized":"2023-10-06T03:03:02.481218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:59.858987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.473425+0000","last_change":"2023-10-06T03:03:01.575712+0000","last_active":"2023-10-06T03:03:02.473425+0000","last_peered":"2023-10-06T03:03:02.473425+0000","last_clean":"2023-10-06T03:03:02.473425+0000","last_became_active":"2023-10-06T03:03:01.575488+0000","last_became_peered":"2023-10-06T03:03:01.575488+0000","last_unstale":"2023-10-06T03:03:02.473425+0000","last_undegraded":"2023-10-06T03:03:02.473425+0000","last_fullsized":"2023-10-06T03:03:02.473425+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:48.973130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491547+0000","last_change":"2023-10-06T03:03:02.491547+0000","last_active":"2023-10-06T03:03:02.491547+0000","last_peered":"2023-10-06T03:03:02.491547+0000","last_clean":"2023-10-06T03:03:02.491547+0000","last_became_active":"2023-10-06T03:03:02.491261+0000","last_became_peered":"2023-10-06T03:03:02.491261+0000","last_unstale":"2023-10-06T03:03:02.491547+0000","last_undegraded":"2023-10-06T03:03:02.491547+0000","last_fullsized":"2023-10-06T03:03:02.491547+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:49.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577890+0000","last_change":"2023-10-06T03:03:01.571046+0000","last_active":"2023-10-06T03:03:01.577890+0000","last_peered":"2023-10-06T03:03:01.577890+0000","last_clean":"2023-10-06T03:03:01.577890+0000","last_became_active":"2023-10-06T03:03:01.570634+0000","last_became_peered":"2023-10-06T03:03:01.570634+0000","last_unstale":"2023-10-06T03:03:01.577890+0000","last_undegraded":"2023-10-06T03:03:01.577890+0000","last_fullsized":"2023-10-06T03:03:01.577890+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:55.922520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.574782+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:01.574782+0000","last_peered":"2023-10-06T03:03:01.574782+0000","last_clean":"2023-10-06T03:03:01.574782+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:01.574782+0000","last_undegraded":"2023-10-06T03:03:01.574782+0000","last_fullsized":"2023-10-06T03:03:01.574782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.496979+0000","last_change":"2023-10-06T03:03:02.492743+0000","last_active":"2023-10-06T03:03:02.496979+0000","last_peered":"2023-10-06T03:03:02.496979+0000","last_clean":"2023-10-06T03:03:02.496979+0000","last_became_active":"2023-10-06T03:03:02.492474+0000","last_became_peered":"2023-10-06T03:03:02.492474+0000","last_unstale":"2023-10-06T03:03:02.496979+0000","last_undegraded":"2023-10-06T03:03:02.496979+0000","last_fullsized":"2023-10-06T03:03:02.496979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:08.881932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"53'1","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481606+0000","last_change":"2023-10-06T03:03:01.578439+0000","last_active":"2023-10-06T03:03:02.481606+0000","last_peered":"2023-10-06T03:03:02.481606+0000","last_clean":"2023-10-06T03:03:02.481606+0000","last_became_active":"2023-10-06T03:03:01.578099+0000","last_became_peered":"2023-10-06T03:03:01.578099+0000","last_unstale":"2023-10-06T03:03:02.481606+0000","last_undegraded":"2023-10-06T03:03:02.481606+0000","last_fullsized":"2023-10-06T03:03:02.481606+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:39.896166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.474290+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:02.474290+0000","last_peered":"2023-10-06T03:03:02.474290+0000","last_clean":"2023-10-06T03:03:02.474290+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:02.474290+0000","last_undegraded":"2023-10-06T03:03:02.474290+0000","last_fullsized":"2023-10-06T03:03:02.474290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.495619+0000","last_change":"2023-10-06T03:03:02.495619+0000","last_active":"2023-10-06T03:03:02.495619+0000","last_peered":"2023-10-06T03:03:02.495619+0000","last_clean":"2023-10-06T03:03:02.495619+0000","last_became_active":"2023-10-06T03:03:02.494843+0000","last_became_peered":"2023-10-06T03:03:02.494843+0000","last_unstale":"2023-10-06T03:03:02.495619+0000","last_undegraded":"2023-10-06T03:03:02.495619+0000","last_fullsized":"2023-10-06T03:03:02.495619+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:17.521000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.568864+0000","last_change":"2023-10-06T03:03:01.579531+0000","last_active":"2023-10-06T03:03:02.568864+0000","last_peered":"2023-10-06T03:03:02.568864+0000","last_clean":"2023-10-06T03:03:02.568864+0000","last_became_active":"2023-10-06T03:03:01.579281+0000","last_became_peered":"2023-10-06T03:03:01.579281+0000","last_unstale":"2023-10-06T03:03:02.568864+0000","last_undegraded":"2023-10-06T03:03:02.568864+0000","last_fullsized":"2023-10-06T03:03:02.568864+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:35.124694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.570742+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:01.570742+0000","last_peered":"2023-10-06T03:03:01.570742+0000","last_clean":"2023-10-06T03:03:01.570742+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:01.570742+0000","last_undegraded":"2023-10-06T03:03:01.570742+0000","last_fullsized":"2023-10-06T03:03:01.570742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.481237+0000","last_change":"2023-10-06T03:03:02.481237+0000","last_active":"2023-10-06T03:03:02.481237+0000","last_peered":"2023-10-06T03:03:02.481237+0000","last_clean":"2023-10-06T03:03:02.481237+0000","last_became_active":"2023-10-06T03:03:02.481024+0000","last_became_peered":"2023-10-06T03:03:02.481024+0000","last_unstale":"2023-10-06T03:03:02.481237+0000","last_undegraded":"2023-10-06T03:03:02.481237+0000","last_fullsized":"2023-10-06T03:03:02.481237+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:24.544569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"53'2","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.567255+0000","last_change":"2023-10-06T03:03:01.575675+0000","last_active":"2023-10-06T03:03:02.567255+0000","last_peered":"2023-10-06T03:03:02.567255+0000","last_clean":"2023-10-06T03:03:02.567255+0000","last_became_active":"2023-10-06T03:03:01.575451+0000","last_became_peered":"2023-10-06T03:03:01.575451+0000","last_unstale":"2023-10-06T03:03:02.567255+0000","last_undegraded":"2023-10-06T03:03:02.567255+0000","last_fullsized":"2023-10-06T03:03:02.567255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:32.008675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.569043+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:01.569043+0000","last_peered":"2023-10-06T03:03:01.569043+0000","last_clean":"2023-10-06T03:03:01.569043+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:01.569043+0000","last_undegraded":"2023-10-06T03:03:01.569043+0000","last_fullsized":"2023-10-06T03:03:01.569043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491152+0000","last_change":"2023-10-06T03:03:02.491152+0000","last_active":"2023-10-06T03:03:02.491152+0000","last_peered":"2023-10-06T03:03:02.491152+0000","last_clean":"2023-10-06T03:03:02.491152+0000","last_became_active":"2023-10-06T03:03:02.490883+0000","last_became_peered":"2023-10-06T03:03:02.490883+0000","last_unstale":"2023-10-06T03:03:02.491152+0000","last_undegraded":"2023-10-06T03:03:02.491152+0000","last_fullsized":"2023-10-06T03:03:02.491152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:40.902722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.577916+0000","last_change":"2023-10-06T03:03:01.570001+0000","last_active":"2023-10-06T03:03:01.577916+0000","last_peered":"2023-10-06T03:03:01.577916+0000","last_clean":"2023-10-06T03:03:01.577916+0000","last_became_active":"2023-10-06T03:03:01.569714+0000","last_became_peered":"2023-10-06T03:03:01.569714+0000","last_unstale":"2023-10-06T03:03:01.577916+0000","last_undegraded":"2023-10-06T03:03:01.577916+0000","last_fullsized":"2023-10-06T03:03:01.577916+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:51.360204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":193,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.569129+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:01.569129+0000","last_peered":"2023-10-06T03:03:01.569129+0000","last_clean":"2023-10-06T03:03:01.569129+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:01.569129+0000","last_undegraded":"2023-10-06T03:03:01.569129+0000","last_fullsized":"2023-10-06T03:03:01.569129+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.477045+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:02.477045+0000","last_peered":"2023-10-06T03:03:02.477045+0000","last_clean":"2023-10-06T03:03:02.477045+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:02.477045+0000","last_undegraded":"2023-10-06T03:03:02.477045+0000","last_fullsized":"2023-10-06T03:03:02.477045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.489236+0000","last_change":"2023-10-06T03:03:02.483349+0000","last_active":"2023-10-06T03:03:02.489236+0000","last_peered":"2023-10-06T03:03:02.489236+0000","last_clean":"2023-10-06T03:03:02.489236+0000","last_became_active":"2023-10-06T03:03:02.483137+0000","last_became_peered":"2023-10-06T03:03:02.483137+0000","last_unstale":"2023-10-06T03:03:02.489236+0000","last_undegraded":"2023-10-06T03:03:02.489236+0000","last_fullsized":"2023-10-06T03:03:02.489236+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:33.846528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.478959+0000","last_change":"2023-10-06T03:03:01.571488+0000","last_active":"2023-10-06T03:03:02.478959+0000","last_peered":"2023-10-06T03:03:02.478959+0000","last_clean":"2023-10-06T03:03:02.478959+0000","last_became_active":"2023-10-06T03:03:01.570904+0000","last_became_peered":"2023-10-06T03:03:01.570904+0000","last_unstale":"2023-10-06T03:03:02.478959+0000","last_undegraded":"2023-10-06T03:03:02.478959+0000","last_fullsized":"2023-10-06T03:03:02.478959+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:02.639969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.566181+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:01.566181+0000","last_peered":"2023-10-06T03:03:01.566181+0000","last_clean":"2023-10-06T03:03:01.566181+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:01.566181+0000","last_undegraded":"2023-10-06T03:03:01.566181+0000","last_fullsized":"2023-10-06T03:03:01.566181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.491934+0000","last_change":"2023-10-06T03:03:02.491934+0000","last_active":"2023-10-06T03:03:02.491934+0000","last_peered":"2023-10-06T03:03:02.491934+0000","last_clean":"2023-10-06T03:03:02.491934+0000","last_became_active":"2023-10-06T03:03:02.491231+0000","last_became_peered":"2023-10-06T03:03:02.491231+0000","last_unstale":"2023-10-06T03:03:02.491934+0000","last_undegraded":"2023-10-06T03:03:02.491934+0000","last_fullsized":"2023-10-06T03:03:02.491934+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:49.090760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-10-06T03:03:01.580689+0000","last_change":"2023-10-06T03:03:01.575645+0000","last_active":"2023-10-06T03:03:01.580689+0000","last_peered":"2023-10-06T03:03:01.580689+0000","last_clean":"2023-10-06T03:03:01.580689+0000","last_became_active":"2023-10-06T03:03:01.575381+0000","last_became_peered":"2023-10-06T03:03:01.575381+0000","last_unstale":"2023-10-06T03:03:01.580689+0000","last_undegraded":"2023-10-06T03:03:01.580689+0000","last_fullsized":"2023-10-06T03:03:01.580689+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:36.328541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.479055+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:02.479055+0000","last_peered":"2023-10-06T03:03:02.479055+0000","last_clean":"2023-10-06T03:03:02.479055+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:02.479055+0000","last_undegraded":"2023-10-06T03:03:02.479055+0000","last_fullsized":"2023-10-06T03:03:02.479055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":8,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.493856+0000","last_change":"2023-10-06T03:03:02.493856+0000","last_active":"2023-10-06T03:03:02.493856+0000","last_peered":"2023-10-06T03:03:02.493856+0000","last_clean":"2023-10-06T03:03:02.493856+0000","last_became_active":"2023-10-06T03:03:02.493611+0000","last_became_peered":"2023-10-06T03:03:02.493611+0000","last_unstale":"2023-10-06T03:03:02.493856+0000","last_undegraded":"2023-10-06T03:03:02.493856+0000","last_fullsized":"2023-10-06T03:03:02.493856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:01.549160+0000","last_clean_scrub_stamp":"2023-10-06T03:03:01.549160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:11.175570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"53'1","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.566099+0000","last_change":"2023-10-06T03:03:01.571117+0000","last_active":"2023-10-06T03:03:02.566099+0000","last_peered":"2023-10-06T03:03:02.566099+0000","last_clean":"2023-10-06T03:03:02.566099+0000","last_became_active":"2023-10-06T03:03:01.570838+0000","last_became_peered":"2023-10-06T03:03:01.570838+0000","last_unstale":"2023-10-06T03:03:02.566099+0000","last_undegraded":"2023-10-06T03:03:02.566099+0000","last_fullsized":"2023-10-06T03:03:02.566099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:00.542604+0000","last_clean_scrub_stamp":"2023-10-06T03:03:00.542604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:34:51.086706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-10-06T03:03:02.473527+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:02.473527+0000","last_peered":"2023-10-06T03:03:02.473527+0000","last_clean":"2023-10-06T03:03:02.473527+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:02.473527+0000","last_undegraded":"2023-10-06T03:03:02.473527+0000","last_fullsized":"2023-10-06T03:03:02.473527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298504,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073336,"statfs":{"total":96636764160,"available":96331096064,"internally_reserved":0,"allocated":1245184,"data_stored":872927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.63},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297912,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073928,"statfs":{"total":96636764160,"available":96331702272,"internally_reserved":0,"allocated":638976,"data_stored":281845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297948,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073892,"statfs":{"total":96636764160,"available":96331665408,"internally_reserved":0,"allocated":675840,"data_stored":290724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.253},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.26100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298488,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073352,"statfs":{"total":96636764160,"available":96331112448,"internally_reserved":0,"allocated":1228800,"data_stored":872179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.505}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.622}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":281811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.312}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.628}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297908,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073932,"statfs":{"total":96636764160,"available":96331706368,"internally_reserved":0,"allocated":634880,"data_stored":281811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"last":0.375}]},{"osd":1,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":5.5220000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.63200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297932,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073908,"statfs":{"total":96636764160,"available":96331681792,"internally_reserved":0,"allocated":659456,"data_stored":289988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.62},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.44}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297924,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073916,"statfs":{"total":96636764160,"available":96331689984,"internally_reserved":0,"allocated":651264,"data_stored":288474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.3869999999999996,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.627}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.69999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:06.989 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:03:06.990 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:03:07.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.143+0000 7f507755c700 1 Processor -- start 2023-10-06T03:03:07.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.143+0000 7f507755c700 1 -- start start 2023-10-06T03:03:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f507755c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5070069c70 0x7f50700cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f507755c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700ce380 con 0x7f50700af320 2023-10-06T03:03:07.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f507755c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700cec20 con 0x7f5070159950 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f507755c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50700afa90 con 0x7f5070069c70 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f5075af9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f50700af320 0x7f50700cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34366/0 (socket says 172.21.15.92:34366) 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f5075af9700 1 -- 172.21.15.92:0/3290140083 learned_addr learned my addr 172.21.15.92:0/3290140083 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f5074af7700 1 --2- 172.21.15.92:0/3290140083 >> v2:172.21.15.136:3300/0 conn(0x7f5070069c70 0x7f50700cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303950885 0 0) 0x7f50700ce380 con 0x7f50700af320 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.144+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f505c003570 con 0x7f50700af320 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053425437 0 0) 0x7f50700cec20 con 0x7f5070159950 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50700ce380 con 0x7f5070159950 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468539889 0 0) 0x7f505c003570 con 0x7f50700af320 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 >> v2:172.21.15.136:3300/0 conn(0x7f5070069c70 msgr2=0x7f50700cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 --2- 172.21.15.92:0/3290140083 >> v2:172.21.15.136:3300/0 conn(0x7f5070069c70 0x7f50700cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 >> v1:172.21.15.136:6789/0 conn(0x7f5070159950 legacy=0x7f5070069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700ce5d0 con 0x7f50700af320 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5068001470 con 0x7f50700af320 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 >> v1:172.21.15.92:6789/0 conn(0x7f50700af320 legacy=0x7f50700cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f505c003c40 0x7f505c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f505c004810 0x7f505c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f505c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700afa90 con 0x7f505c005210 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c00c1b0 con 0x7f505c003c40 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c00c370 con 0x7f505c004810 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f505c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5074af7700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f505c004810 0x7f505c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f50752f8700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f505c003c40 0x7f505c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 -- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f505c004810 msgr2=0x7f505c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f505c004810 0x7f505c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 -- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f505c003c40 msgr2=0x7f505c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f505c003c40 0x7f505c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 -- 172.21.15.92:0/3290140083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c003570 con 0x7f505c005210 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 -- 172.21.15.92:0/3290140083 reap_dead start 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5075af9700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f505c00ba60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5068012970 tx=0x7f50700ceb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3cc202bed91691 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068017070 con 0x7f505c005210 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5068005920 con 0x7f505c005210 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.145+0000 7f5067fff700 1 -- 172.21.15.92:0/3290140083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068007af0 con 0x7f505c005210 2023-10-06T03:03:07.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 -- 172.21.15.92:0/3290140083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 msgr2=0x7f505c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f505c00ba60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5068012970 tx=0x7f50700ceb90 comp rx=0 tx=0).stop 2023-10-06T03:03:07.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 -- 172.21.15.92:0/3290140083 shutdown_connections 2023-10-06T03:03:07.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 --2- 172.21.15.92:0/3290140083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f505c00ba60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 -- 172.21.15.92:0/3290140083 >> 172.21.15.92:0/3290140083 conn(0x7f5070068e00 msgr2=0x7f50700d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:07.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 -- 172.21.15.92:0/3290140083 shutdown_connections 2023-10-06T03:03:07.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.146+0000 7f507755c700 1 -- 172.21.15.92:0/3290140083 wait complete. 2023-10-06T03:03:07.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.147+0000 7f507755c700 1 Processor -- start 2023-10-06T03:03:07.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.147+0000 7f507755c700 1 -- start start 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f507755c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f5070148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f507755c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50700c8cc0 0x7f5070149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f507755c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f50700ca760 0x7f50700b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f507755c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700c9130 con 0x7f505c005210 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f507755c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700c9410 con 0x7f50700c8cc0 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f507755c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700c96f0 con 0x7f50700ca760 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f5070148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f5070148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40282/0 (socket says 172.21.15.92:40282) 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 -- 172.21.15.92:0/3038140685 learned_addr learned my addr 172.21.15.92:0/3038140685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f5075af9700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f50700ca760 0x7f50700b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f5074af7700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50700c8cc0 0x7f5070149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 -- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f50700ca760 msgr2=0x7f50700b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f50700ca760 0x7f50700b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 -- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50700c8cc0 msgr2=0x7f5070149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50700c8cc0 0x7f5070149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.148+0000 7f50752f8700 1 -- 172.21.15.92:0/3038140685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700c99d0 con 0x7f505c005210 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.149+0000 7f50752f8700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f5070148b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f506c00ecf0 tx=0x7f506c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.149+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00eea0 con 0x7f505c005210 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.149+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c006ba0 con 0x7f505c005210 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.149+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700c9cb0 con 0x7f505c005210 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.149+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00cb10 con 0x7f505c005210 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.149+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50700b0e30 con 0x7f505c005210 2023-10-06T03:03:07.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.150+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50700718b0 con 0x7f505c005210 2023-10-06T03:03:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.150+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f506c015430 con 0x7f505c005210 2023-10-06T03:03:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.150+0000 7f50657fa700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5054066f30 0x7f50540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.151+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f506c084d80 con 0x7f505c005210 2023-10-06T03:03:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.151+0000 7f5074af7700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5054066f30 0x7f50540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.151+0000 7f5074af7700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5054066f30 0x7f50540693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5060006ed0 tx=0x7f5060008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:07.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.155+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f506c051e90 con 0x7f505c005210 2023-10-06T03:03:07.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.410+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f50700ce8b0 con 0x7f505c005210 2023-10-06T03:03:07.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.414+0000 7f50657fa700 1 -- 172.21.15.92:0/3038140685 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f506c0554b0 con 0x7f505c005210 2023-10-06T03:03:07.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5054066f30 msgr2=0x7f50540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5054066f30 0x7f50540693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5060006ed0 tx=0x7f5060008040 comp rx=0 tx=0).stop 2023-10-06T03:03:07.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 msgr2=0x7f5070148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f5070148b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f506c00ecf0 tx=0x7f506c0017a0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 shutdown_connections 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5054066f30 0x7f50540693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f505c005210 0x7f5070148b50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f50700ca760 0x7f50700b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 --2- 172.21.15.92:0/3038140685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f50700c8cc0 0x7f5070149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 >> 172.21.15.92:0/3038140685 conn(0x7f5070068e00 msgr2=0x7f50700b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 shutdown_connections 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.416+0000 7f507755c700 1 -- 172.21.15.92:0/3038140685 wait complete. 2023-10-06T03:03:07.419 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:03:07.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:07.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.544+0000 7ffb916b3700 1 Processor -- start 2023-10-06T03:03:07.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.544+0000 7ffb916b3700 1 -- start start 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb916b3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 0x7ffb8c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb916b3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0b2980 con 0x7ffb8c15d230 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb916b3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0b11d0 con 0x7ffb8c1597f0 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb916b3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b14b0 con 0x7ffb8c159420 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb90eb2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffb8c15d230 0x7ffb8c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34368/0 (socket says 172.21.15.92:34368) 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb90eb2700 1 -- 172.21.15.92:0/1213016848 learned_addr learned my addr 172.21.15.92:0/1213016848 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515489345 0 0) 0x7ffb8c0b2980 con 0x7ffb8c15d230 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb8bfff700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 0x7ffb8c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.545+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb74003570 con 0x7ffb8c15d230 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2437007101 0 0) 0x7ffb74003570 con 0x7ffb8c15d230 2023-10-06T03:03:07.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 msgr2=0x7ffb8c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 0x7ffb8c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 >> v1:172.21.15.136:6789/0 conn(0x7ffb8c1597f0 legacy=0x7ffb8c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b1790 con 0x7ffb8c15d230 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7ffb840017f0 con 0x7ffb8c15d230 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 >> v1:172.21.15.92:6789/0 conn(0x7ffb8c15d230 legacy=0x7ffb8c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb74003a90 0x7ffb74003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 0x7ffb740051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b14b0 con 0x7ffb74004810 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7400bb60 con 0x7ffb74003a90 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7400bd20 con 0x7ffb8c159420 2023-10-06T03:03:07.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8bfff700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb74003a90 0x7ffb74003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb90eb2700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 0x7ffb740051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 -- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 msgr2=0x7ffb740051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c159420 0x7ffb740051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 -- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb74003a90 msgr2=0x7ffb74003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb74003a90 0x7ffb74003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 -- 172.21.15.92:0/1213016848 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74003570 con 0x7ffb74004810 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8b7fe700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb74004c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb800018b0 tx=0x7ffb80007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a820d2b865540c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80012420 con 0x7ffb74004810 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb80012580 con 0x7ffb74004810 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.546+0000 7ffb8affd700 1 -- 172.21.15.92:0/1213016848 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80012920 con 0x7ffb74004810 2023-10-06T03:03:07.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb916b3700 1 -- 172.21.15.92:0/1213016848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 msgr2=0x7ffb74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb916b3700 1 --2- 172.21.15.92:0/1213016848 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb74004c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ffb800018b0 tx=0x7ffb80007d30 comp rx=0 tx=0).stop 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb90eb2700 1 -- 172.21.15.92:0/1213016848 reap_dead start 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb916b3700 1 -- 172.21.15.92:0/1213016848 shutdown_connections 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb916b3700 1 -- 172.21.15.92:0/1213016848 >> 172.21.15.92:0/1213016848 conn(0x7ffb8c0d0660 msgr2=0x7ffb8c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb916b3700 1 -- 172.21.15.92:0/1213016848 shutdown_connections 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.547+0000 7ffb916b3700 1 -- 172.21.15.92:0/1213016848 wait complete. 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.548+0000 7ffb916b3700 1 Processor -- start 2023-10-06T03:03:07.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.548+0000 7ffb916b3700 1 -- start start 2023-10-06T03:03:07.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb8c1676f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c16d0b0 0x7ffb8c16d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb8c15baa0 0x7ffb8c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c16da60 con 0x7ffb74004810 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c16bc50 con 0x7ffb8c15baa0 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c16b040 con 0x7ffb8c16d0b0 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb8c1676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb8c1676f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40312/0 (socket says 172.21.15.92:40312) 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 -- 172.21.15.92:0/4282571346 learned_addr learned my addr 172.21.15.92:0/4282571346 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 -- 172.21.15.92:0/4282571346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c16d0b0 msgr2=0x7ffb8c16d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c16d0b0 0x7ffb8c16d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 -- 172.21.15.92:0/4282571346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb8c15baa0 msgr2=0x7ffb8c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb8c15baa0 0x7ffb8c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 -- 172.21.15.92:0/4282571346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c16b320 con 0x7ffb74004810 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb8bfff700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb8c1676f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c011d40 tx=0x7ffb7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00fac0 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c00fc20 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.549+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c16b600 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.550+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c014440 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.550+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c16ac80 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.550+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c16baa0 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.551+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb7c014870 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.551+0000 7ffb73fff700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffb78066f30 0x7ffb780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.551+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb7c018030 con 0x7ffb74004810 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.551+0000 7ffb8b7fe700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffb78066f30 0x7ffb780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:07.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.552+0000 7ffb8b7fe700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffb78066f30 0x7ffb780693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb80001690 tx=0x7ffb80015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:07.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.556+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb7c0522f0 con 0x7ffb74004810 2023-10-06T03:03:07.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.783+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb8c04c3c0 con 0x7ffb74004810 2023-10-06T03:03:07.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.783+0000 7ffb73fff700 1 -- 172.21.15.92:0/4282571346 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ffb7c055910 con 0x7ffb74004810 2023-10-06T03:03:07.785 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:07.785 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:01.941144+0000","modified":"2023-10-06T03:03:05.571224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3249599548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3249599548},{"type":"v1","addr":"172.21.15.92:6837","nonce":3249599548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:03:07.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffb78066f30 msgr2=0x7ffb780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffb78066f30 0x7ffb780693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb80001690 tx=0x7ffb80015000 comp rx=0 tx=0).stop 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 msgr2=0x7ffb8c1676f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb8c1676f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c011d40 tx=0x7ffb7c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 shutdown_connections 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffb78066f30 0x7ffb780693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb74004810 0x7ffb8c1676f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb8c15baa0 0x7ffb8c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 --2- 172.21.15.92:0/4282571346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb8c16d0b0 0x7ffb8c16d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:07.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.785+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 >> 172.21.15.92:0/4282571346 conn(0x7ffb8c0d0660 msgr2=0x7ffb8c155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.786+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 shutdown_connections 2023-10-06T03:03:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:07.786+0000 7ffb916b3700 1 -- 172.21.15.92:0/4282571346 wait complete. 2023-10-06T03:03:07.789 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 29 2023-10-06T03:03:08.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:03:01.941144+0000', 'modified': '2023-10-06T03:03:05.571224+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5364}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5364': {'gid': 5364, 'name': 'a', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/3249599548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 3249599548}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 3249599548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:03:08.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:03:08.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:03:08.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.322+0000 7f83fc2e0700 1 Processor -- start 2023-10-06T03:03:08.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.322+0000 7f83fc2e0700 1 -- start start 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fc2e0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f4159890 0x7f83f415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fc2e0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83f4159c60 0x7f83f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fc2e0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f415a160 con 0x7f83f415db10 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fc2e0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f415df50 con 0x7f83f4159c60 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fc2e0700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f40d13a0 con 0x7f83f4159890 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f987b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83f4159c60 0x7f83f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fa87d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f83f415db10 0x7f83f4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34384/0 (socket says 172.21.15.92:34384) 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fa87d700 1 -- 172.21.15.92:0/2210335353 learned_addr learned my addr 172.21.15.92:0/2210335353 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83fa07c700 1 --2- 172.21.15.92:0/2210335353 >> v2:172.21.15.136:3300/0 conn(0x7f83f4159890 0x7f83f415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f987b700 1 -- 172.21.15.92:0/2210335353 >> v2:172.21.15.136:3300/0 conn(0x7f83f4159890 msgr2=0x7f83f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f987b700 1 --2- 172.21.15.92:0/2210335353 >> v2:172.21.15.136:3300/0 conn(0x7f83f4159890 0x7f83f415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f987b700 1 -- 172.21.15.92:0/2210335353 >> v1:172.21.15.92:6789/0 conn(0x7f83f415db10 legacy=0x7f83f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f907a700 1 -- 172.21.15.92:0/2210335353 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025004869 0 0) 0x7f83f415a160 con 0x7f83f415db10 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f987b700 1 -- 172.21.15.92:0/2210335353 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f415e140 con 0x7f83f4159c60 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.323+0000 7f83f987b700 1 --2- 172.21.15.92:0/2210335353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f4159c60 0x7f83f415d5d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f83f40d1090 tx=0x7f83e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4816cac683a78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83f907a700 1 -- 172.21.15.92:0/2210335353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400b6b0 con 0x7f83f4159c60 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83f907a700 1 -- 172.21.15.92:0/2210335353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400b810 con 0x7f83f4159c60 2023-10-06T03:03:08.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83f907a700 1 -- 172.21.15.92:0/2210335353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400bbb0 con 0x7f83f4159c60 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 -- 172.21.15.92:0/2210335353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f4159c60 msgr2=0x7f83f415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/2210335353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f4159c60 0x7f83f415d5d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f83f40d1090 tx=0x7f83e40091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 -- 172.21.15.92:0/2210335353 shutdown_connections 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/2210335353 >> v2:172.21.15.136:3300/0 conn(0x7f83f4159890 0x7f83f415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/2210335353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f4159c60 0x7f83f415d5d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 -- 172.21.15.92:0/2210335353 >> 172.21.15.92:0/2210335353 conn(0x7f83f4068e20 msgr2=0x7f83f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 -- 172.21.15.92:0/2210335353 shutdown_connections 2023-10-06T03:03:08.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.324+0000 7f83fc2e0700 1 -- 172.21.15.92:0/2210335353 wait complete. 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.325+0000 7f83fc2e0700 1 Processor -- start 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.325+0000 7f83fc2e0700 1 -- start start 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fc2e0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f4159890 0x7f83f40cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fc2e0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83f4159c60 0x7f83f414d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fc2e0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 0x7f83f4152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fc2e0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4151ff0 con 0x7f83f415db10 2023-10-06T03:03:08.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fc2e0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40cdc20 con 0x7f83f4159c60 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fc2e0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40cdec0 con 0x7f83f4159890 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fa87d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 0x7f83f4152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fa87d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 0x7f83f4152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40342/0 (socket says 172.21.15.92:40342) 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fa87d700 1 -- 172.21.15.92:0/673280237 learned_addr learned my addr 172.21.15.92:0/673280237 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83fa07c700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f4159890 0x7f83f40cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.326+0000 7f83f987b700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83f4159c60 0x7f83f414d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fa87d700 1 -- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f4159890 msgr2=0x7f83f40cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fa87d700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f4159890 0x7f83f40cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fa87d700 1 -- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83f4159c60 msgr2=0x7f83f414d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fa87d700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83f4159c60 0x7f83f414d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fa87d700 1 -- 172.21.15.92:0/673280237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4009420 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fa87d700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 0x7f83f4152300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00cbf0 tx=0x7f83ec0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83ea7fc700 1 -- 172.21.15.92:0/673280237 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00a9c0 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83ea7fc700 1 -- 172.21.15.92:0/673280237 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00ab20 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4152e50 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83ea7fc700 1 -- 172.21.15.92:0/673280237 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0070f0 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.327+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f40ce720 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.328+0000 7f83ea7fc700 1 -- 172.21.15.92:0/673280237 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83ec007520 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.329+0000 7f83ea7fc700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d8066f30 0x7f83d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.329+0000 7f83ea7fc700 1 -- 172.21.15.92:0/673280237 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83ec084b10 con 0x7f83f415db10 2023-10-06T03:03:08.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.329+0000 7f83fa07c700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d8066f30 0x7f83d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.329+0000 7f83fa07c700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d8066f30 0x7f83d80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f83f0006fd0 tx=0x7f83f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 Processor -- start 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 -- start start 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 0x55f202ea6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ef66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202dfda70 con 0x55f202ea6840 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202dfb270 con 0x55f202ea7cc0 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f202df7060 con 0x55f202ef6c00 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f83fa07c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 0x55f202ea6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f83fa07c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 0x55f202ea6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40350/0 (socket says 172.21.15.92:40350) 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.331+0000 7f83fa07c700 1 -- 172.21.15.92:0/397842658 learned_addr learned my addr 172.21.15.92:0/397842658 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83fa07c700 1 -- 172.21.15.92:0/397842658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 msgr2=0x55f202ef7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83fa07c700 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83fa07c700 1 -- 172.21.15.92:0/397842658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 msgr2=0x55f202ef66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83fa07c700 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ef66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83fa07c700 1 -- 172.21.15.92:0/397842658 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f202c91270 con 0x55f202ea6840 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83fa07c700 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 0x55f202ea6c10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f83f000fab0 tx=0x7f83f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9811f809a721386c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83d77fe700 1 -- 172.21.15.92:0/397842658 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f000f710 con 0x55f202ea6840 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83d77fe700 1 -- 172.21.15.92:0/397842658 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0019070 con 0x55f202ea6840 2023-10-06T03:03:08.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.332+0000 7f83d77fe700 1 -- 172.21.15.92:0/397842658 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f0014b00 con 0x55f202ea6840 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 -- 172.21.15.92:0/397842658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 msgr2=0x55f202ea6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 0x55f202ea6c10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f83f000fab0 tx=0x7f83f0016000 comp rx=0 tx=0).stop 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 -- 172.21.15.92:0/397842658 shutdown_connections 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea6840 0x55f202ea6c10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 --2- 172.21.15.92:0/397842658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ef66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 -- 172.21.15.92:0/397842658 >> 172.21.15.92:0/397842658 conn(0x55f202ea8160 msgr2=0x55f202ea8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 -- 172.21.15.92:0/397842658 shutdown_connections 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 -- 172.21.15.92:0/397842658 wait complete. 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 -- start start 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.333+0000 7f84055d8b80 1 Processor -- start 2023-10-06T03:03:08.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202eed130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 0x55f202ef4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202da2890 con 0x55f202ea7cc0 2023-10-06T03:03:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202dbdea0 con 0x55f202ea6840 2023-10-06T03:03:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f202c79e30 con 0x55f202ef6c00 2023-10-06T03:03:08.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f84055d8b80 10 client.0 _ll_register_callbacks cb 0x55f202e69e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f83f987b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 0x55f202ef4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f83f987b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 0x55f202ef4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40360/0 (socket says 172.21.15.92:40360) 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.334+0000 7f83f987b700 1 -- 172.21.15.92:0/3107276955 learned_addr learned my addr 172.21.15.92:0/3107276955 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83fa87d700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83fa07c700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202eed130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83f987b700 1 -- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 msgr2=0x55f202ef33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83f987b700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83f987b700 1 -- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 msgr2=0x55f202eed130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83f987b700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202eed130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83f987b700 1 -- 172.21.15.92:0/3107276955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f000e040 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83f987b700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 0x55f202ef4fb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f83e400b0e0 tx=0x7f83e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bce3c7c72ba456 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83d4ff9700 1 -- 172.21.15.92:0/3107276955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4020070 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83d4ff9700 1 -- 172.21.15.92:0/3107276955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83e4026c90 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f202e9b940 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83d4ff9700 1 -- 172.21.15.92:0/3107276955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e401a420 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 10 client.5529 fetch_fsmap learned FSMap version 29 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55f202c33fe0 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83d4ff9700 1 -- 172.21.15.92:0/3107276955 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f202e9b940 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f83d4ff9700 1 -- 172.21.15.92:0/3107276955 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f83e401a580 con 0x55f202ea7cc0 2023-10-06T03:03:08.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 10 client.5529 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 10 client.5529 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 20 client.5529 populate_metadata read hostname 'smithi092' 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ee50b0 0x55f202ee5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 4 client.5529 mds_command: new command op to 5364 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.335+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 --> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f202e78870 con 0x55f202ee50b0 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.336+0000 7f83fa07c700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ee50b0 0x55f202ee5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.336+0000 7f83d4ff9700 10 client.5529 ms_handle_connect on v2:172.21.15.92:6835/3249599548 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.336+0000 7f83fa07c700 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ee50b0 0x55f202ee5480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72868647f206da server_cookie=113eab5a0dcce95b in_seq=0 out_seq=0 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.336+0000 7f83d4ff9700 1 -- 172.21.15.92:0/3107276955 <== mds.0 v2:172.21.15.92:6835/3249599548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f202e78870 con 0x55f202ee50b0 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.336+0000 7f83d4ff9700 10 client.5529 handle_command_reply: tid=0 2023-10-06T03:03:08.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.336+0000 7f84055d8b80 1 client.5529 shutdown 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 msgr2=0x55f202ef4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 0x55f202ef4fb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f83e400b0e0 tx=0x7f83e4017000 comp rx=0 tx=0).stop 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 shutdown_connections 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ee50b0 0x55f202ee5480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ef6c00 0x55f202ef33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f83d4ff9700 0 client.5529 ms_handle_reset on v2:172.21.15.92:6835/3249599548 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202eed130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 --2- 172.21.15.92:0/3107276955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ea7cc0 0x55f202ef4fb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 >> 172.21.15.92:0/3107276955 conn(0x55f202ea8160 msgr2=0x55f202ea71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 shutdown_connections 2023-10-06T03:03:08.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 1 -- 172.21.15.92:0/3107276955 wait complete. 2023-10-06T03:03:08.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.337+0000 7f84055d8b80 20 client.5529 trim_cache size 0 max 16384 2023-10-06T03:03:08.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 Processor -- start 2023-10-06T03:03:08.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 -- start start 2023-10-06T03:03:08.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202ee5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea7cc0 0x55f202ea3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202eeb3b0 con 0x55f202ef6c00 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202f00130 con 0x55f202ea6840 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f202e862e0 con 0x55f202ea7cc0 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.354+0000 7f83fa87d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40368/0 (socket says 172.21.15.92:40368) 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 -- 172.21.15.92:0/854477155 learned_addr learned my addr 172.21.15.92:0/854477155 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:08.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 -- 172.21.15.92:0/854477155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea7cc0 msgr2=0x55f202ea3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea7cc0 0x55f202ea3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 -- 172.21.15.92:0/854477155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 msgr2=0x55f202ee5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202ee5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 -- 172.21.15.92:0/854477155 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f202dc5780 con 0x55f202ef6c00 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83fa87d700 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee6bb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f83ec082ec0 tx=0x7f83ec082d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7a653d8931409 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83d7fff700 1 -- 172.21.15.92:0/854477155 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec055940 con 0x55f202ef6c00 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83d7fff700 1 -- 172.21.15.92:0/854477155 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec058b10 con 0x55f202ef6c00 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f83d7fff700 1 -- 172.21.15.92:0/854477155 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec05c860 con 0x55f202ef6c00 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f84055d8b80 1 -- 172.21.15.92:0/854477155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 msgr2=0x55f202ee6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f84055d8b80 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee6bb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f83ec082ec0 tx=0x7f83ec082d90 comp rx=0 tx=0).stop 2023-10-06T03:03:08.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f84055d8b80 1 -- 172.21.15.92:0/854477155 shutdown_connections 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f84055d8b80 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee6bb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.355+0000 7f84055d8b80 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea6840 0x55f202ee5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.356+0000 7f84055d8b80 1 --2- 172.21.15.92:0/854477155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea7cc0 0x55f202ea3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.356+0000 7f84055d8b80 1 -- 172.21.15.92:0/854477155 >> 172.21.15.92:0/854477155 conn(0x55f202ea8160 msgr2=0x55f202e9f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.356+0000 7f84055d8b80 1 -- 172.21.15.92:0/854477155 shutdown_connections 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.356+0000 7f84055d8b80 1 -- 172.21.15.92:0/854477155 wait complete. 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.356+0000 7f84055d8b80 1 -- start start 2023-10-06T03:03:08.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.356+0000 7f84055d8b80 1 Processor -- start 2023-10-06T03:03:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea6840 0x55f202f53750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ee6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202ea1fb0 con 0x55f202ef6c00 2023-10-06T03:03:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f202ef9ba0 con 0x55f202ea7cc0 2023-10-06T03:03:08.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f202ea7490 con 0x55f202ea6840 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f84055d8b80 10 client.0 _ll_register_callbacks cb 0x55f202e69e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40370/0 (socket says 172.21.15.92:40370) 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 -- 172.21.15.92:0/2372017010 learned_addr learned my addr 172.21.15.92:0/2372017010 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83f987b700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ee6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa07c700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea6840 0x55f202f53750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 -- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea6840 msgr2=0x55f202f53750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea6840 0x55f202f53750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 -- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 msgr2=0x55f202ee6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ee6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.357+0000 7f83fa87d700 1 -- 172.21.15.92:0/2372017010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f40ce720 con 0x55f202ef6c00 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83fa87d700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee7020 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00a3f0 tx=0x7f83ec01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf29508e066d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 1 -- 172.21.15.92:0/2372017010 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0558c0 con 0x55f202ef6c00 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 1 -- 172.21.15.92:0/2372017010 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83ec054db0 con 0x55f202ef6c00 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83f0019070 con 0x55f202ef6c00 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 1 -- 172.21.15.92:0/2372017010 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83ec05c3d0 con 0x55f202ef6c00 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 10 client.5535 fetch_fsmap learned FSMap version 29 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55f202dac470 con 0x55f202ef6c00 2023-10-06T03:03:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 1 -- 172.21.15.92:0/2372017010 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0019070 con 0x55f202ef6c00 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 1 -- 172.21.15.92:0/2372017010 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f83ec004bb0 con 0x55f202ef6c00 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 10 client.5535 fetch_fsmap finished waiting for FSMap version 29 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 10 client.5535 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 20 client.5535 populate_metadata read hostname 'smithi092' 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ea2af0 0x55f202ea2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 4 client.5535 mds_command: new command op to 5364 tid=0[{"prefix": "status"}] 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 --> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f202d99e00 con 0x55f202ea2af0 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83fa07c700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ea2af0 0x55f202ea2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83d4ff9700 10 client.5535 ms_handle_connect on v2:172.21.15.92:6835/3249599548 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.358+0000 7f83fa07c700 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ea2af0 0x55f202ea2ec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e5be2f18bb339d0 server_cookie=73f94d1998861513 in_seq=0 out_seq=0 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83d4ff9700 1 -- 172.21.15.92:0/2372017010 <== mds.0 v2:172.21.15.92:6835/3249599548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f202d99e00 con 0x55f202ea2af0 2023-10-06T03:03:08.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83d4ff9700 10 client.5535 handle_command_reply: tid=0 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 client.5535 shutdown 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 msgr2=0x55f202ee7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee7020 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00a3f0 tx=0x7f83ec01d000 comp rx=0 tx=0).stop 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 shutdown_connections 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:6835/3249599548,v1:172.21.15.92:6837/3249599548] conn(0x55f202ea2af0 0x55f202ea2ec0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f202ef6c00 0x55f202ee7020 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83d4ff9700 0 client.5535 ms_handle_reset on v2:172.21.15.92:6835/3249599548 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f202ea6840 0x55f202f53750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 --2- 172.21.15.92:0/2372017010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f202ea7cc0 0x55f202ee6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 >> 172.21.15.92:0/2372017010 conn(0x55f202ea8160 msgr2=0x55f202ef55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 shutdown_connections 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 1 -- 172.21.15.92:0/2372017010 wait complete. 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f84055d8b80 20 client.5535 trim_cache size 0 max 16384 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5364, 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:03:08.369 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:03:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.8029172339999997, 2023-10-06T03:03:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 29, 2023-10-06T03:03:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 54, 2023-10-06T03:03:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 54, 2023-10-06T03:03:08.370 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.916962599 2023-10-06T03:03:08.370 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d8066f30 msgr2=0x7f83d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d8066f30 0x7f83d80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f83f0006fd0 tx=0x7f83f0008040 comp rx=0 tx=0).stop 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 msgr2=0x7f83f4152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.359+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 0x7f83f4152300 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00cbf0 tx=0x7f83ec0019b0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 shutdown_connections 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83d8066f30 0x7f83d80693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83f415db10 0x7f83f4152300 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f4159890 0x7f83f40cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 --2- 172.21.15.92:0/673280237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83f4159c60 0x7f83f414d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 >> 172.21.15.92:0/673280237 conn(0x7f83f4068e20 msgr2=0x7f83f4069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 shutdown_connections 2023-10-06T03:03:08.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.360+0000 7f83fc2e0700 1 -- 172.21.15.92:0/673280237 wait complete. 2023-10-06T03:03:08.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:03:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.485+0000 7f6eee045700 1 Processor -- start 2023-10-06T03:03:08.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.485+0000 7f6eee045700 1 -- start start 2023-10-06T03:03:08.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6eee045700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6eee045700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6eee045700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee815a160 con 0x7f6ee8159890 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6eee045700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee815df50 con 0x7f6ee8159c60 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6eee045700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80d13a0 con 0x7f6ee815db10 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ee8159890 0x7f6ee815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:34400/0 (socket says 172.21.15.92:34400) 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee77fe700 1 -- 172.21.15.92:0/712209802 learned_addr learned my addr 172.21.15.92:0/712209802 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee6ffd700 1 --2- 172.21.15.92:0/712209802 >> v2:172.21.15.92:3300/0 conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee7fff700 1 --2- 172.21.15.92:0/712209802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee67fc700 1 -- 172.21.15.92:0/712209802 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948213320 0 0) 0x7f6ee815a160 con 0x7f6ee8159890 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee67fc700 1 -- 172.21.15.92:0/712209802 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed0003570 con 0x7f6ee8159890 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/712209802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee815db10 msgr2=0x7f6ee8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee6ffd700 1 --2- 172.21.15.92:0/712209802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/712209802 >> v1:172.21.15.92:6789/0 conn(0x7f6ee8159890 legacy=0x7f6ee815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.486+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/712209802 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee815e140 con 0x7f6ee8159c60 2023-10-06T03:03:08.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6ee6ffd700 1 --2- 172.21.15.92:0/712209802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0098e0 tx=0x7f6edc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3496b36ed1626454 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6ee67fc700 1 -- 172.21.15.92:0/712209802 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc014ca0 con 0x7f6ee8159c60 2023-10-06T03:03:08.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6ee67fc700 1 -- 172.21.15.92:0/712209802 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc014e00 con 0x7f6ee8159c60 2023-10-06T03:03:08.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6ee67fc700 1 -- 172.21.15.92:0/712209802 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01c670 con 0x7f6ee8159c60 2023-10-06T03:03:08.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 -- 172.21.15.92:0/712209802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 msgr2=0x7f6ee815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 --2- 172.21.15.92:0/712209802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0098e0 tx=0x7f6edc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 -- 172.21.15.92:0/712209802 shutdown_connections 2023-10-06T03:03:08.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 --2- 172.21.15.92:0/712209802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 --2- 172.21.15.92:0/712209802 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 -- 172.21.15.92:0/712209802 >> 172.21.15.92:0/712209802 conn(0x7f6ee8068e20 msgr2=0x7f6ee80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 -- 172.21.15.92:0/712209802 shutdown_connections 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.487+0000 7f6eee045700 1 -- 172.21.15.92:0/712209802 wait complete. 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 Processor -- start 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 -- start start 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee8159890 0x7f6ee80c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee815db10 0x7f6ee80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80b0540 con 0x7f6ee8159c60 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80b10c0 con 0x7f6ee815db10 2023-10-06T03:03:08.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6eee045700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80b0790 con 0x7f6ee8159890 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6ee6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6ee6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee80c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40402/0 (socket says 172.21.15.92:40402) 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.489+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/188552578 learned_addr learned my addr 172.21.15.92:0/188552578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee77fe700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee8159890 0x7f6ee80c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee7fff700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee815db10 0x7f6ee80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee8159890 msgr2=0x7f6ee80c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee6ffd700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee8159890 0x7f6ee80c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee815db10 msgr2=0x7f6ee80c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee6ffd700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee815db10 0x7f6ee80c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee6ffd700 1 -- 172.21.15.92:0/188552578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009420 con 0x7f6ee8159c60 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ee6ffd700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee80c9a40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80899c0 tx=0x7f6edc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01b800 con 0x7f6ee8159c60 2023-10-06T03:03:08.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc01b960 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee80b0a70 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01bd60 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.490+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee80ca4c0 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.491+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee804c3c0 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.491+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6edc023070 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.492+0000 7f6ecf7fe700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ed4066f30 0x7f6ed40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.492+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6edc08dd30 con 0x7f6ee8159c60 2023-10-06T03:03:08.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.492+0000 7f6ee77fe700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ed4066f30 0x7f6ed40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:08.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.492+0000 7f6ee77fe700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ed4066f30 0x7f6ed40693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80058d0 tx=0x7f6ed8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:08.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.496+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6edc05aec0 con 0x7f6ee8159c60 2023-10-06T03:03:08.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.723+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ee80b1200 con 0x7f6ee8159c60 2023-10-06T03:03:08.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.724+0000 7f6ecf7fe700 1 -- 172.21.15.92:0/188552578 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6edc05e4e0 con 0x7f6ee8159c60 2023-10-06T03:03:08.725 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ed4066f30 msgr2=0x7f6ed40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ed4066f30 0x7f6ed40693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80058d0 tx=0x7f6ed8009040 comp rx=0 tx=0).stop 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 msgr2=0x7f6ee80c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee80c9a40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80899c0 tx=0x7f6edc01c000 comp rx=0 tx=0).stop 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 shutdown_connections 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ed4066f30 0x7f6ed40693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee815db10 0x7f6ee80c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee8159890 0x7f6ee80c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 --2- 172.21.15.92:0/188552578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ee8159c60 0x7f6ee80c9a40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 >> 172.21.15.92:0/188552578 conn(0x7f6ee8068e20 msgr2=0x7f6ee814e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:08.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 shutdown_connections 2023-10-06T03:03:08.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:08.726+0000 7f6eee045700 1 -- 172.21.15.92:0/188552578 wait complete. 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:03:08.740 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:03:08.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:08.741 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:03:08.756 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:03:08.756 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:08.757 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42417sec preferred_lft 42417sec 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.758 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:08.759 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:03:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:03:08.760 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:08.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:08.760 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:03:08.820 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:03:08.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:08.821 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:03:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:03:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:03:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:03:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:03:08.877 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:03:08.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:08.878 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:03:08.878 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:03:08.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:03:08.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:03:08.878 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:03:08.878 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:03:08.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:08.979 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:03:08.994 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:03:08.994 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:03:08.994 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:03:08.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:08.995 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:03:09.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:09.058 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:09.113 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:03:09.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:09.113 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:09.171 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:03:09.171 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:03:09.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:09.251 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:09.251 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:09.269 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:09.270 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:09.292 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:09.292 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:09.292 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:09.292 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:09.292 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:09.292 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:09.293 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:09.293 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:09.293 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:03:09.293 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:03:09.293 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:03:09.294 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:03:09.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:09.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:09.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:03:09.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.511+0000 7f3cfcde5700 1 Processor -- start 2023-10-06T03:03:09.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.511+0000 7f3cfcde5700 1 -- start start 2023-10-06T03:03:09.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cfcde5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159c60 0x7f3cf815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cfcde5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cf815db10 0x7f3cf8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cfcde5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf815a160 con 0x7f3cf8159890 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cfcde5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf80d13a0 con 0x7f3cf815db10 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cfcde5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf815e140 con 0x7f3cf8159c60 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cf6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cf815db10 0x7f3cf8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cf6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cf815db10 0x7f3cf8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46248/0 (socket says 172.21.15.92:46248) 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/1948951024 learned_addr learned my addr 172.21.15.92:0/1948951024 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:09.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cedd9b700 1 --2- 172.21.15.92:0/1948951024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159c60 0x7f3cf815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/1948951024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159c60 msgr2=0x7f3cf815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.512+0000 7f3cf6d9d700 1 --2- 172.21.15.92:0/1948951024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159c60 0x7f3cf815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/1948951024 >> v1:172.21.15.136:6789/0 conn(0x7f3cf8159890 legacy=0x7f3cf815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/1948951024 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf815a3d0 con 0x7f3cf815db10 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cf6d9d700 1 --2- 172.21.15.92:0/1948951024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf8069560 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009f20 tx=0x7f3cf0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b5ce482d47a72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cf5d9b700 1 -- 172.21.15.92:0/1948951024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000e070 con 0x7f3cf815db10 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cf5d9b700 1 -- 172.21.15.92:0/1948951024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000ba80 con 0x7f3cf815db10 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cf5d9b700 1 -- 172.21.15.92:0/1948951024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00092e0 con 0x7f3cf815db10 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 -- 172.21.15.92:0/1948951024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 msgr2=0x7f3cf8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:09.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/1948951024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf8069560 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009f20 tx=0x7f3cf0009840 comp rx=0 tx=0).stop 2023-10-06T03:03:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 -- 172.21.15.92:0/1948951024 shutdown_connections 2023-10-06T03:03:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/1948951024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf8069560 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/1948951024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159c60 0x7f3cf815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 -- 172.21.15.92:0/1948951024 >> 172.21.15.92:0/1948951024 conn(0x7f3cf8068e20 msgr2=0x7f3cf80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.513+0000 7f3cfcde5700 1 -- 172.21.15.92:0/1948951024 shutdown_connections 2023-10-06T03:03:09.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.514+0000 7f3cfcde5700 1 -- 172.21.15.92:0/1948951024 wait complete. 2023-10-06T03:03:09.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.515+0000 7f3cfcde5700 1 Processor -- start 2023-10-06T03:03:09.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.515+0000 7f3cfcde5700 1 -- start start 2023-10-06T03:03:09.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159890 0x7f3cf80cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:09.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cf8159c60 0x7f3cf80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:09.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf816d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf816abd0 con 0x7f3cf815db10 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf8169f50 con 0x7f3cf8159c60 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf816a230 con 0x7f3cf8159890 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf816d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf816d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46250/0 (socket says 172.21.15.92:46250) 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/2127104552 learned_addr learned my addr 172.21.15.92:0/2127104552 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cedd9b700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cf8159c60 0x7f3cf80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf659c700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159890 0x7f3cf80cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159890 msgr2=0x7f3cf80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159890 0x7f3cf80cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cf8159c60 msgr2=0x7f3cf80cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cf8159c60 0x7f3cf80cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 -- 172.21.15.92:0/2127104552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009ae0 con 0x7f3cf815db10 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cf6d9d700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf816d000 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000b6c0 tx=0x7f3cf0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf00094e0 con 0x7f3cf815db10 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf00032f0 con 0x7f3cf815db10 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0025460 con 0x7f3cf815db10 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf816a510 con 0x7f3cf815db10 2023-10-06T03:03:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.516+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf80c74f0 con 0x7f3cf815db10 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.517+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf804c3c0 con 0x7f3cf815db10 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.518+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cf0003040 con 0x7f3cf815db10 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.518+0000 7f3ceeffd700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cd8066f30 0x7f3cd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.518+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3cf00861f0 con 0x7f3cf815db10 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.518+0000 7f3cf659c700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cd8066f30 0x7f3cd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.519+0000 7f3cf659c700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cd8066f30 0x7f3cd80693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8003a80 tx=0x7f3ce8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.522+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf0053300 con 0x7f3cf815db10 2023-10-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.753+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3cf80b0120 con 0x7f3cf815db10 2023-10-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.754+0000 7f3ceeffd700 1 -- 172.21.15.92:0/2127104552 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3cf0056920 con 0x7f3cf815db10 2023-10-06T03:03:09.755 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cd8066f30 msgr2=0x7f3cd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cd8066f30 0x7f3cd80693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8003a80 tx=0x7f3ce8007ec0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 msgr2=0x7f3cf816d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf816d000 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000b6c0 tx=0x7f3cf0018000 comp rx=0 tx=0).stop 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 shutdown_connections 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cd8066f30 0x7f3cd80693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cf815db10 0x7f3cf816d000 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cf8159890 0x7f3cf80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 --2- 172.21.15.92:0/2127104552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cf8159c60 0x7f3cf80cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 >> 172.21.15.92:0/2127104552 conn(0x7f3cf8068e20 msgr2=0x7f3cf8152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 shutdown_connections 2023-10-06T03:03:09.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:09.756+0000 7f3cfcde5700 1 -- 172.21.15.92:0/2127104552 wait complete. 2023-10-06T03:03:09.769 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:03:09.769 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:03:09.769 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:03:09.769 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:03:09.769 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:09.770 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:03:09.770 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:03:09.770 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:03:09.770 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:03:09.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:09.770 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:09.789 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:03:09.790 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42418sec preferred_lft 42418sec 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:03:09.791 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.792 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:09.792 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:03:09.792 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:03:09.792 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:09.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:09.792 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:03:09.858 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:03:09.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:09.860 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:03:09.918 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:03:09.919 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:03:09.919 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:03:09.919 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:03:09.919 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:03:09.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:09.919 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:03:09.919 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:03:09.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:03:09.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:03:09.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:03:09.920 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:03:10.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:10.025 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:03:10.040 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:03:10.040 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:03:10.040 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:03:10.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:10.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:03:10.105 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:10.106 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:10.160 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:03:10.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:10.161 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:10.215 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:03:10.215 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:03:10.215 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:10.295 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:10.295 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:10.314 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:10.315 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:03:10.337 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:03:10.338 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:03:10.338 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:03:10.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:10.339 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:10.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:03:10.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.542+0000 7f430d133700 1 Processor -- start 2023-10-06T03:03:10.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.542+0000 7f430d133700 1 -- start start 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.542+0000 7f430d133700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43080b55a0 0x7f4308164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430d133700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43080cfa60 con 0x7f43080cd910 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430d133700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43080b5b00 con 0x7f4308159ec0 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430d133700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43080b5da0 con 0x7f43080b55a0 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43080b55a0 0x7f4308164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f42fe59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f43080cd910 0x7f43080ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40650/0 (socket says 172.21.15.92:40650) 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f42fe59c700 1 -- 172.21.15.92:0/2950005564 learned_addr learned my addr 172.21.15.92:0/2950005564 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430759e700 1 -- 172.21.15.92:0/2950005564 >> v1:172.21.15.136:6789/0 conn(0x7f4308159ec0 legacy=0x7f43080b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430759e700 1 -- 172.21.15.92:0/2950005564 >> v1:172.21.15.92:6789/0 conn(0x7f43080cd910 legacy=0x7f43080ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430759e700 1 -- 172.21.15.92:0/2950005564 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43080b6080 con 0x7f43080b55a0 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430659c700 1 -- 172.21.15.92:0/2950005564 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458095054 0 0) 0x7f43080cfa60 con 0x7f43080cd910 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430759e700 1 --2- 172.21.15.92:0/2950005564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080b55a0 0x7f4308164400 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4300009e90 tx=0x7f4300009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2bb2a2f50e7016 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430659c700 1 -- 172.21.15.92:0/2950005564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430000e070 con 0x7f43080b55a0 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430659c700 1 -- 172.21.15.92:0/2950005564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430000bbf0 con 0x7f43080b55a0 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.543+0000 7f430659c700 1 -- 172.21.15.92:0/2950005564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300012430 con 0x7f43080b55a0 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 -- 172.21.15.92:0/2950005564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080b55a0 msgr2=0x7f4308164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 --2- 172.21.15.92:0/2950005564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080b55a0 0x7f4308164400 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4300009e90 tx=0x7f4300009780 comp rx=0 tx=0).stop 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 -- 172.21.15.92:0/2950005564 shutdown_connections 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 --2- 172.21.15.92:0/2950005564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080b55a0 0x7f4308164400 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:10.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 -- 172.21.15.92:0/2950005564 >> 172.21.15.92:0/2950005564 conn(0x7f43080999f0 msgr2=0x7f43080b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:10.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 -- 172.21.15.92:0/2950005564 shutdown_connections 2023-10-06T03:03:10.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.544+0000 7f430d133700 1 -- 172.21.15.92:0/2950005564 wait complete. 2023-10-06T03:03:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.545+0000 7f430d133700 1 Processor -- start 2023-10-06T03:03:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.545+0000 7f430d133700 1 -- start start 2023-10-06T03:03:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430d133700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43080b55a0 0x7f4308147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:10.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430d133700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4308159ec0 0x7f4308157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430d133700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 0x7f4308158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430d133700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430806a980 con 0x7f43080d0c10 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430d133700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43081477a0 con 0x7f4308159ec0 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430d133700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430806ae10 con 0x7f43080b55a0 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 0x7f4308158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 0x7f4308158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46264/0 (socket says 172.21.15.92:46264) 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f430759e700 1 -- 172.21.15.92:0/2950907669 learned_addr learned my addr 172.21.15.92:0/2950907669 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f42fe59c700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4308159ec0 0x7f4308157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.546+0000 7f4306d9d700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43080b55a0 0x7f4308147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430759e700 1 -- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43080b55a0 msgr2=0x7f4308147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430759e700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43080b55a0 0x7f4308147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:10.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430759e700 1 -- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4308159ec0 msgr2=0x7f4308157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430759e700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4308159ec0 0x7f4308157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430759e700 1 -- 172.21.15.92:0/2950907669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4300009a20 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430759e700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 0x7f4308158150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43000099f0 tx=0x7f4300019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430001b420 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430001b580 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430806a050 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430001b920 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.547+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430806a610 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.548+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4308063c90 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.548+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f430001bc90 con 0x7f43080d0c10 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.549+0000 7f42ff7fe700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e8066f30 0x7f42e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.549+0000 7f4306d9d700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e8066f30 0x7f42e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:10.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.549+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4300086520 con 0x7f43080d0c10 2023-10-06T03:03:10.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.549+0000 7f4306d9d700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e8066f30 0x7f42e80693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007fd0 tx=0x7f42f8007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.553+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4300053630 con 0x7f43080d0c10 2023-10-06T03:03:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:10.779+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f43080718b0 con 0x7f43080d0c10 2023-10-06T03:03:11.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.600+0000 7f42ff7fe700 1 -- 172.21.15.92:0/2950907669 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 136+0+0 (secure 0 0 0) 0x7f4300056c50 con 0x7f43080d0c10 2023-10-06T03:03:11.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e8066f30 msgr2=0x7f42e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e8066f30 0x7f42e80693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f42f8007fd0 tx=0x7f42f8007eb0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 msgr2=0x7f4308158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 0x7f4308158150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43000099f0 tx=0x7f4300019000 comp rx=0 tx=0).stop 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 shutdown_connections 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e8066f30 0x7f42e80693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43080d0c10 0x7f4308158150 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43080b55a0 0x7f4308147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 --2- 172.21.15.92:0/2950907669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4308159ec0 0x7f4308157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 >> 172.21.15.92:0/2950907669 conn(0x7f43080999f0 msgr2=0x7f43080c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 shutdown_connections 2023-10-06T03:03:11.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.602+0000 7f430d133700 1 -- 172.21.15.92:0/2950907669 wait complete. 2023-10-06T03:03:11.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:03:11.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.736+0000 7ff5d2d7a700 1 Processor -- start 2023-10-06T03:03:11.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.736+0000 7ff5d2d7a700 1 -- start start 2023-10-06T03:03:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d2d7a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5cc159890 0x7ff5cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d2d7a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc159c60 0x7ff5cc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d2d7a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc15a160 con 0x7ff5cc15db10 2023-10-06T03:03:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d2d7a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc0d13a0 con 0x7ff5cc159890 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d2d7a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc15e140 con 0x7ff5cc159c60 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d0b16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff5cc159890 0x7ff5cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d1317700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff5cc15db10 0x7ff5cc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40656/0 (socket says 172.21.15.92:40656) 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d1317700 1 -- 172.21.15.92:0/3536163094 learned_addr learned my addr 172.21.15.92:0/3536163094 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5c3fff700 1 --2- 172.21.15.92:0/3536163094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc159c60 0x7ff5cc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d0b16700 1 -- 172.21.15.92:0/3536163094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc159c60 msgr2=0x7ff5cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d0b16700 1 --2- 172.21.15.92:0/3536163094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc159c60 0x7ff5cc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d0b16700 1 -- 172.21.15.92:0/3536163094 >> v1:172.21.15.92:6789/0 conn(0x7ff5cc15db10 legacy=0x7ff5cc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5d0b16700 1 -- 172.21.15.92:0/3536163094 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc15a3d0 con 0x7ff5cc159890 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.737+0000 7ff5c37fe700 1 -- 172.21.15.92:0/3536163094 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222746314 0 0) 0x7ff5cc15a160 con 0x7ff5cc15db10 2023-10-06T03:03:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d0b16700 1 --2- 172.21.15.92:0/3536163094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159890 0x7ff5cc15abd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009e60 tx=0x7ff5c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3805dbb7315c632 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5c37fe700 1 -- 172.21.15.92:0/3536163094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800e070 con 0x7ff5cc159890 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5c37fe700 1 -- 172.21.15.92:0/3536163094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800bc30 con 0x7ff5cc159890 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5c37fe700 1 -- 172.21.15.92:0/3536163094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8012430 con 0x7ff5cc159890 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/3536163094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159890 msgr2=0x7ff5cc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/3536163094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159890 0x7ff5cc15abd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009e60 tx=0x7ff5c8009780 comp rx=0 tx=0).stop 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/3536163094 shutdown_connections 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/3536163094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159890 0x7ff5cc15abd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/3536163094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc159c60 0x7ff5cc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/3536163094 >> 172.21.15.92:0/3536163094 conn(0x7ff5cc068e20 msgr2=0x7ff5cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/3536163094 shutdown_connections 2023-10-06T03:03:11.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.738+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/3536163094 wait complete. 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.739+0000 7ff5d2d7a700 1 Processor -- start 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.739+0000 7ff5d2d7a700 1 -- start start 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d2d7a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5cc159890 0x7ff5cc14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d2d7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 0x7ff5cc16a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d2d7a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc15db10 0x7ff5cc14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d2d7a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0cd2e0 con 0x7ff5cc159c60 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d2d7a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc16bbf0 con 0x7ff5cc159890 2023-10-06T03:03:11.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d2d7a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc16afe0 con 0x7ff5cc15db10 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 0x7ff5cc16a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 0x7ff5cc16a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46290/0 (socket says 172.21.15.92:46290) 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5c3fff700 1 -- 172.21.15.92:0/1426781754 learned_addr learned my addr 172.21.15.92:0/1426781754 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d1317700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc15db10 0x7ff5cc14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.740+0000 7ff5d0b16700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5cc159890 0x7ff5cc14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c3fff700 1 -- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc15db10 msgr2=0x7ff5cc14e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c3fff700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc15db10 0x7ff5cc14e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c3fff700 1 -- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5cc159890 msgr2=0x7ff5cc14b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c3fff700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5cc159890 0x7ff5cc14b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c3fff700 1 -- 172.21.15.92:0/1426781754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009a20 con 0x7ff5cc159c60 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5d1317700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc15db10 0x7ff5cc14e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:11.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5d0b16700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5cc159890 0x7ff5cc14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c3fff700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 0x7ff5cc16a560 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c880 tx=0x7ff5bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc011600 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc011780 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc16b320 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc011b40 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.741+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc1513c0 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.742+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc16bd60 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.742+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5bc021350 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.743+0000 7ff5c0ff9700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5b8066f30 0x7ff5b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.743+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5bc0843a0 con 0x7ff5cc159c60 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.743+0000 7ff5d0b16700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5b8066f30 0x7ff5b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:11.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.743+0000 7ff5d0b16700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5b8066f30 0x7ff5b80693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009e60 tx=0x7ff5c8019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.747+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5bc0514b0 con 0x7ff5cc159c60 2023-10-06T03:03:11.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:03:11.791 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:03:11.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.959+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5cc04c3c0 con 0x7ff5b8066f30 2023-10-06T03:03:11.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.960+0000 7ff5c0ff9700 1 -- 172.21.15.92:0/1426781754 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff5cc04c3c0 con 0x7ff5b8066f30 2023-10-06T03:03:11.962 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:03:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:03:11.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:11.963 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:03:11.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5b8066f30 msgr2=0x7ff5b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5b8066f30 0x7ff5b80693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009e60 tx=0x7ff5c8019000 comp rx=0 tx=0).stop 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 msgr2=0x7ff5cc16a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 0x7ff5cc16a560 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00c880 tx=0x7ff5bc008040 comp rx=0 tx=0).stop 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 shutdown_connections 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5b8066f30 0x7ff5b80693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5cc15db10 0x7ff5cc14e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5cc159890 0x7ff5cc14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 --2- 172.21.15.92:0/1426781754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff5cc159c60 0x7ff5cc16a560 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 >> 172.21.15.92:0/1426781754 conn(0x7ff5cc068e20 msgr2=0x7ff5cc0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:11.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 shutdown_connections 2023-10-06T03:03:11.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:11.962+0000 7ff5d2d7a700 1 -- 172.21.15.92:0/1426781754 wait complete. 2023-10-06T03:03:11.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.090+0000 7f7fd66c4700 1 Processor -- start 2023-10-06T03:03:12.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.090+0000 7f7fd66c4700 1 -- start start 2023-10-06T03:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.091+0000 7f7fd66c4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7fd0159890 0x7f7fd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.091+0000 7f7fd66c4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0069560 0x7f7fd00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.091+0000 7f7fd66c4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd0069ee0 con 0x7f7fd0159c60 2023-10-06T03:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.091+0000 7f7fd66c4700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd015a160 con 0x7f7fd0159890 2023-10-06T03:03:12.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.091+0000 7f7fd66c4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00d13a0 con 0x7f7fd0069560 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fcffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7fd0159890 0x7f7fd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fd4c61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0069560 0x7f7fd00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fcf7fe700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f7fd0159c60 0x7f7fd015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:44290/0 (socket says 172.21.15.92:44290) 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fcf7fe700 1 -- 172.21.15.92:0/341362635 learned_addr learned my addr 172.21.15.92:0/341362635 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fd4c61700 1 -- 172.21.15.92:0/341362635 >> v2:172.21.15.136:3300/0 conn(0x7f7fd0159890 msgr2=0x7f7fd015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fd4c61700 1 --2- 172.21.15.92:0/341362635 >> v2:172.21.15.136:3300/0 conn(0x7f7fd0159890 0x7f7fd015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fd4c61700 1 -- 172.21.15.92:0/341362635 >> v1:172.21.15.136:6789/0 conn(0x7f7fd0159c60 legacy=0x7f7fd015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fd4c61700 1 -- 172.21.15.92:0/341362635 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd015e140 con 0x7f7fd0069560 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fcffff700 1 --2- 172.21.15.92:0/341362635 >> v2:172.21.15.136:3300/0 conn(0x7f7fd0159890 0x7f7fd015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fd4c61700 1 --2- 172.21.15.92:0/341362635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0069560 0x7f7fd00699a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009e60 tx=0x7f7fc8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64891b5cb8a3ed9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.092+0000 7f7fceffd700 1 -- 172.21.15.92:0/341362635 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800e070 con 0x7f7fd0069560 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fceffd700 1 -- 172.21.15.92:0/341362635 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc800bb60 con 0x7f7fd0069560 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fceffd700 1 -- 172.21.15.92:0/341362635 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8012430 con 0x7f7fd0069560 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 -- 172.21.15.92:0/341362635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0069560 msgr2=0x7f7fd00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/341362635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0069560 0x7f7fd00699a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009e60 tx=0x7f7fc8009780 comp rx=0 tx=0).stop 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 -- 172.21.15.92:0/341362635 shutdown_connections 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/341362635 >> v2:172.21.15.136:3300/0 conn(0x7f7fd0159890 0x7f7fd015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/341362635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0069560 0x7f7fd00699a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 -- 172.21.15.92:0/341362635 >> 172.21.15.92:0/341362635 conn(0x7f7fd0068e20 msgr2=0x7f7fd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 -- 172.21.15.92:0/341362635 shutdown_connections 2023-10-06T03:03:12.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.093+0000 7f7fd66c4700 1 -- 172.21.15.92:0/341362635 wait complete. 2023-10-06T03:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.094+0000 7f7fd66c4700 1 Processor -- start 2023-10-06T03:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.094+0000 7f7fd66c4700 1 -- start start 2023-10-06T03:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd66c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 0x7f7fd00b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd66c4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd0159890 0x7f7fd00b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd66c4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0159c60 0x7f7fd0151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd66c4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00b6060 con 0x7f7fd0069560 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd66c4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00cd1e0 con 0x7f7fd0159890 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd66c4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00c78a0 con 0x7f7fd0159c60 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fcffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 0x7f7fd00b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fcffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 0x7f7fd00b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46306/0 (socket says 172.21.15.92:46306) 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fcffff700 1 -- 172.21.15.92:0/2632913783 learned_addr learned my addr 172.21.15.92:0/2632913783 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fd4c61700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0159c60 0x7f7fd0151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.095+0000 7f7fcf7fe700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd0159890 0x7f7fd00b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fcffff700 1 -- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0159c60 msgr2=0x7f7fd0151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fcffff700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0159c60 0x7f7fd0151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fcffff700 1 -- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd0159890 msgr2=0x7f7fd00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fcffff700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd0159890 0x7f7fd00b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fcffff700 1 -- 172.21.15.92:0/2632913783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0009420 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fcffff700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 0x7f7fd00b36e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ec80 tx=0x7f7fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000c920 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000ca80 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000ce00 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc8009a20 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.096+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd00cd430 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.097+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd004c3c0 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.097+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fc0013680 con 0x7f7fd0069560 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.098+0000 7f7fb7fff700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fbc066f30 0x7f7fbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.098+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7fc0084a20 con 0x7f7fd0069560 2023-10-06T03:03:12.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.098+0000 7f7fcf7fe700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fbc066f30 0x7f7fbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.098+0000 7f7fcf7fe700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fbc066f30 0x7f7fbc0693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000c00 tx=0x7f7fc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.102+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fc0051b30 con 0x7f7fd0069560 2023-10-06T03:03:12.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.283+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7fd00b1ff0 con 0x7f7fd0069560 2023-10-06T03:03:12.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.283+0000 7f7fb7fff700 1 -- 172.21.15.92:0/2632913783 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7fc0055150 con 0x7f7fd0069560 2023-10-06T03:03:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fbc066f30 msgr2=0x7f7fbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fbc066f30 0x7f7fbc0693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000c00 tx=0x7f7fc4009040 comp rx=0 tx=0).stop 2023-10-06T03:03:12.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 msgr2=0x7f7fd00b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 0x7f7fd00b36e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ec80 tx=0x7f7fc00091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 shutdown_connections 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7fbc066f30 0x7f7fbc0693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fd0069560 0x7f7fd00b36e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd0159c60 0x7f7fd0151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 --2- 172.21.15.92:0/2632913783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd0159890 0x7f7fd00b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.285+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 >> 172.21.15.92:0/2632913783 conn(0x7f7fd0068e20 msgr2=0x7f7fd0152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.286+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 shutdown_connections 2023-10-06T03:03:12.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.286+0000 7f7fd66c4700 1 -- 172.21.15.92:0/2632913783 wait complete. 2023-10-06T03:03:12.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000778789 2023-10-06T03:03:12.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.414+0000 7f1570c16700 1 Processor -- start 2023-10-06T03:03:12.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.414+0000 7f1570c16700 1 -- start start 2023-10-06T03:03:12.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f1570c16700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b16d0 0x7f156c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f1570c16700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0b2080 con 0x7f156c0b33b0 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f1570c16700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0cfb00 con 0x7f156c0b4e70 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f1570c16700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0b5c40 con 0x7f156c0b16d0 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f156bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f156c0b33b0 0x7f156c0b3830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40662/0 (socket says 172.21.15.92:40662) 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f156bfff700 1 -- 172.21.15.92:0/2833741343 learned_addr learned my addr 172.21.15.92:0/2833741343 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f156affd700 1 --2- 172.21.15.92:0/2833741343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b16d0 0x7f156c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445545057 0 0) 0x7f156c0b2080 con 0x7f156c0b33b0 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.415+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003570 con 0x7f156c0b33b0 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186998824 0 0) 0x7f156c0cfb00 con 0x7f156c0b4e70 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c0b2080 con 0x7f156c0b4e70 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717105095 0 0) 0x7f1554003570 con 0x7f156c0b33b0 2023-10-06T03:03:12.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156affd700 1 -- 172.21.15.92:0/2833741343 >> v1:172.21.15.136:6789/0 conn(0x7f156c0b4e70 legacy=0x7f156c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156affd700 1 -- 172.21.15.92:0/2833741343 >> v1:172.21.15.92:6789/0 conn(0x7f156c0b33b0 legacy=0x7f156c0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156affd700 1 -- 172.21.15.92:0/2833741343 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0b5f20 con 0x7f156c0b16d0 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156affd700 1 --2- 172.21.15.92:0/2833741343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b16d0 0x7f156c0b1b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1560009910 tx=0x7f15600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984eedb0ea74aff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640013f0 con 0x7f156c0b16d0 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1560011420 con 0x7f156c0b16d0 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.416+0000 7f156a7fc700 1 -- 172.21.15.92:0/2833741343 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600117a0 con 0x7f156c0b16d0 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 -- 172.21.15.92:0/2833741343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b16d0 msgr2=0x7f156c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 --2- 172.21.15.92:0/2833741343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b16d0 0x7f156c0b1b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1560009910 tx=0x7f15600091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 -- 172.21.15.92:0/2833741343 shutdown_connections 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 --2- 172.21.15.92:0/2833741343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b16d0 0x7f156c0b1b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 -- 172.21.15.92:0/2833741343 >> 172.21.15.92:0/2833741343 conn(0x7f156c0d10c0 msgr2=0x7f156c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:12.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 -- 172.21.15.92:0/2833741343 shutdown_connections 2023-10-06T03:03:12.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.417+0000 7f1570c16700 1 -- 172.21.15.92:0/2833741343 wait complete. 2023-10-06T03:03:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.418+0000 7f1570c16700 1 Processor -- start 2023-10-06T03:03:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.418+0000 7f1570c16700 1 -- start start 2023-10-06T03:03:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 0x7f156c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b33b0 0x7f156c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f156c0b4e70 0x7f156c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0a70a0 con 0x7f156c0b16d0 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0c90c0 con 0x7f156c0b4e70 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0ca7a0 con 0x7f156c0b33b0 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 0x7f156c0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 0x7f156c0c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46320/0 (socket says 172.21.15.92:46320) 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 -- 172.21.15.92:0/2112459317 learned_addr learned my addr 172.21.15.92:0/2112459317 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156bfff700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f156c0b4e70 0x7f156c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156affd700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b33b0 0x7f156c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 -- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b33b0 msgr2=0x7f156c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b33b0 0x7f156c1561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 -- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f156c0b4e70 msgr2=0x7f156c0c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f156c0b4e70 0x7f156c0c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 -- 172.21.15.92:0/2112459317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560009420 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156bfff700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f156c0b4e70 0x7f156c0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f156b7fe700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 0x7f156c0c8b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c960 tx=0x7f155c0027b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00a7a0 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00a920 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c156780 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00ace0 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.419+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c16a580 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.420+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c088990 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.421+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f155c012630 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.421+0000 7f15537fe700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1558066f30 0x7f15580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.421+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f155c084ba0 con 0x7f156c0b16d0 2023-10-06T03:03:12.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.421+0000 7f156affd700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1558066f30 0x7f15580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.422+0000 7f156affd700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1558066f30 0x7f15580693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f156c159840 tx=0x7f1560016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.425+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f155c051d60 con 0x7f156c0b16d0 2023-10-06T03:03:12.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.640+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000778789", "target": ["mon-mgr", ""]}) v1 -- 0x7f156c04bb80 con 0x7f1558066f30 2023-10-06T03:03:12.645 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:12.643+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:12.645 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:12.643+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:12.645 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:12.643+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:12.645 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:12.643+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:12.645 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:12.643+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:12.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.657+0000 7f15537fe700 1 -- 172.21.15.92:0/2112459317 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f156c04bb80 con 0x7f1558066f30 2023-10-06T03:03:12.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1558066f30 msgr2=0x7f15580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1558066f30 0x7f15580693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f156c159840 tx=0x7f1560016000 comp rx=0 tx=0).stop 2023-10-06T03:03:12.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 msgr2=0x7f156c0c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 0x7f156c0c8b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c960 tx=0x7f155c0027b0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 shutdown_connections 2023-10-06T03:03:12.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1558066f30 0x7f15580693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f156c0b16d0 0x7f156c0c8b80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f156c0b4e70 0x7f156c0c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 --2- 172.21.15.92:0/2112459317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156c0b33b0 0x7f156c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 >> 172.21.15.92:0/2112459317 conn(0x7f156c0d10c0 msgr2=0x7f156c159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 shutdown_connections 2023-10-06T03:03:12.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.660+0000 7f1570c16700 1 -- 172.21.15.92:0/2112459317 wait complete. 2023-10-06T03:03:12.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-06T03:03:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.788+0000 7fae785cd700 1 Processor -- start 2023-10-06T03:03:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.788+0000 7fae785cd700 1 -- start start 2023-10-06T03:03:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae785cd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae70159890 0x7fae7015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae785cd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fae70159c60 0x7fae7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae785cd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae7015db10 0x7fae70069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae785cd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7015df50 con 0x7fae70159890 2023-10-06T03:03:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae785cd700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7015a160 con 0x7fae70159c60 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae785cd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700d13a0 con 0x7fae7015db10 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae70159890 0x7fae7015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fae70159890 0x7fae7015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46358/0 (socket says 172.21.15.92:46358) 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 -- 172.21.15.92:0/3982792674 learned_addr learned my addr 172.21.15.92:0/3982792674 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae75b68700 1 --2- 172.21.15.92:0/3982792674 >> v2:172.21.15.136:3300/0 conn(0x7fae70159c60 0x7fae7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 -- 172.21.15.92:0/3982792674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae7015db10 msgr2=0x7fae70069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76b6a700 1 --2- 172.21.15.92:0/3982792674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae7015db10 0x7fae70069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 --2- 172.21.15.92:0/3982792674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae7015db10 0x7fae70069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 -- 172.21.15.92:0/3982792674 >> v2:172.21.15.136:3300/0 conn(0x7fae70159c60 msgr2=0x7fae7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 --2- 172.21.15.92:0/3982792674 >> v2:172.21.15.136:3300/0 conn(0x7fae70159c60 0x7fae7015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae76369700 1 -- 172.21.15.92:0/3982792674 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7015e140 con 0x7fae70159890 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.789+0000 7fae75b68700 1 --2- 172.21.15.92:0/3982792674 >> v2:172.21.15.136:3300/0 conn(0x7fae70159c60 0x7fae7015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae76369700 1 --2- 172.21.15.92:0/3982792674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae70159890 0x7fae7015abd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae700b67f0 tx=0x7fae6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76381c28e2a2e05c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae75367700 1 -- 172.21.15.92:0/3982792674 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00e070 con 0x7fae70159890 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae75367700 1 -- 172.21.15.92:0/3982792674 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c00ba50 con 0x7fae70159890 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae75367700 1 -- 172.21.15.92:0/3982792674 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0092e0 con 0x7fae70159890 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 -- 172.21.15.92:0/3982792674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae70159890 msgr2=0x7fae7015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 --2- 172.21.15.92:0/3982792674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae70159890 0x7fae7015abd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae700b67f0 tx=0x7fae6c009840 comp rx=0 tx=0).stop 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 -- 172.21.15.92:0/3982792674 shutdown_connections 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 --2- 172.21.15.92:0/3982792674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae7015db10 0x7fae70069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 --2- 172.21.15.92:0/3982792674 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae70159890 0x7fae7015abd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 --2- 172.21.15.92:0/3982792674 >> v2:172.21.15.136:3300/0 conn(0x7fae70159c60 0x7fae7015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 -- 172.21.15.92:0/3982792674 >> 172.21.15.92:0/3982792674 conn(0x7fae70068e20 msgr2=0x7fae700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 -- 172.21.15.92:0/3982792674 shutdown_connections 2023-10-06T03:03:12.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.790+0000 7fae785cd700 1 -- 172.21.15.92:0/3982792674 wait complete. 2023-10-06T03:03:12.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.791+0000 7fae785cd700 1 Processor -- start 2023-10-06T03:03:12.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.791+0000 7fae785cd700 1 -- start start 2023-10-06T03:03:12.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae785cd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae70159890 0x7fae70154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae785cd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae70159c60 0x7fae7016a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae785cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 0x7fae700b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae785cd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7016ac20 con 0x7fae7015db10 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae785cd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b3950 con 0x7fae70159890 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae785cd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae700b3c30 con 0x7fae70159c60 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae76b6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 0x7fae700b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae76b6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 0x7fae700b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46368/0 (socket says 172.21.15.92:46368) 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae76b6a700 1 -- 172.21.15.92:0/2107405894 learned_addr learned my addr 172.21.15.92:0/2107405894 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae75b68700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae70159c60 0x7fae7016a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.792+0000 7fae76369700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae70159890 0x7fae70154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae76b6a700 1 -- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae70159c60 msgr2=0x7fae7016a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae76b6a700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae70159c60 0x7fae7016a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae76b6a700 1 -- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae70159890 msgr2=0x7fae70154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:12.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae76b6a700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae70159890 0x7fae70154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae76b6a700 1 -- 172.21.15.92:0/2107405894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c009ae0 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae76b6a700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 0x7fae700b4ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c880 tx=0x7fae68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68011660 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae680117c0 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae700b3f10 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68011b60 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.793+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae700b44d0 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.794+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae700b46f0 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.794+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae680193f0 con 0x7fae7015db10 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.795+0000 7fae667fc700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fae54066f30 0x7fae540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:12.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.795+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fae680849a0 con 0x7fae7015db10 2023-10-06T03:03:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.795+0000 7fae76369700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fae54066f30 0x7fae540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.795+0000 7fae76369700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fae54066f30 0x7fae540693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009880 tx=0x7fae6c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:12.799+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae68055540 con 0x7fae7015db10 2023-10-06T03:03:13.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.010+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fae7004c3c0 con 0x7fae54066f30 2023-10-06T03:03:13.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.012+0000 7fae667fc700 1 -- 172.21.15.92:0/2107405894 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+530 (secure 0 0 0) 0x7fae7004c3c0 con 0x7fae54066f30 2023-10-06T03:03:13.014 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:03:13.014 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_addrs": [ 2023-10-06T03:03:13.014 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6789", 2023-10-06T03:03:13.014 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6789", 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6790" 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "pending_subvolume_deletions": 0, 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": { 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "data": [ 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 365996965888, 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_data", 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 0 2023-10-06T03:03:13.015 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": [ 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 365996965888, 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_metadata", 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 65536 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:13.016 INFO:teuthology.orchestra.run.smithi092.stdout: "used_size": 0 2023-10-06T03:03:13.017 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.014+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fae54066f30 msgr2=0x7fae540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.014+0000 7fae785cd700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fae54066f30 0x7fae540693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009880 tx=0x7fae6c018000 comp rx=0 tx=0).stop 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.014+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 msgr2=0x7fae700b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.014+0000 7fae785cd700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 0x7fae700b4ff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c880 tx=0x7fae68008040 comp rx=0 tx=0).stop 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 shutdown_connections 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fae54066f30 0x7fae540693e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7015db10 0x7fae700b4ff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae70159890 0x7fae70154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 --2- 172.21.15.92:0/2107405894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fae70159c60 0x7fae7016a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 >> 172.21.15.92:0/2107405894 conn(0x7fae70068e20 msgr2=0x7fae700b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:13.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 shutdown_connections 2023-10-06T03:03:13.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.015+0000 7fae785cd700 1 -- 172.21.15.92:0/2107405894 wait complete. 2023-10-06T03:03:13.056 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:03:13.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:03:13.091 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:03:13.138 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:03:13.138 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:03:13.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:13.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:13.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:03:13.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:13.177 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:03:13.232 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:03:13.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:13.233 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:13.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:03:13.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:13.280 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:03:13.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:13.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.409+0000 7f3d5694a700 1 Processor -- start 2023-10-06T03:03:13.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.409+0000 7f3d5694a700 1 -- start start 2023-10-06T03:03:13.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d5694a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d50159c60 0x7f3d5015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d5694a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d5015db10 0x7f3d50069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d5694a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5015a160 con 0x7f3d50159890 2023-10-06T03:03:13.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d5694a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5015df50 con 0x7f3d50159c60 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d5694a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d500d13a0 con 0x7f3d5015db10 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d50159c60 0x7f3d5015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d4f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3d50159c60 0x7f3d5015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46380/0 (socket says 172.21.15.92:46380) 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.410+0000 7f3d4f7fe700 1 -- 172.21.15.92:0/2699214140 learned_addr learned my addr 172.21.15.92:0/2699214140 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d54ee7700 1 --2- 172.21.15.92:0/2699214140 >> v2:172.21.15.136:3300/0 conn(0x7f3d5015db10 0x7f3d50069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4f7fe700 1 -- 172.21.15.92:0/2699214140 >> v2:172.21.15.136:3300/0 conn(0x7f3d5015db10 msgr2=0x7f3d50069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4f7fe700 1 --2- 172.21.15.92:0/2699214140 >> v2:172.21.15.136:3300/0 conn(0x7f3d5015db10 0x7f3d50069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4f7fe700 1 -- 172.21.15.92:0/2699214140 >> v1:172.21.15.136:6789/0 conn(0x7f3d50159890 legacy=0x7f3d5015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4f7fe700 1 -- 172.21.15.92:0/2699214140 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5015e140 con 0x7f3d50159c60 2023-10-06T03:03:13.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4f7fe700 1 --2- 172.21.15.92:0/2699214140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d50159c60 0x7f3d5015d5d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3d5006a890 tx=0x7f3d40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b218583b7032b792 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4effd700 1 -- 172.21.15.92:0/2699214140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4000b820 con 0x7f3d50159c60 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4effd700 1 -- 172.21.15.92:0/2699214140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4000b980 con 0x7f3d50159c60 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d4effd700 1 -- 172.21.15.92:0/2699214140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4000bd20 con 0x7f3d50159c60 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d5694a700 1 -- 172.21.15.92:0/2699214140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d50159c60 msgr2=0x7f3d5015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.411+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2699214140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d50159c60 0x7f3d5015d5d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3d5006a890 tx=0x7f3d40009250 comp rx=0 tx=0).stop 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.412+0000 7f3d5694a700 1 -- 172.21.15.92:0/2699214140 shutdown_connections 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.412+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2699214140 >> v2:172.21.15.136:3300/0 conn(0x7f3d5015db10 0x7f3d50069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.412+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2699214140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d50159c60 0x7f3d5015d5d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.412+0000 7f3d5694a700 1 -- 172.21.15.92:0/2699214140 >> 172.21.15.92:0/2699214140 conn(0x7f3d50068e20 msgr2=0x7f3d500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.412+0000 7f3d5694a700 1 -- 172.21.15.92:0/2699214140 shutdown_connections 2023-10-06T03:03:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.412+0000 7f3d5694a700 1 -- 172.21.15.92:0/2699214140 wait complete. 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 Processor -- start 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 -- start start 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d50159890 0x7f3d5014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d50159c60 0x7f3d50153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 0x7f3d50153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5014d190 con 0x7f3d5015db10 2023-10-06T03:03:13.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500b1830 con 0x7f3d50159890 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.413+0000 7f3d5694a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500b1b10 con 0x7f3d50159c60 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 0x7f3d50153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 0x7f3d50153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46394/0 (socket says 172.21.15.92:46394) 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 -- 172.21.15.92:0/2994574884 learned_addr learned my addr 172.21.15.92:0/2994574884 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d4ffff700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d50159890 0x7f3d5014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 -- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d50159c60 msgr2=0x7f3d50153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d50159c60 0x7f3d50153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 -- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d50159890 msgr2=0x7f3d5014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d50159890 0x7f3d5014cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 -- 172.21.15.92:0/2994574884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d400094f0 con 0x7f3d5015db10 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d4ffff700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d50159890 0x7f3d5014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d54ee7700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 0x7f3d50153e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800cdb0 tx=0x7f3d480077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800aa80 con 0x7f3d5015db10 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4800abe0 con 0x7f3d5015db10 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d500b1df0 con 0x7f3d5015db10 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48017430 con 0x7f3d5015db10 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.414+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d500cde50 con 0x7f3d5015db10 2023-10-06T03:03:13.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.415+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d500b25f0 con 0x7f3d5015db10 2023-10-06T03:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.415+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d48003020 con 0x7f3d5015db10 2023-10-06T03:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.416+0000 7f3d37fff700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d3c066f30 0x7f3d3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.416+0000 7f3d4ffff700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d3c066f30 0x7f3d3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.416+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d48013070 con 0x7f3d5015db10 2023-10-06T03:03:13.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.416+0000 7f3d4ffff700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d3c066f30 0x7f3d3c0693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3d44002880 tx=0x7f3d44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:13.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.420+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d500b25f0 con 0x7f3d5015db10 2023-10-06T03:03:13.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.645+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d500b25f0 con 0x7f3d5015db10 2023-10-06T03:03:13.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.645+0000 7f3d37fff700 1 -- 172.21.15.92:0/2994574884 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f3d500b25f0 con 0x7f3d5015db10 2023-10-06T03:03:13.647 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:13.647 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:01.941144+0000","modified":"2023-10-06T03:03:05.571224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3249599548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3249599548},{"type":"v1","addr":"172.21.15.92:6837","nonce":3249599548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:03:13.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d3c066f30 msgr2=0x7f3d3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d3c066f30 0x7f3d3c0693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3d44002880 tx=0x7f3d44009040 comp rx=0 tx=0).stop 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 msgr2=0x7f3d50153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 0x7f3d50153e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800cdb0 tx=0x7f3d480077d0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 shutdown_connections 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3d3c066f30 0x7f3d3c0693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d5015db10 0x7f3d50153e70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d50159890 0x7f3d5014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 --2- 172.21.15.92:0/2994574884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d50159c60 0x7f3d50153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 >> 172.21.15.92:0/2994574884 conn(0x7f3d50068e20 msgr2=0x7f3d501554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 shutdown_connections 2023-10-06T03:03:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.647+0000 7f3d5694a700 1 -- 172.21.15.92:0/2994574884 wait complete. 2023-10-06T03:03:13.651 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2023-10-06T03:03:13.661 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:13.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.775+0000 7fa7b747f700 1 Processor -- start 2023-10-06T03:03:13.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.775+0000 7fa7b747f700 1 -- start start 2023-10-06T03:03:13.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b747f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0159890 0x7fa7b015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b747f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7b0069560 0x7fa7b00699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b747f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b015a160 con 0x7fa7b0159c60 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b747f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b0069f20 con 0x7fa7b0069560 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b747f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b00d13a0 con 0x7fa7b0159890 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b5a1c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7b0069560 0x7fa7b00699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b4a1a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa7b0159c60 0x7fa7b015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40668/0 (socket says 172.21.15.92:40668) 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.776+0000 7fa7b4a1a700 1 -- 172.21.15.92:0/1977147593 learned_addr learned my addr 172.21.15.92:0/1977147593 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b521b700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0159890 0x7fa7b015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/1977147593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0159890 msgr2=0x7fa7b015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b5a1c700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0159890 0x7fa7b015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/1977147593 >> v1:172.21.15.92:6789/0 conn(0x7fa7b0159c60 legacy=0x7fa7b015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/1977147593 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b015e140 con 0x7fa7b0069560 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7a7fff700 1 -- 172.21.15.92:0/1977147593 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799445718 0 0) 0x7fa7b015a160 con 0x7fa7b0159c60 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b521b700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0159890 0x7fa7b015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b5a1c700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0069560 0x7fa7b00699e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009e60 tx=0x7fa7a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942cdd284a260abe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7a7fff700 1 -- 172.21.15.92:0/1977147593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800e070 con 0x7fa7b0069560 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7a7fff700 1 -- 172.21.15.92:0/1977147593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a800bc30 con 0x7fa7b0069560 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7a7fff700 1 -- 172.21.15.92:0/1977147593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8012430 con 0x7fa7b0069560 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b747f700 1 -- 172.21.15.92:0/1977147593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0069560 msgr2=0x7fa7b00699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.777+0000 7fa7b747f700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0069560 0x7fa7b00699e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009e60 tx=0x7fa7a8009780 comp rx=0 tx=0).stop 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.778+0000 7fa7b747f700 1 -- 172.21.15.92:0/1977147593 shutdown_connections 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.778+0000 7fa7b747f700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0159890 0x7fa7b015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.778+0000 7fa7b747f700 1 --2- 172.21.15.92:0/1977147593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0069560 0x7fa7b00699e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.778+0000 7fa7b747f700 1 -- 172.21.15.92:0/1977147593 >> 172.21.15.92:0/1977147593 conn(0x7fa7b0068e20 msgr2=0x7fa7b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:13.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.778+0000 7fa7b747f700 1 -- 172.21.15.92:0/1977147593 shutdown_connections 2023-10-06T03:03:13.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.778+0000 7fa7b747f700 1 -- 172.21.15.92:0/1977147593 wait complete. 2023-10-06T03:03:13.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.779+0000 7fa7b747f700 1 Processor -- start 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.779+0000 7fa7b747f700 1 -- start start 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0069560 0x7fa7b00b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b0159890 0x7fa7b00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 0x7fa7b00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b00b42d0 con 0x7fa7b0159c60 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b0155540 con 0x7fa7b0159890 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b0155820 con 0x7fa7b0069560 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 0x7fa7b00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 0x7fa7b00cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46412/0 (socket says 172.21.15.92:46412) 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/982312521 learned_addr learned my addr 172.21.15.92:0/982312521 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:13.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b521b700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0069560 0x7fa7b00b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b4a1a700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b0159890 0x7fa7b00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0069560 msgr2=0x7fa7b00b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0069560 0x7fa7b00b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b0159890 msgr2=0x7fa7b00c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b0159890 0x7fa7b00c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 -- 172.21.15.92:0/982312521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8009a20 con 0x7fa7b0159c60 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b4a1a700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b0159890 0x7fa7b00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b5a1c700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 0x7fa7b00cd0c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8013000 tx=0x7fa7a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8027470 con 0x7fa7b0159c60 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a8027630 con 0x7fa7b0159c60 2023-10-06T03:03:13.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b00c92c0 con 0x7fa7b0159c60 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80279f0 con 0x7fa7b0159c60 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.780+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b00b6270 con 0x7fa7b0159c60 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.781+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b00b6490 con 0x7fa7b0159c60 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.782+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7a8024070 con 0x7fa7b0159c60 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.782+0000 7fa7a57fa700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa794066f30 0x7fa7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.782+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7a8086da0 con 0x7fa7b0159c60 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.782+0000 7fa7b521b700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa794066f30 0x7fa7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.782+0000 7fa7b521b700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa794066f30 0x7fa7940693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006f00 tx=0x7fa7ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.786+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7a8053eb0 con 0x7fa7b0159c60 2023-10-06T03:03:13.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.970+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7b01553a0 con 0x7fa7b0159c60 2023-10-06T03:03:13.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.971+0000 7fa7a57fa700 1 -- 172.21.15.92:0/982312521 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16868 (secure 0 0 0) 0x7fa7b01553a0 con 0x7fa7b0159c60 2023-10-06T03:03:13.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:13.974 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":54,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:03.471780+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:59.601586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:00.905218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:13.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa794066f30 msgr2=0x7fa7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa794066f30 0x7fa7940693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac006f00 tx=0x7fa7ac008040 comp rx=0 tx=0).stop 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 msgr2=0x7fa7b00cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 0x7fa7b00cd0c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8013000 tx=0x7fa7a8019000 comp rx=0 tx=0).stop 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 shutdown_connections 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa794066f30 0x7fa7940693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7b0159c60 0x7fa7b00cd0c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7b0069560 0x7fa7b00b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 --2- 172.21.15.92:0/982312521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa7b0159890 0x7fa7b00c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 >> 172.21.15.92:0/982312521 conn(0x7fa7b0068e20 msgr2=0x7fa7b0158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 shutdown_connections 2023-10-06T03:03:13.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:13.974+0000 7fa7b747f700 1 -- 172.21.15.92:0/982312521 wait complete. 2023-10-06T03:03:13.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:02:59.601586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:03:00.905218+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:03:13.988 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:03:14.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.103+0000 7f62227aa700 1 Processor -- start 2023-10-06T03:03:14.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.103+0000 7f62227aa700 1 -- start start 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.104+0000 7f62227aa700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f621c159890 0x7f621c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.104+0000 7f62227aa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c15a160 con 0x7f621c069560 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.104+0000 7f62227aa700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c0d13a0 con 0x7f621c159c60 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.104+0000 7f62227aa700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621c15e140 con 0x7f621c159890 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.104+0000 7f6220d47700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f621c069560 0x7f621c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40674/0 (socket says 172.21.15.92:40674) 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.104+0000 7f6220d47700 1 -- 172.21.15.92:0/1554796400 learned_addr learned my addr 172.21.15.92:0/1554796400 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621bfff700 1 --2- 172.21.15.92:0/1554796400 >> v2:172.21.15.136:3300/0 conn(0x7f621c159890 0x7f621c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146260092 0 0) 0x7f621c15a160 con 0x7f621c069560 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6204003570 con 0x7f621c069560 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764995294 0 0) 0x7f621c0d13a0 con 0x7f621c159c60 2023-10-06T03:03:14.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f621c15a160 con 0x7f621c159c60 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621bfff700 1 -- 172.21.15.92:0/1554796400 >> v1:172.21.15.136:6789/0 conn(0x7f621c159c60 legacy=0x7f621c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485925048 0 0) 0x7f6204003570 con 0x7f621c069560 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621bfff700 1 -- 172.21.15.92:0/1554796400 >> v1:172.21.15.92:6789/0 conn(0x7f621c069560 legacy=0x7f621c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621bfff700 1 -- 172.21.15.92:0/1554796400 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c15a3d0 con 0x7f621c159890 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621bfff700 1 --2- 172.21.15.92:0/1554796400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 0x7f621c15abd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f620c0098e0 tx=0x7f620c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ffea1148c08b18b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62140017f0 con 0x7f621c159890 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c011420 con 0x7f621c159890 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.105+0000 7f621affd700 1 -- 172.21.15.92:0/1554796400 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0117c0 con 0x7f621c159890 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 -- 172.21.15.92:0/1554796400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 msgr2=0x7f621c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 --2- 172.21.15.92:0/1554796400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 0x7f621c15abd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f620c0098e0 tx=0x7f620c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 -- 172.21.15.92:0/1554796400 shutdown_connections 2023-10-06T03:03:14.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 --2- 172.21.15.92:0/1554796400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 0x7f621c15abd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 -- 172.21.15.92:0/1554796400 >> 172.21.15.92:0/1554796400 conn(0x7f621c068e20 msgr2=0x7f621c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:14.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 -- 172.21.15.92:0/1554796400 shutdown_connections 2023-10-06T03:03:14.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.106+0000 7f62227aa700 1 -- 172.21.15.92:0/1554796400 wait complete. 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.107+0000 7f62227aa700 1 Processor -- start 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.107+0000 7f62227aa700 1 -- start start 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f62227aa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c069560 0x7f621c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f62227aa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 0x7f621c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f62227aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 0x7f621c16ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f62227aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0cddb0 con 0x7f621c159c60 2023-10-06T03:03:14.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f62227aa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c140f60 con 0x7f621c159890 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f62227aa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0cd1a0 con 0x7f621c069560 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 0x7f621c16ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 0x7f621c16ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46424/0 (socket says 172.21.15.92:46424) 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 -- 172.21.15.92:0/983588789 learned_addr learned my addr 172.21.15.92:0/983588789 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 -- 172.21.15.92:0/983588789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c069560 msgr2=0x7f621c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c069560 0x7f621c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 -- 172.21.15.92:0/983588789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 msgr2=0x7f621c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 0x7f621c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 -- 172.21.15.92:0/983588789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c009420 con 0x7f621c159c60 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6220d47700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 0x7f621c16ad30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f621400cca0 tx=0x7f6214007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.108+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400ce50 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.109+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6214004dc0 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.109+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c0cd480 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.109+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400ab70 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.109+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c0cda40 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.109+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c04c3c0 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.110+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6214007150 con 0x7f621c159c60 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.110+0000 7f6203fff700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6208066f30 0x7f62080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.110+0000 7f621bfff700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6208066f30 0x7f62080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.111+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6214083f40 con 0x7f621c159c60 2023-10-06T03:03:14.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.111+0000 7f621bfff700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6208066f30 0x7f62080693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f621c167750 tx=0x7f620c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.115+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62140510d0 con 0x7f621c159c60 2023-10-06T03:03:14.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.298+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f621c0cdef0 con 0x7f621c159c60 2023-10-06T03:03:14.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.298+0000 7f6203fff700 1 -- 172.21.15.92:0/983588789 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6214018020 con 0x7f621c159c60 2023-10-06T03:03:14.300 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:03:14.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6208066f30 msgr2=0x7f62080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6208066f30 0x7f62080693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f621c167750 tx=0x7f620c016000 comp rx=0 tx=0).stop 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 msgr2=0x7f621c16ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 0x7f621c16ad30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f621400cca0 tx=0x7f6214007d20 comp rx=0 tx=0).stop 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 shutdown_connections 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6208066f30 0x7f62080693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f621c159c60 0x7f621c16ad30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f621c069560 0x7f621c158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 --2- 172.21.15.92:0/983588789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f621c159890 0x7f621c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.300+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 >> 172.21.15.92:0/983588789 conn(0x7f621c068e20 msgr2=0x7f621c152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.301+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 shutdown_connections 2023-10-06T03:03:14.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.301+0000 7f62227aa700 1 -- 172.21.15.92:0/983588789 wait complete. 2023-10-06T03:03:14.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:03:14.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.429+0000 7f0d0759e700 1 Processor -- start 2023-10-06T03:03:14.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.429+0000 7f0d0759e700 1 -- start start 2023-10-06T03:03:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 0x7f0d080ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0759e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d080b16d0 0x7f0d080b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d080b2080 con 0x7f0d0815d240 2023-10-06T03:03:14.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0759e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d080cfb00 con 0x7f0d080b16d0 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080b5c40 con 0x7f0d080b4e70 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d06d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0d0815d240 0x7f0d0815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40676/0 (socket says 172.21.15.92:40676) 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d06d9d700 1 -- 172.21.15.92:0/1237871874 learned_addr learned my addr 172.21.15.92:0/1237871874 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684261418 0 0) 0x7f0d080b2080 con 0x7f0d0815d240 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf0003570 con 0x7f0d0815d240 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0659c700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 0x7f0d080ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d05d9b700 1 --2- 172.21.15.92:0/1237871874 >> v2:172.21.15.136:3300/0 conn(0x7f0d080b16d0 0x7f0d080b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881050637 0 0) 0x7f0cf0003570 con 0x7f0d0815d240 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 msgr2=0x7f0d080ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 0x7f0d080ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 >> v2:172.21.15.136:3300/0 conn(0x7f0d080b16d0 msgr2=0x7f0d080b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 --2- 172.21.15.92:0/1237871874 >> v2:172.21.15.136:3300/0 conn(0x7f0d080b16d0 0x7f0d080b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d080b5f20 con 0x7f0d0815d240 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f0d000017f0 con 0x7f0d0815d240 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 >> v1:172.21.15.92:6789/0 conn(0x7f0d0815d240 legacy=0x7f0d0815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.430+0000 7f0d0559a700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 0x7f0cf0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0003fe0 0x7f0cf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf0004d50 0x7f0cf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080cfb00 con 0x7f0cf0004d50 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080b5c40 con 0x7f0cf0003fe0 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf000ba90 con 0x7f0d080b4e70 2023-10-06T03:03:14.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf0004d50 0x7f0cf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d05d9b700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0003fe0 0x7f0cf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0659c700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 0x7f0cf0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 -- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 msgr2=0x7f0cf0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080b4e70 0x7f0cf0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 -- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0003fe0 msgr2=0x7f0cf0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0003fe0 0x7f0cf0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 -- 172.21.15.92:0/1237871874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf0003570 con 0x7f0cf0004d50 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf0004d50 0x7f0cf00051c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a970 tx=0x7f0d00002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b1c489e4f8ddae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00012440 con 0x7f0cf0004d50 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d000125c0 con 0x7f0cf0004d50 2023-10-06T03:03:14.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.431+0000 7f0d0559a700 1 -- 172.21.15.92:0/1237871874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00012980 con 0x7f0cf0004d50 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d0759e700 1 -- 172.21.15.92:0/1237871874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf0004d50 msgr2=0x7f0cf00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d0759e700 1 --2- 172.21.15.92:0/1237871874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cf0004d50 0x7f0cf00051c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a970 tx=0x7f0d00002210 comp rx=0 tx=0).stop 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d06d9d700 1 -- 172.21.15.92:0/1237871874 reap_dead start 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d0759e700 1 -- 172.21.15.92:0/1237871874 shutdown_connections 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d0759e700 1 -- 172.21.15.92:0/1237871874 >> 172.21.15.92:0/1237871874 conn(0x7f0d080d10c0 msgr2=0x7f0d080affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d0759e700 1 -- 172.21.15.92:0/1237871874 shutdown_connections 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.432+0000 7f0d0759e700 1 -- 172.21.15.92:0/1237871874 wait complete. 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.433+0000 7f0d0759e700 1 Processor -- start 2023-10-06T03:03:14.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.433+0000 7f0d0759e700 1 -- start start 2023-10-06T03:03:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0004d50 0x7f0d081532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080c6e80 0x7f0d08156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 0x7f0d08157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0816c000 con 0x7f0d080c72f0 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0816b3f0 con 0x7f0cf0004d50 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0816b6d0 con 0x7f0d080c6e80 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 0x7f0d08157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 0x7f0d08157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46434/0 (socket says 172.21.15.92:46434) 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 -- 172.21.15.92:0/3460594883 learned_addr learned my addr 172.21.15.92:0/3460594883 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0659c700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0004d50 0x7f0d081532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d05d9b700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080c6e80 0x7f0d08156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 -- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080c6e80 msgr2=0x7f0d08156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080c6e80 0x7f0d08156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 -- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0004d50 msgr2=0x7f0d081532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0004d50 0x7f0d081532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 -- 172.21.15.92:0/3460594883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0816b9b0 con 0x7f0d080c72f0 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d06d9d700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 0x7f0d08157440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a000 tx=0x7f0d00015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00012440 con 0x7f0d080c72f0 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d000017b0 con 0x7f0d080c72f0 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0816bc90 con 0x7f0d080c72f0 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00012960 con 0x7f0d080c72f0 2023-10-06T03:03:14.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.434+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0816a8f0 con 0x7f0d080c72f0 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.435+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08088990 con 0x7f0d080c72f0 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.435+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d00019070 con 0x7f0d080c72f0 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.436+0000 7f0cf67fc700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ce8066f30 0x7f0ce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.436+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d000860d0 con 0x7f0d080c72f0 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.436+0000 7f0d0659c700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ce8066f30 0x7f0ce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.436+0000 7f0d0659c700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ce8066f30 0x7f0ce80693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006f00 tx=0x7f0cf80091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.440+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d000531e0 con 0x7f0d080c72f0 2023-10-06T03:03:14.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.613+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f0d000535b0 con 0x7f0d080c72f0 2023-10-06T03:03:14.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.626+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d080cf4b0 con 0x7f0d080c72f0 2023-10-06T03:03:14.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.626+0000 7f0cf67fc700 1 -- 172.21.15.92:0/3460594883 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d00056c70 con 0x7f0d080c72f0 2023-10-06T03:03:14.628 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:03:14.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ce8066f30 msgr2=0x7f0ce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ce8066f30 0x7f0ce80693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006f00 tx=0x7f0cf80091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 msgr2=0x7f0d08157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 0x7f0d08157440 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a000 tx=0x7f0d00015000 comp rx=0 tx=0).stop 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 shutdown_connections 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ce8066f30 0x7f0ce80693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d080c72f0 0x7f0d08157440 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cf0004d50 0x7f0d081532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 --2- 172.21.15.92:0/3460594883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d080c6e80 0x7f0d08156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.628+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 >> 172.21.15.92:0/3460594883 conn(0x7f0d080d10c0 msgr2=0x7f0d0815a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:14.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.629+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 shutdown_connections 2023-10-06T03:03:14.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.629+0000 7f0d0759e700 1 -- 172.21.15.92:0/3460594883 wait complete. 2023-10-06T03:03:14.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:03:14.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.757+0000 7f0bd91da700 1 Processor -- start 2023-10-06T03:03:14.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd91da700 1 -- start start 2023-10-06T03:03:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd91da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bd4159890 0x7f0bd415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd91da700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd415a160 con 0x7f0bd4069560 2023-10-06T03:03:14.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd91da700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40d13a0 con 0x7f0bd4159c60 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd91da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd415e140 con 0x7f0bd4159890 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bd4159890 0x7f0bd415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.758+0000 7f0bd359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0bd4069560 0x7f0bd40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40688/0 (socket says 172.21.15.92:40688) 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd359e700 1 -- 172.21.15.92:0/7113176 learned_addr learned my addr 172.21.15.92:0/7113176 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd2d9d700 1 -- 172.21.15.92:0/7113176 >> v1:172.21.15.136:6789/0 conn(0x7f0bd4159c60 legacy=0x7f0bd415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd2d9d700 1 -- 172.21.15.92:0/7113176 >> v1:172.21.15.92:6789/0 conn(0x7f0bd4069560 legacy=0x7f0bd40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd259c700 1 -- 172.21.15.92:0/7113176 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777574383 0 0) 0x7f0bd415a160 con 0x7f0bd4069560 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd2d9d700 1 -- 172.21.15.92:0/7113176 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd415a3d0 con 0x7f0bd4159890 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd2d9d700 1 --2- 172.21.15.92:0/7113176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159890 0x7f0bd415abd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098e0 tx=0x7f0bc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cd68bf29fc06f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd259c700 1 -- 172.21.15.92:0/7113176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4009e80 con 0x7f0bd4159890 2023-10-06T03:03:14.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd259c700 1 -- 172.21.15.92:0/7113176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc400b860 con 0x7f0bd4159890 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.759+0000 7f0bd259c700 1 -- 172.21.15.92:0/7113176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400bbe0 con 0x7f0bd4159890 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 -- 172.21.15.92:0/7113176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159890 msgr2=0x7f0bd415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 --2- 172.21.15.92:0/7113176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159890 0x7f0bd415abd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098e0 tx=0x7f0bc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 -- 172.21.15.92:0/7113176 shutdown_connections 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 --2- 172.21.15.92:0/7113176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159890 0x7f0bd415abd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 -- 172.21.15.92:0/7113176 >> 172.21.15.92:0/7113176 conn(0x7f0bd4068e20 msgr2=0x7f0bd40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 -- 172.21.15.92:0/7113176 shutdown_connections 2023-10-06T03:03:14.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.760+0000 7f0bd91da700 1 -- 172.21.15.92:0/7113176 wait complete. 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.761+0000 7f0bd91da700 1 Processor -- start 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.761+0000 7f0bd91da700 1 -- start start 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd4069560 0x7f0bd414ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd4159890 0x7f0bd4156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 0x7f0bd4157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40ce8d0 con 0x7f0bd4159c60 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd414d4a0 con 0x7f0bd4069560 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd414cf40 con 0x7f0bd4159890 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 0x7f0bd4157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 0x7f0bd4157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46452/0 (socket says 172.21.15.92:46452) 2023-10-06T03:03:14.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 -- 172.21.15.92:0/1906331233 learned_addr learned my addr 172.21.15.92:0/1906331233 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd2d9d700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd4069560 0x7f0bd414ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bca59c700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd4159890 0x7f0bd4156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 -- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd4159890 msgr2=0x7f0bd4156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd4159890 0x7f0bd4156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 -- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd4069560 msgr2=0x7f0bd414ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd4069560 0x7f0bd414ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 -- 172.21.15.92:0/1906331233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc4009420 con 0x7f0bd4159c60 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd359e700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 0x7f0bd4157160 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00cc90 tx=0x7f0bcc001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00a870 con 0x7f0bd4159c60 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc00a9d0 con 0x7f0bd4159c60 2023-10-06T03:03:14.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd40ce5f0 con 0x7f0bd4159c60 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00ad90 con 0x7f0bd4159c60 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.762+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd40cf1e0 con 0x7f0bd4159c60 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.763+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd415c180 con 0x7f0bd4159c60 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.764+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f0bcc007300 con 0x7f0bd4159c60 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.764+0000 7f0bcb7fe700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4066fc0 0x7f0bb4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.764+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0bcc084a80 con 0x7f0bd4159c60 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.764+0000 7f0bd2d9d700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4066fc0 0x7f0bb4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.764+0000 7f0bd2d9d700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4066fc0 0x7f0bb4069470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40093f0 tx=0x7f0bc4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:14.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.768+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bd415c180 con 0x7f0bd4159c60 2023-10-06T03:03:14.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.953+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0bd415c180 con 0x7f0bd4159c60 2023-10-06T03:03:14.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.953+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1906331233 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0bd415c180 con 0x7f0bd4159c60 2023-10-06T03:03:14.955 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:03:14.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4066fc0 msgr2=0x7f0bb4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4066fc0 0x7f0bb4069470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40093f0 tx=0x7f0bc4017000 comp rx=0 tx=0).stop 2023-10-06T03:03:14.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 msgr2=0x7f0bd4157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:14.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 0x7f0bd4157160 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00cc90 tx=0x7f0bcc001a20 comp rx=0 tx=0).stop 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 shutdown_connections 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4066fc0 0x7f0bb4069470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd4159c60 0x7f0bd4157160 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd4069560 0x7f0bd414ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 --2- 172.21.15.92:0/1906331233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd4159890 0x7f0bd4156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 >> 172.21.15.92:0/1906331233 conn(0x7f0bd4068e20 msgr2=0x7f0bd40b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 shutdown_connections 2023-10-06T03:03:14.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:14.955+0000 7f0bd91da700 1 -- 172.21.15.92:0/1906331233 wait complete. 2023-10-06T03:03:14.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:03:15.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.085+0000 7f86b6e3a700 1 Processor -- start 2023-10-06T03:03:15.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.085+0000 7f86b6e3a700 1 -- start start 2023-10-06T03:03:15.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b6e3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 0x7f86b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b6e3a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b00cfb00 con 0x7f86b00b1b40 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b6e3a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b015e060 con 0x7f86b00b16d0 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b6e3a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00b5c40 con 0x7f86b00b4e70 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b6639700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86b00b1b40 0x7f86b015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40702/0 (socket says 172.21.15.92:40702) 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b6639700 1 -- 172.21.15.92:0/3773081489 learned_addr learned my addr 172.21.15.92:0/3773081489 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971166857 0 0) 0x7f86b00cfb00 con 0x7f86b00b1b40 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b5e38700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 0x7f86b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f869c003570 con 0x7f86b00b1b40 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1289514512 0 0) 0x7f869c003570 con 0x7f86b00b1b40 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 msgr2=0x7f86b00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 0x7f86b00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 >> v1:172.21.15.136:6789/0 conn(0x7f86b00b16d0 legacy=0x7f86b015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b00b5f20 con 0x7f86b00b1b40 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.086+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f86a8001470 con 0x7f86b00b1b40 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 >> v1:172.21.15.92:6789/0 conn(0x7f86b00b1b40 legacy=0x7f86b015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869c003a90 0x7f869c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 0x7f869c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869c004d50 0x7f869c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00b5c40 con 0x7f869c003a90 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c00bb70 con 0x7f869c004d50 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c00bd30 con 0x7f86b00b4e70 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869c003a90 0x7f869c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b6639700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869c004d50 0x7f869c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5637700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 0x7f869c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 -- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 msgr2=0x7f869c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86b00b4e70 0x7f869c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 -- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869c004d50 msgr2=0x7f869c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869c004d50 0x7f869c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 -- 172.21.15.92:0/3773081489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c003570 con 0x7f869c003a90 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b6639700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f869c004d50 0x7f869c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b5e38700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869c003a90 0x7f869c003ea0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e210 tx=0x7f86ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730358e275755731 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00eac0 con 0x7f869c003a90 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac00f040 con 0x7f869c003a90 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.087+0000 7f86b4e36700 1 -- 172.21.15.92:0/3773081489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac00ee70 con 0x7f869c003a90 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6e3a700 1 -- 172.21.15.92:0/3773081489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869c003a90 msgr2=0x7f869c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/3773081489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f869c003a90 0x7f869c003ea0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00e210 tx=0x7f86ac00b040 comp rx=0 tx=0).stop 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6639700 1 -- 172.21.15.92:0/3773081489 reap_dead start 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6e3a700 1 -- 172.21.15.92:0/3773081489 shutdown_connections 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6e3a700 1 -- 172.21.15.92:0/3773081489 >> 172.21.15.92:0/3773081489 conn(0x7f86b00d10c0 msgr2=0x7f86b00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6e3a700 1 -- 172.21.15.92:0/3773081489 shutdown_connections 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.088+0000 7f86b6e3a700 1 -- 172.21.15.92:0/3773081489 wait complete. 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.089+0000 7f86b6e3a700 1 Processor -- start 2023-10-06T03:03:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.089+0000 7f86b6e3a700 1 -- start start 2023-10-06T03:03:15.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f869c003a90 0x7f86b0155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b00c92e0 0x7f86b00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 0x7f86b015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cc0d0 con 0x7f86b00cb020 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cb4c0 con 0x7f86b00c92e0 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b00cb7a0 con 0x7f869c003a90 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 0x7f86b015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 0x7f86b015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46476/0 (socket says 172.21.15.92:46476) 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 -- 172.21.15.92:0/1257322059 learned_addr learned my addr 172.21.15.92:0/1257322059 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b5e38700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f869c003a90 0x7f86b0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b5637700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b00c92e0 0x7f86b00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 -- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f869c003a90 msgr2=0x7f86b0155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f869c003a90 0x7f86b0155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 -- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b00c92e0 msgr2=0x7f86b00caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b00c92e0 0x7f86b00caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 -- 172.21.15.92:0/1257322059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b00cba80 con 0x7f86b00cb020 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b5637700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b00c92e0 0x7f86b00caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6639700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 0x7f86b015c7a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f86a8000fd0 tx=0x7f86a8001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a800c600 con 0x7f86b00cb020 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a800c780 con 0x7f86b00cb020 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a800cb40 con 0x7f86b00cb020 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.090+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b00cbd60 con 0x7f86b00cb020 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.091+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b015cf20 con 0x7f86b00cb020 2023-10-06T03:03:15.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.091+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b00c96b0 con 0x7f86b00cb020 2023-10-06T03:03:15.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.092+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f86a801d460 con 0x7f86b00cb020 2023-10-06T03:03:15.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.092+0000 7f86a5ffb700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8694066fc0 0x7f8694069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.092+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f86a80845e0 con 0x7f86b00cb020 2023-10-06T03:03:15.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.092+0000 7f86b5e38700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8694066fc0 0x7f8694069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.093+0000 7f86b5e38700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8694066fc0 0x7f8694069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0094a0 tx=0x7f86ac01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.096+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86a8051700 con 0x7f86b00cb020 2023-10-06T03:03:15.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.276+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86b015d110 con 0x7f86b00cb020 2023-10-06T03:03:15.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.277+0000 7f86a5ffb700 1 -- 172.21.15.92:0/1257322059 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86a8054d20 con 0x7f86b00cb020 2023-10-06T03:03:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8694066fc0 msgr2=0x7f8694069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8694066fc0 0x7f8694069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0094a0 tx=0x7f86ac01a000 comp rx=0 tx=0).stop 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 msgr2=0x7f86b015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 0x7f86b015c7a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f86a8000fd0 tx=0x7f86a8001d90 comp rx=0 tx=0).stop 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 shutdown_connections 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8694066fc0 0x7f8694069470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86b00cb020 0x7f86b015c7a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f869c003a90 0x7f86b0155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 --2- 172.21.15.92:0/1257322059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86b00c92e0 0x7f86b00caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 >> 172.21.15.92:0/1257322059 conn(0x7f86b00d10c0 msgr2=0x7f86b0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 shutdown_connections 2023-10-06T03:03:15.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.279+0000 7f86b6e3a700 1 -- 172.21.15.92:0/1257322059 wait complete. 2023-10-06T03:03:15.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.405+0000 7f5d8fc44700 1 Processor -- start 2023-10-06T03:03:15.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.405+0000 7f5d8fc44700 1 -- start start 2023-10-06T03:03:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.406+0000 7f5d8fc44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 0x7f5d880caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.406+0000 7f5d8fc44700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8815d7e0 con 0x7f5d8806bcd0 2023-10-06T03:03:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.406+0000 7f5d8fc44700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8815dac0 con 0x7f5d8815df50 2023-10-06T03:03:15.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.406+0000 7f5d8fc44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8815aa30 con 0x7f5d8806c0a0 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.406+0000 7f5d8d9e0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5d8806bcd0 0x7f5d8815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40708/0 (socket says 172.21.15.92:40708) 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.406+0000 7f5d8d9e0700 1 -- 172.21.15.92:0/3651174763 learned_addr learned my addr 172.21.15.92:0/3651174763 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d1df700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 0x7f5d880caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027200918 0 0) 0x7f5d8815d7e0 con 0x7f5d8806bcd0 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d74003570 con 0x7f5d8806bcd0 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635755680 0 0) 0x7f5d74003570 con 0x7f5d8806bcd0 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 msgr2=0x7f5d880caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 0x7f5d880caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 >> v1:172.21.15.136:6789/0 conn(0x7f5d8815df50 legacy=0x7f5d880c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8815ad10 con 0x7f5d8806bcd0 2023-10-06T03:03:15.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5d840017f0 con 0x7f5d8806bcd0 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 >> v1:172.21.15.92:6789/0 conn(0x7f5d8806bcd0 legacy=0x7f5d8815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d74003a90 0x7f5d74003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74004810 0x7f5d74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 0x7f5d74005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8815aa30 con 0x7f5d74003a90 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7400bc20 con 0x7f5d74004810 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7400bde0 con 0x7f5d8806c0a0 2023-10-06T03:03:15.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d74003a90 0x7f5d74003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d1df700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74004810 0x7f5d74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8e1e1700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 0x7f5d74005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 -- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 msgr2=0x7f5d74005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806c0a0 0x7f5d74005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 -- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74004810 msgr2=0x7f5d74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74004810 0x7f5d74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 -- 172.21.15.92:0/3651174763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74003570 con 0x7f5d74003a90 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d1df700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74004810 0x7f5d74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.407+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d74003a90 0x7f5d74003ea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5d84002250 tx=0x7f5d880ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40b62a9deca4903 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840070d0 con 0x7f5d74003a90 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84007230 con 0x7f5d74003a90 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8c9de700 1 -- 172.21.15.92:0/3651174763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840075f0 con 0x7f5d74003a90 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8fc44700 1 -- 172.21.15.92:0/3651174763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d74003a90 msgr2=0x7f5d74003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/3651174763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d74003a90 0x7f5d74003ea0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5d84002250 tx=0x7f5d880ce4d0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8e1e1700 1 -- 172.21.15.92:0/3651174763 reap_dead start 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8fc44700 1 -- 172.21.15.92:0/3651174763 shutdown_connections 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8fc44700 1 -- 172.21.15.92:0/3651174763 >> 172.21.15.92:0/3651174763 conn(0x7f5d880ce600 msgr2=0x7f5d880cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8fc44700 1 -- 172.21.15.92:0/3651174763 shutdown_connections 2023-10-06T03:03:15.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.408+0000 7f5d8fc44700 1 -- 172.21.15.92:0/3651174763 wait complete. 2023-10-06T03:03:15.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.409+0000 7f5d8fc44700 1 Processor -- start 2023-10-06T03:03:15.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.409+0000 7f5d8fc44700 1 -- start start 2023-10-06T03:03:15.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8fc44700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74003a90 0x7f5d880c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8fc44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d881553a0 0x7f5d8816a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8fc44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 0x7f5d880c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8fc44700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880c6060 con 0x7f5d880c56b0 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8fc44700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880b1f50 con 0x7f5d74003a90 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8fc44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8816ac30 con 0x7f5d881553a0 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8e1e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 0x7f5d880c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8e1e1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 0x7f5d880c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46506/0 (socket says 172.21.15.92:46506) 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8e1e1700 1 -- 172.21.15.92:0/4027241681 learned_addr learned my addr 172.21.15.92:0/4027241681 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:15.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8e1e1700 1 -- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d881553a0 msgr2=0x7f5d8816a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8d1df700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d881553a0 0x7f5d8816a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74003a90 0x7f5d880c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.410+0000 7f5d8e1e1700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d881553a0 0x7f5d8816a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d8e1e1700 1 -- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74003a90 msgr2=0x7f5d880c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d8e1e1700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74003a90 0x7f5d880c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d8e1e1700 1 -- 172.21.15.92:0/4027241681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8816af10 con 0x7f5d880c56b0 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d8e1e1700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 0x7f5d880c5b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5d8806b640 tx=0x7f5d80009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80013420 con 0x7f5d880c56b0 2023-10-06T03:03:15.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d800135c0 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8816b1f0 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80013980 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.411+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d88157de0 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.412+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d880b2100 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.412+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5d80013cf0 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.413+0000 7f5d7dffb700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d6c066fc0 0x7f5d6c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.413+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d80084480 con 0x7f5d880c56b0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.413+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d6c066fc0 0x7f5d6c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.413+0000 7f5d8d9e0700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d6c066fc0 0x7f5d6c069470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d880b44a0 tx=0x7f5d84015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.417+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d880b2100 con 0x7f5d880c56b0 2023-10-06T03:03:15.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.643+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d880b20c0 con 0x7f5d880c56b0 2023-10-06T03:03:15.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.643+0000 7f5d7dffb700 1 -- 172.21.15.92:0/4027241681 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f5d80055030 con 0x7f5d880c56b0 2023-10-06T03:03:15.645 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:15.645 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:01.941144+0000","modified":"2023-10-06T03:03:05.571224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3249599548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3249599548},{"type":"v1","addr":"172.21.15.92:6837","nonce":3249599548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:03:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d6c066fc0 msgr2=0x7f5d6c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d6c066fc0 0x7f5d6c069470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d880b44a0 tx=0x7f5d84015000 comp rx=0 tx=0).stop 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 msgr2=0x7f5d880c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 0x7f5d880c5b20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5d8806b640 tx=0x7f5d80009040 comp rx=0 tx=0).stop 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 shutdown_connections 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5d6c066fc0 0x7f5d6c069470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d880c56b0 0x7f5d880c5b20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d74003a90 0x7f5d880c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 --2- 172.21.15.92:0/4027241681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d881553a0 0x7f5d8816a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.645+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 >> 172.21.15.92:0/4027241681 conn(0x7f5d880ce600 msgr2=0x7f5d88155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.646+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 shutdown_connections 2023-10-06T03:03:15.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.646+0000 7f5d8fc44700 1 -- 172.21.15.92:0/4027241681 wait complete. 2023-10-06T03:03:15.649 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2023-10-06T03:03:15.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.772+0000 7f8c57cd1700 1 Processor -- start 2023-10-06T03:03:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.772+0000 7f8c57cd1700 1 -- start start 2023-10-06T03:03:15.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c57cd1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8c50159890 0x7f8c5015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c57cd1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50069560 0x7f8c500699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c57cd1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c50069ee0 con 0x7f8c50159c60 2023-10-06T03:03:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c57cd1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5015a160 con 0x7f8c50159890 2023-10-06T03:03:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c57cd1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500d13a0 con 0x7f8c50069560 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c55a6d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8c50159890 0x7f8c5015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c55a6d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8c50159890 0x7f8c5015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:51102/0 (socket says 172.21.15.92:51102) 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c55a6d700 1 -- 172.21.15.92:0/4111920555 learned_addr learned my addr 172.21.15.92:0/4111920555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c5526c700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f8c50159c60 0x7f8c5015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:44350/0 (socket says 172.21.15.92:44350) 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.773+0000 7f8c5626e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50069560 0x7f8c500699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c55a6d700 1 -- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50069560 msgr2=0x7f8c500699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c55a6d700 1 --2- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50069560 0x7f8c500699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c55a6d700 1 -- 172.21.15.92:0/4111920555 >> v1:172.21.15.136:6789/0 conn(0x7f8c50159c60 legacy=0x7f8c5015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c55a6d700 1 -- 172.21.15.92:0/4111920555 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5015e140 con 0x7f8c50159890 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c5626e700 1 --2- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50069560 0x7f8c500699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c55a6d700 1 --2- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159890 0x7f8c5015abd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009f20 tx=0x7f8c4c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5e8e2b1618cd9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c54a6b700 1 -- 172.21.15.92:0/4111920555 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c00e070 con 0x7f8c50159890 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c54a6b700 1 -- 172.21.15.92:0/4111920555 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c00b9d0 con 0x7f8c50159890 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.774+0000 7f8c54a6b700 1 -- 172.21.15.92:0/4111920555 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0092e0 con 0x7f8c50159890 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 -- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159890 msgr2=0x7f8c5015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159890 0x7f8c5015abd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009f20 tx=0x7f8c4c009840 comp rx=0 tx=0).stop 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 -- 172.21.15.92:0/4111920555 shutdown_connections 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159890 0x7f8c5015abd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/4111920555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50069560 0x7f8c500699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 -- 172.21.15.92:0/4111920555 >> 172.21.15.92:0/4111920555 conn(0x7f8c50068e20 msgr2=0x7f8c500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 -- 172.21.15.92:0/4111920555 shutdown_connections 2023-10-06T03:03:15.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.775+0000 7f8c57cd1700 1 -- 172.21.15.92:0/4111920555 wait complete. 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.776+0000 7f8c57cd1700 1 Processor -- start 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.776+0000 7f8c57cd1700 1 -- start start 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 0x7f8c501699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50159890 0x7f8c500ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159c60 0x7f8c500b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4970 con 0x7f8c50069560 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b3d60 con 0x7f8c50159c60 2023-10-06T03:03:15.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4040 con 0x7f8c50159890 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 0x7f8c501699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 0x7f8c501699b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46518/0 (socket says 172.21.15.92:46518) 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 -- 172.21.15.92:0/3723502535 learned_addr learned my addr 172.21.15.92:0/3723502535 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c5626e700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159c60 0x7f8c500b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c5526c700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50159890 0x7f8c500ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 -- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50159890 msgr2=0x7f8c500ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50159890 0x7f8c500ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 -- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159c60 msgr2=0x7f8c500b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159c60 0x7f8c500b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 -- 172.21.15.92:0/3723502535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48009420 con 0x7f8c50069560 2023-10-06T03:03:15.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c55a6d700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 0x7f8c501699b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b610 tx=0x7f8c4c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0089a0 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c008b00 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c009ae0 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c026460 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.778+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c500b4600 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.779+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5004c3c0 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.779+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8c4c003040 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.779+0000 7f8c45ffb700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c34066fc0 0x7f8c34069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.779+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8c4c0864a0 con 0x7f8c50069560 2023-10-06T03:03:15.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.780+0000 7f8c5526c700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c34066fc0 0x7f8c34069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:15.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.780+0000 7f8c5526c700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c34066fc0 0x7f8c34069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c500b6600 tx=0x7f8c40007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:15.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.785+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c4c053620 con 0x7f8c50069560 2023-10-06T03:03:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.969+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c5006a5b0 con 0x7f8c50069560 2023-10-06T03:03:15.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.970+0000 7f8c45ffb700 1 -- 172.21.15.92:0/3723502535 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16868 (secure 0 0 0) 0x7f8c4c056c40 con 0x7f8c50069560 2023-10-06T03:03:15.972 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:15.972 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":54,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:03.471780+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:59.601586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:00.905218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c34066fc0 msgr2=0x7f8c34069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c34066fc0 0x7f8c34069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c500b6600 tx=0x7f8c40007eb0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 msgr2=0x7f8c501699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:15.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 0x7f8c501699b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00b610 tx=0x7f8c4c018000 comp rx=0 tx=0).stop 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 shutdown_connections 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c34066fc0 0x7f8c34069470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c50069560 0x7f8c501699b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c50159c60 0x7f8c500b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 --2- 172.21.15.92:0/3723502535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c50159890 0x7f8c500ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 >> 172.21.15.92:0/3723502535 conn(0x7f8c50068e20 msgr2=0x7f8c500b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 shutdown_connections 2023-10-06T03:03:15.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:15.972+0000 7f8c57cd1700 1 -- 172.21.15.92:0/3723502535 wait complete. 2023-10-06T03:03:15.986 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:03:15.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:16.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.100+0000 7fe0208a3700 1 Processor -- start 2023-10-06T03:03:16.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.100+0000 7fe0208a3700 1 -- start start 2023-10-06T03:03:16.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe0208a3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe01c159890 0x7fe01c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe0208a3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c15a160 con 0x7fe01c159c60 2023-10-06T03:03:16.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe0208a3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe01c0d13a0 con 0x7fe01c069560 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe0208a3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01c15e140 con 0x7fe01c159890 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe019d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe01c159c60 0x7fe01c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40720/0 (socket says 172.21.15.92:40720) 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe019d9b700 1 -- 172.21.15.92:0/70956475 learned_addr learned my addr 172.21.15.92:0/70956475 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 --2- 172.21.15.92:0/70956475 >> v2:172.21.15.136:3300/0 conn(0x7fe01c159890 0x7fe01c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938158304 0 0) 0x7fe01c15a160 con 0x7fe01c159c60 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe008003570 con 0x7fe01c159c60 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668994032 0 0) 0x7fe01c0d13a0 con 0x7fe01c069560 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe01c15a160 con 0x7fe01c069560 2023-10-06T03:03:16.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842770617 0 0) 0x7fe008003570 con 0x7fe01c159c60 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 >> v2:172.21.15.136:3300/0 conn(0x7fe01c159890 msgr2=0x7fe01c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 --2- 172.21.15.92:0/70956475 >> v2:172.21.15.136:3300/0 conn(0x7fe01c159890 0x7fe01c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 >> v1:172.21.15.136:6789/0 conn(0x7fe01c069560 legacy=0x7fe01c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c15a3d0 con 0x7fe01c159c60 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fe004001470 con 0x7fe01c159c60 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 >> v1:172.21.15.92:6789/0 conn(0x7fe01c159c60 legacy=0x7fe01c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe008003bf0 0x7fe008003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008004810 0x7fe008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0080051c0 0x7fe00800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c15e140 con 0x7fe008003bf0 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00800c0b0 con 0x7fe0080051c0 2023-10-06T03:03:16.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00800c270 con 0x7fe008004810 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 --2- 172.21.15.92:0/70956475 >> v2:172.21.15.136:3300/0 conn(0x7fe01c159890 0x7fe01c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe008003bf0 0x7fe008003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01ad9d700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0080051c0 0x7fe00800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe019d9b700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008004810 0x7fe008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 -- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008004810 msgr2=0x7fe008004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008004810 0x7fe008004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 -- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0080051c0 msgr2=0x7fe00800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0080051c0 0x7fe00800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 -- 172.21.15.92:0/70956475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008003570 con 0x7fe008003bf0 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01ad9d700 1 -- 172.21.15.92:0/70956475 reap_dead start 2023-10-06T03:03:16.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01a59c700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe008003bf0 0x7fe008003fe0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe01400e3c0 tx=0x7fe01400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e81ff58576a959 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014009c50 con 0x7fe008003bf0 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01400f040 con 0x7fe008003bf0 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.102+0000 7fe01959a700 1 -- 172.21.15.92:0/70956475 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01400eea0 con 0x7fe008003bf0 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 -- 172.21.15.92:0/70956475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe008003bf0 msgr2=0x7fe008003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe008003bf0 0x7fe008003fe0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe01400e3c0 tx=0x7fe01400b040 comp rx=0 tx=0).stop 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 -- 172.21.15.92:0/70956475 shutdown_connections 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 --2- 172.21.15.92:0/70956475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe008003bf0 0x7fe008003fe0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 -- 172.21.15.92:0/70956475 >> 172.21.15.92:0/70956475 conn(0x7fe01c068e20 msgr2=0x7fe01c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 -- 172.21.15.92:0/70956475 shutdown_connections 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.103+0000 7fe0208a3700 1 -- 172.21.15.92:0/70956475 wait complete. 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.104+0000 7fe0208a3700 1 Processor -- start 2023-10-06T03:03:16.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.104+0000 7fe0208a3700 1 -- start start 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008003bf0 0x7fe01c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 0x7fe01c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe01c0b3670 0x7fe01c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c158e20 con 0x7fe01c0b4990 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0b2180 con 0x7fe01c0b3670 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c0b1570 con 0x7fe008003bf0 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 0x7fe01c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 0x7fe01c155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46536/0 (socket says 172.21.15.92:46536) 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 -- 172.21.15.92:0/3499299734 learned_addr learned my addr 172.21.15.92:0/3499299734 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe01ad9d700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe01c0b3670 0x7fe01c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe01a59c700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008003bf0 0x7fe01c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 -- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008003bf0 msgr2=0x7fe01c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008003bf0 0x7fe01c0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 -- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe01c0b3670 msgr2=0x7fe01c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe01c0b3670 0x7fe01c0b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 -- 172.21.15.92:0/3499299734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c0b1850 con 0x7fe01c0b4990 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe01a59c700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008003bf0 0x7fe01c0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe019d9b700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 0x7fe01c155bb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe004000fd0 tx=0x7fe004001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004018030 con 0x7fe01c0b4990 2023-10-06T03:03:16.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00400f6c0 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c0b1b30 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400fa60 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c1560f0 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.106+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c0b1fd0 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.107+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe00401b070 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.107+0000 7fe0127fc700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdffc066fc0 0x7fdffc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.107+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe004019070 con 0x7fe01c0b4990 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.107+0000 7fe01a59c700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdffc066fc0 0x7fdffc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.107+0000 7fe01a59c700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdffc066fc0 0x7fdffc069470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0140099f0 tx=0x7fe014019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.112+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0040518a0 con 0x7fe01c0b4990 2023-10-06T03:03:16.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.337+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe01c156310 con 0x7fe01c0b4990 2023-10-06T03:03:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.337+0000 7fe0127fc700 1 -- 172.21.15.92:0/3499299734 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fe004054ec0 con 0x7fe01c0b4990 2023-10-06T03:03:16.339 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:16.339 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:01.941144+0000","modified":"2023-10-06T03:03:05.571224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3249599548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3249599548},{"type":"v1","addr":"172.21.15.92:6837","nonce":3249599548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdffc066fc0 msgr2=0x7fdffc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdffc066fc0 0x7fdffc069470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0140099f0 tx=0x7fe014019000 comp rx=0 tx=0).stop 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 msgr2=0x7fe01c155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 0x7fe01c155bb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe004000fd0 tx=0x7fe004001e10 comp rx=0 tx=0).stop 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 shutdown_connections 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdffc066fc0 0x7fdffc069470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe01c0b3670 0x7fe01c0b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe008003bf0 0x7fe01c0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 --2- 172.21.15.92:0/3499299734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe01c0b4990 0x7fe01c155bb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 >> 172.21.15.92:0/3499299734 conn(0x7fe01c068e20 msgr2=0x7fe01c149e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 shutdown_connections 2023-10-06T03:03:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.339+0000 7fe0208a3700 1 -- 172.21.15.92:0/3499299734 wait complete. 2023-10-06T03:03:16.342 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2023-10-06T03:03:16.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:16.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.466+0000 7f90c4340700 1 Processor -- start 2023-10-06T03:03:16.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.466+0000 7f90c4340700 1 -- start start 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c4340700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90bc06aa40 0x7f90bc0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c4340700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f90bc06bac0 0x7f90bc06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c4340700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c4340700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc0cc210 con 0x7f90bc06aa40 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c4340700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc0cc4f0 con 0x7f90bc06bac0 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c4340700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0cc7d0 con 0x7f90bc06b350 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.467+0000 7f90c20dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90bc06aa40 0x7f90bc0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90bc06aa40 0x7f90bc0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46538/0 (socket says 172.21.15.92:46538) 2023-10-06T03:03:16.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 -- 172.21.15.92:0/3205116220 learned_addr learned my addr 172.21.15.92:0/3205116220 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c28dd700 1 --2- 172.21.15.92:0/3205116220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c18db700 1 --2- 172.21.15.92:0/3205116220 >> v2:172.21.15.136:3300/0 conn(0x7f90bc06bac0 0x7f90bc06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 -- 172.21.15.92:0/3205116220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 msgr2=0x7f90bc0c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 --2- 172.21.15.92:0/3205116220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc0c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 -- 172.21.15.92:0/3205116220 >> v2:172.21.15.136:3300/0 conn(0x7f90bc06bac0 msgr2=0x7f90bc06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 --2- 172.21.15.92:0/3205116220 >> v2:172.21.15.136:3300/0 conn(0x7f90bc06bac0 0x7f90bc06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 -- 172.21.15.92:0/3205116220 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc0ccab0 con 0x7f90bc06aa40 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c20dc700 1 --2- 172.21.15.92:0/3205116220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06aa40 0x7f90bc0cbc00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0cc140 tx=0x7f90b800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d380d09b8dff48e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c10da700 1 -- 172.21.15.92:0/3205116220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8005c10 con 0x7f90bc06aa40 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c10da700 1 -- 172.21.15.92:0/3205116220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b8005d70 con 0x7f90bc06aa40 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.468+0000 7f90c10da700 1 -- 172.21.15.92:0/3205116220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b800dc30 con 0x7f90bc06aa40 2023-10-06T03:03:16.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 -- 172.21.15.92:0/3205116220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06aa40 msgr2=0x7f90bc0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 --2- 172.21.15.92:0/3205116220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06aa40 0x7f90bc0cbc00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0cc140 tx=0x7f90b800b4c0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 -- 172.21.15.92:0/3205116220 shutdown_connections 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 --2- 172.21.15.92:0/3205116220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc0c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 --2- 172.21.15.92:0/3205116220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06aa40 0x7f90bc0cbc00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 --2- 172.21.15.92:0/3205116220 >> v2:172.21.15.136:3300/0 conn(0x7f90bc06bac0 0x7f90bc06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 -- 172.21.15.92:0/3205116220 >> 172.21.15.92:0/3205116220 conn(0x7f90bc0cd670 msgr2=0x7f90bc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 -- 172.21.15.92:0/3205116220 shutdown_connections 2023-10-06T03:03:16.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.469+0000 7f90c4340700 1 -- 172.21.15.92:0/3205116220 wait complete. 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.470+0000 7f90c4340700 1 Processor -- start 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.470+0000 7f90c4340700 1 -- start start 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c4340700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06aa40 0x7f90bc0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c4340700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c4340700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 0x7f90bc16a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c4340700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0b3290 con 0x7f90bc06bac0 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c4340700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0b2e00 con 0x7f90bc06aa40 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c4340700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0b21f0 con 0x7f90bc06b350 2023-10-06T03:03:16.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 0x7f90bc16a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 0x7f90bc16a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46542/0 (socket says 172.21.15.92:46542) 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 -- 172.21.15.92:0/3332884042 learned_addr learned my addr 172.21.15.92:0/3332884042 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c20dc700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06aa40 0x7f90bc0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c18db700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 -- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 msgr2=0x7f90bc169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 -- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06aa40 msgr2=0x7f90bc0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06aa40 0x7f90bc0b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 -- 172.21.15.92:0/3332884042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b800b720 con 0x7f90bc06bac0 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c18db700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90c28dd700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 0x7f90bc16a170 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009fd0 tx=0x7f90b4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.471+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b400eca0 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.472+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b400ee00 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.472+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc0b24d0 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.472+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4014670 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.472+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc0b2a90 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.472+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc0b2f90 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.473+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f90b40149e0 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.473+0000 7f90b27fc700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90a0066fc0 0x7f90a0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.473+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f90b4084d90 con 0x7f90bc06bac0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.473+0000 7f90c20dc700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90a0066fc0 0x7f90a0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.474+0000 7f90c20dc700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90a0066fc0 0x7f90a0069470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90b800b500 tx=0x7f90b801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.477+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90b4051eb0 con 0x7f90bc06bac0 2023-10-06T03:03:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.704+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90bc04c3c0 con 0x7f90bc06bac0 2023-10-06T03:03:16.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.705+0000 7f90b27fc700 1 -- 172.21.15.92:0/3332884042 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f90b40554d0 con 0x7f90bc06bac0 2023-10-06T03:03:16.705 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:16.706 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:01.941144+0000","modified":"2023-10-06T03:03:05.571224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5364},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5364":{"gid":5364,"name":"a","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3249599548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3249599548},{"type":"v1","addr":"172.21.15.92:6837","nonce":3249599548}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90a0066fc0 msgr2=0x7f90a0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90a0066fc0 0x7f90a0069470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90b800b500 tx=0x7f90b801a000 comp rx=0 tx=0).stop 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 msgr2=0x7f90bc16a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 0x7f90bc16a170 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009fd0 tx=0x7f90b4009900 comp rx=0 tx=0).stop 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 shutdown_connections 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90a0066fc0 0x7f90a0069470 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90bc06bac0 0x7f90bc16a170 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06aa40 0x7f90bc0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 --2- 172.21.15.92:0/3332884042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06b350 0x7f90bc169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 >> 172.21.15.92:0/3332884042 conn(0x7f90bc0cd670 msgr2=0x7f90bc0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:16.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 shutdown_connections 2023-10-06T03:03:16.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.707+0000 7f90c4340700 1 -- 172.21.15.92:0/3332884042 wait complete. 2023-10-06T03:03:16.709 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 30 2023-10-06T03:03:16.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.831+0000 7fdc08931700 1 Processor -- start 2023-10-06T03:03:16.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.831+0000 7fdc08931700 1 -- start start 2023-10-06T03:03:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdc08931700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc04159c60 0x7fdc0415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdc08931700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0415a160 con 0x7fdc04159890 2023-10-06T03:03:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdc08931700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc040d13a0 con 0x7fdc0415db10 2023-10-06T03:03:16.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdc08931700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0415df50 con 0x7fdc04159c60 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdc0259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc04159890 0x7fdc0415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40732/0 (socket says 172.21.15.92:40732) 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdc0259c700 1 -- 172.21.15.92:0/1179980657 learned_addr learned my addr 172.21.15.92:0/1179980657 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.832+0000 7fdbf9d9b700 1 --2- 172.21.15.92:0/1179980657 >> v2:172.21.15.92:3300/0 conn(0x7fdc04159c60 0x7fdc0415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc01d9b700 1 -- 172.21.15.92:0/1179980657 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427591892 0 0) 0x7fdc0415a160 con 0x7fdc04159890 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc01d9b700 1 -- 172.21.15.92:0/1179980657 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf0003570 con 0x7fdc04159890 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/1179980657 >> v1:172.21.15.136:6789/0 conn(0x7fdc0415db10 legacy=0x7fdc04069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/1179980657 >> v1:172.21.15.92:6789/0 conn(0x7fdc04159890 legacy=0x7fdc0415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/1179980657 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0415e140 con 0x7fdc04159c60 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdbf9d9b700 1 --2- 172.21.15.92:0/1179980657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc0415d5d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0098e0 tx=0x7fdbec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c1bafca2b0a006 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc01d9b700 1 -- 172.21.15.92:0/1179980657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec011630 con 0x7fdc04159c60 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc01d9b700 1 -- 172.21.15.92:0/1179980657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec011790 con 0x7fdc04159c60 2023-10-06T03:03:16.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc01d9b700 1 -- 172.21.15.92:0/1179980657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec011b30 con 0x7fdc04159c60 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc08931700 1 -- 172.21.15.92:0/1179980657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 msgr2=0x7fdc0415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc08931700 1 --2- 172.21.15.92:0/1179980657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc0415d5d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0098e0 tx=0x7fdbec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc08931700 1 -- 172.21.15.92:0/1179980657 shutdown_connections 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc08931700 1 --2- 172.21.15.92:0/1179980657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc0415d5d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.833+0000 7fdc08931700 1 -- 172.21.15.92:0/1179980657 >> 172.21.15.92:0/1179980657 conn(0x7fdc04068e20 msgr2=0x7fdc040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.834+0000 7fdc08931700 1 -- 172.21.15.92:0/1179980657 shutdown_connections 2023-10-06T03:03:16.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.834+0000 7fdc08931700 1 -- 172.21.15.92:0/1179980657 wait complete. 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 Processor -- start 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 -- start start 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc04159890 0x7fdc04155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc040b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc0415db10 0x7fdc040b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0416a300 con 0x7fdc04159c60 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0416a5e0 con 0x7fdc04159890 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.835+0000 7fdc08931700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0416a8c0 con 0x7fdc0415db10 2023-10-06T03:03:16.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc040b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc040b5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46566/0 (socket says 172.21.15.92:46566) 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/215426445 learned_addr learned my addr 172.21.15.92:0/215426445 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdc0259c700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc04159890 0x7fdc04155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc0415db10 msgr2=0x7fdc040b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc0415db10 0x7fdc040b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc04159890 msgr2=0x7fdc04155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc04159890 0x7fdc04155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 -- 172.21.15.92:0/215426445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec009420 con 0x7fdc04159c60 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdc0259c700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc04159890 0x7fdc04155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbf9d9b700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc040b5b40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0030c0 tx=0x7fdbec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec000d10 con 0x7fdc04159c60 2023-10-06T03:03:16.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec020070 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0416aba0 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0276b0 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.836+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc04169f90 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.837+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0416b1c0 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.838+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fdbec01c450 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.838+0000 7fdbfaffd700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4066fc0 0x7fdbe4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.838+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdbec022070 con 0x7fdc04159c60 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.838+0000 7fdc0259c700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4066fc0 0x7fdbe4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:16.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.838+0000 7fdc0259c700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4066fc0 0x7fdbe4069470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007fd0 tx=0x7fdbf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:16.842+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbec0539f0 con 0x7fdc04159c60 2023-10-06T03:03:17.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.027+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc0404c3c0 con 0x7fdc04159c60 2023-10-06T03:03:17.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.028+0000 7fdbfaffd700 1 -- 172.21.15.92:0/215426445 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16868 (secure 0 0 0) 0x7fdbec057010 con 0x7fdc04159c60 2023-10-06T03:03:17.028 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:17.029 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":54,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:03.471780+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:02:59.601586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:00.905218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.029+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4066fc0 msgr2=0x7fdbe4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.029+0000 7fdc08931700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4066fc0 0x7fdbe4069470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007fd0 tx=0x7fdbf4009040 comp rx=0 tx=0).stop 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.029+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 msgr2=0x7fdc040b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc040b5b40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0030c0 tx=0x7fdbec016000 comp rx=0 tx=0).stop 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 shutdown_connections 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4066fc0 0x7fdbe4069470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc0415db10 0x7fdc040b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc04159890 0x7fdc04155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 --2- 172.21.15.92:0/215426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc04159c60 0x7fdc040b5b40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 >> 172.21.15.92:0/215426445 conn(0x7fdc04068e20 msgr2=0x7fdc04153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 shutdown_connections 2023-10-06T03:03:17.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.030+0000 7fdc08931700 1 -- 172.21.15.92:0/215426445 wait complete. 2023-10-06T03:03:17.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:03:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 Processor -- start 2023-10-06T03:03:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 -- start start 2023-10-06T03:03:17.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e34159c60 0x7f6e3415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6e3415db10 0x7f6e34069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415a160 con 0x7f6e34159890 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3415df50 con 0x7f6e34159c60 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.158+0000 7f6e38fd6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340d13a0 con 0x7f6e3415db10 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e34159c60 0x7f6e3415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6e34159c60 0x7f6e3415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46574/0 (socket says 172.21.15.92:46574) 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 -- 172.21.15.92:0/2035654045 learned_addr learned my addr 172.21.15.92:0/2035654045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e32d9d700 1 --2- 172.21.15.92:0/2035654045 >> v2:172.21.15.136:3300/0 conn(0x7f6e3415db10 0x7f6e34069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 -- 172.21.15.92:0/2035654045 >> v2:172.21.15.136:3300/0 conn(0x7f6e3415db10 msgr2=0x7f6e34069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 --2- 172.21.15.92:0/2035654045 >> v2:172.21.15.136:3300/0 conn(0x7f6e3415db10 0x7f6e34069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 -- 172.21.15.92:0/2035654045 >> v1:172.21.15.136:6789/0 conn(0x7f6e34159890 legacy=0x7f6e3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 -- 172.21.15.92:0/2035654045 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415e140 con 0x7f6e34159c60 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e29d9b700 1 --2- 172.21.15.92:0/2035654045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159c60 0x7f6e3415d5d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406a890 tx=0x7f6e1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b74aa82832e6d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e31d9b700 1 -- 172.21.15.92:0/2035654045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00b860 con 0x7f6e34159c60 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e31d9b700 1 -- 172.21.15.92:0/2035654045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00b9c0 con 0x7f6e34159c60 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.159+0000 7f6e31d9b700 1 -- 172.21.15.92:0/2035654045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00bd60 con 0x7f6e34159c60 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2035654045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159c60 msgr2=0x7f6e3415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2035654045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159c60 0x7f6e3415d5d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406a890 tx=0x7f6e1c009250 comp rx=0 tx=0).stop 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2035654045 shutdown_connections 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2035654045 >> v2:172.21.15.136:3300/0 conn(0x7f6e3415db10 0x7f6e34069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2035654045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159c60 0x7f6e3415d5d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2035654045 >> 172.21.15.92:0/2035654045 conn(0x7f6e34068e20 msgr2=0x7f6e340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:17.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2035654045 shutdown_connections 2023-10-06T03:03:17.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.160+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2035654045 wait complete. 2023-10-06T03:03:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.161+0000 7f6e38fd6700 1 Processor -- start 2023-10-06T03:03:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.161+0000 7f6e38fd6700 1 -- start start 2023-10-06T03:03:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 0x7f6e34149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e34159c60 0x7f6e340cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e3415db10 0x7f6e3416d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3416dec0 con 0x7f6e34159890 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34152420 con 0x7f6e34159c60 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34152700 con 0x7f6e3415db10 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 0x7f6e34149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 0x7f6e34149760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46582/0 (socket says 172.21.15.92:46582) 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 -- 172.21.15.92:0/2230456030 learned_addr learned my addr 172.21.15.92:0/2230456030 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e29d9b700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e34159c60 0x7f6e340cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 -- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e3415db10 msgr2=0x7f6e3416d0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e3415db10 0x7f6e3416d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 -- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e34159c60 msgr2=0x7f6e340cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e34159c60 0x7f6e340cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 -- 172.21.15.92:0/2230456030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c0094f0 con 0x7f6e34159890 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e3259c700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 0x7f6e34149760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c590 tx=0x7f6e2400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011590 con 0x7f6e34159890 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e240116f0 con 0x7f6e34159890 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e341529e0 con 0x7f6e34159890 2023-10-06T03:03:17.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011ab0 con 0x7f6e34159890 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.162+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e341520f0 con 0x7f6e34159890 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.163+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340c5590 con 0x7f6e34159890 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.164+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f6e2401b3b0 con 0x7f6e34159890 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.164+0000 7f6e2affd700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e14066fc0 0x7f6e14069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.164+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e24083c60 con 0x7f6e34159890 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.164+0000 7f6e29d9b700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e14066fc0 0x7f6e14069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.164+0000 7f6e29d9b700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e14066fc0 0x7f6e14069470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406a890 tx=0x7f6e1c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.168+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e24050d80 con 0x7f6e34159890 2023-10-06T03:03:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.392+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6e3404c3c0 con 0x7f6e34159890 2023-10-06T03:03:17.393 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:03:17.393+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:03:17.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.396+0000 7f6e2affd700 1 -- 172.21.15.92:0/2230456030 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6e24017070 con 0x7f6e34159890 2023-10-06T03:03:17.398 INFO:tasks.ceph.mds.a.smithi092.stderr: -1809> 2023-10-06T03:03:06.860+0000 7fa852b77700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:03:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.398+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e14066fc0 msgr2=0x7f6e14069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.398+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e14066fc0 0x7f6e14069470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e3406a890 tx=0x7f6e1c017000 comp rx=0 tx=0).stop 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.398+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 msgr2=0x7f6e34149760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.398+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 0x7f6e34149760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c590 tx=0x7f6e2400c950 comp rx=0 tx=0).stop 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 shutdown_connections 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e14066fc0 0x7f6e14069470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e34159890 0x7f6e34149760 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e3415db10 0x7f6e3416d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 --2- 172.21.15.92:0/2230456030 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e34159c60 0x7f6e340cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 >> 172.21.15.92:0/2230456030 conn(0x7f6e34068e20 msgr2=0x7f6e341575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 shutdown_connections 2023-10-06T03:03:17.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.399+0000 7f6e38fd6700 1 -- 172.21.15.92:0/2230456030 wait complete. 2023-10-06T03:03:17.401 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:03:17.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:03:17.421 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:03:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.530+0000 7f97da99f700 1 Processor -- start 2023-10-06T03:03:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.530+0000 7f97da99f700 1 -- start start 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da99f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d41407e0 0x7f97d406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da99f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f97d4140bb0 0x7f97d406a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da99f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40c6820 con 0x7f97d406d4f0 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da99f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d40c6de0 con 0x7f97d4140bb0 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da99f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d4141270 con 0x7f97d41407e0 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d919c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f97d4140bb0 0x7f97d406a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da19e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f97d406d4f0 0x7f97d415b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40740/0 (socket says 172.21.15.92:40740) 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97da19e700 1 -- 172.21.15.92:0/3970420209 learned_addr learned my addr 172.21.15.92:0/3970420209 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d999d700 1 --2- 172.21.15.92:0/3970420209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d41407e0 0x7f97d406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d919c700 1 -- 172.21.15.92:0/3970420209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d41407e0 msgr2=0x7f97d406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d919c700 1 --2- 172.21.15.92:0/3970420209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d41407e0 0x7f97d406a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d919c700 1 -- 172.21.15.92:0/3970420209 >> v1:172.21.15.92:6789/0 conn(0x7f97d406d4f0 legacy=0x7f97d415b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d899b700 1 -- 172.21.15.92:0/3970420209 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306592790 0 0) 0x7f97d40c6820 con 0x7f97d406d4f0 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d919c700 1 -- 172.21.15.92:0/3970420209 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d406acf0 con 0x7f97d4140bb0 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d919c700 1 --2- 172.21.15.92:0/3970420209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d4140bb0 0x7f97d406a6c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f97c40098e0 tx=0x7f97c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0f068be0b2ef8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d899b700 1 -- 172.21.15.92:0/3970420209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400b640 con 0x7f97d4140bb0 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d899b700 1 -- 172.21.15.92:0/3970420209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400b7a0 con 0x7f97d4140bb0 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.531+0000 7f97d899b700 1 -- 172.21.15.92:0/3970420209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400bb40 con 0x7f97d4140bb0 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 -- 172.21.15.92:0/3970420209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d4140bb0 msgr2=0x7f97d406a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 --2- 172.21.15.92:0/3970420209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d4140bb0 0x7f97d406a6c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f97c40098e0 tx=0x7f97c40091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 -- 172.21.15.92:0/3970420209 shutdown_connections 2023-10-06T03:03:17.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 --2- 172.21.15.92:0/3970420209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d41407e0 0x7f97d406a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 --2- 172.21.15.92:0/3970420209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d4140bb0 0x7f97d406a6c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 -- 172.21.15.92:0/3970420209 >> 172.21.15.92:0/3970420209 conn(0x7f97d40cc1e0 msgr2=0x7f97d40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 -- 172.21.15.92:0/3970420209 shutdown_connections 2023-10-06T03:03:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.532+0000 7f97da99f700 1 -- 172.21.15.92:0/3970420209 wait complete. 2023-10-06T03:03:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.533+0000 7f97da99f700 1 Processor -- start 2023-10-06T03:03:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.533+0000 7f97da99f700 1 -- start start 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da99f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d406d4f0 0x7f97d4167e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da99f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 0x7f97d40c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da99f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97d4140bb0 0x7f97d4156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da99f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d4158140 con 0x7f97d41407e0 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da99f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40c6030 con 0x7f97d4140bb0 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da99f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d4157400 con 0x7f97d406d4f0 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 0x7f97d40c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 0x7f97d40c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46616/0 (socket says 172.21.15.92:46616) 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 -- 172.21.15.92:0/2585253130 learned_addr learned my addr 172.21.15.92:0/2585253130 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d999d700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d406d4f0 0x7f97d4167e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da19e700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97d4140bb0 0x7f97d4156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 -- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d406d4f0 msgr2=0x7f97d4167e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d406d4f0 0x7f97d4167e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 -- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97d4140bb0 msgr2=0x7f97d4156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97d4140bb0 0x7f97d4156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 -- 172.21.15.92:0/2585253130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4009420 con 0x7f97d41407e0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97da19e700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97d4140bb0 0x7f97d4156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97d919c700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 0x7f97d40c5af0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f97c40093f0 tx=0x7f97c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.534+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400bb20 con 0x7f97d41407e0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.535+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c4011420 con 0x7f97d41407e0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.535+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d4168480 con 0x7f97d41407e0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.535+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40117e0 con 0x7f97d41407e0 2023-10-06T03:03:17.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.535+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d4168a40 con 0x7f97d41407e0 2023-10-06T03:03:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.536+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d41582b0 con 0x7f97d41407e0 2023-10-06T03:03:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.536+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f97c4011bd0 con 0x7f97d41407e0 2023-10-06T03:03:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.536+0000 7f97c9ffb700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97b8066fc0 0x7f97b8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.536+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f97c4085c10 con 0x7f97d41407e0 2023-10-06T03:03:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.537+0000 7f97d999d700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97b8066fc0 0x7f97b8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:17.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.537+0000 7f97d999d700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97b8066fc0 0x7f97b8069470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f97d406abd0 tx=0x7f97d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:17.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.541+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97c4052c80 con 0x7f97d41407e0 2023-10-06T03:03:17.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:17.769+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f97d406afd0 con 0x7f97d41407e0 2023-10-06T03:03:18.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.397+0000 7f97c9ffb700 1 -- 172.21.15.92:0/2585253130 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f97c40562a0 con 0x7f97d41407e0 2023-10-06T03:03:18.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.399+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97b8066fc0 msgr2=0x7f97b8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.399+0000 7f97da99f700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97b8066fc0 0x7f97b8069470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f97d406abd0 tx=0x7f97d0008040 comp rx=0 tx=0).stop 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.399+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 msgr2=0x7f97d40c5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.399+0000 7f97da99f700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 0x7f97d40c5af0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f97c40093f0 tx=0x7f97c4017000 comp rx=0 tx=0).stop 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.399+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 shutdown_connections 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.399+0000 7f97da99f700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f97b8066fc0 0x7f97b8069470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.400+0000 7f97da99f700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97d4140bb0 0x7f97d4156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.400+0000 7f97da99f700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97d406d4f0 0x7f97d4167e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.400+0000 7f97da99f700 1 --2- 172.21.15.92:0/2585253130 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f97d41407e0 0x7f97d40c5af0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.400+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 >> 172.21.15.92:0/2585253130 conn(0x7f97d40cc1e0 msgr2=0x7f97d40cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.400+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 shutdown_connections 2023-10-06T03:03:18.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.400+0000 7f97da99f700 1 -- 172.21.15.92:0/2585253130 wait complete. 2023-10-06T03:03:18.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:03:18.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.577+0000 7f4cfa55b700 1 Processor -- start 2023-10-06T03:03:18.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.577+0000 7f4cfa55b700 1 -- start start 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cfa55b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cfa55b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4cf40b0790 0x7f4cf40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cfa55b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf40cfa00 con 0x7f4cf41591e0 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cfa55b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf40cb580 con 0x7f4cf40b0790 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cfa55b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40b2de0 con 0x7f4cf40b03c0 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cf9559700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:18.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cf8d58700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4cf40b0790 0x7f4cf40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cf9d5a700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f4cf41591e0 0x7f4cf4159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:44406/0 (socket says 172.21.15.92:44406) 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cf8d58700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4cf40b0790 0x7f4cf40b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:51212/0 (socket says 172.21.15.92:51212) 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cf9d5a700 1 -- 172.21.15.92:0/3779173116 learned_addr learned my addr 172.21.15.92:0/3779173116 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.578+0000 7f4cf9559700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:52012/0 (socket says 172.21.15.92:52012) 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf8d58700 1 -- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 msgr2=0x7f4cf40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf8d58700 1 --2- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf8d58700 1 -- 172.21.15.92:0/3779173116 >> v1:172.21.15.136:6789/0 conn(0x7f4cf41591e0 legacy=0x7f4cf4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf8d58700 1 -- 172.21.15.92:0/3779173116 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40cf160 con 0x7f4cf40b0790 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf9559700 1 --2- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf8d58700 1 --2- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40b28a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80098e0 tx=0x7f4ce80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac18b6e664c8899 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:18.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf3fff700 1 -- 172.21.15.92:0/3779173116 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8011630 con 0x7f4cf40b0790 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf3fff700 1 -- 172.21.15.92:0/3779173116 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce8011790 con 0x7f4cf40b0790 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cf3fff700 1 -- 172.21.15.92:0/3779173116 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8011b30 con 0x7f4cf40b0790 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cfa55b700 1 -- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 msgr2=0x7f4cf40b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.579+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40b28a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80098e0 tx=0x7f4ce80091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.580+0000 7f4cfa55b700 1 -- 172.21.15.92:0/3779173116 shutdown_connections 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.580+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.580+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/3779173116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40b28a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.580+0000 7f4cfa55b700 1 -- 172.21.15.92:0/3779173116 >> 172.21.15.92:0/3779173116 conn(0x7f4cf4063230 msgr2=0x7f4cf4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.580+0000 7f4cfa55b700 1 -- 172.21.15.92:0/3779173116 shutdown_connections 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.580+0000 7f4cfa55b700 1 -- 172.21.15.92:0/3779173116 wait complete. 2023-10-06T03:03:18.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.581+0000 7f4cfa55b700 1 Processor -- start 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.581+0000 7f4cfa55b700 1 -- start start 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf4157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 0x7f4cf415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf415b0f0 con 0x7f4cf41591e0 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4157b90 con 0x7f4cf40b0790 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40c62c0 con 0x7f4cf40b03c0 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 0x7f4cf415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 0x7f4cf415d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46626/0 (socket says 172.21.15.92:46626) 2023-10-06T03:03:18.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 -- 172.21.15.92:0/1362211457 learned_addr learned my addr 172.21.15.92:0/1362211457 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf8d58700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9559700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf4157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 -- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 msgr2=0x7f4cf4157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf4157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 -- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 msgr2=0x7f4cf40c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 -- 172.21.15.92:0/1362211457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec009a20 con 0x7f4cf41591e0 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9559700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf4157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf9d5a700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 0x7f4cf415d500 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c7e0 tx=0x7f4ce4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4005b30 con 0x7f4cf41591e0 2023-10-06T03:03:18.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4005c90 con 0x7f4cf41591e0 2023-10-06T03:03:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4007320 con 0x7f4cf41591e0 2023-10-06T03:03:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.582+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce8009420 con 0x7f4cf41591e0 2023-10-06T03:03:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.583+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf40c6880 con 0x7f4cf41591e0 2023-10-06T03:03:18.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.583+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf404c3c0 con 0x7f4cf41591e0 2023-10-06T03:03:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.584+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f4ce401b3b0 con 0x7f4cf41591e0 2023-10-06T03:03:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.584+0000 7f4cf17fa700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ce0066fc0 0x7f4ce0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.584+0000 7f4cf9559700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ce0066fc0 0x7f4ce0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.584+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f4ce4083cc0 con 0x7f4cf41591e0 2023-10-06T03:03:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.585+0000 7f4cf9559700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ce0066fc0 0x7f4ce0069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0099f0 tx=0x7f4cec00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:18.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.589+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ce408a080 con 0x7f4cf41591e0 2023-10-06T03:03:18.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:18.770+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4cf40cad50 con 0x7f4cf41591e0 2023-10-06T03:03:19.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.639+0000 7f4cf17fa700 1 -- 172.21.15.92:0/1362211457 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4ce4017020 con 0x7f4cf41591e0 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ce0066fc0 msgr2=0x7f4ce0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ce0066fc0 0x7f4ce0069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0099f0 tx=0x7f4cec00b040 comp rx=0 tx=0).stop 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 msgr2=0x7f4cf415d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 0x7f4cf415d500 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400c7e0 tx=0x7f4ce4007ea0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 shutdown_connections 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ce0066fc0 0x7f4ce0069470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cf41591e0 0x7f4cf415d500 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4cf40b03c0 0x7f4cf4157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 --2- 172.21.15.92:0/1362211457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4cf40b0790 0x7f4cf40c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 >> 172.21.15.92:0/1362211457 conn(0x7f4cf4063230 msgr2=0x7f4cf40cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 shutdown_connections 2023-10-06T03:03:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.642+0000 7f4cfa55b700 1 -- 172.21.15.92:0/1362211457 wait complete. 2023-10-06T03:03:19.644 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:03:19.655 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:03:19.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.773+0000 7fddfb382700 1 Processor -- start 2023-10-06T03:03:19.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.773+0000 7fddfb382700 1 -- start start 2023-10-06T03:03:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddfb382700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 0x7fddf40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddfb382700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fddf40b53d0 0x7fddf40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddfb382700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf40b5b00 con 0x7fddf40cce40 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddfb382700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf40b5da0 con 0x7fddf40b53d0 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddfb382700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b6080 con 0x7fddf4159ec0 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddf991f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fddf40cce40 0x7fddf40cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57666/0 (socket says 172.21.15.92:57666) 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.774+0000 7fddf991f700 1 -- 172.21.15.92:0/290595508 learned_addr learned my addr 172.21.15.92:0/290595508 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1294309977 0 0) 0x7fddf40b5b00 con 0x7fddf40cce40 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 0x7fddf40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf891d700 1 --2- 172.21.15.92:0/290595508 >> v2:172.21.15.136:3300/0 conn(0x7fddf40b53d0 0x7fddf40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde0003570 con 0x7fddf40cce40 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554359471 0 0) 0x7fdde0003570 con 0x7fddf40cce40 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 msgr2=0x7fddf40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 0x7fddf40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 >> v2:172.21.15.136:3300/0 conn(0x7fddf40b53d0 msgr2=0x7fddf40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 --2- 172.21.15.92:0/290595508 >> v2:172.21.15.136:3300/0 conn(0x7fddf40b53d0 0x7fddf40cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40cc100 con 0x7fddf40cce40 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fddec001470 con 0x7fddf40cce40 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 >> v1:172.21.15.92:6789/0 conn(0x7fddf40cce40 legacy=0x7fddf40cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdde0003a60 0x7fdde0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdde0004810 0x7fdde0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 0x7fdde00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b5da0 con 0x7fdde0003a60 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b6080 con 0x7fdde0004810 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde000ba60 con 0x7fddf4159ec0 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdde0003a60 0x7fdde0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf991f700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 0x7fdde00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf891d700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdde0004810 0x7fdde0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 -- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 msgr2=0x7fdde00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddf4159ec0 0x7fdde00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 -- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdde0004810 msgr2=0x7fdde0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdde0004810 0x7fdde0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 -- 172.21.15.92:0/290595508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0003570 con 0x7fdde0003a60 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.775+0000 7fddf911e700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdde0003a60 0x7fdde0003e70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e300 tx=0x7fddf000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa09a7361782a0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf000eb50 con 0x7fdde0003a60 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf000f040 con 0x7fdde0003a60 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddebfff700 1 -- 172.21.15.92:0/290595508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0014480 con 0x7fdde0003a60 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddfb382700 1 -- 172.21.15.92:0/290595508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdde0003a60 msgr2=0x7fdde0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddfb382700 1 --2- 172.21.15.92:0/290595508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdde0003a60 0x7fdde0003e70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e300 tx=0x7fddf000b040 comp rx=0 tx=0).stop 2023-10-06T03:03:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddf991f700 1 -- 172.21.15.92:0/290595508 reap_dead start 2023-10-06T03:03:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddfb382700 1 -- 172.21.15.92:0/290595508 shutdown_connections 2023-10-06T03:03:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddfb382700 1 -- 172.21.15.92:0/290595508 >> 172.21.15.92:0/290595508 conn(0x7fddf40999f0 msgr2=0x7fddf40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddfb382700 1 -- 172.21.15.92:0/290595508 shutdown_connections 2023-10-06T03:03:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.776+0000 7fddfb382700 1 -- 172.21.15.92:0/290595508 wait complete. 2023-10-06T03:03:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.777+0000 7fddfb382700 1 Processor -- start 2023-10-06T03:03:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.777+0000 7fddfb382700 1 -- start start 2023-10-06T03:03:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddfb382700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdde0003a60 0x7fddf4151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddfb382700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddf4158e70 0x7fddf4154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddfb382700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 0x7fddf406a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddfb382700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4155c40 con 0x7fddf4155e00 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddfb382700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4153f60 con 0x7fddf4158e70 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddfb382700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4156fc0 con 0x7fdde0003a60 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 0x7fddf406a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 0x7fddf406a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44844/0 (socket says 172.21.15.92:44844) 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 -- 172.21.15.92:0/738091037 learned_addr learned my addr 172.21.15.92:0/738091037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf911e700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdde0003a60 0x7fddf4151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf891d700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddf4158e70 0x7fddf4154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 -- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdde0003a60 msgr2=0x7fddf4151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdde0003a60 0x7fddf4151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 -- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddf4158e70 msgr2=0x7fddf4154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddf4158e70 0x7fddf4154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf991f700 1 -- 172.21.15.92:0/738091037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf41569b0 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf911e700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdde0003a60 0x7fddf4151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.778+0000 7fddf891d700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddf4158e70 0x7fddf4154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.779+0000 7fddf991f700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 0x7fddf406a4e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fddf40ce2f0 tx=0x7fddec0014e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.779+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00c600 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.779+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00c760 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.779+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf4151a80 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.779+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec00cb20 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.779+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf4153c80 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.780+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf4151c10 con 0x7fddf4155e00 2023-10-06T03:03:19.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.780+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fddec0133e0 con 0x7fddf4155e00 2023-10-06T03:03:19.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.780+0000 7fdde97fa700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddd8066fc0 0x7fddd8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:19.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.781+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fddec083e60 con 0x7fddf4155e00 2023-10-06T03:03:19.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.781+0000 7fddf911e700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddd8066fc0 0x7fddd8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:19.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.781+0000 7fddf911e700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddd8066fc0 0x7fddd8069470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e300 tx=0x7fddf001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:19.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.785+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddec051160 con 0x7fddf4155e00 2023-10-06T03:03:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:19.967+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fddf41567f0 con 0x7fddf4155e00 2023-10-06T03:03:20.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.648+0000 7fdde97fa700 1 -- 172.21.15.92:0/738091037 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7fddec054780 con 0x7fddf4155e00 2023-10-06T03:03:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.651+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddd8066fc0 msgr2=0x7fddd8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.651+0000 7fddfb382700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddd8066fc0 0x7fddd8069470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e300 tx=0x7fddf001a000 comp rx=0 tx=0).stop 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.651+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 msgr2=0x7fddf406a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.651+0000 7fddfb382700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 0x7fddf406a4e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fddf40ce2f0 tx=0x7fddec0014e0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 shutdown_connections 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fddd8066fc0 0x7fddd8069470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddf4155e00 0x7fddf406a4e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdde0003a60 0x7fddf4151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 --2- 172.21.15.92:0/738091037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddf4158e70 0x7fddf4154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 >> 172.21.15.92:0/738091037 conn(0x7fddf40999f0 msgr2=0x7fddf4151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 shutdown_connections 2023-10-06T03:03:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.652+0000 7fddfb382700 1 -- 172.21.15.92:0/738091037 wait complete. 2023-10-06T03:03:20.654 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:03:20.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:03:20.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.780+0000 7f2d26bf3700 1 Processor -- start 2023-10-06T03:03:20.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.780+0000 7f2d26bf3700 1 -- start start 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.781+0000 7f2d26bf3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d2006bcd0 0x7f2d20140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.781+0000 7f2d26bf3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2015c2a0 con 0x7f2d200c9510 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.781+0000 7f2d26bf3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2015b690 con 0x7f2d2006c0a0 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.781+0000 7f2d26bf3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d200c9900 con 0x7f2d2006bcd0 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.781+0000 7f2d25190700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2d200c9510 0x7f2d200cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57668/0 (socket says 172.21.15.92:57668) 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.781+0000 7f2d25190700 1 -- 172.21.15.92:0/3210401744 learned_addr learned my addr 172.21.15.92:0/3210401744 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d2498f700 1 --2- 172.21.15.92:0/3210401744 >> v2:172.21.15.136:3300/0 conn(0x7f2d2006bcd0 0x7f2d20140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068421430 0 0) 0x7f2d2015c2a0 con 0x7f2d200c9510 2023-10-06T03:03:20.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d08003570 con 0x7f2d200c9510 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1397219391 0 0) 0x7f2d08003570 con 0x7f2d200c9510 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 >> v2:172.21.15.136:3300/0 conn(0x7f2d2006bcd0 msgr2=0x7f2d20140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 --2- 172.21.15.92:0/3210401744 >> v2:172.21.15.136:3300/0 conn(0x7f2d2006bcd0 0x7f2d20140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 >> v1:172.21.15.136:6789/0 conn(0x7f2d2006c0a0 legacy=0x7f2d200c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2015b970 con 0x7f2d200c9510 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2d18001470 con 0x7f2d200c9510 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d2498f700 1 --2- 172.21.15.92:0/3210401744 >> v2:172.21.15.136:3300/0 conn(0x7f2d2006bcd0 0x7f2d20140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 >> v1:172.21.15.92:6789/0 conn(0x7f2d200c9510 legacy=0x7f2d200cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 0x7f2d08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d08004810 0x7f2d08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d080052e0 0x7f2d0800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200c9900 con 0x7f2d080052e0 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0800c190 con 0x7f2d08003a60 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0800c350 con 0x7f2d08004810 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d25190700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d080052e0 0x7f2d0800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d2498f700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 0x7f2d08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d17fff700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d08004810 0x7f2d08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d25190700 1 -- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d08004810 msgr2=0x7f2d08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d25190700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d08004810 0x7f2d08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d25190700 1 -- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 msgr2=0x7f2d08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d25190700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d08003a60 0x7f2d08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.782+0000 7f2d25190700 1 -- 172.21.15.92:0/3210401744 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08003570 con 0x7f2d080052e0 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d25190700 1 -- 172.21.15.92:0/3210401744 reap_dead start 2023-10-06T03:03:20.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d25190700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d080052e0 0x7f2d0800ba60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c3c0 tx=0x7f2d18001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137bbda2f3cf8fc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800cc90 con 0x7f2d080052e0 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1800ce30 con 0x7f2d080052e0 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d177fe700 1 -- 172.21.15.92:0/3210401744 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18005b70 con 0x7f2d080052e0 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3210401744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d080052e0 msgr2=0x7f2d0800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d080052e0 0x7f2d0800ba60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800c3c0 tx=0x7f2d18001e50 comp rx=0 tx=0).stop 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3210401744 shutdown_connections 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3210401744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d080052e0 0x7f2d0800ba60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3210401744 >> 172.21.15.92:0/3210401744 conn(0x7f2d200ce600 msgr2=0x7f2d200cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3210401744 shutdown_connections 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.783+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3210401744 wait complete. 2023-10-06T03:03:20.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.784+0000 7f2d26bf3700 1 Processor -- start 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 -- start start 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d080052e0 0x7f2d2013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 0x7f2d200c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d200c7790 0x7f2d200c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200b19f0 con 0x7f2d200c7320 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200c7d20 con 0x7f2d200c7790 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d26bf3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200c8000 con 0x7f2d080052e0 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 0x7f2d200c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d17fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 0x7f2d200c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44862/0 (socket says 172.21.15.92:44862) 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d17fff700 1 -- 172.21.15.92:0/3376149400 learned_addr learned my addr 172.21.15.92:0/3376149400 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:20.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d2498f700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d080052e0 0x7f2d2013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.785+0000 7f2d25190700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d200c7790 0x7f2d200c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d17fff700 1 -- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d080052e0 msgr2=0x7f2d2013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d17fff700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d080052e0 0x7f2d2013f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d17fff700 1 -- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d200c7790 msgr2=0x7f2d200c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d17fff700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d200c7790 0x7f2d200c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d17fff700 1 -- 172.21.15.92:0/3376149400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200b13a0 con 0x7f2d200c7320 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d25190700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d200c7790 0x7f2d200c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d17fff700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 0x7f2d200c56b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2d10002980 tx=0x7f2d10007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10002e70 con 0x7f2d200c7320 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1000f870 con 0x7f2d200c7320 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d200b1680 con 0x7f2d200c7320 2023-10-06T03:03:20.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1000fc30 con 0x7f2d200c7320 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.786+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d200b3af0 con 0x7f2d200c7320 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.787+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d200b1b60 con 0x7f2d200c7320 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.787+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2d10006820 con 0x7f2d200c7320 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.788+0000 7f2d14ff9700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d0c066ff0 0x7f2d0c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.788+0000 7f2d2498f700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d0c066ff0 0x7f2d0c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.788+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2d10084460 con 0x7f2d200c7320 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.788+0000 7f2d2498f700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d0c066ff0 0x7f2d0c0694a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009fd0 tx=0x7f2d1c0097d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:20.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.792+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d10051880 con 0x7f2d200c7320 2023-10-06T03:03:20.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.976+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2d2004c3c0 con 0x7f2d200c7320 2023-10-06T03:03:20.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.979+0000 7f2d14ff9700 1 -- 172.21.15.92:0/3376149400 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d10054ea0 con 0x7f2d200c7320 2023-10-06T03:03:20.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.981+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d0c066ff0 msgr2=0x7f2d0c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.981+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d0c066ff0 0x7f2d0c0694a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009fd0 tx=0x7f2d1c0097d0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 msgr2=0x7f2d200c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 0x7f2d200c56b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2d10002980 tx=0x7f2d10007d30 comp rx=0 tx=0).stop 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 shutdown_connections 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d0c066ff0 0x7f2d0c0694a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d200c7790 0x7f2d200c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d080052e0 0x7f2d2013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 --2- 172.21.15.92:0/3376149400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d200c7320 0x7f2d200c56b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 >> 172.21.15.92:0/3376149400 conn(0x7f2d200ce600 msgr2=0x7f2d2013fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 shutdown_connections 2023-10-06T03:03:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:20.982+0000 7f2d26bf3700 1 -- 172.21.15.92:0/3376149400 wait complete. 2023-10-06T03:03:20.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-06T03:03:21.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.114+0000 7f7e03876700 1 Processor -- start 2023-10-06T03:03:21.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.114+0000 7f7e03876700 1 -- start start 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e03876700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 0x7f7dfc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e03876700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7dfc15db10 0x7f7dfc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e03876700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc15a160 con 0x7f7dfc159890 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e03876700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dfc0d13a0 con 0x7f7dfc15db10 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e03876700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc15e140 con 0x7f7dfc159c60 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e01612700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7dfc159890 0x7f7dfc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57680/0 (socket says 172.21.15.92:57680) 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e01612700 1 -- 172.21.15.92:0/1609915 learned_addr learned my addr 172.21.15.92:0/1609915 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e01e13700 1 --2- 172.21.15.92:0/1609915 >> v2:172.21.15.136:3300/0 conn(0x7f7dfc15db10 0x7f7dfc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7e00e11700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 0x7f7dfc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237295775 0 0) 0x7f7dfc15a160 con 0x7f7dfc159890 2023-10-06T03:03:21.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.115+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de8003570 con 0x7f7dfc159890 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049548000 0 0) 0x7f7de8003570 con 0x7f7dfc159890 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 msgr2=0x7f7dfc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 0x7f7dfc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 >> v2:172.21.15.136:3300/0 conn(0x7f7dfc15db10 msgr2=0x7f7dfc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 --2- 172.21.15.92:0/1609915 >> v2:172.21.15.136:3300/0 conn(0x7f7dfc15db10 0x7f7dfc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc15a3d0 con 0x7f7dfc159890 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7df8001470 con 0x7f7dfc159890 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 >> v1:172.21.15.92:6789/0 conn(0x7f7dfc159890 legacy=0x7f7dfc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8003a60 0x7f7de8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 0x7f7de8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8004d50 0x7f7de80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0d13a0 con 0x7f7de8003a60 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc15e140 con 0x7f7de8004d50 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de800ba60 con 0x7f7dfc159c60 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8003a60 0x7f7de8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e00e11700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 0x7f7de8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01e13700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8004d50 0x7f7de80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 -- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 msgr2=0x7f7de8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc159c60 0x7f7de8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 -- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8004d50 msgr2=0x7f7de80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8004d50 0x7f7de80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 -- 172.21.15.92:0/1609915 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de8003570 con 0x7f7de8003a60 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7e01612700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8003a60 0x7f7de8003e70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c3c0 tx=0x7f7df8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be1fa2752563cd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:21.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df800cc30 con 0x7f7de8003a60 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df800cdb0 con 0x7f7de8003a60 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.116+0000 7f7df3fff700 1 -- 172.21.15.92:0/1609915 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8005b70 con 0x7f7de8003a60 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e03876700 1 -- 172.21.15.92:0/1609915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8003a60 msgr2=0x7f7de8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e03876700 1 --2- 172.21.15.92:0/1609915 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7de8003a60 0x7f7de8003e70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c3c0 tx=0x7f7df8001e90 comp rx=0 tx=0).stop 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e01e13700 1 -- 172.21.15.92:0/1609915 reap_dead start 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e03876700 1 -- 172.21.15.92:0/1609915 shutdown_connections 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e03876700 1 -- 172.21.15.92:0/1609915 >> 172.21.15.92:0/1609915 conn(0x7f7dfc068e20 msgr2=0x7f7dfc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e03876700 1 -- 172.21.15.92:0/1609915 shutdown_connections 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.117+0000 7f7e03876700 1 -- 172.21.15.92:0/1609915 wait complete. 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.118+0000 7f7e03876700 1 Processor -- start 2023-10-06T03:03:21.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.118+0000 7f7e03876700 1 -- start start 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e03876700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8003a60 0x7f7dfc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e03876700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 0x7f7dfc0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e03876700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc153a70 0x7f7dfc153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e03876700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc0b3750 con 0x7f7dfc169cd0 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e03876700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc1549a0 con 0x7f7de8003a60 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e03876700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc16ac60 con 0x7f7dfc153a70 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 0x7f7dfc0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 0x7f7dfc0b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44892/0 (socket says 172.21.15.92:44892) 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 -- 172.21.15.92:0/1640263283 learned_addr learned my addr 172.21.15.92:0/1640263283 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e01e13700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc153a70 0x7f7dfc153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e01612700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8003a60 0x7f7dfc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 -- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc153a70 msgr2=0x7f7dfc153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc153a70 0x7f7dfc153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 -- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8003a60 msgr2=0x7f7dfc15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8003a60 0x7f7dfc15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 -- 172.21.15.92:0/1640263283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc16a120 con 0x7f7dfc169cd0 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e01612700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8003a60 0x7f7dfc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e01e13700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc153a70 0x7f7dfc153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.119+0000 7f7e00e11700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 0x7f7dfc0b0960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0119f0 tx=0x7f7dec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.120+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec011e60 con 0x7f7dfc169cd0 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.120+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec00f8e0 con 0x7f7dfc169cd0 2023-10-06T03:03:21.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.120+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc1543f0 con 0x7f7dfc169cd0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.120+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00fca0 con 0x7f7dfc169cd0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.120+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc16a6a0 con 0x7f7dfc169cd0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.121+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc0b3900 con 0x7f7dfc169cd0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.121+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7dec015670 con 0x7f7dfc169cd0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.121+0000 7f7df17fa700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7de0066fc0 0x7f7de0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.122+0000 7f7e01612700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7de0066fc0 0x7f7de0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.122+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7dec084a10 con 0x7f7dfc169cd0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.122+0000 7f7e01612700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7de0066fc0 0x7f7de0069470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7df8001ed0 tx=0x7f7df8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:21.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.126+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dfc0b3900 con 0x7f7dfc169cd0 2023-10-06T03:03:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.383+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f7dfc0b3900 con 0x7f7dfc169cd0 2023-10-06T03:03:21.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.649+0000 7f7df17fa700 1 -- 172.21.15.92:0/1640263283 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v118) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7dfc0b3900 con 0x7f7dfc169cd0 2023-10-06T03:03:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7de0066fc0 msgr2=0x7f7de0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7de0066fc0 0x7f7de0069470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7df8001ed0 tx=0x7f7df8015000 comp rx=0 tx=0).stop 2023-10-06T03:03:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 msgr2=0x7f7dfc0b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 0x7f7dfc0b0960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0119f0 tx=0x7f7dec008040 comp rx=0 tx=0).stop 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 shutdown_connections 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7de0066fc0 0x7f7de0069470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7dfc153a70 0x7f7dfc153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7de8003a60 0x7f7dfc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 --2- 172.21.15.92:0/1640263283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7dfc169cd0 0x7f7dfc0b0960 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 >> 172.21.15.92:0/1640263283 conn(0x7f7dfc068e20 msgr2=0x7f7dfc1588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 shutdown_connections 2023-10-06T03:03:21.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.651+0000 7f7e03876700 1 -- 172.21.15.92:0/1640263283 wait complete. 2023-10-06T03:03:21.664 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2023-10-06T03:03:21.665 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2023-10-06T03:03:21.665 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:03:21.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-06T03:03:21.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.778+0000 7fb4c5e44700 1 Processor -- start 2023-10-06T03:03:21.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.778+0000 7fb4c5e44700 1 -- start start 2023-10-06T03:03:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4c5e44700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb4c0159890 0x7fb4c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4c5e44700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c015a160 con 0x7fb4c0069560 2023-10-06T03:03:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4c5e44700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00d13a0 con 0x7fb4c0159c60 2023-10-06T03:03:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4c5e44700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c015e140 con 0x7fb4c0159890 2023-10-06T03:03:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4bffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb4c0069560 0x7fb4c00ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57696/0 (socket says 172.21.15.92:57696) 2023-10-06T03:03:21.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4bffff700 1 -- 172.21.15.92:0/3624277989 learned_addr learned my addr 172.21.15.92:0/3624277989 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/3624277989 >> v2:172.21.15.136:3300/0 conn(0x7fb4c0159890 0x7fb4c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689481177 0 0) 0x7fb4c015a160 con 0x7fb4c0069560 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.779+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a8003570 con 0x7fb4c0069560 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1533727790 0 0) 0x7fb4a8003570 con 0x7fb4c0069560 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 >> v2:172.21.15.136:3300/0 conn(0x7fb4c0159890 msgr2=0x7fb4c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 --2- 172.21.15.92:0/3624277989 >> v2:172.21.15.136:3300/0 conn(0x7fb4c0159890 0x7fb4c015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 >> v1:172.21.15.136:6789/0 conn(0x7fb4c0159c60 legacy=0x7fb4c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c015a3d0 con 0x7fb4c0069560 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb4b8001470 con 0x7fb4c0069560 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 >> v1:172.21.15.92:6789/0 conn(0x7fb4c0069560 legacy=0x7fb4c00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a8003a60 0x7fb4a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4a8004810 0x7fb4a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a80051c0 0x7fb4a800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c015e140 con 0x7fb4a8003a60 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a800c0d0 con 0x7fb4a8004810 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a800c290 con 0x7fb4a80051c0 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a8003a60 0x7fb4a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4beffd700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4a8004810 0x7fb4a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bffff700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a80051c0 0x7fb4a800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 -- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a80051c0 msgr2=0x7fb4a800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a80051c0 0x7fb4a800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 -- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4a8004810 msgr2=0x7fb4a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4a8004810 0x7fb4a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 -- 172.21.15.92:0/3624277989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8003570 con 0x7fb4a8003a60 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bffff700 1 -- 172.21.15.92:0/3624277989 reap_dead start 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a8003a60 0x7fb4a8003e70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e210 tx=0x7fb4b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b97ef82597bb33d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000eac0 con 0x7fb4a8003a60 2023-10-06T03:03:21.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b000f040 con 0x7fb4a8003a60 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.780+0000 7fb4be7fc700 1 -- 172.21.15.92:0/3624277989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b000ee70 con 0x7fb4a8003a60 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 -- 172.21.15.92:0/3624277989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a8003a60 msgr2=0x7fb4a8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a8003a60 0x7fb4a8003e70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e210 tx=0x7fb4b000b040 comp rx=0 tx=0).stop 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 -- 172.21.15.92:0/3624277989 shutdown_connections 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/3624277989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4a8003a60 0x7fb4a8003e70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 -- 172.21.15.92:0/3624277989 >> 172.21.15.92:0/3624277989 conn(0x7fb4c0068e20 msgr2=0x7fb4c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:21.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 -- 172.21.15.92:0/3624277989 shutdown_connections 2023-10-06T03:03:21.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.781+0000 7fb4c5e44700 1 -- 172.21.15.92:0/3624277989 wait complete. 2023-10-06T03:03:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.782+0000 7fb4c5e44700 1 Processor -- start 2023-10-06T03:03:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.782+0000 7fb4c5e44700 1 -- start start 2023-10-06T03:03:21.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4c5e44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a8003a60 0x7fb4c014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4c5e44700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c00ce180 0x7fb4c00b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4c5e44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 0x7fb4c0152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4c5e44700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00b40d0 con 0x7fb4c00b6070 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4c5e44700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00cf8a0 con 0x7fb4c00ce180 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4c5e44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00b37a0 con 0x7fb4a8003a60 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 0x7fb4c0152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 0x7fb4c0152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44914/0 (socket says 172.21.15.92:44914) 2023-10-06T03:03:21.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 -- 172.21.15.92:0/1149866315 learned_addr learned my addr 172.21.15.92:0/1149866315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4beffd700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c00ce180 0x7fb4c00b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a8003a60 0x7fb4c014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 -- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a8003a60 msgr2=0x7fb4c014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a8003a60 0x7fb4c014c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 -- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c00ce180 msgr2=0x7fb4c00b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c00ce180 0x7fb4c00b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:21.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 -- 172.21.15.92:0/1149866315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c00b3a80 con 0x7fb4c00b6070 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a8003a60 0x7fb4c014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.783+0000 7fb4bffff700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 0x7fb4c0152180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cadc0 tx=0x7fb4b8000fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.784+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800c600 con 0x7fb4c00b6070 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.784+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b800c780 con 0x7fb4c00b6070 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.784+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c00b3d60 con 0x7fb4c00b6070 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.784+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800cb40 con 0x7fb4c00b6070 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.784+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c00b4aa0 con 0x7fb4c00b6070 2023-10-06T03:03:21.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.784+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c00b4240 con 0x7fb4c00b6070 2023-10-06T03:03:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.785+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb4b801d460 con 0x7fb4c00b6070 2023-10-06T03:03:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.785+0000 7fb4a77fe700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4ac066fc0 0x7fb4ac069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.785+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb4b8084280 con 0x7fb4c00b6070 2023-10-06T03:03:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.785+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4ac066fc0 0x7fb4ac069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:21.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.786+0000 7fb4bf7fe700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4ac066fc0 0x7fb4ac069470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000afd0 tx=0x7fb4b001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:21.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:21.790+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4b80516a0 con 0x7fb4c00b6070 2023-10-06T03:03:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.050+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fb4c004c3c0 con 0x7fb4c00b6070 2023-10-06T03:03:22.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.650+0000 7fb4a77fe700 1 -- 172.21.15.92:0/1149866315 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb4b8001c70 con 0x7fb4c00b6070 2023-10-06T03:03:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4ac066fc0 msgr2=0x7fb4ac069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4ac066fc0 0x7fb4ac069470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000afd0 tx=0x7fb4b001a000 comp rx=0 tx=0).stop 2023-10-06T03:03:22.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 msgr2=0x7fb4c0152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 0x7fb4c0152180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb4c00cadc0 tx=0x7fb4b8000fd0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 shutdown_connections 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4ac066fc0 0x7fb4ac069470 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4c00b6070 0x7fb4c0152180 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4a8003a60 0x7fb4c014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 --2- 172.21.15.92:0/1149866315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4c00ce180 0x7fb4c00b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 >> 172.21.15.92:0/1149866315 conn(0x7fb4c0068e20 msgr2=0x7fb4c015cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 shutdown_connections 2023-10-06T03:03:22.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.652+0000 7fb4c5e44700 1 -- 172.21.15.92:0/1149866315 wait complete. 2023-10-06T03:03:22.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:03:22.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.784+0000 7fb4b686f700 1 Processor -- start 2023-10-06T03:03:22.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.784+0000 7fb4b686f700 1 -- start start 2023-10-06T03:03:22.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b686f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 0x7fb4b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b686f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb4b00b1b40 0x7fb4b015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b686f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00cfb00 con 0x7fb4b00b16d0 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b686f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00b5c40 con 0x7fb4b00b1b40 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b686f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b5f20 con 0x7fb4b00b4e70 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b506c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb4b00b16d0 0x7fb4b015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57702/0 (socket says 172.21.15.92:57702) 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b506c700 1 -- 172.21.15.92:0/3932616492 learned_addr learned my addr 172.21.15.92:0/3932616492 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577365331 0 0) 0x7fb4b00cfb00 con 0x7fb4b00b16d0 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b586d700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 0x7fb4b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb498003570 con 0x7fb4b00b16d0 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.785+0000 7fb4b606e700 1 --2- 172.21.15.92:0/3932616492 >> v2:172.21.15.136:3300/0 conn(0x7fb4b00b1b40 0x7fb4b015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408733193 0 0) 0x7fb498003570 con 0x7fb4b00b16d0 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 msgr2=0x7fb4b00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 0x7fb4b00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 >> v2:172.21.15.136:3300/0 conn(0x7fb4b00b1b40 msgr2=0x7fb4b015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 --2- 172.21.15.92:0/3932616492 >> v2:172.21.15.136:3300/0 conn(0x7fb4b00b1b40 0x7fb4b015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00b6200 con 0x7fb4b00b16d0 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb4a4001470 con 0x7fb4b00b16d0 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 >> v1:172.21.15.92:6789/0 conn(0x7fb4b00b16d0 legacy=0x7fb4b015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 0x7fb498003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb498003fe0 0x7fb498004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb498004d50 0x7fb4980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b5c40 con 0x7fb498003fe0 2023-10-06T03:03:22.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b5f20 con 0x7fb498004d50 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49800ba90 con 0x7fb4b00b4e70 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb498003fe0 0x7fb498004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b586d700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 0x7fb498003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b606e700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb498004d50 0x7fb4980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 -- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 msgr2=0x7fb498003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00b4e70 0x7fb498003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 -- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb498004d50 msgr2=0x7fb4980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb498004d50 0x7fb4980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 -- 172.21.15.92:0/3932616492 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb498003570 con 0x7fb498003fe0 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b506c700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb498003fe0 0x7fb498004810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400c3c0 tx=0x7fb4a4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50cd9aa691bf1b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400cc30 con 0x7fb498003fe0 2023-10-06T03:03:22.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a400cdb0 con 0x7fb498003fe0 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.786+0000 7fb4b486b700 1 -- 172.21.15.92:0/3932616492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4005b70 con 0x7fb498003fe0 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b686f700 1 -- 172.21.15.92:0/3932616492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb498003fe0 msgr2=0x7fb498004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b686f700 1 --2- 172.21.15.92:0/3932616492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb498003fe0 0x7fb498004810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400c3c0 tx=0x7fb4a4001e90 comp rx=0 tx=0).stop 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b606e700 1 -- 172.21.15.92:0/3932616492 reap_dead start 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b686f700 1 -- 172.21.15.92:0/3932616492 shutdown_connections 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b686f700 1 -- 172.21.15.92:0/3932616492 >> 172.21.15.92:0/3932616492 conn(0x7fb4b00d10c0 msgr2=0x7fb4b00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b686f700 1 -- 172.21.15.92:0/3932616492 shutdown_connections 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.787+0000 7fb4b686f700 1 -- 172.21.15.92:0/3932616492 wait complete. 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.788+0000 7fb4b686f700 1 Processor -- start 2023-10-06T03:03:22.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.788+0000 7fb4b686f700 1 -- start start 2023-10-06T03:03:22.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b686f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 0x7fb4b016abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b686f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b00cadf0 0x7fb4b00a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b686f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00cc950 0x7fb4b00a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b686f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b016b120 con 0x7fb4b0167790 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b686f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00cb600 con 0x7fb4b00cadf0 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b686f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b0167b60 con 0x7fb4b00cc950 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 0x7fb4b016abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 0x7fb4b016abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44932/0 (socket says 172.21.15.92:44932) 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 -- 172.21.15.92:0/992821923 learned_addr learned my addr 172.21.15.92:0/992821923 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b506c700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b00cadf0 0x7fb4b00a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b606e700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00cc950 0x7fb4b00a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 -- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00cc950 msgr2=0x7fb4b00a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00cc950 0x7fb4b00a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 -- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b00cadf0 msgr2=0x7fb4b00a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b00cadf0 0x7fb4b00a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 -- 172.21.15.92:0/992821923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00cb8b0 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4b586d700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 0x7fb4b016abe0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8011d40 tx=0x7fb4a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.789+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800fa40 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.790+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a800fba0 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.790+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00cbb90 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.790+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8014440 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.790+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b00cc150 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.791+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b0088990 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.791+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb4a8014870 con 0x7fb4b0167790 2023-10-06T03:03:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.792+0000 7fb4adffb700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb49c066fc0 0x7fb49c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.792+0000 7fb4b506c700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb49c066fc0 0x7fb49c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.792+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb4a8018030 con 0x7fb4b0167790 2023-10-06T03:03:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.792+0000 7fb4b506c700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb49c066fc0 0x7fb49c069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4001ed0 tx=0x7fb4a4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.796+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4a80521f0 con 0x7fb4b0167790 2023-10-06T03:03:22.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.980+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb4b00cc7a0 con 0x7fb4b0167790 2023-10-06T03:03:22.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.985+0000 7fb4adffb700 1 -- 172.21.15.92:0/992821923 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4a8055810 con 0x7fb4b0167790 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb49c066fc0 msgr2=0x7fb49c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb49c066fc0 0x7fb49c069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4001ed0 tx=0x7fb4a4015000 comp rx=0 tx=0).stop 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 msgr2=0x7fb4b016abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 0x7fb4b016abe0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8011d40 tx=0x7fb4a80091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 shutdown_connections 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb49c066fc0 0x7fb49c069470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4b0167790 0x7fb4b016abe0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b00cc950 0x7fb4b00a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 --2- 172.21.15.92:0/992821923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b00cadf0 0x7fb4b00a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 >> 172.21.15.92:0/992821923 conn(0x7fb4b00d10c0 msgr2=0x7fb4b01590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 shutdown_connections 2023-10-06T03:03:22.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:22.988+0000 7fb4b686f700 1 -- 172.21.15.92:0/992821923 wait complete. 2023-10-06T03:03:23.000 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:03:23.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:03:23.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:03:23.078 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:03:23.078 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:03:23.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.191+0000 7f749a987700 1 Processor -- start 2023-10-06T03:03:23.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.191+0000 7f749a987700 1 -- start start 2023-10-06T03:03:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.192+0000 7f749a987700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7494159890 0x7f749415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.192+0000 7f749a987700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749415a160 con 0x7f7494069560 2023-10-06T03:03:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.192+0000 7f749a987700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940d13a0 con 0x7f7494159c60 2023-10-06T03:03:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.192+0000 7f749a987700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749415e140 con 0x7f7494159890 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.192+0000 7f7498f24700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7494069560 0x7f74940ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57716/0 (socket says 172.21.15.92:57716) 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.192+0000 7f7498f24700 1 -- 172.21.15.92:0/4168216881 learned_addr learned my addr 172.21.15.92:0/4168216881 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7493fff700 1 --2- 172.21.15.92:0/4168216881 >> v2:172.21.15.136:3300/0 conn(0x7f7494159890 0x7f749415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172197135 0 0) 0x7f749415a160 con 0x7f7494069560 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f747c003570 con 0x7f7494069560 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471283229 0 0) 0x7f747c003570 con 0x7f7494069560 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 >> v2:172.21.15.136:3300/0 conn(0x7f7494159890 msgr2=0x7f749415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 --2- 172.21.15.92:0/4168216881 >> v2:172.21.15.136:3300/0 conn(0x7f7494159890 0x7f749415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 >> v1:172.21.15.136:6789/0 conn(0x7f7494159c60 legacy=0x7f749415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749415a3d0 con 0x7f7494069560 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f748c0017f0 con 0x7f7494069560 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 >> v1:172.21.15.92:6789/0 conn(0x7f7494069560 legacy=0x7f74940ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c003a60 0x7f747c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f747c004810 0x7f747c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f747c0051c0 0x7f747c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749415e140 con 0x7f747c004810 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c00c050 con 0x7f747c003a60 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c00c210 con 0x7f747c0051c0 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f74937fe700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f747c004810 0x7f747c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7493fff700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c003a60 0x7f747c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7498f24700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f747c0051c0 0x7f747c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f74937fe700 1 -- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f747c0051c0 msgr2=0x7f747c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f74937fe700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f747c0051c0 0x7f747c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f74937fe700 1 -- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c003a60 msgr2=0x7f747c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f74937fe700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c003a60 0x7f747c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f74937fe700 1 -- 172.21.15.92:0/4168216881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c003570 con 0x7f747c004810 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.193+0000 7f7498f24700 1 -- 172.21.15.92:0/4168216881 reap_dead start 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f74937fe700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f747c004810 0x7f747c004c80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f74880019a0 tx=0x7f7488007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbb2c922beaec7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800c5d0 con 0x7f747c004810 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748800c750 con 0x7f747c004810 2023-10-06T03:03:23.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f7492ffd700 1 -- 172.21.15.92:0/4168216881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800cb10 con 0x7f747c004810 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 -- 172.21.15.92:0/4168216881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f747c004810 msgr2=0x7f747c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f747c004810 0x7f747c004c80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f74880019a0 tx=0x7f7488007d10 comp rx=0 tx=0).stop 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 -- 172.21.15.92:0/4168216881 shutdown_connections 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 --2- 172.21.15.92:0/4168216881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f747c004810 0x7f747c004c80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 -- 172.21.15.92:0/4168216881 >> 172.21.15.92:0/4168216881 conn(0x7f7494068e20 msgr2=0x7f74940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 -- 172.21.15.92:0/4168216881 shutdown_connections 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.194+0000 7f749a987700 1 -- 172.21.15.92:0/4168216881 wait complete. 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.195+0000 7f749a987700 1 Processor -- start 2023-10-06T03:03:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.195+0000 7f749a987700 1 -- start start 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f749a987700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c004810 0x7f74940c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f749a987700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 0x7f74940b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f749a987700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74940b2690 0x7f74940c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f749a987700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749414a230 con 0x7f7494149dc0 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f749a987700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940b2b00 con 0x7f747c004810 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f749a987700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940c9b20 con 0x7f74940b2690 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f74937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 0x7f74940b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f74937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 0x7f74940b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44948/0 (socket says 172.21.15.92:44948) 2023-10-06T03:03:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f74937fe700 1 -- 172.21.15.92:0/2222909563 learned_addr learned my addr 172.21.15.92:0/2222909563 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f7493fff700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c004810 0x7f74940c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.196+0000 7f7498f24700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74940b2690 0x7f74940c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f74937fe700 1 -- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74940b2690 msgr2=0x7f74940c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f74937fe700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74940b2690 0x7f74940c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f74937fe700 1 -- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c004810 msgr2=0x7f74940c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f74937fe700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c004810 0x7f74940c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f74937fe700 1 -- 172.21.15.92:0/2222909563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940c8f10 con 0x7f7494149dc0 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f74937fe700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 0x7f74940b2150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7488012ec0 tx=0x7f7488015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800c5d0 con 0x7f7494149dc0 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748800ce80 con 0x7f7494149dc0 2023-10-06T03:03:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940c91f0 con 0x7f7494149dc0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800cad0 con 0x7f7494149dc0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.197+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74940c97b0 con 0x7f7494149dc0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.198+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749404c3c0 con 0x7f7494149dc0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.198+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7488019030 con 0x7f7494149dc0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.198+0000 7f747bfff700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7480066fc0 0x7f7480069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.199+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f74880855c0 con 0x7f7494149dc0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.199+0000 7f7493fff700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7480066fc0 0x7f7480069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.199+0000 7f7493fff700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7480066fc0 0x7f7480069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7484006f00 tx=0x7f7484008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:23.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.203+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7488052aa0 con 0x7f7494149dc0 2023-10-06T03:03:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.429+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74940c9c60 con 0x7f7494149dc0 2023-10-06T03:03:23.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.429+0000 7f747bfff700 1 -- 172.21.15.92:0/2222909563 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f7488017de0 con 0x7f7494149dc0 2023-10-06T03:03:23.430 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:23.431 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4886,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/274851395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":274851395},{"type":"v1","addr":"172.21.15.136:6834","nonce":274851395}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1811788018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1811788018},{"type":"v1","addr":"172.21.15.136:6835","nonce":1811788018}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1202939659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1202939659},{"type":"v1","addr":"172.21.15.92:6836","nonce":1202939659}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3735930940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3735930940},{"type":"v1","addr":"172.21.15.92:6837","nonce":3735930940}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[]} 2023-10-06T03:03:23.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.431+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7480066fc0 msgr2=0x7f7480069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.431+0000 7f749a987700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7480066fc0 0x7f7480069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7484006f00 tx=0x7f7484008040 comp rx=0 tx=0).stop 2023-10-06T03:03:23.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.431+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 msgr2=0x7f74940b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 0x7f74940b2150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7488012ec0 tx=0x7f7488015000 comp rx=0 tx=0).stop 2023-10-06T03:03:23.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 shutdown_connections 2023-10-06T03:03:23.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7480066fc0 0x7f7480069470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74940b2690 0x7f74940c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f747c004810 0x7f74940c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 --2- 172.21.15.92:0/2222909563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7494149dc0 0x7f74940b2150 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 >> 172.21.15.92:0/2222909563 conn(0x7f7494068e20 msgr2=0x7f749415d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 shutdown_connections 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.432+0000 7f749a987700 1 -- 172.21.15.92:0/2222909563 wait complete. 2023-10-06T03:03:23.434 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 32 2023-10-06T03:03:23.444 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:03:23.445 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:03:23.445 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:03:23.445 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:03:23.445 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:03:23.445 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:03:23.446 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:03:23.446 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:03:23.446 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:03:23.446 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:03:23.446 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:03:23.446 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:03:23.447 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:03:23.447 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:03:23.447 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:03:23.447 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:03:23.454 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:03:23.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:03:23.455 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:03:23.455 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:03:23.457 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:03:23.457 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:03:23.458 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:03:23.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:03:23.497 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:03:23.497 DEBUG:teuthology.parallel:result is None 2023-10-06T03:03:23.498 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:03:23.498 DEBUG:teuthology.parallel:result is None 2023-10-06T03:03:23.500 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:03:23.500 DEBUG:teuthology.parallel:result is None 2023-10-06T03:03:23.501 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:03:23.501 DEBUG:teuthology.parallel:result is None 2023-10-06T03:03:23.501 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:03:23.569 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:03:23.570 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:03:23.579 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:03:23.580 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:03:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.621+0000 7fe7f26a0700 1 Processor -- start 2023-10-06T03:03:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.621+0000 7fe7f26a0700 1 -- start start 2023-10-06T03:03:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7f26a0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe7ec06bcd0 0x7fe7ec140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7f26a0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec15c2a0 con 0x7fe7ec06c0a0 2023-10-06T03:03:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7f26a0700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec15b690 con 0x7fe7ec0c9510 2023-10-06T03:03:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7f26a0700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0c9900 con 0x7fe7ec06bcd0 2023-10-06T03:03:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7eb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe7ec06c0a0 0x7fe7ec0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57718/0 (socket says 172.21.15.92:57718) 2023-10-06T03:03:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7eb7fe700 1 -- 172.21.15.92:0/1658098327 learned_addr learned my addr 172.21.15.92:0/1658098327 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/1658098327 >> v2:172.21.15.136:3300/0 conn(0x7fe7ec06bcd0 0x7fe7ec140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.622+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858255059 0 0) 0x7fe7ec15c2a0 con 0x7fe7ec06c0a0 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7d4003570 con 0x7fe7ec06c0a0 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677428048 0 0) 0x7fe7d4003570 con 0x7fe7ec06c0a0 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 >> v2:172.21.15.136:3300/0 conn(0x7fe7ec06bcd0 msgr2=0x7fe7ec140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 --2- 172.21.15.92:0/1658098327 >> v2:172.21.15.136:3300/0 conn(0x7fe7ec06bcd0 0x7fe7ec140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 >> v1:172.21.15.136:6789/0 conn(0x7fe7ec0c9510 legacy=0x7fe7ec0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec15b970 con 0x7fe7ec06c0a0 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fe7e0001470 con 0x7fe7ec06c0a0 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 >> v1:172.21.15.92:6789/0 conn(0x7fe7ec06c0a0 legacy=0x7fe7ec0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7d4003a60 0x7fe7d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4004810 0x7fe7d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d40051c0 0x7fe7d400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0c9900 con 0x7fe7d4003a60 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d400c0d0 con 0x7fe7d4004810 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d400c290 con 0x7fe7d40051c0 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7d4003a60 0x7fe7d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eb7fe700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4004810 0x7fe7d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7f0c3d700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d40051c0 0x7fe7d400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 -- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d40051c0 msgr2=0x7fe7d400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d40051c0 0x7fe7d400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 -- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4004810 msgr2=0x7fe7d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4004810 0x7fe7d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 -- 172.21.15.92:0/1658098327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4003570 con 0x7fe7d4003a60 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7f0c3d700 1 -- 172.21.15.92:0/1658098327 reap_dead start 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7eb7fe700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4004810 0x7fe7d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.623+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7d4003a60 0x7fe7d4003e70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009c00 tx=0x7fe7dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb1986762567642 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00e710 con 0x7fe7d4003a60 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7dc00e870 con 0x7fe7d4003a60 2023-10-06T03:03:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7eaffd700 1 -- 172.21.15.92:0/1658098327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00ec30 con 0x7fe7d4003a60 2023-10-06T03:03:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 -- 172.21.15.92:0/1658098327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7d4003a60 msgr2=0x7fe7d4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7d4003a60 0x7fe7d4003e70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009c00 tx=0x7fe7dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 -- 172.21.15.92:0/1658098327 shutdown_connections 2023-10-06T03:03:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/1658098327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7d4003a60 0x7fe7d4003e70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 -- 172.21.15.92:0/1658098327 >> 172.21.15.92:0/1658098327 conn(0x7fe7ec0ce600 msgr2=0x7fe7ec0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:23.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 -- 172.21.15.92:0/1658098327 shutdown_connections 2023-10-06T03:03:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.624+0000 7fe7f26a0700 1 -- 172.21.15.92:0/1658098327 wait complete. 2023-10-06T03:03:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.625+0000 7fe7f26a0700 1 Processor -- start 2023-10-06T03:03:23.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.625+0000 7fe7f26a0700 1 -- start start 2023-10-06T03:03:23.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f26a0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4003a60 0x7fe7ec13d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f26a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 0x7fe7ec13f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f26a0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7ec154030 0x7fe7ec13f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f26a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b1ec0 con 0x7fe7ec13c210 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f26a0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec13fad0 con 0x7fe7d4003a60 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f26a0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1545f0 con 0x7fe7ec154030 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7eb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 0x7fe7ec13f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7eb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 0x7fe7ec13f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44980/0 (socket says 172.21.15.92:44980) 2023-10-06T03:03:23.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7eb7fe700 1 -- 172.21.15.92:0/2298341442 learned_addr learned my addr 172.21.15.92:0/2298341442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7f0c3d700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7ec154030 0x7fe7ec13f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.626+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4003a60 0x7fe7ec13d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7eb7fe700 1 -- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7ec154030 msgr2=0x7fe7ec13f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7eb7fe700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7ec154030 0x7fe7ec13f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7eb7fe700 1 -- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4003a60 msgr2=0x7fe7ec13d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7eb7fe700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4003a60 0x7fe7ec13d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7eb7fe700 1 -- 172.21.15.92:0/2298341442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec1548d0 con 0x7fe7ec13c210 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7eb7fe700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 0x7fe7ec13f050 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000f500 tx=0x7fe7e0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0018030 con 0x7fe7ec13c210 2023-10-06T03:03:23.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e0001430 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0013600 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec154bb0 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.627+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec13db40 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.628+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec155050 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.628+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe7e001b070 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.629+0000 7fe7d3fff700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7d8066fc0 0x7fe7d8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.629+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe7e0019070 con 0x7fe7ec13c210 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.629+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7d8066fc0 0x7fe7d8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:23.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.629+0000 7fe7ebfff700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7d8066fc0 0x7fe7d8069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0091e0 tx=0x7fe7dc01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:23.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.633+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7e00518e0 con 0x7fe7ec13c210 2023-10-06T03:03:23.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:23.836+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe7ec04c3c0 con 0x7fe7ec13c210 2023-10-06T03:03:24.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.672+0000 7fe7d3fff700 1 -- 172.21.15.92:0/2298341442 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe7e0054f00 con 0x7fe7ec13c210 2023-10-06T03:03:24.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7d8066fc0 msgr2=0x7fe7d8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7d8066fc0 0x7fe7d8069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0091e0 tx=0x7fe7dc01a000 comp rx=0 tx=0).stop 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 msgr2=0x7fe7ec13f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 0x7fe7ec13f050 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000f500 tx=0x7fe7e0001e50 comp rx=0 tx=0).stop 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 shutdown_connections 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe7d8066fc0 0x7fe7d8069470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7ec154030 0x7fe7ec13f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d4003a60 0x7fe7ec13d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 --2- 172.21.15.92:0/2298341442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7ec13c210 0x7fe7ec13f050 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 >> 172.21.15.92:0/2298341442 conn(0x7fe7ec0ce600 msgr2=0x7fe7ec13ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 shutdown_connections 2023-10-06T03:03:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.675+0000 7fe7f26a0700 1 -- 172.21.15.92:0/2298341442 wait complete. 2023-10-06T03:03:24.677 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:03:24.689 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:03:24.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.805+0000 7f2714425700 1 Processor -- start 2023-10-06T03:03:24.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.805+0000 7f2714425700 1 -- start start 2023-10-06T03:03:24.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.806+0000 7f2714425700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f270c06bcd0 0x7f270c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:24.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.806+0000 7f2714425700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c0c9510 0x7f270c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.806+0000 7f2714425700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270c15c2a0 con 0x7f270c06c0a0 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.806+0000 7f2714425700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c15b690 con 0x7f270c06bcd0 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.806+0000 7f2714425700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c15b970 con 0x7f270c0c9510 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f270c06bcd0 0x7f270c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f270c06bcd0 0x7f270c140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44990/0 (socket says 172.21.15.92:44990) 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 -- 172.21.15.92:0/4015540913 learned_addr learned my addr 172.21.15.92:0/4015540913 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27129c2700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c0c9510 0x7f270c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 -- 172.21.15.92:0/4015540913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c0c9510 msgr2=0x7f270c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c0c9510 0x7f270c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 -- 172.21.15.92:0/4015540913 >> v1:172.21.15.136:6789/0 conn(0x7f270c06c0a0 legacy=0x7f270c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:24.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 -- 172.21.15.92:0/4015540913 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c15bc50 con 0x7f270c06bcd0 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27129c2700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c0c9510 0x7f270c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27121c1700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c06bcd0 0x7f270c140df0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2708009f20 tx=0x7f2708009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e8057085dceac6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27111bf700 1 -- 172.21.15.92:0/4015540913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270800e070 con 0x7f270c06bcd0 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27111bf700 1 -- 172.21.15.92:0/4015540913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270800ba80 con 0x7f270c06bcd0 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.807+0000 7f27111bf700 1 -- 172.21.15.92:0/4015540913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27080092e0 con 0x7f270c06bcd0 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 -- 172.21.15.92:0/4015540913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c06bcd0 msgr2=0x7f270c140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c06bcd0 0x7f270c140df0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2708009f20 tx=0x7f2708009840 comp rx=0 tx=0).stop 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 -- 172.21.15.92:0/4015540913 shutdown_connections 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c06bcd0 0x7f270c140df0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 --2- 172.21.15.92:0/4015540913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c0c9510 0x7f270c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 -- 172.21.15.92:0/4015540913 >> 172.21.15.92:0/4015540913 conn(0x7f270c0ce600 msgr2=0x7f270c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:24.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 -- 172.21.15.92:0/4015540913 shutdown_connections 2023-10-06T03:03:24.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.808+0000 7f2714425700 1 -- 172.21.15.92:0/4015540913 wait complete. 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.809+0000 7f2714425700 1 Processor -- start 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.809+0000 7f2714425700 1 -- start start 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f270c06bcd0 0x7f270c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c06c0a0 0x7f270c13ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 0x7f270c0c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c13dd40 con 0x7f270c0c9510 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0c78a0 con 0x7f270c06bcd0 2023-10-06T03:03:24.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c069a80 con 0x7f270c06c0a0 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 0x7f270c0c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 0x7f270c0c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44998/0 (socket says 172.21.15.92:44998) 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 -- 172.21.15.92:0/2716604911 learned_addr learned my addr 172.21.15.92:0/2716604911 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27121c1700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f270c06bcd0 0x7f270c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27119c0700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c06c0a0 0x7f270c13ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 -- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c06c0a0 msgr2=0x7f270c13ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c06c0a0 0x7f270c13ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 -- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f270c06bcd0 msgr2=0x7f270c0694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f270c06bcd0 0x7f270c0694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 -- 172.21.15.92:0/2716604911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2708009ae0 con 0x7f270c0c9510 2023-10-06T03:03:24.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27121c1700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f270c06bcd0 0x7f270c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27129c2700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 0x7f270c0c7360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f270400d860 tx=0x7f2704008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27040125f0 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704012750 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c069d60 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704012ad0 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.810+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c06a320 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.811+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c04c3c0 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.812+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2704014430 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.812+0000 7f27027fc700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26f0066fc0 0x7f26f0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.812+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2704084760 con 0x7f270c0c9510 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.812+0000 7f27121c1700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26f0066fc0 0x7f26f0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:24.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.812+0000 7f27121c1700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26f0066fc0 0x7f26f0069470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f270c0cbed0 tx=0x7f2708018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:24.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:24.816+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2704055660 con 0x7f270c0c9510 2023-10-06T03:03:25.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.073+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f270c06c9d0 con 0x7f270c0c9510 2023-10-06T03:03:25.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.073+0000 7f27027fc700 1 -- 172.21.15.92:0/2716604911 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f27040551f0 con 0x7f270c0c9510 2023-10-06T03:03:25.074 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.075 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.076 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.077 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:03:25.078 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.079 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.080 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.081 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:03:25.082 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.083 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.084 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.085 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:03:25.086 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:03:25.087 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.088 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:03:25.089 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:03:25.090 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:03:25.090 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:03:25.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.090 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:25.090 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:03:25.090 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:03:25.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26f0066fc0 msgr2=0x7f26f0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26f0066fc0 0x7f26f0069470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f270c0cbed0 tx=0x7f2708018000 comp rx=0 tx=0).stop 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 msgr2=0x7f270c0c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 0x7f270c0c7360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f270400d860 tx=0x7f2704008040 comp rx=0 tx=0).stop 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 shutdown_connections 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f26f0066fc0 0x7f26f0069470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270c0c9510 0x7f270c0c7360 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f270c06bcd0 0x7f270c0694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 --2- 172.21.15.92:0/2716604911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f270c06c0a0 0x7f270c13ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 >> 172.21.15.92:0/2716604911 conn(0x7f270c0ce600 msgr2=0x7f270c0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 shutdown_connections 2023-10-06T03:03:25.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.075+0000 7f2714425700 1 -- 172.21.15.92:0/2716604911 wait complete. 2023-10-06T03:03:25.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 Processor -- start 2023-10-06T03:03:25.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 -- start start 2023-10-06T03:03:25.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2d40cb5a0 0x7fb2d40cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40c85a0 0x7fb2d40c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d4069d30 con 0x7fb2d4156fb0 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d40c8110 con 0x7fb2d40cb5a0 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.206+0000 7fb2d9bf3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d40cbf30 con 0x7fb2d40c85a0 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2d40cb5a0 0x7fb2d40cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb2d40cb5a0 0x7fb2d40cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45014/0 (socket says 172.21.15.92:45014) 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 -- 172.21.15.92:0/3690081755 learned_addr learned my addr 172.21.15.92:0/3690081755 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d3fff700 1 --2- 172.21.15.92:0/3690081755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40c85a0 0x7fb2d40c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 -- 172.21.15.92:0/3690081755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40c85a0 msgr2=0x7fb2d40c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:25.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 --2- 172.21.15.92:0/3690081755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40c85a0 0x7fb2d40c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 -- 172.21.15.92:0/3690081755 >> v1:172.21.15.136:6789/0 conn(0x7fb2d4156fb0 legacy=0x7fb2d406a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 -- 172.21.15.92:0/3690081755 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d41568e0 con 0x7fb2d40cb5a0 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d2ffd700 1 --2- 172.21.15.92:0/3690081755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40cb5a0 0x7fb2d40cb9f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00bc50 tx=0x7fb2bc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88581d0b00575d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d27fc700 1 -- 172.21.15.92:0/3690081755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00be60 con 0x7fb2d40cb5a0 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d27fc700 1 -- 172.21.15.92:0/3690081755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc00de00 con 0x7fb2d40cb5a0 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.207+0000 7fb2d27fc700 1 -- 172.21.15.92:0/3690081755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0196b0 con 0x7fb2d40cb5a0 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/3690081755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40cb5a0 msgr2=0x7fb2d40cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/3690081755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40cb5a0 0x7fb2d40cb9f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00bc50 tx=0x7fb2bc00b4f0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/3690081755 shutdown_connections 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/3690081755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40c85a0 0x7fb2d40c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/3690081755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40cb5a0 0x7fb2d40cb9f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/3690081755 >> 172.21.15.92:0/3690081755 conn(0x7fb2d40c9870 msgr2=0x7fb2d40c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/3690081755 shutdown_connections 2023-10-06T03:03:25.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.208+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/3690081755 wait complete. 2023-10-06T03:03:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.209+0000 7fb2d9bf3700 1 Processor -- start 2023-10-06T03:03:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.209+0000 7fb2d9bf3700 1 -- start start 2023-10-06T03:03:25.213 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d9bf3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 0x7fb2d40b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d9bf3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40cb5a0 0x7fb2d413cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d9bf3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d4156fb0 0x7fb2d413d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d9bf3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4158390 con 0x7fb2d40c85a0 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d9bf3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4157380 con 0x7fb2d4156fb0 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d9bf3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4157f00 con 0x7fb2d40cb5a0 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 0x7fb2d40b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 0x7fb2d40b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45024/0 (socket says 172.21.15.92:45024) 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 -- 172.21.15.92:0/313473989 learned_addr learned my addr 172.21.15.92:0/313473989 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d3fff700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d4156fb0 0x7fb2d413d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d2ffd700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40cb5a0 0x7fb2d413cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 -- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40cb5a0 msgr2=0x7fb2d413cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40cb5a0 0x7fb2d413cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 -- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d4156fb0 msgr2=0x7fb2d413d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d4156fb0 0x7fb2d413d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 -- 172.21.15.92:0/313473989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc00b790 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.210+0000 7fb2d37fe700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 0x7fb2d40b1eb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c8e0 tx=0x7fb2c4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.211+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012630 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.211+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c40127b0 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.211+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d41575d0 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.211+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4012b70 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.211+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d4157b90 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.212+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d404c3c0 con 0x7fb2d40c85a0 2023-10-06T03:03:25.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.212+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb2c4003030 con 0x7fb2d40c85a0 2023-10-06T03:03:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.212+0000 7fb2cb7fe700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2b4066fc0 0x7fb2b4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.212+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb2c4084240 con 0x7fb2d40c85a0 2023-10-06T03:03:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.212+0000 7fb2d2ffd700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2b4066fc0 0x7fb2b4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.213+0000 7fb2d2ffd700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2b4066fc0 0x7fb2b4069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00bc50 tx=0x7fb2bc01b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.216+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2c4055140 con 0x7fb2d40c85a0 2023-10-06T03:03:25.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.400+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2d41581e0 con 0x7fb2d40c85a0 2023-10-06T03:03:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.401+0000 7fb2cb7fe700 1 -- 172.21.15.92:0/313473989 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+13651 (secure 0 0 0) 0x7fb2c4054cd0 con 0x7fb2d40c85a0 2023-10-06T03:03:25.401 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:25.401 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":62,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:24.665162+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2b4066fc0 msgr2=0x7fb2b4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2b4066fc0 0x7fb2b4069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00bc50 tx=0x7fb2bc01b000 comp rx=0 tx=0).stop 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 msgr2=0x7fb2d40b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 0x7fb2d40b1eb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c8e0 tx=0x7fb2c4001710 comp rx=0 tx=0).stop 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 shutdown_connections 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2b4066fc0 0x7fb2b4069470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2d40c85a0 0x7fb2d40b1eb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2d4156fb0 0x7fb2d413d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 --2- 172.21.15.92:0/313473989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2d40cb5a0 0x7fb2d413cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 >> 172.21.15.92:0/313473989 conn(0x7fb2d40c9870 msgr2=0x7fb2d40c2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 shutdown_connections 2023-10-06T03:03:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.403+0000 7fb2d9bf3700 1 -- 172.21.15.92:0/313473989 wait complete. 2023-10-06T03:03:25.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:03:25.416 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:03:25.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.532+0000 7fbbe5c64700 1 Processor -- start 2023-10-06T03:03:25.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.532+0000 7fbbe5c64700 1 -- start start 2023-10-06T03:03:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5c64700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe0153900 0x7fbbe00c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5c64700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe006a710 0x7fbbe006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5c64700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe006e150 con 0x7fbbe00c9db0 2023-10-06T03:03:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5c64700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe00c7800 con 0x7fbbe006a710 2023-10-06T03:03:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5c64700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00c7ae0 con 0x7fbbe0153900 2023-10-06T03:03:25.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe006a710 0x7fbbe006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbbe006a710 0x7fbbe006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45028/0 (socket says 172.21.15.92:45028) 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 -- 172.21.15.92:0/1874321393 learned_addr learned my addr 172.21.15.92:0/1874321393 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe4c62700 1 --2- 172.21.15.92:0/1874321393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe0153900 0x7fbbe00c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 -- 172.21.15.92:0/1874321393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe0153900 msgr2=0x7fbbe00c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 --2- 172.21.15.92:0/1874321393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe0153900 0x7fbbe00c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 -- 172.21.15.92:0/1874321393 >> v1:172.21.15.136:6789/0 conn(0x7fbbe00c9db0 legacy=0x7fbbe006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 -- 172.21.15.92:0/1874321393 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00c7dc0 con 0x7fbbe006a710 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbe5463700 1 --2- 172.21.15.92:0/1874321393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe006a710 0x7fbbe006e5e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009f20 tx=0x7fbbd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d14219eb53c905 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbdf7fe700 1 -- 172.21.15.92:0/1874321393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800e070 con 0x7fbbe006a710 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbdf7fe700 1 -- 172.21.15.92:0/1874321393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800b9d0 con 0x7fbbe006a710 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.533+0000 7fbbdf7fe700 1 -- 172.21.15.92:0/1874321393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80092e0 con 0x7fbbe006a710 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1874321393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe006a710 msgr2=0x7fbbe006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1874321393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe006a710 0x7fbbe006e5e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009f20 tx=0x7fbbd8009840 comp rx=0 tx=0).stop 2023-10-06T03:03:25.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1874321393 shutdown_connections 2023-10-06T03:03:25.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1874321393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe006a710 0x7fbbe006e5e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1874321393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe0153900 0x7fbbe00c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1874321393 >> 172.21.15.92:0/1874321393 conn(0x7fbbe00cc1e0 msgr2=0x7fbbe00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:25.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1874321393 shutdown_connections 2023-10-06T03:03:25.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.534+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1874321393 wait complete. 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.535+0000 7fbbe5c64700 1 Processor -- start 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.535+0000 7fbbe5c64700 1 -- start start 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5c64700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe006a710 0x7fbbe00b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5c64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 0x7fbbe00ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5c64700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbe0153900 0x7fbbe0148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5c64700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00aef80 con 0x7fbbe00c9db0 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5c64700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00c36a0 con 0x7fbbe0153900 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5c64700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00ae2e0 con 0x7fbbe006a710 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 0x7fbbe00ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 0x7fbbe00ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45044/0 (socket says 172.21.15.92:45044) 2023-10-06T03:03:25.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 -- 172.21.15.92:0/1167367498 learned_addr learned my addr 172.21.15.92:0/1167367498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5463700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbe0153900 0x7fbbe0148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe4c62700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe006a710 0x7fbbe00b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 -- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe006a710 msgr2=0x7fbbe00b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe006a710 0x7fbbe00b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 -- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbe0153900 msgr2=0x7fbbe0148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbe0153900 0x7fbbe0148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbdffff700 1 -- 172.21.15.92:0/1167367498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8009ae0 con 0x7fbbe00c9db0 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe5463700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbe0153900 0x7fbbe0148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.536+0000 7fbbe4c62700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe006a710 0x7fbbe00b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.537+0000 7fbbdffff700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 0x7fbbe00ada90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c590 tx=0x7fbbcc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.537+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc011590 con 0x7fbbe00c9db0 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.537+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc0116f0 con 0x7fbbe00c9db0 2023-10-06T03:03:25.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.537+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc011ab0 con 0x7fbbe00c9db0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.537+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe00ae5c0 con 0x7fbbe00c9db0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.537+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00c3330 con 0x7fbbe00c9db0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.538+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe006c450 con 0x7fbbe00c9db0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.538+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fbbcc01b3b0 con 0x7fbbe00c9db0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.538+0000 7fbbdcff9700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbbc4066fc0 0x7fbbc4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.539+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbbcc083910 con 0x7fbbe00c9db0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.539+0000 7fbbe4c62700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbbc4066fc0 0x7fbbc4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.539+0000 7fbbe4c62700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbbc4066fc0 0x7fbbc4069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009ce0 tx=0x7fbbd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.543+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbcc050d80 con 0x7fbbe00c9db0 2023-10-06T03:03:25.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.727+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbe006dd20 con 0x7fbbe00c9db0 2023-10-06T03:03:25.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.728+0000 7fbbdcff9700 1 -- 172.21.15.92:0/1167367498 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbcc017020 con 0x7fbbe00c9db0 2023-10-06T03:03:25.728 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbbc4066fc0 msgr2=0x7fbbc4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbbc4066fc0 0x7fbbc4069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009ce0 tx=0x7fbbd4008040 comp rx=0 tx=0).stop 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 msgr2=0x7fbbe00ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 0x7fbbe00ada90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00c590 tx=0x7fbbcc00c950 comp rx=0 tx=0).stop 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 shutdown_connections 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbbc4066fc0 0x7fbbc4069470 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbe0153900 0x7fbbe0148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbe006a710 0x7fbbe00b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 --2- 172.21.15.92:0/1167367498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbbe00c9db0 0x7fbbe00ada90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 >> 172.21.15.92:0/1167367498 conn(0x7fbbe00cc1e0 msgr2=0x7fbbe014cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:25.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 shutdown_connections 2023-10-06T03:03:25.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.730+0000 7fbbe5c64700 1 -- 172.21.15.92:0/1167367498 wait complete. 2023-10-06T03:03:25.742 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:03:25.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.860+0000 7fcb7621f700 1 Processor -- start 2023-10-06T03:03:25.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.860+0000 7fcb7621f700 1 -- start start 2023-10-06T03:03:25.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.860+0000 7fcb7621f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb7006aa40 0x7fcb700cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.860+0000 7fcb7621f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb700cc210 con 0x7fcb700c91e0 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.860+0000 7fcb7621f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb700cc4f0 con 0x7fcb7006bac0 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.860+0000 7fcb7621f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700cc7d0 con 0x7fcb7006aa40 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcb700c91e0 0x7fcb70160390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57726/0 (socket says 172.21.15.92:57726) 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6ffff700 1 -- 172.21.15.92:0/1217483874 learned_addr learned my addr 172.21.15.92:0/1217483874 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb7006aa40 0x7fcb700cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103308811 0 0) 0x7fcb700cc210 con 0x7fcb700c91e0 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb58003570 con 0x7fcb700c91e0 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2925043124 0 0) 0x7fcb58003570 con 0x7fcb700c91e0 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb7006aa40 msgr2=0x7fcb700cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:25.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb7006aa40 0x7fcb700cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 >> v1:172.21.15.136:6789/0 conn(0x7fcb7006bac0 legacy=0x7fcb7006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700ccab0 con 0x7fcb700c91e0 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fcb680017f0 con 0x7fcb700c91e0 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 >> v1:172.21.15.92:6789/0 conn(0x7fcb700c91e0 legacy=0x7fcb70160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb58003a90 0x7fcb58003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58004810 0x7fcb58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb580051c0 0x7fcb5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700cc7d0 con 0x7fcb58003a90 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5800bfa0 con 0x7fcb58004810 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5800c160 con 0x7fcb580051c0 2023-10-06T03:03:25.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb7006aa40 0x7fcb700cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6effd700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58004810 0x7fcb58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb58003a90 0x7fcb58003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.861+0000 7fcb6ffff700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb580051c0 0x7fcb5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6f7fe700 1 -- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb580051c0 msgr2=0x7fcb5800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb580051c0 0x7fcb5800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6f7fe700 1 -- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58004810 msgr2=0x7fcb58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58004810 0x7fcb58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6f7fe700 1 -- 172.21.15.92:0/1217483874 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58003570 con 0x7fcb58003a90 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6effd700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58004810 0x7fcb58004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb58003a90 0x7fcb58003e60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcb7006c440 tx=0x7fcb60009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f1066ff5f8e14a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000e710 con 0x7fcb58003a90 2023-10-06T03:03:25.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6000e870 con 0x7fcb58003a90 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6e7fc700 1 -- 172.21.15.92:0/1217483874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000ec10 con 0x7fcb58003a90 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb7621f700 1 -- 172.21.15.92:0/1217483874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb58003a90 msgr2=0x7fcb58003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb7621f700 1 --2- 172.21.15.92:0/1217483874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb58003a90 0x7fcb58003e60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcb7006c440 tx=0x7fcb60009120 comp rx=0 tx=0).stop 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb6ffff700 1 -- 172.21.15.92:0/1217483874 reap_dead start 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb7621f700 1 -- 172.21.15.92:0/1217483874 shutdown_connections 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.862+0000 7fcb7621f700 1 -- 172.21.15.92:0/1217483874 >> 172.21.15.92:0/1217483874 conn(0x7fcb700cd670 msgr2=0x7fcb700cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.863+0000 7fcb7621f700 1 -- 172.21.15.92:0/1217483874 shutdown_connections 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.863+0000 7fcb7621f700 1 -- 172.21.15.92:0/1217483874 wait complete. 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.864+0000 7fcb7621f700 1 Processor -- start 2023-10-06T03:03:25.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.864+0000 7fcb7621f700 1 -- start start 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58003a90 0x7fcb700b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb700b0e20 0x7fcb70149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 0x7fcb70147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700b12b0 con 0x7fcb700685e0 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700b1590 con 0x7fcb58003a90 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700b1870 con 0x7fcb700b0e20 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 0x7fcb70147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 0x7fcb70147ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45074/0 (socket says 172.21.15.92:45074) 2023-10-06T03:03:25.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 -- 172.21.15.92:0/3620881190 learned_addr learned my addr 172.21.15.92:0/3620881190 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6effd700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb700b0e20 0x7fcb70149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58003a90 0x7fcb700b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 -- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb700b0e20 msgr2=0x7fcb70149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb700b0e20 0x7fcb70149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 -- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58003a90 msgr2=0x7fcb700b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58003a90 0x7fcb700b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 -- 172.21.15.92:0/3620881190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700b1b50 con 0x7fcb700685e0 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb6ffff700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 0x7fcb70147ca0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcb7006e510 tx=0x7fcb680021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800ae30 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb680133f0 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb700b1e30 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680137b0 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.865+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb70068a80 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.866+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb700c45f0 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.867+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fcb68013ba0 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.867+0000 7fcb577fe700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb5c066fc0 0x7fcb5c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.867+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcb68084bf0 con 0x7fcb700685e0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.867+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb5c066fc0 0x7fcb5c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:25.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.868+0000 7fcb6f7fe700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb5c066fc0 0x7fcb5c069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009160 tx=0x7fcb60019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:25.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:25.871+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb68052120 con 0x7fcb700685e0 2023-10-06T03:03:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.054+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcb7004c3c0 con 0x7fcb700685e0 2023-10-06T03:03:26.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.054+0000 7fcb577fe700 1 -- 172.21.15.92:0/3620881190 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcb68055740 con 0x7fcb700685e0 2023-10-06T03:03:26.055 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:03:26.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb5c066fc0 msgr2=0x7fcb5c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb5c066fc0 0x7fcb5c069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009160 tx=0x7fcb60019000 comp rx=0 tx=0).stop 2023-10-06T03:03:26.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 msgr2=0x7fcb70147ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 0x7fcb70147ca0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcb7006e510 tx=0x7fcb680021f0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 shutdown_connections 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcb5c066fc0 0x7fcb5c069470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb700685e0 0x7fcb70147ca0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb58003a90 0x7fcb700b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 --2- 172.21.15.92:0/3620881190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcb700b0e20 0x7fcb70149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 >> 172.21.15.92:0/3620881190 conn(0x7fcb700cd670 msgr2=0x7fcb701538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 shutdown_connections 2023-10-06T03:03:26.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.056+0000 7fcb7621f700 1 -- 172.21.15.92:0/3620881190 wait complete. 2023-10-06T03:03:26.069 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:03:26.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.183+0000 7eff21d38700 1 Processor -- start 2023-10-06T03:03:26.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.183+0000 7eff21d38700 1 -- start start 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff21d38700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7eff1c15d230 0x7eff1c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff21d38700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1c0b2980 con 0x7eff1c1597f0 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff21d38700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1c15db70 con 0x7eff1c159420 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff21d38700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c0b11d0 con 0x7eff1c15d230 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff1bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7eff1c1597f0 0x7eff1c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57732/0 (socket says 172.21.15.92:57732) 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff1bfff700 1 -- 172.21.15.92:0/2924916974 learned_addr learned my addr 172.21.15.92:0/2924916974 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff21537700 1 --2- 172.21.15.92:0/2924916974 >> v2:172.21.15.136:3300/0 conn(0x7eff1c15d230 0x7eff1c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316138767 0 0) 0x7eff1c0b2980 con 0x7eff1c1597f0 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff0c003570 con 0x7eff1c1597f0 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.184+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3289014031 0 0) 0x7eff0c003570 con 0x7eff1c1597f0 2023-10-06T03:03:26.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 >> v2:172.21.15.136:3300/0 conn(0x7eff1c15d230 msgr2=0x7eff1c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 --2- 172.21.15.92:0/2924916974 >> v2:172.21.15.136:3300/0 conn(0x7eff1c15d230 0x7eff1c15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 >> v1:172.21.15.136:6789/0 conn(0x7eff1c159420 legacy=0x7eff1c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c0b14b0 con 0x7eff1c1597f0 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7eff08001470 con 0x7eff1c1597f0 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 >> v1:172.21.15.92:6789/0 conn(0x7eff1c1597f0 legacy=0x7eff1c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c003a60 0x7eff0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff0c004810 0x7eff0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff0c0051c0 0x7eff0c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0b11d0 con 0x7eff0c0051c0 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c00c0d0 con 0x7eff0c003a60 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c00c290 con 0x7eff0c004810 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff0c0051c0 0x7eff0c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1bfff700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff0c004810 0x7eff0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff20d36700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c003a60 0x7eff0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 -- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff0c004810 msgr2=0x7eff0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff0c004810 0x7eff0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 -- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c003a60 msgr2=0x7eff0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c003a60 0x7eff0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 -- 172.21.15.92:0/2924916974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c003570 con 0x7eff0c0051c0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 -- 172.21.15.92:0/2924916974 reap_dead start 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff21537700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff0c0051c0 0x7eff0c00ba60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7eff1400e210 tx=0x7eff1400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa5bcca35880a67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400eac0 con 0x7eff0c0051c0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400f040 con 0x7eff0c0051c0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.185+0000 7eff1b7fe700 1 -- 172.21.15.92:0/2924916974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400ee70 con 0x7eff0c0051c0 2023-10-06T03:03:26.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 -- 172.21.15.92:0/2924916974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff0c0051c0 msgr2=0x7eff0c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff0c0051c0 0x7eff0c00ba60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7eff1400e210 tx=0x7eff1400b040 comp rx=0 tx=0).stop 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 -- 172.21.15.92:0/2924916974 shutdown_connections 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 --2- 172.21.15.92:0/2924916974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff0c0051c0 0x7eff0c00ba60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 -- 172.21.15.92:0/2924916974 >> 172.21.15.92:0/2924916974 conn(0x7eff1c0d0660 msgr2=0x7eff1c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 -- 172.21.15.92:0/2924916974 shutdown_connections 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.186+0000 7eff21d38700 1 -- 172.21.15.92:0/2924916974 wait complete. 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.187+0000 7eff21d38700 1 Processor -- start 2023-10-06T03:03:26.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.187+0000 7eff21d38700 1 -- start start 2023-10-06T03:03:26.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21d38700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c0051c0 0x7eff1c16db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21d38700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff1c14d0e0 0x7eff1c0ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21d38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 0x7eff1c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21d38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0cf870 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21d38700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c16e040 con 0x7eff0c0051c0 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21d38700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c15ca30 con 0x7eff1c14d0e0 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 0x7eff1c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 0x7eff1c159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45100/0 (socket says 172.21.15.92:45100) 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 -- 172.21.15.92:0/427159826 learned_addr learned my addr 172.21.15.92:0/427159826 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff1bfff700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff1c14d0e0 0x7eff1c0ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff20d36700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c0051c0 0x7eff1c16db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 -- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff1c14d0e0 msgr2=0x7eff1c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff1c14d0e0 0x7eff1c0ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 -- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c0051c0 msgr2=0x7eff1c16db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c0051c0 0x7eff1c16db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 -- 172.21.15.92:0/427159826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c15cec0 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff20d36700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c0051c0 0x7eff1c16db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.188+0000 7eff21537700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 0x7eff1c159ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff1c15e340 tx=0x7eff14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.189+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1401d660 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.189+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff14022070 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.189+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c15be20 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.189+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1401da30 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.189+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c15c3e0 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.189+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c04c3c0 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.190+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7eff14003020 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.190+0000 7eff18ff9700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff00066fc0 0x7eff00069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.190+0000 7eff20d36700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff00066fc0 0x7eff00069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.190+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5206+0+0 (secure 0 0 0) 0x7eff14024070 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.191+0000 7eff20d36700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff00066fc0 0x7eff00069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff10007e40 tx=0x7eff10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:26.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.195+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff14053c30 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.380+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7eff1c15cb70 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.381+0000 7eff18ff9700 1 -- 172.21.15.92:0/427159826 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+126 (secure 0 0 0) 0x7eff1401de60 con 0x7eff1c0cf3f0 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:03:26.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff00066fc0 msgr2=0x7eff00069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff00066fc0 0x7eff00069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7eff10007e40 tx=0x7eff10009040 comp rx=0 tx=0).stop 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 msgr2=0x7eff1c159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 0x7eff1c159ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff1c15e340 tx=0x7eff14019000 comp rx=0 tx=0).stop 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 shutdown_connections 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff00066fc0 0x7eff00069470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff1c0cf3f0 0x7eff1c159ca0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0c0051c0 0x7eff1c16db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 --2- 172.21.15.92:0/427159826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff1c14d0e0 0x7eff1c0ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 >> 172.21.15.92:0/427159826 conn(0x7eff1c0d0660 msgr2=0x7eff1c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 shutdown_connections 2023-10-06T03:03:26.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.383+0000 7eff21d38700 1 -- 172.21.15.92:0/427159826 wait complete. 2023-10-06T03:03:26.395 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:03:26.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:03:26.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.513+0000 7fd92f23a700 1 Processor -- start 2023-10-06T03:03:26.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.513+0000 7fd92f23a700 1 -- start start 2023-10-06T03:03:26.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92f23a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd928159ec0 0x7fd9280b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92f23a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9280b53d0 0x7fd9280cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92f23a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd9280cce40 0x7fd9280cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92f23a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280b5b00 con 0x7fd928159ec0 2023-10-06T03:03:26.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92f23a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280b5850 con 0x7fd9280cce40 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92f23a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280b5da0 con 0x7fd9280b53d0 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd928159ec0 0x7fd9280b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd928159ec0 0x7fd9280b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45104/0 (socket says 172.21.15.92:45104) 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 -- 172.21.15.92:0/14545165 learned_addr learned my addr 172.21.15.92:0/14545165 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 -- 172.21.15.92:0/14545165 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9280b53d0 msgr2=0x7fd9280cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 --2- 172.21.15.92:0/14545165 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9280b53d0 0x7fd9280cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 -- 172.21.15.92:0/14545165 >> v2:172.21.15.136:3300/0 conn(0x7fd9280cce40 msgr2=0x7fd9280cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.514+0000 7fd92cfd6700 1 --2- 172.21.15.92:0/14545165 >> v2:172.21.15.136:3300/0 conn(0x7fd9280cce40 0x7fd9280cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92cfd6700 1 -- 172.21.15.92:0/14545165 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280b6080 con 0x7fd928159ec0 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92cfd6700 1 --2- 172.21.15.92:0/14545165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd928159ec0 0x7fd9280b4e90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd9280c75b0 tx=0x7fd918009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b8a2114e4f3a72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd9277fe700 1 -- 172.21.15.92:0/14545165 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918005c10 con 0x7fd928159ec0 2023-10-06T03:03:26.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd9277fe700 1 -- 172.21.15.92:0/14545165 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918005d70 con 0x7fd928159ec0 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd9277fe700 1 -- 172.21.15.92:0/14545165 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800b880 con 0x7fd928159ec0 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 -- 172.21.15.92:0/14545165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd928159ec0 msgr2=0x7fd9280b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 --2- 172.21.15.92:0/14545165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd928159ec0 0x7fd9280b4e90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd9280c75b0 tx=0x7fd918009220 comp rx=0 tx=0).stop 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 -- 172.21.15.92:0/14545165 shutdown_connections 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 --2- 172.21.15.92:0/14545165 >> v2:172.21.15.136:3300/0 conn(0x7fd9280cce40 0x7fd9280cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 --2- 172.21.15.92:0/14545165 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd928159ec0 0x7fd9280b4e90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 --2- 172.21.15.92:0/14545165 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9280b53d0 0x7fd9280cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 -- 172.21.15.92:0/14545165 >> 172.21.15.92:0/14545165 conn(0x7fd9280999f0 msgr2=0x7fd9280b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 -- 172.21.15.92:0/14545165 shutdown_connections 2023-10-06T03:03:26.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.515+0000 7fd92f23a700 1 -- 172.21.15.92:0/14545165 wait complete. 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.516+0000 7fd92f23a700 1 Processor -- start 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 -- start start 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 0x7fd928156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9280cce40 0x7fd928154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd928159ec0 0x7fd9281535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92806ae70 con 0x7fd9280b53d0 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928068430 con 0x7fd9280cce40 2023-10-06T03:03:26.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92f23a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd928068710 con 0x7fd928159ec0 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92cfd6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 0x7fd928156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92cfd6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 0x7fd928156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45112/0 (socket says 172.21.15.92:45112) 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.517+0000 7fd92cfd6700 1 -- 172.21.15.92:0/661807886 learned_addr learned my addr 172.21.15.92:0/661807886 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd927fff700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9280cce40 0x7fd928154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92cfd6700 1 -- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd928159ec0 msgr2=0x7fd9281535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92d7d7700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd928159ec0 0x7fd9281535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92cfd6700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd928159ec0 0x7fd9281535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92cfd6700 1 -- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9280cce40 msgr2=0x7fd928154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92cfd6700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9280cce40 0x7fd928154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92cfd6700 1 -- 172.21.15.92:0/661807886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918009480 con 0x7fd9280b53d0 2023-10-06T03:03:26.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd927fff700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9280cce40 0x7fd928154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92cfd6700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 0x7fd928156bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd928089390 tx=0x7fd918016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180118e0 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918020070 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918011cb0 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9280689f0 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.518+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd928068d80 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.519+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd928063c90 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.519+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd918022030 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.520+0000 7fd924ff9700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd914066fc0 0x7fd914069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.520+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd918085fb0 con 0x7fd9280b53d0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.520+0000 7fd927fff700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd914066fc0 0x7fd914069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:26.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.520+0000 7fd927fff700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd914066fc0 0x7fd914069470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd92815f5c0 tx=0x7fd91c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:26.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.524+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd918053420 con 0x7fd9280b53d0 2023-10-06T03:03:26.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:26.706+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9280691f0 con 0x7fd9280b53d0 2023-10-06T03:03:27.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.689+0000 7fd924ff9700 1 -- 172.21.15.92:0/661807886 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd918056a40 con 0x7fd9280b53d0 2023-10-06T03:03:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.692+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd914066fc0 msgr2=0x7fd914069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.692+0000 7fd92f23a700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd914066fc0 0x7fd914069470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd92815f5c0 tx=0x7fd91c008040 comp rx=0 tx=0).stop 2023-10-06T03:03:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.692+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 msgr2=0x7fd928156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.692+0000 7fd92f23a700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 0x7fd928156bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd928089390 tx=0x7fd918016000 comp rx=0 tx=0).stop 2023-10-06T03:03:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 shutdown_connections 2023-10-06T03:03:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd914066fc0 0x7fd914069470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9280b53d0 0x7fd928156bf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd928159ec0 0x7fd9281535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 --2- 172.21.15.92:0/661807886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9280cce40 0x7fd928154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 >> 172.21.15.92:0/661807886 conn(0x7fd9280999f0 msgr2=0x7fd9281520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:27.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.693+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 shutdown_connections 2023-10-06T03:03:27.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.694+0000 7fd92f23a700 1 -- 172.21.15.92:0/661807886 wait complete. 2023-10-06T03:03:27.695 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:03:27.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.828+0000 7f5edc808700 1 Processor -- start 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.828+0000 7f5edc808700 1 -- start start 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5edc808700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ed806bcd0 0x7f5ed80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5edc808700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed815b790 con 0x7f5ed81414c0 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5edc808700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed815ba70 con 0x7f5ed8158d40 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5edc808700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed815bd50 con 0x7f5ed806bcd0 2023-10-06T03:03:27.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5ed659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ed806bcd0 0x7f5ed80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5ed6d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5ed81414c0 0x7f5ed80cc290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57744/0 (socket says 172.21.15.92:57744) 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.829+0000 7f5ed6d9d700 1 -- 172.21.15.92:0/254748597 learned_addr learned my addr 172.21.15.92:0/254748597 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed659c700 1 -- 172.21.15.92:0/254748597 >> v1:172.21.15.136:6789/0 conn(0x7f5ed8158d40 legacy=0x7f5ed8140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed659c700 1 -- 172.21.15.92:0/254748597 >> v1:172.21.15.92:6789/0 conn(0x7f5ed81414c0 legacy=0x7f5ed80cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed5d9b700 1 -- 172.21.15.92:0/254748597 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026780820 0 0) 0x7f5ed815b790 con 0x7f5ed81414c0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed659c700 1 -- 172.21.15.92:0/254748597 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed815c030 con 0x7f5ed806bcd0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed659c700 1 --2- 172.21.15.92:0/254748597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed80c9230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80098e0 tx=0x7f5ec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fc2000554341e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed5d9b700 1 -- 172.21.15.92:0/254748597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8009e80 con 0x7f5ed806bcd0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed5d9b700 1 -- 172.21.15.92:0/254748597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec800b860 con 0x7f5ed806bcd0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5ed5d9b700 1 -- 172.21.15.92:0/254748597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800bbe0 con 0x7f5ed806bcd0 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 -- 172.21.15.92:0/254748597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 msgr2=0x7f5ed80c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 --2- 172.21.15.92:0/254748597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed80c9230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80098e0 tx=0x7f5ec80091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 -- 172.21.15.92:0/254748597 shutdown_connections 2023-10-06T03:03:27.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 --2- 172.21.15.92:0/254748597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed80c9230 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 -- 172.21.15.92:0/254748597 >> 172.21.15.92:0/254748597 conn(0x7f5ed80ce5c0 msgr2=0x7f5ed80ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:27.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 -- 172.21.15.92:0/254748597 shutdown_connections 2023-10-06T03:03:27.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.830+0000 7f5edc808700 1 -- 172.21.15.92:0/254748597 wait complete. 2023-10-06T03:03:27.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 Processor -- start 2023-10-06T03:03:27.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 -- start start 2023-10-06T03:03:27.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed816ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed81414c0 0x7f5ed81580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ed8158d40 0x7f5ed81585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8168e20 con 0x7f5ed806bcd0 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8168210 con 0x7f5ed81414c0 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5edc808700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed81684f0 con 0x7f5ed8158d40 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed816ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed816ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45138/0 (socket says 172.21.15.92:45138) 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.832+0000 7f5ed659c700 1 -- 172.21.15.92:0/3564853587 learned_addr learned my addr 172.21.15.92:0/3564853587 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed6d9d700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ed8158d40 0x7f5ed81585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ecdd9b700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed81414c0 0x7f5ed81580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed659c700 1 -- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ed8158d40 msgr2=0x7f5ed81585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed659c700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ed8158d40 0x7f5ed81585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed659c700 1 -- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed81414c0 msgr2=0x7f5ed81580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed659c700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed81414c0 0x7f5ed81580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed659c700 1 -- 172.21.15.92:0/3564853587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8009420 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5ed659c700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed816ae00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8069220 tx=0x7f5ec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec801f070 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec8011420 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec800bbe0 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed81687d0 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.833+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed816b340 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.834+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed80718b0 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.834+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5ec8011a50 con 0x7f5ed806bcd0 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.835+0000 7f5eceffd700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5eb8066fc0 0x7f5eb8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:27.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.835+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f5ec8085a30 con 0x7f5ed806bcd0 2023-10-06T03:03:27.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.835+0000 7f5ecdd9b700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5eb8066fc0 0x7f5eb8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:27.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.835+0000 7f5ecdd9b700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5eb8066fc0 0x7f5eb8069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8165cf0 tx=0x7f5ec0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:27.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:27.839+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ec8052d30 con 0x7f5ed806bcd0 2023-10-06T03:03:28.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.024+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5ed813c910 con 0x7f5ed806bcd0 2023-10-06T03:03:28.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.689+0000 7f5eceffd700 1 -- 172.21.15.92:0/3564853587 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5ec80150d0 con 0x7f5ed806bcd0 2023-10-06T03:03:28.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.694+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5eb8066fc0 msgr2=0x7f5eb8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:28.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.694+0000 7f5edc808700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5eb8066fc0 0x7f5eb8069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8165cf0 tx=0x7f5ec0009040 comp rx=0 tx=0).stop 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.694+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 msgr2=0x7f5ed816ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.694+0000 7f5edc808700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed816ae00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8069220 tx=0x7f5ec8017000 comp rx=0 tx=0).stop 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.695+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 shutdown_connections 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.695+0000 7f5edc808700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5eb8066fc0 0x7f5eb8069470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.695+0000 7f5edc808700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ed806bcd0 0x7f5ed816ae00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.695+0000 7f5edc808700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ed8158d40 0x7f5ed81585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.695+0000 7f5edc808700 1 --2- 172.21.15.92:0/3564853587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ed81414c0 0x7f5ed81580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.695+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 >> 172.21.15.92:0/3564853587 conn(0x7f5ed80ce5c0 msgr2=0x7f5ed80cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:28.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.696+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 shutdown_connections 2023-10-06T03:03:28.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.697+0000 7f5edc808700 1 -- 172.21.15.92:0/3564853587 wait complete. 2023-10-06T03:03:28.706 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:03:28.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:03:28.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.838+0000 7fe562fb8700 1 Processor -- start 2023-10-06T03:03:28.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.838+0000 7fe562fb8700 1 -- start start 2023-10-06T03:03:28.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe562fb8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe55c0b4e70 0x7fe55c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:28.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe562fb8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe55c0b1b40 0x7fe55c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:28.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe562fb8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c0cfb00 con 0x7fe55c0b16d0 2023-10-06T03:03:28.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe562fb8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c0b5c40 con 0x7fe55c0b1b40 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe562fb8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c0b5f20 con 0x7fe55c0b4e70 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe55c0b1b40 0x7fe55c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5617b5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe55c0b16d0 0x7fe55c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57752/0 (socket says 172.21.15.92:57752) 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe55c0b1b40 0x7fe55c15d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45152/0 (socket says 172.21.15.92:45152) 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5617b5700 1 -- 172.21.15.92:0/2671554230 learned_addr learned my addr 172.21.15.92:0/2671554230 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe561fb6700 1 --2- 172.21.15.92:0/2671554230 >> v2:172.21.15.136:3300/0 conn(0x7fe55c0b4e70 0x7fe55c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 -- 172.21.15.92:0/2671554230 >> v2:172.21.15.136:3300/0 conn(0x7fe55c0b4e70 msgr2=0x7fe55c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 --2- 172.21.15.92:0/2671554230 >> v2:172.21.15.136:3300/0 conn(0x7fe55c0b4e70 0x7fe55c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 -- 172.21.15.92:0/2671554230 >> v1:172.21.15.92:6789/0 conn(0x7fe55c0b16d0 legacy=0x7fe55c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 -- 172.21.15.92:0/2671554230 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0b6200 con 0x7fe55c0b1b40 2023-10-06T03:03:28.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe560fb4700 1 -- 172.21.15.92:0/2671554230 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448170979 0 0) 0x7fe55c0cfb00 con 0x7fe55c0b16d0 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.839+0000 7fe5627b7700 1 --2- 172.21.15.92:0/2671554230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c15d950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe55c064910 tx=0x7fe558009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09b0c60242a1a37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe560fb4700 1 -- 172.21.15.92:0/2671554230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800e070 con 0x7fe55c0b1b40 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe560fb4700 1 -- 172.21.15.92:0/2671554230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800bc30 con 0x7fe55c0b1b40 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe560fb4700 1 -- 172.21.15.92:0/2671554230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558012430 con 0x7fe55c0b1b40 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 -- 172.21.15.92:0/2671554230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 msgr2=0x7fe55c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2671554230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c15d950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe55c064910 tx=0x7fe558009780 comp rx=0 tx=0).stop 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 -- 172.21.15.92:0/2671554230 shutdown_connections 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2671554230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c15d950 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2671554230 >> v2:172.21.15.136:3300/0 conn(0x7fe55c0b4e70 0x7fe55c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 -- 172.21.15.92:0/2671554230 >> 172.21.15.92:0/2671554230 conn(0x7fe55c0d10c0 msgr2=0x7fe55c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 -- 172.21.15.92:0/2671554230 shutdown_connections 2023-10-06T03:03:28.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.840+0000 7fe562fb8700 1 -- 172.21.15.92:0/2671554230 wait complete. 2023-10-06T03:03:28.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.841+0000 7fe562fb8700 1 Processor -- start 2023-10-06T03:03:28.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.841+0000 7fe562fb8700 1 -- start start 2023-10-06T03:03:28.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe562fb8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0b16d0 0x7fe55c145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:28.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe562fb8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:28.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe562fb8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c0b4e70 0x7fe55c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe562fb8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c158810 con 0x7fe55c0b1b40 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe562fb8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0c9030 con 0x7fe55c0b4e70 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe562fb8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0c6bf0 con 0x7fe55c0b16d0 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c0ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45154/0 (socket says 172.21.15.92:45154) 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 -- 172.21.15.92:0/2452811575 learned_addr learned my addr 172.21.15.92:0/2452811575 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 -- 172.21.15.92:0/2452811575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0b16d0 msgr2=0x7fe55c145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0b16d0 0x7fe55c145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 -- 172.21.15.92:0/2452811575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c0b4e70 msgr2=0x7fe55c0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c0b4e70 0x7fe55c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:28.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.842+0000 7fe5617b5700 1 -- 172.21.15.92:0/2452811575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe558009a20 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.843+0000 7fe5617b5700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c0ca760 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00c7e0 tx=0x7fe54c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.843+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c005b30 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.843+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c005c90 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.843+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c007300 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.843+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c0c8700 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.843+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c0c8cc0 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.844+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c088990 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.844+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe54c01b400 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.844+0000 7fe551ffb700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540066fc0 0x7fe540069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.844+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fe54c0841d0 con 0x7fe55c0b1b40 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.845+0000 7fe561fb6700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540066fc0 0x7fe540069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:28.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.845+0000 7fe561fb6700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540066fc0 0x7fe540069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe554009a40 tx=0x7fe554008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:28.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:28.849+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe54c0513d0 con 0x7fe55c0b1b40 2023-10-06T03:03:29.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.074+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe55c0c9170 con 0x7fe55c0b1b40 2023-10-06T03:03:29.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.718+0000 7fe551ffb700 1 -- 172.21.15.92:0/2452811575 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v34) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe55c0c9170 con 0x7fe55c0b1b40 2023-10-06T03:03:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.722+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540066fc0 msgr2=0x7fe540069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.722+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540066fc0 0x7fe540069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe554009a40 tx=0x7fe554008040 comp rx=0 tx=0).stop 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.722+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 msgr2=0x7fe55c0ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.722+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c0ca760 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00c7e0 tx=0x7fe54c007df0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.723+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 shutdown_connections 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.723+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540066fc0 0x7fe540069470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.723+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c0b4e70 0x7fe55c0caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.723+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0b16d0 0x7fe55c145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.723+0000 7fe562fb8700 1 --2- 172.21.15.92:0/2452811575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c0b1b40 0x7fe55c0ca760 secure :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00c7e0 tx=0x7fe54c007df0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.723+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 >> 172.21.15.92:0/2452811575 conn(0x7fe55c0d10c0 msgr2=0x7fe55c15c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:29.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.731+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 shutdown_connections 2023-10-06T03:03:29.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.731+0000 7fe562fb8700 1 -- 172.21.15.92:0/2452811575 wait complete. 2023-10-06T03:03:29.733 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T03:03:29.745 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.859+0000 7fd92265b700 1 Processor -- start 2023-10-06T03:03:29.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.859+0000 7fd92265b700 1 -- start start 2023-10-06T03:03:29.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd92265b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 0x7fd91c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd92265b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c15a160 con 0x7fd91c069560 2023-10-06T03:03:29.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd92265b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0d13a0 con 0x7fd91c159c60 2023-10-06T03:03:29.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd92265b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c15e140 con 0x7fd91c159890 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd920bf8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd91c069560 0x7fd91c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59496/0 (socket says 172.21.15.92:59496) 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd920bf8700 1 -- 172.21.15.92:0/1027894362 learned_addr learned my addr 172.21.15.92:0/1027894362 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.860+0000 7fd91bfff700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 0x7fd91c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245070732 0 0) 0x7fd91c15a160 con 0x7fd91c069560 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd904003570 con 0x7fd91c069560 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271068445 0 0) 0x7fd91c0d13a0 con 0x7fd91c159c60 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd91c15a160 con 0x7fd91c159c60 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3048488123 0 0) 0x7fd904003570 con 0x7fd91c069560 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 msgr2=0x7fd91c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 0x7fd91c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 >> v1:172.21.15.136:6789/0 conn(0x7fd91c159c60 legacy=0x7fd91c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c15a3d0 con 0x7fd91c069560 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd9140017f0 con 0x7fd91c069560 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 >> v1:172.21.15.92:6789/0 conn(0x7fd91c069560 legacy=0x7fd91c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd904004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd904004810 0x7fd904004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 0x7fd904005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c15e140 con 0x7fd904003c20 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90400bbf0 con 0x7fd904004810 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90400bdb0 con 0x7fd91c159890 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91bfff700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd904004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91b7fe700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd904004810 0x7fd904004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd920bf8700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 0x7fd904005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91bfff700 1 -- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 msgr2=0x7fd904005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91bfff700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c159890 0x7fd904005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91bfff700 1 -- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd904004810 msgr2=0x7fd904004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91bfff700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd904004810 0x7fd904004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91bfff700 1 -- 172.21.15.92:0/1027894362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904003570 con 0x7fd904003c20 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.861+0000 7fd91b7fe700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd904004810 0x7fd904004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd91bfff700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd904004010 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009c00 tx=0x7fd90c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfc1f3264d94213 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00e710 con 0x7fd904003c20 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00e870 con 0x7fd904003c20 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd91affd700 1 -- 172.21.15.92:0/1027894362 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00ec30 con 0x7fd904003c20 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd92265b700 1 -- 172.21.15.92:0/1027894362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 msgr2=0x7fd904004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd92265b700 1 --2- 172.21.15.92:0/1027894362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd904004010 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009c00 tx=0x7fd90c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd920bf8700 1 -- 172.21.15.92:0/1027894362 reap_dead start 2023-10-06T03:03:29.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd92265b700 1 -- 172.21.15.92:0/1027894362 shutdown_connections 2023-10-06T03:03:29.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd92265b700 1 -- 172.21.15.92:0/1027894362 >> 172.21.15.92:0/1027894362 conn(0x7fd91c068e20 msgr2=0x7fd91c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:29.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd92265b700 1 -- 172.21.15.92:0/1027894362 shutdown_connections 2023-10-06T03:03:29.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.862+0000 7fd92265b700 1 -- 172.21.15.92:0/1027894362 wait complete. 2023-10-06T03:03:29.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.863+0000 7fd92265b700 1 Processor -- start 2023-10-06T03:03:29.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.863+0000 7fd92265b700 1 -- start start 2023-10-06T03:03:29.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd92265b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd91c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd92265b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c158400 0x7fd91c16d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd92265b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd91c158870 0x7fd91c16d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd92265b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0ca0d0 con 0x7fd904003c20 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd92265b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0b1550 con 0x7fd91c158870 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd92265b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c155c40 con 0x7fd91c158400 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd91c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd91c0cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50164/0 (socket says 172.21.15.92:50164) 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd91bfff700 1 -- 172.21.15.92:0/675710361 learned_addr learned my addr 172.21.15.92:0/675710361 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd91b7fe700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c158400 0x7fd91c16d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.864+0000 7fd920bf8700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd91c158870 0x7fd91c16d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91bfff700 1 -- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c158400 msgr2=0x7fd91c16d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91bfff700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c158400 0x7fd91c16d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91bfff700 1 -- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd91c158870 msgr2=0x7fd91c16d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:29.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91bfff700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd91c158870 0x7fd91c16d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91bfff700 1 -- 172.21.15.92:0/675710361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0c8a60 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd920bf8700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd91c158870 0x7fd91c16d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91b7fe700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c158400 0x7fd91c16d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd91bfff700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd91c0cf860 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9040057f0 tx=0x7fd90c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c022070 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00ebf0 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c0c9b10 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c01c6a0 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.865+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c0b11e0 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.866+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c0b1830 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.866+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fd90c01ca90 con 0x7fd904003c20 2023-10-06T03:03:29.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.867+0000 7fd903fff700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd908066fc0 0x7fd908069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:29.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.867+0000 7fd91b7fe700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd908066fc0 0x7fd908069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:29.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.867+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd90c085fc0 con 0x7fd904003c20 2023-10-06T03:03:29.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.867+0000 7fd91b7fe700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd908066fc0 0x7fd908069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd910007fd0 tx=0x7fd910009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:29.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:29.871+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd90c0530e0 con 0x7fd904003c20 2023-10-06T03:03:30.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.056+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd91c0ca030 con 0x7fd904003c20 2023-10-06T03:03:30.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.057+0000 7fd903fff700 1 -- 172.21.15.92:0/675710361 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16882 (secure 0 0 0) 0x7fd90c056700 con 0x7fd904003c20 2023-10-06T03:03:30.058 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:30.059 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":65,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:29.075177+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:03:26.724124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:28.041787+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd908066fc0 msgr2=0x7fd908069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd908066fc0 0x7fd908069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd910007fd0 tx=0x7fd910009040 comp rx=0 tx=0).stop 2023-10-06T03:03:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 msgr2=0x7fd91c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd91c0cf860 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9040057f0 tx=0x7fd90c01a000 comp rx=0 tx=0).stop 2023-10-06T03:03:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 shutdown_connections 2023-10-06T03:03:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd908066fc0 0x7fd908069470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd904003c20 0x7fd91c0cf860 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd91c158870 0x7fd91c16d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 --2- 172.21.15.92:0/675710361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd91c158400 0x7fd91c16d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 >> 172.21.15.92:0/675710361 conn(0x7fd91c068e20 msgr2=0x7fd91c149f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:30.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 shutdown_connections 2023-10-06T03:03:30.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.060+0000 7fd92265b700 1 -- 172.21.15.92:0/675710361 wait complete. 2023-10-06T03:03:30.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.187+0000 7f4706805700 1 Processor -- start 2023-10-06T03:03:30.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.187+0000 7f4706805700 1 -- start start 2023-10-06T03:03:30.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f4706805700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f470006bcd0 0x7f4700140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f4706805700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47000c9510 0x7f4700141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f4706805700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470015c2a0 con 0x7f470006c0a0 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f4706805700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f470015b690 con 0x7f470006bcd0 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f4706805700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470015b970 con 0x7f47000c9510 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f46fffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f470006bcd0 0x7f4700140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f46fffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f470006bcd0 0x7f4700140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50180/0 (socket says 172.21.15.92:50180) 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.188+0000 7f46fffff700 1 -- 172.21.15.92:0/1222740710 learned_addr learned my addr 172.21.15.92:0/1222740710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f4704da2700 1 --2- 172.21.15.92:0/1222740710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47000c9510 0x7f4700141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46fffff700 1 -- 172.21.15.92:0/1222740710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47000c9510 msgr2=0x7f4700141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:30.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46fffff700 1 --2- 172.21.15.92:0/1222740710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47000c9510 0x7f4700141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46fffff700 1 -- 172.21.15.92:0/1222740710 >> v1:172.21.15.136:6789/0 conn(0x7f470006c0a0 legacy=0x7f47000c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46fffff700 1 -- 172.21.15.92:0/1222740710 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470015bc50 con 0x7f470006bcd0 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46fffff700 1 --2- 172.21.15.92:0/1222740710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006bcd0 0x7f4700140df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f46f00099b0 tx=0x7f46f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fdf11c63cb5a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46feffd700 1 -- 172.21.15.92:0/1222740710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000b860 con 0x7f470006bcd0 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46feffd700 1 -- 172.21.15.92:0/1222740710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000b9c0 con 0x7f470006bcd0 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.189+0000 7f46feffd700 1 -- 172.21.15.92:0/1222740710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000bd60 con 0x7f470006bcd0 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 -- 172.21.15.92:0/1222740710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006bcd0 msgr2=0x7f4700140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 --2- 172.21.15.92:0/1222740710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006bcd0 0x7f4700140df0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f46f00099b0 tx=0x7f46f0009250 comp rx=0 tx=0).stop 2023-10-06T03:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 -- 172.21.15.92:0/1222740710 shutdown_connections 2023-10-06T03:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 --2- 172.21.15.92:0/1222740710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006bcd0 0x7f4700140df0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 --2- 172.21.15.92:0/1222740710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47000c9510 0x7f4700141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 -- 172.21.15.92:0/1222740710 >> 172.21.15.92:0/1222740710 conn(0x7f47000ce600 msgr2=0x7f47000cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 -- 172.21.15.92:0/1222740710 shutdown_connections 2023-10-06T03:03:30.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.190+0000 7f4706805700 1 -- 172.21.15.92:0/1222740710 wait complete. 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.191+0000 7f4706805700 1 Processor -- start 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.191+0000 7f4706805700 1 -- start start 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470006bcd0 0x7f47000afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 0x7f47000c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47000c9510 0x7f47000c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000c8980 con 0x7f470006c0a0 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000b2310 con 0x7f47000c9510 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4700168d90 con 0x7f470006bcd0 2023-10-06T03:03:30.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 0x7f47000c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 0x7f47000c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50192/0 (socket says 172.21.15.92:50192) 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 -- 172.21.15.92:0/2462274250 learned_addr learned my addr 172.21.15.92:0/2462274250 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46fffff700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470006bcd0 0x7f47000afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4704da2700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47000c9510 0x7f47000c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 -- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470006bcd0 msgr2=0x7f47000afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470006bcd0 0x7f47000afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 -- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47000c9510 msgr2=0x7f47000c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47000c9510 0x7f47000c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 -- 172.21.15.92:0/2462274250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f00094f0 con 0x7f470006c0a0 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4704da2700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47000c9510 0x7f47000c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46ff7fe700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 0x7f47000c4310 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c790 tx=0x7f46f4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4011590 con 0x7f470006c0a0 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f40116f0 con 0x7f470006c0a0 2023-10-06T03:03:30.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4700167d80 con 0x7f470006c0a0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4011ab0 con 0x7f470006c0a0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.192+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47001682b0 con 0x7f470006c0a0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.193+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470004c3c0 con 0x7f470006c0a0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.194+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f46f401b3b0 con 0x7f470006c0a0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.194+0000 7f46e7fff700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46ec066fc0 0x7f46ec069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.194+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f46f4083a70 con 0x7f470006c0a0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.194+0000 7f46fffff700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46ec066fc0 0x7f46ec069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.195+0000 7f46fffff700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46ec066fc0 0x7f46ec069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46f00099b0 tx=0x7f46f0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:30.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.198+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46f4050b90 con 0x7f470006c0a0 2023-10-06T03:03:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.383+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f470006c9d0 con 0x7f470006c0a0 2023-10-06T03:03:30.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.383+0000 7f46e7fff700 1 -- 172.21.15.92:0/2462274250 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16882 (secure 0 0 0) 0x7f46f4017020 con 0x7f470006c0a0 2023-10-06T03:03:30.384 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:30.385 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":65,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:29.075177+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:03:26.724124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:28.041787+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:30.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.385+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46ec066fc0 msgr2=0x7f46ec069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.385+0000 7f4706805700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46ec066fc0 0x7f46ec069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46f00099b0 tx=0x7f46f0017000 comp rx=0 tx=0).stop 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.385+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 msgr2=0x7f47000c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.385+0000 7f4706805700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 0x7f47000c4310 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c790 tx=0x7f46f4007f40 comp rx=0 tx=0).stop 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 shutdown_connections 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46ec066fc0 0x7f46ec069470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47000c9510 0x7f47000c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470006bcd0 0x7f47000afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 --2- 172.21.15.92:0/2462274250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f470006c0a0 0x7f47000c4310 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 >> 172.21.15.92:0/2462274250 conn(0x7f47000ce600 msgr2=0x7f470013ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 shutdown_connections 2023-10-06T03:03:30.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.386+0000 7f4706805700 1 -- 172.21.15.92:0/2462274250 wait complete. 2023-10-06T03:03:30.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.515+0000 7f6c2c38d700 1 Processor -- start 2023-10-06T03:03:30.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.515+0000 7f6c2c38d700 1 -- start start 2023-10-06T03:03:30.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2c38d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c24159c60 0x7f6c2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2c38d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6c2415db10 0x7f6c24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2c38d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2415a160 con 0x7f6c24159890 2023-10-06T03:03:30.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2c38d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2415df50 con 0x7f6c24159c60 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2c38d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240d13a0 con 0x7f6c2415db10 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c29928700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6c24159c60 0x7f6c2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2a129700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c24159890 0x7f6c2415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59502/0 (socket says 172.21.15.92:59502) 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2a129700 1 -- 172.21.15.92:0/2524622087 learned_addr learned my addr 172.21.15.92:0/2524622087 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.516+0000 7f6c2a92a700 1 --2- 172.21.15.92:0/2524622087 >> v2:172.21.15.136:3300/0 conn(0x7f6c2415db10 0x7f6c24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29928700 1 -- 172.21.15.92:0/2524622087 >> v2:172.21.15.136:3300/0 conn(0x7f6c2415db10 msgr2=0x7f6c24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29928700 1 --2- 172.21.15.92:0/2524622087 >> v2:172.21.15.136:3300/0 conn(0x7f6c2415db10 0x7f6c24069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29928700 1 -- 172.21.15.92:0/2524622087 >> v1:172.21.15.92:6789/0 conn(0x7f6c24159890 legacy=0x7f6c2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29928700 1 -- 172.21.15.92:0/2524622087 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2415e140 con 0x7f6c24159c60 2023-10-06T03:03:30.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29127700 1 -- 172.21.15.92:0/2524622087 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020031078 0 0) 0x7f6c2415a160 con 0x7f6c24159890 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29928700 1 --2- 172.21.15.92:0/2524622087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c2415d5d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6c2406a890 tx=0x7f6c140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f7a4e768f888fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29127700 1 -- 172.21.15.92:0/2524622087 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400b6b0 con 0x7f6c24159c60 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29127700 1 -- 172.21.15.92:0/2524622087 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400b810 con 0x7f6c24159c60 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c29127700 1 -- 172.21.15.92:0/2524622087 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400bbb0 con 0x7f6c24159c60 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 -- 172.21.15.92:0/2524622087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 msgr2=0x7f6c2415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/2524622087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c2415d5d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6c2406a890 tx=0x7f6c140091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 -- 172.21.15.92:0/2524622087 shutdown_connections 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/2524622087 >> v2:172.21.15.136:3300/0 conn(0x7f6c2415db10 0x7f6c24069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/2524622087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c2415d5d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 -- 172.21.15.92:0/2524622087 >> 172.21.15.92:0/2524622087 conn(0x7f6c24068e20 msgr2=0x7f6c240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 -- 172.21.15.92:0/2524622087 shutdown_connections 2023-10-06T03:03:30.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.517+0000 7f6c2c38d700 1 -- 172.21.15.92:0/2524622087 wait complete. 2023-10-06T03:03:30.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 Processor -- start 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 -- start start 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c24159890 0x7f6c240b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c240c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c2415db10 0x7f6c2414c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2416d300 con 0x7f6c24159c60 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cfde0 con 0x7f6c2415db10 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2c38d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2416d550 con 0x7f6c24159890 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c29928700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c240c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c29928700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c240c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50202/0 (socket says 172.21.15.92:50202) 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c29928700 1 -- 172.21.15.92:0/153254049 learned_addr learned my addr 172.21.15.92:0/153254049 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:30.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2a92a700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c2415db10 0x7f6c2414c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.519+0000 7f6c2a129700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c24159890 0x7f6c240b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c29928700 1 -- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c24159890 msgr2=0x7f6c240b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c29928700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c24159890 0x7f6c240b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c29928700 1 -- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c2415db10 msgr2=0x7f6c2414c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c29928700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c2415db10 0x7f6c2414c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c29928700 1 -- 172.21.15.92:0/153254049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c14009420 con 0x7f6c24159c60 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c29928700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c240c6a00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6c24070e20 tx=0x7f6c14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400b980 con 0x7f6c24159c60 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400bb00 con 0x7f6c24159c60 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2416d830 con 0x7f6c24159c60 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14025670 con 0x7f6c24159c60 2023-10-06T03:03:30.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.520+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2416ddf0 con 0x7f6c24159c60 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.521+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2404c3c0 con 0x7f6c24159c60 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.521+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f6c14023070 con 0x7f6c24159c60 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.522+0000 7f6c1a7fc700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c08066fc0 0x7f6c08069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.522+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6c14085d50 con 0x7f6c24159c60 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.522+0000 7f6c2a129700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c08066fc0 0x7f6c08069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.522+0000 7f6c2a129700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c08066fc0 0x7f6c08069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c240c7450 tx=0x7f6c20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:30.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.526+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c14052e70 con 0x7f6c24159c60 2023-10-06T03:03:30.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:30.757+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6c241536c0 con 0x7f6c24159c60 2023-10-06T03:03:31.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.717+0000 7f6c1a7fc700 1 -- 172.21.15.92:0/153254049 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6c140150b0 con 0x7f6c24159c60 2023-10-06T03:03:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c08066fc0 msgr2=0x7f6c08069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:31.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c08066fc0 0x7f6c08069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c240c7450 tx=0x7f6c20009040 comp rx=0 tx=0).stop 2023-10-06T03:03:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 msgr2=0x7f6c240c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c240c6a00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6c24070e20 tx=0x7f6c14017000 comp rx=0 tx=0).stop 2023-10-06T03:03:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 shutdown_connections 2023-10-06T03:03:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c08066fc0 0x7f6c08069470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c2415db10 0x7f6c2414c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c24159890 0x7f6c240b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 --2- 172.21.15.92:0/153254049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c24159c60 0x7f6c240c6a00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 >> 172.21.15.92:0/153254049 conn(0x7f6c24068e20 msgr2=0x7f6c240cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.719+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 shutdown_connections 2023-10-06T03:03:31.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.720+0000 7f6c2c38d700 1 -- 172.21.15.92:0/153254049 wait complete. 2023-10-06T03:03:31.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:03:31.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.856+0000 7fde013ac700 1 Processor -- start 2023-10-06T03:03:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.857+0000 7fde013ac700 1 -- start start 2023-10-06T03:03:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.857+0000 7fde013ac700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fddfc159c60 0x7fddfc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.857+0000 7fde013ac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fddfc15db10 0x7fddfc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.857+0000 7fde013ac700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddfc15a160 con 0x7fddfc159890 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.857+0000 7fde013ac700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc15df50 con 0x7fddfc15db10 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.857+0000 7fde013ac700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddfc0d13a0 con 0x7fddfc159c60 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fddfc15db10 0x7fddfc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fddfc15db10 0x7fddfc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50212/0 (socket says 172.21.15.92:50212) 2023-10-06T03:03:31.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 -- 172.21.15.92:0/2569932072 learned_addr learned my addr 172.21.15.92:0/2569932072 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfa7fc700 1 --2- 172.21.15.92:0/2569932072 >> v2:172.21.15.136:3300/0 conn(0x7fddfc159c60 0x7fddfc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 -- 172.21.15.92:0/2569932072 >> v2:172.21.15.136:3300/0 conn(0x7fddfc159c60 msgr2=0x7fddfc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 --2- 172.21.15.92:0/2569932072 >> v2:172.21.15.136:3300/0 conn(0x7fddfc159c60 0x7fddfc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 -- 172.21.15.92:0/2569932072 >> v1:172.21.15.136:6789/0 conn(0x7fddfc159890 legacy=0x7fddfc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 -- 172.21.15.92:0/2569932072 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc15e140 con 0x7fddfc15db10 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddfb7fe700 1 --2- 172.21.15.92:0/2569932072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc15db10 0x7fddfc069560 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b67f0 tx=0x7fddf4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cc11d9181f8ed0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddf9ffb700 1 -- 172.21.15.92:0/2569932072 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf400e070 con 0x7fddfc15db10 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddf9ffb700 1 -- 172.21.15.92:0/2569932072 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf400ba10 con 0x7fddfc15db10 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.858+0000 7fddf9ffb700 1 -- 172.21.15.92:0/2569932072 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40092e0 con 0x7fddfc15db10 2023-10-06T03:03:31.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 -- 172.21.15.92:0/2569932072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc15db10 msgr2=0x7fddfc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 --2- 172.21.15.92:0/2569932072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc15db10 0x7fddfc069560 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b67f0 tx=0x7fddf4009840 comp rx=0 tx=0).stop 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 -- 172.21.15.92:0/2569932072 shutdown_connections 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 --2- 172.21.15.92:0/2569932072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc15db10 0x7fddfc069560 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 --2- 172.21.15.92:0/2569932072 >> v2:172.21.15.136:3300/0 conn(0x7fddfc159c60 0x7fddfc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 -- 172.21.15.92:0/2569932072 >> 172.21.15.92:0/2569932072 conn(0x7fddfc068e20 msgr2=0x7fddfc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 -- 172.21.15.92:0/2569932072 shutdown_connections 2023-10-06T03:03:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.859+0000 7fde013ac700 1 -- 172.21.15.92:0/2569932072 wait complete. 2023-10-06T03:03:31.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.860+0000 7fde013ac700 1 Processor -- start 2023-10-06T03:03:31.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.860+0000 7fde013ac700 1 -- start start 2023-10-06T03:03:31.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddfc159890 0x7fddfc14a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:31.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 0x7fddfc14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:31.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddfc15db10 0x7fddfc14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc158070 con 0x7fddfc159c60 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0c6310 con 0x7fddfc159890 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0c65f0 con 0x7fddfc15db10 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 0x7fddfc14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 0x7fddfc14ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50226/0 (socket says 172.21.15.92:50226) 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 -- 172.21.15.92:0/2085745181 learned_addr learned my addr 172.21.15.92:0/2085745181 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfaffd700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddfc159890 0x7fddfc14a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfb7fe700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddfc15db10 0x7fddfc14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 -- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddfc15db10 msgr2=0x7fddfc14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddfc15db10 0x7fddfc14b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 -- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddfc159890 msgr2=0x7fddfc14a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddfc159890 0x7fddfc14a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 -- 172.21.15.92:0/2085745181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009ae0 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfaffd700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddfc159890 0x7fddfc14a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fddfa7fc700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 0x7fddfc14ab80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d890 tx=0x7fddf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf000b600 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf000b760 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc0c68d0 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf000bae0 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.861+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc0c6e90 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.862+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddfc04c3c0 con 0x7fddfc159c60 2023-10-06T03:03:31.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.863+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fddf001e460 con 0x7fddfc159c60 2023-10-06T03:03:31.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.863+0000 7fdde2ffd700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdde8066fc0 0x7fdde8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:31.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.863+0000 7fddfaffd700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdde8066fc0 0x7fdde8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:31.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.863+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fddf0084d70 con 0x7fddfc159c60 2023-10-06T03:03:31.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.863+0000 7fddfaffd700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdde8066fc0 0x7fdde8069470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fddec007fd0 tx=0x7fddec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:31.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:31.867+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddf0051e90 con 0x7fddfc159c60 2023-10-06T03:03:32.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.092+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fddfc153490 con 0x7fddfc159c60 2023-10-06T03:03:32.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.722+0000 7fdde2ffd700 1 -- 172.21.15.92:0/2085745181 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7fddf00554b0 con 0x7fddfc159c60 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdde8066fc0 msgr2=0x7fdde8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdde8066fc0 0x7fdde8069470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fddec007fd0 tx=0x7fddec009040 comp rx=0 tx=0).stop 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 msgr2=0x7fddfc14ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 0x7fddfc14ab80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fddf000d890 tx=0x7fddf0008040 comp rx=0 tx=0).stop 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 shutdown_connections 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdde8066fc0 0x7fdde8069470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddfc15db10 0x7fddfc14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddfc159890 0x7fddfc14a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 --2- 172.21.15.92:0/2085745181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fddfc159c60 0x7fddfc14ab80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 >> 172.21.15.92:0/2085745181 conn(0x7fddfc068e20 msgr2=0x7fddfc1584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:32.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 shutdown_connections 2023-10-06T03:03:32.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.725+0000 7fde013ac700 1 -- 172.21.15.92:0/2085745181 wait complete. 2023-10-06T03:03:32.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:32.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.859+0000 7f0996051700 1 Processor -- start 2023-10-06T03:03:32.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.859+0000 7f0996051700 1 -- start start 2023-10-06T03:03:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f0996051700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0990159890 0x7f099015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f0996051700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0990159c60 0x7f099015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f0996051700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f0990069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f0996051700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099015df50 con 0x7f0990159c60 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f0996051700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099015a160 con 0x7f0990159890 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f0996051700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09900d13a0 con 0x7f099015db10 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0990159c60 0x7f099015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0990159c60 0x7f099015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50240/0 (socket says 172.21.15.92:50240) 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 -- 172.21.15.92:0/3351071479 learned_addr learned my addr 172.21.15.92:0/3351071479 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098f7fe700 1 --2- 172.21.15.92:0/3351071479 >> v2:172.21.15.136:3300/0 conn(0x7f0990159890 0x7f099015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098ffff700 1 --2- 172.21.15.92:0/3351071479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f0990069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 -- 172.21.15.92:0/3351071479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 msgr2=0x7f0990069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 --2- 172.21.15.92:0/3351071479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f0990069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 -- 172.21.15.92:0/3351071479 >> v2:172.21.15.136:3300/0 conn(0x7f0990159890 msgr2=0x7f099015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 --2- 172.21.15.92:0/3351071479 >> v2:172.21.15.136:3300/0 conn(0x7f0990159890 0x7f099015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 -- 172.21.15.92:0/3351071479 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099015e140 con 0x7f0990159c60 2023-10-06T03:03:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.860+0000 7f098effd700 1 --2- 172.21.15.92:0/3351071479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159c60 0x7f099015d5d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f09900d1090 tx=0x7f0984009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be442a0fe1455dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f098e7fc700 1 -- 172.21.15.92:0/3351071479 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098400b640 con 0x7f0990159c60 2023-10-06T03:03:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f098e7fc700 1 -- 172.21.15.92:0/3351071479 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098400b7a0 con 0x7f0990159c60 2023-10-06T03:03:32.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f098e7fc700 1 -- 172.21.15.92:0/3351071479 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098400bb60 con 0x7f0990159c60 2023-10-06T03:03:32.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 -- 172.21.15.92:0/3351071479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159c60 msgr2=0x7f099015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 --2- 172.21.15.92:0/3351071479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159c60 0x7f099015d5d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f09900d1090 tx=0x7f0984009250 comp rx=0 tx=0).stop 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 -- 172.21.15.92:0/3351071479 shutdown_connections 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 --2- 172.21.15.92:0/3351071479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f0990069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 --2- 172.21.15.92:0/3351071479 >> v2:172.21.15.136:3300/0 conn(0x7f0990159890 0x7f099015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 --2- 172.21.15.92:0/3351071479 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159c60 0x7f099015d5d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 -- 172.21.15.92:0/3351071479 >> 172.21.15.92:0/3351071479 conn(0x7f0990068e20 msgr2=0x7f09900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 -- 172.21.15.92:0/3351071479 shutdown_connections 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.861+0000 7f0996051700 1 -- 172.21.15.92:0/3351071479 wait complete. 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.862+0000 7f0996051700 1 Processor -- start 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.862+0000 7f0996051700 1 -- start start 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f0996051700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 0x7f09900b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f0996051700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0990159c60 0x7f09900b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f0996051700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f09900b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f0996051700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900b05d0 con 0x7f0990159890 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f0996051700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900b46a0 con 0x7f0990159c60 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f0996051700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09900b4980 con 0x7f099015db10 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f098f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 0x7f09900b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f098f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 0x7f09900b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50250/0 (socket says 172.21.15.92:50250) 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f098f7fe700 1 -- 172.21.15.92:0/1215390359 learned_addr learned my addr 172.21.15.92:0/1215390359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f098ffff700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f09900b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.863+0000 7f098effd700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0990159c60 0x7f09900b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f098f7fe700 1 -- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 msgr2=0x7f09900b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:32.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f098f7fe700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f09900b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f098f7fe700 1 -- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0990159c60 msgr2=0x7f09900b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f098f7fe700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0990159c60 0x7f09900b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f098f7fe700 1 -- 172.21.15.92:0/1215390359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840094f0 con 0x7f0990159890 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f098f7fe700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 0x7f09900b3fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0980009e50 tx=0x7f098000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000caa0 con 0x7f0990159890 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098000f040 con 0x7f0990159890 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09900b07e0 con 0x7f0990159890 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000ce30 con 0x7f0990159890 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.864+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09900c8560 con 0x7f0990159890 2023-10-06T03:03:32.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.865+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09900b4ba0 con 0x7f0990159890 2023-10-06T03:03:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.865+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f0980014690 con 0x7f0990159890 2023-10-06T03:03:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.866+0000 7f09777fe700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f097c066fc0 0x7f097c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.866+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0980085070 con 0x7f0990159890 2023-10-06T03:03:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.866+0000 7f098effd700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f097c066fc0 0x7f097c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:32.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.866+0000 7f098effd700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f097c066fc0 0x7f097c069470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f09900d1090 tx=0x7f0984017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:32.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:32.870+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0980052080 con 0x7f0990159890 2023-10-06T03:03:33.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.094+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f099004c3c0 con 0x7f0990159890 2023-10-06T03:03:33.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.094+0000 7f09777fe700 1 -- 172.21.15.92:0/1215390359 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f09800556a0 con 0x7f0990159890 2023-10-06T03:03:33.096 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:33.096 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:29.075556+0000","modified":"2023-10-06T03:03:32.719393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/812630670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":812630670},{"type":"v1","addr":"172.21.15.92:6837","nonce":812630670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:03:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f097c066fc0 msgr2=0x7f097c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f097c066fc0 0x7f097c069470 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f09900d1090 tx=0x7f0984017000 comp rx=0 tx=0).stop 2023-10-06T03:03:33.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 msgr2=0x7f09900b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 0x7f09900b3fa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0980009e50 tx=0x7f098000b040 comp rx=0 tx=0).stop 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 shutdown_connections 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f097c066fc0 0x7f097c069470 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0990159890 0x7f09900b3fa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f099015db10 0x7f09900b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 --2- 172.21.15.92:0/1215390359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0990159c60 0x7f09900b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 >> 172.21.15.92:0/1215390359 conn(0x7f0990068e20 msgr2=0x7f09900cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 shutdown_connections 2023-10-06T03:03:33.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.096+0000 7f0996051700 1 -- 172.21.15.92:0/1215390359 wait complete. 2023-10-06T03:03:33.100 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 38 2023-10-06T03:03:33.110 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:03:33.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.222+0000 7f8837ff5700 1 Processor -- start 2023-10-06T03:03:33.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.223+0000 7f8837ff5700 1 -- start start 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.223+0000 7f8837ff5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8830159890 0x7f883015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8837ff5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883015a160 con 0x7f8830159c60 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8837ff5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88300d13a0 con 0x7f8830069560 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8837ff5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883015e140 con 0x7f8830159890 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835d91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8830159890 0x7f883015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835590700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8830159c60 0x7f883015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59512/0 (socket says 172.21.15.92:59512) 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835590700 1 -- 172.21.15.92:0/1528563598 learned_addr learned my addr 172.21.15.92:0/1528563598 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835d91700 1 -- 172.21.15.92:0/1528563598 >> v1:172.21.15.136:6789/0 conn(0x7f8830069560 legacy=0x7f88300ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:33.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835d91700 1 -- 172.21.15.92:0/1528563598 >> v1:172.21.15.92:6789/0 conn(0x7f8830159c60 legacy=0x7f883015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835d91700 1 -- 172.21.15.92:0/1528563598 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883015a3d0 con 0x7f8830159890 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8834d8f700 1 -- 172.21.15.92:0/1528563598 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563748814 0 0) 0x7f883015a160 con 0x7f8830159c60 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8835d91700 1 --2- 172.21.15.92:0/1528563598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830159890 0x7f883015abd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f882c009e60 tx=0x7f882c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b03a76eadbc9073 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8834d8f700 1 -- 172.21.15.92:0/1528563598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00e070 con 0x7f8830159890 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8834d8f700 1 -- 172.21.15.92:0/1528563598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c00bbf0 con 0x7f8830159890 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.224+0000 7f8834d8f700 1 -- 172.21.15.92:0/1528563598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c012430 con 0x7f8830159890 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 -- 172.21.15.92:0/1528563598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830159890 msgr2=0x7f883015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 --2- 172.21.15.92:0/1528563598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830159890 0x7f883015abd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f882c009e60 tx=0x7f882c009780 comp rx=0 tx=0).stop 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 -- 172.21.15.92:0/1528563598 shutdown_connections 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 --2- 172.21.15.92:0/1528563598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830159890 0x7f883015abd0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 -- 172.21.15.92:0/1528563598 >> 172.21.15.92:0/1528563598 conn(0x7f8830068e20 msgr2=0x7f88300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 -- 172.21.15.92:0/1528563598 shutdown_connections 2023-10-06T03:03:33.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.225+0000 7f8837ff5700 1 -- 172.21.15.92:0/1528563598 wait complete. 2023-10-06T03:03:33.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.226+0000 7f8837ff5700 1 Processor -- start 2023-10-06T03:03:33.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.226+0000 7f8837ff5700 1 -- start start 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 0x7f883014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8830159890 0x7f883015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8830159c60 0x7f88300ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88300b4850 con 0x7f8830069560 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88300b4ac0 con 0x7f8830159c60 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88300b4da0 con 0x7f8830159890 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 0x7f883014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 0x7f883014d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50266/0 (socket says 172.21.15.92:50266) 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 -- 172.21.15.92:0/2983480803 learned_addr learned my addr 172.21.15.92:0/2983480803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835590700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8830159890 0x7f883015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8836592700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8830159c60 0x7f88300ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 -- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8830159890 msgr2=0x7f883015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8830159890 0x7f883015d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 -- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8830159c60 msgr2=0x7f88300ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8830159c60 0x7f88300ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 -- 172.21.15.92:0/2983480803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c009a20 con 0x7f8830069560 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835590700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8830159890 0x7f883015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8836592700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8830159c60 0x7f88300ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8835d91700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 0x7f883014d3f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b6c0 tx=0x7f882c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c012e80 con 0x7f8830069560 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c01e640 con 0x7f8830069560 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88300b5080 con 0x7f8830069560 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.227+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c01ea00 con 0x7f8830069560 2023-10-06T03:03:33.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.228+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883014d930 con 0x7f8830069560 2023-10-06T03:03:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.228+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88300b55a0 con 0x7f8830069560 2023-10-06T03:03:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.229+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f882c024070 con 0x7f8830069560 2023-10-06T03:03:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.229+0000 7f8825ffb700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8814066fc0 0x7f8814069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.229+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f882c087270 con 0x7f8830069560 2023-10-06T03:03:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.229+0000 7f8835590700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8814066fc0 0x7f8814069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.230+0000 7f8835590700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8814066fc0 0x7f8814069470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88300caa20 tx=0x7f8820009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.234+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88300b55a0 con 0x7f8830069560 2023-10-06T03:03:33.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.416+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88300cc820 con 0x7f8830069560 2023-10-06T03:03:33.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.417+0000 7f8825ffb700 1 -- 172.21.15.92:0/2983480803 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f882c054290 con 0x7f8830069560 2023-10-06T03:03:33.419 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:33.419 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:30.711121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:03:26.724124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:28.041787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:03:33.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8814066fc0 msgr2=0x7f8814069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8814066fc0 0x7f8814069470 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88300caa20 tx=0x7f8820009040 comp rx=0 tx=0).stop 2023-10-06T03:03:33.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 msgr2=0x7f883014d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 0x7f883014d3f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b6c0 tx=0x7f882c018000 comp rx=0 tx=0).stop 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 shutdown_connections 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8814066fc0 0x7f8814069470 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8830069560 0x7f883014d3f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8830159c60 0x7f88300ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 --2- 172.21.15.92:0/2983480803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8830159890 0x7f883015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 >> 172.21.15.92:0/2983480803 conn(0x7f8830068e20 msgr2=0x7f88300cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 shutdown_connections 2023-10-06T03:03:33.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.419+0000 7f8837ff5700 1 -- 172.21.15.92:0/2983480803 wait complete. 2023-10-06T03:03:33.433 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:03:33.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:33.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.545+0000 7fb82af2b700 1 Processor -- start 2023-10-06T03:03:33.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.545+0000 7fb82af2b700 1 -- start start 2023-10-06T03:03:33.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb82af2b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb824069560 0x7fb8240699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb82af2b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb824069ee0 con 0x7fb824159c60 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb82af2b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82415a160 con 0x7fb824159890 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb82af2b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8240d13a0 con 0x7fb824069560 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb8294c8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb824069560 0x7fb8240699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb81bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb824159c60 0x7fb82415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59524/0 (socket says 172.21.15.92:59524) 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb8294c8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb824069560 0x7fb8240699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50268/0 (socket says 172.21.15.92:50268) 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.546+0000 7fb81bfff700 1 -- 172.21.15.92:0/4109573467 learned_addr learned my addr 172.21.15.92:0/4109573467 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb8294c8700 1 -- 172.21.15.92:0/4109573467 >> v1:172.21.15.136:6789/0 conn(0x7fb824159890 legacy=0x7fb82415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:33.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb8294c8700 1 -- 172.21.15.92:0/4109573467 >> v1:172.21.15.92:6789/0 conn(0x7fb824159c60 legacy=0x7fb82415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb8294c8700 1 -- 172.21.15.92:0/4109573467 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82415e140 con 0x7fb824069560 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb81b7fe700 1 -- 172.21.15.92:0/4109573467 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964928810 0 0) 0x7fb824069ee0 con 0x7fb824159c60 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb8294c8700 1 --2- 172.21.15.92:0/4109573467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824069560 0x7fb8240699a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098e0 tx=0x7fb81c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e5787d4eff6cf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb81b7fe700 1 -- 172.21.15.92:0/4109573467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c009e80 con 0x7fb824069560 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb81b7fe700 1 -- 172.21.15.92:0/4109573467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00b820 con 0x7fb824069560 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb81b7fe700 1 -- 172.21.15.92:0/4109573467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00bba0 con 0x7fb824069560 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 -- 172.21.15.92:0/4109573467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824069560 msgr2=0x7fb8240699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 --2- 172.21.15.92:0/4109573467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824069560 0x7fb8240699a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0098e0 tx=0x7fb81c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 -- 172.21.15.92:0/4109573467 shutdown_connections 2023-10-06T03:03:33.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 --2- 172.21.15.92:0/4109573467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824069560 0x7fb8240699a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 -- 172.21.15.92:0/4109573467 >> 172.21.15.92:0/4109573467 conn(0x7fb824068e20 msgr2=0x7fb8240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:33.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 -- 172.21.15.92:0/4109573467 shutdown_connections 2023-10-06T03:03:33.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.547+0000 7fb82af2b700 1 -- 172.21.15.92:0/4109573467 wait complete. 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.548+0000 7fb82af2b700 1 Processor -- start 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 -- start start 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb824069560 0x7fb82416af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 0x7fb82416dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb824159c60 0x7fb824140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8240cffc0 con 0x7fb824159890 2023-10-06T03:03:33.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82416ad80 con 0x7fb824159c60 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb82af2b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82416e280 con 0x7fb824069560 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.549+0000 7fb81bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 0x7fb82416dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 0x7fb82416dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50274/0 (socket says 172.21.15.92:50274) 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 -- 172.21.15.92:0/205086331 learned_addr learned my addr 172.21.15.92:0/205086331 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb8294c8700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb824159c60 0x7fb824140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb828cc7700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb824069560 0x7fb82416af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 -- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb824069560 msgr2=0x7fb82416af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb824069560 0x7fb82416af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 -- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb824159c60 msgr2=0x7fb824140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb824159c60 0x7fb824140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 -- 172.21.15.92:0/205086331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009420 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb8294c8700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb824159c60 0x7fb824140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb81bfff700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 0x7fb82416dd40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c5b0 tx=0x7fb814001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814005b30 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb814005c90 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814007300 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82416a170 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.550+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82416a730 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.551+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82404c3c0 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.551+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb81401b3b0 con 0x7fb824159890 2023-10-06T03:03:33.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.552+0000 7fb818ff9700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb810066fc0 0x7fb810069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.552+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb814083bf0 con 0x7fb824159890 2023-10-06T03:03:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.552+0000 7fb828cc7700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb810066fc0 0x7fb810069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.552+0000 7fb828cc7700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb810066fc0 0x7fb810069470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.556+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb814050d10 con 0x7fb824159890 2023-10-06T03:03:33.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.737+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb82406a5b0 con 0x7fb810066fc0 2023-10-06T03:03:33.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.746+0000 7fb818ff9700 1 -- 172.21.15.92:0/205086331 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308773 (secure 0 0 0) 0x7fb82406a5b0 con 0x7fb810066fc0 2023-10-06T03:03:33.751 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb810066fc0 msgr2=0x7fb810069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb810066fc0 0x7fb810069470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820009040 comp rx=0 tx=0).stop 2023-10-06T03:03:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 msgr2=0x7fb82416dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 0x7fb82416dd40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb81400c5b0 tx=0x7fb814001070 comp rx=0 tx=0).stop 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 shutdown_connections 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb810066fc0 0x7fb810069470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb824159c60 0x7fb824140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb824069560 0x7fb82416af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 --2- 172.21.15.92:0/205086331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb824159890 0x7fb82416dd40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.749+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 >> 172.21.15.92:0/205086331 conn(0x7fb824068e20 msgr2=0x7fb824152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.750+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 shutdown_connections 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.750+0000 7fb82af2b700 1 -- 172.21.15.92:0/205086331 wait complete. 2023-10-06T03:03:33.755 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:33.763 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-10-06T03:03:33.204443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385508,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589212,"statfs":{"total":773094113280,"available":770651353088,"internally_reserved":0,"allocated":7376896,"data_stored":3939304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.516622"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693879+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699590+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697919+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:32.855430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691881+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692588+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:39.984353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692696+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.702535+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:10.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694123+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:00.982412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694721+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691732+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697013+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691407+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692094+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:40.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692263+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698568+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695189+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693372+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:36.017816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695286+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:04.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694599+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693997+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:27.525933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696022+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697572+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:22.851295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694020+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695339+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697401+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691119+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693754+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693530+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:45.043748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696604+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693725+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692458+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694658+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692841+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705118+0000","last_change":"2023-10-06T03:03:28.705118+0000","last_active":"2023-10-06T03:03:28.705118+0000","last_peered":"2023-10-06T03:03:28.705118+0000","last_clean":"2023-10-06T03:03:28.705118+0000","last_became_active":"2023-10-06T03:03:28.704360+0000","last_became_peered":"2023-10-06T03:03:28.704360+0000","last_unstale":"2023-10-06T03:03:28.705118+0000","last_undegraded":"2023-10-06T03:03:28.705118+0000","last_fullsized":"2023-10-06T03:03:28.705118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:51.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698103+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709814+0000","last_change":"2023-10-06T03:03:28.709814+0000","last_active":"2023-10-06T03:03:28.709814+0000","last_peered":"2023-10-06T03:03:28.709814+0000","last_clean":"2023-10-06T03:03:28.709814+0000","last_became_active":"2023-10-06T03:03:28.708830+0000","last_became_peered":"2023-10-06T03:03:28.708830+0000","last_unstale":"2023-10-06T03:03:28.709814+0000","last_undegraded":"2023-10-06T03:03:28.709814+0000","last_fullsized":"2023-10-06T03:03:28.709814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694566+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.710421+0000","last_change":"2023-10-06T03:03:28.710421+0000","last_active":"2023-10-06T03:03:28.710421+0000","last_peered":"2023-10-06T03:03:28.710421+0000","last_clean":"2023-10-06T03:03:28.710421+0000","last_became_active":"2023-10-06T03:03:28.710171+0000","last_became_peered":"2023-10-06T03:03:28.710171+0000","last_unstale":"2023-10-06T03:03:28.710421+0000","last_undegraded":"2023-10-06T03:03:28.710421+0000","last_fullsized":"2023-10-06T03:03:28.710421+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694208+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711375+0000","last_change":"2023-10-06T03:03:28.711375+0000","last_active":"2023-10-06T03:03:28.711375+0000","last_peered":"2023-10-06T03:03:28.711375+0000","last_clean":"2023-10-06T03:03:28.711375+0000","last_became_active":"2023-10-06T03:03:28.711092+0000","last_became_peered":"2023-10-06T03:03:28.711092+0000","last_unstale":"2023-10-06T03:03:28.711375+0000","last_undegraded":"2023-10-06T03:03:28.711375+0000","last_fullsized":"2023-10-06T03:03:28.711375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:32.012077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698481+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709567+0000","last_change":"2023-10-06T03:03:28.709567+0000","last_active":"2023-10-06T03:03:28.709567+0000","last_peered":"2023-10-06T03:03:28.709567+0000","last_clean":"2023-10-06T03:03:28.709567+0000","last_became_active":"2023-10-06T03:03:28.709325+0000","last_became_peered":"2023-10-06T03:03:28.709325+0000","last_unstale":"2023-10-06T03:03:28.709567+0000","last_undegraded":"2023-10-06T03:03:28.709567+0000","last_fullsized":"2023-10-06T03:03:28.709567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:41.961709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697580+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.697823+0000","last_change":"2023-10-06T03:03:28.697823+0000","last_active":"2023-10-06T03:03:28.697823+0000","last_peered":"2023-10-06T03:03:28.697823+0000","last_clean":"2023-10-06T03:03:28.697823+0000","last_became_active":"2023-10-06T03:03:28.697466+0000","last_became_peered":"2023-10-06T03:03:28.697466+0000","last_unstale":"2023-10-06T03:03:28.697823+0000","last_undegraded":"2023-10-06T03:03:28.697823+0000","last_fullsized":"2023-10-06T03:03:28.697823+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694957+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711882+0000","last_change":"2023-10-06T03:03:28.711882+0000","last_active":"2023-10-06T03:03:28.711882+0000","last_peered":"2023-10-06T03:03:28.711882+0000","last_clean":"2023-10-06T03:03:28.711882+0000","last_became_active":"2023-10-06T03:03:28.711670+0000","last_became_peered":"2023-10-06T03:03:28.711670+0000","last_unstale":"2023-10-06T03:03:28.711882+0000","last_undegraded":"2023-10-06T03:03:28.711882+0000","last_fullsized":"2023-10-06T03:03:28.711882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:43.144129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697635+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.702078+0000","last_change":"2023-10-06T03:03:28.702078+0000","last_active":"2023-10-06T03:03:28.702078+0000","last_peered":"2023-10-06T03:03:28.702078+0000","last_clean":"2023-10-06T03:03:28.702078+0000","last_became_active":"2023-10-06T03:03:28.701828+0000","last_became_peered":"2023-10-06T03:03:28.701828+0000","last_unstale":"2023-10-06T03:03:28.702078+0000","last_undegraded":"2023-10-06T03:03:28.702078+0000","last_fullsized":"2023-10-06T03:03:28.702078+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:50.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699075+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709569+0000","last_change":"2023-10-06T03:03:28.709569+0000","last_active":"2023-10-06T03:03:28.709569+0000","last_peered":"2023-10-06T03:03:28.709569+0000","last_clean":"2023-10-06T03:03:28.709569+0000","last_became_active":"2023-10-06T03:03:28.709216+0000","last_became_peered":"2023-10-06T03:03:28.709216+0000","last_unstale":"2023-10-06T03:03:28.709569+0000","last_undegraded":"2023-10-06T03:03:28.709569+0000","last_fullsized":"2023-10-06T03:03:28.709569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.690677+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704341+0000","last_change":"2023-10-06T03:03:28.704341+0000","last_active":"2023-10-06T03:03:28.704341+0000","last_peered":"2023-10-06T03:03:28.704341+0000","last_clean":"2023-10-06T03:03:28.704341+0000","last_became_active":"2023-10-06T03:03:28.704116+0000","last_became_peered":"2023-10-06T03:03:28.704116+0000","last_unstale":"2023-10-06T03:03:28.704341+0000","last_undegraded":"2023-10-06T03:03:28.704341+0000","last_fullsized":"2023-10-06T03:03:28.704341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691849+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707321+0000","last_change":"2023-10-06T03:03:28.707321+0000","last_active":"2023-10-06T03:03:28.707321+0000","last_peered":"2023-10-06T03:03:28.707321+0000","last_clean":"2023-10-06T03:03:28.707321+0000","last_became_active":"2023-10-06T03:03:28.706986+0000","last_became_peered":"2023-10-06T03:03:28.706986+0000","last_unstale":"2023-10-06T03:03:28.707321+0000","last_undegraded":"2023-10-06T03:03:28.707321+0000","last_fullsized":"2023-10-06T03:03:28.707321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.749372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694345+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.702854+0000","last_change":"2023-10-06T03:03:28.702854+0000","last_active":"2023-10-06T03:03:28.702854+0000","last_peered":"2023-10-06T03:03:28.702854+0000","last_clean":"2023-10-06T03:03:28.702854+0000","last_became_active":"2023-10-06T03:03:28.702541+0000","last_became_peered":"2023-10-06T03:03:28.702541+0000","last_unstale":"2023-10-06T03:03:28.702854+0000","last_undegraded":"2023-10-06T03:03:28.702854+0000","last_fullsized":"2023-10-06T03:03:28.702854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.690704+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.701142+0000","last_change":"2023-10-06T03:03:28.701142+0000","last_active":"2023-10-06T03:03:28.701142+0000","last_peered":"2023-10-06T03:03:28.701142+0000","last_clean":"2023-10-06T03:03:28.701142+0000","last_became_active":"2023-10-06T03:03:28.700888+0000","last_became_peered":"2023-10-06T03:03:28.700888+0000","last_unstale":"2023-10-06T03:03:28.701142+0000","last_undegraded":"2023-10-06T03:03:28.701142+0000","last_fullsized":"2023-10-06T03:03:28.701142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697568+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:48.016154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709750+0000","last_change":"2023-10-06T03:03:28.709750+0000","last_active":"2023-10-06T03:03:28.709750+0000","last_peered":"2023-10-06T03:03:28.709750+0000","last_clean":"2023-10-06T03:03:28.709750+0000","last_became_active":"2023-10-06T03:03:28.708794+0000","last_became_peered":"2023-10-06T03:03:28.708794+0000","last_unstale":"2023-10-06T03:03:28.709750+0000","last_undegraded":"2023-10-06T03:03:28.709750+0000","last_fullsized":"2023-10-06T03:03:28.709750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:31.256052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696447+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.198535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707056+0000","last_change":"2023-10-06T03:03:28.707056+0000","last_active":"2023-10-06T03:03:28.707056+0000","last_peered":"2023-10-06T03:03:28.707056+0000","last_clean":"2023-10-06T03:03:28.707056+0000","last_became_active":"2023-10-06T03:03:28.706778+0000","last_became_peered":"2023-10-06T03:03:28.706778+0000","last_unstale":"2023-10-06T03:03:28.707056+0000","last_undegraded":"2023-10-06T03:03:28.707056+0000","last_fullsized":"2023-10-06T03:03:28.707056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:18.282708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699410+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707737+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:28.707737+0000","last_peered":"2023-10-06T03:03:28.707737+0000","last_clean":"2023-10-06T03:03:28.707737+0000","last_became_active":"2023-10-06T03:03:28.707467+0000","last_became_peered":"2023-10-06T03:03:28.707467+0000","last_unstale":"2023-10-06T03:03:28.707737+0000","last_undegraded":"2023-10-06T03:03:28.707737+0000","last_fullsized":"2023-10-06T03:03:28.707737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694509+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.703808+0000","last_change":"2023-10-06T03:03:28.703808+0000","last_active":"2023-10-06T03:03:28.703808+0000","last_peered":"2023-10-06T03:03:28.703808+0000","last_clean":"2023-10-06T03:03:28.703808+0000","last_became_active":"2023-10-06T03:03:28.703078+0000","last_became_peered":"2023-10-06T03:03:28.703078+0000","last_unstale":"2023-10-06T03:03:28.703808+0000","last_undegraded":"2023-10-06T03:03:28.703808+0000","last_fullsized":"2023-10-06T03:03:28.703808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699563+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705716+0000","last_change":"2023-10-06T03:03:28.705716+0000","last_active":"2023-10-06T03:03:28.705716+0000","last_peered":"2023-10-06T03:03:28.705716+0000","last_clean":"2023-10-06T03:03:28.705716+0000","last_became_active":"2023-10-06T03:03:28.705508+0000","last_became_peered":"2023-10-06T03:03:28.705508+0000","last_unstale":"2023-10-06T03:03:28.705716+0000","last_undegraded":"2023-10-06T03:03:28.705716+0000","last_fullsized":"2023-10-06T03:03:28.705716+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:47.795438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693842+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709286+0000","last_change":"2023-10-06T03:03:28.709286+0000","last_active":"2023-10-06T03:03:28.709286+0000","last_peered":"2023-10-06T03:03:28.709286+0000","last_clean":"2023-10-06T03:03:28.709286+0000","last_became_active":"2023-10-06T03:03:28.708990+0000","last_became_peered":"2023-10-06T03:03:28.708990+0000","last_unstale":"2023-10-06T03:03:28.709286+0000","last_undegraded":"2023-10-06T03:03:28.709286+0000","last_fullsized":"2023-10-06T03:03:28.709286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696163+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709344+0000","last_change":"2023-10-06T03:03:28.709344+0000","last_active":"2023-10-06T03:03:28.709344+0000","last_peered":"2023-10-06T03:03:28.709344+0000","last_clean":"2023-10-06T03:03:28.709344+0000","last_became_active":"2023-10-06T03:03:28.709005+0000","last_became_peered":"2023-10-06T03:03:28.709005+0000","last_unstale":"2023-10-06T03:03:28.709344+0000","last_undegraded":"2023-10-06T03:03:28.709344+0000","last_fullsized":"2023-10-06T03:03:28.709344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697127+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:46.396502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711750+0000","last_change":"2023-10-06T03:03:28.711750+0000","last_active":"2023-10-06T03:03:28.711750+0000","last_peered":"2023-10-06T03:03:28.711750+0000","last_clean":"2023-10-06T03:03:28.711750+0000","last_became_active":"2023-10-06T03:03:28.711510+0000","last_became_peered":"2023-10-06T03:03:28.711510+0000","last_unstale":"2023-10-06T03:03:28.711750+0000","last_undegraded":"2023-10-06T03:03:28.711750+0000","last_fullsized":"2023-10-06T03:03:28.711750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:59.402917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697067+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:42.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.706921+0000","last_change":"2023-10-06T03:03:28.706921+0000","last_active":"2023-10-06T03:03:28.706921+0000","last_peered":"2023-10-06T03:03:28.706921+0000","last_clean":"2023-10-06T03:03:28.706921+0000","last_became_active":"2023-10-06T03:03:28.706676+0000","last_became_peered":"2023-10-06T03:03:28.706676+0000","last_unstale":"2023-10-06T03:03:28.706921+0000","last_undegraded":"2023-10-06T03:03:28.706921+0000","last_fullsized":"2023-10-06T03:03:28.706921+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.778082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694542+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:11.768132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707082+0000","last_change":"2023-10-06T03:03:28.707082+0000","last_active":"2023-10-06T03:03:28.707082+0000","last_peered":"2023-10-06T03:03:28.707082+0000","last_clean":"2023-10-06T03:03:28.707082+0000","last_became_active":"2023-10-06T03:03:28.706254+0000","last_became_peered":"2023-10-06T03:03:28.706254+0000","last_unstale":"2023-10-06T03:03:28.707082+0000","last_undegraded":"2023-10-06T03:03:28.707082+0000","last_fullsized":"2023-10-06T03:03:28.707082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:08.966317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693721+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.699446+0000","last_change":"2023-10-06T03:03:28.699446+0000","last_active":"2023-10-06T03:03:28.699446+0000","last_peered":"2023-10-06T03:03:28.699446+0000","last_clean":"2023-10-06T03:03:28.699446+0000","last_became_active":"2023-10-06T03:03:28.699197+0000","last_became_peered":"2023-10-06T03:03:28.699197+0000","last_unstale":"2023-10-06T03:03:28.699446+0000","last_undegraded":"2023-10-06T03:03:28.699446+0000","last_fullsized":"2023-10-06T03:03:28.699446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:45.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697426+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:27.714687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705759+0000","last_change":"2023-10-06T03:03:28.705759+0000","last_active":"2023-10-06T03:03:28.705759+0000","last_peered":"2023-10-06T03:03:28.705759+0000","last_clean":"2023-10-06T03:03:28.705759+0000","last_became_active":"2023-10-06T03:03:28.705443+0000","last_became_peered":"2023-10-06T03:03:28.705443+0000","last_unstale":"2023-10-06T03:03:28.705759+0000","last_undegraded":"2023-10-06T03:03:28.705759+0000","last_fullsized":"2023-10-06T03:03:28.705759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689309+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:27.689309+0000","last_peered":"2023-10-06T03:03:27.689309+0000","last_clean":"2023-10-06T03:03:27.689309+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:27.689309+0000","last_undegraded":"2023-10-06T03:03:27.689309+0000","last_fullsized":"2023-10-06T03:03:27.689309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691637+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709130+0000","last_change":"2023-10-06T03:03:28.709130+0000","last_active":"2023-10-06T03:03:28.709130+0000","last_peered":"2023-10-06T03:03:28.709130+0000","last_clean":"2023-10-06T03:03:28.709130+0000","last_became_active":"2023-10-06T03:03:28.708759+0000","last_became_peered":"2023-10-06T03:03:28.708759+0000","last_unstale":"2023-10-06T03:03:28.709130+0000","last_undegraded":"2023-10-06T03:03:28.709130+0000","last_fullsized":"2023-10-06T03:03:28.709130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.690347+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:27.690347+0000","last_peered":"2023-10-06T03:03:27.690347+0000","last_clean":"2023-10-06T03:03:27.690347+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:27.690347+0000","last_undegraded":"2023-10-06T03:03:27.690347+0000","last_fullsized":"2023-10-06T03:03:27.690347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691452+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707737+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:28.707737+0000","last_peered":"2023-10-06T03:03:28.707737+0000","last_clean":"2023-10-06T03:03:28.707737+0000","last_became_active":"2023-10-06T03:03:28.707489+0000","last_became_peered":"2023-10-06T03:03:28.707489+0000","last_unstale":"2023-10-06T03:03:28.707737+0000","last_undegraded":"2023-10-06T03:03:28.707737+0000","last_fullsized":"2023-10-06T03:03:28.707737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:41.067368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691712+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:27.691712+0000","last_peered":"2023-10-06T03:03:27.691712+0000","last_clean":"2023-10-06T03:03:27.691712+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:27.691712+0000","last_undegraded":"2023-10-06T03:03:27.691712+0000","last_fullsized":"2023-10-06T03:03:27.691712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699137+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.708268+0000","last_change":"2023-10-06T03:03:28.708268+0000","last_active":"2023-10-06T03:03:28.708268+0000","last_peered":"2023-10-06T03:03:28.708268+0000","last_clean":"2023-10-06T03:03:28.708268+0000","last_became_active":"2023-10-06T03:03:28.707997+0000","last_became_peered":"2023-10-06T03:03:28.707997+0000","last_unstale":"2023-10-06T03:03:28.708268+0000","last_undegraded":"2023-10-06T03:03:28.708268+0000","last_fullsized":"2023-10-06T03:03:28.708268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691489+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:27.691489+0000","last_peered":"2023-10-06T03:03:27.691489+0000","last_clean":"2023-10-06T03:03:27.691489+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:27.691489+0000","last_undegraded":"2023-10-06T03:03:27.691489+0000","last_fullsized":"2023-10-06T03:03:27.691489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694989+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.706968+0000","last_change":"2023-10-06T03:03:28.706968+0000","last_active":"2023-10-06T03:03:28.706968+0000","last_peered":"2023-10-06T03:03:28.706968+0000","last_clean":"2023-10-06T03:03:28.706968+0000","last_became_active":"2023-10-06T03:03:28.706720+0000","last_became_peered":"2023-10-06T03:03:28.706720+0000","last_unstale":"2023-10-06T03:03:28.706968+0000","last_undegraded":"2023-10-06T03:03:28.706968+0000","last_fullsized":"2023-10-06T03:03:28.706968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691536+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:27.691536+0000","last_peered":"2023-10-06T03:03:27.691536+0000","last_clean":"2023-10-06T03:03:27.691536+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:27.691536+0000","last_undegraded":"2023-10-06T03:03:27.691536+0000","last_fullsized":"2023-10-06T03:03:27.691536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.694163+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:27.694163+0000","last_peered":"2023-10-06T03:03:27.694163+0000","last_clean":"2023-10-06T03:03:27.694163+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:27.694163+0000","last_undegraded":"2023-10-06T03:03:27.694163+0000","last_fullsized":"2023-10-06T03:03:27.694163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698738+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707199+0000","last_change":"2023-10-06T03:03:28.707199+0000","last_active":"2023-10-06T03:03:28.707199+0000","last_peered":"2023-10-06T03:03:28.707199+0000","last_clean":"2023-10-06T03:03:28.707199+0000","last_became_active":"2023-10-06T03:03:28.706953+0000","last_became_peered":"2023-10-06T03:03:28.706953+0000","last_unstale":"2023-10-06T03:03:28.707199+0000","last_undegraded":"2023-10-06T03:03:28.707199+0000","last_fullsized":"2023-10-06T03:03:28.707199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689112+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:27.689112+0000","last_peered":"2023-10-06T03:03:27.689112+0000","last_clean":"2023-10-06T03:03:27.689112+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:27.689112+0000","last_undegraded":"2023-10-06T03:03:27.689112+0000","last_fullsized":"2023-10-06T03:03:27.689112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697722+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704304+0000","last_change":"2023-10-06T03:03:28.704304+0000","last_active":"2023-10-06T03:03:28.704304+0000","last_peered":"2023-10-06T03:03:28.704304+0000","last_clean":"2023-10-06T03:03:28.704304+0000","last_became_active":"2023-10-06T03:03:28.704042+0000","last_became_peered":"2023-10-06T03:03:28.704042+0000","last_unstale":"2023-10-06T03:03:28.704304+0000","last_undegraded":"2023-10-06T03:03:28.704304+0000","last_fullsized":"2023-10-06T03:03:28.704304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.694049+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:27.694049+0000","last_peered":"2023-10-06T03:03:27.694049+0000","last_clean":"2023-10-06T03:03:27.694049+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:27.694049+0000","last_undegraded":"2023-10-06T03:03:27.694049+0000","last_fullsized":"2023-10-06T03:03:27.694049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697103+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:35.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704994+0000","last_change":"2023-10-06T03:03:28.704994+0000","last_active":"2023-10-06T03:03:28.704994+0000","last_peered":"2023-10-06T03:03:28.704994+0000","last_clean":"2023-10-06T03:03:28.704994+0000","last_became_active":"2023-10-06T03:03:28.704702+0000","last_became_peered":"2023-10-06T03:03:28.704702+0000","last_unstale":"2023-10-06T03:03:28.704994+0000","last_undegraded":"2023-10-06T03:03:28.704994+0000","last_fullsized":"2023-10-06T03:03:28.704994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:18.494652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689696+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:27.689696+0000","last_peered":"2023-10-06T03:03:27.689696+0000","last_clean":"2023-10-06T03:03:27.689696+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:27.689696+0000","last_undegraded":"2023-10-06T03:03:27.689696+0000","last_fullsized":"2023-10-06T03:03:27.689696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073212,"statfs":{"total":96636764160,"available":96330969088,"internally_reserved":0,"allocated":1372160,"data_stored":937670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.496}]},{"osd":2,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.624},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.505},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.374}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.251},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.20200000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.626},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073212,"statfs":{"total":96636764160,"available":96330969088,"internally_reserved":0,"allocated":1372160,"data_stored":937670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.246},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.22900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.308}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.249},{"interface":"front","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.29499999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.311},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.374}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.253},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.40999999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.745}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.3869999999999996,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.495}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:33.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.917+0000 7f36e92cf700 1 Processor -- start 2023-10-06T03:03:33.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.917+0000 7f36e92cf700 1 -- start start 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e92cf700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f36e4159420 0x7f36e40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e92cf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e40b2980 con 0x7f36e41597f0 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e92cf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e40b11d0 con 0x7f36e415d230 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e92cf700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e40b14b0 con 0x7f36e4159420 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f36e41597f0 0x7f36e40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59536/0 (socket says 172.21.15.92:59536) 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e37fe700 1 -- 172.21.15.92:0/1349944897 learned_addr learned my addr 172.21.15.92:0/1349944897 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e3fff700 1 --2- 172.21.15.92:0/1349944897 >> v2:172.21.15.136:3300/0 conn(0x7f36e4159420 0x7f36e40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141448975 0 0) 0x7f36e40b2980 con 0x7f36e41597f0 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.918+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36cc003570 con 0x7f36e41597f0 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1478610646 0 0) 0x7f36cc003570 con 0x7f36e41597f0 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 >> v2:172.21.15.136:3300/0 conn(0x7f36e4159420 msgr2=0x7f36e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 --2- 172.21.15.92:0/1349944897 >> v2:172.21.15.136:3300/0 conn(0x7f36e4159420 0x7f36e40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 >> v1:172.21.15.136:6789/0 conn(0x7f36e415d230 legacy=0x7f36e415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40b1790 con 0x7f36e41597f0 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f36d8001470 con 0x7f36e41597f0 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 --2- 172.21.15.92:0/1349944897 >> v2:172.21.15.136:3300/0 conn(0x7f36e4159420 0x7f36e40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 >> v1:172.21.15.92:6789/0 conn(0x7f36e41597f0 legacy=0x7f36e40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36cc004810 0x7f36cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36cc0051c0 0x7f36cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40b14b0 con 0x7f36cc003a60 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc00c0d0 con 0x7f36cc004810 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36cc00c290 con 0x7f36cc0051c0 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e37fe700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36cc004810 0x7f36cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e8ace700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36cc0051c0 0x7f36cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 -- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36cc0051c0 msgr2=0x7f36cc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36cc0051c0 0x7f36cc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 -- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36cc004810 msgr2=0x7f36cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36cc004810 0x7f36cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 -- 172.21.15.92:0/1349944897 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc003570 con 0x7f36cc003a60 2023-10-06T03:03:33.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e8ace700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36cc0051c0 0x7f36cc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e37fe700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36cc004810 0x7f36cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e8ace700 1 -- 172.21.15.92:0/1349944897 reap_dead start 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e3fff700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36cc003e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009df0 tx=0x7f36d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de3616d7e979fa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4009740 con 0x7f36cc003a60 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d400e920 con 0x7f36cc003a60 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.919+0000 7f36e2ffd700 1 -- 172.21.15.92:0/1349944897 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d400ece0 con 0x7f36cc003a60 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 -- 172.21.15.92:0/1349944897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 msgr2=0x7f36cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36cc003e70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009df0 tx=0x7f36d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 -- 172.21.15.92:0/1349944897 shutdown_connections 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 --2- 172.21.15.92:0/1349944897 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36cc003e70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 -- 172.21.15.92:0/1349944897 >> 172.21.15.92:0/1349944897 conn(0x7f36e40d0660 msgr2=0x7f36e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:33.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 -- 172.21.15.92:0/1349944897 shutdown_connections 2023-10-06T03:03:33.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.920+0000 7f36e92cf700 1 -- 172.21.15.92:0/1349944897 wait complete. 2023-10-06T03:03:33.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.921+0000 7f36e92cf700 1 Processor -- start 2023-10-06T03:03:33.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.921+0000 7f36e92cf700 1 -- start start 2023-10-06T03:03:33.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e92cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36e40c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e92cf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36e41552c0 0x7f36e40cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e92cf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36e4158050 0x7f36e41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e92cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40c6b10 con 0x7f36cc003a60 2023-10-06T03:03:33.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e92cf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e41589e0 con 0x7f36e41552c0 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e92cf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40cc170 con 0x7f36e4158050 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36e40c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36e40c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50298/0 (socket says 172.21.15.92:50298) 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 -- 172.21.15.92:0/3385314650 learned_addr learned my addr 172.21.15.92:0/3385314650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e8ace700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36e4158050 0x7f36e41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e37fe700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36e41552c0 0x7f36e40cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 -- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36e4158050 msgr2=0x7f36e41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36e4158050 0x7f36e41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 -- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36e41552c0 msgr2=0x7f36e40cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36e41552c0 0x7f36e40cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:33.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 -- 172.21.15.92:0/3385314650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40cc450 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e8ace700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36e4158050 0x7f36e41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e37fe700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36e41552c0 0x7f36e40cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.922+0000 7f36e3fff700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36e40c95d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00c0a0 tx=0x7f36d401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.923+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d4022070 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.923+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d4013be0 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.923+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d401c450 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.923+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e40c64c0 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.923+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e40cbe50 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.924+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e404c3c0 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.924+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f36d4026070 con 0x7f36cc003a60 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.924+0000 7f36cbfff700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36d0066fc0 0x7f36d0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:33.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.924+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f36d4085fe0 con 0x7f36cc003a60 2023-10-06T03:03:33.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.924+0000 7f36e37fe700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36d0066fc0 0x7f36d0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:33.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.925+0000 7f36e37fe700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36d0066fc0 0x7f36d0069470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000e10 tx=0x7f36d8001d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:33.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:33.928+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36d4053040 con 0x7f36cc003a60 2023-10-06T03:03:33.993 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T03:03:33.992+0000 7fbcc1e84700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:03:34.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.108+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36e40b1a70 con 0x7f36d0066fc0 2023-10-06T03:03:34.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.116+0000 7f36cbfff700 1 -- 172.21.15.92:0/3385314650 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308773 (secure 0 0 0) 0x7f36e40b1a70 con 0x7f36d0066fc0 2023-10-06T03:03:34.121 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36d0066fc0 msgr2=0x7f36d0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36d0066fc0 0x7f36d0069470 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000e10 tx=0x7f36d8001d90 comp rx=0 tx=0).stop 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 msgr2=0x7f36e40c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36e40c95d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00c0a0 tx=0x7f36d401a000 comp rx=0 tx=0).stop 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 shutdown_connections 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f36d0066fc0 0x7f36d0069470 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36cc003a60 0x7f36e40c95d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36e4158050 0x7f36e41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 --2- 172.21.15.92:0/3385314650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36e41552c0 0x7f36e40cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 >> 172.21.15.92:0/3385314650 conn(0x7f36e40d0660 msgr2=0x7f36e4156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 shutdown_connections 2023-10-06T03:03:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.120+0000 7f36e92cf700 1 -- 172.21.15.92:0/3385314650 wait complete. 2023-10-06T03:03:34.125 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:34.133 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-10-06T03:03:33.204443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385508,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589212,"statfs":{"total":773094113280,"available":770651353088,"internally_reserved":0,"allocated":7376896,"data_stored":3939304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.516622"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693879+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699590+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697919+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:32.855430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691881+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692588+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:39.984353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692696+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.702535+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:10.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694123+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:00.982412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694721+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691732+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697013+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691407+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692094+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:40.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692263+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698568+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695189+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693372+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:36.017816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695286+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:04.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694599+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693997+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:27.525933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696022+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697572+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:22.851295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694020+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695339+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697401+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691119+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693754+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693530+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:45.043748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696604+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693725+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692458+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694658+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692841+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705118+0000","last_change":"2023-10-06T03:03:28.705118+0000","last_active":"2023-10-06T03:03:28.705118+0000","last_peered":"2023-10-06T03:03:28.705118+0000","last_clean":"2023-10-06T03:03:28.705118+0000","last_became_active":"2023-10-06T03:03:28.704360+0000","last_became_peered":"2023-10-06T03:03:28.704360+0000","last_unstale":"2023-10-06T03:03:28.705118+0000","last_undegraded":"2023-10-06T03:03:28.705118+0000","last_fullsized":"2023-10-06T03:03:28.705118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:51.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698103+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709814+0000","last_change":"2023-10-06T03:03:28.709814+0000","last_active":"2023-10-06T03:03:28.709814+0000","last_peered":"2023-10-06T03:03:28.709814+0000","last_clean":"2023-10-06T03:03:28.709814+0000","last_became_active":"2023-10-06T03:03:28.708830+0000","last_became_peered":"2023-10-06T03:03:28.708830+0000","last_unstale":"2023-10-06T03:03:28.709814+0000","last_undegraded":"2023-10-06T03:03:28.709814+0000","last_fullsized":"2023-10-06T03:03:28.709814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694566+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.710421+0000","last_change":"2023-10-06T03:03:28.710421+0000","last_active":"2023-10-06T03:03:28.710421+0000","last_peered":"2023-10-06T03:03:28.710421+0000","last_clean":"2023-10-06T03:03:28.710421+0000","last_became_active":"2023-10-06T03:03:28.710171+0000","last_became_peered":"2023-10-06T03:03:28.710171+0000","last_unstale":"2023-10-06T03:03:28.710421+0000","last_undegraded":"2023-10-06T03:03:28.710421+0000","last_fullsized":"2023-10-06T03:03:28.710421+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694208+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711375+0000","last_change":"2023-10-06T03:03:28.711375+0000","last_active":"2023-10-06T03:03:28.711375+0000","last_peered":"2023-10-06T03:03:28.711375+0000","last_clean":"2023-10-06T03:03:28.711375+0000","last_became_active":"2023-10-06T03:03:28.711092+0000","last_became_peered":"2023-10-06T03:03:28.711092+0000","last_unstale":"2023-10-06T03:03:28.711375+0000","last_undegraded":"2023-10-06T03:03:28.711375+0000","last_fullsized":"2023-10-06T03:03:28.711375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:32.012077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698481+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709567+0000","last_change":"2023-10-06T03:03:28.709567+0000","last_active":"2023-10-06T03:03:28.709567+0000","last_peered":"2023-10-06T03:03:28.709567+0000","last_clean":"2023-10-06T03:03:28.709567+0000","last_became_active":"2023-10-06T03:03:28.709325+0000","last_became_peered":"2023-10-06T03:03:28.709325+0000","last_unstale":"2023-10-06T03:03:28.709567+0000","last_undegraded":"2023-10-06T03:03:28.709567+0000","last_fullsized":"2023-10-06T03:03:28.709567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:41.961709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697580+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.697823+0000","last_change":"2023-10-06T03:03:28.697823+0000","last_active":"2023-10-06T03:03:28.697823+0000","last_peered":"2023-10-06T03:03:28.697823+0000","last_clean":"2023-10-06T03:03:28.697823+0000","last_became_active":"2023-10-06T03:03:28.697466+0000","last_became_peered":"2023-10-06T03:03:28.697466+0000","last_unstale":"2023-10-06T03:03:28.697823+0000","last_undegraded":"2023-10-06T03:03:28.697823+0000","last_fullsized":"2023-10-06T03:03:28.697823+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694957+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711882+0000","last_change":"2023-10-06T03:03:28.711882+0000","last_active":"2023-10-06T03:03:28.711882+0000","last_peered":"2023-10-06T03:03:28.711882+0000","last_clean":"2023-10-06T03:03:28.711882+0000","last_became_active":"2023-10-06T03:03:28.711670+0000","last_became_peered":"2023-10-06T03:03:28.711670+0000","last_unstale":"2023-10-06T03:03:28.711882+0000","last_undegraded":"2023-10-06T03:03:28.711882+0000","last_fullsized":"2023-10-06T03:03:28.711882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:43.144129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697635+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.702078+0000","last_change":"2023-10-06T03:03:28.702078+0000","last_active":"2023-10-06T03:03:28.702078+0000","last_peered":"2023-10-06T03:03:28.702078+0000","last_clean":"2023-10-06T03:03:28.702078+0000","last_became_active":"2023-10-06T03:03:28.701828+0000","last_became_peered":"2023-10-06T03:03:28.701828+0000","last_unstale":"2023-10-06T03:03:28.702078+0000","last_undegraded":"2023-10-06T03:03:28.702078+0000","last_fullsized":"2023-10-06T03:03:28.702078+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:50.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699075+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709569+0000","last_change":"2023-10-06T03:03:28.709569+0000","last_active":"2023-10-06T03:03:28.709569+0000","last_peered":"2023-10-06T03:03:28.709569+0000","last_clean":"2023-10-06T03:03:28.709569+0000","last_became_active":"2023-10-06T03:03:28.709216+0000","last_became_peered":"2023-10-06T03:03:28.709216+0000","last_unstale":"2023-10-06T03:03:28.709569+0000","last_undegraded":"2023-10-06T03:03:28.709569+0000","last_fullsized":"2023-10-06T03:03:28.709569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.690677+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704341+0000","last_change":"2023-10-06T03:03:28.704341+0000","last_active":"2023-10-06T03:03:28.704341+0000","last_peered":"2023-10-06T03:03:28.704341+0000","last_clean":"2023-10-06T03:03:28.704341+0000","last_became_active":"2023-10-06T03:03:28.704116+0000","last_became_peered":"2023-10-06T03:03:28.704116+0000","last_unstale":"2023-10-06T03:03:28.704341+0000","last_undegraded":"2023-10-06T03:03:28.704341+0000","last_fullsized":"2023-10-06T03:03:28.704341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691849+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707321+0000","last_change":"2023-10-06T03:03:28.707321+0000","last_active":"2023-10-06T03:03:28.707321+0000","last_peered":"2023-10-06T03:03:28.707321+0000","last_clean":"2023-10-06T03:03:28.707321+0000","last_became_active":"2023-10-06T03:03:28.706986+0000","last_became_peered":"2023-10-06T03:03:28.706986+0000","last_unstale":"2023-10-06T03:03:28.707321+0000","last_undegraded":"2023-10-06T03:03:28.707321+0000","last_fullsized":"2023-10-06T03:03:28.707321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.749372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694345+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.702854+0000","last_change":"2023-10-06T03:03:28.702854+0000","last_active":"2023-10-06T03:03:28.702854+0000","last_peered":"2023-10-06T03:03:28.702854+0000","last_clean":"2023-10-06T03:03:28.702854+0000","last_became_active":"2023-10-06T03:03:28.702541+0000","last_became_peered":"2023-10-06T03:03:28.702541+0000","last_unstale":"2023-10-06T03:03:28.702854+0000","last_undegraded":"2023-10-06T03:03:28.702854+0000","last_fullsized":"2023-10-06T03:03:28.702854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.690704+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.701142+0000","last_change":"2023-10-06T03:03:28.701142+0000","last_active":"2023-10-06T03:03:28.701142+0000","last_peered":"2023-10-06T03:03:28.701142+0000","last_clean":"2023-10-06T03:03:28.701142+0000","last_became_active":"2023-10-06T03:03:28.700888+0000","last_became_peered":"2023-10-06T03:03:28.700888+0000","last_unstale":"2023-10-06T03:03:28.701142+0000","last_undegraded":"2023-10-06T03:03:28.701142+0000","last_fullsized":"2023-10-06T03:03:28.701142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697568+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:48.016154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709750+0000","last_change":"2023-10-06T03:03:28.709750+0000","last_active":"2023-10-06T03:03:28.709750+0000","last_peered":"2023-10-06T03:03:28.709750+0000","last_clean":"2023-10-06T03:03:28.709750+0000","last_became_active":"2023-10-06T03:03:28.708794+0000","last_became_peered":"2023-10-06T03:03:28.708794+0000","last_unstale":"2023-10-06T03:03:28.709750+0000","last_undegraded":"2023-10-06T03:03:28.709750+0000","last_fullsized":"2023-10-06T03:03:28.709750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:31.256052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696447+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.198535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707056+0000","last_change":"2023-10-06T03:03:28.707056+0000","last_active":"2023-10-06T03:03:28.707056+0000","last_peered":"2023-10-06T03:03:28.707056+0000","last_clean":"2023-10-06T03:03:28.707056+0000","last_became_active":"2023-10-06T03:03:28.706778+0000","last_became_peered":"2023-10-06T03:03:28.706778+0000","last_unstale":"2023-10-06T03:03:28.707056+0000","last_undegraded":"2023-10-06T03:03:28.707056+0000","last_fullsized":"2023-10-06T03:03:28.707056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:18.282708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699410+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707737+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:28.707737+0000","last_peered":"2023-10-06T03:03:28.707737+0000","last_clean":"2023-10-06T03:03:28.707737+0000","last_became_active":"2023-10-06T03:03:28.707467+0000","last_became_peered":"2023-10-06T03:03:28.707467+0000","last_unstale":"2023-10-06T03:03:28.707737+0000","last_undegraded":"2023-10-06T03:03:28.707737+0000","last_fullsized":"2023-10-06T03:03:28.707737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694509+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.703808+0000","last_change":"2023-10-06T03:03:28.703808+0000","last_active":"2023-10-06T03:03:28.703808+0000","last_peered":"2023-10-06T03:03:28.703808+0000","last_clean":"2023-10-06T03:03:28.703808+0000","last_became_active":"2023-10-06T03:03:28.703078+0000","last_became_peered":"2023-10-06T03:03:28.703078+0000","last_unstale":"2023-10-06T03:03:28.703808+0000","last_undegraded":"2023-10-06T03:03:28.703808+0000","last_fullsized":"2023-10-06T03:03:28.703808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699563+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705716+0000","last_change":"2023-10-06T03:03:28.705716+0000","last_active":"2023-10-06T03:03:28.705716+0000","last_peered":"2023-10-06T03:03:28.705716+0000","last_clean":"2023-10-06T03:03:28.705716+0000","last_became_active":"2023-10-06T03:03:28.705508+0000","last_became_peered":"2023-10-06T03:03:28.705508+0000","last_unstale":"2023-10-06T03:03:28.705716+0000","last_undegraded":"2023-10-06T03:03:28.705716+0000","last_fullsized":"2023-10-06T03:03:28.705716+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:47.795438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693842+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709286+0000","last_change":"2023-10-06T03:03:28.709286+0000","last_active":"2023-10-06T03:03:28.709286+0000","last_peered":"2023-10-06T03:03:28.709286+0000","last_clean":"2023-10-06T03:03:28.709286+0000","last_became_active":"2023-10-06T03:03:28.708990+0000","last_became_peered":"2023-10-06T03:03:28.708990+0000","last_unstale":"2023-10-06T03:03:28.709286+0000","last_undegraded":"2023-10-06T03:03:28.709286+0000","last_fullsized":"2023-10-06T03:03:28.709286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696163+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709344+0000","last_change":"2023-10-06T03:03:28.709344+0000","last_active":"2023-10-06T03:03:28.709344+0000","last_peered":"2023-10-06T03:03:28.709344+0000","last_clean":"2023-10-06T03:03:28.709344+0000","last_became_active":"2023-10-06T03:03:28.709005+0000","last_became_peered":"2023-10-06T03:03:28.709005+0000","last_unstale":"2023-10-06T03:03:28.709344+0000","last_undegraded":"2023-10-06T03:03:28.709344+0000","last_fullsized":"2023-10-06T03:03:28.709344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697127+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:46.396502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711750+0000","last_change":"2023-10-06T03:03:28.711750+0000","last_active":"2023-10-06T03:03:28.711750+0000","last_peered":"2023-10-06T03:03:28.711750+0000","last_clean":"2023-10-06T03:03:28.711750+0000","last_became_active":"2023-10-06T03:03:28.711510+0000","last_became_peered":"2023-10-06T03:03:28.711510+0000","last_unstale":"2023-10-06T03:03:28.711750+0000","last_undegraded":"2023-10-06T03:03:28.711750+0000","last_fullsized":"2023-10-06T03:03:28.711750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:59.402917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697067+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:42.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.706921+0000","last_change":"2023-10-06T03:03:28.706921+0000","last_active":"2023-10-06T03:03:28.706921+0000","last_peered":"2023-10-06T03:03:28.706921+0000","last_clean":"2023-10-06T03:03:28.706921+0000","last_became_active":"2023-10-06T03:03:28.706676+0000","last_became_peered":"2023-10-06T03:03:28.706676+0000","last_unstale":"2023-10-06T03:03:28.706921+0000","last_undegraded":"2023-10-06T03:03:28.706921+0000","last_fullsized":"2023-10-06T03:03:28.706921+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.778082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694542+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:11.768132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707082+0000","last_change":"2023-10-06T03:03:28.707082+0000","last_active":"2023-10-06T03:03:28.707082+0000","last_peered":"2023-10-06T03:03:28.707082+0000","last_clean":"2023-10-06T03:03:28.707082+0000","last_became_active":"2023-10-06T03:03:28.706254+0000","last_became_peered":"2023-10-06T03:03:28.706254+0000","last_unstale":"2023-10-06T03:03:28.707082+0000","last_undegraded":"2023-10-06T03:03:28.707082+0000","last_fullsized":"2023-10-06T03:03:28.707082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:08.966317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693721+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.699446+0000","last_change":"2023-10-06T03:03:28.699446+0000","last_active":"2023-10-06T03:03:28.699446+0000","last_peered":"2023-10-06T03:03:28.699446+0000","last_clean":"2023-10-06T03:03:28.699446+0000","last_became_active":"2023-10-06T03:03:28.699197+0000","last_became_peered":"2023-10-06T03:03:28.699197+0000","last_unstale":"2023-10-06T03:03:28.699446+0000","last_undegraded":"2023-10-06T03:03:28.699446+0000","last_fullsized":"2023-10-06T03:03:28.699446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:45.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697426+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:27.714687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705759+0000","last_change":"2023-10-06T03:03:28.705759+0000","last_active":"2023-10-06T03:03:28.705759+0000","last_peered":"2023-10-06T03:03:28.705759+0000","last_clean":"2023-10-06T03:03:28.705759+0000","last_became_active":"2023-10-06T03:03:28.705443+0000","last_became_peered":"2023-10-06T03:03:28.705443+0000","last_unstale":"2023-10-06T03:03:28.705759+0000","last_undegraded":"2023-10-06T03:03:28.705759+0000","last_fullsized":"2023-10-06T03:03:28.705759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689309+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:27.689309+0000","last_peered":"2023-10-06T03:03:27.689309+0000","last_clean":"2023-10-06T03:03:27.689309+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:27.689309+0000","last_undegraded":"2023-10-06T03:03:27.689309+0000","last_fullsized":"2023-10-06T03:03:27.689309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691637+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709130+0000","last_change":"2023-10-06T03:03:28.709130+0000","last_active":"2023-10-06T03:03:28.709130+0000","last_peered":"2023-10-06T03:03:28.709130+0000","last_clean":"2023-10-06T03:03:28.709130+0000","last_became_active":"2023-10-06T03:03:28.708759+0000","last_became_peered":"2023-10-06T03:03:28.708759+0000","last_unstale":"2023-10-06T03:03:28.709130+0000","last_undegraded":"2023-10-06T03:03:28.709130+0000","last_fullsized":"2023-10-06T03:03:28.709130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.690347+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:27.690347+0000","last_peered":"2023-10-06T03:03:27.690347+0000","last_clean":"2023-10-06T03:03:27.690347+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:27.690347+0000","last_undegraded":"2023-10-06T03:03:27.690347+0000","last_fullsized":"2023-10-06T03:03:27.690347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691452+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707737+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:28.707737+0000","last_peered":"2023-10-06T03:03:28.707737+0000","last_clean":"2023-10-06T03:03:28.707737+0000","last_became_active":"2023-10-06T03:03:28.707489+0000","last_became_peered":"2023-10-06T03:03:28.707489+0000","last_unstale":"2023-10-06T03:03:28.707737+0000","last_undegraded":"2023-10-06T03:03:28.707737+0000","last_fullsized":"2023-10-06T03:03:28.707737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:41.067368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691712+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:27.691712+0000","last_peered":"2023-10-06T03:03:27.691712+0000","last_clean":"2023-10-06T03:03:27.691712+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:27.691712+0000","last_undegraded":"2023-10-06T03:03:27.691712+0000","last_fullsized":"2023-10-06T03:03:27.691712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699137+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.708268+0000","last_change":"2023-10-06T03:03:28.708268+0000","last_active":"2023-10-06T03:03:28.708268+0000","last_peered":"2023-10-06T03:03:28.708268+0000","last_clean":"2023-10-06T03:03:28.708268+0000","last_became_active":"2023-10-06T03:03:28.707997+0000","last_became_peered":"2023-10-06T03:03:28.707997+0000","last_unstale":"2023-10-06T03:03:28.708268+0000","last_undegraded":"2023-10-06T03:03:28.708268+0000","last_fullsized":"2023-10-06T03:03:28.708268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691489+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:27.691489+0000","last_peered":"2023-10-06T03:03:27.691489+0000","last_clean":"2023-10-06T03:03:27.691489+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:27.691489+0000","last_undegraded":"2023-10-06T03:03:27.691489+0000","last_fullsized":"2023-10-06T03:03:27.691489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694989+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.706968+0000","last_change":"2023-10-06T03:03:28.706968+0000","last_active":"2023-10-06T03:03:28.706968+0000","last_peered":"2023-10-06T03:03:28.706968+0000","last_clean":"2023-10-06T03:03:28.706968+0000","last_became_active":"2023-10-06T03:03:28.706720+0000","last_became_peered":"2023-10-06T03:03:28.706720+0000","last_unstale":"2023-10-06T03:03:28.706968+0000","last_undegraded":"2023-10-06T03:03:28.706968+0000","last_fullsized":"2023-10-06T03:03:28.706968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691536+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:27.691536+0000","last_peered":"2023-10-06T03:03:27.691536+0000","last_clean":"2023-10-06T03:03:27.691536+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:27.691536+0000","last_undegraded":"2023-10-06T03:03:27.691536+0000","last_fullsized":"2023-10-06T03:03:27.691536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.694163+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:27.694163+0000","last_peered":"2023-10-06T03:03:27.694163+0000","last_clean":"2023-10-06T03:03:27.694163+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:27.694163+0000","last_undegraded":"2023-10-06T03:03:27.694163+0000","last_fullsized":"2023-10-06T03:03:27.694163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698738+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707199+0000","last_change":"2023-10-06T03:03:28.707199+0000","last_active":"2023-10-06T03:03:28.707199+0000","last_peered":"2023-10-06T03:03:28.707199+0000","last_clean":"2023-10-06T03:03:28.707199+0000","last_became_active":"2023-10-06T03:03:28.706953+0000","last_became_peered":"2023-10-06T03:03:28.706953+0000","last_unstale":"2023-10-06T03:03:28.707199+0000","last_undegraded":"2023-10-06T03:03:28.707199+0000","last_fullsized":"2023-10-06T03:03:28.707199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689112+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:27.689112+0000","last_peered":"2023-10-06T03:03:27.689112+0000","last_clean":"2023-10-06T03:03:27.689112+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:27.689112+0000","last_undegraded":"2023-10-06T03:03:27.689112+0000","last_fullsized":"2023-10-06T03:03:27.689112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697722+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704304+0000","last_change":"2023-10-06T03:03:28.704304+0000","last_active":"2023-10-06T03:03:28.704304+0000","last_peered":"2023-10-06T03:03:28.704304+0000","last_clean":"2023-10-06T03:03:28.704304+0000","last_became_active":"2023-10-06T03:03:28.704042+0000","last_became_peered":"2023-10-06T03:03:28.704042+0000","last_unstale":"2023-10-06T03:03:28.704304+0000","last_undegraded":"2023-10-06T03:03:28.704304+0000","last_fullsized":"2023-10-06T03:03:28.704304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.694049+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:27.694049+0000","last_peered":"2023-10-06T03:03:27.694049+0000","last_clean":"2023-10-06T03:03:27.694049+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:27.694049+0000","last_undegraded":"2023-10-06T03:03:27.694049+0000","last_fullsized":"2023-10-06T03:03:27.694049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697103+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:35.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704994+0000","last_change":"2023-10-06T03:03:28.704994+0000","last_active":"2023-10-06T03:03:28.704994+0000","last_peered":"2023-10-06T03:03:28.704994+0000","last_clean":"2023-10-06T03:03:28.704994+0000","last_became_active":"2023-10-06T03:03:28.704702+0000","last_became_peered":"2023-10-06T03:03:28.704702+0000","last_unstale":"2023-10-06T03:03:28.704994+0000","last_undegraded":"2023-10-06T03:03:28.704994+0000","last_fullsized":"2023-10-06T03:03:28.704994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:18.494652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689696+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:27.689696+0000","last_peered":"2023-10-06T03:03:27.689696+0000","last_clean":"2023-10-06T03:03:27.689696+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:27.689696+0000","last_undegraded":"2023-10-06T03:03:27.689696+0000","last_fullsized":"2023-10-06T03:03:27.689696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073212,"statfs":{"total":96636764160,"available":96330969088,"internally_reserved":0,"allocated":1372160,"data_stored":937670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.496}]},{"osd":2,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.624},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.505},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.374}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.251},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.20200000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.626},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073212,"statfs":{"total":96636764160,"available":96330969088,"internally_reserved":0,"allocated":1372160,"data_stored":937670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.246},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.22900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.308}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.249},{"interface":"front","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.29499999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.311},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.374}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.253},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.40999999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.745}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.3869999999999996,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.495}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:34.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:34.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.288+0000 7f5232e6b700 1 Processor -- start 2023-10-06T03:03:34.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.288+0000 7f5232e6b700 1 -- start start 2023-10-06T03:03:34.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5232e6b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c159db0 0x7f522c069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5232e6b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f522c06a2b0 0x7f522c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5232e6b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522c0c96a0 con 0x7f522c1599e0 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5232e6b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c0cb9e0 con 0x7f522c06a2b0 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5232e6b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0cab60 con 0x7f522c159db0 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5231408700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f522c06a2b0 0x7f522c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5231408700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f522c06a2b0 0x7f522c06a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50310/0 (socket says 172.21.15.92:50310) 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5231408700 1 -- 172.21.15.92:0/2162570215 learned_addr learned my addr 172.21.15.92:0/2162570215 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.289+0000 7f5223fff700 1 --2- 172.21.15.92:0/2162570215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c159db0 0x7f522c069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5231408700 1 -- 172.21.15.92:0/2162570215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c159db0 msgr2=0x7f522c069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5231408700 1 --2- 172.21.15.92:0/2162570215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c159db0 0x7f522c069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5231408700 1 -- 172.21.15.92:0/2162570215 >> v1:172.21.15.136:6789/0 conn(0x7f522c1599e0 legacy=0x7f522c069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5231408700 1 -- 172.21.15.92:0/2162570215 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0cb420 con 0x7f522c06a2b0 2023-10-06T03:03:34.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5231408700 1 --2- 172.21.15.92:0/2162570215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c06a2b0 0x7f522c06a710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5224009f20 tx=0x7f5224009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc012d02b262fe07 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f52237fe700 1 -- 172.21.15.92:0/2162570215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400e070 con 0x7f522c06a2b0 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f52237fe700 1 -- 172.21.15.92:0/2162570215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522400b9d0 con 0x7f522c06a2b0 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f52237fe700 1 -- 172.21.15.92:0/2162570215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240092e0 con 0x7f522c06a2b0 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 -- 172.21.15.92:0/2162570215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c06a2b0 msgr2=0x7f522c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 --2- 172.21.15.92:0/2162570215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c06a2b0 0x7f522c06a710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5224009f20 tx=0x7f5224009840 comp rx=0 tx=0).stop 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 -- 172.21.15.92:0/2162570215 shutdown_connections 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 --2- 172.21.15.92:0/2162570215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c06a2b0 0x7f522c06a710 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 --2- 172.21.15.92:0/2162570215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c159db0 0x7f522c069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 -- 172.21.15.92:0/2162570215 >> 172.21.15.92:0/2162570215 conn(0x7f522c0c9860 msgr2=0x7f522c0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.290+0000 7f5232e6b700 1 -- 172.21.15.92:0/2162570215 shutdown_connections 2023-10-06T03:03:34.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.291+0000 7f5232e6b700 1 -- 172.21.15.92:0/2162570215 wait complete. 2023-10-06T03:03:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 Processor -- start 2023-10-06T03:03:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 -- start start 2023-10-06T03:03:34.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c06a2b0 0x7f522c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 0x7f522c0c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f522c159db0 0x7f522c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0c7160 con 0x7f522c1599e0 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c0c6e80 con 0x7f522c159db0 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.292+0000 7f5232e6b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c15b290 con 0x7f522c06a2b0 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 0x7f522c0c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 0x7f522c0c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50314/0 (socket says 172.21.15.92:50314) 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 -- 172.21.15.92:0/1161413598 learned_addr learned my addr 172.21.15.92:0/1161413598 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5230c07700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c06a2b0 0x7f522c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5231408700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f522c159db0 0x7f522c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 -- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c06a2b0 msgr2=0x7f522c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c06a2b0 0x7f522c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 -- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f522c159db0 msgr2=0x7f522c15bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f522c159db0 0x7f522c15bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 -- 172.21.15.92:0/1161413598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224009ae0 con 0x7f522c1599e0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5231408700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f522c159db0 0x7f522c15bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5223fff700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 0x7f522c0c6510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f521c00e930 tx=0x7f521c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00eed0 con 0x7f522c1599e0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c00c850 con 0x7f522c1599e0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c15b570 con 0x7f522c1599e0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c00cbf0 con 0x7f522c1599e0 2023-10-06T03:03:34.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.293+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c155bb0 con 0x7f522c1599e0 2023-10-06T03:03:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.294+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c15ba10 con 0x7f522c1599e0 2023-10-06T03:03:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.295+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f521c013470 con 0x7f522c1599e0 2023-10-06T03:03:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.295+0000 7f5220ff9700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5218066fc0 0x7f5218069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.295+0000 7f5230c07700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5218066fc0 0x7f5218069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.295+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f521c0848a0 con 0x7f522c1599e0 2023-10-06T03:03:34.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.296+0000 7f5230c07700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5218066fc0 0x7f5218069470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f522c15cfe0 tx=0x7f5228009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:34.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.299+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f521c0519c0 con 0x7f522c1599e0 2023-10-06T03:03:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.477+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f522c04c3c0 con 0x7f5218066fc0 2023-10-06T03:03:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.486+0000 7f5220ff9700 1 -- 172.21.15.92:0/1161413598 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308773 (secure 0 0 0) 0x7f522c04c3c0 con 0x7f5218066fc0 2023-10-06T03:03:34.490 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5218066fc0 msgr2=0x7f5218069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5218066fc0 0x7f5218069470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f522c15cfe0 tx=0x7f5228009040 comp rx=0 tx=0).stop 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 msgr2=0x7f522c0c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 0x7f522c0c6510 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f521c00e930 tx=0x7f521c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 shutdown_connections 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5218066fc0 0x7f5218069470 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f522c159db0 0x7f522c15bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f522c06a2b0 0x7f522c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 --2- 172.21.15.92:0/1161413598 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f522c1599e0 0x7f522c0c6510 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 >> 172.21.15.92:0/1161413598 conn(0x7f522c0c9860 msgr2=0x7f522c157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:34.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 shutdown_connections 2023-10-06T03:03:34.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.489+0000 7f5232e6b700 1 -- 172.21.15.92:0/1161413598 wait complete. 2023-10-06T03:03:34.494 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:34.502 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-10-06T03:03:33.204443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385508,"kb_used_data":7204,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752589212,"statfs":{"total":773094113280,"available":770651353088,"internally_reserved":0,"allocated":7376896,"data_stored":3939304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.516622"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693879+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699590+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697919+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:32.855430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691881+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692588+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:39.984353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692696+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.702535+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:10.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694123+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:09:00.982412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694721+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691732+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697013+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691407+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692094+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:40.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692263+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698568+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695189+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693372+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:36.017816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695286+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:04.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694599+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693997+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:27.525933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696022+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697572+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:22.851295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694020+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.695339+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:36.288303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697401+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691119+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693754+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693530+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:45.043748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696604+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693725+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692458+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694658+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.692841+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705118+0000","last_change":"2023-10-06T03:03:28.705118+0000","last_active":"2023-10-06T03:03:28.705118+0000","last_peered":"2023-10-06T03:03:28.705118+0000","last_clean":"2023-10-06T03:03:28.705118+0000","last_became_active":"2023-10-06T03:03:28.704360+0000","last_became_peered":"2023-10-06T03:03:28.704360+0000","last_unstale":"2023-10-06T03:03:28.705118+0000","last_undegraded":"2023-10-06T03:03:28.705118+0000","last_fullsized":"2023-10-06T03:03:28.705118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:51.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698103+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709814+0000","last_change":"2023-10-06T03:03:28.709814+0000","last_active":"2023-10-06T03:03:28.709814+0000","last_peered":"2023-10-06T03:03:28.709814+0000","last_clean":"2023-10-06T03:03:28.709814+0000","last_became_active":"2023-10-06T03:03:28.708830+0000","last_became_peered":"2023-10-06T03:03:28.708830+0000","last_unstale":"2023-10-06T03:03:28.709814+0000","last_undegraded":"2023-10-06T03:03:28.709814+0000","last_fullsized":"2023-10-06T03:03:28.709814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694566+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.710421+0000","last_change":"2023-10-06T03:03:28.710421+0000","last_active":"2023-10-06T03:03:28.710421+0000","last_peered":"2023-10-06T03:03:28.710421+0000","last_clean":"2023-10-06T03:03:28.710421+0000","last_became_active":"2023-10-06T03:03:28.710171+0000","last_became_peered":"2023-10-06T03:03:28.710171+0000","last_unstale":"2023-10-06T03:03:28.710421+0000","last_undegraded":"2023-10-06T03:03:28.710421+0000","last_fullsized":"2023-10-06T03:03:28.710421+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694208+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711375+0000","last_change":"2023-10-06T03:03:28.711375+0000","last_active":"2023-10-06T03:03:28.711375+0000","last_peered":"2023-10-06T03:03:28.711375+0000","last_clean":"2023-10-06T03:03:28.711375+0000","last_became_active":"2023-10-06T03:03:28.711092+0000","last_became_peered":"2023-10-06T03:03:28.711092+0000","last_unstale":"2023-10-06T03:03:28.711375+0000","last_undegraded":"2023-10-06T03:03:28.711375+0000","last_fullsized":"2023-10-06T03:03:28.711375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:32.012077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698481+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709567+0000","last_change":"2023-10-06T03:03:28.709567+0000","last_active":"2023-10-06T03:03:28.709567+0000","last_peered":"2023-10-06T03:03:28.709567+0000","last_clean":"2023-10-06T03:03:28.709567+0000","last_became_active":"2023-10-06T03:03:28.709325+0000","last_became_peered":"2023-10-06T03:03:28.709325+0000","last_unstale":"2023-10-06T03:03:28.709567+0000","last_undegraded":"2023-10-06T03:03:28.709567+0000","last_fullsized":"2023-10-06T03:03:28.709567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:41.961709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697580+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.697823+0000","last_change":"2023-10-06T03:03:28.697823+0000","last_active":"2023-10-06T03:03:28.697823+0000","last_peered":"2023-10-06T03:03:28.697823+0000","last_clean":"2023-10-06T03:03:28.697823+0000","last_became_active":"2023-10-06T03:03:28.697466+0000","last_became_peered":"2023-10-06T03:03:28.697466+0000","last_unstale":"2023-10-06T03:03:28.697823+0000","last_undegraded":"2023-10-06T03:03:28.697823+0000","last_fullsized":"2023-10-06T03:03:28.697823+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694957+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711882+0000","last_change":"2023-10-06T03:03:28.711882+0000","last_active":"2023-10-06T03:03:28.711882+0000","last_peered":"2023-10-06T03:03:28.711882+0000","last_clean":"2023-10-06T03:03:28.711882+0000","last_became_active":"2023-10-06T03:03:28.711670+0000","last_became_peered":"2023-10-06T03:03:28.711670+0000","last_unstale":"2023-10-06T03:03:28.711882+0000","last_undegraded":"2023-10-06T03:03:28.711882+0000","last_fullsized":"2023-10-06T03:03:28.711882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:43.144129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697635+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.702078+0000","last_change":"2023-10-06T03:03:28.702078+0000","last_active":"2023-10-06T03:03:28.702078+0000","last_peered":"2023-10-06T03:03:28.702078+0000","last_clean":"2023-10-06T03:03:28.702078+0000","last_became_active":"2023-10-06T03:03:28.701828+0000","last_became_peered":"2023-10-06T03:03:28.701828+0000","last_unstale":"2023-10-06T03:03:28.702078+0000","last_undegraded":"2023-10-06T03:03:28.702078+0000","last_fullsized":"2023-10-06T03:03:28.702078+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:50.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699075+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709569+0000","last_change":"2023-10-06T03:03:28.709569+0000","last_active":"2023-10-06T03:03:28.709569+0000","last_peered":"2023-10-06T03:03:28.709569+0000","last_clean":"2023-10-06T03:03:28.709569+0000","last_became_active":"2023-10-06T03:03:28.709216+0000","last_became_peered":"2023-10-06T03:03:28.709216+0000","last_unstale":"2023-10-06T03:03:28.709569+0000","last_undegraded":"2023-10-06T03:03:28.709569+0000","last_fullsized":"2023-10-06T03:03:28.709569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.690677+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704341+0000","last_change":"2023-10-06T03:03:28.704341+0000","last_active":"2023-10-06T03:03:28.704341+0000","last_peered":"2023-10-06T03:03:28.704341+0000","last_clean":"2023-10-06T03:03:28.704341+0000","last_became_active":"2023-10-06T03:03:28.704116+0000","last_became_peered":"2023-10-06T03:03:28.704116+0000","last_unstale":"2023-10-06T03:03:28.704341+0000","last_undegraded":"2023-10-06T03:03:28.704341+0000","last_fullsized":"2023-10-06T03:03:28.704341+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691849+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707321+0000","last_change":"2023-10-06T03:03:28.707321+0000","last_active":"2023-10-06T03:03:28.707321+0000","last_peered":"2023-10-06T03:03:28.707321+0000","last_clean":"2023-10-06T03:03:28.707321+0000","last_became_active":"2023-10-06T03:03:28.706986+0000","last_became_peered":"2023-10-06T03:03:28.706986+0000","last_unstale":"2023-10-06T03:03:28.707321+0000","last_undegraded":"2023-10-06T03:03:28.707321+0000","last_fullsized":"2023-10-06T03:03:28.707321+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.749372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694345+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.702854+0000","last_change":"2023-10-06T03:03:28.702854+0000","last_active":"2023-10-06T03:03:28.702854+0000","last_peered":"2023-10-06T03:03:28.702854+0000","last_clean":"2023-10-06T03:03:28.702854+0000","last_became_active":"2023-10-06T03:03:28.702541+0000","last_became_peered":"2023-10-06T03:03:28.702541+0000","last_unstale":"2023-10-06T03:03:28.702854+0000","last_undegraded":"2023-10-06T03:03:28.702854+0000","last_fullsized":"2023-10-06T03:03:28.702854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.690704+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.701142+0000","last_change":"2023-10-06T03:03:28.701142+0000","last_active":"2023-10-06T03:03:28.701142+0000","last_peered":"2023-10-06T03:03:28.701142+0000","last_clean":"2023-10-06T03:03:28.701142+0000","last_became_active":"2023-10-06T03:03:28.700888+0000","last_became_peered":"2023-10-06T03:03:28.700888+0000","last_unstale":"2023-10-06T03:03:28.701142+0000","last_undegraded":"2023-10-06T03:03:28.701142+0000","last_fullsized":"2023-10-06T03:03:28.701142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697568+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:48.016154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709750+0000","last_change":"2023-10-06T03:03:28.709750+0000","last_active":"2023-10-06T03:03:28.709750+0000","last_peered":"2023-10-06T03:03:28.709750+0000","last_clean":"2023-10-06T03:03:28.709750+0000","last_became_active":"2023-10-06T03:03:28.708794+0000","last_became_peered":"2023-10-06T03:03:28.708794+0000","last_unstale":"2023-10-06T03:03:28.709750+0000","last_undegraded":"2023-10-06T03:03:28.709750+0000","last_fullsized":"2023-10-06T03:03:28.709750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:31.256052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696447+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:57.198535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707056+0000","last_change":"2023-10-06T03:03:28.707056+0000","last_active":"2023-10-06T03:03:28.707056+0000","last_peered":"2023-10-06T03:03:28.707056+0000","last_clean":"2023-10-06T03:03:28.707056+0000","last_became_active":"2023-10-06T03:03:28.706778+0000","last_became_peered":"2023-10-06T03:03:28.706778+0000","last_unstale":"2023-10-06T03:03:28.707056+0000","last_undegraded":"2023-10-06T03:03:28.707056+0000","last_fullsized":"2023-10-06T03:03:28.707056+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:18.282708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699410+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707737+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:28.707737+0000","last_peered":"2023-10-06T03:03:28.707737+0000","last_clean":"2023-10-06T03:03:28.707737+0000","last_became_active":"2023-10-06T03:03:28.707467+0000","last_became_peered":"2023-10-06T03:03:28.707467+0000","last_unstale":"2023-10-06T03:03:28.707737+0000","last_undegraded":"2023-10-06T03:03:28.707737+0000","last_fullsized":"2023-10-06T03:03:28.707737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694509+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.703808+0000","last_change":"2023-10-06T03:03:28.703808+0000","last_active":"2023-10-06T03:03:28.703808+0000","last_peered":"2023-10-06T03:03:28.703808+0000","last_clean":"2023-10-06T03:03:28.703808+0000","last_became_active":"2023-10-06T03:03:28.703078+0000","last_became_peered":"2023-10-06T03:03:28.703078+0000","last_unstale":"2023-10-06T03:03:28.703808+0000","last_undegraded":"2023-10-06T03:03:28.703808+0000","last_fullsized":"2023-10-06T03:03:28.703808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699563+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705716+0000","last_change":"2023-10-06T03:03:28.705716+0000","last_active":"2023-10-06T03:03:28.705716+0000","last_peered":"2023-10-06T03:03:28.705716+0000","last_clean":"2023-10-06T03:03:28.705716+0000","last_became_active":"2023-10-06T03:03:28.705508+0000","last_became_peered":"2023-10-06T03:03:28.705508+0000","last_unstale":"2023-10-06T03:03:28.705716+0000","last_undegraded":"2023-10-06T03:03:28.705716+0000","last_fullsized":"2023-10-06T03:03:28.705716+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:47.795438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693842+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709286+0000","last_change":"2023-10-06T03:03:28.709286+0000","last_active":"2023-10-06T03:03:28.709286+0000","last_peered":"2023-10-06T03:03:28.709286+0000","last_clean":"2023-10-06T03:03:28.709286+0000","last_became_active":"2023-10-06T03:03:28.708990+0000","last_became_peered":"2023-10-06T03:03:28.708990+0000","last_unstale":"2023-10-06T03:03:28.709286+0000","last_undegraded":"2023-10-06T03:03:28.709286+0000","last_fullsized":"2023-10-06T03:03:28.709286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.696163+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709344+0000","last_change":"2023-10-06T03:03:28.709344+0000","last_active":"2023-10-06T03:03:28.709344+0000","last_peered":"2023-10-06T03:03:28.709344+0000","last_clean":"2023-10-06T03:03:28.709344+0000","last_became_active":"2023-10-06T03:03:28.709005+0000","last_became_peered":"2023-10-06T03:03:28.709005+0000","last_unstale":"2023-10-06T03:03:28.709344+0000","last_undegraded":"2023-10-06T03:03:28.709344+0000","last_fullsized":"2023-10-06T03:03:28.709344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697127+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:46.396502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.711750+0000","last_change":"2023-10-06T03:03:28.711750+0000","last_active":"2023-10-06T03:03:28.711750+0000","last_peered":"2023-10-06T03:03:28.711750+0000","last_clean":"2023-10-06T03:03:28.711750+0000","last_became_active":"2023-10-06T03:03:28.711510+0000","last_became_peered":"2023-10-06T03:03:28.711510+0000","last_unstale":"2023-10-06T03:03:28.711750+0000","last_undegraded":"2023-10-06T03:03:28.711750+0000","last_fullsized":"2023-10-06T03:03:28.711750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:59.402917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697067+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:42.075360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.706921+0000","last_change":"2023-10-06T03:03:28.706921+0000","last_active":"2023-10-06T03:03:28.706921+0000","last_peered":"2023-10-06T03:03:28.706921+0000","last_clean":"2023-10-06T03:03:28.706921+0000","last_became_active":"2023-10-06T03:03:28.706676+0000","last_became_peered":"2023-10-06T03:03:28.706676+0000","last_unstale":"2023-10-06T03:03:28.706921+0000","last_undegraded":"2023-10-06T03:03:28.706921+0000","last_fullsized":"2023-10-06T03:03:28.706921+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.778082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694542+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:11.768132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707082+0000","last_change":"2023-10-06T03:03:28.707082+0000","last_active":"2023-10-06T03:03:28.707082+0000","last_peered":"2023-10-06T03:03:28.707082+0000","last_clean":"2023-10-06T03:03:28.707082+0000","last_became_active":"2023-10-06T03:03:28.706254+0000","last_became_peered":"2023-10-06T03:03:28.706254+0000","last_unstale":"2023-10-06T03:03:28.707082+0000","last_undegraded":"2023-10-06T03:03:28.707082+0000","last_fullsized":"2023-10-06T03:03:28.707082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:08.966317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.693721+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.699446+0000","last_change":"2023-10-06T03:03:28.699446+0000","last_active":"2023-10-06T03:03:28.699446+0000","last_peered":"2023-10-06T03:03:28.699446+0000","last_clean":"2023-10-06T03:03:28.699446+0000","last_became_active":"2023-10-06T03:03:28.699197+0000","last_became_peered":"2023-10-06T03:03:28.699197+0000","last_unstale":"2023-10-06T03:03:28.699446+0000","last_undegraded":"2023-10-06T03:03:28.699446+0000","last_fullsized":"2023-10-06T03:03:28.699446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:45.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697426+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:27.714687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.705759+0000","last_change":"2023-10-06T03:03:28.705759+0000","last_active":"2023-10-06T03:03:28.705759+0000","last_peered":"2023-10-06T03:03:28.705759+0000","last_clean":"2023-10-06T03:03:28.705759+0000","last_became_active":"2023-10-06T03:03:28.705443+0000","last_became_peered":"2023-10-06T03:03:28.705443+0000","last_unstale":"2023-10-06T03:03:28.705759+0000","last_undegraded":"2023-10-06T03:03:28.705759+0000","last_fullsized":"2023-10-06T03:03:28.705759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689309+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:27.689309+0000","last_peered":"2023-10-06T03:03:27.689309+0000","last_clean":"2023-10-06T03:03:27.689309+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:27.689309+0000","last_undegraded":"2023-10-06T03:03:27.689309+0000","last_fullsized":"2023-10-06T03:03:27.689309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691637+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.709130+0000","last_change":"2023-10-06T03:03:28.709130+0000","last_active":"2023-10-06T03:03:28.709130+0000","last_peered":"2023-10-06T03:03:28.709130+0000","last_clean":"2023-10-06T03:03:28.709130+0000","last_became_active":"2023-10-06T03:03:28.708759+0000","last_became_peered":"2023-10-06T03:03:28.708759+0000","last_unstale":"2023-10-06T03:03:28.709130+0000","last_undegraded":"2023-10-06T03:03:28.709130+0000","last_fullsized":"2023-10-06T03:03:28.709130+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.690347+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:27.690347+0000","last_peered":"2023-10-06T03:03:27.690347+0000","last_clean":"2023-10-06T03:03:27.690347+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:27.690347+0000","last_undegraded":"2023-10-06T03:03:27.690347+0000","last_fullsized":"2023-10-06T03:03:27.690347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.691452+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707737+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:28.707737+0000","last_peered":"2023-10-06T03:03:28.707737+0000","last_clean":"2023-10-06T03:03:28.707737+0000","last_became_active":"2023-10-06T03:03:28.707489+0000","last_became_peered":"2023-10-06T03:03:28.707489+0000","last_unstale":"2023-10-06T03:03:28.707737+0000","last_undegraded":"2023-10-06T03:03:28.707737+0000","last_fullsized":"2023-10-06T03:03:28.707737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:41.067368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691712+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:27.691712+0000","last_peered":"2023-10-06T03:03:27.691712+0000","last_clean":"2023-10-06T03:03:27.691712+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:27.691712+0000","last_undegraded":"2023-10-06T03:03:27.691712+0000","last_fullsized":"2023-10-06T03:03:27.691712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.699137+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.708268+0000","last_change":"2023-10-06T03:03:28.708268+0000","last_active":"2023-10-06T03:03:28.708268+0000","last_peered":"2023-10-06T03:03:28.708268+0000","last_clean":"2023-10-06T03:03:28.708268+0000","last_became_active":"2023-10-06T03:03:28.707997+0000","last_became_peered":"2023-10-06T03:03:28.707997+0000","last_unstale":"2023-10-06T03:03:28.708268+0000","last_undegraded":"2023-10-06T03:03:28.708268+0000","last_fullsized":"2023-10-06T03:03:28.708268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691489+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:27.691489+0000","last_peered":"2023-10-06T03:03:27.691489+0000","last_clean":"2023-10-06T03:03:27.691489+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:27.691489+0000","last_undegraded":"2023-10-06T03:03:27.691489+0000","last_fullsized":"2023-10-06T03:03:27.691489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.694989+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.706968+0000","last_change":"2023-10-06T03:03:28.706968+0000","last_active":"2023-10-06T03:03:28.706968+0000","last_peered":"2023-10-06T03:03:28.706968+0000","last_clean":"2023-10-06T03:03:28.706968+0000","last_became_active":"2023-10-06T03:03:28.706720+0000","last_became_peered":"2023-10-06T03:03:28.706720+0000","last_unstale":"2023-10-06T03:03:28.706968+0000","last_undegraded":"2023-10-06T03:03:28.706968+0000","last_fullsized":"2023-10-06T03:03:28.706968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.691536+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:27.691536+0000","last_peered":"2023-10-06T03:03:27.691536+0000","last_clean":"2023-10-06T03:03:27.691536+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:27.691536+0000","last_undegraded":"2023-10-06T03:03:27.691536+0000","last_fullsized":"2023-10-06T03:03:27.691536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.694163+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:27.694163+0000","last_peered":"2023-10-06T03:03:27.694163+0000","last_clean":"2023-10-06T03:03:27.694163+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:27.694163+0000","last_undegraded":"2023-10-06T03:03:27.694163+0000","last_fullsized":"2023-10-06T03:03:27.694163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.698738+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.707199+0000","last_change":"2023-10-06T03:03:28.707199+0000","last_active":"2023-10-06T03:03:28.707199+0000","last_peered":"2023-10-06T03:03:28.707199+0000","last_clean":"2023-10-06T03:03:28.707199+0000","last_became_active":"2023-10-06T03:03:28.706953+0000","last_became_peered":"2023-10-06T03:03:28.706953+0000","last_unstale":"2023-10-06T03:03:28.707199+0000","last_undegraded":"2023-10-06T03:03:28.707199+0000","last_fullsized":"2023-10-06T03:03:28.707199+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689112+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:27.689112+0000","last_peered":"2023-10-06T03:03:27.689112+0000","last_clean":"2023-10-06T03:03:27.689112+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:27.689112+0000","last_undegraded":"2023-10-06T03:03:27.689112+0000","last_fullsized":"2023-10-06T03:03:27.689112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697722+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704304+0000","last_change":"2023-10-06T03:03:28.704304+0000","last_active":"2023-10-06T03:03:28.704304+0000","last_peered":"2023-10-06T03:03:28.704304+0000","last_clean":"2023-10-06T03:03:28.704304+0000","last_became_active":"2023-10-06T03:03:28.704042+0000","last_became_peered":"2023-10-06T03:03:28.704042+0000","last_unstale":"2023-10-06T03:03:28.704304+0000","last_undegraded":"2023-10-06T03:03:28.704304+0000","last_fullsized":"2023-10-06T03:03:28.704304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.694049+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:27.694049+0000","last_peered":"2023-10-06T03:03:27.694049+0000","last_clean":"2023-10-06T03:03:27.694049+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:27.694049+0000","last_undegraded":"2023-10-06T03:03:27.694049+0000","last_fullsized":"2023-10-06T03:03:27.694049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:03:28.685044+0000","last_change":"2023-10-06T03:03:28.697103+0000","last_active":"2023-10-06T03:03:28.685044+0000","last_peered":"2023-10-06T03:03:28.685044+0000","last_clean":"2023-10-06T03:03:28.685044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:03:28.685044+0000","last_undegraded":"2023-10-06T03:03:28.685044+0000","last_fullsized":"2023-10-06T03:03:28.685044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:35.564522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-10-06T03:03:28.704994+0000","last_change":"2023-10-06T03:03:28.704994+0000","last_active":"2023-10-06T03:03:28.704994+0000","last_peered":"2023-10-06T03:03:28.704994+0000","last_clean":"2023-10-06T03:03:28.704994+0000","last_became_active":"2023-10-06T03:03:28.704702+0000","last_became_peered":"2023-10-06T03:03:28.704702+0000","last_unstale":"2023-10-06T03:03:28.704994+0000","last_undegraded":"2023-10-06T03:03:28.704994+0000","last_fullsized":"2023-10-06T03:03:28.704994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:18.494652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":49,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:03:27.689696+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:27.689696+0000","last_peered":"2023-10-06T03:03:27.689696+0000","last_clean":"2023-10-06T03:03:27.689696+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:27.689696+0000","last_undegraded":"2023-10-06T03:03:27.689696+0000","last_fullsized":"2023-10-06T03:03:27.689696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073212,"statfs":{"total":96636764160,"available":96330969088,"internally_reserved":0,"allocated":1372160,"data_stored":937670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.496}]},{"osd":2,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.624},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.505},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.374}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.251},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.20200000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.30099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.626},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298628,"kb_used_data":1340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073212,"statfs":{"total":96636764160,"available":96330969088,"internally_reserved":0,"allocated":1372160,"data_stored":937670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.246},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.22900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.308}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.249},{"interface":"front","average":{"1min":0.311,"5min":0.311,"15min":0.311},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.29499999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.311},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.374}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"last":0.253},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.313,"15min":0.313},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.30399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.627},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:02:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":340686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3,"5min":3,"15min":3},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":10.084,"5min":10.084,"15min":10.084},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.65300000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.40999999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.745}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298048,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073792,"statfs":{"total":96636764160,"available":96331563008,"internally_reserved":0,"allocated":778240,"data_stored":347302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":5.3869999999999996,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":5.4630000000000001,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.495}]},{"osd":7,"last update":"Fri Oct 6 03:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:34.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693879+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.699590+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697919+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:22:32.855430+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691881+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T03:03:34.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.692588+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:34:39.984353+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.692696+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.702535+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:56:10.609021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694123+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:09:00.982412+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2023-10-06T03:03:34.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694721+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691732+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697013+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691407+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.692094+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:52:40.111000+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.692263+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.698568+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.695189+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693372+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:05:36.017816+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.695286+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:00:04.975316+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694599+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693997+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:17:27.525933+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2023-10-06T03:03:34.514 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.696022+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697572+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:22:22.851295+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694020+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.695339+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:40:36.288303+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2023-10-06T03:03:34.515 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697401+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:20:37.178591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691119+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693754+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693530+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:29:45.043748+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2023-10-06T03:03:34.516 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.696604+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T09:23:47.802160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693725+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.692458+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694658+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.692841+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2023-10-06T03:03:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.698103+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694566+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694208+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T10:06:59.890516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.698481+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:50:55.761960+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697580+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.518 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694957+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:26:40.088277+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697635+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:07:05.070770+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.699075+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.690677+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2023-10-06T03:03:34.519 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691849+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694345+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.690704+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697568+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:34:48.016154+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.696447+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:08:57.198535+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2023-10-06T03:03:34.520 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.699410+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694509+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T08:23:43.178741+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.699563+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T12:56:16.496726+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693842+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.696163+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T05:47:32.264045+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.521 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697127+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:20:46.396502+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697067+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T11:49:42.075360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694542+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:42:11.768132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.693721+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T07:57:57.642545+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2023-10-06T03:03:34.522 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697426+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:15:27.714687+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691637+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.691452+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.699137+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.694989+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-06T03:03:34.523 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2023-10-06T03:03:34.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.698738+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T04:11:00.468822+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-06T03:03:34.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2023-10-06T03:03:34.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697722+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T14:00:59.278665+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-06T03:03:34.524 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2023-10-06T03:03:34.524 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-06T03:03:28.685044+0000', 'last_change': '2023-10-06T03:03:28.697103+0000', 'last_active': '2023-10-06T03:03:28.685044+0000', 'last_peered': '2023-10-06T03:03:28.685044+0000', 'last_clean': '2023-10-06T03:03:28.685044+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T03:03:28.685044+0000', 'last_undegraded': '2023-10-06T03:03:28.685044+0000', 'last_fullsized': '2023-10-06T03:03:28.685044+0000', 'mapping_epoch': 64, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 64, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'last_clean_scrub_stamp': '2023-10-06T03:03:28.685044+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-07T13:51:35.564522+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-06T03:03:34.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T03:03:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.654+0000 7f262e3f5700 1 Processor -- start 2023-10-06T03:03:34.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.654+0000 7f262e3f5700 1 -- start start 2023-10-06T03:03:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.655+0000 7f262e3f5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 0x7f262815abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.655+0000 7f262e3f5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262815b130 con 0x7f2628154320 2023-10-06T03:03:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.655+0000 7f262e3f5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262815db50 con 0x7f26280b21f0 2023-10-06T03:03:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.655+0000 7f262e3f5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262815de30 con 0x7f2628153f50 2023-10-06T03:03:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.655+0000 7f26277fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2628154320 0x7f26280b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59544/0 (socket says 172.21.15.92:59544) 2023-10-06T03:03:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.655+0000 7f26277fe700 1 -- 172.21.15.92:0/38777369 learned_addr learned my addr 172.21.15.92:0/38777369 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2627fff700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 0x7f262815abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813180686 0 0) 0x7f261c001110 con 0x7f2628154320 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2610003570 con 0x7f2628154320 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324865420 0 0) 0x7f262815db50 con 0x7f26280b21f0 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261c001110 con 0x7f26280b21f0 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2838026040 0 0) 0x7f2610003570 con 0x7f2628154320 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 msgr2=0x7f262815abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 0x7f262815abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 >> v1:172.21.15.136:6789/0 conn(0x7f26280b21f0 legacy=0x7f2628069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262815e110 con 0x7f2628154320 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f261c001600 con 0x7f2628154320 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 >> v1:172.21.15.92:6789/0 conn(0x7f2628154320 legacy=0x7f26280b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610003c20 0x7f2610004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f2610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 0x7f26100052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262815de30 con 0x7f2610004810 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261000bc40 con 0x7f2610003c20 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261000be00 con 0x7f2628153f50 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f26277fe700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f2610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f2627fff700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610003c20 0x7f2610004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f262c992700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 0x7f26100052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f26277fe700 1 -- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 msgr2=0x7f26100052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f26277fe700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2628153f50 0x7f26100052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f26277fe700 1 -- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610003c20 msgr2=0x7f2610004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f26277fe700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2610003c20 0x7f2610004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.656+0000 7f26277fe700 1 -- 172.21.15.92:0/38777369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610003570 con 0x7f2610004810 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f26277fe700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f2610004c80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c3c0 tx=0x7f262815e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150cfd253b4bb496 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00cc30 con 0x7f2610004810 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00cdb0 con 0x7f2610004810 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f2626ffd700 1 -- 172.21.15.92:0/38777369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c005b70 con 0x7f2610004810 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262e3f5700 1 -- 172.21.15.92:0/38777369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 msgr2=0x7f2610004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262e3f5700 1 --2- 172.21.15.92:0/38777369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f2610004c80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f261c00c3c0 tx=0x7f262815e6d0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262c992700 1 -- 172.21.15.92:0/38777369 reap_dead start 2023-10-06T03:03:34.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262e3f5700 1 -- 172.21.15.92:0/38777369 shutdown_connections 2023-10-06T03:03:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262e3f5700 1 -- 172.21.15.92:0/38777369 >> 172.21.15.92:0/38777369 conn(0x7f262806baf0 msgr2=0x7f262806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262e3f5700 1 -- 172.21.15.92:0/38777369 shutdown_connections 2023-10-06T03:03:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.657+0000 7f262e3f5700 1 -- 172.21.15.92:0/38777369 wait complete. 2023-10-06T03:03:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.658+0000 7f262e3f5700 1 Processor -- start 2023-10-06T03:03:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.658+0000 7f262e3f5700 1 -- start start 2023-10-06T03:03:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262e3f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f262816afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262e3f5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628159950 0x7f2628154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262e3f5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26280cdbe0 0x7f2628155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262e3f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2628152e90 con 0x7f2610004810 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262e3f5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280ce180 con 0x7f2628159950 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262e3f5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280ce460 con 0x7f26280cdbe0 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f2627fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f262816afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f2627fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f262816afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50326/0 (socket says 172.21.15.92:50326) 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f2627fff700 1 -- 172.21.15.92:0/1329494423 learned_addr learned my addr 172.21.15.92:0/1329494423 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f262c992700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26280cdbe0 0x7f2628155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.659+0000 7f26277fe700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628159950 0x7f2628154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f2627fff700 1 -- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26280cdbe0 msgr2=0x7f2628155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f2627fff700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26280cdbe0 0x7f2628155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f2627fff700 1 -- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628159950 msgr2=0x7f2628154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f2627fff700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628159950 0x7f2628154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f2627fff700 1 -- 172.21.15.92:0/1329494423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280ce740 con 0x7f2610004810 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f2627fff700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f262816afe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2618011d40 tx=0x7f26180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261800fa10 con 0x7f2610004810 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261800fb70 con 0x7f2610004810 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26280cea20 con 0x7f2610004810 2023-10-06T03:03:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618014440 con 0x7f2610004810 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.660+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262816b520 con 0x7f2610004810 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.661+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262804c3c0 con 0x7f2610004810 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.662+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2618027480 con 0x7f2610004810 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.662+0000 7f260ffff700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2614066fc0 0x7f2614069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.662+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2618018030 con 0x7f2610004810 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.662+0000 7f26277fe700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2614066fc0 0x7f2614069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.663+0000 7f26277fe700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2614066fc0 0x7f2614069470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f261c007d40 tx=0x7f261c015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.666+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26180525f0 con 0x7f2610004810 2023-10-06T03:03:34.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.924+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f26280cd9c0 con 0x7f2610004810 2023-10-06T03:03:34.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.924+0000 7f260ffff700 1 -- 172.21.15.92:0/1329494423 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1456 (secure 0 0 0) 0x7f26180225c0 con 0x7f2610004810 2023-10-06T03:03:34.926 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:34.926 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1696561288,"num_in_osds":8,"osd_in_since":1696561280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":41}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":2442760192,"bytes_avail":770651353088,"bytes_total":773094113280,"inactive_pgs_ratio":0.60952383279800415},"fsmap":{"epoch":38,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":5733}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-06T03:02:51.196787+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T03:03:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2614066fc0 msgr2=0x7f2614069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2614066fc0 0x7f2614069470 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f261c007d40 tx=0x7f261c015000 comp rx=0 tx=0).stop 2023-10-06T03:03:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 msgr2=0x7f262816afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f262816afe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2618011d40 tx=0x7f26180091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 shutdown_connections 2023-10-06T03:03:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2614066fc0 0x7f2614069470 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2610004810 0x7f262816afe0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26280cdbe0 0x7f2628155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 --2- 172.21.15.92:0/1329494423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2628159950 0x7f2628154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:34.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 >> 172.21.15.92:0/1329494423 conn(0x7f262806baf0 msgr2=0x7f262815c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:34.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 shutdown_connections 2023-10-06T03:03:34.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:34.926+0000 7f262e3f5700 1 -- 172.21.15.92:0/1329494423 wait complete. 2023-10-06T03:03:34.940 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T03:03:34.940 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:35.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.051+0000 7fbb77433700 1 Processor -- start 2023-10-06T03:03:35.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.051+0000 7fbb77433700 1 -- start start 2023-10-06T03:03:35.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb77433700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb77433700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7015a160 con 0x7fbb7015db10 2023-10-06T03:03:35.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb77433700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb700d13a0 con 0x7fbb70159890 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb77433700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7015e140 con 0x7fbb70159c60 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb759d0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbb7015db10 0x7fbb70069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59556/0 (socket says 172.21.15.92:59556) 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb759d0700 1 -- 172.21.15.92:0/3206510346 learned_addr learned my addr 172.21.15.92:0/3206510346 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb749ce700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190800168 0 0) 0x7fbb7015a160 con 0x7fbb7015db10 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.052+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c003570 con 0x7fbb7015db10 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655572653 0 0) 0x7fbb5c003570 con 0x7fbb7015db10 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 msgr2=0x7fbb7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb7015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 >> v1:172.21.15.136:6789/0 conn(0x7fbb70159890 legacy=0x7fbb7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7015a3d0 con 0x7fbb7015db10 2023-10-06T03:03:35.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fbb68001470 con 0x7fbb7015db10 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 >> v1:172.21.15.92:6789/0 conn(0x7fbb7015db10 legacy=0x7fbb70069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb5c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb5c004010 0x7fbb5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb5c004d50 0x7fbb5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7015e140 con 0x7fbb5c004d50 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00bbf0 con 0x7fbb5c004010 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c00bdb0 con 0x7fbb70159c60 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb5c004d50 0x7fbb5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb749ce700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb5c004010 0x7fbb5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb751cf700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb5c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 -- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 msgr2=0x7fbb5c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb5c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 -- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb5c004010 msgr2=0x7fbb5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb5c004010 0x7fbb5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 -- 172.21.15.92:0/3206510346 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c003570 con 0x7fbb5c004d50 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb751cf700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb70159c60 0x7fbb5c003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb759d0700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb5c004d50 0x7fbb5c0051c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c390 tx=0x7fbb68001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1350628bb7d33c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6800cc60 con 0x7fbb5c004d50 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6800cdc0 con 0x7fbb5c004d50 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.053+0000 7fbb67fff700 1 -- 172.21.15.92:0/3206510346 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68005b70 con 0x7fbb5c004d50 2023-10-06T03:03:35.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb77433700 1 -- 172.21.15.92:0/3206510346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb5c004d50 msgr2=0x7fbb5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb77433700 1 --2- 172.21.15.92:0/3206510346 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb5c004d50 0x7fbb5c0051c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c390 tx=0x7fbb68001e50 comp rx=0 tx=0).stop 2023-10-06T03:03:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb759d0700 1 -- 172.21.15.92:0/3206510346 reap_dead start 2023-10-06T03:03:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb77433700 1 -- 172.21.15.92:0/3206510346 shutdown_connections 2023-10-06T03:03:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb77433700 1 -- 172.21.15.92:0/3206510346 >> 172.21.15.92:0/3206510346 conn(0x7fbb70068e20 msgr2=0x7fbb700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb77433700 1 -- 172.21.15.92:0/3206510346 shutdown_connections 2023-10-06T03:03:35.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.054+0000 7fbb77433700 1 -- 172.21.15.92:0/3206510346 wait complete. 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.055+0000 7fbb77433700 1 Processor -- start 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.055+0000 7fbb77433700 1 -- start start 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb77433700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb7016c9f0 0x7fbb7016cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb77433700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 0x7fbb7016af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb77433700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb7016a880 0x7fbb700c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb77433700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700ca800 con 0x7fbb700ca390 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb77433700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700caa80 con 0x7fbb7016c9f0 2023-10-06T03:03:35.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb77433700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700cd970 con 0x7fbb7016a880 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 0x7fbb7016af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 0x7fbb7016af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50342/0 (socket says 172.21.15.92:50342) 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 -- 172.21.15.92:0/1668503832 learned_addr learned my addr 172.21.15.92:0/1668503832 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb751cf700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb7016c9f0 0x7fbb7016cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb759d0700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb7016a880 0x7fbb700c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 -- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb7016a880 msgr2=0x7fbb700c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb7016a880 0x7fbb700c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 -- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb7016c9f0 msgr2=0x7fbb7016cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb7016c9f0 0x7fbb7016cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb749ce700 1 -- 172.21.15.92:0/1668503832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700cdc50 con 0x7fbb700ca390 2023-10-06T03:03:35.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.056+0000 7fbb751cf700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb7016c9f0 0x7fbb7016cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.057+0000 7fbb749ce700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 0x7fbb7016af90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbb60011b40 tx=0x7fbb600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.057+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000fa10 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.057+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6000fb70 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.057+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb700cdf30 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.057+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60014440 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.057+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7016ad20 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.058+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb700ce3d0 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.058+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fbb60014870 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.058+0000 7fbb657fa700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbb54066fc0 0x7fbb54069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.058+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbb60018030 con 0x7fbb700ca390 2023-10-06T03:03:35.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.059+0000 7fbb751cf700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbb54066fc0 0x7fbb54069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.059+0000 7fbb751cf700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbb54066fc0 0x7fbb54069470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c007fd0 tx=0x7fbb6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:35.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.063+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb60052640 con 0x7fbb700ca390 2023-10-06T03:03:35.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.241+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb7004c3c0 con 0x7fbb54066fc0 2023-10-06T03:03:35.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.249+0000 7fbb657fa700 1 -- 172.21.15.92:0/1668503832 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312021 (secure 0 0 0) 0x7fbb7004c3c0 con 0x7fbb54066fc0 2023-10-06T03:03:35.254 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbb54066fc0 msgr2=0x7fbb54069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbb54066fc0 0x7fbb54069470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c007fd0 tx=0x7fbb6c009040 comp rx=0 tx=0).stop 2023-10-06T03:03:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 msgr2=0x7fbb7016af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 0x7fbb7016af90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbb60011b40 tx=0x7fbb600091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 shutdown_connections 2023-10-06T03:03:35.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbb54066fc0 0x7fbb54069470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb7016a880 0x7fbb700c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb7016c9f0 0x7fbb7016cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 --2- 172.21.15.92:0/1668503832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb700ca390 0x7fbb7016af90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 >> 172.21.15.92:0/1668503832 conn(0x7fbb70068e20 msgr2=0x7fbb70152ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 shutdown_connections 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:35.252+0000 7fbb77433700 1 -- 172.21.15.92:0/1668503832 wait complete. 2023-10-06T03:03:35.258 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:35.266 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2023-10-06T03:03:35.204771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385800,"kb_used_data":7496,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588920,"statfs":{"total":773094113280,"available":770651054080,"internally_reserved":0,"allocated":7675904,"data_stored":4073820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.517069"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730680+0000","last_change":"2023-10-06T03:03:29.723778+0000","last_active":"2023-10-06T03:03:29.730680+0000","last_peered":"2023-10-06T03:03:29.730680+0000","last_clean":"2023-10-06T03:03:29.730680+0000","last_became_active":"2023-10-06T03:03:29.723169+0000","last_became_peered":"2023-10-06T03:03:29.723169+0000","last_unstale":"2023-10-06T03:03:29.730680+0000","last_undegraded":"2023-10-06T03:03:29.730680+0000","last_fullsized":"2023-10-06T03:03:29.730680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728864+0000","last_change":"2023-10-06T03:03:29.718001+0000","last_active":"2023-10-06T03:03:29.728864+0000","last_peered":"2023-10-06T03:03:29.728864+0000","last_clean":"2023-10-06T03:03:29.728864+0000","last_became_active":"2023-10-06T03:03:29.714605+0000","last_became_peered":"2023-10-06T03:03:29.714605+0000","last_unstale":"2023-10-06T03:03:29.728864+0000","last_undegraded":"2023-10-06T03:03:29.728864+0000","last_fullsized":"2023-10-06T03:03:29.728864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732232+0000","last_change":"2023-10-06T03:03:29.728789+0000","last_active":"2023-10-06T03:03:29.732232+0000","last_peered":"2023-10-06T03:03:29.732232+0000","last_clean":"2023-10-06T03:03:29.732232+0000","last_became_active":"2023-10-06T03:03:29.728534+0000","last_became_peered":"2023-10-06T03:03:29.728534+0000","last_unstale":"2023-10-06T03:03:29.732232+0000","last_undegraded":"2023-10-06T03:03:29.732232+0000","last_fullsized":"2023-10-06T03:03:29.732232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:30.417220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727754+0000","last_change":"2023-10-06T03:03:29.720422+0000","last_active":"2023-10-06T03:03:29.727754+0000","last_peered":"2023-10-06T03:03:29.727754+0000","last_clean":"2023-10-06T03:03:29.727754+0000","last_became_active":"2023-10-06T03:03:29.720058+0000","last_became_peered":"2023-10-06T03:03:29.720058+0000","last_unstale":"2023-10-06T03:03:29.727754+0000","last_undegraded":"2023-10-06T03:03:29.727754+0000","last_fullsized":"2023-10-06T03:03:29.727754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.729436+0000","last_change":"2023-10-06T03:03:29.719553+0000","last_active":"2023-10-06T03:03:29.729436+0000","last_peered":"2023-10-06T03:03:29.729436+0000","last_clean":"2023-10-06T03:03:29.729436+0000","last_became_active":"2023-10-06T03:03:29.719334+0000","last_became_peered":"2023-10-06T03:03:29.719334+0000","last_unstale":"2023-10-06T03:03:29.729436+0000","last_undegraded":"2023-10-06T03:03:29.729436+0000","last_fullsized":"2023-10-06T03:03:29.729436+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:06.879475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736426+0000","last_change":"2023-10-06T03:03:29.731299+0000","last_active":"2023-10-06T03:03:29.736426+0000","last_peered":"2023-10-06T03:03:29.736426+0000","last_clean":"2023-10-06T03:03:29.736426+0000","last_became_active":"2023-10-06T03:03:29.727621+0000","last_became_peered":"2023-10-06T03:03:29.727621+0000","last_unstale":"2023-10-06T03:03:29.736426+0000","last_undegraded":"2023-10-06T03:03:29.736426+0000","last_fullsized":"2023-10-06T03:03:29.736426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725723+0000","last_change":"2023-10-06T03:03:29.718340+0000","last_active":"2023-10-06T03:03:29.725723+0000","last_peered":"2023-10-06T03:03:29.725723+0000","last_clean":"2023-10-06T03:03:29.725723+0000","last_became_active":"2023-10-06T03:03:29.718072+0000","last_became_peered":"2023-10-06T03:03:29.718072+0000","last_unstale":"2023-10-06T03:03:29.725723+0000","last_undegraded":"2023-10-06T03:03:29.725723+0000","last_fullsized":"2023-10-06T03:03:29.725723+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:42.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731440+0000","last_change":"2023-10-06T03:03:29.724404+0000","last_active":"2023-10-06T03:03:29.731440+0000","last_peered":"2023-10-06T03:03:29.731440+0000","last_clean":"2023-10-06T03:03:29.731440+0000","last_became_active":"2023-10-06T03:03:29.724159+0000","last_became_peered":"2023-10-06T03:03:29.724159+0000","last_unstale":"2023-10-06T03:03:29.731440+0000","last_undegraded":"2023-10-06T03:03:29.731440+0000","last_fullsized":"2023-10-06T03:03:29.731440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:20.821206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726859+0000","last_change":"2023-10-06T03:03:29.721567+0000","last_active":"2023-10-06T03:03:29.726859+0000","last_peered":"2023-10-06T03:03:29.726859+0000","last_clean":"2023-10-06T03:03:29.726859+0000","last_became_active":"2023-10-06T03:03:29.720653+0000","last_became_peered":"2023-10-06T03:03:29.720653+0000","last_unstale":"2023-10-06T03:03:29.726859+0000","last_undegraded":"2023-10-06T03:03:29.726859+0000","last_fullsized":"2023-10-06T03:03:29.726859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722753+0000","last_change":"2023-10-06T03:03:29.717201+0000","last_active":"2023-10-06T03:03:29.722753+0000","last_peered":"2023-10-06T03:03:29.722753+0000","last_clean":"2023-10-06T03:03:29.722753+0000","last_became_active":"2023-10-06T03:03:29.716949+0000","last_became_peered":"2023-10-06T03:03:29.716949+0000","last_unstale":"2023-10-06T03:03:29.722753+0000","last_undegraded":"2023-10-06T03:03:29.722753+0000","last_fullsized":"2023-10-06T03:03:29.722753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727527+0000","last_change":"2023-10-06T03:03:29.712576+0000","last_active":"2023-10-06T03:03:29.727527+0000","last_peered":"2023-10-06T03:03:29.727527+0000","last_clean":"2023-10-06T03:03:29.727527+0000","last_became_active":"2023-10-06T03:03:29.712317+0000","last_became_peered":"2023-10-06T03:03:29.712317+0000","last_unstale":"2023-10-06T03:03:29.727527+0000","last_undegraded":"2023-10-06T03:03:29.727527+0000","last_fullsized":"2023-10-06T03:03:29.727527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:37.515635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722817+0000","last_change":"2023-10-06T03:03:29.716677+0000","last_active":"2023-10-06T03:03:29.722817+0000","last_peered":"2023-10-06T03:03:29.722817+0000","last_clean":"2023-10-06T03:03:29.722817+0000","last_became_active":"2023-10-06T03:03:29.716387+0000","last_became_peered":"2023-10-06T03:03:29.716387+0000","last_unstale":"2023-10-06T03:03:29.722817+0000","last_undegraded":"2023-10-06T03:03:29.722817+0000","last_fullsized":"2023-10-06T03:03:29.722817+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731906+0000","last_change":"2023-10-06T03:03:29.722034+0000","last_active":"2023-10-06T03:03:29.731906+0000","last_peered":"2023-10-06T03:03:29.731906+0000","last_clean":"2023-10-06T03:03:29.731906+0000","last_became_active":"2023-10-06T03:03:29.721750+0000","last_became_peered":"2023-10-06T03:03:29.721750+0000","last_unstale":"2023-10-06T03:03:29.731906+0000","last_undegraded":"2023-10-06T03:03:29.731906+0000","last_fullsized":"2023-10-06T03:03:29.731906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:31.502226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736461+0000","last_change":"2023-10-06T03:03:29.731966+0000","last_active":"2023-10-06T03:03:29.736461+0000","last_peered":"2023-10-06T03:03:29.736461+0000","last_clean":"2023-10-06T03:03:29.736461+0000","last_became_active":"2023-10-06T03:03:29.727297+0000","last_became_peered":"2023-10-06T03:03:29.727297+0000","last_unstale":"2023-10-06T03:03:29.736461+0000","last_undegraded":"2023-10-06T03:03:29.736461+0000","last_fullsized":"2023-10-06T03:03:29.736461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730867+0000","last_change":"2023-10-06T03:03:29.726517+0000","last_active":"2023-10-06T03:03:29.730867+0000","last_peered":"2023-10-06T03:03:29.730867+0000","last_clean":"2023-10-06T03:03:29.730867+0000","last_became_active":"2023-10-06T03:03:29.726030+0000","last_became_peered":"2023-10-06T03:03:29.726030+0000","last_unstale":"2023-10-06T03:03:29.730867+0000","last_undegraded":"2023-10-06T03:03:29.730867+0000","last_fullsized":"2023-10-06T03:03:29.730867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730357+0000","last_change":"2023-10-06T03:03:29.723878+0000","last_active":"2023-10-06T03:03:29.730357+0000","last_peered":"2023-10-06T03:03:29.730357+0000","last_clean":"2023-10-06T03:03:29.730357+0000","last_became_active":"2023-10-06T03:03:29.723154+0000","last_became_peered":"2023-10-06T03:03:29.723154+0000","last_unstale":"2023-10-06T03:03:29.730357+0000","last_undegraded":"2023-10-06T03:03:29.730357+0000","last_fullsized":"2023-10-06T03:03:29.730357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.719190+0000","last_change":"2023-10-06T03:03:29.716857+0000","last_active":"2023-10-06T03:03:29.719190+0000","last_peered":"2023-10-06T03:03:29.719190+0000","last_clean":"2023-10-06T03:03:29.719190+0000","last_became_active":"2023-10-06T03:03:29.716624+0000","last_became_peered":"2023-10-06T03:03:29.716624+0000","last_unstale":"2023-10-06T03:03:29.719190+0000","last_undegraded":"2023-10-06T03:03:29.719190+0000","last_fullsized":"2023-10-06T03:03:29.719190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:33.517523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728951+0000","last_change":"2023-10-06T03:03:29.722266+0000","last_active":"2023-10-06T03:03:29.728951+0000","last_peered":"2023-10-06T03:03:29.728951+0000","last_clean":"2023-10-06T03:03:29.728951+0000","last_became_active":"2023-10-06T03:03:29.722042+0000","last_became_peered":"2023-10-06T03:03:29.722042+0000","last_unstale":"2023-10-06T03:03:29.728951+0000","last_undegraded":"2023-10-06T03:03:29.728951+0000","last_fullsized":"2023-10-06T03:03:29.728951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:42.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.724572+0000","last_change":"2023-10-06T03:03:29.719109+0000","last_active":"2023-10-06T03:03:29.724572+0000","last_peered":"2023-10-06T03:03:29.724572+0000","last_clean":"2023-10-06T03:03:29.724572+0000","last_became_active":"2023-10-06T03:03:29.718855+0000","last_became_peered":"2023-10-06T03:03:29.718855+0000","last_unstale":"2023-10-06T03:03:29.724572+0000","last_undegraded":"2023-10-06T03:03:29.724572+0000","last_fullsized":"2023-10-06T03:03:29.724572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.729932+0000","last_change":"2023-10-06T03:03:29.721942+0000","last_active":"2023-10-06T03:03:29.729932+0000","last_peered":"2023-10-06T03:03:29.729932+0000","last_clean":"2023-10-06T03:03:29.729932+0000","last_became_active":"2023-10-06T03:03:29.721655+0000","last_became_peered":"2023-10-06T03:03:29.721655+0000","last_unstale":"2023-10-06T03:03:29.729932+0000","last_undegraded":"2023-10-06T03:03:29.729932+0000","last_fullsized":"2023-10-06T03:03:29.729932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.061645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730489+0000","last_change":"2023-10-06T03:03:29.727560+0000","last_active":"2023-10-06T03:03:29.730489+0000","last_peered":"2023-10-06T03:03:29.730489+0000","last_clean":"2023-10-06T03:03:29.730489+0000","last_became_active":"2023-10-06T03:03:29.726841+0000","last_became_peered":"2023-10-06T03:03:29.726841+0000","last_unstale":"2023-10-06T03:03:29.730489+0000","last_undegraded":"2023-10-06T03:03:29.730489+0000","last_fullsized":"2023-10-06T03:03:29.730489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730336+0000","last_change":"2023-10-06T03:03:29.722110+0000","last_active":"2023-10-06T03:03:29.730336+0000","last_peered":"2023-10-06T03:03:29.730336+0000","last_clean":"2023-10-06T03:03:29.730336+0000","last_became_active":"2023-10-06T03:03:29.721847+0000","last_became_peered":"2023-10-06T03:03:29.721847+0000","last_unstale":"2023-10-06T03:03:29.730336+0000","last_undegraded":"2023-10-06T03:03:29.730336+0000","last_fullsized":"2023-10-06T03:03:29.730336+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:33.599591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.723179+0000","last_change":"2023-10-06T03:03:29.718942+0000","last_active":"2023-10-06T03:03:29.723179+0000","last_peered":"2023-10-06T03:03:29.723179+0000","last_clean":"2023-10-06T03:03:29.723179+0000","last_became_active":"2023-10-06T03:03:29.718682+0000","last_became_peered":"2023-10-06T03:03:29.718682+0000","last_unstale":"2023-10-06T03:03:29.723179+0000","last_undegraded":"2023-10-06T03:03:29.723179+0000","last_fullsized":"2023-10-06T03:03:29.723179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727957+0000","last_change":"2023-10-06T03:03:29.716417+0000","last_active":"2023-10-06T03:03:29.727957+0000","last_peered":"2023-10-06T03:03:29.727957+0000","last_clean":"2023-10-06T03:03:29.727957+0000","last_became_active":"2023-10-06T03:03:29.716136+0000","last_became_peered":"2023-10-06T03:03:29.716136+0000","last_unstale":"2023-10-06T03:03:29.727957+0000","last_undegraded":"2023-10-06T03:03:29.727957+0000","last_fullsized":"2023-10-06T03:03:29.727957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728816+0000","last_change":"2023-10-06T03:03:29.725908+0000","last_active":"2023-10-06T03:03:29.728816+0000","last_peered":"2023-10-06T03:03:29.728816+0000","last_clean":"2023-10-06T03:03:29.728816+0000","last_became_active":"2023-10-06T03:03:29.720807+0000","last_became_peered":"2023-10-06T03:03:29.720807+0000","last_unstale":"2023-10-06T03:03:29.728816+0000","last_undegraded":"2023-10-06T03:03:29.728816+0000","last_fullsized":"2023-10-06T03:03:29.728816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.717426+0000","last_change":"2023-10-06T03:03:29.711792+0000","last_active":"2023-10-06T03:03:29.717426+0000","last_peered":"2023-10-06T03:03:29.717426+0000","last_clean":"2023-10-06T03:03:29.717426+0000","last_became_active":"2023-10-06T03:03:29.711514+0000","last_became_peered":"2023-10-06T03:03:29.711514+0000","last_unstale":"2023-10-06T03:03:29.717426+0000","last_undegraded":"2023-10-06T03:03:29.717426+0000","last_fullsized":"2023-10-06T03:03:29.717426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728591+0000","last_change":"2023-10-06T03:03:29.722633+0000","last_active":"2023-10-06T03:03:29.728591+0000","last_peered":"2023-10-06T03:03:29.728591+0000","last_clean":"2023-10-06T03:03:29.728591+0000","last_became_active":"2023-10-06T03:03:29.722331+0000","last_became_peered":"2023-10-06T03:03:29.722331+0000","last_unstale":"2023-10-06T03:03:29.728591+0000","last_undegraded":"2023-10-06T03:03:29.728591+0000","last_fullsized":"2023-10-06T03:03:29.728591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.719760+0000","last_change":"2023-10-06T03:03:29.716929+0000","last_active":"2023-10-06T03:03:29.719760+0000","last_peered":"2023-10-06T03:03:29.719760+0000","last_clean":"2023-10-06T03:03:29.719760+0000","last_became_active":"2023-10-06T03:03:29.716689+0000","last_became_peered":"2023-10-06T03:03:29.716689+0000","last_unstale":"2023-10-06T03:03:29.719760+0000","last_undegraded":"2023-10-06T03:03:29.719760+0000","last_fullsized":"2023-10-06T03:03:29.719760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:43:26.246835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727929+0000","last_change":"2023-10-06T03:03:29.716732+0000","last_active":"2023-10-06T03:03:29.727929+0000","last_peered":"2023-10-06T03:03:29.727929+0000","last_clean":"2023-10-06T03:03:29.727929+0000","last_became_active":"2023-10-06T03:03:29.716488+0000","last_became_peered":"2023-10-06T03:03:29.716488+0000","last_unstale":"2023-10-06T03:03:29.727929+0000","last_undegraded":"2023-10-06T03:03:29.727929+0000","last_fullsized":"2023-10-06T03:03:29.727929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:46.103255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732766+0000","last_change":"2023-10-06T03:03:29.729334+0000","last_active":"2023-10-06T03:03:29.732766+0000","last_peered":"2023-10-06T03:03:29.732766+0000","last_clean":"2023-10-06T03:03:29.732766+0000","last_became_active":"2023-10-06T03:03:29.728999+0000","last_became_peered":"2023-10-06T03:03:29.728999+0000","last_unstale":"2023-10-06T03:03:29.732766+0000","last_undegraded":"2023-10-06T03:03:29.732766+0000","last_fullsized":"2023-10-06T03:03:29.732766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731004+0000","last_change":"2023-10-06T03:03:29.726465+0000","last_active":"2023-10-06T03:03:29.731004+0000","last_peered":"2023-10-06T03:03:29.731004+0000","last_clean":"2023-10-06T03:03:29.731004+0000","last_became_active":"2023-10-06T03:03:29.726231+0000","last_became_peered":"2023-10-06T03:03:29.726231+0000","last_unstale":"2023-10-06T03:03:29.731004+0000","last_undegraded":"2023-10-06T03:03:29.731004+0000","last_fullsized":"2023-10-06T03:03:29.731004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:44.152122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730336+0000","last_change":"2023-10-06T03:03:29.723868+0000","last_active":"2023-10-06T03:03:29.730336+0000","last_peered":"2023-10-06T03:03:29.730336+0000","last_clean":"2023-10-06T03:03:29.730336+0000","last_became_active":"2023-10-06T03:03:29.722945+0000","last_became_peered":"2023-10-06T03:03:29.722945+0000","last_unstale":"2023-10-06T03:03:29.730336+0000","last_undegraded":"2023-10-06T03:03:29.730336+0000","last_fullsized":"2023-10-06T03:03:29.730336+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728887+0000","last_change":"2023-10-06T03:03:29.721810+0000","last_active":"2023-10-06T03:03:29.728887+0000","last_peered":"2023-10-06T03:03:29.728887+0000","last_clean":"2023-10-06T03:03:29.728887+0000","last_became_active":"2023-10-06T03:03:29.721567+0000","last_became_peered":"2023-10-06T03:03:29.721567+0000","last_unstale":"2023-10-06T03:03:29.728887+0000","last_undegraded":"2023-10-06T03:03:29.728887+0000","last_fullsized":"2023-10-06T03:03:29.728887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:29.446863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.734684+0000","last_change":"2023-10-06T03:03:28.705118+0000","last_active":"2023-10-06T03:03:29.734684+0000","last_peered":"2023-10-06T03:03:29.734684+0000","last_clean":"2023-10-06T03:03:29.734684+0000","last_became_active":"2023-10-06T03:03:28.704360+0000","last_became_peered":"2023-10-06T03:03:28.704360+0000","last_unstale":"2023-10-06T03:03:29.734684+0000","last_undegraded":"2023-10-06T03:03:29.734684+0000","last_fullsized":"2023-10-06T03:03:29.734684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:51.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733224+0000","last_change":"2023-10-06T03:03:29.728561+0000","last_active":"2023-10-06T03:03:29.733224+0000","last_peered":"2023-10-06T03:03:29.733224+0000","last_clean":"2023-10-06T03:03:29.733224+0000","last_became_active":"2023-10-06T03:03:29.726346+0000","last_became_peered":"2023-10-06T03:03:29.726346+0000","last_unstale":"2023-10-06T03:03:29.733224+0000","last_undegraded":"2023-10-06T03:03:29.733224+0000","last_fullsized":"2023-10-06T03:03:29.733224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730231+0000","last_change":"2023-10-06T03:03:28.709814+0000","last_active":"2023-10-06T03:03:29.730231+0000","last_peered":"2023-10-06T03:03:29.730231+0000","last_clean":"2023-10-06T03:03:29.730231+0000","last_became_active":"2023-10-06T03:03:28.708830+0000","last_became_peered":"2023-10-06T03:03:28.708830+0000","last_unstale":"2023-10-06T03:03:29.730231+0000","last_undegraded":"2023-10-06T03:03:29.730231+0000","last_fullsized":"2023-10-06T03:03:29.730231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733217+0000","last_change":"2023-10-06T03:03:29.730127+0000","last_active":"2023-10-06T03:03:29.733217+0000","last_peered":"2023-10-06T03:03:29.733217+0000","last_clean":"2023-10-06T03:03:29.733217+0000","last_became_active":"2023-10-06T03:03:29.729670+0000","last_became_peered":"2023-10-06T03:03:29.729670+0000","last_unstale":"2023-10-06T03:03:29.733217+0000","last_undegraded":"2023-10-06T03:03:29.733217+0000","last_fullsized":"2023-10-06T03:03:29.733217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732178+0000","last_change":"2023-10-06T03:03:28.710421+0000","last_active":"2023-10-06T03:03:29.732178+0000","last_peered":"2023-10-06T03:03:29.732178+0000","last_clean":"2023-10-06T03:03:29.732178+0000","last_became_active":"2023-10-06T03:03:28.710171+0000","last_became_peered":"2023-10-06T03:03:28.710171+0000","last_unstale":"2023-10-06T03:03:29.732178+0000","last_undegraded":"2023-10-06T03:03:29.732178+0000","last_fullsized":"2023-10-06T03:03:29.732178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727157+0000","last_change":"2023-10-06T03:03:29.722570+0000","last_active":"2023-10-06T03:03:29.727157+0000","last_peered":"2023-10-06T03:03:29.727157+0000","last_clean":"2023-10-06T03:03:29.727157+0000","last_became_active":"2023-10-06T03:03:29.722301+0000","last_became_peered":"2023-10-06T03:03:29.722301+0000","last_unstale":"2023-10-06T03:03:29.727157+0000","last_undegraded":"2023-10-06T03:03:29.727157+0000","last_fullsized":"2023-10-06T03:03:29.727157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733140+0000","last_change":"2023-10-06T03:03:28.711375+0000","last_active":"2023-10-06T03:03:29.733140+0000","last_peered":"2023-10-06T03:03:29.733140+0000","last_clean":"2023-10-06T03:03:29.733140+0000","last_became_active":"2023-10-06T03:03:28.711092+0000","last_became_peered":"2023-10-06T03:03:28.711092+0000","last_unstale":"2023-10-06T03:03:29.733140+0000","last_undegraded":"2023-10-06T03:03:29.733140+0000","last_fullsized":"2023-10-06T03:03:29.733140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:32.012077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727483+0000","last_change":"2023-10-06T03:03:29.716098+0000","last_active":"2023-10-06T03:03:29.727483+0000","last_peered":"2023-10-06T03:03:29.727483+0000","last_clean":"2023-10-06T03:03:29.727483+0000","last_became_active":"2023-10-06T03:03:29.715856+0000","last_became_peered":"2023-10-06T03:03:29.715856+0000","last_unstale":"2023-10-06T03:03:29.727483+0000","last_undegraded":"2023-10-06T03:03:29.727483+0000","last_fullsized":"2023-10-06T03:03:29.727483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730534+0000","last_change":"2023-10-06T03:03:28.709567+0000","last_active":"2023-10-06T03:03:29.730534+0000","last_peered":"2023-10-06T03:03:29.730534+0000","last_clean":"2023-10-06T03:03:29.730534+0000","last_became_active":"2023-10-06T03:03:28.709325+0000","last_became_peered":"2023-10-06T03:03:28.709325+0000","last_unstale":"2023-10-06T03:03:29.730534+0000","last_undegraded":"2023-10-06T03:03:29.730534+0000","last_fullsized":"2023-10-06T03:03:29.730534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:41.961709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727551+0000","last_change":"2023-10-06T03:03:29.715609+0000","last_active":"2023-10-06T03:03:29.727551+0000","last_peered":"2023-10-06T03:03:29.727551+0000","last_clean":"2023-10-06T03:03:29.727551+0000","last_became_active":"2023-10-06T03:03:29.715358+0000","last_became_peered":"2023-10-06T03:03:29.715358+0000","last_unstale":"2023-10-06T03:03:29.727551+0000","last_undegraded":"2023-10-06T03:03:29.727551+0000","last_fullsized":"2023-10-06T03:03:29.727551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:12.612570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736693+0000","last_change":"2023-10-06T03:03:28.697823+0000","last_active":"2023-10-06T03:03:29.736693+0000","last_peered":"2023-10-06T03:03:29.736693+0000","last_clean":"2023-10-06T03:03:29.736693+0000","last_became_active":"2023-10-06T03:03:28.697466+0000","last_became_peered":"2023-10-06T03:03:28.697466+0000","last_unstale":"2023-10-06T03:03:29.736693+0000","last_undegraded":"2023-10-06T03:03:29.736693+0000","last_fullsized":"2023-10-06T03:03:29.736693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727960+0000","last_change":"2023-10-06T03:03:29.725473+0000","last_active":"2023-10-06T03:03:29.727960+0000","last_peered":"2023-10-06T03:03:29.727960+0000","last_clean":"2023-10-06T03:03:29.727960+0000","last_became_active":"2023-10-06T03:03:29.725215+0000","last_became_peered":"2023-10-06T03:03:29.725215+0000","last_unstale":"2023-10-06T03:03:29.727960+0000","last_undegraded":"2023-10-06T03:03:29.727960+0000","last_fullsized":"2023-10-06T03:03:29.727960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.703927+0000","last_change":"2023-10-06T03:03:28.711882+0000","last_active":"2023-10-06T03:03:29.703927+0000","last_peered":"2023-10-06T03:03:29.703927+0000","last_clean":"2023-10-06T03:03:29.703927+0000","last_became_active":"2023-10-06T03:03:28.711670+0000","last_became_peered":"2023-10-06T03:03:28.711670+0000","last_unstale":"2023-10-06T03:03:29.703927+0000","last_undegraded":"2023-10-06T03:03:29.703927+0000","last_fullsized":"2023-10-06T03:03:29.703927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:43.144129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726252+0000","last_change":"2023-10-06T03:03:29.718657+0000","last_active":"2023-10-06T03:03:29.726252+0000","last_peered":"2023-10-06T03:03:29.726252+0000","last_clean":"2023-10-06T03:03:29.726252+0000","last_became_active":"2023-10-06T03:03:29.718357+0000","last_became_peered":"2023-10-06T03:03:29.718357+0000","last_unstale":"2023-10-06T03:03:29.726252+0000","last_undegraded":"2023-10-06T03:03:29.726252+0000","last_fullsized":"2023-10-06T03:03:29.726252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:44.152122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.707806+0000","last_change":"2023-10-06T03:03:28.702078+0000","last_active":"2023-10-06T03:03:29.707806+0000","last_peered":"2023-10-06T03:03:29.707806+0000","last_clean":"2023-10-06T03:03:29.707806+0000","last_became_active":"2023-10-06T03:03:28.701828+0000","last_became_peered":"2023-10-06T03:03:28.701828+0000","last_unstale":"2023-10-06T03:03:29.707806+0000","last_undegraded":"2023-10-06T03:03:29.707806+0000","last_fullsized":"2023-10-06T03:03:29.707806+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:50.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728841+0000","last_change":"2023-10-06T03:03:29.725737+0000","last_active":"2023-10-06T03:03:29.728841+0000","last_peered":"2023-10-06T03:03:29.728841+0000","last_clean":"2023-10-06T03:03:29.728841+0000","last_became_active":"2023-10-06T03:03:29.725491+0000","last_became_peered":"2023-10-06T03:03:29.725491+0000","last_unstale":"2023-10-06T03:03:29.728841+0000","last_undegraded":"2023-10-06T03:03:29.728841+0000","last_fullsized":"2023-10-06T03:03:29.728841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711967+0000","last_change":"2023-10-06T03:03:28.709569+0000","last_active":"2023-10-06T03:03:29.711967+0000","last_peered":"2023-10-06T03:03:29.711967+0000","last_clean":"2023-10-06T03:03:29.711967+0000","last_became_active":"2023-10-06T03:03:28.709216+0000","last_became_peered":"2023-10-06T03:03:28.709216+0000","last_unstale":"2023-10-06T03:03:29.711967+0000","last_undegraded":"2023-10-06T03:03:29.711967+0000","last_fullsized":"2023-10-06T03:03:29.711967+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727702+0000","last_change":"2023-10-06T03:03:29.720004+0000","last_active":"2023-10-06T03:03:29.727702+0000","last_peered":"2023-10-06T03:03:29.727702+0000","last_clean":"2023-10-06T03:03:29.727702+0000","last_became_active":"2023-10-06T03:03:29.719665+0000","last_became_peered":"2023-10-06T03:03:29.719665+0000","last_unstale":"2023-10-06T03:03:29.727702+0000","last_undegraded":"2023-10-06T03:03:29.727702+0000","last_fullsized":"2023-10-06T03:03:29.727702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708267+0000","last_change":"2023-10-06T03:03:28.704341+0000","last_active":"2023-10-06T03:03:29.708267+0000","last_peered":"2023-10-06T03:03:29.708267+0000","last_clean":"2023-10-06T03:03:29.708267+0000","last_became_active":"2023-10-06T03:03:28.704116+0000","last_became_peered":"2023-10-06T03:03:28.704116+0000","last_unstale":"2023-10-06T03:03:29.708267+0000","last_undegraded":"2023-10-06T03:03:29.708267+0000","last_fullsized":"2023-10-06T03:03:29.708267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726855+0000","last_change":"2023-10-06T03:03:29.720059+0000","last_active":"2023-10-06T03:03:29.726855+0000","last_peered":"2023-10-06T03:03:29.726855+0000","last_clean":"2023-10-06T03:03:29.726855+0000","last_became_active":"2023-10-06T03:03:29.719808+0000","last_became_peered":"2023-10-06T03:03:29.719808+0000","last_unstale":"2023-10-06T03:03:29.726855+0000","last_undegraded":"2023-10-06T03:03:29.726855+0000","last_fullsized":"2023-10-06T03:03:29.726855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731482+0000","last_change":"2023-10-06T03:03:28.707321+0000","last_active":"2023-10-06T03:03:29.731482+0000","last_peered":"2023-10-06T03:03:29.731482+0000","last_clean":"2023-10-06T03:03:29.731482+0000","last_became_active":"2023-10-06T03:03:28.706986+0000","last_became_peered":"2023-10-06T03:03:28.706986+0000","last_unstale":"2023-10-06T03:03:29.731482+0000","last_undegraded":"2023-10-06T03:03:29.731482+0000","last_fullsized":"2023-10-06T03:03:29.731482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.749372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732256+0000","last_change":"2023-10-06T03:03:29.728887+0000","last_active":"2023-10-06T03:03:29.732256+0000","last_peered":"2023-10-06T03:03:29.732256+0000","last_clean":"2023-10-06T03:03:29.732256+0000","last_became_active":"2023-10-06T03:03:29.728331+0000","last_became_peered":"2023-10-06T03:03:29.728331+0000","last_unstale":"2023-10-06T03:03:29.732256+0000","last_undegraded":"2023-10-06T03:03:29.732256+0000","last_fullsized":"2023-10-06T03:03:29.732256+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.272664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"65'6","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.738004+0000","last_change":"2023-10-06T03:03:28.702854+0000","last_active":"2023-10-06T03:03:29.738004+0000","last_peered":"2023-10-06T03:03:29.738004+0000","last_clean":"2023-10-06T03:03:29.738004+0000","last_became_active":"2023-10-06T03:03:28.702541+0000","last_became_peered":"2023-10-06T03:03:28.702541+0000","last_unstale":"2023-10-06T03:03:29.738004+0000","last_undegraded":"2023-10-06T03:03:29.738004+0000","last_fullsized":"2023-10-06T03:03:29.738004+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727735+0000","last_change":"2023-10-06T03:03:29.720289+0000","last_active":"2023-10-06T03:03:29.727735+0000","last_peered":"2023-10-06T03:03:29.727735+0000","last_clean":"2023-10-06T03:03:29.727735+0000","last_became_active":"2023-10-06T03:03:29.719700+0000","last_became_peered":"2023-10-06T03:03:29.719700+0000","last_unstale":"2023-10-06T03:03:29.727735+0000","last_undegraded":"2023-10-06T03:03:29.727735+0000","last_fullsized":"2023-10-06T03:03:29.727735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733446+0000","last_change":"2023-10-06T03:03:28.701142+0000","last_active":"2023-10-06T03:03:29.733446+0000","last_peered":"2023-10-06T03:03:29.733446+0000","last_clean":"2023-10-06T03:03:29.733446+0000","last_became_active":"2023-10-06T03:03:28.700888+0000","last_became_peered":"2023-10-06T03:03:28.700888+0000","last_unstale":"2023-10-06T03:03:29.733446+0000","last_undegraded":"2023-10-06T03:03:29.733446+0000","last_fullsized":"2023-10-06T03:03:29.733446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725958+0000","last_change":"2023-10-06T03:03:29.719247+0000","last_active":"2023-10-06T03:03:29.725958+0000","last_peered":"2023-10-06T03:03:29.725958+0000","last_clean":"2023-10-06T03:03:29.725958+0000","last_became_active":"2023-10-06T03:03:29.719000+0000","last_became_peered":"2023-10-06T03:03:29.719000+0000","last_unstale":"2023-10-06T03:03:29.725958+0000","last_undegraded":"2023-10-06T03:03:29.725958+0000","last_fullsized":"2023-10-06T03:03:29.725958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.272664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711971+0000","last_change":"2023-10-06T03:03:28.709750+0000","last_active":"2023-10-06T03:03:29.711971+0000","last_peered":"2023-10-06T03:03:29.711971+0000","last_clean":"2023-10-06T03:03:29.711971+0000","last_became_active":"2023-10-06T03:03:28.708794+0000","last_became_peered":"2023-10-06T03:03:28.708794+0000","last_unstale":"2023-10-06T03:03:29.711971+0000","last_undegraded":"2023-10-06T03:03:29.711971+0000","last_fullsized":"2023-10-06T03:03:29.711971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:31.256052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725898+0000","last_change":"2023-10-06T03:03:29.722007+0000","last_active":"2023-10-06T03:03:29.725898+0000","last_peered":"2023-10-06T03:03:29.725898+0000","last_clean":"2023-10-06T03:03:29.725898+0000","last_became_active":"2023-10-06T03:03:29.721770+0000","last_became_peered":"2023-10-06T03:03:29.721770+0000","last_unstale":"2023-10-06T03:03:29.725898+0000","last_undegraded":"2023-10-06T03:03:29.725898+0000","last_fullsized":"2023-10-06T03:03:29.725898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:04.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.709703+0000","last_change":"2023-10-06T03:03:28.707056+0000","last_active":"2023-10-06T03:03:29.709703+0000","last_peered":"2023-10-06T03:03:29.709703+0000","last_clean":"2023-10-06T03:03:29.709703+0000","last_became_active":"2023-10-06T03:03:28.706778+0000","last_became_peered":"2023-10-06T03:03:28.706778+0000","last_unstale":"2023-10-06T03:03:29.709703+0000","last_undegraded":"2023-10-06T03:03:29.709703+0000","last_fullsized":"2023-10-06T03:03:29.709703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:18.282708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726300+0000","last_change":"2023-10-06T03:03:29.721620+0000","last_active":"2023-10-06T03:03:29.726300+0000","last_peered":"2023-10-06T03:03:29.726300+0000","last_clean":"2023-10-06T03:03:29.726300+0000","last_became_active":"2023-10-06T03:03:29.721371+0000","last_became_peered":"2023-10-06T03:03:29.721371+0000","last_unstale":"2023-10-06T03:03:29.726300+0000","last_undegraded":"2023-10-06T03:03:29.726300+0000","last_fullsized":"2023-10-06T03:03:29.726300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:29.446863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"65'8","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.738156+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:29.738156+0000","last_peered":"2023-10-06T03:03:29.738156+0000","last_clean":"2023-10-06T03:03:29.738156+0000","last_became_active":"2023-10-06T03:03:28.707467+0000","last_became_peered":"2023-10-06T03:03:28.707467+0000","last_unstale":"2023-10-06T03:03:29.738156+0000","last_undegraded":"2023-10-06T03:03:29.738156+0000","last_fullsized":"2023-10-06T03:03:29.738156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726915+0000","last_change":"2023-10-06T03:03:29.721553+0000","last_active":"2023-10-06T03:03:29.726915+0000","last_peered":"2023-10-06T03:03:29.726915+0000","last_clean":"2023-10-06T03:03:29.726915+0000","last_became_active":"2023-10-06T03:03:29.721102+0000","last_became_peered":"2023-10-06T03:03:29.721102+0000","last_unstale":"2023-10-06T03:03:29.726915+0000","last_undegraded":"2023-10-06T03:03:29.726915+0000","last_fullsized":"2023-10-06T03:03:29.726915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731305+0000","last_change":"2023-10-06T03:03:28.703808+0000","last_active":"2023-10-06T03:03:29.731305+0000","last_peered":"2023-10-06T03:03:29.731305+0000","last_clean":"2023-10-06T03:03:29.731305+0000","last_became_active":"2023-10-06T03:03:28.703078+0000","last_became_peered":"2023-10-06T03:03:28.703078+0000","last_unstale":"2023-10-06T03:03:29.731305+0000","last_undegraded":"2023-10-06T03:03:29.731305+0000","last_fullsized":"2023-10-06T03:03:29.731305+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725981+0000","last_change":"2023-10-06T03:03:29.723143+0000","last_active":"2023-10-06T03:03:29.725981+0000","last_peered":"2023-10-06T03:03:29.725981+0000","last_clean":"2023-10-06T03:03:29.725981+0000","last_became_active":"2023-10-06T03:03:29.722899+0000","last_became_peered":"2023-10-06T03:03:29.722899+0000","last_unstale":"2023-10-06T03:03:29.725981+0000","last_undegraded":"2023-10-06T03:03:29.725981+0000","last_fullsized":"2023-10-06T03:03:29.725981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:48.803431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-06T03:03:30.717830+0000","last_change":"2023-10-06T03:03:28.705716+0000","last_active":"2023-10-06T03:03:30.717830+0000","last_peered":"2023-10-06T03:03:30.717830+0000","last_clean":"2023-10-06T03:03:30.717830+0000","last_became_active":"2023-10-06T03:03:28.705508+0000","last_became_peered":"2023-10-06T03:03:28.705508+0000","last_unstale":"2023-10-06T03:03:30.717830+0000","last_undegraded":"2023-10-06T03:03:30.717830+0000","last_fullsized":"2023-10-06T03:03:30.717830+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:47.795438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728139+0000","last_change":"2023-10-06T03:03:29.725809+0000","last_active":"2023-10-06T03:03:29.728139+0000","last_peered":"2023-10-06T03:03:29.728139+0000","last_clean":"2023-10-06T03:03:29.728139+0000","last_became_active":"2023-10-06T03:03:29.720674+0000","last_became_peered":"2023-10-06T03:03:29.720674+0000","last_unstale":"2023-10-06T03:03:29.728139+0000","last_undegraded":"2023-10-06T03:03:29.728139+0000","last_fullsized":"2023-10-06T03:03:29.728139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.712156+0000","last_change":"2023-10-06T03:03:28.709286+0000","last_active":"2023-10-06T03:03:29.712156+0000","last_peered":"2023-10-06T03:03:29.712156+0000","last_clean":"2023-10-06T03:03:29.712156+0000","last_became_active":"2023-10-06T03:03:28.708990+0000","last_became_peered":"2023-10-06T03:03:28.708990+0000","last_unstale":"2023-10-06T03:03:29.712156+0000","last_undegraded":"2023-10-06T03:03:29.712156+0000","last_fullsized":"2023-10-06T03:03:29.712156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728818+0000","last_change":"2023-10-06T03:03:29.725295+0000","last_active":"2023-10-06T03:03:29.728818+0000","last_peered":"2023-10-06T03:03:29.728818+0000","last_clean":"2023-10-06T03:03:29.728818+0000","last_became_active":"2023-10-06T03:03:29.725125+0000","last_became_peered":"2023-10-06T03:03:29.725125+0000","last_unstale":"2023-10-06T03:03:29.728818+0000","last_undegraded":"2023-10-06T03:03:29.728818+0000","last_fullsized":"2023-10-06T03:03:29.728818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:18.809274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711255+0000","last_change":"2023-10-06T03:03:28.709344+0000","last_active":"2023-10-06T03:03:29.711255+0000","last_peered":"2023-10-06T03:03:29.711255+0000","last_clean":"2023-10-06T03:03:29.711255+0000","last_became_active":"2023-10-06T03:03:28.709005+0000","last_became_peered":"2023-10-06T03:03:28.709005+0000","last_unstale":"2023-10-06T03:03:29.711255+0000","last_undegraded":"2023-10-06T03:03:29.711255+0000","last_fullsized":"2023-10-06T03:03:29.711255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725988+0000","last_change":"2023-10-06T03:03:29.721504+0000","last_active":"2023-10-06T03:03:29.725988+0000","last_peered":"2023-10-06T03:03:29.725988+0000","last_clean":"2023-10-06T03:03:29.725988+0000","last_became_active":"2023-10-06T03:03:29.721198+0000","last_became_peered":"2023-10-06T03:03:29.721198+0000","last_unstale":"2023-10-06T03:03:29.725988+0000","last_undegraded":"2023-10-06T03:03:29.725988+0000","last_fullsized":"2023-10-06T03:03:29.725988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:11.768132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711510+0000","last_change":"2023-10-06T03:03:28.711750+0000","last_active":"2023-10-06T03:03:29.711510+0000","last_peered":"2023-10-06T03:03:29.711510+0000","last_clean":"2023-10-06T03:03:29.711510+0000","last_became_active":"2023-10-06T03:03:28.711510+0000","last_became_peered":"2023-10-06T03:03:28.711510+0000","last_unstale":"2023-10-06T03:03:29.711510+0000","last_undegraded":"2023-10-06T03:03:29.711510+0000","last_fullsized":"2023-10-06T03:03:29.711510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:59.402917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725934+0000","last_change":"2023-10-06T03:03:29.718707+0000","last_active":"2023-10-06T03:03:29.725934+0000","last_peered":"2023-10-06T03:03:29.725934+0000","last_clean":"2023-10-06T03:03:29.725934+0000","last_became_active":"2023-10-06T03:03:29.718416+0000","last_became_peered":"2023-10-06T03:03:29.718416+0000","last_unstale":"2023-10-06T03:03:29.725934+0000","last_undegraded":"2023-10-06T03:03:29.725934+0000","last_fullsized":"2023-10-06T03:03:29.725934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:30.417220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.710081+0000","last_change":"2023-10-06T03:03:28.706921+0000","last_active":"2023-10-06T03:03:29.710081+0000","last_peered":"2023-10-06T03:03:29.710081+0000","last_clean":"2023-10-06T03:03:29.710081+0000","last_became_active":"2023-10-06T03:03:28.706676+0000","last_became_peered":"2023-10-06T03:03:28.706676+0000","last_unstale":"2023-10-06T03:03:29.710081+0000","last_undegraded":"2023-10-06T03:03:29.710081+0000","last_fullsized":"2023-10-06T03:03:29.710081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.778082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732255+0000","last_change":"2023-10-06T03:03:29.729338+0000","last_active":"2023-10-06T03:03:29.732255+0000","last_peered":"2023-10-06T03:03:29.732255+0000","last_clean":"2023-10-06T03:03:29.732255+0000","last_became_active":"2023-10-06T03:03:29.729096+0000","last_became_peered":"2023-10-06T03:03:29.729096+0000","last_unstale":"2023-10-06T03:03:29.732255+0000","last_undegraded":"2023-10-06T03:03:29.732255+0000","last_fullsized":"2023-10-06T03:03:29.732255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:48.803431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.714450+0000","last_change":"2023-10-06T03:03:28.707082+0000","last_active":"2023-10-06T03:03:29.714450+0000","last_peered":"2023-10-06T03:03:29.714450+0000","last_clean":"2023-10-06T03:03:29.714450+0000","last_became_active":"2023-10-06T03:03:28.706254+0000","last_became_peered":"2023-10-06T03:03:28.706254+0000","last_unstale":"2023-10-06T03:03:29.714450+0000","last_undegraded":"2023-10-06T03:03:29.714450+0000","last_fullsized":"2023-10-06T03:03:29.714450+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:08.966317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728619+0000","last_change":"2023-10-06T03:03:29.724012+0000","last_active":"2023-10-06T03:03:29.728619+0000","last_peered":"2023-10-06T03:03:29.728619+0000","last_clean":"2023-10-06T03:03:29.728619+0000","last_became_active":"2023-10-06T03:03:29.723782+0000","last_became_peered":"2023-10-06T03:03:29.723782+0000","last_unstale":"2023-10-06T03:03:29.728619+0000","last_undegraded":"2023-10-06T03:03:29.728619+0000","last_fullsized":"2023-10-06T03:03:29.728619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:10.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.709773+0000","last_change":"2023-10-06T03:03:28.699446+0000","last_active":"2023-10-06T03:03:29.709773+0000","last_peered":"2023-10-06T03:03:29.709773+0000","last_clean":"2023-10-06T03:03:29.709773+0000","last_became_active":"2023-10-06T03:03:28.699197+0000","last_became_peered":"2023-10-06T03:03:28.699197+0000","last_unstale":"2023-10-06T03:03:29.709773+0000","last_undegraded":"2023-10-06T03:03:29.709773+0000","last_fullsized":"2023-10-06T03:03:29.709773+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:45.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722644+0000","last_change":"2023-10-06T03:03:29.718640+0000","last_active":"2023-10-06T03:03:29.722644+0000","last_peered":"2023-10-06T03:03:29.722644+0000","last_clean":"2023-10-06T03:03:29.722644+0000","last_became_active":"2023-10-06T03:03:29.718359+0000","last_became_peered":"2023-10-06T03:03:29.718359+0000","last_unstale":"2023-10-06T03:03:29.722644+0000","last_undegraded":"2023-10-06T03:03:29.722644+0000","last_fullsized":"2023-10-06T03:03:29.722644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:28.073193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731275+0000","last_change":"2023-10-06T03:03:28.705759+0000","last_active":"2023-10-06T03:03:29.731275+0000","last_peered":"2023-10-06T03:03:29.731275+0000","last_clean":"2023-10-06T03:03:29.731275+0000","last_became_active":"2023-10-06T03:03:28.705443+0000","last_became_peered":"2023-10-06T03:03:28.705443+0000","last_unstale":"2023-10-06T03:03:29.731275+0000","last_undegraded":"2023-10-06T03:03:29.731275+0000","last_fullsized":"2023-10-06T03:03:29.731275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.713938+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:29.713938+0000","last_peered":"2023-10-06T03:03:29.713938+0000","last_clean":"2023-10-06T03:03:29.713938+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:29.713938+0000","last_undegraded":"2023-10-06T03:03:29.713938+0000","last_fullsized":"2023-10-06T03:03:29.713938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.720253+0000","last_change":"2023-10-06T03:03:29.717723+0000","last_active":"2023-10-06T03:03:29.720253+0000","last_peered":"2023-10-06T03:03:29.720253+0000","last_clean":"2023-10-06T03:03:29.720253+0000","last_became_active":"2023-10-06T03:03:29.717452+0000","last_became_peered":"2023-10-06T03:03:29.717452+0000","last_unstale":"2023-10-06T03:03:29.720253+0000","last_undegraded":"2023-10-06T03:03:29.720253+0000","last_fullsized":"2023-10-06T03:03:29.720253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732353+0000","last_change":"2023-10-06T03:03:28.709130+0000","last_active":"2023-10-06T03:03:29.732353+0000","last_peered":"2023-10-06T03:03:29.732353+0000","last_clean":"2023-10-06T03:03:29.732353+0000","last_became_active":"2023-10-06T03:03:28.708759+0000","last_became_peered":"2023-10-06T03:03:28.708759+0000","last_unstale":"2023-10-06T03:03:29.732353+0000","last_undegraded":"2023-10-06T03:03:29.732353+0000","last_fullsized":"2023-10-06T03:03:29.732353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.713535+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:29.713535+0000","last_peered":"2023-10-06T03:03:29.713535+0000","last_clean":"2023-10-06T03:03:29.713535+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:29.713535+0000","last_undegraded":"2023-10-06T03:03:29.713535+0000","last_fullsized":"2023-10-06T03:03:29.713535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.724345+0000","last_change":"2023-10-06T03:03:29.718676+0000","last_active":"2023-10-06T03:03:29.724345+0000","last_peered":"2023-10-06T03:03:29.724345+0000","last_clean":"2023-10-06T03:03:29.724345+0000","last_became_active":"2023-10-06T03:03:29.718418+0000","last_became_peered":"2023-10-06T03:03:29.718418+0000","last_unstale":"2023-10-06T03:03:29.724345+0000","last_undegraded":"2023-10-06T03:03:29.724345+0000","last_fullsized":"2023-10-06T03:03:29.724345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:30.688614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711119+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:29.711119+0000","last_peered":"2023-10-06T03:03:29.711119+0000","last_clean":"2023-10-06T03:03:29.711119+0000","last_became_active":"2023-10-06T03:03:28.707489+0000","last_became_peered":"2023-10-06T03:03:28.707489+0000","last_unstale":"2023-10-06T03:03:29.711119+0000","last_undegraded":"2023-10-06T03:03:29.711119+0000","last_fullsized":"2023-10-06T03:03:29.711119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:41.067368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.712577+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:29.712577+0000","last_peered":"2023-10-06T03:03:29.712577+0000","last_clean":"2023-10-06T03:03:29.712577+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:29.712577+0000","last_undegraded":"2023-10-06T03:03:29.712577+0000","last_fullsized":"2023-10-06T03:03:29.712577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731785+0000","last_change":"2023-10-06T03:03:29.728078+0000","last_active":"2023-10-06T03:03:29.731785+0000","last_peered":"2023-10-06T03:03:29.731785+0000","last_clean":"2023-10-06T03:03:29.731785+0000","last_became_active":"2023-10-06T03:03:29.727835+0000","last_became_peered":"2023-10-06T03:03:29.727835+0000","last_unstale":"2023-10-06T03:03:29.731785+0000","last_undegraded":"2023-10-06T03:03:29.731785+0000","last_fullsized":"2023-10-06T03:03:29.731785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:30.688614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.707786+0000","last_change":"2023-10-06T03:03:28.708268+0000","last_active":"2023-10-06T03:03:29.707786+0000","last_peered":"2023-10-06T03:03:29.707786+0000","last_clean":"2023-10-06T03:03:29.707786+0000","last_became_active":"2023-10-06T03:03:28.707997+0000","last_became_peered":"2023-10-06T03:03:28.707997+0000","last_unstale":"2023-10-06T03:03:29.707786+0000","last_undegraded":"2023-10-06T03:03:29.707786+0000","last_fullsized":"2023-10-06T03:03:29.707786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.714294+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:29.714294+0000","last_peered":"2023-10-06T03:03:29.714294+0000","last_clean":"2023-10-06T03:03:29.714294+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:29.714294+0000","last_undegraded":"2023-10-06T03:03:29.714294+0000","last_fullsized":"2023-10-06T03:03:29.714294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728614+0000","last_change":"2023-10-06T03:03:29.722687+0000","last_active":"2023-10-06T03:03:29.728614+0000","last_peered":"2023-10-06T03:03:29.728614+0000","last_clean":"2023-10-06T03:03:29.728614+0000","last_became_active":"2023-10-06T03:03:29.722429+0000","last_became_peered":"2023-10-06T03:03:29.722429+0000","last_unstale":"2023-10-06T03:03:29.728614+0000","last_undegraded":"2023-10-06T03:03:29.728614+0000","last_fullsized":"2023-10-06T03:03:29.728614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.707656+0000","last_change":"2023-10-06T03:03:28.706968+0000","last_active":"2023-10-06T03:03:29.707656+0000","last_peered":"2023-10-06T03:03:29.707656+0000","last_clean":"2023-10-06T03:03:29.707656+0000","last_became_active":"2023-10-06T03:03:28.706720+0000","last_became_peered":"2023-10-06T03:03:28.706720+0000","last_unstale":"2023-10-06T03:03:29.707656+0000","last_undegraded":"2023-10-06T03:03:29.707656+0000","last_fullsized":"2023-10-06T03:03:29.707656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.714381+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:29.714381+0000","last_peered":"2023-10-06T03:03:29.714381+0000","last_clean":"2023-10-06T03:03:29.714381+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:29.714381+0000","last_undegraded":"2023-10-06T03:03:29.714381+0000","last_fullsized":"2023-10-06T03:03:29.714381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708229+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:29.708229+0000","last_peered":"2023-10-06T03:03:29.708229+0000","last_clean":"2023-10-06T03:03:29.708229+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:29.708229+0000","last_undegraded":"2023-10-06T03:03:29.708229+0000","last_fullsized":"2023-10-06T03:03:29.708229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726284+0000","last_change":"2023-10-06T03:03:29.720101+0000","last_active":"2023-10-06T03:03:29.726284+0000","last_peered":"2023-10-06T03:03:29.726284+0000","last_clean":"2023-10-06T03:03:29.726284+0000","last_became_active":"2023-10-06T03:03:29.719853+0000","last_became_peered":"2023-10-06T03:03:29.719853+0000","last_unstale":"2023-10-06T03:03:29.726284+0000","last_undegraded":"2023-10-06T03:03:29.726284+0000","last_fullsized":"2023-10-06T03:03:29.726284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:32.855430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708615+0000","last_change":"2023-10-06T03:03:28.707199+0000","last_active":"2023-10-06T03:03:29.708615+0000","last_peered":"2023-10-06T03:03:29.708615+0000","last_clean":"2023-10-06T03:03:29.708615+0000","last_became_active":"2023-10-06T03:03:28.706953+0000","last_became_peered":"2023-10-06T03:03:28.706953+0000","last_unstale":"2023-10-06T03:03:29.708615+0000","last_undegraded":"2023-10-06T03:03:29.708615+0000","last_fullsized":"2023-10-06T03:03:29.708615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.709678+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:29.709678+0000","last_peered":"2023-10-06T03:03:29.709678+0000","last_clean":"2023-10-06T03:03:29.709678+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:29.709678+0000","last_undegraded":"2023-10-06T03:03:29.709678+0000","last_fullsized":"2023-10-06T03:03:29.709678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728618+0000","last_change":"2023-10-06T03:03:29.725709+0000","last_active":"2023-10-06T03:03:29.728618+0000","last_peered":"2023-10-06T03:03:29.728618+0000","last_clean":"2023-10-06T03:03:29.728618+0000","last_became_active":"2023-10-06T03:03:29.720819+0000","last_became_peered":"2023-10-06T03:03:29.720819+0000","last_unstale":"2023-10-06T03:03:29.728618+0000","last_undegraded":"2023-10-06T03:03:29.728618+0000","last_fullsized":"2023-10-06T03:03:29.728618+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736550+0000","last_change":"2023-10-06T03:03:28.704304+0000","last_active":"2023-10-06T03:03:29.736550+0000","last_peered":"2023-10-06T03:03:29.736550+0000","last_clean":"2023-10-06T03:03:29.736550+0000","last_became_active":"2023-10-06T03:03:28.704042+0000","last_became_peered":"2023-10-06T03:03:28.704042+0000","last_unstale":"2023-10-06T03:03:29.736550+0000","last_undegraded":"2023-10-06T03:03:29.736550+0000","last_fullsized":"2023-10-06T03:03:29.736550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708146+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:29.708146+0000","last_peered":"2023-10-06T03:03:29.708146+0000","last_clean":"2023-10-06T03:03:29.708146+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:29.708146+0000","last_undegraded":"2023-10-06T03:03:29.708146+0000","last_fullsized":"2023-10-06T03:03:29.708146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722617+0000","last_change":"2023-10-06T03:03:29.718087+0000","last_active":"2023-10-06T03:03:29.722617+0000","last_peered":"2023-10-06T03:03:29.722617+0000","last_clean":"2023-10-06T03:03:29.722617+0000","last_became_active":"2023-10-06T03:03:29.717875+0000","last_became_peered":"2023-10-06T03:03:29.717875+0000","last_unstale":"2023-10-06T03:03:29.722617+0000","last_undegraded":"2023-10-06T03:03:29.722617+0000","last_fullsized":"2023-10-06T03:03:29.722617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:52.287277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730484+0000","last_change":"2023-10-06T03:03:28.704994+0000","last_active":"2023-10-06T03:03:29.730484+0000","last_peered":"2023-10-06T03:03:29.730484+0000","last_clean":"2023-10-06T03:03:29.730484+0000","last_became_active":"2023-10-06T03:03:28.704702+0000","last_became_peered":"2023-10-06T03:03:28.704702+0000","last_unstale":"2023-10-06T03:03:29.730484+0000","last_undegraded":"2023-10-06T03:03:29.730484+0000","last_fullsized":"2023-10-06T03:03:29.730484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:18.494652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711197+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:29.711197+0000","last_peered":"2023-10-06T03:03:29.711197+0000","last_clean":"2023-10-06T03:03:29.711197+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:29.711197+0000","last_undegraded":"2023-10-06T03:03:29.711197+0000","last_fullsized":"2023-10-06T03:03:29.711197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298656,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073184,"statfs":{"total":96636764160,"available":96330940416,"internally_reserved":0,"allocated":1400832,"data_stored":951449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.38800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71699999999999997,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.499}]},{"osd":1,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.60899999999999999,"5min":0.875,"15min":0.875},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":0.999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64900000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68999999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.80200000000000005,"5min":1.026,"15min":1.026},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.75700000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76900000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77700000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":362595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.504}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":361171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32700000000000001},"min":{"1min":0.23799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.33500000000000002},"min":{"1min":0.251,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51400000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.309},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.33800000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.45200000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33600000000000002,"15min":0.33900000000000002},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.42299999999999999,"5min":0.628,"15min":0.628},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54900000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":1.115,"15min":1.115},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78500000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51000000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85799999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81999999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81299999999999994,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.443},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79100000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298652,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073188,"statfs":{"total":96636764160,"available":96330944512,"internally_reserved":0,"allocated":1396736,"data_stored":951427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.31,"15min":0.311},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.31},{"interface":"front","average":{"1min":0.316,"5min":0.32800000000000001,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.25900000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34799999999999998,"15min":0.34699999999999998},"min":{"1min":0.27000000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38}]},{"osd":4,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":0.871,"15min":0.871},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77100000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84699999999999998,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80800000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83899999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91200000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.629,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":361773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.41699999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32600000000000001},"min":{"1min":0.245,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51100000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81100000000000005,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84999999999999998,"5min":1.903,"15min":1.903},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":361613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.309,"15min":0.311},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.38300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.246},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32700000000000001,"15min":0.32800000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.39800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.42399999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35599999999999998},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.318}]},{"osd":4,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.61699999999999999,"15min":0.58199999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70699999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69499999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":362699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.78600000000000003,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72399999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67700000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":3,"15min":3},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.84999999999999998,"15min":0.89800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":10.084,"15min":10.084},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70799999999999996,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.71699999999999997,"15min":0.74099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68899999999999995,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.64700000000000002,"5min":1.54,"15min":1.54},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.439},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.63900000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76500000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":361093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78200000000000003,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61899999999999999,"15min":0.63100000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.73199999999999998,"5min":2.371,"15min":2.371},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69199999999999995,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64500000000000002,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64600000000000002,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65400000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.61299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67100000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.48799999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73999999999999999,"5min":1.052,"15min":1.052},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71999999999999997,"5min":1.175,"15min":1.175},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.44600000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71099999999999997,"5min":1.145,"15min":1.145},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:38.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:03:38.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.406+0000 7f9f3a935700 1 Processor -- start 2023-10-06T03:03:38.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.406+0000 7f9f3a935700 1 -- start start 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.406+0000 7f9f3a935700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 0x7f9f3415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.406+0000 7f9f3a935700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3415a160 con 0x7f9f34159c60 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.406+0000 7f9f3a935700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f340d13a0 con 0x7f9f34069560 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.406+0000 7f9f3a935700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415e140 con 0x7f9f34159890 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f337fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9f34159c60 0x7f9f3415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59566/0 (socket says 172.21.15.92:59566) 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f337fe700 1 -- 172.21.15.92:0/3398712137 learned_addr learned my addr 172.21.15.92:0/3398712137 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:38.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f33fff700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 0x7f9f3415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697091999 0 0) 0x7f9f3415a160 con 0x7f9f34159c60 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f1c003570 con 0x7f9f34159c60 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2375839370 0 0) 0x7f9f1c003570 con 0x7f9f34159c60 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 msgr2=0x7f9f3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 0x7f9f3415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 >> v1:172.21.15.136:6789/0 conn(0x7f9f34069560 legacy=0x7f9f340ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3415a3d0 con 0x7f9f34159c60 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f9f28001470 con 0x7f9f34159c60 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 >> v1:172.21.15.92:6789/0 conn(0x7f9f34159c60 legacy=0x7f9f3415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f1c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c004810 0x7f9f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 0x7f9f1c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415e140 con 0x7f9f1c003a90 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c00bb60 con 0x7f9f1c004810 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c00bd20 con 0x7f9f34159890 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.407+0000 7f9f33fff700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f1c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f337fe700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c004810 0x7f9f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f38ed2700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 0x7f9f1c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f33fff700 1 -- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 msgr2=0x7f9f1c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f33fff700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f34159890 0x7f9f1c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f33fff700 1 -- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c004810 msgr2=0x7f9f1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f33fff700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c004810 0x7f9f1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f33fff700 1 -- 172.21.15.92:0/3398712137 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c003570 con 0x7f9f1c003a90 2023-10-06T03:03:38.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f33fff700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f1c003ea0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009c00 tx=0x7f9f240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0157b5ec45abcce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400e710 con 0x7f9f1c003a90 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2400e870 con 0x7f9f1c003a90 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.408+0000 7f9f32ffd700 1 -- 172.21.15.92:0/3398712137 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400ec30 con 0x7f9f1c003a90 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f3a935700 1 -- 172.21.15.92:0/3398712137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 msgr2=0x7f9f1c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f3a935700 1 --2- 172.21.15.92:0/3398712137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f1c003ea0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009c00 tx=0x7f9f240091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f38ed2700 1 -- 172.21.15.92:0/3398712137 reap_dead start 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f3a935700 1 -- 172.21.15.92:0/3398712137 shutdown_connections 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f3a935700 1 -- 172.21.15.92:0/3398712137 >> 172.21.15.92:0/3398712137 conn(0x7f9f34068e20 msgr2=0x7f9f340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f3a935700 1 -- 172.21.15.92:0/3398712137 shutdown_connections 2023-10-06T03:03:38.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.409+0000 7f9f3a935700 1 -- 172.21.15.92:0/3398712137 wait complete. 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.410+0000 7f9f3a935700 1 Processor -- start 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.410+0000 7f9f3a935700 1 -- start start 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f3a935700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f340cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f3a935700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f34152460 0x7f9f34169800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f3a935700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f341528d0 0x7f9f34169d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f3a935700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3416a280 con 0x7f9f1c003a90 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f3a935700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3414afd0 con 0x7f9f34152460 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f3a935700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3414b2b0 con 0x7f9f341528d0 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f340cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f340cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50356/0 (socket says 172.21.15.92:50356) 2023-10-06T03:03:38.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 -- 172.21.15.92:0/2226548974 learned_addr learned my addr 172.21.15.92:0/2226548974 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f337fe700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f34152460 0x7f9f34169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f38ed2700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f341528d0 0x7f9f34169d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 -- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f341528d0 msgr2=0x7f9f34169d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f341528d0 0x7f9f34169d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 -- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f34152460 msgr2=0x7f9f34169800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f34152460 0x7f9f34169800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 -- 172.21.15.92:0/2226548974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3414b590 con 0x7f9f1c003a90 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.411+0000 7f9f33fff700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f340cf380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400e4b0 tx=0x7f9f2401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.412+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24022070 con 0x7f9f1c003a90 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.412+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2400e9e0 con 0x7f9f1c003a90 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.412+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3414b870 con 0x7f9f1c003a90 2023-10-06T03:03:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.412+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24027470 con 0x7f9f1c003a90 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.412+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f340cf8c0 con 0x7f9f1c003a90 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.413+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3404c3c0 con 0x7f9f1c003a90 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.413+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9f24027880 con 0x7f9f1c003a90 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.413+0000 7f9f1bfff700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f20066fc0 0x7f9f20069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.414+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f240861d0 con 0x7f9f1c003a90 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.414+0000 7f9f337fe700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f20066fc0 0x7f9f20069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.414+0000 7f9f337fe700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f20066fc0 0x7f9f20069470 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f28001620 tx=0x7f9f28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.418+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f240533e0 con 0x7f9f1c003a90 2023-10-06T03:03:38.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.601+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f3416a280 con 0x7f9f20066fc0 2023-10-06T03:03:38.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.609+0000 7f9f1bfff700 1 -- 172.21.15.92:0/2226548974 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312021 (secure 0 0 0) 0x7f9f3416a280 con 0x7f9f20066fc0 2023-10-06T03:03:38.614 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f20066fc0 msgr2=0x7f9f20069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f20066fc0 0x7f9f20069470 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f28001620 tx=0x7f9f28009040 comp rx=0 tx=0).stop 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 msgr2=0x7f9f340cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f340cf380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400e4b0 tx=0x7f9f2401a000 comp rx=0 tx=0).stop 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 shutdown_connections 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f20066fc0 0x7f9f20069470 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f1c003a90 0x7f9f340cf380 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f341528d0 0x7f9f34169d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 --2- 172.21.15.92:0/2226548974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f34152460 0x7f9f34169800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 >> 172.21.15.92:0/2226548974 conn(0x7f9f34068e20 msgr2=0x7f9f3415d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 shutdown_connections 2023-10-06T03:03:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.612+0000 7f9f3a935700 1 -- 172.21.15.92:0/2226548974 wait complete. 2023-10-06T03:03:38.618 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:03:38.626 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-10-06T03:03:37.205364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385800,"kb_used_data":7496,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752588920,"statfs":{"total":773094113280,"available":770651054080,"internally_reserved":0,"allocated":7675904,"data_stored":4073820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001694"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730680+0000","last_change":"2023-10-06T03:03:29.723778+0000","last_active":"2023-10-06T03:03:29.730680+0000","last_peered":"2023-10-06T03:03:29.730680+0000","last_clean":"2023-10-06T03:03:29.730680+0000","last_became_active":"2023-10-06T03:03:29.723169+0000","last_became_peered":"2023-10-06T03:03:29.723169+0000","last_unstale":"2023-10-06T03:03:29.730680+0000","last_undegraded":"2023-10-06T03:03:29.730680+0000","last_fullsized":"2023-10-06T03:03:29.730680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728864+0000","last_change":"2023-10-06T03:03:29.718001+0000","last_active":"2023-10-06T03:03:29.728864+0000","last_peered":"2023-10-06T03:03:29.728864+0000","last_clean":"2023-10-06T03:03:29.728864+0000","last_became_active":"2023-10-06T03:03:29.714605+0000","last_became_peered":"2023-10-06T03:03:29.714605+0000","last_unstale":"2023-10-06T03:03:29.728864+0000","last_undegraded":"2023-10-06T03:03:29.728864+0000","last_fullsized":"2023-10-06T03:03:29.728864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732232+0000","last_change":"2023-10-06T03:03:29.728789+0000","last_active":"2023-10-06T03:03:29.732232+0000","last_peered":"2023-10-06T03:03:29.732232+0000","last_clean":"2023-10-06T03:03:29.732232+0000","last_became_active":"2023-10-06T03:03:29.728534+0000","last_became_peered":"2023-10-06T03:03:29.728534+0000","last_unstale":"2023-10-06T03:03:29.732232+0000","last_undegraded":"2023-10-06T03:03:29.732232+0000","last_fullsized":"2023-10-06T03:03:29.732232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:30.417220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727754+0000","last_change":"2023-10-06T03:03:29.720422+0000","last_active":"2023-10-06T03:03:29.727754+0000","last_peered":"2023-10-06T03:03:29.727754+0000","last_clean":"2023-10-06T03:03:29.727754+0000","last_became_active":"2023-10-06T03:03:29.720058+0000","last_became_peered":"2023-10-06T03:03:29.720058+0000","last_unstale":"2023-10-06T03:03:29.727754+0000","last_undegraded":"2023-10-06T03:03:29.727754+0000","last_fullsized":"2023-10-06T03:03:29.727754+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.729436+0000","last_change":"2023-10-06T03:03:29.719553+0000","last_active":"2023-10-06T03:03:29.729436+0000","last_peered":"2023-10-06T03:03:29.729436+0000","last_clean":"2023-10-06T03:03:29.729436+0000","last_became_active":"2023-10-06T03:03:29.719334+0000","last_became_peered":"2023-10-06T03:03:29.719334+0000","last_unstale":"2023-10-06T03:03:29.729436+0000","last_undegraded":"2023-10-06T03:03:29.729436+0000","last_fullsized":"2023-10-06T03:03:29.729436+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:44:06.879475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736426+0000","last_change":"2023-10-06T03:03:29.731299+0000","last_active":"2023-10-06T03:03:29.736426+0000","last_peered":"2023-10-06T03:03:29.736426+0000","last_clean":"2023-10-06T03:03:29.736426+0000","last_became_active":"2023-10-06T03:03:29.727621+0000","last_became_peered":"2023-10-06T03:03:29.727621+0000","last_unstale":"2023-10-06T03:03:29.736426+0000","last_undegraded":"2023-10-06T03:03:29.736426+0000","last_fullsized":"2023-10-06T03:03:29.736426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725723+0000","last_change":"2023-10-06T03:03:29.718340+0000","last_active":"2023-10-06T03:03:29.725723+0000","last_peered":"2023-10-06T03:03:29.725723+0000","last_clean":"2023-10-06T03:03:29.725723+0000","last_became_active":"2023-10-06T03:03:29.718072+0000","last_became_peered":"2023-10-06T03:03:29.718072+0000","last_unstale":"2023-10-06T03:03:29.725723+0000","last_undegraded":"2023-10-06T03:03:29.725723+0000","last_fullsized":"2023-10-06T03:03:29.725723+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:42.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731440+0000","last_change":"2023-10-06T03:03:29.724404+0000","last_active":"2023-10-06T03:03:29.731440+0000","last_peered":"2023-10-06T03:03:29.731440+0000","last_clean":"2023-10-06T03:03:29.731440+0000","last_became_active":"2023-10-06T03:03:29.724159+0000","last_became_peered":"2023-10-06T03:03:29.724159+0000","last_unstale":"2023-10-06T03:03:29.731440+0000","last_undegraded":"2023-10-06T03:03:29.731440+0000","last_fullsized":"2023-10-06T03:03:29.731440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:20.821206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726859+0000","last_change":"2023-10-06T03:03:29.721567+0000","last_active":"2023-10-06T03:03:29.726859+0000","last_peered":"2023-10-06T03:03:29.726859+0000","last_clean":"2023-10-06T03:03:29.726859+0000","last_became_active":"2023-10-06T03:03:29.720653+0000","last_became_peered":"2023-10-06T03:03:29.720653+0000","last_unstale":"2023-10-06T03:03:29.726859+0000","last_undegraded":"2023-10-06T03:03:29.726859+0000","last_fullsized":"2023-10-06T03:03:29.726859+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:44.423282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722753+0000","last_change":"2023-10-06T03:03:29.717201+0000","last_active":"2023-10-06T03:03:29.722753+0000","last_peered":"2023-10-06T03:03:29.722753+0000","last_clean":"2023-10-06T03:03:29.722753+0000","last_became_active":"2023-10-06T03:03:29.716949+0000","last_became_peered":"2023-10-06T03:03:29.716949+0000","last_unstale":"2023-10-06T03:03:29.722753+0000","last_undegraded":"2023-10-06T03:03:29.722753+0000","last_fullsized":"2023-10-06T03:03:29.722753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727527+0000","last_change":"2023-10-06T03:03:29.712576+0000","last_active":"2023-10-06T03:03:29.727527+0000","last_peered":"2023-10-06T03:03:29.727527+0000","last_clean":"2023-10-06T03:03:29.727527+0000","last_became_active":"2023-10-06T03:03:29.712317+0000","last_became_peered":"2023-10-06T03:03:29.712317+0000","last_unstale":"2023-10-06T03:03:29.727527+0000","last_undegraded":"2023-10-06T03:03:29.727527+0000","last_fullsized":"2023-10-06T03:03:29.727527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:37.515635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722817+0000","last_change":"2023-10-06T03:03:29.716677+0000","last_active":"2023-10-06T03:03:29.722817+0000","last_peered":"2023-10-06T03:03:29.722817+0000","last_clean":"2023-10-06T03:03:29.722817+0000","last_became_active":"2023-10-06T03:03:29.716387+0000","last_became_peered":"2023-10-06T03:03:29.716387+0000","last_unstale":"2023-10-06T03:03:29.722817+0000","last_undegraded":"2023-10-06T03:03:29.722817+0000","last_fullsized":"2023-10-06T03:03:29.722817+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731906+0000","last_change":"2023-10-06T03:03:29.722034+0000","last_active":"2023-10-06T03:03:29.731906+0000","last_peered":"2023-10-06T03:03:29.731906+0000","last_clean":"2023-10-06T03:03:29.731906+0000","last_became_active":"2023-10-06T03:03:29.721750+0000","last_became_peered":"2023-10-06T03:03:29.721750+0000","last_unstale":"2023-10-06T03:03:29.731906+0000","last_undegraded":"2023-10-06T03:03:29.731906+0000","last_fullsized":"2023-10-06T03:03:29.731906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:31.502226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736461+0000","last_change":"2023-10-06T03:03:29.731966+0000","last_active":"2023-10-06T03:03:29.736461+0000","last_peered":"2023-10-06T03:03:29.736461+0000","last_clean":"2023-10-06T03:03:29.736461+0000","last_became_active":"2023-10-06T03:03:29.727297+0000","last_became_peered":"2023-10-06T03:03:29.727297+0000","last_unstale":"2023-10-06T03:03:29.736461+0000","last_undegraded":"2023-10-06T03:03:29.736461+0000","last_fullsized":"2023-10-06T03:03:29.736461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730867+0000","last_change":"2023-10-06T03:03:29.726517+0000","last_active":"2023-10-06T03:03:29.730867+0000","last_peered":"2023-10-06T03:03:29.730867+0000","last_clean":"2023-10-06T03:03:29.730867+0000","last_became_active":"2023-10-06T03:03:29.726030+0000","last_became_peered":"2023-10-06T03:03:29.726030+0000","last_unstale":"2023-10-06T03:03:29.730867+0000","last_undegraded":"2023-10-06T03:03:29.730867+0000","last_fullsized":"2023-10-06T03:03:29.730867+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730357+0000","last_change":"2023-10-06T03:03:29.723878+0000","last_active":"2023-10-06T03:03:29.730357+0000","last_peered":"2023-10-06T03:03:29.730357+0000","last_clean":"2023-10-06T03:03:29.730357+0000","last_became_active":"2023-10-06T03:03:29.723154+0000","last_became_peered":"2023-10-06T03:03:29.723154+0000","last_unstale":"2023-10-06T03:03:29.730357+0000","last_undegraded":"2023-10-06T03:03:29.730357+0000","last_fullsized":"2023-10-06T03:03:29.730357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.719190+0000","last_change":"2023-10-06T03:03:29.716857+0000","last_active":"2023-10-06T03:03:29.719190+0000","last_peered":"2023-10-06T03:03:29.719190+0000","last_clean":"2023-10-06T03:03:29.719190+0000","last_became_active":"2023-10-06T03:03:29.716624+0000","last_became_peered":"2023-10-06T03:03:29.716624+0000","last_unstale":"2023-10-06T03:03:29.719190+0000","last_undegraded":"2023-10-06T03:03:29.719190+0000","last_fullsized":"2023-10-06T03:03:29.719190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:33.517523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728951+0000","last_change":"2023-10-06T03:03:29.722266+0000","last_active":"2023-10-06T03:03:29.728951+0000","last_peered":"2023-10-06T03:03:29.728951+0000","last_clean":"2023-10-06T03:03:29.728951+0000","last_became_active":"2023-10-06T03:03:29.722042+0000","last_became_peered":"2023-10-06T03:03:29.722042+0000","last_unstale":"2023-10-06T03:03:29.728951+0000","last_undegraded":"2023-10-06T03:03:29.728951+0000","last_fullsized":"2023-10-06T03:03:29.728951+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:42.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.724572+0000","last_change":"2023-10-06T03:03:29.719109+0000","last_active":"2023-10-06T03:03:29.724572+0000","last_peered":"2023-10-06T03:03:29.724572+0000","last_clean":"2023-10-06T03:03:29.724572+0000","last_became_active":"2023-10-06T03:03:29.718855+0000","last_became_peered":"2023-10-06T03:03:29.718855+0000","last_unstale":"2023-10-06T03:03:29.724572+0000","last_undegraded":"2023-10-06T03:03:29.724572+0000","last_fullsized":"2023-10-06T03:03:29.724572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.729932+0000","last_change":"2023-10-06T03:03:29.721942+0000","last_active":"2023-10-06T03:03:29.729932+0000","last_peered":"2023-10-06T03:03:29.729932+0000","last_clean":"2023-10-06T03:03:29.729932+0000","last_became_active":"2023-10-06T03:03:29.721655+0000","last_became_peered":"2023-10-06T03:03:29.721655+0000","last_unstale":"2023-10-06T03:03:29.729932+0000","last_undegraded":"2023-10-06T03:03:29.729932+0000","last_fullsized":"2023-10-06T03:03:29.729932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:24.061645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730489+0000","last_change":"2023-10-06T03:03:29.727560+0000","last_active":"2023-10-06T03:03:29.730489+0000","last_peered":"2023-10-06T03:03:29.730489+0000","last_clean":"2023-10-06T03:03:29.730489+0000","last_became_active":"2023-10-06T03:03:29.726841+0000","last_became_peered":"2023-10-06T03:03:29.726841+0000","last_unstale":"2023-10-06T03:03:29.730489+0000","last_undegraded":"2023-10-06T03:03:29.730489+0000","last_fullsized":"2023-10-06T03:03:29.730489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730336+0000","last_change":"2023-10-06T03:03:29.722110+0000","last_active":"2023-10-06T03:03:29.730336+0000","last_peered":"2023-10-06T03:03:29.730336+0000","last_clean":"2023-10-06T03:03:29.730336+0000","last_became_active":"2023-10-06T03:03:29.721847+0000","last_became_peered":"2023-10-06T03:03:29.721847+0000","last_unstale":"2023-10-06T03:03:29.730336+0000","last_undegraded":"2023-10-06T03:03:29.730336+0000","last_fullsized":"2023-10-06T03:03:29.730336+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:33.599591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.723179+0000","last_change":"2023-10-06T03:03:29.718942+0000","last_active":"2023-10-06T03:03:29.723179+0000","last_peered":"2023-10-06T03:03:29.723179+0000","last_clean":"2023-10-06T03:03:29.723179+0000","last_became_active":"2023-10-06T03:03:29.718682+0000","last_became_peered":"2023-10-06T03:03:29.718682+0000","last_unstale":"2023-10-06T03:03:29.723179+0000","last_undegraded":"2023-10-06T03:03:29.723179+0000","last_fullsized":"2023-10-06T03:03:29.723179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727957+0000","last_change":"2023-10-06T03:03:29.716417+0000","last_active":"2023-10-06T03:03:29.727957+0000","last_peered":"2023-10-06T03:03:29.727957+0000","last_clean":"2023-10-06T03:03:29.727957+0000","last_became_active":"2023-10-06T03:03:29.716136+0000","last_became_peered":"2023-10-06T03:03:29.716136+0000","last_unstale":"2023-10-06T03:03:29.727957+0000","last_undegraded":"2023-10-06T03:03:29.727957+0000","last_fullsized":"2023-10-06T03:03:29.727957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728816+0000","last_change":"2023-10-06T03:03:29.725908+0000","last_active":"2023-10-06T03:03:29.728816+0000","last_peered":"2023-10-06T03:03:29.728816+0000","last_clean":"2023-10-06T03:03:29.728816+0000","last_became_active":"2023-10-06T03:03:29.720807+0000","last_became_peered":"2023-10-06T03:03:29.720807+0000","last_unstale":"2023-10-06T03:03:29.728816+0000","last_undegraded":"2023-10-06T03:03:29.728816+0000","last_fullsized":"2023-10-06T03:03:29.728816+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.717426+0000","last_change":"2023-10-06T03:03:29.711792+0000","last_active":"2023-10-06T03:03:29.717426+0000","last_peered":"2023-10-06T03:03:29.717426+0000","last_clean":"2023-10-06T03:03:29.717426+0000","last_became_active":"2023-10-06T03:03:29.711514+0000","last_became_peered":"2023-10-06T03:03:29.711514+0000","last_unstale":"2023-10-06T03:03:29.717426+0000","last_undegraded":"2023-10-06T03:03:29.717426+0000","last_fullsized":"2023-10-06T03:03:29.717426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728591+0000","last_change":"2023-10-06T03:03:29.722633+0000","last_active":"2023-10-06T03:03:29.728591+0000","last_peered":"2023-10-06T03:03:29.728591+0000","last_clean":"2023-10-06T03:03:29.728591+0000","last_became_active":"2023-10-06T03:03:29.722331+0000","last_became_peered":"2023-10-06T03:03:29.722331+0000","last_unstale":"2023-10-06T03:03:29.728591+0000","last_undegraded":"2023-10-06T03:03:29.728591+0000","last_fullsized":"2023-10-06T03:03:29.728591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:32.267883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.719760+0000","last_change":"2023-10-06T03:03:29.716929+0000","last_active":"2023-10-06T03:03:29.719760+0000","last_peered":"2023-10-06T03:03:29.719760+0000","last_clean":"2023-10-06T03:03:29.719760+0000","last_became_active":"2023-10-06T03:03:29.716689+0000","last_became_peered":"2023-10-06T03:03:29.716689+0000","last_unstale":"2023-10-06T03:03:29.719760+0000","last_undegraded":"2023-10-06T03:03:29.719760+0000","last_fullsized":"2023-10-06T03:03:29.719760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:43:26.246835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727929+0000","last_change":"2023-10-06T03:03:29.716732+0000","last_active":"2023-10-06T03:03:29.727929+0000","last_peered":"2023-10-06T03:03:29.727929+0000","last_clean":"2023-10-06T03:03:29.727929+0000","last_became_active":"2023-10-06T03:03:29.716488+0000","last_became_peered":"2023-10-06T03:03:29.716488+0000","last_unstale":"2023-10-06T03:03:29.727929+0000","last_undegraded":"2023-10-06T03:03:29.727929+0000","last_fullsized":"2023-10-06T03:03:29.727929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:46.103255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732766+0000","last_change":"2023-10-06T03:03:29.729334+0000","last_active":"2023-10-06T03:03:29.732766+0000","last_peered":"2023-10-06T03:03:29.732766+0000","last_clean":"2023-10-06T03:03:29.732766+0000","last_became_active":"2023-10-06T03:03:29.728999+0000","last_became_peered":"2023-10-06T03:03:29.728999+0000","last_unstale":"2023-10-06T03:03:29.732766+0000","last_undegraded":"2023-10-06T03:03:29.732766+0000","last_fullsized":"2023-10-06T03:03:29.732766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731004+0000","last_change":"2023-10-06T03:03:29.726465+0000","last_active":"2023-10-06T03:03:29.731004+0000","last_peered":"2023-10-06T03:03:29.731004+0000","last_clean":"2023-10-06T03:03:29.731004+0000","last_became_active":"2023-10-06T03:03:29.726231+0000","last_became_peered":"2023-10-06T03:03:29.726231+0000","last_unstale":"2023-10-06T03:03:29.731004+0000","last_undegraded":"2023-10-06T03:03:29.731004+0000","last_fullsized":"2023-10-06T03:03:29.731004+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:44.152122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730336+0000","last_change":"2023-10-06T03:03:29.723868+0000","last_active":"2023-10-06T03:03:29.730336+0000","last_peered":"2023-10-06T03:03:29.730336+0000","last_clean":"2023-10-06T03:03:29.730336+0000","last_became_active":"2023-10-06T03:03:29.722945+0000","last_became_peered":"2023-10-06T03:03:29.722945+0000","last_unstale":"2023-10-06T03:03:29.730336+0000","last_undegraded":"2023-10-06T03:03:29.730336+0000","last_fullsized":"2023-10-06T03:03:29.730336+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728887+0000","last_change":"2023-10-06T03:03:29.721810+0000","last_active":"2023-10-06T03:03:29.728887+0000","last_peered":"2023-10-06T03:03:29.728887+0000","last_clean":"2023-10-06T03:03:29.728887+0000","last_became_active":"2023-10-06T03:03:29.721567+0000","last_became_peered":"2023-10-06T03:03:29.721567+0000","last_unstale":"2023-10-06T03:03:29.728887+0000","last_undegraded":"2023-10-06T03:03:29.728887+0000","last_fullsized":"2023-10-06T03:03:29.728887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:29.446863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"65'4","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.734684+0000","last_change":"2023-10-06T03:03:28.705118+0000","last_active":"2023-10-06T03:03:29.734684+0000","last_peered":"2023-10-06T03:03:29.734684+0000","last_clean":"2023-10-06T03:03:29.734684+0000","last_became_active":"2023-10-06T03:03:28.704360+0000","last_became_peered":"2023-10-06T03:03:28.704360+0000","last_unstale":"2023-10-06T03:03:29.734684+0000","last_undegraded":"2023-10-06T03:03:29.734684+0000","last_fullsized":"2023-10-06T03:03:29.734684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:51.855537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733224+0000","last_change":"2023-10-06T03:03:29.728561+0000","last_active":"2023-10-06T03:03:29.733224+0000","last_peered":"2023-10-06T03:03:29.733224+0000","last_clean":"2023-10-06T03:03:29.733224+0000","last_became_active":"2023-10-06T03:03:29.726346+0000","last_became_peered":"2023-10-06T03:03:29.726346+0000","last_unstale":"2023-10-06T03:03:29.733224+0000","last_undegraded":"2023-10-06T03:03:29.733224+0000","last_fullsized":"2023-10-06T03:03:29.733224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.605546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730231+0000","last_change":"2023-10-06T03:03:28.709814+0000","last_active":"2023-10-06T03:03:29.730231+0000","last_peered":"2023-10-06T03:03:29.730231+0000","last_clean":"2023-10-06T03:03:29.730231+0000","last_became_active":"2023-10-06T03:03:28.708830+0000","last_became_peered":"2023-10-06T03:03:28.708830+0000","last_unstale":"2023-10-06T03:03:29.730231+0000","last_undegraded":"2023-10-06T03:03:29.730231+0000","last_fullsized":"2023-10-06T03:03:29.730231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.794167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733217+0000","last_change":"2023-10-06T03:03:29.730127+0000","last_active":"2023-10-06T03:03:29.733217+0000","last_peered":"2023-10-06T03:03:29.733217+0000","last_clean":"2023-10-06T03:03:29.733217+0000","last_became_active":"2023-10-06T03:03:29.729670+0000","last_became_peered":"2023-10-06T03:03:29.729670+0000","last_unstale":"2023-10-06T03:03:29.733217+0000","last_undegraded":"2023-10-06T03:03:29.733217+0000","last_fullsized":"2023-10-06T03:03:29.733217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.110311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732178+0000","last_change":"2023-10-06T03:03:28.710421+0000","last_active":"2023-10-06T03:03:29.732178+0000","last_peered":"2023-10-06T03:03:29.732178+0000","last_clean":"2023-10-06T03:03:29.732178+0000","last_became_active":"2023-10-06T03:03:28.710171+0000","last_became_peered":"2023-10-06T03:03:28.710171+0000","last_unstale":"2023-10-06T03:03:29.732178+0000","last_undegraded":"2023-10-06T03:03:29.732178+0000","last_fullsized":"2023-10-06T03:03:29.732178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727157+0000","last_change":"2023-10-06T03:03:29.722570+0000","last_active":"2023-10-06T03:03:29.727157+0000","last_peered":"2023-10-06T03:03:29.727157+0000","last_clean":"2023-10-06T03:03:29.727157+0000","last_became_active":"2023-10-06T03:03:29.722301+0000","last_became_peered":"2023-10-06T03:03:29.722301+0000","last_unstale":"2023-10-06T03:03:29.727157+0000","last_undegraded":"2023-10-06T03:03:29.727157+0000","last_fullsized":"2023-10-06T03:03:29.727157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733140+0000","last_change":"2023-10-06T03:03:28.711375+0000","last_active":"2023-10-06T03:03:29.733140+0000","last_peered":"2023-10-06T03:03:29.733140+0000","last_clean":"2023-10-06T03:03:29.733140+0000","last_became_active":"2023-10-06T03:03:28.711092+0000","last_became_peered":"2023-10-06T03:03:28.711092+0000","last_unstale":"2023-10-06T03:03:29.733140+0000","last_undegraded":"2023-10-06T03:03:29.733140+0000","last_fullsized":"2023-10-06T03:03:29.733140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:32.012077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727483+0000","last_change":"2023-10-06T03:03:29.716098+0000","last_active":"2023-10-06T03:03:29.727483+0000","last_peered":"2023-10-06T03:03:29.727483+0000","last_clean":"2023-10-06T03:03:29.727483+0000","last_became_active":"2023-10-06T03:03:29.715856+0000","last_became_peered":"2023-10-06T03:03:29.715856+0000","last_unstale":"2023-10-06T03:03:29.727483+0000","last_undegraded":"2023-10-06T03:03:29.727483+0000","last_fullsized":"2023-10-06T03:03:29.727483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:12.936793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730534+0000","last_change":"2023-10-06T03:03:28.709567+0000","last_active":"2023-10-06T03:03:29.730534+0000","last_peered":"2023-10-06T03:03:29.730534+0000","last_clean":"2023-10-06T03:03:29.730534+0000","last_became_active":"2023-10-06T03:03:28.709325+0000","last_became_peered":"2023-10-06T03:03:28.709325+0000","last_unstale":"2023-10-06T03:03:29.730534+0000","last_undegraded":"2023-10-06T03:03:29.730534+0000","last_fullsized":"2023-10-06T03:03:29.730534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:41.961709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727551+0000","last_change":"2023-10-06T03:03:29.715609+0000","last_active":"2023-10-06T03:03:29.727551+0000","last_peered":"2023-10-06T03:03:29.727551+0000","last_clean":"2023-10-06T03:03:29.727551+0000","last_became_active":"2023-10-06T03:03:29.715358+0000","last_became_peered":"2023-10-06T03:03:29.715358+0000","last_unstale":"2023-10-06T03:03:29.727551+0000","last_undegraded":"2023-10-06T03:03:29.727551+0000","last_fullsized":"2023-10-06T03:03:29.727551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:12.612570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736693+0000","last_change":"2023-10-06T03:03:28.697823+0000","last_active":"2023-10-06T03:03:29.736693+0000","last_peered":"2023-10-06T03:03:29.736693+0000","last_clean":"2023-10-06T03:03:29.736693+0000","last_became_active":"2023-10-06T03:03:28.697466+0000","last_became_peered":"2023-10-06T03:03:28.697466+0000","last_unstale":"2023-10-06T03:03:29.736693+0000","last_undegraded":"2023-10-06T03:03:29.736693+0000","last_fullsized":"2023-10-06T03:03:29.736693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727960+0000","last_change":"2023-10-06T03:03:29.725473+0000","last_active":"2023-10-06T03:03:29.727960+0000","last_peered":"2023-10-06T03:03:29.727960+0000","last_clean":"2023-10-06T03:03:29.727960+0000","last_became_active":"2023-10-06T03:03:29.725215+0000","last_became_peered":"2023-10-06T03:03:29.725215+0000","last_unstale":"2023-10-06T03:03:29.727960+0000","last_undegraded":"2023-10-06T03:03:29.727960+0000","last_fullsized":"2023-10-06T03:03:29.727960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:31.004510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.703927+0000","last_change":"2023-10-06T03:03:28.711882+0000","last_active":"2023-10-06T03:03:29.703927+0000","last_peered":"2023-10-06T03:03:29.703927+0000","last_clean":"2023-10-06T03:03:29.703927+0000","last_became_active":"2023-10-06T03:03:28.711670+0000","last_became_peered":"2023-10-06T03:03:28.711670+0000","last_unstale":"2023-10-06T03:03:29.703927+0000","last_undegraded":"2023-10-06T03:03:29.703927+0000","last_fullsized":"2023-10-06T03:03:29.703927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:43.144129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726252+0000","last_change":"2023-10-06T03:03:29.718657+0000","last_active":"2023-10-06T03:03:29.726252+0000","last_peered":"2023-10-06T03:03:29.726252+0000","last_clean":"2023-10-06T03:03:29.726252+0000","last_became_active":"2023-10-06T03:03:29.718357+0000","last_became_peered":"2023-10-06T03:03:29.718357+0000","last_unstale":"2023-10-06T03:03:29.726252+0000","last_undegraded":"2023-10-06T03:03:29.726252+0000","last_fullsized":"2023-10-06T03:03:29.726252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:44.152122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.707806+0000","last_change":"2023-10-06T03:03:28.702078+0000","last_active":"2023-10-06T03:03:29.707806+0000","last_peered":"2023-10-06T03:03:29.707806+0000","last_clean":"2023-10-06T03:03:29.707806+0000","last_became_active":"2023-10-06T03:03:28.701828+0000","last_became_peered":"2023-10-06T03:03:28.701828+0000","last_unstale":"2023-10-06T03:03:29.707806+0000","last_undegraded":"2023-10-06T03:03:29.707806+0000","last_fullsized":"2023-10-06T03:03:29.707806+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:50.672461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728841+0000","last_change":"2023-10-06T03:03:29.725737+0000","last_active":"2023-10-06T03:03:29.728841+0000","last_peered":"2023-10-06T03:03:29.728841+0000","last_clean":"2023-10-06T03:03:29.728841+0000","last_became_active":"2023-10-06T03:03:29.725491+0000","last_became_peered":"2023-10-06T03:03:29.725491+0000","last_unstale":"2023-10-06T03:03:29.728841+0000","last_undegraded":"2023-10-06T03:03:29.728841+0000","last_fullsized":"2023-10-06T03:03:29.728841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711967+0000","last_change":"2023-10-06T03:03:28.709569+0000","last_active":"2023-10-06T03:03:29.711967+0000","last_peered":"2023-10-06T03:03:29.711967+0000","last_clean":"2023-10-06T03:03:29.711967+0000","last_became_active":"2023-10-06T03:03:28.709216+0000","last_became_peered":"2023-10-06T03:03:28.709216+0000","last_unstale":"2023-10-06T03:03:29.711967+0000","last_undegraded":"2023-10-06T03:03:29.711967+0000","last_fullsized":"2023-10-06T03:03:29.711967+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727702+0000","last_change":"2023-10-06T03:03:29.720004+0000","last_active":"2023-10-06T03:03:29.727702+0000","last_peered":"2023-10-06T03:03:29.727702+0000","last_clean":"2023-10-06T03:03:29.727702+0000","last_became_active":"2023-10-06T03:03:29.719665+0000","last_became_peered":"2023-10-06T03:03:29.719665+0000","last_unstale":"2023-10-06T03:03:29.727702+0000","last_undegraded":"2023-10-06T03:03:29.727702+0000","last_fullsized":"2023-10-06T03:03:29.727702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708267+0000","last_change":"2023-10-06T03:03:28.704341+0000","last_active":"2023-10-06T03:03:29.708267+0000","last_peered":"2023-10-06T03:03:29.708267+0000","last_clean":"2023-10-06T03:03:29.708267+0000","last_became_active":"2023-10-06T03:03:28.704116+0000","last_became_peered":"2023-10-06T03:03:28.704116+0000","last_unstale":"2023-10-06T03:03:29.708267+0000","last_undegraded":"2023-10-06T03:03:29.708267+0000","last_fullsized":"2023-10-06T03:03:29.708267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726855+0000","last_change":"2023-10-06T03:03:29.720059+0000","last_active":"2023-10-06T03:03:29.726855+0000","last_peered":"2023-10-06T03:03:29.726855+0000","last_clean":"2023-10-06T03:03:29.726855+0000","last_became_active":"2023-10-06T03:03:29.719808+0000","last_became_peered":"2023-10-06T03:03:29.719808+0000","last_unstale":"2023-10-06T03:03:29.726855+0000","last_undegraded":"2023-10-06T03:03:29.726855+0000","last_fullsized":"2023-10-06T03:03:29.726855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"65'3","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731482+0000","last_change":"2023-10-06T03:03:28.707321+0000","last_active":"2023-10-06T03:03:29.731482+0000","last_peered":"2023-10-06T03:03:29.731482+0000","last_clean":"2023-10-06T03:03:29.731482+0000","last_became_active":"2023-10-06T03:03:28.706986+0000","last_became_peered":"2023-10-06T03:03:28.706986+0000","last_unstale":"2023-10-06T03:03:29.731482+0000","last_undegraded":"2023-10-06T03:03:29.731482+0000","last_fullsized":"2023-10-06T03:03:29.731482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:41.749372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732256+0000","last_change":"2023-10-06T03:03:29.728887+0000","last_active":"2023-10-06T03:03:29.732256+0000","last_peered":"2023-10-06T03:03:29.732256+0000","last_clean":"2023-10-06T03:03:29.732256+0000","last_became_active":"2023-10-06T03:03:29.728331+0000","last_became_peered":"2023-10-06T03:03:29.728331+0000","last_unstale":"2023-10-06T03:03:29.732256+0000","last_undegraded":"2023-10-06T03:03:29.732256+0000","last_fullsized":"2023-10-06T03:03:29.732256+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.272664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"65'6","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.738004+0000","last_change":"2023-10-06T03:03:28.702854+0000","last_active":"2023-10-06T03:03:29.738004+0000","last_peered":"2023-10-06T03:03:29.738004+0000","last_clean":"2023-10-06T03:03:29.738004+0000","last_became_active":"2023-10-06T03:03:28.702541+0000","last_became_peered":"2023-10-06T03:03:28.702541+0000","last_unstale":"2023-10-06T03:03:29.738004+0000","last_undegraded":"2023-10-06T03:03:29.738004+0000","last_fullsized":"2023-10-06T03:03:29.738004+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:40.260251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.727735+0000","last_change":"2023-10-06T03:03:29.720289+0000","last_active":"2023-10-06T03:03:29.727735+0000","last_peered":"2023-10-06T03:03:29.727735+0000","last_clean":"2023-10-06T03:03:29.727735+0000","last_became_active":"2023-10-06T03:03:29.719700+0000","last_became_peered":"2023-10-06T03:03:29.719700+0000","last_unstale":"2023-10-06T03:03:29.727735+0000","last_undegraded":"2023-10-06T03:03:29.727735+0000","last_fullsized":"2023-10-06T03:03:29.727735+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:15.821768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.733446+0000","last_change":"2023-10-06T03:03:28.701142+0000","last_active":"2023-10-06T03:03:29.733446+0000","last_peered":"2023-10-06T03:03:29.733446+0000","last_clean":"2023-10-06T03:03:29.733446+0000","last_became_active":"2023-10-06T03:03:28.700888+0000","last_became_peered":"2023-10-06T03:03:28.700888+0000","last_unstale":"2023-10-06T03:03:29.733446+0000","last_undegraded":"2023-10-06T03:03:29.733446+0000","last_fullsized":"2023-10-06T03:03:29.733446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725958+0000","last_change":"2023-10-06T03:03:29.719247+0000","last_active":"2023-10-06T03:03:29.725958+0000","last_peered":"2023-10-06T03:03:29.725958+0000","last_clean":"2023-10-06T03:03:29.725958+0000","last_became_active":"2023-10-06T03:03:29.719000+0000","last_became_peered":"2023-10-06T03:03:29.719000+0000","last_unstale":"2023-10-06T03:03:29.725958+0000","last_undegraded":"2023-10-06T03:03:29.725958+0000","last_fullsized":"2023-10-06T03:03:29.725958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:37.272664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711971+0000","last_change":"2023-10-06T03:03:28.709750+0000","last_active":"2023-10-06T03:03:29.711971+0000","last_peered":"2023-10-06T03:03:29.711971+0000","last_clean":"2023-10-06T03:03:29.711971+0000","last_became_active":"2023-10-06T03:03:28.708794+0000","last_became_peered":"2023-10-06T03:03:28.708794+0000","last_unstale":"2023-10-06T03:03:29.711971+0000","last_undegraded":"2023-10-06T03:03:29.711971+0000","last_fullsized":"2023-10-06T03:03:29.711971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:31.256052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725898+0000","last_change":"2023-10-06T03:03:29.722007+0000","last_active":"2023-10-06T03:03:29.725898+0000","last_peered":"2023-10-06T03:03:29.725898+0000","last_clean":"2023-10-06T03:03:29.725898+0000","last_became_active":"2023-10-06T03:03:29.721770+0000","last_became_peered":"2023-10-06T03:03:29.721770+0000","last_unstale":"2023-10-06T03:03:29.725898+0000","last_undegraded":"2023-10-06T03:03:29.725898+0000","last_fullsized":"2023-10-06T03:03:29.725898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:04.975316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.709703+0000","last_change":"2023-10-06T03:03:28.707056+0000","last_active":"2023-10-06T03:03:29.709703+0000","last_peered":"2023-10-06T03:03:29.709703+0000","last_clean":"2023-10-06T03:03:29.709703+0000","last_became_active":"2023-10-06T03:03:28.706778+0000","last_became_peered":"2023-10-06T03:03:28.706778+0000","last_unstale":"2023-10-06T03:03:29.709703+0000","last_undegraded":"2023-10-06T03:03:29.709703+0000","last_fullsized":"2023-10-06T03:03:29.709703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:18.282708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726300+0000","last_change":"2023-10-06T03:03:29.721620+0000","last_active":"2023-10-06T03:03:29.726300+0000","last_peered":"2023-10-06T03:03:29.726300+0000","last_clean":"2023-10-06T03:03:29.726300+0000","last_became_active":"2023-10-06T03:03:29.721371+0000","last_became_peered":"2023-10-06T03:03:29.721371+0000","last_unstale":"2023-10-06T03:03:29.726300+0000","last_undegraded":"2023-10-06T03:03:29.726300+0000","last_fullsized":"2023-10-06T03:03:29.726300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:29.446863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"65'8","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.738156+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:29.738156+0000","last_peered":"2023-10-06T03:03:29.738156+0000","last_clean":"2023-10-06T03:03:29.738156+0000","last_became_active":"2023-10-06T03:03:28.707467+0000","last_became_peered":"2023-10-06T03:03:28.707467+0000","last_unstale":"2023-10-06T03:03:29.738156+0000","last_undegraded":"2023-10-06T03:03:29.738156+0000","last_fullsized":"2023-10-06T03:03:29.738156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726915+0000","last_change":"2023-10-06T03:03:29.721553+0000","last_active":"2023-10-06T03:03:29.726915+0000","last_peered":"2023-10-06T03:03:29.726915+0000","last_clean":"2023-10-06T03:03:29.726915+0000","last_became_active":"2023-10-06T03:03:29.721102+0000","last_became_peered":"2023-10-06T03:03:29.721102+0000","last_unstale":"2023-10-06T03:03:29.726915+0000","last_undegraded":"2023-10-06T03:03:29.726915+0000","last_fullsized":"2023-10-06T03:03:29.726915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:50.441521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731305+0000","last_change":"2023-10-06T03:03:28.703808+0000","last_active":"2023-10-06T03:03:29.731305+0000","last_peered":"2023-10-06T03:03:29.731305+0000","last_clean":"2023-10-06T03:03:29.731305+0000","last_became_active":"2023-10-06T03:03:28.703078+0000","last_became_peered":"2023-10-06T03:03:28.703078+0000","last_unstale":"2023-10-06T03:03:29.731305+0000","last_undegraded":"2023-10-06T03:03:29.731305+0000","last_fullsized":"2023-10-06T03:03:29.731305+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725981+0000","last_change":"2023-10-06T03:03:29.723143+0000","last_active":"2023-10-06T03:03:29.725981+0000","last_peered":"2023-10-06T03:03:29.725981+0000","last_clean":"2023-10-06T03:03:29.725981+0000","last_became_active":"2023-10-06T03:03:29.722899+0000","last_became_peered":"2023-10-06T03:03:29.722899+0000","last_unstale":"2023-10-06T03:03:29.725981+0000","last_undegraded":"2023-10-06T03:03:29.725981+0000","last_fullsized":"2023-10-06T03:03:29.725981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:48.803431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"66'1","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-06T03:03:30.717830+0000","last_change":"2023-10-06T03:03:28.705716+0000","last_active":"2023-10-06T03:03:30.717830+0000","last_peered":"2023-10-06T03:03:30.717830+0000","last_clean":"2023-10-06T03:03:30.717830+0000","last_became_active":"2023-10-06T03:03:28.705508+0000","last_became_peered":"2023-10-06T03:03:28.705508+0000","last_unstale":"2023-10-06T03:03:30.717830+0000","last_undegraded":"2023-10-06T03:03:30.717830+0000","last_fullsized":"2023-10-06T03:03:30.717830+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:47.795438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728139+0000","last_change":"2023-10-06T03:03:29.725809+0000","last_active":"2023-10-06T03:03:29.728139+0000","last_peered":"2023-10-06T03:03:29.728139+0000","last_clean":"2023-10-06T03:03:29.728139+0000","last_became_active":"2023-10-06T03:03:29.720674+0000","last_became_peered":"2023-10-06T03:03:29.720674+0000","last_unstale":"2023-10-06T03:03:29.728139+0000","last_undegraded":"2023-10-06T03:03:29.728139+0000","last_fullsized":"2023-10-06T03:03:29.728139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.051205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.712156+0000","last_change":"2023-10-06T03:03:28.709286+0000","last_active":"2023-10-06T03:03:29.712156+0000","last_peered":"2023-10-06T03:03:29.712156+0000","last_clean":"2023-10-06T03:03:29.712156+0000","last_became_active":"2023-10-06T03:03:28.708990+0000","last_became_peered":"2023-10-06T03:03:28.708990+0000","last_unstale":"2023-10-06T03:03:29.712156+0000","last_undegraded":"2023-10-06T03:03:29.712156+0000","last_fullsized":"2023-10-06T03:03:29.712156+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728818+0000","last_change":"2023-10-06T03:03:29.725295+0000","last_active":"2023-10-06T03:03:29.728818+0000","last_peered":"2023-10-06T03:03:29.728818+0000","last_clean":"2023-10-06T03:03:29.728818+0000","last_became_active":"2023-10-06T03:03:29.725125+0000","last_became_peered":"2023-10-06T03:03:29.725125+0000","last_unstale":"2023-10-06T03:03:29.728818+0000","last_undegraded":"2023-10-06T03:03:29.728818+0000","last_fullsized":"2023-10-06T03:03:29.728818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:18.809274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711255+0000","last_change":"2023-10-06T03:03:28.709344+0000","last_active":"2023-10-06T03:03:29.711255+0000","last_peered":"2023-10-06T03:03:29.711255+0000","last_clean":"2023-10-06T03:03:29.711255+0000","last_became_active":"2023-10-06T03:03:28.709005+0000","last_became_peered":"2023-10-06T03:03:28.709005+0000","last_unstale":"2023-10-06T03:03:29.711255+0000","last_undegraded":"2023-10-06T03:03:29.711255+0000","last_fullsized":"2023-10-06T03:03:29.711255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:06:58.882523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725988+0000","last_change":"2023-10-06T03:03:29.721504+0000","last_active":"2023-10-06T03:03:29.725988+0000","last_peered":"2023-10-06T03:03:29.725988+0000","last_clean":"2023-10-06T03:03:29.725988+0000","last_became_active":"2023-10-06T03:03:29.721198+0000","last_became_peered":"2023-10-06T03:03:29.721198+0000","last_unstale":"2023-10-06T03:03:29.725988+0000","last_undegraded":"2023-10-06T03:03:29.725988+0000","last_fullsized":"2023-10-06T03:03:29.725988+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:11.768132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711510+0000","last_change":"2023-10-06T03:03:28.711750+0000","last_active":"2023-10-06T03:03:29.711510+0000","last_peered":"2023-10-06T03:03:29.711510+0000","last_clean":"2023-10-06T03:03:29.711510+0000","last_became_active":"2023-10-06T03:03:28.711510+0000","last_became_peered":"2023-10-06T03:03:28.711510+0000","last_unstale":"2023-10-06T03:03:29.711510+0000","last_undegraded":"2023-10-06T03:03:29.711510+0000","last_fullsized":"2023-10-06T03:03:29.711510+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:59.402917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.725934+0000","last_change":"2023-10-06T03:03:29.718707+0000","last_active":"2023-10-06T03:03:29.725934+0000","last_peered":"2023-10-06T03:03:29.725934+0000","last_clean":"2023-10-06T03:03:29.725934+0000","last_became_active":"2023-10-06T03:03:29.718416+0000","last_became_peered":"2023-10-06T03:03:29.718416+0000","last_unstale":"2023-10-06T03:03:29.725934+0000","last_undegraded":"2023-10-06T03:03:29.725934+0000","last_fullsized":"2023-10-06T03:03:29.725934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:30.417220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.710081+0000","last_change":"2023-10-06T03:03:28.706921+0000","last_active":"2023-10-06T03:03:29.710081+0000","last_peered":"2023-10-06T03:03:29.710081+0000","last_clean":"2023-10-06T03:03:29.710081+0000","last_became_active":"2023-10-06T03:03:28.706676+0000","last_became_peered":"2023-10-06T03:03:28.706676+0000","last_unstale":"2023-10-06T03:03:29.710081+0000","last_undegraded":"2023-10-06T03:03:29.710081+0000","last_fullsized":"2023-10-06T03:03:29.710081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:59.778082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732255+0000","last_change":"2023-10-06T03:03:29.729338+0000","last_active":"2023-10-06T03:03:29.732255+0000","last_peered":"2023-10-06T03:03:29.732255+0000","last_clean":"2023-10-06T03:03:29.732255+0000","last_became_active":"2023-10-06T03:03:29.729096+0000","last_became_peered":"2023-10-06T03:03:29.729096+0000","last_unstale":"2023-10-06T03:03:29.732255+0000","last_undegraded":"2023-10-06T03:03:29.732255+0000","last_fullsized":"2023-10-06T03:03:29.732255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:48.803431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.714450+0000","last_change":"2023-10-06T03:03:28.707082+0000","last_active":"2023-10-06T03:03:29.714450+0000","last_peered":"2023-10-06T03:03:29.714450+0000","last_clean":"2023-10-06T03:03:29.714450+0000","last_became_active":"2023-10-06T03:03:28.706254+0000","last_became_peered":"2023-10-06T03:03:28.706254+0000","last_unstale":"2023-10-06T03:03:29.714450+0000","last_undegraded":"2023-10-06T03:03:29.714450+0000","last_fullsized":"2023-10-06T03:03:29.714450+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:46:08.966317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728619+0000","last_change":"2023-10-06T03:03:29.724012+0000","last_active":"2023-10-06T03:03:29.728619+0000","last_peered":"2023-10-06T03:03:29.728619+0000","last_clean":"2023-10-06T03:03:29.728619+0000","last_became_active":"2023-10-06T03:03:29.723782+0000","last_became_peered":"2023-10-06T03:03:29.723782+0000","last_unstale":"2023-10-06T03:03:29.728619+0000","last_undegraded":"2023-10-06T03:03:29.728619+0000","last_fullsized":"2023-10-06T03:03:29.728619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:56:10.609021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.709773+0000","last_change":"2023-10-06T03:03:28.699446+0000","last_active":"2023-10-06T03:03:29.709773+0000","last_peered":"2023-10-06T03:03:29.709773+0000","last_clean":"2023-10-06T03:03:29.709773+0000","last_became_active":"2023-10-06T03:03:28.699197+0000","last_became_peered":"2023-10-06T03:03:28.699197+0000","last_unstale":"2023-10-06T03:03:29.709773+0000","last_undegraded":"2023-10-06T03:03:29.709773+0000","last_fullsized":"2023-10-06T03:03:29.709773+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:45.388509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722644+0000","last_change":"2023-10-06T03:03:29.718640+0000","last_active":"2023-10-06T03:03:29.722644+0000","last_peered":"2023-10-06T03:03:29.722644+0000","last_clean":"2023-10-06T03:03:29.722644+0000","last_became_active":"2023-10-06T03:03:29.718359+0000","last_became_peered":"2023-10-06T03:03:29.718359+0000","last_unstale":"2023-10-06T03:03:29.722644+0000","last_undegraded":"2023-10-06T03:03:29.722644+0000","last_fullsized":"2023-10-06T03:03:29.722644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:28.073193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731275+0000","last_change":"2023-10-06T03:03:28.705759+0000","last_active":"2023-10-06T03:03:29.731275+0000","last_peered":"2023-10-06T03:03:29.731275+0000","last_clean":"2023-10-06T03:03:29.731275+0000","last_became_active":"2023-10-06T03:03:28.705443+0000","last_became_peered":"2023-10-06T03:03:28.705443+0000","last_unstale":"2023-10-06T03:03:29.731275+0000","last_undegraded":"2023-10-06T03:03:29.731275+0000","last_fullsized":"2023-10-06T03:03:29.731275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:28.438871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.713938+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:03:29.713938+0000","last_peered":"2023-10-06T03:03:29.713938+0000","last_clean":"2023-10-06T03:03:29.713938+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:03:29.713938+0000","last_undegraded":"2023-10-06T03:03:29.713938+0000","last_fullsized":"2023-10-06T03:03:29.713938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.720253+0000","last_change":"2023-10-06T03:03:29.717723+0000","last_active":"2023-10-06T03:03:29.720253+0000","last_peered":"2023-10-06T03:03:29.720253+0000","last_clean":"2023-10-06T03:03:29.720253+0000","last_became_active":"2023-10-06T03:03:29.717452+0000","last_became_peered":"2023-10-06T03:03:29.717452+0000","last_unstale":"2023-10-06T03:03:29.720253+0000","last_undegraded":"2023-10-06T03:03:29.720253+0000","last_fullsized":"2023-10-06T03:03:29.720253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:14.638908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.732353+0000","last_change":"2023-10-06T03:03:28.709130+0000","last_active":"2023-10-06T03:03:29.732353+0000","last_peered":"2023-10-06T03:03:29.732353+0000","last_clean":"2023-10-06T03:03:29.732353+0000","last_became_active":"2023-10-06T03:03:28.708759+0000","last_became_peered":"2023-10-06T03:03:28.708759+0000","last_unstale":"2023-10-06T03:03:29.732353+0000","last_undegraded":"2023-10-06T03:03:29.732353+0000","last_fullsized":"2023-10-06T03:03:29.732353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.713535+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:03:29.713535+0000","last_peered":"2023-10-06T03:03:29.713535+0000","last_clean":"2023-10-06T03:03:29.713535+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:03:29.713535+0000","last_undegraded":"2023-10-06T03:03:29.713535+0000","last_fullsized":"2023-10-06T03:03:29.713535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.724345+0000","last_change":"2023-10-06T03:03:29.718676+0000","last_active":"2023-10-06T03:03:29.724345+0000","last_peered":"2023-10-06T03:03:29.724345+0000","last_clean":"2023-10-06T03:03:29.724345+0000","last_became_active":"2023-10-06T03:03:29.718418+0000","last_became_peered":"2023-10-06T03:03:29.718418+0000","last_unstale":"2023-10-06T03:03:29.724345+0000","last_undegraded":"2023-10-06T03:03:29.724345+0000","last_fullsized":"2023-10-06T03:03:29.724345+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:30.688614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711119+0000","last_change":"2023-10-06T03:03:28.707737+0000","last_active":"2023-10-06T03:03:29.711119+0000","last_peered":"2023-10-06T03:03:29.711119+0000","last_clean":"2023-10-06T03:03:29.711119+0000","last_became_active":"2023-10-06T03:03:28.707489+0000","last_became_peered":"2023-10-06T03:03:28.707489+0000","last_unstale":"2023-10-06T03:03:29.711119+0000","last_undegraded":"2023-10-06T03:03:29.711119+0000","last_fullsized":"2023-10-06T03:03:29.711119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:41.067368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.712577+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:03:29.712577+0000","last_peered":"2023-10-06T03:03:29.712577+0000","last_clean":"2023-10-06T03:03:29.712577+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:03:29.712577+0000","last_undegraded":"2023-10-06T03:03:29.712577+0000","last_fullsized":"2023-10-06T03:03:29.712577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.731785+0000","last_change":"2023-10-06T03:03:29.728078+0000","last_active":"2023-10-06T03:03:29.731785+0000","last_peered":"2023-10-06T03:03:29.731785+0000","last_clean":"2023-10-06T03:03:29.731785+0000","last_became_active":"2023-10-06T03:03:29.727835+0000","last_became_peered":"2023-10-06T03:03:29.727835+0000","last_unstale":"2023-10-06T03:03:29.731785+0000","last_undegraded":"2023-10-06T03:03:29.731785+0000","last_fullsized":"2023-10-06T03:03:29.731785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:30.688614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.707786+0000","last_change":"2023-10-06T03:03:28.708268+0000","last_active":"2023-10-06T03:03:29.707786+0000","last_peered":"2023-10-06T03:03:29.707786+0000","last_clean":"2023-10-06T03:03:29.707786+0000","last_became_active":"2023-10-06T03:03:28.707997+0000","last_became_peered":"2023-10-06T03:03:28.707997+0000","last_unstale":"2023-10-06T03:03:29.707786+0000","last_undegraded":"2023-10-06T03:03:29.707786+0000","last_fullsized":"2023-10-06T03:03:29.707786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:36.170599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.714294+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:03:29.714294+0000","last_peered":"2023-10-06T03:03:29.714294+0000","last_clean":"2023-10-06T03:03:29.714294+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:03:29.714294+0000","last_undegraded":"2023-10-06T03:03:29.714294+0000","last_fullsized":"2023-10-06T03:03:29.714294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728614+0000","last_change":"2023-10-06T03:03:29.722687+0000","last_active":"2023-10-06T03:03:29.728614+0000","last_peered":"2023-10-06T03:03:29.728614+0000","last_clean":"2023-10-06T03:03:29.728614+0000","last_became_active":"2023-10-06T03:03:29.722429+0000","last_became_peered":"2023-10-06T03:03:29.722429+0000","last_unstale":"2023-10-06T03:03:29.728614+0000","last_undegraded":"2023-10-06T03:03:29.728614+0000","last_fullsized":"2023-10-06T03:03:29.728614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:20:01.225363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.707656+0000","last_change":"2023-10-06T03:03:28.706968+0000","last_active":"2023-10-06T03:03:29.707656+0000","last_peered":"2023-10-06T03:03:29.707656+0000","last_clean":"2023-10-06T03:03:29.707656+0000","last_became_active":"2023-10-06T03:03:28.706720+0000","last_became_peered":"2023-10-06T03:03:28.706720+0000","last_unstale":"2023-10-06T03:03:29.707656+0000","last_undegraded":"2023-10-06T03:03:29.707656+0000","last_fullsized":"2023-10-06T03:03:29.707656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.080285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":204,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.714381+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:03:29.714381+0000","last_peered":"2023-10-06T03:03:29.714381+0000","last_clean":"2023-10-06T03:03:29.714381+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:03:29.714381+0000","last_undegraded":"2023-10-06T03:03:29.714381+0000","last_fullsized":"2023-10-06T03:03:29.714381+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708229+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:03:29.708229+0000","last_peered":"2023-10-06T03:03:29.708229+0000","last_clean":"2023-10-06T03:03:29.708229+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:03:29.708229+0000","last_undegraded":"2023-10-06T03:03:29.708229+0000","last_fullsized":"2023-10-06T03:03:29.708229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.726284+0000","last_change":"2023-10-06T03:03:29.720101+0000","last_active":"2023-10-06T03:03:29.726284+0000","last_peered":"2023-10-06T03:03:29.726284+0000","last_clean":"2023-10-06T03:03:29.726284+0000","last_became_active":"2023-10-06T03:03:29.719853+0000","last_became_peered":"2023-10-06T03:03:29.719853+0000","last_unstale":"2023-10-06T03:03:29.726284+0000","last_undegraded":"2023-10-06T03:03:29.726284+0000","last_fullsized":"2023-10-06T03:03:29.726284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:32.855430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708615+0000","last_change":"2023-10-06T03:03:28.707199+0000","last_active":"2023-10-06T03:03:29.708615+0000","last_peered":"2023-10-06T03:03:29.708615+0000","last_clean":"2023-10-06T03:03:29.708615+0000","last_became_active":"2023-10-06T03:03:28.706953+0000","last_became_peered":"2023-10-06T03:03:28.706953+0000","last_unstale":"2023-10-06T03:03:29.708615+0000","last_undegraded":"2023-10-06T03:03:29.708615+0000","last_fullsized":"2023-10-06T03:03:29.708615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:42.170748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.709678+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:03:29.709678+0000","last_peered":"2023-10-06T03:03:29.709678+0000","last_clean":"2023-10-06T03:03:29.709678+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:03:29.709678+0000","last_undegraded":"2023-10-06T03:03:29.709678+0000","last_fullsized":"2023-10-06T03:03:29.709678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.728618+0000","last_change":"2023-10-06T03:03:29.725709+0000","last_active":"2023-10-06T03:03:29.728618+0000","last_peered":"2023-10-06T03:03:29.728618+0000","last_clean":"2023-10-06T03:03:29.728618+0000","last_became_active":"2023-10-06T03:03:29.720819+0000","last_became_peered":"2023-10-06T03:03:29.720819+0000","last_unstale":"2023-10-06T03:03:29.728618+0000","last_undegraded":"2023-10-06T03:03:29.728618+0000","last_fullsized":"2023-10-06T03:03:29.728618+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:26.827247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"65'2","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.736550+0000","last_change":"2023-10-06T03:03:28.704304+0000","last_active":"2023-10-06T03:03:29.736550+0000","last_peered":"2023-10-06T03:03:29.736550+0000","last_clean":"2023-10-06T03:03:29.736550+0000","last_became_active":"2023-10-06T03:03:28.704042+0000","last_became_peered":"2023-10-06T03:03:28.704042+0000","last_unstale":"2023-10-06T03:03:29.736550+0000","last_undegraded":"2023-10-06T03:03:29.736550+0000","last_fullsized":"2023-10-06T03:03:29.736550+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:58.270672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.708146+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:03:29.708146+0000","last_peered":"2023-10-06T03:03:29.708146+0000","last_clean":"2023-10-06T03:03:29.708146+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:03:29.708146+0000","last_undegraded":"2023-10-06T03:03:29.708146+0000","last_fullsized":"2023-10-06T03:03:29.708146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.722617+0000","last_change":"2023-10-06T03:03:29.718087+0000","last_active":"2023-10-06T03:03:29.722617+0000","last_peered":"2023-10-06T03:03:29.722617+0000","last_clean":"2023-10-06T03:03:29.722617+0000","last_became_active":"2023-10-06T03:03:29.717875+0000","last_became_peered":"2023-10-06T03:03:29.717875+0000","last_unstale":"2023-10-06T03:03:29.722617+0000","last_undegraded":"2023-10-06T03:03:29.722617+0000","last_fullsized":"2023-10-06T03:03:29.722617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:28.685044+0000","last_clean_scrub_stamp":"2023-10-06T03:03:28.685044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:52.287277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.730484+0000","last_change":"2023-10-06T03:03:28.704994+0000","last_active":"2023-10-06T03:03:29.730484+0000","last_peered":"2023-10-06T03:03:29.730484+0000","last_clean":"2023-10-06T03:03:29.730484+0000","last_became_active":"2023-10-06T03:03:28.704702+0000","last_became_peered":"2023-10-06T03:03:28.704702+0000","last_unstale":"2023-10-06T03:03:29.730484+0000","last_undegraded":"2023-10-06T03:03:29.730484+0000","last_fullsized":"2023-10-06T03:03:29.730484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:03:27.677052+0000","last_clean_scrub_stamp":"2023-10-06T03:03:27.677052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:29:18.494652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-06T03:03:29.711197+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:03:29.711197+0000","last_peered":"2023-10-06T03:03:29.711197+0000","last_clean":"2023-10-06T03:03:29.711197+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:03:29.711197+0000","last_undegraded":"2023-10-06T03:03:29.711197+0000","last_fullsized":"2023-10-06T03:03:29.711197+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298656,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073184,"statfs":{"total":96636764160,"available":96330940416,"internally_reserved":0,"allocated":1400832,"data_stored":951449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.752,"15min":0.79400000000000004},"min":{"1min":0.38800000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71699999999999997,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.499}]},{"osd":1,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67300000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.60899999999999999,"5min":0.875,"15min":0.875},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.55600000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70399999999999996,"5min":0.999,"15min":0.999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52700000000000002,"15min":0.52900000000000003},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.64900000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.68999999999999995,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58399999999999996,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.51000000000000001,"15min":0.502},"min":{"1min":0.34399999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.46700000000000003,"5min":0.439,"15min":0.439},"max":{"1min":0.80200000000000005,"5min":1.026,"15min":1.026},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.316,"15min":0.316},"max":{"1min":0.75700000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.45000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:03:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76900000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77700000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":362595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.504}]},{"osd":3,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:02:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":361171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32700000000000001},"min":{"1min":0.23799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53400000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.377},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.58899999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33600000000000002,"15min":0.33500000000000002},"min":{"1min":0.251,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.51400000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.309},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.33600000000000002,"15min":0.33800000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.45200000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33600000000000002,"15min":0.33900000000000002},"min":{"1min":0.26200000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.42299999999999999,"5min":0.628,"15min":0.628},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.37,"15min":0.371},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54900000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.878,"5min":1.115,"15min":1.115},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.40300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78500000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51000000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.57899999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.85799999999999998,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.38},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.81999999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.443,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81299999999999994,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.443},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80900000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.79100000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.47699999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298652,"kb_used_data":1364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073188,"statfs":{"total":96636764160,"available":96330944512,"internally_reserved":0,"allocated":1396736,"data_stored":951427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.31,"15min":0.311},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.31},{"interface":"front","average":{"1min":0.316,"5min":0.32800000000000001,"15min":0.33000000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.499,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.25900000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.56000000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35499999999999998,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.34799999999999998,"15min":0.34699999999999998},"min":{"1min":0.27000000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.48899999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38900000000000001,"15min":0.39300000000000002},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.38}]},{"osd":4,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":0.871,"15min":0.871},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.77100000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84699999999999998,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59699999999999998,"15min":0.60199999999999998},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80800000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42499999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83899999999999997,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.78500000000000003,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91200000000000003,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.629,"15min":0.63},"min":{"1min":0.46000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":361773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.313,"15min":0.311},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.41699999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32800000000000001,"15min":0.32600000000000001},"min":{"1min":0.245,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.29299999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.51100000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.34100000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32000000000000001},"min":{"1min":0.24099999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53000000000000003,"15min":0.52400000000000002},"min":{"1min":0.45100000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51100000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.5},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.501}]},{"osd":6,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54100000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.81100000000000005,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.623,"15min":0.629},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84999999999999998,"5min":1.903,"15min":1.903},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.53200000000000003},"min":{"1min":0.45000000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.46400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":361613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.29399999999999998,"5min":0.309,"15min":0.311},"min":{"1min":0.23400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.38300000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.246},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.32700000000000001,"15min":0.32800000000000001},"min":{"1min":0.26000000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.39800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.42399999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35599999999999998},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.318}]},{"osd":4,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72499999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.40400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.61699999999999999,"15min":0.58199999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70699999999999996,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.69499999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":362699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.503,"15min":0.49099999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.78600000000000003,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72399999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67700000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61099999999999999,"15min":0.61499999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.751,"5min":3,"15min":3},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.84999999999999998,"15min":0.89800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":10.084,"15min":10.084},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70799999999999996,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.71699999999999997,"15min":0.74099999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.68899999999999995,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67600000000000005,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.59199999999999997,"15min":0.60799999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.64700000000000002,"5min":1.54,"15min":1.54},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.439},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.63900000000000001,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76500000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42899999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.79300000000000004,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298076,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073764,"statfs":{"total":96636764160,"available":96331534336,"internally_reserved":0,"allocated":806912,"data_stored":361093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78200000000000003,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.61899999999999999,"15min":0.63100000000000001},"min":{"1min":0.44700000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.73199999999999998,"5min":2.371,"15min":2.371},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69199999999999995,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.64500000000000002,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.59899999999999998,"15min":0.61099999999999999},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64600000000000002,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":0.76100000000000001,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78500000000000003,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.74199999999999999,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.52400000000000002,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65400000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.61299999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.57099999999999995,"15min":0.58399999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67100000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63600000000000001,"15min":0.64400000000000002},"min":{"1min":0.48799999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.73999999999999999,"5min":1.052,"15min":1.052},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:03:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.64000000000000001,"15min":0.65500000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71999999999999997,"5min":1.175,"15min":1.175},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.623,"15min":0.63300000000000001},"min":{"1min":0.44600000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.71099999999999997,"5min":1.145,"15min":1.145},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:03:38.632 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:03:38.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:03:38.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.787+0000 7f4494b50700 1 Processor -- start 2023-10-06T03:03:38.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.787+0000 7f4494b50700 1 -- start start 2023-10-06T03:03:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4494b50700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4490159950 0x7f44900cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4494b50700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44900af320 0x7f4490069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4494b50700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4490069aa0 0x7f449015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4494b50700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4490069f20 con 0x7f44900af320 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4494b50700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900ce380 con 0x7f4490069aa0 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4494b50700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44900cec20 con 0x7f4490159950 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44900af320 0x7f4490069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f44900af320 0x7f4490069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50364/0 (socket says 172.21.15.92:50364) 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 -- 172.21.15.92:0/1868856554 learned_addr learned my addr 172.21.15.92:0/1868856554 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f448ed9d700 1 --2- 172.21.15.92:0/1868856554 >> v2:172.21.15.136:3300/0 conn(0x7f4490069aa0 0x7f449015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f448e59c700 1 --2- 172.21.15.92:0/1868856554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4490159950 0x7f44900cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 -- 172.21.15.92:0/1868856554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4490159950 msgr2=0x7f44900cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 --2- 172.21.15.92:0/1868856554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4490159950 0x7f44900cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 -- 172.21.15.92:0/1868856554 >> v2:172.21.15.136:3300/0 conn(0x7f4490069aa0 msgr2=0x7f449015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 --2- 172.21.15.92:0/1868856554 >> v2:172.21.15.136:3300/0 conn(0x7f4490069aa0 0x7f449015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 -- 172.21.15.92:0/1868856554 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900afa90 con 0x7f44900af320 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.788+0000 7f4485d9b700 1 --2- 172.21.15.92:0/1868856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900af320 0x7f4490069560 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f44900d0760 tx=0x7f4478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19580959607132f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f448dd9b700 1 -- 172.21.15.92:0/1868856554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800b640 con 0x7f44900af320 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f448dd9b700 1 -- 172.21.15.92:0/1868856554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800b7a0 con 0x7f44900af320 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f448dd9b700 1 -- 172.21.15.92:0/1868856554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800bb60 con 0x7f44900af320 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 -- 172.21.15.92:0/1868856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900af320 msgr2=0x7f4490069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 --2- 172.21.15.92:0/1868856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900af320 0x7f4490069560 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f44900d0760 tx=0x7f4478009250 comp rx=0 tx=0).stop 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 -- 172.21.15.92:0/1868856554 shutdown_connections 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 --2- 172.21.15.92:0/1868856554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4490159950 0x7f44900cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 --2- 172.21.15.92:0/1868856554 >> v2:172.21.15.136:3300/0 conn(0x7f4490069aa0 0x7f449015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 --2- 172.21.15.92:0/1868856554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44900af320 0x7f4490069560 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 -- 172.21.15.92:0/1868856554 >> 172.21.15.92:0/1868856554 conn(0x7f4490068e00 msgr2=0x7f44900d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 -- 172.21.15.92:0/1868856554 shutdown_connections 2023-10-06T03:03:38.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.789+0000 7f4494b50700 1 -- 172.21.15.92:0/1868856554 wait complete. 2023-10-06T03:03:38.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.790+0000 7f4494b50700 1 Processor -- start 2023-10-06T03:03:38.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.790+0000 7f4494b50700 1 -- start start 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4494b50700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4490069aa0 0x7f449014e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4494b50700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44900af320 0x7f44900b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4494b50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 0x7f44900b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4494b50700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900c7e10 con 0x7f4490159950 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4494b50700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490151d70 con 0x7f4490069aa0 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4494b50700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4490152050 con 0x7f44900af320 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f448ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 0x7f44900b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f448ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 0x7f44900b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50378/0 (socket says 172.21.15.92:50378) 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f448ed9d700 1 -- 172.21.15.92:0/2336113424 learned_addr learned my addr 172.21.15.92:0/2336113424 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f4485d9b700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44900af320 0x7f44900b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.791+0000 7f448e59c700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4490069aa0 0x7f449014e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f448ed9d700 1 -- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44900af320 msgr2=0x7f44900b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f448ed9d700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44900af320 0x7f44900b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f448ed9d700 1 -- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4490069aa0 msgr2=0x7f449014e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f448ed9d700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4490069aa0 0x7f449014e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f448ed9d700 1 -- 172.21.15.92:0/2336113424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780094f0 con 0x7f4490159950 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f448ed9d700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 0x7f44900b5ae0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e90 tx=0x7f448800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800cb70 con 0x7f4490159950 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800f040 con 0x7f4490159950 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4490152330 con 0x7f4490159950 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488014480 con 0x7f4490159950 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.792+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449014e770 con 0x7f4490159950 2023-10-06T03:03:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.793+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44900718b0 con 0x7f4490159950 2023-10-06T03:03:38.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.793+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f4488014830 con 0x7f4490159950 2023-10-06T03:03:38.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.794+0000 7f4486ffd700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4470066fc0 0x7f4470069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:38.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.794+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4488085340 con 0x7f4490159950 2023-10-06T03:03:38.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.794+0000 7f448e59c700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4470066fc0 0x7f4470069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:38.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.794+0000 7f448e59c700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4470066fc0 0x7f4470069470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4480009d10 tx=0x7f44800091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:38.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:38.798+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4488052250 con 0x7f4490159950 2023-10-06T03:03:39.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.054+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f44901550c0 con 0x7f4490159950 2023-10-06T03:03:39.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.057+0000 7f4486ffd700 1 -- 172.21.15.92:0/2336113424 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4488055870 con 0x7f4490159950 2023-10-06T03:03:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4470066fc0 msgr2=0x7f4470069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4470066fc0 0x7f4470069470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4480009d10 tx=0x7f44800091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 msgr2=0x7f44900b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 0x7f44900b5ae0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4488009e90 tx=0x7f448800b040 comp rx=0 tx=0).stop 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 shutdown_connections 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4470066fc0 0x7f4470069470 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4490159950 0x7f44900b5ae0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4490069aa0 0x7f449014e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 --2- 172.21.15.92:0/2336113424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44900af320 0x7f44900b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 >> 172.21.15.92:0/2336113424 conn(0x7f4490068e00 msgr2=0x7f44900cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 shutdown_connections 2023-10-06T03:03:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.059+0000 7f4494b50700 1 -- 172.21.15.92:0/2336113424 wait complete. 2023-10-06T03:03:39.063 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:03:39.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:03:39.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.185+0000 7fb1c6a51700 1 Processor -- start 2023-10-06T03:03:39.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.185+0000 7fb1c6a51700 1 -- start start 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1c6a51700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb1c0159890 0x7fb1c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1c6a51700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1c0159c60 0x7fb1c015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1c6a51700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c015db10 0x7fb1c0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1c6a51700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c015df50 con 0x7fb1c0159c60 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1c6a51700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c015a160 con 0x7fb1c0159890 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1c6a51700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00d13a0 con 0x7fb1c015db10 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1bf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1c0159c60 0x7fb1c015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1bf7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb1c0159c60 0x7fb1c015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50388/0 (socket says 172.21.15.92:50388) 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.186+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/2225959617 learned_addr learned my addr 172.21.15.92:0/2225959617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1c4fee700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c015db10 0x7fb1c0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bffff700 1 --2- 172.21.15.92:0/2225959617 >> v2:172.21.15.136:3300/0 conn(0x7fb1c0159890 0x7fb1c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/2225959617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c015db10 msgr2=0x7fb1c0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c015db10 0x7fb1c0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/2225959617 >> v2:172.21.15.136:3300/0 conn(0x7fb1c0159890 msgr2=0x7fb1c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/2225959617 >> v2:172.21.15.136:3300/0 conn(0x7fb1c0159890 0x7fb1c015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/2225959617 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c015e140 con 0x7fb1c0159c60 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1c4fee700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c015db10 0x7fb1c0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bffff700 1 --2- 172.21.15.92:0/2225959617 >> v2:172.21.15.136:3300/0 conn(0x7fb1c0159890 0x7fb1c015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c015d5d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00d1090 tx=0x7fb1b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7c5d72bde8e36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1beffd700 1 -- 172.21.15.92:0/2225959617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b400b860 con 0x7fb1c0159c60 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1beffd700 1 -- 172.21.15.92:0/2225959617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b400b9c0 con 0x7fb1c0159c60 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.187+0000 7fb1beffd700 1 -- 172.21.15.92:0/2225959617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b400bd60 con 0x7fb1c0159c60 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 -- 172.21.15.92:0/2225959617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 msgr2=0x7fb1c015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c015d5d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00d1090 tx=0x7fb1b4009250 comp rx=0 tx=0).stop 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 -- 172.21.15.92:0/2225959617 shutdown_connections 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c015db10 0x7fb1c0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/2225959617 >> v2:172.21.15.136:3300/0 conn(0x7fb1c0159890 0x7fb1c015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/2225959617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c015d5d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 -- 172.21.15.92:0/2225959617 >> 172.21.15.92:0/2225959617 conn(0x7fb1c0068e20 msgr2=0x7fb1c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 -- 172.21.15.92:0/2225959617 shutdown_connections 2023-10-06T03:03:39.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.188+0000 7fb1c6a51700 1 -- 172.21.15.92:0/2225959617 wait complete. 2023-10-06T03:03:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.189+0000 7fb1c6a51700 1 Processor -- start 2023-10-06T03:03:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.189+0000 7fb1c6a51700 1 -- start start 2023-10-06T03:03:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c0159890 0x7fb1c00b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c01473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c015db10 0x7fb1c00b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0144b90 con 0x7fb1c0159c60 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00b4e60 con 0x7fb1c015db10 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c00b48a0 con 0x7fb1c0159890 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c01473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c01473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50400/0 (socket says 172.21.15.92:50400) 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/86682198 learned_addr learned my addr 172.21.15.92:0/86682198 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bffff700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c0159890 0x7fb1c00b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c4fee700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c015db10 0x7fb1c00b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c0159890 msgr2=0x7fb1c00b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c0159890 0x7fb1c00b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c015db10 msgr2=0x7fb1c00b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c015db10 0x7fb1c00b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 -- 172.21.15.92:0/86682198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40094f0 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1bf7fe700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c01473d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0070e20 tx=0x7fb1b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4011dc0 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4024440 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c00b4b80 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4024800 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.190+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c00c7c20 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.191+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c00c7e40 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.192+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb1b401a420 con 0x7fb1c0159c60 2023-10-06T03:03:39.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.192+0000 7fb1a7fff700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb1ac066fc0 0x7fb1ac069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.192+0000 7fb1bffff700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb1ac066fc0 0x7fb1ac069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.192+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb1b40864d0 con 0x7fb1c0159c60 2023-10-06T03:03:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.193+0000 7fb1bffff700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb1ac066fc0 0x7fb1ac069470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00b6630 tx=0x7fb1b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.197+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1b40535f0 con 0x7fb1c0159c60 2023-10-06T03:03:39.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.423+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c004c3c0 con 0x7fb1c0159c60 2023-10-06T03:03:39.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.423+0000 7fb1a7fff700 1 -- 172.21.15.92:0/86682198 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fb1b4056c10 con 0x7fb1c0159c60 2023-10-06T03:03:39.425 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:03:39.425 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:29.075556+0000","modified":"2023-10-06T03:03:32.719393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/812630670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":812630670},{"type":"v1","addr":"172.21.15.92:6837","nonce":812630670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb1ac066fc0 msgr2=0x7fb1ac069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb1ac066fc0 0x7fb1ac069470 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00b6630 tx=0x7fb1b0008040 comp rx=0 tx=0).stop 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 msgr2=0x7fb1c01473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c01473d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0070e20 tx=0x7fb1b4017000 comp rx=0 tx=0).stop 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 shutdown_connections 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb1ac066fc0 0x7fb1ac069470 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c015db10 0x7fb1c00b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c0159890 0x7fb1c00b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 --2- 172.21.15.92:0/86682198 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb1c0159c60 0x7fb1c01473d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 >> 172.21.15.92:0/86682198 conn(0x7fb1c0068e20 msgr2=0x7fb1c015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 shutdown_connections 2023-10-06T03:03:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.426+0000 7fb1c6a51700 1 -- 172.21.15.92:0/86682198 wait complete. 2023-10-06T03:03:39.429 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 38 2023-10-06T03:03:39.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:03:29.075556+0000', 'modified': '2023-10-06T03:03:32.719393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5733}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5733': {'gid': 5733, 'name': 'c', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/812630670', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 812630670}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 812630670}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:03:39.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:03:39.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:03:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.957+0000 7f42078c2700 1 Processor -- start 2023-10-06T03:03:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.957+0000 7f42078c2700 1 -- start start 2023-10-06T03:03:39.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f42078c2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f42000879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f42078c2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4200068aa0 0x7f4200068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f42078c2700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4200069450 con 0x7f42000875d0 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f42078c2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42000abfd0 con 0x7f4200068aa0 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f42078c2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42000acdf0 con 0x7f42000b1c50 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f4205e5f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4200068aa0 0x7f4200068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f4205e5f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4200068aa0 0x7f4200068f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32834/0 (socket says 172.21.15.92:32834) 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.958+0000 7f4205e5f700 1 -- 172.21.15.92:0/785236455 learned_addr learned my addr 172.21.15.92:0/785236455 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f4204e5d700 1 --2- 172.21.15.92:0/785236455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f42000879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f41f7fff700 1 -- 172.21.15.92:0/785236455 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769192389 0 0) 0x7f4200069450 con 0x7f42000875d0 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f41f7fff700 1 -- 172.21.15.92:0/785236455 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41ec003550 con 0x7f42000875d0 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f4205e5f700 1 -- 172.21.15.92:0/785236455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 msgr2=0x7f42000879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f4205e5f700 1 --2- 172.21.15.92:0/785236455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f42000879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f4205e5f700 1 -- 172.21.15.92:0/785236455 >> v1:172.21.15.136:6789/0 conn(0x7f42000875d0 legacy=0x7f42000ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f4205e5f700 1 -- 172.21.15.92:0/785236455 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000ac1e0 con 0x7f4200068aa0 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.959+0000 7f4205e5f700 1 --2- 172.21.15.92:0/785236455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4200068aa0 0x7f4200068f10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41f800afb0 tx=0x7f41f800c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21d997462186131 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f41f7fff700 1 -- 172.21.15.92:0/785236455 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800a2e0 con 0x7f4200068aa0 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f41f7fff700 1 -- 172.21.15.92:0/785236455 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800b9c0 con 0x7f4200068aa0 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f41f7fff700 1 -- 172.21.15.92:0/785236455 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f800bd40 con 0x7f4200068aa0 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 -- 172.21.15.92:0/785236455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4200068aa0 msgr2=0x7f4200068f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 --2- 172.21.15.92:0/785236455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4200068aa0 0x7f4200068f10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f41f800afb0 tx=0x7f41f800c040 comp rx=0 tx=0).stop 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 -- 172.21.15.92:0/785236455 shutdown_connections 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 --2- 172.21.15.92:0/785236455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4200068aa0 0x7f4200068f10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 --2- 172.21.15.92:0/785236455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f42000879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 -- 172.21.15.92:0/785236455 >> 172.21.15.92:0/785236455 conn(0x7f42000cd000 msgr2=0x7f4200068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 -- 172.21.15.92:0/785236455 shutdown_connections 2023-10-06T03:03:39.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.960+0000 7f42078c2700 1 -- 172.21.15.92:0/785236455 wait complete. 2023-10-06T03:03:39.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.961+0000 7f42078c2700 1 Processor -- start 2023-10-06T03:03:39.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.961+0000 7f42078c2700 1 -- start start 2023-10-06T03:03:39.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f42078c2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200068aa0 0x7f4200177940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f42078c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 0x7f42000aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f42078c2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f420016c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f42078c2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420016bca0 con 0x7f42000875d0 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f42078c2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420016b370 con 0x7f4200068aa0 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f42078c2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420016b650 con 0x7f42000b1c50 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f4204e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 0x7f42000aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f4204e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 0x7f42000aa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32842/0 (socket says 172.21.15.92:32842) 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.962+0000 7f4204e5d700 1 -- 172.21.15.92:0/3916914376 learned_addr learned my addr 172.21.15.92:0/3916914376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4205e5f700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f420016c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f420565e700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200068aa0 0x7f4200177940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4204e5d700 1 -- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 msgr2=0x7f420016c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f420016c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4204e5d700 1 -- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200068aa0 msgr2=0x7f4200177940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200068aa0 0x7f4200177940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4204e5d700 1 -- 172.21.15.92:0/3916914376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f800aaf0 con 0x7f42000875d0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f420565e700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200068aa0 0x7f4200177940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 0x7f42000aa970 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f41f000de60 tx=0x7f41f000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f41f57fa700 1 -- 172.21.15.92:0/3916914376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f000d2e0 con 0x7f42000875d0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f41f57fa700 1 -- 172.21.15.92:0/3916914376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0013040 con 0x7f42000875d0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420016b930 con 0x7f42000875d0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f41f57fa700 1 -- 172.21.15.92:0/3916914376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0010c20 con 0x7f42000875d0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.963+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4200177e80 con 0x7f42000875d0 2023-10-06T03:03:39.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.964+0000 7f41f57fa700 1 -- 172.21.15.92:0/3916914376 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f41f0018480 con 0x7f42000875d0 2023-10-06T03:03:39.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.965+0000 7f41f57fa700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f41e4066fc0 0x7f41e4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.965+0000 7f41f57fa700 1 -- 172.21.15.92:0/3916914376 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41f0088b00 con 0x7f42000875d0 2023-10-06T03:03:39.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.965+0000 7f420565e700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f41e4066fc0 0x7f41e4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.965+0000 7f420565e700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f41e4066fc0 0x7f41e4069470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f420006e680 tx=0x7f41fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 Processor -- start 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 -- start start 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce80dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce812b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 0x559ce812c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8032a70 con 0x559ce812bc00 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8030270 con 0x559ce80dccc0 2023-10-06T03:03:39.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.967+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce802c060 con 0x559ce80db840 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 0x559ce812c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 0x559ce812c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32846/0 (socket says 172.21.15.92:32846) 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 -- 172.21.15.92:0/3008474061 learned_addr learned my addr 172.21.15.92:0/3008474061 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 -- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 msgr2=0x559ce80dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f420565e700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce80dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce812b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce80dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 -- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 msgr2=0x559ce812b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce812b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 -- 172.21.15.92:0/3008474061 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce7ec6270 con 0x559ce812bc00 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce812b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f420565e700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce80dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f4205e5f700 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 0x559ce812c050 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b7b0 tx=0x7f41f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecac17fd7a096cae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f41ea7fc700 1 -- 172.21.15.92:0/3008474061 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8007000 con 0x559ce812bc00 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f41ea7fc700 1 -- 172.21.15.92:0/3008474061 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800a3b0 con 0x559ce812bc00 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.968+0000 7f41ea7fc700 1 -- 172.21.15.92:0/3008474061 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8008720 con 0x559ce812bc00 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 -- 172.21.15.92:0/3008474061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 msgr2=0x559ce812c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 0x559ce812c050 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f41f800b7b0 tx=0x7f41f8015000 comp rx=0 tx=0).stop 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 -- 172.21.15.92:0/3008474061 shutdown_connections 2023-10-06T03:03:39.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce812bc00 0x559ce812c050 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce80dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3008474061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce812b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 -- 172.21.15.92:0/3008474061 >> 172.21.15.92:0/3008474061 conn(0x559ce80dd160 msgr2=0x559ce80dd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 -- 172.21.15.92:0/3008474061 shutdown_connections 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 -- 172.21.15.92:0/3008474061 wait complete. 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.969+0000 7f4210bbab80 1 -- start start 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 Processor -- start 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80db840 0x559ce8101390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce811e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce7fd7890 con 0x559ce80dccc0 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce7ff2ea0 con 0x559ce80db840 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce7eaee30 con 0x559ce812bc00 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4210bbab80 10 client.0 _ll_register_callbacks cb 0x559ce809ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.970+0000 7f4204e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce811e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce811e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32862/0 (socket says 172.21.15.92:32862) 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 -- 172.21.15.92:0/3529761135 learned_addr learned my addr 172.21.15.92:0/3529761135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f420565e700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80db840 0x559ce8101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4205e5f700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 -- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 msgr2=0x559ce811aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 -- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80db840 msgr2=0x559ce8101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80db840 0x559ce8101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 -- 172.21.15.92:0/3529761135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42000ac1e0 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4205e5f700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4204e5d700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce811e6f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f41f0010730 tx=0x7f41f00595a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44577af0855db47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f41df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f41df7fe700 1 -- 172.21.15.92:0/3529761135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f005ca20 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f41df7fe700 1 -- 172.21.15.92:0/3529761135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41f0060770 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ce80d0940 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f41df7fe700 1 -- 172.21.15.92:0/3529761135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41f00604f0 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 10 client.5907 fetch_fsmap learned FSMap version 38 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x559ce7e68fe0 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f41df7fe700 1 -- 172.21.15.92:0/3529761135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0086030 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f41df7fe700 1 -- 172.21.15.92:0/3529761135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f41f0082d20 con 0x559ce80dccc0 2023-10-06T03:03:39.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 10 client.5907 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 10 client.5907 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 20 client.5907 populate_metadata read hostname 'smithi092' 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8105810 0x559ce8105be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 4 client.5907 mds_command: new command op to 5733 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.971+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 --> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ce80ad870 con 0x559ce8105810 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.972+0000 7f420565e700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8105810 0x559ce8105be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.972+0000 7f41df7fe700 10 client.5907 ms_handle_connect on v2:172.21.15.92:6835/812630670 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.972+0000 7f420565e700 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8105810 0x559ce8105be0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763af47f5829ace0 server_cookie=86ca832f97ad9009 in_seq=0 out_seq=0 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.972+0000 7f41df7fe700 1 -- 172.21.15.92:0/3529761135 <== mds.0 v2:172.21.15.92:6835/812630670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559ce80ad870 con 0x559ce8105810 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.972+0000 7f41df7fe700 10 client.5907 handle_command_reply: tid=0 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.972+0000 7f4210bbab80 1 client.5907 shutdown 2023-10-06T03:03:39.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 msgr2=0x559ce811e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce811e6f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f41f0010730 tx=0x7f41f00595a0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 shutdown_connections 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8105810 0x559ce8105be0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f41df7fe700 0 client.5907 ms_handle_reset on v2:172.21.15.92:6835/812630670 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80db840 0x559ce8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 --2- 172.21.15.92:0/3529761135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce811e6f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 >> 172.21.15.92:0/3529761135 conn(0x559ce80dd160 msgr2=0x559ce80dc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 shutdown_connections 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 1 -- 172.21.15.92:0/3529761135 wait complete. 2023-10-06T03:03:39.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.973+0000 7f4210bbab80 20 client.5907 trim_cache size 0 max 16384 2023-10-06T03:03:39.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.989+0000 7f4210bbab80 1 Processor -- start 2023-10-06T03:03:39.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.989+0000 7f4210bbab80 1 -- start start 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce81213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce80d9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce812bc00 0x559ce811c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4210bbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce80d6620 con 0x559ce80dccc0 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8131a80 con 0x559ce812bc00 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce8120d00 con 0x559ce80db840 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4204e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce80d9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4204e5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce80d9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32876/0 (socket says 172.21.15.92:32876) 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4204e5d700 1 -- 172.21.15.92:0/2516787560 learned_addr learned my addr 172.21.15.92:0/2516787560 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:39.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f420565e700 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce81213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.990+0000 7f4205e5f700 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce812bc00 0x559ce811c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4204e5d700 1 -- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 msgr2=0x559ce81213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4204e5d700 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce81213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4204e5d700 1 -- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce812bc00 msgr2=0x559ce811c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4204e5d700 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce812bc00 0x559ce811c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4204e5d700 1 -- 172.21.15.92:0/2516787560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ce7ffa780 con 0x559ce80dccc0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4204e5d700 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce80d9860 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f41f00559c0 tx=0x7f41f0072000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfae09f377a6136 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f41eaffd700 1 -- 172.21.15.92:0/2516787560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0086da0 con 0x559ce80dccc0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f41eaffd700 1 -- 172.21.15.92:0/2516787560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0086040 con 0x559ce80dccc0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f41eaffd700 1 -- 172.21.15.92:0/2516787560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f00595a0 con 0x559ce80dccc0 2023-10-06T03:03:39.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 -- 172.21.15.92:0/2516787560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 msgr2=0x559ce80d9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce80d9860 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f41f00559c0 tx=0x7f41f0072000 comp rx=0 tx=0).stop 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 -- 172.21.15.92:0/2516787560 shutdown_connections 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce812bc00 0x559ce811c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce80db840 0x559ce81213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 --2- 172.21.15.92:0/2516787560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80dccc0 0x559ce80d9860 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 -- 172.21.15.92:0/2516787560 >> 172.21.15.92:0/2516787560 conn(0x559ce80dd160 msgr2=0x559ce811a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 -- 172.21.15.92:0/2516787560 shutdown_connections 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.991+0000 7f4210bbab80 1 -- 172.21.15.92:0/2516787560 wait complete. 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.992+0000 7f4210bbab80 1 -- start start 2023-10-06T03:03:39.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.992+0000 7f4210bbab80 1 Processor -- start 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 0x559ce8126960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce8187800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce811c0f0 con 0x559ce80db840 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ce8130f50 con 0x559ce80dccc0 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ce8125f10 con 0x559ce812bc00 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4210bbab80 10 client.0 _ll_register_callbacks cb 0x559ce809ee70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 0x559ce8126960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 0x559ce8126960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32886/0 (socket says 172.21.15.92:32886) 2023-10-06T03:03:40.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 -- 172.21.15.92:0/1318980811 learned_addr learned my addr 172.21.15.92:0/1318980811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4205e5f700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4204e5d700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce8187800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 -- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 msgr2=0x559ce811b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 -- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 msgr2=0x559ce8187800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce8187800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 -- 172.21.15.92:0/1318980811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ce7e68fe0 con 0x559ce80db840 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f4204e5d700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce8187800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f420565e700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 0x559ce8126960 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x559ce811db40 tx=0x7f41fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b57e1ea2004140d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.993+0000 7f41df7fe700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:03:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f41df7fe700 1 -- 172.21.15.92:0/1318980811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc014e20 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f41df7fe700 1 -- 172.21.15.92:0/1318980811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41fc0219f0 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41f800a3b0 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f41df7fe700 1 -- 172.21.15.92:0/1318980811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41fc021dc0 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 10 client.5913 fetch_fsmap learned FSMap version 38 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x559ce7fe1470 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f41df7fe700 1 -- 172.21.15.92:0/1318980811 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f800a3b0 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f41df7fe700 1 -- 172.21.15.92:0/1318980811 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 38) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f41fc02a440 con 0x559ce80db840 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 10 client.5913 fetch_fsmap finished waiting for FSMap version 38 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 10 client.5913 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 20 client.5913 populate_metadata read hostname 'smithi092' 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8104b70 0x559ce8104f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 4 client.5913 mds_command: new command op to 5733 tid=0[{"prefix": "status"}] 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 --> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559ce7fcee00 con 0x559ce8104b70 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4204e5d700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8104b70 0x559ce8104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f41df7fe700 10 client.5913 ms_handle_connect on v2:172.21.15.92:6835/812630670 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.994+0000 7f4204e5d700 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8104b70 0x559ce8104f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af0054a9c1745f7 server_cookie=e7330556f07ecf70 in_seq=0 out_seq=0 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f41df7fe700 1 -- 172.21.15.92:0/1318980811 <== mds.0 v2:172.21.15.92:6835/812630670 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559ce7fcee00 con 0x559ce8104b70 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f41df7fe700 10 client.5913 handle_command_reply: tid=0 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 client.5913 shutdown 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 msgr2=0x559ce8126960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 0x559ce8126960 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x559ce811db40 tx=0x7f41fc014000 comp rx=0 tx=0).stop 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 shutdown_connections 2023-10-06T03:03:40.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:6835/812630670,v1:172.21.15.92:6837/812630670] conn(0x559ce8104b70 0x559ce8104f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559ce80db840 0x559ce8126960 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f41df7fe700 0 client.5913 ms_handle_reset on v2:172.21.15.92:6835/812630670 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ce812bc00 0x559ce811b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 --2- 172.21.15.92:0/1318980811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ce80dccc0 0x559ce8187800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 >> 172.21.15.92:0/1318980811 conn(0x559ce80dd160 msgr2=0x559ce80d9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 shutdown_connections 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 1 -- 172.21.15.92:0/1318980811 wait complete. 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f4210bbab80 20 client.5913 trim_cache size 0 max 16384 2023-10-06T03:03:40.004 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 5733, 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 10.284346586, 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 38, 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 66, 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 66, 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.425259997000001 2023-10-06T03:03:40.005 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:03:40.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f41e4066fc0 msgr2=0x7f41e4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f41e4066fc0 0x7f41e4069470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f420006e680 tx=0x7f41fc009040 comp rx=0 tx=0).stop 2023-10-06T03:03:40.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 msgr2=0x7f42000aa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 0x7f42000aa970 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f41f000de60 tx=0x7f41f000f040 comp rx=0 tx=0).stop 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 shutdown_connections 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f41e4066fc0 0x7f41e4069470 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42000b1c50 0x7f420016c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200068aa0 0x7f4200177940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 --2- 172.21.15.92:0/3916914376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42000875d0 0x7f42000aa970 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.995+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 >> 172.21.15.92:0/3916914376 conn(0x7f42000cd000 msgr2=0x7f4200168d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.996+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 shutdown_connections 2023-10-06T03:03:40.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:39.996+0000 7f42078c2700 1 -- 172.21.15.92:0/3916914376 wait complete. 2023-10-06T03:03:40.009 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:03:40.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.122+0000 7f67acef5700 1 Processor -- start 2023-10-06T03:03:40.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.122+0000 7f67acef5700 1 -- start start 2023-10-06T03:03:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67acef5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67a8159890 0x7f67a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67acef5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159c60 0x7f67a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67acef5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67a815db10 0x7f67a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67acef5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a815a160 con 0x7f67a815db10 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67acef5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a80d13a0 con 0x7f67a8159890 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67acef5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a815e140 con 0x7f67a8159c60 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67a815db10 0x7f67a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f67a815db10 0x7f67a8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32888/0 (socket says 172.21.15.92:32888) 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 -- 172.21.15.92:0/660345828 learned_addr learned my addr 172.21.15.92:0/660345828 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a659c700 1 --2- 172.21.15.92:0/660345828 >> v2:172.21.15.136:3300/0 conn(0x7f67a8159890 0x7f67a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a5d9b700 1 --2- 172.21.15.92:0/660345828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159c60 0x7f67a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 -- 172.21.15.92:0/660345828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159c60 msgr2=0x7f67a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 --2- 172.21.15.92:0/660345828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159c60 0x7f67a815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 -- 172.21.15.92:0/660345828 >> v2:172.21.15.136:3300/0 conn(0x7f67a8159890 msgr2=0x7f67a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:40.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 --2- 172.21.15.92:0/660345828 >> v2:172.21.15.136:3300/0 conn(0x7f67a8159890 0x7f67a815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.123+0000 7f67a6d9d700 1 -- 172.21.15.92:0/660345828 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a815a3d0 con 0x7f67a815db10 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67a6d9d700 1 --2- 172.21.15.92:0/660345828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a815db10 0x7f67a8069560 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f67a80d1090 tx=0x7f6798009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b5df4d7fd99e1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67a559a700 1 -- 172.21.15.92:0/660345828 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800b640 con 0x7f67a815db10 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67a559a700 1 -- 172.21.15.92:0/660345828 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679800b7a0 con 0x7f67a815db10 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67a559a700 1 -- 172.21.15.92:0/660345828 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679800bb60 con 0x7f67a815db10 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 -- 172.21.15.92:0/660345828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a815db10 msgr2=0x7f67a8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 --2- 172.21.15.92:0/660345828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a815db10 0x7f67a8069560 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f67a80d1090 tx=0x7f6798009250 comp rx=0 tx=0).stop 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 -- 172.21.15.92:0/660345828 shutdown_connections 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 --2- 172.21.15.92:0/660345828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a815db10 0x7f67a8069560 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 --2- 172.21.15.92:0/660345828 >> v2:172.21.15.136:3300/0 conn(0x7f67a8159890 0x7f67a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 --2- 172.21.15.92:0/660345828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159c60 0x7f67a815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 -- 172.21.15.92:0/660345828 >> 172.21.15.92:0/660345828 conn(0x7f67a8068e20 msgr2=0x7f67a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:40.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 -- 172.21.15.92:0/660345828 shutdown_connections 2023-10-06T03:03:40.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.124+0000 7f67acef5700 1 -- 172.21.15.92:0/660345828 wait complete. 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.125+0000 7f67acef5700 1 Processor -- start 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.125+0000 7f67acef5700 1 -- start start 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67acef5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159890 0x7f67a80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67acef5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 0x7f67a80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67acef5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67a815db10 0x7f67a80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67acef5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8153e70 con 0x7f67a8159c60 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67acef5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a8153860 con 0x7f67a815db10 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67acef5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80cfbc0 con 0x7f67a8159890 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 0x7f67a80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 0x7f67a80cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32892/0 (socket says 172.21.15.92:32892) 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 -- 172.21.15.92:0/1175986715 learned_addr learned my addr 172.21.15.92:0/1175986715 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a659c700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159890 0x7f67a80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a6d9d700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67a815db10 0x7f67a80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 -- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159890 msgr2=0x7f67a80b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159890 0x7f67a80b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 -- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67a815db10 msgr2=0x7f67a80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67a815db10 0x7f67a80cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.126+0000 7f67a5d9b700 1 -- 172.21.15.92:0/1175986715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67980094f0 con 0x7f67a8159c60 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.127+0000 7f67a5d9b700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 0x7f67a80cf0b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f679c00ec50 tx=0x7f679c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.127+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00c9d0 con 0x7f67a8159c60 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.127+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c00cb30 con 0x7f67a8159c60 2023-10-06T03:03:40.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.127+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a8153b00 con 0x7f67a8159c60 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.127+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c013470 con 0x7f67a8159c60 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.127+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a8151010 con 0x7f67a8159c60 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.128+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a804c3c0 con 0x7f67a8159c60 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.128+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f679c013860 con 0x7f67a8159c60 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.128+0000 7f67967fc700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6788066fc0 0x7f6788069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.129+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f679c084a70 con 0x7f67a8159c60 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.129+0000 7f67a659c700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6788066fc0 0x7f6788069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:40.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.129+0000 7f67a659c700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6788066fc0 0x7f6788069470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f67a816cab0 tx=0x7f67a000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:40.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.133+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f679c051b90 con 0x7f67a8159c60 2023-10-06T03:03:40.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.357+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67a8153fb0 con 0x7f67a8159c60 2023-10-06T03:03:40.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.357+0000 7f67967fc700 1 -- 172.21.15.92:0/1175986715 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f679c0551b0 con 0x7f67a8159c60 2023-10-06T03:03:40.359 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:03:40.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6788066fc0 msgr2=0x7f6788069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6788066fc0 0x7f6788069470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f67a816cab0 tx=0x7f67a000b040 comp rx=0 tx=0).stop 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 msgr2=0x7f67a80cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 0x7f67a80cf0b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f679c00ec50 tx=0x7f679c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 shutdown_connections 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6788066fc0 0x7f6788069470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67a815db10 0x7f67a80cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67a8159890 0x7f67a80b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 --2- 172.21.15.92:0/1175986715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a8159c60 0x7f67a80cf0b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 >> 172.21.15.92:0/1175986715 conn(0x7f67a8068e20 msgr2=0x7f67a8152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 shutdown_connections 2023-10-06T03:03:40.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:40.359+0000 7f67acef5700 1 -- 172.21.15.92:0/1175986715 wait complete. 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:03:40.373 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:03:40.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.374 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:03:40.389 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:40.390 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42386sec preferred_lft 42386sec 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:03:40.391 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:03:40.392 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:40.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.393 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:03:40.452 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:03:40.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.454 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:03:40.509 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:03:40.510 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:03:40.510 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:03:40.510 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:03:40.510 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:03:40.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.510 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:03:40.510 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:03:40.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:03:40.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:03:40.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:03:40.511 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:03:40.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.609 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:03:40.623 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:03:40.623 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:03:40.624 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:03:40.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.624 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:03:40.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:40.688 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:40.742 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:03:40.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:40.742 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:40.796 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:03:40.796 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:03:40.797 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:40.875 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:40.876 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:40.894 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:40.894 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:40.894 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:40.895 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:40.895 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:40.895 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:40.895 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:40.895 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:03:40.917 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:03:40.918 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:03:40.919 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:03:40.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:40.920 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:03:40.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:03:41.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.136+0000 7f5f9eeb6700 1 Processor -- start 2023-10-06T03:03:41.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.136+0000 7f5f9eeb6700 1 -- start start 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9eeb6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5f98159420 0x7f5f980b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9eeb6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f981597f0 0x7f5f980b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9eeb6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f9815d230 0x7f5f9815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9eeb6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9815dbd0 con 0x7f5f9815d230 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9eeb6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f980b11d0 con 0x7f5f98159420 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9eeb6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f980b14b0 con 0x7f5f981597f0 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f9815d230 0x7f5f9815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5f9815d230 0x7f5f9815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32896/0 (socket says 172.21.15.92:32896) 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 -- 172.21.15.92:0/3584173914 learned_addr learned my addr 172.21.15.92:0/3584173914 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:41.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9deb4700 1 --2- 172.21.15.92:0/3584173914 >> v2:172.21.15.136:3300/0 conn(0x7f5f98159420 0x7f5f980b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9d6b3700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f981597f0 0x7f5f980b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 -- 172.21.15.92:0/3584173914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f981597f0 msgr2=0x7f5f980b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f981597f0 0x7f5f980b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 -- 172.21.15.92:0/3584173914 >> v2:172.21.15.136:3300/0 conn(0x7f5f98159420 msgr2=0x7f5f980b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 --2- 172.21.15.92:0/3584173914 >> v2:172.21.15.136:3300/0 conn(0x7f5f98159420 0x7f5f980b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9e6b5700 1 -- 172.21.15.92:0/3584173914 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f980b1790 con 0x7f5f9815d230 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.137+0000 7f5f9deb4700 1 --2- 172.21.15.92:0/3584173914 >> v2:172.21.15.136:3300/0 conn(0x7f5f98159420 0x7f5f980b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9d6b3700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f981597f0 0x7f5f980b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9e6b5700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f9815d230 0x7f5f9815d690 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5f980b2f20 tx=0x7f5f90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62de4fec572b50e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:41.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9ceb2700 1 -- 172.21.15.92:0/3584173914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000e070 con 0x7f5f9815d230 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9ceb2700 1 -- 172.21.15.92:0/3584173914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9000ba50 con 0x7f5f9815d230 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9ceb2700 1 -- 172.21.15.92:0/3584173914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f900092e0 con 0x7f5f9815d230 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3584173914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f9815d230 msgr2=0x7f5f9815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f9815d230 0x7f5f9815d690 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5f980b2f20 tx=0x7f5f90009840 comp rx=0 tx=0).stop 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3584173914 shutdown_connections 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f9815d230 0x7f5f9815d690 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3584173914 >> v2:172.21.15.136:3300/0 conn(0x7f5f98159420 0x7f5f980b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3584173914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f981597f0 0x7f5f980b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3584173914 >> 172.21.15.92:0/3584173914 conn(0x7f5f980d0660 msgr2=0x7f5f980d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3584173914 shutdown_connections 2023-10-06T03:03:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.138+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3584173914 wait complete. 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.139+0000 7f5f9eeb6700 1 Processor -- start 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 -- start start 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 0x7f5f98169b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f981597f0 0x7f5f98069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f9815d230 0x7f5f98152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9815ca30 con 0x7f5f98159420 2023-10-06T03:03:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9815cec0 con 0x7f5f981597f0 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9eeb6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9815be20 con 0x7f5f9815d230 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9deb4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 0x7f5f98169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9deb4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 0x7f5f98169b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32904/0 (socket says 172.21.15.92:32904) 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.140+0000 7f5f9deb4700 1 -- 172.21.15.92:0/3223388662 learned_addr learned my addr 172.21.15.92:0/3223388662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9d6b3700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f981597f0 0x7f5f98069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9e6b5700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f9815d230 0x7f5f98152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9deb4700 1 -- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f9815d230 msgr2=0x7f5f98152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9deb4700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f9815d230 0x7f5f98152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9deb4700 1 -- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f981597f0 msgr2=0x7f5f98069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9deb4700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f981597f0 0x7f5f98069520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9deb4700 1 -- 172.21.15.92:0/3223388662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90009ae0 con 0x7f5f98159420 2023-10-06T03:03:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9deb4700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 0x7f5f98169b30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e8d0 tx=0x7f5f94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400ee70 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9400c5e0 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9815c100 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400c980 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.141+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9815c6c0 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.142+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9804c3c0 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.142+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5f94013410 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.143+0000 7f5f8dffb700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5f7c067040 0x7f5f7c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.143+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5f94084950 con 0x7f5f98159420 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.143+0000 7f5f9d6b3700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5f7c067040 0x7f5f7c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:41.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.143+0000 7f5f9d6b3700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5f7c067040 0x7f5f7c0694f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f9806a070 tx=0x7f5f88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:41.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.147+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f94051860 con 0x7f5f98159420 2023-10-06T03:03:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.372+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f980b1a70 con 0x7f5f98159420 2023-10-06T03:03:41.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.373+0000 7f5f8dffb700 1 -- 172.21.15.92:0/3223388662 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5f94020070 con 0x7f5f98159420 2023-10-06T03:03:41.375 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5f7c067040 msgr2=0x7f5f7c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5f7c067040 0x7f5f7c0694f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5f9806a070 tx=0x7f5f88008040 comp rx=0 tx=0).stop 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 msgr2=0x7f5f98169b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 0x7f5f98169b30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400e8d0 tx=0x7f5f94008040 comp rx=0 tx=0).stop 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 shutdown_connections 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5f7c067040 0x7f5f7c0694f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f98159420 0x7f5f98169b30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f9815d230 0x7f5f98152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 --2- 172.21.15.92:0/3223388662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f981597f0 0x7f5f98069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 >> 172.21.15.92:0/3223388662 conn(0x7f5f980d0660 msgr2=0x7f5f980b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 shutdown_connections 2023-10-06T03:03:41.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:41.375+0000 7f5f9eeb6700 1 -- 172.21.15.92:0/3223388662 wait complete. 2023-10-06T03:03:41.388 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:03:41.388 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:03:41.389 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:03:41.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.389 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:03:41.407 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:03:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:03:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:03:41.407 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:41.408 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42386sec preferred_lft 42386sec 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.409 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:03:41.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.410 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:03:41.474 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:03:41.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.476 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:03:41.533 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:03:41.534 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:03:41.534 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:03:41.534 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:03:41.534 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:03:41.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.534 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:03:41.534 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:03:41.534 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:03:41.535 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:03:41.535 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:03:41.535 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:03:41.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.639 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:03:41.653 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:03:41.653 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:03:41.654 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:03:41.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:03:41.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:41.718 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:41.775 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:03:41.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:03:41.775 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:41.830 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:03:41.830 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:03:41.830 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:41.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:03:41.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:03:41.910 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:41.910 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:41.929 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:41.930 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:41.954 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:03:41.954 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:03:41.954 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:03:41.954 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:03:41.954 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:03:41.954 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:03:41.955 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:03:41.955 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:03:41.955 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:03:41.955 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:03:41.955 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:03:41.956 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:03:41.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:03:41.956 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:03:42.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:03:42.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.152+0000 7fc8e468e700 1 Processor -- start 2023-10-06T03:03:42.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.152+0000 7fc8e468e700 1 -- start start 2023-10-06T03:03:42.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e468e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc0cd910 0x7fc8dc0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e468e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8dc0b55a0 0x7fc8dc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e468e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc0b5b00 con 0x7fc8dc159ec0 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e468e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b5da0 con 0x7fc8dc0b55a0 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e468e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b6080 con 0x7fc8dc0cd910 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e242a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc8dc159ec0 0x7fc8dc0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48758/0 (socket says 172.21.15.92:48758) 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.153+0000 7fc8e242a700 1 -- 172.21.15.92:0/376385480 learned_addr learned my addr 172.21.15.92:0/376385480 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc0cd910 0x7fc8dc0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e2c2b700 1 --2- 172.21.15.92:0/376385480 >> v2:172.21.15.136:3300/0 conn(0x7fc8dc0b55a0 0x7fc8dc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906954242 0 0) 0x7fc8dc0b5b00 con 0x7fc8dc159ec0 2023-10-06T03:03:42.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8c8003570 con 0x7fc8dc159ec0 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369783460 0 0) 0x7fc8c8003570 con 0x7fc8dc159ec0 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc0cd910 msgr2=0x7fc8dc0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc0cd910 0x7fc8dc0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 >> v2:172.21.15.136:3300/0 conn(0x7fc8dc0b55a0 msgr2=0x7fc8dc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 --2- 172.21.15.92:0/376385480 >> v2:172.21.15.136:3300/0 conn(0x7fc8dc0b55a0 0x7fc8dc0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0cc100 con 0x7fc8dc159ec0 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc8d80015f0 con 0x7fc8dc159ec0 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 >> v1:172.21.15.92:6789/0 conn(0x7fc8dc159ec0 legacy=0x7fc8dc0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8003a60 0x7fc8c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8004810 0x7fc8c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8c80051c0 0x7fc8c800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b5da0 con 0x7fc8c8004810 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b6080 con 0x7fc8c8003a60 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c800bfa0 con 0x7fc8c80051c0 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc0cd910 0x7fc8dc0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e2c2b700 1 --2- 172.21.15.92:0/376385480 >> v2:172.21.15.136:3300/0 conn(0x7fc8dc0b55a0 0x7fc8dc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e242a700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8003a60 0x7fc8c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8004810 0x7fc8c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e2c2b700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8c80051c0 0x7fc8c800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 -- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8c80051c0 msgr2=0x7fc8c800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8c80051c0 0x7fc8c800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 -- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8003a60 msgr2=0x7fc8c8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8003a60 0x7fc8c8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.154+0000 7fc8e1c29700 1 -- 172.21.15.92:0/376385480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c8003570 con 0x7fc8c8004810 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8004810 0x7fc8c8004c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009c50 tx=0x7fc8cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f60912cfc356670 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00e710 con 0x7fc8c8004810 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc00e870 con 0x7fc8c8004810 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e1428700 1 -- 172.21.15.92:0/376385480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00ec10 con 0x7fc8c8004810 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e468e700 1 -- 172.21.15.92:0/376385480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8004810 msgr2=0x7fc8c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e468e700 1 --2- 172.21.15.92:0/376385480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8c8004810 0x7fc8c8004c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009c50 tx=0x7fc8cc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e2c2b700 1 -- 172.21.15.92:0/376385480 reap_dead start 2023-10-06T03:03:42.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e468e700 1 -- 172.21.15.92:0/376385480 shutdown_connections 2023-10-06T03:03:42.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e468e700 1 -- 172.21.15.92:0/376385480 >> 172.21.15.92:0/376385480 conn(0x7fc8dc0999f0 msgr2=0x7fc8dc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:42.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e468e700 1 -- 172.21.15.92:0/376385480 shutdown_connections 2023-10-06T03:03:42.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.155+0000 7fc8e468e700 1 -- 172.21.15.92:0/376385480 wait complete. 2023-10-06T03:03:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.156+0000 7fc8e468e700 1 Processor -- start 2023-10-06T03:03:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 -- start start 2023-10-06T03:03:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8004810 0x7fc8dc169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 0x7fc8dc169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc1557e0 0x7fc8dc16a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc1560f0 con 0x7fc8dc1553f0 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0c8880 con 0x7fc8c8004810 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e468e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc09a7a0 con 0x7fc8dc1557e0 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e1c29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 0x7fc8dc169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e1c29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 0x7fc8dc169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32908/0 (socket says 172.21.15.92:32908) 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.157+0000 7fc8e1c29700 1 -- 172.21.15.92:0/229578092 learned_addr learned my addr 172.21.15.92:0/229578092 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e242a700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8004810 0x7fc8dc169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e1c29700 1 -- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc1557e0 msgr2=0x7fc8dc16a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e2c2b700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc1557e0 0x7fc8dc16a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc1557e0 0x7fc8dc16a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e1c29700 1 -- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8004810 msgr2=0x7fc8dc169780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8004810 0x7fc8dc169780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e1c29700 1 -- 172.21.15.92:0/229578092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc09ac30 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e1c29700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 0x7fc8dc169cc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80046c0 tx=0x7fc8cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc023070 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc00ebf0 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc06b360 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc01c650 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.158+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc09af70 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.159+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc063c90 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.159+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fc8cc01ca80 con 0x7fc8dc1553f0 2023-10-06T03:03:42.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.160+0000 7fc8d27fc700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc8c0066fc0 0x7fc8c0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.160+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc8cc085e80 con 0x7fc8dc1553f0 2023-10-06T03:03:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.160+0000 7fc8e242a700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc8c0066fc0 0x7fc8c0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.160+0000 7fc8e242a700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc8c0066fc0 0x7fc8c0069470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8000ef0 tx=0x7fc8d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.164+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8cc052f20 con 0x7fc8dc1553f0 2023-10-06T03:03:42.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.389+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc8dc09a910 con 0x7fc8dc1553f0 2023-10-06T03:03:42.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.489+0000 7fc8d27fc700 1 -- 172.21.15.92:0/229578092 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 136+0+0 (secure 0 0 0) 0x7fc8cc056540 con 0x7fc8dc1553f0 2023-10-06T03:03:42.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc8c0066fc0 msgr2=0x7fc8c0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc8c0066fc0 0x7fc8c0069470 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8000ef0 tx=0x7fc8d8009040 comp rx=0 tx=0).stop 2023-10-06T03:03:42.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 msgr2=0x7fc8dc169cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 0x7fc8dc169cc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80046c0 tx=0x7fc8cc019000 comp rx=0 tx=0).stop 2023-10-06T03:03:42.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 shutdown_connections 2023-10-06T03:03:42.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc8c0066fc0 0x7fc8c0069470 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8dc1557e0 0x7fc8dc16a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8c8004810 0x7fc8dc169780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 --2- 172.21.15.92:0/229578092 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8dc1553f0 0x7fc8dc169cc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 >> 172.21.15.92:0/229578092 conn(0x7fc8dc0999f0 msgr2=0x7fc8dc0c5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 shutdown_connections 2023-10-06T03:03:42.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.491+0000 7fc8e468e700 1 -- 172.21.15.92:0/229578092 wait complete. 2023-10-06T03:03:42.504 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:03:42.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.619+0000 7f6eb84e4700 1 Processor -- start 2023-10-06T03:03:42.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.619+0000 7f6eb84e4700 1 -- start start 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb84e4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6eb006bcd0 0x7f6eb0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb84e4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6eb00c9510 0x7f6eb0141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb84e4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb015c2a0 con 0x7f6eb006c0a0 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb84e4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb00c9900 con 0x7f6eb006bcd0 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb84e4700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb015b690 con 0x7f6eb00c9510 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb6280700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6eb006bcd0 0x7f6eb0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb6280700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6eb006bcd0 0x7f6eb0140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32918/0 (socket says 172.21.15.92:32918) 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.620+0000 7f6eb6280700 1 -- 172.21.15.92:0/1323081844 learned_addr learned my addr 172.21.15.92:0/1323081844 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6a81700 1 --2- 172.21.15.92:0/1323081844 >> v2:172.21.15.136:3300/0 conn(0x7f6eb00c9510 0x7f6eb0141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6280700 1 -- 172.21.15.92:0/1323081844 >> v2:172.21.15.136:3300/0 conn(0x7f6eb00c9510 msgr2=0x7f6eb0141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:42.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6280700 1 --2- 172.21.15.92:0/1323081844 >> v2:172.21.15.136:3300/0 conn(0x7f6eb00c9510 0x7f6eb0141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6280700 1 -- 172.21.15.92:0/1323081844 >> v1:172.21.15.136:6789/0 conn(0x7f6eb006c0a0 legacy=0x7f6eb00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6280700 1 -- 172.21.15.92:0/1323081844 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb015b970 con 0x7f6eb006bcd0 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6a81700 1 --2- 172.21.15.92:0/1323081844 >> v2:172.21.15.136:3300/0 conn(0x7f6eb00c9510 0x7f6eb0141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb6280700 1 --2- 172.21.15.92:0/1323081844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb006bcd0 0x7f6eb0140df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00ce190 tx=0x7f6eac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6480dcd700105d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb527e700 1 -- 172.21.15.92:0/1323081844 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00e070 con 0x7f6eb006bcd0 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb527e700 1 -- 172.21.15.92:0/1323081844 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac00ba50 con 0x7f6eb006bcd0 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb527e700 1 -- 172.21.15.92:0/1323081844 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0092e0 con 0x7f6eb006bcd0 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb84e4700 1 -- 172.21.15.92:0/1323081844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb006bcd0 msgr2=0x7f6eb0140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.621+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/1323081844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb006bcd0 0x7f6eb0140df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00ce190 tx=0x7f6eac009840 comp rx=0 tx=0).stop 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.622+0000 7f6eb84e4700 1 -- 172.21.15.92:0/1323081844 shutdown_connections 2023-10-06T03:03:42.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.622+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/1323081844 >> v2:172.21.15.136:3300/0 conn(0x7f6eb00c9510 0x7f6eb0141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.622+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/1323081844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb006bcd0 0x7f6eb0140df0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.622+0000 7f6eb84e4700 1 -- 172.21.15.92:0/1323081844 >> 172.21.15.92:0/1323081844 conn(0x7f6eb00ce600 msgr2=0x7f6eb00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:42.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.622+0000 7f6eb84e4700 1 -- 172.21.15.92:0/1323081844 shutdown_connections 2023-10-06T03:03:42.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.622+0000 7f6eb84e4700 1 -- 172.21.15.92:0/1323081844 wait complete. 2023-10-06T03:03:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 Processor -- start 2023-10-06T03:03:42.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 -- start start 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6eb006bcd0 0x7f6eb0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6eb006c0a0 0x7f6eb00b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 0x7f6eb00b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00b21f0 con 0x7f6eb00c9510 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0157ba0 con 0x7f6eb006c0a0 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.623+0000 7f6eb84e4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00b3b10 con 0x7f6eb006bcd0 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 0x7f6eb00b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 0x7f6eb00b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32922/0 (socket says 172.21.15.92:32922) 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 -- 172.21.15.92:0/3204876166 learned_addr learned my addr 172.21.15.92:0/3204876166 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6280700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6eb006bcd0 0x7f6eb0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb5a7f700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6eb006c0a0 0x7f6eb00b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 -- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6eb006bcd0 msgr2=0x7f6eb0157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6eb006bcd0 0x7f6eb0157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 -- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6eb006c0a0 msgr2=0x7f6eb00b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6eb006c0a0 0x7f6eb00b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 -- 172.21.15.92:0/3204876166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac009ae0 con 0x7f6eb00c9510 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb6a81700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 0x7f6eb00b18e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800d860 tx=0x7f6ea8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea80125f0 con 0x7f6eb00c9510 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea8012750 con 0x7f6eb00c9510 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb013cdc0 con 0x7f6eb00c9510 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8012ad0 con 0x7f6eb00c9510 2023-10-06T03:03:42.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.624+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb00b34c0 con 0x7f6eb00c9510 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.625+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb00b3cc0 con 0x7f6eb00c9510 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.626+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f6ea8013480 con 0x7f6eb00c9510 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.626+0000 7f6ea67fc700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e94066fc0 0x7f6e94069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.626+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ea8084760 con 0x7f6eb00c9510 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.626+0000 7f6eb6280700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e94066fc0 0x7f6e94069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.626+0000 7f6eb6280700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e94066fc0 0x7f6e94069470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009880 tx=0x7f6eac018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.630+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ea8055310 con 0x7f6eb00c9510 2023-10-06T03:03:42.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.840+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb004c3c0 con 0x7f6e94066fc0 2023-10-06T03:03:42.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.842+0000 7f6ea67fc700 1 -- 172.21.15.92:0/3204876166 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6eb004c3c0 con 0x7f6e94066fc0 2023-10-06T03:03:42.843 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:03:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:03:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:03:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:03:42.844 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:03:42.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.843+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e94066fc0 msgr2=0x7f6e94069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.843+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e94066fc0 0x7f6e94069470 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009880 tx=0x7f6eac018000 comp rx=0 tx=0).stop 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.843+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 msgr2=0x7f6eb00b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.843+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 0x7f6eb00b18e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800d860 tx=0x7f6ea8008040 comp rx=0 tx=0).stop 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 shutdown_connections 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6e94066fc0 0x7f6e94069470 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6eb00c9510 0x7f6eb00b18e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6eb006bcd0 0x7f6eb0157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 --2- 172.21.15.92:0/3204876166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6eb006c0a0 0x7f6eb00b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 >> 172.21.15.92:0/3204876166 conn(0x7f6eb00ce600 msgr2=0x7f6eb00cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 shutdown_connections 2023-10-06T03:03:42.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.844+0000 7f6eb84e4700 1 -- 172.21.15.92:0/3204876166 wait complete. 2023-10-06T03:03:42.857 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:03:42.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.972+0000 7f9f13ce8700 1 Processor -- start 2023-10-06T03:03:42.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.972+0000 7f9f13ce8700 1 -- start start 2023-10-06T03:03:42.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f13ce8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9f0c159c60 0x7f9f0c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f13ce8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9f0c15db10 0x7f9f0c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f13ce8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c15a160 con 0x7f9f0c159890 2023-10-06T03:03:42.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f13ce8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c15df50 con 0x7f9f0c15db10 2023-10-06T03:03:42.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f13ce8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c0d13a0 con 0x7f9f0c159c60 2023-10-06T03:03:42.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f11a84700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9f0c159890 0x7f9f0c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48762/0 (socket says 172.21.15.92:48762) 2023-10-06T03:03:42.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f11a84700 1 -- 172.21.15.92:0/3736042135 learned_addr learned my addr 172.21.15.92:0/3736042135 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:42.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f12285700 1 --2- 172.21.15.92:0/3736042135 >> v2:172.21.15.92:3300/0 conn(0x7f9f0c15db10 0x7f9f0c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.973+0000 7f9f11283700 1 --2- 172.21.15.92:0/3736042135 >> v2:172.21.15.136:3300/0 conn(0x7f9f0c159c60 0x7f9f0c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f10a82700 1 -- 172.21.15.92:0/3736042135 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158040998 0 0) 0x7f9f0c15a160 con 0x7f9f0c159890 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f10a82700 1 -- 172.21.15.92:0/3736042135 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef8003570 con 0x7f9f0c159890 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f12285700 1 -- 172.21.15.92:0/3736042135 >> v2:172.21.15.136:3300/0 conn(0x7f9f0c159c60 msgr2=0x7f9f0c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f12285700 1 --2- 172.21.15.92:0/3736042135 >> v2:172.21.15.136:3300/0 conn(0x7f9f0c159c60 0x7f9f0c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f12285700 1 -- 172.21.15.92:0/3736042135 >> v1:172.21.15.92:6789/0 conn(0x7f9f0c159890 legacy=0x7f9f0c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f12285700 1 -- 172.21.15.92:0/3736042135 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c15e140 con 0x7f9f0c15db10 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f12285700 1 --2- 172.21.15.92:0/3736042135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c15db10 0x7f9f0c069560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0b67f0 tx=0x7f9f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4c8ea531216100 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f10a82700 1 -- 172.21.15.92:0/3736042135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04011630 con 0x7f9f0c15db10 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f10a82700 1 -- 172.21.15.92:0/3736042135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f040117b0 con 0x7f9f0c15db10 2023-10-06T03:03:42.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f10a82700 1 -- 172.21.15.92:0/3736042135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04011b70 con 0x7f9f0c15db10 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f13ce8700 1 -- 172.21.15.92:0/3736042135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c15db10 msgr2=0x7f9f0c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.974+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/3736042135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c15db10 0x7f9f0c069560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0b67f0 tx=0x7f9f040091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.975+0000 7f9f13ce8700 1 -- 172.21.15.92:0/3736042135 shutdown_connections 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.975+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/3736042135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c15db10 0x7f9f0c069560 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.975+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/3736042135 >> v2:172.21.15.136:3300/0 conn(0x7f9f0c159c60 0x7f9f0c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.975+0000 7f9f13ce8700 1 -- 172.21.15.92:0/3736042135 >> 172.21.15.92:0/3736042135 conn(0x7f9f0c068e20 msgr2=0x7f9f0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.975+0000 7f9f13ce8700 1 -- 172.21.15.92:0/3736042135 shutdown_connections 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.975+0000 7f9f13ce8700 1 -- 172.21.15.92:0/3736042135 wait complete. 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 Processor -- start 2023-10-06T03:03:42.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 -- start start 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f0c159890 0x7f9f0c0c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 0x7f9f0c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0c15db10 0x7f9f0c16d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c16d1c0 con 0x7f9f0c159c60 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0c7460 con 0x7f9f0c15db10 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.976+0000 7f9f13ce8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c14c430 con 0x7f9f0c159890 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 0x7f9f0c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 0x7f9f0c0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32946/0 (socket says 172.21.15.92:32946) 2023-10-06T03:03:42.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 -- 172.21.15.92:0/2278930417 learned_addr learned my addr 172.21.15.92:0/2278930417 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11a84700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f0c159890 0x7f9f0c0c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f12285700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0c15db10 0x7f9f0c16d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 -- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f0c159890 msgr2=0x7f9f0c0c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f0c159890 0x7f9f0c0c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 -- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0c15db10 msgr2=0x7f9f0c16d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0c15db10 0x7f9f0c16d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 -- 172.21.15.92:0/2278930417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f04009420 con 0x7f9f0c159c60 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f11283700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 0x7f9f0c0cd6b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00ec80 tx=0x7f9efc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00c850 con 0x7f9f0c159c60 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc00c9b0 con 0x7f9f0c159c60 2023-10-06T03:03:42.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c0cf6d0 con 0x7f9f0c159c60 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00cd50 con 0x7f9f0c159c60 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.977+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c0cfc90 con 0x7f9f0c159c60 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.978+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c16cf90 con 0x7f9f0c159c60 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.979+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f9efc013680 con 0x7f9f0c159c60 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.979+0000 7f9f01ffb700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ef0066fc0 0x7f9ef0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.979+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9efc0848a0 con 0x7f9f0c159c60 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.979+0000 7f9f11a84700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ef0066fc0 0x7f9ef0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.980+0000 7f9f11a84700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ef0066fc0 0x7f9ef0069470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0d00d0 tx=0x7f9f08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:42.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:42.983+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f0c16cf90 con 0x7f9f0c159c60 2023-10-06T03:03:43.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.165+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9f0c16cf90 con 0x7f9f0c159c60 2023-10-06T03:03:43.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.166+0000 7f9f01ffb700 1 -- 172.21.15.92:0/2278930417 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f0c16cf90 con 0x7f9f0c159c60 2023-10-06T03:03:43.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ef0066fc0 msgr2=0x7f9ef0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ef0066fc0 0x7f9ef0069470 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0d00d0 tx=0x7f9f08009040 comp rx=0 tx=0).stop 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 msgr2=0x7f9f0c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 0x7f9f0c0cd6b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00ec80 tx=0x7f9efc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 shutdown_connections 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ef0066fc0 0x7f9ef0069470 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0c15db10 0x7f9f0c16d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f0c159890 0x7f9f0c0c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 --2- 172.21.15.92:0/2278930417 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f0c159c60 0x7f9f0c0cd6b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 >> 172.21.15.92:0/2278930417 conn(0x7f9f0c068e20 msgr2=0x7f9f0c154f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:43.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 shutdown_connections 2023-10-06T03:03:43.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.168+0000 7f9f13ce8700 1 -- 172.21.15.92:0/2278930417 wait complete. 2023-10-06T03:03:43.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000163772 --mode=777 2023-10-06T03:03:43.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.298+0000 7f5aa9569700 1 Processor -- start 2023-10-06T03:03:43.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.298+0000 7f5aa9569700 1 -- start start 2023-10-06T03:03:43.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.299+0000 7f5aa9569700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 0x7f5aa40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.299+0000 7f5aa9569700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa415c320 con 0x7f5aa4155360 2023-10-06T03:03:43.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.299+0000 7f5aa9569700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa40cefc0 con 0x7f5aa40ceb60 2023-10-06T03:03:43.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.299+0000 7f5aa9569700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa415b710 con 0x7f5aa40b42c0 2023-10-06T03:03:43.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.299+0000 7f5aa2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5aa4155360 0x7f5aa40b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48768/0 (socket says 172.21.15.92:48768) 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.299+0000 7f5aa2ffd700 1 -- 172.21.15.92:0/2511583275 learned_addr learned my addr 172.21.15.92:0/2511583275 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973677692 0 0) 0x7f5aa415c320 con 0x7f5aa4155360 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a90003570 con 0x7f5aa4155360 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5a9a7fc700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 0x7f5aa40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057582196 0 0) 0x7f5a90003570 con 0x7f5aa4155360 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 msgr2=0x7f5aa40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 0x7f5aa40ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 >> v1:172.21.15.136:6789/0 conn(0x7f5aa40ceb60 legacy=0x7f5aa40cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa415b9f0 con 0x7f5aa4155360 2023-10-06T03:03:43.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5a94001470 con 0x7f5aa4155360 2023-10-06T03:03:43.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 >> v1:172.21.15.92:6789/0 conn(0x7f5aa4155360 legacy=0x7f5aa40b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 0x7f5a90003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a90004010 0x7f5a90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a90004d50 0x7f5a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa415b710 con 0x7f5a90004d50 2023-10-06T03:03:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9000bbf0 con 0x7f5a90004010 2023-10-06T03:03:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9000bdb0 con 0x7f5aa40b42c0 2023-10-06T03:03:43.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a90004d50 0x7f5a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa2ffd700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 0x7f5a90003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5a9a7fc700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a90004010 0x7f5a90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 -- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 msgr2=0x7f5a90003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa40b42c0 0x7f5a90003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 -- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a90004010 msgr2=0x7f5a90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a90004010 0x7f5a90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 -- 172.21.15.92:0/2511583275 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a90003570 con 0x7f5a90004d50 2023-10-06T03:03:43.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.300+0000 7f5aa37fe700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a90004d50 0x7f5a900051c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c001f90 tx=0x7f5a9c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8de4c23b1409be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c001950 con 0x7f5a90004d50 2023-10-06T03:03:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c00c8b0 con 0x7f5a90004d50 2023-10-06T03:03:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa27fc700 1 -- 172.21.15.92:0/2511583275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00cc70 con 0x7f5a90004d50 2023-10-06T03:03:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa9569700 1 -- 172.21.15.92:0/2511583275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a90004d50 msgr2=0x7f5a900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa9569700 1 --2- 172.21.15.92:0/2511583275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a90004d50 0x7f5a900051c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c001f90 tx=0x7f5a9c001640 comp rx=0 tx=0).stop 2023-10-06T03:03:43.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa37fe700 1 -- 172.21.15.92:0/2511583275 reap_dead start 2023-10-06T03:03:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa9569700 1 -- 172.21.15.92:0/2511583275 shutdown_connections 2023-10-06T03:03:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa9569700 1 -- 172.21.15.92:0/2511583275 >> 172.21.15.92:0/2511583275 conn(0x7f5aa406baf0 msgr2=0x7f5aa406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa9569700 1 -- 172.21.15.92:0/2511583275 shutdown_connections 2023-10-06T03:03:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.301+0000 7f5aa9569700 1 -- 172.21.15.92:0/2511583275 wait complete. 2023-10-06T03:03:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.302+0000 7f5aa9569700 1 Processor -- start 2023-10-06T03:03:43.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.302+0000 7f5aa9569700 1 -- start start 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa9569700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa414b650 0x7f5aa4156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa9569700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 0x7f5aa415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa9569700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aa415cb40 0x7f5aa40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa9569700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa4167f80 con 0x7f5aa40aff10 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa9569700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa415cfb0 con 0x7f5aa415cb40 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa9569700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40b1f80 con 0x7f5aa414b650 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5a9a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 0x7f5aa415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5a9a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 0x7f5aa415c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:32960/0 (socket says 172.21.15.92:32960) 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5a9a7fc700 1 -- 172.21.15.92:0/461324803 learned_addr learned my addr 172.21.15.92:0/461324803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:43.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa2ffd700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa414b650 0x7f5aa4156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.303+0000 7f5aa37fe700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aa415cb40 0x7f5aa40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9a7fc700 1 -- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa414b650 msgr2=0x7f5aa4156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9a7fc700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa414b650 0x7f5aa4156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9a7fc700 1 -- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aa415cb40 msgr2=0x7f5aa40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9a7fc700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aa415cb40 0x7f5aa40b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9a7fc700 1 -- 172.21.15.92:0/461324803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa414ba20 con 0x7f5aa40aff10 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9a7fc700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 0x7f5aa415c600 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c011b40 tx=0x7f5a8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00fa10 con 0x7f5aa40aff10 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c00fb70 con 0x7f5aa40aff10 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c014440 con 0x7f5aa40aff10 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa40b06b0 con 0x7f5aa40aff10 2023-10-06T03:03:43.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.304+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa40b0c70 con 0x7f5aa40aff10 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.305+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa40b2110 con 0x7f5aa40aff10 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.305+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5a8c014870 con 0x7f5aa40aff10 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.306+0000 7f5a9b7fe700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a84066fc0 0x7f5a84069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.306+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5a8c018030 con 0x7f5aa40aff10 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.306+0000 7f5aa2ffd700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a84066fc0 0x7f5a84069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.306+0000 7f5aa2ffd700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a84066fc0 0x7f5a84069470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001ce0 tx=0x7f5a94001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:43.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.310+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a8c052220 con 0x7f5aa40aff10 2023-10-06T03:03:43.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.522+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000163772", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f5aa40718b0 con 0x7f5a84066fc0 2023-10-06T03:03:43.525 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:43.523+0000 7fdd0eb80700 -1 client.5583 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-06T03:03:12.652957+0000 mtime=2023-10-06T03:03:12.652957+0000 ctime=2023-10-06T03:03:12.659044+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000778789"] has_dir_layout 0x555aeb1b0000) 2023-10-06T03:03:43.528 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:43.527+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:43.528 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:43.527+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:43.528 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:43.527+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:43.528 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:43.527+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:43.529 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:03:43.527+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:03:43.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.559+0000 7f5a9b7fe700 1 -- 172.21.15.92:0/461324803 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5aa40718b0 con 0x7f5a84066fc0 2023-10-06T03:03:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a84066fc0 msgr2=0x7f5a84069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a84066fc0 0x7f5a84069470 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5a94001ce0 tx=0x7f5a94001e50 comp rx=0 tx=0).stop 2023-10-06T03:03:43.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 msgr2=0x7f5aa415c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 0x7f5aa415c600 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c011b40 tx=0x7f5a8c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 shutdown_connections 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a84066fc0 0x7f5a84069470 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5aa415cb40 0x7f5aa40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5aa414b650 0x7f5aa4156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 --2- 172.21.15.92:0/461324803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5aa40aff10 0x7f5aa415c600 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 >> 172.21.15.92:0/461324803 conn(0x7f5aa406baf0 msgr2=0x7f5aa40ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 shutdown_connections 2023-10-06T03:03:43.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.561+0000 7f5aa9569700 1 -- 172.21.15.92:0/461324803 wait complete. 2023-10-06T03:03:43.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000163772 2023-10-06T03:03:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.690+0000 7f4e19707700 1 Processor -- start 2023-10-06T03:03:43.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.690+0000 7f4e19707700 1 -- start start 2023-10-06T03:03:43.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e19707700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e14159a60 0x7f4e140c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e19707700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e19707700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140c9cb0 con 0x7f4e140ccc00 2023-10-06T03:03:43.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e19707700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140cbbd0 con 0x7f4e14159a60 2023-10-06T03:03:43.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e19707700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1406dfc0 con 0x7f4e140cb230 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e13fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e14159a60 0x7f4e140c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e137fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4e140ccc00 0x7f4e14069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48772/0 (socket says 172.21.15.92:48772) 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e137fe700 1 -- 172.21.15.92:0/1463545450 learned_addr learned my addr 172.21.15.92:0/1463545450 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e18f06700 1 --2- 172.21.15.92:0/1463545450 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e13fff700 1 -- 172.21.15.92:0/1463545450 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 msgr2=0x7f4e140cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e13fff700 1 --2- 172.21.15.92:0/1463545450 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e13fff700 1 -- 172.21.15.92:0/1463545450 >> v1:172.21.15.92:6789/0 conn(0x7f4e140ccc00 legacy=0x7f4e14069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e13fff700 1 -- 172.21.15.92:0/1463545450 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140c9f30 con 0x7f4e14159a60 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e12ffd700 1 -- 172.21.15.92:0/1463545450 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898586909 0 0) 0x7f4e140c9cb0 con 0x7f4e140ccc00 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e13fff700 1 --2- 172.21.15.92:0/1463545450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14159a60 0x7f4e140c9770 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009910 tx=0x7f4e040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cea4e9ee1f7ceed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:43.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.691+0000 7f4e12ffd700 1 -- 172.21.15.92:0/1463545450 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400b6d0 con 0x7f4e14159a60 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e12ffd700 1 -- 172.21.15.92:0/1463545450 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0400b830 con 0x7f4e14159a60 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e12ffd700 1 -- 172.21.15.92:0/1463545450 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400bbf0 con 0x7f4e14159a60 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 -- 172.21.15.92:0/1463545450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14159a60 msgr2=0x7f4e140c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 --2- 172.21.15.92:0/1463545450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14159a60 0x7f4e140c9770 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009910 tx=0x7f4e040091a0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 -- 172.21.15.92:0/1463545450 shutdown_connections 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 --2- 172.21.15.92:0/1463545450 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 --2- 172.21.15.92:0/1463545450 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e14159a60 0x7f4e140c9770 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 -- 172.21.15.92:0/1463545450 >> 172.21.15.92:0/1463545450 conn(0x7f4e1406baa0 msgr2=0x7f4e1406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 -- 172.21.15.92:0/1463545450 shutdown_connections 2023-10-06T03:03:43.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.692+0000 7f4e19707700 1 -- 172.21.15.92:0/1463545450 wait complete. 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.693+0000 7f4e19707700 1 Processor -- start 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.693+0000 7f4e19707700 1 -- start start 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e19707700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e19707700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 0x7f4e140a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e19707700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14159a60 0x7f4e140a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e19707700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140c8a30 con 0x7f4e140ccc00 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e19707700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140b2d70 con 0x7f4e14159a60 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e19707700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140b3050 con 0x7f4e140cb230 2023-10-06T03:03:43.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 0x7f4e140a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 0x7f4e140a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33004/0 (socket says 172.21.15.92:33004) 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 -- 172.21.15.92:0/1537276998 learned_addr learned my addr 172.21.15.92:0/1537276998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 -- 172.21.15.92:0/1537276998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 msgr2=0x7f4e140b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 -- 172.21.15.92:0/1537276998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14159a60 msgr2=0x7f4e140a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14159a60 0x7f4e140a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 -- 172.21.15.92:0/1537276998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009420 con 0x7f4e140ccc00 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.694+0000 7f4e137fe700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 0x7f4e140a54c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c7e0 tx=0x7f4e08007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.695+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08005b30 con 0x7f4e140ccc00 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.695+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08005c90 con 0x7f4e140ccc00 2023-10-06T03:03:43.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.695+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e140b3330 con 0x7f4e140ccc00 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.695+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08007300 con 0x7f4e140ccc00 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.695+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e140b38f0 con 0x7f4e140ccc00 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.696+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e140ca240 con 0x7f4e140ccc00 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.696+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f4e0801b400 con 0x7f4e140ccc00 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.696+0000 7f4dfbfff700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4e00066fc0 0x7f4e00069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.697+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4e08084230 con 0x7f4e140ccc00 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.697+0000 7f4e13fff700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4e00066fc0 0x7f4e00069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:03:43.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.697+0000 7f4e13fff700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4e00066fc0 0x7f4e00069470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e14171840 tx=0x7f4e04017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:03:43.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.701+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e080513d0 con 0x7f4e140ccc00 2023-10-06T03:03:43.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.915+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000163772", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e140b3b10 con 0x7f4e00066fc0 2023-10-06T03:03:43.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.920+0000 7f4dfbfff700 1 -- 172.21.15.92:0/1537276998 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f4e140b3b10 con 0x7f4e00066fc0 2023-10-06T03:03:43.922 INFO:teuthology.orchestra.run.smithi092.stdout:/volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17 2023-10-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4e00066fc0 msgr2=0x7f4e00069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4e00066fc0 0x7f4e00069470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e14171840 tx=0x7f4e04017000 comp rx=0 tx=0).stop 2023-10-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 msgr2=0x7f4e140a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 0x7f4e140a54c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c7e0 tx=0x7f4e08007df0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 shutdown_connections 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4e00066fc0 0x7f4e00069470 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e14159a60 0x7f4e140a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e140cb230 0x7f4e140b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 --2- 172.21.15.92:0/1537276998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e140ccc00 0x7f4e140a54c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 >> 172.21.15.92:0/1537276998 conn(0x7f4e1406baa0 msgr2=0x7f4e14158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 shutdown_connections 2023-10-06T03:03:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:03:43.922+0000 7f4e19707700 1 -- 172.21.15.92:0/1537276998 wait complete. 2023-10-06T03:03:43.936 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000163772 with 3000 files each 0.1MB size under directory volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17 2023-10-06T03:03:43.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:43.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.0 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:43.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:43.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:43.957 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5615e-05 s, 0.0 kB/s 2023-10-06T03:03:43.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:43.957 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3596e-05 s, 0.0 kB/s 2023-10-06T03:03:44.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2608e-05 s, 0.0 kB/s 2023-10-06T03:03:44.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.3 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7758e-05 s, 0.0 kB/s 2023-10-06T03:03:44.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.4 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2428e-05 s, 0.0 kB/s 2023-10-06T03:03:44.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.180 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.5 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.235 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.006e-05 s, 0.0 kB/s 2023-10-06T03:03:44.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.236 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.6 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.5939e-05 s, 0.0 kB/s 2023-10-06T03:03:44.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.291 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.7 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.347 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8301e-05 s, 0.0 kB/s 2023-10-06T03:03:44.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.8 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.403 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8739e-05 s, 0.0 kB/s 2023-10-06T03:03:44.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.403 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.9 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2342e-05 s, 0.0 kB/s 2023-10-06T03:03:44.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.459 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.10 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.514 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0506e-05 s, 0.0 kB/s 2023-10-06T03:03:44.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.515 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.11 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.570 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2635e-05 s, 0.0 kB/s 2023-10-06T03:03:44.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.570 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.12 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.626 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9644e-05 s, 0.0 kB/s 2023-10-06T03:03:44.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.627 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.13 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.682 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8563e-05 s, 0.0 kB/s 2023-10-06T03:03:44.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.683 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.14 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.738 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.738 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.738 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0023e-05 s, 0.0 kB/s 2023-10-06T03:03:44.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.739 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.15 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.794 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0468e-05 s, 0.0 kB/s 2023-10-06T03:03:44.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.16 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.850 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6066e-05 s, 0.0 kB/s 2023-10-06T03:03:44.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.850 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.17 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.906 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.906 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.906 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8803e-05 s, 0.0 kB/s 2023-10-06T03:03:44.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.906 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.18 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:44.962 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6673e-05 s, 0.0 kB/s 2023-10-06T03:03:44.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:44.962 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.19 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.017 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3111e-05 s, 0.0 kB/s 2023-10-06T03:03:45.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.017 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.20 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3709e-05 s, 0.0 kB/s 2023-10-06T03:03:45.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.072 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.21 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.127 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8995e-05 s, 0.0 kB/s 2023-10-06T03:03:45.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.127 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.22 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-06T03:03:45.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.182 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.23 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.238 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.238 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.238 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0482e-05 s, 0.0 kB/s 2023-10-06T03:03:45.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.239 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.24 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.294 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.294 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.294 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2081e-05 s, 0.0 kB/s 2023-10-06T03:03:45.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.25 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.350 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.350 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.350 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9654e-05 s, 0.0 kB/s 2023-10-06T03:03:45.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.351 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.26 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.406 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1101e-05 s, 0.0 kB/s 2023-10-06T03:03:45.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.407 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.27 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7226e-05 s, 0.0 kB/s 2023-10-06T03:03:45.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.28 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.519 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2195e-05 s, 0.0 kB/s 2023-10-06T03:03:45.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.519 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.29 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.575 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3464e-05 s, 0.0 kB/s 2023-10-06T03:03:45.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.576 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.30 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.631 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.631 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.632 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2609e-05 s, 0.0 kB/s 2023-10-06T03:03:45.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.31 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.687 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.687 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.688 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2258e-05 s, 0.0 kB/s 2023-10-06T03:03:45.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.688 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.32 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1023e-05 s, 0.0 kB/s 2023-10-06T03:03:45.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.744 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.33 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.799 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.178e-05 s, 0.0 kB/s 2023-10-06T03:03:45.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.799 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.34 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.855 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.855 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.855 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1294e-05 s, 0.0 kB/s 2023-10-06T03:03:45.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.856 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.35 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.910 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.911 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9277e-05 s, 0.0 kB/s 2023-10-06T03:03:45.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.911 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.36 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:45.965 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:45.966 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8862e-05 s, 0.0 kB/s 2023-10-06T03:03:45.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:45.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.37 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.022 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.022 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2709e-05 s, 0.0 kB/s 2023-10-06T03:03:46.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.023 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.38 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.077 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.078 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7982e-05 s, 0.0 kB/s 2023-10-06T03:03:46.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.078 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.39 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.133 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.164e-05 s, 0.0 kB/s 2023-10-06T03:03:46.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.40 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.190 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3257e-05 s, 0.0 kB/s 2023-10-06T03:03:46.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.190 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.41 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.245 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0898e-05 s, 0.0 kB/s 2023-10-06T03:03:46.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.42 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.301 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8275e-05 s, 0.0 kB/s 2023-10-06T03:03:46.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.301 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.43 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1086e-05 s, 0.0 kB/s 2023-10-06T03:03:46.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.357 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.44 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.411 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7729e-05 s, 0.0 kB/s 2023-10-06T03:03:46.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.412 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.45 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.468 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9166e-05 s, 0.0 kB/s 2023-10-06T03:03:46.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.468 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.46 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.523 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.524 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.838e-05 s, 0.0 kB/s 2023-10-06T03:03:46.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.524 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.47 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.579 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.580 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8369e-05 s, 0.0 kB/s 2023-10-06T03:03:46.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.580 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.48 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.636 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0445e-05 s, 0.0 kB/s 2023-10-06T03:03:46.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.636 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.49 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.691 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7092e-05 s, 0.0 kB/s 2023-10-06T03:03:46.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.50 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1806e-05 s, 0.0 kB/s 2023-10-06T03:03:46.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.746 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.51 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.801 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.801 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9719e-05 s, 0.0 kB/s 2023-10-06T03:03:46.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.801 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.52 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.858 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6877e-05 s, 0.0 kB/s 2023-10-06T03:03:46.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.858 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.53 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.912 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-06T03:03:46.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.913 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.54 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:46.967 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:46.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:46.968 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2109e-05 s, 0.0 kB/s 2023-10-06T03:03:46.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:46.968 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.55 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.022 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.023 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8139e-05 s, 0.0 kB/s 2023-10-06T03:03:47.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.023 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.56 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8447e-05 s, 0.0 kB/s 2023-10-06T03:03:47.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.079 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.57 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.134 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6168e-05 s, 0.0 kB/s 2023-10-06T03:03:47.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.134 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.58 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.188 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.188 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.188 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7573e-05 s, 0.0 kB/s 2023-10-06T03:03:47.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.189 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.59 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.244 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9057e-05 s, 0.0 kB/s 2023-10-06T03:03:47.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.244 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.60 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6991e-05 s, 0.0 kB/s 2023-10-06T03:03:47.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.300 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.61 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.355 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7826e-05 s, 0.0 kB/s 2023-10-06T03:03:47.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.356 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.62 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7134e-05 s, 0.0 kB/s 2023-10-06T03:03:47.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.411 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.63 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.465 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.466 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5274e-05 s, 0.0 kB/s 2023-10-06T03:03:47.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.466 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.64 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.521 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2138e-05 s, 0.0 kB/s 2023-10-06T03:03:47.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.522 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.65 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.577 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.73e-05 s, 0.0 kB/s 2023-10-06T03:03:47.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.577 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.66 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.633 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9651e-05 s, 0.0 kB/s 2023-10-06T03:03:47.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.633 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.67 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0805e-05 s, 0.0 kB/s 2023-10-06T03:03:47.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.690 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.68 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.745 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.046e-05 s, 0.0 kB/s 2023-10-06T03:03:47.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.745 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.69 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.799 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8005e-05 s, 0.0 kB/s 2023-10-06T03:03:47.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.800 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.70 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.854 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.854 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.855 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0509e-05 s, 0.0 kB/s 2023-10-06T03:03:47.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.855 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.71 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.910 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.910 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0408e-05 s, 0.0 kB/s 2023-10-06T03:03:47.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.910 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.72 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:47.964 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:47.965 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8414e-05 s, 0.0 kB/s 2023-10-06T03:03:47.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:47.965 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.73 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.020 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9522e-05 s, 0.0 kB/s 2023-10-06T03:03:48.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.020 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.74 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.076 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.046e-05 s, 0.0 kB/s 2023-10-06T03:03:48.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.076 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.75 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.131 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0222e-05 s, 0.0 kB/s 2023-10-06T03:03:48.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.132 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.76 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.187 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0667e-05 s, 0.0 kB/s 2023-10-06T03:03:48.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.187 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.77 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.242 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0248e-05 s, 0.0 kB/s 2023-10-06T03:03:48.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.242 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.78 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.297 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.298 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9924e-05 s, 0.0 kB/s 2023-10-06T03:03:48.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.298 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.79 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.353 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.353 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7834e-05 s, 0.0 kB/s 2023-10-06T03:03:48.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.80 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8756e-05 s, 0.0 kB/s 2023-10-06T03:03:48.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.81 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7991e-05 s, 0.0 kB/s 2023-10-06T03:03:48.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.82 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.517 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-10-06T03:03:48.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.518 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.83 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.573 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9658e-05 s, 0.0 kB/s 2023-10-06T03:03:48.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.84 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.628 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0286e-05 s, 0.0 kB/s 2023-10-06T03:03:48.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.629 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.85 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.685 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3921e-05 s, 0.0 kB/s 2023-10-06T03:03:48.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.685 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.86 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.740 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.874e-05 s, 0.0 kB/s 2023-10-06T03:03:48.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.741 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.87 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.796 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8962e-05 s, 0.0 kB/s 2023-10-06T03:03:48.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.88 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.852 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.181e-05 s, 0.0 kB/s 2023-10-06T03:03:48.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.852 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.89 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.910 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.910 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.910 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2114e-05 s, 0.0 kB/s 2023-10-06T03:03:48.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.910 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.90 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:48.965 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:48.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:48.966 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2201e-05 s, 0.0 kB/s 2023-10-06T03:03:48.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:48.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.91 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.021 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0964e-05 s, 0.0 kB/s 2023-10-06T03:03:49.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.022 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.92 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.077 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9988e-05 s, 0.0 kB/s 2023-10-06T03:03:49.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.078 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.93 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.133 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.088e-05 s, 0.0 kB/s 2023-10-06T03:03:49.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.94 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.189 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9338e-05 s, 0.0 kB/s 2023-10-06T03:03:49.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.190 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.95 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.245 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.047e-05 s, 0.0 kB/s 2023-10-06T03:03:49.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.96 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2456e-05 s, 0.0 kB/s 2023-10-06T03:03:49.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.301 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.97 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.356 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.356 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1189e-05 s, 0.0 kB/s 2023-10-06T03:03:49.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.356 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.98 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.411 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.411 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.174e-05 s, 0.0 kB/s 2023-10-06T03:03:49.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.411 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.99 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.466 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0936e-05 s, 0.0 kB/s 2023-10-06T03:03:49.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.466 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.521 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6279e-05 s, 0.0 kB/s 2023-10-06T03:03:49.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.522 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.577 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.743e-05 s, 0.0 kB/s 2023-10-06T03:03:49.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.578 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.633 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2943e-05 s, 0.0 kB/s 2023-10-06T03:03:49.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.634 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.690 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8205e-05 s, 0.0 kB/s 2023-10-06T03:03:49.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.690 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.745 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9693e-05 s, 0.0 kB/s 2023-10-06T03:03:49.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.746 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.801 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.801 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.801 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0124e-05 s, 0.0 kB/s 2023-10-06T03:03:49.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.801 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.858 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0948e-05 s, 0.0 kB/s 2023-10-06T03:03:49.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.858 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.914 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.847e-05 s, 0.0 kB/s 2023-10-06T03:03:49.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8317e-05 s, 0.0 kB/s 2023-10-06T03:03:49.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:49.970 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.279e-05 s, 0.0 kB/s 2023-10-06T03:03:50.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.025 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.081 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1972e-05 s, 0.0 kB/s 2023-10-06T03:03:50.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.082 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.138 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.139 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8274e-05 s, 0.0 kB/s 2023-10-06T03:03:50.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.139 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.194 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.194 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.194 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9094e-05 s, 0.0 kB/s 2023-10-06T03:03:50.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.194 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.250 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.250 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8478e-05 s, 0.0 kB/s 2023-10-06T03:03:50.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.250 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8878e-05 s, 0.0 kB/s 2023-10-06T03:03:50.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.305 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.361 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-06T03:03:50.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.362 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.417 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1523e-05 s, 0.0 kB/s 2023-10-06T03:03:50.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.417 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.472 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.967e-05 s, 0.0 kB/s 2023-10-06T03:03:50.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.472 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.527 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0095e-05 s, 0.0 kB/s 2023-10-06T03:03:50.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.527 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.583 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.584 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.584 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0026e-05 s, 0.0 kB/s 2023-10-06T03:03:50.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.585 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.638 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.962e-05 s, 0.0 kB/s 2023-10-06T03:03:50.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.639 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.694 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.695 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8985e-05 s, 0.0 kB/s 2023-10-06T03:03:50.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.695 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.750 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.751 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.751 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7941e-05 s, 0.0 kB/s 2023-10-06T03:03:50.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.751 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.805 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.805 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.805 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1555e-05 s, 0.0 kB/s 2023-10-06T03:03:50.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.806 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.860 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.861 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.861 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8794e-05 s, 0.0 kB/s 2023-10-06T03:03:50.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.861 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.915 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8343e-05 s, 0.0 kB/s 2023-10-06T03:03:50.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.916 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:50.971 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-10-06T03:03:50.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:50.971 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.027 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.027 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.027 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1287e-05 s, 0.0 kB/s 2023-10-06T03:03:51.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.027 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.083 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.083 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.083 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9406e-05 s, 0.0 kB/s 2023-10-06T03:03:51.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.084 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.139 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7649e-05 s, 0.0 kB/s 2023-10-06T03:03:51.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.139 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.194 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.195 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.195 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1405e-05 s, 0.0 kB/s 2023-10-06T03:03:51.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.195 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.250 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.250 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.251 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0758e-05 s, 0.0 kB/s 2023-10-06T03:03:51.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.251 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0036e-05 s, 0.0 kB/s 2023-10-06T03:03:51.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.307 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.362 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.363 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9357e-05 s, 0.0 kB/s 2023-10-06T03:03:51.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.363 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.418 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0967e-05 s, 0.0 kB/s 2023-10-06T03:03:51.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.418 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.473 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6893e-05 s, 0.0 kB/s 2023-10-06T03:03:51.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.473 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.528 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9037e-05 s, 0.0 kB/s 2023-10-06T03:03:51.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.529 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.584 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9353e-05 s, 0.0 kB/s 2023-10-06T03:03:51.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.585 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.640 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0793e-05 s, 0.0 kB/s 2023-10-06T03:03:51.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.640 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.695 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.696 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9012e-05 s, 0.0 kB/s 2023-10-06T03:03:51.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.696 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.751 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8081e-05 s, 0.0 kB/s 2023-10-06T03:03:51.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.752 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.807 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8845e-05 s, 0.0 kB/s 2023-10-06T03:03:51.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.807 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.862 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.863 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.148e-05 s, 0.0 kB/s 2023-10-06T03:03:51.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.863 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.918 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.918 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.918 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6589e-05 s, 0.0 kB/s 2023-10-06T03:03:51.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.919 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:51.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:51.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:51.974 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7449e-05 s, 0.0 kB/s 2023-10-06T03:03:51.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:51.974 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.030 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7585e-05 s, 0.0 kB/s 2023-10-06T03:03:52.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.030 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.085 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.085 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.085 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.9913e-05 s, 0.0 kB/s 2023-10-06T03:03:52.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.085 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.140 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.141 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3118e-05 s, 0.0 kB/s 2023-10-06T03:03:52.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.141 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.196 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.196 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.196 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6887e-05 s, 0.0 kB/s 2023-10-06T03:03:52.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.196 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.252 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8344e-05 s, 0.0 kB/s 2023-10-06T03:03:52.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.307 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.308 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8113e-05 s, 0.0 kB/s 2023-10-06T03:03:52.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.308 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.363 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0177e-05 s, 0.0 kB/s 2023-10-06T03:03:52.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.363 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.418 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.306e-05 s, 0.0 kB/s 2023-10-06T03:03:52.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.419 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.473 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0779e-05 s, 0.0 kB/s 2023-10-06T03:03:52.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.474 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.529 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2078e-05 s, 0.0 kB/s 2023-10-06T03:03:52.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.530 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.586 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.590 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.3952e-05 s, 0.0 kB/s 2023-10-06T03:03:52.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.590 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.642 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.942e-05 s, 0.0 kB/s 2023-10-06T03:03:52.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.642 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.697 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9645e-05 s, 0.0 kB/s 2023-10-06T03:03:52.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.698 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.753 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.753 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.753 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8604e-05 s, 0.0 kB/s 2023-10-06T03:03:52.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.753 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.809 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8012e-05 s, 0.0 kB/s 2023-10-06T03:03:52.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.809 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.865 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0232e-05 s, 0.0 kB/s 2023-10-06T03:03:52.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.865 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.920 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.921 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6198e-05 s, 0.0 kB/s 2023-10-06T03:03:52.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.921 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:52.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:52.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:52.976 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1765e-05 s, 0.0 kB/s 2023-10-06T03:03:52.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:52.977 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.031 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.862e-05 s, 0.0 kB/s 2023-10-06T03:03:53.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.032 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.087 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.088 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.088 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1943e-05 s, 0.0 kB/s 2023-10-06T03:03:53.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.088 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.143 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.143 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.143 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7804e-05 s, 0.0 kB/s 2023-10-06T03:03:53.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.143 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.197 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.197 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.198 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0231e-05 s, 0.0 kB/s 2023-10-06T03:03:53.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.198 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.253 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8849e-05 s, 0.0 kB/s 2023-10-06T03:03:53.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.254 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0751e-05 s, 0.0 kB/s 2023-10-06T03:03:53.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.309 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.363 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0526e-05 s, 0.0 kB/s 2023-10-06T03:03:53.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.364 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.421 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.421 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.421 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1177e-05 s, 0.0 kB/s 2023-10-06T03:03:53.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.421 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.476 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.476 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.476 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8655e-05 s, 0.0 kB/s 2023-10-06T03:03:53.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.477 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.532 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.532 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0236e-05 s, 0.0 kB/s 2023-10-06T03:03:53.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0221e-05 s, 0.0 kB/s 2023-10-06T03:03:53.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.588 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.644 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9023e-05 s, 0.0 kB/s 2023-10-06T03:03:53.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.644 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.699 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.700 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6295e-05 s, 0.0 kB/s 2023-10-06T03:03:53.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.700 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1436e-05 s, 0.0 kB/s 2023-10-06T03:03:53.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.755 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.811 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.811 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.084e-05 s, 0.0 kB/s 2023-10-06T03:03:53.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.811 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.866 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8136e-05 s, 0.0 kB/s 2023-10-06T03:03:53.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.866 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.920 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.920 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.921 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0889e-05 s, 0.0 kB/s 2023-10-06T03:03:53.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.921 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:53.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:53.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:53.976 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7863e-05 s, 0.0 kB/s 2023-10-06T03:03:53.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:53.976 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.032 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-06T03:03:54.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.033 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.088 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7514e-05 s, 0.0 kB/s 2023-10-06T03:03:54.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.088 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.144 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8284e-05 s, 0.0 kB/s 2023-10-06T03:03:54.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.200 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0667e-05 s, 0.0 kB/s 2023-10-06T03:03:54.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.200 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.255 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.255 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.256 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3027e-05 s, 0.0 kB/s 2023-10-06T03:03:54.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.256 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7958e-05 s, 0.0 kB/s 2023-10-06T03:03:54.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.366 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.367 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1387e-05 s, 0.0 kB/s 2023-10-06T03:03:54.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.367 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.421 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.422 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2509e-05 s, 0.0 kB/s 2023-10-06T03:03:54.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.422 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.476 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.477 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0574e-05 s, 0.0 kB/s 2023-10-06T03:03:54.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.477 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9222e-05 s, 0.0 kB/s 2023-10-06T03:03:54.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7027e-05 s, 0.0 kB/s 2023-10-06T03:03:54.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1118e-05 s, 0.0 kB/s 2023-10-06T03:03:54.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.643 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.698 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1167e-05 s, 0.0 kB/s 2023-10-06T03:03:54.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.699 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.753 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.753 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.753 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1434e-05 s, 0.0 kB/s 2023-10-06T03:03:54.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.754 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.808 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.808 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.808 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8536e-05 s, 0.0 kB/s 2023-10-06T03:03:54.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.809 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.864 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.864 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.864 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-06T03:03:54.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.865 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.919 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.919 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.919 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9223e-05 s, 0.0 kB/s 2023-10-06T03:03:54.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.920 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:54.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3635e-05 s, 0.0 kB/s 2023-10-06T03:03:54.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:54.975 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.029 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.029 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0251e-05 s, 0.0 kB/s 2023-10-06T03:03:55.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.030 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.085 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.085 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.085 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1151e-05 s, 0.0 kB/s 2023-10-06T03:03:55.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.085 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.141 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.141 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.141 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1153e-05 s, 0.0 kB/s 2023-10-06T03:03:55.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.141 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.196 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.197 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0356e-05 s, 0.0 kB/s 2023-10-06T03:03:55.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.197 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.252 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.252 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.252 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7642e-05 s, 0.0 kB/s 2023-10-06T03:03:55.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.253 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.307 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.307 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.308 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7025e-05 s, 0.0 kB/s 2023-10-06T03:03:55.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.308 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.362 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.363 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0522e-05 s, 0.0 kB/s 2023-10-06T03:03:55.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.363 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.418 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7081e-05 s, 0.0 kB/s 2023-10-06T03:03:55.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.418 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.473 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.473 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1195e-05 s, 0.0 kB/s 2023-10-06T03:03:55.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.473 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.528 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9207e-05 s, 0.0 kB/s 2023-10-06T03:03:55.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.528 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.582 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1321e-05 s, 0.0 kB/s 2023-10-06T03:03:55.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.583 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.637 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.637 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.638 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0828e-05 s, 0.0 kB/s 2023-10-06T03:03:55.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.638 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.692 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.693 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.693 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.235e-05 s, 0.0 kB/s 2023-10-06T03:03:55.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.748 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8334e-05 s, 0.0 kB/s 2023-10-06T03:03:55.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.748 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2514e-05 s, 0.0 kB/s 2023-10-06T03:03:55.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.804 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.858 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.137e-05 s, 0.0 kB/s 2023-10-06T03:03:55.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.859 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.914 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9304e-05 s, 0.0 kB/s 2023-10-06T03:03:55.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.914 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:55.968 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1772e-05 s, 0.0 kB/s 2023-10-06T03:03:55.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:55.969 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9809e-05 s, 0.0 kB/s 2023-10-06T03:03:56.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.025 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.081 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.0075e-05 s, 0.0 kB/s 2023-10-06T03:03:56.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.137 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8686e-05 s, 0.0 kB/s 2023-10-06T03:03:56.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.138 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.192 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.192 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.193 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1293e-05 s, 0.0 kB/s 2023-10-06T03:03:56.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.193 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.248 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.248 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.248 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7203e-05 s, 0.0 kB/s 2023-10-06T03:03:56.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.249 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.303 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.304 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9138e-05 s, 0.0 kB/s 2023-10-06T03:03:56.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.304 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.358 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.358 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.358 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3038e-05 s, 0.0 kB/s 2023-10-06T03:03:56.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.359 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.413 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.413 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.413 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8065e-05 s, 0.0 kB/s 2023-10-06T03:03:56.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.469 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9766e-05 s, 0.0 kB/s 2023-10-06T03:03:56.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.470 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.524 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.525 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0057e-05 s, 0.0 kB/s 2023-10-06T03:03:56.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.525 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.580 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.580 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.580 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2421e-05 s, 0.0 kB/s 2023-10-06T03:03:56.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.581 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.636 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.636 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.399e-05 s, 0.0 kB/s 2023-10-06T03:03:56.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.636 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.691 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7048e-05 s, 0.0 kB/s 2023-10-06T03:03:56.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.691 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.747 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1652e-05 s, 0.0 kB/s 2023-10-06T03:03:56.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.747 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.802 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0464e-05 s, 0.0 kB/s 2023-10-06T03:03:56.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.803 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.858 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8203e-05 s, 0.0 kB/s 2023-10-06T03:03:56.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.858 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.913 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.832e-05 s, 0.0 kB/s 2023-10-06T03:03:56.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.914 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:56.967 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1567e-05 s, 0.0 kB/s 2023-10-06T03:03:56.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:56.968 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.023 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9517e-05 s, 0.0 kB/s 2023-10-06T03:03:57.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.023 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.079 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-10-06T03:03:57.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.079 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.134 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1345e-05 s, 0.0 kB/s 2023-10-06T03:03:57.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.135 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.190 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.190 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.190 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6936e-05 s, 0.0 kB/s 2023-10-06T03:03:57.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.190 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.246 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7737e-05 s, 0.0 kB/s 2023-10-06T03:03:57.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.302 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2894e-05 s, 0.0 kB/s 2023-10-06T03:03:57.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.302 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.357 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-06T03:03:57.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.358 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.413 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.413 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.413 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1761e-05 s, 0.0 kB/s 2023-10-06T03:03:57.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.414 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.469 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.470 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7984e-05 s, 0.0 kB/s 2023-10-06T03:03:57.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.470 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9547e-05 s, 0.0 kB/s 2023-10-06T03:03:57.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.526 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.581 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1156e-05 s, 0.0 kB/s 2023-10-06T03:03:57.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.582 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.636 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.637 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2801e-05 s, 0.0 kB/s 2023-10-06T03:03:57.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.637 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.692 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.692 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7657e-05 s, 0.0 kB/s 2023-10-06T03:03:57.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.746 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8212e-05 s, 0.0 kB/s 2023-10-06T03:03:57.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.747 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.803 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9077e-05 s, 0.0 kB/s 2023-10-06T03:03:57.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.804 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.859 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9911e-05 s, 0.0 kB/s 2023-10-06T03:03:57.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.859 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.914 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1084e-05 s, 0.0 kB/s 2023-10-06T03:03:57.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.914 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:57.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:57.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:57.969 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7479e-05 s, 0.0 kB/s 2023-10-06T03:03:57.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:57.969 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.024 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.902e-05 s, 0.0 kB/s 2023-10-06T03:03:58.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.024 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8843e-05 s, 0.0 kB/s 2023-10-06T03:03:58.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.079 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.134 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9752e-05 s, 0.0 kB/s 2023-10-06T03:03:58.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.134 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.188 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.188 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.189 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9817e-05 s, 0.0 kB/s 2023-10-06T03:03:58.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.189 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.243 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.244 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.244 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9177e-05 s, 0.0 kB/s 2023-10-06T03:03:58.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.244 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.299 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2833e-05 s, 0.0 kB/s 2023-10-06T03:03:58.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.299 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.353 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.353 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.353 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0652e-05 s, 0.0 kB/s 2023-10-06T03:03:58.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.354 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8062e-05 s, 0.0 kB/s 2023-10-06T03:03:58.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.409 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-06T03:03:58.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.464 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.519 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2432e-05 s, 0.0 kB/s 2023-10-06T03:03:58.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.519 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.575 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.576 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.576 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4052e-05 s, 0.0 kB/s 2023-10-06T03:03:58.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.576 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.631 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.631 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.631 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1213e-05 s, 0.0 kB/s 2023-10-06T03:03:58.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.687 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1064e-05 s, 0.0 kB/s 2023-10-06T03:03:58.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.687 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.742 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6981e-05 s, 0.0 kB/s 2023-10-06T03:03:58.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.746 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.798 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.799 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-06T03:03:58.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.799 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.854 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.854 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.854 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9607e-05 s, 0.0 kB/s 2023-10-06T03:03:58.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.854 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.909 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.910 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0585e-05 s, 0.0 kB/s 2023-10-06T03:03:58.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.910 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-10-06T03:03:58.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:58.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.021 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2893e-05 s, 0.0 kB/s 2023-10-06T03:03:59.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.022 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.077 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.077 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.077 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0951e-05 s, 0.0 kB/s 2023-10-06T03:03:59.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.077 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.133 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9074e-05 s, 0.0 kB/s 2023-10-06T03:03:59.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.189 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7552e-05 s, 0.0 kB/s 2023-10-06T03:03:59.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.189 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.245 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0812e-05 s, 0.0 kB/s 2023-10-06T03:03:59.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.301 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1059e-05 s, 0.0 kB/s 2023-10-06T03:03:59.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.301 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.357 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.357 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.357 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9414e-05 s, 0.0 kB/s 2023-10-06T03:03:59.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.358 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.412 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.413 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1886e-05 s, 0.0 kB/s 2023-10-06T03:03:59.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.467 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.467 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.468 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6781e-05 s, 0.0 kB/s 2023-10-06T03:03:59.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.468 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.523 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.523 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.523 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1457e-05 s, 0.0 kB/s 2023-10-06T03:03:59.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.523 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.932e-05 s, 0.0 kB/s 2023-10-06T03:03:59.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.579 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.635 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9764e-05 s, 0.0 kB/s 2023-10-06T03:03:59.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.636 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.691 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.94e-05 s, 0.0 kB/s 2023-10-06T03:03:59.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.692 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.747 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.747 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.747 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7817e-05 s, 0.0 kB/s 2023-10-06T03:03:59.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.747 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.802 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8216e-05 s, 0.0 kB/s 2023-10-06T03:03:59.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.804 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.859 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.859 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.859 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9587e-05 s, 0.0 kB/s 2023-10-06T03:03:59.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.860 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6899e-05 s, 0.0 kB/s 2023-10-06T03:03:59.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:03:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:03:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:03:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7911e-05 s, 0.0 kB/s 2023-10-06T03:03:59.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:03:59.971 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.026 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.026 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.027 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0522e-05 s, 0.0 kB/s 2023-10-06T03:04:00.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.027 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.5562e-05 s, 0.0 kB/s 2023-10-06T03:04:00.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.082 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.136 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.137 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2583e-05 s, 0.0 kB/s 2023-10-06T03:04:00.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.137 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.192 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.192 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.192 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.935e-05 s, 0.0 kB/s 2023-10-06T03:04:00.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.192 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.247 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.248 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8559e-05 s, 0.0 kB/s 2023-10-06T03:04:00.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.248 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.302 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.303 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7773e-05 s, 0.0 kB/s 2023-10-06T03:04:00.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.303 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.357 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.358 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.85e-05 s, 0.0 kB/s 2023-10-06T03:04:00.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.358 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.412 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.413 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4739e-05 s, 0.0 kB/s 2023-10-06T03:04:00.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7216e-05 s, 0.0 kB/s 2023-10-06T03:04:00.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.468 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.523 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.524 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1117e-05 s, 0.0 kB/s 2023-10-06T03:04:00.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.524 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.579 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.579 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.579 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1539e-05 s, 0.0 kB/s 2023-10-06T03:04:00.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.579 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.635 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9709e-05 s, 0.0 kB/s 2023-10-06T03:04:00.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.636 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.690 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.691 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.691 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9452e-05 s, 0.0 kB/s 2023-10-06T03:04:00.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.691 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.746 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2352e-05 s, 0.0 kB/s 2023-10-06T03:04:00.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.747 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.802 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.802 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.802 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8696e-05 s, 0.0 kB/s 2023-10-06T03:04:00.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.802 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.858 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9688e-05 s, 0.0 kB/s 2023-10-06T03:04:00.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.858 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5324e-05 s, 0.0 kB/s 2023-10-06T03:04:00.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.914 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:00.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:00.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:00.970 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.167e-05 s, 0.0 kB/s 2023-10-06T03:04:00.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:00.970 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7387e-05 s, 0.0 kB/s 2023-10-06T03:04:01.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.025 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.081 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1863e-05 s, 0.0 kB/s 2023-10-06T03:04:01.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.135 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.136 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.136 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2146e-05 s, 0.0 kB/s 2023-10-06T03:04:01.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.136 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.190 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.191 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9914e-05 s, 0.0 kB/s 2023-10-06T03:04:01.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.191 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7237e-05 s, 0.0 kB/s 2023-10-06T03:04:01.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.300 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.300 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.301 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8324e-05 s, 0.0 kB/s 2023-10-06T03:04:01.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.301 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.356 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7596e-05 s, 0.0 kB/s 2023-10-06T03:04:01.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.356 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.411 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.411 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6656e-05 s, 0.0 kB/s 2023-10-06T03:04:01.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.411 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.465 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.028e-05 s, 0.0 kB/s 2023-10-06T03:04:01.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.466 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.522 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.522 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7734e-05 s, 0.0 kB/s 2023-10-06T03:04:01.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.522 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.578 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.578 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.578 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1348e-05 s, 0.0 kB/s 2023-10-06T03:04:01.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.579 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.634 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.287e-05 s, 0.0 kB/s 2023-10-06T03:04:01.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.634 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.688 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0971e-05 s, 0.0 kB/s 2023-10-06T03:04:01.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.689 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.744 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0412e-05 s, 0.0 kB/s 2023-10-06T03:04:01.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.744 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.800 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0549e-05 s, 0.0 kB/s 2023-10-06T03:04:01.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.800 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.856 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0161e-05 s, 0.0 kB/s 2023-10-06T03:04:01.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.856 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.912 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8575e-05 s, 0.0 kB/s 2023-10-06T03:04:01.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.912 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9615e-05 s, 0.0 kB/s 2023-10-06T03:04:01.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:01.968 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7538e-05 s, 0.0 kB/s 2023-10-06T03:04:02.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.026 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.079 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.079 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.080 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1336e-05 s, 0.0 kB/s 2023-10-06T03:04:02.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.080 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.135 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.135 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.135 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8375e-05 s, 0.0 kB/s 2023-10-06T03:04:02.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.135 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.190 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.191 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.191 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8192e-05 s, 0.0 kB/s 2023-10-06T03:04:02.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.191 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.245 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.246 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.246 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.991e-05 s, 0.0 kB/s 2023-10-06T03:04:02.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.246 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.301 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.301 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9207e-05 s, 0.0 kB/s 2023-10-06T03:04:02.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.301 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.356 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.357 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.357 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.958e-05 s, 0.0 kB/s 2023-10-06T03:04:02.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.357 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.411 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.412 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.412 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0617e-05 s, 0.0 kB/s 2023-10-06T03:04:02.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.412 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.467 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.467 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0843e-05 s, 0.0 kB/s 2023-10-06T03:04:02.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.467 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.522 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1063e-05 s, 0.0 kB/s 2023-10-06T03:04:02.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.522 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.578 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.578 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.578 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9949e-05 s, 0.0 kB/s 2023-10-06T03:04:02.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.578 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.634 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0809e-05 s, 0.0 kB/s 2023-10-06T03:04:02.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.634 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.688 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3023e-05 s, 0.0 kB/s 2023-10-06T03:04:02.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.689 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.744 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0008e-05 s, 0.0 kB/s 2023-10-06T03:04:02.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.744 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.800 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8989e-05 s, 0.0 kB/s 2023-10-06T03:04:02.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.800 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.854 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.855 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.855 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1428e-05 s, 0.0 kB/s 2023-10-06T03:04:02.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.855 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.909 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9984e-05 s, 0.0 kB/s 2023-10-06T03:04:02.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.910 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:02.965 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2279e-05 s, 0.0 kB/s 2023-10-06T03:04:02.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:02.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.021 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7276e-05 s, 0.0 kB/s 2023-10-06T03:04:03.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.022 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.076 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.076 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.077 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8701e-05 s, 0.0 kB/s 2023-10-06T03:04:03.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.077 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.132 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9169e-05 s, 0.0 kB/s 2023-10-06T03:04:03.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.132 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.187 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2446e-05 s, 0.0 kB/s 2023-10-06T03:04:03.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.187 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.241 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.242 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9261e-05 s, 0.0 kB/s 2023-10-06T03:04:03.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.242 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7081e-05 s, 0.0 kB/s 2023-10-06T03:04:03.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.353 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.981e-05 s, 0.0 kB/s 2023-10-06T03:04:03.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1652e-05 s, 0.0 kB/s 2023-10-06T03:04:03.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7729e-05 s, 0.0 kB/s 2023-10-06T03:04:03.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.542 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.542 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.5876e-05 s, 0.0 kB/s 2023-10-06T03:04:03.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.543 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.573 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.586 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9489e-05 s, 0.0 kB/s 2023-10-06T03:04:03.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.629 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8134e-05 s, 0.0 kB/s 2023-10-06T03:04:03.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.629 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.685 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.261e-05 s, 0.0 kB/s 2023-10-06T03:04:03.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.685 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-10-06T03:04:03.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.740 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.796 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6905e-05 s, 0.0 kB/s 2023-10-06T03:04:03.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.852 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.852 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.0596e-05 s, 0.0 kB/s 2023-10-06T03:04:03.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.852 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.906 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8435e-05 s, 0.0 kB/s 2023-10-06T03:04:03.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.907 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:03.961 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:03.962 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:03.962 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7789e-05 s, 0.0 kB/s 2023-10-06T03:04:03.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:03.962 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.017 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8679e-05 s, 0.0 kB/s 2023-10-06T03:04:04.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.018 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.073 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1702e-05 s, 0.0 kB/s 2023-10-06T03:04:04.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.074 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.129 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1637e-05 s, 0.0 kB/s 2023-10-06T03:04:04.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.129 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7919e-05 s, 0.0 kB/s 2023-10-06T03:04:04.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.239 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6221e-05 s, 0.0 kB/s 2023-10-06T03:04:04.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.241 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.295 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6518e-05 s, 0.0 kB/s 2023-10-06T03:04:04.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.296 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.350 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.350 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.350 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0556e-05 s, 0.0 kB/s 2023-10-06T03:04:04.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.351 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9211e-05 s, 0.0 kB/s 2023-10-06T03:04:04.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.406 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.461 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2218e-05 s, 0.0 kB/s 2023-10-06T03:04:04.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.461 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.515 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9376e-05 s, 0.0 kB/s 2023-10-06T03:04:04.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.516 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.571 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9479e-05 s, 0.0 kB/s 2023-10-06T03:04:04.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.571 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.626 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1097e-05 s, 0.0 kB/s 2023-10-06T03:04:04.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.626 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.682 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0524e-05 s, 0.0 kB/s 2023-10-06T03:04:04.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.682 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8806e-05 s, 0.0 kB/s 2023-10-06T03:04:04.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.007e-05 s, 0.0 kB/s 2023-10-06T03:04:04.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.849 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.849 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.12e-05 s, 0.0 kB/s 2023-10-06T03:04:04.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-10-06T03:04:04.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:04.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:04.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:04.960 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5921e-05 s, 0.0 kB/s 2023-10-06T03:04:04.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:04.960 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.015 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0486e-05 s, 0.0 kB/s 2023-10-06T03:04:05.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9619e-05 s, 0.0 kB/s 2023-10-06T03:04:05.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.072 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.128 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2418e-05 s, 0.0 kB/s 2023-10-06T03:04:05.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.130 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.183 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.997e-05 s, 0.0 kB/s 2023-10-06T03:04:05.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.183 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.239 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1129e-05 s, 0.0 kB/s 2023-10-06T03:04:05.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.295 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1668e-05 s, 0.0 kB/s 2023-10-06T03:04:05.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.350 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.350 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.350 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8553e-05 s, 0.0 kB/s 2023-10-06T03:04:05.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.351 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.406 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-06T03:04:05.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.407 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.462 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1113e-05 s, 0.0 kB/s 2023-10-06T03:04:05.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.518 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0507e-05 s, 0.0 kB/s 2023-10-06T03:04:05.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.519 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.575 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.916e-05 s, 0.0 kB/s 2023-10-06T03:04:05.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.575 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.630 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.630 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.631 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8672e-05 s, 0.0 kB/s 2023-10-06T03:04:05.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.631 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.686 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.686 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9024e-05 s, 0.0 kB/s 2023-10-06T03:04:05.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.686 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.741 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1002e-05 s, 0.0 kB/s 2023-10-06T03:04:05.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.742 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.798 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.798 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.798 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7745e-05 s, 0.0 kB/s 2023-10-06T03:04:05.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.799 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0439e-05 s, 0.0 kB/s 2023-10-06T03:04:05.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.855 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.910 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.911 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9028e-05 s, 0.0 kB/s 2023-10-06T03:04:05.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.911 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:05.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:05.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:05.966 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8439e-05 s, 0.0 kB/s 2023-10-06T03:04:05.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:05.966 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.021 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.021 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2034e-05 s, 0.0 kB/s 2023-10-06T03:04:06.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.022 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.076 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.076 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.076 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0985e-05 s, 0.0 kB/s 2023-10-06T03:04:06.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.077 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.133 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0138e-05 s, 0.0 kB/s 2023-10-06T03:04:06.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.189 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.189 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0409e-05 s, 0.0 kB/s 2023-10-06T03:04:06.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.189 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.243 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.244 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.244 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9807e-05 s, 0.0 kB/s 2023-10-06T03:04:06.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.244 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.299 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7955e-05 s, 0.0 kB/s 2023-10-06T03:04:06.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.299 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.354 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.355 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9858e-05 s, 0.0 kB/s 2023-10-06T03:04:06.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.355 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.410 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9599e-05 s, 0.0 kB/s 2023-10-06T03:04:06.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.410 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.466 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1036e-05 s, 0.0 kB/s 2023-10-06T03:04:06.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.467 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.522 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.522 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8343e-05 s, 0.0 kB/s 2023-10-06T03:04:06.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.522 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.578 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0563e-05 s, 0.0 kB/s 2023-10-06T03:04:06.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.578 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0566e-05 s, 0.0 kB/s 2023-10-06T03:04:06.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.633 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-10-06T03:04:06.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.690 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.745 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1427e-05 s, 0.0 kB/s 2023-10-06T03:04:06.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.746 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.801 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7902e-05 s, 0.0 kB/s 2023-10-06T03:04:06.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.801 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.857 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2152e-05 s, 0.0 kB/s 2023-10-06T03:04:06.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.858 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.913 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0892e-05 s, 0.0 kB/s 2023-10-06T03:04:06.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.913 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:06.969 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9757e-05 s, 0.0 kB/s 2023-10-06T03:04:06.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:06.970 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.148e-05 s, 0.0 kB/s 2023-10-06T03:04:07.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.025 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.081 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9155e-05 s, 0.0 kB/s 2023-10-06T03:04:07.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.138 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.138 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2021e-05 s, 0.0 kB/s 2023-10-06T03:04:07.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.138 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7947e-05 s, 0.0 kB/s 2023-10-06T03:04:07.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.193 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.249 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-06T03:04:07.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.250 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.305 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.305 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8852e-05 s, 0.0 kB/s 2023-10-06T03:04:07.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.306 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.361 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0956e-05 s, 0.0 kB/s 2023-10-06T03:04:07.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.362 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.417 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.417 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0944e-05 s, 0.0 kB/s 2023-10-06T03:04:07.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.418 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.473 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8356e-05 s, 0.0 kB/s 2023-10-06T03:04:07.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.474 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.531 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.332e-05 s, 0.0 kB/s 2023-10-06T03:04:07.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.531 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8803e-05 s, 0.0 kB/s 2023-10-06T03:04:07.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.153e-05 s, 0.0 kB/s 2023-10-06T03:04:07.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.643 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.698 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7579e-05 s, 0.0 kB/s 2023-10-06T03:04:07.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.699 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.754 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.754 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.754 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0854e-05 s, 0.0 kB/s 2023-10-06T03:04:07.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.754 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.810 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7794e-05 s, 0.0 kB/s 2023-10-06T03:04:07.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.811 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.865 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8049e-05 s, 0.0 kB/s 2023-10-06T03:04:07.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.866 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.921 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.922 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8841e-05 s, 0.0 kB/s 2023-10-06T03:04:07.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.922 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:07.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:07.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:07.976 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2246e-05 s, 0.0 kB/s 2023-10-06T03:04:07.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:07.977 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-10-06T03:04:08.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.033 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.087 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.087 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.087 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.037e-05 s, 0.0 kB/s 2023-10-06T03:04:08.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.088 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.143 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.143 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.143 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9534e-05 s, 0.0 kB/s 2023-10-06T03:04:08.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.144 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.201 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.201 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0424e-05 s, 0.0 kB/s 2023-10-06T03:04:08.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.202 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.255 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0085e-05 s, 0.0 kB/s 2023-10-06T03:04:08.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.256 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.310 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.311 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.311 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9271e-05 s, 0.0 kB/s 2023-10-06T03:04:08.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.311 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.365 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.366 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3761e-05 s, 0.0 kB/s 2023-10-06T03:04:08.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.366 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.421 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.421 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.421 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7669e-05 s, 0.0 kB/s 2023-10-06T03:04:08.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.421 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.477 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8273e-05 s, 0.0 kB/s 2023-10-06T03:04:08.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.477 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.532 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.533 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9881e-05 s, 0.0 kB/s 2023-10-06T03:04:08.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.591 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1985e-05 s, 0.0 kB/s 2023-10-06T03:04:08.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.592 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.647 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.647 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.647 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9587e-05 s, 0.0 kB/s 2023-10-06T03:04:08.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.648 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.703 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0987e-05 s, 0.0 kB/s 2023-10-06T03:04:08.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.704 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.759 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0958e-05 s, 0.0 kB/s 2023-10-06T03:04:08.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.759 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.815 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.815 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.815 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.755e-05 s, 0.0 kB/s 2023-10-06T03:04:08.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.871 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.871 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.871 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.414e-05 s, 0.0 kB/s 2023-10-06T03:04:08.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.872 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.926 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.927 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7519e-05 s, 0.0 kB/s 2023-10-06T03:04:08.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.927 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:08.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:08.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:08.982 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2258e-05 s, 0.0 kB/s 2023-10-06T03:04:08.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:08.982 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.037 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9612e-05 s, 0.0 kB/s 2023-10-06T03:04:09.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.038 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.092 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.092 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.093 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8943e-05 s, 0.0 kB/s 2023-10-06T03:04:09.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.148 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1053e-05 s, 0.0 kB/s 2023-10-06T03:04:09.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.148 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.206 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.206 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.206 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0782e-05 s, 0.0 kB/s 2023-10-06T03:04:09.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.206 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9072e-05 s, 0.0 kB/s 2023-10-06T03:04:09.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.263 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.318 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.318 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.318 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.072e-05 s, 0.0 kB/s 2023-10-06T03:04:09.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.318 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.373 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.374 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9814e-05 s, 0.0 kB/s 2023-10-06T03:04:09.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.428 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0523e-05 s, 0.0 kB/s 2023-10-06T03:04:09.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.429 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.484 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7474e-05 s, 0.0 kB/s 2023-10-06T03:04:09.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.485 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.539 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8109e-05 s, 0.0 kB/s 2023-10-06T03:04:09.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.540 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.595 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1689e-05 s, 0.0 kB/s 2023-10-06T03:04:09.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.595 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.649 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.650 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9789e-05 s, 0.0 kB/s 2023-10-06T03:04:09.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.650 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.705 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.705 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.705 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8036e-05 s, 0.0 kB/s 2023-10-06T03:04:09.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.706 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.760 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.761 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2905e-05 s, 0.0 kB/s 2023-10-06T03:04:09.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.817 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-10-06T03:04:09.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.873 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8035e-05 s, 0.0 kB/s 2023-10-06T03:04:09.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.929 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-06T03:04:09.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.930 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:09.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:09.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:09.985 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1678e-05 s, 0.0 kB/s 2023-10-06T03:04:09.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:09.985 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.041 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.248e-05 s, 0.0 kB/s 2023-10-06T03:04:10.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.041 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.096 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8438e-05 s, 0.0 kB/s 2023-10-06T03:04:10.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.097 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8449e-05 s, 0.0 kB/s 2023-10-06T03:04:10.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.153 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.207 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.208 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6877e-05 s, 0.0 kB/s 2023-10-06T03:04:10.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.208 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.263 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0282e-05 s, 0.0 kB/s 2023-10-06T03:04:10.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.264 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7543e-05 s, 0.0 kB/s 2023-10-06T03:04:10.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.319 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.374 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.375 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6723e-05 s, 0.0 kB/s 2023-10-06T03:04:10.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.375 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.429 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.431 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.431 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.095e-05 s, 0.0 kB/s 2023-10-06T03:04:10.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.431 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.485 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.485 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6211e-05 s, 0.0 kB/s 2023-10-06T03:04:10.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.485 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.540 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.039e-05 s, 0.0 kB/s 2023-10-06T03:04:10.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.540 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.596 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2024e-05 s, 0.0 kB/s 2023-10-06T03:04:10.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.597 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.652 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7564e-05 s, 0.0 kB/s 2023-10-06T03:04:10.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.707 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.236e-05 s, 0.0 kB/s 2023-10-06T03:04:10.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.707 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.763 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8145e-05 s, 0.0 kB/s 2023-10-06T03:04:10.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.763 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.818 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1322e-05 s, 0.0 kB/s 2023-10-06T03:04:10.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.074e-05 s, 0.0 kB/s 2023-10-06T03:04:10.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.930 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2306e-05 s, 0.0 kB/s 2023-10-06T03:04:10.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.930 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:10.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:10.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:10.985 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0083e-05 s, 0.0 kB/s 2023-10-06T03:04:10.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:10.985 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.040 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6967e-05 s, 0.0 kB/s 2023-10-06T03:04:11.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.040 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2641e-05 s, 0.0 kB/s 2023-10-06T03:04:11.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.095 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.150 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9076e-05 s, 0.0 kB/s 2023-10-06T03:04:11.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.150 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9234e-05 s, 0.0 kB/s 2023-10-06T03:04:11.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.262 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3656e-05 s, 0.0 kB/s 2023-10-06T03:04:11.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.262 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.317 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9365e-05 s, 0.0 kB/s 2023-10-06T03:04:11.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.317 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.372 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.372 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.372 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0266e-05 s, 0.0 kB/s 2023-10-06T03:04:11.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.428 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7875e-05 s, 0.0 kB/s 2023-10-06T03:04:11.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.428 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.484 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6559e-05 s, 0.0 kB/s 2023-10-06T03:04:11.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.484 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.540 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7621e-05 s, 0.0 kB/s 2023-10-06T03:04:11.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.540 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.595 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8574e-05 s, 0.0 kB/s 2023-10-06T03:04:11.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.596 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.651 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-10-06T03:04:11.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.651 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.705 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.706 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0315e-05 s, 0.0 kB/s 2023-10-06T03:04:11.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.706 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.761 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9224e-05 s, 0.0 kB/s 2023-10-06T03:04:11.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.817 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7899e-05 s, 0.0 kB/s 2023-10-06T03:04:11.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.873 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9345e-05 s, 0.0 kB/s 2023-10-06T03:04:11.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.928 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.779e-05 s, 0.0 kB/s 2023-10-06T03:04:11.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:04:11.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:11.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:11.983 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9952e-05 s, 0.0 kB/s 2023-10-06T03:04:11.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:11.983 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:11.984 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:04:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1242e-05 s, 0.0 kB/s 2023-10-06T03:04:12.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.040 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7019e-05 s, 0.0 kB/s 2023-10-06T03:04:12.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.095 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.150 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7299e-05 s, 0.0 kB/s 2023-10-06T03:04:12.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.150 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7352e-05 s, 0.0 kB/s 2023-10-06T03:04:12.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.261 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7226e-05 s, 0.0 kB/s 2023-10-06T03:04:12.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.261 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.317 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9167e-05 s, 0.0 kB/s 2023-10-06T03:04:12.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.318 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.373 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.373 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.373 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9515e-05 s, 0.0 kB/s 2023-10-06T03:04:12.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.429 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.92e-05 s, 0.0 kB/s 2023-10-06T03:04:12.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.430 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.485 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0845e-05 s, 0.0 kB/s 2023-10-06T03:04:12.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.485 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.541 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8273e-05 s, 0.0 kB/s 2023-10-06T03:04:12.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.542 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.597 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8601e-05 s, 0.0 kB/s 2023-10-06T03:04:12.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.598 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7805e-05 s, 0.0 kB/s 2023-10-06T03:04:12.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.709 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9512e-05 s, 0.0 kB/s 2023-10-06T03:04:12.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.709 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.765 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0333e-05 s, 0.0 kB/s 2023-10-06T03:04:12.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.765 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.820 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.821 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8606e-05 s, 0.0 kB/s 2023-10-06T03:04:12.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.821 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.876 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8849e-05 s, 0.0 kB/s 2023-10-06T03:04:12.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.876 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.932 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.932 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.932 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8949e-05 s, 0.0 kB/s 2023-10-06T03:04:12.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.932 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:12.987 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:12.988 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2471e-05 s, 0.0 kB/s 2023-10-06T03:04:12.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:12.988 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.043 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.043 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.043 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-10-06T03:04:13.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.043 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.099 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.099 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.099 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8881e-05 s, 0.0 kB/s 2023-10-06T03:04:13.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.099 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.155 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9581e-05 s, 0.0 kB/s 2023-10-06T03:04:13.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.155 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8673e-05 s, 0.0 kB/s 2023-10-06T03:04:13.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.266 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7341e-05 s, 0.0 kB/s 2023-10-06T03:04:13.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.267 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9874e-05 s, 0.0 kB/s 2023-10-06T03:04:13.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.322 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.377 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8129e-05 s, 0.0 kB/s 2023-10-06T03:04:13.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.431 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.432 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7902e-05 s, 0.0 kB/s 2023-10-06T03:04:13.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.432 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.487 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7154e-05 s, 0.0 kB/s 2023-10-06T03:04:13.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.487 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.542 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1819e-05 s, 0.0 kB/s 2023-10-06T03:04:13.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.542 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.598 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.598 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.598 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1627e-05 s, 0.0 kB/s 2023-10-06T03:04:13.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.599 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.654 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.225e-05 s, 0.0 kB/s 2023-10-06T03:04:13.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.710 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8686e-05 s, 0.0 kB/s 2023-10-06T03:04:13.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.710 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.765 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9615e-05 s, 0.0 kB/s 2023-10-06T03:04:13.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.765 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.820 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.821 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1384e-05 s, 0.0 kB/s 2023-10-06T03:04:13.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.821 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.875 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.876 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0997e-05 s, 0.0 kB/s 2023-10-06T03:04:13.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.876 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.931 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.931 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.931 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7792e-05 s, 0.0 kB/s 2023-10-06T03:04:13.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.931 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:13.987 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8144e-05 s, 0.0 kB/s 2023-10-06T03:04:13.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:13.987 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8141e-05 s, 0.0 kB/s 2023-10-06T03:04:14.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.043 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.097 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8439e-05 s, 0.0 kB/s 2023-10-06T03:04:14.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.098 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.153 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.897e-05 s, 0.0 kB/s 2023-10-06T03:04:14.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.153 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.208 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.209 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0465e-05 s, 0.0 kB/s 2023-10-06T03:04:14.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.265 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4018e-05 s, 0.0 kB/s 2023-10-06T03:04:14.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.265 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.321 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9187e-05 s, 0.0 kB/s 2023-10-06T03:04:14.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.321 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.377 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8714e-05 s, 0.0 kB/s 2023-10-06T03:04:14.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.432 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.879e-05 s, 0.0 kB/s 2023-10-06T03:04:14.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.433 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.488 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.491 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0223e-05 s, 0.0 kB/s 2023-10-06T03:04:14.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.544 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2221e-05 s, 0.0 kB/s 2023-10-06T03:04:14.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.544 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1551e-05 s, 0.0 kB/s 2023-10-06T03:04:14.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.656 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.845e-05 s, 0.0 kB/s 2023-10-06T03:04:14.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.656 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.712 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3669e-05 s, 0.0 kB/s 2023-10-06T03:04:14.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1037e-05 s, 0.0 kB/s 2023-10-06T03:04:14.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.769 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1612e-05 s, 0.0 kB/s 2023-10-06T03:04:14.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.825 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.880 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3644e-05 s, 0.0 kB/s 2023-10-06T03:04:14.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.881 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.936 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3353e-05 s, 0.0 kB/s 2023-10-06T03:04:14.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.937 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:14.992 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.069e-05 s, 0.0 kB/s 2023-10-06T03:04:14.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:14.992 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.048 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0939e-05 s, 0.0 kB/s 2023-10-06T03:04:15.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.104 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0544e-05 s, 0.0 kB/s 2023-10-06T03:04:15.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.104 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.160 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0881e-05 s, 0.0 kB/s 2023-10-06T03:04:15.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.160 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.215 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0859e-05 s, 0.0 kB/s 2023-10-06T03:04:15.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.216 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.271 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.783e-05 s, 0.0 kB/s 2023-10-06T03:04:15.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.272 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.327 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8883e-05 s, 0.0 kB/s 2023-10-06T03:04:15.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.383 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.383 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.383 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8693e-05 s, 0.0 kB/s 2023-10-06T03:04:15.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.383 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.438 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.439 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9069e-05 s, 0.0 kB/s 2023-10-06T03:04:15.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.439 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.494 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0078e-05 s, 0.0 kB/s 2023-10-06T03:04:15.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.494 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.550 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.550 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.550 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0191e-05 s, 0.0 kB/s 2023-10-06T03:04:15.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.550 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.605 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.605 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.605 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2055e-05 s, 0.0 kB/s 2023-10-06T03:04:15.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.606 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.661 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0367e-05 s, 0.0 kB/s 2023-10-06T03:04:15.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.661 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.716 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8744e-05 s, 0.0 kB/s 2023-10-06T03:04:15.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.033e-05 s, 0.0 kB/s 2023-10-06T03:04:15.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.933e-05 s, 0.0 kB/s 2023-10-06T03:04:15.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.827 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.826e-05 s, 0.0 kB/s 2023-10-06T03:04:15.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.938 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0014e-05 s, 0.0 kB/s 2023-10-06T03:04:15.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.938 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:15.993 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2308e-05 s, 0.0 kB/s 2023-10-06T03:04:15.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:15.993 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-06T03:04:16.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.103 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9054e-05 s, 0.0 kB/s 2023-10-06T03:04:16.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.159 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8932e-05 s, 0.0 kB/s 2023-10-06T03:04:16.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.214 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7619e-05 s, 0.0 kB/s 2023-10-06T03:04:16.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.269 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8417e-05 s, 0.0 kB/s 2023-10-06T03:04:16.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.270 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.325 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9766e-05 s, 0.0 kB/s 2023-10-06T03:04:16.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.381 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9153e-05 s, 0.0 kB/s 2023-10-06T03:04:16.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.436 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-06T03:04:16.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.436 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0346e-05 s, 0.0 kB/s 2023-10-06T03:04:16.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7027e-05 s, 0.0 kB/s 2023-10-06T03:04:16.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.548 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.603 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6135e-05 s, 0.0 kB/s 2023-10-06T03:04:16.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.659 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1149e-05 s, 0.0 kB/s 2023-10-06T03:04:16.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.660 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.715 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8081e-05 s, 0.0 kB/s 2023-10-06T03:04:16.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.716 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.770 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.932e-05 s, 0.0 kB/s 2023-10-06T03:04:16.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.771 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.825 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.5823e-05 s, 0.0 kB/s 2023-10-06T03:04:16.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.826 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2018e-05 s, 0.0 kB/s 2023-10-06T03:04:16.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.881 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.936 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8043e-05 s, 0.0 kB/s 2023-10-06T03:04:16.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:16.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:16.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:16.991 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-10-06T03:04:16.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:16.991 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.046 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1209e-05 s, 0.0 kB/s 2023-10-06T03:04:17.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.101 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0466e-05 s, 0.0 kB/s 2023-10-06T03:04:17.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.101 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.156 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1646e-05 s, 0.0 kB/s 2023-10-06T03:04:17.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.156 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.212 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6715e-05 s, 0.0 kB/s 2023-10-06T03:04:17.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.268 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3412e-05 s, 0.0 kB/s 2023-10-06T03:04:17.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.268 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.324 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0041e-05 s, 0.0 kB/s 2023-10-06T03:04:17.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.325 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.380 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7318e-05 s, 0.0 kB/s 2023-10-06T03:04:17.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.435 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6666e-05 s, 0.0 kB/s 2023-10-06T03:04:17.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.436 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3396e-05 s, 0.0 kB/s 2023-10-06T03:04:17.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6203e-05 s, 0.0 kB/s 2023-10-06T03:04:17.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.548 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.603 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9067e-05 s, 0.0 kB/s 2023-10-06T03:04:17.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.659 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7508e-05 s, 0.0 kB/s 2023-10-06T03:04:17.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.659 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.715 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.412e-05 s, 0.0 kB/s 2023-10-06T03:04:17.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.715 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.603e-05 s, 0.0 kB/s 2023-10-06T03:04:17.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1603e-05 s, 0.0 kB/s 2023-10-06T03:04:17.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7049e-05 s, 0.0 kB/s 2023-10-06T03:04:17.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.938 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6805e-05 s, 0.0 kB/s 2023-10-06T03:04:17.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.938 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:17.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:17.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:17.992 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3542e-05 s, 0.0 kB/s 2023-10-06T03:04:17.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:17.993 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.048 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6493e-05 s, 0.0 kB/s 2023-10-06T03:04:18.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.103 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1098e-05 s, 0.0 kB/s 2023-10-06T03:04:18.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.159 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1198e-05 s, 0.0 kB/s 2023-10-06T03:04:18.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.160 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.215 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.5465e-05 s, 0.0 kB/s 2023-10-06T03:04:18.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.271 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8177e-05 s, 0.0 kB/s 2023-10-06T03:04:18.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.271 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7327e-05 s, 0.0 kB/s 2023-10-06T03:04:18.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.381 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2429e-05 s, 0.0 kB/s 2023-10-06T03:04:18.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1879e-05 s, 0.0 kB/s 2023-10-06T03:04:18.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1323e-05 s, 0.0 kB/s 2023-10-06T03:04:18.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0646e-05 s, 0.0 kB/s 2023-10-06T03:04:18.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.549 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1776e-05 s, 0.0 kB/s 2023-10-06T03:04:18.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9329e-05 s, 0.0 kB/s 2023-10-06T03:04:18.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7395e-05 s, 0.0 kB/s 2023-10-06T03:04:18.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.768 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.289e-05 s, 0.0 kB/s 2023-10-06T03:04:18.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0926e-05 s, 0.0 kB/s 2023-10-06T03:04:18.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0064e-05 s, 0.0 kB/s 2023-10-06T03:04:18.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.933 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.934 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.9124e-05 s, 0.0 kB/s 2023-10-06T03:04:18.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:18.988 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2186e-05 s, 0.0 kB/s 2023-10-06T03:04:18.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:18.989 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.043 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.044 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9566e-05 s, 0.0 kB/s 2023-10-06T03:04:19.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.044 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.099 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.072e-05 s, 0.0 kB/s 2023-10-06T03:04:19.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.099 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1136e-05 s, 0.0 kB/s 2023-10-06T03:04:19.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.154 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.209 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7508e-05 s, 0.0 kB/s 2023-10-06T03:04:19.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.209 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.265 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.147e-05 s, 0.0 kB/s 2023-10-06T03:04:19.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.320 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.922e-05 s, 0.0 kB/s 2023-10-06T03:04:19.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.321 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.376 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9368e-05 s, 0.0 kB/s 2023-10-06T03:04:19.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.376 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.431 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.432 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8489e-05 s, 0.0 kB/s 2023-10-06T03:04:19.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.432 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.487 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.487 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5581e-05 s, 0.0 kB/s 2023-10-06T03:04:19.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.487 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.542 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.542 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.542 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6697e-05 s, 0.0 kB/s 2023-10-06T03:04:19.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.542 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.598 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0218e-05 s, 0.0 kB/s 2023-10-06T03:04:19.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.599 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.654 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.654 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.654 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1165e-05 s, 0.0 kB/s 2023-10-06T03:04:19.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.710 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.710 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.009e-05 s, 0.0 kB/s 2023-10-06T03:04:19.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.710 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.765 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0699e-05 s, 0.0 kB/s 2023-10-06T03:04:19.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.766 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.821 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.821 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.822 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9627e-05 s, 0.0 kB/s 2023-10-06T03:04:19.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.822 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.877 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.877 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.877 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9255e-05 s, 0.0 kB/s 2023-10-06T03:04:19.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.878 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.932 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.932 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.933 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8802e-05 s, 0.0 kB/s 2023-10-06T03:04:19.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:19.988 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:19.988 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:19.989 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2273e-05 s, 0.0 kB/s 2023-10-06T03:04:19.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:19.989 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.044 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9471e-05 s, 0.0 kB/s 2023-10-06T03:04:20.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.044 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.099 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.099 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.11e-05 s, 0.0 kB/s 2023-10-06T03:04:20.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.100 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3514e-05 s, 0.0 kB/s 2023-10-06T03:04:20.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.155 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0368e-05 s, 0.0 kB/s 2023-10-06T03:04:20.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.264 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0155e-05 s, 0.0 kB/s 2023-10-06T03:04:20.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.265 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.321 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.947e-05 s, 0.0 kB/s 2023-10-06T03:04:20.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.321 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.375 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8494e-05 s, 0.0 kB/s 2023-10-06T03:04:20.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.376 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.430 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.430 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.430 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0411e-05 s, 0.0 kB/s 2023-10-06T03:04:20.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.431 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.486 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0558e-05 s, 0.0 kB/s 2023-10-06T03:04:20.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.486 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.542 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6371e-05 s, 0.0 kB/s 2023-10-06T03:04:20.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.542 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.597 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0201e-05 s, 0.0 kB/s 2023-10-06T03:04:20.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.598 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.653 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2876e-05 s, 0.0 kB/s 2023-10-06T03:04:20.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.708 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1591e-05 s, 0.0 kB/s 2023-10-06T03:04:20.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.708 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.762 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9572e-05 s, 0.0 kB/s 2023-10-06T03:04:20.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.763 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.817 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8378e-05 s, 0.0 kB/s 2023-10-06T03:04:20.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.873 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.941e-05 s, 0.0 kB/s 2023-10-06T03:04:20.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0328e-05 s, 0.0 kB/s 2023-10-06T03:04:20.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:20.982 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:20.983 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7863e-05 s, 0.0 kB/s 2023-10-06T03:04:20.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:20.983 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.038 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0215e-05 s, 0.0 kB/s 2023-10-06T03:04:21.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.039 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.096 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1873e-05 s, 0.0 kB/s 2023-10-06T03:04:21.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.096 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.151 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8596e-05 s, 0.0 kB/s 2023-10-06T03:04:21.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.151 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0972e-05 s, 0.0 kB/s 2023-10-06T03:04:21.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.206 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.261 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1441e-05 s, 0.0 kB/s 2023-10-06T03:04:21.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.261 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.316 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7938e-05 s, 0.0 kB/s 2023-10-06T03:04:21.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.316 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1215e-05 s, 0.0 kB/s 2023-10-06T03:04:21.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.428 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1097e-05 s, 0.0 kB/s 2023-10-06T03:04:21.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.429 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.484 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0134e-05 s, 0.0 kB/s 2023-10-06T03:04:21.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.484 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8972e-05 s, 0.0 kB/s 2023-10-06T03:04:21.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.540 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.596 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1195e-05 s, 0.0 kB/s 2023-10-06T03:04:21.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.597 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.652 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9058e-05 s, 0.0 kB/s 2023-10-06T03:04:21.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.708 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.708 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0408e-05 s, 0.0 kB/s 2023-10-06T03:04:21.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.708 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.763 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6445e-05 s, 0.0 kB/s 2023-10-06T03:04:21.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.763 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.3731e-05 s, 0.0 kB/s 2023-10-06T03:04:21.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.819 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.874 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1376e-05 s, 0.0 kB/s 2023-10-06T03:04:21.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.929 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1019e-05 s, 0.0 kB/s 2023-10-06T03:04:21.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.929 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:21.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:21.984 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0097e-05 s, 0.0 kB/s 2023-10-06T03:04:21.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:21.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.039 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9173e-05 s, 0.0 kB/s 2023-10-06T03:04:22.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.039 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1927e-05 s, 0.0 kB/s 2023-10-06T03:04:22.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.095 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.151 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.788e-05 s, 0.0 kB/s 2023-10-06T03:04:22.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.151 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.206 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0737e-05 s, 0.0 kB/s 2023-10-06T03:04:22.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.206 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.262 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9793e-05 s, 0.0 kB/s 2023-10-06T03:04:22.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.262 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.318 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.046e-05 s, 0.0 kB/s 2023-10-06T03:04:22.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.318 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.373 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.374 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1624e-05 s, 0.0 kB/s 2023-10-06T03:04:22.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.428 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.864e-05 s, 0.0 kB/s 2023-10-06T03:04:22.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.429 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.484 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2667e-05 s, 0.0 kB/s 2023-10-06T03:04:22.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.484 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.539 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9134e-05 s, 0.0 kB/s 2023-10-06T03:04:22.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.540 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.596 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2681e-05 s, 0.0 kB/s 2023-10-06T03:04:22.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.596 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.651 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7435e-05 s, 0.0 kB/s 2023-10-06T03:04:22.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.651 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.707 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.081e-05 s, 0.0 kB/s 2023-10-06T03:04:22.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.707 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.763 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1528e-05 s, 0.0 kB/s 2023-10-06T03:04:22.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.763 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.818 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7872e-05 s, 0.0 kB/s 2023-10-06T03:04:22.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.874 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.857e-05 s, 0.0 kB/s 2023-10-06T03:04:22.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6848e-05 s, 0.0 kB/s 2023-10-06T03:04:22.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.929 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-06T03:04:22.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:22.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.039 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8148e-05 s, 0.0 kB/s 2023-10-06T03:04:23.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.039 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-10-06T03:04:23.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.094 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.149 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2947e-05 s, 0.0 kB/s 2023-10-06T03:04:23.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.150 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-06T03:04:23.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9904e-05 s, 0.0 kB/s 2023-10-06T03:04:23.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.261 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.316 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1281e-05 s, 0.0 kB/s 2023-10-06T03:04:23.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.316 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.372 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.431e-05 s, 0.0 kB/s 2023-10-06T03:04:23.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.426 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0343e-05 s, 0.0 kB/s 2023-10-06T03:04:23.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.483 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9782e-05 s, 0.0 kB/s 2023-10-06T03:04:23.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.483 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.537 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.538 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.538 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1146e-05 s, 0.0 kB/s 2023-10-06T03:04:23.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.538 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.593 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1578e-05 s, 0.0 kB/s 2023-10-06T03:04:23.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.594 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.650 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.312e-05 s, 0.0 kB/s 2023-10-06T03:04:23.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.651 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.706 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5484e-05 s, 0.0 kB/s 2023-10-06T03:04:23.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.707 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.762 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0001e-05 s, 0.0 kB/s 2023-10-06T03:04:23.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8204e-05 s, 0.0 kB/s 2023-10-06T03:04:23.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.873 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8459e-05 s, 0.0 kB/s 2023-10-06T03:04:23.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.928 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.104e-05 s, 0.0 kB/s 2023-10-06T03:04:23.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:23.982 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:23.983 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1478e-05 s, 0.0 kB/s 2023-10-06T03:04:23.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:23.983 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.038 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1349e-05 s, 0.0 kB/s 2023-10-06T03:04:24.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.038 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8794e-05 s, 0.0 kB/s 2023-10-06T03:04:24.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.094 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.149 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.116e-05 s, 0.0 kB/s 2023-10-06T03:04:24.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.150 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6364e-05 s, 0.0 kB/s 2023-10-06T03:04:24.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.261 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8796e-05 s, 0.0 kB/s 2023-10-06T03:04:24.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.262 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.317 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.317 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.317 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.879e-05 s, 0.0 kB/s 2023-10-06T03:04:24.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.318 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.374 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0669e-05 s, 0.0 kB/s 2023-10-06T03:04:24.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.375 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.430 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.430 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.430 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9838e-05 s, 0.0 kB/s 2023-10-06T03:04:24.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.430 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.485 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.486 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7922e-05 s, 0.0 kB/s 2023-10-06T03:04:24.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.486 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.540 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9258e-05 s, 0.0 kB/s 2023-10-06T03:04:24.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.541 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.596 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2585e-05 s, 0.0 kB/s 2023-10-06T03:04:24.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.597 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.652 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.792e-05 s, 0.0 kB/s 2023-10-06T03:04:24.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.707 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0979e-05 s, 0.0 kB/s 2023-10-06T03:04:24.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.708 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.764 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-06T03:04:24.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.765 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.819 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.820 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8882e-05 s, 0.0 kB/s 2023-10-06T03:04:24.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.820 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.875 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9079e-05 s, 0.0 kB/s 2023-10-06T03:04:24.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.875 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.931 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.931 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.931 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0568e-05 s, 0.0 kB/s 2023-10-06T03:04:24.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.932 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9308e-05 s, 0.0 kB/s 2023-10-06T03:04:24.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:24.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.042 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.043 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1033e-05 s, 0.0 kB/s 2023-10-06T03:04:25.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.043 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.098 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0076e-05 s, 0.0 kB/s 2023-10-06T03:04:25.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.098 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.154 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.075e-05 s, 0.0 kB/s 2023-10-06T03:04:25.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.154 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4008e-05 s, 0.0 kB/s 2023-10-06T03:04:25.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2847e-05 s, 0.0 kB/s 2023-10-06T03:04:25.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.267 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1499e-05 s, 0.0 kB/s 2023-10-06T03:04:25.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.322 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.377 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6863e-05 s, 0.0 kB/s 2023-10-06T03:04:25.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.432 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0804e-05 s, 0.0 kB/s 2023-10-06T03:04:25.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.432 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.487 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9175e-05 s, 0.0 kB/s 2023-10-06T03:04:25.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.487 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.543 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.543 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.543 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8831e-05 s, 0.0 kB/s 2023-10-06T03:04:25.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.543 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.599 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.599 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.599 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1164e-05 s, 0.0 kB/s 2023-10-06T03:04:25.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.600 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.655 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2576e-05 s, 0.0 kB/s 2023-10-06T03:04:25.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.656 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.711 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.711 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.711 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1896e-05 s, 0.0 kB/s 2023-10-06T03:04:25.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.711 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.767 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1489e-05 s, 0.0 kB/s 2023-10-06T03:04:25.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8291e-05 s, 0.0 kB/s 2023-10-06T03:04:25.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-10-06T03:04:25.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.934 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-06T03:04:25.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:25.989 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.835e-05 s, 0.0 kB/s 2023-10-06T03:04:25.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:25.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2364e-05 s, 0.0 kB/s 2023-10-06T03:04:26.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.045 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.100 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1034e-05 s, 0.0 kB/s 2023-10-06T03:04:26.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.100 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.156 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9405e-05 s, 0.0 kB/s 2023-10-06T03:04:26.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.156 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.211 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.091e-05 s, 0.0 kB/s 2023-10-06T03:04:26.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.212 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.268 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-10-06T03:04:26.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.268 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.323 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2844e-05 s, 0.0 kB/s 2023-10-06T03:04:26.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.378 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1391e-05 s, 0.0 kB/s 2023-10-06T03:04:26.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.378 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.433 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0117e-05 s, 0.0 kB/s 2023-10-06T03:04:26.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.488 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7975e-05 s, 0.0 kB/s 2023-10-06T03:04:26.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.489 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.544 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.544 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.544 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8058e-05 s, 0.0 kB/s 2023-10-06T03:04:26.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.545 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9059e-05 s, 0.0 kB/s 2023-10-06T03:04:26.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7013e-05 s, 0.0 kB/s 2023-10-06T03:04:26.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.659 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.714 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8855e-05 s, 0.0 kB/s 2023-10-06T03:04:26.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.923e-05 s, 0.0 kB/s 2023-10-06T03:04:26.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.769 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2452e-05 s, 0.0 kB/s 2023-10-06T03:04:26.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.880 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.041e-05 s, 0.0 kB/s 2023-10-06T03:04:26.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.067e-05 s, 0.0 kB/s 2023-10-06T03:04:26.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:26.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:26.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:26.989 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.911e-05 s, 0.0 kB/s 2023-10-06T03:04:26.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:26.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.045 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1966e-05 s, 0.0 kB/s 2023-10-06T03:04:27.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.045 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.099 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.100 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2323e-05 s, 0.0 kB/s 2023-10-06T03:04:27.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.100 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.155 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.155 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.155 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0072e-05 s, 0.0 kB/s 2023-10-06T03:04:27.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.155 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.211 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7032e-05 s, 0.0 kB/s 2023-10-06T03:04:27.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.033e-05 s, 0.0 kB/s 2023-10-06T03:04:27.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.267 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.323 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7063e-05 s, 0.0 kB/s 2023-10-06T03:04:27.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.324 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.379 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9029e-05 s, 0.0 kB/s 2023-10-06T03:04:27.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.379 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9419e-05 s, 0.0 kB/s 2023-10-06T03:04:27.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8501e-05 s, 0.0 kB/s 2023-10-06T03:04:27.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.489 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0813e-05 s, 0.0 kB/s 2023-10-06T03:04:27.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.546 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7459e-05 s, 0.0 kB/s 2023-10-06T03:04:27.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9042e-05 s, 0.0 kB/s 2023-10-06T03:04:27.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8371e-05 s, 0.0 kB/s 2023-10-06T03:04:27.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2671e-05 s, 0.0 kB/s 2023-10-06T03:04:27.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.825 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3397e-05 s, 0.0 kB/s 2023-10-06T03:04:27.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.826 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7889e-05 s, 0.0 kB/s 2023-10-06T03:04:27.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.882 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.936 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9946e-05 s, 0.0 kB/s 2023-10-06T03:04:27.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.937 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:27.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:27.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:27.991 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7186e-05 s, 0.0 kB/s 2023-10-06T03:04:27.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:27.992 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.047 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.74e-05 s, 0.0 kB/s 2023-10-06T03:04:28.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.047 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.102 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.102 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6463e-05 s, 0.0 kB/s 2023-10-06T03:04:28.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.102 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.157 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1647e-05 s, 0.0 kB/s 2023-10-06T03:04:28.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.157 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.212 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9435e-05 s, 0.0 kB/s 2023-10-06T03:04:28.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.212 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7615e-05 s, 0.0 kB/s 2023-10-06T03:04:28.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.271 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.323 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.113e-05 s, 0.0 kB/s 2023-10-06T03:04:28.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.379 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0049e-05 s, 0.0 kB/s 2023-10-06T03:04:28.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.380 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.435 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8506e-05 s, 0.0 kB/s 2023-10-06T03:04:28.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.435 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.491 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8145e-05 s, 0.0 kB/s 2023-10-06T03:04:28.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.491 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9136e-05 s, 0.0 kB/s 2023-10-06T03:04:28.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.547 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2961e-05 s, 0.0 kB/s 2023-10-06T03:04:28.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8144e-05 s, 0.0 kB/s 2023-10-06T03:04:28.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0115e-05 s, 0.0 kB/s 2023-10-06T03:04:28.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.770 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.770 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7415e-05 s, 0.0 kB/s 2023-10-06T03:04:28.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.825 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3852e-05 s, 0.0 kB/s 2023-10-06T03:04:28.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.825 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.880 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9562e-05 s, 0.0 kB/s 2023-10-06T03:04:28.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9166e-05 s, 0.0 kB/s 2023-10-06T03:04:28.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:28.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:28.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:28.991 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8334e-05 s, 0.0 kB/s 2023-10-06T03:04:28.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:28.991 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8062e-05 s, 0.0 kB/s 2023-10-06T03:04:29.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.100 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2495e-05 s, 0.0 kB/s 2023-10-06T03:04:29.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.101 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.156 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7374e-05 s, 0.0 kB/s 2023-10-06T03:04:29.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.156 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0016e-05 s, 0.0 kB/s 2023-10-06T03:04:29.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.266 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9932e-05 s, 0.0 kB/s 2023-10-06T03:04:29.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7776e-05 s, 0.0 kB/s 2023-10-06T03:04:29.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.322 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.377 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7971e-05 s, 0.0 kB/s 2023-10-06T03:04:29.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.378 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.433 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0135e-05 s, 0.0 kB/s 2023-10-06T03:04:29.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.433 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8072e-05 s, 0.0 kB/s 2023-10-06T03:04:29.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.490 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9465e-05 s, 0.0 kB/s 2023-10-06T03:04:29.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.546 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7803e-05 s, 0.0 kB/s 2023-10-06T03:04:29.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.602 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3869e-05 s, 0.0 kB/s 2023-10-06T03:04:29.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2317e-05 s, 0.0 kB/s 2023-10-06T03:04:29.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.917e-05 s, 0.0 kB/s 2023-10-06T03:04:29.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.932e-05 s, 0.0 kB/s 2023-10-06T03:04:29.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.826 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.881 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7399e-05 s, 0.0 kB/s 2023-10-06T03:04:29.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.881 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.936 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0501e-05 s, 0.0 kB/s 2023-10-06T03:04:29.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:29.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:29.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:29.991 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9199e-05 s, 0.0 kB/s 2023-10-06T03:04:29.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:29.991 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.046 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7703e-05 s, 0.0 kB/s 2023-10-06T03:04:30.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.101 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8307e-05 s, 0.0 kB/s 2023-10-06T03:04:30.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.102 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.157 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.157 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7999e-05 s, 0.0 kB/s 2023-10-06T03:04:30.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.157 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.212 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.212 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2448e-05 s, 0.0 kB/s 2023-10-06T03:04:30.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.212 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.266 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0408e-05 s, 0.0 kB/s 2023-10-06T03:04:30.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.267 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3507e-05 s, 0.0 kB/s 2023-10-06T03:04:30.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.322 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.377 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7937e-05 s, 0.0 kB/s 2023-10-06T03:04:30.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.433 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8809e-05 s, 0.0 kB/s 2023-10-06T03:04:30.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0213e-05 s, 0.0 kB/s 2023-10-06T03:04:30.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.489 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0749e-05 s, 0.0 kB/s 2023-10-06T03:04:30.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.545 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0709e-05 s, 0.0 kB/s 2023-10-06T03:04:30.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.656 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7458e-05 s, 0.0 kB/s 2023-10-06T03:04:30.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9628e-05 s, 0.0 kB/s 2023-10-06T03:04:30.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.712 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.768 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0978e-05 s, 0.0 kB/s 2023-10-06T03:04:30.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0052e-05 s, 0.0 kB/s 2023-10-06T03:04:30.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.880 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8104e-05 s, 0.0 kB/s 2023-10-06T03:04:30.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-06T03:04:30.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:30.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:30.990 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9019e-05 s, 0.0 kB/s 2023-10-06T03:04:30.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:30.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.044 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.045 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7389e-05 s, 0.0 kB/s 2023-10-06T03:04:31.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.045 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.101 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9047e-05 s, 0.0 kB/s 2023-10-06T03:04:31.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.101 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.157 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7143e-05 s, 0.0 kB/s 2023-10-06T03:04:31.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.157 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.212 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7875e-05 s, 0.0 kB/s 2023-10-06T03:04:31.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.212 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.268 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8633e-05 s, 0.0 kB/s 2023-10-06T03:04:31.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.268 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.323 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8289e-05 s, 0.0 kB/s 2023-10-06T03:04:31.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.324 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.378 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7844e-05 s, 0.0 kB/s 2023-10-06T03:04:31.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.378 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6544e-05 s, 0.0 kB/s 2023-10-06T03:04:31.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.490 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.490 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7753e-05 s, 0.0 kB/s 2023-10-06T03:04:31.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.490 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.546 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8094e-05 s, 0.0 kB/s 2023-10-06T03:04:31.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.546 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0146e-05 s, 0.0 kB/s 2023-10-06T03:04:31.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7965e-05 s, 0.0 kB/s 2023-10-06T03:04:31.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9452e-05 s, 0.0 kB/s 2023-10-06T03:04:31.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8965e-05 s, 0.0 kB/s 2023-10-06T03:04:31.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.769 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.824 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8718e-05 s, 0.0 kB/s 2023-10-06T03:04:31.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.880 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7764e-05 s, 0.0 kB/s 2023-10-06T03:04:31.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8678e-05 s, 0.0 kB/s 2023-10-06T03:04:31.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:31.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:31.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:31.990 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.051e-05 s, 0.0 kB/s 2023-10-06T03:04:31.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:31.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.046 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1441e-05 s, 0.0 kB/s 2023-10-06T03:04:32.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8926e-05 s, 0.0 kB/s 2023-10-06T03:04:32.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.101 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.156 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1358e-05 s, 0.0 kB/s 2023-10-06T03:04:32.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.156 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.211 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.211 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8462e-05 s, 0.0 kB/s 2023-10-06T03:04:32.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.267 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6517e-05 s, 0.0 kB/s 2023-10-06T03:04:32.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.268 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0863e-05 s, 0.0 kB/s 2023-10-06T03:04:32.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.380 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1825e-05 s, 0.0 kB/s 2023-10-06T03:04:32.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.435 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.436 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7405e-05 s, 0.0 kB/s 2023-10-06T03:04:32.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.436 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.490 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8697e-05 s, 0.0 kB/s 2023-10-06T03:04:32.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.491 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6583e-05 s, 0.0 kB/s 2023-10-06T03:04:32.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.547 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.603 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0534e-05 s, 0.0 kB/s 2023-10-06T03:04:32.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.604 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.659 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0186e-05 s, 0.0 kB/s 2023-10-06T03:04:32.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.659 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.714 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5362e-05 s, 0.0 kB/s 2023-10-06T03:04:32.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8184e-05 s, 0.0 kB/s 2023-10-06T03:04:32.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.825 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1364e-05 s, 0.0 kB/s 2023-10-06T03:04:32.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.826 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.880 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.882 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7458e-05 s, 0.0 kB/s 2023-10-06T03:04:32.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.882 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.936 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.041e-05 s, 0.0 kB/s 2023-10-06T03:04:32.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.245e-05 s, 0.0 kB/s 2023-10-06T03:04:32.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:32.992 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.048 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4978e-05 s, 0.0 kB/s 2023-10-06T03:04:33.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.103 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0328e-05 s, 0.0 kB/s 2023-10-06T03:04:33.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.104 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3334e-05 s, 0.0 kB/s 2023-10-06T03:04:33.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.213 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.214 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9596e-05 s, 0.0 kB/s 2023-10-06T03:04:33.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.269 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9991e-05 s, 0.0 kB/s 2023-10-06T03:04:33.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.270 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7699e-05 s, 0.0 kB/s 2023-10-06T03:04:33.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.381 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2324e-05 s, 0.0 kB/s 2023-10-06T03:04:33.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.382 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9138e-05 s, 0.0 kB/s 2023-10-06T03:04:33.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.834e-05 s, 0.0 kB/s 2023-10-06T03:04:33.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9933e-05 s, 0.0 kB/s 2023-10-06T03:04:33.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.548 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8396e-05 s, 0.0 kB/s 2023-10-06T03:04:33.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0242e-05 s, 0.0 kB/s 2023-10-06T03:04:33.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0666e-05 s, 0.0 kB/s 2023-10-06T03:04:33.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.768 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9497e-05 s, 0.0 kB/s 2023-10-06T03:04:33.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9022e-05 s, 0.0 kB/s 2023-10-06T03:04:33.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.823 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0913e-05 s, 0.0 kB/s 2023-10-06T03:04:33.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.933 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.934 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.895e-05 s, 0.0 kB/s 2023-10-06T03:04:33.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:33.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:33.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:33.991 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0852e-05 s, 0.0 kB/s 2023-10-06T03:04:33.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:33.991 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.045 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3092e-05 s, 0.0 kB/s 2023-10-06T03:04:34.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.101 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1673e-05 s, 0.0 kB/s 2023-10-06T03:04:34.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.102 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8789e-05 s, 0.0 kB/s 2023-10-06T03:04:34.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.157 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.212 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.213 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.213 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8945e-05 s, 0.0 kB/s 2023-10-06T03:04:34.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.213 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.268 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0028e-05 s, 0.0 kB/s 2023-10-06T03:04:34.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.268 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.323 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.324 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5035e-05 s, 0.0 kB/s 2023-10-06T03:04:34.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.324 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0869e-05 s, 0.0 kB/s 2023-10-06T03:04:34.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.379 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0085e-05 s, 0.0 kB/s 2023-10-06T03:04:34.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9602e-05 s, 0.0 kB/s 2023-10-06T03:04:34.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.489 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7365e-05 s, 0.0 kB/s 2023-10-06T03:04:34.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.546 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0609e-05 s, 0.0 kB/s 2023-10-06T03:04:34.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.103e-05 s, 0.0 kB/s 2023-10-06T03:04:34.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.244e-05 s, 0.0 kB/s 2023-10-06T03:04:34.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7937e-05 s, 0.0 kB/s 2023-10-06T03:04:34.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.825 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8e-05 s, 0.0 kB/s 2023-10-06T03:04:34.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.825 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.881 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.821e-05 s, 0.0 kB/s 2023-10-06T03:04:34.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.882 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.937 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9184e-05 s, 0.0 kB/s 2023-10-06T03:04:34.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.937 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:34.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:34.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:34.992 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.99e-05 s, 0.0 kB/s 2023-10-06T03:04:34.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:34.992 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.047 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9321e-05 s, 0.0 kB/s 2023-10-06T03:04:35.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.047 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.101 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.101 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1959e-05 s, 0.0 kB/s 2023-10-06T03:04:35.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.102 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.158 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0485e-05 s, 0.0 kB/s 2023-10-06T03:04:35.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.213 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.213 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.213 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8656e-05 s, 0.0 kB/s 2023-10-06T03:04:35.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.269 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.971e-05 s, 0.0 kB/s 2023-10-06T03:04:35.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.269 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.324 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9908e-05 s, 0.0 kB/s 2023-10-06T03:04:35.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.324 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.379 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2097e-05 s, 0.0 kB/s 2023-10-06T03:04:35.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.380 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9462e-05 s, 0.0 kB/s 2023-10-06T03:04:35.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.490 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.141e-05 s, 0.0 kB/s 2023-10-06T03:04:35.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.490 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.544 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9882e-05 s, 0.0 kB/s 2023-10-06T03:04:35.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.545 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.600 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1417e-05 s, 0.0 kB/s 2023-10-06T03:04:35.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.600 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8564e-05 s, 0.0 kB/s 2023-10-06T03:04:35.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.712 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9469e-05 s, 0.0 kB/s 2023-10-06T03:04:35.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3673e-05 s, 0.0 kB/s 2023-10-06T03:04:35.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.769 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9401e-05 s, 0.0 kB/s 2023-10-06T03:04:35.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2162e-05 s, 0.0 kB/s 2023-10-06T03:04:35.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.933 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.933 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.933 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7308e-05 s, 0.0 kB/s 2023-10-06T03:04:35.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:35.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:35.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:35.989 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9103e-05 s, 0.0 kB/s 2023-10-06T03:04:35.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:35.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.045 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.045 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7719e-05 s, 0.0 kB/s 2023-10-06T03:04:36.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.100 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.100 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8824e-05 s, 0.0 kB/s 2023-10-06T03:04:36.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.101 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.155 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.155 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.155 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8851e-05 s, 0.0 kB/s 2023-10-06T03:04:36.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.156 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9153e-05 s, 0.0 kB/s 2023-10-06T03:04:36.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.266 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.997e-05 s, 0.0 kB/s 2023-10-06T03:04:36.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.321 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9816e-05 s, 0.0 kB/s 2023-10-06T03:04:36.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.321 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.376 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8817e-05 s, 0.0 kB/s 2023-10-06T03:04:36.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.433 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9803e-05 s, 0.0 kB/s 2023-10-06T03:04:36.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0284e-05 s, 0.0 kB/s 2023-10-06T03:04:36.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.490 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9208e-05 s, 0.0 kB/s 2023-10-06T03:04:36.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.545 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0163e-05 s, 0.0 kB/s 2023-10-06T03:04:36.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.602 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1113e-05 s, 0.0 kB/s 2023-10-06T03:04:36.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.712 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1196e-05 s, 0.0 kB/s 2023-10-06T03:04:36.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.767 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1042e-05 s, 0.0 kB/s 2023-10-06T03:04:36.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2588e-05 s, 0.0 kB/s 2023-10-06T03:04:36.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.648e-05 s, 0.0 kB/s 2023-10-06T03:04:36.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1105e-05 s, 0.0 kB/s 2023-10-06T03:04:36.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.936 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:36.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:36.991 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:36.991 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9668e-05 s, 0.0 kB/s 2023-10-06T03:04:36.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:36.992 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.047 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9671e-05 s, 0.0 kB/s 2023-10-06T03:04:37.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.103 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0423e-05 s, 0.0 kB/s 2023-10-06T03:04:37.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.159 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7604e-05 s, 0.0 kB/s 2023-10-06T03:04:37.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.213 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-06T03:04:37.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.268 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.268 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.748e-05 s, 0.0 kB/s 2023-10-06T03:04:37.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.269 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.324 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.324 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1421e-05 s, 0.0 kB/s 2023-10-06T03:04:37.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.325 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.380 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.380 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2224e-05 s, 0.0 kB/s 2023-10-06T03:04:37.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1179e-05 s, 0.0 kB/s 2023-10-06T03:04:37.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.494 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.494 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.494 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1566e-05 s, 0.0 kB/s 2023-10-06T03:04:37.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.494 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.549 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1193e-05 s, 0.0 kB/s 2023-10-06T03:04:37.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.550 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.606 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.606 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.606 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2026e-05 s, 0.0 kB/s 2023-10-06T03:04:37.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.607 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.661 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9721e-05 s, 0.0 kB/s 2023-10-06T03:04:37.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.662 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9427e-05 s, 0.0 kB/s 2023-10-06T03:04:37.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.773 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.774 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.14e-05 s, 0.0 kB/s 2023-10-06T03:04:37.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.828 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.829 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.829 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7821e-05 s, 0.0 kB/s 2023-10-06T03:04:37.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.829 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2924e-05 s, 0.0 kB/s 2023-10-06T03:04:37.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.884 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1243e-05 s, 0.0 kB/s 2023-10-06T03:04:37.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:37.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:37.995 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2667e-05 s, 0.0 kB/s 2023-10-06T03:04:37.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:37.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.050 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.050 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.050 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9113e-05 s, 0.0 kB/s 2023-10-06T03:04:38.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.051 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.106 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2302e-05 s, 0.0 kB/s 2023-10-06T03:04:38.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.107 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.162 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6085e-05 s, 0.0 kB/s 2023-10-06T03:04:38.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.162 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9556e-05 s, 0.0 kB/s 2023-10-06T03:04:38.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.218 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.273 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8239e-05 s, 0.0 kB/s 2023-10-06T03:04:38.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.273 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.329 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.329 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.329 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.999e-05 s, 0.0 kB/s 2023-10-06T03:04:38.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.329 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.385 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8798e-05 s, 0.0 kB/s 2023-10-06T03:04:38.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.442 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8935e-05 s, 0.0 kB/s 2023-10-06T03:04:38.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.442 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.497 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8114e-05 s, 0.0 kB/s 2023-10-06T03:04:38.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.497 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.552 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0709e-05 s, 0.0 kB/s 2023-10-06T03:04:38.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.552 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.607 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8763e-05 s, 0.0 kB/s 2023-10-06T03:04:38.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.608 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.662 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.663 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8274e-05 s, 0.0 kB/s 2023-10-06T03:04:38.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.663 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.718 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5451e-05 s, 0.0 kB/s 2023-10-06T03:04:38.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.719 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.774 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1235e-05 s, 0.0 kB/s 2023-10-06T03:04:38.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.774 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.830 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8094e-05 s, 0.0 kB/s 2023-10-06T03:04:38.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.831 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.885 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.885 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.886 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3339e-05 s, 0.0 kB/s 2023-10-06T03:04:38.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.886 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.941 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.941 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.941 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.274e-05 s, 0.0 kB/s 2023-10-06T03:04:38.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.942 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:38.997 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9609e-05 s, 0.0 kB/s 2023-10-06T03:04:38.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:38.997 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.052 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.052 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.052 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.095e-05 s, 0.0 kB/s 2023-10-06T03:04:39.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.107 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.855e-05 s, 0.0 kB/s 2023-10-06T03:04:39.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.108 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.163 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8246e-05 s, 0.0 kB/s 2023-10-06T03:04:39.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.163 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.218 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.218 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.218 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0389e-05 s, 0.0 kB/s 2023-10-06T03:04:39.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.218 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.273 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9733e-05 s, 0.0 kB/s 2023-10-06T03:04:39.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.274 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.330 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1893e-05 s, 0.0 kB/s 2023-10-06T03:04:39.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.330 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.385 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.386 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9104e-05 s, 0.0 kB/s 2023-10-06T03:04:39.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0399e-05 s, 0.0 kB/s 2023-10-06T03:04:39.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.441 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.497 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0662e-05 s, 0.0 kB/s 2023-10-06T03:04:39.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.497 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.553 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.553 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.553 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9429e-05 s, 0.0 kB/s 2023-10-06T03:04:39.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.553 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.609 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9394e-05 s, 0.0 kB/s 2023-10-06T03:04:39.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.609 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.665 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7172e-05 s, 0.0 kB/s 2023-10-06T03:04:39.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.665 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.721 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9966e-05 s, 0.0 kB/s 2023-10-06T03:04:39.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.722 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.777 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9341e-05 s, 0.0 kB/s 2023-10-06T03:04:39.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.778 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.832 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.833 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.833 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1593e-05 s, 0.0 kB/s 2023-10-06T03:04:39.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.833 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.888 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9041e-05 s, 0.0 kB/s 2023-10-06T03:04:39.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.889 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:39.945 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:39.945 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:39.945 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7279e-05 s, 0.0 kB/s 2023-10-06T03:04:39.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:39.946 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.001 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2425e-05 s, 0.0 kB/s 2023-10-06T03:04:40.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.002 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.057 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.057 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.057 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8588e-05 s, 0.0 kB/s 2023-10-06T03:04:40.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.058 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.112 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9988e-05 s, 0.0 kB/s 2023-10-06T03:04:40.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.114 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.126e-05 s, 0.0 kB/s 2023-10-06T03:04:40.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.169 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.223 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.224 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8587e-05 s, 0.0 kB/s 2023-10-06T03:04:40.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.224 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.279 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.279 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-10-06T03:04:40.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.279 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.333 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1504e-05 s, 0.0 kB/s 2023-10-06T03:04:40.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.334 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.388 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.389 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.389 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7998e-05 s, 0.0 kB/s 2023-10-06T03:04:40.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.390 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.444 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.444 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9651e-05 s, 0.0 kB/s 2023-10-06T03:04:40.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.444 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.499 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6178e-05 s, 0.0 kB/s 2023-10-06T03:04:40.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.499 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.554 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.555 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.555 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9022e-05 s, 0.0 kB/s 2023-10-06T03:04:40.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.555 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1493e-05 s, 0.0 kB/s 2023-10-06T03:04:40.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.610 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.665 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8917e-05 s, 0.0 kB/s 2023-10-06T03:04:40.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.666 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.721 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9729e-05 s, 0.0 kB/s 2023-10-06T03:04:40.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.721 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.777 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.165e-05 s, 0.0 kB/s 2023-10-06T03:04:40.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.777 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.831 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.832 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0082e-05 s, 0.0 kB/s 2023-10-06T03:04:40.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.832 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.887 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7955e-05 s, 0.0 kB/s 2023-10-06T03:04:40.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.887 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.942 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.942 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.942 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9913e-05 s, 0.0 kB/s 2023-10-06T03:04:40.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.943 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:40.998 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:40.998 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:40.998 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.128e-05 s, 0.0 kB/s 2023-10-06T03:04:40.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:40.999 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.053 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.053 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.053 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9705e-05 s, 0.0 kB/s 2023-10-06T03:04:41.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.108 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0601e-05 s, 0.0 kB/s 2023-10-06T03:04:41.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.109 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.163 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.164 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.164 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.4544e-05 s, 0.0 kB/s 2023-10-06T03:04:41.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.164 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.219 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3546e-05 s, 0.0 kB/s 2023-10-06T03:04:41.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.219 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.273 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7287e-05 s, 0.0 kB/s 2023-10-06T03:04:41.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.274 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.328 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.328 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.328 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1044e-05 s, 0.0 kB/s 2023-10-06T03:04:41.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.329 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.383 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.384 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8578e-05 s, 0.0 kB/s 2023-10-06T03:04:41.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.384 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.439 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1532e-05 s, 0.0 kB/s 2023-10-06T03:04:41.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.440 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.495 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0506e-05 s, 0.0 kB/s 2023-10-06T03:04:41.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.496 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.551 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6689e-05 s, 0.0 kB/s 2023-10-06T03:04:41.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.551 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.608 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.058e-05 s, 0.0 kB/s 2023-10-06T03:04:41.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.609 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8029e-05 s, 0.0 kB/s 2023-10-06T03:04:41.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.664 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.719 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7911e-05 s, 0.0 kB/s 2023-10-06T03:04:41.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.720 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.775 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.775 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.775 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2169e-05 s, 0.0 kB/s 2023-10-06T03:04:41.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.776 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.831 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.831 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.831 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1357e-05 s, 0.0 kB/s 2023-10-06T03:04:41.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.832 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.887 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0181e-05 s, 0.0 kB/s 2023-10-06T03:04:41.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.888 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:41.944 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:41.944 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:41.944 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4014e-05 s, 0.0 kB/s 2023-10-06T03:04:41.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:41.945 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.001 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.002 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3443e-05 s, 0.0 kB/s 2023-10-06T03:04:42.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.002 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:04:42.058 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.058 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.058 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0993e-05 s, 0.0 kB/s 2023-10-06T03:04:42.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.059 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.060 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:04:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3416e-05 s, 0.0 kB/s 2023-10-06T03:04:42.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.116 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.171 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.171 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.171 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3398e-05 s, 0.0 kB/s 2023-10-06T03:04:42.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.171 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1206e-05 s, 0.0 kB/s 2023-10-06T03:04:42.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.228 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.283 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.087e-05 s, 0.0 kB/s 2023-10-06T03:04:42.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.284 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.340 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.340 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2093e-05 s, 0.0 kB/s 2023-10-06T03:04:42.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.340 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.395 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0909e-05 s, 0.0 kB/s 2023-10-06T03:04:42.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.395 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0298e-05 s, 0.0 kB/s 2023-10-06T03:04:42.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.451 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.507 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1305e-05 s, 0.0 kB/s 2023-10-06T03:04:42.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.508 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2068e-05 s, 0.0 kB/s 2023-10-06T03:04:42.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.563 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.620 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2166e-05 s, 0.0 kB/s 2023-10-06T03:04:42.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.621 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.677 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0164e-05 s, 0.0 kB/s 2023-10-06T03:04:42.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.677 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.732 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.732 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.732 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.045e-05 s, 0.0 kB/s 2023-10-06T03:04:42.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.788 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.789 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.109e-05 s, 0.0 kB/s 2023-10-06T03:04:42.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.789 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.844 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0753e-05 s, 0.0 kB/s 2023-10-06T03:04:42.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.844 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.900 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2392e-05 s, 0.0 kB/s 2023-10-06T03:04:42.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.900 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:42.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:42.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:42.955 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0132e-05 s, 0.0 kB/s 2023-10-06T03:04:42.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:42.955 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.011 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8798e-05 s, 0.0 kB/s 2023-10-06T03:04:43.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.012 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.067 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0756e-05 s, 0.0 kB/s 2023-10-06T03:04:43.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.068 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1318e-05 s, 0.0 kB/s 2023-10-06T03:04:43.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.179 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2704e-05 s, 0.0 kB/s 2023-10-06T03:04:43.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.234 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1787e-05 s, 0.0 kB/s 2023-10-06T03:04:43.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.290 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2381e-05 s, 0.0 kB/s 2023-10-06T03:04:43.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.291 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.346 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.211e-05 s, 0.0 kB/s 2023-10-06T03:04:43.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.402 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0754e-05 s, 0.0 kB/s 2023-10-06T03:04:43.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.403 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1099e-05 s, 0.0 kB/s 2023-10-06T03:04:43.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.515 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0993e-05 s, 0.0 kB/s 2023-10-06T03:04:43.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.515 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.570 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.032e-05 s, 0.0 kB/s 2023-10-06T03:04:43.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.571 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.626 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7417e-05 s, 0.0 kB/s 2023-10-06T03:04:43.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.626 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.682 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8766e-05 s, 0.0 kB/s 2023-10-06T03:04:43.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.683 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.738 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.738 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.738 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0785e-05 s, 0.0 kB/s 2023-10-06T03:04:43.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.738 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.821 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9516e-05 s, 0.0 kB/s 2023-10-06T03:04:43.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.821 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.849 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8808e-05 s, 0.0 kB/s 2023-10-06T03:04:43.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.903 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8661e-05 s, 0.0 kB/s 2023-10-06T03:04:43.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:43.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:43.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:43.959 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8457e-05 s, 0.0 kB/s 2023-10-06T03:04:43.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:43.959 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.015 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1701e-05 s, 0.0 kB/s 2023-10-06T03:04:44.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.070 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1557e-05 s, 0.0 kB/s 2023-10-06T03:04:44.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.070 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.124 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1228e-05 s, 0.0 kB/s 2023-10-06T03:04:44.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.180 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-06T03:04:44.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.180 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.235 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1409e-05 s, 0.0 kB/s 2023-10-06T03:04:44.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.235 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.291 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9719e-05 s, 0.0 kB/s 2023-10-06T03:04:44.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.346 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0205e-05 s, 0.0 kB/s 2023-10-06T03:04:44.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.402 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0083e-05 s, 0.0 kB/s 2023-10-06T03:04:44.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3679e-05 s, 0.0 kB/s 2023-10-06T03:04:44.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9972e-05 s, 0.0 kB/s 2023-10-06T03:04:44.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.512 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.566 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9501e-05 s, 0.0 kB/s 2023-10-06T03:04:44.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.622 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0113e-05 s, 0.0 kB/s 2023-10-06T03:04:44.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.622 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.678 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.425e-05 s, 0.0 kB/s 2023-10-06T03:04:44.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.733 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.734 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8323e-05 s, 0.0 kB/s 2023-10-06T03:04:44.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.734 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.788 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.789 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.789 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.4014e-05 s, 0.0 kB/s 2023-10-06T03:04:44.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.789 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.844 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9358e-05 s, 0.0 kB/s 2023-10-06T03:04:44.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.844 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.898 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.899 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.81e-05 s, 0.0 kB/s 2023-10-06T03:04:44.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.899 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:44.953 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:44.953 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:44.953 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9205e-05 s, 0.0 kB/s 2023-10-06T03:04:44.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:44.954 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.008 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.008 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.009 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7441e-05 s, 0.0 kB/s 2023-10-06T03:04:45.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.009 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.064 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.064 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.064 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1164e-05 s, 0.0 kB/s 2023-10-06T03:04:45.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.065 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.120 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.120 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.120 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1708e-05 s, 0.0 kB/s 2023-10-06T03:04:45.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.120 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2155e-05 s, 0.0 kB/s 2023-10-06T03:04:45.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.177 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.232 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.09e-05 s, 0.0 kB/s 2023-10-06T03:04:45.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.232 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.287 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2548e-05 s, 0.0 kB/s 2023-10-06T03:04:45.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.287 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.342 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-10-06T03:04:45.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.399 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0846e-05 s, 0.0 kB/s 2023-10-06T03:04:45.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.400 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.454 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8622e-05 s, 0.0 kB/s 2023-10-06T03:04:45.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1823e-05 s, 0.0 kB/s 2023-10-06T03:04:45.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.510 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.565 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.565 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1538e-05 s, 0.0 kB/s 2023-10-06T03:04:45.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.565 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.620 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0035e-05 s, 0.0 kB/s 2023-10-06T03:04:45.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.620 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.676 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0657e-05 s, 0.0 kB/s 2023-10-06T03:04:45.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.676 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.732 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.958e-05 s, 0.0 kB/s 2023-10-06T03:04:45.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.732 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.788 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1039e-05 s, 0.0 kB/s 2023-10-06T03:04:45.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.789 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.844 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9528e-05 s, 0.0 kB/s 2023-10-06T03:04:45.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.844 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.900 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0375e-05 s, 0.0 kB/s 2023-10-06T03:04:45.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.900 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:45.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:45.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:45.955 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.874e-05 s, 0.0 kB/s 2023-10-06T03:04:45.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:45.955 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.010 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.826e-05 s, 0.0 kB/s 2023-10-06T03:04:46.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.011 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.067 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8425e-05 s, 0.0 kB/s 2023-10-06T03:04:46.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.067 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.975e-05 s, 0.0 kB/s 2023-10-06T03:04:46.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.179 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0841e-05 s, 0.0 kB/s 2023-10-06T03:04:46.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.235 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1905e-05 s, 0.0 kB/s 2023-10-06T03:04:46.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.235 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9499e-05 s, 0.0 kB/s 2023-10-06T03:04:46.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.291 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.346 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9746e-05 s, 0.0 kB/s 2023-10-06T03:04:46.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.346 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.401 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8313e-05 s, 0.0 kB/s 2023-10-06T03:04:46.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8659e-05 s, 0.0 kB/s 2023-10-06T03:04:46.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.457 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5426e-05 s, 0.0 kB/s 2023-10-06T03:04:46.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.568 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5276e-05 s, 0.0 kB/s 2023-10-06T03:04:46.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.568 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1872e-05 s, 0.0 kB/s 2023-10-06T03:04:46.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.679 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9324e-05 s, 0.0 kB/s 2023-10-06T03:04:46.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.680 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.735 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7931e-05 s, 0.0 kB/s 2023-10-06T03:04:46.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.790 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5453e-05 s, 0.0 kB/s 2023-10-06T03:04:46.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.790 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2738e-05 s, 0.0 kB/s 2023-10-06T03:04:46.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.846 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.901 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-10-06T03:04:46.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:46.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:46.956 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9511e-05 s, 0.0 kB/s 2023-10-06T03:04:46.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:46.956 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.011 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.011 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8848e-05 s, 0.0 kB/s 2023-10-06T03:04:47.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.011 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.065 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8519e-05 s, 0.0 kB/s 2023-10-06T03:04:47.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.066 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.122 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9907e-05 s, 0.0 kB/s 2023-10-06T03:04:47.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.178 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0005e-05 s, 0.0 kB/s 2023-10-06T03:04:47.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.178 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.234 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0763e-05 s, 0.0 kB/s 2023-10-06T03:04:47.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.289 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.197e-05 s, 0.0 kB/s 2023-10-06T03:04:47.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.343 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.196e-05 s, 0.0 kB/s 2023-10-06T03:04:47.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.344 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9962e-05 s, 0.0 kB/s 2023-10-06T03:04:47.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.399 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.453 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9562e-05 s, 0.0 kB/s 2023-10-06T03:04:47.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.509 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.509 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.509 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.142e-05 s, 0.0 kB/s 2023-10-06T03:04:47.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.510 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.565 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0445e-05 s, 0.0 kB/s 2023-10-06T03:04:47.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.565 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.620 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9338e-05 s, 0.0 kB/s 2023-10-06T03:04:47.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.621 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.677 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3715e-05 s, 0.0 kB/s 2023-10-06T03:04:47.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.677 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.762 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7481e-05 s, 0.0 kB/s 2023-10-06T03:04:47.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.778 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2162e-05 s, 0.0 kB/s 2023-10-06T03:04:47.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.779 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.834 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7304e-05 s, 0.0 kB/s 2023-10-06T03:04:47.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.834 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.889 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.890 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8766e-05 s, 0.0 kB/s 2023-10-06T03:04:47.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.890 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:47.944 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:47.944 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:47.944 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0353e-05 s, 0.0 kB/s 2023-10-06T03:04:47.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:47.945 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.000 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7577e-05 s, 0.0 kB/s 2023-10-06T03:04:48.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.001 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.056 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1339e-05 s, 0.0 kB/s 2023-10-06T03:04:48.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.057 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.111 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1e-05 s, 0.0 kB/s 2023-10-06T03:04:48.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.112 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9105e-05 s, 0.0 kB/s 2023-10-06T03:04:48.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.169 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9626e-05 s, 0.0 kB/s 2023-10-06T03:04:48.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.224 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.279 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.279 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.279 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7637e-05 s, 0.0 kB/s 2023-10-06T03:04:48.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.280 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.335 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.412 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0829e-05 s, 0.0 kB/s 2023-10-06T03:04:48.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.413 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0368e-05 s, 0.0 kB/s 2023-10-06T03:04:48.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.484 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2088e-05 s, 0.0 kB/s 2023-10-06T03:04:48.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.485 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.540 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9888e-05 s, 0.0 kB/s 2023-10-06T03:04:48.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.540 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.595 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2325e-05 s, 0.0 kB/s 2023-10-06T03:04:48.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.596 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.651 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1586e-05 s, 0.0 kB/s 2023-10-06T03:04:48.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.651 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.707 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.3696e-05 s, 0.0 kB/s 2023-10-06T03:04:48.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.708 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.767 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8271e-05 s, 0.0 kB/s 2023-10-06T03:04:48.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.767 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1691e-05 s, 0.0 kB/s 2023-10-06T03:04:48.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.823 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1745e-05 s, 0.0 kB/s 2023-10-06T03:04:48.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.933 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.954 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.955 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8332e-05 s, 0.0 kB/s 2023-10-06T03:04:48.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.955 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:48.988 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:48.989 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:48.989 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0357e-05 s, 0.0 kB/s 2023-10-06T03:04:48.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:48.989 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.043 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.044 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1727e-05 s, 0.0 kB/s 2023-10-06T03:04:49.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.044 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.099 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.238e-05 s, 0.0 kB/s 2023-10-06T03:04:49.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.099 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.154 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1154e-05 s, 0.0 kB/s 2023-10-06T03:04:49.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.155 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9859e-05 s, 0.0 kB/s 2023-10-06T03:04:49.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.266 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.778e-05 s, 0.0 kB/s 2023-10-06T03:04:49.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9443e-05 s, 0.0 kB/s 2023-10-06T03:04:49.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.378 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8811e-05 s, 0.0 kB/s 2023-10-06T03:04:49.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.379 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9553e-05 s, 0.0 kB/s 2023-10-06T03:04:49.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.435 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.490 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.490 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1412e-05 s, 0.0 kB/s 2023-10-06T03:04:49.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.490 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8462e-05 s, 0.0 kB/s 2023-10-06T03:04:49.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.545 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1286e-05 s, 0.0 kB/s 2023-10-06T03:04:49.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.602 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.656 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9953e-05 s, 0.0 kB/s 2023-10-06T03:04:49.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.657 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.712 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9474e-05 s, 0.0 kB/s 2023-10-06T03:04:49.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.712 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.767 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1889e-05 s, 0.0 kB/s 2023-10-06T03:04:49.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.184e-05 s, 0.0 kB/s 2023-10-06T03:04:49.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1364e-05 s, 0.0 kB/s 2023-10-06T03:04:49.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.934 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.785e-05 s, 0.0 kB/s 2023-10-06T03:04:49.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.934 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:49.990 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.285e-05 s, 0.0 kB/s 2023-10-06T03:04:49.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:49.990 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.046 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.046 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6958e-05 s, 0.0 kB/s 2023-10-06T03:04:50.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.046 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.102 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.102 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.102 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.09e-05 s, 0.0 kB/s 2023-10-06T03:04:50.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.102 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.158 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2508e-05 s, 0.0 kB/s 2023-10-06T03:04:50.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.214 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1089e-05 s, 0.0 kB/s 2023-10-06T03:04:50.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7066e-05 s, 0.0 kB/s 2023-10-06T03:04:50.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.270 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.182e-05 s, 0.0 kB/s 2023-10-06T03:04:50.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.382 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6991e-05 s, 0.0 kB/s 2023-10-06T03:04:50.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.382 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.438 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0042e-05 s, 0.0 kB/s 2023-10-06T03:04:50.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.438 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.496 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.496 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8254e-05 s, 0.0 kB/s 2023-10-06T03:04:50.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.496 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.551 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9716e-05 s, 0.0 kB/s 2023-10-06T03:04:50.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.551 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9698e-05 s, 0.0 kB/s 2023-10-06T03:04:50.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.608 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.663 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8385e-05 s, 0.0 kB/s 2023-10-06T03:04:50.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.663 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.718 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.718 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.718 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8585e-05 s, 0.0 kB/s 2023-10-06T03:04:50.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.719 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.774 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.774 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.774 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.885e-05 s, 0.0 kB/s 2023-10-06T03:04:50.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.776 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.830 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.830 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.830 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1186e-05 s, 0.0 kB/s 2023-10-06T03:04:50.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.830 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.886 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.886 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.886 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9711e-05 s, 0.0 kB/s 2023-10-06T03:04:50.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.886 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.940 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0859e-05 s, 0.0 kB/s 2023-10-06T03:04:50.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.941 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:50.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:50.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:50.996 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.987e-05 s, 0.0 kB/s 2023-10-06T03:04:50.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:50.996 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.051 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.025e-05 s, 0.0 kB/s 2023-10-06T03:04:51.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.052 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.106 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.107 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7051e-05 s, 0.0 kB/s 2023-10-06T03:04:51.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.107 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.162 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3997e-05 s, 0.0 kB/s 2023-10-06T03:04:51.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.162 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.217 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0698e-05 s, 0.0 kB/s 2023-10-06T03:04:51.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.217 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.273 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3602e-05 s, 0.0 kB/s 2023-10-06T03:04:51.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.274 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.328 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.329 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4292e-05 s, 0.0 kB/s 2023-10-06T03:04:51.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.329 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.384 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0401e-05 s, 0.0 kB/s 2023-10-06T03:04:51.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.384 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.440 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.440 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.440 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7779e-05 s, 0.0 kB/s 2023-10-06T03:04:51.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.440 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.496 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8578e-05 s, 0.0 kB/s 2023-10-06T03:04:51.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.497 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.552 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7226e-05 s, 0.0 kB/s 2023-10-06T03:04:51.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.553 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.608 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0254e-05 s, 0.0 kB/s 2023-10-06T03:04:51.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.608 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8221e-05 s, 0.0 kB/s 2023-10-06T03:04:51.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.664 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.719 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.719 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.719 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8216e-05 s, 0.0 kB/s 2023-10-06T03:04:51.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.719 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.775 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.775 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.775 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0998e-05 s, 0.0 kB/s 2023-10-06T03:04:51.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.775 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.831 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.831 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.831 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0673e-05 s, 0.0 kB/s 2023-10-06T03:04:51.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.831 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.887 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9881e-05 s, 0.0 kB/s 2023-10-06T03:04:51.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.887 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.943 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7818e-05 s, 0.0 kB/s 2023-10-06T03:04:51.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.944 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:51.998 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:51.999 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8669e-05 s, 0.0 kB/s 2023-10-06T03:04:51.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:51.999 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.053 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.054 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.876e-05 s, 0.0 kB/s 2023-10-06T03:04:52.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.108 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0665e-05 s, 0.0 kB/s 2023-10-06T03:04:52.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.109 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.163 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.163 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.164 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-06T03:04:52.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.164 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.219 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8251e-05 s, 0.0 kB/s 2023-10-06T03:04:52.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.220 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.275 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.875e-05 s, 0.0 kB/s 2023-10-06T03:04:52.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.276 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.331 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7727e-05 s, 0.0 kB/s 2023-10-06T03:04:52.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.387 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9044e-05 s, 0.0 kB/s 2023-10-06T03:04:52.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.387 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.443 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9279e-05 s, 0.0 kB/s 2023-10-06T03:04:52.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.443 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.499 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.499 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.499 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1124e-05 s, 0.0 kB/s 2023-10-06T03:04:52.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.499 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.555 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1514e-05 s, 0.0 kB/s 2023-10-06T03:04:52.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.555 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.611 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7793e-05 s, 0.0 kB/s 2023-10-06T03:04:52.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.612 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.667 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.31e-05 s, 0.0 kB/s 2023-10-06T03:04:52.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.667 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.722 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0311e-05 s, 0.0 kB/s 2023-10-06T03:04:52.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.722 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.777 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0405e-05 s, 0.0 kB/s 2023-10-06T03:04:52.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.777 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.832 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.832 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.833 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0358e-05 s, 0.0 kB/s 2023-10-06T03:04:52.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.833 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.888 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.888 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1273e-05 s, 0.0 kB/s 2023-10-06T03:04:52.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.888 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.943 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.943 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.943 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8206e-05 s, 0.0 kB/s 2023-10-06T03:04:52.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.943 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:52.999 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6279e-05 s, 0.0 kB/s 2023-10-06T03:04:52.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:52.999 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.053 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.053 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.054 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9129e-05 s, 0.0 kB/s 2023-10-06T03:04:53.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.054 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.109 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7509e-05 s, 0.0 kB/s 2023-10-06T03:04:53.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.109 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.164 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.165 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.165 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7441e-05 s, 0.0 kB/s 2023-10-06T03:04:53.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.165 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.221 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.221 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.221 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8935e-05 s, 0.0 kB/s 2023-10-06T03:04:53.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.221 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.288 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1766e-05 s, 0.0 kB/s 2023-10-06T03:04:53.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.343 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1644e-05 s, 0.0 kB/s 2023-10-06T03:04:53.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.401 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2413e-05 s, 0.0 kB/s 2023-10-06T03:04:53.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9656e-05 s, 0.0 kB/s 2023-10-06T03:04:53.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1905e-05 s, 0.0 kB/s 2023-10-06T03:04:53.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.567 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1029e-05 s, 0.0 kB/s 2023-10-06T03:04:53.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.568 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.623 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.915e-05 s, 0.0 kB/s 2023-10-06T03:04:53.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.624 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7046e-05 s, 0.0 kB/s 2023-10-06T03:04:53.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.735 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2447e-05 s, 0.0 kB/s 2023-10-06T03:04:53.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0981e-05 s, 0.0 kB/s 2023-10-06T03:04:53.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.792 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9265e-05 s, 0.0 kB/s 2023-10-06T03:04:53.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.902 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9485e-05 s, 0.0 kB/s 2023-10-06T03:04:53.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.902 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:53.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:53.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:53.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0318e-05 s, 0.0 kB/s 2023-10-06T03:04:53.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:53.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.014 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8312e-05 s, 0.0 kB/s 2023-10-06T03:04:54.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8568e-05 s, 0.0 kB/s 2023-10-06T03:04:54.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.125 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0225e-05 s, 0.0 kB/s 2023-10-06T03:04:54.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.179 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9748e-05 s, 0.0 kB/s 2023-10-06T03:04:54.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.181 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.235 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2255e-05 s, 0.0 kB/s 2023-10-06T03:04:54.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.236 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.291 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1721e-05 s, 0.0 kB/s 2023-10-06T03:04:54.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.291 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.347 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1361e-05 s, 0.0 kB/s 2023-10-06T03:04:54.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.401 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9531e-05 s, 0.0 kB/s 2023-10-06T03:04:54.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-10-06T03:04:54.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.457 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0533e-05 s, 0.0 kB/s 2023-10-06T03:04:54.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.569 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.569 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5067e-05 s, 0.0 kB/s 2023-10-06T03:04:54.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2801e-05 s, 0.0 kB/s 2023-10-06T03:04:54.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.679 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-10-06T03:04:54.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.680 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.736 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0866e-05 s, 0.0 kB/s 2023-10-06T03:04:54.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.736 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-10-06T03:04:54.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.792 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2522e-05 s, 0.0 kB/s 2023-10-06T03:04:54.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.903 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-06T03:04:54.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.903 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:54.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:54.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0878e-05 s, 0.0 kB/s 2023-10-06T03:04:54.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:54.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9064e-05 s, 0.0 kB/s 2023-10-06T03:04:55.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.269e-05 s, 0.0 kB/s 2023-10-06T03:04:55.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.124 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0255e-05 s, 0.0 kB/s 2023-10-06T03:04:55.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.180 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8318e-05 s, 0.0 kB/s 2023-10-06T03:04:55.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.180 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.237 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1977e-05 s, 0.0 kB/s 2023-10-06T03:04:55.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.237 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.293 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.052e-05 s, 0.0 kB/s 2023-10-06T03:04:55.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.348 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0532e-05 s, 0.0 kB/s 2023-10-06T03:04:55.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2996e-05 s, 0.0 kB/s 2023-10-06T03:04:55.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.404 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.459 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8302e-05 s, 0.0 kB/s 2023-10-06T03:04:55.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.459 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.514 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3813e-05 s, 0.0 kB/s 2023-10-06T03:04:55.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.514 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.569 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.569 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2709e-05 s, 0.0 kB/s 2023-10-06T03:04:55.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.943e-05 s, 0.0 kB/s 2023-10-06T03:04:55.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1061e-05 s, 0.0 kB/s 2023-10-06T03:04:55.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.681 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.736 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7964e-05 s, 0.0 kB/s 2023-10-06T03:04:55.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.736 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8795e-05 s, 0.0 kB/s 2023-10-06T03:04:55.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.792 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7516e-05 s, 0.0 kB/s 2023-10-06T03:04:55.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.848 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.902 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.308e-05 s, 0.0 kB/s 2023-10-06T03:04:55.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.903 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:55.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3739e-05 s, 0.0 kB/s 2023-10-06T03:04:55.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:55.959 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2906e-05 s, 0.0 kB/s 2023-10-06T03:04:56.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0155e-05 s, 0.0 kB/s 2023-10-06T03:04:56.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.124 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.97e-05 s, 0.0 kB/s 2023-10-06T03:04:56.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.124 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9616e-05 s, 0.0 kB/s 2023-10-06T03:04:56.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.233 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9197e-05 s, 0.0 kB/s 2023-10-06T03:04:56.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.289 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.07e-05 s, 0.0 kB/s 2023-10-06T03:04:56.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.344 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.345 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.384e-05 s, 0.0 kB/s 2023-10-06T03:04:56.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.345 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.399 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1225e-05 s, 0.0 kB/s 2023-10-06T03:04:56.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.400 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.455 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.455 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.455 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9186e-05 s, 0.0 kB/s 2023-10-06T03:04:56.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.511 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.628e-05 s, 0.0 kB/s 2023-10-06T03:04:56.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.511 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.567 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.817e-05 s, 0.0 kB/s 2023-10-06T03:04:56.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.622 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-10-06T03:04:56.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.623 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1417e-05 s, 0.0 kB/s 2023-10-06T03:04:56.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.734 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1407e-05 s, 0.0 kB/s 2023-10-06T03:04:56.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.789 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.790 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8959e-05 s, 0.0 kB/s 2023-10-06T03:04:56.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.791 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.846 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8417e-05 s, 0.0 kB/s 2023-10-06T03:04:56.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2911e-05 s, 0.0 kB/s 2023-10-06T03:04:56.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.902 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3949e-05 s, 0.0 kB/s 2023-10-06T03:04:56.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:56.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.012 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1262e-05 s, 0.0 kB/s 2023-10-06T03:04:57.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.067 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9982e-05 s, 0.0 kB/s 2023-10-06T03:04:57.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.068 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1195e-05 s, 0.0 kB/s 2023-10-06T03:04:57.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.177 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3499e-05 s, 0.0 kB/s 2023-10-06T03:04:57.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.178 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.232 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.232 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.233 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7749e-05 s, 0.0 kB/s 2023-10-06T03:04:57.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.233 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.288 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9811e-05 s, 0.0 kB/s 2023-10-06T03:04:57.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.344 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.344 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.344 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2895e-05 s, 0.0 kB/s 2023-10-06T03:04:57.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.345 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.400 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9301e-05 s, 0.0 kB/s 2023-10-06T03:04:57.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.400 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.456 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.456 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.456 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0953e-05 s, 0.0 kB/s 2023-10-06T03:04:57.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.456 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.005e-05 s, 0.0 kB/s 2023-10-06T03:04:57.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.512 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.567 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8951e-05 s, 0.0 kB/s 2023-10-06T03:04:57.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.622 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2352e-05 s, 0.0 kB/s 2023-10-06T03:04:57.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.623 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.677 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.678 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3389e-05 s, 0.0 kB/s 2023-10-06T03:04:57.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.678 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.734 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1222e-05 s, 0.0 kB/s 2023-10-06T03:04:57.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.734 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.790 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3e-05 s, 0.0 kB/s 2023-10-06T03:04:57.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.791 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0498e-05 s, 0.0 kB/s 2023-10-06T03:04:57.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.901 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.102e-05 s, 0.0 kB/s 2023-10-06T03:04:57.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:57.956 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9283e-05 s, 0.0 kB/s 2023-10-06T03:04:57.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:57.956 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.011 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.012 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.012 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1109e-05 s, 0.0 kB/s 2023-10-06T03:04:58.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.012 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.066 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.067 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3599e-05 s, 0.0 kB/s 2023-10-06T03:04:58.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.067 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.122 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.122 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.122 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1371e-05 s, 0.0 kB/s 2023-10-06T03:04:58.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.177 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8204e-05 s, 0.0 kB/s 2023-10-06T03:04:58.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.178 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.234 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9528e-05 s, 0.0 kB/s 2023-10-06T03:04:58.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.289 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0316e-05 s, 0.0 kB/s 2023-10-06T03:04:58.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.344 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2256e-05 s, 0.0 kB/s 2023-10-06T03:04:58.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.344 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.400 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7651e-05 s, 0.0 kB/s 2023-10-06T03:04:58.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.400 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.455 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1189e-05 s, 0.0 kB/s 2023-10-06T03:04:58.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.456 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1757e-05 s, 0.0 kB/s 2023-10-06T03:04:58.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1691e-05 s, 0.0 kB/s 2023-10-06T03:04:58.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.097e-05 s, 0.0 kB/s 2023-10-06T03:04:58.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9228e-05 s, 0.0 kB/s 2023-10-06T03:04:58.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.680 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1089e-05 s, 0.0 kB/s 2023-10-06T03:04:58.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9714e-05 s, 0.0 kB/s 2023-10-06T03:04:58.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.792 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0226e-05 s, 0.0 kB/s 2023-10-06T03:04:58.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.903 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9382e-05 s, 0.0 kB/s 2023-10-06T03:04:58.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.903 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:58.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7405e-05 s, 0.0 kB/s 2023-10-06T03:04:58.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:58.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.014 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9126e-05 s, 0.0 kB/s 2023-10-06T03:04:59.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.070 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8026e-05 s, 0.0 kB/s 2023-10-06T03:04:59.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.071 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.126 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9239e-05 s, 0.0 kB/s 2023-10-06T03:04:59.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.126 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.182 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8537e-05 s, 0.0 kB/s 2023-10-06T03:04:59.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.183 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.238 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0785e-05 s, 0.0 kB/s 2023-10-06T03:04:59.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.238 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.292 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1599e-05 s, 0.0 kB/s 2023-10-06T03:04:59.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.348 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5535e-05 s, 0.0 kB/s 2023-10-06T03:04:59.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.403 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.947e-05 s, 0.0 kB/s 2023-10-06T03:04:59.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.404 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.459 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1187e-05 s, 0.0 kB/s 2023-10-06T03:04:59.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.460 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.515 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7266e-05 s, 0.0 kB/s 2023-10-06T03:04:59.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.515 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7061e-05 s, 0.0 kB/s 2023-10-06T03:04:59.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.570 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7971e-05 s, 0.0 kB/s 2023-10-06T03:04:59.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.628 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.682 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9086e-05 s, 0.0 kB/s 2023-10-06T03:04:59.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.682 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.034e-05 s, 0.0 kB/s 2023-10-06T03:04:59.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.738 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9085e-05 s, 0.0 kB/s 2023-10-06T03:04:59.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.849 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.849 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.849 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.673e-05 s, 0.0 kB/s 2023-10-06T03:04:59.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.905 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9243e-05 s, 0.0 kB/s 2023-10-06T03:04:59.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:04:59.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:04:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:04:59.961 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0679e-05 s, 0.0 kB/s 2023-10-06T03:04:59.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:04:59.961 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.016 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0641e-05 s, 0.0 kB/s 2023-10-06T03:05:00.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.072 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3535e-05 s, 0.0 kB/s 2023-10-06T03:05:00.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.072 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.127 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0925e-05 s, 0.0 kB/s 2023-10-06T03:05:00.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.127 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.184 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8918e-05 s, 0.0 kB/s 2023-10-06T03:05:00.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.184 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.240 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0601e-05 s, 0.0 kB/s 2023-10-06T03:05:00.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.296 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.5869e-05 s, 0.0 kB/s 2023-10-06T03:05:00.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.296 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.351 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8666e-05 s, 0.0 kB/s 2023-10-06T03:05:00.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.351 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.407 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8739e-05 s, 0.0 kB/s 2023-10-06T03:05:00.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0654e-05 s, 0.0 kB/s 2023-10-06T03:05:00.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.518 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7915e-05 s, 0.0 kB/s 2023-10-06T03:05:00.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.519 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.573 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.9208e-05 s, 0.0 kB/s 2023-10-06T03:05:00.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.574 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.629 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1039e-05 s, 0.0 kB/s 2023-10-06T03:05:00.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.629 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.685 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2804e-05 s, 0.0 kB/s 2023-10-06T03:05:00.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.686 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.740 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9977e-05 s, 0.0 kB/s 2023-10-06T03:05:00.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.741 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.795 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.796 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2033e-05 s, 0.0 kB/s 2023-10-06T03:05:00.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.851 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0057e-05 s, 0.0 kB/s 2023-10-06T03:05:00.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.851 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3462e-05 s, 0.0 kB/s 2023-10-06T03:05:00.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.907 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:00.962 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:00.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:00.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8952e-05 s, 0.0 kB/s 2023-10-06T03:05:00.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:00.963 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.017 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9047e-05 s, 0.0 kB/s 2023-10-06T03:05:01.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.018 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.073 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.074 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-10-06T03:05:01.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.074 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.128 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0315e-05 s, 0.0 kB/s 2023-10-06T03:05:01.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.129 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.185 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9615e-05 s, 0.0 kB/s 2023-10-06T03:05:01.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.239 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-06T03:05:01.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.294 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.294 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.295 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0253e-05 s, 0.0 kB/s 2023-10-06T03:05:01.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.349 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0175e-05 s, 0.0 kB/s 2023-10-06T03:05:01.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.350 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.405 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1473e-05 s, 0.0 kB/s 2023-10-06T03:05:01.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.405 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.459 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2823e-05 s, 0.0 kB/s 2023-10-06T03:05:01.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.460 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.515 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-06T03:05:01.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.516 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.571 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.571 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1275e-05 s, 0.0 kB/s 2023-10-06T03:05:01.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.571 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.625 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.927e-05 s, 0.0 kB/s 2023-10-06T03:05:01.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.626 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.683 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1426e-05 s, 0.0 kB/s 2023-10-06T03:05:01.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.683 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8457e-05 s, 0.0 kB/s 2023-10-06T03:05:01.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.738 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9592e-05 s, 0.0 kB/s 2023-10-06T03:05:01.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.794 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.849 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1558e-05 s, 0.0 kB/s 2023-10-06T03:05:01.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.850 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.905 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.905 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.905 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0381e-05 s, 0.0 kB/s 2023-10-06T03:05:01.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:01.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:01.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:01.961 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0367e-05 s, 0.0 kB/s 2023-10-06T03:05:01.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:01.961 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.016 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6852e-05 s, 0.0 kB/s 2023-10-06T03:05:02.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.070 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3629e-05 s, 0.0 kB/s 2023-10-06T03:05:02.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.071 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.126 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0158e-05 s, 0.0 kB/s 2023-10-06T03:05:02.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.127 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.181 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1898e-05 s, 0.0 kB/s 2023-10-06T03:05:02.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.182 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.238 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9454e-05 s, 0.0 kB/s 2023-10-06T03:05:02.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.238 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.292 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2139e-05 s, 0.0 kB/s 2023-10-06T03:05:02.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.347 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7403e-05 s, 0.0 kB/s 2023-10-06T03:05:02.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.403 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0751e-05 s, 0.0 kB/s 2023-10-06T03:05:02.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.403 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8866e-05 s, 0.0 kB/s 2023-10-06T03:05:02.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.514 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.514 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0286e-05 s, 0.0 kB/s 2023-10-06T03:05:02.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.515 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.569 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.570 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.125e-05 s, 0 B/s 2023-10-06T03:05:02.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.570 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.625 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0652e-05 s, 0.0 kB/s 2023-10-06T03:05:02.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.626 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.681 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1184e-05 s, 0.0 kB/s 2023-10-06T03:05:02.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.682 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8528e-05 s, 0.0 kB/s 2023-10-06T03:05:02.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.792 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4208e-05 s, 0.0 kB/s 2023-10-06T03:05:02.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8216e-05 s, 0.0 kB/s 2023-10-06T03:05:02.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7938e-05 s, 0.0 kB/s 2023-10-06T03:05:02.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:02.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:02.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:02.959 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8518e-05 s, 0.0 kB/s 2023-10-06T03:05:02.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:02.959 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9828e-05 s, 0.0 kB/s 2023-10-06T03:05:03.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7932e-05 s, 0.0 kB/s 2023-10-06T03:05:03.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.126 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8281e-05 s, 0.0 kB/s 2023-10-06T03:05:03.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.127 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.180 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1479e-05 s, 0.0 kB/s 2023-10-06T03:05:03.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.180 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.234 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9083e-05 s, 0.0 kB/s 2023-10-06T03:05:03.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.235 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.291 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0306e-05 s, 0.0 kB/s 2023-10-06T03:05:03.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.291 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.345 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7732e-05 s, 0.0 kB/s 2023-10-06T03:05:03.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.346 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.401 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9623e-05 s, 0.0 kB/s 2023-10-06T03:05:03.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.075e-05 s, 0.0 kB/s 2023-10-06T03:05:03.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.457 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2916e-05 s, 0.0 kB/s 2023-10-06T03:05:03.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.568 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8053e-05 s, 0.0 kB/s 2023-10-06T03:05:03.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.645 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7932e-05 s, 0.0 kB/s 2023-10-06T03:05:03.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.646 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.701 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.701 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.701 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9649e-05 s, 0.0 kB/s 2023-10-06T03:05:03.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.757 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0317e-05 s, 0.0 kB/s 2023-10-06T03:05:03.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0864e-05 s, 0.0 kB/s 2023-10-06T03:05:03.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9093e-05 s, 0.0 kB/s 2023-10-06T03:05:03.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.924 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2929e-05 s, 0.0 kB/s 2023-10-06T03:05:03.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.924 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:03.978 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:03.978 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:03.978 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.816e-05 s, 0.0 kB/s 2023-10-06T03:05:03.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:03.979 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.033 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.033 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.033 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0987e-05 s, 0.0 kB/s 2023-10-06T03:05:04.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.034 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.089 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.090 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0072e-05 s, 0.0 kB/s 2023-10-06T03:05:04.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.090 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.145 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-06T03:05:04.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.200 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1696e-05 s, 0.0 kB/s 2023-10-06T03:05:04.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.200 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.895e-05 s, 0.0 kB/s 2023-10-06T03:05:04.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0055e-05 s, 0.0 kB/s 2023-10-06T03:05:04.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.367 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.056e-05 s, 0.0 kB/s 2023-10-06T03:05:04.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0141e-05 s, 0.0 kB/s 2023-10-06T03:05:04.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.424 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.479 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.479 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.479 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1485e-05 s, 0.0 kB/s 2023-10-06T03:05:04.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.480 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.535 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.535 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.535 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-10-06T03:05:04.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.536 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.591 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8493e-05 s, 0.0 kB/s 2023-10-06T03:05:04.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.591 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.965e-05 s, 0.0 kB/s 2023-10-06T03:05:04.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.647 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.702 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.702 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.702 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1427e-05 s, 0.0 kB/s 2023-10-06T03:05:04.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.702 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1456e-05 s, 0.0 kB/s 2023-10-06T03:05:04.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.758 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.814 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.814 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.814 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5495e-05 s, 0.0 kB/s 2023-10-06T03:05:04.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.869 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.869 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.869 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0148e-05 s, 0.0 kB/s 2023-10-06T03:05:04.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.871 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.926 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0465e-05 s, 0.0 kB/s 2023-10-06T03:05:04.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.926 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:04.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:04.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:04.982 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1263e-05 s, 0.0 kB/s 2023-10-06T03:05:04.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:04.982 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.036 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0997e-05 s, 0.0 kB/s 2023-10-06T03:05:05.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.037 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.091 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9967e-05 s, 0.0 kB/s 2023-10-06T03:05:05.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.092 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.147 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7344e-05 s, 0.0 kB/s 2023-10-06T03:05:05.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.147 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.201 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.201 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.201 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0338e-05 s, 0.0 kB/s 2023-10-06T03:05:05.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.202 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.9729e-05 s, 0.0 kB/s 2023-10-06T03:05:05.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.311 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.311 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.4844e-05 s, 0.0 kB/s 2023-10-06T03:05:05.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.367 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8615e-05 s, 0.0 kB/s 2023-10-06T03:05:05.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.367 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.422 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9173e-05 s, 0.0 kB/s 2023-10-06T03:05:05.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.423 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.477 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7851e-05 s, 0.0 kB/s 2023-10-06T03:05:05.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.478 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.534 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.781e-05 s, 0.0 kB/s 2023-10-06T03:05:05.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.588 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.589 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.296e-05 s, 0.0 kB/s 2023-10-06T03:05:05.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.589 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.644 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1738e-05 s, 0.0 kB/s 2023-10-06T03:05:05.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.645 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.700 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.974e-05 s, 0.0 kB/s 2023-10-06T03:05:05.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.756 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0086e-05 s, 0.0 kB/s 2023-10-06T03:05:05.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0055e-05 s, 0.0 kB/s 2023-10-06T03:05:05.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.812 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0105e-05 s, 0.0 kB/s 2023-10-06T03:05:05.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1919e-05 s, 0.0 kB/s 2023-10-06T03:05:05.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.925 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:05.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:05.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:05.980 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7851e-05 s, 0.0 kB/s 2023-10-06T03:05:05.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:05.980 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.035 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9322e-05 s, 0.0 kB/s 2023-10-06T03:05:06.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.036 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.091 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0387e-05 s, 0.0 kB/s 2023-10-06T03:05:06.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.092 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.148 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9706e-05 s, 0.0 kB/s 2023-10-06T03:05:06.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.148 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.202 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.203 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.203 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.82e-05 s, 0.0 kB/s 2023-10-06T03:05:06.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.203 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.258 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0858e-05 s, 0.0 kB/s 2023-10-06T03:05:06.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.258 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9814e-05 s, 0.0 kB/s 2023-10-06T03:05:06.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.314 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.369 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.369 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.369 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7411e-05 s, 0.0 kB/s 2023-10-06T03:05:06.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.370 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.425 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.425 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.425 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6958e-05 s, 0.0 kB/s 2023-10-06T03:05:06.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.425 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.480 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.480 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.480 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8428e-05 s, 0.0 kB/s 2023-10-06T03:05:06.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.481 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.536 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.536 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.536 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7559e-05 s, 0.0 kB/s 2023-10-06T03:05:06.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.537 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.592 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9857e-05 s, 0.0 kB/s 2023-10-06T03:05:06.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.592 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.648 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.648 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.648 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2567e-05 s, 0.0 kB/s 2023-10-06T03:05:06.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.649 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.703 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.704 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.704 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4167e-05 s, 0.0 kB/s 2023-10-06T03:05:06.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.704 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2602e-05 s, 0.0 kB/s 2023-10-06T03:05:06.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.759 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.814 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0017e-05 s, 0.0 kB/s 2023-10-06T03:05:06.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.814 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.869 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.870 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.870 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0243e-05 s, 0.0 kB/s 2023-10-06T03:05:06.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.870 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.925 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9983e-05 s, 0.0 kB/s 2023-10-06T03:05:06.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.926 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:06.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:06.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:06.981 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9559e-05 s, 0.0 kB/s 2023-10-06T03:05:06.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:06.982 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.037 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0196e-05 s, 0.0 kB/s 2023-10-06T03:05:07.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.037 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.093 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.093 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.093 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2104e-05 s, 0.0 kB/s 2023-10-06T03:05:07.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.149 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9712e-05 s, 0.0 kB/s 2023-10-06T03:05:07.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.149 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1426e-05 s, 0.0 kB/s 2023-10-06T03:05:07.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.260 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8949e-05 s, 0.0 kB/s 2023-10-06T03:05:07.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.260 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.315 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.315 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8626e-05 s, 0.0 kB/s 2023-10-06T03:05:07.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.316 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.372 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.372 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.846e-05 s, 0.0 kB/s 2023-10-06T03:05:07.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.427 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.427 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4456e-05 s, 0.0 kB/s 2023-10-06T03:05:07.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.482 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8103e-05 s, 0.0 kB/s 2023-10-06T03:05:07.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.482 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.538 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.538 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.538 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2295e-05 s, 0.0 kB/s 2023-10-06T03:05:07.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.539 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.594 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8733e-05 s, 0.0 kB/s 2023-10-06T03:05:07.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.595 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.651 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.918e-05 s, 0.0 kB/s 2023-10-06T03:05:07.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.707 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9224e-05 s, 0.0 kB/s 2023-10-06T03:05:07.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.707 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.763 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2009e-05 s, 0.0 kB/s 2023-10-06T03:05:07.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.764 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.819 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.819 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0752e-05 s, 0.0 kB/s 2023-10-06T03:05:07.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.819 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.874 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8237e-05 s, 0.0 kB/s 2023-10-06T03:05:07.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.929 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8141e-05 s, 0.0 kB/s 2023-10-06T03:05:07.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.930 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:07.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:07.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:07.985 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1423e-05 s, 0.0 kB/s 2023-10-06T03:05:07.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:07.985 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.040 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2136e-05 s, 0.0 kB/s 2023-10-06T03:05:08.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.040 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2695e-05 s, 0.0 kB/s 2023-10-06T03:05:08.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.095 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.150 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2399e-05 s, 0.0 kB/s 2023-10-06T03:05:08.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.151 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.206 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6589e-05 s, 0.0 kB/s 2023-10-06T03:05:08.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.206 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3328e-05 s, 0.0 kB/s 2023-10-06T03:05:08.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.262 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.317 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7256e-05 s, 0.0 kB/s 2023-10-06T03:05:08.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.317 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.371 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9651e-05 s, 0.0 kB/s 2023-10-06T03:05:08.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.427 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9765e-05 s, 0.0 kB/s 2023-10-06T03:05:08.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.481 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2684e-05 s, 0.0 kB/s 2023-10-06T03:05:08.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.482 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.539 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.211e-05 s, 0.0 kB/s 2023-10-06T03:05:08.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.539 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.0561e-05 s, 0.0 kB/s 2023-10-06T03:05:08.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.595 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.651 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.652 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1455e-05 s, 0.0 kB/s 2023-10-06T03:05:08.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.652 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.707 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8201e-05 s, 0.0 kB/s 2023-10-06T03:05:08.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.707 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.763 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0739e-05 s, 0.0 kB/s 2023-10-06T03:05:08.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.764 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.819 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8534e-05 s, 0.0 kB/s 2023-10-06T03:05:08.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.819 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.874 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-06T03:05:08.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.875 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.930 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.930 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.930 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9824e-05 s, 0.0 kB/s 2023-10-06T03:05:08.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.931 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:08.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:08.986 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:08.986 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4112e-05 s, 0.0 kB/s 2023-10-06T03:05:08.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:08.986 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.041 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0539e-05 s, 0.0 kB/s 2023-10-06T03:05:09.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.042 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.097 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2565e-05 s, 0.0 kB/s 2023-10-06T03:05:09.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.097 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.153 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0082e-05 s, 0.0 kB/s 2023-10-06T03:05:09.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.153 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.209 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.121e-05 s, 0.0 kB/s 2023-10-06T03:05:09.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.265 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3577e-05 s, 0.0 kB/s 2023-10-06T03:05:09.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.321 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.321 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3969e-05 s, 0.0 kB/s 2023-10-06T03:05:09.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.322 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.1749e-05 s, 0.0 kB/s 2023-10-06T03:05:09.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.378 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.433 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.512e-05 s, 0.0 kB/s 2023-10-06T03:05:09.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.434 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.489 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2536e-05 s, 0.0 kB/s 2023-10-06T03:05:09.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.489 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.545 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8076e-05 s, 0.0 kB/s 2023-10-06T03:05:09.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.545 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3596e-05 s, 0.0 kB/s 2023-10-06T03:05:09.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.602 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2084e-05 s, 0.0 kB/s 2023-10-06T03:05:09.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.660 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.714 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1882e-05 s, 0.0 kB/s 2023-10-06T03:05:09.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.715 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.4929e-05 s, 0.0 kB/s 2023-10-06T03:05:09.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4902e-05 s, 0.0 kB/s 2023-10-06T03:05:09.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2674e-05 s, 0.0 kB/s 2023-10-06T03:05:09.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1804e-05 s, 0.0 kB/s 2023-10-06T03:05:09.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:09.960 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.016 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3275e-05 s, 0.0 kB/s 2023-10-06T03:05:10.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1944e-05 s, 0.0 kB/s 2023-10-06T03:05:10.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.072 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.128 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9646e-05 s, 0.0 kB/s 2023-10-06T03:05:10.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.128 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.183 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0882e-05 s, 0.0 kB/s 2023-10-06T03:05:10.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.183 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.239 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9705e-05 s, 0.0 kB/s 2023-10-06T03:05:10.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.239 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.295 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1793e-05 s, 0.0 kB/s 2023-10-06T03:05:10.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.351 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0119e-05 s, 0.0 kB/s 2023-10-06T03:05:10.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.407 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.986e-05 s, 0.0 kB/s 2023-10-06T03:05:10.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.407 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.462 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0328e-05 s, 0.0 kB/s 2023-10-06T03:05:10.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.462 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.518 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0671e-05 s, 0.0 kB/s 2023-10-06T03:05:10.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.518 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.573 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9607e-05 s, 0.0 kB/s 2023-10-06T03:05:10.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.574 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.629 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.228e-05 s, 0.0 kB/s 2023-10-06T03:05:10.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.629 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.686 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0287e-05 s, 0.0 kB/s 2023-10-06T03:05:10.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.686 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.741 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9097e-05 s, 0.0 kB/s 2023-10-06T03:05:10.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.741 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.797 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1002e-05 s, 0.0 kB/s 2023-10-06T03:05:10.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.798 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.852 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.853 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.853 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2183e-05 s, 0.0 kB/s 2023-10-06T03:05:10.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.853 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.908 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.997e-05 s, 0.0 kB/s 2023-10-06T03:05:10.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.909 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:10.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:10.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:10.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-06T03:05:10.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:10.963 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0141e-05 s, 0.0 kB/s 2023-10-06T03:05:11.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.019 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.074 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1207e-05 s, 0.0 kB/s 2023-10-06T03:05:11.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.074 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.129 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.130 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8999e-05 s, 0.0 kB/s 2023-10-06T03:05:11.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.130 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.185 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7139e-05 s, 0.0 kB/s 2023-10-06T03:05:11.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.241 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.092e-05 s, 0.0 kB/s 2023-10-06T03:05:11.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.242 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1174e-05 s, 0.0 kB/s 2023-10-06T03:05:11.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.351 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0523e-05 s, 0.0 kB/s 2023-10-06T03:05:11.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.957e-05 s, 0.0 kB/s 2023-10-06T03:05:11.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.462 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1669e-05 s, 0.0 kB/s 2023-10-06T03:05:11.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.463 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.517 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5719e-05 s, 0.0 kB/s 2023-10-06T03:05:11.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.518 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.573 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7809e-05 s, 0.0 kB/s 2023-10-06T03:05:11.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.628 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1245e-05 s, 0.0 kB/s 2023-10-06T03:05:11.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.628 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.684 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.336e-05 s, 0.0 kB/s 2023-10-06T03:05:11.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.685 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.741 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1563e-05 s, 0.0 kB/s 2023-10-06T03:05:11.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.741 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.795 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.796 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6859e-05 s, 0.0 kB/s 2023-10-06T03:05:11.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.851 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7549e-05 s, 0.0 kB/s 2023-10-06T03:05:11.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.851 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1455e-05 s, 0.0 kB/s 2023-10-06T03:05:11.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.907 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:11.962 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:11.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:11.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9347e-05 s, 0.0 kB/s 2023-10-06T03:05:11.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:11.963 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.018 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1192e-05 s, 0.0 kB/s 2023-10-06T03:05:12.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.018 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-06T03:05:12.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.073 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:05:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.128 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-06T03:05:12.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.129 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.170 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:05:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.185 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9442e-05 s, 0.0 kB/s 2023-10-06T03:05:12.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.282 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0196e-05 s, 0.0 kB/s 2023-10-06T03:05:12.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.283 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.338 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0669e-05 s, 0.0 kB/s 2023-10-06T03:05:12.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.338 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.393 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.394 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.394 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3016e-05 s, 0.0 kB/s 2023-10-06T03:05:12.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.394 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.448 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.449 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.449 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7855e-05 s, 0.0 kB/s 2023-10-06T03:05:12.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.449 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.503 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.504 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.9971e-05 s, 0.0 kB/s 2023-10-06T03:05:12.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.504 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.560 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7897e-05 s, 0.0 kB/s 2023-10-06T03:05:12.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.561 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.616 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.616 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.616 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4716e-05 s, 0.0 kB/s 2023-10-06T03:05:12.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.617 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.672 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.672 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.672 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8501e-05 s, 0.0 kB/s 2023-10-06T03:05:12.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.673 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.729 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2853e-05 s, 0.0 kB/s 2023-10-06T03:05:12.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.730 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.202e-05 s, 0.0 kB/s 2023-10-06T03:05:12.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.786 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.841 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.841 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.841 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7311e-05 s, 0.0 kB/s 2023-10-06T03:05:12.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.842 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.897 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8165e-05 s, 0.0 kB/s 2023-10-06T03:05:12.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.897 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:12.954 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0584e-05 s, 0.0 kB/s 2023-10-06T03:05:12.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:12.954 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.010 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0617e-05 s, 0.0 kB/s 2023-10-06T03:05:13.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.010 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.064 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.064 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.065 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9279e-05 s, 0.0 kB/s 2023-10-06T03:05:13.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.065 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-10-06T03:05:13.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.120 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.174 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.175 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9971e-05 s, 0.0 kB/s 2023-10-06T03:05:13.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.175 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.230 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.230 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.230 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7213e-05 s, 0.0 kB/s 2023-10-06T03:05:13.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.230 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.285 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2144e-05 s, 0.0 kB/s 2023-10-06T03:05:13.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.286 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.342 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7565e-05 s, 0.0 kB/s 2023-10-06T03:05:13.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.342 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.397 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.397 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.397 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-06T03:05:13.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.398 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.452 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8199e-05 s, 0.0 kB/s 2023-10-06T03:05:13.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.508 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.509 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.509 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9127e-05 s, 0.0 kB/s 2023-10-06T03:05:13.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.509 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.564 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3061e-05 s, 0.0 kB/s 2023-10-06T03:05:13.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.564 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.619 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9033e-05 s, 0.0 kB/s 2023-10-06T03:05:13.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.619 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.675 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.675 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.675 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9618e-05 s, 0.0 kB/s 2023-10-06T03:05:13.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.675 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.731 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9544e-05 s, 0.0 kB/s 2023-10-06T03:05:13.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.731 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.787 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3139e-05 s, 0.0 kB/s 2023-10-06T03:05:13.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.788 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.843 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9355e-05 s, 0.0 kB/s 2023-10-06T03:05:13.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.843 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0906e-05 s, 0.0 kB/s 2023-10-06T03:05:13.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.899 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:13.955 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0476e-05 s, 0.0 kB/s 2023-10-06T03:05:13.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:13.955 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.011 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.011 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8649e-05 s, 0.0 kB/s 2023-10-06T03:05:14.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.011 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.066 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.066 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8049e-05 s, 0.0 kB/s 2023-10-06T03:05:14.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.066 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.120 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.120 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.120 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9537e-05 s, 0.0 kB/s 2023-10-06T03:05:14.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.121 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0507e-05 s, 0.0 kB/s 2023-10-06T03:05:14.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.176 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.231 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2358e-05 s, 0.0 kB/s 2023-10-06T03:05:14.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.231 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.286 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.287 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2111e-05 s, 0.0 kB/s 2023-10-06T03:05:14.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.287 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.343 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1176e-05 s, 0.0 kB/s 2023-10-06T03:05:14.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.397 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8445e-05 s, 0.0 kB/s 2023-10-06T03:05:14.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.398 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.452 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.453 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.453 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5056e-05 s, 0.0 kB/s 2023-10-06T03:05:14.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.508 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7987e-05 s, 0.0 kB/s 2023-10-06T03:05:14.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.508 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.563 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.268e-05 s, 0.0 kB/s 2023-10-06T03:05:14.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.564 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.619 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8444e-05 s, 0.0 kB/s 2023-10-06T03:05:14.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.620 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.674 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.0962e-05 s, 0.0 kB/s 2023-10-06T03:05:14.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.676 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.731 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.731 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.731 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1248e-05 s, 0.0 kB/s 2023-10-06T03:05:14.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.731 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.786 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3241e-05 s, 0.0 kB/s 2023-10-06T03:05:14.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.787 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.844 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0871e-05 s, 0.0 kB/s 2023-10-06T03:05:14.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.844 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.899 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0691e-05 s, 0.0 kB/s 2023-10-06T03:05:14.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.899 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:14.954 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:14.954 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:14.955 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.178e-05 s, 0.0 kB/s 2023-10-06T03:05:14.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:14.955 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.011 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.011 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0564e-05 s, 0.0 kB/s 2023-10-06T03:05:15.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.011 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.065 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0741e-05 s, 0.0 kB/s 2023-10-06T03:05:15.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.066 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.121 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.121 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.121 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7659e-05 s, 0.0 kB/s 2023-10-06T03:05:15.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.122 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.176 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.176 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.176 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9074e-05 s, 0.0 kB/s 2023-10-06T03:05:15.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.177 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.232 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.232 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.232 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9099e-05 s, 0.0 kB/s 2023-10-06T03:05:15.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.232 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.288 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8942e-05 s, 0.0 kB/s 2023-10-06T03:05:15.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.343 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1575e-05 s, 0.0 kB/s 2023-10-06T03:05:15.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.441 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.0127e-05 s, 0.0 kB/s 2023-10-06T03:05:15.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.442 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1889e-05 s, 0.0 kB/s 2023-10-06T03:05:15.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3191e-05 s, 0.0 kB/s 2023-10-06T03:05:15.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.568 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3663e-05 s, 0.0 kB/s 2023-10-06T03:05:15.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.625 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2548e-05 s, 0.0 kB/s 2023-10-06T03:05:15.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.681 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-10-06T03:05:15.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.682 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.736 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9759e-05 s, 0.0 kB/s 2023-10-06T03:05:15.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-10-06T03:05:15.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7956e-05 s, 0.0 kB/s 2023-10-06T03:05:15.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.848 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.942 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.942 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7716e-05 s, 0.0 kB/s 2023-10-06T03:05:15.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.942 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:15.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:15.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:15.959 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8809e-05 s, 0.0 kB/s 2023-10-06T03:05:15.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:15.959 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7989e-05 s, 0.0 kB/s 2023-10-06T03:05:16.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7788e-05 s, 0.0 kB/s 2023-10-06T03:05:16.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.124 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1957e-05 s, 0.0 kB/s 2023-10-06T03:05:16.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.124 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.178 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.179 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8883e-05 s, 0.0 kB/s 2023-10-06T03:05:16.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.180 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.236 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.236 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.236 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1157e-05 s, 0.0 kB/s 2023-10-06T03:05:16.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.236 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.292 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8431e-05 s, 0.0 kB/s 2023-10-06T03:05:16.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.348 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7665e-05 s, 0.0 kB/s 2023-10-06T03:05:16.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.403 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7538e-05 s, 0.0 kB/s 2023-10-06T03:05:16.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.403 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4901e-05 s, 0.0 kB/s 2023-10-06T03:05:16.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8479e-05 s, 0.0 kB/s 2023-10-06T03:05:16.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.567 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.323e-05 s, 0.0 kB/s 2023-10-06T03:05:16.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.568 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.623 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.623 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8529e-05 s, 0.0 kB/s 2023-10-06T03:05:16.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.624 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.678 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3595e-05 s, 0.0 kB/s 2023-10-06T03:05:16.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.735 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0212e-05 s, 0.0 kB/s 2023-10-06T03:05:16.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7424e-05 s, 0.0 kB/s 2023-10-06T03:05:16.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.791 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.942e-05 s, 0.0 kB/s 2023-10-06T03:05:16.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.848 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.903 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.098e-05 s, 0.0 kB/s 2023-10-06T03:05:16.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.903 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:16.959 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2253e-05 s, 0.0 kB/s 2023-10-06T03:05:16.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:16.959 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.234e-05 s, 0.0 kB/s 2023-10-06T03:05:17.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.070 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.070 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.070 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2258e-05 s, 0.0 kB/s 2023-10-06T03:05:17.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.070 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.125 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7927e-05 s, 0.0 kB/s 2023-10-06T03:05:17.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.126 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.181 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.972e-05 s, 0.0 kB/s 2023-10-06T03:05:17.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.181 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.235 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8276e-05 s, 0.0 kB/s 2023-10-06T03:05:17.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.236 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.292 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.178e-05 s, 0.0 kB/s 2023-10-06T03:05:17.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.347 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.347 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2692e-05 s, 0.0 kB/s 2023-10-06T03:05:17.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.401 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9758e-05 s, 0.0 kB/s 2023-10-06T03:05:17.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.456 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2945e-05 s, 0.0 kB/s 2023-10-06T03:05:17.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.457 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8213e-05 s, 0.0 kB/s 2023-10-06T03:05:17.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.514 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.569 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.569 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1629e-05 s, 0.0 kB/s 2023-10-06T03:05:17.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1844e-05 s, 0.0 kB/s 2023-10-06T03:05:17.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0302e-05 s, 0.0 kB/s 2023-10-06T03:05:17.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.681 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.736 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.4084e-05 s, 0.0 kB/s 2023-10-06T03:05:17.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.736 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.847e-05 s, 0.0 kB/s 2023-10-06T03:05:17.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.792 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1289e-05 s, 0.0 kB/s 2023-10-06T03:05:17.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.903 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3087e-05 s, 0.0 kB/s 2023-10-06T03:05:17.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:17.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:17.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:17.959 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.4662e-05 s, 0.0 kB/s 2023-10-06T03:05:17.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:17.959 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.015 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8539e-05 s, 0.0 kB/s 2023-10-06T03:05:18.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7398e-05 s, 0.0 kB/s 2023-10-06T03:05:18.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.071 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.127 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3144e-05 s, 0.0 kB/s 2023-10-06T03:05:18.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.128 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.182 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.182 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.183 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7769e-05 s, 0.0 kB/s 2023-10-06T03:05:18.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.183 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.238 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1311e-05 s, 0.0 kB/s 2023-10-06T03:05:18.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.238 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.293 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2647e-05 s, 0.0 kB/s 2023-10-06T03:05:18.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.349 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8714e-05 s, 0.0 kB/s 2023-10-06T03:05:18.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.350 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.404 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.405 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8944e-05 s, 0.0 kB/s 2023-10-06T03:05:18.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.405 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.459 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.115e-05 s, 0.0 kB/s 2023-10-06T03:05:18.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.460 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.515 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.303e-05 s, 0.0 kB/s 2023-10-06T03:05:18.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.516 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.571 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8749e-05 s, 0.0 kB/s 2023-10-06T03:05:18.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.572 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.627 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.628 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.017e-05 s, 0.0 kB/s 2023-10-06T03:05:18.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.629 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.683 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.683 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.683 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5858e-05 s, 0.0 kB/s 2023-10-06T03:05:18.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.684 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.739 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1831e-05 s, 0.0 kB/s 2023-10-06T03:05:18.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.740 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.794 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.795 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.795 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3988e-05 s, 0.0 kB/s 2023-10-06T03:05:18.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.795 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.851 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9241e-05 s, 0.0 kB/s 2023-10-06T03:05:18.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.852 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7339e-05 s, 0.0 kB/s 2023-10-06T03:05:18.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.908 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:18.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:18.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:18.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1058e-05 s, 0.0 kB/s 2023-10-06T03:05:18.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:18.964 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.019 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1601e-05 s, 0.0 kB/s 2023-10-06T03:05:19.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.020 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.075 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3383e-05 s, 0.0 kB/s 2023-10-06T03:05:19.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.076 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.131 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9891e-05 s, 0.0 kB/s 2023-10-06T03:05:19.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.131 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.186 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0006e-05 s, 0.0 kB/s 2023-10-06T03:05:19.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.187 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.243 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.243 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.243 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.209e-05 s, 0.0 kB/s 2023-10-06T03:05:19.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.244 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0076e-05 s, 0.0 kB/s 2023-10-06T03:05:19.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.300 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.355 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9304e-05 s, 0.0 kB/s 2023-10-06T03:05:19.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.355 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.411 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9834e-05 s, 0.0 kB/s 2023-10-06T03:05:19.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.411 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.466 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2038e-05 s, 0.0 kB/s 2023-10-06T03:05:19.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.466 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.522 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.522 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.522 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8937e-05 s, 0.0 kB/s 2023-10-06T03:05:19.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.523 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.578 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7269e-05 s, 0.0 kB/s 2023-10-06T03:05:19.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.578 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.632 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.633 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9305e-05 s, 0.0 kB/s 2023-10-06T03:05:19.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.633 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.688 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3224e-05 s, 0.0 kB/s 2023-10-06T03:05:19.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.689 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.745 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.745 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3089e-05 s, 0.0 kB/s 2023-10-06T03:05:19.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.745 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.800 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1381e-05 s, 0.0 kB/s 2023-10-06T03:05:19.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.801 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.856 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.857 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9322e-05 s, 0.0 kB/s 2023-10-06T03:05:19.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.857 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.912 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.913 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4108e-05 s, 0.0 kB/s 2023-10-06T03:05:19.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.913 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:19.968 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9474e-05 s, 0.0 kB/s 2023-10-06T03:05:19.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:19.968 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1741e-05 s, 0.0 kB/s 2023-10-06T03:05:20.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.026 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.081 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1144e-05 s, 0.0 kB/s 2023-10-06T03:05:20.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.136 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3577e-05 s, 0.0 kB/s 2023-10-06T03:05:20.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.137 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.193 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3195e-05 s, 0.0 kB/s 2023-10-06T03:05:20.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.194 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.248 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.249 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2952e-05 s, 0.0 kB/s 2023-10-06T03:05:20.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.249 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1776e-05 s, 0.0 kB/s 2023-10-06T03:05:20.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.305 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.359 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.360 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2359e-05 s, 0.0 kB/s 2023-10-06T03:05:20.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.360 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.414 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.415 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5621e-05 s, 0.0 kB/s 2023-10-06T03:05:20.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.415 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.469 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.470 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2195e-05 s, 0.0 kB/s 2023-10-06T03:05:20.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.470 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.531 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2346e-05 s, 0.0 kB/s 2023-10-06T03:05:20.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1684e-05 s, 0.0 kB/s 2023-10-06T03:05:20.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.643 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.643 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.643 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0021e-05 s, 0.0 kB/s 2023-10-06T03:05:20.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.644 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.699 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.699 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.699 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8506e-05 s, 0.0 kB/s 2023-10-06T03:05:20.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.700 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.755 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2751e-05 s, 0.0 kB/s 2023-10-06T03:05:20.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.756 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.811 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.811 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9446e-05 s, 0.0 kB/s 2023-10-06T03:05:20.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.811 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.866 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.866 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0314e-05 s, 0.0 kB/s 2023-10-06T03:05:20.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.866 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.921 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2223e-05 s, 0.0 kB/s 2023-10-06T03:05:20.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.921 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:20.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:20.977 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:20.977 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0627e-05 s, 0.0 kB/s 2023-10-06T03:05:20.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:20.977 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.031 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.031 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.031 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7793e-05 s, 0.0 kB/s 2023-10-06T03:05:21.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.032 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.087 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.087 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.087 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9372e-05 s, 0.0 kB/s 2023-10-06T03:05:21.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.087 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.142 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.143 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.143 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8157e-05 s, 0.0 kB/s 2023-10-06T03:05:21.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.144 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.199 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0632e-05 s, 0.0 kB/s 2023-10-06T03:05:21.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.200 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.255 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8461e-05 s, 0.0 kB/s 2023-10-06T03:05:21.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.256 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.311 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.311 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.311 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2414e-05 s, 0.0 kB/s 2023-10-06T03:05:21.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.368 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.027e-05 s, 0.0 kB/s 2023-10-06T03:05:21.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.423 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3686e-05 s, 0.0 kB/s 2023-10-06T03:05:21.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.423 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.478 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.478 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.478 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1192e-05 s, 0.0 kB/s 2023-10-06T03:05:21.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.479 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.534 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1339e-05 s, 0.0 kB/s 2023-10-06T03:05:21.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.588 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.589 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.107e-05 s, 0.0 kB/s 2023-10-06T03:05:21.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.589 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.645 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1166e-05 s, 0.0 kB/s 2023-10-06T03:05:21.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.645 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.700 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9514e-05 s, 0.0 kB/s 2023-10-06T03:05:21.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.756 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4802e-05 s, 0.0 kB/s 2023-10-06T03:05:21.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0158e-05 s, 0.0 kB/s 2023-10-06T03:05:21.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.867 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.867 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9109e-05 s, 0.0 kB/s 2023-10-06T03:05:21.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.923 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8052e-05 s, 0.0 kB/s 2023-10-06T03:05:21.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.924 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:21.978 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:21.979 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0628e-05 s, 0.0 kB/s 2023-10-06T03:05:21.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:21.979 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.034 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0251e-05 s, 0.0 kB/s 2023-10-06T03:05:22.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.034 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.088 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1312e-05 s, 0.0 kB/s 2023-10-06T03:05:22.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.089 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0565e-05 s, 0.0 kB/s 2023-10-06T03:05:22.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.199 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3528e-05 s, 0.0 kB/s 2023-10-06T03:05:22.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.200 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.256 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7934e-05 s, 0.0 kB/s 2023-10-06T03:05:22.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.256 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1214e-05 s, 0.0 kB/s 2023-10-06T03:05:22.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.368 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9014e-05 s, 0.0 kB/s 2023-10-06T03:05:22.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.423 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8046e-05 s, 0.0 kB/s 2023-10-06T03:05:22.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.424 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.479 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.479 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.479 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7458e-05 s, 0.0 kB/s 2023-10-06T03:05:22.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.480 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.535 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-06T03:05:22.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.536 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.591 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9428e-05 s, 0.0 kB/s 2023-10-06T03:05:22.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.593 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.647 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7983e-05 s, 0.0 kB/s 2023-10-06T03:05:22.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.647 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.703 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2179e-05 s, 0.0 kB/s 2023-10-06T03:05:22.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.704 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.758 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8794e-05 s, 0.0 kB/s 2023-10-06T03:05:22.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.759 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.815 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.816 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.816 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1373e-05 s, 0.0 kB/s 2023-10-06T03:05:22.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.817 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.871 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.871 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.872 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1019e-05 s, 0.0 kB/s 2023-10-06T03:05:22.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0773e-05 s, 0.0 kB/s 2023-10-06T03:05:22.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.929 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:22.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:22.984 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8817e-05 s, 0.0 kB/s 2023-10-06T03:05:22.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:22.985 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.041 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.854e-05 s, 0.0 kB/s 2023-10-06T03:05:23.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.041 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0851e-05 s, 0.0 kB/s 2023-10-06T03:05:23.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.096 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.151 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.151 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.151 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9366e-05 s, 0.0 kB/s 2023-10-06T03:05:23.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.152 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.209 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2802e-05 s, 0.0 kB/s 2023-10-06T03:05:23.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.264 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3675e-05 s, 0.0 kB/s 2023-10-06T03:05:23.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.264 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.319 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2753e-05 s, 0.0 kB/s 2023-10-06T03:05:23.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.319 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.374 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.375 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8821e-05 s, 0.0 kB/s 2023-10-06T03:05:23.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.375 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.429 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.429 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.429 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2172e-05 s, 0.0 kB/s 2023-10-06T03:05:23.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.430 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.486 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9761e-05 s, 0.0 kB/s 2023-10-06T03:05:23.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.486 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.542 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.542 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.542 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8079e-05 s, 0.0 kB/s 2023-10-06T03:05:23.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.542 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.599 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.447e-05 s, 0.0 kB/s 2023-10-06T03:05:23.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.600 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.655 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.656 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0895e-05 s, 0.0 kB/s 2023-10-06T03:05:23.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.656 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.711 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.711 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.711 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1845e-05 s, 0.0 kB/s 2023-10-06T03:05:23.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.712 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.767 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.081e-05 s, 0.0 kB/s 2023-10-06T03:05:23.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.767 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8185e-05 s, 0.0 kB/s 2023-10-06T03:05:23.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.823 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.878 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.043e-05 s, 0.0 kB/s 2023-10-06T03:05:23.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.879 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.934 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0292e-05 s, 0.0 kB/s 2023-10-06T03:05:23.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:23.990 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.781e-05 s, 0.0 kB/s 2023-10-06T03:05:23.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:23.991 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.048 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8126e-05 s, 0.0 kB/s 2023-10-06T03:05:24.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.103 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1058e-05 s, 0.0 kB/s 2023-10-06T03:05:24.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.104 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.158 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.159 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4854e-05 s, 0.0 kB/s 2023-10-06T03:05:24.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.214 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9857e-05 s, 0.0 kB/s 2023-10-06T03:05:24.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.214 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9174e-05 s, 0.0 kB/s 2023-10-06T03:05:24.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.270 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3532e-05 s, 0.0 kB/s 2023-10-06T03:05:24.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.381 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6678e-05 s, 0.0 kB/s 2023-10-06T03:05:24.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1865e-05 s, 0.0 kB/s 2023-10-06T03:05:24.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1202e-05 s, 0.0 kB/s 2023-10-06T03:05:24.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.549 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0928e-05 s, 0.0 kB/s 2023-10-06T03:05:24.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.549 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.604 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.605 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.605 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1647e-05 s, 0.0 kB/s 2023-10-06T03:05:24.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.605 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.660 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0946e-05 s, 0.0 kB/s 2023-10-06T03:05:24.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.660 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.715 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2373e-05 s, 0.0 kB/s 2023-10-06T03:05:24.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.715 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1732e-05 s, 0.0 kB/s 2023-10-06T03:05:24.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.771 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5546e-05 s, 0.0 kB/s 2023-10-06T03:05:24.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1211e-05 s, 0.0 kB/s 2023-10-06T03:05:24.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8803e-05 s, 0.0 kB/s 2023-10-06T03:05:24.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7539e-05 s, 0.0 kB/s 2023-10-06T03:05:24.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:24.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.050 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.051 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7182e-05 s, 0.0 kB/s 2023-10-06T03:05:25.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.051 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.106 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.106 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.106 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9619e-05 s, 0.0 kB/s 2023-10-06T03:05:25.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.107 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.161 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9712e-05 s, 0.0 kB/s 2023-10-06T03:05:25.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.162 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.218 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.218 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.218 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5147e-05 s, 0.0 kB/s 2023-10-06T03:05:25.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.219 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0691e-05 s, 0.0 kB/s 2023-10-06T03:05:25.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.274 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.330 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.5479e-05 s, 0.0 kB/s 2023-10-06T03:05:25.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.330 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.387 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9261e-05 s, 0.0 kB/s 2023-10-06T03:05:25.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.387 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.441 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9359e-05 s, 0.0 kB/s 2023-10-06T03:05:25.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.442 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.496 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.497 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3533e-05 s, 0.0 kB/s 2023-10-06T03:05:25.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.497 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.552 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7304e-05 s, 0.0 kB/s 2023-10-06T03:05:25.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.552 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.606 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.607 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1423e-05 s, 0.0 kB/s 2023-10-06T03:05:25.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.607 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.662 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1956e-05 s, 0.0 kB/s 2023-10-06T03:05:25.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.662 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.717 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7796e-05 s, 0.0 kB/s 2023-10-06T03:05:25.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.772 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7484e-05 s, 0.0 kB/s 2023-10-06T03:05:25.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9229e-05 s, 0.0 kB/s 2023-10-06T03:05:25.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.827 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0459e-05 s, 0.0 kB/s 2023-10-06T03:05:25.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.938 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1074e-05 s, 0.0 kB/s 2023-10-06T03:05:25.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.938 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1755e-05 s, 0.0 kB/s 2023-10-06T03:05:25.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:25.994 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.049 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7902e-05 s, 0.0 kB/s 2023-10-06T03:05:26.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9328e-05 s, 0.0 kB/s 2023-10-06T03:05:26.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.105 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.161 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8482e-05 s, 0.0 kB/s 2023-10-06T03:05:26.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.162 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.216 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7825e-05 s, 0.0 kB/s 2023-10-06T03:05:26.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.217 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.272 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8708e-05 s, 0.0 kB/s 2023-10-06T03:05:26.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.272 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9848e-05 s, 0.0 kB/s 2023-10-06T03:05:26.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.383 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.383 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.383 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.5729e-05 s, 0.0 kB/s 2023-10-06T03:05:26.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.383 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.438 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.439 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3739e-05 s, 0.0 kB/s 2023-10-06T03:05:26.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.439 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.493 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9464e-05 s, 0.0 kB/s 2023-10-06T03:05:26.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.494 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.549 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2791e-05 s, 0.0 kB/s 2023-10-06T03:05:26.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.550 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.605 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.605 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.605 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-06T03:05:26.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.606 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.661 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-06T03:05:26.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.661 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.717 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4318e-05 s, 0.0 kB/s 2023-10-06T03:05:26.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.773 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7437e-05 s, 0.0 kB/s 2023-10-06T03:05:26.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.828 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.828 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.239e-05 s, 0.0 kB/s 2023-10-06T03:05:26.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1198e-05 s, 0.0 kB/s 2023-10-06T03:05:26.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.938 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4495e-05 s, 0.0 kB/s 2023-10-06T03:05:26.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.938 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:26.993 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9452e-05 s, 0.0 kB/s 2023-10-06T03:05:26.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:26.993 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.049 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1329e-05 s, 0.0 kB/s 2023-10-06T03:05:27.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0778e-05 s, 0.0 kB/s 2023-10-06T03:05:27.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.105 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.160 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3051e-05 s, 0.0 kB/s 2023-10-06T03:05:27.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.160 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.215 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7962e-05 s, 0.0 kB/s 2023-10-06T03:05:27.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8824e-05 s, 0.0 kB/s 2023-10-06T03:05:27.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.270 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3513e-05 s, 0.0 kB/s 2023-10-06T03:05:27.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.381 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.381 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8094e-05 s, 0.0 kB/s 2023-10-06T03:05:27.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.381 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1193e-05 s, 0.0 kB/s 2023-10-06T03:05:27.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9124e-05 s, 0.0 kB/s 2023-10-06T03:05:27.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.548 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-10-06T03:05:27.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.549 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.604 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.604 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-06T03:05:27.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.604 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.660 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.660 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.660 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1716e-05 s, 0.0 kB/s 2023-10-06T03:05:27.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.661 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.716 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2717e-05 s, 0.0 kB/s 2023-10-06T03:05:27.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.716 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.772 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3666e-05 s, 0.0 kB/s 2023-10-06T03:05:27.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.826 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9887e-05 s, 0.0 kB/s 2023-10-06T03:05:27.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.827 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.881 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8199e-05 s, 0.0 kB/s 2023-10-06T03:05:27.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.882 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.938 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8956e-05 s, 0.0 kB/s 2023-10-06T03:05:27.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:27.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:27.994 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9522e-05 s, 0.0 kB/s 2023-10-06T03:05:27.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:27.994 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.049 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9565e-05 s, 0.0 kB/s 2023-10-06T03:05:28.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.050 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.105 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.105 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.2017e-05 s, 0.0 kB/s 2023-10-06T03:05:28.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.105 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.160 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9056e-05 s, 0.0 kB/s 2023-10-06T03:05:28.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.160 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.215 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9334e-05 s, 0.0 kB/s 2023-10-06T03:05:28.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.271 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1866e-05 s, 0.0 kB/s 2023-10-06T03:05:28.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.271 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.326 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3049e-05 s, 0.0 kB/s 2023-10-06T03:05:28.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.382 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8596e-05 s, 0.0 kB/s 2023-10-06T03:05:28.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.382 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1728e-05 s, 0.0 kB/s 2023-10-06T03:05:28.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.438 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.493 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.493 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2944e-05 s, 0.0 kB/s 2023-10-06T03:05:28.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.179e-05 s, 0.0 kB/s 2023-10-06T03:05:28.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.548 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.603 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0388e-05 s, 0.0 kB/s 2023-10-06T03:05:28.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0539e-05 s, 0.0 kB/s 2023-10-06T03:05:28.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.659 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1613e-05 s, 0.0 kB/s 2023-10-06T03:05:28.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.770 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.770 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.770 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8271e-05 s, 0.0 kB/s 2023-10-06T03:05:28.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.771 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.826 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.826 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.826 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0262e-05 s, 0.0 kB/s 2023-10-06T03:05:28.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.826 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.884 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0809e-05 s, 0.0 kB/s 2023-10-06T03:05:28.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.885 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8624e-05 s, 0.0 kB/s 2023-10-06T03:05:28.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8954e-05 s, 0.0 kB/s 2023-10-06T03:05:28.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:28.994 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7969e-05 s, 0.0 kB/s 2023-10-06T03:05:29.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0888e-05 s, 0.0 kB/s 2023-10-06T03:05:29.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.104 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.160 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7416e-05 s, 0.0 kB/s 2023-10-06T03:05:29.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.160 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.215 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1766e-05 s, 0.0 kB/s 2023-10-06T03:05:29.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.216 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.271 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.272 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2322e-05 s, 0.0 kB/s 2023-10-06T03:05:29.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.272 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.326 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.327 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2514e-05 s, 0.0 kB/s 2023-10-06T03:05:29.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.327 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.382 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.254e-05 s, 0.0 kB/s 2023-10-06T03:05:29.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.382 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0124e-05 s, 0.0 kB/s 2023-10-06T03:05:29.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.491 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.993e-05 s, 0.0 kB/s 2023-10-06T03:05:29.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.546 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0288e-05 s, 0.0 kB/s 2023-10-06T03:05:29.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.547 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.602 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0612e-05 s, 0.0 kB/s 2023-10-06T03:05:29.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.602 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.658 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1311e-05 s, 0.0 kB/s 2023-10-06T03:05:29.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8076e-05 s, 0.0 kB/s 2023-10-06T03:05:29.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.768 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.768 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1692e-05 s, 0.0 kB/s 2023-10-06T03:05:29.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.824 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3265e-05 s, 0.0 kB/s 2023-10-06T03:05:29.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.878 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.878 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4707e-05 s, 0.0 kB/s 2023-10-06T03:05:29.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.935 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9151e-05 s, 0.0 kB/s 2023-10-06T03:05:29.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.935 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:29.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:29.990 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:29.990 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3074e-05 s, 0.0 kB/s 2023-10-06T03:05:29.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:29.991 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.047 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.047 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1176e-05 s, 0.0 kB/s 2023-10-06T03:05:30.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.048 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.103 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.103 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0428e-05 s, 0.0 kB/s 2023-10-06T03:05:30.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.103 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.159 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.159 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.919e-05 s, 0.0 kB/s 2023-10-06T03:05:30.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.159 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.214 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.215 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.215 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0864e-05 s, 0.0 kB/s 2023-10-06T03:05:30.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.215 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.270 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.270 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9653e-05 s, 0.0 kB/s 2023-10-06T03:05:30.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.270 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.325 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.325 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.838e-05 s, 0.0 kB/s 2023-10-06T03:05:30.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.326 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.382 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.383 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6821e-05 s, 0.0 kB/s 2023-10-06T03:05:30.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.384 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.437 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.438 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.438 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1166e-05 s, 0.0 kB/s 2023-10-06T03:05:30.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.438 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.682e-05 s, 0.0 kB/s 2023-10-06T03:05:30.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.549 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9746e-05 s, 0.0 kB/s 2023-10-06T03:05:30.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.549 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.604 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6424e-05 s, 0.0 kB/s 2023-10-06T03:05:30.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.604 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.659 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0227e-05 s, 0.0 kB/s 2023-10-06T03:05:30.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.660 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1518e-05 s, 0.0 kB/s 2023-10-06T03:05:30.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.716 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9812e-05 s, 0.0 kB/s 2023-10-06T03:05:30.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1853e-05 s, 0.0 kB/s 2023-10-06T03:05:30.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.827 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.882 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8359e-05 s, 0.0 kB/s 2023-10-06T03:05:30.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8288e-05 s, 0.0 kB/s 2023-10-06T03:05:30.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:30.994 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.016e-05 s, 0.0 kB/s 2023-10-06T03:05:30.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:30.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.050 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.051 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8945e-05 s, 0.0 kB/s 2023-10-06T03:05:31.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.051 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.105 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.98e-05 s, 0.0 kB/s 2023-10-06T03:05:31.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.106 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9277e-05 s, 0.0 kB/s 2023-10-06T03:05:31.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.161 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.220 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-10-06T03:05:31.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.220 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.275 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9989e-05 s, 0.0 kB/s 2023-10-06T03:05:31.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.276 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.331 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8514e-05 s, 0.0 kB/s 2023-10-06T03:05:31.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.387 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7219e-05 s, 0.0 kB/s 2023-10-06T03:05:31.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.387 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.442 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0723e-05 s, 0.0 kB/s 2023-10-06T03:05:31.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.443 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.498 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1822e-05 s, 0.0 kB/s 2023-10-06T03:05:31.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.498 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.553 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.554 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0661e-05 s, 0.0 kB/s 2023-10-06T03:05:31.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.554 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.609 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2676e-05 s, 0.0 kB/s 2023-10-06T03:05:31.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.609 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.664 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.664 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.664 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2189e-05 s, 0.0 kB/s 2023-10-06T03:05:31.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.665 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.719 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1387e-05 s, 0.0 kB/s 2023-10-06T03:05:31.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.720 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.803 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.804 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2995e-05 s, 0.0 kB/s 2023-10-06T03:05:31.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.804 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2584e-05 s, 0.0 kB/s 2023-10-06T03:05:31.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.860 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.915 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1256e-05 s, 0.0 kB/s 2023-10-06T03:05:31.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:31.971 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1795e-05 s, 0.0 kB/s 2023-10-06T03:05:31.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:31.971 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.026 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.027 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.027 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3057e-05 s, 0.0 kB/s 2023-10-06T03:05:32.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.027 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.082 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1617e-05 s, 0.0 kB/s 2023-10-06T03:05:32.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.082 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9707e-05 s, 0.0 kB/s 2023-10-06T03:05:32.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.138 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.193 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2875e-05 s, 0.0 kB/s 2023-10-06T03:05:32.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.193 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.249 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1447e-05 s, 0.0 kB/s 2023-10-06T03:05:32.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.249 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.305 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.305 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.305 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1397e-05 s, 0.0 kB/s 2023-10-06T03:05:32.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.305 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.361 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9212e-05 s, 0.0 kB/s 2023-10-06T03:05:32.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.361 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.416 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.416 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.416 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-10-06T03:05:32.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.417 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.472 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0354e-05 s, 0.0 kB/s 2023-10-06T03:05:32.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.472 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.528 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.929e-05 s, 0.0 kB/s 2023-10-06T03:05:32.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.529 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.583 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.584 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.584 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8346e-05 s, 0.0 kB/s 2023-10-06T03:05:32.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.584 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.638 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.639 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.639 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9147e-05 s, 0.0 kB/s 2023-10-06T03:05:32.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.639 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.695 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.695 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.695 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1527e-05 s, 0.0 kB/s 2023-10-06T03:05:32.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.696 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.751 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.751 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.751 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9195e-05 s, 0.0 kB/s 2023-10-06T03:05:32.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.752 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.807 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1444e-05 s, 0.0 kB/s 2023-10-06T03:05:32.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.808 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.862 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.863 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0103e-05 s, 0.0 kB/s 2023-10-06T03:05:32.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.863 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.918 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0666e-05 s, 0.0 kB/s 2023-10-06T03:05:32.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.919 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:32.975 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.085e-05 s, 0.0 kB/s 2023-10-06T03:05:32.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:32.975 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.029 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.030 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.030 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8978e-05 s, 0.0 kB/s 2023-10-06T03:05:33.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.030 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.085 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.085 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.085 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0838e-05 s, 0.0 kB/s 2023-10-06T03:05:33.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.086 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.141 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9748e-05 s, 0.0 kB/s 2023-10-06T03:05:33.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.142 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.197 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1056e-05 s, 0.0 kB/s 2023-10-06T03:05:33.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.198 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.253 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9147e-05 s, 0.0 kB/s 2023-10-06T03:05:33.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.254 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.309 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-06T03:05:33.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.310 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.364 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.365 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.365 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1535e-05 s, 0.0 kB/s 2023-10-06T03:05:33.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.365 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.419 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.420 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.420 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8734e-05 s, 0.0 kB/s 2023-10-06T03:05:33.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.420 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.475 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.981e-05 s, 0.0 kB/s 2023-10-06T03:05:33.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.475 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.530 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1233e-05 s, 0.0 kB/s 2023-10-06T03:05:33.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.588 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.588 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.588 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.3706e-05 s, 0.0 kB/s 2023-10-06T03:05:33.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.588 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.644 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1623e-05 s, 0.0 kB/s 2023-10-06T03:05:33.644 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.644 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.700 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2903e-05 s, 0.0 kB/s 2023-10-06T03:05:33.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.756 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3956e-05 s, 0.0 kB/s 2023-10-06T03:05:33.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0601e-05 s, 0.0 kB/s 2023-10-06T03:05:33.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.812 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9809e-05 s, 0.0 kB/s 2023-10-06T03:05:33.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.869 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.924 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.77e-05 s, 0.0 kB/s 2023-10-06T03:05:33.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.925 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:33.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.159e-05 s, 0.0 kB/s 2023-10-06T03:05:33.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:33.980 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.035 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.035 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.035 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0842e-05 s, 0.0 kB/s 2023-10-06T03:05:34.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.036 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.091 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0063e-05 s, 0.0 kB/s 2023-10-06T03:05:34.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.092 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.293e-05 s, 0.0 kB/s 2023-10-06T03:05:34.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.147 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.202 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.203 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.203 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0953e-05 s, 0.0 kB/s 2023-10-06T03:05:34.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.203 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.258 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9328e-05 s, 0.0 kB/s 2023-10-06T03:05:34.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.259 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.049e-05 s, 0.0 kB/s 2023-10-06T03:05:34.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.314 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.369 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.369 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.369 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9418e-05 s, 0.0 kB/s 2023-10-06T03:05:34.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.369 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.425 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.425 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.425 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9125e-05 s, 0.0 kB/s 2023-10-06T03:05:34.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.425 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.480 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.481 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8231e-05 s, 0.0 kB/s 2023-10-06T03:05:34.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.481 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.536 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.536 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.536 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0783e-05 s, 0.0 kB/s 2023-10-06T03:05:34.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.536 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.592 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.592 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.592 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.237e-05 s, 0.0 kB/s 2023-10-06T03:05:34.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.592 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.124e-05 s, 0.0 kB/s 2023-10-06T03:05:34.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.649 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.704 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2363e-05 s, 0.0 kB/s 2023-10-06T03:05:34.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.705 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.761 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8555e-05 s, 0.0 kB/s 2023-10-06T03:05:34.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.816 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1851e-05 s, 0.0 kB/s 2023-10-06T03:05:34.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.816 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.872 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2966e-05 s, 0.0 kB/s 2023-10-06T03:05:34.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.928 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3175e-05 s, 0.0 kB/s 2023-10-06T03:05:34.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.929 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:34.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:34.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:34.984 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1624e-05 s, 0.0 kB/s 2023-10-06T03:05:34.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:34.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1644e-05 s, 0.0 kB/s 2023-10-06T03:05:35.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.041 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.096 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0231e-05 s, 0.0 kB/s 2023-10-06T03:05:35.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.096 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.152 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.153 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2126e-05 s, 0.0 kB/s 2023-10-06T03:05:35.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.153 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.208 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.208 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.208 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-10-06T03:05:35.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.208 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.263 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0546e-05 s, 0.0 kB/s 2023-10-06T03:05:35.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.264 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.1999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.320 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7969e-05 s, 0.0 kB/s 2023-10-06T03:05:35.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.321 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.376 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8749e-05 s, 0.0 kB/s 2023-10-06T03:05:35.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.376 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.431 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0881e-05 s, 0.0 kB/s 2023-10-06T03:05:35.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.588 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.610 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1627e-05 s, 0.0 kB/s 2023-10-06T03:05:35.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.610 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.658 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4444e-05 s, 0.0 kB/s 2023-10-06T03:05:35.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.792 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7557e-05 s, 0.0 kB/s 2023-10-06T03:05:35.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.813 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3139e-05 s, 0.0 kB/s 2023-10-06T03:05:35.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0646e-05 s, 0.0 kB/s 2023-10-06T03:05:35.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.924 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9983e-05 s, 0.0 kB/s 2023-10-06T03:05:35.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.924 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:35.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:35.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:35.980 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1979e-05 s, 0.0 kB/s 2023-10-06T03:05:35.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:35.980 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.034 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.036 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.036 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3334e-05 s, 0.0 kB/s 2023-10-06T03:05:36.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.036 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.089 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.089 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.089 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0445e-05 s, 0.0 kB/s 2023-10-06T03:05:36.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.090 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.145 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5038e-05 s, 0.0 kB/s 2023-10-06T03:05:36.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.200 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.894e-05 s, 0.0 kB/s 2023-10-06T03:05:36.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.201 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.256 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4167e-05 s, 0.0 kB/s 2023-10-06T03:05:36.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0569e-05 s, 0.0 kB/s 2023-10-06T03:05:36.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.367 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.367 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7375e-05 s, 0.0 kB/s 2023-10-06T03:05:36.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.423 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0245e-05 s, 0.0 kB/s 2023-10-06T03:05:36.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.423 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.478 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.478 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.479 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8735e-05 s, 0.0 kB/s 2023-10-06T03:05:36.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.479 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.534 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.534 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.534 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7865e-05 s, 0.0 kB/s 2023-10-06T03:05:36.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.590 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.590 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5085e-05 s, 0.0 kB/s 2023-10-06T03:05:36.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.590 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.645 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8503e-05 s, 0.0 kB/s 2023-10-06T03:05:36.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.646 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.701 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.701 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.701 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0746e-05 s, 0.0 kB/s 2023-10-06T03:05:36.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.702 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.757 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1301e-05 s, 0.0 kB/s 2023-10-06T03:05:36.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.758 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.813 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.813 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.813 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8851e-05 s, 0.0 kB/s 2023-10-06T03:05:36.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.869 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.869 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.869 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5516e-05 s, 0.0 kB/s 2023-10-06T03:05:36.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.869 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.925 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9478e-05 s, 0.0 kB/s 2023-10-06T03:05:36.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.926 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:36.981 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3261e-05 s, 0.0 kB/s 2023-10-06T03:05:36.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:36.982 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.037 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9356e-05 s, 0.0 kB/s 2023-10-06T03:05:37.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.037 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.093 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.093 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.093 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1804e-05 s, 0.0 kB/s 2023-10-06T03:05:37.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.149 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.149 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0895e-05 s, 0.0 kB/s 2023-10-06T03:05:37.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.150 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.205 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3591e-05 s, 0.0 kB/s 2023-10-06T03:05:37.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.206 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.262 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0278e-05 s, 0.0 kB/s 2023-10-06T03:05:37.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.262 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.319 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2403e-05 s, 0.0 kB/s 2023-10-06T03:05:37.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.320 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.376 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1676e-05 s, 0.0 kB/s 2023-10-06T03:05:37.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.376 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.437 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3568e-05 s, 0.0 kB/s 2023-10-06T03:05:37.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3333e-05 s, 0.0 kB/s 2023-10-06T03:05:37.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.493 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.548 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2834e-05 s, 0.0 kB/s 2023-10-06T03:05:37.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.548 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.604 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9983e-05 s, 0.0 kB/s 2023-10-06T03:05:37.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.719 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.735 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1714e-05 s, 0.0 kB/s 2023-10-06T03:05:37.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.791 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.791 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9819e-05 s, 0.0 kB/s 2023-10-06T03:05:37.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.791 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.847 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0009e-05 s, 0.0 kB/s 2023-10-06T03:05:37.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.903 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.885e-05 s, 0.0 kB/s 2023-10-06T03:05:37.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.903 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3029e-05 s, 0.0 kB/s 2023-10-06T03:05:37.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:37.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.002e-05 s, 0.0 kB/s 2023-10-06T03:05:38.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.914e-05 s, 0.0 kB/s 2023-10-06T03:05:38.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.353 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1818e-05 s, 0.0 kB/s 2023-10-06T03:05:38.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.370 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.370 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.370 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-06T03:05:38.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.371 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.426 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-10-06T03:05:38.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.426 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.481 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.482 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0933e-05 s, 0.0 kB/s 2023-10-06T03:05:38.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.482 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.537 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.537 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.537 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7466e-05 s, 0.0 kB/s 2023-10-06T03:05:38.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.537 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.594 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3774e-05 s, 0.0 kB/s 2023-10-06T03:05:38.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.594 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.650 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4584e-05 s, 0.0 kB/s 2023-10-06T03:05:38.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.651 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.706 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2438e-05 s, 0.0 kB/s 2023-10-06T03:05:38.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.707 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.761 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1798e-05 s, 0.0 kB/s 2023-10-06T03:05:38.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.818 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.818 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.109e-05 s, 0.0 kB/s 2023-10-06T03:05:38.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.872 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7201e-05 s, 0.0 kB/s 2023-10-06T03:05:38.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.928 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.9025e-05 s, 0.0 kB/s 2023-10-06T03:05:38.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:38.982 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:38.983 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6095e-05 s, 0.0 kB/s 2023-10-06T03:05:38.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:38.983 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.037 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.038 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8633e-05 s, 0.0 kB/s 2023-10-06T03:05:39.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.038 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.092 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.871e-05 s, 0.0 kB/s 2023-10-06T03:05:39.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.147 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9714e-05 s, 0.0 kB/s 2023-10-06T03:05:39.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.148 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.203 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.204 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8685e-05 s, 0.0 kB/s 2023-10-06T03:05:39.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.204 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.259 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8948e-05 s, 0.0 kB/s 2023-10-06T03:05:39.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.259 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.315 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.315 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.315 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0573e-05 s, 0.0 kB/s 2023-10-06T03:05:39.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.316 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.371 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3184e-05 s, 0.0 kB/s 2023-10-06T03:05:39.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.372 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.427 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.451 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8126e-05 s, 0.0 kB/s 2023-10-06T03:05:39.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.451 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.482 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.483 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8514e-05 s, 0.0 kB/s 2023-10-06T03:05:39.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.483 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.537 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.538 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.538 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8043e-05 s, 0.0 kB/s 2023-10-06T03:05:39.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.538 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.593 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.601 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.838e-05 s, 0.0 kB/s 2023-10-06T03:05:39.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.648 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.656 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.656 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8316e-05 s, 0.0 kB/s 2023-10-06T03:05:39.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.656 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.803e-05 s, 0.0 kB/s 2023-10-06T03:05:39.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.705 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.760 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.760 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.760 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0536e-05 s, 0.0 kB/s 2023-10-06T03:05:39.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.761 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.819 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.819 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3271e-05 s, 0.0 kB/s 2023-10-06T03:05:39.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.819 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.872 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.873 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.437e-05 s, 0.0 kB/s 2023-10-06T03:05:39.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.873 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.928 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8718e-05 s, 0.0 kB/s 2023-10-06T03:05:39.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6051e-05 s, 0.0 kB/s 2023-10-06T03:05:39.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:39.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.040 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.040 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1762e-05 s, 0.0 kB/s 2023-10-06T03:05:40.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.040 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8792e-05 s, 0.0 kB/s 2023-10-06T03:05:40.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.095 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.150 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0053e-05 s, 0.0 kB/s 2023-10-06T03:05:40.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.150 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.205 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.227 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.903e-05 s, 0.0 kB/s 2023-10-06T03:05:40.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.227 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.385 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8335e-05 s, 0.0 kB/s 2023-10-06T03:05:40.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.442 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.442 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.443 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2248e-05 s, 0.0 kB/s 2023-10-06T03:05:40.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.443 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.498 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1878e-05 s, 0.0 kB/s 2023-10-06T03:05:40.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.498 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.554 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.554 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.554 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-06T03:05:40.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.554 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.610 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0701e-05 s, 0.0 kB/s 2023-10-06T03:05:40.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.610 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.666 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2923e-05 s, 0.0 kB/s 2023-10-06T03:05:40.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.667 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.722 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.722 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.722 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0982e-05 s, 0.0 kB/s 2023-10-06T03:05:40.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.723 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.778 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0986e-05 s, 0.0 kB/s 2023-10-06T03:05:40.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.779 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.836 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.836 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.836 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8083e-05 s, 0.0 kB/s 2023-10-06T03:05:40.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.837 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.891 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.892 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.892 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9303e-05 s, 0.0 kB/s 2023-10-06T03:05:40.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.892 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:40.946 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:40.946 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:40.947 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.052e-05 s, 0.0 kB/s 2023-10-06T03:05:40.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:40.947 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.002 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.002 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.002 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1078e-05 s, 0.0 kB/s 2023-10-06T03:05:41.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.002 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.058 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.059 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0659e-05 s, 0.0 kB/s 2023-10-06T03:05:41.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.059 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.113 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.114 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.114 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1207e-05 s, 0.0 kB/s 2023-10-06T03:05:41.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.114 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.168 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.169 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.094e-05 s, 0.0 kB/s 2023-10-06T03:05:41.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.169 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.223 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.224 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7931e-05 s, 0.0 kB/s 2023-10-06T03:05:41.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.224 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.278 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7581e-05 s, 0.0 kB/s 2023-10-06T03:05:41.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.279 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.334 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.334 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.334 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9305e-05 s, 0.0 kB/s 2023-10-06T03:05:41.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.334 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.389 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.390 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2872e-05 s, 0.0 kB/s 2023-10-06T03:05:41.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.390 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.445 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.446 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9813e-05 s, 0.0 kB/s 2023-10-06T03:05:41.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.446 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.501 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.501 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.502 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.1417e-05 s, 0.0 kB/s 2023-10-06T03:05:41.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.502 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.557 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.094e-05 s, 0.0 kB/s 2023-10-06T03:05:41.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.557 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.612 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.613 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.613 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7965e-05 s, 0.0 kB/s 2023-10-06T03:05:41.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.668 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.668 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.668 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9417e-05 s, 0.0 kB/s 2023-10-06T03:05:41.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.669 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.724 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9796e-05 s, 0.0 kB/s 2023-10-06T03:05:41.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.724 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.780 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.780 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.780 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1122e-05 s, 0.0 kB/s 2023-10-06T03:05:41.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.780 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.836 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.836 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.836 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9837e-05 s, 0.0 kB/s 2023-10-06T03:05:41.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.837 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.892 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0778e-05 s, 0.0 kB/s 2023-10-06T03:05:41.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.893 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:41.948 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:41.948 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:41.948 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3e-05 s, 0.0 kB/s 2023-10-06T03:05:41.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:41.948 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.004 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2383e-05 s, 0.0 kB/s 2023-10-06T03:05:42.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.005 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.062 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.062 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.062 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9558e-05 s, 0.0 kB/s 2023-10-06T03:05:42.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.063 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.313e-05 s, 0.0 kB/s 2023-10-06T03:05:42.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.118 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.173 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2539e-05 s, 0.0 kB/s 2023-10-06T03:05:42.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.228 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2615e-05 s, 0.0 kB/s 2023-10-06T03:05:42.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.228 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:05:42.270 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:05:42.282 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.282 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.282 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9947e-05 s, 0.0 kB/s 2023-10-06T03:05:42.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.283 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.339 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2816e-05 s, 0.0 kB/s 2023-10-06T03:05:42.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.340 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.396 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.167e-05 s, 0.0 kB/s 2023-10-06T03:05:42.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.397 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3716e-05 s, 0.0 kB/s 2023-10-06T03:05:42.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.453 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.508 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.508 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8853e-05 s, 0.0 kB/s 2023-10-06T03:05:42.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.508 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.563 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0426e-05 s, 0.0 kB/s 2023-10-06T03:05:42.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.563 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.619 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0075e-05 s, 0.0 kB/s 2023-10-06T03:05:42.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.619 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.675 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0476e-05 s, 0.0 kB/s 2023-10-06T03:05:42.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.675 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.731 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.731 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.731 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2369e-05 s, 0.0 kB/s 2023-10-06T03:05:42.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.732 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.787 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0891e-05 s, 0.0 kB/s 2023-10-06T03:05:42.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.787 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.843 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9417e-05 s, 0.0 kB/s 2023-10-06T03:05:42.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.843 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.898 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.899 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8204e-05 s, 0.0 kB/s 2023-10-06T03:05:42.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.899 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:42.953 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:42.953 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:42.953 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.073e-05 s, 0.0 kB/s 2023-10-06T03:05:42.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:42.954 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.010 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.010 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7464e-05 s, 0.0 kB/s 2023-10-06T03:05:43.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.010 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.068 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9882e-05 s, 0.0 kB/s 2023-10-06T03:05:43.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.068 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.122 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9285e-05 s, 0.0 kB/s 2023-10-06T03:05:43.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.177 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8108e-05 s, 0.0 kB/s 2023-10-06T03:05:43.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.178 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.233 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.861e-05 s, 0.0 kB/s 2023-10-06T03:05:43.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.233 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.289 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8727e-05 s, 0.0 kB/s 2023-10-06T03:05:43.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.345 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1221e-05 s, 0.0 kB/s 2023-10-06T03:05:43.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.345 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9732e-05 s, 0.0 kB/s 2023-10-06T03:05:43.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.401 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.456 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7866e-05 s, 0.0 kB/s 2023-10-06T03:05:43.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.457 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7826e-05 s, 0.0 kB/s 2023-10-06T03:05:43.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.512 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.567 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.567 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1371e-05 s, 0.0 kB/s 2023-10-06T03:05:43.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.622 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9699e-05 s, 0.0 kB/s 2023-10-06T03:05:43.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.622 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.678 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8654e-05 s, 0.0 kB/s 2023-10-06T03:05:43.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.734 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8595e-05 s, 0.0 kB/s 2023-10-06T03:05:43.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.734 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.790 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0473e-05 s, 0.0 kB/s 2023-10-06T03:05:43.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.791 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.846 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4439e-05 s, 0.0 kB/s 2023-10-06T03:05:43.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.902 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.302e-05 s, 0.0 kB/s 2023-10-06T03:05:43.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.903 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:43.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:43.958 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:43.958 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7461e-05 s, 0.0 kB/s 2023-10-06T03:05:43.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:43.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.014 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2099e-05 s, 0.0 kB/s 2023-10-06T03:05:44.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1916e-05 s, 0.0 kB/s 2023-10-06T03:05:44.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.070 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0126e-05 s, 0.0 kB/s 2023-10-06T03:05:44.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.126 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.181 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3461e-05 s, 0.0 kB/s 2023-10-06T03:05:44.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.181 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.237 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3236e-05 s, 0.0 kB/s 2023-10-06T03:05:44.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.237 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.293 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.293 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.293 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7967e-05 s, 0.0 kB/s 2023-10-06T03:05:44.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.349 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0725e-05 s, 0.0 kB/s 2023-10-06T03:05:44.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.349 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.403 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.158e-05 s, 0.0 kB/s 2023-10-06T03:05:44.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.404 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-10-06T03:05:44.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.460 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.516 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9844e-05 s, 0.0 kB/s 2023-10-06T03:05:44.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.516 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.572 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.973e-05 s, 0.0 kB/s 2023-10-06T03:05:44.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.572 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.628 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9665e-05 s, 0.0 kB/s 2023-10-06T03:05:44.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.629 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.683 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.684 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.684 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3495e-05 s, 0.0 kB/s 2023-10-06T03:05:44.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.684 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.740 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.740 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.359e-05 s, 0.0 kB/s 2023-10-06T03:05:44.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.740 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.796 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1198e-05 s, 0.0 kB/s 2023-10-06T03:05:44.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.797 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.852 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.852 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.852 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9132e-05 s, 0.0 kB/s 2023-10-06T03:05:44.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.852 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.908 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.908 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.908 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8562e-05 s, 0.0 kB/s 2023-10-06T03:05:44.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.909 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:44.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:44.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:44.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0989e-05 s, 0.0 kB/s 2023-10-06T03:05:44.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:44.964 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.020 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0429e-05 s, 0.0 kB/s 2023-10-06T03:05:45.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.020 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1783e-05 s, 0.0 kB/s 2023-10-06T03:05:45.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.075 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.129 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.974e-05 s, 0.0 kB/s 2023-10-06T03:05:45.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.130 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.185 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7426e-05 s, 0.0 kB/s 2023-10-06T03:05:45.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.185 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.240 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.772e-05 s, 0.0 kB/s 2023-10-06T03:05:45.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.296 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0375e-05 s, 0.0 kB/s 2023-10-06T03:05:45.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.296 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.352 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1196e-05 s, 0.0 kB/s 2023-10-06T03:05:45.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.0536e-05 s, 0.0 kB/s 2023-10-06T03:05:45.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.464 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.464 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.464 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8761e-05 s, 0.0 kB/s 2023-10-06T03:05:45.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.464 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.519 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0054e-05 s, 0.0 kB/s 2023-10-06T03:05:45.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.520 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.576 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.576 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.576 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0768e-05 s, 0.0 kB/s 2023-10-06T03:05:45.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.576 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.631 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.632 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.632 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0443e-05 s, 0.0 kB/s 2023-10-06T03:05:45.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.686 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.686 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.687 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2817e-05 s, 0.0 kB/s 2023-10-06T03:05:45.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.688 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2304e-05 s, 0.0 kB/s 2023-10-06T03:05:45.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.743 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.797 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.798 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6291e-05 s, 0.0 kB/s 2023-10-06T03:05:45.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.798 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.853 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.853 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.853 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9768e-05 s, 0.0 kB/s 2023-10-06T03:05:45.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.853 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.908 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.909 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9666e-05 s, 0.0 kB/s 2023-10-06T03:05:45.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.909 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:45.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1591e-05 s, 0.0 kB/s 2023-10-06T03:05:45.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:45.964 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.020 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.020 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9499e-05 s, 0.0 kB/s 2023-10-06T03:05:46.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.020 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.075 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7075e-05 s, 0.0 kB/s 2023-10-06T03:05:46.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.075 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.130 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.131 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2362e-05 s, 0.0 kB/s 2023-10-06T03:05:46.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.131 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.187 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9394e-05 s, 0.0 kB/s 2023-10-06T03:05:46.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.187 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.242 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2772e-05 s, 0.0 kB/s 2023-10-06T03:05:46.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.243 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.299 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0187e-05 s, 0.0 kB/s 2023-10-06T03:05:46.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.299 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.354 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.355 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.355 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8697e-05 s, 0.0 kB/s 2023-10-06T03:05:46.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.355 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.409 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.410 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0592e-05 s, 0.0 kB/s 2023-10-06T03:05:46.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.410 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.465 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.465 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.465 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1868e-05 s, 0.0 kB/s 2023-10-06T03:05:46.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.465 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.520 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.520 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.520 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6463e-05 s, 0.0 kB/s 2023-10-06T03:05:46.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.521 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.577 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0968e-05 s, 0.0 kB/s 2023-10-06T03:05:46.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.577 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.633 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.698 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3083e-05 s, 0.0 kB/s 2023-10-06T03:05:46.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.699 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.715 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3723e-05 s, 0.0 kB/s 2023-10-06T03:05:46.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.715 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.064e-05 s, 0.0 kB/s 2023-10-06T03:05:46.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.771 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9413e-05 s, 0.0 kB/s 2023-10-06T03:05:46.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.087e-05 s, 0.0 kB/s 2023-10-06T03:05:46.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0319e-05 s, 0.0 kB/s 2023-10-06T03:05:46.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:46.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8113e-05 s, 0.0 kB/s 2023-10-06T03:05:46.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:46.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.050 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.050 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2488e-05 s, 0.0 kB/s 2023-10-06T03:05:47.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.050 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.105 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.106 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2419e-05 s, 0.0 kB/s 2023-10-06T03:05:47.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.106 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.161 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.941e-05 s, 0.0 kB/s 2023-10-06T03:05:47.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.161 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.216 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.217 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2555e-05 s, 0.0 kB/s 2023-10-06T03:05:47.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.217 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.272 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.272 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.272 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0527e-05 s, 0.0 kB/s 2023-10-06T03:05:47.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.272 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3162e-05 s, 0.0 kB/s 2023-10-06T03:05:47.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.329 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.384 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9779e-05 s, 0.0 kB/s 2023-10-06T03:05:47.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.384 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.440 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.440 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0751e-05 s, 0.0 kB/s 2023-10-06T03:05:47.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.440 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.496 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0004e-05 s, 0.0 kB/s 2023-10-06T03:05:47.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.496 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.214e-05 s, 0.0 kB/s 2023-10-06T03:05:47.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.552 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.607 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1741e-05 s, 0.0 kB/s 2023-10-06T03:05:47.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.607 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.667 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.668 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4862e-05 s, 0.0 kB/s 2023-10-06T03:05:47.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.668 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.723 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.723 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.723 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3477e-05 s, 0.0 kB/s 2023-10-06T03:05:47.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.724 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2945e-05 s, 0.0 kB/s 2023-10-06T03:05:47.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.780 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.835 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8655e-05 s, 0.0 kB/s 2023-10-06T03:05:47.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.835 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.685e-05 s, 0.0 kB/s 2023-10-06T03:05:47.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.892 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:47.947 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:47.947 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:47.947 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.19e-05 s, 0.0 kB/s 2023-10-06T03:05:47.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:47.948 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.003 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.008e-05 s, 0.0 kB/s 2023-10-06T03:05:48.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.004 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.059 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9473e-05 s, 0.0 kB/s 2023-10-06T03:05:48.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.059 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.114 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.115 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8818e-05 s, 0.0 kB/s 2023-10-06T03:05:48.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.115 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.170 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.170 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.170 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2131e-05 s, 0.0 kB/s 2023-10-06T03:05:48.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.170 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7893e-05 s, 0.0 kB/s 2023-10-06T03:05:48.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.226 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.282 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.282 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.282 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.003e-05 s, 0.0 kB/s 2023-10-06T03:05:48.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.283 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.338 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0871e-05 s, 0.0 kB/s 2023-10-06T03:05:48.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.339 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.393 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.394 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.394 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1542e-05 s, 0.0 kB/s 2023-10-06T03:05:48.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.394 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.448 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8894e-05 s, 0.0 kB/s 2023-10-06T03:05:48.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.449 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.505 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.505 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.505 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9703e-05 s, 0.0 kB/s 2023-10-06T03:05:48.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.506 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.560 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.561 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.561 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7301e-05 s, 0.0 kB/s 2023-10-06T03:05:48.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.561 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.616 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.617 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.617 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2851e-05 s, 0.0 kB/s 2023-10-06T03:05:48.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.617 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.672 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0393e-05 s, 0.0 kB/s 2023-10-06T03:05:48.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.673 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.728 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.728 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.728 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2241e-05 s, 0.0 kB/s 2023-10-06T03:05:48.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.729 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.784 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.945e-05 s, 0.0 kB/s 2023-10-06T03:05:48.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.784 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.839 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.840 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.840 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4757e-05 s, 0.0 kB/s 2023-10-06T03:05:48.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.840 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.895 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.895 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.895 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1561e-05 s, 0.0 kB/s 2023-10-06T03:05:48.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.895 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:48.951 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:48.951 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:48.951 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.234e-05 s, 0.0 kB/s 2023-10-06T03:05:48.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:48.951 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.007 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.107e-05 s, 0.0 kB/s 2023-10-06T03:05:49.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.008 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.063 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.064 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9771e-05 s, 0.0 kB/s 2023-10-06T03:05:49.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.064 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.119 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.119 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.119 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0632e-05 s, 0.0 kB/s 2023-10-06T03:05:49.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.119 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.175 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.175 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.175 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7882e-05 s, 0.0 kB/s 2023-10-06T03:05:49.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.176 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.231 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2052e-05 s, 0.0 kB/s 2023-10-06T03:05:49.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.232 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.288 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4077e-05 s, 0.0 kB/s 2023-10-06T03:05:49.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.343 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1277e-05 s, 0.0 kB/s 2023-10-06T03:05:49.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1194e-05 s, 0.0 kB/s 2023-10-06T03:05:49.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.399 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.454 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.455 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9654e-05 s, 0.0 kB/s 2023-10-06T03:05:49.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.510 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7436e-05 s, 0.0 kB/s 2023-10-06T03:05:49.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.510 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8436e-05 s, 0.0 kB/s 2023-10-06T03:05:49.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.622 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1416e-05 s, 0.0 kB/s 2023-10-06T03:05:49.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.622 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.679 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1395e-05 s, 0.0 kB/s 2023-10-06T03:05:49.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.734 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-10-06T03:05:49.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.789 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.790 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0154e-05 s, 0.0 kB/s 2023-10-06T03:05:49.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.790 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.845 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9912e-05 s, 0.0 kB/s 2023-10-06T03:05:49.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.846 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.901 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7627e-05 s, 0.0 kB/s 2023-10-06T03:05:49.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:49.957 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9216e-05 s, 0.0 kB/s 2023-10-06T03:05:49.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:49.957 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1506e-05 s, 0.0 kB/s 2023-10-06T03:05:50.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8828e-05 s, 0.0 kB/s 2023-10-06T03:05:50.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2581e-05 s, 0.0 kB/s 2023-10-06T03:05:50.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.179 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0264e-05 s, 0.0 kB/s 2023-10-06T03:05:50.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.180 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.236 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0552e-05 s, 0.0 kB/s 2023-10-06T03:05:50.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.237 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.292 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-10-06T03:05:50.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.293 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.348 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9799e-05 s, 0.0 kB/s 2023-10-06T03:05:50.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.348 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.404 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.404 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.404 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-10-06T03:05:50.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.404 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.460 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0603e-05 s, 0.0 kB/s 2023-10-06T03:05:50.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.461 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.517 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.517 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-06T03:05:50.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.517 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.572 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.835e-05 s, 0.0 kB/s 2023-10-06T03:05:50.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.572 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.627 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.628 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7841e-05 s, 0.0 kB/s 2023-10-06T03:05:50.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.628 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.682 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.683 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1331e-05 s, 0.0 kB/s 2023-10-06T03:05:50.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.684 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.739 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0745e-05 s, 0.0 kB/s 2023-10-06T03:05:50.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.740 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.794 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.795 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0965e-05 s, 0.0 kB/s 2023-10-06T03:05:50.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.851 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.244e-05 s, 0.0 kB/s 2023-10-06T03:05:50.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.852 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0845e-05 s, 0.0 kB/s 2023-10-06T03:05:50.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.907 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:50.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2295e-05 s, 0.0 kB/s 2023-10-06T03:05:50.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:50.964 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.020 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0776e-05 s, 0.0 kB/s 2023-10-06T03:05:51.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.020 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.075 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0741e-05 s, 0.0 kB/s 2023-10-06T03:05:51.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.076 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.131 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1597e-05 s, 0.0 kB/s 2023-10-06T03:05:51.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.131 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.186 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.187 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.187 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3064e-05 s, 0.0 kB/s 2023-10-06T03:05:51.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.187 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.241 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.242 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.083e-05 s, 0.0 kB/s 2023-10-06T03:05:51.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.242 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.297 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.297 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0091e-05 s, 0.0 kB/s 2023-10-06T03:05:51.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.352 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2137e-05 s, 0.0 kB/s 2023-10-06T03:05:51.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8917e-05 s, 0.0 kB/s 2023-10-06T03:05:51.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.465 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.465 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7135e-05 s, 0.0 kB/s 2023-10-06T03:05:51.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.465 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.518 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.519 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3177e-05 s, 0.0 kB/s 2023-10-06T03:05:51.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.520 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.574 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.575 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.878e-05 s, 0.0 kB/s 2023-10-06T03:05:51.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.575 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0987e-05 s, 0.0 kB/s 2023-10-06T03:05:51.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.630 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.685 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1705e-05 s, 0.0 kB/s 2023-10-06T03:05:51.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.686 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.741 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4132e-05 s, 0.0 kB/s 2023-10-06T03:05:51.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.742 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.796 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.797 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.797 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2409e-05 s, 0.0 kB/s 2023-10-06T03:05:51.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.797 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.851 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3046e-05 s, 0.0 kB/s 2023-10-06T03:05:51.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.852 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9397e-05 s, 0.0 kB/s 2023-10-06T03:05:51.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.907 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3292e-05 s, 0.0 kB/s 2023-10-06T03:05:51.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:51.963 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.017 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9128e-05 s, 0.0 kB/s 2023-10-06T03:05:52.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.019 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.074 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4237e-05 s, 0.0 kB/s 2023-10-06T03:05:52.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.075 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.130 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.130 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.130 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.133e-05 s, 0.0 kB/s 2023-10-06T03:05:52.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.130 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.185 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3482e-05 s, 0.0 kB/s 2023-10-06T03:05:52.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.186 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.240 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0323e-05 s, 0.0 kB/s 2023-10-06T03:05:52.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.241 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.296 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2046e-05 s, 0.0 kB/s 2023-10-06T03:05:52.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.352 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2398e-05 s, 0.0 kB/s 2023-10-06T03:05:52.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3073e-05 s, 0.0 kB/s 2023-10-06T03:05:52.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.408 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.464 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1252e-05 s, 0.0 kB/s 2023-10-06T03:05:52.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.465 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.521 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.521 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9844e-05 s, 0.0 kB/s 2023-10-06T03:05:52.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.521 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.576 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.577 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3718e-05 s, 0.0 kB/s 2023-10-06T03:05:52.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.577 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.632 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.632 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.632 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8354e-05 s, 0.0 kB/s 2023-10-06T03:05:52.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.632 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2909e-05 s, 0.0 kB/s 2023-10-06T03:05:52.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.690 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.744 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0105e-05 s, 0.0 kB/s 2023-10-06T03:05:52.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.745 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.799 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.799 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1861e-05 s, 0.0 kB/s 2023-10-06T03:05:52.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.800 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.855 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.856 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.856 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.879e-05 s, 0.0 kB/s 2023-10-06T03:05:52.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.856 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.911 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.911 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.911 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8318e-05 s, 0.0 kB/s 2023-10-06T03:05:52.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.911 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:52.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:52.966 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:52.966 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2888e-05 s, 0.0 kB/s 2023-10-06T03:05:52.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:52.967 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.023 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.023 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-06T03:05:53.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.023 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.079 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.079 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.079 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0184e-05 s, 0.0 kB/s 2023-10-06T03:05:53.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.079 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.135 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9547e-05 s, 0.0 kB/s 2023-10-06T03:05:53.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.136 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.191 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.191 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.191 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6871e-05 s, 0.0 kB/s 2023-10-06T03:05:53.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.192 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.247 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.247 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.247 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8688e-05 s, 0.0 kB/s 2023-10-06T03:05:53.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.247 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.303 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.303 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.303 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9117e-05 s, 0.0 kB/s 2023-10-06T03:05:53.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.303 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.359 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.360 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3347e-05 s, 0.0 kB/s 2023-10-06T03:05:53.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.360 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.415 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.415 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.415 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.153e-05 s, 0.0 kB/s 2023-10-06T03:05:53.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.416 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.474 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.474 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.474 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1455e-05 s, 0.0 kB/s 2023-10-06T03:05:53.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.475 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.530 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.531 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0057e-05 s, 0.0 kB/s 2023-10-06T03:05:53.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.531 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.586 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.586 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.0532e-05 s, 0.0 kB/s 2023-10-06T03:05:53.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.643 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0326e-05 s, 0.0 kB/s 2023-10-06T03:05:53.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.643 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.698 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.004e-05 s, 0.0 kB/s 2023-10-06T03:05:53.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.699 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.756 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1446e-05 s, 0.0 kB/s 2023-10-06T03:05:53.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.756 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.841 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.842 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.842 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2731e-05 s, 0.0 kB/s 2023-10-06T03:05:53.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.842 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.858 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.859 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0582e-05 s, 0.0 kB/s 2023-10-06T03:05:53.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.859 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.914 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1749e-05 s, 0.0 kB/s 2023-10-06T03:05:53.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.915 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:53.970 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:53.970 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:53.970 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1951e-05 s, 0.0 kB/s 2023-10-06T03:05:53.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:53.971 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.026 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9425e-05 s, 0.0 kB/s 2023-10-06T03:05:54.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.027 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2806e-05 s, 0.0 kB/s 2023-10-06T03:05:54.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.083 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.138 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.217e-05 s, 0.0 kB/s 2023-10-06T03:05:54.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.139 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.194 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.194 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.194 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.187e-05 s, 0.0 kB/s 2023-10-06T03:05:54.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.194 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.250 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3267e-05 s, 0.0 kB/s 2023-10-06T03:05:54.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.251 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.306 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1046e-05 s, 0.0 kB/s 2023-10-06T03:05:54.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.306 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.362 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.362 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.362 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2852e-05 s, 0.0 kB/s 2023-10-06T03:05:54.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.363 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.419 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.419 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.419 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1904e-05 s, 0.0 kB/s 2023-10-06T03:05:54.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.420 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9806e-05 s, 0.0 kB/s 2023-10-06T03:05:54.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.476 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.531 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.532 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1485e-05 s, 0.0 kB/s 2023-10-06T03:05:54.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.532 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.587 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3447e-05 s, 0.0 kB/s 2023-10-06T03:05:54.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.587 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.643 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2179e-05 s, 0.0 kB/s 2023-10-06T03:05:54.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.643 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.699 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1463e-05 s, 0.0 kB/s 2023-10-06T03:05:54.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.700 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.755 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.755 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2418e-05 s, 0.0 kB/s 2023-10-06T03:05:54.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.756 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9291e-05 s, 0.0 kB/s 2023-10-06T03:05:54.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.812 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2476e-05 s, 0.0 kB/s 2023-10-06T03:05:54.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.924 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1047e-05 s, 0.0 kB/s 2023-10-06T03:05:54.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.925 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:54.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:54.981 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3473e-05 s, 0.0 kB/s 2023-10-06T03:05:54.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:54.981 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.036 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.036 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.036 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2575e-05 s, 0.0 kB/s 2023-10-06T03:05:55.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.037 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.092 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0831e-05 s, 0.0 kB/s 2023-10-06T03:05:55.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.093 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9978e-05 s, 0.0 kB/s 2023-10-06T03:05:55.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.149 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9657e-05 s, 0.0 kB/s 2023-10-06T03:05:55.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.205 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.260 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.260 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.203e-05 s, 0.0 kB/s 2023-10-06T03:05:55.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.261 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.316 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.316 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1111e-05 s, 0.0 kB/s 2023-10-06T03:05:55.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.316 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.372 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2597e-05 s, 0.0 kB/s 2023-10-06T03:05:55.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.373 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.428 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2389e-05 s, 0.0 kB/s 2023-10-06T03:05:55.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.429 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0972e-05 s, 0.0 kB/s 2023-10-06T03:05:55.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.485 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.540 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.541 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1188e-05 s, 0.0 kB/s 2023-10-06T03:05:55.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.541 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.596 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2018e-05 s, 0.0 kB/s 2023-10-06T03:05:55.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.597 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9735e-05 s, 0.0 kB/s 2023-10-06T03:05:55.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.653 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.708 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.708 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.709 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.139e-05 s, 0.0 kB/s 2023-10-06T03:05:55.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.709 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.764 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1766e-05 s, 0.0 kB/s 2023-10-06T03:05:55.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.764 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9676e-05 s, 0.0 kB/s 2023-10-06T03:05:55.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.820 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.876 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.876 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.876 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8339e-05 s, 0.0 kB/s 2023-10-06T03:05:55.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.877 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.932 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.932 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.932 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1693e-05 s, 0.0 kB/s 2023-10-06T03:05:55.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.933 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:55.987 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0332e-05 s, 0.0 kB/s 2023-10-06T03:05:55.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:55.988 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.042 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.043 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.043 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1667e-05 s, 0.0 kB/s 2023-10-06T03:05:56.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.043 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.098 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7179e-05 s, 0.0 kB/s 2023-10-06T03:05:56.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.098 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.154 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1679e-05 s, 0.0 kB/s 2023-10-06T03:05:56.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.155 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.210 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.210 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1695e-05 s, 0.0 kB/s 2023-10-06T03:05:56.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.211 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.266 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.267 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9052e-05 s, 0.0 kB/s 2023-10-06T03:05:56.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.267 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1928e-05 s, 0.0 kB/s 2023-10-06T03:05:56.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.378 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.262e-05 s, 0.0 kB/s 2023-10-06T03:05:56.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.379 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0671e-05 s, 0.0 kB/s 2023-10-06T03:05:56.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.435 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.490 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2041e-05 s, 0.0 kB/s 2023-10-06T03:05:56.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.491 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.547 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.547 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1806e-05 s, 0.0 kB/s 2023-10-06T03:05:56.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.547 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.604 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.604 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4185e-05 s, 0.0 kB/s 2023-10-06T03:05:56.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.604 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.659 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2206e-05 s, 0.0 kB/s 2023-10-06T03:05:56.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.659 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.716 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1016e-05 s, 0.0 kB/s 2023-10-06T03:05:56.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.716 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0211e-05 s, 0.0 kB/s 2023-10-06T03:05:56.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.826 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0097e-05 s, 0.0 kB/s 2023-10-06T03:05:56.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.884 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.884 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8688e-05 s, 0.0 kB/s 2023-10-06T03:05:56.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.884 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9987e-05 s, 0.0 kB/s 2023-10-06T03:05:56.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.939 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2577e-05 s, 0.0 kB/s 2023-10-06T03:05:56.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:56.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.052 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3366e-05 s, 0.0 kB/s 2023-10-06T03:05:57.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.052 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.107 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2749e-05 s, 0.0 kB/s 2023-10-06T03:05:57.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.108 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.163 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.163 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.163 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0906e-05 s, 0.0 kB/s 2023-10-06T03:05:57.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.163 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.219 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.071e-05 s, 0.0 kB/s 2023-10-06T03:05:57.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.219 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.275 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0952e-05 s, 0.0 kB/s 2023-10-06T03:05:57.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.276 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.331 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.129e-05 s, 0.0 kB/s 2023-10-06T03:05:57.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.332 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.387 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0604e-05 s, 0.0 kB/s 2023-10-06T03:05:57.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.387 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.443 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2881e-05 s, 0.0 kB/s 2023-10-06T03:05:57.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.444 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.500 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.500 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.1552e-05 s, 0.0 kB/s 2023-10-06T03:05:57.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.501 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.557 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.558 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6179e-05 s, 0.0 kB/s 2023-10-06T03:05:57.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.558 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.613 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.613 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.614 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0113e-05 s, 0.0 kB/s 2023-10-06T03:05:57.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.614 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.669 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.669 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.669 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1007e-05 s, 0.0 kB/s 2023-10-06T03:05:57.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.670 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.725 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.726 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.726 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.2241e-05 s, 0.0 kB/s 2023-10-06T03:05:57.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.726 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.781 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.781 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.781 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7108e-05 s, 0.0 kB/s 2023-10-06T03:05:57.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.782 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.838 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.839 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.337e-05 s, 0.0 kB/s 2023-10-06T03:05:57.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.839 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.894 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.894 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.895 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3521e-05 s, 0.0 kB/s 2023-10-06T03:05:57.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.895 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:57.950 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:57.950 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:57.950 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0705e-05 s, 0.0 kB/s 2023-10-06T03:05:57.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:57.950 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.006 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3898e-05 s, 0.0 kB/s 2023-10-06T03:05:58.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.007 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2088e-05 s, 0.0 kB/s 2023-10-06T03:05:58.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.062 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.118 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.118 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.118 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8563e-05 s, 0.0 kB/s 2023-10-06T03:05:58.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.119 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.174 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.174 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.174 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7729e-05 s, 0.0 kB/s 2023-10-06T03:05:58.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.175 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.231 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.231 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2753e-05 s, 0.0 kB/s 2023-10-06T03:05:58.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.232 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.286 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.287 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.287 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4097e-05 s, 0.0 kB/s 2023-10-06T03:05:58.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.288 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.343 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.343 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9893e-05 s, 0.0 kB/s 2023-10-06T03:05:58.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.344 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.399 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.399 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2436e-05 s, 0.0 kB/s 2023-10-06T03:05:58.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.400 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.455 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.455 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.455 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9988e-05 s, 0.0 kB/s 2023-10-06T03:05:58.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.455 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.511 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.511 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1759e-05 s, 0.0 kB/s 2023-10-06T03:05:58.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.512 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.568 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9771e-05 s, 0.0 kB/s 2023-10-06T03:05:58.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.568 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.624 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3694e-05 s, 0.0 kB/s 2023-10-06T03:05:58.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.068e-05 s, 0.0 kB/s 2023-10-06T03:05:58.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.681 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9976e-05 s, 0.0 kB/s 2023-10-06T03:05:58.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0515e-05 s, 0.0 kB/s 2023-10-06T03:05:58.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.214e-05 s, 0.0 kB/s 2023-10-06T03:05:58.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2267e-05 s, 0.0 kB/s 2023-10-06T03:05:58.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:58.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:58.960 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1558e-05 s, 0.0 kB/s 2023-10-06T03:05:58.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:58.960 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5488e-05 s, 0.0 kB/s 2023-10-06T03:05:59.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.015 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.070 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.070 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.070 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2065e-05 s, 0.0 kB/s 2023-10-06T03:05:59.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.071 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.127 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9597e-05 s, 0.0 kB/s 2023-10-06T03:05:59.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.127 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.183 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.957e-05 s, 0.0 kB/s 2023-10-06T03:05:59.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.184 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.239 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0062e-05 s, 0.0 kB/s 2023-10-06T03:05:59.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.239 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.295 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1184e-05 s, 0.0 kB/s 2023-10-06T03:05:59.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.351 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2113e-05 s, 0.0 kB/s 2023-10-06T03:05:59.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.407 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0702e-05 s, 0.0 kB/s 2023-10-06T03:05:59.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.407 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.469 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3076e-05 s, 0.0 kB/s 2023-10-06T03:05:59.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.470 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.525 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.525 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.525 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1574e-05 s, 0.0 kB/s 2023-10-06T03:05:59.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.526 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.581 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1244e-05 s, 0.0 kB/s 2023-10-06T03:05:59.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.582 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.636 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.636 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.636 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0259e-05 s, 0.0 kB/s 2023-10-06T03:05:59.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.638 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8561e-05 s, 0.0 kB/s 2023-10-06T03:05:59.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.748 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.748 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.748 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1461e-05 s, 0.0 kB/s 2023-10-06T03:05:59.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.749 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.804 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.804 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.804 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.854e-05 s, 0.0 kB/s 2023-10-06T03:05:59.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.804 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.860 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.860 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.860 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.92e-05 s, 0.0 kB/s 2023-10-06T03:05:59.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.861 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.915 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.916 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0919e-05 s, 0.0 kB/s 2023-10-06T03:05:59.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.916 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:05:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:05:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:05:59.971 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9789e-05 s, 0.0 kB/s 2023-10-06T03:05:59.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:05:59.971 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.026 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.027 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.027 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3338e-05 s, 0.0 kB/s 2023-10-06T03:06:00.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.027 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.319e-05 s, 0.0 kB/s 2023-10-06T03:06:00.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.082 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.138 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.138 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1092e-05 s, 0.0 kB/s 2023-10-06T03:06:00.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.138 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.193 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0842e-05 s, 0.0 kB/s 2023-10-06T03:06:00.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.193 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.249 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8699e-05 s, 0.0 kB/s 2023-10-06T03:06:00.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.250 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.304 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.304 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.305 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9589e-05 s, 0.0 kB/s 2023-10-06T03:06:00.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.305 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.360 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1407e-05 s, 0.0 kB/s 2023-10-06T03:06:00.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.361 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.416 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.416 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.416 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9612e-05 s, 0.0 kB/s 2023-10-06T03:06:00.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.416 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.472 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0508e-05 s, 0.0 kB/s 2023-10-06T03:06:00.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.472 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.528 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7174e-05 s, 0.0 kB/s 2023-10-06T03:06:00.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.528 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.583 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.583 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.583 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6645e-05 s, 0.0 kB/s 2023-10-06T03:06:00.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.584 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.639 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.067e-05 s, 0.0 kB/s 2023-10-06T03:06:00.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.640 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3548e-05 s, 0.0 kB/s 2023-10-06T03:06:00.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.697 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.752 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.752 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.752 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8942e-05 s, 0.0 kB/s 2023-10-06T03:06:00.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.753 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.808 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1589e-05 s, 0.0 kB/s 2023-10-06T03:06:00.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.808 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.863 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.864 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.864 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2518e-05 s, 0.0 kB/s 2023-10-06T03:06:00.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.864 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.919 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2344e-05 s, 0.0 kB/s 2023-10-06T03:06:00.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.920 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:00.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:00.974 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:00.975 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9608e-05 s, 0.0 kB/s 2023-10-06T03:06:00.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:00.976 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8894e-05 s, 0.0 kB/s 2023-10-06T03:06:01.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.031 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.086 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.086 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.086 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9342e-05 s, 0.0 kB/s 2023-10-06T03:06:01.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.086 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.142 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.143 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0083e-05 s, 0.0 kB/s 2023-10-06T03:06:01.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.143 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.198 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.198 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.198 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9876e-05 s, 0.0 kB/s 2023-10-06T03:06:01.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.198 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.84e-05 s, 0.0 kB/s 2023-10-06T03:06:01.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.255 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.310 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.310 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.310 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9528e-05 s, 0.0 kB/s 2023-10-06T03:06:01.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.311 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.366 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.366 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.366 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9398e-05 s, 0.0 kB/s 2023-10-06T03:06:01.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.366 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.422 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1632e-05 s, 0.0 kB/s 2023-10-06T03:06:01.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.423 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.478 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.478 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0858e-05 s, 0.0 kB/s 2023-10-06T03:06:01.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.478 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.533 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0413e-05 s, 0.0 kB/s 2023-10-06T03:06:01.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.533 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.589 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9578e-05 s, 0.0 kB/s 2023-10-06T03:06:01.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.590 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.645 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1888e-05 s, 0.0 kB/s 2023-10-06T03:06:01.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.646 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.701 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-06T03:06:01.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.702 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.757 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1092e-05 s, 0.0 kB/s 2023-10-06T03:06:01.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2849e-05 s, 0.0 kB/s 2023-10-06T03:06:01.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.812 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0002e-05 s, 0.0 kB/s 2023-10-06T03:06:01.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.869 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.924 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8926e-05 s, 0.0 kB/s 2023-10-06T03:06:01.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.924 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:01.980 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-06T03:06:01.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:01.981 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.035 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.036 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.036 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9444e-05 s, 0.0 kB/s 2023-10-06T03:06:02.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.036 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.091 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.091 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9835e-05 s, 0.0 kB/s 2023-10-06T03:06:02.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.092 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.147 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9657e-05 s, 0.0 kB/s 2023-10-06T03:06:02.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.148 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.203 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.203 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.203 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8719e-05 s, 0.0 kB/s 2023-10-06T03:06:02.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.203 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.259 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-06T03:06:02.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.260 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9066e-05 s, 0.0 kB/s 2023-10-06T03:06:02.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.315 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.371 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7692e-05 s, 0.0 kB/s 2023-10-06T03:06:02.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.371 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.427 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.427 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.427 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0396e-05 s, 0.0 kB/s 2023-10-06T03:06:02.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.482 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.483 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9962e-05 s, 0.0 kB/s 2023-10-06T03:06:02.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.483 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.538 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.539 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.539 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8519e-05 s, 0.0 kB/s 2023-10-06T03:06:02.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.539 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.594 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2726e-05 s, 0.0 kB/s 2023-10-06T03:06:02.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.595 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1947e-05 s, 0.0 kB/s 2023-10-06T03:06:02.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.650 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.706 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0819e-05 s, 0.0 kB/s 2023-10-06T03:06:02.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.706 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1954e-05 s, 0.0 kB/s 2023-10-06T03:06:02.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.817 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1019e-05 s, 0.0 kB/s 2023-10-06T03:06:02.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.817 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.873 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7612e-05 s, 0.0 kB/s 2023-10-06T03:06:02.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.930 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.931 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.931 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8154e-05 s, 0.0 kB/s 2023-10-06T03:06:02.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.931 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:02.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:02.986 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:02.986 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.711e-05 s, 0.0 kB/s 2023-10-06T03:06:02.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:02.986 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.041 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0673e-05 s, 0.0 kB/s 2023-10-06T03:06:03.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.041 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.097 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2561e-05 s, 0.0 kB/s 2023-10-06T03:06:03.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.097 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.153 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8875e-05 s, 0.0 kB/s 2023-10-06T03:06:03.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.154 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.209 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0494e-05 s, 0.0 kB/s 2023-10-06T03:06:03.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.210 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.265 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.265 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3158e-05 s, 0.0 kB/s 2023-10-06T03:06:03.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.266 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.322 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.322 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1613e-05 s, 0.0 kB/s 2023-10-06T03:06:03.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.323 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.378 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.379 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.379 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2493e-05 s, 0.0 kB/s 2023-10-06T03:06:03.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.379 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.434 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1858e-05 s, 0.0 kB/s 2023-10-06T03:06:03.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.435 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.489 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.490 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.490 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.949e-05 s, 0.0 kB/s 2023-10-06T03:06:03.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.491 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.545 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.546 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3196e-05 s, 0.0 kB/s 2023-10-06T03:06:03.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.546 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.603 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.603 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4663e-05 s, 0.0 kB/s 2023-10-06T03:06:03.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.603 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.659 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.660 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.1701e-05 s, 0.0 kB/s 2023-10-06T03:06:03.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.661 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.715 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.716 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4271e-05 s, 0.0 kB/s 2023-10-06T03:06:03.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.716 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0576e-05 s, 0.0 kB/s 2023-10-06T03:06:03.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.772 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3162e-05 s, 0.0 kB/s 2023-10-06T03:06:03.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.77e-05 s, 0.0 kB/s 2023-10-06T03:06:03.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.883 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.221e-05 s, 0.0 kB/s 2023-10-06T03:06:03.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.940 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:03.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:03.995 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:03.995 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8736e-05 s, 0.0 kB/s 2023-10-06T03:06:03.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:03.995 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.051 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.051 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9761e-05 s, 0.0 kB/s 2023-10-06T03:06:04.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.052 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.107 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.108 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1876e-05 s, 0.0 kB/s 2023-10-06T03:06:04.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.108 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.162 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.162 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8391e-05 s, 0.0 kB/s 2023-10-06T03:06:04.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.163 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.219 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2641e-05 s, 0.0 kB/s 2023-10-06T03:06:04.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.220 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.275 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.4069e-05 s, 0.0 kB/s 2023-10-06T03:06:04.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.275 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.330 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0923e-05 s, 0.0 kB/s 2023-10-06T03:06:04.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.387 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8254e-05 s, 0.0 kB/s 2023-10-06T03:06:04.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.387 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.443 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.443 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7687e-05 s, 0.0 kB/s 2023-10-06T03:06:04.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.443 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.499 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2251e-05 s, 0.0 kB/s 2023-10-06T03:06:04.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.500 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.566 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.567 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8186e-05 s, 0.0 kB/s 2023-10-06T03:06:04.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.567 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8523e-05 s, 0.0 kB/s 2023-10-06T03:06:04.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.623 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.678 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.679 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9636e-05 s, 0.0 kB/s 2023-10-06T03:06:04.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.679 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.734 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.734 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1503e-05 s, 0.0 kB/s 2023-10-06T03:06:04.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.735 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.013e-05 s, 0.0 kB/s 2023-10-06T03:06:04.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.790 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.846 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7937e-05 s, 0.0 kB/s 2023-10-06T03:06:04.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.847 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-06T03:06:04.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.902 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:04.957 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0057e-05 s, 0.0 kB/s 2023-10-06T03:06:04.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:04.958 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0842e-05 s, 0.0 kB/s 2023-10-06T03:06:05.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8618e-05 s, 0.0 kB/s 2023-10-06T03:06:05.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.124 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0816e-05 s, 0.0 kB/s 2023-10-06T03:06:05.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.125 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.180 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.181 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3167e-05 s, 0.0 kB/s 2023-10-06T03:06:05.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.181 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.236 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.236 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.236 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8453e-05 s, 0.0 kB/s 2023-10-06T03:06:05.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.236 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.291 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.292 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.292 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0197e-05 s, 0.0 kB/s 2023-10-06T03:06:05.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.292 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.83e-05 s, 0.0 kB/s 2023-10-06T03:06:05.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.403 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.403 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8364e-05 s, 0.0 kB/s 2023-10-06T03:06:05.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.403 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.458 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.458 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7638e-05 s, 0.0 kB/s 2023-10-06T03:06:05.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.976e-05 s, 0.0 kB/s 2023-10-06T03:06:05.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.514 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.568 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.02e-05 s, 0.0 kB/s 2023-10-06T03:06:05.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.625 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9232e-05 s, 0.0 kB/s 2023-10-06T03:06:05.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1168e-05 s, 0.0 kB/s 2023-10-06T03:06:05.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.681 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.736 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1687e-05 s, 0.0 kB/s 2023-10-06T03:06:05.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.736 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.792 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0194e-05 s, 0.0 kB/s 2023-10-06T03:06:05.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1079e-05 s, 0.0 kB/s 2023-10-06T03:06:05.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.848 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0151e-05 s, 0.0 kB/s 2023-10-06T03:06:05.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:05.959 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:05.960 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6139e-05 s, 0.0 kB/s 2023-10-06T03:06:05.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:05.960 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.015 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.015 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8777e-05 s, 0.0 kB/s 2023-10-06T03:06:06.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.071 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.071 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.337e-05 s, 0.0 kB/s 2023-10-06T03:06:06.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.071 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.127 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.127 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2259e-05 s, 0.0 kB/s 2023-10-06T03:06:06.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.127 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8594e-05 s, 0.0 kB/s 2023-10-06T03:06:06.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.183 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.239 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.240 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3351e-05 s, 0.0 kB/s 2023-10-06T03:06:06.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.295 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1328e-05 s, 0.0 kB/s 2023-10-06T03:06:06.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.351 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0907e-05 s, 0.0 kB/s 2023-10-06T03:06:06.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.352 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.407 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.407 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.897e-05 s, 0.0 kB/s 2023-10-06T03:06:06.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.407 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.461 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.461 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.462 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1353e-05 s, 0.0 kB/s 2023-10-06T03:06:06.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.462 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.517 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1624e-05 s, 0.0 kB/s 2023-10-06T03:06:06.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.517 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.571 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.572 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2341e-05 s, 0.0 kB/s 2023-10-06T03:06:06.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.572 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.627 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2149e-05 s, 0.0 kB/s 2023-10-06T03:06:06.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.628 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.683 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.683 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.683 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2134e-05 s, 0.0 kB/s 2023-10-06T03:06:06.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.684 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.739 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.739 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3927e-05 s, 0.0 kB/s 2023-10-06T03:06:06.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.740 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.795 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.795 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.795 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0972e-05 s, 0.0 kB/s 2023-10-06T03:06:06.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.796 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.851 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.851 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5299e-05 s, 0.0 kB/s 2023-10-06T03:06:06.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.851 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.907 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.206e-05 s, 0.0 kB/s 2023-10-06T03:06:06.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.907 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0115e-05 s, 0.0 kB/s 2023-10-06T03:06:06.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:06.963 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.019 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3625e-05 s, 0.0 kB/s 2023-10-06T03:06:07.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.019 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.075 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8816e-05 s, 0.0 kB/s 2023-10-06T03:06:07.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.075 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.133 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9629e-05 s, 0.0 kB/s 2023-10-06T03:06:07.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.133 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.194 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9603e-05 s, 0.0 kB/s 2023-10-06T03:06:07.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.194 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.253 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.32e-05 s, 0.0 kB/s 2023-10-06T03:06:07.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.254 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.310 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0381e-05 s, 0.0 kB/s 2023-10-06T03:06:07.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.311 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.365 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.365 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.365 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-10-06T03:06:07.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.366 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.421 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.422 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.422 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8755e-05 s, 0.0 kB/s 2023-10-06T03:06:07.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.422 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.477 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2604e-05 s, 0.0 kB/s 2023-10-06T03:06:07.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.477 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.533 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1449e-05 s, 0.0 kB/s 2023-10-06T03:06:07.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.588 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.589 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.589 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.75e-05 s, 0.0 kB/s 2023-10-06T03:06:07.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.589 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.039e-05 s, 0.0 kB/s 2023-10-06T03:06:07.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.645 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.700 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.700 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0898e-05 s, 0.0 kB/s 2023-10-06T03:06:07.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.701 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.756 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.756 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0835e-05 s, 0.0 kB/s 2023-10-06T03:06:07.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.757 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.812 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.812 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2691e-05 s, 0.0 kB/s 2023-10-06T03:06:07.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.813 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.868 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.868 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0552e-05 s, 0.0 kB/s 2023-10-06T03:06:07.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.868 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.174e-05 s, 0.0 kB/s 2023-10-06T03:06:07.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.924 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0449e-05 s, 0.0 kB/s 2023-10-06T03:06:07.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:07.980 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.034 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.035 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.035 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3728e-05 s, 0.0 kB/s 2023-10-06T03:06:08.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.035 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.090 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2107e-05 s, 0.0 kB/s 2023-10-06T03:06:08.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.090 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.146 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2894e-05 s, 0.0 kB/s 2023-10-06T03:06:08.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.146 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.201 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.201 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.201 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9963e-05 s, 0.0 kB/s 2023-10-06T03:06:08.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.202 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0628e-05 s, 0.0 kB/s 2023-10-06T03:06:08.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.259 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.314 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.121e-05 s, 0.0 kB/s 2023-10-06T03:06:08.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.314 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.370 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.370 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.370 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2201e-05 s, 0.0 kB/s 2023-10-06T03:06:08.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.371 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.425 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.426 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9809e-05 s, 0.0 kB/s 2023-10-06T03:06:08.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.482 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.482 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.482 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9088e-05 s, 0.0 kB/s 2023-10-06T03:06:08.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.483 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.537 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.538 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.538 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8186e-05 s, 0.0 kB/s 2023-10-06T03:06:08.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.538 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2673e-05 s, 0.0 kB/s 2023-10-06T03:06:08.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.594 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.653 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3566e-05 s, 0.0 kB/s 2023-10-06T03:06:08.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.710 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8004e-05 s, 0.0 kB/s 2023-10-06T03:06:08.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.711 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.767 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.768 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5632e-05 s, 0.0 kB/s 2023-10-06T03:06:08.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.768 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.823 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.823 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3008e-05 s, 0.0 kB/s 2023-10-06T03:06:08.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.824 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.879 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.879 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2061e-05 s, 0.0 kB/s 2023-10-06T03:06:08.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.880 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.936 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.936 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0541e-05 s, 0.0 kB/s 2023-10-06T03:06:08.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.937 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:08.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:08.992 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:08.992 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4109e-05 s, 0.0 kB/s 2023-10-06T03:06:08.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:08.993 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.048 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1289e-05 s, 0.0 kB/s 2023-10-06T03:06:09.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3393e-05 s, 0.0 kB/s 2023-10-06T03:06:09.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.105 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.160 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0955e-05 s, 0.0 kB/s 2023-10-06T03:06:09.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.161 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.216 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.216 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.216 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0558e-05 s, 0.0 kB/s 2023-10-06T03:06:09.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.216 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.273 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.273 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1481e-05 s, 0.0 kB/s 2023-10-06T03:06:09.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.273 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.329 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.329 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.329 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2239e-05 s, 0.0 kB/s 2023-10-06T03:06:09.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.329 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.384 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.385 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0346e-05 s, 0.0 kB/s 2023-10-06T03:06:09.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.385 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.439 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0729e-05 s, 0.0 kB/s 2023-10-06T03:06:09.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.440 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.495 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.495 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1807e-05 s, 0.0 kB/s 2023-10-06T03:06:09.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.495 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.551 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.552 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.552 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1847e-05 s, 0.0 kB/s 2023-10-06T03:06:09.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.552 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.607 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0099e-05 s, 0.0 kB/s 2023-10-06T03:06:09.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.607 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.703 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.704 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.704 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.6006e-05 s, 0.0 kB/s 2023-10-06T03:06:09.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.704 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.720 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.720 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.721 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1392e-05 s, 0.0 kB/s 2023-10-06T03:06:09.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.721 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.776 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.777 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2053e-05 s, 0.0 kB/s 2023-10-06T03:06:09.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.777 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.832 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.832 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.832 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2891e-05 s, 0.0 kB/s 2023-10-06T03:06:09.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.832 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.887 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.888 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.888 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.18e-05 s, 0.0 kB/s 2023-10-06T03:06:09.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.888 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.943 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0919e-05 s, 0.0 kB/s 2023-10-06T03:06:09.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:09.943 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:09.999 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:09.999 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:09.999 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9355e-05 s, 0.0 kB/s 2023-10-06T03:06:09.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.000 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.055 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.055 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.055 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2852e-05 s, 0.0 kB/s 2023-10-06T03:06:10.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.056 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9239e-05 s, 0.0 kB/s 2023-10-06T03:06:10.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.111 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.165 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.166 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.166 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-06T03:06:10.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.166 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.222 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.222 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.222 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1898e-05 s, 0.0 kB/s 2023-10-06T03:06:10.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.223 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.278 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.04e-05 s, 0.0 kB/s 2023-10-06T03:06:10.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.279 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.334 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3361e-05 s, 0.0 kB/s 2023-10-06T03:06:10.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.335 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.390 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.390 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.390 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2014e-05 s, 0.0 kB/s 2023-10-06T03:06:10.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.390 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.446 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.446 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.446 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1511e-05 s, 0.0 kB/s 2023-10-06T03:06:10.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.447 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.501 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.084e-05 s, 0.0 kB/s 2023-10-06T03:06:10.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.502 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.952e-05 s, 0.0 kB/s 2023-10-06T03:06:10.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.558 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.612 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.613 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.613 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1281e-05 s, 0.0 kB/s 2023-10-06T03:06:10.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.614 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.668 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.668 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.668 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8247e-05 s, 0.0 kB/s 2023-10-06T03:06:10.669 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.669 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.726 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.726 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.726 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4489e-05 s, 0.0 kB/s 2023-10-06T03:06:10.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.726 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.781 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.782 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0585e-05 s, 0.0 kB/s 2023-10-06T03:06:10.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.782 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.837 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9824e-05 s, 0.0 kB/s 2023-10-06T03:06:10.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.837 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.893 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.893 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.893 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9188e-05 s, 0.0 kB/s 2023-10-06T03:06:10.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.894 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:10.949 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:10.949 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:10.949 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9852e-05 s, 0.0 kB/s 2023-10-06T03:06:10.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:10.949 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.005 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9627e-05 s, 0.0 kB/s 2023-10-06T03:06:11.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.006 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.061 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.154e-05 s, 0.0 kB/s 2023-10-06T03:06:11.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.062 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9666e-05 s, 0.0 kB/s 2023-10-06T03:06:11.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.118 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.172 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1418e-05 s, 0.0 kB/s 2023-10-06T03:06:11.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.228 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.229 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1634e-05 s, 0.0 kB/s 2023-10-06T03:06:11.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.230 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.284 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.285 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.285 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8415e-05 s, 0.0 kB/s 2023-10-06T03:06:11.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.285 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.400 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.400 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2579e-05 s, 0.0 kB/s 2023-10-06T03:06:11.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.400 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.415 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.415 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.415 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9902e-05 s, 0.0 kB/s 2023-10-06T03:06:11.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.416 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.470 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.470 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.470 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0745e-05 s, 0.0 kB/s 2023-10-06T03:06:11.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.471 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.526 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.527 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8443e-05 s, 0.0 kB/s 2023-10-06T03:06:11.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.527 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.581 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.582 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8564e-05 s, 0.0 kB/s 2023-10-06T03:06:11.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.582 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.637 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.637 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.637 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1689e-05 s, 0.0 kB/s 2023-10-06T03:06:11.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.638 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.693 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9071e-05 s, 0.0 kB/s 2023-10-06T03:06:11.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.693 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.750 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.750 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.750 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8994e-05 s, 0.0 kB/s 2023-10-06T03:06:11.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.750 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.806 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.186e-05 s, 0.0 kB/s 2023-10-06T03:06:11.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.806 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.862 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7999e-05 s, 0.0 kB/s 2023-10-06T03:06:11.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.862 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.917 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.918 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.918 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1362e-05 s, 0.0 kB/s 2023-10-06T03:06:11.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.918 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:11.972 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:11.972 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:11.973 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9479e-05 s, 0.0 kB/s 2023-10-06T03:06:11.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:11.973 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.027 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.028 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9335e-05 s, 0.0 kB/s 2023-10-06T03:06:12.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.028 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.083 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2097e-05 s, 0.0 kB/s 2023-10-06T03:06:12.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.083 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.139 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.139 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.139 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0648e-05 s, 0.0 kB/s 2023-10-06T03:06:12.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.140 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.195 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.195 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.195 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9588e-05 s, 0.0 kB/s 2023-10-06T03:06:12.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.196 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.251 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.251 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.251 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.919e-05 s, 0.0 kB/s 2023-10-06T03:06:12.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.252 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3588e-05 s, 0.0 kB/s 2023-10-06T03:06:12.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.313 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.344 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:06:12.360 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:06:12.371 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.373 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.373 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0838e-05 s, 0.0 kB/s 2023-10-06T03:06:12.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.374 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.463 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1245e-05 s, 0.0 kB/s 2023-10-06T03:06:12.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.464 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.519 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.519 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4487e-05 s, 0.0 kB/s 2023-10-06T03:06:12.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.520 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.577 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.578 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9785e-05 s, 0.0 kB/s 2023-10-06T03:06:12.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.578 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.632 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.632 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.632 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1637e-05 s, 0.0 kB/s 2023-10-06T03:06:12.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.633 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.688 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.689 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0587e-05 s, 0.0 kB/s 2023-10-06T03:06:12.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.689 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.744 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.744 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-10-06T03:06:12.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.745 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.800 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1398e-05 s, 0.0 kB/s 2023-10-06T03:06:12.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.801 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.857 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9699e-05 s, 0.0 kB/s 2023-10-06T03:06:12.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.858 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.913 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.914 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2411e-05 s, 0.0 kB/s 2023-10-06T03:06:12.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.914 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:12.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:12.969 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:12.969 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9772e-05 s, 0.0 kB/s 2023-10-06T03:06:12.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:12.970 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.025 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7676e-05 s, 0.0 kB/s 2023-10-06T03:06:13.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.026 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.080 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1787e-05 s, 0.0 kB/s 2023-10-06T03:06:13.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.081 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1368e-05 s, 0.0 kB/s 2023-10-06T03:06:13.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.137 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.193 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.193 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8415e-05 s, 0.0 kB/s 2023-10-06T03:06:13.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.193 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.249 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8182e-05 s, 0.0 kB/s 2023-10-06T03:06:13.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.249 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.304 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.305 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8643e-05 s, 0.0 kB/s 2023-10-06T03:06:13.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.305 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.360 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.360 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2719e-05 s, 0.0 kB/s 2023-10-06T03:06:13.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.360 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.415 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.416 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.416 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.01e-05 s, 0.0 kB/s 2023-10-06T03:06:13.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.416 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.470 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.471 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-06T03:06:13.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.471 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0963e-05 s, 0.0 kB/s 2023-10-06T03:06:13.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.526 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.582 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.583 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1759e-05 s, 0.0 kB/s 2023-10-06T03:06:13.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.584 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.640 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3989e-05 s, 0.0 kB/s 2023-10-06T03:06:13.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.641 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.695 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.696 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1419e-05 s, 0.0 kB/s 2023-10-06T03:06:13.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.697 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.752 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.752 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.752 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2676e-05 s, 0.0 kB/s 2023-10-06T03:06:13.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.753 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.809 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.809 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.809 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0165e-05 s, 0.0 kB/s 2023-10-06T03:06:13.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.810 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.865 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.865 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1242e-05 s, 0.0 kB/s 2023-10-06T03:06:13.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.866 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.920 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.921 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.921e-05 s, 0.0 kB/s 2023-10-06T03:06:13.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.921 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:13.976 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7849e-05 s, 0.0 kB/s 2023-10-06T03:06:13.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:13.976 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.032 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.033 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3484e-05 s, 0.0 kB/s 2023-10-06T03:06:14.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.033 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.088 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.089 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3529e-05 s, 0.0 kB/s 2023-10-06T03:06:14.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.089 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.144 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.145 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.145 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0513e-05 s, 0.0 kB/s 2023-10-06T03:06:14.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.145 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.200 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0742e-05 s, 0.0 kB/s 2023-10-06T03:06:14.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.201 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.256 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.257 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9114e-05 s, 0.0 kB/s 2023-10-06T03:06:14.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.312 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9922e-05 s, 0.0 kB/s 2023-10-06T03:06:14.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.312 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1393e-05 s, 0.0 kB/s 2023-10-06T03:06:14.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.423 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.424 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.424 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9951e-05 s, 0.0 kB/s 2023-10-06T03:06:14.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.424 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.479 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.479 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.479 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.806e-05 s, 0.0 kB/s 2023-10-06T03:06:14.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.479 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.534 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4256e-05 s, 0.0 kB/s 2023-10-06T03:06:14.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.535 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.590 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.590 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.590 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0775e-05 s, 0.0 kB/s 2023-10-06T03:06:14.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.590 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.645 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-06T03:06:14.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.646 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.702 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.702 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.702 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1441e-05 s, 0.0 kB/s 2023-10-06T03:06:14.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.702 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.758 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.758 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3952e-05 s, 0.0 kB/s 2023-10-06T03:06:14.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.759 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.814 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.814 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.814 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1269e-05 s, 0.0 kB/s 2023-10-06T03:06:14.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.815 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.870 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.870 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.870 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1619e-05 s, 0.0 kB/s 2023-10-06T03:06:14.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.871 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.927 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.927 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0668e-05 s, 0.0 kB/s 2023-10-06T03:06:14.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.928 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:14.983 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:14.984 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:14.984 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2579e-05 s, 0.0 kB/s 2023-10-06T03:06:14.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:14.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1486e-05 s, 0.0 kB/s 2023-10-06T03:06:15.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.039 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.094 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.095 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.095 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0909e-05 s, 0.0 kB/s 2023-10-06T03:06:15.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.096 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.151 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.151 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.151 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9871e-05 s, 0.0 kB/s 2023-10-06T03:06:15.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.151 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.207 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.207 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.207 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8922e-05 s, 0.0 kB/s 2023-10-06T03:06:15.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.207 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.263 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.263 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9442e-05 s, 0.0 kB/s 2023-10-06T03:06:15.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.264 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.318 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.319 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0214e-05 s, 0.0 kB/s 2023-10-06T03:06:15.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.320 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.375 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.375 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.096e-05 s, 0.0 kB/s 2023-10-06T03:06:15.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.375 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.430 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.430 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.431 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2882e-05 s, 0.0 kB/s 2023-10-06T03:06:15.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.431 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.485 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.486 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9146e-05 s, 0.0 kB/s 2023-10-06T03:06:15.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.486 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.541 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8155e-05 s, 0.0 kB/s 2023-10-06T03:06:15.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.541 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.596 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.597 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0432e-05 s, 0.0 kB/s 2023-10-06T03:06:15.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.598 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.653 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0562e-05 s, 0.0 kB/s 2023-10-06T03:06:15.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.654 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.709 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9188e-05 s, 0.0 kB/s 2023-10-06T03:06:15.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.709 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.765 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.766 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4814e-05 s, 0.0 kB/s 2023-10-06T03:06:15.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.766 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.839 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.8543e-05 s, 0.0 kB/s 2023-10-06T03:06:15.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.839 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.894 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.895 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3319e-05 s, 0.0 kB/s 2023-10-06T03:06:15.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.896 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:15.950 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:15.951 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:15.951 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8711e-05 s, 0.0 kB/s 2023-10-06T03:06:15.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:15.951 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.005 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.006 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.006 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8695e-05 s, 0.0 kB/s 2023-10-06T03:06:16.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.006 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.060 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.061 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7832e-05 s, 0.0 kB/s 2023-10-06T03:06:16.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.061 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.116 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.116 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.116 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9654e-05 s, 0.0 kB/s 2023-10-06T03:06:16.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.116 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.171 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.172 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9855e-05 s, 0.0 kB/s 2023-10-06T03:06:16.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.172 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.227 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.227 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0829e-05 s, 0.0 kB/s 2023-10-06T03:06:16.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.227 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.283 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.284 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2147e-05 s, 0.0 kB/s 2023-10-06T03:06:16.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.285 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.339 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0484e-05 s, 0.0 kB/s 2023-10-06T03:06:16.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.339 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.395 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.395 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.395 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.23e-05 s, 0.0 kB/s 2023-10-06T03:06:16.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.396 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.451 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5261e-05 s, 0.0 kB/s 2023-10-06T03:06:16.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.451 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.507 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2943e-05 s, 0.0 kB/s 2023-10-06T03:06:16.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.507 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.563 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.563 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0435e-05 s, 0.0 kB/s 2023-10-06T03:06:16.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.563 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.618 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.619 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.619 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1769e-05 s, 0.0 kB/s 2023-10-06T03:06:16.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.619 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.674 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.675 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7793e-05 s, 0.0 kB/s 2023-10-06T03:06:16.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.675 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.730 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6908e-05 s, 0.0 kB/s 2023-10-06T03:06:16.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.731 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2532e-05 s, 0.0 kB/s 2023-10-06T03:06:16.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.787 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.842 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.843 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3142e-05 s, 0.0 kB/s 2023-10-06T03:06:16.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.843 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.897 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.898 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.898 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2167e-05 s, 0.0 kB/s 2023-10-06T03:06:16.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.898 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:16.953 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9004e-05 s, 0.0 kB/s 2023-10-06T03:06:16.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:16.953 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.009 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9066e-05 s, 0.0 kB/s 2023-10-06T03:06:17.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.010 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.064 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.064 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.064 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8544e-05 s, 0.0 kB/s 2023-10-06T03:06:17.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.065 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.121 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0583e-05 s, 0.0 kB/s 2023-10-06T03:06:17.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.121 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.177 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-10-06T03:06:17.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.177 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.233 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2049e-05 s, 0.0 kB/s 2023-10-06T03:06:17.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.289 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2712e-05 s, 0.0 kB/s 2023-10-06T03:06:17.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.420 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2046e-05 s, 0.0 kB/s 2023-10-06T03:06:17.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.421 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.436 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0933e-05 s, 0.0 kB/s 2023-10-06T03:06:17.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.437 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.492 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8047e-05 s, 0.0 kB/s 2023-10-06T03:06:17.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.492 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.548 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9886e-05 s, 0.0 kB/s 2023-10-06T03:06:17.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.548 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.604 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.604 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.604 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6766e-05 s, 0.0 kB/s 2023-10-06T03:06:17.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.604 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.660 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.660 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.660 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.28e-05 s, 0.0 kB/s 2023-10-06T03:06:17.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.660 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.716 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.717 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5227e-05 s, 0.0 kB/s 2023-10-06T03:06:17.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.717 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.772 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2012e-05 s, 0.0 kB/s 2023-10-06T03:06:17.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.773 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.828 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.828 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.828 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.097e-05 s, 0.0 kB/s 2023-10-06T03:06:17.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.884 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.884 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.884 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1624e-05 s, 0.0 kB/s 2023-10-06T03:06:17.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.885 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7652e-05 s, 0.0 kB/s 2023-10-06T03:06:17.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.941 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:17.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:17.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:17.996 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8083e-05 s, 0.0 kB/s 2023-10-06T03:06:17.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:17.996 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.052 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.052 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.052 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0693e-05 s, 0.0 kB/s 2023-10-06T03:06:18.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.052 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.108 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7854e-05 s, 0.0 kB/s 2023-10-06T03:06:18.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.109 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.164 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9615e-05 s, 0.0 kB/s 2023-10-06T03:06:18.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.164 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.220 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.220 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.220 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1138e-05 s, 0.0 kB/s 2023-10-06T03:06:18.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.220 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.275 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.276 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8567e-05 s, 0.0 kB/s 2023-10-06T03:06:18.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.276 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.331 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4095e-05 s, 0.0 kB/s 2023-10-06T03:06:18.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.385 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1471e-05 s, 0.0 kB/s 2023-10-06T03:06:18.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.386 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.441 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8117e-05 s, 0.0 kB/s 2023-10-06T03:06:18.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.442 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3115e-05 s, 0.0 kB/s 2023-10-06T03:06:18.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.498 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.552 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.553 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2966e-05 s, 0.0 kB/s 2023-10-06T03:06:18.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.553 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.609 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.609 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.609 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1196e-05 s, 0.0 kB/s 2023-10-06T03:06:18.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.609 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.665 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.665 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0862e-05 s, 0.0 kB/s 2023-10-06T03:06:18.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.666 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.721 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0307e-05 s, 0.0 kB/s 2023-10-06T03:06:18.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.721 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.776 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.777 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.777 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1628e-05 s, 0.0 kB/s 2023-10-06T03:06:18.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.777 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.833 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.833 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.833 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.25e-05 s, 0.0 kB/s 2023-10-06T03:06:18.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.833 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.889 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9429e-05 s, 0.0 kB/s 2023-10-06T03:06:18.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.889 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:18.945 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:18.945 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:18.945 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9676e-05 s, 0.0 kB/s 2023-10-06T03:06:18.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:18.946 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.001 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.391e-05 s, 0.0 kB/s 2023-10-06T03:06:19.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.002 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.056 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.057 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.057 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.95e-05 s, 0.0 kB/s 2023-10-06T03:06:19.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.058 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.114 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.114 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.114 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0308e-05 s, 0.0 kB/s 2023-10-06T03:06:19.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.114 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1279e-05 s, 0.0 kB/s 2023-10-06T03:06:19.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.170 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.224 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.225 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.225 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3361e-05 s, 0.0 kB/s 2023-10-06T03:06:19.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.226 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.281 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.281 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.281 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9721e-05 s, 0.0 kB/s 2023-10-06T03:06:19.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.282 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.336 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.337 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1675e-05 s, 0.0 kB/s 2023-10-06T03:06:19.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.339 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.393 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.393 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.393 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8767e-05 s, 0.0 kB/s 2023-10-06T03:06:19.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.393 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8516e-05 s, 0.0 kB/s 2023-10-06T03:06:19.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.450 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.504 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.505 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.505 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7165e-05 s, 0.0 kB/s 2023-10-06T03:06:19.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.505 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.560 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.560 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.560 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1125e-05 s, 0.0 kB/s 2023-10-06T03:06:19.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.561 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.615 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.616 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0419e-05 s, 0.0 kB/s 2023-10-06T03:06:19.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.617 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.671 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7917e-05 s, 0.0 kB/s 2023-10-06T03:06:19.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.671 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.727 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1582e-05 s, 0.0 kB/s 2023-10-06T03:06:19.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.728 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.784 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1655e-05 s, 0.0 kB/s 2023-10-06T03:06:19.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.785 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.840 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.841 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.841 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1575e-05 s, 0.0 kB/s 2023-10-06T03:06:19.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.841 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.896 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.897 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.897 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8548e-05 s, 0.0 kB/s 2023-10-06T03:06:19.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.897 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:19.952 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:19.953 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2908e-05 s, 0.0 kB/s 2023-10-06T03:06:19.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:19.953 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 5.0698e-05 s, 0.0 kB/s 2023-10-06T03:06:20.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.010 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.065 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.065 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1441e-05 s, 0.0 kB/s 2023-10-06T03:06:20.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.065 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.121 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9779e-05 s, 0.0 kB/s 2023-10-06T03:06:20.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.122 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.891e-05 s, 0.0 kB/s 2023-10-06T03:06:20.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.177 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.233 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.746e-05 s, 0.0 kB/s 2023-10-06T03:06:20.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.234 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.289 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0849e-05 s, 0.0 kB/s 2023-10-06T03:06:20.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.289 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.346 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6919e-05 s, 0.0 kB/s 2023-10-06T03:06:20.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.347 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.402 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.402 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9862e-05 s, 0.0 kB/s 2023-10-06T03:06:20.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.402 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8635e-05 s, 0.0 kB/s 2023-10-06T03:06:20.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.458 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.512 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1484e-05 s, 0.0 kB/s 2023-10-06T03:06:20.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.568 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.569 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2686e-05 s, 0.0 kB/s 2023-10-06T03:06:20.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.624 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.625 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9373e-05 s, 0.0 kB/s 2023-10-06T03:06:20.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.625 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.680 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.995e-05 s, 0.0 kB/s 2023-10-06T03:06:20.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.680 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.738 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0369e-05 s, 0.0 kB/s 2023-10-06T03:06:20.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.739 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.794 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.794 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.794 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1428e-05 s, 0.0 kB/s 2023-10-06T03:06:20.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.795 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.849 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.850 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.850 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.004e-05 s, 0.0 kB/s 2023-10-06T03:06:20.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.850 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.905 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.905 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.905 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2049e-05 s, 0.0 kB/s 2023-10-06T03:06:20.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.906 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:20.961 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:20.961 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:20.961 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0646e-05 s, 0.0 kB/s 2023-10-06T03:06:20.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:20.962 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.017 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.017 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.339e-05 s, 0.0 kB/s 2023-10-06T03:06:21.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.018 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.073 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.073 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.073 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3703e-05 s, 0.0 kB/s 2023-10-06T03:06:21.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.073 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.129 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.914e-05 s, 0.0 kB/s 2023-10-06T03:06:21.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.129 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.185 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4207e-05 s, 0.0 kB/s 2023-10-06T03:06:21.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.186 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.241 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8417e-05 s, 0.0 kB/s 2023-10-06T03:06:21.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.241 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9821e-05 s, 0.0 kB/s 2023-10-06T03:06:21.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.353 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9736e-05 s, 0.0 kB/s 2023-10-06T03:06:21.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8438e-05 s, 0.0 kB/s 2023-10-06T03:06:21.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.409 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.472 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.472 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4133e-05 s, 0.0 kB/s 2023-10-06T03:06:21.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.472 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.528 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2165e-05 s, 0.0 kB/s 2023-10-06T03:06:21.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.529 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.584 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.584 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.584 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.89e-05 s, 0.0 kB/s 2023-10-06T03:06:21.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.584 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.640 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2663e-05 s, 0.0 kB/s 2023-10-06T03:06:21.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.641 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1275e-05 s, 0.0 kB/s 2023-10-06T03:06:21.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.696 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.752 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.753 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9611e-05 s, 0.0 kB/s 2023-10-06T03:06:21.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.753 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.808 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.808 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.809 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0551e-05 s, 0.0 kB/s 2023-10-06T03:06:21.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.809 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1939e-05 s, 0.0 kB/s 2023-10-06T03:06:21.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.865 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.920 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.921 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2756e-05 s, 0.0 kB/s 2023-10-06T03:06:21.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.921 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:21.976 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:21.977 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:21.977 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2968e-05 s, 0.0 kB/s 2023-10-06T03:06:21.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:21.977 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.033 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2323e-05 s, 0.0 kB/s 2023-10-06T03:06:22.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.033 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.090 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1888e-05 s, 0.0 kB/s 2023-10-06T03:06:22.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.090 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.146 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2759e-05 s, 0.0 kB/s 2023-10-06T03:06:22.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.147 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.202 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1524e-05 s, 0.0 kB/s 2023-10-06T03:06:22.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.203 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.258 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.259 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1671e-05 s, 0.0 kB/s 2023-10-06T03:06:22.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.259 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.314 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.349e-05 s, 0.0 kB/s 2023-10-06T03:06:22.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.315 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.370 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.371 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.046e-05 s, 0.0 kB/s 2023-10-06T03:06:22.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.371 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.426 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.427 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.427 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.5049e-05 s, 0.0 kB/s 2023-10-06T03:06:22.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.427 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.483 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2081e-05 s, 0.0 kB/s 2023-10-06T03:06:22.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.484 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.541 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.541 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2246e-05 s, 0.0 kB/s 2023-10-06T03:06:22.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.542 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.600 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.151e-05 s, 0.0 kB/s 2023-10-06T03:06:22.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2538e-05 s, 0.0 kB/s 2023-10-06T03:06:22.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.714 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.714 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1794e-05 s, 0.0 kB/s 2023-10-06T03:06:22.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.714 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.770 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.771 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2844e-05 s, 0.0 kB/s 2023-10-06T03:06:22.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.771 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.827 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0479e-05 s, 0.0 kB/s 2023-10-06T03:06:22.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.828 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.883 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.883 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1866e-05 s, 0.0 kB/s 2023-10-06T03:06:22.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.884 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.939 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.939 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2747e-05 s, 0.0 kB/s 2023-10-06T03:06:22.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.940 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:22.996 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1914e-05 s, 0.0 kB/s 2023-10-06T03:06:22.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:22.997 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.052 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.052 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.052 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1395e-05 s, 0.0 kB/s 2023-10-06T03:06:23.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.053 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.108 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.109 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6749e-05 s, 0.0 kB/s 2023-10-06T03:06:23.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.109 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.164 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.165 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1682e-05 s, 0.0 kB/s 2023-10-06T03:06:23.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.165 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.219 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.220 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1983e-05 s, 0.0 kB/s 2023-10-06T03:06:23.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.221 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.277 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.277 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.278 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0079e-05 s, 0.0 kB/s 2023-10-06T03:06:23.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.278 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.333 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2592e-05 s, 0.0 kB/s 2023-10-06T03:06:23.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.334 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.389 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.389 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.389 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8127e-05 s, 0.0 kB/s 2023-10-06T03:06:23.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.390 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.444 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9808e-05 s, 0.0 kB/s 2023-10-06T03:06:23.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.445 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.501 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.502 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2641e-05 s, 0.0 kB/s 2023-10-06T03:06:23.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.502 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.557 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2376e-05 s, 0.0 kB/s 2023-10-06T03:06:23.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.557 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.613 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3484e-05 s, 0.0 kB/s 2023-10-06T03:06:23.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.613 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.669 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.670 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2008e-05 s, 0.0 kB/s 2023-10-06T03:06:23.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.670 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.725 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2841e-05 s, 0.0 kB/s 2023-10-06T03:06:23.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.725 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.781 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.088e-05 s, 0.0 kB/s 2023-10-06T03:06:23.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.781 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.837 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.837 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.837 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8403e-05 s, 0.0 kB/s 2023-10-06T03:06:23.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.838 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.893 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.893 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.893 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0204e-05 s, 0.0 kB/s 2023-10-06T03:06:23.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.893 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1794e-05 s, 0.0 kB/s 2023-10-06T03:06:23.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:23.949 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.005 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.85e-05 s, 0.0 kB/s 2023-10-06T03:06:24.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.005 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.061 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.021e-05 s, 0.0 kB/s 2023-10-06T03:06:24.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.061 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.116 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7971e-05 s, 0.0 kB/s 2023-10-06T03:06:24.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.116 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.172 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.172 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.962e-05 s, 0.0 kB/s 2023-10-06T03:06:24.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.172 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.228 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.228 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.229 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3446e-05 s, 0.0 kB/s 2023-10-06T03:06:24.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.229 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.284 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6443e-05 s, 0.0 kB/s 2023-10-06T03:06:24.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.284 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.339 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.339 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8724e-05 s, 0.0 kB/s 2023-10-06T03:06:24.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.340 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.395 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.396 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.396 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7862e-05 s, 0.0 kB/s 2023-10-06T03:06:24.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.397 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.451 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.452 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9249e-05 s, 0.0 kB/s 2023-10-06T03:06:24.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.452 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.507 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.507 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1999e-05 s, 0.0 kB/s 2023-10-06T03:06:24.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.507 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8159e-05 s, 0.0 kB/s 2023-10-06T03:06:24.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.564 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.620 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.620 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2655e-05 s, 0.0 kB/s 2023-10-06T03:06:24.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.620 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.676 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2147e-05 s, 0.0 kB/s 2023-10-06T03:06:24.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.676 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.732 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3116e-05 s, 0.0 kB/s 2023-10-06T03:06:24.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.733 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.788 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.788 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.788 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2853e-05 s, 0.0 kB/s 2023-10-06T03:06:24.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.789 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.843 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.844 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.193e-05 s, 0.0 kB/s 2023-10-06T03:06:24.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.844 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.899 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.900 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.900 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1098e-05 s, 0.0 kB/s 2023-10-06T03:06:24.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.900 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:24.956 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:24.956 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:24.956 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3635e-05 s, 0.0 kB/s 2023-10-06T03:06:24.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:24.956 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.012 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.012 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.013 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3622e-05 s, 0.0 kB/s 2023-10-06T03:06:25.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.013 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.067 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.068 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.068 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0481e-05 s, 0.0 kB/s 2023-10-06T03:06:25.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.068 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.123 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.123 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0829e-05 s, 0.0 kB/s 2023-10-06T03:06:25.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.123 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.179 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0473e-05 s, 0.0 kB/s 2023-10-06T03:06:25.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.179 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.234 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.235 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.235 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7939e-05 s, 0.0 kB/s 2023-10-06T03:06:25.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.235 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.290 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0705e-05 s, 0.0 kB/s 2023-10-06T03:06:25.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.291 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.345 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.346 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9758e-05 s, 0.0 kB/s 2023-10-06T03:06:25.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.346 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.400 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.401 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8771e-05 s, 0.0 kB/s 2023-10-06T03:06:25.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.401 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.457 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.457 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1374e-05 s, 0.0 kB/s 2023-10-06T03:06:25.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.457 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.513 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.513 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.6748e-05 s, 0.0 kB/s 2023-10-06T03:06:25.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.513 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.569 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.569 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.569 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3259e-05 s, 0.0 kB/s 2023-10-06T03:06:25.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.569 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.625 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.625 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7428e-05 s, 0.0 kB/s 2023-10-06T03:06:25.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.626 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.681 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0754e-05 s, 0.0 kB/s 2023-10-06T03:06:25.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.681 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.737 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3137e-05 s, 0.0 kB/s 2023-10-06T03:06:25.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.738 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.793 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8194e-05 s, 0.0 kB/s 2023-10-06T03:06:25.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1324e-05 s, 0.0 kB/s 2023-10-06T03:06:25.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.848 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.086e-05 s, 0.0 kB/s 2023-10-06T03:06:25.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.904 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:25.960 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.225e-05 s, 0.0 kB/s 2023-10-06T03:06:25.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:25.961 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.016 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1482e-05 s, 0.0 kB/s 2023-10-06T03:06:26.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.016 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.072 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.072 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2975e-05 s, 0.0 kB/s 2023-10-06T03:06:26.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.073 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.128 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8191e-05 s, 0.0 kB/s 2023-10-06T03:06:26.128 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.128 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.184 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.184 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0688e-05 s, 0.0 kB/s 2023-10-06T03:06:26.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.184 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.240 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.240 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.058e-05 s, 0.0 kB/s 2023-10-06T03:06:26.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.240 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.296 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.296 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.084e-05 s, 0.0 kB/s 2023-10-06T03:06:26.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.297 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.352 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.324e-05 s, 0.0 kB/s 2023-10-06T03:06:26.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.408 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0989e-05 s, 0.0 kB/s 2023-10-06T03:06:26.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.409 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.464 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4308e-05 s, 0.0 kB/s 2023-10-06T03:06:26.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.464 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.520 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.520 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.520 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9313e-05 s, 0.0 kB/s 2023-10-06T03:06:26.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.520 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.586 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.586 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.586 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3144e-05 s, 0.0 kB/s 2023-10-06T03:06:26.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.586 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.642 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.642 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3228e-05 s, 0.0 kB/s 2023-10-06T03:06:26.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.643 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.698 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0063e-05 s, 0.0 kB/s 2023-10-06T03:06:26.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.699 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.754 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.754 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.755 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9118e-05 s, 0.0 kB/s 2023-10-06T03:06:26.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.755 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.810 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.810 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1104e-05 s, 0.0 kB/s 2023-10-06T03:06:26.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.810 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.866 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.866 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.866 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1211e-05 s, 0.0 kB/s 2023-10-06T03:06:26.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.867 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.922 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.922 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.922 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1976e-05 s, 0.0 kB/s 2023-10-06T03:06:26.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.923 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:26.978 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:26.979 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:26.979 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1729e-05 s, 0.0 kB/s 2023-10-06T03:06:26.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:26.979 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0906e-05 s, 0.0 kB/s 2023-10-06T03:06:27.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.035 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.090 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3124e-05 s, 0.0 kB/s 2023-10-06T03:06:27.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.091 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.146 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.146 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0159e-05 s, 0.0 kB/s 2023-10-06T03:06:27.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.147 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.201 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.202 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.202 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2651e-05 s, 0.0 kB/s 2023-10-06T03:06:27.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.202 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.257 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9582e-05 s, 0.0 kB/s 2023-10-06T03:06:27.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.257 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.312 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.313 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.313 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0079e-05 s, 0.0 kB/s 2023-10-06T03:06:27.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.313 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.368 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.126e-05 s, 0.0 kB/s 2023-10-06T03:06:27.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.368 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.424 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.424 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.424 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9849e-05 s, 0.0 kB/s 2023-10-06T03:06:27.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.424 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1628e-05 s, 0.0 kB/s 2023-10-06T03:06:27.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.480 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.536 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.537 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4231e-05 s, 0.0 kB/s 2023-10-06T03:06:27.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.537 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.591 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.2253e-05 s, 0.0 kB/s 2023-10-06T03:06:27.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.593 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.649 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.649 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.650 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2084e-05 s, 0.0 kB/s 2023-10-06T03:06:27.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.650 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.705 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.706 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4298e-05 s, 0.0 kB/s 2023-10-06T03:06:27.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.706 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.761 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.762 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1456e-05 s, 0.0 kB/s 2023-10-06T03:06:27.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.762 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.817 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.817 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.369e-05 s, 0.0 kB/s 2023-10-06T03:06:27.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.818 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.873 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.874 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0714e-05 s, 0.0 kB/s 2023-10-06T03:06:27.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.874 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.929 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0457e-05 s, 0.0 kB/s 2023-10-06T03:06:27.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.930 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:27.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:27.985 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:27.985 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3851e-05 s, 0.0 kB/s 2023-10-06T03:06:27.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:27.986 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.041 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8665e-05 s, 0.0 kB/s 2023-10-06T03:06:28.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.041 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.097 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.097 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0938e-05 s, 0.0 kB/s 2023-10-06T03:06:28.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.097 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.153 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.153 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0256e-05 s, 0.0 kB/s 2023-10-06T03:06:28.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.153 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.208 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.209 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9701e-05 s, 0.0 kB/s 2023-10-06T03:06:28.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.209 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.264 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.264 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.264 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.993e-05 s, 0.0 kB/s 2023-10-06T03:06:28.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.265 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.320 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.320 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0274e-05 s, 0.0 kB/s 2023-10-06T03:06:28.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.320 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.376 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.376 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4696e-05 s, 0.0 kB/s 2023-10-06T03:06:28.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.377 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.432 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.432 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9449e-05 s, 0.0 kB/s 2023-10-06T03:06:28.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.432 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.488 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.488 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.488 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7445e-05 s, 0.0 kB/s 2023-10-06T03:06:28.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.488 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.543 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.544 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.544 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9253e-05 s, 0.0 kB/s 2023-10-06T03:06:28.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.544 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.600 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.600 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2253e-05 s, 0.0 kB/s 2023-10-06T03:06:28.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.601 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.657 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0581e-05 s, 0.0 kB/s 2023-10-06T03:06:28.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.658 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.712 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0719e-05 s, 0.0 kB/s 2023-10-06T03:06:28.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.713 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.769 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.769 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2955e-05 s, 0.0 kB/s 2023-10-06T03:06:28.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.770 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.825 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.825 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.161e-05 s, 0.0 kB/s 2023-10-06T03:06:28.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.825 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.881 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.881 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4932e-05 s, 0.0 kB/s 2023-10-06T03:06:28.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.881 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.937 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1811e-05 s, 0.0 kB/s 2023-10-06T03:06:28.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.938 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:28.993 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:28.994 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1481e-05 s, 0.0 kB/s 2023-10-06T03:06:28.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:28.994 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2837e-05 s, 0.0 kB/s 2023-10-06T03:06:29.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.049 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8983e-05 s, 0.0 kB/s 2023-10-06T03:06:29.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.106 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.161 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1308e-05 s, 0.0 kB/s 2023-10-06T03:06:29.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.162 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.217 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8261e-05 s, 0.0 kB/s 2023-10-06T03:06:29.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.218 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.274 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.274 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.274 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4341e-05 s, 0.0 kB/s 2023-10-06T03:06:29.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.275 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.330 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.331 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.247e-05 s, 0.0 kB/s 2023-10-06T03:06:29.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.331 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.386 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.387 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9414e-05 s, 0.0 kB/s 2023-10-06T03:06:29.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.387 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.442 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2544e-05 s, 0.0 kB/s 2023-10-06T03:06:29.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.443 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.498 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.498 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3674e-05 s, 0.0 kB/s 2023-10-06T03:06:29.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.499 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.554 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.932e-05 s, 0.0 kB/s 2023-10-06T03:06:29.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.555 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.610 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.610 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.831e-05 s, 0.0 kB/s 2023-10-06T03:06:29.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.611 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.666 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.666 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.666 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.022e-05 s, 0.0 kB/s 2023-10-06T03:06:29.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.666 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.722 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2707e-05 s, 0.0 kB/s 2023-10-06T03:06:29.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.722 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3469e-05 s, 0.0 kB/s 2023-10-06T03:06:29.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.779 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.836 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.836 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2967e-05 s, 0.0 kB/s 2023-10-06T03:06:29.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.836 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4418e-05 s, 0.0 kB/s 2023-10-06T03:06:29.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.892 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:29.947 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:29.947 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:29.947 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.8814e-05 s, 0.0 kB/s 2023-10-06T03:06:29.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:29.948 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.004 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.005 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.005 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2087e-05 s, 0.0 kB/s 2023-10-06T03:06:30.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.005 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.061 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.6621e-05 s, 0.0 kB/s 2023-10-06T03:06:30.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.061 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.117 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.117 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.118 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4225e-05 s, 0.0 kB/s 2023-10-06T03:06:30.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.118 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.173 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.173 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.173 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3989e-05 s, 0.0 kB/s 2023-10-06T03:06:30.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.173 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.229 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.230 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.230 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.7123e-05 s, 0.0 kB/s 2023-10-06T03:06:30.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.230 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.285 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1892e-05 s, 0.0 kB/s 2023-10-06T03:06:30.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.286 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.341 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.342 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.342 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2121e-05 s, 0.0 kB/s 2023-10-06T03:06:30.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.343 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.397 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.397 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.398 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9331e-05 s, 0.0 kB/s 2023-10-06T03:06:30.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.398 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.453 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.36e-05 s, 0.0 kB/s 2023-10-06T03:06:30.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.454 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.509 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.510 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.510 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3207e-05 s, 0.0 kB/s 2023-10-06T03:06:30.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.510 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1307e-05 s, 0.0 kB/s 2023-10-06T03:06:30.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.565 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.621 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.621 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.621 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2814e-05 s, 0.0 kB/s 2023-10-06T03:06:30.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.622 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.677 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0529e-05 s, 0.0 kB/s 2023-10-06T03:06:30.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.678 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.733 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.733 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.733 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3404e-05 s, 0.0 kB/s 2023-10-06T03:06:30.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.734 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.789 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.789 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.789 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1339e-05 s, 0.0 kB/s 2023-10-06T03:06:30.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.790 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.845 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.845 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1124e-05 s, 0.0 kB/s 2023-10-06T03:06:30.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.846 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.7166e-05 s, 0.0 kB/s 2023-10-06T03:06:30.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.901 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:30.957 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2024e-05 s, 0.0 kB/s 2023-10-06T03:06:30.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:30.957 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.013 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.014 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.014 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.946e-05 s, 0.0 kB/s 2023-10-06T03:06:31.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.014 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.069 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9256e-05 s, 0.0 kB/s 2023-10-06T03:06:31.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.069 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.125 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1596e-05 s, 0.0 kB/s 2023-10-06T03:06:31.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.126 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.181 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.182 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.23e-05 s, 0.0 kB/s 2023-10-06T03:06:31.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.182 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.237 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.237 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-10-06T03:06:31.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.237 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.294 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.294 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.294 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1821e-05 s, 0.0 kB/s 2023-10-06T03:06:31.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.295 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.349 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.349 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3948e-05 s, 0.0 kB/s 2023-10-06T03:06:31.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.350 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.405 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0287e-05 s, 0.0 kB/s 2023-10-06T03:06:31.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.406 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.460 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.461 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 2.9635e-05 s, 0.0 kB/s 2023-10-06T03:06:31.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.461 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.516 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.516 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1907e-05 s, 0.0 kB/s 2023-10-06T03:06:31.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.517 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.572 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.572 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.0744e-05 s, 0.0 kB/s 2023-10-06T03:06:31.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.573 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1556e-05 s, 0.0 kB/s 2023-10-06T03:06:31.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.630 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.720 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 4.074e-05 s, 0.0 kB/s 2023-10-06T03:06:31.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.721 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.736 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.737 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.4412e-05 s, 0.0 kB/s 2023-10-06T03:06:31.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.737 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.792 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.793 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3653e-05 s, 0.0 kB/s 2023-10-06T03:06:31.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.793 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.848 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.2959e-05 s, 0.0 kB/s 2023-10-06T03:06:31.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.849 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.904 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.904 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.1124e-05 s, 0.0 kB/s 2023-10-06T03:06:31.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:31.905 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000163772/a8df94d0-d22e-4149-b76b-033211511e17/subvolume_file.2999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-06T03:06:31.960 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records in 2023-10-06T03:06:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:0+0 records out 2023-10-06T03:06:31.961 INFO:teuthology.orchestra.run.smithi092.stderr:0 bytes copied, 3.3332e-05 s, 0.0 kB/s 2023-10-06T03:06:31.961 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000163772 2023-10-06T03:06:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.116+0000 7fdaf9a06700 1 Processor -- start 2023-10-06T03:06:32.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.116+0000 7fdaf9a06700 1 -- start start 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf9a06700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159890 0x7fdaf415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf9a06700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdaf4069560 0x7fdaf40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf9a06700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf415a160 con 0x7fdaf4159c60 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf9a06700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf4069f20 con 0x7fdaf4069560 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf9a06700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40d13a0 con 0x7fdaf4159890 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdaf4069560 0x7fdaf40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdaf4069560 0x7fdaf40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58700/0 (socket says 172.21.15.92:58700) 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.117+0000 7fdaf37fe700 1 -- 172.21.15.92:0/161745357 learned_addr learned my addr 172.21.15.92:0/161745357 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159890 0x7fdaf415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf37fe700 1 -- 172.21.15.92:0/161745357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159890 msgr2=0x7fdaf415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159890 0x7fdaf415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf37fe700 1 -- 172.21.15.92:0/161745357 >> v1:172.21.15.136:6789/0 conn(0x7fdaf4159c60 legacy=0x7fdaf415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf37fe700 1 -- 172.21.15.92:0/161745357 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf415e140 con 0x7fdaf4069560 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159890 0x7fdaf415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf40699e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0099b0 tx=0x7fdaec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabda4be0622ac41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/161745357 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00b860 con 0x7fdaf4069560 2023-10-06T03:06:32.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/161745357 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec00b9c0 con 0x7fdaf4069560 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf1ffb700 1 -- 172.21.15.92:0/161745357 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec00bd60 con 0x7fdaf4069560 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf9a06700 1 -- 172.21.15.92:0/161745357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 msgr2=0x7fdaf40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.118+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf40699e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdaec0099b0 tx=0x7fdaec009250 comp rx=0 tx=0).stop 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.119+0000 7fdaf9a06700 1 -- 172.21.15.92:0/161745357 shutdown_connections 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.119+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf40699e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.119+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/161745357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159890 0x7fdaf415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.119+0000 7fdaf9a06700 1 -- 172.21.15.92:0/161745357 >> 172.21.15.92:0/161745357 conn(0x7fdaf4068e20 msgr2=0x7fdaf40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.119+0000 7fdaf9a06700 1 -- 172.21.15.92:0/161745357 shutdown_connections 2023-10-06T03:06:32.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.119+0000 7fdaf9a06700 1 -- 172.21.15.92:0/161745357 wait complete. 2023-10-06T03:06:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.120+0000 7fdaf9a06700 1 Processor -- start 2023-10-06T03:06:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.120+0000 7fdaf9a06700 1 -- start start 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.120+0000 7fdaf9a06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf4149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.120+0000 7fdaf9a06700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf4159890 0x7fdaf415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.120+0000 7fdaf9a06700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159c60 0x7fdaf415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf9a06700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40b5de0 con 0x7fdaf4069560 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf9a06700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4169a30 con 0x7fdaf4159890 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf9a06700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf416a5a0 con 0x7fdaf4159c60 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf4149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf4149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58716/0 (socket says 172.21.15.92:58716) 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/468645220 learned_addr learned my addr 172.21.15.92:0/468645220 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf37fe700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159c60 0x7fdaf415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf4159890 0x7fdaf415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159c60 msgr2=0x7fdaf415eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159c60 0x7fdaf415eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf4159890 msgr2=0x7fdaf415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf4159890 0x7fdaf415e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 -- 172.21.15.92:0/468645220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec0094f0 con 0x7fdaf4069560 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf2ffd700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf4149170 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdae400d1d0 tx=0x7fdae4009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae400dcb0 con 0x7fdaf4069560 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae400f040 con 0x7fdaf4069560 2023-10-06T03:06:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf4169c70 con 0x7fdaf4069560 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae4015670 con 0x7fdaf4069560 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.121+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf416a230 con 0x7fdaf4069560 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.122+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf416a880 con 0x7fdaf4069560 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.123+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fdae4015a60 con 0x7fdaf4069560 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.123+0000 7fdadaffd700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdae0066fc0 0x7fdae0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.123+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdae0066fc0 0x7fdae0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.123+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdae4084aa0 con 0x7fdaf4069560 2023-10-06T03:06:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.124+0000 7fdaf27fc700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdae0066fc0 0x7fdae0069470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40d0f60 tx=0x7fdae8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.128+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdae404e5a0 con 0x7fdaf4069560 2023-10-06T03:06:32.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.340+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000163772", "target": ["mon-mgr", ""]}) v1 -- 0x7fdaf404c3c0 con 0x7fdae0066fc0 2023-10-06T03:06:32.345 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.344+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.346 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.344+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.346 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.344+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.346 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.344+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.346 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.344+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.394+0000 7fdadaffd700 1 -- 172.21.15.92:0/468645220 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdaf404c3c0 con 0x7fdae0066fc0 2023-10-06T03:06:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdae0066fc0 msgr2=0x7fdae0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdae0066fc0 0x7fdae0069470 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40d0f60 tx=0x7fdae8007ec0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 msgr2=0x7fdaf4149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf4149170 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdae400d1d0 tx=0x7fdae4009a90 comp rx=0 tx=0).stop 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 shutdown_connections 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdae0066fc0 0x7fdae0069470 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaf4069560 0x7fdaf4149170 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdaf4159c60 0x7fdaf415eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 --2- 172.21.15.92:0/468645220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaf4159890 0x7fdaf415e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 >> 172.21.15.92:0/468645220 conn(0x7fdaf4068e20 msgr2=0x7fdaf40c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 shutdown_connections 2023-10-06T03:06:32.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.396+0000 7fdaf9a06700 1 -- 172.21.15.92:0/468645220 wait complete. 2023-10-06T03:06:32.400 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.397+0000 7fdd11b86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.400 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.397+0000 7fdd11b86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.400 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.397+0000 7fdd11b86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.400 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.397+0000 7fdd11b86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.400 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.397+0000 7fdd11b86700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.411 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.409+0000 7fdd11385700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.411 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.409+0000 7fdd11385700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.411 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.409+0000 7fdd11385700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.411 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.409+0000 7fdd11385700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.411 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:06:32.409+0000 7fdd11385700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:06:32.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-06T03:06:32.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.538+0000 7f8a75bb1700 1 Processor -- start 2023-10-06T03:06:32.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.538+0000 7f8a75bb1700 1 -- start start 2023-10-06T03:06:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a75bb1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8a7006aa40 0x7f8a700cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a75bb1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006bac0 0x7f8a7006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a75bb1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8a7006b350 0x7f8a700c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a75bb1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700cc210 con 0x7f8a7006aa40 2023-10-06T03:06:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a75bb1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700cc4f0 con 0x7f8a7006b350 2023-10-06T03:06:32.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a75bb1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700cc7d0 con 0x7f8a7006bac0 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8a7006aa40 0x7f8a700cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8a7006aa40 0x7f8a700cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58744/0 (socket says 172.21.15.92:58744) 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/1911166919 learned_addr learned my addr 172.21.15.92:0/1911166919 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6ffff700 1 --2- 172.21.15.92:0/1911166919 >> v2:172.21.15.136:3300/0 conn(0x7f8a7006b350 0x7f8a700c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6effd700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006bac0 0x7f8a7006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/1911166919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006bac0 msgr2=0x7f8a7006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006bac0 0x7f8a7006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/1911166919 >> v2:172.21.15.136:3300/0 conn(0x7f8a7006b350 msgr2=0x7f8a700c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 --2- 172.21.15.92:0/1911166919 >> v2:172.21.15.136:3300/0 conn(0x7f8a7006b350 0x7f8a700c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/1911166919 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700ccab0 con 0x7f8a7006aa40 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6ffff700 1 --2- 172.21.15.92:0/1911166919 >> v2:172.21.15.136:3300/0 conn(0x7f8a7006b350 0x7f8a700c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.539+0000 7f8a6effd700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006bac0 0x7f8a7006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.540+0000 7f8a6f7fe700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a700cbc00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c7c40 tx=0x7f8a60009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533def00c135f546 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.540+0000 7f8a6e7fc700 1 -- 172.21.15.92:0/1911166919 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60005c10 con 0x7f8a7006aa40 2023-10-06T03:06:32.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.540+0000 7f8a6e7fc700 1 -- 172.21.15.92:0/1911166919 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60005d70 con 0x7f8a7006aa40 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.540+0000 7f8a6e7fc700 1 -- 172.21.15.92:0/1911166919 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000b880 con 0x7f8a7006aa40 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.540+0000 7f8a75bb1700 1 -- 172.21.15.92:0/1911166919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 msgr2=0x7f8a700cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.540+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a700cbc00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8a700c7c40 tx=0x7f8a60009220 comp rx=0 tx=0).stop 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 -- 172.21.15.92:0/1911166919 shutdown_connections 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/1911166919 >> v2:172.21.15.136:3300/0 conn(0x7f8a7006b350 0x7f8a700c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a700cbc00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/1911166919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006bac0 0x7f8a7006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 -- 172.21.15.92:0/1911166919 >> 172.21.15.92:0/1911166919 conn(0x7f8a700cd670 msgr2=0x7f8a700cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 -- 172.21.15.92:0/1911166919 shutdown_connections 2023-10-06T03:06:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.541+0000 7f8a75bb1700 1 -- 172.21.15.92:0/1911166919 wait complete. 2023-10-06T03:06:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.542+0000 7f8a75bb1700 1 Processor -- start 2023-10-06T03:06:32.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.542+0000 7f8a75bb1700 1 -- start start 2023-10-06T03:06:32.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a75bb1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a7014ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a75bb1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006b350 0x7f8a701673d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a75bb1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a7006bac0 0x7f8a70169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a75bb1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70166990 con 0x7f8a7006aa40 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a75bb1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70166f40 con 0x7f8a7006bac0 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a75bb1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7014f320 con 0x7f8a7006b350 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a7014ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a7014ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58750/0 (socket says 172.21.15.92:58750) 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/3443753125 learned_addr learned my addr 172.21.15.92:0/3443753125 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:32.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6ffff700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a7006bac0 0x7f8a70169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6effd700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006b350 0x7f8a701673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006b350 msgr2=0x7f8a701673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006b350 0x7f8a701673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a7006bac0 msgr2=0x7f8a70169fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a7006bac0 0x7f8a70169fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 -- 172.21.15.92:0/3443753125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60009480 con 0x7f8a7006aa40 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6effd700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006b350 0x7f8a701673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.543+0000 7f8a6f7fe700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a7014ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b3c0 tx=0x7f8a60016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.544+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60011920 con 0x7f8a7006aa40 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.544+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60020070 con 0x7f8a7006aa40 2023-10-06T03:06:32.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.544+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7016a9c0 con 0x7f8a7006aa40 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.544+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60011cb0 con 0x7f8a7006aa40 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.544+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a70166bd0 con 0x7f8a7006aa40 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.545+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70167220 con 0x7f8a7006aa40 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.546+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8a60022030 con 0x7f8a7006aa40 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.546+0000 7f8a577fe700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8a5c066fc0 0x7f8a5c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.546+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a600866c0 con 0x7f8a7006aa40 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.546+0000 7f8a6effd700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8a5c066fc0 0x7f8a5c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.547+0000 7f8a6effd700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8a5c066fc0 0x7f8a5c069470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:32.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.550+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a60053860 con 0x7f8a7006aa40 2023-10-06T03:06:32.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.784+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a70167080 con 0x7f8a5c066fc0 2023-10-06T03:06:32.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.796+0000 7f8a577fe700 1 -- 172.21.15.92:0/3443753125 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+538 (secure 0 0 0) 0x7f8a64009e70 con 0x7f8a5c066fc0 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_addrs": [ 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6789", 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6789", 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6790" 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "pending_subvolume_deletions": 1, 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": { 2023-10-06T03:06:32.798 INFO:teuthology.orchestra.run.smithi092.stdout: "data": [ 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 365965180928, 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_data", 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 8192 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": [ 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 365965180928, 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_metadata", 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 20766720 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:32.799 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:06:32.800 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:32.800 INFO:teuthology.orchestra.run.smithi092.stdout: "used_size": 145 2023-10-06T03:06:32.800 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8a5c066fc0 msgr2=0x7f8a5c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8a5c066fc0 0x7f8a5c069470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 msgr2=0x7f8a7014ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a7014ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b3c0 tx=0x7f8a60016000 comp rx=0 tx=0).stop 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 shutdown_connections 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8a5c066fc0 0x7f8a5c069470 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a7006aa40 0x7f8a7014ede0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a7006bac0 0x7f8a70169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 --2- 172.21.15.92:0/3443753125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8a7006b350 0x7f8a701673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 >> 172.21.15.92:0/3443753125 conn(0x7f8a700cd670 msgr2=0x7f8a701523d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 shutdown_connections 2023-10-06T03:06:32.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:32.798+0000 7f8a75bb1700 1 -- 172.21.15.92:0/3443753125 wait complete. 2023-10-06T03:06:32.815 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:06:32.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:06:32.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:06:32.899 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:06:32.899 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:06:32.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:32.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:06:33.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:06:33.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:33.422 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:06:33.437 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:06:33.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:06:33.437 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:06:33.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:06:33.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:34.000 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:06:34.017 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.165+0000 7ff770eb2700 1 Processor -- start 2023-10-06T03:06:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.166+0000 7ff770eb2700 1 -- start start 2023-10-06T03:06:34.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff770eb2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff76c06bcd0 0x7ff76c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff770eb2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff770eb2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c15d7e0 con 0x7ff76c06c0a0 2023-10-06T03:06:34.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff770eb2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c15dac0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff770eb2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15aa30 con 0x7ff76c15df50 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff769d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff76c06c0a0 0x7ff76c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39320/0 (socket says 172.21.15.92:39320) 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff769d9b700 1 -- 172.21.15.92:0/2627932353 learned_addr learned my addr 172.21.15.92:0/2627932353 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76a59c700 1 --2- 172.21.15.92:0/2627932353 >> v2:172.21.15.136:3300/0 conn(0x7ff76c06bcd0 0x7ff76c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76ad9d700 1 --2- 172.21.15.92:0/2627932353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76959a700 1 -- 172.21.15.92:0/2627932353 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582248673 0 0) 0x7ff76c15d7e0 con 0x7ff76c06c0a0 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76959a700 1 -- 172.21.15.92:0/2627932353 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff754003570 con 0x7ff76c06c0a0 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76959a700 1 -- 172.21.15.92:0/2627932353 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 739512131 0 0) 0x7ff754003570 con 0x7ff76c06c0a0 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76ad9d700 1 -- 172.21.15.92:0/2627932353 >> v2:172.21.15.136:3300/0 conn(0x7ff76c06bcd0 msgr2=0x7ff76c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76ad9d700 1 --2- 172.21.15.92:0/2627932353 >> v2:172.21.15.136:3300/0 conn(0x7ff76c06bcd0 0x7ff76c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76ad9d700 1 -- 172.21.15.92:0/2627932353 >> v1:172.21.15.92:6789/0 conn(0x7ff76c06c0a0 legacy=0x7ff76c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.167+0000 7ff76ad9d700 1 -- 172.21.15.92:0/2627932353 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c15ad10 con 0x7ff76c15df50 2023-10-06T03:06:34.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.168+0000 7ff76ad9d700 1 --2- 172.21.15.92:0/2627932353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c15e3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff764009e60 tx=0x7ff764009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd2881b2e8335cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:34.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.168+0000 7ff76959a700 1 -- 172.21.15.92:0/2627932353 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76400e070 con 0x7ff76c15df50 2023-10-06T03:06:34.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.168+0000 7ff76959a700 1 -- 172.21.15.92:0/2627932353 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76400b7a0 con 0x7ff76c15df50 2023-10-06T03:06:34.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.168+0000 7ff76959a700 1 -- 172.21.15.92:0/2627932353 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760001470 con 0x7ff76c15df50 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 -- 172.21.15.92:0/2627932353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 msgr2=0x7ff76c15e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 --2- 172.21.15.92:0/2627932353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c15e3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff764009e60 tx=0x7ff764009780 comp rx=0 tx=0).stop 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 -- 172.21.15.92:0/2627932353 shutdown_connections 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 --2- 172.21.15.92:0/2627932353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c15e3f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 --2- 172.21.15.92:0/2627932353 >> v2:172.21.15.136:3300/0 conn(0x7ff76c06bcd0 0x7ff76c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 -- 172.21.15.92:0/2627932353 >> 172.21.15.92:0/2627932353 conn(0x7ff76c0ce600 msgr2=0x7ff76c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 -- 172.21.15.92:0/2627932353 shutdown_connections 2023-10-06T03:06:34.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.169+0000 7ff770eb2700 1 -- 172.21.15.92:0/2627932353 wait complete. 2023-10-06T03:06:34.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.171+0000 7ff770eb2700 1 Processor -- start 2023-10-06T03:06:34.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.171+0000 7ff770eb2700 1 -- start start 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff770eb2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 0x7ff76c156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff770eb2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c06c0a0 0x7ff76c0c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff770eb2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff770eb2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c14b8b0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff770eb2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15c5d0 con 0x7ff76c06c0a0 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff770eb2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15c8b0 con 0x7ff76c15df50 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff76a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 0x7ff76c156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff76a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 0x7ff76c156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58766/0 (socket says 172.21.15.92:58766) 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff76a59c700 1 -- 172.21.15.92:0/61205065 learned_addr learned my addr 172.21.15.92:0/61205065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff76ad9d700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.172+0000 7ff769d9b700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c06c0a0 0x7ff76c0c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff76a59c700 1 -- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 msgr2=0x7ff76c14b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff76a59c700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c14b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff76a59c700 1 -- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c06c0a0 msgr2=0x7ff76c0c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff76a59c700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c06c0a0 0x7ff76c0c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff76a59c700 1 -- 172.21.15.92:0/61205065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff764009a20 con 0x7ff76c06bcd0 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff76a59c700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 0x7ff76c156410 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00e930 tx=0x7ff75c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c013680 con 0x7ff76c06bcd0 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c0137e0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c0b40a0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c013ba0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.173+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c0b3cc0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.174+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c04c3c0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.175+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff75c00c3e0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.175+0000 7ff75a7fc700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff74c067170 0x7ff74c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.175+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff75c084a00 con 0x7ff76c06bcd0 2023-10-06T03:06:34.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.175+0000 7ff769d9b700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff74c067170 0x7ff74c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.175+0000 7ff769d9b700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff74c067170 0x7ff74c069620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff76c167790 tx=0x7ff760001a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:34.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.180+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff75c051880 con 0x7ff76c06bcd0 2023-10-06T03:06:34.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.494+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff76c15aff0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.494+0000 7ff75a7fc700 1 -- 172.21.15.92:0/61205065 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4599 (secure 0 0 0) 0x7ff75c054ea0 con 0x7ff76c06bcd0 2023-10-06T03:06:34.496 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:34.496 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:29.075556+0000","modified":"2023-10-06T03:03:32.719393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/812630670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":812630670},{"type":"v1","addr":"172.21.15.92:6837","nonce":812630670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:06:34.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff74c067170 msgr2=0x7ff74c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff74c067170 0x7ff74c069620 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff76c167790 tx=0x7ff760001a00 comp rx=0 tx=0).stop 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 msgr2=0x7ff76c156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 0x7ff76c156410 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00e930 tx=0x7ff75c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 shutdown_connections 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff74c067170 0x7ff74c069620 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff76c06bcd0 0x7ff76c156410 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff76c15df50 0x7ff76c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 --2- 172.21.15.92:0/61205065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff76c06c0a0 0x7ff76c0c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.497+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 >> 172.21.15.92:0/61205065 conn(0x7ff76c0ce600 msgr2=0x7ff76c159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.498+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 shutdown_connections 2023-10-06T03:06:34.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.498+0000 7ff770eb2700 1 -- 172.21.15.92:0/61205065 wait complete. 2023-10-06T03:06:34.501 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 39 2023-10-06T03:06:34.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:34.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.638+0000 7f93c8b99700 1 Processor -- start 2023-10-06T03:06:34.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.638+0000 7f93c8b99700 1 -- start start 2023-10-06T03:06:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c8b99700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159890 0x7f93c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c8b99700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93c4069560 0x7f93c40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c8b99700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c415a160 con 0x7f93c4159c60 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c8b99700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c4069f20 con 0x7f93c4069560 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c8b99700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40d13a0 con 0x7f93c4159890 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93c4069560 0x7f93c40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93c4069560 0x7f93c40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58776/0 (socket says 172.21.15.92:58776) 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 -- 172.21.15.92:0/4005911827 learned_addr learned my addr 172.21.15.92:0/4005911827 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c259c700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159890 0x7f93c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 -- 172.21.15.92:0/4005911827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159890 msgr2=0x7f93c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159890 0x7f93c415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 -- 172.21.15.92:0/4005911827 >> v1:172.21.15.136:6789/0 conn(0x7f93c4159c60 legacy=0x7f93c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c159a700 1 -- 172.21.15.92:0/4005911827 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34056051 0 0) 0x7f93c415a160 con 0x7f93c4159c60 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c2d9d700 1 -- 172.21.15.92:0/4005911827 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c415e140 con 0x7f93c4069560 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.639+0000 7f93c259c700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159890 0x7f93c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.640+0000 7f93c2d9d700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4069560 0x7f93c40699e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009f20 tx=0x7f93bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d6c0ce62bfd7b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.640+0000 7f93c159a700 1 -- 172.21.15.92:0/4005911827 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00e070 con 0x7f93c4069560 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.640+0000 7f93c159a700 1 -- 172.21.15.92:0/4005911827 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc00b9d0 con 0x7f93c4069560 2023-10-06T03:06:34.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.640+0000 7f93c159a700 1 -- 172.21.15.92:0/4005911827 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0092e0 con 0x7f93c4069560 2023-10-06T03:06:34.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.640+0000 7f93c8b99700 1 -- 172.21.15.92:0/4005911827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4069560 msgr2=0x7f93c40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.640+0000 7f93c8b99700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4069560 0x7f93c40699e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009f20 tx=0x7f93bc009840 comp rx=0 tx=0).stop 2023-10-06T03:06:34.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.641+0000 7f93c8b99700 1 -- 172.21.15.92:0/4005911827 shutdown_connections 2023-10-06T03:06:34.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.641+0000 7f93c8b99700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4069560 0x7f93c40699e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.641+0000 7f93c8b99700 1 --2- 172.21.15.92:0/4005911827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159890 0x7f93c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.641+0000 7f93c8b99700 1 -- 172.21.15.92:0/4005911827 >> 172.21.15.92:0/4005911827 conn(0x7f93c4068e20 msgr2=0x7f93c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.641+0000 7f93c8b99700 1 -- 172.21.15.92:0/4005911827 shutdown_connections 2023-10-06T03:06:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.641+0000 7f93c8b99700 1 -- 172.21.15.92:0/4005911827 wait complete. 2023-10-06T03:06:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.642+0000 7f93c8b99700 1 Processor -- start 2023-10-06T03:06:34.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.642+0000 7f93c8b99700 1 -- start start 2023-10-06T03:06:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c8b99700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93c4069560 0x7f93c40b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c8b99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 0x7f93c40c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c8b99700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159c60 0x7f93c4145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c8b99700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c99c0 con 0x7f93c4159890 2023-10-06T03:06:34.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c8b99700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c9ca0 con 0x7f93c4069560 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c8b99700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c9f80 con 0x7f93c4159c60 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 0x7f93c40c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 0x7f93c40c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58790/0 (socket says 172.21.15.92:58790) 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 -- 172.21.15.92:0/936285585 learned_addr learned my addr 172.21.15.92:0/936285585 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c259c700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93c4069560 0x7f93c40b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c2d9d700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159c60 0x7f93c4145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 -- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159c60 msgr2=0x7f93c4145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159c60 0x7f93c4145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 -- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93c4069560 msgr2=0x7f93c40b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93c4069560 0x7f93c40b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.643+0000 7f93c1d9b700 1 -- 172.21.15.92:0/936285585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009420 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.644+0000 7f93c1d9b700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 0x7f93c40c9020 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f93b800c7e0 tx=0x7f93b8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.644+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8011590 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.644+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b80116f0 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.644+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc009ae0 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.644+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8011ab0 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.644+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c40ca540 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.645+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c40ca760 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.645+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f93b801b3b0 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.645+0000 7f93b27fc700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a4067170 0x7f93a4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.646+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f93b8083fb0 con 0x7f93c4159890 2023-10-06T03:06:34.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.646+0000 7f93c259c700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a4067170 0x7f93a4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:34.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.646+0000 7f93c259c700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a4067170 0x7f93a4069620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93b40093f0 tx=0x7f93b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:34.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.650+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b8050e30 con 0x7f93c4159890 2023-10-06T03:06:34.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.881+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93c404c3c0 con 0x7f93c4159890 2023-10-06T03:06:34.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.881+0000 7f93b27fc700 1 -- 172.21.15.92:0/936285585 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f93b8017070 con 0x7f93c4159890 2023-10-06T03:06:34.883 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:34.883 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:30.711121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:03:26.724124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:28.041787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:34.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a4067170 msgr2=0x7f93a4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a4067170 0x7f93a4069620 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93b40093f0 tx=0x7f93b4008040 comp rx=0 tx=0).stop 2023-10-06T03:06:34.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 msgr2=0x7f93c40c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:34.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 0x7f93c40c9020 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f93b800c7e0 tx=0x7f93b8007dc0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 shutdown_connections 2023-10-06T03:06:34.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f93a4067170 0x7f93a4069620 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93c4159c60 0x7f93c4145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93c4069560 0x7f93c40b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 --2- 172.21.15.92:0/936285585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c4159890 0x7f93c40c9020 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:34.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 >> 172.21.15.92:0/936285585 conn(0x7f93c4068e20 msgr2=0x7f93c41551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:34.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 shutdown_connections 2023-10-06T03:06:34.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:34.884+0000 7f93c8b99700 1 -- 172.21.15.92:0/936285585 wait complete. 2023-10-06T03:06:34.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:03:26.724124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:03:28.041787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:06:34.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:06:35.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.034+0000 7f7f18efc700 1 Processor -- start 2023-10-06T03:06:35.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.034+0000 7f7f18efc700 1 -- start start 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f18efc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7f140af250 0x7f7f14068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f18efc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f140688d0 0x7f7f14157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f18efc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1406b030 con 0x7f7f14157960 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f18efc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140ac760 con 0x7f7f140688d0 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f18efc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140af830 con 0x7f7f140af250 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f140688d0 0x7f7f14157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7f140688d0 0x7f7f14157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58800/0 (socket says 172.21.15.92:58800) 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 -- 172.21.15.92:0/1051271000 learned_addr learned my addr 172.21.15.92:0/1051271000 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f11d9b700 1 --2- 172.21.15.92:0/1051271000 >> v2:172.21.15.136:3300/0 conn(0x7f7f140af250 0x7f7f14068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 -- 172.21.15.92:0/1051271000 >> v2:172.21.15.136:3300/0 conn(0x7f7f140af250 msgr2=0x7f7f14068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 --2- 172.21.15.92:0/1051271000 >> v2:172.21.15.136:3300/0 conn(0x7f7f140af250 0x7f7f14068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 -- 172.21.15.92:0/1051271000 >> v1:172.21.15.136:6789/0 conn(0x7f7f14157960 legacy=0x7f7f1406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 -- 172.21.15.92:0/1051271000 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1406bbe0 con 0x7f7f140688d0 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f11d9b700 1 --2- 172.21.15.92:0/1051271000 >> v2:172.21.15.136:3300/0 conn(0x7f7f140af250 0x7f7f14068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f12d9d700 1 --2- 172.21.15.92:0/1051271000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140688d0 0x7f7f14157d30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7f1406fe40 tx=0x7f7f0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d8067f71c879c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.035+0000 7f7f1159a700 1 -- 172.21.15.92:0/1051271000 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00e070 con 0x7f7f140688d0 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f1159a700 1 -- 172.21.15.92:0/1051271000 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c00ba10 con 0x7f7f140688d0 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f1159a700 1 -- 172.21.15.92:0/1051271000 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0092e0 con 0x7f7f140688d0 2023-10-06T03:06:35.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 -- 172.21.15.92:0/1051271000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140688d0 msgr2=0x7f7f14157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 --2- 172.21.15.92:0/1051271000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140688d0 0x7f7f14157d30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7f1406fe40 tx=0x7f7f0c009840 comp rx=0 tx=0).stop 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 -- 172.21.15.92:0/1051271000 shutdown_connections 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 --2- 172.21.15.92:0/1051271000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140688d0 0x7f7f14157d30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 --2- 172.21.15.92:0/1051271000 >> v2:172.21.15.136:3300/0 conn(0x7f7f140af250 0x7f7f14068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 -- 172.21.15.92:0/1051271000 >> 172.21.15.92:0/1051271000 conn(0x7f7f140c9870 msgr2=0x7f7f140c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 -- 172.21.15.92:0/1051271000 shutdown_connections 2023-10-06T03:06:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.036+0000 7f7f18efc700 1 -- 172.21.15.92:0/1051271000 wait complete. 2023-10-06T03:06:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.038+0000 7f7f18efc700 1 Processor -- start 2023-10-06T03:06:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.038+0000 7f7f18efc700 1 -- start start 2023-10-06T03:06:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f140688d0 0x7f7f1414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 0x7f7f140ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f14157960 0x7f7f14154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140c5c30 con 0x7f7f140af250 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140c5ed0 con 0x7f7f140688d0 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140c61b0 con 0x7f7f14157960 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 0x7f7f140ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 0x7f7f140ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58804/0 (socket says 172.21.15.92:58804) 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 -- 172.21.15.92:0/243184386 learned_addr learned my addr 172.21.15.92:0/243184386 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f12d9d700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f14157960 0x7f7f14154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f1259c700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f140688d0 0x7f7f1414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 -- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f14157960 msgr2=0x7f7f14154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f14157960 0x7f7f14154fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 -- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f140688d0 msgr2=0x7f7f1414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f140688d0 0x7f7f1414de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 -- 172.21.15.92:0/243184386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009ae0 con 0x7f7f140af250 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f11d9b700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 0x7f7f140ad670 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800c830 tx=0x7f7f08008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08011600 con 0x7f7f140af250 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08011760 con 0x7f7f140af250 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.039+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f140c6490 con 0x7f7f140af250 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.040+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08011b20 con 0x7f7f140af250 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.040+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1414e360 con 0x7f7f140af250 2023-10-06T03:06:35.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.041+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f140c1c30 con 0x7f7f140af250 2023-10-06T03:06:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.041+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7f08018360 con 0x7f7f140af250 2023-10-06T03:06:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.041+0000 7f7f027fc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ef4067170 0x7f7ef4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.042+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f080849b0 con 0x7f7f140af250 2023-10-06T03:06:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.042+0000 7f7f1259c700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ef4067170 0x7f7ef4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.042+0000 7f7f1259c700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ef4067170 0x7f7ef4069620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009af0 tx=0x7f7f04009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.049+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f140c1c30 con 0x7f7f140af250 2023-10-06T03:06:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.291+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f140c1c30 con 0x7f7f140af250 2023-10-06T03:06:35.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.291+0000 7f7f027fc700 1 -- 172.21.15.92:0/243184386 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f140c1c30 con 0x7f7f140af250 2023-10-06T03:06:35.293 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ef4067170 msgr2=0x7f7ef4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ef4067170 0x7f7ef4069620 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009af0 tx=0x7f7f04009060 comp rx=0 tx=0).stop 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 msgr2=0x7f7f140ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 0x7f7f140ad670 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800c830 tx=0x7f7f08008040 comp rx=0 tx=0).stop 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 shutdown_connections 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ef4067170 0x7f7ef4069620 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f14157960 0x7f7f14154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f140688d0 0x7f7f1414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 --2- 172.21.15.92:0/243184386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f140af250 0x7f7f140ad670 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 >> 172.21.15.92:0/243184386 conn(0x7f7f140c9870 msgr2=0x7f7f14159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:35.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 shutdown_connections 2023-10-06T03:06:35.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.294+0000 7f7f18efc700 1 -- 172.21.15.92:0/243184386 wait complete. 2023-10-06T03:06:35.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:06:35.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.438+0000 7f4576d9a700 1 Processor -- start 2023-10-06T03:06:35.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4576d9a700 1 -- start start 2023-10-06T03:06:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4576d9a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45700aef00 0x7f457015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4576d9a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f457006e470 0x7f457006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4576d9a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700aea70 con 0x7f457015a3b0 2023-10-06T03:06:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4576d9a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457006ee30 con 0x7f45700aef00 2023-10-06T03:06:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4576d9a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45700ae480 con 0x7f457006e470 2023-10-06T03:06:35.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.439+0000 7f4575597700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45700aef00 0x7f457015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575d98700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f457015a3b0 0x7f457015a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39330/0 (socket says 172.21.15.92:39330) 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575d98700 1 -- 172.21.15.92:0/853636634 learned_addr learned my addr 172.21.15.92:0/853636634 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575597700 1 -- 172.21.15.92:0/853636634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f457006e470 msgr2=0x7f457006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4576599700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f457006e470 0x7f457006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575597700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f457006e470 0x7f457006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575597700 1 -- 172.21.15.92:0/853636634 >> v1:172.21.15.92:6789/0 conn(0x7f457015a3b0 legacy=0x7f457015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4574d96700 1 -- 172.21.15.92:0/853636634 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118050829 0 0) 0x7f45700aea70 con 0x7f457015a3b0 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575597700 1 -- 172.21.15.92:0/853636634 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457015dd30 con 0x7f45700aef00 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4576599700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f457006e470 0x7f457006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4575597700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45700aef00 0x7f457015d3c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f45600098e0 tx=0x7f45600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71eaba010b0a0462 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4574d96700 1 -- 172.21.15.92:0/853636634 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560009e80 con 0x7f45700aef00 2023-10-06T03:06:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4574d96700 1 -- 172.21.15.92:0/853636634 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456000b860 con 0x7f45700aef00 2023-10-06T03:06:35.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.440+0000 7f4574d96700 1 -- 172.21.15.92:0/853636634 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456000bbe0 con 0x7f45700aef00 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 -- 172.21.15.92:0/853636634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45700aef00 msgr2=0x7f457015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45700aef00 0x7f457015d3c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f45600098e0 tx=0x7f45600091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 -- 172.21.15.92:0/853636634 shutdown_connections 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f457006e470 0x7f457006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 --2- 172.21.15.92:0/853636634 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45700aef00 0x7f457015d3c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 -- 172.21.15.92:0/853636634 >> 172.21.15.92:0/853636634 conn(0x7f45700c9f70 msgr2=0x7f457006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 -- 172.21.15.92:0/853636634 shutdown_connections 2023-10-06T03:06:35.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.441+0000 7f4576d9a700 1 -- 172.21.15.92:0/853636634 wait complete. 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.442+0000 7f4576d9a700 1 Processor -- start 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.442+0000 7f4576d9a700 1 -- start start 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576d9a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f457006e470 0x7f45700c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576d9a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45700aef00 0x7f45701543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576d9a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 0x7f45700af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576d9a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45700b08f0 con 0x7f457015a3b0 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576d9a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45700b0330 con 0x7f457006e470 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576d9a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45700b0610 con 0x7f45700aef00 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 0x7f45700af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 0x7f45700af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58816/0 (socket says 172.21.15.92:58816) 2023-10-06T03:06:35.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 -- 172.21.15.92:0/3229479209 learned_addr learned my addr 172.21.15.92:0/3229479209 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4575d98700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f457006e470 0x7f45700c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4575597700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45700aef00 0x7f45701543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 -- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45700aef00 msgr2=0x7f45701543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45700aef00 0x7f45701543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 -- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f457006e470 msgr2=0x7f45700c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f457006e470 0x7f45700c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 -- 172.21.15.92:0/3229479209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4560009420 con 0x7f457015a3b0 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.443+0000 7f4576599700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 0x7f45700af870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f456800a380 tx=0x7f4568007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.444+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800ae10 con 0x7f457015a3b0 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.444+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4570157ac0 con 0x7f457015a3b0 2023-10-06T03:06:35.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.444+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800d650 con 0x7f457015a3b0 2023-10-06T03:06:35.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.444+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800da10 con 0x7f457015a3b0 2023-10-06T03:06:35.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.444+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45700b1ad0 con 0x7f457015a3b0 2023-10-06T03:06:35.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.445+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4570157920 con 0x7f457015a3b0 2023-10-06T03:06:35.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.445+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f4568015440 con 0x7f457015a3b0 2023-10-06T03:06:35.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.446+0000 7f4565ffb700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4554067170 0x7f4554069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.446+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4568085280 con 0x7f457015a3b0 2023-10-06T03:06:35.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.446+0000 7f4575d98700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4554067170 0x7f4554069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.446+0000 7f4575d98700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4554067170 0x7f4554069620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c8f0 tx=0x7f456c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.451+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4570157920 con 0x7f457015a3b0 2023-10-06T03:06:35.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.698+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4570157920 con 0x7f457015a3b0 2023-10-06T03:06:35.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.698+0000 7f4565ffb700 1 -- 172.21.15.92:0/3229479209 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4570157920 con 0x7f457015a3b0 2023-10-06T03:06:35.700 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:06:35.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4554067170 msgr2=0x7f4554069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4554067170 0x7f4554069620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c8f0 tx=0x7f456c009040 comp rx=0 tx=0).stop 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 msgr2=0x7f45700af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 0x7f45700af870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f456800a380 tx=0x7f4568007900 comp rx=0 tx=0).stop 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 shutdown_connections 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4554067170 0x7f4554069620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015a3b0 0x7f45700af870 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f457006e470 0x7f45700c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 --2- 172.21.15.92:0/3229479209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45700aef00 0x7f45701543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.700+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 >> 172.21.15.92:0/3229479209 conn(0x7f45700c9f70 msgr2=0x7f4570153570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:35.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.701+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 shutdown_connections 2023-10-06T03:06:35.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.701+0000 7f4576d9a700 1 -- 172.21.15.92:0/3229479209 wait complete. 2023-10-06T03:06:35.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:06:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.861+0000 7fe85759e700 1 Processor -- start 2023-10-06T03:06:35.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.861+0000 7fe85759e700 1 -- start start 2023-10-06T03:06:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.862+0000 7fe85759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b2020 0x7fe85806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.862+0000 7fe85759e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe8580b23f0 0x7fe85806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.862+0000 7fe85759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580cbf40 con 0x7fe858159980 2023-10-06T03:06:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.862+0000 7fe85759e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85815a4c0 con 0x7fe8580b23f0 2023-10-06T03:06:35.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.862+0000 7fe85759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85815a6b0 con 0x7fe8580b2020 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe856d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe858159980 0x7fe858159db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39332/0 (socket says 172.21.15.92:39332) 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe856d9d700 1 -- 172.21.15.92:0/4134933846 learned_addr learned my addr 172.21.15.92:0/4134933846 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe85659c700 1 --2- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b2020 0x7fe85806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe855d9b700 1 --2- 172.21.15.92:0/4134933846 >> v2:172.21.15.136:3300/0 conn(0x7fe8580b23f0 0x7fe85806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe85559a700 1 -- 172.21.15.92:0/4134933846 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516918476 0 0) 0x7fe8580cbf40 con 0x7fe858159980 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe85559a700 1 -- 172.21.15.92:0/4134933846 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe840003570 con 0x7fe858159980 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe855d9b700 1 -- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b2020 msgr2=0x7fe85806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe855d9b700 1 --2- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b2020 0x7fe85806e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe855d9b700 1 -- 172.21.15.92:0/4134933846 >> v1:172.21.15.92:6789/0 conn(0x7fe858159980 legacy=0x7fe858159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe855d9b700 1 -- 172.21.15.92:0/4134933846 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580cb610 con 0x7fe8580b23f0 2023-10-06T03:06:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.863+0000 7fe855d9b700 1 --2- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b23f0 0x7fe85806e9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0098e0 tx=0x7fe84c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d7f065067089d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85559a700 1 -- 172.21.15.92:0/4134933846 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c011630 con 0x7fe8580b23f0 2023-10-06T03:06:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85559a700 1 -- 172.21.15.92:0/4134933846 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c011790 con 0x7fe8580b23f0 2023-10-06T03:06:35.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85559a700 1 -- 172.21.15.92:0/4134933846 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c011b30 con 0x7fe8580b23f0 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 -- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b23f0 msgr2=0x7fe85806e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 --2- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b23f0 0x7fe85806e9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0098e0 tx=0x7fe84c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 -- 172.21.15.92:0/4134933846 shutdown_connections 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 --2- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b2020 0x7fe85806e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 --2- 172.21.15.92:0/4134933846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b23f0 0x7fe85806e9b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 -- 172.21.15.92:0/4134933846 >> 172.21.15.92:0/4134933846 conn(0x7fe8580c9f70 msgr2=0x7fe85806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 -- 172.21.15.92:0/4134933846 shutdown_connections 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.864+0000 7fe85759e700 1 -- 172.21.15.92:0/4134933846 wait complete. 2023-10-06T03:06:35.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.866+0000 7fe85759e700 1 Processor -- start 2023-10-06T03:06:35.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.866+0000 7fe85759e700 1 -- start start 2023-10-06T03:06:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b2020 0x7fe858154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b23f0 0x7fe8580ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 0x7fe8580ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580ac580 con 0x7fe858159980 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858156500 con 0x7fe8580b2020 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581512f0 con 0x7fe8580b23f0 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 0x7fe8580ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 0x7fe8580ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58824/0 (socket says 172.21.15.92:58824) 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 -- 172.21.15.92:0/2017682480 learned_addr learned my addr 172.21.15.92:0/2017682480 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe855d9b700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b23f0 0x7fe8580ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe85659c700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b2020 0x7fe858154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 -- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b23f0 msgr2=0x7fe8580ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b23f0 0x7fe8580ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 -- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b2020 msgr2=0x7fe858154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b2020 0x7fe858154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 -- 172.21.15.92:0/2017682480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c009420 con 0x7fe858159980 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe856d9d700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 0x7fe8580ab640 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe850001460 tx=0x7fe850001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000a870 con 0x7fe858159980 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.867+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85000a9d0 con 0x7fe858159980 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.868+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85000ad50 con 0x7fe858159980 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.868+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8580abce0 con 0x7fe858159980 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.868+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8580ac2a0 con 0x7fe858159980 2023-10-06T03:06:35.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.869+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe858156670 con 0x7fe858159980 2023-10-06T03:06:35.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.869+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe850007300 con 0x7fe858159980 2023-10-06T03:06:35.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.870+0000 7fe8467fc700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe838067170 0x7fe838069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:35.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.870+0000 7fe85659c700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe838067170 0x7fe838069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.870+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe850084cb0 con 0x7fe858159980 2023-10-06T03:06:35.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.870+0000 7fe85659c700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe838067170 0x7fe838069620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe848009db0 tx=0x7fe8480091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:35.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:35.875+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe850051c60 con 0x7fe858159980 2023-10-06T03:06:36.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.123+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8580cb8f0 con 0x7fe858159980 2023-10-06T03:06:36.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.123+0000 7fe8467fc700 1 -- 172.21.15.92:0/2017682480 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe850055280 con 0x7fe858159980 2023-10-06T03:06:36.125 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:06:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe838067170 msgr2=0x7fe838069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe838067170 0x7fe838069620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe848009db0 tx=0x7fe8480091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 msgr2=0x7fe8580ab640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 0x7fe8580ab640 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe850001460 tx=0x7fe850001a40 comp rx=0 tx=0).stop 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 shutdown_connections 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe838067170 0x7fe838069620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe858159980 0x7fe8580ab640 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8580b2020 0x7fe858154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 --2- 172.21.15.92:0/2017682480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8580b23f0 0x7fe8580ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 >> 172.21.15.92:0/2017682480 conn(0x7fe8580c9f70 msgr2=0x7fe8580c2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 shutdown_connections 2023-10-06T03:06:36.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.125+0000 7fe85759e700 1 -- 172.21.15.92:0/2017682480 wait complete. 2023-10-06T03:06:36.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:06:36.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.290+0000 7fb589563700 1 Processor -- start 2023-10-06T03:06:36.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.290+0000 7fb589563700 1 -- start start 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.291+0000 7fb589563700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 0x7fb584068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.291+0000 7fb589563700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58406b030 con 0x7fb584157960 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.291+0000 7fb589563700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58406bbe0 con 0x7fb5840688d0 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.291+0000 7fb589563700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840ac760 con 0x7fb5840af250 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb582ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb584157960 0x7fb58406b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39344/0 (socket says 172.21.15.92:39344) 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb582ffd700 1 -- 172.21.15.92:0/3588194939 learned_addr learned my addr 172.21.15.92:0/3588194939 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb5827fc700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 0x7fb584068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954769595 0 0) 0x7fb58406b030 con 0x7fb584157960 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb56c003570 con 0x7fb584157960 2023-10-06T03:06:36.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298610258 0 0) 0x7fb56c003570 con 0x7fb584157960 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 msgr2=0x7fb584068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 0x7fb584068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 >> v1:172.21.15.136:6789/0 conn(0x7fb5840688d0 legacy=0x7fb584157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840c7f60 con 0x7fb584157960 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb574001470 con 0x7fb584157960 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 >> v1:172.21.15.92:6789/0 conn(0x7fb584157960 legacy=0x7fb58406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 0x7fb56c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56c003fd0 0x7fb56c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56c004d50 0x7fb56c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840ac760 con 0x7fb56c004d50 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c00bb40 con 0x7fb56c003fd0 2023-10-06T03:06:36.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c00bd00 con 0x7fb5840af250 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb582ffd700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 0x7fb56c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb5837fe700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56c004d50 0x7fb56c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.292+0000 7fb5827fc700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56c003fd0 0x7fb56c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb5837fe700 1 -- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 msgr2=0x7fb56c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb5837fe700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5840af250 0x7fb56c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb5837fe700 1 -- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56c003fd0 msgr2=0x7fb56c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb5837fe700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb56c003fd0 0x7fb56c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb5837fe700 1 -- 172.21.15.92:0/3588194939 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c003570 con 0x7fb56c004d50 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb5837fe700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56c004d50 0x7fb56c0051c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0020d0 tx=0x7fb57c001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a0d558cbab462d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00c8b0 con 0x7fb56c004d50 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00ca10 con 0x7fb56c004d50 2023-10-06T03:06:36.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.293+0000 7fb581ffb700 1 -- 172.21.15.92:0/3588194939 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00cdd0 con 0x7fb56c004d50 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb589563700 1 -- 172.21.15.92:0/3588194939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56c004d50 msgr2=0x7fb56c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb589563700 1 --2- 172.21.15.92:0/3588194939 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb56c004d50 0x7fb56c0051c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0020d0 tx=0x7fb57c001720 comp rx=0 tx=0).stop 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb5837fe700 1 -- 172.21.15.92:0/3588194939 reap_dead start 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb589563700 1 -- 172.21.15.92:0/3588194939 shutdown_connections 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb589563700 1 -- 172.21.15.92:0/3588194939 >> 172.21.15.92:0/3588194939 conn(0x7fb5840c9870 msgr2=0x7fb5840c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb589563700 1 -- 172.21.15.92:0/3588194939 shutdown_connections 2023-10-06T03:06:36.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.294+0000 7fb589563700 1 -- 172.21.15.92:0/3588194939 wait complete. 2023-10-06T03:06:36.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.295+0000 7fb589563700 1 Processor -- start 2023-10-06T03:06:36.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.295+0000 7fb589563700 1 -- start start 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb589563700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58414db50 0x7fb584157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb589563700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 0x7fb584152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb589563700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5841528c0 0x7fb584156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb589563700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840af830 con 0x7fb58414df20 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb589563700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58416bbc0 con 0x7fb58414db50 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb589563700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58416bea0 con 0x7fb5841528c0 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 0x7fb584152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 0x7fb584152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58842/0 (socket says 172.21.15.92:58842) 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 -- 172.21.15.92:0/1487015048 learned_addr learned my addr 172.21.15.92:0/1487015048 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb582ffd700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58414db50 0x7fb584157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5837fe700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5841528c0 0x7fb584156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 -- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5841528c0 msgr2=0x7fb584156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5841528c0 0x7fb584156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 -- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58414db50 msgr2=0x7fb584157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58414db50 0x7fb584157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.296+0000 7fb5827fc700 1 -- 172.21.15.92:0/1487015048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58416c180 con 0x7fb58414df20 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.297+0000 7fb5827fc700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 0x7fb584152380 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb578006ac0 tx=0x7fb5780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.297+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800f940 con 0x7fb58414df20 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.297+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57800faa0 con 0x7fb58414df20 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.297+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57800fe20 con 0x7fb58414df20 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.297+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58416c460 con 0x7fb58414df20 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.297+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb584158270 con 0x7fb58414df20 2023-10-06T03:06:36.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.298+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58404c3c0 con 0x7fb58414df20 2023-10-06T03:06:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.298+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb578014650 con 0x7fb58414df20 2023-10-06T03:06:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.299+0000 7fb56affd700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb570067170 0x7fb570069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.299+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb578018030 con 0x7fb58414df20 2023-10-06T03:06:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.299+0000 7fb582ffd700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb570067170 0x7fb570069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.299+0000 7fb582ffd700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb570067170 0x7fb570069620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb584151880 tx=0x7fb574001e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:36.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.303+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5780521b0 con 0x7fb58414df20 2023-10-06T03:06:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.512+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5840678d0 con 0x7fb58414df20 2023-10-06T03:06:36.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.513+0000 7fb56affd700 1 -- 172.21.15.92:0/1487015048 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5780557d0 con 0x7fb58414df20 2023-10-06T03:06:36.515 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb570067170 msgr2=0x7fb570069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb570067170 0x7fb570069620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb584151880 tx=0x7fb574001e70 comp rx=0 tx=0).stop 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 msgr2=0x7fb584152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 0x7fb584152380 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb578006ac0 tx=0x7fb5780091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 shutdown_connections 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb570067170 0x7fb570069620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5841528c0 0x7fb584156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb58414db50 0x7fb584157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 --2- 172.21.15.92:0/1487015048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb58414df20 0x7fb584152380 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 >> 172.21.15.92:0/1487015048 conn(0x7fb5840c9870 msgr2=0x7fb58406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 shutdown_connections 2023-10-06T03:06:36.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.515+0000 7fb589563700 1 -- 172.21.15.92:0/1487015048 wait complete. 2023-10-06T03:06:36.529 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.661+0000 7f1bd9e41700 1 Processor -- start 2023-10-06T03:06:36.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.661+0000 7f1bd9e41700 1 -- start start 2023-10-06T03:06:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9e41700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd4157380 0x7f1bd4157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9e41700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1bd40ae840 0x7f1bd40aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9e41700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd40af1a0 con 0x7f1bd40c6b30 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9e41700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd406ea60 con 0x7f1bd40ae840 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9e41700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd406ed40 con 0x7f1bd4157380 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9640700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1bd40ae840 0x7f1bd40aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd8e3f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd4157380 0x7f1bd4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9640700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1bd40ae840 0x7f1bd40aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:59934/0 (socket says 172.21.15.92:59934) 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd9640700 1 -- 172.21.15.92:0/372534128 learned_addr learned my addr 172.21.15.92:0/372534128 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.662+0000 7f1bd3fff700 1 --1- 172.21.15.92:0/372534128 >> v1:172.21.15.136:6789/0 conn(0x7f1bd40c6b30 0x7f1bd415bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:60936/0 (socket says 172.21.15.92:60936) 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd9640700 1 -- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd4157380 msgr2=0x7f1bd4157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd9640700 1 --2- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd4157380 0x7f1bd4157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd9640700 1 -- 172.21.15.92:0/372534128 >> v1:172.21.15.136:6789/0 conn(0x7f1bd40c6b30 legacy=0x7f1bd415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd9640700 1 -- 172.21.15.92:0/372534128 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd40afe60 con 0x7f1bd40ae840 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd8e3f700 1 --2- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd4157380 0x7f1bd4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd37fe700 1 -- 172.21.15.92:0/372534128 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580652096 0 0) 0x7f1bd40af1a0 con 0x7f1bd40c6b30 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd9640700 1 --2- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40ae840 0x7f1bd40aec60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009f20 tx=0x7f1bcc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bdc38945e9d08b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd37fe700 1 -- 172.21.15.92:0/372534128 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc00e070 con 0x7f1bd40ae840 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd37fe700 1 -- 172.21.15.92:0/372534128 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc00b9d0 con 0x7f1bd40ae840 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.663+0000 7f1bd37fe700 1 -- 172.21.15.92:0/372534128 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0092e0 con 0x7f1bd40ae840 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 -- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40ae840 msgr2=0x7f1bd40aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40ae840 0x7f1bd40aec60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009f20 tx=0x7f1bcc009840 comp rx=0 tx=0).stop 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 -- 172.21.15.92:0/372534128 shutdown_connections 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40ae840 0x7f1bd40aec60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/372534128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd4157380 0x7f1bd4157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 -- 172.21.15.92:0/372534128 >> 172.21.15.92:0/372534128 conn(0x7f1bd406ddb0 msgr2=0x7f1bd406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 -- 172.21.15.92:0/372534128 shutdown_connections 2023-10-06T03:06:36.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.664+0000 7f1bd9e41700 1 -- 172.21.15.92:0/372534128 wait complete. 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.665+0000 7f1bd9e41700 1 Processor -- start 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.666+0000 7f1bd9e41700 1 -- start start 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9e41700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd40ae840 0x7f1bd414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9e41700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40c6b30 0x7f1bd414eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9e41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 0x7f1bd40c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9e41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd41527d0 con 0x7f1bd4157380 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9e41700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd4151bc0 con 0x7f1bd40c6b30 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9e41700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd4151ea0 con 0x7f1bd40ae840 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 0x7f1bd40c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 0x7f1bd40c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58848/0 (socket says 172.21.15.92:58848) 2023-10-06T03:06:36.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 -- 172.21.15.92:0/2181065680 learned_addr learned my addr 172.21.15.92:0/2181065680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd8e3f700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd40ae840 0x7f1bd414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd3fff700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40c6b30 0x7f1bd414eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 -- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd40ae840 msgr2=0x7f1bd414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd40ae840 0x7f1bd414c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 -- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40c6b30 msgr2=0x7f1bd414eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40c6b30 0x7f1bd414eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 -- 172.21.15.92:0/2181065680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc8009420 con 0x7f1bd4157380 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.667+0000 7f1bd9640700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 0x7f1bd40c5d60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1bd406d320 tx=0x7f1bcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.668+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc008c70 con 0x7f1bd4157380 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.668+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc008df0 con 0x7f1bd4157380 2023-10-06T03:06:36.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.668+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc009ae0 con 0x7f1bd4157380 2023-10-06T03:06:36.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.668+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0266b0 con 0x7f1bd4157380 2023-10-06T03:06:36.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.668+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd4152460 con 0x7f1bd4157380 2023-10-06T03:06:36.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.669+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd4152980 con 0x7f1bd4157380 2023-10-06T03:06:36.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.669+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f1bcc003040 con 0x7f1bd4157380 2023-10-06T03:06:36.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.669+0000 7f1bd0ff9700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1bb8067170 0x7f1bb8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:36.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.670+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1bcc086a70 con 0x7f1bd4157380 2023-10-06T03:06:36.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.670+0000 7f1bd8e3f700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1bb8067170 0x7f1bb8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:36.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.670+0000 7f1bd8e3f700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1bb8067170 0x7f1bb8069620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8006f00 tx=0x7f1bc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:36.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.674+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bd4152980 con 0x7f1bd4157380 2023-10-06T03:06:36.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.943+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1bd404c3c0 con 0x7f1bd4157380 2023-10-06T03:06:36.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.944+0000 7f1bd0ff9700 1 -- 172.21.15.92:0/2181065680 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f1bcc0537e0 con 0x7f1bd4157380 2023-10-06T03:06:36.946 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:36.946 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:29.075556+0000","modified":"2023-10-06T03:03:32.719393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/812630670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":812630670},{"type":"v1","addr":"172.21.15.92:6837","nonce":812630670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:06:36.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1bb8067170 msgr2=0x7f1bb8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1bb8067170 0x7f1bb8069620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8006f00 tx=0x7f1bc8008040 comp rx=0 tx=0).stop 2023-10-06T03:06:36.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 msgr2=0x7f1bd40c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:36.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 0x7f1bd40c5d60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1bd406d320 tx=0x7f1bcc018000 comp rx=0 tx=0).stop 2023-10-06T03:06:36.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 shutdown_connections 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1bb8067170 0x7f1bb8069620 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1bd4157380 0x7f1bd40c5d60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1bd40ae840 0x7f1bd414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 --2- 172.21.15.92:0/2181065680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd40c6b30 0x7f1bd414eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 >> 172.21.15.92:0/2181065680 conn(0x7f1bd406ddb0 msgr2=0x7f1bd415b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 shutdown_connections 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:36.946+0000 7f1bd9e41700 1 -- 172.21.15.92:0/2181065680 wait complete. 2023-10-06T03:06:36.949 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 39 2023-10-06T03:06:36.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:37.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.095+0000 7f2d55b12700 1 Processor -- start 2023-10-06T03:06:37.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.095+0000 7f2d55b12700 1 -- start start 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d55b12700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d50159b10 0x7f2d500cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d55b12700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d5015d730 0x7f2d5015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d55b12700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d500c74b0 con 0x7f2d50159ee0 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d55b12700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d500c8700 con 0x7f2d50159b10 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d55b12700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500c9070 con 0x7f2d5015d730 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d4f7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d50159b10 0x7f2d500cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d5015d730 0x7f2d5015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d4f7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d50159b10 0x7f2d500cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:59958/0 (socket says 172.21.15.92:59958) 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.096+0000 7f2d4f7fe700 1 -- 172.21.15.92:0/2990698128 learned_addr learned my addr 172.21.15.92:0/2990698128 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:37.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4f7fe700 1 -- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d5015d730 msgr2=0x7f2d5015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4f7fe700 1 --2- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d5015d730 0x7f2d5015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4f7fe700 1 -- 172.21.15.92:0/2990698128 >> v1:172.21.15.136:6789/0 conn(0x7f2d50159ee0 legacy=0x7f2d5015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4f7fe700 1 -- 172.21.15.92:0/2990698128 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5015e0e0 con 0x7f2d50159b10 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4effd700 1 --2- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d5015d730 0x7f2d5015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4f7fe700 1 --2- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d50159b10 0x7f2d500cb3e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009f20 tx=0x7f2d40009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e0f3fe85e31051 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4e7fc700 1 -- 172.21.15.92:0/2990698128 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000e070 con 0x7f2d50159b10 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4e7fc700 1 -- 172.21.15.92:0/2990698128 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000ba60 con 0x7f2d50159b10 2023-10-06T03:06:37.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.097+0000 7f2d4e7fc700 1 -- 172.21.15.92:0/2990698128 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d400092e0 con 0x7f2d50159b10 2023-10-06T03:06:37.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 -- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d50159b10 msgr2=0x7f2d500cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 --2- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d50159b10 0x7f2d500cb3e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009f20 tx=0x7f2d40009840 comp rx=0 tx=0).stop 2023-10-06T03:06:37.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 -- 172.21.15.92:0/2990698128 shutdown_connections 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 --2- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d50159b10 0x7f2d500cb3e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 --2- 172.21.15.92:0/2990698128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d5015d730 0x7f2d5015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 -- 172.21.15.92:0/2990698128 >> 172.21.15.92:0/2990698128 conn(0x7f2d500c9870 msgr2=0x7f2d500c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 -- 172.21.15.92:0/2990698128 shutdown_connections 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.098+0000 7f2d55b12700 1 -- 172.21.15.92:0/2990698128 wait complete. 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.099+0000 7f2d55b12700 1 Processor -- start 2023-10-06T03:06:37.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.099+0000 7f2d55b12700 1 -- start start 2023-10-06T03:06:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d55b12700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d50159b10 0x7f2d500b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d55b12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 0x7f2d5015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d55b12700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d5015d730 0x7f2d50152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d55b12700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015b670 con 0x7f2d50159ee0 2023-10-06T03:06:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d55b12700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015aa60 con 0x7f2d5015d730 2023-10-06T03:06:37.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d55b12700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015ad40 con 0x7f2d50159b10 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 0x7f2d5015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d4effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 0x7f2d5015a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58860/0 (socket says 172.21.15.92:58860) 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.100+0000 7f2d4effd700 1 -- 172.21.15.92:0/3378209617 learned_addr learned my addr 172.21.15.92:0/3378209617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4f7fe700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d50159b10 0x7f2d500b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4ffff700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d5015d730 0x7f2d50152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4effd700 1 -- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d50159b10 msgr2=0x7f2d500b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4effd700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d50159b10 0x7f2d500b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4effd700 1 -- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d5015d730 msgr2=0x7f2d50152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4effd700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d5015d730 0x7f2d50152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4effd700 1 -- 172.21.15.92:0/3378209617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38009420 con 0x7f2d50159ee0 2023-10-06T03:06:37.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4f7fe700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d50159b10 0x7f2d500b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4ffff700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d5015d730 0x7f2d50152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d4effd700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 0x7f2d5015a360 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800ec80 tx=0x7f2d380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800c9d0 con 0x7f2d50159ee0 2023-10-06T03:06:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3800cb30 con 0x7f2d50159ee0 2023-10-06T03:06:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d40009ae0 con 0x7f2d50159ee0 2023-10-06T03:06:37.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800ceb0 con 0x7f2d50159ee0 2023-10-06T03:06:37.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.101+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5015b300 con 0x7f2d50159ee0 2023-10-06T03:06:37.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.102+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5015b7e0 con 0x7f2d50159ee0 2023-10-06T03:06:37.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.103+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2d38013680 con 0x7f2d50159ee0 2023-10-06T03:06:37.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.103+0000 7f2d477fe700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d30067170 0x7f2d30069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.103+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d38085500 con 0x7f2d50159ee0 2023-10-06T03:06:37.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.103+0000 7f2d4f7fe700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d30067170 0x7f2d30069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.104+0000 7f2d4f7fe700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d30067170 0x7f2d30069620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009880 tx=0x7f2d40018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.108+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d38052400 con 0x7f2d50159ee0 2023-10-06T03:06:37.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.341+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d5004c3c0 con 0x7f2d50159ee0 2023-10-06T03:06:37.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.342+0000 7f2d477fe700 1 -- 172.21.15.92:0/3378209617 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f2d38055a20 con 0x7f2d50159ee0 2023-10-06T03:06:37.343 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:37.344 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:30.711121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:03:26.724124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:28.041787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:37.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d30067170 msgr2=0x7f2d30069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d30067170 0x7f2d30069620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d40009880 tx=0x7f2d40018000 comp rx=0 tx=0).stop 2023-10-06T03:06:37.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 msgr2=0x7f2d5015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 0x7f2d5015a360 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800ec80 tx=0x7f2d380091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 shutdown_connections 2023-10-06T03:06:37.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d30067170 0x7f2d30069620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d5015d730 0x7f2d50152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d50159b10 0x7f2d500b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 --2- 172.21.15.92:0/3378209617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d50159ee0 0x7f2d5015a360 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 >> 172.21.15.92:0/3378209617 conn(0x7f2d500c9870 msgr2=0x7f2d50158340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 shutdown_connections 2023-10-06T03:06:37.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.344+0000 7f2d55b12700 1 -- 172.21.15.92:0/3378209617 wait complete. 2023-10-06T03:06:37.359 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:06:37.359 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.477+0000 7f68e700f700 1 Processor -- start 2023-10-06T03:06:37.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.478+0000 7f68e700f700 1 -- start start 2023-10-06T03:06:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.478+0000 7f68e700f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 0x7f68e00cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e700f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f68e006ec80 0x7f68e00ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e700f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e00adcf0 con 0x7f68e006bbe0 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e700f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e006cbf0 con 0x7f68e006ec80 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e700f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00aceb0 con 0x7f68e0156950 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 0x7f68e00cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e680e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f68e006ec80 0x7f68e00ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 0x7f68e00cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:49362/0 (socket says 172.21.15.92:49362) 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 -- 172.21.15.92:0/175755814 learned_addr learned my addr 172.21.15.92:0/175755814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 -- 172.21.15.92:0/175755814 >> v2:172.21.15.136:3300/0 conn(0x7f68e006ec80 msgr2=0x7f68e00ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 --2- 172.21.15.92:0/175755814 >> v2:172.21.15.136:3300/0 conn(0x7f68e006ec80 0x7f68e00ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 -- 172.21.15.92:0/175755814 >> v1:172.21.15.136:6789/0 conn(0x7f68e006bbe0 legacy=0x7f68e006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.479+0000 7f68e600d700 1 -- 172.21.15.92:0/175755814 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00adfb0 con 0x7f68e0156950 2023-10-06T03:06:37.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e600d700 1 --2- 172.21.15.92:0/175755814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 0x7f68e00cda10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0099b0 tx=0x7f68dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8fd75b3ec829119 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e500b700 1 -- 172.21.15.92:0/175755814 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc009ea0 con 0x7f68e0156950 2023-10-06T03:06:37.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e500b700 1 -- 172.21.15.92:0/175755814 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc00b8f0 con 0x7f68e0156950 2023-10-06T03:06:37.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e500b700 1 -- 172.21.15.92:0/175755814 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00bc90 con 0x7f68e0156950 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 -- 172.21.15.92:0/175755814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 msgr2=0x7f68e00cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 --2- 172.21.15.92:0/175755814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 0x7f68e00cda10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0099b0 tx=0x7f68dc009250 comp rx=0 tx=0).stop 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 -- 172.21.15.92:0/175755814 shutdown_connections 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 --2- 172.21.15.92:0/175755814 >> v2:172.21.15.136:3300/0 conn(0x7f68e006ec80 0x7f68e00ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 --2- 172.21.15.92:0/175755814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e0156950 0x7f68e00cda10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 -- 172.21.15.92:0/175755814 >> 172.21.15.92:0/175755814 conn(0x7f68e006b490 msgr2=0x7f68e00cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.480+0000 7f68e700f700 1 -- 172.21.15.92:0/175755814 shutdown_connections 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.481+0000 7f68e700f700 1 -- 172.21.15.92:0/175755814 wait complete. 2023-10-06T03:06:37.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.482+0000 7f68e700f700 1 Processor -- start 2023-10-06T03:06:37.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.482+0000 7f68e700f700 1 -- start start 2023-10-06T03:06:37.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.483+0000 7f68e700f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e006bbe0 0x7f68e00c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.483+0000 7f68e700f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e006ec80 0x7f68e014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.483+0000 7f68e700f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 0x7f68e00cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.483+0000 7f68e700f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e014e830 con 0x7f68e0156950 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.483+0000 7f68e700f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00cc2c0 con 0x7f68e006ec80 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.483+0000 7f68e700f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00c5060 con 0x7f68e006bbe0 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 0x7f68e00cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 0x7f68e00cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58862/0 (socket says 172.21.15.92:58862) 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 -- 172.21.15.92:0/4147839754 learned_addr learned my addr 172.21.15.92:0/4147839754 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e580c700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e006ec80 0x7f68e014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e600d700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e006bbe0 0x7f68e00c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 -- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e006bbe0 msgr2=0x7f68e00c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e006bbe0 0x7f68e00c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 -- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e006ec80 msgr2=0x7f68e014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e006ec80 0x7f68e014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 -- 172.21.15.92:0/4147839754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc0094f0 con 0x7f68e0156950 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e680e700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 0x7f68e00cae30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009e90 tx=0x7f68d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800caa0 con 0x7f68e0156950 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d800f040 con 0x7f68e0156950 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e00c5340 con 0x7f68e0156950 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800ce50 con 0x7f68e0156950 2023-10-06T03:06:37.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.484+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e00c5900 con 0x7f68e0156950 2023-10-06T03:06:37.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.485+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e00cc470 con 0x7f68e0156950 2023-10-06T03:06:37.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.486+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f68d8014690 con 0x7f68e0156950 2023-10-06T03:06:37.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.486+0000 7f68d67fc700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c4067170 0x7f68c4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.487+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f68d80859d0 con 0x7f68e0156950 2023-10-06T03:06:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.487+0000 7f68e600d700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c4067170 0x7f68c4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.487+0000 7f68e600d700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c4067170 0x7f68c4069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009290 tx=0x7f68dc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.492+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68d8056250 con 0x7f68e0156950 2023-10-06T03:06:37.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.801+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68e00c5ac0 con 0x7f68e0156950 2023-10-06T03:06:37.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.802+0000 7f68d67fc700 1 -- 172.21.15.92:0/4147839754 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f68e00c5ac0 con 0x7f68e0156950 2023-10-06T03:06:37.803 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:37.803 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:29.075556+0000","modified":"2023-10-06T03:03:32.719393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/812630670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":812630670},{"type":"v1","addr":"172.21.15.92:6837","nonce":812630670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c4067170 msgr2=0x7f68c4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c4067170 0x7f68c4069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009290 tx=0x7f68dc017000 comp rx=0 tx=0).stop 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 msgr2=0x7f68e00cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 0x7f68e00cae30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f68d8009e90 tx=0x7f68d800b040 comp rx=0 tx=0).stop 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 shutdown_connections 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f68c4067170 0x7f68c4069620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68e0156950 0x7f68e00cae30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f68e006bbe0 0x7f68e00c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 --2- 172.21.15.92:0/4147839754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68e006ec80 0x7f68e014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 >> 172.21.15.92:0/4147839754 conn(0x7f68e006b490 msgr2=0x7f68e0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 shutdown_connections 2023-10-06T03:06:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.804+0000 7f68e700f700 1 -- 172.21.15.92:0/4147839754 wait complete. 2023-10-06T03:06:37.807 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 39 2023-10-06T03:06:37.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.947+0000 7f3c652d5700 1 Processor -- start 2023-10-06T03:06:37.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.947+0000 7f3c652d5700 1 -- start start 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c652d5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3c600cd0f0 0x7f3c601598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c652d5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c60159de0 0x7f3c6015a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c652d5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c600d01f0 con 0x7f3c601594d0 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c652d5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600cd540 con 0x7f3c60159de0 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c652d5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600cf5e0 con 0x7f3c600cd0f0 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c5f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c60159de0 0x7f3c6015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c5f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c60159de0 0x7f3c6015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58872/0 (socket says 172.21.15.92:58872) 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/1806317803 learned_addr learned my addr 172.21.15.92:0/1806317803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.948+0000 7f3c5e7fc700 1 --2- 172.21.15.92:0/1806317803 >> v2:172.21.15.136:3300/0 conn(0x7f3c600cd0f0 0x7f3c601598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/1806317803 >> v2:172.21.15.136:3300/0 conn(0x7f3c600cd0f0 msgr2=0x7f3c601598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:37.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5f7fe700 1 --2- 172.21.15.92:0/1806317803 >> v2:172.21.15.136:3300/0 conn(0x7f3c600cd0f0 0x7f3c601598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/1806317803 >> v1:172.21.15.136:6789/0 conn(0x7f3c601594d0 legacy=0x7f3c600b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/1806317803 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600cf8c0 con 0x7f3c60159de0 2023-10-06T03:06:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5f7fe700 1 --2- 172.21.15.92:0/1806317803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c6015a220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3c60064810 tx=0x7f3c58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2038cd2c61971c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5dffb700 1 -- 172.21.15.92:0/1806317803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800e070 con 0x7f3c60159de0 2023-10-06T03:06:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5dffb700 1 -- 172.21.15.92:0/1806317803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5800ba20 con 0x7f3c60159de0 2023-10-06T03:06:37.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.949+0000 7f3c5dffb700 1 -- 172.21.15.92:0/1806317803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580092e0 con 0x7f3c60159de0 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 -- 172.21.15.92:0/1806317803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 msgr2=0x7f3c6015a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 --2- 172.21.15.92:0/1806317803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c6015a220 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3c60064810 tx=0x7f3c58009840 comp rx=0 tx=0).stop 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 -- 172.21.15.92:0/1806317803 shutdown_connections 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 --2- 172.21.15.92:0/1806317803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c6015a220 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 --2- 172.21.15.92:0/1806317803 >> v2:172.21.15.136:3300/0 conn(0x7f3c600cd0f0 0x7f3c601598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 -- 172.21.15.92:0/1806317803 >> 172.21.15.92:0/1806317803 conn(0x7f3c600d05f0 msgr2=0x7f3c600d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:37.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 -- 172.21.15.92:0/1806317803 shutdown_connections 2023-10-06T03:06:37.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.950+0000 7f3c652d5700 1 -- 172.21.15.92:0/1806317803 wait complete. 2023-10-06T03:06:37.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.952+0000 7f3c652d5700 1 Processor -- start 2023-10-06T03:06:37.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.952+0000 7f3c652d5700 1 -- start start 2023-10-06T03:06:37.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c652d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c600cd0f0 0x7f3c600c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c652d5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c601594d0 0x7f3c600c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c652d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c601474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c652d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601584b0 con 0x7f3c60159de0 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c652d5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600c6280 con 0x7f3c601594d0 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c652d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60157830 con 0x7f3c600cd0f0 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c601474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c601474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58878/0 (socket says 172.21.15.92:58878) 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/2075298865 learned_addr learned my addr 172.21.15.92:0/2075298865 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5e7fc700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c601594d0 0x7f3c600c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5effd700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c600cd0f0 0x7f3c600c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c600cd0f0 msgr2=0x7f3c600c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c600cd0f0 0x7f3c600c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c601594d0 msgr2=0x7f3c600c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c601594d0 0x7f3c600c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 -- 172.21.15.92:0/2075298865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58009ae0 con 0x7f3c60159de0 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5effd700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c600cd0f0 0x7f3c600c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.953+0000 7f3c5f7fe700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c601474b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009df0 tx=0x7f3c58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.954+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58008b30 con 0x7f3c60159de0 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.954+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c60157b10 con 0x7f3c60159de0 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.954+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58008c90 con 0x7f3c60159de0 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.954+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580256b0 con 0x7f3c60159de0 2023-10-06T03:06:37.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.954+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c601580d0 con 0x7f3c60159de0 2023-10-06T03:06:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.954+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c601585f0 con 0x7f3c60159de0 2023-10-06T03:06:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.955+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f3c58003040 con 0x7f3c60159de0 2023-10-06T03:06:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.955+0000 7f3c46ffd700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3c4c067170 0x7f3c4c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:37.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.955+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3c580861a0 con 0x7f3c60159de0 2023-10-06T03:06:37.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.955+0000 7f3c5effd700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3c4c067170 0x7f3c4c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:37.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.956+0000 7f3c5effd700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3c4c067170 0x7f3c4c069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007fd0 tx=0x7f3c50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:37.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:37.960+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c601585f0 con 0x7f3c60159de0 2023-10-06T03:06:38.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.240+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c601585f0 con 0x7f3c60159de0 2023-10-06T03:06:38.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.240+0000 7f3c46ffd700 1 -- 172.21.15.92:0/2075298865 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f3c601585f0 con 0x7f3c60159de0 2023-10-06T03:06:38.243 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:38.243 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:03:29.075556+0000","modified":"2023-10-06T03:03:32.719393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"c","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/812630670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":812630670},{"type":"v1","addr":"172.21.15.92:6837","nonce":812630670}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:06:38.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3c4c067170 msgr2=0x7f3c4c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3c4c067170 0x7f3c4c069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007fd0 tx=0x7f3c50009040 comp rx=0 tx=0).stop 2023-10-06T03:06:38.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 msgr2=0x7f3c601474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c601474b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009df0 tx=0x7f3c58018000 comp rx=0 tx=0).stop 2023-10-06T03:06:38.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 shutdown_connections 2023-10-06T03:06:38.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3c4c067170 0x7f3c4c069620 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c60159de0 0x7f3c601474b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c600cd0f0 0x7f3c600c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 --2- 172.21.15.92:0/2075298865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c601594d0 0x7f3c600c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 >> 172.21.15.92:0/2075298865 conn(0x7f3c600d05f0 msgr2=0x7f3c600b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:38.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 shutdown_connections 2023-10-06T03:06:38.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.243+0000 7f3c652d5700 1 -- 172.21.15.92:0/2075298865 wait complete. 2023-10-06T03:06:38.247 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 39 2023-10-06T03:06:38.260 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:38.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 Processor -- start 2023-10-06T03:06:38.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 -- start start 2023-10-06T03:06:38.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c78159420 0x7f6c780b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6c781597f0 0x7f6c780b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780b11d0 con 0x7f6c7815d230 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780b14b0 con 0x7f6c781597f0 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.388+0000 7f6c7c8ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780b1790 con 0x7f6c78159420 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c77fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6c7815d230 0x7f6c780b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39354/0 (socket says 172.21.15.92:39354) 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c77fff700 1 -- 172.21.15.92:0/549395484 learned_addr learned my addr 172.21.15.92:0/549395484 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c777fe700 1 --2- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c78159420 0x7f6c780b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c76ffd700 1 --2- 172.21.15.92:0/549395484 >> v2:172.21.15.136:3300/0 conn(0x7f6c781597f0 0x7f6c780b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c767fc700 1 -- 172.21.15.92:0/549395484 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617546534 0 0) 0x7f6c780b11d0 con 0x7f6c7815d230 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c767fc700 1 -- 172.21.15.92:0/549395484 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c60003570 con 0x7f6c7815d230 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c76ffd700 1 -- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c78159420 msgr2=0x7f6c780b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c76ffd700 1 --2- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c78159420 0x7f6c780b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c76ffd700 1 -- 172.21.15.92:0/549395484 >> v1:172.21.15.92:6789/0 conn(0x7f6c7815d230 legacy=0x7f6c780b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c76ffd700 1 -- 172.21.15.92:0/549395484 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780b1a70 con 0x7f6c781597f0 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.389+0000 7f6c76ffd700 1 --2- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c781597f0 0x7f6c780b2270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0098e0 tx=0x7f6c6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac83181c733e4be2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c767fc700 1 -- 172.21.15.92:0/549395484 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c011630 con 0x7f6c781597f0 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c767fc700 1 -- 172.21.15.92:0/549395484 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c011790 con 0x7f6c781597f0 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c767fc700 1 -- 172.21.15.92:0/549395484 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c011b10 con 0x7f6c781597f0 2023-10-06T03:06:38.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c781597f0 msgr2=0x7f6c780b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c781597f0 0x7f6c780b2270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0098e0 tx=0x7f6c6c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/549395484 shutdown_connections 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c78159420 0x7f6c780b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/549395484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c781597f0 0x7f6c780b2270 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/549395484 >> 172.21.15.92:0/549395484 conn(0x7f6c780d0660 msgr2=0x7f6c780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/549395484 shutdown_connections 2023-10-06T03:06:38.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.390+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/549395484 wait complete. 2023-10-06T03:06:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.392+0000 7f6c7c8ac700 1 Processor -- start 2023-10-06T03:06:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.392+0000 7f6c7c8ac700 1 -- start start 2023-10-06T03:06:38.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c7c8ac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78159420 0x7f6c7814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c7c8ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c781597f0 0x7f6c7816a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c7c8ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 0x7f6c7815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c7c8ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78153670 con 0x7f6c7815d230 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c7c8ac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c781537e0 con 0x7f6c78159420 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c7c8ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78151690 con 0x7f6c781597f0 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c77fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 0x7f6c7815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c77fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 0x7f6c7815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58892/0 (socket says 172.21.15.92:58892) 2023-10-06T03:06:38.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c77fff700 1 -- 172.21.15.92:0/406101141 learned_addr learned my addr 172.21.15.92:0/406101141 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c777fe700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78159420 0x7f6c7814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.393+0000 7f6c76ffd700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c781597f0 0x7f6c7816a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c77fff700 1 -- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c781597f0 msgr2=0x7f6c7816a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c77fff700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c781597f0 0x7f6c7816a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c77fff700 1 -- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78159420 msgr2=0x7f6c7814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c77fff700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78159420 0x7f6c7814dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c77fff700 1 -- 172.21.15.92:0/406101141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009420 con 0x7f6c7815d230 2023-10-06T03:06:38.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c777fe700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78159420 0x7f6c7814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c77fff700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 0x7f6c7815abd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c70001460 tx=0x7f6c70001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000a950 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7000aab0 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c78151a70 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000ae30 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.394+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c78152030 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.395+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c78152250 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.396+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6c70007300 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.396+0000 7f6c5f7fe700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c64067170 0x7f6c64069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.396+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6c70084e60 con 0x7f6c7815d230 2023-10-06T03:06:38.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.396+0000 7f6c777fe700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c64067170 0x7f6c64069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.397+0000 7f6c777fe700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c64067170 0x7f6c64069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c68006f00 tx=0x7f6c68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:38.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.401+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c78152250 con 0x7f6c7815d230 2023-10-06T03:06:38.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.621+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c78152250 con 0x7f6c7815d230 2023-10-06T03:06:38.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.621+0000 7f6c5f7fe700 1 -- 172.21.15.92:0/406101141 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f6c78152250 con 0x7f6c7815d230 2023-10-06T03:06:38.624 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:38.624 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":66,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:03:30.711121+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:03:26.724124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:03:28.041787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:38.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c64067170 msgr2=0x7f6c64069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c64067170 0x7f6c64069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c68006f00 tx=0x7f6c68008040 comp rx=0 tx=0).stop 2023-10-06T03:06:38.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 msgr2=0x7f6c7815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 0x7f6c7815abd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6c70001460 tx=0x7f6c70001a40 comp rx=0 tx=0).stop 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 shutdown_connections 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6c64067170 0x7f6c64069620 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6c7815d230 0x7f6c7815abd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c78159420 0x7f6c7814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 --2- 172.21.15.92:0/406101141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c781597f0 0x7f6c7816a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 >> 172.21.15.92:0/406101141 conn(0x7f6c780d0660 msgr2=0x7f6c780b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 shutdown_connections 2023-10-06T03:06:38.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.624+0000 7f6c7c8ac700 1 -- 172.21.15.92:0/406101141 wait complete. 2023-10-06T03:06:38.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:06:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.789+0000 7fc66d0b1700 1 Processor -- start 2023-10-06T03:06:38.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.789+0000 7fc66d0b1700 1 -- start start 2023-10-06T03:06:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc66d0b1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 0x7fc6680af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc66d0b1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 0x7fc6680af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc66d0b1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6680aff10 con 0x7fc6680c8120 2023-10-06T03:06:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc66d0b1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6681563e0 con 0x7fc66813f880 2023-10-06T03:06:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc66d0b1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680b22f0 con 0x7fc66813f4b0 2023-10-06T03:06:38.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc667fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 0x7fc6680af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.790+0000 7fc6677fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 0x7fc6680af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc6677fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 0x7fc6680af9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:60032/0 (socket says 172.21.15.92:60032) 2023-10-06T03:06:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc6677fe700 1 -- 172.21.15.92:0/981822597 learned_addr learned my addr 172.21.15.92:0/981822597 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:38.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc667fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 0x7fc6680af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:49400/0 (socket says 172.21.15.92:49400) 2023-10-06T03:06:38.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc66c8b0700 1 --1- 172.21.15.92:0/981822597 >> v1:172.21.15.136:6789/0 conn(0x7fc6680c8120 0x7fc668159960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:60980/0 (socket says 172.21.15.92:60980) 2023-10-06T03:06:38.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc667fff700 1 -- 172.21.15.92:0/981822597 >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 msgr2=0x7fc6680af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:38.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc667fff700 1 --2- 172.21.15.92:0/981822597 >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 0x7fc6680af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc667fff700 1 -- 172.21.15.92:0/981822597 >> v1:172.21.15.136:6789/0 conn(0x7fc6680c8120 legacy=0x7fc668159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc667fff700 1 -- 172.21.15.92:0/981822597 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6680b25d0 con 0x7fc66813f4b0 2023-10-06T03:06:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc666ffd700 1 -- 172.21.15.92:0/981822597 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103311773 0 0) 0x7fc6680aff10 con 0x7fc6680c8120 2023-10-06T03:06:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc6677fe700 1 --2- 172.21.15.92:0/981822597 >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 0x7fc6680af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc667fff700 1 --2- 172.21.15.92:0/981822597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 0x7fc6680af260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6580098e0 tx=0x7fc6580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d47d12ff6ceca08 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.791+0000 7fc666ffd700 1 -- 172.21.15.92:0/981822597 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800b640 con 0x7fc66813f4b0 2023-10-06T03:06:38.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc666ffd700 1 -- 172.21.15.92:0/981822597 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65800b7a0 con 0x7fc66813f4b0 2023-10-06T03:06:38.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc666ffd700 1 -- 172.21.15.92:0/981822597 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800bb40 con 0x7fc66813f4b0 2023-10-06T03:06:38.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 -- 172.21.15.92:0/981822597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 msgr2=0x7fc6680af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/981822597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 0x7fc6680af260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6580098e0 tx=0x7fc6580091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 -- 172.21.15.92:0/981822597 shutdown_connections 2023-10-06T03:06:38.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/981822597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f4b0 0x7fc6680af260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/981822597 >> v2:172.21.15.136:3300/0 conn(0x7fc66813f880 0x7fc6680af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 -- 172.21.15.92:0/981822597 >> 172.21.15.92:0/981822597 conn(0x7fc66806ddb0 msgr2=0x7fc66806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:38.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 -- 172.21.15.92:0/981822597 shutdown_connections 2023-10-06T03:06:38.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.792+0000 7fc66d0b1700 1 -- 172.21.15.92:0/981822597 wait complete. 2023-10-06T03:06:38.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.793+0000 7fc66d0b1700 1 Processor -- start 2023-10-06T03:06:38.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 -- start start 2023-10-06T03:06:38.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6680c8120 0x7fc6681671a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 0x7fc668169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f880 0x7fc668165f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc668167c70 con 0x7fc66813f4b0 2023-10-06T03:06:38.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6681676e0 con 0x7fc6680c8120 2023-10-06T03:06:38.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc66d0b1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc668167900 con 0x7fc66813f880 2023-10-06T03:06:38.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc6677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 0x7fc668169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.794+0000 7fc6677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 0x7fc668169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58894/0 (socket says 172.21.15.92:58894) 2023-10-06T03:06:38.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 -- 172.21.15.92:0/452941519 learned_addr learned my addr 172.21.15.92:0/452941519 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:38.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc66c8b0700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f880 0x7fc668165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc667fff700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6680c8120 0x7fc6681671a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 -- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f880 msgr2=0x7fc668165f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f880 0x7fc668165f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 -- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6680c8120 msgr2=0x7fc6681671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:38.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6680c8120 0x7fc6681671a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:38.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 -- 172.21.15.92:0/452941519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0094f0 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc6677fe700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 0x7fc668169d30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc65c010ec0 tx=0x7fc65c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c009730 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c00ed40 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc658009420 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c011660 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.795+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66816a270 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.796+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66804c3c0 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.797+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc65c011a50 con 0x7fc66813f4b0 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.797+0000 7fc64ffff700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc654067170 0x7fc654069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:38.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.797+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc65c086c50 con 0x7fc66813f4b0 2023-10-06T03:06:38.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.797+0000 7fc667fff700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc654067170 0x7fc654069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:38.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.798+0000 7fc667fff700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc654067170 0x7fc654069620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc6580093f0 tx=0x7fc658017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:38.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:38.802+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc65c08c020 con 0x7fc66813f4b0 2023-10-06T03:06:39.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.091+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc6680b28b0 con 0x7fc66813f4b0 2023-10-06T03:06:39.093 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:06:39.092+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:06:39.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.099+0000 7fc64ffff700 1 -- 172.21.15.92:0/452941519 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc65c01e090 con 0x7fc66813f4b0 2023-10-06T03:06:39.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc654067170 msgr2=0x7fc654069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc654067170 0x7fc654069620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc6580093f0 tx=0x7fc658017000 comp rx=0 tx=0).stop 2023-10-06T03:06:39.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 msgr2=0x7fc668169d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 0x7fc668169d30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc65c010ec0 tx=0x7fc65c009250 comp rx=0 tx=0).stop 2023-10-06T03:06:39.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 shutdown_connections 2023-10-06T03:06:39.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc654067170 0x7fc654069620 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc66813f880 0x7fc668165f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6680c8120 0x7fc6681671a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 --2- 172.21.15.92:0/452941519 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc66813f4b0 0x7fc668169d30 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 >> 172.21.15.92:0/452941519 conn(0x7fc66806ddb0 msgr2=0x7fc6680b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 shutdown_connections 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.101+0000 7fc66d0b1700 1 -- 172.21.15.92:0/452941519 wait complete. 2023-10-06T03:06:39.104 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:06:39.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:06:39.135 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:06:39.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.232+0000 7fad64105700 1 Processor -- start 2023-10-06T03:06:39.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.232+0000 7fad64105700 1 -- start start 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.233+0000 7fad64105700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.233+0000 7fad64105700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c06bdb0 con 0x7fad5c069d60 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.233+0000 7fad64105700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c0cb370 con 0x7fad5c0c83c0 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.233+0000 7fad64105700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c159f60 con 0x7fad5c0aca80 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.233+0000 7fad616a0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad5c069d60 0x7fad5c0c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39358/0 (socket says 172.21.15.92:39358) 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.233+0000 7fad616a0700 1 -- 172.21.15.92:0/1564416387 learned_addr learned my addr 172.21.15.92:0/1564416387 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad61ea1700 1 --2- 172.21.15.92:0/1564416387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad60e9f700 1 -- 172.21.15.92:0/1564416387 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661722362 0 0) 0x7fad5c06bdb0 con 0x7fad5c069d60 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad60e9f700 1 -- 172.21.15.92:0/1564416387 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad48003570 con 0x7fad5c069d60 2023-10-06T03:06:39.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad60e9f700 1 -- 172.21.15.92:0/1564416387 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262404619 0 0) 0x7fad5c0cb370 con 0x7fad5c0c83c0 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad61ea1700 1 -- 172.21.15.92:0/1564416387 >> v1:172.21.15.136:6789/0 conn(0x7fad5c0c83c0 legacy=0x7fad5c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad61ea1700 1 -- 172.21.15.92:0/1564416387 >> v1:172.21.15.92:6789/0 conn(0x7fad5c069d60 legacy=0x7fad5c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad61ea1700 1 -- 172.21.15.92:0/1564416387 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c06b510 con 0x7fad5c0aca80 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad61ea1700 1 --2- 172.21.15.92:0/1564416387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c06aba0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad58009ee0 tx=0x7fad58009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f60459ad572d1a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad60e9f700 1 -- 172.21.15.92:0/1564416387 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800e070 con 0x7fad5c0aca80 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad60e9f700 1 -- 172.21.15.92:0/1564416387 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5800bc30 con 0x7fad5c0aca80 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.234+0000 7fad60e9f700 1 -- 172.21.15.92:0/1564416387 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58012430 con 0x7fad5c0aca80 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 -- 172.21.15.92:0/1564416387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 msgr2=0x7fad5c06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 --2- 172.21.15.92:0/1564416387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c06aba0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad58009ee0 tx=0x7fad58009780 comp rx=0 tx=0).stop 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 -- 172.21.15.92:0/1564416387 shutdown_connections 2023-10-06T03:06:39.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 --2- 172.21.15.92:0/1564416387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c06aba0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 -- 172.21.15.92:0/1564416387 >> 172.21.15.92:0/1564416387 conn(0x7fad5c0cd670 msgr2=0x7fad5c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 -- 172.21.15.92:0/1564416387 shutdown_connections 2023-10-06T03:06:39.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.235+0000 7fad64105700 1 -- 172.21.15.92:0/1564416387 wait complete. 2023-10-06T03:06:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.236+0000 7fad64105700 1 Processor -- start 2023-10-06T03:06:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.236+0000 7fad64105700 1 -- start start 2023-10-06T03:06:39.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5c069d60 0x7fad5c0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 0x7fad5c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0c5760 con 0x7fad5c0c83c0 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0c4b50 con 0x7fad5c069d60 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0c4e30 con 0x7fad5c0aca80 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 0x7fad5c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 0x7fad5c0b0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58924/0 (socket says 172.21.15.92:58924) 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 -- 172.21.15.92:0/1219449205 learned_addr learned my addr 172.21.15.92:0/1219449205 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad616a0700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad61ea1700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5c069d60 0x7fad5c0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:39.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 -- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 msgr2=0x7fad5c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 -- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5c069d60 msgr2=0x7fad5c0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5c069d60 0x7fad5c0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 -- 172.21.15.92:0/1219449205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58009a20 con 0x7fad5c0c83c0 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad61ea1700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5c069d60 0x7fad5c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad626a2700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 0x7fad5c0b0950 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad54001eb0 tx=0x7fad54007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54011590 con 0x7fad5c0c83c0 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad54011710 con 0x7fad5c0c83c0 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c0c5110 con 0x7fad5c0c83c0 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.237+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54011ad0 con 0x7fad5c0c83c0 2023-10-06T03:06:39.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.238+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c169e00 con 0x7fad5c0c83c0 2023-10-06T03:06:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.239+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c0c5910 con 0x7fad5c0c83c0 2023-10-06T03:06:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.239+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fad5401b3b0 con 0x7fad5c0c83c0 2023-10-06T03:06:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.239+0000 7fad51ffb700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad40067170 0x7fad40069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.239+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fad54083e40 con 0x7fad5c0c83c0 2023-10-06T03:06:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.239+0000 7fad61ea1700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad40067170 0x7fad40069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:39.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.240+0000 7fad61ea1700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad40067170 0x7fad40069620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fad580099f0 tx=0x7fad58019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:39.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.243+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad5c0c5910 con 0x7fad5c0c83c0 2023-10-06T03:06:39.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:39.475+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fad5c04c3c0 con 0x7fad5c0c83c0 2023-10-06T03:06:40.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.099+0000 7fad51ffb700 1 -- 172.21.15.92:0/1219449205 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7fad54017020 con 0x7fad5c0c83c0 2023-10-06T03:06:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad40067170 msgr2=0x7fad40069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad40067170 0x7fad40069620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fad580099f0 tx=0x7fad58019000 comp rx=0 tx=0).stop 2023-10-06T03:06:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 msgr2=0x7fad5c0b0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 0x7fad5c0b0950 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad54001eb0 tx=0x7fad54007eb0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 shutdown_connections 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad40067170 0x7fad40069620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad5c0c83c0 0x7fad5c0b0950 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5c069d60 0x7fad5c0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 --2- 172.21.15.92:0/1219449205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5c0aca80 0x7fad5c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 >> 172.21.15.92:0/1219449205 conn(0x7fad5c0cd670 msgr2=0x7fad5c159220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 shutdown_connections 2023-10-06T03:06:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.101+0000 7fad64105700 1 -- 172.21.15.92:0/1219449205 wait complete. 2023-10-06T03:06:40.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:06:40.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.272+0000 7f5d056bf700 1 Processor -- start 2023-10-06T03:06:40.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.272+0000 7f5d056bf700 1 -- start start 2023-10-06T03:06:40.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5d056bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5d00155360 0x7f5d000b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5d056bf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0015c320 con 0x7f5d000ced30 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5d056bf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0015b710 con 0x7f5d000b40f0 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5d056bf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d000b4590 con 0x7f5d00155360 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5cff7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5d000ced30 0x7f5d000cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37958/0 (socket says 172.21.15.92:37958) 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5cff7fe700 1 -- 172.21.15.92:0/2966570901 learned_addr learned my addr 172.21.15.92:0/2966570901 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.273+0000 7f5cfeffd700 1 --2- 172.21.15.92:0/2966570901 >> v2:172.21.15.92:3300/0 conn(0x7f5d00155360 0x7f5d000b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfe7fc700 1 -- 172.21.15.92:0/2966570901 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521403025 0 0) 0x7f5d0015c320 con 0x7f5d000ced30 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfe7fc700 1 -- 172.21.15.92:0/2966570901 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cec003570 con 0x7f5d000ced30 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfeffd700 1 -- 172.21.15.92:0/2966570901 >> v1:172.21.15.136:6789/0 conn(0x7f5d000b40f0 legacy=0x7f5d000ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfeffd700 1 -- 172.21.15.92:0/2966570901 >> v1:172.21.15.92:6789/0 conn(0x7f5d000ced30 legacy=0x7f5d000cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfeffd700 1 -- 172.21.15.92:0/2966570901 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0015b9f0 con 0x7f5d00155360 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfeffd700 1 --2- 172.21.15.92:0/2966570901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d000b3bb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098e0 tx=0x7f5cf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d55bb8339d4a73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfe7fc700 1 -- 172.21.15.92:0/2966570901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000b640 con 0x7f5d00155360 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfe7fc700 1 -- 172.21.15.92:0/2966570901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf000b7a0 con 0x7f5d00155360 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5cfe7fc700 1 -- 172.21.15.92:0/2966570901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000bb40 con 0x7f5d00155360 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5d056bf700 1 -- 172.21.15.92:0/2966570901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 msgr2=0x7f5d000b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2966570901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d000b3bb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098e0 tx=0x7f5cf00091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5d056bf700 1 -- 172.21.15.92:0/2966570901 shutdown_connections 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2966570901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d000b3bb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.274+0000 7f5d056bf700 1 -- 172.21.15.92:0/2966570901 >> 172.21.15.92:0/2966570901 conn(0x7f5d0006baf0 msgr2=0x7f5d0006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.275+0000 7f5d056bf700 1 -- 172.21.15.92:0/2966570901 shutdown_connections 2023-10-06T03:06:40.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.275+0000 7f5d056bf700 1 -- 172.21.15.92:0/2966570901 wait complete. 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 Processor -- start 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 -- start start 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d000b40f0 0x7f5d000b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d000ced30 0x7f5d00168c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d00165e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0016a580 con 0x7f5d00155360 2023-10-06T03:06:40.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00169170 con 0x7f5d000ced30 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.276+0000 7f5d056bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00169970 con 0x7f5d000b40f0 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d00165e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d00165e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51126/0 (socket says 172.21.15.92:51126) 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 -- 172.21.15.92:0/2047205561 learned_addr learned my addr 172.21.15.92:0/2047205561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cf67fc700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d000ced30 0x7f5d00168c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cfeffd700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d000b40f0 0x7f5d000b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 -- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d000b40f0 msgr2=0x7f5d000b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d000b40f0 0x7f5d000b0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 -- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d000ced30 msgr2=0x7f5d00168c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d000ced30 0x7f5d00168c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:40.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 -- 172.21.15.92:0/2047205561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009420 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cff7fe700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d00165e20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8001460 tx=0x7f5cf8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800a890 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf800a9f0 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf800ad70 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d00169c50 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.277+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0016a210 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.278+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0016a6f0 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.279+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5cf8007300 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.279+0000 7f5cf77fe700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5ce0067170 0x7f5ce0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.279+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5cf8084db0 con 0x7f5d00155360 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.279+0000 7f5cfeffd700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5ce0067170 0x7f5ce0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:40.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.279+0000 7f5cfeffd700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5ce0067170 0x7f5ce0069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00093f0 tx=0x7f5cf0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:40.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.283+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cf8051bd0 con 0x7f5d00155360 2023-10-06T03:06:40.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:40.467+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d000718b0 con 0x7f5d00155360 2023-10-06T03:06:41.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.101+0000 7f5cf77fe700 1 -- 172.21.15.92:0/2047205561 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5cf80551f0 con 0x7f5d00155360 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5ce0067170 msgr2=0x7f5ce0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5ce0067170 0x7f5ce0069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00093f0 tx=0x7f5cf0017000 comp rx=0 tx=0).stop 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 msgr2=0x7f5d00165e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d00165e20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8001460 tx=0x7f5cf8001ae0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 shutdown_connections 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5ce0067170 0x7f5ce0069620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5d00155360 0x7f5d00165e20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d000b40f0 0x7f5d000b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 --2- 172.21.15.92:0/2047205561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d000ced30 0x7f5d00168c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.104+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 >> 172.21.15.92:0/2047205561 conn(0x7f5d0006baf0 msgr2=0x7f5d00159460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.105+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 shutdown_connections 2023-10-06T03:06:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.105+0000 7f5d056bf700 1 -- 172.21.15.92:0/2047205561 wait complete. 2023-10-06T03:06:41.107 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:06:41.119 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:06:41.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.235+0000 7fd05b112700 1 Processor -- start 2023-10-06T03:06:41.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.235+0000 7fd05b112700 1 -- start start 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd05b112700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd054159ec0 0x7fd0540b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd05b112700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd0540b53d0 0x7fd0540cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd05b112700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540b5b00 con 0x7fd0540cce40 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd05b112700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540b5850 con 0x7fd054159ec0 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd05b112700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540b5da0 con 0x7fd0540b53d0 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd054159ec0 0x7fd0540b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd054159ec0 0x7fd0540b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51130/0 (socket says 172.21.15.92:51130) 2023-10-06T03:06:41.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 -- 172.21.15.92:0/116525578 learned_addr learned my addr 172.21.15.92:0/116525578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 -- 172.21.15.92:0/116525578 >> v2:172.21.15.136:3300/0 conn(0x7fd0540b53d0 msgr2=0x7fd0540cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 --2- 172.21.15.92:0/116525578 >> v2:172.21.15.136:3300/0 conn(0x7fd0540b53d0 0x7fd0540cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 -- 172.21.15.92:0/116525578 >> v1:172.21.15.136:6789/0 conn(0x7fd0540cce40 legacy=0x7fd0540cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.236+0000 7fd058eae700 1 -- 172.21.15.92:0/116525578 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540b6080 con 0x7fd054159ec0 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd058eae700 1 --2- 172.21.15.92:0/116525578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540b4e90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b64d0 tx=0x7fd050009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca740675e51073e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd04b7fe700 1 -- 172.21.15.92:0/116525578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050005c10 con 0x7fd054159ec0 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd04b7fe700 1 -- 172.21.15.92:0/116525578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050005d70 con 0x7fd054159ec0 2023-10-06T03:06:41.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd04b7fe700 1 -- 172.21.15.92:0/116525578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05000b840 con 0x7fd054159ec0 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 -- 172.21.15.92:0/116525578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 msgr2=0x7fd0540b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 --2- 172.21.15.92:0/116525578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540b4e90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b64d0 tx=0x7fd050009220 comp rx=0 tx=0).stop 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 -- 172.21.15.92:0/116525578 shutdown_connections 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 --2- 172.21.15.92:0/116525578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540b4e90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 --2- 172.21.15.92:0/116525578 >> v2:172.21.15.136:3300/0 conn(0x7fd0540b53d0 0x7fd0540cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 -- 172.21.15.92:0/116525578 >> 172.21.15.92:0/116525578 conn(0x7fd0540999f0 msgr2=0x7fd0540b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 -- 172.21.15.92:0/116525578 shutdown_connections 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.237+0000 7fd05b112700 1 -- 172.21.15.92:0/116525578 wait complete. 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.238+0000 7fd05b112700 1 Processor -- start 2023-10-06T03:06:41.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 -- start start 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0540b53d0 0x7fd05414b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0540cce40 0x7fd054069260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd054069e20 con 0x7fd054159ec0 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05409aec0 con 0x7fd0540b53d0 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd05b112700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd054068630 con 0x7fd0540cce40 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd0596af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd0596af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540697a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51142/0 (socket says 172.21.15.92:51142) 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.239+0000 7fd0596af700 1 -- 172.21.15.92:0/280516922 learned_addr learned my addr 172.21.15.92:0/280516922 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd058eae700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0540b53d0 0x7fd05414b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd04bfff700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0540cce40 0x7fd054069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:41.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd0596af700 1 -- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0540cce40 msgr2=0x7fd054069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd0596af700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0540cce40 0x7fd054069260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd0596af700 1 -- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0540b53d0 msgr2=0x7fd05414b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd0596af700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0540b53d0 0x7fd05414b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd0596af700 1 -- 172.21.15.92:0/280516922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd050009480 con 0x7fd054159ec0 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd04bfff700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0540cce40 0x7fd054069260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd0596af700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540697a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00bbe0 tx=0x7fd04c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0092e0 con 0x7fd054159ec0 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c00c6f0 con 0x7fd054159ec0 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00cab0 con 0x7fd054159ec0 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05409a590 con 0x7fd054159ec0 2023-10-06T03:06:41.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.240+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05409ab50 con 0x7fd054159ec0 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.241+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0540718b0 con 0x7fd054159ec0 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.241+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd04c012070 con 0x7fd054159ec0 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.242+0000 7fd048ff9700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd040067170 0x7fd040069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.242+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fd04c0844e0 con 0x7fd054159ec0 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.242+0000 7fd058eae700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd040067170 0x7fd040069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.242+0000 7fd058eae700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd040067170 0x7fd040069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd050003080 tx=0x7fd050015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:41.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.246+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd04c0514c0 con 0x7fd054159ec0 2023-10-06T03:06:41.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:41.435+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd054063c40 con 0x7fd054159ec0 2023-10-06T03:06:42.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.105+0000 7fd048ff9700 1 -- 172.21.15.92:0/280516922 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd04c054ae0 con 0x7fd054159ec0 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd040067170 msgr2=0x7fd040069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd040067170 0x7fd040069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd050003080 tx=0x7fd050015000 comp rx=0 tx=0).stop 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 msgr2=0x7fd0540697a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540697a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00bbe0 tx=0x7fd04c009840 comp rx=0 tx=0).stop 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 shutdown_connections 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd040067170 0x7fd040069620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd054159ec0 0x7fd0540697a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0540b53d0 0x7fd05414b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 --2- 172.21.15.92:0/280516922 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0540cce40 0x7fd054069260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 >> 172.21.15.92:0/280516922 conn(0x7fd0540999f0 msgr2=0x7fd054159390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 shutdown_connections 2023-10-06T03:06:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.109+0000 7fd05b112700 1 -- 172.21.15.92:0/280516922 wait complete. 2023-10-06T03:06:42.112 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:06:42.125 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:06:42.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.240+0000 7fb2ab59e700 1 Processor -- start 2023-10-06T03:06:42.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.240+0000 7fb2ab59e700 1 -- start start 2023-10-06T03:06:42.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2ab59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 0x7fb2ac0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2ab59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac0b2080 con 0x7fb2ac0b4e70 2023-10-06T03:06:42.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2ab59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac0cfb00 con 0x7fb2ac0b33b0 2023-10-06T03:06:42.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2ab59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0b5c40 con 0x7fb2ac0b16d0 2023-10-06T03:06:42.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2aa59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb2ac0b4e70 0x7fb2ac0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37974/0 (socket says 172.21.15.92:37974) 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2aa59c700 1 -- 172.21.15.92:0/3610033000 learned_addr learned my addr 172.21.15.92:0/3610033000 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685144165 0 0) 0x7fb2ac0b2080 con 0x7fb2ac0b4e70 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb298003570 con 0x7fb2ac0b4e70 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a1d9b700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 0x7fb2ac0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 732213140 0 0) 0x7fb298003570 con 0x7fb2ac0b4e70 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 msgr2=0x7fb2ac0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 0x7fb2ac0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 >> v1:172.21.15.136:6789/0 conn(0x7fb2ac0b33b0 legacy=0x7fb2ac0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac0b5f20 con 0x7fb2ac0b4e70 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb29c001470 con 0x7fb2ac0b4e70 2023-10-06T03:06:42.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 >> v1:172.21.15.92:6789/0 conn(0x7fb2ac0b4e70 legacy=0x7fb2ac0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 0x7fb298003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2980040a0 0x7fb298004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298004da0 0x7fb298005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0b5c40 con 0x7fb298004da0 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29800bd10 con 0x7fb2980040a0 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29800bed0 con 0x7fb2ac0b16d0 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.241+0000 7fb2aad9d700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298004da0 0x7fb298005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aa59c700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 0x7fb298003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2a1d9b700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2980040a0 0x7fb298004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 -- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 msgr2=0x7fb298003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0b16d0 0x7fb298003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 -- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2980040a0 msgr2=0x7fb298004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2980040a0 0x7fb298004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 -- 172.21.15.92:0/3610033000 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb298003570 con 0x7fb298004da0 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298004da0 0x7fb298005210 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4002f00 tx=0x7fb2a4002530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144042c9bcc102d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4016070 con 0x7fb298004da0 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a4006ba0 con 0x7fb298004da0 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2a9d9b700 1 -- 172.21.15.92:0/3610033000 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4002840 con 0x7fb298004da0 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2ab59e700 1 -- 172.21.15.92:0/3610033000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298004da0 msgr2=0x7fb298005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/3610033000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb298004da0 0x7fb298005210 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4002f00 tx=0x7fb2a4002530 comp rx=0 tx=0).stop 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2aad9d700 1 -- 172.21.15.92:0/3610033000 reap_dead start 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2ab59e700 1 -- 172.21.15.92:0/3610033000 shutdown_connections 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.242+0000 7fb2ab59e700 1 -- 172.21.15.92:0/3610033000 >> 172.21.15.92:0/3610033000 conn(0x7fb2ac0d10c0 msgr2=0x7fb2ac0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:42.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.243+0000 7fb2ab59e700 1 -- 172.21.15.92:0/3610033000 shutdown_connections 2023-10-06T03:06:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.243+0000 7fb2ab59e700 1 -- 172.21.15.92:0/3610033000 wait complete. 2023-10-06T03:06:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.244+0000 7fb2ab59e700 1 Processor -- start 2023-10-06T03:06:42.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.244+0000 7fb2ab59e700 1 -- start start 2023-10-06T03:06:42.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.244+0000 7fb2ab59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2ac1533e0 0x7fb2ac0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.244+0000 7fb2ab59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 0x7fb2ac0a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.244+0000 7fb2ab59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0cc140 0x7fb2ac0a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2ab59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0a8330 con 0x7fb2ac1537b0 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2ab59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0cbe40 con 0x7fb2ac1533e0 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2ab59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0a7a40 con 0x7fb2ac0cc140 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 0x7fb2ac0a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 0x7fb2ac0a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51164/0 (socket says 172.21.15.92:51164) 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 -- 172.21.15.92:0/1215627052 learned_addr learned my addr 172.21.15.92:0/1215627052 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2aa59c700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2ac1533e0 0x7fb2ac0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2aad9d700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0cc140 0x7fb2ac0a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 -- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0cc140 msgr2=0x7fb2ac0a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0cc140 0x7fb2ac0a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 -- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2ac1533e0 msgr2=0x7fb2ac0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2ac1533e0 0x7fb2ac0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 -- 172.21.15.92:0/1215627052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac0a7ce0 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a1d9b700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 0x7fb2ac0a8850 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb294010aa0 tx=0x7fb294008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294010ee0 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29400e8d0 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac0a7fc0 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400ec70 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.245+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac16d210 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.246+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac0cbfb0 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.247+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb294014640 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.247+0000 7fb2a2ffd700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb28c067170 0x7fb28c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.247+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb294084d80 con 0x7fb2ac1537b0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.247+0000 7fb2aa59c700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb28c067170 0x7fb28c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.247+0000 7fb2aa59c700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb28c067170 0x7fb28c069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0010e0 tx=0x7fb29c001e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:42.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.251+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb294051f00 con 0x7fb2ac1537b0 2023-10-06T03:06:42.429 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:06:42.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:06:42.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.442+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb2ac0c92d0 con 0x7fb2ac1537b0 2023-10-06T03:06:42.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.448+0000 7fb2a2ffd700 1 -- 172.21.15.92:0/1215627052 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb294055520 con 0x7fb2ac1537b0 2023-10-06T03:06:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb28c067170 msgr2=0x7fb28c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb28c067170 0x7fb28c069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0010e0 tx=0x7fb29c001e70 comp rx=0 tx=0).stop 2023-10-06T03:06:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 msgr2=0x7fb2ac0a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 0x7fb2ac0a8850 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb294010aa0 tx=0x7fb294008040 comp rx=0 tx=0).stop 2023-10-06T03:06:42.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 shutdown_connections 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb28c067170 0x7fb28c069620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2ac0cc140 0x7fb2ac0a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2ac1533e0 0x7fb2ac0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 --2- 172.21.15.92:0/1215627052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2ac1537b0 0x7fb2ac0a8850 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 >> 172.21.15.92:0/1215627052 conn(0x7fb2ac0d10c0 msgr2=0x7fb2ac0c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 shutdown_connections 2023-10-06T03:06:42.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.450+0000 7fb2ab59e700 1 -- 172.21.15.92:0/1215627052 wait complete. 2023-10-06T03:06:42.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-06T03:06:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.650+0000 7fb54ca77700 1 Processor -- start 2023-10-06T03:06:42.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.651+0000 7fb54ca77700 1 -- start start 2023-10-06T03:06:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.651+0000 7fb54ca77700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb5480b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.651+0000 7fb54ca77700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb5480b0a00 0x7fb54815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.651+0000 7fb54ca77700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54815a980 con 0x7fb548157270 2023-10-06T03:06:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.651+0000 7fb54ca77700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54815a050 con 0x7fb5480b0a00 2023-10-06T03:06:42.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.651+0000 7fb54ca77700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815a330 con 0x7fb548068aa0 2023-10-06T03:06:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb546ffd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb5480b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb547fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb5480b0a00 0x7fb54815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb5477fe700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb548157270 0x7fb548068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:37686/0 (socket says 172.21.15.92:37686) 2023-10-06T03:06:42.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb5477fe700 1 -- 172.21.15.92:0/3849156341 learned_addr learned my addr 172.21.15.92:0/3849156341 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb546ffd700 1 -- 172.21.15.92:0/3849156341 >> v2:172.21.15.136:3300/0 conn(0x7fb5480b0a00 msgr2=0x7fb54815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb546ffd700 1 --2- 172.21.15.92:0/3849156341 >> v2:172.21.15.136:3300/0 conn(0x7fb5480b0a00 0x7fb54815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb546ffd700 1 -- 172.21.15.92:0/3849156341 >> v1:172.21.15.136:6789/0 conn(0x7fb548157270 legacy=0x7fb548068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb546ffd700 1 -- 172.21.15.92:0/3849156341 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54815a610 con 0x7fb548068aa0 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb547fff700 1 --2- 172.21.15.92:0/3849156341 >> v2:172.21.15.136:3300/0 conn(0x7fb5480b0a00 0x7fb54815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb546ffd700 1 --2- 172.21.15.92:0/3849156341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb5480b04c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0098e0 tx=0x7fb53c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17790354b5afcf4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.652+0000 7fb5467fc700 1 -- 172.21.15.92:0/3849156341 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00b640 con 0x7fb548068aa0 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb5467fc700 1 -- 172.21.15.92:0/3849156341 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c00b7a0 con 0x7fb548068aa0 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb5467fc700 1 -- 172.21.15.92:0/3849156341 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00bb40 con 0x7fb548068aa0 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 -- 172.21.15.92:0/3849156341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 msgr2=0x7fb5480b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 --2- 172.21.15.92:0/3849156341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb5480b04c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0098e0 tx=0x7fb53c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 -- 172.21.15.92:0/3849156341 shutdown_connections 2023-10-06T03:06:42.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 --2- 172.21.15.92:0/3849156341 >> v2:172.21.15.136:3300/0 conn(0x7fb5480b0a00 0x7fb54815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 --2- 172.21.15.92:0/3849156341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb5480b04c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 -- 172.21.15.92:0/3849156341 >> 172.21.15.92:0/3849156341 conn(0x7fb5480cd310 msgr2=0x7fb5480cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:42.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 -- 172.21.15.92:0/3849156341 shutdown_connections 2023-10-06T03:06:42.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.653+0000 7fb54ca77700 1 -- 172.21.15.92:0/3849156341 wait complete. 2023-10-06T03:06:42.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.654+0000 7fb54ca77700 1 Processor -- start 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.654+0000 7fb54ca77700 1 -- start start 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb54ca77700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb548151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb54ca77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 0x7fb548155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb54ca77700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548157270 0x7fb54806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb54ca77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54816d9d0 con 0x7fb5480b0a00 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb54ca77700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548155560 con 0x7fb548157270 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb54ca77700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54816d380 con 0x7fb548068aa0 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 0x7fb548155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 0x7fb548155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51180/0 (socket says 172.21.15.92:51180) 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 -- 172.21.15.92:0/2214130027 learned_addr learned my addr 172.21.15.92:0/2214130027 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:42.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb5477fe700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb548151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb547fff700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548157270 0x7fb54806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 -- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 msgr2=0x7fb548151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb548151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 -- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548157270 msgr2=0x7fb54806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548157270 0x7fb54806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.655+0000 7fb546ffd700 1 -- 172.21.15.92:0/2214130027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540009420 con 0x7fb5480b0a00 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb5477fe700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb548151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb547fff700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548157270 0x7fb54806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb546ffd700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 0x7fb548155a70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb54815a470 tx=0x7fb53c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c01bda0 con 0x7fb5480b0a00 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c011420 con 0x7fb5480b0a00 2023-10-06T03:06:42.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c009420 con 0x7fb5480b0a00 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0117e0 con 0x7fb5480b0a00 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.656+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548155fb0 con 0x7fb5480b0a00 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.657+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54816d1c0 con 0x7fb5480b0a00 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.657+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb53c011b90 con 0x7fb5480b0a00 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.658+0000 7fb52f7fe700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb534067170 0x7fb534069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.658+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb53c085e50 con 0x7fb5480b0a00 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.658+0000 7fb5477fe700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb534067170 0x7fb534069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:42.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.658+0000 7fb5477fe700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb534067170 0x7fb534069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb5380021c0 tx=0x7fb538009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:42.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.662+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb53c052fe0 con 0x7fb5480b0a00 2023-10-06T03:06:42.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:42.917+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fb54804c3c0 con 0x7fb5480b0a00 2023-10-06T03:06:43.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.106+0000 7fb52f7fe700 1 -- 172.21.15.92:0/2214130027 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v235) v1 ==== 158+0+0 (secure 0 0 0) 0x7fb53c056600 con 0x7fb5480b0a00 2023-10-06T03:06:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb534067170 msgr2=0x7fb534069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb534067170 0x7fb534069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb5380021c0 tx=0x7fb538009040 comp rx=0 tx=0).stop 2023-10-06T03:06:43.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 msgr2=0x7fb548155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 0x7fb548155a70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fb54815a470 tx=0x7fb53c017000 comp rx=0 tx=0).stop 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 shutdown_connections 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb534067170 0x7fb534069620 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb548157270 0x7fb54806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb548068aa0 0x7fb548151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 --2- 172.21.15.92:0/2214130027 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5480b0a00 0x7fb548155a70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 >> 172.21.15.92:0/2214130027 conn(0x7fb5480cd310 msgr2=0x7fb5481591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 shutdown_connections 2023-10-06T03:06:43.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.108+0000 7fb54ca77700 1 -- 172.21.15.92:0/2214130027 wait complete. 2023-10-06T03:06:43.122 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ok 2023-10-06T03:06:43.123 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command 2023-10-06T03:06:43.123 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:06:43.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-06T03:06:43.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.236+0000 7fc175f27700 1 Processor -- start 2023-10-06T03:06:43.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.236+0000 7fc175f27700 1 -- start start 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc175f27700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc170159c60 0x7fc17015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc175f27700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17015db10 0x7fc170069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc175f27700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17015a160 con 0x7fc170159890 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc175f27700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17015df50 con 0x7fc170159c60 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc175f27700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700d13a0 con 0x7fc17015db10 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc170159c60 0x7fc17015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc170159890 0x7fc17015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37980/0 (socket says 172.21.15.92:37980) 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16f7fe700 1 -- 172.21.15.92:0/3136642846 learned_addr learned my addr 172.21.15.92:0/3136642846 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16effd700 1 -- 172.21.15.92:0/3136642846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17015db10 msgr2=0x7fc170069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:43.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16ffff700 1 --2- 172.21.15.92:0/3136642846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17015db10 0x7fc170069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16effd700 1 --2- 172.21.15.92:0/3136642846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17015db10 0x7fc170069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16effd700 1 -- 172.21.15.92:0/3136642846 >> v1:172.21.15.92:6789/0 conn(0x7fc170159890 legacy=0x7fc17015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16e7fc700 1 -- 172.21.15.92:0/3136642846 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247312937 0 0) 0x7fc17015a160 con 0x7fc170159890 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16effd700 1 -- 172.21.15.92:0/3136642846 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17015e140 con 0x7fc170159c60 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16effd700 1 --2- 172.21.15.92:0/3136642846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc170159c60 0x7fc17015d5d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc1580098e0 tx=0x7fc1580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30722ff56f8d3275 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16e7fc700 1 -- 172.21.15.92:0/3136642846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158009e80 con 0x7fc170159c60 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16e7fc700 1 -- 172.21.15.92:0/3136642846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15800b860 con 0x7fc170159c60 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.237+0000 7fc16e7fc700 1 -- 172.21.15.92:0/3136642846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15800bbe0 con 0x7fc170159c60 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 -- 172.21.15.92:0/3136642846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc170159c60 msgr2=0x7fc17015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:43.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 --2- 172.21.15.92:0/3136642846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc170159c60 0x7fc17015d5d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc1580098e0 tx=0x7fc1580091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 -- 172.21.15.92:0/3136642846 shutdown_connections 2023-10-06T03:06:43.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 --2- 172.21.15.92:0/3136642846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc17015db10 0x7fc170069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 --2- 172.21.15.92:0/3136642846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc170159c60 0x7fc17015d5d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 -- 172.21.15.92:0/3136642846 >> 172.21.15.92:0/3136642846 conn(0x7fc170068e20 msgr2=0x7fc1700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:43.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 -- 172.21.15.92:0/3136642846 shutdown_connections 2023-10-06T03:06:43.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.238+0000 7fc175f27700 1 -- 172.21.15.92:0/3136642846 wait complete. 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.239+0000 7fc175f27700 1 Processor -- start 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.239+0000 7fc175f27700 1 -- start start 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc175f27700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc170159890 0x7fc1700b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc175f27700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc170159c60 0x7fc1701536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc175f27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 0x7fc1700b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc175f27700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170153c30 con 0x7fc17015db10 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc175f27700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700b0a80 con 0x7fc170159890 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc175f27700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700b0cb0 con 0x7fc170159c60 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 0x7fc1700b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 0x7fc1700b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51188/0 (socket says 172.21.15.92:51188) 2023-10-06T03:06:43.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 -- 172.21.15.92:0/2566913247 learned_addr learned my addr 172.21.15.92:0/2566913247 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16effd700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc170159c60 0x7fc1701536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16f7fe700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc170159890 0x7fc1700b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 -- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc170159c60 msgr2=0x7fc1701536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc170159c60 0x7fc1701536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 -- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc170159890 msgr2=0x7fc1700b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc170159890 0x7fc1700b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 -- 172.21.15.92:0/2566913247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc158009420 con 0x7fc17015db10 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.240+0000 7fc16ffff700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 0x7fc1700b0540 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c880 tx=0x7fc160008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.241+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160011600 con 0x7fc17015db10 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.241+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160011760 con 0x7fc17015db10 2023-10-06T03:06:43.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.241+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1700b0f90 con 0x7fc17015db10 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.241+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160011b00 con 0x7fc17015db10 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.241+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1700b4d70 con 0x7fc17015db10 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.242+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1700b4f90 con 0x7fc17015db10 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.242+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc160018360 con 0x7fc17015db10 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.242+0000 7fc1677fe700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc150067170 0x7fc150069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.242+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc160084570 con 0x7fc17015db10 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.243+0000 7fc16f7fe700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc150067170 0x7fc150069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:43.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.243+0000 7fc16f7fe700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc150067170 0x7fc150069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc168001560 tx=0x7fc168009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:43.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.247+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc160054fd0 con 0x7fc17015db10 2023-10-06T03:06:43.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:43.507+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7fc17004c3c0 con 0x7fc17015db10 2023-10-06T03:06:44.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.110+0000 7fc1677fe700 1 -- 172.21.15.92:0/2566913247 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v236) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc160054b60 con 0x7fc17015db10 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc150067170 msgr2=0x7fc150069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc150067170 0x7fc150069620 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc168001560 tx=0x7fc168009040 comp rx=0 tx=0).stop 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 msgr2=0x7fc1700b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 0x7fc1700b0540 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc16000c880 tx=0x7fc160008040 comp rx=0 tx=0).stop 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 shutdown_connections 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc150067170 0x7fc150069620 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc17015db10 0x7fc1700b0540 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc170159890 0x7fc1700b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 --2- 172.21.15.92:0/2566913247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc170159c60 0x7fc1701536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 >> 172.21.15.92:0/2566913247 conn(0x7fc170068e20 msgr2=0x7fc170155440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:44.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 shutdown_connections 2023-10-06T03:06:44.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.113+0000 7fc175f27700 1 -- 172.21.15.92:0/2566913247 wait complete. 2023-10-06T03:06:44.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:06:44.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.244+0000 7efd6f31a700 1 Processor -- start 2023-10-06T03:06:44.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.244+0000 7efd6f31a700 1 -- start start 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6f31a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 0x7efd680c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6f31a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efd6806c0a0 0x7efd680c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6f31a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd6815c2a0 con 0x7efd6806bcd0 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6f31a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd680c9880 con 0x7efd6806c0a0 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6f31a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6815b690 con 0x7efd68141500 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6d0b6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efd6806bcd0 0x7efd68140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37996/0 (socket says 172.21.15.92:37996) 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.245+0000 7efd6d0b6700 1 -- 172.21.15.92:0/4257262228 learned_addr learned my addr 172.21.15.92:0/4257262228 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:44.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574905414 0 0) 0x7efd6815c2a0 con 0x7efd6806bcd0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd54003570 con 0x7efd6806bcd0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d8b7700 1 --2- 172.21.15.92:0/4257262228 >> v2:172.21.15.136:3300/0 conn(0x7efd6806c0a0 0x7efd680c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6c8b5700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 0x7efd680c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3007458123 0 0) 0x7efd54003570 con 0x7efd6806bcd0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 msgr2=0x7efd680c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 0x7efd680c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 >> v2:172.21.15.136:3300/0 conn(0x7efd6806c0a0 msgr2=0x7efd680c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 --2- 172.21.15.92:0/4257262228 >> v2:172.21.15.136:3300/0 conn(0x7efd6806c0a0 0x7efd680c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6815b970 con 0x7efd6806bcd0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7efd64001470 con 0x7efd6806bcd0 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 >> v1:172.21.15.92:6789/0 conn(0x7efd6806bcd0 legacy=0x7efd68140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003a60 0x7efd54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 0x7efd54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54004d50 0x7efd540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd680c9880 con 0x7efd54003a60 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6815b690 con 0x7efd54004d50 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5400ba60 con 0x7efd68141500 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003a60 0x7efd54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6c8b5700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 0x7efd54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d8b7700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54004d50 0x7efd540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d0b6700 1 -- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 msgr2=0x7efd54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd68141500 0x7efd54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d0b6700 1 -- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54004d50 msgr2=0x7efd540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54004d50 0x7efd540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d0b6700 1 -- 172.21.15.92:0/4257262228 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54003570 con 0x7efd54003a60 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.246+0000 7efd6d8b7700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54004d50 0x7efd540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003a60 0x7efd54003e70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c3c0 tx=0x7efd6806e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcce1980caa3d545 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6400cc30 con 0x7efd54003a60 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6400cd90 con 0x7efd54003a60 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd5ffff700 1 -- 172.21.15.92:0/4257262228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64005b70 con 0x7efd54003a60 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6f31a700 1 -- 172.21.15.92:0/4257262228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003a60 msgr2=0x7efd54003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6f31a700 1 --2- 172.21.15.92:0/4257262228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd54003a60 0x7efd54003e70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c3c0 tx=0x7efd6806e470 comp rx=0 tx=0).stop 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6d8b7700 1 -- 172.21.15.92:0/4257262228 reap_dead start 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6f31a700 1 -- 172.21.15.92:0/4257262228 shutdown_connections 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6f31a700 1 -- 172.21.15.92:0/4257262228 >> 172.21.15.92:0/4257262228 conn(0x7efd680ce600 msgr2=0x7efd680cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:44.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6f31a700 1 -- 172.21.15.92:0/4257262228 shutdown_connections 2023-10-06T03:06:44.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.247+0000 7efd6f31a700 1 -- 172.21.15.92:0/4257262228 wait complete. 2023-10-06T03:06:44.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.248+0000 7efd6f31a700 1 Processor -- start 2023-10-06T03:06:44.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.248+0000 7efd6f31a700 1 -- start start 2023-10-06T03:06:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6f31a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54003a60 0x7efd6813f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6f31a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd6815a6a0 0x7efd6815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6f31a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 0x7efd6806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6f31a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6815a210 con 0x7efd6806abb0 2023-10-06T03:06:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6f31a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd680c4ff0 con 0x7efd54003a60 2023-10-06T03:06:44.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6f31a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd68159600 con 0x7efd6815a6a0 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6d8b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 0x7efd6806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6d8b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 0x7efd6806b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51220/0 (socket says 172.21.15.92:51220) 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.249+0000 7efd6d8b7700 1 -- 172.21.15.92:0/884424892 learned_addr learned my addr 172.21.15.92:0/884424892 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6c8b5700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd6815a6a0 0x7efd6815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54003a60 0x7efd6813f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d8b7700 1 -- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd6815a6a0 msgr2=0x7efd6815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d8b7700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd6815a6a0 0x7efd6815ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d8b7700 1 -- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54003a60 msgr2=0x7efd6813f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d8b7700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54003a60 0x7efd6813f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d8b7700 1 -- 172.21.15.92:0/884424892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd681598e0 con 0x7efd6806abb0 2023-10-06T03:06:44.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6c8b5700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd6815a6a0 0x7efd6815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54003a60 0x7efd6813f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6d8b7700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 0x7efd6806b020 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efd680cf210 tx=0x7efd6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6000e050 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6000f040 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd68159bc0 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd60010ec0 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.250+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6813f890 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.251+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6815a380 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.251+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7efd60003020 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.252+0000 7efd5d7fa700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efd4c067170 0x7efd4c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.252+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5292+0+0 (secure 0 0 0) 0x7efd60084b80 con 0x7efd6806abb0 2023-10-06T03:06:44.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.252+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efd4c067170 0x7efd4c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.252+0000 7efd6d0b6700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efd4c067170 0x7efd4c069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c3c0 tx=0x7efd64015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:44.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.256+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd60051b20 con 0x7efd6806abb0 2023-10-06T03:06:44.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.440+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efd6804c3c0 con 0x7efd6806abb0 2023-10-06T03:06:44.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.446+0000 7efd5d7fa700 1 -- 172.21.15.92:0/884424892 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7efd6001a0c0 con 0x7efd6806abb0 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efd4c067170 msgr2=0x7efd4c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efd4c067170 0x7efd4c069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efd6400c3c0 tx=0x7efd64015000 comp rx=0 tx=0).stop 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 msgr2=0x7efd6806b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 0x7efd6806b020 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efd680cf210 tx=0x7efd6000b040 comp rx=0 tx=0).stop 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 shutdown_connections 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efd4c067170 0x7efd4c069620 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd6806abb0 0x7efd6806b020 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd54003a60 0x7efd6813f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 --2- 172.21.15.92:0/884424892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd6815a6a0 0x7efd6815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 >> 172.21.15.92:0/884424892 conn(0x7efd680ce600 msgr2=0x7efd6813fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:44.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 shutdown_connections 2023-10-06T03:06:44.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.448+0000 7efd6f31a700 1 -- 172.21.15.92:0/884424892 wait complete. 2023-10-06T03:06:44.462 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:06:44.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:06:44.501 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:06:44.548 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:06:44.548 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:06:44.548 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:44.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.661+0000 7f4282614700 1 Processor -- start 2023-10-06T03:06:44.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.661+0000 7f4282614700 1 -- start start 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f4282614700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f427c159890 0x7f427c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f4282614700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427c15a160 con 0x7f427c069560 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f4282614700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427c0d13a0 con 0x7f427c159c60 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f4282614700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427c15e140 con 0x7f427c159890 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f4280bb1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f427c069560 0x7f427c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38008/0 (socket says 172.21.15.92:38008) 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f4280bb1700 1 -- 172.21.15.92:0/707021721 learned_addr learned my addr 172.21.15.92:0/707021721 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360686097 0 0) 0x7f427c15a160 con 0x7f427c069560 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.662+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4264003570 con 0x7f427c069560 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 --2- 172.21.15.92:0/707021721 >> v2:172.21.15.136:3300/0 conn(0x7f427c159890 0x7f427c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853829590 0 0) 0x7f4264003570 con 0x7f427c069560 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 >> v2:172.21.15.136:3300/0 conn(0x7f427c159890 msgr2=0x7f427c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 --2- 172.21.15.92:0/707021721 >> v2:172.21.15.136:3300/0 conn(0x7f427c159890 0x7f427c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 >> v1:172.21.15.136:6789/0 conn(0x7f427c159c60 legacy=0x7f427c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c15a3d0 con 0x7f427c069560 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f42740017f0 con 0x7f427c069560 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 >> v1:172.21.15.92:6789/0 conn(0x7f427c069560 legacy=0x7f427c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f4264003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4264004810 0x7f4264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4264005230 0x7f426400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c15e140 con 0x7f4264003a60 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426400c160 con 0x7f4264005230 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426400c320 con 0x7f4264004810 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f4264003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f4280bb1700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4264005230 0x7f426400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427b7fe700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4264004810 0x7f4264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 -- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4264004810 msgr2=0x7f4264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4264004810 0x7f4264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 -- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4264005230 msgr2=0x7f426400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4264005230 0x7f426400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 -- 172.21.15.92:0/707021721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4264003570 con 0x7f4264003a60 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f4280bb1700 1 -- 172.21.15.92:0/707021721 reap_dead start 2023-10-06T03:06:44.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427bfff700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f4264003e50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f426c00bb90 tx=0x7f426c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4022a5fec4f67f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00d5d0 con 0x7f4264003a60 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c00d730 con 0x7f4264003a60 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.663+0000 7f427affd700 1 -- 172.21.15.92:0/707021721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00daf0 con 0x7f4264003a60 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 -- 172.21.15.92:0/707021721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 msgr2=0x7f4264003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f4264003e50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f426c00bb90 tx=0x7f426c008040 comp rx=0 tx=0).stop 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 -- 172.21.15.92:0/707021721 shutdown_connections 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 --2- 172.21.15.92:0/707021721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f4264003e50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 -- 172.21.15.92:0/707021721 >> 172.21.15.92:0/707021721 conn(0x7f427c068e20 msgr2=0x7f427c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 -- 172.21.15.92:0/707021721 shutdown_connections 2023-10-06T03:06:44.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.664+0000 7f4282614700 1 -- 172.21.15.92:0/707021721 wait complete. 2023-10-06T03:06:44.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.665+0000 7f4282614700 1 Processor -- start 2023-10-06T03:06:44.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.665+0000 7f4282614700 1 -- start start 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4282614700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f427c149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4282614700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c0cf6f0 0x7f427c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4282614700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f427c0b3590 0x7f427c0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4282614700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0cf260 con 0x7f4264003a60 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4282614700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c169f90 con 0x7f427c0cf6f0 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4282614700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0cdca0 con 0x7f427c0b3590 2023-10-06T03:06:44.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f427c149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f427c149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51236/0 (socket says 172.21.15.92:51236) 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 -- 172.21.15.92:0/3257872885 learned_addr learned my addr 172.21.15.92:0/3257872885 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f4280bb1700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f427c0b3590 0x7f427c0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427b7fe700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c0cf6f0 0x7f427c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 -- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f427c0b3590 msgr2=0x7f427c0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f427c0b3590 0x7f427c0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 -- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c0cf6f0 msgr2=0x7f427c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c0cf6f0 0x7f427c0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.666+0000 7f427bfff700 1 -- 172.21.15.92:0/3257872885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0cdf80 con 0x7f4264003a60 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.667+0000 7f427bfff700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f427c149c20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f426c00bfd0 tx=0x7f426c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:44.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.667+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c014790 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.667+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c014950 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.667+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c0cec10 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.667+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c014d10 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.667+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c0cd0c0 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.668+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c0cf0b0 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.668+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f426c01e400 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.668+0000 7f4263fff700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4268067170 0x7f4268069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.668+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f426c086090 con 0x7f4264003a60 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.668+0000 7f427b7fe700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4268067170 0x7f4268069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:44.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.669+0000 7f427b7fe700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4268067170 0x7f4268069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4270003910 tx=0x7f4270009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:44.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.672+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f426c053170 con 0x7f4264003a60 2023-10-06T03:06:44.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.899+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f427c04c3c0 con 0x7f4264003a60 2023-10-06T03:06:44.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.900+0000 7f4263fff700 1 -- 172.21.15.92:0/3257872885 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f426c056790 con 0x7f4264003a60 2023-10-06T03:06:44.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:44.902 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4244903556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4244903556},{"type":"v1","addr":"172.21.15.136:6834","nonce":4244903556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/370241984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":370241984},{"type":"v1","addr":"172.21.15.136:6835","nonce":370241984}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2315446547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2315446547},{"type":"v1","addr":"172.21.15.92:6836","nonce":2315446547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6072,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2631145051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2631145051},{"type":"v1","addr":"172.21.15.92:6837","nonce":2631145051}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[]} 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4268067170 msgr2=0x7f4268069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4268067170 0x7f4268069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4270003910 tx=0x7f4270009040 comp rx=0 tx=0).stop 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 msgr2=0x7f427c149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f427c149c20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f426c00bfd0 tx=0x7f426c01b000 comp rx=0 tx=0).stop 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 shutdown_connections 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4268067170 0x7f4268069620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4264003a60 0x7f427c149c20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f427c0b3590 0x7f427c0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 --2- 172.21.15.92:0/3257872885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f427c0cf6f0 0x7f427c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 >> 172.21.15.92:0/3257872885 conn(0x7f427c068e20 msgr2=0x7f427c0c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 shutdown_connections 2023-10-06T03:06:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:44.902+0000 7f4282614700 1 -- 172.21.15.92:0/3257872885 wait complete. 2023-10-06T03:06:44.905 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 41 2023-10-06T03:06:44.916 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:06:44.916 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:06:44.916 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:06:44.916 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:06:44.916 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:06:44.917 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:06:44.917 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:06:44.917 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:06:44.917 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:06:44.917 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:06:44.917 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:06:44.918 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:06:44.918 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:06:44.918 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:06:44.918 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:06:44.918 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:06:44.925 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:06:44.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:06:44.926 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:06:44.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:06:44.929 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:06:44.929 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:06:44.929 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:06:44.930 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:06:44.968 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:06:44.968 DEBUG:teuthology.parallel:result is None 2023-10-06T03:06:44.969 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:06:44.969 DEBUG:teuthology.parallel:result is None 2023-10-06T03:06:44.971 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:06:44.971 DEBUG:teuthology.parallel:result is None 2023-10-06T03:06:44.972 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:06:44.972 DEBUG:teuthology.parallel:result is None 2023-10-06T03:06:44.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:06:45.041 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:06:45.041 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:06:45.054 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:06:45.055 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:06:45.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.092+0000 7f2cb2f32700 1 Processor -- start 2023-10-06T03:06:45.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.092+0000 7f2cb2f32700 1 -- start start 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb2f32700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2cac159890 0x7f2cac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb2f32700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2cac159c60 0x7f2cac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb2f32700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb2f32700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac15df50 con 0x7f2cac159890 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb2f32700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac15a160 con 0x7f2cac159c60 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb2f32700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0d13a0 con 0x7f2cac15db10 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb0cce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2cac159890 0x7f2cac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb0cce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2cac159890 0x7f2cac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51286/0 (socket says 172.21.15.92:51286) 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.093+0000 7f2cb0cce700 1 -- 172.21.15.92:0/2291627288 learned_addr learned my addr 172.21.15.92:0/2291627288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:45.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cabfff700 1 --2- 172.21.15.92:0/2291627288 >> v2:172.21.15.136:3300/0 conn(0x7f2cac159c60 0x7f2cac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb14cf700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb0cce700 1 -- 172.21.15.92:0/2291627288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 msgr2=0x7f2cac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb0cce700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb0cce700 1 -- 172.21.15.92:0/2291627288 >> v2:172.21.15.136:3300/0 conn(0x7f2cac159c60 msgr2=0x7f2cac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb0cce700 1 --2- 172.21.15.92:0/2291627288 >> v2:172.21.15.136:3300/0 conn(0x7f2cac159c60 0x7f2cac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb0cce700 1 -- 172.21.15.92:0/2291627288 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac15e140 con 0x7f2cac159890 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb14cf700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cb0cce700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159890 0x7f2cac15abd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0b67f0 tx=0x7f2c9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d4b1c45a7567ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cab7fe700 1 -- 172.21.15.92:0/2291627288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c009e70 con 0x7f2cac159890 2023-10-06T03:06:45.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cab7fe700 1 -- 172.21.15.92:0/2291627288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c00b860 con 0x7f2cac159890 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.094+0000 7f2cab7fe700 1 -- 172.21.15.92:0/2291627288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00bc00 con 0x7f2cac159890 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2291627288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159890 msgr2=0x7f2cac15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159890 0x7f2cac15abd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0b67f0 tx=0x7f2c9c009250 comp rx=0 tx=0).stop 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2291627288 shutdown_connections 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2291627288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159890 0x7f2cac15abd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2291627288 >> v2:172.21.15.136:3300/0 conn(0x7f2cac159c60 0x7f2cac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2291627288 >> 172.21.15.92:0/2291627288 conn(0x7f2cac068e20 msgr2=0x7f2cac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2291627288 shutdown_connections 2023-10-06T03:06:45.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.095+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2291627288 wait complete. 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.096+0000 7f2cb2f32700 1 Processor -- start 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.096+0000 7f2cb2f32700 1 -- start start 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cac159890 0x7f2cac151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 0x7f2cac0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac0b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac169b40 con 0x7f2cac159c60 2023-10-06T03:06:45.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac169d50 con 0x7f2cac159890 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac16a030 con 0x7f2cac15db10 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 0x7f2cac0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 0x7f2cac0b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51302/0 (socket says 172.21.15.92:51302) 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 -- 172.21.15.92:0/2219111331 learned_addr learned my addr 172.21.15.92:0/2219111331 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb0cce700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cac159890 0x7f2cac151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb14cf700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac0b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 -- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 msgr2=0x7f2cac0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac0b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 -- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cac159890 msgr2=0x7f2cac151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cac159890 0x7f2cac151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:45.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 -- 172.21.15.92:0/2219111331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0094f0 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cabfff700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 0x7f2cac0b5940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000ec80 tx=0x7f2ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000c850 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca000c9b0 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac16a310 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.097+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000cd50 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.098+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac16a8d0 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.098+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac16aaf0 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.099+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2ca0013680 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.099+0000 7f2ca8ff9700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c98067170 0x7f2c98069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.099+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2ca00848c0 con 0x7f2cac159c60 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.099+0000 7f2cb0cce700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c98067170 0x7f2c98069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:45.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.100+0000 7f2cb0cce700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c98067170 0x7f2c98069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0b67f0 tx=0x7f2c9c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:45.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.103+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ca0051a40 con 0x7f2cac159c60 2023-10-06T03:06:45.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:45.306+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2cac04c3c0 con 0x7f2cac159c60 2023-10-06T03:06:46.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.133+0000 7f2ca8ff9700 1 -- 172.21.15.92:0/2219111331 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2ca0055060 con 0x7f2cac159c60 2023-10-06T03:06:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c98067170 msgr2=0x7f2c98069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c98067170 0x7f2c98069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0b67f0 tx=0x7f2c9c017000 comp rx=0 tx=0).stop 2023-10-06T03:06:46.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 msgr2=0x7f2cac0b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 0x7f2cac0b5940 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000ec80 tx=0x7f2ca00091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 shutdown_connections 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c98067170 0x7f2c98069620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2cac15db10 0x7f2cac0b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2cac159890 0x7f2cac151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 --2- 172.21.15.92:0/2219111331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2cac159c60 0x7f2cac0b5940 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.135+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 >> 172.21.15.92:0/2219111331 conn(0x7f2cac068e20 msgr2=0x7f2cac0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.136+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 shutdown_connections 2023-10-06T03:06:46.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.136+0000 7f2cb2f32700 1 -- 172.21.15.92:0/2219111331 wait complete. 2023-10-06T03:06:46.139 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:06:46.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:06:46.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.266+0000 7f78c93a3700 1 Processor -- start 2023-10-06T03:06:46.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.266+0000 7f78c93a3700 1 -- start start 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c93a3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78c4159890 0x7f78c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c93a3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f78c4159c60 0x7f78c415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c93a3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c93a3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c415df50 con 0x7f78c4159890 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c93a3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c415a160 con 0x7f78c4159c60 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c93a3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c40d13a0 con 0x7f78c415db10 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78c4159890 0x7f78c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78c4159890 0x7f78c415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51310/0 (socket says 172.21.15.92:51310) 2023-10-06T03:06:46.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.267+0000 7f78c2ffd700 1 -- 172.21.15.92:0/2029287181 learned_addr learned my addr 172.21.15.92:0/2029287181 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78ba7fc700 1 --2- 172.21.15.92:0/2029287181 >> v2:172.21.15.136:3300/0 conn(0x7f78c4159c60 0x7f78c415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c37fe700 1 --2- 172.21.15.92:0/2029287181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c2ffd700 1 -- 172.21.15.92:0/2029287181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 msgr2=0x7f78c4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c2ffd700 1 --2- 172.21.15.92:0/2029287181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c2ffd700 1 -- 172.21.15.92:0/2029287181 >> v2:172.21.15.136:3300/0 conn(0x7f78c4159c60 msgr2=0x7f78c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c2ffd700 1 --2- 172.21.15.92:0/2029287181 >> v2:172.21.15.136:3300/0 conn(0x7f78c4159c60 0x7f78c415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c2ffd700 1 -- 172.21.15.92:0/2029287181 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c415e140 con 0x7f78c4159890 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78ba7fc700 1 --2- 172.21.15.92:0/2029287181 >> v2:172.21.15.136:3300/0 conn(0x7f78c4159c60 0x7f78c415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c2ffd700 1 --2- 172.21.15.92:0/2029287181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159890 0x7f78c415abd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78c40b67f0 tx=0x7f78b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1504868b64289e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c27fc700 1 -- 172.21.15.92:0/2029287181 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400b860 con 0x7f78c4159890 2023-10-06T03:06:46.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c27fc700 1 -- 172.21.15.92:0/2029287181 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b400b9c0 con 0x7f78c4159890 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c27fc700 1 -- 172.21.15.92:0/2029287181 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400bd60 con 0x7f78c4159890 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c93a3700 1 -- 172.21.15.92:0/2029287181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159890 msgr2=0x7f78c415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.268+0000 7f78c93a3700 1 --2- 172.21.15.92:0/2029287181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159890 0x7f78c415abd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78c40b67f0 tx=0x7f78b4009250 comp rx=0 tx=0).stop 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 -- 172.21.15.92:0/2029287181 shutdown_connections 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 --2- 172.21.15.92:0/2029287181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 --2- 172.21.15.92:0/2029287181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159890 0x7f78c415abd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 --2- 172.21.15.92:0/2029287181 >> v2:172.21.15.136:3300/0 conn(0x7f78c4159c60 0x7f78c415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 -- 172.21.15.92:0/2029287181 >> 172.21.15.92:0/2029287181 conn(0x7f78c4068e20 msgr2=0x7f78c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 -- 172.21.15.92:0/2029287181 shutdown_connections 2023-10-06T03:06:46.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.269+0000 7f78c93a3700 1 -- 172.21.15.92:0/2029287181 wait complete. 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.270+0000 7f78c93a3700 1 Processor -- start 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.270+0000 7f78c93a3700 1 -- start start 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c4159890 0x7f78c4150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 0x7f78c4169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c416a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c4153e30 con 0x7f78c4159c60 2023-10-06T03:06:46.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c41511d0 con 0x7f78c4159890 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c416a7a0 con 0x7f78c415db10 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 0x7f78c4169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 0x7f78c4169bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51314/0 (socket says 172.21.15.92:51314) 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 -- 172.21.15.92:0/1486324502 learned_addr learned my addr 172.21.15.92:0/1486324502 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c2ffd700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c4159890 0x7f78c4150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c37fe700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c416a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 -- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 msgr2=0x7f78c416a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c416a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 -- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c4159890 msgr2=0x7f78c4150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c4159890 0x7f78c4150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 -- 172.21.15.92:0/1486324502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40094f0 con 0x7f78c4159c60 2023-10-06T03:06:46.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78ba7fc700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 0x7f78c4169bc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00c830 tx=0x7f78ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac011600 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac011760 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c416aa80 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac011b20 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.271+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c4153ac0 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.272+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c404c3c0 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.273+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f78ac018360 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.273+0000 7f78bb7fe700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78a4067170 0x7f78a4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.273+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f78ac084750 con 0x7f78c4159c60 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.273+0000 7f78c2ffd700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78a4067170 0x7f78a4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.273+0000 7f78c2ffd700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78a4067170 0x7f78a4069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78c40b67f0 tx=0x7f78b4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.277+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78ac051810 con 0x7f78c4159c60 2023-10-06T03:06:46.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.534+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f78c4153f70 con 0x7f78c4159c60 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.534+0000 7f78bb7fe700 1 -- 172.21.15.92:0/1486324502 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f78ac054e30 con 0x7f78c4159c60 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:06:46.536 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.537 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:06:46.538 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.539 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:06:46.540 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.542 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.543 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:06:46.545 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:06:46.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.547 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:06:46.548 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:06:46.549 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.550 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:46.551 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:06:46.551 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:06:46.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78a4067170 msgr2=0x7f78a4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78a4067170 0x7f78a4069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78c40b67f0 tx=0x7f78b4017000 comp rx=0 tx=0).stop 2023-10-06T03:06:46.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 msgr2=0x7f78c4169bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 0x7f78c4169bc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00c830 tx=0x7f78ac008040 comp rx=0 tx=0).stop 2023-10-06T03:06:46.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 shutdown_connections 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78a4067170 0x7f78a4069620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78c415db10 0x7f78c416a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78c4159890 0x7f78c4150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 --2- 172.21.15.92:0/1486324502 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78c4159c60 0x7f78c4169bc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 >> 172.21.15.92:0/1486324502 conn(0x7f78c4068e20 msgr2=0x7f78c40c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 shutdown_connections 2023-10-06T03:06:46.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.536+0000 7f78c93a3700 1 -- 172.21.15.92:0/1486324502 wait complete. 2023-10-06T03:06:46.554 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:46.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.665+0000 7f0316f19700 1 Processor -- start 2023-10-06T03:06:46.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.665+0000 7f0316f19700 1 -- start start 2023-10-06T03:06:46.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0316f19700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f031006bcd0 0x7f03100caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0316f19700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f031015df50 0x7f031015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0316f19700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031015d7e0 con 0x7f031006c0a0 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0316f19700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031015dac0 con 0x7f031006bcd0 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0316f19700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031015aa30 con 0x7f031015df50 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0314cb5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f031006bcd0 0x7f03100caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0314cb5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f031006bcd0 0x7f03100caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51322/0 (socket says 172.21.15.92:51322) 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.666+0000 7f0314cb5700 1 -- 172.21.15.92:0/3492745811 learned_addr learned my addr 172.21.15.92:0/3492745811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:46.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f03154b6700 1 --2- 172.21.15.92:0/3492745811 >> v2:172.21.15.136:3300/0 conn(0x7f031015df50 0x7f031015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f0314cb5700 1 -- 172.21.15.92:0/3492745811 >> v2:172.21.15.136:3300/0 conn(0x7f031015df50 msgr2=0x7f031015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f0314cb5700 1 --2- 172.21.15.92:0/3492745811 >> v2:172.21.15.136:3300/0 conn(0x7f031015df50 0x7f031015e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f0314cb5700 1 -- 172.21.15.92:0/3492745811 >> v1:172.21.15.136:6789/0 conn(0x7f031006c0a0 legacy=0x7f031015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f0314cb5700 1 -- 172.21.15.92:0/3492745811 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031015ad10 con 0x7f031006bcd0 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f03154b6700 1 --2- 172.21.15.92:0/3492745811 >> v2:172.21.15.136:3300/0 conn(0x7f031015df50 0x7f031015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f0314cb5700 1 --2- 172.21.15.92:0/3492745811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006bcd0 0x7f03100caa90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f031015d720 tx=0x7f030c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca8112b6961fdf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f03077fe700 1 -- 172.21.15.92:0/3492745811 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00e070 con 0x7f031006bcd0 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f03077fe700 1 -- 172.21.15.92:0/3492745811 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00ba50 con 0x7f031006bcd0 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.667+0000 7f03077fe700 1 -- 172.21.15.92:0/3492745811 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0092e0 con 0x7f031006bcd0 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 -- 172.21.15.92:0/3492745811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006bcd0 msgr2=0x7f03100caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 --2- 172.21.15.92:0/3492745811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006bcd0 0x7f03100caa90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f031015d720 tx=0x7f030c009840 comp rx=0 tx=0).stop 2023-10-06T03:06:46.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 -- 172.21.15.92:0/3492745811 shutdown_connections 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 --2- 172.21.15.92:0/3492745811 >> v2:172.21.15.136:3300/0 conn(0x7f031015df50 0x7f031015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 --2- 172.21.15.92:0/3492745811 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006bcd0 0x7f03100caa90 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 -- 172.21.15.92:0/3492745811 >> 172.21.15.92:0/3492745811 conn(0x7f03100ce600 msgr2=0x7f03100cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 -- 172.21.15.92:0/3492745811 shutdown_connections 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.668+0000 7f0316f19700 1 -- 172.21.15.92:0/3492745811 wait complete. 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.669+0000 7f0316f19700 1 Processor -- start 2023-10-06T03:06:46.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.669+0000 7f0316f19700 1 -- start start 2023-10-06T03:06:46.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031006bcd0 0x7f03100c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 0x7f031015edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f031015df50 0x7f03100b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100b2150 con 0x7f031006c0a0 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100b1540 con 0x7f031006bcd0 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03100b1820 con 0x7f031015df50 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 0x7f031015edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 0x7f031015edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51326/0 (socket says 172.21.15.92:51326) 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 -- 172.21.15.92:0/238574305 learned_addr learned my addr 172.21.15.92:0/238574305 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0314cb5700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031006bcd0 0x7f03100c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f03154b6700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f031015df50 0x7f03100b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 -- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f031015df50 msgr2=0x7f03100b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f031015df50 0x7f03100b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 -- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031006bcd0 msgr2=0x7f03100c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031006bcd0 0x7f03100c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 -- 172.21.15.92:0/238574305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009ae0 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0307fff700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 0x7f031015edc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f030000c4e0 tx=0x7f030000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300011590 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0300011710 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03100b1b00 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300011ad0 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.670+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03100c8940 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.671+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03100b2300 con 0x7f031006c0a0 2023-10-06T03:06:46.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.672+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f030001b3b0 con 0x7f031006c0a0 2023-10-06T03:06:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.672+0000 7f0304ff9700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02fc067170 0x7f02fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.672+0000 7f0314cb5700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02fc067170 0x7f02fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.672+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f03000839f0 con 0x7f031006c0a0 2023-10-06T03:06:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.673+0000 7f0314cb5700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02fc067170 0x7f02fc069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f030c009a60 tx=0x7f030c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.677+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03100b2300 con 0x7f031006c0a0 2023-10-06T03:06:46.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.858+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f031004c3c0 con 0x7f031006c0a0 2023-10-06T03:06:46.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.859+0000 7f0304ff9700 1 -- 172.21.15.92:0/238574305 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13651 (secure 0 0 0) 0x7f0300017070 con 0x7f031006c0a0 2023-10-06T03:06:46.861 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:46.861 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":74,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:46.131668+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.861+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02fc067170 msgr2=0x7f02fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.861+0000 7f0316f19700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02fc067170 0x7f02fc069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f030c009a60 tx=0x7f030c018000 comp rx=0 tx=0).stop 2023-10-06T03:06:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.861+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 msgr2=0x7f031015edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.861+0000 7f0316f19700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 0x7f031015edc0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f030000c4e0 tx=0x7f030000c7f0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.861+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 shutdown_connections 2023-10-06T03:06:46.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02fc067170 0x7f02fc069620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f031015df50 0x7f03100b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031006bcd0 0x7f03100c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 --2- 172.21.15.92:0/238574305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f031006c0a0 0x7f031015edc0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 >> 172.21.15.92:0/238574305 conn(0x7f03100ce600 msgr2=0x7f0310069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 shutdown_connections 2023-10-06T03:06:46.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.862+0000 7f0316f19700 1 -- 172.21.15.92:0/238574305 wait complete. 2023-10-06T03:06:46.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:06:46.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:06:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.991+0000 7f348dcae700 1 Processor -- start 2023-10-06T03:06:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.991+0000 7f348dcae700 1 -- start start 2023-10-06T03:06:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f348dcae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3488159c60 0x7f348815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f348dcae700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f3488069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f348dcae700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348815a160 con 0x7f3488159890 2023-10-06T03:06:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f348dcae700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348815df50 con 0x7f3488159c60 2023-10-06T03:06:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f348dcae700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880d13a0 con 0x7f348815db10 2023-10-06T03:06:46.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f34877fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3488159890 0x7f348815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38020/0 (socket says 172.21.15.92:38020) 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f34877fe700 1 -- 172.21.15.92:0/3353720798 learned_addr learned my addr 172.21.15.92:0/3353720798 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3353720798 >> v2:172.21.15.92:3300/0 conn(0x7f3488159c60 0x7f348815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f3487fff700 1 --2- 172.21.15.92:0/3353720798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f3488069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f34867fc700 1 -- 172.21.15.92:0/3353720798 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25597707 0 0) 0x7f348815a160 con 0x7f3488159890 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f34867fc700 1 -- 172.21.15.92:0/3353720798 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3470003570 con 0x7f3488159890 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f3486ffd700 1 -- 172.21.15.92:0/3353720798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 msgr2=0x7f3488069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3353720798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f3488069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f3486ffd700 1 -- 172.21.15.92:0/3353720798 >> v1:172.21.15.92:6789/0 conn(0x7f3488159890 legacy=0x7f348815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.992+0000 7f3486ffd700 1 -- 172.21.15.92:0/3353720798 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348815e140 con 0x7f3488159c60 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f3486ffd700 1 --2- 172.21.15.92:0/3353720798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159c60 0x7f348815d5d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f347c0098e0 tx=0x7f347c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda86a498115257a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:46.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f34867fc700 1 -- 172.21.15.92:0/3353720798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00b640 con 0x7f3488159c60 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f34867fc700 1 -- 172.21.15.92:0/3353720798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c00b7a0 con 0x7f3488159c60 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f34867fc700 1 -- 172.21.15.92:0/3353720798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00bb40 con 0x7f3488159c60 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 -- 172.21.15.92:0/3353720798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159c60 msgr2=0x7f348815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 --2- 172.21.15.92:0/3353720798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159c60 0x7f348815d5d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f347c0098e0 tx=0x7f347c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 -- 172.21.15.92:0/3353720798 shutdown_connections 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 --2- 172.21.15.92:0/3353720798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f3488069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 --2- 172.21.15.92:0/3353720798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159c60 0x7f348815d5d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 -- 172.21.15.92:0/3353720798 >> 172.21.15.92:0/3353720798 conn(0x7f3488068e20 msgr2=0x7f34880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 -- 172.21.15.92:0/3353720798 shutdown_connections 2023-10-06T03:06:46.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.993+0000 7f348dcae700 1 -- 172.21.15.92:0/3353720798 wait complete. 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 Processor -- start 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 -- start start 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 0x7f348814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488159c60 0x7f3488151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f348814aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348815ba70 con 0x7f3488159890 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34880cd680 con 0x7f3488159c60 2023-10-06T03:06:47.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f348dcae700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3488152160 con 0x7f348815db10 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f34877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 0x7f348814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f34877fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 0x7f348814c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51330/0 (socket says 172.21.15.92:51330) 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.995+0000 7f34877fe700 1 -- 172.21.15.92:0/1221747954 learned_addr learned my addr 172.21.15.92:0/1221747954 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f34877fe700 1 -- 172.21.15.92:0/1221747954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 msgr2=0x7f348814aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f34877fe700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f348814aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f34877fe700 1 -- 172.21.15.92:0/1221747954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488159c60 msgr2=0x7f3488151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f34877fe700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488159c60 0x7f3488151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f34877fe700 1 -- 172.21.15.92:0/1221747954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009420 con 0x7f3488159890 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f34877fe700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 0x7f348814c660 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f347800c7c0 tx=0x7f3478007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478005b30 con 0x7f3488159890 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478005c90 con 0x7f3488159890 2023-10-06T03:06:47.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3488152440 con 0x7f3488159890 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478007340 con 0x7f3488159890 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.996+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348815b700 con 0x7f3488159890 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.997+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348804c3c0 con 0x7f3488159890 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.997+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f347801b400 con 0x7f3488159890 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.998+0000 7f346f7fe700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3474067170 0x7f3474069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.998+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3478084240 con 0x7f3488159890 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.998+0000 7f3486ffd700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3474067170 0x7f3474069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:46.998+0000 7f3486ffd700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3474067170 0x7f3474069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f34880b6630 tx=0x7f347c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.002+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34780513c0 con 0x7f3488159890 2023-10-06T03:06:47.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.186+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34880b2de0 con 0x7f3488159890 2023-10-06T03:06:47.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.187+0000 7f346f7fe700 1 -- 172.21.15.92:0/1221747954 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34780549e0 con 0x7f3488159890 2023-10-06T03:06:47.189 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3474067170 msgr2=0x7f3474069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3474067170 0x7f3474069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f34880b6630 tx=0x7f347c017000 comp rx=0 tx=0).stop 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 msgr2=0x7f348814c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 0x7f348814c660 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f347800c7c0 tx=0x7f3478007e00 comp rx=0 tx=0).stop 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 shutdown_connections 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3474067170 0x7f3474069620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3488159890 0x7f348814c660 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f348815db10 0x7f348814aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 --2- 172.21.15.92:0/1221747954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3488159c60 0x7f3488151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 >> 172.21.15.92:0/1221747954 conn(0x7f3488068e20 msgr2=0x7f3488069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 shutdown_connections 2023-10-06T03:06:47.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.189+0000 7f348dcae700 1 -- 172.21.15.92:0/1221747954 wait complete. 2023-10-06T03:06:47.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:06:47.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.318+0000 7f1b4efdf700 1 Processor -- start 2023-10-06T03:06:47.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.318+0000 7f1b4efdf700 1 -- start start 2023-10-06T03:06:47.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4efdf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480b4e70 0x7f1b480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4efdf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b480cfb00 con 0x7f1b480b16d0 2023-10-06T03:06:47.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4efdf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4815e060 con 0x7f1b480b1b40 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4efdf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480b5c40 con 0x7f1b480b4e70 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4dfdd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b480b4e70 0x7f1b480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4d7dc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b480b16d0 0x7f1b4815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:38024/0 (socket says 172.21.15.92:38024) 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4d7dc700 1 -- 172.21.15.92:0/1291271272 learned_addr learned my addr 172.21.15.92:0/1291271272 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4dfdd700 1 -- 172.21.15.92:0/1291271272 >> v1:172.21.15.136:6789/0 conn(0x7f1b480b1b40 legacy=0x7f1b4815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4dfdd700 1 -- 172.21.15.92:0/1291271272 >> v1:172.21.15.92:6789/0 conn(0x7f1b480b16d0 legacy=0x7f1b4815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4cfdb700 1 -- 172.21.15.92:0/1291271272 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458404971 0 0) 0x7f1b480cfb00 con 0x7f1b480b16d0 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.319+0000 7f1b4dfdd700 1 -- 172.21.15.92:0/1291271272 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480b5f20 con 0x7f1b480b4e70 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4dfdd700 1 --2- 172.21.15.92:0/1291271272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b4e70 0x7f1b480ccff0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1b440098e0 tx=0x7f1b440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4612ded1e9f124 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4cfdb700 1 -- 172.21.15.92:0/1291271272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44009e80 con 0x7f1b480b4e70 2023-10-06T03:06:47.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4cfdb700 1 -- 172.21.15.92:0/1291271272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4400b820 con 0x7f1b480b4e70 2023-10-06T03:06:47.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4cfdb700 1 -- 172.21.15.92:0/1291271272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400bba0 con 0x7f1b480b4e70 2023-10-06T03:06:47.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 -- 172.21.15.92:0/1291271272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b4e70 msgr2=0x7f1b480ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/1291271272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b4e70 0x7f1b480ccff0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1b440098e0 tx=0x7f1b440091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 -- 172.21.15.92:0/1291271272 shutdown_connections 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/1291271272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b4e70 0x7f1b480ccff0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 -- 172.21.15.92:0/1291271272 >> 172.21.15.92:0/1291271272 conn(0x7f1b480d10c0 msgr2=0x7f1b480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 -- 172.21.15.92:0/1291271272 shutdown_connections 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.320+0000 7f1b4efdf700 1 -- 172.21.15.92:0/1291271272 wait complete. 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.321+0000 7f1b4efdf700 1 Processor -- start 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.321+0000 7f1b4efdf700 1 -- start start 2023-10-06T03:06:47.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4efdf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480b16d0 0x7f1b48148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4efdf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 0x7f1b48148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4efdf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b480b4e70 0x7f1b48148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4efdf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48145eb0 con 0x7f1b480b1b40 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4efdf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480c94e0 con 0x7f1b480b16d0 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4efdf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480c88d0 con 0x7f1b480b4e70 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4d7dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 0x7f1b48148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4d7dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 0x7f1b48148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51346/0 (socket says 172.21.15.92:51346) 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4d7dc700 1 -- 172.21.15.92:0/508323067 learned_addr learned my addr 172.21.15.92:0/508323067 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4dfdd700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480b16d0 0x7f1b48148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.322+0000 7f1b4e7de700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b480b4e70 0x7f1b48148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4d7dc700 1 -- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b480b4e70 msgr2=0x7f1b48148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4d7dc700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b480b4e70 0x7f1b48148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4d7dc700 1 -- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480b16d0 msgr2=0x7f1b48148020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4d7dc700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480b16d0 0x7f1b48148020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4d7dc700 1 -- 172.21.15.92:0/508323067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009420 con 0x7f1b480b1b40 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4e7de700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b480b4e70 0x7f1b48148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4dfdd700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480b16d0 0x7f1b48148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4d7dc700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 0x7f1b48148560 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c7e0 tx=0x7f1b38007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38005b30 con 0x7f1b480b1b40 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b38005c90 con 0x7f1b480b1b40 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007300 con 0x7f1b480b1b40 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480c8bb0 con 0x7f1b480b1b40 2023-10-06T03:06:47.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.323+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480c9170 con 0x7f1b480b1b40 2023-10-06T03:06:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.324+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b480c9670 con 0x7f1b480b1b40 2023-10-06T03:06:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.324+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f1b3801b400 con 0x7f1b480b1b40 2023-10-06T03:06:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.325+0000 7f1b3dffb700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b2c067170 0x7f1b2c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.325+0000 7f1b4dfdd700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b2c067170 0x7f1b2c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.325+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1b38084260 con 0x7f1b480b1b40 2023-10-06T03:06:47.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.325+0000 7f1b4dfdd700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b2c067170 0x7f1b2c069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b440030c0 tx=0x7f1b44016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.329+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b380513e0 con 0x7f1b480b1b40 2023-10-06T03:06:47.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.510+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1b480af990 con 0x7f1b480b1b40 2023-10-06T03:06:47.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.510+0000 7f1b3dffb700 1 -- 172.21.15.92:0/508323067 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1b38054a00 con 0x7f1b480b1b40 2023-10-06T03:06:47.512 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:06:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b2c067170 msgr2=0x7f1b2c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b2c067170 0x7f1b2c069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1b440030c0 tx=0x7f1b44016000 comp rx=0 tx=0).stop 2023-10-06T03:06:47.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 msgr2=0x7f1b48148560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 0x7f1b48148560 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c7e0 tx=0x7f1b38007df0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 shutdown_connections 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b2c067170 0x7f1b2c069620 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b480b4e70 0x7f1b48148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b480b16d0 0x7f1b48148020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 --2- 172.21.15.92:0/508323067 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b480b1b40 0x7f1b48148560 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.512+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 >> 172.21.15.92:0/508323067 conn(0x7f1b480d10c0 msgr2=0x7f1b480ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.513+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 shutdown_connections 2023-10-06T03:06:47.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.513+0000 7f1b4efdf700 1 -- 172.21.15.92:0/508323067 wait complete. 2023-10-06T03:06:47.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:06:47.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.640+0000 7f707ffff700 1 Processor -- start 2023-10-06T03:06:47.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.640+0000 7f707ffff700 1 -- start start 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f707ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 0x7f708006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f707ffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 0x7f708006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f707ffff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70800b00f0 con 0x7f7080157680 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f707ffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708006cae0 con 0x7f70800af5c0 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f707ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708006ebe0 con 0x7f7080157a50 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f707f7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 0x7f708006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.641+0000 7f70767fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 0x7f708006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 0x7f708006e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:54792/0 (socket says 172.21.15.92:54792) 2023-10-06T03:06:47.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f707f7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 0x7f708006c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:49630/0 (socket says 172.21.15.92:49630) 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f707effd700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f7080157680 0x7f70800aeeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:37770/0 (socket says 172.21.15.92:37770) 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 -- 172.21.15.92:0/924170162 learned_addr learned my addr 172.21.15.92:0/924170162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 -- 172.21.15.92:0/924170162 >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 msgr2=0x7f708006c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 --2- 172.21.15.92:0/924170162 >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 0x7f708006c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 -- 172.21.15.92:0/924170162 >> v1:172.21.15.136:6789/0 conn(0x7f7080157680 legacy=0x7f70800aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 -- 172.21.15.92:0/924170162 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708006eec0 con 0x7f7080157a50 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f707f7fe700 1 --2- 172.21.15.92:0/924170162 >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 0x7f708006c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f70767fc700 1 --2- 172.21.15.92:0/924170162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 0x7f708006e570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7068009990 tx=0x7f7068009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe59dee303434e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f707e7fc700 1 -- 172.21.15.92:0/924170162 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068009e80 con 0x7f7080157a50 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f707e7fc700 1 -- 172.21.15.92:0/924170162 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7068005c10 con 0x7f7080157a50 2023-10-06T03:06:47.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.642+0000 7f707e7fc700 1 -- 172.21.15.92:0/924170162 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706800b650 con 0x7f7080157a50 2023-10-06T03:06:47.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 -- 172.21.15.92:0/924170162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 msgr2=0x7f708006e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 --2- 172.21.15.92:0/924170162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 0x7f708006e570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7068009990 tx=0x7f7068009220 comp rx=0 tx=0).stop 2023-10-06T03:06:47.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 -- 172.21.15.92:0/924170162 shutdown_connections 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 --2- 172.21.15.92:0/924170162 >> v2:172.21.15.136:3300/0 conn(0x7f70800af5c0 0x7f708006c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 --2- 172.21.15.92:0/924170162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157a50 0x7f708006e570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 -- 172.21.15.92:0/924170162 >> 172.21.15.92:0/924170162 conn(0x7f708006b450 msgr2=0x7f70800cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 -- 172.21.15.92:0/924170162 shutdown_connections 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.643+0000 7f707ffff700 1 -- 172.21.15.92:0/924170162 wait complete. 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.644+0000 7f707ffff700 1 Processor -- start 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.644+0000 7f707ffff700 1 -- start start 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70800af5c0 0x7f70801688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157680 0x7f70800c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 0x7f70800c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c6430 con 0x7f7080157a50 2023-10-06T03:06:47.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c4d70 con 0x7f70800af5c0 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70800c61b0 con 0x7f7080157680 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 0x7f70800c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 0x7f70800c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51356/0 (socket says 172.21.15.92:51356) 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 -- 172.21.15.92:0/3767180440 learned_addr learned my addr 172.21.15.92:0/3767180440 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707effd700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70800af5c0 0x7f70801688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f70767fc700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157680 0x7f70800c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 -- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157680 msgr2=0x7f70800c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157680 0x7f70800c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 -- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70800af5c0 msgr2=0x7f70801688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70800af5c0 0x7f70801688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 -- 172.21.15.92:0/3767180440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009a20 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707effd700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70800af5c0 0x7f70801688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707f7fe700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 0x7f70800c5200 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f707800c0f0 tx=0x7f707800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800cd90 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800f040 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7068009480 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.645+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70780146d0 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.646+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70800c4720 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.646+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708004c3c0 con 0x7f7080157a50 2023-10-06T03:06:47.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.647+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f7078003020 con 0x7f7080157a50 2023-10-06T03:06:47.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.647+0000 7f70777fe700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7060067170 0x7f7060069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.647+0000 7f707effd700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7060067170 0x7f7060069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.647+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7078085380 con 0x7f7080157a50 2023-10-06T03:06:47.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.648+0000 7f707effd700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7060067170 0x7f7060069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7070007fd0 tx=0x7f7070007e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.652+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7078052370 con 0x7f7080157a50 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.836+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f70800cc620 con 0x7f7080157a50 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.836+0000 7f70777fe700 1 -- 172.21.15.92:0/3767180440 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7078055990 con 0x7f7080157a50 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:06:47.838 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:06:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:06:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:06:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:06:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:06:47.839 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:06:47.839 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:06:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7060067170 msgr2=0x7f7060069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7060067170 0x7f7060069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7070007fd0 tx=0x7f7070007e10 comp rx=0 tx=0).stop 2023-10-06T03:06:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 msgr2=0x7f70800c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 0x7f70800c5200 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f707800c0f0 tx=0x7f707800b040 comp rx=0 tx=0).stop 2023-10-06T03:06:47.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 shutdown_connections 2023-10-06T03:06:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7060067170 0x7f7060069620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7080157a50 0x7f70800c5200 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70800af5c0 0x7f70801688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 --2- 172.21.15.92:0/3767180440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7080157680 0x7f70800c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 >> 172.21.15.92:0/3767180440 conn(0x7f708006b450 msgr2=0x7f70801567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 shutdown_connections 2023-10-06T03:06:47.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.838+0000 7f707ffff700 1 -- 172.21.15.92:0/3767180440 wait complete. 2023-10-06T03:06:47.852 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:06:47.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:06:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.968+0000 7feb8d18f700 1 Processor -- start 2023-10-06T03:06:47.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.968+0000 7feb8d18f700 1 -- start start 2023-10-06T03:06:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8d18f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8d18f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feb880b0790 0x7feb880b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8d18f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb880cfa00 con 0x7feb881591e0 2023-10-06T03:06:47.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8d18f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb880cb580 con 0x7feb880b0790 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8d18f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb880b2de0 con 0x7feb880b03c0 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb877fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feb880b0790 0x7feb880b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb87fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8c98e700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7feb881591e0 0x7feb88159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:37778/0 (socket says 172.21.15.92:37778) 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.969+0000 7feb8c98e700 1 -- 172.21.15.92:0/2380276410 learned_addr learned my addr 172.21.15.92:0/2380276410 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb877fe700 1 -- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 msgr2=0x7feb880cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb877fe700 1 --2- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb880cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb877fe700 1 -- 172.21.15.92:0/2380276410 >> v1:172.21.15.136:6789/0 conn(0x7feb881591e0 legacy=0x7feb88159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb86ffd700 1 -- 172.21.15.92:0/2380276410 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092421086 0 0) 0x7feb880cfa00 con 0x7feb881591e0 2023-10-06T03:06:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb877fe700 1 -- 172.21.15.92:0/2380276410 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880cf160 con 0x7feb880b0790 2023-10-06T03:06:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb877fe700 1 --2- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb880b28a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0098e0 tx=0x7feb7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3341b0256e4892c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb86ffd700 1 -- 172.21.15.92:0/2380276410 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c00b640 con 0x7feb880b0790 2023-10-06T03:06:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb86ffd700 1 -- 172.21.15.92:0/2380276410 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c00b7a0 con 0x7feb880b0790 2023-10-06T03:06:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb86ffd700 1 -- 172.21.15.92:0/2380276410 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c00bb40 con 0x7feb880b0790 2023-10-06T03:06:47.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb8d18f700 1 -- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 msgr2=0x7feb880b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.970+0000 7feb8d18f700 1 --2- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb880b28a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0098e0 tx=0x7feb7c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.971+0000 7feb8d18f700 1 -- 172.21.15.92:0/2380276410 shutdown_connections 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.971+0000 7feb8d18f700 1 --2- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb880cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.971+0000 7feb8d18f700 1 --2- 172.21.15.92:0/2380276410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb880b28a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.971+0000 7feb8d18f700 1 -- 172.21.15.92:0/2380276410 >> 172.21.15.92:0/2380276410 conn(0x7feb88063230 msgr2=0x7feb88063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.971+0000 7feb8d18f700 1 -- 172.21.15.92:0/2380276410 shutdown_connections 2023-10-06T03:06:47.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.971+0000 7feb8d18f700 1 -- 172.21.15.92:0/2380276410 wait complete. 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.972+0000 7feb8d18f700 1 Processor -- start 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.972+0000 7feb8d18f700 1 -- start start 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.972+0000 7feb8d18f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb881697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.972+0000 7feb8d18f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb8814c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.972+0000 7feb8d18f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 0x7feb8815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.972+0000 7feb8d18f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8815b0d0 con 0x7feb881591e0 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8d18f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb880c76d0 con 0x7feb880b0790 2023-10-06T03:06:47.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8d18f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb880c79b0 con 0x7feb880b03c0 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 0x7feb8815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb877fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb8814c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb87fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb881697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 0x7feb8815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51366/0 (socket says 172.21.15.92:51366) 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 -- 172.21.15.92:0/1436378201 learned_addr learned my addr 172.21.15.92:0/1436378201 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 -- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 msgr2=0x7feb881697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb881697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 -- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 msgr2=0x7feb8814c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb8814c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 -- 172.21.15.92:0/1436378201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009420 con 0x7feb881591e0 2023-10-06T03:06:47.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb877fe700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb8814c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8c98e700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 0x7feb8815a7a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feb8000cc90 tx=0x7feb80001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8000a870 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb880c7c90 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8000a9d0 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8000ad90 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.973+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8814cea0 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.974+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb880c81b0 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.975+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7feb80007300 con 0x7feb881591e0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.975+0000 7feb6ffff700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feb74067170 0x7feb74069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.975+0000 7feb87fff700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feb74067170 0x7feb74069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:47.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.975+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5206+0+0 (secure 0 0 0) 0x7feb800849c0 con 0x7feb881591e0 2023-10-06T03:06:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.976+0000 7feb87fff700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feb74067170 0x7feb74069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb78009d10 tx=0x7feb780091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:47.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:47.979+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb880c81b0 con 0x7feb881591e0 2023-10-06T03:06:48.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:48.166+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feb880c81b0 con 0x7feb881591e0 2023-10-06T03:06:49.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.142+0000 7feb6ffff700 1 -- 172.21.15.92:0/1436378201 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7feb880c81b0 con 0x7feb881591e0 2023-10-06T03:06:49.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feb74067170 msgr2=0x7feb74069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feb74067170 0x7feb74069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb78009d10 tx=0x7feb780091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 msgr2=0x7feb8815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 0x7feb8815a7a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feb8000cc90 tx=0x7feb80001a20 comp rx=0 tx=0).stop 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 shutdown_connections 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feb74067170 0x7feb74069620 secure :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feb78009d10 tx=0x7feb780091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feb881591e0 0x7feb8815a7a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb880b03c0 0x7feb881697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 --2- 172.21.15.92:0/1436378201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb880b0790 0x7feb8814c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.146+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 >> 172.21.15.92:0/1436378201 conn(0x7feb88063230 msgr2=0x7feb88158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:49.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.147+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 shutdown_connections 2023-10-06T03:06:49.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.147+0000 7feb8d18f700 1 -- 172.21.15.92:0/1436378201 wait complete. 2023-10-06T03:06:49.150 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:06:49.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:06:49.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.288+0000 7f98afe63700 1 Processor -- start 2023-10-06T03:06:49.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.289+0000 7f98afe63700 1 -- start start 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98afe63700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98afe63700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98a8069c70 0x7f98a80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98afe63700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a80ce380 con 0x7f98a80af320 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98afe63700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a80cec20 con 0x7f98a8069c70 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98afe63700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80afa90 con 0x7f98a8159950 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98a8069c70 0x7f98a80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f98a8069c70 0x7f98a80cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51370/0 (socket says 172.21.15.92:51370) 2023-10-06T03:06:49.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 -- 172.21.15.92:0/2209782407 learned_addr learned my addr 172.21.15.92:0/2209782407 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 -- 172.21.15.92:0/2209782407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 msgr2=0x7f98a80cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 --2- 172.21.15.92:0/2209782407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 -- 172.21.15.92:0/2209782407 >> v1:172.21.15.136:6789/0 conn(0x7f98a80af320 legacy=0x7f98a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 -- 172.21.15.92:0/2209782407 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a80ce5d0 con 0x7f98a8069c70 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98ae400700 1 --2- 172.21.15.92:0/2209782407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a8069c70 0x7f98a80cf0b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009f20 tx=0x7f98a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16bbe24cb755d69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98acbfd700 1 -- 172.21.15.92:0/2209782407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000e070 con 0x7f98a8069c70 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98acbfd700 1 -- 172.21.15.92:0/2209782407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a000ba40 con 0x7f98a8069c70 2023-10-06T03:06:49.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.290+0000 7f98acbfd700 1 -- 172.21.15.92:0/2209782407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00092e0 con 0x7f98a8069c70 2023-10-06T03:06:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 -- 172.21.15.92:0/2209782407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a8069c70 msgr2=0x7f98a80cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 --2- 172.21.15.92:0/2209782407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a8069c70 0x7f98a80cf0b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009f20 tx=0x7f98a0009840 comp rx=0 tx=0).stop 2023-10-06T03:06:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 -- 172.21.15.92:0/2209782407 shutdown_connections 2023-10-06T03:06:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 --2- 172.21.15.92:0/2209782407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a8069c70 0x7f98a80cf0b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 --2- 172.21.15.92:0/2209782407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 -- 172.21.15.92:0/2209782407 >> 172.21.15.92:0/2209782407 conn(0x7f98a8068e00 msgr2=0x7f98a80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 -- 172.21.15.92:0/2209782407 shutdown_connections 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.291+0000 7f98afe63700 1 -- 172.21.15.92:0/2209782407 wait complete. 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.292+0000 7f98afe63700 1 Processor -- start 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.292+0000 7f98afe63700 1 -- start start 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98afe63700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a8069c70 0x7f98a8169e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98afe63700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 0x7f98a8152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98afe63700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98afe63700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80c82b0 con 0x7f98a80af320 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98afe63700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80c8590 con 0x7f98a8069c70 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98afe63700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8159d20 con 0x7f98a8159950 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 0x7f98a8152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:49.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 0x7f98a8152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:51380/0 (socket says 172.21.15.92:51380) 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 -- 172.21.15.92:0/2910559927 learned_addr learned my addr 172.21.15.92:0/2910559927 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ae400700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98adbff700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a8069c70 0x7f98a8169e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 -- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 msgr2=0x7f98a80c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 -- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a8069c70 msgr2=0x7f98a8169e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a8069c70 0x7f98a8169e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:49.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 -- 172.21.15.92:0/2910559927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a0009ae0 con 0x7f98a80af320 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98adbff700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a8069c70 0x7f98a8169e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.293+0000 7f98ad3fe700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 0x7f98a8152240 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f989800c5d0 tx=0x7f9898007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.294+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898011590 con 0x7f98a80af320 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.294+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898011710 con 0x7f98a80af320 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.294+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a815a000 con 0x7f98a80af320 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.294+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898011ad0 con 0x7f98a80af320 2023-10-06T03:06:49.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.294+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a80c98b0 con 0x7f98a80af320 2023-10-06T03:06:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.295+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a815a520 con 0x7f98a80af320 2023-10-06T03:06:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.295+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f989801c460 con 0x7f98a80af320 2023-10-06T03:06:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.295+0000 7f989dffb700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f988c067170 0x7f988c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.296+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f9898084510 con 0x7f98a80af320 2023-10-06T03:06:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.296+0000 7f98adbff700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f988c067170 0x7f988c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:49.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.296+0000 7f98adbff700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f988c067170 0x7f988c069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98a4006fd0 tx=0x7f98a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:49.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.300+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98a815a520 con 0x7f98a80af320 2023-10-06T03:06:49.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:49.483+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f98a815a520 con 0x7f98a80af320 2023-10-06T03:06:50.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.150+0000 7f989dffb700 1 -- 172.21.15.92:0/2910559927 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f98a815a520 con 0x7f98a80af320 2023-10-06T03:06:50.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.159+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f988c067170 msgr2=0x7f988c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:50.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.159+0000 7f98afe63700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f988c067170 0x7f988c069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98a4006fd0 tx=0x7f98a4008040 comp rx=0 tx=0).stop 2023-10-06T03:06:50.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.159+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 msgr2=0x7f98a8152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:50.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.159+0000 7f98afe63700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 0x7f98a8152240 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f989800c5d0 tx=0x7f9898007f60 comp rx=0 tx=0).stop 2023-10-06T03:06:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 shutdown_connections 2023-10-06T03:06:50.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f988c067170 0x7f988c069620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98a8159950 0x7f98a80c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a8069c70 0x7f98a8169e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 --2- 172.21.15.92:0/2910559927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98a80af320 0x7f98a8152240 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 >> 172.21.15.92:0/2910559927 conn(0x7f98a8068e00 msgr2=0x7f98a80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 shutdown_connections 2023-10-06T03:06:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.161+0000 7f98afe63700 1 -- 172.21.15.92:0/2910559927 wait complete. 2023-10-06T03:06:50.168 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:06:50.178 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:06:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.295+0000 7efeded8c700 1 Processor -- start 2023-10-06T03:06:50.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.295+0000 7efeded8c700 1 -- start start 2023-10-06T03:06:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 0x7efed80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:50.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efed80b1b40 0x7efed815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed80cfb00 con 0x7efed80b16d0 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed80b5c40 con 0x7efed80b1b40 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80b5f20 con 0x7efed80b4e70 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 0x7efed80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efede58b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efed80b1b40 0x7efed815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 0x7efed80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:55530/0 (socket says 172.21.15.92:55530) 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 -- 172.21.15.92:0/3070609856 learned_addr learned my addr 172.21.15.92:0/3070609856 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:50.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efedd589700 1 --1- 172.21.15.92:0/3070609856 >> v1:172.21.15.136:6789/0 conn(0x7efed80b16d0 0x7efed815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:53894/0 (socket says 172.21.15.92:53894) 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 -- 172.21.15.92:0/3070609856 >> v2:172.21.15.136:3300/0 conn(0x7efed80b1b40 msgr2=0x7efed815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 --2- 172.21.15.92:0/3070609856 >> v2:172.21.15.136:3300/0 conn(0x7efed80b1b40 0x7efed815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 -- 172.21.15.92:0/3070609856 >> v1:172.21.15.136:6789/0 conn(0x7efed80b16d0 legacy=0x7efed815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 -- 172.21.15.92:0/3070609856 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80b6200 con 0x7efed80b4e70 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeddd8a700 1 --2- 172.21.15.92:0/3070609856 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 0x7efed80ccff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efed0009f20 tx=0x7efed0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80dc3d6e731b5b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efedcd88700 1 -- 172.21.15.92:0/3070609856 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000e070 con 0x7efed80b4e70 2023-10-06T03:06:50.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efedcd88700 1 -- 172.21.15.92:0/3070609856 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000b9d0 con 0x7efed80b4e70 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efedcd88700 1 -- 172.21.15.92:0/3070609856 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed00092e0 con 0x7efed80b4e70 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 -- 172.21.15.92:0/3070609856 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 msgr2=0x7efed80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 --2- 172.21.15.92:0/3070609856 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 0x7efed80ccff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efed0009f20 tx=0x7efed0009840 comp rx=0 tx=0).stop 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 -- 172.21.15.92:0/3070609856 shutdown_connections 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 --2- 172.21.15.92:0/3070609856 >> v2:172.21.15.136:3300/0 conn(0x7efed80b1b40 0x7efed815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 --2- 172.21.15.92:0/3070609856 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b4e70 0x7efed80ccff0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.296+0000 7efeded8c700 1 -- 172.21.15.92:0/3070609856 >> 172.21.15.92:0/3070609856 conn(0x7efed80d10c0 msgr2=0x7efed80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.297+0000 7efeded8c700 1 -- 172.21.15.92:0/3070609856 shutdown_connections 2023-10-06T03:06:50.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.297+0000 7efeded8c700 1 -- 172.21.15.92:0/3070609856 wait complete. 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 Processor -- start 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 -- start start 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 0x7efed8147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b1b40 0x7efed816af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed80b4e70 0x7efed816c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed816b900 con 0x7efed80b16d0 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed816bbe0 con 0x7efed80b4e70 2023-10-06T03:06:50.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeded8c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed816bec0 con 0x7efed80b1b40 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeddd8a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 0x7efed8147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeddd8a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 0x7efed8147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42946/0 (socket says 172.21.15.92:42946) 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efeddd8a700 1 -- 172.21.15.92:0/2767864463 learned_addr learned my addr 172.21.15.92:0/2767864463 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.298+0000 7efedd589700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b1b40 0x7efed816af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efede58b700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed80b4e70 0x7efed816c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeddd8a700 1 -- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b1b40 msgr2=0x7efed816af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeddd8a700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b1b40 0x7efed816af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeddd8a700 1 -- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed80b4e70 msgr2=0x7efed816c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeddd8a700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed80b4e70 0x7efed816c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:50.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeddd8a700 1 -- 172.21.15.92:0/2767864463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0009ae0 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeddd8a700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 0x7efed8147550 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efed000b610 tx=0x7efed0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0008a20 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed0008b80 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0026460 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed816c1a0 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.299+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed816b4d0 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.300+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed8088990 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.301+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7efed0003040 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.301+0000 7efecdffb700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efebc067170 0x7efebc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.301+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5958+0+0 (secure 0 0 0) 0x7efed0086650 con 0x7efed80b16d0 2023-10-06T03:06:50.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.301+0000 7efedd589700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efebc067170 0x7efebc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.301+0000 7efedd589700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efebc067170 0x7efebc069620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efed8068fd0 tx=0x7efec8007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:50.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.306+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efed0053420 con 0x7efed80b16d0 2023-10-06T03:06:50.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:50.529+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efed80cf4b0 con 0x7efed80b16d0 2023-10-06T03:06:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.154+0000 7efecdffb700 1 -- 172.21.15.92:0/2767864463 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7efed0056a40 con 0x7efed80b16d0 2023-10-06T03:06:51.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.160+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efebc067170 msgr2=0x7efebc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.160+0000 7efeded8c700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efebc067170 0x7efebc069620 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efed8068fd0 tx=0x7efec8007eb0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.160+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 msgr2=0x7efed8147550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.160+0000 7efeded8c700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 0x7efed8147550 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efed000b610 tx=0x7efed0018000 comp rx=0 tx=0).stop 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.162+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 shutdown_connections 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.162+0000 7efeded8c700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efebc067170 0x7efebc069620 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.162+0000 7efeded8c700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efed80b16d0 0x7efed8147550 secure :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efed000b610 tx=0x7efed0018000 comp rx=0 tx=0).stop 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.162+0000 7efeded8c700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efed80b4e70 0x7efed816c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.162+0000 7efeded8c700 1 --2- 172.21.15.92:0/2767864463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efed80b1b40 0x7efed816af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.162+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 >> 172.21.15.92:0/2767864463 conn(0x7efed80d10c0 msgr2=0x7efed8069520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:51.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.164+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 shutdown_connections 2023-10-06T03:06:51.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.165+0000 7efeded8c700 1 -- 172.21.15.92:0/2767864463 wait complete. 2023-10-06T03:06:51.173 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T03:06:51.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:51.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.302+0000 7f1987fff700 1 Processor -- start 2023-10-06T03:06:51.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.302+0000 7f1987fff700 1 -- start start 2023-10-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1987fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f19880b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1987fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19880b2920 0x7f198815a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1987fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f198815a5d0 0x7f19880ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1987fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880ce260 con 0x7f198815a5d0 2023-10-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1987fff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880b5b40 con 0x7f19880b2920 2023-10-06T03:06:51.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1987fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19880b5e20 con 0x7f1988159cc0 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f198815a5d0 0x7f19880ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f198815a5d0 0x7f19880ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42954/0 (socket says 172.21.15.92:42954) 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 -- 172.21.15.92:0/3193996476 learned_addr learned my addr 172.21.15.92:0/3193996476 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1986ffd700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f19880b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f197e7fc700 1 --2- 172.21.15.92:0/3193996476 >> v2:172.21.15.136:3300/0 conn(0x7f19880b2920 0x7f198815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 -- 172.21.15.92:0/3193996476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 msgr2=0x7f19880b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f19880b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 -- 172.21.15.92:0/3193996476 >> v2:172.21.15.136:3300/0 conn(0x7f19880b2920 msgr2=0x7f198815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 --2- 172.21.15.92:0/3193996476 >> v2:172.21.15.136:3300/0 conn(0x7f19880b2920 0x7f198815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f19877fe700 1 -- 172.21.15.92:0/3193996476 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880b6100 con 0x7f198815a5d0 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f197e7fc700 1 --2- 172.21.15.92:0/3193996476 >> v2:172.21.15.136:3300/0 conn(0x7f19880b2920 0x7f198815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:51.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.303+0000 7f1986ffd700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f19880b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f19877fe700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198815a5d0 0x7f19880ce6b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f19880679b0 tx=0x7f1980009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ea47ab8c6636ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f19867fc700 1 -- 172.21.15.92:0/3193996476 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000e070 con 0x7f198815a5d0 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f19867fc700 1 -- 172.21.15.92:0/3193996476 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198000ba50 con 0x7f198815a5d0 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f19867fc700 1 -- 172.21.15.92:0/3193996476 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800092e0 con 0x7f198815a5d0 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 -- 172.21.15.92:0/3193996476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198815a5d0 msgr2=0x7f19880ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198815a5d0 0x7f19880ce6b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f19880679b0 tx=0x7f1980009840 comp rx=0 tx=0).stop 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 -- 172.21.15.92:0/3193996476 shutdown_connections 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f19880b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 --2- 172.21.15.92:0/3193996476 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f198815a5d0 0x7f19880ce6b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 --2- 172.21.15.92:0/3193996476 >> v2:172.21.15.136:3300/0 conn(0x7f19880b2920 0x7f198815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 -- 172.21.15.92:0/3193996476 >> 172.21.15.92:0/3193996476 conn(0x7f19880d0560 msgr2=0x7f19880d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:51.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 -- 172.21.15.92:0/3193996476 shutdown_connections 2023-10-06T03:06:51.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.304+0000 7f1987fff700 1 -- 172.21.15.92:0/3193996476 wait complete. 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.305+0000 7f1987fff700 1 Processor -- start 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 -- start start 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 0x7f1988156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f198815fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f198815a5d0 0x7f1988151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881568b0 con 0x7f19880b2920 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988155ca0 con 0x7f198815a5d0 2023-10-06T03:06:51.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1987fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1988155f80 con 0x7f1988159cc0 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1986ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 0x7f1988156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1986ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 0x7f1988156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42960/0 (socket says 172.21.15.92:42960) 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.306+0000 7f1986ffd700 1 -- 172.21.15.92:0/1775045710 learned_addr learned my addr 172.21.15.92:0/1775045710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1986ffd700 1 -- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 msgr2=0x7f198815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1986ffd700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f198815fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1986ffd700 1 -- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f198815a5d0 msgr2=0x7f1988151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f19877fe700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f198815a5d0 0x7f1988151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1986ffd700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f198815a5d0 0x7f1988151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1986ffd700 1 -- 172.21.15.92:0/1775045710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009ae0 con 0x7f19880b2920 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1986ffd700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 0x7f1988156d40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f197800d810 tx=0x7f1978008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978005a60 con 0x7f19880b2920 2023-10-06T03:06:51.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1978005bc0 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1988156260 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780125f0 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.307+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1988160040 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.308+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1988156b90 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.308+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f19780129e0 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.309+0000 7f197f7fe700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1968067170 0x7f1968069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.309+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1978084650 con 0x7f19880b2920 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.309+0000 7f197e7fc700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1968067170 0x7f1968069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.309+0000 7f197e7fc700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1968067170 0x7f1968069620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f1970008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.313+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f197804d5a0 con 0x7f19880b2920 2023-10-06T03:06:51.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.498+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f198804c3c0 con 0x7f19880b2920 2023-10-06T03:06:51.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.499+0000 7f197f7fe700 1 -- 172.21.15.92:0/1775045710 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16882 (secure 0 0 0) 0x7f1978054f60 con 0x7f19880b2920 2023-10-06T03:06:51.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:51.500 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:50.531635+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:06:48.185180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:06:49.502363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1968067170 msgr2=0x7f1968069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1968067170 0x7f1968069620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1970006fd0 tx=0x7f1970008040 comp rx=0 tx=0).stop 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 msgr2=0x7f1988156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 0x7f1988156d40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f197800d810 tx=0x7f1978008040 comp rx=0 tx=0).stop 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 shutdown_connections 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1968067170 0x7f1968069620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19880b2920 0x7f1988156d40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f198815a5d0 0x7f1988151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 --2- 172.21.15.92:0/1775045710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1988159cc0 0x7f198815fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 >> 172.21.15.92:0/1775045710 conn(0x7f19880d0560 msgr2=0x7f19880ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:51.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 shutdown_connections 2023-10-06T03:06:51.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.501+0000 7f1987fff700 1 -- 172.21.15.92:0/1775045710 wait complete. 2023-10-06T03:06:51.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.627+0000 7f283ee9b700 1 Processor -- start 2023-10-06T03:06:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.627+0000 7f283ee9b700 1 -- start start 2023-10-06T03:06:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283ee9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2838159890 0x7f283815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283ee9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2838159c60 0x7f283815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283ee9b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283815a160 con 0x7f283815db10 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283ee9b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283815df50 con 0x7f2838159890 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283ee9b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28380d13a0 con 0x7f2838159c60 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283cc37700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2838159890 0x7f283815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283cc37700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2838159890 0x7f283815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42970/0 (socket says 172.21.15.92:42970) 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.628+0000 7f283cc37700 1 -- 172.21.15.92:0/3857408504 learned_addr learned my addr 172.21.15.92:0/3857408504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283cc37700 1 -- 172.21.15.92:0/3857408504 >> v2:172.21.15.136:3300/0 conn(0x7f2838159c60 msgr2=0x7f283815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283cc37700 1 --2- 172.21.15.92:0/3857408504 >> v2:172.21.15.136:3300/0 conn(0x7f2838159c60 0x7f283815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283cc37700 1 -- 172.21.15.92:0/3857408504 >> v1:172.21.15.136:6789/0 conn(0x7f283815db10 legacy=0x7f2838069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283cc37700 1 -- 172.21.15.92:0/3857408504 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283815e140 con 0x7f2838159890 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283cc37700 1 --2- 172.21.15.92:0/3857408504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f283815abd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f28380b67f0 tx=0x7f2834009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac64ea5c791bd243 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f282f7fe700 1 -- 172.21.15.92:0/3857408504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283400e070 con 0x7f2838159890 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f282f7fe700 1 -- 172.21.15.92:0/3857408504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283400ba10 con 0x7f2838159890 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f282f7fe700 1 -- 172.21.15.92:0/3857408504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340092e0 con 0x7f2838159890 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283ee9b700 1 -- 172.21.15.92:0/3857408504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 msgr2=0x7f283815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.629+0000 7f283ee9b700 1 --2- 172.21.15.92:0/3857408504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f283815abd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f28380b67f0 tx=0x7f2834009840 comp rx=0 tx=0).stop 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.630+0000 7f283ee9b700 1 -- 172.21.15.92:0/3857408504 shutdown_connections 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.630+0000 7f283ee9b700 1 --2- 172.21.15.92:0/3857408504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f283815abd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.630+0000 7f283ee9b700 1 --2- 172.21.15.92:0/3857408504 >> v2:172.21.15.136:3300/0 conn(0x7f2838159c60 0x7f283815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.630+0000 7f283ee9b700 1 -- 172.21.15.92:0/3857408504 >> 172.21.15.92:0/3857408504 conn(0x7f2838068e20 msgr2=0x7f28380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.630+0000 7f283ee9b700 1 -- 172.21.15.92:0/3857408504 shutdown_connections 2023-10-06T03:06:51.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.630+0000 7f283ee9b700 1 -- 172.21.15.92:0/3857408504 wait complete. 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.631+0000 7f283ee9b700 1 Processor -- start 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.631+0000 7f283ee9b700 1 -- start start 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f28380cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2838159c60 0x7f28381440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f283815db10 0x7f28380c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380cf920 con 0x7f2838159890 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380b3f00 con 0x7f2838159c60 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380b41e0 con 0x7f283815db10 2023-10-06T03:06:51.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f28380cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f28380cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42976/0 (socket says 172.21.15.92:42976) 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 -- 172.21.15.92:0/2113447337 learned_addr learned my addr 172.21.15.92:0/2113447337 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f282ffff700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2838159c60 0x7f28381440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283d438700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f283815db10 0x7f28380c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 -- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f283815db10 msgr2=0x7f28380c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f283815db10 0x7f28380c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 -- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2838159c60 msgr2=0x7f28381440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2838159c60 0x7f28381440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 -- 172.21.15.92:0/2113447337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009ae0 con 0x7f2838159890 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283cc37700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f28380cefb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2838070e20 tx=0x7f2834017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340129a0 con 0x7f2838159890 2023-10-06T03:06:51.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834012b20 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28380b44c0 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283401e430 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.632+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28380cf4f0 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.633+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283804c3c0 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.634+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f283401e860 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.634+0000 7f282cff9700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2824067170 0x7f2824069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.634+0000 7f282ffff700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2824067170 0x7f2824069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.634+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f28340863c0 con 0x7f2838159890 2023-10-06T03:06:51.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.635+0000 7f282ffff700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2824067170 0x7f2824069620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f28380b3930 tx=0x7f28280091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.638+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2834053130 con 0x7f2838159890 2023-10-06T03:06:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.822+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f283806a5b0 con 0x7f2838159890 2023-10-06T03:06:51.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.823+0000 7f282cff9700 1 -- 172.21.15.92:0/2113447337 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16882 (secure 0 0 0) 0x7f2834056750 con 0x7f2838159890 2023-10-06T03:06:51.824 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:51.824 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:50.531635+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:06:48.185180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:06:49.502363+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2824067170 msgr2=0x7f2824069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2824067170 0x7f2824069620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f28380b3930 tx=0x7f28280091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 msgr2=0x7f28380cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f28380cefb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2838070e20 tx=0x7f2834017000 comp rx=0 tx=0).stop 2023-10-06T03:06:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 shutdown_connections 2023-10-06T03:06:51.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2824067170 0x7f2824069620 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2838159890 0x7f28380cefb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f283815db10 0x7f28380c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 --2- 172.21.15.92:0/2113447337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2838159c60 0x7f28381440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 >> 172.21.15.92:0/2113447337 conn(0x7f2838068e20 msgr2=0x7f2838153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 shutdown_connections 2023-10-06T03:06:51.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.825+0000 7f283ee9b700 1 -- 172.21.15.92:0/2113447337 wait complete. 2023-10-06T03:06:51.838 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:06:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.955+0000 7f9ddc3ab700 1 Processor -- start 2023-10-06T03:06:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.955+0000 7f9ddc3ab700 1 -- start start 2023-10-06T03:06:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9ddc3ab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159890 0x7f9dd415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9ddc3ab700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd415a160 con 0x7f9dd4069560 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9ddc3ab700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd40d13a0 con 0x7f9dd4159c60 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9ddc3ab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd415e140 con 0x7f9dd4159890 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9dda948700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9dd4069560 0x7f9dd40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56564/0 (socket says 172.21.15.92:56564) 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9dda948700 1 -- 172.21.15.92:0/1763096933 learned_addr learned my addr 172.21.15.92:0/1763096933 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.956+0000 7f9dda147700 1 --2- 172.21.15.92:0/1763096933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159890 0x7f9dd415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390357669 0 0) 0x7f9dd415a160 con 0x7f9dd4069560 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc0003570 con 0x7f9dd4069560 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300700812 0 0) 0x7f9dd40d13a0 con 0x7f9dd4159c60 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd415a160 con 0x7f9dd4159c60 2023-10-06T03:06:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dda147700 1 -- 172.21.15.92:0/1763096933 >> v1:172.21.15.136:6789/0 conn(0x7f9dd4159c60 legacy=0x7f9dd415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dda147700 1 -- 172.21.15.92:0/1763096933 >> v1:172.21.15.92:6789/0 conn(0x7f9dd4069560 legacy=0x7f9dd40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dda147700 1 -- 172.21.15.92:0/1763096933 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd415a3d0 con 0x7f9dd4159890 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dda147700 1 --2- 172.21.15.92:0/1763096933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159890 0x7f9dd415abd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009e90 tx=0x7f9dd0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab512a8a0e8368 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000e070 con 0x7f9dd4159890 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd000bb80 con 0x7f9dd4159890 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.957+0000 7f9dd9145700 1 -- 172.21.15.92:0/1763096933 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0012430 con 0x7f9dd4159890 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/1763096933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159890 msgr2=0x7f9dd415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/1763096933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159890 0x7f9dd415abd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009e90 tx=0x7f9dd0009780 comp rx=0 tx=0).stop 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/1763096933 shutdown_connections 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/1763096933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159890 0x7f9dd415abd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/1763096933 >> 172.21.15.92:0/1763096933 conn(0x7f9dd4068e20 msgr2=0x7f9dd40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/1763096933 shutdown_connections 2023-10-06T03:06:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.958+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/1763096933 wait complete. 2023-10-06T03:06:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.959+0000 7f9ddc3ab700 1 Processor -- start 2023-10-06T03:06:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.959+0000 7f9ddc3ab700 1 -- start start 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 0x7f9dd40b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dd4159890 0x7f9dd414d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159c60 0x7f9dd40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40b5f20 con 0x7f9dd4069560 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd40b63b0 con 0x7f9dd4159890 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd414d790 con 0x7f9dd4159c60 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 0x7f9dd40b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 0x7f9dd40b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:42990/0 (socket says 172.21.15.92:42990) 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 -- 172.21.15.92:0/3359709584 learned_addr learned my addr 172.21.15.92:0/3359709584 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda948700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159c60 0x7f9dd40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 -- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159c60 msgr2=0x7f9dd40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dd9946700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dd4159890 0x7f9dd414d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159c60 0x7f9dd40b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 -- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dd4159890 msgr2=0x7f9dd414d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dd4159890 0x7f9dd414d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 -- 172.21.15.92:0/3359709584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0009a20 con 0x7f9dd4069560 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dd9946700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dd4159890 0x7f9dd414d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda948700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159c60 0x7f9dd40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dda147700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 0x7f9dd40b4cc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097c0 tx=0x7f9dd0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd000be40 con 0x7f9dd4069560 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0012e00 con 0x7f9dd4069560 2023-10-06T03:06:51.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd40b48b0 con 0x7f9dd4069560 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd001d880 con 0x7f9dd4069560 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.960+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd40b55f0 con 0x7f9dd4069560 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.961+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd404c3c0 con 0x7f9dd4069560 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.962+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9dd0024070 con 0x7f9dd4069560 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.962+0000 7f9dca7fc700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9db8067170 0x7f9db8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.962+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9dd00870e0 con 0x7f9dd4069560 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.962+0000 7f9dd9946700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9db8067170 0x7f9db8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:51.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.962+0000 7f9dd9946700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9db8067170 0x7f9db8069620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40b3640 tx=0x7f9dc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:51.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:51.966+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dd0053f60 con 0x7f9dd4069560 2023-10-06T03:06:52.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:52.193+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9dd40b6200 con 0x7f9dd4069560 2023-10-06T03:06:53.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.172+0000 7f9dca7fc700 1 -- 172.21.15.92:0/3359709584 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9dd0057580 con 0x7f9dd4069560 2023-10-06T03:06:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9db8067170 msgr2=0x7f9db8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9db8067170 0x7f9db8069620 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40b3640 tx=0x7f9dc4009040 comp rx=0 tx=0).stop 2023-10-06T03:06:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 msgr2=0x7f9dd40b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 0x7f9dd40b4cc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097c0 tx=0x7f9dd0019000 comp rx=0 tx=0).stop 2023-10-06T03:06:53.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 shutdown_connections 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9db8067170 0x7f9db8069620 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9dd4069560 0x7f9dd40b4cc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dd4159c60 0x7f9dd40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 --2- 172.21.15.92:0/3359709584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dd4159890 0x7f9dd414d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 >> 172.21.15.92:0/3359709584 conn(0x7f9dd4068e20 msgr2=0x7f9dd415c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 shutdown_connections 2023-10-06T03:06:53.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.174+0000 7f9ddc3ab700 1 -- 172.21.15.92:0/3359709584 wait complete. 2023-10-06T03:06:53.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:06:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.307+0000 7f99b0dc7700 1 Processor -- start 2023-10-06T03:06:53.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.307+0000 7f99b0dc7700 1 -- start start 2023-10-06T03:06:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.308+0000 7f99b0dc7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.308+0000 7f99b0dc7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f99ac15db10 0x7f99ac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.308+0000 7f99b0dc7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac15a160 con 0x7f99ac159890 2023-10-06T03:06:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.308+0000 7f99b0dc7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0d13a0 con 0x7f99ac15db10 2023-10-06T03:06:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.308+0000 7f99b0dc7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac15e140 con 0x7f99ac159c60 2023-10-06T03:06:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99a1d9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aad9d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f99ac15db10 0x7f99ac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aa59c700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f99ac159890 0x7f99ac15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:53916/0 (socket says 172.21.15.92:53916) 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aa59c700 1 -- 172.21.15.92:0/2744061449 learned_addr learned my addr 172.21.15.92:0/2744061449 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aad9d700 1 -- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 msgr2=0x7f99ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aad9d700 1 --2- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aad9d700 1 -- 172.21.15.92:0/2744061449 >> v1:172.21.15.136:6789/0 conn(0x7f99ac159890 legacy=0x7f99ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aad9d700 1 -- 172.21.15.92:0/2744061449 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac15a3d0 con 0x7f99ac15db10 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99a9d9b700 1 -- 172.21.15.92:0/2744061449 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171757728 0 0) 0x7f99ac15a160 con 0x7f99ac159890 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99a1d9b700 1 --2- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99aad9d700 1 --2- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac069560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009e60 tx=0x7f99a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf89b95b77c31bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99a9d9b700 1 -- 172.21.15.92:0/2744061449 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400e070 con 0x7f99ac15db10 2023-10-06T03:06:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99a9d9b700 1 -- 172.21.15.92:0/2744061449 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a400bb60 con 0x7f99ac15db10 2023-10-06T03:06:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.309+0000 7f99a9d9b700 1 -- 172.21.15.92:0/2744061449 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4012430 con 0x7f99ac15db10 2023-10-06T03:06:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 msgr2=0x7f99ac069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac069560 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009e60 tx=0x7f99a4009780 comp rx=0 tx=0).stop 2023-10-06T03:06:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2744061449 shutdown_connections 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac069560 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2744061449 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2744061449 >> 172.21.15.92:0/2744061449 conn(0x7f99ac068e20 msgr2=0x7f99ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2744061449 shutdown_connections 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.310+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2744061449 wait complete. 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.311+0000 7f99b0dc7700 1 Processor -- start 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.311+0000 7f99b0dc7700 1 -- start start 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99b0dc7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 0x7f99ac0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99b0dc7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac0cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99b0dc7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99b0dc7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac16d030 con 0x7f99ac159890 2023-10-06T03:06:53.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99b0dc7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0cf4d0 con 0x7f99ac15db10 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99b0dc7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0cf7b0 con 0x7f99ac159c60 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 0x7f99ac0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 0x7f99ac0ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43000/0 (socket says 172.21.15.92:43000) 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 -- 172.21.15.92:0/2683865283 learned_addr learned my addr 172.21.15.92:0/2683865283 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99a1d9b700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac0cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aad9d700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 -- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 msgr2=0x7f99ac0cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac0cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 -- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 msgr2=0x7f99ac158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.312+0000 7f99aa59c700 1 -- 172.21.15.92:0/2683865283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994009420 con 0x7f99ac159890 2023-10-06T03:06:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.313+0000 7f99aa59c700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 0x7f99ac0ceed0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c7e0 tx=0x7f999c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.313+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c005b30 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.313+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c005c90 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.313+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a4009a20 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.313+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c007300 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.313+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac0cd8b0 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.314+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac0cfba0 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.314+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f999c01b3b0 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.315+0000 7f99a2ffd700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f998c067170 0x7f998c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.315+0000 7f99a1d9b700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f998c067170 0x7f998c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.315+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f999c083f80 con 0x7f99ac159890 2023-10-06T03:06:53.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.315+0000 7f99a1d9b700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f998c067170 0x7f998c069620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f99ac15c8f0 tx=0x7f99940091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:53.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.319+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f999c050d30 con 0x7f99ac159890 2023-10-06T03:06:53.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:53.551+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f99ac0cff30 con 0x7f99ac159890 2023-10-06T03:06:54.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.182+0000 7f99a2ffd700 1 -- 172.21.15.92:0/2683865283 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99ac0cff30 con 0x7f99ac159890 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f998c067170 msgr2=0x7f998c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f998c067170 0x7f998c069620 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f99ac15c8f0 tx=0x7f99940091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 msgr2=0x7f99ac0ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 0x7f99ac0ceed0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c7e0 tx=0x7f999c007df0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 shutdown_connections 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f998c067170 0x7f998c069620 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99ac159890 0x7f99ac0ceed0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99ac15db10 0x7f99ac158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 --2- 172.21.15.92:0/2683865283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99ac159c60 0x7f99ac0cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 >> 172.21.15.92:0/2683865283 conn(0x7f99ac068e20 msgr2=0x7f99ac15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:54.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 shutdown_connections 2023-10-06T03:06:54.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.185+0000 7f99b0dc7700 1 -- 172.21.15.92:0/2683865283 wait complete. 2023-10-06T03:06:54.201 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:54.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.326+0000 7f72cb170700 1 Processor -- start 2023-10-06T03:06:54.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.326+0000 7f72cb170700 1 -- start start 2023-10-06T03:06:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72cb170700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f72c4159c10 0x7f72c415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72cb170700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c415c910 0x7f72c415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72cb170700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c415c480 con 0x7f72c40b5ea0 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72cb170700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c40b2a40 con 0x7f72c4159c10 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72cb170700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c415d290 con 0x7f72c415c910 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72c3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f72c40b5ea0 0x7f72c4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56566/0 (socket says 172.21.15.92:56566) 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72c3fff700 1 -- 172.21.15.92:0/3645455651 learned_addr learned my addr 172.21.15.92:0/3645455651 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3645455651 >> v2:172.21.15.136:3300/0 conn(0x7f72c4159c10 0x7f72c415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72c970d700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c415c910 0x7f72c415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009847388 0 0) 0x7f72c415c480 con 0x7f72c40b5ea0 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.327+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b4003570 con 0x7f72c40b5ea0 2023-10-06T03:06:54.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3524270420 0 0) 0x7f72b4003570 con 0x7f72c40b5ea0 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c415c910 msgr2=0x7f72c415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c415c910 0x7f72c415cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 >> v2:172.21.15.136:3300/0 conn(0x7f72c4159c10 msgr2=0x7f72c415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 --2- 172.21.15.92:0/3645455651 >> v2:172.21.15.136:3300/0 conn(0x7f72c4159c10 0x7f72c415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c415b5d0 con 0x7f72c40b5ea0 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f72b0001470 con 0x7f72c40b5ea0 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 >> v1:172.21.15.92:6789/0 conn(0x7f72c40b5ea0 legacy=0x7f72c4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72b4003a60 0x7f72b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4004810 0x7f72b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72b40051c0 0x7f72b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c415d290 con 0x7f72b4003a60 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b400bfa0 con 0x7f72b4004810 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b400c0e0 con 0x7f72b40051c0 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c970d700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c415c910 0x7f72c415cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3645455651 >> v2:172.21.15.136:3300/0 conn(0x7f72c4159c10 0x7f72c415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72b4003a60 0x7f72b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c3fff700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4004810 0x7f72b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c970d700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72b40051c0 0x7f72b400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 -- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72b40051c0 msgr2=0x7f72b400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72b40051c0 0x7f72b400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 -- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4004810 msgr2=0x7f72b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4004810 0x7f72b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 -- 172.21.15.92:0/3645455651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b4003570 con 0x7f72b4003a60 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72b4003a60 0x7f72b4003e70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009db0 tx=0x7f72b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263aec3ab3eae2ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.328+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8009740 con 0x7f72b4003a60 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800e920 con 0x7f72b4003a60 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72c37fe700 1 -- 172.21.15.92:0/3645455651 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800eca0 con 0x7f72b4003a60 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72cb170700 1 -- 172.21.15.92:0/3645455651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72b4003a60 msgr2=0x7f72b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72cb170700 1 --2- 172.21.15.92:0/3645455651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72b4003a60 0x7f72b4003e70 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009db0 tx=0x7f72b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72c970d700 1 -- 172.21.15.92:0/3645455651 reap_dead start 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72cb170700 1 -- 172.21.15.92:0/3645455651 shutdown_connections 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72cb170700 1 -- 172.21.15.92:0/3645455651 >> 172.21.15.92:0/3645455651 conn(0x7f72c4099a20 msgr2=0x7f72c40af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:54.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72cb170700 1 -- 172.21.15.92:0/3645455651 shutdown_connections 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.329+0000 7f72cb170700 1 -- 172.21.15.92:0/3645455651 wait complete. 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.330+0000 7f72cb170700 1 Processor -- start 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.330+0000 7f72cb170700 1 -- start start 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72cb170700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4003a60 0x7f72c4144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72cb170700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c40aeff0 0x7f72c40ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72cb170700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 0x7f72c40ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72cb170700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40aeb60 con 0x7f72c40af460 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72cb170700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4159180 con 0x7f72b4003a60 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72cb170700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c40adf50 con 0x7f72c40aeff0 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72c970d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 0x7f72c40ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72c970d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 0x7f72c40ad8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43018/0 (socket says 172.21.15.92:43018) 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72c970d700 1 -- 172.21.15.92:0/3027739472 learned_addr learned my addr 172.21.15.92:0/3027739472 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4003a60 0x7f72c4144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.331+0000 7f72c3fff700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c40aeff0 0x7f72c40ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c970d700 1 -- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c40aeff0 msgr2=0x7f72c40ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c970d700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c40aeff0 0x7f72c40ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c970d700 1 -- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4003a60 msgr2=0x7f72c4144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c970d700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4003a60 0x7f72c4144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c970d700 1 -- 172.21.15.92:0/3027739472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40ae230 con 0x7f72c40af460 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c970d700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 0x7f72c40ad8d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f72c415c410 tx=0x7f72bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00e040 con 0x7f72c40af460 2023-10-06T03:06:54.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc00f040 con 0x7f72c40af460 2023-10-06T03:06:54.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c40ae510 con 0x7f72c40af460 2023-10-06T03:06:54.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00fd90 con 0x7f72c40af460 2023-10-06T03:06:54.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.332+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c4145400 con 0x7f72c40af460 2023-10-06T03:06:54.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.333+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c40aecd0 con 0x7f72c40af460 2023-10-06T03:06:54.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.333+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f72bc015640 con 0x7f72c40af460 2023-10-06T03:06:54.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.334+0000 7f72c0ff9700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a8067170 0x7f72a8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.334+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72bc084a60 con 0x7f72c40af460 2023-10-06T03:06:54.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.334+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a8067170 0x7f72a8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.334+0000 7f72c8f0c700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a8067170 0x7f72a8069620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f72b80091e0 tx=0x7f72b8019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:54.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.338+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72bc0517d0 con 0x7f72c40af460 2023-10-06T03:06:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.566+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72c404c3c0 con 0x7f72c40af460 2023-10-06T03:06:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.566+0000 7f72c0ff9700 1 -- 172.21.15.92:0/3027739472 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f72bc01e020 con 0x7f72c40af460 2023-10-06T03:06:54.568 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:54.568 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:06:50.531995+0000","modified":"2023-10-06T03:06:54.171754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/2746794570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2746794570},{"type":"v1","addr":"172.21.15.136:6835","nonce":2746794570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:06:54.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a8067170 msgr2=0x7f72a8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a8067170 0x7f72a8069620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f72b80091e0 tx=0x7f72b8019000 comp rx=0 tx=0).stop 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 msgr2=0x7f72c40ad8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 0x7f72c40ad8d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f72c415c410 tx=0x7f72bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 shutdown_connections 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a8067170 0x7f72a8069620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c40af460 0x7f72c40ad8d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72b4003a60 0x7f72c4144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 --2- 172.21.15.92:0/3027739472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c40aeff0 0x7f72c40ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.568+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 >> 172.21.15.92:0/3027739472 conn(0x7f72c4099a20 msgr2=0x7f72c415a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.569+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 shutdown_connections 2023-10-06T03:06:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.569+0000 7f72cb170700 1 -- 172.21.15.92:0/3027739472 wait complete. 2023-10-06T03:06:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 47 2023-10-06T03:06:54.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:06:54.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.696+0000 7fb6032ed700 1 Processor -- start 2023-10-06T03:06:54.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.697+0000 7fb6032ed700 1 -- start start 2023-10-06T03:06:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.697+0000 7fb6032ed700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb5fc159890 0x7fb5fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.697+0000 7fb6032ed700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 0x7fb5fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.697+0000 7fb6032ed700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5fc15a160 con 0x7fb5fc15db10 2023-10-06T03:06:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb6032ed700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5fc0d13a0 con 0x7fb5fc159890 2023-10-06T03:06:54.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb6032ed700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc15e140 con 0x7fb5fc159c60 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb60188a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb5fc15db10 0x7fb5fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56576/0 (socket says 172.21.15.92:56576) 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb60188a700 1 -- 172.21.15.92:0/52885436 learned_addr learned my addr 172.21.15.92:0/52885436 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb601089700 1 --2- 172.21.15.92:0/52885436 >> v2:172.21.15.136:3300/0 conn(0x7fb5fc159890 0x7fb5fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb600888700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 0x7fb5fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186464099 0 0) 0x7fb5fc15a160 con 0x7fb5fc15db10 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e8003570 con 0x7fb5fc15db10 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1523544454 0 0) 0x7fb5e8003570 con 0x7fb5fc15db10 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 msgr2=0x7fb5fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 0x7fb5fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 >> v2:172.21.15.136:3300/0 conn(0x7fb5fc159890 msgr2=0x7fb5fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/52885436 >> v2:172.21.15.136:3300/0 conn(0x7fb5fc159890 0x7fb5fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc15a3d0 con 0x7fb5fc15db10 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb601089700 1 --2- 172.21.15.92:0/52885436 >> v2:172.21.15.136:3300/0 conn(0x7fb5fc159890 0x7fb5fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb5f4001470 con 0x7fb5fc15db10 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 >> v1:172.21.15.92:6789/0 conn(0x7fb5fc15db10 legacy=0x7fb5fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 0x7fb5e8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e8003fe0 0x7fb5e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8004d50 0x7fb5e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc0d13a0 con 0x7fb5e8003fe0 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc15e140 con 0x7fb5e8004d50 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e800ba90 con 0x7fb5fc159c60 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb600888700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e8003fe0 0x7fb5e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.698+0000 7fb601089700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 0x7fb5e8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb60188a700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8004d50 0x7fb5e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb600888700 1 -- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 msgr2=0x7fb5e8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb600888700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc159c60 0x7fb5e8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb600888700 1 -- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8004d50 msgr2=0x7fb5e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb600888700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8004d50 0x7fb5e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb600888700 1 -- 172.21.15.92:0/52885436 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8003570 con 0x7fb5e8003fe0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb600888700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e8003fe0 0x7fb5e8004810 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc15a090 tx=0x7fb5ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d4bab4bf0afe9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00e6b0 con 0x7fb5e8003fe0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec00e810 con 0x7fb5e8003fe0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.699+0000 7fb5f3fff700 1 -- 172.21.15.92:0/52885436 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00ebd0 con 0x7fb5e8003fe0 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb6032ed700 1 -- 172.21.15.92:0/52885436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e8003fe0 msgr2=0x7fb5e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb6032ed700 1 --2- 172.21.15.92:0/52885436 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e8003fe0 0x7fb5e8004810 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc15a090 tx=0x7fb5ec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb60188a700 1 -- 172.21.15.92:0/52885436 reap_dead start 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb6032ed700 1 -- 172.21.15.92:0/52885436 shutdown_connections 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb6032ed700 1 -- 172.21.15.92:0/52885436 >> 172.21.15.92:0/52885436 conn(0x7fb5fc068e20 msgr2=0x7fb5fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb6032ed700 1 -- 172.21.15.92:0/52885436 shutdown_connections 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.700+0000 7fb6032ed700 1 -- 172.21.15.92:0/52885436 wait complete. 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.701+0000 7fb6032ed700 1 Processor -- start 2023-10-06T03:06:54.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.701+0000 7fb6032ed700 1 -- start start 2023-10-06T03:06:54.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8003fe0 0x7fb5fc0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc14d270 0x7fb5fc0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 0x7fb5fc152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc152850 con 0x7fb5fc151ea0 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc16d180 con 0x7fb5e8003fe0 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc16dfe0 con 0x7fb5fc14d270 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 0x7fb5fc152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 0x7fb5fc152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43036/0 (socket says 172.21.15.92:43036) 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 -- 172.21.15.92:0/327704909 learned_addr learned my addr 172.21.15.92:0/327704909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb601089700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8003fe0 0x7fb5fc0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb600888700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc14d270 0x7fb5fc0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 -- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc14d270 msgr2=0x7fb5fc0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc14d270 0x7fb5fc0ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 -- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8003fe0 msgr2=0x7fb5fc0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8003fe0 0x7fb5fc0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 -- 172.21.15.92:0/327704909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc16d3d0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb60188a700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 0x7fb5fc152310 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4001050 tx=0x7fb5f4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400f8a0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f400fa20 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc16d6b0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400fde0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.702+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc16dc70 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.703+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc04c3c0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.704+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb5f40135c0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.704+0000 7fb5f17fa700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb5e0067170 0x7fb5e0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:54.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.704+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb5f4084610 con 0x7fb5fc151ea0 2023-10-06T03:06:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.704+0000 7fb601089700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb5e0067170 0x7fb5e0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.705+0000 7fb601089700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb5e0067170 0x7fb5e0069620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80094d0 tx=0x7fb5f800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:54.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.709+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5f4051490 con 0x7fb5fc151ea0 2023-10-06T03:06:54.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.892+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5fc0cfa40 con 0x7fb5fc151ea0 2023-10-06T03:06:54.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.893+0000 7fb5f17fa700 1 -- 172.21.15.92:0/327704909 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16869 (secure 0 0 0) 0x7fb5f4054ab0 con 0x7fb5fc151ea0 2023-10-06T03:06:54.894 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:54.894 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:52.159838+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:06:48.185180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:06:49.502363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:06:54.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.895+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb5e0067170 msgr2=0x7fb5e0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb5e0067170 0x7fb5e0069620 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb5f80094d0 tx=0x7fb5f800b040 comp rx=0 tx=0).stop 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 msgr2=0x7fb5fc152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 0x7fb5fc152310 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4001050 tx=0x7fb5f4001e90 comp rx=0 tx=0).stop 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 shutdown_connections 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb5e0067170 0x7fb5e0069620 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5fc151ea0 0x7fb5fc152310 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5e8003fe0 0x7fb5fc0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 --2- 172.21.15.92:0/327704909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5fc14d270 0x7fb5fc0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 >> 172.21.15.92:0/327704909 conn(0x7fb5fc068e20 msgr2=0x7fb5fc154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 shutdown_connections 2023-10-06T03:06:54.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:54.896+0000 7fb6032ed700 1 -- 172.21.15.92:0/327704909 wait complete. 2023-10-06T03:06:54.909 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:06:54.909 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:06:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.023+0000 7f851ca60700 1 Processor -- start 2023-10-06T03:06:55.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.023+0000 7f851ca60700 1 -- start start 2023-10-06T03:06:55.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f851ca60700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8518143050 0x7f8518159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f851ca60700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f85180b3f20 0x7f851806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f851ca60700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851806c390 con 0x7f851806b9e0 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f851ca60700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8518143420 con 0x7f85180b3f20 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f851ca60700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851806a1e0 con 0x7f8518143050 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f85180b3f20 0x7f851806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f85180b3f20 0x7f851806be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43042/0 (socket says 172.21.15.92:43042) 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 -- 172.21.15.92:0/2149827690 learned_addr learned my addr 172.21.15.92:0/2149827690 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f85177fe700 1 --2- 172.21.15.92:0/2149827690 >> v2:172.21.15.136:3300/0 conn(0x7f8518143050 0x7f8518159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 -- 172.21.15.92:0/2149827690 >> v2:172.21.15.136:3300/0 conn(0x7f8518143050 msgr2=0x7f8518159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 --2- 172.21.15.92:0/2149827690 >> v2:172.21.15.136:3300/0 conn(0x7f8518143050 0x7f8518159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 -- 172.21.15.92:0/2149827690 >> v1:172.21.15.136:6789/0 conn(0x7f851806b9e0 legacy=0x7f85180b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.024+0000 7f8517fff700 1 -- 172.21.15.92:0/2149827690 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180695d0 con 0x7f85180b3f20 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f8517fff700 1 --2- 172.21.15.92:0/2149827690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851806be50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f851806eec0 tx=0x7f8510009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b14f18c8baa1c23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f85167fc700 1 -- 172.21.15.92:0/2149827690 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510009e70 con 0x7f85180b3f20 2023-10-06T03:06:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f85167fc700 1 -- 172.21.15.92:0/2149827690 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851000b860 con 0x7f85180b3f20 2023-10-06T03:06:55.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f85167fc700 1 -- 172.21.15.92:0/2149827690 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000bc20 con 0x7f85180b3f20 2023-10-06T03:06:55.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 -- 172.21.15.92:0/2149827690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 msgr2=0x7f851806be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 --2- 172.21.15.92:0/2149827690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851806be50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f851806eec0 tx=0x7f8510009250 comp rx=0 tx=0).stop 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 -- 172.21.15.92:0/2149827690 shutdown_connections 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 --2- 172.21.15.92:0/2149827690 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851806be50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 --2- 172.21.15.92:0/2149827690 >> v2:172.21.15.136:3300/0 conn(0x7f8518143050 0x7f8518159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 -- 172.21.15.92:0/2149827690 >> 172.21.15.92:0/2149827690 conn(0x7f85180ce430 msgr2=0x7f85180ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 -- 172.21.15.92:0/2149827690 shutdown_connections 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.025+0000 7f851ca60700 1 -- 172.21.15.92:0/2149827690 wait complete. 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 Processor -- start 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 -- start start 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851806b9e0 0x7f851806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8518143050 0x7f8518138980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851815c770 con 0x7f85180b3f20 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851815ca50 con 0x7f8518143050 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f851ca60700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851815cd30 con 0x7f851806b9e0 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f8516ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851815bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43044/0 (socket says 172.21.15.92:43044) 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f8516ffd700 1 -- 172.21.15.92:0/117242890 learned_addr learned my addr 172.21.15.92:0/117242890 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f8517fff700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8518143050 0x7f8518138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.027+0000 7f85177fe700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851806b9e0 0x7f851806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f8516ffd700 1 -- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851806b9e0 msgr2=0x7f851806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f8516ffd700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851806b9e0 0x7f851806d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f8516ffd700 1 -- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8518143050 msgr2=0x7f8518138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f8516ffd700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8518143050 0x7f8518138980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f8516ffd700 1 -- 172.21.15.92:0/117242890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85100094f0 con 0x7f85180b3f20 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f8516ffd700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851815bd20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c4e0 tx=0x7f850c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c011590 con 0x7f85180b3f20 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c011710 con 0x7f85180b3f20 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851815d010 con 0x7f85180b3f20 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c011ad0 con 0x7f85180b3f20 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.028+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518158810 con 0x7f85180b3f20 2023-10-06T03:06:55.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.029+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851815d530 con 0x7f85180b3f20 2023-10-06T03:06:55.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.029+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f850c01b3b0 con 0x7f85180b3f20 2023-10-06T03:06:55.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.030+0000 7f84ff7fe700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8504067170 0x7f8504069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.030+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f850c083da0 con 0x7f85180b3f20 2023-10-06T03:06:55.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.030+0000 7f85177fe700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8504067170 0x7f8504069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.030+0000 7f85177fe700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8504067170 0x7f8504069620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f851815d780 tx=0x7f850800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.034+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f851815d530 con 0x7f85180b3f20 2023-10-06T03:06:55.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.216+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f851813a470 con 0x7f8504067170 2023-10-06T03:06:55.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.225+0000 7f84ff7fe700 1 -- 172.21.15.92:0/117242890 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301066 (secure 0 0 0) 0x7f851813a470 con 0x7f8504067170 2023-10-06T03:06:55.229 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8504067170 msgr2=0x7f8504069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8504067170 0x7f8504069620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f851815d780 tx=0x7f850800b040 comp rx=0 tx=0).stop 2023-10-06T03:06:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 msgr2=0x7f851815bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851815bd20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c4e0 tx=0x7f850c00c7f0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 shutdown_connections 2023-10-06T03:06:55.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8504067170 0x7f8504069620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8518143050 0x7f8518138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851806b9e0 0x7f851806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 --2- 172.21.15.92:0/117242890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f85180b3f20 0x7f851815bd20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 >> 172.21.15.92:0/117242890 conn(0x7f85180ce430 msgr2=0x7f8518159c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 shutdown_connections 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.228+0000 7f851ca60700 1 -- 172.21.15.92:0/117242890 wait complete. 2023-10-06T03:06:55.232 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:06:55.241 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2023-10-06T03:06:53.246822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2390664,"kb_used_data":8264,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752584056,"statfs":{"total":773094113280,"available":770646073344,"internally_reserved":0,"allocated":8462336,"data_stored":4437816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.103849"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.151593+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:09.909611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.146477+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:44.813810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147931+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:43.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.150025+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:22.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.146479+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:48.490492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147014+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:03.061634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.144894+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:10.184603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147431+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:16.337717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149864+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:44.813810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.144867+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:16.652174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.148204+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:43.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149833+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:15.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147130+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:00.101158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149643+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:38.875838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147111+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:56.481915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147016+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:03.971534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147265+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:44.813810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147782+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:16.337717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.146472+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:31.109495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.150661+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:31.779112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149642+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:58.127368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.150349+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:00.101158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.151801+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:20.583705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.146881+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:31.109495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149853+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:27.268883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211906+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:06:46.211906+0000","last_peered":"2023-10-06T03:06:46.211906+0000","last_clean":"2023-10-06T03:06:46.211906+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:06:46.211906+0000","last_undegraded":"2023-10-06T03:06:46.211906+0000","last_fullsized":"2023-10-06T03:06:46.211906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149844+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:31.109495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211377+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:06:46.211377+0000","last_peered":"2023-10-06T03:06:46.211377+0000","last_clean":"2023-10-06T03:06:46.211377+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:06:46.211377+0000","last_undegraded":"2023-10-06T03:06:46.211377+0000","last_fullsized":"2023-10-06T03:06:46.211377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.149905+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:29:05.233111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211837+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:06:46.211837+0000","last_peered":"2023-10-06T03:06:46.211837+0000","last_clean":"2023-10-06T03:06:46.211837+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:06:46.211837+0000","last_undegraded":"2023-10-06T03:06:46.211837+0000","last_fullsized":"2023-10-06T03:06:46.211837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.150989+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:04.488958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211971+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:06:46.211971+0000","last_peered":"2023-10-06T03:06:46.211971+0000","last_clean":"2023-10-06T03:06:46.211971+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:06:46.211971+0000","last_undegraded":"2023-10-06T03:06:46.211971+0000","last_fullsized":"2023-10-06T03:06:46.211971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147005+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:20.569533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":210,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211840+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:06:46.211840+0000","last_peered":"2023-10-06T03:06:46.211840+0000","last_clean":"2023-10-06T03:06:46.211840+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:06:46.211840+0000","last_undegraded":"2023-10-06T03:06:46.211840+0000","last_fullsized":"2023-10-06T03:06:46.211840+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.136862+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:06:46.136862+0000","last_peered":"2023-10-06T03:06:46.136862+0000","last_clean":"2023-10-06T03:06:46.136862+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:06:46.136862+0000","last_undegraded":"2023-10-06T03:06:46.136862+0000","last_fullsized":"2023-10-06T03:06:46.136862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147072+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:48.490492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211366+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:06:46.211366+0000","last_peered":"2023-10-06T03:06:46.211366+0000","last_clean":"2023-10-06T03:06:46.211366+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:06:46.211366+0000","last_undegraded":"2023-10-06T03:06:46.211366+0000","last_fullsized":"2023-10-06T03:06:46.211366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.147260+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:00.101158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.136862+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:06:46.136862+0000","last_peered":"2023-10-06T03:06:46.136862+0000","last_clean":"2023-10-06T03:06:46.136862+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:06:46.136862+0000","last_undegraded":"2023-10-06T03:06:46.136862+0000","last_fullsized":"2023-10-06T03:06:46.136862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:06:50.143044+0000","last_change":"2023-10-06T03:06:50.143044+0000","last_active":"2023-10-06T03:06:50.143044+0000","last_peered":"2023-10-06T03:06:50.143044+0000","last_clean":"2023-10-06T03:06:50.143044+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:50.143044+0000","last_undegraded":"2023-10-06T03:06:50.143044+0000","last_fullsized":"2023-10-06T03:06:50.143044+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:06:49.138384+0000","last_change":"2023-10-06T03:06:49.150096+0000","last_active":"2023-10-06T03:06:49.138384+0000","last_peered":"2023-10-06T03:06:49.138384+0000","last_clean":"2023-10-06T03:06:49.138384+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:06:49.138384+0000","last_undegraded":"2023-10-06T03:06:49.138384+0000","last_fullsized":"2023-10-06T03:06:49.138384+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:48.490492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-10-06T03:06:46.211361+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:06:46.211361+0000","last_peered":"2023-10-06T03:06:46.211361+0000","last_clean":"2023-10-06T03:06:46.211361+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:06:46.211361+0000","last_undegraded":"2023-10-06T03:06:46.211361+0000","last_fullsized":"2023-10-06T03:06:46.211361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574917,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":997503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55500000000000005,"15min":0.72799999999999998},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76200000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.433},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.38,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.501,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.374},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71599999999999997,"5min":0.875,"15min":0.875},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81299999999999994,"5min":0.999,"15min":0.999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":1.026,"15min":1.026},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574916,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":407135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.59099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.435}]},{"osd":5,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.55100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":407135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.372},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55900000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32300000000000001,"15min":0.33100000000000002},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48599999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.372},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.25700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.50800000000000001,"5min":0.628,"15min":0.628},"last":0.311},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.32600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":1.115,"15min":1.115},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84699999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.434},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.5},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82699999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298756,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073084,"statfs":{"total":96636764160,"available":96330838016,"internally_reserved":0,"allocated":1503232,"data_stored":997503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.307,"15min":0.31},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.443},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.31900000000000001,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40300000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.26600000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.433,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.374},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.48499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63500000000000001,"5min":0.871,"15min":0.871},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68000000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.754,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.746},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71099999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76300000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":407135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.32200000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.498,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.373},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.315},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.25800000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34599999999999997,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.504,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.31900000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.42599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.87,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.55500000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":1.903,"15min":1.903},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":407135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.30499999999999999},"min":{"1min":0.215,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.503,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.308},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.25600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.49299999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.314,"5min":0.33300000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.38700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35699999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.625,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77000000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.997,"15min":0.997},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73799999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.747}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":407135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52400000000000002,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64300000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54800000000000004,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70799999999999996,"5min":3,"15min":3},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.60999999999999999,"15min":0.81799999999999995},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69199999999999995,"5min":10.084,"15min":10.084},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59199999999999997,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77600000000000002,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.373},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.58999999999999997},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":1.54,"15min":1.54},"last":0.42799999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.40999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.59199999999999997},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76800000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.61599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574917,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298176,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073664,"statfs":{"total":96636764160,"available":96331431936,"internally_reserved":0,"allocated":909312,"data_stored":407135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":2.371,"15min":2.371},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.505,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79200000000000004,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78200000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67700000000000005,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.72599999999999998},"min":{"1min":0.373,"5min":0.315,"15min":0.315},"max":{"1min":0.72099999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68400000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72999999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.63400000000000001},"min":{"1min":0.45400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.746,"5min":1.052,"15min":1.052},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.63700000000000001},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":1.175,"15min":1.175},"last":0.497},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":1.145,"15min":1.145},"last":0.61699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:06:55.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:06:55.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.396+0000 7f65527ac700 1 Processor -- start 2023-10-06T03:06:55.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.396+0000 7f65527ac700 1 -- start start 2023-10-06T03:06:55.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65527ac700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f654c1597f0 0x7f654c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65527ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c15d230 0x7f654c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65527ac700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c15dbb0 con 0x7f654c159420 2023-10-06T03:06:55.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65527ac700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654c0b11d0 con 0x7f654c1597f0 2023-10-06T03:06:55.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65527ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c0b14b0 con 0x7f654c15d230 2023-10-06T03:06:55.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f6551fab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c15d230 0x7f654c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f6550fa9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f654c1597f0 0x7f654c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65517aa700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f654c159420 0x7f654c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:53928/0 (socket says 172.21.15.92:53928) 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.397+0000 7f65517aa700 1 -- 172.21.15.92:0/3670182261 learned_addr learned my addr 172.21.15.92:0/3670182261 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6551fab700 1 -- 172.21.15.92:0/3670182261 >> v2:172.21.15.136:3300/0 conn(0x7f654c1597f0 msgr2=0x7f654c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6551fab700 1 --2- 172.21.15.92:0/3670182261 >> v2:172.21.15.136:3300/0 conn(0x7f654c1597f0 0x7f654c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6551fab700 1 -- 172.21.15.92:0/3670182261 >> v1:172.21.15.136:6789/0 conn(0x7f654c159420 legacy=0x7f654c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6551fab700 1 -- 172.21.15.92:0/3670182261 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c0b1790 con 0x7f654c15d230 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6550fa9700 1 --2- 172.21.15.92:0/3670182261 >> v2:172.21.15.136:3300/0 conn(0x7f654c1597f0 0x7f654c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6551fab700 1 --2- 172.21.15.92:0/3670182261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c15d230 0x7f654c15d670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65440098e0 tx=0x7f65440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f14bbf3182c506c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6543fff700 1 -- 172.21.15.92:0/3670182261 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400b640 con 0x7f654c15d230 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6543fff700 1 -- 172.21.15.92:0/3670182261 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654400b7a0 con 0x7f654c15d230 2023-10-06T03:06:55.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f6543fff700 1 -- 172.21.15.92:0/3670182261 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654400bb40 con 0x7f654c15d230 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f65527ac700 1 -- 172.21.15.92:0/3670182261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c15d230 msgr2=0x7f654c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.398+0000 7f65527ac700 1 --2- 172.21.15.92:0/3670182261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c15d230 0x7f654c15d670 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65440098e0 tx=0x7f65440091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.399+0000 7f65527ac700 1 -- 172.21.15.92:0/3670182261 shutdown_connections 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.399+0000 7f65527ac700 1 --2- 172.21.15.92:0/3670182261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c15d230 0x7f654c15d670 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.399+0000 7f65527ac700 1 --2- 172.21.15.92:0/3670182261 >> v2:172.21.15.136:3300/0 conn(0x7f654c1597f0 0x7f654c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.399+0000 7f65527ac700 1 -- 172.21.15.92:0/3670182261 >> 172.21.15.92:0/3670182261 conn(0x7f654c0d0660 msgr2=0x7f654c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.399+0000 7f65527ac700 1 -- 172.21.15.92:0/3670182261 shutdown_connections 2023-10-06T03:06:55.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.399+0000 7f65527ac700 1 -- 172.21.15.92:0/3670182261 wait complete. 2023-10-06T03:06:55.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.400+0000 7f65527ac700 1 Processor -- start 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.400+0000 7f65527ac700 1 -- start start 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c159420 0x7f654c156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c1597f0 0x7f654c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 0x7f654c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c13aa20 con 0x7f654c15d230 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c156780 con 0x7f654c159420 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c155b70 con 0x7f654c1597f0 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 0x7f654c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 0x7f654c15af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43050/0 (socket says 172.21.15.92:43050) 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 -- 172.21.15.92:0/2711731461 learned_addr learned my addr 172.21.15.92:0/2711731461 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65517aa700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c159420 0x7f654c156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6550fa9700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c1597f0 0x7f654c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 -- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c1597f0 msgr2=0x7f654c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c1597f0 0x7f654c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 -- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c159420 msgr2=0x7f654c156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c159420 0x7f654c156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 -- 172.21.15.92:0/2711731461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c009420 con 0x7f654c15d230 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65517aa700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c159420 0x7f654c156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f6551fab700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 0x7f654c15af30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f654c15dd40 tx=0x7f6544017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544011420 con 0x7f654c15d230 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544011580 con 0x7f654c15d230 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6544009420 con 0x7f654c15d230 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544011940 con 0x7f654c15d230 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.401+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c156130 con 0x7f654c15d230 2023-10-06T03:06:55.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.402+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c1568f0 con 0x7f654c15d230 2023-10-06T03:06:55.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.403+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6544011cb0 con 0x7f654c15d230 2023-10-06T03:06:55.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.403+0000 7f65417fa700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6530067170 0x7f6530069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.403+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6544086540 con 0x7f654c15d230 2023-10-06T03:06:55.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.403+0000 7f65517aa700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6530067170 0x7f6530069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.403+0000 7f65517aa700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6530067170 0x7f6530069620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6548002230 tx=0x7f6548009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.407+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65440533c0 con 0x7f654c15d230 2023-10-06T03:06:55.453 INFO:tasks.ceph.mds.d.smithi136.stderr:2023-10-06T03:06:55.453+0000 7ffbdf7f3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:06:55.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.585+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f654c04c3c0 con 0x7f6530067170 2023-10-06T03:06:55.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.593+0000 7f65417fa700 1 -- 172.21.15.92:0/2711731461 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311788 (secure 0 0 0) 0x7f654c04c3c0 con 0x7f6530067170 2023-10-06T03:06:55.598 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6530067170 msgr2=0x7f6530069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6530067170 0x7f6530069620 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6548002230 tx=0x7f6548009040 comp rx=0 tx=0).stop 2023-10-06T03:06:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 msgr2=0x7f654c15af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 0x7f654c15af30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f654c15dd40 tx=0x7f6544017000 comp rx=0 tx=0).stop 2023-10-06T03:06:55.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 shutdown_connections 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6530067170 0x7f6530069620 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f654c15d230 0x7f654c15af30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c159420 0x7f654c156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 --2- 172.21.15.92:0/2711731461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c1597f0 0x7f654c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 >> 172.21.15.92:0/2711731461 conn(0x7f654c0d0660 msgr2=0x7f654c0697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 shutdown_connections 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.597+0000 7f65527ac700 1 -- 172.21.15.92:0/2711731461 wait complete. 2023-10-06T03:06:55.603 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:06:55.609 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2023-10-06T03:06:55.247167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395196,"kb_used_data":8700,"kb_used_omap":0,"kb_used_meta":2386432,"kb_avail":752579524,"statfs":{"total":773094113280,"available":770641432576,"internally_reserved":0,"allocated":8908800,"data_stored":4648035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2443706368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001052"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177364+0000","last_change":"2023-10-06T03:06:51.171028+0000","last_active":"2023-10-06T03:06:51.177364+0000","last_peered":"2023-10-06T03:06:51.177364+0000","last_clean":"2023-10-06T03:06:51.177364+0000","last_became_active":"2023-10-06T03:06:51.170776+0000","last_became_peered":"2023-10-06T03:06:51.170776+0000","last_unstale":"2023-10-06T03:06:51.177364+0000","last_undegraded":"2023-10-06T03:06:51.177364+0000","last_fullsized":"2023-10-06T03:06:51.177364+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:32.579039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177201+0000","last_change":"2023-10-06T03:06:51.170530+0000","last_active":"2023-10-06T03:06:51.177201+0000","last_peered":"2023-10-06T03:06:51.177201+0000","last_clean":"2023-10-06T03:06:51.177201+0000","last_became_active":"2023-10-06T03:06:51.170284+0000","last_became_peered":"2023-10-06T03:06:51.170284+0000","last_unstale":"2023-10-06T03:06:51.177201+0000","last_undegraded":"2023-10-06T03:06:51.177201+0000","last_fullsized":"2023-10-06T03:06:51.177201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:14.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173646+0000","last_change":"2023-10-06T03:06:51.167533+0000","last_active":"2023-10-06T03:06:51.173646+0000","last_peered":"2023-10-06T03:06:51.173646+0000","last_clean":"2023-10-06T03:06:51.173646+0000","last_became_active":"2023-10-06T03:06:51.167277+0000","last_became_peered":"2023-10-06T03:06:51.167277+0000","last_unstale":"2023-10-06T03:06:51.173646+0000","last_undegraded":"2023-10-06T03:06:51.173646+0000","last_fullsized":"2023-10-06T03:06:51.173646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:32.169059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174912+0000","last_change":"2023-10-06T03:06:51.167471+0000","last_active":"2023-10-06T03:06:51.174912+0000","last_peered":"2023-10-06T03:06:51.174912+0000","last_clean":"2023-10-06T03:06:51.174912+0000","last_became_active":"2023-10-06T03:06:51.167215+0000","last_became_peered":"2023-10-06T03:06:51.167215+0000","last_unstale":"2023-10-06T03:06:51.174912+0000","last_undegraded":"2023-10-06T03:06:51.174912+0000","last_fullsized":"2023-10-06T03:06:51.174912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:23.153702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.171632+0000","last_change":"2023-10-06T03:06:51.165742+0000","last_active":"2023-10-06T03:06:51.171632+0000","last_peered":"2023-10-06T03:06:51.171632+0000","last_clean":"2023-10-06T03:06:51.171632+0000","last_became_active":"2023-10-06T03:06:51.165468+0000","last_became_peered":"2023-10-06T03:06:51.165468+0000","last_unstale":"2023-10-06T03:06:51.171632+0000","last_undegraded":"2023-10-06T03:06:51.171632+0000","last_fullsized":"2023-10-06T03:06:51.171632+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:20.175386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185045+0000","last_change":"2023-10-06T03:06:51.175912+0000","last_active":"2023-10-06T03:06:51.185045+0000","last_peered":"2023-10-06T03:06:51.185045+0000","last_clean":"2023-10-06T03:06:51.185045+0000","last_became_active":"2023-10-06T03:06:51.175627+0000","last_became_peered":"2023-10-06T03:06:51.175627+0000","last_unstale":"2023-10-06T03:06:51.185045+0000","last_undegraded":"2023-10-06T03:06:51.185045+0000","last_fullsized":"2023-10-06T03:06:51.185045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:40.334839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177724+0000","last_change":"2023-10-06T03:06:51.174059+0000","last_active":"2023-10-06T03:06:51.177724+0000","last_peered":"2023-10-06T03:06:51.177724+0000","last_clean":"2023-10-06T03:06:51.177724+0000","last_became_active":"2023-10-06T03:06:51.173809+0000","last_became_peered":"2023-10-06T03:06:51.173809+0000","last_unstale":"2023-10-06T03:06:51.177724+0000","last_undegraded":"2023-10-06T03:06:51.177724+0000","last_fullsized":"2023-10-06T03:06:51.177724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:03.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174648+0000","last_change":"2023-10-06T03:06:51.165826+0000","last_active":"2023-10-06T03:06:51.174648+0000","last_peered":"2023-10-06T03:06:51.174648+0000","last_clean":"2023-10-06T03:06:51.174648+0000","last_became_active":"2023-10-06T03:06:51.165453+0000","last_became_peered":"2023-10-06T03:06:51.165453+0000","last_unstale":"2023-10-06T03:06:51.174648+0000","last_undegraded":"2023-10-06T03:06:51.174648+0000","last_fullsized":"2023-10-06T03:06:51.174648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:45.818470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.172872+0000","last_change":"2023-10-06T03:06:51.167244+0000","last_active":"2023-10-06T03:06:51.172872+0000","last_peered":"2023-10-06T03:06:51.172872+0000","last_clean":"2023-10-06T03:06:51.172872+0000","last_became_active":"2023-10-06T03:06:51.166823+0000","last_became_peered":"2023-10-06T03:06:51.166823+0000","last_unstale":"2023-10-06T03:06:51.172872+0000","last_undegraded":"2023-10-06T03:06:51.172872+0000","last_fullsized":"2023-10-06T03:06:51.172872+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:03.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.184812+0000","last_change":"2023-10-06T03:06:51.174254+0000","last_active":"2023-10-06T03:06:51.184812+0000","last_peered":"2023-10-06T03:06:51.184812+0000","last_clean":"2023-10-06T03:06:51.184812+0000","last_became_active":"2023-10-06T03:06:51.173956+0000","last_became_peered":"2023-10-06T03:06:51.173956+0000","last_unstale":"2023-10-06T03:06:51.184812+0000","last_undegraded":"2023-10-06T03:06:51.184812+0000","last_fullsized":"2023-10-06T03:06:51.184812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:24.772251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.176516+0000","last_change":"2023-10-06T03:06:51.164478+0000","last_active":"2023-10-06T03:06:51.176516+0000","last_peered":"2023-10-06T03:06:51.176516+0000","last_clean":"2023-10-06T03:06:51.176516+0000","last_became_active":"2023-10-06T03:06:51.164171+0000","last_became_peered":"2023-10-06T03:06:51.164171+0000","last_unstale":"2023-10-06T03:06:51.176516+0000","last_undegraded":"2023-10-06T03:06:51.176516+0000","last_fullsized":"2023-10-06T03:06:51.176516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:24.654659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.172339+0000","last_change":"2023-10-06T03:06:51.169841+0000","last_active":"2023-10-06T03:06:51.172339+0000","last_peered":"2023-10-06T03:06:51.172339+0000","last_clean":"2023-10-06T03:06:51.172339+0000","last_became_active":"2023-10-06T03:06:51.169579+0000","last_became_peered":"2023-10-06T03:06:51.169579+0000","last_unstale":"2023-10-06T03:06:51.172339+0000","last_undegraded":"2023-10-06T03:06:51.172339+0000","last_fullsized":"2023-10-06T03:06:51.172339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:46.890267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174888+0000","last_change":"2023-10-06T03:06:51.171235+0000","last_active":"2023-10-06T03:06:51.174888+0000","last_peered":"2023-10-06T03:06:51.174888+0000","last_clean":"2023-10-06T03:06:51.174888+0000","last_became_active":"2023-10-06T03:06:51.170984+0000","last_became_peered":"2023-10-06T03:06:51.170984+0000","last_unstale":"2023-10-06T03:06:51.174888+0000","last_undegraded":"2023-10-06T03:06:51.174888+0000","last_fullsized":"2023-10-06T03:06:51.174888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:17.656834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180517+0000","last_change":"2023-10-06T03:06:51.175952+0000","last_active":"2023-10-06T03:06:51.180517+0000","last_peered":"2023-10-06T03:06:51.180517+0000","last_clean":"2023-10-06T03:06:51.180517+0000","last_became_active":"2023-10-06T03:06:51.175683+0000","last_became_peered":"2023-10-06T03:06:51.175683+0000","last_unstale":"2023-10-06T03:06:51.180517+0000","last_undegraded":"2023-10-06T03:06:51.180517+0000","last_fullsized":"2023-10-06T03:06:51.180517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:33.842760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174823+0000","last_change":"2023-10-06T03:06:51.170079+0000","last_active":"2023-10-06T03:06:51.174823+0000","last_peered":"2023-10-06T03:06:51.174823+0000","last_clean":"2023-10-06T03:06:51.174823+0000","last_became_active":"2023-10-06T03:06:51.169827+0000","last_became_peered":"2023-10-06T03:06:51.169827+0000","last_unstale":"2023-10-06T03:06:51.174823+0000","last_undegraded":"2023-10-06T03:06:51.174823+0000","last_fullsized":"2023-10-06T03:06:51.174823+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:01.105818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185074+0000","last_change":"2023-10-06T03:06:51.177590+0000","last_active":"2023-10-06T03:06:51.185074+0000","last_peered":"2023-10-06T03:06:51.185074+0000","last_clean":"2023-10-06T03:06:51.185074+0000","last_became_active":"2023-10-06T03:06:51.177353+0000","last_became_peered":"2023-10-06T03:06:51.177353+0000","last_unstale":"2023-10-06T03:06:51.185074+0000","last_undegraded":"2023-10-06T03:06:51.185074+0000","last_fullsized":"2023-10-06T03:06:51.185074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:42.831744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185119+0000","last_change":"2023-10-06T03:06:51.176746+0000","last_active":"2023-10-06T03:06:51.185119+0000","last_peered":"2023-10-06T03:06:51.185119+0000","last_clean":"2023-10-06T03:06:51.185119+0000","last_became_active":"2023-10-06T03:06:51.176054+0000","last_became_peered":"2023-10-06T03:06:51.176054+0000","last_unstale":"2023-10-06T03:06:51.185119+0000","last_undegraded":"2023-10-06T03:06:51.185119+0000","last_fullsized":"2023-10-06T03:06:51.185119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.374699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173694+0000","last_change":"2023-10-06T03:06:51.167023+0000","last_active":"2023-10-06T03:06:51.173694+0000","last_peered":"2023-10-06T03:06:51.173694+0000","last_clean":"2023-10-06T03:06:51.173694+0000","last_became_active":"2023-10-06T03:06:51.166442+0000","last_became_peered":"2023-10-06T03:06:51.166442+0000","last_unstale":"2023-10-06T03:06:51.173694+0000","last_undegraded":"2023-10-06T03:06:51.173694+0000","last_fullsized":"2023-10-06T03:06:51.173694+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:16.746131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.182708+0000","last_change":"2023-10-06T03:06:51.177518+0000","last_active":"2023-10-06T03:06:51.182708+0000","last_peered":"2023-10-06T03:06:51.182708+0000","last_clean":"2023-10-06T03:06:51.182708+0000","last_became_active":"2023-10-06T03:06:51.176327+0000","last_became_peered":"2023-10-06T03:06:51.176327+0000","last_unstale":"2023-10-06T03:06:51.182708+0000","last_undegraded":"2023-10-06T03:06:51.182708+0000","last_fullsized":"2023-10-06T03:06:51.182708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:47.374699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174965+0000","last_change":"2023-10-06T03:06:51.171243+0000","last_active":"2023-10-06T03:06:51.174965+0000","last_peered":"2023-10-06T03:06:51.174965+0000","last_clean":"2023-10-06T03:06:51.174965+0000","last_became_active":"2023-10-06T03:06:51.170003+0000","last_became_peered":"2023-10-06T03:06:51.170003+0000","last_unstale":"2023-10-06T03:06:51.174965+0000","last_undegraded":"2023-10-06T03:06:51.174965+0000","last_fullsized":"2023-10-06T03:06:51.174965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:11.189263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.175307+0000","last_change":"2023-10-06T03:06:51.171906+0000","last_active":"2023-10-06T03:06:51.175307+0000","last_peered":"2023-10-06T03:06:51.175307+0000","last_clean":"2023-10-06T03:06:51.175307+0000","last_became_active":"2023-10-06T03:06:51.171355+0000","last_became_peered":"2023-10-06T03:06:51.171355+0000","last_unstale":"2023-10-06T03:06:51.175307+0000","last_undegraded":"2023-10-06T03:06:51.175307+0000","last_fullsized":"2023-10-06T03:06:51.175307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:12:47.932091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177573+0000","last_change":"2023-10-06T03:06:51.173528+0000","last_active":"2023-10-06T03:06:51.177573+0000","last_peered":"2023-10-06T03:06:51.177573+0000","last_clean":"2023-10-06T03:06:51.177573+0000","last_became_active":"2023-10-06T03:06:51.173270+0000","last_became_peered":"2023-10-06T03:06:51.173270+0000","last_unstale":"2023-10-06T03:06:51.177573+0000","last_undegraded":"2023-10-06T03:06:51.177573+0000","last_fullsized":"2023-10-06T03:06:51.177573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:35.617433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180263+0000","last_change":"2023-10-06T03:06:51.176128+0000","last_active":"2023-10-06T03:06:51.180263+0000","last_peered":"2023-10-06T03:06:51.180263+0000","last_clean":"2023-10-06T03:06:51.180263+0000","last_became_active":"2023-10-06T03:06:51.175838+0000","last_became_peered":"2023-10-06T03:06:51.175838+0000","last_unstale":"2023-10-06T03:06:51.180263+0000","last_undegraded":"2023-10-06T03:06:51.180263+0000","last_fullsized":"2023-10-06T03:06:51.180263+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:32.114155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173487+0000","last_change":"2023-10-06T03:06:51.165692+0000","last_active":"2023-10-06T03:06:51.173487+0000","last_peered":"2023-10-06T03:06:51.173487+0000","last_clean":"2023-10-06T03:06:51.173487+0000","last_became_active":"2023-10-06T03:06:51.165463+0000","last_became_peered":"2023-10-06T03:06:51.165463+0000","last_unstale":"2023-10-06T03:06:51.173487+0000","last_undegraded":"2023-10-06T03:06:51.173487+0000","last_fullsized":"2023-10-06T03:06:51.173487+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:16.818485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.168950+0000","last_change":"2023-10-06T03:06:51.166420+0000","last_active":"2023-10-06T03:06:51.168950+0000","last_peered":"2023-10-06T03:06:51.168950+0000","last_clean":"2023-10-06T03:06:51.168950+0000","last_became_active":"2023-10-06T03:06:51.166130+0000","last_became_peered":"2023-10-06T03:06:51.166130+0000","last_unstale":"2023-10-06T03:06:51.168950+0000","last_undegraded":"2023-10-06T03:06:51.168950+0000","last_fullsized":"2023-10-06T03:06:51.168950+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:23.606316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180461+0000","last_change":"2023-10-06T03:06:51.175867+0000","last_active":"2023-10-06T03:06:51.180461+0000","last_peered":"2023-10-06T03:06:51.180461+0000","last_clean":"2023-10-06T03:06:51.180461+0000","last_became_active":"2023-10-06T03:06:51.175611+0000","last_became_peered":"2023-10-06T03:06:51.175611+0000","last_unstale":"2023-10-06T03:06:51.180461+0000","last_undegraded":"2023-10-06T03:06:51.180461+0000","last_fullsized":"2023-10-06T03:06:51.180461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:26:42.683549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.168828+0000","last_change":"2023-10-06T03:06:51.164676+0000","last_active":"2023-10-06T03:06:51.168828+0000","last_peered":"2023-10-06T03:06:51.168828+0000","last_clean":"2023-10-06T03:06:51.168828+0000","last_became_active":"2023-10-06T03:06:51.164344+0000","last_became_peered":"2023-10-06T03:06:51.164344+0000","last_unstale":"2023-10-06T03:06:51.168828+0000","last_undegraded":"2023-10-06T03:06:51.168828+0000","last_fullsized":"2023-10-06T03:06:51.168828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:05.951668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.172867+0000","last_change":"2023-10-06T03:06:51.162751+0000","last_active":"2023-10-06T03:06:51.172867+0000","last_peered":"2023-10-06T03:06:51.172867+0000","last_clean":"2023-10-06T03:06:51.172867+0000","last_became_active":"2023-10-06T03:06:51.162425+0000","last_became_peered":"2023-10-06T03:06:51.162425+0000","last_unstale":"2023-10-06T03:06:51.172867+0000","last_undegraded":"2023-10-06T03:06:51.172867+0000","last_fullsized":"2023-10-06T03:06:51.172867+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:44.894501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177661+0000","last_change":"2023-10-06T03:06:51.171283+0000","last_active":"2023-10-06T03:06:51.177661+0000","last_peered":"2023-10-06T03:06:51.177661+0000","last_clean":"2023-10-06T03:06:51.177661+0000","last_became_active":"2023-10-06T03:06:51.171029+0000","last_became_peered":"2023-10-06T03:06:51.171029+0000","last_unstale":"2023-10-06T03:06:51.177661+0000","last_undegraded":"2023-10-06T03:06:51.177661+0000","last_fullsized":"2023-10-06T03:06:51.177661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.332365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.172898+0000","last_change":"2023-10-06T03:06:51.167782+0000","last_active":"2023-10-06T03:06:51.172898+0000","last_peered":"2023-10-06T03:06:51.172898+0000","last_clean":"2023-10-06T03:06:51.172898+0000","last_became_active":"2023-10-06T03:06:51.167523+0000","last_became_peered":"2023-10-06T03:06:51.167523+0000","last_unstale":"2023-10-06T03:06:51.172898+0000","last_undegraded":"2023-10-06T03:06:51.172898+0000","last_fullsized":"2023-10-06T03:06:51.172898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:40.334839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173125+0000","last_change":"2023-10-06T03:06:51.168677+0000","last_active":"2023-10-06T03:06:51.173125+0000","last_peered":"2023-10-06T03:06:51.173125+0000","last_clean":"2023-10-06T03:06:51.173125+0000","last_became_active":"2023-10-06T03:06:51.168442+0000","last_became_peered":"2023-10-06T03:06:51.168442+0000","last_unstale":"2023-10-06T03:06:51.173125+0000","last_undegraded":"2023-10-06T03:06:51.173125+0000","last_fullsized":"2023-10-06T03:06:51.173125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:43.292433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185050+0000","last_change":"2023-10-06T03:06:51.176407+0000","last_active":"2023-10-06T03:06:51.185050+0000","last_peered":"2023-10-06T03:06:51.185050+0000","last_clean":"2023-10-06T03:06:51.185050+0000","last_became_active":"2023-10-06T03:06:51.175670+0000","last_became_peered":"2023-10-06T03:06:51.175670+0000","last_unstale":"2023-10-06T03:06:51.185050+0000","last_undegraded":"2023-10-06T03:06:51.185050+0000","last_fullsized":"2023-10-06T03:06:51.185050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:28.497261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.169613+0000","last_change":"2023-10-06T03:06:51.168170+0000","last_active":"2023-10-06T03:06:51.169613+0000","last_peered":"2023-10-06T03:06:51.169613+0000","last_clean":"2023-10-06T03:06:51.169613+0000","last_became_active":"2023-10-06T03:06:51.167981+0000","last_became_peered":"2023-10-06T03:06:51.167981+0000","last_unstale":"2023-10-06T03:06:51.169613+0000","last_undegraded":"2023-10-06T03:06:51.169613+0000","last_fullsized":"2023-10-06T03:06:51.169613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:55.172681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"77'4","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180371+0000","last_change":"2023-10-06T03:06:50.162239+0000","last_active":"2023-10-06T03:06:51.180371+0000","last_peered":"2023-10-06T03:06:51.180371+0000","last_clean":"2023-10-06T03:06:51.180371+0000","last_became_active":"2023-10-06T03:06:50.161971+0000","last_became_peered":"2023-10-06T03:06:50.161971+0000","last_unstale":"2023-10-06T03:06:51.180371+0000","last_undegraded":"2023-10-06T03:06:51.180371+0000","last_fullsized":"2023-10-06T03:06:51.180371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:20.178316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177477+0000","last_change":"2023-10-06T03:06:51.169035+0000","last_active":"2023-10-06T03:06:51.177477+0000","last_peered":"2023-10-06T03:06:51.177477+0000","last_clean":"2023-10-06T03:06:51.177477+0000","last_became_active":"2023-10-06T03:06:51.168803+0000","last_became_peered":"2023-10-06T03:06:51.168803+0000","last_unstale":"2023-10-06T03:06:51.177477+0000","last_undegraded":"2023-10-06T03:06:51.177477+0000","last_fullsized":"2023-10-06T03:06:51.177477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:24.654659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.176013+0000","last_change":"2023-10-06T03:06:50.164209+0000","last_active":"2023-10-06T03:06:51.176013+0000","last_peered":"2023-10-06T03:06:51.176013+0000","last_clean":"2023-10-06T03:06:51.176013+0000","last_became_active":"2023-10-06T03:06:50.163939+0000","last_became_peered":"2023-10-06T03:06:50.163939+0000","last_unstale":"2023-10-06T03:06:51.176013+0000","last_undegraded":"2023-10-06T03:06:51.176013+0000","last_fullsized":"2023-10-06T03:06:51.176013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:31.574379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180265+0000","last_change":"2023-10-06T03:06:51.175996+0000","last_active":"2023-10-06T03:06:51.180265+0000","last_peered":"2023-10-06T03:06:51.180265+0000","last_clean":"2023-10-06T03:06:51.180265+0000","last_became_active":"2023-10-06T03:06:51.175754+0000","last_became_peered":"2023-10-06T03:06:51.175754+0000","last_unstale":"2023-10-06T03:06:51.180265+0000","last_undegraded":"2023-10-06T03:06:51.180265+0000","last_fullsized":"2023-10-06T03:06:51.180265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:08.989278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180252+0000","last_change":"2023-10-06T03:06:50.162370+0000","last_active":"2023-10-06T03:06:51.180252+0000","last_peered":"2023-10-06T03:06:51.180252+0000","last_clean":"2023-10-06T03:06:51.180252+0000","last_became_active":"2023-10-06T03:06:50.162087+0000","last_became_peered":"2023-10-06T03:06:50.162087+0000","last_unstale":"2023-10-06T03:06:51.180252+0000","last_undegraded":"2023-10-06T03:06:51.180252+0000","last_fullsized":"2023-10-06T03:06:51.180252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:58.127368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.179481+0000","last_change":"2023-10-06T03:06:51.171072+0000","last_active":"2023-10-06T03:06:51.179481+0000","last_peered":"2023-10-06T03:06:51.179481+0000","last_clean":"2023-10-06T03:06:51.179481+0000","last_became_active":"2023-10-06T03:06:51.170836+0000","last_became_peered":"2023-10-06T03:06:51.170836+0000","last_unstale":"2023-10-06T03:06:51.179481+0000","last_undegraded":"2023-10-06T03:06:51.179481+0000","last_fullsized":"2023-10-06T03:06:51.179481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:20.175386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180036+0000","last_change":"2023-10-06T03:06:50.166874+0000","last_active":"2023-10-06T03:06:51.180036+0000","last_peered":"2023-10-06T03:06:51.180036+0000","last_clean":"2023-10-06T03:06:51.180036+0000","last_became_active":"2023-10-06T03:06:50.166594+0000","last_became_peered":"2023-10-06T03:06:50.166594+0000","last_unstale":"2023-10-06T03:06:51.180036+0000","last_undegraded":"2023-10-06T03:06:51.180036+0000","last_fullsized":"2023-10-06T03:06:51.180036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:55.998589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.175170+0000","last_change":"2023-10-06T03:06:51.169219+0000","last_active":"2023-10-06T03:06:51.175170+0000","last_peered":"2023-10-06T03:06:51.175170+0000","last_clean":"2023-10-06T03:06:51.175170+0000","last_became_active":"2023-10-06T03:06:51.168984+0000","last_became_peered":"2023-10-06T03:06:51.168984+0000","last_unstale":"2023-10-06T03:06:51.175170+0000","last_undegraded":"2023-10-06T03:06:51.175170+0000","last_fullsized":"2023-10-06T03:06:51.175170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:17.342377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162894+0000","last_change":"2023-10-06T03:06:50.162597+0000","last_active":"2023-10-06T03:06:51.162894+0000","last_peered":"2023-10-06T03:06:51.162894+0000","last_clean":"2023-10-06T03:06:51.162894+0000","last_became_active":"2023-10-06T03:06:50.162332+0000","last_became_peered":"2023-10-06T03:06:50.162332+0000","last_unstale":"2023-10-06T03:06:51.162894+0000","last_undegraded":"2023-10-06T03:06:51.162894+0000","last_fullsized":"2023-10-06T03:06:51.162894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:15.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.175080+0000","last_change":"2023-10-06T03:06:51.172094+0000","last_active":"2023-10-06T03:06:51.175080+0000","last_peered":"2023-10-06T03:06:51.175080+0000","last_clean":"2023-10-06T03:06:51.175080+0000","last_became_active":"2023-10-06T03:06:51.171850+0000","last_became_peered":"2023-10-06T03:06:51.171850+0000","last_unstale":"2023-10-06T03:06:51.175080+0000","last_undegraded":"2023-10-06T03:06:51.175080+0000","last_fullsized":"2023-10-06T03:06:51.175080+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:44.894501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.172992+0000","last_change":"2023-10-06T03:06:50.163527+0000","last_active":"2023-10-06T03:06:51.172992+0000","last_peered":"2023-10-06T03:06:51.172992+0000","last_clean":"2023-10-06T03:06:51.172992+0000","last_became_active":"2023-10-06T03:06:50.163046+0000","last_became_peered":"2023-10-06T03:06:50.163046+0000","last_unstale":"2023-10-06T03:06:51.172992+0000","last_undegraded":"2023-10-06T03:06:51.172992+0000","last_fullsized":"2023-10-06T03:06:51.172992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:34.691242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185068+0000","last_change":"2023-10-06T03:06:51.176204+0000","last_active":"2023-10-06T03:06:51.185068+0000","last_peered":"2023-10-06T03:06:51.185068+0000","last_clean":"2023-10-06T03:06:51.185068+0000","last_became_active":"2023-10-06T03:06:51.175961+0000","last_became_peered":"2023-10-06T03:06:51.175961+0000","last_unstale":"2023-10-06T03:06:51.185068+0000","last_undegraded":"2023-10-06T03:06:51.185068+0000","last_fullsized":"2023-10-06T03:06:51.185068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:24.499960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.159693+0000","last_change":"2023-10-06T03:06:50.162378+0000","last_active":"2023-10-06T03:06:51.159693+0000","last_peered":"2023-10-06T03:06:51.159693+0000","last_clean":"2023-10-06T03:06:51.159693+0000","last_became_active":"2023-10-06T03:06:50.162126+0000","last_became_peered":"2023-10-06T03:06:50.162126+0000","last_unstale":"2023-10-06T03:06:51.159693+0000","last_undegraded":"2023-10-06T03:06:51.159693+0000","last_fullsized":"2023-10-06T03:06:51.159693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:44.813810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174869+0000","last_change":"2023-10-06T03:06:51.167472+0000","last_active":"2023-10-06T03:06:51.174869+0000","last_peered":"2023-10-06T03:06:51.174869+0000","last_clean":"2023-10-06T03:06:51.174869+0000","last_became_active":"2023-10-06T03:06:51.167240+0000","last_became_peered":"2023-10-06T03:06:51.167240+0000","last_unstale":"2023-10-06T03:06:51.174869+0000","last_undegraded":"2023-10-06T03:06:51.174869+0000","last_fullsized":"2023-10-06T03:06:51.174869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:32.114155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.164008+0000","last_change":"2023-10-06T03:06:50.167270+0000","last_active":"2023-10-06T03:06:51.164008+0000","last_peered":"2023-10-06T03:06:51.164008+0000","last_clean":"2023-10-06T03:06:51.164008+0000","last_became_active":"2023-10-06T03:06:50.167042+0000","last_became_peered":"2023-10-06T03:06:50.167042+0000","last_unstale":"2023-10-06T03:06:51.164008+0000","last_undegraded":"2023-10-06T03:06:51.164008+0000","last_fullsized":"2023-10-06T03:06:51.164008+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:30.260946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.176099+0000","last_change":"2023-10-06T03:06:51.171774+0000","last_active":"2023-10-06T03:06:51.176099+0000","last_peered":"2023-10-06T03:06:51.176099+0000","last_clean":"2023-10-06T03:06:51.176099+0000","last_became_active":"2023-10-06T03:06:51.171439+0000","last_became_peered":"2023-10-06T03:06:51.171439+0000","last_unstale":"2023-10-06T03:06:51.176099+0000","last_undegraded":"2023-10-06T03:06:51.176099+0000","last_fullsized":"2023-10-06T03:06:51.176099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:12.821049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.158779+0000","last_change":"2023-10-06T03:06:50.159355+0000","last_active":"2023-10-06T03:06:51.158779+0000","last_peered":"2023-10-06T03:06:51.158779+0000","last_clean":"2023-10-06T03:06:51.158779+0000","last_became_active":"2023-10-06T03:06:50.159100+0000","last_became_peered":"2023-10-06T03:06:50.159100+0000","last_unstale":"2023-10-06T03:06:51.158779+0000","last_undegraded":"2023-10-06T03:06:51.158779+0000","last_fullsized":"2023-10-06T03:06:51.158779+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:16.337717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180309+0000","last_change":"2023-10-06T03:06:51.177399+0000","last_active":"2023-10-06T03:06:51.180309+0000","last_peered":"2023-10-06T03:06:51.180309+0000","last_clean":"2023-10-06T03:06:51.180309+0000","last_became_active":"2023-10-06T03:06:51.177128+0000","last_became_peered":"2023-10-06T03:06:51.177128+0000","last_unstale":"2023-10-06T03:06:51.180309+0000","last_undegraded":"2023-10-06T03:06:51.180309+0000","last_fullsized":"2023-10-06T03:06:51.180309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:05.952410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.157871+0000","last_change":"2023-10-06T03:06:50.165433+0000","last_active":"2023-10-06T03:06:51.157871+0000","last_peered":"2023-10-06T03:06:51.157871+0000","last_clean":"2023-10-06T03:06:51.157871+0000","last_became_active":"2023-10-06T03:06:50.165151+0000","last_became_peered":"2023-10-06T03:06:50.165151+0000","last_unstale":"2023-10-06T03:06:51.157871+0000","last_undegraded":"2023-10-06T03:06:51.157871+0000","last_fullsized":"2023-10-06T03:06:51.157871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:31.109495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173715+0000","last_change":"2023-10-06T03:06:51.167175+0000","last_active":"2023-10-06T03:06:51.173715+0000","last_peered":"2023-10-06T03:06:51.173715+0000","last_clean":"2023-10-06T03:06:51.173715+0000","last_became_active":"2023-10-06T03:06:51.166778+0000","last_became_peered":"2023-10-06T03:06:51.166778+0000","last_unstale":"2023-10-06T03:06:51.173715+0000","last_undegraded":"2023-10-06T03:06:51.173715+0000","last_fullsized":"2023-10-06T03:06:51.173715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:22.521227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"77'3","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.176750+0000","last_change":"2023-10-06T03:06:50.165581+0000","last_active":"2023-10-06T03:06:51.176750+0000","last_peered":"2023-10-06T03:06:51.176750+0000","last_clean":"2023-10-06T03:06:51.176750+0000","last_became_active":"2023-10-06T03:06:50.165268+0000","last_became_peered":"2023-10-06T03:06:50.165268+0000","last_unstale":"2023-10-06T03:06:51.176750+0000","last_undegraded":"2023-10-06T03:06:51.176750+0000","last_fullsized":"2023-10-06T03:06:51.176750+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:38.875838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.175178+0000","last_change":"2023-10-06T03:06:51.166293+0000","last_active":"2023-10-06T03:06:51.175178+0000","last_peered":"2023-10-06T03:06:51.175178+0000","last_clean":"2023-10-06T03:06:51.175178+0000","last_became_active":"2023-10-06T03:06:51.166029+0000","last_became_peered":"2023-10-06T03:06:51.166029+0000","last_unstale":"2023-10-06T03:06:51.175178+0000","last_undegraded":"2023-10-06T03:06:51.175178+0000","last_fullsized":"2023-10-06T03:06:51.175178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:36.340761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"77'6","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185282+0000","last_change":"2023-10-06T03:06:50.157172+0000","last_active":"2023-10-06T03:06:51.185282+0000","last_peered":"2023-10-06T03:06:51.185282+0000","last_clean":"2023-10-06T03:06:51.185282+0000","last_became_active":"2023-10-06T03:06:50.156919+0000","last_became_peered":"2023-10-06T03:06:50.156919+0000","last_unstale":"2023-10-06T03:06:51.185282+0000","last_undegraded":"2023-10-06T03:06:51.185282+0000","last_fullsized":"2023-10-06T03:06:51.185282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:31.164399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174628+0000","last_change":"2023-10-06T03:06:51.165707+0000","last_active":"2023-10-06T03:06:51.174628+0000","last_peered":"2023-10-06T03:06:51.174628+0000","last_clean":"2023-10-06T03:06:51.174628+0000","last_became_active":"2023-10-06T03:06:51.165117+0000","last_became_peered":"2023-10-06T03:06:51.165117+0000","last_unstale":"2023-10-06T03:06:51.174628+0000","last_undegraded":"2023-10-06T03:06:51.174628+0000","last_fullsized":"2023-10-06T03:06:51.174628+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:49.495152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180046+0000","last_change":"2023-10-06T03:06:50.162377+0000","last_active":"2023-10-06T03:06:51.180046+0000","last_peered":"2023-10-06T03:06:51.180046+0000","last_clean":"2023-10-06T03:06:51.180046+0000","last_became_active":"2023-10-06T03:06:50.162050+0000","last_became_peered":"2023-10-06T03:06:50.162050+0000","last_unstale":"2023-10-06T03:06:51.180046+0000","last_undegraded":"2023-10-06T03:06:51.180046+0000","last_fullsized":"2023-10-06T03:06:51.180046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:15.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.171564+0000","last_change":"2023-10-06T03:06:51.165161+0000","last_active":"2023-10-06T03:06:51.171564+0000","last_peered":"2023-10-06T03:06:51.171564+0000","last_clean":"2023-10-06T03:06:51.171564+0000","last_became_active":"2023-10-06T03:06:51.164894+0000","last_became_peered":"2023-10-06T03:06:51.164894+0000","last_unstale":"2023-10-06T03:06:51.171564+0000","last_undegraded":"2023-10-06T03:06:51.171564+0000","last_fullsized":"2023-10-06T03:06:51.171564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:24.499960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.158215+0000","last_change":"2023-10-06T03:06:50.159738+0000","last_active":"2023-10-06T03:06:51.158215+0000","last_peered":"2023-10-06T03:06:51.158215+0000","last_clean":"2023-10-06T03:06:51.158215+0000","last_became_active":"2023-10-06T03:06:50.159552+0000","last_became_peered":"2023-10-06T03:06:50.159552+0000","last_unstale":"2023-10-06T03:06:51.158215+0000","last_undegraded":"2023-10-06T03:06:51.158215+0000","last_fullsized":"2023-10-06T03:06:51.158215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:21.516567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.170438+0000","last_change":"2023-10-06T03:06:51.167275+0000","last_active":"2023-10-06T03:06:51.170438+0000","last_peered":"2023-10-06T03:06:51.170438+0000","last_clean":"2023-10-06T03:06:51.170438+0000","last_became_active":"2023-10-06T03:06:51.167019+0000","last_became_peered":"2023-10-06T03:06:51.167019+0000","last_unstale":"2023-10-06T03:06:51.170438+0000","last_undegraded":"2023-10-06T03:06:51.170438+0000","last_fullsized":"2023-10-06T03:06:51.170438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:37.362697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.153346+0000","last_change":"2023-10-06T03:06:50.165952+0000","last_active":"2023-10-06T03:06:51.153346+0000","last_peered":"2023-10-06T03:06:51.153346+0000","last_clean":"2023-10-06T03:06:51.153346+0000","last_became_active":"2023-10-06T03:06:50.165473+0000","last_became_peered":"2023-10-06T03:06:50.165473+0000","last_unstale":"2023-10-06T03:06:51.153346+0000","last_undegraded":"2023-10-06T03:06:51.153346+0000","last_fullsized":"2023-10-06T03:06:51.153346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:04.488958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.168335+0000","last_change":"2023-10-06T03:06:51.162636+0000","last_active":"2023-10-06T03:06:51.168335+0000","last_peered":"2023-10-06T03:06:51.168335+0000","last_clean":"2023-10-06T03:06:51.168335+0000","last_became_active":"2023-10-06T03:06:51.162376+0000","last_became_peered":"2023-10-06T03:06:51.162376+0000","last_unstale":"2023-10-06T03:06:51.168335+0000","last_undegraded":"2023-10-06T03:06:51.168335+0000","last_fullsized":"2023-10-06T03:06:51.168335+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:44.646176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"77'8","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.175116+0000","last_change":"2023-10-06T03:06:50.168820+0000","last_active":"2023-10-06T03:06:51.175116+0000","last_peered":"2023-10-06T03:06:51.175116+0000","last_clean":"2023-10-06T03:06:51.175116+0000","last_became_active":"2023-10-06T03:06:50.168603+0000","last_became_peered":"2023-10-06T03:06:50.168603+0000","last_unstale":"2023-10-06T03:06:51.175116+0000","last_undegraded":"2023-10-06T03:06:51.175116+0000","last_fullsized":"2023-10-06T03:06:51.175116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:31.779112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173310+0000","last_change":"2023-10-06T03:06:51.171655+0000","last_active":"2023-10-06T03:06:51.173310+0000","last_peered":"2023-10-06T03:06:51.173310+0000","last_clean":"2023-10-06T03:06:51.173310+0000","last_became_active":"2023-10-06T03:06:51.171013+0000","last_became_peered":"2023-10-06T03:06:51.171013+0000","last_unstale":"2023-10-06T03:06:51.173310+0000","last_undegraded":"2023-10-06T03:06:51.173310+0000","last_fullsized":"2023-10-06T03:06:51.173310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:17.656834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174056+0000","last_change":"2023-10-06T03:06:50.163596+0000","last_active":"2023-10-06T03:06:51.174056+0000","last_peered":"2023-10-06T03:06:51.174056+0000","last_clean":"2023-10-06T03:06:51.174056+0000","last_became_active":"2023-10-06T03:06:50.163331+0000","last_became_peered":"2023-10-06T03:06:50.163331+0000","last_unstale":"2023-10-06T03:06:51.174056+0000","last_undegraded":"2023-10-06T03:06:51.174056+0000","last_fullsized":"2023-10-06T03:06:51.174056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:31.574379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185017+0000","last_change":"2023-10-06T03:06:51.176028+0000","last_active":"2023-10-06T03:06:51.185017+0000","last_peered":"2023-10-06T03:06:51.185017+0000","last_clean":"2023-10-06T03:06:51.185017+0000","last_became_active":"2023-10-06T03:06:51.175785+0000","last_became_peered":"2023-10-06T03:06:51.175785+0000","last_unstale":"2023-10-06T03:06:51.185017+0000","last_undegraded":"2023-10-06T03:06:51.185017+0000","last_fullsized":"2023-10-06T03:06:51.185017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:07.748790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.155127+0000","last_change":"2023-10-06T03:06:50.168235+0000","last_active":"2023-10-06T03:06:51.155127+0000","last_peered":"2023-10-06T03:06:51.155127+0000","last_clean":"2023-10-06T03:06:51.155127+0000","last_became_active":"2023-10-06T03:06:50.167950+0000","last_became_peered":"2023-10-06T03:06:50.167950+0000","last_unstale":"2023-10-06T03:06:51.155127+0000","last_undegraded":"2023-10-06T03:06:51.155127+0000","last_fullsized":"2023-10-06T03:06:51.155127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:30.260946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.179503+0000","last_change":"2023-10-06T03:06:51.170779+0000","last_active":"2023-10-06T03:06:51.179503+0000","last_peered":"2023-10-06T03:06:51.179503+0000","last_clean":"2023-10-06T03:06:51.179503+0000","last_became_active":"2023-10-06T03:06:51.170518+0000","last_became_peered":"2023-10-06T03:06:51.170518+0000","last_unstale":"2023-10-06T03:06:51.179503+0000","last_undegraded":"2023-10-06T03:06:51.179503+0000","last_fullsized":"2023-10-06T03:06:51.179503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:35.617433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.156279+0000","last_change":"2023-10-06T03:06:50.162762+0000","last_active":"2023-10-06T03:06:51.156279+0000","last_peered":"2023-10-06T03:06:51.156279+0000","last_clean":"2023-10-06T03:06:51.156279+0000","last_became_active":"2023-10-06T03:06:50.162333+0000","last_became_peered":"2023-10-06T03:06:50.162333+0000","last_unstale":"2023-10-06T03:06:51.156279+0000","last_undegraded":"2023-10-06T03:06:51.156279+0000","last_fullsized":"2023-10-06T03:06:51.156279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:58.127368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.168926+0000","last_change":"2023-10-06T03:06:51.165240+0000","last_active":"2023-10-06T03:06:51.168926+0000","last_peered":"2023-10-06T03:06:51.168926+0000","last_clean":"2023-10-06T03:06:51.168926+0000","last_became_active":"2023-10-06T03:06:51.164933+0000","last_became_peered":"2023-10-06T03:06:51.164933+0000","last_unstale":"2023-10-06T03:06:51.168926+0000","last_undegraded":"2023-10-06T03:06:51.168926+0000","last_fullsized":"2023-10-06T03:06:51.168926+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:14.332365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162431+0000","last_change":"2023-10-06T03:06:50.159592+0000","last_active":"2023-10-06T03:06:51.162431+0000","last_peered":"2023-10-06T03:06:51.162431+0000","last_clean":"2023-10-06T03:06:51.162431+0000","last_became_active":"2023-10-06T03:06:50.159279+0000","last_became_peered":"2023-10-06T03:06:50.159279+0000","last_unstale":"2023-10-06T03:06:51.162431+0000","last_undegraded":"2023-10-06T03:06:51.162431+0000","last_fullsized":"2023-10-06T03:06:51.162431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:33.305496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.179693+0000","last_change":"2023-10-06T03:06:51.171425+0000","last_active":"2023-10-06T03:06:51.179693+0000","last_peered":"2023-10-06T03:06:51.179693+0000","last_clean":"2023-10-06T03:06:51.179693+0000","last_became_active":"2023-10-06T03:06:51.171042+0000","last_became_peered":"2023-10-06T03:06:51.171042+0000","last_unstale":"2023-10-06T03:06:51.179693+0000","last_undegraded":"2023-10-06T03:06:51.179693+0000","last_fullsized":"2023-10-06T03:06:51.179693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:24.991546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.159231+0000","last_change":"2023-10-06T03:06:50.160340+0000","last_active":"2023-10-06T03:06:51.159231+0000","last_peered":"2023-10-06T03:06:51.159231+0000","last_clean":"2023-10-06T03:06:51.159231+0000","last_became_active":"2023-10-06T03:06:50.160125+0000","last_became_peered":"2023-10-06T03:06:50.160125+0000","last_unstale":"2023-10-06T03:06:51.159231+0000","last_undegraded":"2023-10-06T03:06:51.159231+0000","last_fullsized":"2023-10-06T03:06:51.159231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:31.574379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177466+0000","last_change":"2023-10-06T03:06:51.173322+0000","last_active":"2023-10-06T03:06:51.177466+0000","last_peered":"2023-10-06T03:06:51.177466+0000","last_clean":"2023-10-06T03:06:51.177466+0000","last_became_active":"2023-10-06T03:06:51.172824+0000","last_became_peered":"2023-10-06T03:06:51.172824+0000","last_unstale":"2023-10-06T03:06:51.177466+0000","last_undegraded":"2023-10-06T03:06:51.177466+0000","last_fullsized":"2023-10-06T03:06:51.177466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:43.292433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.161924+0000","last_change":"2023-10-06T03:06:50.161609+0000","last_active":"2023-10-06T03:06:51.161924+0000","last_peered":"2023-10-06T03:06:51.161924+0000","last_clean":"2023-10-06T03:06:51.161924+0000","last_became_active":"2023-10-06T03:06:50.161325+0000","last_became_peered":"2023-10-06T03:06:50.161325+0000","last_unstale":"2023-10-06T03:06:51.161924+0000","last_undegraded":"2023-10-06T03:06:51.161924+0000","last_fullsized":"2023-10-06T03:06:51.161924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:43.294217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.180200+0000","last_change":"2023-10-06T03:06:51.176259+0000","last_active":"2023-10-06T03:06:51.180200+0000","last_peered":"2023-10-06T03:06:51.180200+0000","last_clean":"2023-10-06T03:06:51.180200+0000","last_became_active":"2023-10-06T03:06:51.175881+0000","last_became_peered":"2023-10-06T03:06:51.175881+0000","last_unstale":"2023-10-06T03:06:51.180200+0000","last_undegraded":"2023-10-06T03:06:51.180200+0000","last_fullsized":"2023-10-06T03:06:51.180200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:45.818470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.164010+0000","last_change":"2023-10-06T03:06:50.159741+0000","last_active":"2023-10-06T03:06:51.164010+0000","last_peered":"2023-10-06T03:06:51.164010+0000","last_clean":"2023-10-06T03:06:51.164010+0000","last_became_active":"2023-10-06T03:06:50.159479+0000","last_became_peered":"2023-10-06T03:06:50.159479+0000","last_unstale":"2023-10-06T03:06:51.164010+0000","last_undegraded":"2023-10-06T03:06:51.164010+0000","last_fullsized":"2023-10-06T03:06:51.164010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:31.484478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.184813+0000","last_change":"2023-10-06T03:06:51.175736+0000","last_active":"2023-10-06T03:06:51.184813+0000","last_peered":"2023-10-06T03:06:51.184813+0000","last_clean":"2023-10-06T03:06:51.184813+0000","last_became_active":"2023-10-06T03:06:51.175424+0000","last_became_peered":"2023-10-06T03:06:51.175424+0000","last_unstale":"2023-10-06T03:06:51.184813+0000","last_undegraded":"2023-10-06T03:06:51.184813+0000","last_fullsized":"2023-10-06T03:06:51.184813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:33.842760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162740+0000","last_change":"2023-10-06T03:06:50.169549+0000","last_active":"2023-10-06T03:06:51.162740+0000","last_peered":"2023-10-06T03:06:51.162740+0000","last_clean":"2023-10-06T03:06:51.162740+0000","last_became_active":"2023-10-06T03:06:50.169321+0000","last_became_peered":"2023-10-06T03:06:50.169321+0000","last_unstale":"2023-10-06T03:06:51.162740+0000","last_undegraded":"2023-10-06T03:06:51.162740+0000","last_fullsized":"2023-10-06T03:06:51.162740+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:31.532307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.169645+0000","last_change":"2023-10-06T03:06:51.167275+0000","last_active":"2023-10-06T03:06:51.169645+0000","last_peered":"2023-10-06T03:06:51.169645+0000","last_clean":"2023-10-06T03:06:51.169645+0000","last_became_active":"2023-10-06T03:06:51.167037+0000","last_became_peered":"2023-10-06T03:06:51.167037+0000","last_unstale":"2023-10-06T03:06:51.169645+0000","last_undegraded":"2023-10-06T03:06:51.169645+0000","last_fullsized":"2023-10-06T03:06:51.169645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:39.686927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.158749+0000","last_change":"2023-10-06T03:06:50.165117+0000","last_active":"2023-10-06T03:06:51.158749+0000","last_peered":"2023-10-06T03:06:51.158749+0000","last_clean":"2023-10-06T03:06:51.158749+0000","last_became_active":"2023-10-06T03:06:50.164891+0000","last_became_peered":"2023-10-06T03:06:50.164891+0000","last_unstale":"2023-10-06T03:06:51.158749+0000","last_undegraded":"2023-10-06T03:06:51.158749+0000","last_fullsized":"2023-10-06T03:06:51.158749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:11.831457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162519+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:06:51.162519+0000","last_peered":"2023-10-06T03:06:51.162519+0000","last_clean":"2023-10-06T03:06:51.162519+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:06:51.162519+0000","last_undegraded":"2023-10-06T03:06:51.162519+0000","last_fullsized":"2023-10-06T03:06:51.162519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173197+0000","last_change":"2023-10-06T03:06:51.168571+0000","last_active":"2023-10-06T03:06:51.173197+0000","last_peered":"2023-10-06T03:06:51.173197+0000","last_clean":"2023-10-06T03:06:51.173197+0000","last_became_active":"2023-10-06T03:06:51.168302+0000","last_became_peered":"2023-10-06T03:06:51.168302+0000","last_unstale":"2023-10-06T03:06:51.173197+0000","last_undegraded":"2023-10-06T03:06:51.173197+0000","last_fullsized":"2023-10-06T03:06:51.173197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:03.614360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.157326+0000","last_change":"2023-10-06T03:06:50.154985+0000","last_active":"2023-10-06T03:06:51.157326+0000","last_peered":"2023-10-06T03:06:51.157326+0000","last_clean":"2023-10-06T03:06:51.157326+0000","last_became_active":"2023-10-06T03:06:50.154690+0000","last_became_peered":"2023-10-06T03:06:50.154690+0000","last_unstale":"2023-10-06T03:06:51.157326+0000","last_undegraded":"2023-10-06T03:06:51.157326+0000","last_fullsized":"2023-10-06T03:06:51.157326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:30.260946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.153307+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:06:51.153307+0000","last_peered":"2023-10-06T03:06:51.153307+0000","last_clean":"2023-10-06T03:06:51.153307+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:06:51.153307+0000","last_undegraded":"2023-10-06T03:06:51.153307+0000","last_fullsized":"2023-10-06T03:06:51.153307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173275+0000","last_change":"2023-10-06T03:06:51.169637+0000","last_active":"2023-10-06T03:06:51.173275+0000","last_peered":"2023-10-06T03:06:51.173275+0000","last_clean":"2023-10-06T03:06:51.173275+0000","last_became_active":"2023-10-06T03:06:51.169418+0000","last_became_peered":"2023-10-06T03:06:51.169418+0000","last_unstale":"2023-10-06T03:06:51.173275+0000","last_undegraded":"2023-10-06T03:06:51.173275+0000","last_fullsized":"2023-10-06T03:06:51.173275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:05.952410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.157431+0000","last_change":"2023-10-06T03:06:50.166795+0000","last_active":"2023-10-06T03:06:51.157431+0000","last_peered":"2023-10-06T03:06:51.157431+0000","last_clean":"2023-10-06T03:06:51.157431+0000","last_became_active":"2023-10-06T03:06:50.166530+0000","last_became_peered":"2023-10-06T03:06:50.166530+0000","last_unstale":"2023-10-06T03:06:51.157431+0000","last_undegraded":"2023-10-06T03:06:51.157431+0000","last_fullsized":"2023-10-06T03:06:51.157431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:58:59.407109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.163799+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:06:51.163799+0000","last_peered":"2023-10-06T03:06:51.163799+0000","last_clean":"2023-10-06T03:06:51.163799+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:06:51.163799+0000","last_undegraded":"2023-10-06T03:06:51.163799+0000","last_fullsized":"2023-10-06T03:06:51.163799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185102+0000","last_change":"2023-10-06T03:06:51.177020+0000","last_active":"2023-10-06T03:06:51.185102+0000","last_peered":"2023-10-06T03:06:51.185102+0000","last_clean":"2023-10-06T03:06:51.185102+0000","last_became_active":"2023-10-06T03:06:51.176565+0000","last_became_peered":"2023-10-06T03:06:51.176565+0000","last_unstale":"2023-10-06T03:06:51.185102+0000","last_undegraded":"2023-10-06T03:06:51.185102+0000","last_fullsized":"2023-10-06T03:06:51.185102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:18.011336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177824+0000","last_change":"2023-10-06T03:06:50.162816+0000","last_active":"2023-10-06T03:06:51.177824+0000","last_peered":"2023-10-06T03:06:51.177824+0000","last_clean":"2023-10-06T03:06:51.177824+0000","last_became_active":"2023-10-06T03:06:50.162522+0000","last_became_peered":"2023-10-06T03:06:50.162522+0000","last_unstale":"2023-10-06T03:06:51.177824+0000","last_undegraded":"2023-10-06T03:06:51.177824+0000","last_fullsized":"2023-10-06T03:06:51.177824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:59.716654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162847+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:06:51.162847+0000","last_peered":"2023-10-06T03:06:51.162847+0000","last_clean":"2023-10-06T03:06:51.162847+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:06:51.162847+0000","last_undegraded":"2023-10-06T03:06:51.162847+0000","last_fullsized":"2023-10-06T03:06:51.162847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.177958+0000","last_change":"2023-10-06T03:06:51.174612+0000","last_active":"2023-10-06T03:06:51.177958+0000","last_peered":"2023-10-06T03:06:51.177958+0000","last_clean":"2023-10-06T03:06:51.177958+0000","last_became_active":"2023-10-06T03:06:51.174306+0000","last_became_peered":"2023-10-06T03:06:51.174306+0000","last_unstale":"2023-10-06T03:06:51.177958+0000","last_undegraded":"2023-10-06T03:06:51.177958+0000","last_fullsized":"2023-10-06T03:06:51.177958+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:14.192650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.153927+0000","last_change":"2023-10-06T03:06:50.165138+0000","last_active":"2023-10-06T03:06:51.153927+0000","last_peered":"2023-10-06T03:06:51.153927+0000","last_clean":"2023-10-06T03:06:51.153927+0000","last_became_active":"2023-10-06T03:06:50.164896+0000","last_became_peered":"2023-10-06T03:06:50.164896+0000","last_unstale":"2023-10-06T03:06:51.153927+0000","last_undegraded":"2023-10-06T03:06:51.153927+0000","last_fullsized":"2023-10-06T03:06:51.153927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:55:22.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":213,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162933+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:06:51.162933+0000","last_peered":"2023-10-06T03:06:51.162933+0000","last_clean":"2023-10-06T03:06:51.162933+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:06:51.162933+0000","last_undegraded":"2023-10-06T03:06:51.162933+0000","last_fullsized":"2023-10-06T03:06:51.162933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.156928+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:06:51.156928+0000","last_peered":"2023-10-06T03:06:51.156928+0000","last_clean":"2023-10-06T03:06:51.156928+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:06:51.156928+0000","last_undegraded":"2023-10-06T03:06:51.156928+0000","last_fullsized":"2023-10-06T03:06:51.156928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173221+0000","last_change":"2023-10-06T03:06:51.171506+0000","last_active":"2023-10-06T03:06:51.173221+0000","last_peered":"2023-10-06T03:06:51.173221+0000","last_clean":"2023-10-06T03:06:51.173221+0000","last_became_active":"2023-10-06T03:06:51.167362+0000","last_became_peered":"2023-10-06T03:06:51.167362+0000","last_unstale":"2023-10-06T03:06:51.173221+0000","last_undegraded":"2023-10-06T03:06:51.173221+0000","last_fullsized":"2023-10-06T03:06:51.173221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:11.189263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.174592+0000","last_change":"2023-10-06T03:06:50.165103+0000","last_active":"2023-10-06T03:06:51.174592+0000","last_peered":"2023-10-06T03:06:51.174592+0000","last_clean":"2023-10-06T03:06:51.174592+0000","last_became_active":"2023-10-06T03:06:50.164840+0000","last_became_peered":"2023-10-06T03:06:50.164840+0000","last_unstale":"2023-10-06T03:06:51.174592+0000","last_undegraded":"2023-10-06T03:06:51.174592+0000","last_fullsized":"2023-10-06T03:06:51.174592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:38.875838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.156341+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:06:51.156341+0000","last_peered":"2023-10-06T03:06:51.156341+0000","last_clean":"2023-10-06T03:06:51.156341+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:06:51.156341+0000","last_undegraded":"2023-10-06T03:06:51.156341+0000","last_fullsized":"2023-10-06T03:06:51.156341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.167558+0000","last_change":"2023-10-06T03:06:51.163281+0000","last_active":"2023-10-06T03:06:51.167558+0000","last_peered":"2023-10-06T03:06:51.167558+0000","last_clean":"2023-10-06T03:06:51.167558+0000","last_became_active":"2023-10-06T03:06:51.162981+0000","last_became_peered":"2023-10-06T03:06:51.162981+0000","last_unstale":"2023-10-06T03:06:51.167558+0000","last_undegraded":"2023-10-06T03:06:51.167558+0000","last_fullsized":"2023-10-06T03:06:51.167558+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:39.686927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.176468+0000","last_change":"2023-10-06T03:06:50.166643+0000","last_active":"2023-10-06T03:06:51.176468+0000","last_peered":"2023-10-06T03:06:51.176468+0000","last_clean":"2023-10-06T03:06:51.176468+0000","last_became_active":"2023-10-06T03:06:50.166332+0000","last_became_peered":"2023-10-06T03:06:50.166332+0000","last_unstale":"2023-10-06T03:06:51.176468+0000","last_undegraded":"2023-10-06T03:06:51.176468+0000","last_fullsized":"2023-10-06T03:06:51.176468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:31.532307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.162009+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:06:51.162009+0000","last_peered":"2023-10-06T03:06:51.162009+0000","last_clean":"2023-10-06T03:06:51.162009+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:06:51.162009+0000","last_undegraded":"2023-10-06T03:06:51.162009+0000","last_fullsized":"2023-10-06T03:06:51.162009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.185092+0000","last_change":"2023-10-06T03:06:51.178285+0000","last_active":"2023-10-06T03:06:51.185092+0000","last_peered":"2023-10-06T03:06:51.185092+0000","last_clean":"2023-10-06T03:06:51.185092+0000","last_became_active":"2023-10-06T03:06:51.178027+0000","last_became_peered":"2023-10-06T03:06:51.178027+0000","last_unstale":"2023-10-06T03:06:51.185092+0000","last_undegraded":"2023-10-06T03:06:51.185092+0000","last_fullsized":"2023-10-06T03:06:51.185092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:50.143044+0000","last_clean_scrub_stamp":"2023-10-06T03:06:50.143044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:25.267719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"77'2","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.173925+0000","last_change":"2023-10-06T03:06:50.161527+0000","last_active":"2023-10-06T03:06:51.173925+0000","last_peered":"2023-10-06T03:06:51.173925+0000","last_clean":"2023-10-06T03:06:51.173925+0000","last_became_active":"2023-10-06T03:06:50.161239+0000","last_became_peered":"2023-10-06T03:06:50.161239+0000","last_unstale":"2023-10-06T03:06:51.173925+0000","last_undegraded":"2023-10-06T03:06:51.173925+0000","last_fullsized":"2023-10-06T03:06:51.173925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:06:49.138384+0000","last_clean_scrub_stamp":"2023-10-06T03:06:49.138384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:31.532307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-06T03:06:51.154021+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:06:51.154021+0000","last_peered":"2023-10-06T03:06:51.154021+0000","last_clean":"2023-10-06T03:06:51.154021+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:06:51.154021+0000","last_undegraded":"2023-10-06T03:06:51.154021+0000","last_fullsized":"2023-10-06T03:06:51.154021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574918,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302908,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068932,"statfs":{"total":96636764160,"available":96326586368,"internally_reserved":0,"allocated":1560576,"data_stored":1024090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55500000000000005,"15min":0.72799999999999998},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76200000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.38,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.501,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71599999999999997,"5min":0.875,"15min":0.875},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81299999999999994,"5min":0.999,"15min":0.999},"last":0.749},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":1.026,"15min":1.026},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.439},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36699999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298244,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073596,"statfs":{"total":96636764160,"available":96331362304,"internally_reserved":0,"allocated":978944,"data_stored":435778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.747}]},{"osd":3,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.62},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.86499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.55100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.81699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":427095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.372},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55900000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.317}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32300000000000001,"15min":0.33100000000000002},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48599999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.50800000000000001,"5min":0.628,"15min":0.628},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":1.115,"15min":1.115},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84699999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82699999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574916,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298828,"kb_used_data":1540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073012,"statfs":{"total":96636764160,"available":96330764288,"internally_reserved":0,"allocated":1576960,"data_stored":1026248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.307,"15min":0.31},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.31900000000000001,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40300000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.26600000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.433,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.433}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.48499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.40899999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63500000000000001,"5min":0.871,"15min":0.871},"last":0.629},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68000000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.754,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71099999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76300000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574916,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":433666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.32200000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.498,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.315},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.38}]},{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.31},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34599999999999997,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.504,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.501}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.42599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.87,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":1.903,"15min":1.903},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":433666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.30499999999999999},"min":{"1min":0.215,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.503,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.253},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.25600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.49299999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.314,"5min":0.33300000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33400000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.625,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77000000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.997,"15min":0.997},"last":0.39800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73799999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574916,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302328,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069512,"statfs":{"total":96636764160,"available":96327180288,"internally_reserved":0,"allocated":966656,"data_stored":433826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.629}]},{"osd":1,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52400000000000002,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64300000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54800000000000004,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70799999999999996,"5min":3,"15min":3},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.60999999999999999,"15min":0.81799999999999995},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69199999999999995,"5min":10.084,"15min":10.084},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59199999999999997,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77600000000000002,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.58999999999999997},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":1.54,"15min":1.54},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.40999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.59199999999999997},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76800000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.61599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.69099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574918,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298224,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073616,"statfs":{"total":96636764160,"available":96331382784,"internally_reserved":0,"allocated":958464,"data_stored":433666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":2.371,"15min":2.371},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.505,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79200000000000004,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78200000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67700000000000005,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.72599999999999998},"min":{"1min":0.373,"5min":0.315,"15min":0.315},"max":{"1min":0.72099999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68400000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72999999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.63400000000000001},"min":{"1min":0.45400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.746,"5min":1.052,"15min":1.052},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.63700000000000001},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":1.175,"15min":1.175},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":1.145,"15min":1.145},"last":0.53600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:06:55.616 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:06:55.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:06:55.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.771+0000 7ffa95364700 1 Processor -- start 2023-10-06T03:06:55.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.771+0000 7ffa95364700 1 -- start start 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa95364700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffa90069c70 0x7ffa900cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa95364700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa900af320 0x7ffa900cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa95364700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa900ce380 con 0x7ffa90159950 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa95364700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa900cec20 con 0x7ffa90069c70 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa95364700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa900afa90 con 0x7ffa900af320 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ffa90159950 0x7ffa90069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56584/0 (socket says 172.21.15.92:56584) 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8effd700 1 -- 172.21.15.92:0/4238363999 learned_addr learned my addr 172.21.15.92:0/4238363999 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8e7fc700 1 --2- 172.21.15.92:0/4238363999 >> v2:172.21.15.92:3300/0 conn(0x7ffa90069c70 0x7ffa900cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8f7fe700 1 --2- 172.21.15.92:0/4238363999 >> v2:172.21.15.136:3300/0 conn(0x7ffa900af320 0x7ffa900cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8dffb700 1 -- 172.21.15.92:0/4238363999 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529188447 0 0) 0x7ffa900ce380 con 0x7ffa90159950 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8dffb700 1 -- 172.21.15.92:0/4238363999 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa7c003570 con 0x7ffa90159950 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8e7fc700 1 -- 172.21.15.92:0/4238363999 >> v2:172.21.15.136:3300/0 conn(0x7ffa900af320 msgr2=0x7ffa900cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8e7fc700 1 --2- 172.21.15.92:0/4238363999 >> v2:172.21.15.136:3300/0 conn(0x7ffa900af320 0x7ffa900cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8e7fc700 1 -- 172.21.15.92:0/4238363999 >> v1:172.21.15.92:6789/0 conn(0x7ffa90159950 legacy=0x7ffa90069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8e7fc700 1 -- 172.21.15.92:0/4238363999 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa900ce5d0 con 0x7ffa90069c70 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8e7fc700 1 --2- 172.21.15.92:0/4238363999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa900cbb90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa900cb5c0 tx=0x7ffa780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10e08f55cdb8c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8dffb700 1 -- 172.21.15.92:0/4238363999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78011630 con 0x7ffa90069c70 2023-10-06T03:06:55.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8dffb700 1 -- 172.21.15.92:0/4238363999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa780117b0 con 0x7ffa90069c70 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.772+0000 7ffa8dffb700 1 -- 172.21.15.92:0/4238363999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78011b30 con 0x7ffa90069c70 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 -- 172.21.15.92:0/4238363999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 msgr2=0x7ffa900cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 --2- 172.21.15.92:0/4238363999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa900cbb90 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ffa900cb5c0 tx=0x7ffa780091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 -- 172.21.15.92:0/4238363999 shutdown_connections 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 --2- 172.21.15.92:0/4238363999 >> v2:172.21.15.136:3300/0 conn(0x7ffa900af320 0x7ffa900cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 --2- 172.21.15.92:0/4238363999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa900cbb90 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 -- 172.21.15.92:0/4238363999 >> 172.21.15.92:0/4238363999 conn(0x7ffa90068e00 msgr2=0x7ffa900d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 -- 172.21.15.92:0/4238363999 shutdown_connections 2023-10-06T03:06:55.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.773+0000 7ffa95364700 1 -- 172.21.15.92:0/4238363999 wait complete. 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.774+0000 7ffa95364700 1 Processor -- start 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.774+0000 7ffa95364700 1 -- start start 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa95364700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa9016a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa95364700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa900af320 0x7ffa900c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa95364700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa90159950 0x7ffa9015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa95364700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900c8ec0 con 0x7ffa90069c70 2023-10-06T03:06:55.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa95364700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900c91a0 con 0x7ffa90159950 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa95364700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa900c9480 con 0x7ffa900af320 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa9016a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa9016a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43058/0 (socket says 172.21.15.92:43058) 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 -- 172.21.15.92:0/2146156309 learned_addr learned my addr 172.21.15.92:0/2146156309 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 -- 172.21.15.92:0/2146156309 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa900af320 msgr2=0x7ffa900c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa900af320 0x7ffa900c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 -- 172.21.15.92:0/2146156309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa90159950 msgr2=0x7ffa9015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa90159950 0x7ffa9015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 -- 172.21.15.92:0/2146156309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78009420 con 0x7ffa90069c70 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa8effd700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa9016a540 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800cc40 tx=0x7ffa88001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.775+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8800a980 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.776+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8800aae0 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.776+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa900c9760 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.776+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8800ae80 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.776+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa900c8bb0 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.777+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa900c9c60 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.777+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ffa88007300 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.777+0000 7ffa86ffd700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa70067170 0x7ffa70069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.777+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa88084de0 con 0x7ffa90069c70 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.777+0000 7ffa8e7fc700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa70067170 0x7ffa70069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:55.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.778+0000 7ffa8e7fc700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa70067170 0x7ffa70069620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffa900cb5c0 tx=0x7ffa78017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:55.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:55.782+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa88051ce0 con 0x7ffa90069c70 2023-10-06T03:06:56.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.041+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffa900718b0 con 0x7ffa90069c70 2023-10-06T03:06:56.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.045+0000 7ffa86ffd700 1 -- 172.21.15.92:0/2146156309 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7ffa88055300 con 0x7ffa90069c70 2023-10-06T03:06:56.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa70067170 msgr2=0x7ffa70069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa70067170 0x7ffa70069620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffa900cb5c0 tx=0x7ffa78017000 comp rx=0 tx=0).stop 2023-10-06T03:06:56.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 msgr2=0x7ffa9016a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa9016a540 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800cc40 tx=0x7ffa88001a60 comp rx=0 tx=0).stop 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 shutdown_connections 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa70067170 0x7ffa70069620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa90069c70 0x7ffa9016a540 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa90159950 0x7ffa9015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 --2- 172.21.15.92:0/2146156309 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa900af320 0x7ffa900c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 >> 172.21.15.92:0/2146156309 conn(0x7ffa90068e00 msgr2=0x7ffa900d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 shutdown_connections 2023-10-06T03:06:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.047+0000 7ffa95364700 1 -- 172.21.15.92:0/2146156309 wait complete. 2023-10-06T03:06:56.050 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:06:56.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:06:56.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.175+0000 7f93a9519700 1 Processor -- start 2023-10-06T03:06:56.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.175+0000 7f93a9519700 1 -- start start 2023-10-06T03:06:56.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a9519700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a4159c60 0x7f93a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a9519700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93a415db10 0x7f93a4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a9519700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a415a160 con 0x7f93a4159890 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a9519700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a40d13a0 con 0x7f93a415db10 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a9519700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a415e140 con 0x7f93a4159c60 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93a415db10 0x7f93a4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93a415db10 0x7f93a4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43062/0 (socket says 172.21.15.92:43062) 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 -- 172.21.15.92:0/1874046492 learned_addr learned my addr 172.21.15.92:0/1874046492 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a27fc700 1 --2- 172.21.15.92:0/1874046492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a4159c60 0x7f93a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 -- 172.21.15.92:0/1874046492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a4159c60 msgr2=0x7f93a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 --2- 172.21.15.92:0/1874046492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a4159c60 0x7f93a415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 -- 172.21.15.92:0/1874046492 >> v1:172.21.15.136:6789/0 conn(0x7f93a4159890 legacy=0x7f93a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 -- 172.21.15.92:0/1874046492 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a415a3d0 con 0x7f93a415db10 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a37fe700 1 --2- 172.21.15.92:0/1874046492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a415db10 0x7f93a4069560 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f93940099b0 tx=0x7f9394009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83fd9ba0ee27ba9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a1ffb700 1 -- 172.21.15.92:0/1874046492 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400b860 con 0x7f93a415db10 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a1ffb700 1 -- 172.21.15.92:0/1874046492 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400b9c0 con 0x7f93a415db10 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.176+0000 7f93a1ffb700 1 -- 172.21.15.92:0/1874046492 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939400bd60 con 0x7f93a415db10 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 -- 172.21.15.92:0/1874046492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a415db10 msgr2=0x7f93a4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 --2- 172.21.15.92:0/1874046492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a415db10 0x7f93a4069560 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f93940099b0 tx=0x7f9394009250 comp rx=0 tx=0).stop 2023-10-06T03:06:56.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 -- 172.21.15.92:0/1874046492 shutdown_connections 2023-10-06T03:06:56.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 --2- 172.21.15.92:0/1874046492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a415db10 0x7f93a4069560 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 --2- 172.21.15.92:0/1874046492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a4159c60 0x7f93a415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 -- 172.21.15.92:0/1874046492 >> 172.21.15.92:0/1874046492 conn(0x7f93a4068e20 msgr2=0x7f93a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 -- 172.21.15.92:0/1874046492 shutdown_connections 2023-10-06T03:06:56.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.177+0000 7f93a9519700 1 -- 172.21.15.92:0/1874046492 wait complete. 2023-10-06T03:06:56.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.178+0000 7f93a9519700 1 Processor -- start 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.178+0000 7f93a9519700 1 -- start start 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a9519700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 0x7f93a4152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a9519700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93a4159c60 0x7f93a40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a9519700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a415db10 0x7f93a416a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a9519700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4152a40 con 0x7f93a4159890 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a9519700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40cd810 con 0x7f93a4159c60 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a9519700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a40ce650 con 0x7f93a415db10 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 0x7f93a4152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 0x7f93a4152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43064/0 (socket says 172.21.15.92:43064) 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 -- 172.21.15.92:0/1204400107 learned_addr learned my addr 172.21.15.92:0/1204400107 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a37fe700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a415db10 0x7f93a416a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a27fc700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93a4159c60 0x7f93a40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 -- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a415db10 msgr2=0x7f93a416a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a415db10 0x7f93a416a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 -- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93a4159c60 msgr2=0x7f93a40cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93a4159c60 0x7f93a40cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.179+0000 7f93a2ffd700 1 -- 172.21.15.92:0/1204400107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940094f0 con 0x7f93a4159890 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.180+0000 7f93a2ffd700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 0x7f93a4152500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f939c00c5c0 tx=0x7f939c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.180+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c011590 con 0x7f93a4159890 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.180+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c011710 con 0x7f93a4159890 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.180+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c011ad0 con 0x7f93a4159890 2023-10-06T03:06:56.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.180+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a40cda40 con 0x7f93a4159890 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.180+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a40ce000 con 0x7f93a4159890 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.181+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a404c3c0 con 0x7f93a4159890 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.181+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f939c01b3b0 con 0x7f93a4159890 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.181+0000 7f938affd700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9390067170 0x7f9390069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.181+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f939c083fa0 con 0x7f93a4159890 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.181+0000 7f93a27fc700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9390067170 0x7f9390069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.182+0000 7f93a27fc700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9390067170 0x7f9390069620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93a414d300 tx=0x7f939800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.185+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f939c050e20 con 0x7f93a4159890 2023-10-06T03:06:56.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.414+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93a40b2880 con 0x7f93a4159890 2023-10-06T03:06:56.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.415+0000 7f938affd700 1 -- 172.21.15.92:0/1204400107 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f939c017020 con 0x7f93a4159890 2023-10-06T03:06:56.416 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:06:56.416 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:06:50.531995+0000","modified":"2023-10-06T03:06:54.171754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/2746794570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2746794570},{"type":"v1","addr":"172.21.15.136:6835","nonce":2746794570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9390067170 msgr2=0x7f9390069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9390067170 0x7f9390069620 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f93a414d300 tx=0x7f939800b040 comp rx=0 tx=0).stop 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 msgr2=0x7f93a4152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 0x7f93a4152500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f939c00c5c0 tx=0x7f939c00c980 comp rx=0 tx=0).stop 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 shutdown_connections 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9390067170 0x7f9390069620 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93a4159890 0x7f93a4152500 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93a415db10 0x7f93a416a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 --2- 172.21.15.92:0/1204400107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93a4159c60 0x7f93a40cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 >> 172.21.15.92:0/1204400107 conn(0x7f93a4068e20 msgr2=0x7f93a4158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 shutdown_connections 2023-10-06T03:06:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.417+0000 7f93a9519700 1 -- 172.21.15.92:0/1204400107 wait complete. 2023-10-06T03:06:56.419 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 47 2023-10-06T03:06:56.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:06:50.531995+0000', 'modified': '2023-10-06T03:06:54.171754+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6147}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6147': {'gid': 6147, 'name': 'd', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.136:6835/2746794570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6833', 'nonce': 2746794570}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 2746794570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:06:56.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:06:56.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T03:06:56.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.949+0000 7fc20359e700 1 Processor -- start 2023-10-06T03:06:56.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.949+0000 7fc20359e700 1 -- start start 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc20359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc20415d230 0x7fc20415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc20359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2040b2980 con 0x7fc2041597f0 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc20359e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20415db70 con 0x7fc204159420 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc20359e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2040b11d0 con 0x7fc20415d230 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc202d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc20415d230 0x7fc20415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc1f9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc2041597f0 0x7fc2040b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56588/0 (socket says 172.21.15.92:56588) 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc1f9d9b700 1 -- 172.21.15.92:0/1972605715 learned_addr learned my addr 172.21.15.92:0/1972605715 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc202d9d700 1 -- 172.21.15.92:0/1972605715 >> v1:172.21.15.136:6789/0 conn(0x7fc204159420 legacy=0x7fc2040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc202d9d700 1 -- 172.21.15.92:0/1972605715 >> v1:172.21.15.92:6789/0 conn(0x7fc2041597f0 legacy=0x7fc2040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc202d9d700 1 -- 172.21.15.92:0/1972605715 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040b14b0 con 0x7fc20415d230 2023-10-06T03:06:56.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc202d9d700 1 --2- 172.21.15.92:0/1972605715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc20415d230 0x7fc20415d630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009e60 tx=0x7fc1fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b894d9718670d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.950+0000 7fc201d9b700 1 -- 172.21.15.92:0/1972605715 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00e070 con 0x7fc20415d230 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc201d9b700 1 -- 172.21.15.92:0/1972605715 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc00bc30 con 0x7fc20415d230 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc201d9b700 1 -- 172.21.15.92:0/1972605715 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc012430 con 0x7fc20415d230 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 -- 172.21.15.92:0/1972605715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc20415d230 msgr2=0x7fc20415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 --2- 172.21.15.92:0/1972605715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc20415d230 0x7fc20415d630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009e60 tx=0x7fc1fc009780 comp rx=0 tx=0).stop 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 -- 172.21.15.92:0/1972605715 shutdown_connections 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 --2- 172.21.15.92:0/1972605715 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc20415d230 0x7fc20415d630 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 -- 172.21.15.92:0/1972605715 >> 172.21.15.92:0/1972605715 conn(0x7fc2040d0660 msgr2=0x7fc2040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 -- 172.21.15.92:0/1972605715 shutdown_connections 2023-10-06T03:06:56.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.951+0000 7fc20359e700 1 -- 172.21.15.92:0/1972605715 wait complete. 2023-10-06T03:06:56.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.952+0000 7fc20359e700 1 Processor -- start 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.952+0000 7fc20359e700 1 -- start start 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 0x7fc204154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2041597f0 0x7fc2040cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20359e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc20415d230 0x7fc2040cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040cc340 con 0x7fc204159420 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20359e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040cbaa0 con 0x7fc20415d230 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2040cbd80 con 0x7fc2041597f0 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 0x7fc204154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 0x7fc204154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43088/0 (socket says 172.21.15.92:43088) 2023-10-06T03:06:56.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 -- 172.21.15.92:0/575234384 learned_addr learned my addr 172.21.15.92:0/575234384 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc202d9d700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc20415d230 0x7fc2040cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2041597f0 0x7fc2040cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 -- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2041597f0 msgr2=0x7fc2040cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2041597f0 0x7fc2040cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 -- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc20415d230 msgr2=0x7fc2040cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc20415d230 0x7fc2040cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.953+0000 7fc20259c700 1 -- 172.21.15.92:0/575234384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc009a20 con 0x7fc204159420 2023-10-06T03:06:56.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.954+0000 7fc20259c700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 0x7fc204154ea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c790 tx=0x7fc1f4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.954+0000 7fc1faffd700 1 -- 172.21.15.92:0/575234384 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4011590 con 0x7fc204159420 2023-10-06T03:06:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.954+0000 7fc1faffd700 1 -- 172.21.15.92:0/575234384 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4011710 con 0x7fc204159420 2023-10-06T03:06:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.954+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2040cc060 con 0x7fc204159420 2023-10-06T03:06:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.954+0000 7fc1faffd700 1 -- 172.21.15.92:0/575234384 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4011ad0 con 0x7fc204159420 2023-10-06T03:06:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.954+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2040cb7c0 con 0x7fc204159420 2023-10-06T03:06:56.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.955+0000 7fc1faffd700 1 -- 172.21.15.92:0/575234384 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc1f401b3b0 con 0x7fc204159420 2023-10-06T03:06:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.955+0000 7fc1faffd700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e4067170 0x7fc1e4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.956+0000 7fc1faffd700 1 -- 172.21.15.92:0/575234384 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1f4083d80 con 0x7fc204159420 2023-10-06T03:06:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.956+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e4067170 0x7fc1e4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.956+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e4067170 0x7fc1e4069620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc2040b22e0 tx=0x7fc1ec007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 Processor -- start 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 -- start start 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac302c0 0x560e6ac2b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac2b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ab85a70 con 0x560e6ac4fe50 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ab83270 con 0x560e6ac302c0 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e6ab7f060 con 0x560e6ac31a10 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac2acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43100/0 (socket says 172.21.15.92:43100) 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.958+0000 7fc20259c700 1 -- 172.21.15.92:0/362121538 learned_addr learned my addr 172.21.15.92:0/362121538 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac302c0 0x560e6ac2b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc202d9d700 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac2b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc20259c700 1 -- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 msgr2=0x560e6ac2b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc20259c700 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac2b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc20259c700 1 -- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac302c0 msgr2=0x560e6ac2b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc20259c700 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac302c0 0x560e6ac2b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc20259c700 1 -- 172.21.15.92:0/362121538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e6aa19270 con 0x560e6ac4fe50 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc20259c700 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac2acc0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40545c0 tx=0x7fc1f4050c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b0b95ab3ef7d35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc1e37fe700 1 -- 172.21.15.92:0/362121538 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40548c0 con 0x560e6ac4fe50 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc1e37fe700 1 -- 172.21.15.92:0/362121538 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4057b10 con 0x560e6ac4fe50 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.959+0000 7fc1e37fe700 1 -- 172.21.15.92:0/362121538 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f405b860 con 0x560e6ac4fe50 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 -- 172.21.15.92:0/362121538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 msgr2=0x560e6ac2acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac2acc0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40545c0 tx=0x7fc1f4050c90 comp rx=0 tx=0).stop 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 -- 172.21.15.92:0/362121538 shutdown_connections 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac2acc0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac2b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/362121538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac302c0 0x560e6ac2b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 -- 172.21.15.92:0/362121538 >> 172.21.15.92:0/362121538 conn(0x560e6ac2fe80 msgr2=0x560e6ac306b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 -- 172.21.15.92:0/362121538 shutdown_connections 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 -- 172.21.15.92:0/362121538 wait complete. 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 -- start start 2023-10-06T03:06:56.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.960+0000 7fc211cdbb80 1 Processor -- start 2023-10-06T03:06:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac4dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac84de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac79330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ab2a890 con 0x560e6ac302c0 2023-10-06T03:06:56.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ab45ea0 con 0x560e6ac4fe50 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e6aa01e30 con 0x560e6ac31a10 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc211cdbb80 10 client.0 _ll_register_callbacks cb 0x560e6abf2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac4dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac4dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43114/0 (socket says 172.21.15.92:43114) 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc20259c700 1 -- 172.21.15.92:0/2882291112 learned_addr learned my addr 172.21.15.92:0/2882291112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac84de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.961+0000 7fc202d9d700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac79330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc20259c700 1 -- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 msgr2=0x560e6ac84de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc20259c700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac84de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc20259c700 1 -- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 msgr2=0x560e6ac79330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc20259c700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac79330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc20259c700 1 -- 172.21.15.92:0/2882291112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2040cb7c0 con 0x560e6ac302c0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc20259c700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac4dab0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40545c0 tx=0x7fc1f4003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cecd8c5cf6418e80 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc1e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/2882291112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40548c0 con 0x560e6ac302c0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/2882291112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1f4052c60 con 0x560e6ac302c0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e6ac23940 con 0x560e6ac302c0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/2882291112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f4011c30 con 0x560e6ac302c0 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 10 client.6273 fetch_fsmap learned FSMap version 47 2023-10-06T03:06:56.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x560e6a9bbfe0 con 0x560e6ac302c0 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/2882291112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560e6ac23940 con 0x560e6ac302c0 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/2882291112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 47) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc1f4070250 con 0x560e6ac302c0 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 10 client.6273 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 10 client.6273 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 20 client.6273 populate_metadata read hostname 'smithi092' 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac5d2c0 0x560e6ac5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 4 client.6273 mds_command: new command op to 6147 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.962+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 --> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e6ac00870 con 0x560e6ac5d2c0 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.963+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac5d2c0 0x560e6ac5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.963+0000 7fc1e0ff9700 10 client.6273 ms_handle_connect on v2:172.21.15.136:6833/2746794570 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.963+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac5d2c0 0x560e6ac5d690 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c491a156d7ab409f server_cookie=ceace35fb656fc97 in_seq=0 out_seq=0 2023-10-06T03:06:56.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/2882291112 <== mds.0 v2:172.21.15.136:6833/2746794570 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560e6ac00870 con 0x560e6ac5d2c0 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc1e0ff9700 10 client.6273 handle_command_reply: tid=0 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 client.6273 shutdown 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 msgr2=0x560e6ac4dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac4dab0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40545c0 tx=0x7fc1f4003000 comp rx=0 tx=0).stop 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 shutdown_connections 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac5d2c0 0x560e6ac5d690 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac4dab0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc1e0ff9700 0 client.6273 ms_handle_reset on v2:172.21.15.136:6833/2746794570 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac79330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2882291112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac84de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 >> 172.21.15.92:0/2882291112 conn(0x560e6ac2fe80 msgr2=0x560e6ac30d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 shutdown_connections 2023-10-06T03:06:56.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2882291112 wait complete. 2023-10-06T03:06:56.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.964+0000 7fc211cdbb80 20 client.6273 trim_cache size 0 max 16384 2023-10-06T03:06:56.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 Processor -- start 2023-10-06T03:06:56.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 -- start start 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac75a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac2e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ac32370 con 0x560e6ac302c0 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ac2bd00 con 0x560e6ac4fe50 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.981+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e6ac23700 con 0x560e6ac31a10 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43120/0 (socket says 172.21.15.92:43120) 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 -- 172.21.15.92:0/2165867768 learned_addr learned my addr 172.21.15.92:0/2165867768 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac75a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc202d9d700 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac2e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 -- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 msgr2=0x560e6ac75a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac75a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 -- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 msgr2=0x560e6ac2e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac2e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 -- 172.21.15.92:0/2165867768 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e6ab4d780 con 0x560e6ac302c0 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac75a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc20259c700 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac571c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4054240 tx=0x7fc1f407e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b52a61030e2ea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc1e3fff700 1 -- 172.21.15.92:0/2165867768 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4054690 con 0x560e6ac302c0 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc1e3fff700 1 -- 172.21.15.92:0/2165867768 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4052c60 con 0x560e6ac302c0 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc1e3fff700 1 -- 172.21.15.92:0/2165867768 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f405b860 con 0x560e6ac302c0 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2165867768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 msgr2=0x560e6ac571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.982+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac571c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4054240 tx=0x7fc1f407e070 comp rx=0 tx=0).stop 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2165867768 shutdown_connections 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac302c0 0x560e6ac571c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac4fe50 0x560e6ac2e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/2165867768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac31a10 0x560e6ac75a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2165867768 >> 172.21.15.92:0/2165867768 conn(0x560e6ac2fe80 msgr2=0x560e6ac51640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2165867768 shutdown_connections 2023-10-06T03:06:56.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 -- 172.21.15.92:0/2165867768 wait complete. 2023-10-06T03:06:56.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 -- start start 2023-10-06T03:06:56.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.983+0000 7fc211cdbb80 1 Processor -- start 2023-10-06T03:06:56.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac302c0 0x560e6ac5df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac31a10 0x560e6ac84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac84f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6acc3d20 con 0x560e6ac4fe50 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6ac5e440 con 0x560e6ac31a10 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e6ac553b0 con 0x560e6ac302c0 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc211cdbb80 10 client.0 _ll_register_callbacks cb 0x560e6abf2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac84f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac84f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43124/0 (socket says 172.21.15.92:43124) 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 -- 172.21.15.92:0/772705280 learned_addr learned my addr 172.21.15.92:0/772705280 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac31a10 0x560e6ac84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc20259c700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac302c0 0x560e6ac5df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 -- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac302c0 msgr2=0x560e6ac5df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac302c0 0x560e6ac5df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 -- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac31a10 msgr2=0x560e6ac84a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac31a10 0x560e6ac84a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 -- 172.21.15.92:0/772705280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e6a9bbfe0 con 0x560e6ac4fe50 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc1f9d9b700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac31a10 0x560e6ac84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc202d9d700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac84f50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc204089340 tx=0x7fc1fc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509aa3dbee8e4adf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.984+0000 7fc1e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/772705280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc023070 con 0x560e6ac4fe50 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/772705280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1fc01e9e0 con 0x560e6ac4fe50 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1f4057b10 con 0x560e6ac4fe50 2023-10-06T03:06:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/772705280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1fc01edb0 con 0x560e6ac4fe50 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 10 client.6279 fetch_fsmap learned FSMap version 47 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x560e6ab34470 con 0x560e6ac4fe50 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/772705280 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4057b10 con 0x560e6ac4fe50 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/772705280 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 47) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc1fc029460 con 0x560e6ac4fe50 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 10 client.6279 fetch_fsmap finished waiting for FSMap version 47 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 10 client.6279 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 20 client.6279 populate_metadata read hostname 'smithi092' 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac8ecb0 0x560e6ac4dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 4 client.6279 mds_command: new command op to 6147 tid=0[{"prefix": "status"}] 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 --> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560e6ab21e00 con 0x560e6ac8ecb0 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.985+0000 7fc20259c700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac8ecb0 0x560e6ac4dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc1e0ff9700 10 client.6279 ms_handle_connect on v2:172.21.15.136:6833/2746794570 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc20259c700 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac8ecb0 0x560e6ac4dd20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f720bbc42f8fa3ee server_cookie=f0b0efcc1dde7dfe in_seq=0 out_seq=0 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc1e0ff9700 1 -- 172.21.15.92:0/772705280 <== mds.0 v2:172.21.15.136:6833/2746794570 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560e6ab21e00 con 0x560e6ac8ecb0 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc1e0ff9700 10 client.6279 handle_command_reply: tid=0 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 client.6279 shutdown 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 msgr2=0x560e6ac84f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac84f50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc204089340 tx=0x7fc1fc019000 comp rx=0 tx=0).stop 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 shutdown_connections 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:6833/2746794570,v1:172.21.15.136:6835/2746794570] conn(0x560e6ac8ecb0 0x560e6ac4dd20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560e6ac4fe50 0x560e6ac84f50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x560e6ac302c0 0x560e6ac5df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 --2- 172.21.15.92:0/772705280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560e6ac31a10 0x560e6ac84a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc1e0ff9700 0 client.6279 ms_handle_reset on v2:172.21.15.136:6833/2746794570 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 >> 172.21.15.92:0/772705280 conn(0x560e6ac2fe80 msgr2=0x560e6ac30f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 shutdown_connections 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 1 -- 172.21.15.92:0/772705280 wait complete. 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.986+0000 7fc211cdbb80 20 client.6279 trim_cache size 0 max 16384 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:06:56.995 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6147, 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.8372059930000004, 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 47, 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 78, 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 78, 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.930693031000001 2023-10-06T03:06:56.996 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:06:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e4067170 msgr2=0x7fc1e4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e4067170 0x7fc1e4069620 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc2040b22e0 tx=0x7fc1ec007eb0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 msgr2=0x7fc204154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 0x7fc204154ea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400c790 tx=0x7fc1f4007f40 comp rx=0 tx=0).stop 2023-10-06T03:06:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 shutdown_connections 2023-10-06T03:06:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1e4067170 0x7fc1e4069620 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc204159420 0x7fc204154ea0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc20415d230 0x7fc2040cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 --2- 172.21.15.92:0/575234384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2041597f0 0x7fc2040cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 >> 172.21.15.92:0/575234384 conn(0x7fc2040d0660 msgr2=0x7fc204155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 shutdown_connections 2023-10-06T03:06:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:56.987+0000 7fc20359e700 1 -- 172.21.15.92:0/575234384 wait complete. 2023-10-06T03:06:56.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:06:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.112+0000 7fd18f238700 1 Processor -- start 2023-10-06T03:06:57.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.112+0000 7fd18f238700 1 -- start start 2023-10-06T03:06:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.113+0000 7fd18f238700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188069560 0x7fd1880699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:57.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.113+0000 7fd18f238700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd188069ee0 con 0x7fd188159c60 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.113+0000 7fd18f238700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18815a160 con 0x7fd188159890 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.113+0000 7fd18f238700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880d13a0 con 0x7fd188069560 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.113+0000 7fd17ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd188159c60 0x7fd18815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56592/0 (socket says 172.21.15.92:56592) 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.113+0000 7fd17ffff700 1 -- 172.21.15.92:0/1813501280 learned_addr learned my addr 172.21.15.92:0/1813501280 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd18d7d5700 1 --2- 172.21.15.92:0/1813501280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188069560 0x7fd1880699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861172382 0 0) 0x7fd188069ee0 con 0x7fd188159c60 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd170003570 con 0x7fd188159c60 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391387964 0 0) 0x7fd18815a160 con 0x7fd188159890 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd188069ee0 con 0x7fd188159890 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343169898 0 0) 0x7fd170003570 con 0x7fd188159c60 2023-10-06T03:06:57.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd18d7d5700 1 -- 172.21.15.92:0/1813501280 >> v1:172.21.15.136:6789/0 conn(0x7fd188159890 legacy=0x7fd18815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd18d7d5700 1 -- 172.21.15.92:0/1813501280 >> v1:172.21.15.92:6789/0 conn(0x7fd188159c60 legacy=0x7fd18815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd18d7d5700 1 -- 172.21.15.92:0/1813501280 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18815e140 con 0x7fd188069560 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd18d7d5700 1 --2- 172.21.15.92:0/1813501280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188069560 0x7fd1880699a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd180009e90 tx=0x7fd180009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d4b3f5a993f09d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000e070 con 0x7fd188069560 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000b870 con 0x7fd188069560 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.114+0000 7fd17f7fe700 1 -- 172.21.15.92:0/1813501280 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178001470 con 0x7fd188069560 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 -- 172.21.15.92:0/1813501280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188069560 msgr2=0x7fd1880699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 --2- 172.21.15.92:0/1813501280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188069560 0x7fd1880699a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd180009e90 tx=0x7fd180009780 comp rx=0 tx=0).stop 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 -- 172.21.15.92:0/1813501280 shutdown_connections 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 --2- 172.21.15.92:0/1813501280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188069560 0x7fd1880699a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 -- 172.21.15.92:0/1813501280 >> 172.21.15.92:0/1813501280 conn(0x7fd188068e20 msgr2=0x7fd1880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:57.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 -- 172.21.15.92:0/1813501280 shutdown_connections 2023-10-06T03:06:57.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.115+0000 7fd18f238700 1 -- 172.21.15.92:0/1813501280 wait complete. 2023-10-06T03:06:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.116+0000 7fd18f238700 1 Processor -- start 2023-10-06T03:06:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.116+0000 7fd18f238700 1 -- start start 2023-10-06T03:06:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18f238700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 0x7fd1880c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18f238700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188159890 0x7fd1880cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:57.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18f238700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd188159c60 0x7fd188157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18f238700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880c88b0 con 0x7fd188069560 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18f238700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18816cfd0 con 0x7fd188159c60 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18f238700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18816d2b0 con 0x7fd188159890 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 0x7fd1880c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 0x7fd1880c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43140/0 (socket says 172.21.15.92:43140) 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 -- 172.21.15.92:0/3567744294 learned_addr learned my addr 172.21.15.92:0/3567744294 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 -- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188159890 msgr2=0x7fd1880cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd17ffff700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188159890 0x7fd1880cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188159890 0x7fd1880cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18d7d5700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd188159c60 0x7fd188157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:57.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 -- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd188159c60 msgr2=0x7fd188157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd188159c60 0x7fd188157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 -- 172.21.15.92:0/3567744294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180009a20 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd18cfd4700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 0x7fd1880c8370 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7c0 tx=0x7fd184007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.117+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184011680 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.118+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184011800 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.118+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18816d590 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.118+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184011bc0 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.118+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18816db50 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.118+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18804c3c0 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.119+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fd184012440 con 0x7fd188069560 2023-10-06T03:06:57.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.119+0000 7fd17cff9700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd174067170 0x7fd174069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.119+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd184084990 con 0x7fd188069560 2023-10-06T03:06:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.119+0000 7fd17ffff700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd174067170 0x7fd174069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:57.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.120+0000 7fd17ffff700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd174067170 0x7fd174069620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd1780015a0 tx=0x7fd178009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:57.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.123+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd184051810 con 0x7fd188069560 2023-10-06T03:06:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.352+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd18806a5b0 con 0x7fd188069560 2023-10-06T03:06:57.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.352+0000 7fd17cff9700 1 -- 172.21.15.92:0/3567744294 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd184084030 con 0x7fd188069560 2023-10-06T03:06:57.353 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:06:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd174067170 msgr2=0x7fd174069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd174067170 0x7fd174069620 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd1780015a0 tx=0x7fd178009040 comp rx=0 tx=0).stop 2023-10-06T03:06:57.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 msgr2=0x7fd1880c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 0x7fd1880c8370 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c7c0 tx=0x7fd184007de0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 shutdown_connections 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd174067170 0x7fd174069620 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd188069560 0x7fd1880c8370 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd188159c60 0x7fd188157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 --2- 172.21.15.92:0/3567744294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd188159890 0x7fd1880cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.354+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 >> 172.21.15.92:0/3567744294 conn(0x7fd188068e20 msgr2=0x7fd1880cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.355+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 shutdown_connections 2023-10-06T03:06:57.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:57.355+0000 7fd18f238700 1 -- 172.21.15.92:0/3567744294 wait complete. 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:06:57.367 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:06:57.368 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:06:57.368 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:06:57.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.368 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:06:57.383 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:06:57.383 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:06:57.384 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42189sec preferred_lft 42189sec 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.385 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:57.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.386 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:06:57.447 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:06:57.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.448 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:06:57.505 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:06:57.505 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:06:57.505 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:06:57.505 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:06:57.505 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:06:57.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.506 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:06:57.506 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:06:57.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:06:57.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:06:57.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:06:57.506 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:06:57.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.605 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:06:57.620 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:06:57.620 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:06:57.621 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:06:57.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:06:57.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:06:57.685 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:06:57.739 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:06:57.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:06:57.739 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:06:57.793 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:06:57.794 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:06:57.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:57.872 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:57.873 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:06:57.891 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:57.891 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:06:57.891 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:06:57.892 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:06:57.892 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:06:57.892 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:06:57.892 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:06:57.892 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:06:57.914 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:57.914 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:06:57.915 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:06:57.916 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:06:57.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:57.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:06:57.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:06:58.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.135+0000 7fda219ca700 1 Processor -- start 2023-10-06T03:06:58.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.135+0000 7fda219ca700 1 -- start start 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda219ca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda1c159890 0x7fda1c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda219ca700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fda1c069560 0x7fda1c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda219ca700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1c069ee0 con 0x7fda1c159c60 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda219ca700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c15a160 con 0x7fda1c159890 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda219ca700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1c0d13a0 con 0x7fda1c069560 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda1affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda1c159890 0x7fda1c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda127fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fda1c159c60 0x7fda1c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56600/0 (socket says 172.21.15.92:56600) 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.136+0000 7fda127fc700 1 -- 172.21.15.92:0/531507018 learned_addr learned my addr 172.21.15.92:0/531507018 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1affd700 1 -- 172.21.15.92:0/531507018 >> v2:172.21.15.136:3300/0 conn(0x7fda1c069560 msgr2=0x7fda1c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:58.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1b7fe700 1 --2- 172.21.15.92:0/531507018 >> v2:172.21.15.136:3300/0 conn(0x7fda1c069560 0x7fda1c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1affd700 1 --2- 172.21.15.92:0/531507018 >> v2:172.21.15.136:3300/0 conn(0x7fda1c069560 0x7fda1c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1affd700 1 -- 172.21.15.92:0/531507018 >> v1:172.21.15.92:6789/0 conn(0x7fda1c159c60 legacy=0x7fda1c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1affd700 1 -- 172.21.15.92:0/531507018 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c15e140 con 0x7fda1c159890 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1a7fc700 1 -- 172.21.15.92:0/531507018 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805555234 0 0) 0x7fda1c069ee0 con 0x7fda1c159c60 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1affd700 1 --2- 172.21.15.92:0/531507018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159890 0x7fda1c15abd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fda1c06a890 tx=0x7fda0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cd2ca9ff2b99dc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1a7fc700 1 -- 172.21.15.92:0/531507018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00b6b0 con 0x7fda1c159890 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1a7fc700 1 -- 172.21.15.92:0/531507018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c00b810 con 0x7fda1c159890 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda1a7fc700 1 -- 172.21.15.92:0/531507018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00bbb0 con 0x7fda1c159890 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda219ca700 1 -- 172.21.15.92:0/531507018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159890 msgr2=0x7fda1c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda219ca700 1 --2- 172.21.15.92:0/531507018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159890 0x7fda1c15abd0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fda1c06a890 tx=0x7fda0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda219ca700 1 -- 172.21.15.92:0/531507018 shutdown_connections 2023-10-06T03:06:58.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda219ca700 1 --2- 172.21.15.92:0/531507018 >> v2:172.21.15.136:3300/0 conn(0x7fda1c069560 0x7fda1c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda219ca700 1 --2- 172.21.15.92:0/531507018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159890 0x7fda1c15abd0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.137+0000 7fda219ca700 1 -- 172.21.15.92:0/531507018 >> 172.21.15.92:0/531507018 conn(0x7fda1c068e20 msgr2=0x7fda1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.138+0000 7fda219ca700 1 -- 172.21.15.92:0/531507018 shutdown_connections 2023-10-06T03:06:58.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.138+0000 7fda219ca700 1 -- 172.21.15.92:0/531507018 wait complete. 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 Processor -- start 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 -- start start 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda1c069560 0x7fda1c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda1c159890 0x7fda1c0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 0x7fda1c0b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c0ce720 con 0x7fda1c159c60 2023-10-06T03:06:58.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c0b60c0 con 0x7fda1c069560 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.139+0000 7fda219ca700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1c0b63a0 con 0x7fda1c159890 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 0x7fda1c0b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 0x7fda1c0b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43156/0 (socket says 172.21.15.92:43156) 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 -- 172.21.15.92:0/2209241218 learned_addr learned my addr 172.21.15.92:0/2209241218 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1affd700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda1c069560 0x7fda1c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda127fc700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda1c159890 0x7fda1c0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 -- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda1c159890 msgr2=0x7fda1c0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda1c159890 0x7fda1c0b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 -- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda1c069560 msgr2=0x7fda1c1528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:58.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda1c069560 0x7fda1c1528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 -- 172.21.15.92:0/2209241218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c009420 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1affd700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda1c069560 0x7fda1c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda1b7fe700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 0x7fda1c0b5a80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fda1400ceb0 tx=0x7fda14007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1400ae40 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14016430 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1c0ce0d0 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140167d0 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.140+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1c0cdcd0 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.141+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1c0ce570 con 0x7fda1c159c60 2023-10-06T03:06:58.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.141+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fda14003020 con 0x7fda1c159c60 2023-10-06T03:06:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.142+0000 7fda137fe700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9fc067170 0x7fd9fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.142+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fda14012070 con 0x7fda1c159c60 2023-10-06T03:06:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.142+0000 7fda1affd700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9fc067170 0x7fd9fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.142+0000 7fda1affd700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9fc067170 0x7fd9fc069620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0093f0 tx=0x7fda0c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:58.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.146+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda140527d0 con 0x7fda1c159c60 2023-10-06T03:06:58.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.376+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fda1c04c3c0 con 0x7fda1c159c60 2023-10-06T03:06:58.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.376+0000 7fda137fe700 1 -- 172.21.15.92:0/2209241218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fda14055df0 con 0x7fda1c159c60 2023-10-06T03:06:58.377 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9fc067170 msgr2=0x7fd9fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9fc067170 0x7fd9fc069620 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0093f0 tx=0x7fda0c017000 comp rx=0 tx=0).stop 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 msgr2=0x7fda1c0b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 0x7fda1c0b5a80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fda1400ceb0 tx=0x7fda14007870 comp rx=0 tx=0).stop 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 shutdown_connections 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd9fc067170 0x7fd9fc069620 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda1c159c60 0x7fda1c0b5a80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda1c069560 0x7fda1c1528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 --2- 172.21.15.92:0/2209241218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda1c159890 0x7fda1c0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:58.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 >> 172.21.15.92:0/2209241218 conn(0x7fda1c068e20 msgr2=0x7fda1c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:58.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 shutdown_connections 2023-10-06T03:06:58.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:58.378+0000 7fda219ca700 1 -- 172.21.15.92:0/2209241218 wait complete. 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:06:58.391 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:06:58.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.392 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:06:58.410 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:06:58.410 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:06:58.411 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42189sec preferred_lft 42189sec 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.412 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:06:58.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.413 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:06:58.478 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:06:58.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.480 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:06:58.539 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:06:58.539 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:06:58.539 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:06:58.539 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:06:58.539 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:06:58.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.540 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:06:58.540 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:06:58.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:06:58.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:06:58.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:06:58.540 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:06:58.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.646 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:06:58.661 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:06:58.662 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:06:58.662 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:06:58.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:06:58.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:06:58.726 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:06:58.781 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:06:58.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:06:58.782 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:06:58.835 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:06:58.836 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:06:58.836 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:58.914 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:58.915 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:06:58.933 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:06:58.934 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:06:58.956 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:06:58.956 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:06:58.956 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:06:58.956 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:06:58.957 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:06:58.959 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:06:58.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:06:58.959 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:06:59.023 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:06:59.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.144+0000 7ff173590700 1 Processor -- start 2023-10-06T03:06:59.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff173590700 1 -- start start 2023-10-06T03:06:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff173590700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff16c0c8490 0x7ff16c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff173590700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff16c069e20 0x7ff16c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff173590700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c0ac030 con 0x7ff16c069a50 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff173590700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c0ab820 con 0x7ff16c0c8490 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff173590700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c06dbb0 con 0x7ff16c069e20 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.145+0000 7ff170b2b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff16c0c8490 0x7ff16c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff16c0c8490 0x7ff16c0c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43162/0 (socket says 172.21.15.92:43162) 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 -- 172.21.15.92:0/2564413371 learned_addr learned my addr 172.21.15.92:0/2564413371 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff171b2d700 1 --2- 172.21.15.92:0/2564413371 >> v2:172.21.15.136:3300/0 conn(0x7ff16c069e20 0x7ff16c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 -- 172.21.15.92:0/2564413371 >> v2:172.21.15.136:3300/0 conn(0x7ff16c069e20 msgr2=0x7ff16c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 --2- 172.21.15.92:0/2564413371 >> v2:172.21.15.136:3300/0 conn(0x7ff16c069e20 0x7ff16c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 -- 172.21.15.92:0/2564413371 >> v1:172.21.15.136:6789/0 conn(0x7ff16c069a50 legacy=0x7ff16c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 -- 172.21.15.92:0/2564413371 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0aba70 con 0x7ff16c0c8490 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff170b2b700 1 --2- 172.21.15.92:0/2564413371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c0c8490 0x7ff16c0c6ac0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06db20 tx=0x7ff15c00a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25de6e23c4aa768 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff163fff700 1 -- 172.21.15.92:0/2564413371 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00ca40 con 0x7ff16c0c8490 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff163fff700 1 -- 172.21.15.92:0/2564413371 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c00cba0 con 0x7ff16c0c8490 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.146+0000 7ff163fff700 1 -- 172.21.15.92:0/2564413371 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c018400 con 0x7ff16c0c8490 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 -- 172.21.15.92:0/2564413371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c0c8490 msgr2=0x7ff16c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 --2- 172.21.15.92:0/2564413371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c0c8490 0x7ff16c0c6ac0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06db20 tx=0x7ff15c00a3f0 comp rx=0 tx=0).stop 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 -- 172.21.15.92:0/2564413371 shutdown_connections 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 --2- 172.21.15.92:0/2564413371 >> v2:172.21.15.136:3300/0 conn(0x7ff16c069e20 0x7ff16c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 --2- 172.21.15.92:0/2564413371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c0c8490 0x7ff16c0c6ac0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 -- 172.21.15.92:0/2564413371 >> 172.21.15.92:0/2564413371 conn(0x7ff16c0c8f90 msgr2=0x7ff16c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:06:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 -- 172.21.15.92:0/2564413371 shutdown_connections 2023-10-06T03:06:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.147+0000 7ff173590700 1 -- 172.21.15.92:0/2564413371 wait complete. 2023-10-06T03:06:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.148+0000 7ff173590700 1 Processor -- start 2023-10-06T03:06:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.148+0000 7ff173590700 1 -- start start 2023-10-06T03:06:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff173590700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 0x7ff16c157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff173590700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff16c069e20 0x7ff16c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff173590700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff16c0c8490 0x7ff16c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff173590700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b00c0 con 0x7ff16c069a50 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff173590700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b1950 con 0x7ff16c069e20 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff173590700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c158800 con 0x7ff16c0c8490 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 0x7ff16c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 0x7ff16c157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43172/0 (socket says 172.21.15.92:43172) 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 -- 172.21.15.92:0/4132822917 learned_addr learned my addr 172.21.15.92:0/4132822917 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff170b2b700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff16c069e20 0x7ff16c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff171b2d700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff16c0c8490 0x7ff16c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 -- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff16c0c8490 msgr2=0x7ff16c0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff16c0c8490 0x7ff16c0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 -- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff16c069e20 msgr2=0x7ff16c0b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff16c069e20 0x7ff16c0b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 -- 172.21.15.92:0/4132822917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c00a690 con 0x7ff16c069a50 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff170b2b700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff16c069e20 0x7ff16c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.149+0000 7ff17132c700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 0x7ff16c157610 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff16800c4e0 tx=0x7ff16800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.150+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168011590 con 0x7ff16c069a50 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.150+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff168011710 con 0x7ff16c069a50 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.150+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c157bf0 con 0x7ff16c069a50 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.150+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168011ad0 con 0x7ff16c069a50 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.150+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1581b0 con 0x7ff16c069a50 2023-10-06T03:06:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.151+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c158ae0 con 0x7ff16c069a50 2023-10-06T03:06:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.151+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7ff16801b3b0 con 0x7ff16c069a50 2023-10-06T03:06:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.151+0000 7ff1617fa700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff150067170 0x7ff150069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:06:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.151+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff168083d90 con 0x7ff16c069a50 2023-10-06T03:06:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.151+0000 7ff170b2b700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff150067170 0x7ff150069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:06:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.152+0000 7ff170b2b700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff150067170 0x7ff150069620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06db20 tx=0x7ff15c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:06:59.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.156+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff168050b70 con 0x7ff16c069a50 2023-10-06T03:06:59.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:06:59.390+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff16c0ac030 con 0x7ff16c069a50 2023-10-06T03:07:00.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.205+0000 7ff1617fa700 1 -- 172.21.15.92:0/4132822917 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff16c0ac030 con 0x7ff16c069a50 2023-10-06T03:07:00.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff150067170 msgr2=0x7ff150069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff150067170 0x7ff150069620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06db20 tx=0x7ff15c01a000 comp rx=0 tx=0).stop 2023-10-06T03:07:00.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 msgr2=0x7ff16c157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 0x7ff16c157610 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff16800c4e0 tx=0x7ff16800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 shutdown_connections 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff150067170 0x7ff150069620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff16c069a50 0x7ff16c157610 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff16c0c8490 0x7ff16c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 --2- 172.21.15.92:0/4132822917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff16c069e20 0x7ff16c0b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 >> 172.21.15.92:0/4132822917 conn(0x7ff16c0c8f90 msgr2=0x7ff16c155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:00.210 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 shutdown_connections 2023-10-06T03:07:00.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.208+0000 7ff173590700 1 -- 172.21.15.92:0/4132822917 wait complete. 2023-10-06T03:07:00.223 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:07:00.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.344+0000 7f308ce78700 1 Processor -- start 2023-10-06T03:07:00.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.344+0000 7f308ce78700 1 -- start start 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308ce78700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308806c0a0 0x7f30880caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308ce78700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f308815df50 0x7f308815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308ce78700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308815d7e0 con 0x7f308806bcd0 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308ce78700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308815dac0 con 0x7f308815df50 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308ce78700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308815aa30 con 0x7f308806c0a0 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f3086d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f308815df50 0x7f308815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f308806bcd0 0x7f308815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51938/0 (socket says 172.21.15.92:51938) 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.345+0000 7f308659c700 1 -- 172.21.15.92:0/3500865874 learned_addr learned my addr 172.21.15.92:0/3500865874 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f307dd9b700 1 --2- 172.21.15.92:0/3500865874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308806c0a0 0x7f30880caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3086d9d700 1 -- 172.21.15.92:0/3500865874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308806c0a0 msgr2=0x7f30880caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3086d9d700 1 --2- 172.21.15.92:0/3500865874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308806c0a0 0x7f30880caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3086d9d700 1 -- 172.21.15.92:0/3500865874 >> v1:172.21.15.92:6789/0 conn(0x7f308806bcd0 legacy=0x7f308815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3086d9d700 1 -- 172.21.15.92:0/3500865874 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308815ad10 con 0x7f308815df50 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3085d9b700 1 -- 172.21.15.92:0/3500865874 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745770368 0 0) 0x7f308815d7e0 con 0x7f308806bcd0 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3086d9d700 1 --2- 172.21.15.92:0/3500865874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308815df50 0x7f308815e3c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3080009e60 tx=0x7f3080009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527b6a3eff0c4e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3085d9b700 1 -- 172.21.15.92:0/3500865874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000e070 con 0x7f308815df50 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3085d9b700 1 -- 172.21.15.92:0/3500865874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308000bc30 con 0x7f308815df50 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f3085d9b700 1 -- 172.21.15.92:0/3500865874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080012430 con 0x7f308815df50 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 -- 172.21.15.92:0/3500865874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308815df50 msgr2=0x7f308815e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 --2- 172.21.15.92:0/3500865874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308815df50 0x7f308815e3c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3080009e60 tx=0x7f3080009780 comp rx=0 tx=0).stop 2023-10-06T03:07:00.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 -- 172.21.15.92:0/3500865874 shutdown_connections 2023-10-06T03:07:00.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 --2- 172.21.15.92:0/3500865874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308815df50 0x7f308815e3c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 --2- 172.21.15.92:0/3500865874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308806c0a0 0x7f30880caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 -- 172.21.15.92:0/3500865874 >> 172.21.15.92:0/3500865874 conn(0x7f30880ce600 msgr2=0x7f30880cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:00.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 -- 172.21.15.92:0/3500865874 shutdown_connections 2023-10-06T03:07:00.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.346+0000 7f308ce78700 1 -- 172.21.15.92:0/3500865874 wait complete. 2023-10-06T03:07:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 Processor -- start 2023-10-06T03:07:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 -- start start 2023-10-06T03:07:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f308806bcd0 0x7f30880c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 0x7f308814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308815df50 0x7f30880b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30880b3540 con 0x7f308806c0a0 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30880b2930 con 0x7f308806bcd0 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.348+0000 7f308ce78700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30880b2c10 con 0x7f308815df50 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 0x7f308814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 0x7f308814b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48084/0 (socket says 172.21.15.92:48084) 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 -- 172.21.15.92:0/754382593 learned_addr learned my addr 172.21.15.92:0/754382593 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f3086d9d700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308815df50 0x7f30880b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f308659c700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f308806bcd0 0x7f30880c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 -- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308815df50 msgr2=0x7f30880b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308815df50 0x7f30880b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 -- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f308806bcd0 msgr2=0x7f30880c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f308806bcd0 0x7f30880c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 -- 172.21.15.92:0/754382593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080009a20 con 0x7f308806c0a0 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f3086d9d700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308815df50 0x7f30880b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f308659c700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f308806bcd0 0x7f30880c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307dd9b700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 0x7f308814b1e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f307000d860 tx=0x7f3070008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070009610 con 0x7f308806c0a0 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070009770 con 0x7f308806c0a0 2023-10-06T03:07:00.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30880b2ef0 con 0x7f308806c0a0 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070009b30 con 0x7f308806c0a0 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.349+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30880b25e0 con 0x7f308806c0a0 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.350+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30880b36b0 con 0x7f308806c0a0 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.350+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f307001e460 con 0x7f308806c0a0 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.351+0000 7f307effd700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068067170 0x7f3068069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.351+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3070084fb0 con 0x7f308806c0a0 2023-10-06T03:07:00.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.351+0000 7f308659c700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068067170 0x7f3068069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.351+0000 7f308659c700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068067170 0x7f3068069620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3078001890 tx=0x7f3078009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:00.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.355+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3070051e30 con 0x7f308806c0a0 2023-10-06T03:07:00.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.562+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f308804c3c0 con 0x7f3068067170 2023-10-06T03:07:00.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.564+0000 7f307effd700 1 -- 172.21.15.92:0/754382593 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f308804c3c0 con 0x7f3068067170 2023-10-06T03:07:00.564 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:07:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:00.564 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:07:00.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:00.565 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068067170 msgr2=0x7f3068069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068067170 0x7f3068069620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3078001890 tx=0x7f3078009040 comp rx=0 tx=0).stop 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 msgr2=0x7f308814b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 0x7f308814b1e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f307000d860 tx=0x7f3070008040 comp rx=0 tx=0).stop 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 shutdown_connections 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3068067170 0x7f3068069620 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308815df50 0x7f30880b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f308806bcd0 0x7f30880c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 --2- 172.21.15.92:0/754382593 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f308806c0a0 0x7f308814b1e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 >> 172.21.15.92:0/754382593 conn(0x7f30880ce600 msgr2=0x7f3088157370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 shutdown_connections 2023-10-06T03:07:00.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.566+0000 7f308ce78700 1 -- 172.21.15.92:0/754382593 wait complete. 2023-10-06T03:07:00.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:07:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.695+0000 7fe2ecab8700 1 Processor -- start 2023-10-06T03:07:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.695+0000 7fe2ecab8700 1 -- start start 2023-10-06T03:07:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2ecab8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2e80b4e70 0x7fe2e80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2ecab8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b1b40 0x7fe2e815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2ecab8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80cfb00 con 0x7fe2e80b16d0 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2ecab8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e80b5c40 con 0x7fe2e80b4e70 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2ecab8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80b5f20 con 0x7fe2e80b1b40 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2e77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2e80b4e70 0x7fe2e80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.696+0000 7fe2e77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2e80b4e70 0x7fe2e80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48090/0 (socket says 172.21.15.92:48090) 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e77fe700 1 -- 172.21.15.92:0/2767934489 learned_addr learned my addr 172.21.15.92:0/2767934489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e7fff700 1 --2- 172.21.15.92:0/2767934489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b1b40 0x7fe2e815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e77fe700 1 -- 172.21.15.92:0/2767934489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b1b40 msgr2=0x7fe2e815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e77fe700 1 --2- 172.21.15.92:0/2767934489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b1b40 0x7fe2e815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e77fe700 1 -- 172.21.15.92:0/2767934489 >> v1:172.21.15.136:6789/0 conn(0x7fe2e80b16d0 legacy=0x7fe2e815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:00.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e77fe700 1 -- 172.21.15.92:0/2767934489 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80b6200 con 0x7fe2e80b4e70 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e77fe700 1 --2- 172.21.15.92:0/2767934489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b4e70 0x7fe2e80ccff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009f20 tx=0x7fe2d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dedc39351e2e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e67fc700 1 -- 172.21.15.92:0/2767934489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d800e070 con 0x7fe2e80b4e70 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e67fc700 1 -- 172.21.15.92:0/2767934489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d800ba40 con 0x7fe2e80b4e70 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.697+0000 7fe2e67fc700 1 -- 172.21.15.92:0/2767934489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80092e0 con 0x7fe2e80b4e70 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 -- 172.21.15.92:0/2767934489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b4e70 msgr2=0x7fe2e80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/2767934489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b4e70 0x7fe2e80ccff0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009f20 tx=0x7fe2d8009840 comp rx=0 tx=0).stop 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 -- 172.21.15.92:0/2767934489 shutdown_connections 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/2767934489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b4e70 0x7fe2e80ccff0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/2767934489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b1b40 0x7fe2e815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 -- 172.21.15.92:0/2767934489 >> 172.21.15.92:0/2767934489 conn(0x7fe2e80d10c0 msgr2=0x7fe2e80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 -- 172.21.15.92:0/2767934489 shutdown_connections 2023-10-06T03:07:00.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.698+0000 7fe2ecab8700 1 -- 172.21.15.92:0/2767934489 wait complete. 2023-10-06T03:07:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.699+0000 7fe2ecab8700 1 Processor -- start 2023-10-06T03:07:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.699+0000 7fe2ecab8700 1 -- start start 2023-10-06T03:07:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e80b16d0 0x7fe2e816aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 0x7fe2e816b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b4e70 0x7fe2e8167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e816ba30 con 0x7fe2e80b1b40 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e816bd10 con 0x7fe2e80b16d0 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e816bff0 con 0x7fe2e80b4e70 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 0x7fe2e816b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 0x7fe2e816b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48102/0 (socket says 172.21.15.92:48102) 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 -- 172.21.15.92:0/710273940 learned_addr learned my addr 172.21.15.92:0/710273940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e77fe700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e80b16d0 0x7fe2e816aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e7fff700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b4e70 0x7fe2e8167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 -- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b4e70 msgr2=0x7fe2e8167dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b4e70 0x7fe2e8167dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 -- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e80b16d0 msgr2=0x7fe2e816aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e80b16d0 0x7fe2e816aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 -- 172.21.15.92:0/710273940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d8009ae0 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e7fff700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b4e70 0x7fe2e8167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2e6ffd700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 0x7fe2e816b330 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00b5e0 tx=0x7fe2dc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00f510 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc00f670 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e816c2d0 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc00fa30 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.700+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e8168300 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.701+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e816c7f0 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.702+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe2dc01a400 con 0x7fe2e80b1b40 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.702+0000 7fe2cf7fe700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2d4067170 0x7fe2d4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.702+0000 7fe2e77fe700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2d4067170 0x7fe2d4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:00.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.702+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe2dc084430 con 0x7fe2e80b1b40 2023-10-06T03:07:00.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.703+0000 7fe2e77fe700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2d4067170 0x7fe2d4069620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009880 tx=0x7fe2d8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:00.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.707+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2e816c7f0 con 0x7fe2e80b1b40 2023-10-06T03:07:00.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.891+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe2e816c7f0 con 0x7fe2e80b1b40 2023-10-06T03:07:00.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.891+0000 7fe2cf7fe700 1 -- 172.21.15.92:0/710273940 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe2e816c7f0 con 0x7fe2e80b1b40 2023-10-06T03:07:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2d4067170 msgr2=0x7fe2d4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2d4067170 0x7fe2d4069620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009880 tx=0x7fe2d8017000 comp rx=0 tx=0).stop 2023-10-06T03:07:00.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 msgr2=0x7fe2e816b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 0x7fe2e816b330 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc00b5e0 tx=0x7fe2dc009120 comp rx=0 tx=0).stop 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 shutdown_connections 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2d4067170 0x7fe2d4069620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80b4e70 0x7fe2e8167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e80b16d0 0x7fe2e816aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 --2- 172.21.15.92:0/710273940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e80b1b40 0x7fe2e816b330 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 >> 172.21.15.92:0/710273940 conn(0x7fe2e80d10c0 msgr2=0x7fe2e80af470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 shutdown_connections 2023-10-06T03:07:00.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:00.893+0000 7fe2ecab8700 1 -- 172.21.15.92:0/710273940 wait complete. 2023-10-06T03:07:00.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000371993 2023-10-06T03:07:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.023+0000 7fcdfd534700 1 Processor -- start 2023-10-06T03:07:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.023+0000 7fcdfd534700 1 -- start start 2023-10-06T03:07:01.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.024+0000 7fcdfd534700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf8159a60 0x7fcdf80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.024+0000 7fcdfd534700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcdf80cb230 0x7fcdf80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.024+0000 7fcdfd534700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80c9cb0 con 0x7fcdf80ccc00 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.024+0000 7fcdfd534700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80cbbd0 con 0x7fcdf80cb230 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.024+0000 7fcdfd534700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf806dfc0 con 0x7fcdf8159a60 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdf77fe700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fcdf80ccc00 0x7fcdf8069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:52692/0 (socket says 172.21.15.92:52692) 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdfcd33700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcdf80cb230 0x7fcdf80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdf77fe700 1 -- 172.21.15.92:0/3132419384 learned_addr learned my addr 172.21.15.92:0/3132419384 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:01.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf8159a60 0x7fcdf80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdfcd33700 1 -- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf8159a60 msgr2=0x7fcdf80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdfcd33700 1 --2- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf8159a60 0x7fcdf80c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdfcd33700 1 -- 172.21.15.92:0/3132419384 >> v1:172.21.15.136:6789/0 conn(0x7fcdf80ccc00 legacy=0x7fcdf8069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdfcd33700 1 -- 172.21.15.92:0/3132419384 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80c9f30 con 0x7fcdf80cb230 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.025+0000 7fcdf6ffd700 1 -- 172.21.15.92:0/3132419384 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334241802 0 0) 0x7fcdf80c9cb0 con 0x7fcdf80ccc00 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfcd33700 1 --2- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf80cb230 0x7fcdf80cb690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009e90 tx=0x7fcdf0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9cdfaf3fcf9287e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdf6ffd700 1 -- 172.21.15.92:0/3132419384 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf000e070 con 0x7fcdf80cb230 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdf6ffd700 1 -- 172.21.15.92:0/3132419384 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf000bc80 con 0x7fcdf80cb230 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdf6ffd700 1 -- 172.21.15.92:0/3132419384 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0012430 con 0x7fcdf80cb230 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 -- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf80cb230 msgr2=0x7fcdf80cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf80cb230 0x7fcdf80cb690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009e90 tx=0x7fcdf0009780 comp rx=0 tx=0).stop 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 -- 172.21.15.92:0/3132419384 shutdown_connections 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf80cb230 0x7fcdf80cb690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3132419384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf8159a60 0x7fcdf80c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 -- 172.21.15.92:0/3132419384 >> 172.21.15.92:0/3132419384 conn(0x7fcdf806baa0 msgr2=0x7fcdf806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 -- 172.21.15.92:0/3132419384 shutdown_connections 2023-10-06T03:07:01.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.026+0000 7fcdfd534700 1 -- 172.21.15.92:0/3132419384 wait complete. 2023-10-06T03:07:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.027+0000 7fcdfd534700 1 Processor -- start 2023-10-06T03:07:01.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 -- start start 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 0x7fcdf8152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf80ccc00 0x7fcdf81573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf8159a60 0x7fcdf8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8155fa0 con 0x7fcdf80cb230 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80c3c90 con 0x7fcdf8159a60 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfd534700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8157e40 con 0x7fcdf80ccc00 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 0x7fcdf8152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 0x7fcdf8152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48106/0 (socket says 172.21.15.92:48106) 2023-10-06T03:07:01.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdf7fff700 1 -- 172.21.15.92:0/3453261627 learned_addr learned my addr 172.21.15.92:0/3453261627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdfcd33700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf8159a60 0x7fcdf8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.028+0000 7fcdf77fe700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf80ccc00 0x7fcdf81573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdf7fff700 1 -- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf80ccc00 msgr2=0x7fcdf81573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdf7fff700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf80ccc00 0x7fcdf81573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdf7fff700 1 -- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf8159a60 msgr2=0x7fcdf8157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdf7fff700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf8159a60 0x7fcdf8157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdf7fff700 1 -- 172.21.15.92:0/3453261627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0009a20 con 0x7fcdf80cb230 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdf7fff700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 0x7fcdf8152280 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e930 tx=0x7fcde80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8013680 con 0x7fcdf80cb230 2023-10-06T03:07:01.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80137e0 con 0x7fcdf80cb230 2023-10-06T03:07:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf81559e0 con 0x7fcdf80cb230 2023-10-06T03:07:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8013b80 con 0x7fcdf80cb230 2023-10-06T03:07:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.029+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf80c3920 con 0x7fcdf80cb230 2023-10-06T03:07:01.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.030+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf80718b0 con 0x7fcdf80cb230 2023-10-06T03:07:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.030+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fcde800c3e0 con 0x7fcdf80cb230 2023-10-06T03:07:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.031+0000 7fcddffff700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcde4067170 0x7fcde4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.031+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcde8084ac0 con 0x7fcdf80cb230 2023-10-06T03:07:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.031+0000 7fcdf77fe700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcde4067170 0x7fcde4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.031+0000 7fcdf77fe700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcde4067170 0x7fcde4069620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcdec007fd0 tx=0x7fcdec007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.035+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcde8051940 con 0x7fcdf80cb230 2023-10-06T03:07:01.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.251+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000371993", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdf80c3dd0 con 0x7fcde4067170 2023-10-06T03:07:01.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:01.255+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:01.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:01.255+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:01.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:01.255+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:01.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:01.255+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:01.256 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:01.255+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:01.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.270+0000 7fcddffff700 1 -- 172.21.15.92:0/3453261627 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcdec001400 con 0x7fcde4067170 2023-10-06T03:07:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcde4067170 msgr2=0x7fcde4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcde4067170 0x7fcde4069620 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcdec007fd0 tx=0x7fcdec007d10 comp rx=0 tx=0).stop 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 msgr2=0x7fcdf8152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 0x7fcdf8152280 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e930 tx=0x7fcde80091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 shutdown_connections 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcde4067170 0x7fcde4069620 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcdf80cb230 0x7fcdf8152280 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf8159a60 0x7fcdf8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 --2- 172.21.15.92:0/3453261627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf80ccc00 0x7fcdf81573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 >> 172.21.15.92:0/3453261627 conn(0x7fcdf806baa0 msgr2=0x7fcdf8159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 shutdown_connections 2023-10-06T03:07:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.272+0000 7fcdfd534700 1 -- 172.21.15.92:0/3453261627 wait complete. 2023-10-06T03:07:01.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-06T03:07:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.401+0000 7feec092d700 1 Processor -- start 2023-10-06T03:07:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.401+0000 7feec092d700 1 -- start start 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feec092d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feebc0ccc00 0x7feebc069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feec092d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feebc0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feec092d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0c9f30 con 0x7feebc159a60 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feec092d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc06dfc0 con 0x7feebc0ccc00 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feec092d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0ca470 con 0x7feebc06a020 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feebad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7feebc159a60 0x7feebc0c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51942/0 (socket says 172.21.15.92:51942) 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feebad9d700 1 -- 172.21.15.92:0/1113500708 learned_addr learned my addr 172.21.15.92:0/1113500708 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302872862 0 0) 0x7feebc0c9f30 con 0x7feebc159a60 2023-10-06T03:07:01.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.402+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea8003570 con 0x7feebc159a60 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeba59c700 1 --2- 172.21.15.92:0/1113500708 >> v2:172.21.15.136:3300/0 conn(0x7feebc0ccc00 0x7feebc069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebb59e700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feebc0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825417699 0 0) 0x7feea8003570 con 0x7feebc159a60 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 msgr2=0x7feebc0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feebc0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 >> v2:172.21.15.136:3300/0 conn(0x7feebc0ccc00 msgr2=0x7feebc069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 --2- 172.21.15.92:0/1113500708 >> v2:172.21.15.136:3300/0 conn(0x7feebc0ccc00 0x7feebc069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0ca750 con 0x7feebc159a60 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7feeb4001470 con 0x7feebc159a60 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebb59e700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feebc0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 >> v1:172.21.15.92:6789/0 conn(0x7feebc159a60 legacy=0x7feebc0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feea8003a60 0x7feea8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feea8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feea8004d50 0x7feea80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc06dfc0 con 0x7feea8003a60 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0ca470 con 0x7feea8004d50 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea800bab0 con 0x7feebc06a020 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feea8003a60 0x7feea8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebb59e700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feea8004d50 0x7feea80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feeba59c700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feea8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 -- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 msgr2=0x7feea8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feebc06a020 0x7feea8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 -- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feea8004d50 msgr2=0x7feea80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feea8004d50 0x7feea80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 -- 172.21.15.92:0/1113500708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea8003570 con 0x7feea8003a60 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.403+0000 7feebad9d700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feea8003a60 0x7feea8003e70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3c0 tx=0x7feeb4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e894938e427811c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb400cc90 con 0x7feea8003a60 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb400ce30 con 0x7feea8003a60 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feeb9d9b700 1 -- 172.21.15.92:0/1113500708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4005b70 con 0x7feea8003a60 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feec092d700 1 -- 172.21.15.92:0/1113500708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feea8003a60 msgr2=0x7feea8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feec092d700 1 --2- 172.21.15.92:0/1113500708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feea8003a60 0x7feea8003e70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3c0 tx=0x7feeb4001e90 comp rx=0 tx=0).stop 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feebb59e700 1 -- 172.21.15.92:0/1113500708 reap_dead start 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feec092d700 1 -- 172.21.15.92:0/1113500708 shutdown_connections 2023-10-06T03:07:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feec092d700 1 -- 172.21.15.92:0/1113500708 >> 172.21.15.92:0/1113500708 conn(0x7feebc06baa0 msgr2=0x7feebc06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feec092d700 1 -- 172.21.15.92:0/1113500708 shutdown_connections 2023-10-06T03:07:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.404+0000 7feec092d700 1 -- 172.21.15.92:0/1113500708 wait complete. 2023-10-06T03:07:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.405+0000 7feec092d700 1 Processor -- start 2023-10-06T03:07:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.405+0000 7feec092d700 1 -- start start 2023-10-06T03:07:01.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feec092d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feea8003a60 0x7feebc155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feec092d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 0x7feebc0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feec092d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feebc0b2dd0 0x7feebc0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feec092d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc155860 con 0x7feebc156e20 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feec092d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b2940 con 0x7feebc0b2dd0 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feec092d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0b1d30 con 0x7feea8003a60 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 0x7feebc0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 0x7feebc0b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48158/0 (socket says 172.21.15.92:48158) 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 -- 172.21.15.92:0/1501318024 learned_addr learned my addr 172.21.15.92:0/1501318024 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feebad9d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feea8003a60 0x7feebc155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feebb59e700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feebc0b2dd0 0x7feebc0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 -- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feea8003a60 msgr2=0x7feebc155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feea8003a60 0x7feebc155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 -- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feebc0b2dd0 msgr2=0x7feebc0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feebc0b2dd0 0x7feebc0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feeba59c700 1 -- 172.21.15.92:0/1501318024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0b2010 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feebad9d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feea8003a60 0x7feebc155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.406+0000 7feebb59e700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feebc0b2dd0 0x7feebc0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.407+0000 7feeba59c700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 0x7feebc0b1430 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7feea4011d70 tx=0x7feea40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.407+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4009e20 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.407+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400fcd0 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.407+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0b22f0 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.407+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4014650 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.407+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc0b1970 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.408+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc06eba0 con 0x7feebc156e20 2023-10-06T03:07:01.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.408+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7feea4014a80 con 0x7feebc156e20 2023-10-06T03:07:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.409+0000 7feeb2ffd700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee9c067170 0x7fee9c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.409+0000 7feebad9d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee9c067170 0x7fee9c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.409+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7feea4018030 con 0x7feebc156e20 2023-10-06T03:07:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.409+0000 7feebad9d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee9c067170 0x7fee9c069620 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3c0 tx=0x7feeb4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.413+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feea40522e0 con 0x7feebc156e20 2023-10-06T03:07:01.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.620+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7feebc0b2790 con 0x7fee9c067170 2023-10-06T03:07:01.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.622+0000 7feeb2ffd700 1 -- 172.21.15.92:0/1501318024 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+534 (secure 0 0 0) 0x7feebc0b2790 con 0x7fee9c067170 2023-10-06T03:07:01.623 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_addrs": [ 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6789", 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6789", 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6790" 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "pending_subvolume_deletions": 0, 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": { 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "data": [ 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": " 340G", 2023-10-06T03:07:01.624 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_data", 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "used": " 0 " 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": [ 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": " 340G", 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_metadata", 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "used": "64.0k" 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:01.625 INFO:teuthology.orchestra.run.smithi092.stdout: "used_size": " 0 " 2023-10-06T03:07:01.626 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:07:01.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee9c067170 msgr2=0x7fee9c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee9c067170 0x7fee9c069620 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c3c0 tx=0x7feeb4015000 comp rx=0 tx=0).stop 2023-10-06T03:07:01.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 msgr2=0x7feebc0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 0x7feebc0b1430 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7feea4011d70 tx=0x7feea40091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 shutdown_connections 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee9c067170 0x7fee9c069620 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feebc0b2dd0 0x7feebc0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feea8003a60 0x7feebc155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 --2- 172.21.15.92:0/1501318024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7feebc156e20 0x7feebc0b1430 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 >> 172.21.15.92:0/1501318024 conn(0x7feebc06baa0 msgr2=0x7feebc15a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 shutdown_connections 2023-10-06T03:07:01.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.625+0000 7feec092d700 1 -- 172.21.15.92:0/1501318024 wait complete. 2023-10-06T03:07:01.638 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:07:01.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:01.676 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:01.719 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:07:01.719 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:07:01.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:07:01.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:07:01.765 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:07:01.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:01.766 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:07:01.780 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:07:01.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:07:01.781 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:07:01.827 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:07:01.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:01.828 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:07:01.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:01.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.958+0000 7f08cf59e700 1 Processor -- start 2023-10-06T03:07:01.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.958+0000 7f08cf59e700 1 -- start start 2023-10-06T03:07:01.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cf59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f08d0159420 0x7f08d00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cf59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cf59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08d015d230 0x7f08d015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cf59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d015dbd0 con 0x7f08d015d230 2023-10-06T03:07:01.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cf59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d00b11d0 con 0x7f08d0159420 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cf59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d00b14b0 con 0x7f08d01597f0 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08d015d230 0x7f08d015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08d015d230 0x7f08d015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48160/0 (socket says 172.21.15.92:48160) 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 -- 172.21.15.92:0/133790917 learned_addr learned my addr 172.21.15.92:0/133790917 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ce59c700 1 --2- 172.21.15.92:0/133790917 >> v2:172.21.15.136:3300/0 conn(0x7f08d0159420 0x7f08d00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08c5d9b700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 -- 172.21.15.92:0/133790917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 msgr2=0x7f08d00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 -- 172.21.15.92:0/133790917 >> v2:172.21.15.136:3300/0 conn(0x7f08d0159420 msgr2=0x7f08d00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:01.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 --2- 172.21.15.92:0/133790917 >> v2:172.21.15.136:3300/0 conn(0x7f08d0159420 0x7f08d00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 -- 172.21.15.92:0/133790917 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d00b1790 con 0x7f08d015d230 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ce59c700 1 --2- 172.21.15.92:0/133790917 >> v2:172.21.15.136:3300/0 conn(0x7f08d0159420 0x7f08d00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08c5d9b700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08ced9d700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d015d230 0x7f08d015d690 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f08d00b2f20 tx=0x7f08c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff4e6c5aee27a14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.959+0000 7f08cdd9b700 1 -- 172.21.15.92:0/133790917 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800e070 con 0x7f08d015d230 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cdd9b700 1 -- 172.21.15.92:0/133790917 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c800ba50 con 0x7f08d015d230 2023-10-06T03:07:01.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cdd9b700 1 -- 172.21.15.92:0/133790917 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80092e0 con 0x7f08d015d230 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 -- 172.21.15.92:0/133790917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d015d230 msgr2=0x7f08d015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d015d230 0x7f08d015d690 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f08d00b2f20 tx=0x7f08c8009840 comp rx=0 tx=0).stop 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 -- 172.21.15.92:0/133790917 shutdown_connections 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d015d230 0x7f08d015d690 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 --2- 172.21.15.92:0/133790917 >> v2:172.21.15.136:3300/0 conn(0x7f08d0159420 0x7f08d00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 --2- 172.21.15.92:0/133790917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 -- 172.21.15.92:0/133790917 >> 172.21.15.92:0/133790917 conn(0x7f08d00d0660 msgr2=0x7f08d00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:01.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 -- 172.21.15.92:0/133790917 shutdown_connections 2023-10-06T03:07:01.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.960+0000 7f08cf59e700 1 -- 172.21.15.92:0/133790917 wait complete. 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.961+0000 7f08cf59e700 1 Processor -- start 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.961+0000 7f08cf59e700 1 -- start start 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08cf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 0x7f08d016dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08cf59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08cf59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d015d230 0x7f08d0150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08cf59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0159c60 con 0x7f08d0159420 2023-10-06T03:07:01.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08cf59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d015c670 con 0x7f08d015d230 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08cf59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0159eb0 con 0x7f08d01597f0 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08ce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 0x7f08d016dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08ce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 0x7f08d016dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48166/0 (socket says 172.21.15.92:48166) 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08ce59c700 1 -- 172.21.15.92:0/2524144452 learned_addr learned my addr 172.21.15.92:0/2524144452 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08ce59c700 1 -- 172.21.15.92:0/2524144452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 msgr2=0x7f08d015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.962+0000 7f08ce59c700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08ce59c700 1 -- 172.21.15.92:0/2524144452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d015d230 msgr2=0x7f08d0150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08ce59c700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d015d230 0x7f08d0150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08ce59c700 1 -- 172.21.15.92:0/2524144452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009ae0 con 0x7f08d0159420 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08ce59c700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 0x7f08d016dcf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e900 tx=0x7f08c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c000eea0 con 0x7f08d0159420 2023-10-06T03:07:01.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c000c5e0 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d015a190 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c000c980 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.963+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d015a750 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.964+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d004c3c0 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.964+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f08c000cd30 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.965+0000 7f08c6ffd700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08b0067170 0x7f08b0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.965+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08c00849b0 con 0x7f08d0159420 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.965+0000 7f08c5d9b700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08b0067170 0x7f08b0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:01.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.965+0000 7f08c5d9b700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08b0067170 0x7f08b0069620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f08b8006fd0 tx=0x7f08b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:01.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:01.969+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08c0051680 con 0x7f08d0159420 2023-10-06T03:07:02.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.192+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d00b1a70 con 0x7f08d0159420 2023-10-06T03:07:02.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.193+0000 7f08c6ffd700 1 -- 172.21.15.92:0/2524144452 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f08c0054ca0 con 0x7f08d0159420 2023-10-06T03:07:02.194 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:02.194 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:06:50.531995+0000","modified":"2023-10-06T03:06:54.171754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/2746794570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2746794570},{"type":"v1","addr":"172.21.15.136:6835","nonce":2746794570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08b0067170 msgr2=0x7f08b0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08b0067170 0x7f08b0069620 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f08b8006fd0 tx=0x7f08b8008040 comp rx=0 tx=0).stop 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 msgr2=0x7f08d016dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 0x7f08d016dcf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e900 tx=0x7f08c0008040 comp rx=0 tx=0).stop 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 shutdown_connections 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08b0067170 0x7f08b0069620 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08d0159420 0x7f08d016dcf0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08d015d230 0x7f08d0150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 --2- 172.21.15.92:0/2524144452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08d01597f0 0x7f08d015c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 >> 172.21.15.92:0/2524144452 conn(0x7f08d00d0660 msgr2=0x7f08d00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 shutdown_connections 2023-10-06T03:07:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.195+0000 7f08cf59e700 1 -- 172.21.15.92:0/2524144452 wait complete. 2023-10-06T03:07:02.197 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 48 2023-10-06T03:07:02.208 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:02.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.322+0000 7f20dd15a700 1 Processor -- start 2023-10-06T03:07:02.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.322+0000 7f20dd15a700 1 -- start start 2023-10-06T03:07:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20dd15a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20d8159420 0x7f20d80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20dd15a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20dd15a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d80b11d0 con 0x7f20d815d230 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20dd15a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80b14b0 con 0x7f20d8159420 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20dd15a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d80b1790 con 0x7f20d81597f0 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20d8159420 0x7f20d80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f20d8159420 0x7f20d80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48178/0 (socket says 172.21.15.92:48178) 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 -- 172.21.15.92:0/2714341261 learned_addr learned my addr 172.21.15.92:0/2714341261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d77fe700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 -- 172.21.15.92:0/2714341261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 msgr2=0x7f20d80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 -- 172.21.15.92:0/2714341261 >> v1:172.21.15.136:6789/0 conn(0x7f20d815d230 legacy=0x7f20d80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.323+0000 7f20d7fff700 1 -- 172.21.15.92:0/2714341261 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80b1a70 con 0x7f20d8159420 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20d77fe700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20d7fff700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d80b5f40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f20c80099b0 tx=0x7f20c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a51f7708d6b202 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20d6ffd700 1 -- 172.21.15.92:0/2714341261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c800b860 con 0x7f20d8159420 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20d6ffd700 1 -- 172.21.15.92:0/2714341261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c800b9c0 con 0x7f20d8159420 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20d6ffd700 1 -- 172.21.15.92:0/2714341261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c800bd60 con 0x7f20d8159420 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 -- 172.21.15.92:0/2714341261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 msgr2=0x7f20d80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d80b5f40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f20c80099b0 tx=0x7f20c8009250 comp rx=0 tx=0).stop 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 -- 172.21.15.92:0/2714341261 shutdown_connections 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d80b5f40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 --2- 172.21.15.92:0/2714341261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 -- 172.21.15.92:0/2714341261 >> 172.21.15.92:0/2714341261 conn(0x7f20d80d0660 msgr2=0x7f20d80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 -- 172.21.15.92:0/2714341261 shutdown_connections 2023-10-06T03:07:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.324+0000 7f20dd15a700 1 -- 172.21.15.92:0/2714341261 wait complete. 2023-10-06T03:07:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.325+0000 7f20dd15a700 1 Processor -- start 2023-10-06T03:07:02.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 -- start start 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d816d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20d815d230 0x7f20d8143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d816da30 con 0x7f20d8159420 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d816dd10 con 0x7f20d815d230 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20dd15a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d816ad40 con 0x7f20d81597f0 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.326+0000 7f20d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d816d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d816d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48190/0 (socket says 172.21.15.92:48190) 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 -- 172.21.15.92:0/414936952 learned_addr learned my addr 172.21.15.92:0/414936952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:02.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20dc959700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20d815d230 0x7f20d8143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d77fe700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 -- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 msgr2=0x7f20d80b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 -- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20d815d230 msgr2=0x7f20d8143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20d815d230 0x7f20d8143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 -- 172.21.15.92:0/414936952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80094f0 con 0x7f20d8159420 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d77fe700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20d7fff700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d816d170 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009290 tx=0x7f20c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8026480 con 0x7f20d8159420 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c801f070 con 0x7f20d8159420 2023-10-06T03:07:02.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d816b020 con 0x7f20d8159420 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8026830 con 0x7f20d8159420 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.327+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d80b2900 con 0x7f20d8159420 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.328+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d80b2af0 con 0x7f20d8159420 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.329+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f20c8026ba0 con 0x7f20d8159420 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.329+0000 7f20bffff700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20c4067170 0x7f20c4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.329+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f20c8021070 con 0x7f20d8159420 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.329+0000 7f20d77fe700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20c4067170 0x7f20c4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.329+0000 7f20d77fe700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20c4067170 0x7f20c4069620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006f00 tx=0x7f20cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.333+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20c8053620 con 0x7f20d8159420 2023-10-06T03:07:02.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.516+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20d804c3c0 con 0x7f20d8159420 2023-10-06T03:07:02.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.517+0000 7f20bffff700 1 -- 172.21.15.92:0/414936952 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f20c8056c40 con 0x7f20d8159420 2023-10-06T03:07:02.518 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:02.518 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:52.159838+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:06:48.185180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:06:49.502363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:02.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20c4067170 msgr2=0x7f20c4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20c4067170 0x7f20c4069620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f20cc006f00 tx=0x7f20cc008040 comp rx=0 tx=0).stop 2023-10-06T03:07:02.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 msgr2=0x7f20d816d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d816d170 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009290 tx=0x7f20c8017000 comp rx=0 tx=0).stop 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 shutdown_connections 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f20c4067170 0x7f20c4069620 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20d8159420 0x7f20d816d170 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20d815d230 0x7f20d8143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 --2- 172.21.15.92:0/414936952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20d81597f0 0x7f20d80b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 >> 172.21.15.92:0/414936952 conn(0x7f20d80d0660 msgr2=0x7f20d80b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 shutdown_connections 2023-10-06T03:07:02.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.519+0000 7f20dd15a700 1 -- 172.21.15.92:0/414936952 wait complete. 2023-10-06T03:07:02.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:06:48.185180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:06:49.502363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:07:02.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:07:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.649+0000 7f430682b700 1 Processor -- start 2023-10-06T03:07:02.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.649+0000 7f430682b700 1 -- start start 2023-10-06T03:07:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430682b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f43000b4e70 0x7f43000ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430682b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43000b1b40 0x7f430015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430682b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43000cfb00 con 0x7f43000b16d0 2023-10-06T03:07:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430682b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43000b5c40 con 0x7f43000b1b40 2023-10-06T03:07:02.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430682b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43000b5f20 con 0x7f43000b4e70 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430602a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43000b1b40 0x7f430015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430602a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43000b1b40 0x7f430015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48198/0 (socket says 172.21.15.92:48198) 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430602a700 1 -- 172.21.15.92:0/3920381326 learned_addr learned my addr 172.21.15.92:0/3920381326 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f4305829700 1 --2- 172.21.15.92:0/3920381326 >> v2:172.21.15.136:3300/0 conn(0x7f43000b4e70 0x7f43000ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430602a700 1 -- 172.21.15.92:0/3920381326 >> v2:172.21.15.136:3300/0 conn(0x7f43000b4e70 msgr2=0x7f43000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430602a700 1 --2- 172.21.15.92:0/3920381326 >> v2:172.21.15.136:3300/0 conn(0x7f43000b4e70 0x7f43000ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.650+0000 7f430602a700 1 -- 172.21.15.92:0/3920381326 >> v1:172.21.15.136:6789/0 conn(0x7f43000b16d0 legacy=0x7f430015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430602a700 1 -- 172.21.15.92:0/3920381326 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000b6200 con 0x7f43000b1b40 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f4305829700 1 --2- 172.21.15.92:0/3920381326 >> v2:172.21.15.136:3300/0 conn(0x7f43000b4e70 0x7f43000ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430602a700 1 --2- 172.21.15.92:0/3920381326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b1b40 0x7f430015d950 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4300064910 tx=0x7f42f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aceeed5040ebe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f4304827700 1 -- 172.21.15.92:0/3920381326 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800b860 con 0x7f43000b1b40 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f4304827700 1 -- 172.21.15.92:0/3920381326 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f800b9c0 con 0x7f43000b1b40 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f4304827700 1 -- 172.21.15.92:0/3920381326 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800bd60 con 0x7f43000b1b40 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 -- 172.21.15.92:0/3920381326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b1b40 msgr2=0x7f430015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 --2- 172.21.15.92:0/3920381326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b1b40 0x7f430015d950 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4300064910 tx=0x7f42f8009250 comp rx=0 tx=0).stop 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 -- 172.21.15.92:0/3920381326 shutdown_connections 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 --2- 172.21.15.92:0/3920381326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b1b40 0x7f430015d950 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 --2- 172.21.15.92:0/3920381326 >> v2:172.21.15.136:3300/0 conn(0x7f43000b4e70 0x7f43000ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 -- 172.21.15.92:0/3920381326 >> 172.21.15.92:0/3920381326 conn(0x7f43000d10c0 msgr2=0x7f43000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 -- 172.21.15.92:0/3920381326 shutdown_connections 2023-10-06T03:07:02.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.651+0000 7f430682b700 1 -- 172.21.15.92:0/3920381326 wait complete. 2023-10-06T03:07:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.652+0000 7f430682b700 1 Processor -- start 2023-10-06T03:07:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.652+0000 7f430682b700 1 -- start start 2023-10-06T03:07:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430682b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43000b16d0 0x7f4300149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430682b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43000b1b40 0x7f430015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430682b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 0x7f4300068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430682b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430015b730 con 0x7f43000b4e70 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430682b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430016b380 con 0x7f43000b1b40 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430682b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430015b170 con 0x7f43000b16d0 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430602a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 0x7f4300068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430602a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 0x7f4300068a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48210/0 (socket says 172.21.15.92:48210) 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f430602a700 1 -- 172.21.15.92:0/1391490059 learned_addr learned my addr 172.21.15.92:0/1391490059 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f4305028700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43000b1b40 0x7f430015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.653+0000 7f4305829700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43000b16d0 0x7f4300149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430602a700 1 -- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43000b16d0 msgr2=0x7f4300149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430602a700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43000b16d0 0x7f4300149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430602a700 1 -- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43000b1b40 msgr2=0x7f430015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430602a700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43000b1b40 0x7f430015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430602a700 1 -- 172.21.15.92:0/1391490059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80094f0 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430602a700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 0x7f4300068a50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f42f8006f50 tx=0x7f42f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800b860 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f801a660 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430015b450 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f801aa20 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.654+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430015c310 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.655+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4300088990 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.655+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f42f801b430 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.656+0000 7f42f5ffb700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e4067170 0x7f42e4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.656+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f42f80861b0 con 0x7f43000b4e70 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.656+0000 7f4305829700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e4067170 0x7f42e4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.656+0000 7f4305829700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e4067170 0x7f42e4069620 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0076a0 tx=0x7f42fc009930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.660+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42f8053030 con 0x7f43000b4e70 2023-10-06T03:07:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.841+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4300069840 con 0x7f43000b4e70 2023-10-06T03:07:02.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.841+0000 7f42f5ffb700 1 -- 172.21.15.92:0/1391490059 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42f8056650 con 0x7f43000b4e70 2023-10-06T03:07:02.842 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:07:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.843+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e4067170 msgr2=0x7f42e4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.843+0000 7f430682b700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e4067170 0x7f42e4069620 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0076a0 tx=0x7f42fc009930 comp rx=0 tx=0).stop 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.843+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 msgr2=0x7f4300068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.843+0000 7f430682b700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 0x7f4300068a50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f42f8006f50 tx=0x7f42f8017000 comp rx=0 tx=0).stop 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 shutdown_connections 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f42e4067170 0x7f42e4069620 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43000b4e70 0x7f4300068a50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43000b16d0 0x7f4300149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 --2- 172.21.15.92:0/1391490059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43000b1b40 0x7f430015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 >> 172.21.15.92:0/1391490059 conn(0x7f43000d10c0 msgr2=0x7f4300159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 shutdown_connections 2023-10-06T03:07:02.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.844+0000 7f430682b700 1 -- 172.21.15.92:0/1391490059 wait complete. 2023-10-06T03:07:02.856 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:07:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.972+0000 7f87ecc9e700 1 Processor -- start 2023-10-06T03:07:02.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.972+0000 7f87ecc9e700 1 -- start start 2023-10-06T03:07:02.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ecc9e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87e8159c60 0x7f87e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ecc9e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e815a160 con 0x7f87e8159890 2023-10-06T03:07:02.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ecc9e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80d13a0 con 0x7f87e815db10 2023-10-06T03:07:02.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ecc9e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e815df50 con 0x7f87e8159c60 2023-10-06T03:07:02.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ddd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f87e8159c60 0x7f87e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87e659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f87e8159890 0x7f87e815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51956/0 (socket says 172.21.15.92:51956) 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87e659c700 1 -- 172.21.15.92:0/4122426445 learned_addr learned my addr 172.21.15.92:0/4122426445 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ddd9b700 1 -- 172.21.15.92:0/4122426445 >> v1:172.21.15.136:6789/0 conn(0x7f87e815db10 legacy=0x7f87e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ddd9b700 1 -- 172.21.15.92:0/4122426445 >> v1:172.21.15.92:6789/0 conn(0x7f87e8159890 legacy=0x7f87e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87e5d9b700 1 -- 172.21.15.92:0/4122426445 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357398465 0 0) 0x7f87e815a160 con 0x7f87e8159890 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ddd9b700 1 -- 172.21.15.92:0/4122426445 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e815e140 con 0x7f87e8159c60 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.973+0000 7f87ddd9b700 1 --2- 172.21.15.92:0/4122426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159c60 0x7f87e815d5d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098e0 tx=0x7f87d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aee2955a7ff5223 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87e5d9b700 1 -- 172.21.15.92:0/4122426445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0009e80 con 0x7f87e8159c60 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87e5d9b700 1 -- 172.21.15.92:0/4122426445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d000b860 con 0x7f87e8159c60 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87e5d9b700 1 -- 172.21.15.92:0/4122426445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000bbe0 con 0x7f87e8159c60 2023-10-06T03:07:02.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 -- 172.21.15.92:0/4122426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159c60 msgr2=0x7f87e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/4122426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159c60 0x7f87e815d5d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f87d00098e0 tx=0x7f87d00091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 -- 172.21.15.92:0/4122426445 shutdown_connections 2023-10-06T03:07:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/4122426445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159c60 0x7f87e815d5d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 -- 172.21.15.92:0/4122426445 >> 172.21.15.92:0/4122426445 conn(0x7f87e8068e20 msgr2=0x7f87e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 -- 172.21.15.92:0/4122426445 shutdown_connections 2023-10-06T03:07:02.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.974+0000 7f87ecc9e700 1 -- 172.21.15.92:0/4122426445 wait complete. 2023-10-06T03:07:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.975+0000 7f87ecc9e700 1 Processor -- start 2023-10-06T03:07:02.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.975+0000 7f87ecc9e700 1 -- start start 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ecc9e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 0x7f87e80b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ecc9e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e8159c60 0x7f87e80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ecc9e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87e815db10 0x7f87e80b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ecc9e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80b2090 con 0x7f87e8159890 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ecc9e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80b64b0 con 0x7f87e8159c60 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ecc9e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80b1480 con 0x7f87e815db10 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 0x7f87e80b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 0x7f87e80b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48226/0 (socket says 172.21.15.92:48226) 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 -- 172.21.15.92:0/1430049079 learned_addr learned my addr 172.21.15.92:0/1430049079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 -- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87e815db10 msgr2=0x7f87e80b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87ddd9b700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e8159c60 0x7f87e80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87e815db10 0x7f87e80b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 -- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e8159c60 msgr2=0x7f87e80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e8159c60 0x7f87e80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.976+0000 7f87e659c700 1 -- 172.21.15.92:0/1430049079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009420 con 0x7f87e8159890 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87ddd9b700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e8159c60 0x7f87e80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87e659c700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 0x7f87e80b5f70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f87d800c810 tx=0x7f87d8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8005c00 con 0x7f87e8159890 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8005d60 con 0x7f87e8159890 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8007340 con 0x7f87e8159890 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e80b1760 con 0x7f87e8159890 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.977+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e80b1d20 con 0x7f87e8159890 2023-10-06T03:07:02.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.978+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e804c3c0 con 0x7f87e8159890 2023-10-06T03:07:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.978+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f87d801b400 con 0x7f87e8159890 2023-10-06T03:07:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.979+0000 7f87deffd700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f87c8067170 0x7f87c8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.979+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f87d8084460 con 0x7f87e8159890 2023-10-06T03:07:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.979+0000 7f87ddd9b700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f87c8067170 0x7f87c8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:02.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.979+0000 7f87ddd9b700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f87c8067170 0x7f87c8069620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87e8157310 tx=0x7f87d0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:02.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:02.983+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87d8051360 con 0x7f87e8159890 2023-10-06T03:07:03.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.164+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87e80caba0 con 0x7f87e8159890 2023-10-06T03:07:03.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.164+0000 7f87deffd700 1 -- 172.21.15.92:0/1430049079 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v78) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87d8054980 con 0x7f87e8159890 2023-10-06T03:07:03.165 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:07:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f87c8067170 msgr2=0x7f87c8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f87c8067170 0x7f87c8069620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f87e8157310 tx=0x7f87d0017000 comp rx=0 tx=0).stop 2023-10-06T03:07:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 msgr2=0x7f87e80b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 0x7f87e80b5f70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f87d800c810 tx=0x7f87d8007d90 comp rx=0 tx=0).stop 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 shutdown_connections 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f87c8067170 0x7f87c8069620 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87e8159890 0x7f87e80b5f70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87e815db10 0x7f87e80b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 --2- 172.21.15.92:0/1430049079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87e8159c60 0x7f87e80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 >> 172.21.15.92:0/1430049079 conn(0x7f87e8068e20 msgr2=0x7f87e8153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 shutdown_connections 2023-10-06T03:07:03.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.166+0000 7f87ecc9e700 1 -- 172.21.15.92:0/1430049079 wait complete. 2023-10-06T03:07:03.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:07:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.295+0000 7fedd5dfc700 1 Processor -- start 2023-10-06T03:07:03.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedd5dfc700 1 -- start start 2023-10-06T03:07:03.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedd5dfc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd0159890 0x7fedd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedd5dfc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedd0159c60 0x7fedd015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedd5dfc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd015a160 con 0x7fedd015db10 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedd5dfc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd00d13a0 con 0x7fedd0159c60 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedd5dfc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd015e140 con 0x7fedd0159890 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedceffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedd0159c60 0x7fedd015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedcffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fedd015db10 0x7fedd0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51972/0 (socket says 172.21.15.92:51972) 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.296+0000 7fedcffff700 1 -- 172.21.15.92:0/757891243 learned_addr learned my addr 172.21.15.92:0/757891243 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedcf7fe700 1 --2- 172.21.15.92:0/757891243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd0159890 0x7fedd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedceffd700 1 -- 172.21.15.92:0/757891243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd0159890 msgr2=0x7fedd015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedceffd700 1 --2- 172.21.15.92:0/757891243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd0159890 0x7fedd015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedceffd700 1 -- 172.21.15.92:0/757891243 >> v1:172.21.15.92:6789/0 conn(0x7fedd015db10 legacy=0x7fedd0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedceffd700 1 -- 172.21.15.92:0/757891243 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd015a3d0 con 0x7fedd0159c60 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedce7fc700 1 -- 172.21.15.92:0/757891243 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238243169 0 0) 0x7fedd015a160 con 0x7fedd015db10 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedceffd700 1 --2- 172.21.15.92:0/757891243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd015d5d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fedc40098e0 tx=0x7fedc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91b03f714d9f995 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedce7fc700 1 -- 172.21.15.92:0/757891243 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4009e80 con 0x7fedd0159c60 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedce7fc700 1 -- 172.21.15.92:0/757891243 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc400b860 con 0x7fedd0159c60 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedce7fc700 1 -- 172.21.15.92:0/757891243 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400bbe0 con 0x7fedd0159c60 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedd5dfc700 1 -- 172.21.15.92:0/757891243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 msgr2=0x7fedd015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.297+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/757891243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd015d5d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fedc40098e0 tx=0x7fedc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.298+0000 7fedd5dfc700 1 -- 172.21.15.92:0/757891243 shutdown_connections 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.298+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/757891243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd0159890 0x7fedd015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.298+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/757891243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd015d5d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.298+0000 7fedd5dfc700 1 -- 172.21.15.92:0/757891243 >> 172.21.15.92:0/757891243 conn(0x7fedd0068e20 msgr2=0x7fedd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:03.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.298+0000 7fedd5dfc700 1 -- 172.21.15.92:0/757891243 shutdown_connections 2023-10-06T03:07:03.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.298+0000 7fedd5dfc700 1 -- 172.21.15.92:0/757891243 wait complete. 2023-10-06T03:07:03.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.299+0000 7fedd5dfc700 1 Processor -- start 2023-10-06T03:07:03.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.299+0000 7fedd5dfc700 1 -- start start 2023-10-06T03:07:03.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd0159890 0x7fedd00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd01472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd015db10 0x7fedd00cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd0147830 con 0x7fedd0159c60 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00cfd50 con 0x7fedd0159890 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00b5b00 con 0x7fedd015db10 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd01472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd01472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48244/0 (socket says 172.21.15.92:48244) 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 -- 172.21.15.92:0/595571756 learned_addr learned my addr 172.21.15.92:0/595571756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:03.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedcf7fe700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd0159890 0x7fedd00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedcffff700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd015db10 0x7fedd00cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 -- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd015db10 msgr2=0x7fedd00cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd015db10 0x7fedd00cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 -- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd0159890 msgr2=0x7fedd00b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd0159890 0x7fedd00b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 -- 172.21.15.92:0/595571756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4009420 con 0x7fedd0159c60 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedcf7fe700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd0159890 0x7fedd00b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:03.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedceffd700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd01472f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fedc4013000 tx=0x7fedc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400bbe0 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4011420 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40117c0 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd00b51d0 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.300+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd00b5790 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.301+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd00b64d0 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.302+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fedc4011b70 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.302+0000 7fedb77fe700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedbc067170 0x7fedbc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.302+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fedc40863b0 con 0x7fedd0159c60 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.302+0000 7fedcf7fe700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedbc067170 0x7fedbc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.302+0000 7fedcf7fe700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedbc067170 0x7fedbc069620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fedc00075e0 tx=0x7fedc00097e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.306+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedc4053230 con 0x7fedd0159c60 2023-10-06T03:07:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.487+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fedd00b5c70 con 0x7fedd0159c60 2023-10-06T03:07:03.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.487+0000 7fedb77fe700 1 -- 172.21.15.92:0/595571756 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fedc4056850 con 0x7fedd0159c60 2023-10-06T03:07:03.488 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:07:03.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedbc067170 msgr2=0x7fedbc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedbc067170 0x7fedbc069620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fedc00075e0 tx=0x7fedc00097e0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 msgr2=0x7fedd01472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd01472f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fedc4013000 tx=0x7fedc4017000 comp rx=0 tx=0).stop 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 shutdown_connections 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedbc067170 0x7fedbc069620 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedd015db10 0x7fedd00cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedd0159890 0x7fedd00b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 --2- 172.21.15.92:0/595571756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedd0159c60 0x7fedd01472f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 >> 172.21.15.92:0/595571756 conn(0x7fedd0068e20 msgr2=0x7fedd01530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 shutdown_connections 2023-10-06T03:07:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.489+0000 7fedd5dfc700 1 -- 172.21.15.92:0/595571756 wait complete. 2023-10-06T03:07:03.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:07:03.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.621+0000 7fd84c4de700 1 Processor -- start 2023-10-06T03:07:03.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.621+0000 7fd84c4de700 1 -- start start 2023-10-06T03:07:03.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd84c4de700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd844069560 0x7fd8440699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd84c4de700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd844069ee0 con 0x7fd844159c60 2023-10-06T03:07:03.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd84c4de700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84415a160 con 0x7fd844159890 2023-10-06T03:07:03.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd84c4de700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8440d13a0 con 0x7fd844069560 2023-10-06T03:07:03.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849a79700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd844159c60 0x7fd84415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51984/0 (socket says 172.21.15.92:51984) 2023-10-06T03:07:03.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849a79700 1 -- 172.21.15.92:0/2260730372 learned_addr learned my addr 172.21.15.92:0/2260730372 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd84aa7b700 1 --2- 172.21.15.92:0/2260730372 >> v2:172.21.15.136:3300/0 conn(0x7fd844069560 0x7fd8440699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642985857 0 0) 0x7fd844069ee0 con 0x7fd844159c60 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd830003570 con 0x7fd844159c60 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218626303 0 0) 0x7fd830003570 con 0x7fd844159c60 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 >> v2:172.21.15.136:3300/0 conn(0x7fd844069560 msgr2=0x7fd8440699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 --2- 172.21.15.92:0/2260730372 >> v2:172.21.15.136:3300/0 conn(0x7fd844069560 0x7fd8440699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 >> v1:172.21.15.136:6789/0 conn(0x7fd844159890 legacy=0x7fd84415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84415e140 con 0x7fd844159c60 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd834001470 con 0x7fd844159c60 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 >> v1:172.21.15.92:6789/0 conn(0x7fd844159c60 legacy=0x7fd84415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd830003a60 0x7fd830003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.622+0000 7fd849278700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830004810 0x7fd830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd83000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440d13a0 con 0x7fd8300051c0 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83000c0d0 con 0x7fd830004810 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83000c290 con 0x7fd830003a60 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd83000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849a79700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830004810 0x7fd830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84a27a700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd830003a60 0x7fd830003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 -- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd830003a60 msgr2=0x7fd830003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd830003a60 0x7fd830003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 -- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830004810 msgr2=0x7fd830004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:03.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd830004810 0x7fd830004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 -- 172.21.15.92:0/2260730372 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830003570 con 0x7fd8300051c0 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 -- 172.21.15.92:0/2260730372 reap_dead start 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd84aa7b700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd83000ba60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e210 tx=0x7fd83c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7c9ff655e43156 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00eac0 con 0x7fd8300051c0 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00f040 con 0x7fd8300051c0 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.623+0000 7fd849278700 1 -- 172.21.15.92:0/2260730372 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00ee70 con 0x7fd8300051c0 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 -- 172.21.15.92:0/2260730372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 msgr2=0x7fd83000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd83000ba60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e210 tx=0x7fd83c00b040 comp rx=0 tx=0).stop 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 -- 172.21.15.92:0/2260730372 shutdown_connections 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2260730372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd83000ba60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 -- 172.21.15.92:0/2260730372 >> 172.21.15.92:0/2260730372 conn(0x7fd844068e20 msgr2=0x7fd8440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:03.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 -- 172.21.15.92:0/2260730372 shutdown_connections 2023-10-06T03:07:03.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.624+0000 7fd84c4de700 1 -- 172.21.15.92:0/2260730372 wait complete. 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.625+0000 7fd84c4de700 1 Processor -- start 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.625+0000 7fd84c4de700 1 -- start start 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd844152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8440ce1c0 0x7fd8441538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8440ce630 0x7fd844153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440cf7b0 con 0x7fd8300051c0 2023-10-06T03:07:03.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441526b0 con 0x7fd8440ce1c0 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440ceba0 con 0x7fd8440ce630 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd844152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd844152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48268/0 (socket says 172.21.15.92:48268) 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 -- 172.21.15.92:0/2284693810 learned_addr learned my addr 172.21.15.92:0/2284693810 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 -- 172.21.15.92:0/2284693810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8440ce630 msgr2=0x7fd844153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8440ce630 0x7fd844153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 -- 172.21.15.92:0/2284693810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8440ce1c0 msgr2=0x7fd8441538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8440ce1c0 0x7fd8441538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 -- 172.21.15.92:0/2284693810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440cee80 con 0x7fd8300051c0 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84a27a700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd844152170 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd840002950 tx=0x7fd840007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840002e90 con 0x7fd8300051c0 2023-10-06T03:07:03.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000f8d0 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8440cf160 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000fc70 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.626+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8440b6030 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.627+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84404c3c0 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.628+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd84001c400 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.628+0000 7fd83a7fc700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd828067200 0x7fd8280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.628+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8400850e0 con 0x7fd8300051c0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.628+0000 7fd849a79700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd828067200 0x7fd8280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.628+0000 7fd849a79700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd828067200 0x7fd8280696b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd834000fd0 tx=0x7fd834001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.632+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd840051e70 con 0x7fd8300051c0 2023-10-06T03:07:03.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.814+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8440cfa90 con 0x7fd8300051c0 2023-10-06T03:07:03.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.815+0000 7fd83a7fc700 1 -- 172.21.15.92:0/2284693810 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd840055490 con 0x7fd8300051c0 2023-10-06T03:07:03.816 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd828067200 msgr2=0x7fd8280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd828067200 0x7fd8280696b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd834000fd0 tx=0x7fd834001e50 comp rx=0 tx=0).stop 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 msgr2=0x7fd844152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd844152170 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd840002950 tx=0x7fd840007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 shutdown_connections 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd828067200 0x7fd8280696b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8300051c0 0x7fd844152170 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8440ce630 0x7fd844153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 --2- 172.21.15.92:0/2284693810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8440ce1c0 0x7fd8441538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 >> 172.21.15.92:0/2284693810 conn(0x7fd844068e20 msgr2=0x7fd8440b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:03.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 shutdown_connections 2023-10-06T03:07:03.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.817+0000 7fd84c4de700 1 -- 172.21.15.92:0/2284693810 wait complete. 2023-10-06T03:07:03.829 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:03.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.943+0000 7f1b816cc700 1 Processor -- start 2023-10-06T03:07:03.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 -- start start 2023-10-06T03:07:03.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1b7c159890 0x7f1b7c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b7c15db10 0x7f1b7c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7c15a160 con 0x7f1b7c15db10 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7c0d13a0 con 0x7f1b7c159890 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b816cc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c15e140 con 0x7f1b7c159c60 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.944+0000 7f1b7b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b7c15db10 0x7f1b7c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1b7c15db10 0x7f1b7c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48284/0 (socket says 172.21.15.92:48284) 2023-10-06T03:07:03.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/4098803809 learned_addr learned my addr 172.21.15.92:0/4098803809 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:03.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/4098803809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 msgr2=0x7f1b7c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:03.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/4098803809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/4098803809 >> v2:172.21.15.136:3300/0 conn(0x7f1b7c159890 msgr2=0x7f1b7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:03.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7affd700 1 --2- 172.21.15.92:0/4098803809 >> v2:172.21.15.136:3300/0 conn(0x7f1b7c159890 0x7f1b7c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/4098803809 >> v2:172.21.15.136:3300/0 conn(0x7f1b7c159890 0x7f1b7c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/4098803809 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c15a3d0 con 0x7f1b7c15db10 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/4098803809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c15db10 0x7f1b7c069560 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0d1090 tx=0x7f1b74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae00c44ab0b9aa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4098803809 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400e070 con 0x7f1b7c15db10 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4098803809 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7400ba50 con 0x7f1b7c15db10 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4098803809 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740092e0 con 0x7f1b7c15db10 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.945+0000 7f1b816cc700 1 -- 172.21.15.92:0/4098803809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c15db10 msgr2=0x7f1b7c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 --2- 172.21.15.92:0/4098803809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c15db10 0x7f1b7c069560 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0d1090 tx=0x7f1b74009840 comp rx=0 tx=0).stop 2023-10-06T03:07:03.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 -- 172.21.15.92:0/4098803809 shutdown_connections 2023-10-06T03:07:03.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 --2- 172.21.15.92:0/4098803809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c15db10 0x7f1b7c069560 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 --2- 172.21.15.92:0/4098803809 >> v2:172.21.15.136:3300/0 conn(0x7f1b7c159890 0x7f1b7c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 --2- 172.21.15.92:0/4098803809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 -- 172.21.15.92:0/4098803809 >> 172.21.15.92:0/4098803809 conn(0x7f1b7c068e20 msgr2=0x7f1b7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:03.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 -- 172.21.15.92:0/4098803809 shutdown_connections 2023-10-06T03:07:03.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.946+0000 7f1b816cc700 1 -- 172.21.15.92:0/4098803809 wait complete. 2023-10-06T03:07:03.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.947+0000 7f1b816cc700 1 Processor -- start 2023-10-06T03:07:03.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.947+0000 7f1b816cc700 1 -- start start 2023-10-06T03:07:03.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 0x7f1b7c14c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c15db10 0x7f1b7c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c0c6680 con 0x7f1b7c159890 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c15c780 con 0x7f1b7c15db10 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c14c3b0 con 0x7f1b7c159c60 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 0x7f1b7c14c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 0x7f1b7c14c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48300/0 (socket says 172.21.15.92:48300) 2023-10-06T03:07:03.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 -- 172.21.15.92:0/1484454835 learned_addr learned my addr 172.21.15.92:0/1484454835 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c15db10 0x7f1b7c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b727fc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 -- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 msgr2=0x7f1b7c14b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c14b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 -- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c15db10 msgr2=0x7f1b7c1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c15db10 0x7f1b7c1511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:03.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 -- 172.21.15.92:0/1484454835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b74009ae0 con 0x7f1b7c159890 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c15db10 0x7f1b7c1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b7affd700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 0x7f1b7c14c650 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c800 tx=0x7f1b6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c011600 con 0x7f1b7c159890 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c011760 con 0x7f1b7c159890 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c0c6890 con 0x7f1b7c159890 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c011b20 con 0x7f1b7c159890 2023-10-06T03:07:03.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.948+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c14b800 con 0x7f1b7c159890 2023-10-06T03:07:03.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.949+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c0c6d30 con 0x7f1b7c159890 2023-10-06T03:07:03.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.950+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f1b6c0193f0 con 0x7f1b7c159890 2023-10-06T03:07:03.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.950+0000 7f1b737fe700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b5c067200 0x7f1b5c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:03.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.950+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1b6c085020 con 0x7f1b7c159890 2023-10-06T03:07:03.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.950+0000 7f1b727fc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b5c067200 0x7f1b5c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:03.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.950+0000 7f1b727fc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b5c067200 0x7f1b5c0696b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1b64006fd0 tx=0x7f1b64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:03.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:03.954+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b6c051cd0 con 0x7f1b7c159890 2023-10-06T03:07:04.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.179+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b7c04c3c0 con 0x7f1b7c159890 2023-10-06T03:07:04.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.179+0000 7f1b737fe700 1 -- 172.21.15.92:0/1484454835 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1b6c0552f0 con 0x7f1b7c159890 2023-10-06T03:07:04.180 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:04.180 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:06:50.531995+0000","modified":"2023-10-06T03:06:54.171754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/2746794570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2746794570},{"type":"v1","addr":"172.21.15.136:6835","nonce":2746794570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:07:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b5c067200 msgr2=0x7f1b5c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b5c067200 0x7f1b5c0696b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1b64006fd0 tx=0x7f1b64008040 comp rx=0 tx=0).stop 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 msgr2=0x7f1b7c14c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 0x7f1b7c14c650 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c800 tx=0x7f1b6c008040 comp rx=0 tx=0).stop 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 shutdown_connections 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b5c067200 0x7f1b5c0696b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c159890 0x7f1b7c14c650 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c15db10 0x7f1b7c1511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 --2- 172.21.15.92:0/1484454835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159c60 0x7f1b7c14b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 >> 172.21.15.92:0/1484454835 conn(0x7f1b7c068e20 msgr2=0x7f1b7c14d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 shutdown_connections 2023-10-06T03:07:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.181+0000 7f1b816cc700 1 -- 172.21.15.92:0/1484454835 wait complete. 2023-10-06T03:07:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 48 2023-10-06T03:07:04.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:04.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.308+0000 7f4dfa6a8700 1 Processor -- start 2023-10-06T03:07:04.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.308+0000 7f4dfa6a8700 1 -- start start 2023-10-06T03:07:04.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4dfa6a8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4df4157960 0x7f4df40af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4dfa6a8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4df406b970 0x7f4df4157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4dfa6a8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df406b030 con 0x7f4df406b4d0 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4dfa6a8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df40ac760 con 0x7f4df406b970 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4dfa6a8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df40af830 con 0x7f4df4157960 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4df8c45700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4df406b970 0x7f4df4157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4df37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4df406b4d0 0x7f4df4068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51994/0 (socket says 172.21.15.92:51994) 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.309+0000 7f4df37fe700 1 -- 172.21.15.92:0/3705449355 learned_addr learned my addr 172.21.15.92:0/3705449355 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df3fff700 1 --2- 172.21.15.92:0/3705449355 >> v2:172.21.15.136:3300/0 conn(0x7f4df4157960 0x7f4df40af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df8c45700 1 -- 172.21.15.92:0/3705449355 >> v2:172.21.15.136:3300/0 conn(0x7f4df4157960 msgr2=0x7f4df40af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df8c45700 1 --2- 172.21.15.92:0/3705449355 >> v2:172.21.15.136:3300/0 conn(0x7f4df4157960 0x7f4df40af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df8c45700 1 -- 172.21.15.92:0/3705449355 >> v1:172.21.15.92:6789/0 conn(0x7f4df406b4d0 legacy=0x7f4df4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df8c45700 1 -- 172.21.15.92:0/3705449355 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40c7f60 con 0x7f4df406b970 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df3fff700 1 --2- 172.21.15.92:0/3705449355 >> v2:172.21.15.136:3300/0 conn(0x7f4df4157960 0x7f4df40af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df8c45700 1 --2- 172.21.15.92:0/3705449355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df406b970 0x7f4df4157d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4df40ccdd0 tx=0x7f4dec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59485c81d675ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df2ffd700 1 -- 172.21.15.92:0/3705449355 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00e070 con 0x7f4df406b970 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df2ffd700 1 -- 172.21.15.92:0/3705449355 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec00bc30 con 0x7f4df406b970 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.310+0000 7f4df2ffd700 1 -- 172.21.15.92:0/3705449355 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec012430 con 0x7f4df406b970 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3705449355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df406b970 msgr2=0x7f4df4157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3705449355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df406b970 0x7f4df4157d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4df40ccdd0 tx=0x7f4dec009780 comp rx=0 tx=0).stop 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3705449355 shutdown_connections 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3705449355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df406b970 0x7f4df4157d30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3705449355 >> v2:172.21.15.136:3300/0 conn(0x7f4df4157960 0x7f4df40af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3705449355 >> 172.21.15.92:0/3705449355 conn(0x7f4df40c9870 msgr2=0x7f4df40c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3705449355 shutdown_connections 2023-10-06T03:07:04.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.311+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3705449355 wait complete. 2023-10-06T03:07:04.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.312+0000 7f4dfa6a8700 1 Processor -- start 2023-10-06T03:07:04.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.312+0000 7f4dfa6a8700 1 -- start start 2023-10-06T03:07:04.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df406b4d0 0x7f4df415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df406b970 0x7f4df40b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 0x7f4df4168360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40b2940 con 0x7f4df4157960 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df415b670 con 0x7f4df406b970 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4168bb0 con 0x7f4df406b4d0 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 0x7f4df4168360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 0x7f4df4168360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48318/0 (socket says 172.21.15.92:48318) 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 -- 172.21.15.92:0/3066763901 learned_addr learned my addr 172.21.15.92:0/3066763901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df37fe700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df406b970 0x7f4df40b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df3fff700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df406b4d0 0x7f4df415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 -- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df406b4d0 msgr2=0x7f4df415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df406b4d0 0x7f4df415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 -- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df406b970 msgr2=0x7f4df40b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df406b970 0x7f4df40b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 -- 172.21.15.92:0/3066763901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009a20 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4df8c45700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 0x7f4df4168360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4dec013000 tx=0x7f4dec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec01e630 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec01e7d0 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4168e90 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec01eb90 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.313+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df415b300 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.314+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df40c7420 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.315+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4dec024070 con 0x7f4df4157960 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.315+0000 7f4ddbfff700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4de0067200 0x7f4de00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.315+0000 7f4df3fff700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4de0067200 0x7f4de00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.315+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4dec087440 con 0x7f4df4157960 2023-10-06T03:07:04.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.315+0000 7f4df3fff700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4de0067200 0x7f4de00696b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4de4006fd0 tx=0x7f4de4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:04.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.319+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dec054200 con 0x7f4df4157960 2023-10-06T03:07:04.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.504+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4df404c3c0 con 0x7f4df4157960 2023-10-06T03:07:04.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.505+0000 7f4ddbfff700 1 -- 172.21.15.92:0/3066763901 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f4dec057820 con 0x7f4df4157960 2023-10-06T03:07:04.506 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:04.506 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:52.159838+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:06:48.185180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:06:49.502363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.507+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4de0067200 msgr2=0x7f4de00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.507+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4de0067200 0x7f4de00696b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4de4006fd0 tx=0x7f4de4008040 comp rx=0 tx=0).stop 2023-10-06T03:07:04.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.507+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 msgr2=0x7f4df4168360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.507+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 0x7f4df4168360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4dec013000 tx=0x7f4dec019000 comp rx=0 tx=0).stop 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 shutdown_connections 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4de0067200 0x7f4de00696b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4df4157960 0x7f4df4168360 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df406b4d0 0x7f4df415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 --2- 172.21.15.92:0/3066763901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df406b970 0x7f4df40b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 >> 172.21.15.92:0/3066763901 conn(0x7f4df40c9870 msgr2=0x7f4df41596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 shutdown_connections 2023-10-06T03:07:04.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.508+0000 7f4dfa6a8700 1 -- 172.21.15.92:0/3066763901 wait complete. 2023-10-06T03:07:04.521 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:07:04.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:04.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.635+0000 7f5552396700 1 Processor -- start 2023-10-06T03:07:04.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.635+0000 7f5552396700 1 -- start start 2023-10-06T03:07:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f5552396700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f554c0ccf90 0x7f554c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f5552396700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c0cc300 con 0x7f554c0b1420 2023-10-06T03:07:04.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f5552396700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c0ad370 con 0x7f554c0cbf30 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f5552396700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554c156bf0 con 0x7f554c0ccf90 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f554c0b1420 0x7f554c0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51996/0 (socket says 172.21.15.92:51996) 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554b7fe700 1 -- 172.21.15.92:0/2349074661 learned_addr learned my addr 172.21.15.92:0/2349074661 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f5550933700 1 --2- 172.21.15.92:0/2349074661 >> v2:172.21.15.136:3300/0 conn(0x7f554c0ccf90 0x7f554c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388765085 0 0) 0x7f554c0cc300 con 0x7f554c0b1420 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5534003570 con 0x7f554c0b1420 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2710576336 0 0) 0x7f5534003570 con 0x7f554c0b1420 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 >> v2:172.21.15.136:3300/0 conn(0x7f554c0ccf90 msgr2=0x7f554c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 --2- 172.21.15.92:0/2349074661 >> v2:172.21.15.136:3300/0 conn(0x7f554c0ccf90 0x7f554c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 >> v1:172.21.15.136:6789/0 conn(0x7f554c0cbf30 legacy=0x7f554c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0ad650 con 0x7f554c0b1420 2023-10-06T03:07:04.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5540001470 con 0x7f554c0b1420 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 >> v1:172.21.15.92:6789/0 conn(0x7f554c0b1420 legacy=0x7f554c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5534003a60 0x7f5534003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534004810 0x7f5534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55340051c0 0x7f553400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c156bf0 con 0x7f5534003a60 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553400c0d0 con 0x7f5534004810 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553400c290 con 0x7f55340051c0 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.636+0000 7f554bfff700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5534003a60 0x7f5534003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554b7fe700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534004810 0x7f5534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5550933700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55340051c0 0x7f553400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554bfff700 1 -- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55340051c0 msgr2=0x7f553400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554bfff700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55340051c0 0x7f553400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554bfff700 1 -- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534004810 msgr2=0x7f5534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554bfff700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534004810 0x7f5534004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554bfff700 1 -- 172.21.15.92:0/2349074661 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534003570 con 0x7f5534003a60 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5550933700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55340051c0 0x7f553400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5550933700 1 -- 172.21.15.92:0/2349074661 reap_dead start 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554bfff700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5534003a60 0x7f5534003e70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f553c0018b0 tx=0x7f553c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d07fb2cb9d86a81 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c012420 con 0x7f5534003a60 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c0125a0 con 0x7f5534003a60 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f554affd700 1 -- 172.21.15.92:0/2349074661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c012960 con 0x7f5534003a60 2023-10-06T03:07:04.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5552396700 1 -- 172.21.15.92:0/2349074661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5534003a60 msgr2=0x7f5534003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5552396700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5534003a60 0x7f5534003e70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f553c0018b0 tx=0x7f553c007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5552396700 1 -- 172.21.15.92:0/2349074661 shutdown_connections 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5552396700 1 --2- 172.21.15.92:0/2349074661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5534003a60 0x7f5534003e70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.637+0000 7f5552396700 1 -- 172.21.15.92:0/2349074661 >> 172.21.15.92:0/2349074661 conn(0x7f554c06a320 msgr2=0x7f554c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.638+0000 7f5552396700 1 -- 172.21.15.92:0/2349074661 shutdown_connections 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.638+0000 7f5552396700 1 -- 172.21.15.92:0/2349074661 wait complete. 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 Processor -- start 2023-10-06T03:07:04.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 -- start start 2023-10-06T03:07:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534003a60 0x7f554c0aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f554c0a9760 0x7f554c1688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 0x7f554c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0a9bd0 con 0x7f554c152f20 2023-10-06T03:07:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c168de0 con 0x7f5534003a60 2023-10-06T03:07:04.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.639+0000 7f5552396700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0a9e40 con 0x7f554c0a9760 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 0x7f554c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 0x7f554c0abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48328/0 (socket says 172.21.15.92:48328) 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 -- 172.21.15.92:0/1757237534 learned_addr learned my addr 172.21.15.92:0/1757237534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f554bfff700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534003a60 0x7f554c0aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f554b7fe700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f554c0a9760 0x7f554c1688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 -- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f554c0a9760 msgr2=0x7f554c1688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f554c0a9760 0x7f554c1688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 -- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534003a60 msgr2=0x7f554c0aaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534003a60 0x7f554c0aaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 -- 172.21.15.92:0/1757237534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0aa120 con 0x7f554c152f20 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5550933700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 0x7f554c0abef0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f554c0b03d0 tx=0x7f554400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:04.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554401b070 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554400f040 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c0aa400 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400e050 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.640+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c0aa9c0 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.641+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c0aabe0 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.641+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5544003020 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.642+0000 7f5533fff700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5538067200 0x7f55380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.642+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5544085020 con 0x7f554c152f20 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.642+0000 7f554bfff700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5538067200 0x7f55380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.642+0000 7f554bfff700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5538067200 0x7f55380696b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f554c153900 tx=0x7f553c015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:04.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.646+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5544051cd0 con 0x7f554c152f20 2023-10-06T03:07:04.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.870+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f554c04c3c0 con 0x7f554c152f20 2023-10-06T03:07:04.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.871+0000 7f5533fff700 1 -- 172.21.15.92:0/1757237534 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f55440552f0 con 0x7f554c152f20 2023-10-06T03:07:04.877 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:04.877 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:06:50.531995+0000","modified":"2023-10-06T03:06:54.171754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/2746794570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2746794570},{"type":"v1","addr":"172.21.15.136:6835","nonce":2746794570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:07:04.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5538067200 msgr2=0x7f55380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5538067200 0x7f55380696b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f554c153900 tx=0x7f553c015000 comp rx=0 tx=0).stop 2023-10-06T03:07:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 msgr2=0x7f554c0abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 0x7f554c0abef0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f554c0b03d0 tx=0x7f554400b040 comp rx=0 tx=0).stop 2023-10-06T03:07:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 shutdown_connections 2023-10-06T03:07:04.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5538067200 0x7f55380696b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f554c152f20 0x7f554c0abef0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5534003a60 0x7f554c0aaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 --2- 172.21.15.92:0/1757237534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f554c0a9760 0x7f554c1688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 >> 172.21.15.92:0/1757237534 conn(0x7f554c06a320 msgr2=0x7f554c1560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 shutdown_connections 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:04.873+0000 7f5552396700 1 -- 172.21.15.92:0/1757237534 wait complete. 2023-10-06T03:07:04.902 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 48 2023-10-06T03:07:04.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:05.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.017+0000 7f5612236700 1 Processor -- start 2023-10-06T03:07:05.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.017+0000 7f5612236700 1 -- start start 2023-10-06T03:07:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f5612236700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f5612236700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f560c069560 0x7f560c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f5612236700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560c15a160 con 0x7f560c159c60 2023-10-06T03:07:05.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f5612236700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560c069f20 con 0x7f560c069560 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f5612236700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0d13a0 con 0x7f560c159890 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560b7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f560c069560 0x7f560c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f560c069560 0x7f560c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:40376/0 (socket says 172.21.15.92:40376) 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560b7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:56384/0 (socket says 172.21.15.92:56384) 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 -- 172.21.15.92:0/3355624863 learned_addr learned my addr 172.21.15.92:0/3355624863 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f5603fff700 1 --1- 172.21.15.92:0/3355624863 >> v1:172.21.15.136:6789/0 conn(0x7f560c159c60 0x7f560c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:52756/0 (socket says 172.21.15.92:52756) 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 -- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 msgr2=0x7f560c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 --2- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 -- 172.21.15.92:0/3355624863 >> v1:172.21.15.136:6789/0 conn(0x7f560c159c60 legacy=0x7f560c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560bfff700 1 -- 172.21.15.92:0/3355624863 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560c15e140 con 0x7f560c069560 2023-10-06T03:07:05.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.018+0000 7f560affd700 1 -- 172.21.15.92:0/3355624863 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854426136 0 0) 0x7f560c15a160 con 0x7f560c159c60 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f560bfff700 1 --2- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c069560 0x7f560c0699e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5604009f20 tx=0x7f5604009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=334454c149203c52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f560affd700 1 -- 172.21.15.92:0/3355624863 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400e070 con 0x7f560c069560 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f560affd700 1 -- 172.21.15.92:0/3355624863 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400b9d0 con 0x7f560c069560 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f560affd700 1 -- 172.21.15.92:0/3355624863 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040092e0 con 0x7f560c069560 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 -- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c069560 msgr2=0x7f560c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 --2- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c069560 0x7f560c0699e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5604009f20 tx=0x7f5604009840 comp rx=0 tx=0).stop 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 -- 172.21.15.92:0/3355624863 shutdown_connections 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 --2- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c069560 0x7f560c0699e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 --2- 172.21.15.92:0/3355624863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 -- 172.21.15.92:0/3355624863 >> 172.21.15.92:0/3355624863 conn(0x7f560c068e20 msgr2=0x7f560c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:05.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 -- 172.21.15.92:0/3355624863 shutdown_connections 2023-10-06T03:07:05.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.019+0000 7f5612236700 1 -- 172.21.15.92:0/3355624863 wait complete. 2023-10-06T03:07:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.020+0000 7f5612236700 1 Processor -- start 2023-10-06T03:07:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.020+0000 7f5612236700 1 -- start start 2023-10-06T03:07:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5612236700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 0x7f560c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5612236700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5612236700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c159c60 0x7f560c145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5612236700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c99c0 con 0x7f560c069560 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5612236700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c9ca0 con 0x7f560c159c60 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5612236700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560c0c9f80 con 0x7f560c159890 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f560b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 0x7f560c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f560b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 0x7f560c0b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48332/0 (socket says 172.21.15.92:48332) 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f560b7fe700 1 -- 172.21.15.92:0/942099795 learned_addr learned my addr 172.21.15.92:0/942099795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f5603fff700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.021+0000 7f560bfff700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c159c60 0x7f560c145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f560b7fe700 1 -- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 msgr2=0x7f560c0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f560b7fe700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c0c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f560b7fe700 1 -- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c159c60 msgr2=0x7f560c145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f560b7fe700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c159c60 0x7f560c145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f560b7fe700 1 -- 172.21.15.92:0/942099795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604009ae0 con 0x7f560c069560 2023-10-06T03:07:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f560b7fe700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 0x7f560c0b5450 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00ec80 tx=0x7f55fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00c850 con 0x7f560c069560 2023-10-06T03:07:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc00c9b0 con 0x7f560c069560 2023-10-06T03:07:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c0ca260 con 0x7f560c069560 2023-10-06T03:07:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00cd50 con 0x7f560c069560 2023-10-06T03:07:05.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.022+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c0caa60 con 0x7f560c069560 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.023+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c0ca780 con 0x7f560c069560 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.023+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f55fc013680 con 0x7f560c069560 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.024+0000 7f56037fe700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55ec067200 0x7f55ec0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.024+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55fc084d60 con 0x7f560c069560 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.024+0000 7f5603fff700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55ec067200 0x7f55ec0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.024+0000 7f5603fff700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55ec067200 0x7f55ec0696b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f560c154ac0 tx=0x7f55f4007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.028+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55fc051b20 con 0x7f560c069560 2023-10-06T03:07:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.251+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f560c04c3c0 con 0x7f560c069560 2023-10-06T03:07:05.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.251+0000 7f56037fe700 1 -- 172.21.15.92:0/942099795 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f55fc055140 con 0x7f560c069560 2023-10-06T03:07:05.254 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:05.254 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:06:50.531995+0000","modified":"2023-10-06T03:06:54.171754+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6835/2746794570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2746794570},{"type":"v1","addr":"172.21.15.136:6835","nonce":2746794570}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:07:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55ec067200 msgr2=0x7f55ec0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55ec067200 0x7f55ec0696b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f560c154ac0 tx=0x7f55f4007d50 comp rx=0 tx=0).stop 2023-10-06T03:07:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 msgr2=0x7f560c0b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 0x7f560c0b5450 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00ec80 tx=0x7f55fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 shutdown_connections 2023-10-06T03:07:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55ec067200 0x7f55ec0696b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f560c069560 0x7f560c0b5450 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f560c159c60 0x7f560c145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 --2- 172.21.15.92:0/942099795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f560c159890 0x7f560c0c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 >> 172.21.15.92:0/942099795 conn(0x7f560c068e20 msgr2=0x7f560c1551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 shutdown_connections 2023-10-06T03:07:05.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.254+0000 7f5612236700 1 -- 172.21.15.92:0/942099795 wait complete. 2023-10-06T03:07:05.259 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 48 2023-10-06T03:07:05.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:05.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.381+0000 7ff9e759e700 1 Processor -- start 2023-10-06T03:07:05.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.381+0000 7ff9e759e700 1 -- start start 2023-10-06T03:07:05.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9e81593b0 0x7ff9e8159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80ceaa0 0x7ff9e80b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e759e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff9e80b1740 0x7ff9e80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e80b2100 con 0x7ff9e81593b0 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e759e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e80b6570 con 0x7ff9e80b1740 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e80cfe20 con 0x7ff9e80ceaa0 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9e81593b0 0x7ff9e8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9e81593b0 0x7ff9e8159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48346/0 (socket says 172.21.15.92:48346) 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 -- 172.21.15.92:0/2724740694 learned_addr learned my addr 172.21.15.92:0/2724740694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 -- 172.21.15.92:0/2724740694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80ceaa0 msgr2=0x7ff9e80b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:05.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 --2- 172.21.15.92:0/2724740694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80ceaa0 0x7ff9e80b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 -- 172.21.15.92:0/2724740694 >> v2:172.21.15.136:3300/0 conn(0x7ff9e80b1740 msgr2=0x7ff9e80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e6d9d700 1 --2- 172.21.15.92:0/2724740694 >> v2:172.21.15.136:3300/0 conn(0x7ff9e80b1740 0x7ff9e80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 --2- 172.21.15.92:0/2724740694 >> v2:172.21.15.136:3300/0 conn(0x7ff9e80b1740 0x7ff9e80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 -- 172.21.15.92:0/2724740694 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80cf210 con 0x7ff9e81593b0 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e6d9d700 1 --2- 172.21.15.92:0/2724740694 >> v2:172.21.15.136:3300/0 conn(0x7ff9e80b1740 0x7ff9e80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.382+0000 7ff9e659c700 1 --2- 172.21.15.92:0/2724740694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8159780 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b24b0 tx=0x7ff9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3105c0119cf32f24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e5d9b700 1 -- 172.21.15.92:0/2724740694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800b860 con 0x7ff9e81593b0 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e5d9b700 1 -- 172.21.15.92:0/2724740694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d800b9c0 con 0x7ff9e81593b0 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e5d9b700 1 -- 172.21.15.92:0/2724740694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800bd60 con 0x7ff9e81593b0 2023-10-06T03:07:05.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 -- 172.21.15.92:0/2724740694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 msgr2=0x7ff9e8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 --2- 172.21.15.92:0/2724740694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8159780 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b24b0 tx=0x7ff9d8009250 comp rx=0 tx=0).stop 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 -- 172.21.15.92:0/2724740694 shutdown_connections 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 --2- 172.21.15.92:0/2724740694 >> v2:172.21.15.136:3300/0 conn(0x7ff9e80b1740 0x7ff9e80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 --2- 172.21.15.92:0/2724740694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8159780 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 --2- 172.21.15.92:0/2724740694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80ceaa0 0x7ff9e80b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 -- 172.21.15.92:0/2724740694 >> 172.21.15.92:0/2724740694 conn(0x7ff9e80d1130 msgr2=0x7ff9e80b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 -- 172.21.15.92:0/2724740694 shutdown_connections 2023-10-06T03:07:05.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.383+0000 7ff9e759e700 1 -- 172.21.15.92:0/2724740694 wait complete. 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.384+0000 7ff9e759e700 1 Processor -- start 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.384+0000 7ff9e759e700 1 -- start start 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80b1740 0x7ff9e814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9e80ceaa0 0x7ff9e816a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e814db80 con 0x7ff9e81593b0 2023-10-06T03:07:05.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e815b070 con 0x7ff9e80ceaa0 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e815b350 con 0x7ff9e80b1740 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8069360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48362/0 (socket says 172.21.15.92:48362) 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e6d9d700 1 -- 172.21.15.92:0/975293683 learned_addr learned my addr 172.21.15.92:0/975293683 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9ddd9b700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9e80ceaa0 0x7ff9e816a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.385+0000 7ff9e659c700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80b1740 0x7ff9e814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e6d9d700 1 -- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80b1740 msgr2=0x7ff9e814d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e6d9d700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80b1740 0x7ff9e814d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e6d9d700 1 -- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9e80ceaa0 msgr2=0x7ff9e816a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e6d9d700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9e80ceaa0 0x7ff9e816a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e6d9d700 1 -- 172.21.15.92:0/975293683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d80094f0 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e6d9d700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8069360 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000a380 tx=0x7ff9e0007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0015440 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e00155a0 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e815b630 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0015960 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.386+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e815bbf0 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.387+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e815be10 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.387+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff9e000d440 con 0x7ff9e81593b0 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.388+0000 7ff9deffd700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff9c8067200 0x7ff9c80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.388+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff9e00855c0 con 0x7ff9e81593b0 2023-10-06T03:07:05.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.388+0000 7ff9e659c700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff9c8067200 0x7ff9c80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.388+0000 7ff9e659c700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff9c8067200 0x7ff9c80696b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b24b0 tx=0x7ff9d8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.392+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9e0052380 con 0x7ff9e81593b0 2023-10-06T03:07:05.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.578+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9e804c3c0 con 0x7ff9e81593b0 2023-10-06T03:07:05.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.579+0000 7ff9deffd700 1 -- 172.21.15.92:0/975293683 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16869 (secure 0 0 0) 0x7ff9e00559a0 con 0x7ff9e81593b0 2023-10-06T03:07:05.581 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:05.581 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":78,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:06:52.159838+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:06:48.185180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:06:49.502363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff9c8067200 msgr2=0x7ff9c80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff9c8067200 0x7ff9c80696b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b24b0 tx=0x7ff9d8017000 comp rx=0 tx=0).stop 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 msgr2=0x7ff9e8069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8069360 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000a380 tx=0x7ff9e0007900 comp rx=0 tx=0).stop 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 shutdown_connections 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff9c8067200 0x7ff9c80696b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9e81593b0 0x7ff9e8069360 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9e80b1740 0x7ff9e814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 --2- 172.21.15.92:0/975293683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9e80ceaa0 0x7ff9e816a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 >> 172.21.15.92:0/975293683 conn(0x7ff9e80d1130 msgr2=0x7ff9e80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 shutdown_connections 2023-10-06T03:07:05.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.581+0000 7ff9e759e700 1 -- 172.21.15.92:0/975293683 wait complete. 2023-10-06T03:07:05.595 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:07:05.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.708+0000 7fe2baabb700 1 Processor -- start 2023-10-06T03:07:05.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.708+0000 7fe2baabb700 1 -- start start 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2baabb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2b4159420 0x7fe2b40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2baabb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2baabb700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b40b2980 con 0x7fe2b41597f0 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2baabb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b415dbb0 con 0x7fe2b4159420 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2baabb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40b11d0 con 0x7fe2b415d230 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2b9ab9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2b4159420 0x7fe2b40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2b9ab9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2b4159420 0x7fe2b40b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48370/0 (socket says 172.21.15.92:48370) 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2b9ab9700 1 -- 172.21.15.92:0/596448561 learned_addr learned my addr 172.21.15.92:0/596448561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2ba2ba700 1 --2- 172.21.15.92:0/596448561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2b9ab9700 1 -- 172.21.15.92:0/596448561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 msgr2=0x7fe2b415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.709+0000 7fe2b9ab9700 1 --2- 172.21.15.92:0/596448561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2b9ab9700 1 -- 172.21.15.92:0/596448561 >> v1:172.21.15.136:6789/0 conn(0x7fe2b41597f0 legacy=0x7fe2b40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2b9ab9700 1 -- 172.21.15.92:0/596448561 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b40b14b0 con 0x7fe2b4159420 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2b9ab9700 1 --2- 172.21.15.92:0/596448561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159420 0x7fe2b40b5f40 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009f20 tx=0x7fe2b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91de522cee8f6c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2b8ab7700 1 -- 172.21.15.92:0/596448561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000e070 con 0x7fe2b4159420 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2b8ab7700 1 -- 172.21.15.92:0/596448561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000ba80 con 0x7fe2b4159420 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2b8ab7700 1 -- 172.21.15.92:0/596448561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00092e0 con 0x7fe2b4159420 2023-10-06T03:07:05.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 -- 172.21.15.92:0/596448561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159420 msgr2=0x7fe2b40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 --2- 172.21.15.92:0/596448561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159420 0x7fe2b40b5f40 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009f20 tx=0x7fe2b0009840 comp rx=0 tx=0).stop 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 -- 172.21.15.92:0/596448561 shutdown_connections 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 --2- 172.21.15.92:0/596448561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159420 0x7fe2b40b5f40 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 --2- 172.21.15.92:0/596448561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 -- 172.21.15.92:0/596448561 >> 172.21.15.92:0/596448561 conn(0x7fe2b40d0660 msgr2=0x7fe2b40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 -- 172.21.15.92:0/596448561 shutdown_connections 2023-10-06T03:07:05.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.710+0000 7fe2baabb700 1 -- 172.21.15.92:0/596448561 wait complete. 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.711+0000 7fe2baabb700 1 Processor -- start 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 -- start start 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159420 0x7fe2b41551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 0x7fe2b40cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c6ba0 con 0x7fe2b41597f0 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c9ba0 con 0x7fe2b4159420 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2baabb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c9870 con 0x7fe2b415d230 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2b92b8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 0x7fe2b40cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2b92b8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 0x7fe2b40cf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48376/0 (socket says 172.21.15.92:48376) 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2b92b8700 1 -- 172.21.15.92:0/1756494901 learned_addr learned my addr 172.21.15.92:0/1756494901 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2b9ab9700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159420 0x7fe2b41551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.712+0000 7fe2ba2ba700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2b92b8700 1 -- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 msgr2=0x7fe2b40cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2b92b8700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b40cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2b92b8700 1 -- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159420 msgr2=0x7fe2b41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2b92b8700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159420 0x7fe2b41551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2b92b8700 1 -- 172.21.15.92:0/1756494901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009ae0 con 0x7fe2b41597f0 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2b92b8700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 0x7fe2b40cf950 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400c7f0 tx=0x7fe2a4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4011590 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a4011710 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b40c6300 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4011ad0 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.713+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b40c68c0 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.714+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b404c3c0 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.714+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe2a401b3b0 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.715+0000 7fe2a9ffb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe298067200 0x7fe2980696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.715+0000 7fe2b9ab9700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe298067200 0x7fe2980696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.715+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe2a4083f50 con 0x7fe2b41597f0 2023-10-06T03:07:05.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.715+0000 7fe2b9ab9700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe298067200 0x7fe2980696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009880 tx=0x7fe2b0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:05.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.719+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2a4050d10 con 0x7fe2b41597f0 2023-10-06T03:07:05.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.945+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe2b40c9ce0 con 0x7fe2b41597f0 2023-10-06T03:07:05.973 INFO:tasks.ceph.mds.d.smithi136.stderr: -1812> 2023-10-06T03:06:55.453+0000 7ffbdf7f3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:07:05.973 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:07:05.945+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.949+0000 7fe2a9ffb700 1 -- 172.21.15.92:0/1756494901 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe2a4017070 con 0x7fe2b41597f0 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe298067200 msgr2=0x7fe2980696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe298067200 0x7fe2980696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009880 tx=0x7fe2b0018000 comp rx=0 tx=0).stop 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 msgr2=0x7fe2b40cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 0x7fe2b40cf950 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400c7f0 tx=0x7fe2a4007f50 comp rx=0 tx=0).stop 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 shutdown_connections 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe298067200 0x7fe2980696b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b415d230 0x7fe2b40cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159420 0x7fe2b41551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 --2- 172.21.15.92:0/1756494901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b41597f0 0x7fe2b40cf950 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:05.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 >> 172.21.15.92:0/1756494901 conn(0x7fe2b40d0660 msgr2=0x7fe2b4156000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:05.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.951+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 shutdown_connections 2023-10-06T03:07:05.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:05.952+0000 7fe2baabb700 1 -- 172.21.15.92:0/1756494901 wait complete. 2023-10-06T03:07:05.976 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:07:05.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:07:05.978 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:07:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.092+0000 7f59560da700 1 Processor -- start 2023-10-06T03:07:06.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.092+0000 7f59560da700 1 -- start start 2023-10-06T03:07:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f59560da700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5950159c60 0x7f595015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f59560da700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f595015db10 0x7f5950069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f59560da700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595015a160 con 0x7f5950159890 2023-10-06T03:07:06.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f59560da700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595015df50 con 0x7f595015db10 2023-10-06T03:07:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f59560da700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500d13a0 con 0x7f5950159c60 2023-10-06T03:07:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f594ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f595015db10 0x7f5950069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f594ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f595015db10 0x7f5950069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48384/0 (socket says 172.21.15.92:48384) 2023-10-06T03:07:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.093+0000 7f594ffff700 1 -- 172.21.15.92:0/4027029934 learned_addr learned my addr 172.21.15.92:0/4027029934 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594effd700 1 --2- 172.21.15.92:0/4027029934 >> v2:172.21.15.136:3300/0 conn(0x7f5950159c60 0x7f595015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:06.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594ffff700 1 -- 172.21.15.92:0/4027029934 >> v2:172.21.15.136:3300/0 conn(0x7f5950159c60 msgr2=0x7f595015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594ffff700 1 --2- 172.21.15.92:0/4027029934 >> v2:172.21.15.136:3300/0 conn(0x7f5950159c60 0x7f595015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594ffff700 1 -- 172.21.15.92:0/4027029934 >> v1:172.21.15.136:6789/0 conn(0x7f5950159890 legacy=0x7f595015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594ffff700 1 -- 172.21.15.92:0/4027029934 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595015e140 con 0x7f595015db10 2023-10-06T03:07:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594ffff700 1 --2- 172.21.15.92:0/4027029934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f595015db10 0x7f5950069560 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f59500b67f0 tx=0x7f5948009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27917e7863c6e438 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594e7fc700 1 -- 172.21.15.92:0/4027029934 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948009e70 con 0x7f595015db10 2023-10-06T03:07:06.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594e7fc700 1 -- 172.21.15.92:0/4027029934 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800b860 con 0x7f595015db10 2023-10-06T03:07:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f594e7fc700 1 -- 172.21.15.92:0/4027029934 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800bc00 con 0x7f595015db10 2023-10-06T03:07:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f59560da700 1 -- 172.21.15.92:0/4027029934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f595015db10 msgr2=0x7f5950069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:06.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.094+0000 7f59560da700 1 --2- 172.21.15.92:0/4027029934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f595015db10 0x7f5950069560 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f59500b67f0 tx=0x7f5948009250 comp rx=0 tx=0).stop 2023-10-06T03:07:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.095+0000 7f59560da700 1 -- 172.21.15.92:0/4027029934 shutdown_connections 2023-10-06T03:07:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.095+0000 7f59560da700 1 --2- 172.21.15.92:0/4027029934 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f595015db10 0x7f5950069560 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.095+0000 7f59560da700 1 --2- 172.21.15.92:0/4027029934 >> v2:172.21.15.136:3300/0 conn(0x7f5950159c60 0x7f595015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.095+0000 7f59560da700 1 -- 172.21.15.92:0/4027029934 >> 172.21.15.92:0/4027029934 conn(0x7f5950068e20 msgr2=0x7f59500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.095+0000 7f59560da700 1 -- 172.21.15.92:0/4027029934 shutdown_connections 2023-10-06T03:07:06.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.095+0000 7f59560da700 1 -- 172.21.15.92:0/4027029934 wait complete. 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.096+0000 7f59560da700 1 Processor -- start 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.096+0000 7f59560da700 1 -- start start 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 0x7f59500cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595015db10 0x7f595015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500c64f0 con 0x7f5950159890 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500cd600 con 0x7f595015db10 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500c5730 con 0x7f5950159c60 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 0x7f59500cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 0x7f59500cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48392/0 (socket says 172.21.15.92:48392) 2023-10-06T03:07:06.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 -- 172.21.15.92:0/1555359737 learned_addr learned my addr 172.21.15.92:0/1555359737 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594ffff700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595015db10 0x7f595015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594effd700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 -- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 msgr2=0x7f595015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 -- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595015db10 msgr2=0x7f595015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595015db10 0x7f595015fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 -- 172.21.15.92:0/1555359737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480094f0 con 0x7f5950159890 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f594f7fe700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 0x7f59500cd0c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f594000cb10 tx=0x7f59400017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000ce20 con 0x7f5950159890 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594000a7e0 con 0x7f5950159890 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59500c5a10 con 0x7f5950159890 2023-10-06T03:07:06.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000aba0 con 0x7f5950159890 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.097+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59500c5fd0 con 0x7f5950159890 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.098+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59500c61f0 con 0x7f5950159890 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.099+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5940003030 con 0x7f5950159890 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.099+0000 7f59377fe700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067200 0x7f593c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.099+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5940084ac0 con 0x7f5950159890 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.099+0000 7f594effd700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067200 0x7f593c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.099+0000 7f594effd700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067200 0x7f593c0696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5944009db0 tx=0x7f59440091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:06.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.103+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5940051830 con 0x7f5950159890 2023-10-06T03:07:06.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.332+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f595004c3c0 con 0x7f5950159890 2023-10-06T03:07:06.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.950+0000 7f59377fe700 1 -- 172.21.15.92:0/1555359737 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5940054e50 con 0x7f5950159890 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067200 msgr2=0x7f593c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067200 0x7f593c0696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5944009db0 tx=0x7f59440091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 msgr2=0x7f59500cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 0x7f59500cd0c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f594000cb10 tx=0x7f59400017b0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 shutdown_connections 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067200 0x7f593c0696b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950159890 0x7f59500cd0c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f595015db10 0x7f595015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 --2- 172.21.15.92:0/1555359737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 >> 172.21.15.92:0/1555359737 conn(0x7f5950068e20 msgr2=0x7f5950154f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 shutdown_connections 2023-10-06T03:07:06.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:06.953+0000 7f59560da700 1 -- 172.21.15.92:0/1555359737 wait complete. 2023-10-06T03:07:06.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:07:07.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.091+0000 7ff93fa97700 1 Processor -- start 2023-10-06T03:07:07.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.092+0000 7ff93fa97700 1 -- start start 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.092+0000 7ff93fa97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff938159ec0 0x7ff9380b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.092+0000 7ff93fa97700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380b5b00 con 0x7ff9380b53d0 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.092+0000 7ff93fa97700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380b5850 con 0x7ff9380cf350 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.092+0000 7ff93fa97700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380b5da0 con 0x7ff938159ec0 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.092+0000 7ff93d833700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff938159ec0 0x7ff9380b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d032700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff9380b53d0 0x7ff9380cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52006/0 (socket says 172.21.15.92:52006) 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d833700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff938159ec0 0x7ff9380b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48398/0 (socket says 172.21.15.92:48398) 2023-10-06T03:07:07.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d032700 1 -- 172.21.15.92:0/4213832040 learned_addr learned my addr 172.21.15.92:0/4213832040 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d833700 1 -- 172.21.15.92:0/4213832040 >> v1:172.21.15.136:6789/0 conn(0x7ff9380cf350 legacy=0x7ff9380cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d833700 1 -- 172.21.15.92:0/4213832040 >> v1:172.21.15.92:6789/0 conn(0x7ff9380b53d0 legacy=0x7ff9380cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d833700 1 -- 172.21.15.92:0/4213832040 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380b6080 con 0x7ff938159ec0 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93c831700 1 -- 172.21.15.92:0/4213832040 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224103488 0 0) 0x7ff9380b5b00 con 0x7ff9380b53d0 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93d833700 1 --2- 172.21.15.92:0/4213832040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff938159ec0 0x7ff9380b4e90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff9300098d0 tx=0x7ff930009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856be323466ce584 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93c831700 1 -- 172.21.15.92:0/4213832040 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930009e70 con 0x7ff938159ec0 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93c831700 1 -- 172.21.15.92:0/4213832040 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930005c10 con 0x7ff938159ec0 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.093+0000 7ff93c831700 1 -- 172.21.15.92:0/4213832040 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000b610 con 0x7ff938159ec0 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 -- 172.21.15.92:0/4213832040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff938159ec0 msgr2=0x7ff9380b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 --2- 172.21.15.92:0/4213832040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff938159ec0 0x7ff9380b4e90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff9300098d0 tx=0x7ff930009120 comp rx=0 tx=0).stop 2023-10-06T03:07:07.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 -- 172.21.15.92:0/4213832040 shutdown_connections 2023-10-06T03:07:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 --2- 172.21.15.92:0/4213832040 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff938159ec0 0x7ff9380b4e90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 -- 172.21.15.92:0/4213832040 >> 172.21.15.92:0/4213832040 conn(0x7ff9380999f0 msgr2=0x7ff9380b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 -- 172.21.15.92:0/4213832040 shutdown_connections 2023-10-06T03:07:07.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.094+0000 7ff93fa97700 1 -- 172.21.15.92:0/4213832040 wait complete. 2023-10-06T03:07:07.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.095+0000 7ff93fa97700 1 Processor -- start 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.095+0000 7ff93fa97700 1 -- start start 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.095+0000 7ff93fa97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 0x7ff938156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9380cf350 0x7ff938069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff938159ec0 0x7ff9380698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93806aac0 con 0x7ff9380b53d0 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938157470 con 0x7ff9380cf350 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93816a130 con 0x7ff938159ec0 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 0x7ff938156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 0x7ff938156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48408/0 (socket says 172.21.15.92:48408) 2023-10-06T03:07:07.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 -- 172.21.15.92:0/2441233952 learned_addr learned my addr 172.21.15.92:0/2441233952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d032700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9380cf350 0x7ff938069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93e034700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff938159ec0 0x7ff9380698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 -- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff938159ec0 msgr2=0x7ff9380698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff938159ec0 0x7ff9380698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 -- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9380cf350 msgr2=0x7ff938069360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9380cf350 0x7ff938069360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 -- 172.21.15.92:0/2441233952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300093c0 con 0x7ff9380b53d0 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d032700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9380cf350 0x7ff938069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93e034700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff938159ec0 0x7ff9380698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93d833700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 0x7ff938156f30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff930009160 tx=0x7ff930015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930006640 con 0x7ff9380b53d0 2023-10-06T03:07:07.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9300067a0 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300266c0 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff938069eb0 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.096+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93806a470 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.097+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff938063c90 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.098+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff930023070 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.098+0000 7ff935ffb700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff924067200 0x7ff9240696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.098+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6144+0+0 (secure 0 0 0) 0x7ff9300870b0 con 0x7ff9380b53d0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.098+0000 7ff93d032700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff924067200 0x7ff9240696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:07.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.098+0000 7ff93d032700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff924067200 0x7ff9240696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff93809cbb0 tx=0x7ff92c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:07.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.102+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff930053d40 con 0x7ff9380b53d0 2023-10-06T03:07:07.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.288+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff9380718b0 con 0x7ff9380b53d0 2023-10-06T03:07:07.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.961+0000 7ff935ffb700 1 -- 172.21.15.92:0/2441233952 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff930057360 con 0x7ff9380b53d0 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff924067200 msgr2=0x7ff9240696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff924067200 0x7ff9240696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff93809cbb0 tx=0x7ff92c009040 comp rx=0 tx=0).stop 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 msgr2=0x7ff938156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 0x7ff938156f30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff930009160 tx=0x7ff930015000 comp rx=0 tx=0).stop 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 shutdown_connections 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff924067200 0x7ff9240696b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9380b53d0 0x7ff938156f30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff938159ec0 0x7ff9380698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 --2- 172.21.15.92:0/2441233952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9380cf350 0x7ff938069360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:07.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 >> 172.21.15.92:0/2441233952 conn(0x7ff9380999f0 msgr2=0x7ff9380b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:07.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 shutdown_connections 2023-10-06T03:07:07.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:07.964+0000 7ff93fa97700 1 -- 172.21.15.92:0/2441233952 wait complete. 2023-10-06T03:07:07.967 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:07:07.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:07:08.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.096+0000 7f1b81fc1700 1 Processor -- start 2023-10-06T03:07:08.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.096+0000 7f1b81fc1700 1 -- start start 2023-10-06T03:07:08.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b81fc1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:08.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b81fc1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1b7c0b53d0 0x7f1b7c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:08.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b81fc1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7c0b5b00 con 0x7f1b7c0cce40 2023-10-06T03:07:08.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b81fc1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7c0b5da0 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b81fc1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c0b6080 con 0x7f1b7c159ec0 2023-10-06T03:07:08.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b7bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b7c0cce40 0x7f1b7c0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52020/0 (socket says 172.21.15.92:52020) 2023-10-06T03:07:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.097+0000 7f1b7bfff700 1 -- 172.21.15.92:0/4067757724 learned_addr learned my addr 172.21.15.92:0/4067757724 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7affd700 1 --2- 172.21.15.92:0/4067757724 >> v2:172.21.15.136:3300/0 conn(0x7f1b7c0b53d0 0x7f1b7c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4067757724 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679157359 0 0) 0x7f1b7c0b5b00 con 0x7f1b7c0cce40 2023-10-06T03:07:08.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4067757724 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b64003570 con 0x7f1b7c0cce40 2023-10-06T03:07:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7affd700 1 -- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 msgr2=0x7f1b7c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7affd700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4067757724 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218823794 0 0) 0x7f1b64003570 con 0x7f1b7c0cce40 2023-10-06T03:07:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7affd700 1 -- 172.21.15.92:0/4067757724 >> v1:172.21.15.92:6789/0 conn(0x7f1b7c0cce40 legacy=0x7f1b7c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7affd700 1 -- 172.21.15.92:0/4067757724 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c0cc100 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7affd700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c0cc900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1b700098e0 tx=0x7f1b700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f132846e12625b57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4067757724 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74001470 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4067757724 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b70011420 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.098+0000 7f1b7a7fc700 1 -- 172.21.15.92:0/4067757724 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700117a0 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 -- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0b53d0 msgr2=0x7f1b7c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c0cc900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1b700098e0 tx=0x7f1b700091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 -- 172.21.15.92:0/4067757724 shutdown_connections 2023-10-06T03:07:08.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/4067757724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c0cc900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 -- 172.21.15.92:0/4067757724 >> 172.21.15.92:0/4067757724 conn(0x7f1b7c0999f0 msgr2=0x7f1b7c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:08.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 -- 172.21.15.92:0/4067757724 shutdown_connections 2023-10-06T03:07:08.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.099+0000 7f1b81fc1700 1 -- 172.21.15.92:0/4067757724 wait complete. 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.100+0000 7f1b81fc1700 1 Processor -- start 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.100+0000 7f1b81fc1700 1 -- start start 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b81fc1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b81fc1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0cce40 0x7f1b7c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b81fc1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b81fc1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c169db0 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b81fc1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c16a010 con 0x7f1b7c0cce40 2023-10-06T03:07:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b81fc1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c156cc0 con 0x7f1b7c159ec0 2023-10-06T03:07:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48416/0 (socket says 172.21.15.92:48416) 2023-10-06T03:07:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/1308918069 learned_addr learned my addr 172.21.15.92:0/1308918069 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7affd700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0cce40 0x7f1b7c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7bfff700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 msgr2=0x7f1b7c0cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0cce40 msgr2=0x7f1b7c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0cce40 0x7f1b7c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 -- 172.21.15.92:0/1308918069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c009a20 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7bfff700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.101+0000 7f1b7b7fe700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c152150 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c1f0 tx=0x7f1b6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.102+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c00ce30 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.102+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c00f040 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.102+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b70009420 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.102+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c014690 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.102+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7c16a590 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.103+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b7c063c90 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.103+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f1b6c014a00 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.103+0000 7f1b637fe700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b68067200 0x7f1b680696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:08.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.104+0000 7f1b7affd700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b68067200 0x7f1b680696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:08.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.104+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f1b6c085af0 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.104+0000 7f1b7affd700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b68067200 0x7f1b680696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b700091e0 tx=0x7f1b70016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:08.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.108+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b6c052900 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.290+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b7c0718b0 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.964+0000 7f1b637fe700 1 -- 172.21.15.92:0/1308918069 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1b6c014cb0 con 0x7f1b7c0b53d0 2023-10-06T03:07:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.967+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b68067200 msgr2=0x7f1b680696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.967+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b68067200 0x7f1b680696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1b700091e0 tx=0x7f1b70016000 comp rx=0 tx=0).stop 2023-10-06T03:07:08.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.967+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 msgr2=0x7f1b7c152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.967+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c152150 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00c1f0 tx=0x7f1b6c00b040 comp rx=0 tx=0).stop 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 shutdown_connections 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b68067200 0x7f1b680696b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b7c0b53d0 0x7f1b7c152150 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b7c159ec0 0x7f1b7c0cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 --2- 172.21.15.92:0/1308918069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b7c0cce40 0x7f1b7c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 >> 172.21.15.92:0/1308918069 conn(0x7f1b7c0999f0 msgr2=0x7f1b7c0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 shutdown_connections 2023-10-06T03:07:08.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:08.968+0000 7f1b81fc1700 1 -- 172.21.15.92:0/1308918069 wait complete. 2023-10-06T03:07:08.971 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:07:08.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:07:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.098+0000 7fa11d542700 1 Processor -- start 2023-10-06T03:07:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.098+0000 7fa11d542700 1 -- start start 2023-10-06T03:07:09.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa11d542700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa11806bcd0 0x7fa1180caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa11d542700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa11815df50 0x7fa11815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa11d542700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11815d7e0 con 0x7fa11806c0a0 2023-10-06T03:07:09.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa11d542700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11815dac0 con 0x7fa11815df50 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa11d542700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11815aa30 con 0x7fa11806bcd0 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa1177fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa11815df50 0x7fa11815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa10e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa11806c0a0 0x7fa11815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52024/0 (socket says 172.21.15.92:52024) 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa10e7fc700 1 -- 172.21.15.92:0/3104525431 learned_addr learned my addr 172.21.15.92:0/3104525431 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.099+0000 7fa116ffd700 1 --2- 172.21.15.92:0/3104525431 >> v2:172.21.15.136:3300/0 conn(0x7fa11806bcd0 0x7fa1180caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1177fe700 1 -- 172.21.15.92:0/3104525431 >> v2:172.21.15.136:3300/0 conn(0x7fa11806bcd0 msgr2=0x7fa1180caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1177fe700 1 --2- 172.21.15.92:0/3104525431 >> v2:172.21.15.136:3300/0 conn(0x7fa11806bcd0 0x7fa1180caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1177fe700 1 -- 172.21.15.92:0/3104525431 >> v1:172.21.15.92:6789/0 conn(0x7fa11806c0a0 legacy=0x7fa11815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1167fc700 1 -- 172.21.15.92:0/3104525431 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381803823 0 0) 0x7fa11815d7e0 con 0x7fa11806c0a0 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1177fe700 1 -- 172.21.15.92:0/3104525431 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11815ad10 con 0x7fa11815df50 2023-10-06T03:07:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1177fe700 1 --2- 172.21.15.92:0/3104525431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11815df50 0x7fa11815e3b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa1180691a0 tx=0x7fa110009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e4ab6279ce4d99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1167fc700 1 -- 172.21.15.92:0/3104525431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11000e070 con 0x7fa11815df50 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1167fc700 1 -- 172.21.15.92:0/3104525431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11000bb60 con 0x7fa11815df50 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa1167fc700 1 -- 172.21.15.92:0/3104525431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110012430 con 0x7fa11815df50 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa11d542700 1 -- 172.21.15.92:0/3104525431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11815df50 msgr2=0x7fa11815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa11d542700 1 --2- 172.21.15.92:0/3104525431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11815df50 0x7fa11815e3b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa1180691a0 tx=0x7fa110009780 comp rx=0 tx=0).stop 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa11d542700 1 -- 172.21.15.92:0/3104525431 shutdown_connections 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa11d542700 1 --2- 172.21.15.92:0/3104525431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11815df50 0x7fa11815e3b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa11d542700 1 --2- 172.21.15.92:0/3104525431 >> v2:172.21.15.136:3300/0 conn(0x7fa11806bcd0 0x7fa1180caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.100+0000 7fa11d542700 1 -- 172.21.15.92:0/3104525431 >> 172.21.15.92:0/3104525431 conn(0x7fa1180ce600 msgr2=0x7fa1180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.101+0000 7fa11d542700 1 -- 172.21.15.92:0/3104525431 shutdown_connections 2023-10-06T03:07:09.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.101+0000 7fa11d542700 1 -- 172.21.15.92:0/3104525431 wait complete. 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 Processor -- start 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 -- start start 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa11806bcd0 0x7fa11816caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 0x7fa118169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa11815df50 0x7fa1180c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180b1710 con 0x7fa11806c0a0 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11816d030 con 0x7fa11815df50 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.102+0000 7fa11d542700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11816a540 con 0x7fa11806bcd0 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 0x7fa118169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 0x7fa118169c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:48434/0 (socket says 172.21.15.92:48434) 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 -- 172.21.15.92:0/471049601 learned_addr learned my addr 172.21.15.92:0/471049601 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa116ffd700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa11806bcd0 0x7fa11816caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa1177fe700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa11815df50 0x7fa1180c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 -- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa11806bcd0 msgr2=0x7fa11816caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa11806bcd0 0x7fa11816caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 -- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa11815df50 msgr2=0x7fa1180c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa11815df50 0x7fa1180c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 -- 172.21.15.92:0/471049601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110009a20 con 0x7fa11806c0a0 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa1177fe700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa11815df50 0x7fa1180c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10e7fc700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 0x7fa118169c80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c5e0 tx=0x7fa100001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:09.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100005b30 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa100005c90 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11816a820 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100007300 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.103+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1180b13a0 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.104+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1180b18c0 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.105+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa10001b3b0 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.105+0000 7fa10f7fe700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa0f8067200 0x7fa0f80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.105+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa100083cc0 con 0x7fa11806c0a0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.105+0000 7fa116ffd700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa0f8067200 0x7fa0f80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.105+0000 7fa116ffd700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa0f8067200 0x7fa0f80696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa11806ae30 tx=0x7fa1080091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:09.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.109+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1180b18c0 con 0x7fa11806c0a0 2023-10-06T03:07:09.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.306+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa1180b18c0 con 0x7fa11806c0a0 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.309+0000 7fa10f7fe700 1 -- 172.21.15.92:0/471049601 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa1180b18c0 con 0x7fa11806c0a0 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa0f8067200 msgr2=0x7fa0f80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa0f8067200 0x7fa0f80696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa11806ae30 tx=0x7fa1080091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 msgr2=0x7fa118169c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 0x7fa118169c80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c5e0 tx=0x7fa100001070 comp rx=0 tx=0).stop 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 shutdown_connections 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa0f8067200 0x7fa0f80696b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa11815df50 0x7fa1180c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa11806bcd0 0x7fa11816caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 --2- 172.21.15.92:0/471049601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa11806c0a0 0x7fa118169c80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 >> 172.21.15.92:0/471049601 conn(0x7fa1180ce600 msgr2=0x7fa118155350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:09.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 shutdown_connections 2023-10-06T03:07:09.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.311+0000 7fa11d542700 1 -- 172.21.15.92:0/471049601 wait complete. 2023-10-06T03:07:09.325 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-06T03:07:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.438+0000 7f5fc77d5700 1 Processor -- start 2023-10-06T03:07:09.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.439+0000 7f5fc77d5700 1 -- start start 2023-10-06T03:07:09.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc77d5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc77d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc77d5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc015a160 con 0x7f5fc0159890 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc77d5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc015df50 con 0x7f5fc0159c60 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc77d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00d13a0 con 0x7f5fc015db10 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc4d70700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:43702/0 (socket says 172.21.15.92:43702) 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 -- 172.21.15.92:0/285388638 learned_addr learned my addr 172.21.15.92:0/285388638 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5571700 1 --1- 172.21.15.92:0/285388638 >> v1:172.21.15.136:6789/0 conn(0x7f5fc0159890 0x7f5fc015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:58556/0 (socket says 172.21.15.92:58556) 2023-10-06T03:07:09.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 -- 172.21.15.92:0/285388638 >> v2:172.21.15.136:3300/0 conn(0x7f5fc0159c60 msgr2=0x7f5fc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 --2- 172.21.15.92:0/285388638 >> v2:172.21.15.136:3300/0 conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 -- 172.21.15.92:0/285388638 >> v1:172.21.15.136:6789/0 conn(0x7f5fc0159890 legacy=0x7f5fc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 -- 172.21.15.92:0/285388638 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc015e140 con 0x7f5fc015db10 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.440+0000 7f5fc5d72700 1 --2- 172.21.15.92:0/285388638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 0x7f5fc0069560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80099b0 tx=0x7f5fb8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6003765586c0dc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fb7fff700 1 -- 172.21.15.92:0/285388638 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8009ea0 con 0x7f5fc015db10 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fb7fff700 1 -- 172.21.15.92:0/285388638 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb800b860 con 0x7f5fc015db10 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fb7fff700 1 -- 172.21.15.92:0/285388638 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb800bc00 con 0x7f5fc015db10 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 -- 172.21.15.92:0/285388638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 msgr2=0x7f5fc0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/285388638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 0x7f5fc0069560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80099b0 tx=0x7f5fb8009250 comp rx=0 tx=0).stop 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 -- 172.21.15.92:0/285388638 shutdown_connections 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/285388638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc015db10 0x7f5fc0069560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/285388638 >> v2:172.21.15.136:3300/0 conn(0x7f5fc0159c60 0x7f5fc015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 -- 172.21.15.92:0/285388638 >> 172.21.15.92:0/285388638 conn(0x7f5fc0068e20 msgr2=0x7f5fc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 -- 172.21.15.92:0/285388638 shutdown_connections 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.441+0000 7f5fc77d5700 1 -- 172.21.15.92:0/285388638 wait complete. 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.442+0000 7f5fc77d5700 1 Processor -- start 2023-10-06T03:07:09.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.442+0000 7f5fc77d5700 1 -- start start 2023-10-06T03:07:09.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc77d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc0159890 0x7f5fc014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc77d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 0x7f5fc0151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc77d5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc015db10 0x7f5fc0154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc77d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0151b80 con 0x7f5fc0159c60 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc77d5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0151e60 con 0x7f5fc015db10 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc77d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0152140 con 0x7f5fc0159890 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc4d70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 0x7f5fc0151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc4d70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 0x7f5fc0151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50290/0 (socket says 172.21.15.92:50290) 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc4d70700 1 -- 172.21.15.92:0/1931962828 learned_addr learned my addr 172.21.15.92:0/1931962828 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc5571700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc0159890 0x7f5fc014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.443+0000 7f5fc5d72700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc015db10 0x7f5fc0154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc4d70700 1 -- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc0159890 msgr2=0x7f5fc014c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc4d70700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc0159890 0x7f5fc014c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc4d70700 1 -- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc015db10 msgr2=0x7f5fc0154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc4d70700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc015db10 0x7f5fc0154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc4d70700 1 -- 172.21.15.92:0/1931962828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb80094f0 con 0x7f5fc0159c60 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc4d70700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 0x7f5fc0151280 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000ec80 tx=0x7f5fb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000c850 con 0x7f5fc0159c60 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb000c9b0 con 0x7f5fc0159c60 2023-10-06T03:07:09.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc0152420 con 0x7f5fc0159c60 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000cd50 con 0x7f5fc0159c60 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.444+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc01517c0 con 0x7f5fc0159c60 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.445+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc00cd150 con 0x7f5fc0159c60 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.445+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5fb0013680 con 0x7f5fc0159c60 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.446+0000 7f5fb57fa700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fa4067200 0x7f5fa40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.446+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f5fb0084a10 con 0x7f5fc0159c60 2023-10-06T03:07:09.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.446+0000 7f5fc5571700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fa4067200 0x7f5fa40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.446+0000 7f5fc5571700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fa4067200 0x7f5fa40696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0020c0 tx=0x7f5fbc001ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.450+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fc00cd150 con 0x7f5fc0159c60 2023-10-06T03:07:09.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.707+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7f5fc00cd150 con 0x7f5fc0159c60 2023-10-06T03:07:09.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.964+0000 7f5fb57fa700 1 -- 172.21.15.92:0/1931962828 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v261) v1 ==== 158+0+0 (secure 0 0 0) 0x7f5fc00cd150 con 0x7f5fc0159c60 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fa4067200 msgr2=0x7f5fa40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fa4067200 0x7f5fa40696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc0020c0 tx=0x7f5fbc001ad0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 msgr2=0x7f5fc0151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 0x7f5fc0151280 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000ec80 tx=0x7f5fb00091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 shutdown_connections 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fa4067200 0x7f5fa40696b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fc015db10 0x7f5fc0154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fc0159890 0x7f5fc014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 --2- 172.21.15.92:0/1931962828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fc0159c60 0x7f5fc0151280 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 >> 172.21.15.92:0/1931962828 conn(0x7f5fc0068e20 msgr2=0x7f5fc0157300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 shutdown_connections 2023-10-06T03:07:09.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:09.967+0000 7f5fc77d5700 1 -- 172.21.15.92:0/1931962828 wait complete. 2023-10-06T03:07:09.981 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command ... ok 2023-10-06T03:07:09.981 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command without subvolume group 2023-10-06T03:07:09.981 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:07:09.981 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-06T03:07:10.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.097+0000 7f7138dac700 1 Processor -- start 2023-10-06T03:07:10.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.097+0000 7f7138dac700 1 -- start start 2023-10-06T03:07:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.097+0000 7f7138dac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 0x7f713415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.097+0000 7f7138dac700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713415a160 con 0x7f713415db10 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.097+0000 7f7138dac700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340d13a0 con 0x7f7134159890 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.097+0000 7f7138dac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713415e140 con 0x7f7134159c60 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7132d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f713415db10 0x7f7134069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40716/0 (socket says 172.21.15.92:40716) 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7132d9d700 1 -- 172.21.15.92:0/2149684234 learned_addr learned my addr 172.21.15.92:0/2149684234 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201374345 0 0) 0x7f713415a160 con 0x7f713415db10 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f712bfff700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 0x7f713415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7120003570 con 0x7f713415db10 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347757335 0 0) 0x7f71340d13a0 con 0x7f7134159890 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f713415a160 con 0x7f7134159890 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 801253217 0 0) 0x7f7120003570 con 0x7f713415db10 2023-10-06T03:07:10.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 msgr2=0x7f713415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 0x7f713415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 >> v1:172.21.15.136:6789/0 conn(0x7f7134159890 legacy=0x7f713415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713415a3d0 con 0x7f713415db10 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f712c0017f0 con 0x7f713415db10 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 >> v1:172.21.15.92:6789/0 conn(0x7f713415db10 legacy=0x7f7134069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f7120004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120004810 0x7f7120004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 0x7f7120005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713415e140 con 0x7f7120003c20 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712000bbf0 con 0x7f7120004810 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712000bdb0 con 0x7f7134159c60 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f713259c700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f7120004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:10.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f712bfff700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120004810 0x7f7120004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.098+0000 7f7132d9d700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 0x7f7120005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f713259c700 1 -- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 msgr2=0x7f7120005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f713259c700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7134159c60 0x7f7120005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f713259c700 1 -- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120004810 msgr2=0x7f7120004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f713259c700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120004810 0x7f7120004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f713259c700 1 -- 172.21.15.92:0/2149684234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120003570 con 0x7f7120003c20 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f712bfff700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120004810 0x7f7120004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f713259c700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f7120004010 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7124001a10 tx=0x7f7124007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8644f8ea9b52488e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400c5d0 con 0x7f7120003c20 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712400c730 con 0x7f7120003c20 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7131d9b700 1 -- 172.21.15.92:0/2149684234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400cad0 con 0x7f7120003c20 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7138dac700 1 -- 172.21.15.92:0/2149684234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 msgr2=0x7f7120004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7138dac700 1 --2- 172.21.15.92:0/2149684234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f7120004010 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7124001a10 tx=0x7f7124007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7132d9d700 1 -- 172.21.15.92:0/2149684234 reap_dead start 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7138dac700 1 -- 172.21.15.92:0/2149684234 shutdown_connections 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7138dac700 1 -- 172.21.15.92:0/2149684234 >> 172.21.15.92:0/2149684234 conn(0x7f7134068e20 msgr2=0x7f71340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7138dac700 1 -- 172.21.15.92:0/2149684234 shutdown_connections 2023-10-06T03:07:10.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.099+0000 7f7138dac700 1 -- 172.21.15.92:0/2149684234 wait complete. 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.100+0000 7f7138dac700 1 Processor -- start 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 -- start start 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f713413e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71340ca3a0 0x7f71340b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71340b6120 0x7f71340b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713413e600 con 0x7f7120003c20 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c9f10 con 0x7f71340ca3a0 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f7138dac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340cfec0 con 0x7f71340b6120 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f713259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f713413e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:10.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f713259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f713413e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50298/0 (socket says 172.21.15.92:50298) 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.101+0000 7f713259c700 1 -- 172.21.15.92:0/1428434341 learned_addr learned my addr 172.21.15.92:0/1428434341 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f713259c700 1 -- 172.21.15.92:0/1428434341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71340b6120 msgr2=0x7f71340b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f713259c700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71340b6120 0x7f71340b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f713259c700 1 -- 172.21.15.92:0/1428434341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71340ca3a0 msgr2=0x7f71340b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f713259c700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71340ca3a0 0x7f71340b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f713259c700 1 -- 172.21.15.92:0/1428434341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340c95e0 con 0x7f7120003c20 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f713259c700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f713413e0c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f712400c3c0 tx=0x7f7124015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400c5d0 con 0x7f7120003c20 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712400cab0 con 0x7f7120003c20 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340c98c0 con 0x7f7120003c20 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124006a50 con 0x7f7120003c20 2023-10-06T03:07:10.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.102+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71340cf370 con 0x7f7120003c20 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.103+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713404c3c0 con 0x7f7120003c20 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.103+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7124019070 con 0x7f7120003c20 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.103+0000 7f712a7fc700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7114067230 0x7f71140696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.104+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7124085c60 con 0x7f7120003c20 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.104+0000 7f712bfff700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7114067230 0x7f71140696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.104+0000 7f712bfff700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7114067230 0x7f71140696e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71340b3a90 tx=0x7f711c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:10.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.108+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7124052d20 con 0x7f7120003c20 2023-10-06T03:07:10.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.364+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7f713406a5b0 con 0x7f7120003c20 2023-10-06T03:07:10.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.966+0000 7f712a7fc700 1 -- 172.21.15.92:0/1428434341 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v262) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7124056340 con 0x7f7120003c20 2023-10-06T03:07:10.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7114067230 msgr2=0x7f71140696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7114067230 0x7f71140696e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71340b3a90 tx=0x7f711c008040 comp rx=0 tx=0).stop 2023-10-06T03:07:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 msgr2=0x7f713413e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f713413e0c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f712400c3c0 tx=0x7f7124015000 comp rx=0 tx=0).stop 2023-10-06T03:07:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 shutdown_connections 2023-10-06T03:07:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7114067230 0x7f71140696e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120003c20 0x7f713413e0c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71340b6120 0x7f71340b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 --2- 172.21.15.92:0/1428434341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71340ca3a0 0x7f71340b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:10.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 >> 172.21.15.92:0/1428434341 conn(0x7f7134068e20 msgr2=0x7f71340b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:10.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.968+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 shutdown_connections 2023-10-06T03:07:10.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:10.969+0000 7f7138dac700 1 -- 172.21.15.92:0/1428434341 wait complete. 2023-10-06T03:07:10.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:07:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.098+0000 7f059ea68700 1 Processor -- start 2023-10-06T03:07:11.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.099+0000 7f059ea68700 1 -- start start 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.099+0000 7f059ea68700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0598159890 0x7f059815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.099+0000 7f059ea68700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05980699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.099+0000 7f059ea68700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0598069ee0 con 0x7f0598159c60 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.099+0000 7f059ea68700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059815a160 con 0x7f0598159890 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.099+0000 7f059ea68700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980d13a0 con 0x7f0598069560 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0598159890 0x7f059815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0598159890 0x7f059815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50312/0 (socket says 172.21.15.92:50312) 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 -- 172.21.15.92:0/2863702710 learned_addr learned my addr 172.21.15.92:0/2863702710 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059d005700 1 --2- 172.21.15.92:0/2863702710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05980699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 -- 172.21.15.92:0/2863702710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 msgr2=0x7f05980699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 --2- 172.21.15.92:0/2863702710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05980699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 -- 172.21.15.92:0/2863702710 >> v1:172.21.15.136:6789/0 conn(0x7f0598159c60 legacy=0x7f059815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 -- 172.21.15.92:0/2863702710 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059815e140 con 0x7f0598159890 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f059c804700 1 --2- 172.21.15.92:0/2863702710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f059815abd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0594009f20 tx=0x7f0594009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f43d0161d592c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f058f7fe700 1 -- 172.21.15.92:0/2863702710 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400e070 con 0x7f0598159890 2023-10-06T03:07:11.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f058f7fe700 1 -- 172.21.15.92:0/2863702710 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059400b9d0 con 0x7f0598159890 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.100+0000 7f058f7fe700 1 -- 172.21.15.92:0/2863702710 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940092e0 con 0x7f0598159890 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 -- 172.21.15.92:0/2863702710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 msgr2=0x7f059815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 --2- 172.21.15.92:0/2863702710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f059815abd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0594009f20 tx=0x7f0594009840 comp rx=0 tx=0).stop 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 -- 172.21.15.92:0/2863702710 shutdown_connections 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 --2- 172.21.15.92:0/2863702710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f059815abd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 --2- 172.21.15.92:0/2863702710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05980699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 -- 172.21.15.92:0/2863702710 >> 172.21.15.92:0/2863702710 conn(0x7f0598068e20 msgr2=0x7f05980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 -- 172.21.15.92:0/2863702710 shutdown_connections 2023-10-06T03:07:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.101+0000 7f059ea68700 1 -- 172.21.15.92:0/2863702710 wait complete. 2023-10-06T03:07:11.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.102+0000 7f059ea68700 1 Processor -- start 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.102+0000 7f059ea68700 1 -- start start 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059ea68700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05981471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059ea68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f05980b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059ea68700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0598159c60 0x7f05980b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059ea68700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980c78a0 con 0x7f0598159890 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059ea68700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980b41c0 con 0x7f0598159c60 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059ea68700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980b44a0 con 0x7f0598069560 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f05980b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f05980b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50314/0 (socket says 172.21.15.92:50314) 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 -- 172.21.15.92:0/1226433393 learned_addr learned my addr 172.21.15.92:0/1226433393 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 -- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 msgr2=0x7f05981471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059c804700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05981471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f059d005700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0598159c60 0x7f05980b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05981471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 -- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0598159c60 msgr2=0x7f05980b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0598159c60 0x7f05980b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 -- 172.21.15.92:0/1226433393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594009ae0 con 0x7f0598159890 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.103+0000 7f058ffff700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f05980b4e00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f058800c590 tx=0x7f058800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.104+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588011590 con 0x7f0598159890 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.104+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05880116f0 con 0x7f0598159890 2023-10-06T03:07:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.104+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05980c7af0 con 0x7f0598159890 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.104+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588011ab0 con 0x7f0598159890 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.104+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0598147730 con 0x7f0598159890 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.105+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059804c3c0 con 0x7f0598159890 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.105+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f058801b3b0 con 0x7f0598159890 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.105+0000 7f058cff9700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0584067200 0x7f05840696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.105+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0588083e60 con 0x7f0598159890 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.105+0000 7f059c804700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0584067200 0x7f05840696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.106+0000 7f059c804700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0584067200 0x7f05840696b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0594009880 tx=0x7f0594018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.109+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0588050e00 con 0x7f0598159890 2023-10-06T03:07:11.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.295+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f059806a5b0 con 0x7f0598159890 2023-10-06T03:07:11.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.299+0000 7f058cff9700 1 -- 172.21.15.92:0/1226433393 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0588017020 con 0x7f0598159890 2023-10-06T03:07:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0584067200 msgr2=0x7f05840696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0584067200 0x7f05840696b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0594009880 tx=0x7f0594018000 comp rx=0 tx=0).stop 2023-10-06T03:07:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 msgr2=0x7f05980b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f05980b4e00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f058800c590 tx=0x7f058800c950 comp rx=0 tx=0).stop 2023-10-06T03:07:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 shutdown_connections 2023-10-06T03:07:11.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0584067200 0x7f05840696b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0598159c60 0x7f05980b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0598069560 0x7f05981471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 --2- 172.21.15.92:0/1226433393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0598159890 0x7f05980b4e00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 >> 172.21.15.92:0/1226433393 conn(0x7f0598068e20 msgr2=0x7f05980b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 shutdown_connections 2023-10-06T03:07:11.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.301+0000 7f059ea68700 1 -- 172.21.15.92:0/1226433393 wait complete. 2023-10-06T03:07:11.315 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:07:11.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:11.351 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:11.393 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:07:11.393 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:07:11.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:11.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.507+0000 7ff056474700 1 Processor -- start 2023-10-06T03:07:11.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.507+0000 7ff056474700 1 -- start start 2023-10-06T03:07:11.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff056474700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff050159890 0x7ff05015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff056474700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05015a160 con 0x7ff050069560 2023-10-06T03:07:11.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff056474700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500d13a0 con 0x7ff050159c60 2023-10-06T03:07:11.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff056474700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05015e140 con 0x7ff050159890 2023-10-06T03:07:11.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff054a11700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff050069560 0x7ff0500ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40720/0 (socket says 172.21.15.92:40720) 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff054a11700 1 -- 172.21.15.92:0/195628308 learned_addr learned my addr 172.21.15.92:0/195628308 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04ffff700 1 --2- 172.21.15.92:0/195628308 >> v2:172.21.15.136:3300/0 conn(0x7ff050159890 0x7ff05015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236619306 0 0) 0x7ff05015a160 con 0x7ff050069560 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff038003570 con 0x7ff050069560 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216020657 0 0) 0x7ff038003570 con 0x7ff050069560 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 >> v2:172.21.15.136:3300/0 conn(0x7ff050159890 msgr2=0x7ff05015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 --2- 172.21.15.92:0/195628308 >> v2:172.21.15.136:3300/0 conn(0x7ff050159890 0x7ff05015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 >> v1:172.21.15.136:6789/0 conn(0x7ff050159c60 legacy=0x7ff05015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05015a3d0 con 0x7ff050069560 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7ff0480017f0 con 0x7ff050069560 2023-10-06T03:07:11.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 >> v1:172.21.15.92:6789/0 conn(0x7ff050069560 legacy=0x7ff0500ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff038003a60 0x7ff038003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038004810 0x7ff038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0380051c0 0x7ff03800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05015e140 con 0x7ff038004810 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03800c0d0 con 0x7ff038003a60 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03800c290 con 0x7ff0380051c0 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.508+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038004810 0x7ff038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04ffff700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff038003a60 0x7ff038003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff054a11700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0380051c0 0x7ff03800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04f7fe700 1 -- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0380051c0 msgr2=0x7ff03800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0380051c0 0x7ff03800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04f7fe700 1 -- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff038003a60 msgr2=0x7ff038003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff038003a60 0x7ff038003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04f7fe700 1 -- 172.21.15.92:0/195628308 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038003570 con 0x7ff038004810 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff054a11700 1 -- 172.21.15.92:0/195628308 reap_dead start 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038004810 0x7ff038004c80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff0440018b0 tx=0x7ff044007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84d1c573bcdc208 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044012420 con 0x7ff038004810 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044012580 con 0x7ff038004810 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff04effd700 1 -- 172.21.15.92:0/195628308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044012920 con 0x7ff038004810 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff056474700 1 -- 172.21.15.92:0/195628308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038004810 msgr2=0x7ff038004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff056474700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038004810 0x7ff038004c80 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff0440018b0 tx=0x7ff044007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:11.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.509+0000 7ff056474700 1 -- 172.21.15.92:0/195628308 shutdown_connections 2023-10-06T03:07:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.510+0000 7ff056474700 1 --2- 172.21.15.92:0/195628308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff038004810 0x7ff038004c80 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.510+0000 7ff056474700 1 -- 172.21.15.92:0/195628308 >> 172.21.15.92:0/195628308 conn(0x7ff050068e20 msgr2=0x7ff0500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.510+0000 7ff056474700 1 -- 172.21.15.92:0/195628308 shutdown_connections 2023-10-06T03:07:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.510+0000 7ff056474700 1 -- 172.21.15.92:0/195628308 wait complete. 2023-10-06T03:07:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 Processor -- start 2023-10-06T03:07:11.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 -- start start 2023-10-06T03:07:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff038004810 0x7ff050151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 0x7ff0500b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0500b3590 0x7ff0500b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500b3f40 con 0x7ff0500b1ec0 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500b1a30 con 0x7ff0500b3590 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.511+0000 7ff056474700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500b0e20 con 0x7ff038004810 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 0x7ff0500b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 0x7ff0500b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50334/0 (socket says 172.21.15.92:50334) 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 -- 172.21.15.92:0/4168703590 learned_addr learned my addr 172.21.15.92:0/4168703590 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04ffff700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff038004810 0x7ff050151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff054a11700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0500b3590 0x7ff0500b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 -- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff038004810 msgr2=0x7ff050151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff038004810 0x7ff050151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 -- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0500b3590 msgr2=0x7ff0500b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0500b3590 0x7ff0500b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 -- 172.21.15.92:0/4168703590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500b1100 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff04f7fe700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 0x7ff0500b2330 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff04400cfa0 tx=0x7ff044015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff044012420 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff044001200 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0500b13e0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0440126f0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.512+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0501520f0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.513+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500b1bc0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.513+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff04401f620 con 0x7ff0500b1ec0 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.514+0000 7ff037fff700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff03c067200 0x7ff03c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.514+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ff0440861b0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.514+0000 7ff04ffff700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff03c067200 0x7ff03c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.514+0000 7ff04ffff700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff03c067200 0x7ff03c0696b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff040006f00 tx=0x7ff0400091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.518+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff044053270 con 0x7ff0500b1ec0 2023-10-06T03:07:11.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.747+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff05004c3c0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.747+0000 7ff037fff700 1 -- 172.21.15.92:0/4168703590 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ff044017de0 con 0x7ff0500b1ec0 2023-10-06T03:07:11.749 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:11.749 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3548371136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3548371136},{"type":"v1","addr":"172.21.15.136:6835","nonce":3548371136}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50},{"gid":5797,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1619501943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1619501943},{"type":"v1","addr":"172.21.15.136:6834","nonce":1619501943}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2733535798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2733535798},{"type":"v1","addr":"172.21.15.92:6836","nonce":2733535798}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3964110614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3964110614},{"type":"v1","addr":"172.21.15.92:6837","nonce":3964110614}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[]} 2023-10-06T03:07:11.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff03c067200 msgr2=0x7ff03c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff03c067200 0x7ff03c0696b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff040006f00 tx=0x7ff0400091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 msgr2=0x7ff0500b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 0x7ff0500b2330 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff04400cfa0 tx=0x7ff044015000 comp rx=0 tx=0).stop 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 shutdown_connections 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff03c067200 0x7ff03c0696b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0500b3590 0x7ff0500b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff038004810 0x7ff050151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 --2- 172.21.15.92:0/4168703590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0500b1ec0 0x7ff0500b2330 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.749+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 >> 172.21.15.92:0/4168703590 conn(0x7ff050068e20 msgr2=0x7ff05015d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.750+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 shutdown_connections 2023-10-06T03:07:11.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.750+0000 7ff056474700 1 -- 172.21.15.92:0/4168703590 wait complete. 2023-10-06T03:07:11.753 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 50 2023-10-06T03:07:11.763 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:07:11.763 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:07:11.764 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:07:11.764 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:11.764 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:07:11.764 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:07:11.764 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:07:11.764 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:11.765 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:07:11.765 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:07:11.765 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:07:11.765 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:11.765 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:07:11.765 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:07:11.765 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:07:11.766 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:11.772 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:07:11.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:07:11.773 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:07:11.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:07:11.775 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:07:11.775 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:07:11.776 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:07:11.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:07:11.815 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:07:11.815 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:11.816 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:07:11.816 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:11.818 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:07:11.818 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:11.818 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:07:11.819 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:11.820 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:07:11.888 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:07:11.888 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:07:11.898 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:07:11.898 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:07:11.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.940+0000 7fd919b05700 1 Processor -- start 2023-10-06T03:07:11.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.940+0000 7fd919b05700 1 -- start start 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd919b05700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9141597f0 0x7fd9140b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd919b05700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd91415d230 0x7fd91415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd919b05700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91415db70 con 0x7fd914159420 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd919b05700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140b11d0 con 0x7fd9141597f0 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd919b05700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140b14b0 con 0x7fd91415d230 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9141597f0 0x7fd9140b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd9141597f0 0x7fd9140b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50364/0 (socket says 172.21.15.92:50364) 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 -- 172.21.15.92:0/927206636 learned_addr learned my addr 172.21.15.92:0/927206636 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd919304700 1 --2- 172.21.15.92:0/927206636 >> v2:172.21.15.136:3300/0 conn(0x7fd91415d230 0x7fd91415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 -- 172.21.15.92:0/927206636 >> v2:172.21.15.136:3300/0 conn(0x7fd91415d230 msgr2=0x7fd91415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 --2- 172.21.15.92:0/927206636 >> v2:172.21.15.136:3300/0 conn(0x7fd91415d230 0x7fd91415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 -- 172.21.15.92:0/927206636 >> v1:172.21.15.136:6789/0 conn(0x7fd914159420 legacy=0x7fd9140b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.941+0000 7fd913fff700 1 -- 172.21.15.92:0/927206636 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140b1790 con 0x7fd9141597f0 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd913fff700 1 --2- 172.21.15.92:0/927206636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9141597f0 0x7fd9140b2270 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd91415d170 tx=0x7fd908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaad187509cc00e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd9137fe700 1 -- 172.21.15.92:0/927206636 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908009e70 con 0x7fd9141597f0 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd9137fe700 1 -- 172.21.15.92:0/927206636 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90800b860 con 0x7fd9141597f0 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd9137fe700 1 -- 172.21.15.92:0/927206636 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800bc00 con 0x7fd9141597f0 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 -- 172.21.15.92:0/927206636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9141597f0 msgr2=0x7fd9140b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 --2- 172.21.15.92:0/927206636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9141597f0 0x7fd9140b2270 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd91415d170 tx=0x7fd908009250 comp rx=0 tx=0).stop 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 -- 172.21.15.92:0/927206636 shutdown_connections 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 --2- 172.21.15.92:0/927206636 >> v2:172.21.15.136:3300/0 conn(0x7fd91415d230 0x7fd91415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 --2- 172.21.15.92:0/927206636 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9141597f0 0x7fd9140b2270 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 -- 172.21.15.92:0/927206636 >> 172.21.15.92:0/927206636 conn(0x7fd9140d0660 msgr2=0x7fd9140d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 -- 172.21.15.92:0/927206636 shutdown_connections 2023-10-06T03:07:11.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.942+0000 7fd919b05700 1 -- 172.21.15.92:0/927206636 wait complete. 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.943+0000 7fd919b05700 1 Processor -- start 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 -- start start 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd914159420 0x7fd914169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9141597f0 0x7fd9140c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 0x7fd914155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914159c30 con 0x7fd91415d230 2023-10-06T03:07:11.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91415a7b0 con 0x7fd9141597f0 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919b05700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914159e80 con 0x7fd914159420 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919304700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 0x7fd914155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919304700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 0x7fd914155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50370/0 (socket says 172.21.15.92:50370) 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd919304700 1 -- 172.21.15.92:0/2963741482 learned_addr learned my addr 172.21.15.92:0/2963741482 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd918b03700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd914159420 0x7fd914169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.944+0000 7fd913fff700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9141597f0 0x7fd9140c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919304700 1 -- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd914159420 msgr2=0x7fd914169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919304700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd914159420 0x7fd914169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919304700 1 -- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9141597f0 msgr2=0x7fd9140c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:11.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919304700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9141597f0 0x7fd9140c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919304700 1 -- 172.21.15.92:0/2963741482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080094f0 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919304700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 0x7fd914155fa0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e900 tx=0x7fd9040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904013680 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040137e0 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91415a160 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904013ba0 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.945+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91416a410 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.946+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91415a940 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.946+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd9040123d0 con 0x7fd91415d230 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.947+0000 7fd910ff9700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd900067200 0x7fd9000696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:11.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.947+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd904084860 con 0x7fd91415d230 2023-10-06T03:07:11.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.947+0000 7fd918b03700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd900067200 0x7fd9000696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:11.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.947+0000 7fd918b03700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd900067200 0x7fd9000696b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd90c007fd0 tx=0x7fd90c0016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:11.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:11.951+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd904051920 con 0x7fd91415d230 2023-10-06T03:07:12.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.154+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd91404c3c0 con 0x7fd91415d230 2023-10-06T03:07:12.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:07:12.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:07:12.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.979+0000 7fd910ff9700 1 -- 172.21.15.92:0/2963741482 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v86) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd904054f40 con 0x7fd91415d230 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd900067200 msgr2=0x7fd9000696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd900067200 0x7fd9000696b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd90c007fd0 tx=0x7fd90c0016f0 comp rx=0 tx=0).stop 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 msgr2=0x7fd914155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 0x7fd914155fa0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd90400e900 tx=0x7fd9040091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 shutdown_connections 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd900067200 0x7fd9000696b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:12.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd91415d230 0x7fd914155fa0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd914159420 0x7fd914169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 --2- 172.21.15.92:0/2963741482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9141597f0 0x7fd9140c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 >> 172.21.15.92:0/2963741482 conn(0x7fd9140d0660 msgr2=0x7fd9140699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 shutdown_connections 2023-10-06T03:07:12.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:12.982+0000 7fd919b05700 1 -- 172.21.15.92:0/2963741482 wait complete. 2023-10-06T03:07:12.985 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:07:12.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:07:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd581c67700 1 Processor -- start 2023-10-06T03:07:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd581c67700 1 -- start start 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd581c67700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 0x7fd57c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd581c67700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57c069ee0 con 0x7fd57c159890 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd581c67700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57c15a160 con 0x7fd57c159c60 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd581c67700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0d13a0 con 0x7fd57c069560 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.112+0000 7fd57b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd57c159890 0x7fd57c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40724/0 (socket says 172.21.15.92:40724) 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57b7fe700 1 -- 172.21.15.92:0/1090043396 learned_addr learned my addr 172.21.15.92:0/1090043396 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839162168 0 0) 0x7fd57c069ee0 con 0x7fd57c159890 2023-10-06T03:07:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd564003570 con 0x7fd57c159890 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 0x7fd57c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261450137 0 0) 0x7fd564003570 con 0x7fd57c159890 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 msgr2=0x7fd57c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 0x7fd57c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 >> v1:172.21.15.136:6789/0 conn(0x7fd57c159c60 legacy=0x7fd57c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c15e140 con 0x7fd57c159890 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd56c0017f0 con 0x7fd57c159890 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 >> v1:172.21.15.92:6789/0 conn(0x7fd57c159890 legacy=0x7fd57c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 0x7fd564003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd564003fd0 0x7fd564004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd564004d50 0x7fd5640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0d13a0 con 0x7fd564004d50 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56400bb40 con 0x7fd564003fd0 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56400bd00 con 0x7fd57c069560 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd564004d50 0x7fd5640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 0x7fd564003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57affd700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd564003fd0 0x7fd564004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 -- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 msgr2=0x7fd564003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd57c069560 0x7fd564003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 -- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd564003fd0 msgr2=0x7fd564004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd564003fd0 0x7fd564004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57bfff700 1 -- 172.21.15.92:0/1090043396 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564003570 con 0x7fd564004d50 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.113+0000 7fd57affd700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd564003fd0 0x7fd564004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd57bfff700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd564004d50 0x7fd5640051c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd574009c00 tx=0x7fd5740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9256be28b09d842 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400e710 con 0x7fd564004d50 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57400e870 con 0x7fd564004d50 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd57a7fc700 1 -- 172.21.15.92:0/1090043396 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400ec30 con 0x7fd564004d50 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd581c67700 1 -- 172.21.15.92:0/1090043396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd564004d50 msgr2=0x7fd5640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd581c67700 1 --2- 172.21.15.92:0/1090043396 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd564004d50 0x7fd5640051c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd574009c00 tx=0x7fd5740091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd57bfff700 1 -- 172.21.15.92:0/1090043396 reap_dead start 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd581c67700 1 -- 172.21.15.92:0/1090043396 shutdown_connections 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd581c67700 1 -- 172.21.15.92:0/1090043396 >> 172.21.15.92:0/1090043396 conn(0x7fd57c068e20 msgr2=0x7fd57c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:13.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd581c67700 1 -- 172.21.15.92:0/1090043396 shutdown_connections 2023-10-06T03:07:13.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.114+0000 7fd581c67700 1 -- 172.21.15.92:0/1090043396 wait complete. 2023-10-06T03:07:13.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.115+0000 7fd581c67700 1 Processor -- start 2023-10-06T03:07:13.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 -- start start 2023-10-06T03:07:13.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd564004d50 0x7fd57c14e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 0x7fd57c0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd57c0b1f30 0x7fd57c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0b23a0 con 0x7fd57c15d600 2023-10-06T03:07:13.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c15d170 con 0x7fd57c0b1f30 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd581c67700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c15c560 con 0x7fd564004d50 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd57affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 0x7fd57c0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd57affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 0x7fd57c0b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50388/0 (socket says 172.21.15.92:50388) 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd57affd700 1 -- 172.21.15.92:0/3475227703 learned_addr learned my addr 172.21.15.92:0/3475227703 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd564004d50 0x7fd57c14e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.116+0000 7fd57bfff700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd57c0b1f30 0x7fd57c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd57affd700 1 -- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd564004d50 msgr2=0x7fd57c14e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd57affd700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd564004d50 0x7fd57c14e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd57affd700 1 -- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd57c0b1f30 msgr2=0x7fd57c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd57affd700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd57c0b1f30 0x7fd57c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd57affd700 1 -- 172.21.15.92:0/3475227703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c15c840 con 0x7fd57c15d600 2023-10-06T03:07:13.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd57affd700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 0x7fd57c0b19f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd570007a80 tx=0x7fd570009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570006820 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570010040 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c15cb20 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570006bb0 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.117+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c14eaf0 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.118+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c0b01a0 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.118+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd570007030 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.119+0000 7fd5637fe700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd568067200 0x7fd5680696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.119+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd568067200 0x7fd5680696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.119+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd570083b30 con 0x7fd57c15d600 2023-10-06T03:07:13.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.119+0000 7fd57b7fe700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd568067200 0x7fd5680696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0011d0 tx=0x7fd56c0021f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.123+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd57004d5a0 con 0x7fd57c15d600 2023-10-06T03:07:13.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.381+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd57c0cce70 con 0x7fd57c15d600 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.382+0000 7fd5637fe700 1 -- 172.21.15.92:0/3475227703 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd570050b30 con 0x7fd57c15d600 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.384 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:13.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.386 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.387 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:07:13.388 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.389 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.390 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:07:13.391 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.392 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.393 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.394 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:07:13.395 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.396 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:07:13.397 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:07:13.398 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:07:13.399 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:13.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.399 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:13.399 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:07:13.399 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:07:13.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd568067200 msgr2=0x7fd5680696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd568067200 0x7fd5680696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0011d0 tx=0x7fd56c0021f0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 msgr2=0x7fd57c0b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 0x7fd57c0b19f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd570007a80 tx=0x7fd570009040 comp rx=0 tx=0).stop 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 shutdown_connections 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd568067200 0x7fd5680696b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd57c0b1f30 0x7fd57c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd564004d50 0x7fd57c14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 --2- 172.21.15.92:0/3475227703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd57c15d600 0x7fd57c0b19f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 >> 172.21.15.92:0/3475227703 conn(0x7fd57c068e20 msgr2=0x7fd57c0c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 shutdown_connections 2023-10-06T03:07:13.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.384+0000 7fd581c67700 1 -- 172.21.15.92:0/3475227703 wait complete. 2023-10-06T03:07:13.402 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:13.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.514+0000 7f65c1b6e700 1 Processor -- start 2023-10-06T03:07:13.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.514+0000 7f65c1b6e700 1 -- start start 2023-10-06T03:07:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65c1b6e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc068550 0x7f65bc068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65c1b6e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65bc15e010 0x7f65bc06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65c1b6e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f65bc06f640 0x7f65bc06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65c1b6e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc0b3430 con 0x7f65bc15e010 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65c1b6e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65bc06ec70 con 0x7f65bc06f640 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65c1b6e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc0b2e70 con 0x7f65bc068550 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65baffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65bc15e010 0x7f65bc06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65baffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f65bc15e010 0x7f65bc06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50392/0 (socket says 172.21.15.92:50392) 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65baffd700 1 -- 172.21.15.92:0/3138881575 learned_addr learned my addr 172.21.15.92:0/3138881575 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65bbfff700 1 --2- 172.21.15.92:0/3138881575 >> v2:172.21.15.136:3300/0 conn(0x7f65bc06f640 0x7f65bc06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.515+0000 7f65bb7fe700 1 --2- 172.21.15.92:0/3138881575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc068550 0x7f65bc068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65baffd700 1 -- 172.21.15.92:0/3138881575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc068550 msgr2=0x7f65bc068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65baffd700 1 --2- 172.21.15.92:0/3138881575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc068550 0x7f65bc068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65baffd700 1 -- 172.21.15.92:0/3138881575 >> v2:172.21.15.136:3300/0 conn(0x7f65bc06f640 msgr2=0x7f65bc06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65baffd700 1 --2- 172.21.15.92:0/3138881575 >> v2:172.21.15.136:3300/0 conn(0x7f65bc06f640 0x7f65bc06a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65baffd700 1 -- 172.21.15.92:0/3138881575 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc0b3150 con 0x7f65bc15e010 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65baffd700 1 --2- 172.21.15.92:0/3138881575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc15e010 0x7f65bc06f100 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f65bc06ebe0 tx=0x7f65b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7aa293b4a4c7959 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65ba7fc700 1 -- 172.21.15.92:0/3138881575 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b000b860 con 0x7f65bc15e010 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65ba7fc700 1 -- 172.21.15.92:0/3138881575 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b000b9c0 con 0x7f65bc15e010 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65ba7fc700 1 -- 172.21.15.92:0/3138881575 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b000bd60 con 0x7f65bc15e010 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3138881575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc15e010 msgr2=0x7f65bc06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3138881575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc15e010 0x7f65bc06f100 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f65bc06ebe0 tx=0x7f65b0009250 comp rx=0 tx=0).stop 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3138881575 shutdown_connections 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3138881575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc068550 0x7f65bc068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3138881575 >> v2:172.21.15.136:3300/0 conn(0x7f65bc06f640 0x7f65bc06a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3138881575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc15e010 0x7f65bc06f100 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3138881575 >> 172.21.15.92:0/3138881575 conn(0x7f65bc0cd480 msgr2=0x7f65bc0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:13.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3138881575 shutdown_connections 2023-10-06T03:07:13.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.516+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3138881575 wait complete. 2023-10-06T03:07:13.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 Processor -- start 2023-10-06T03:07:13.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 -- start start 2023-10-06T03:07:13.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 0x7f65bc146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc06f640 0x7f65bc154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65bc15e010 0x7f65bc1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc15aa80 con 0x7f65bc068550 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc147500 con 0x7f65bc15e010 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.518+0000 7f65c1b6e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc159e70 con 0x7f65bc06f640 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 0x7f65bc146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 0x7f65bc146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50394/0 (socket says 172.21.15.92:50394) 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 -- 172.21.15.92:0/3904055113 learned_addr learned my addr 172.21.15.92:0/3904055113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65baffd700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc06f640 0x7f65bc154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bbfff700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65bc15e010 0x7f65bc1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 -- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc06f640 msgr2=0x7f65bc154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc06f640 0x7f65bc154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 -- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65bc15e010 msgr2=0x7f65bc1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65bc15e010 0x7f65bc1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 -- 172.21.15.92:0/3904055113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b00094f0 con 0x7f65bc068550 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65bb7fe700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 0x7f65bc146fc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e190 tx=0x7f65ac009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00ecd0 con 0x7f65bc068550 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac00ee30 con 0x7f65bc068550 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc15a150 con 0x7f65bc068550 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac014670 con 0x7f65bc068550 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.519+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65bc15a710 con 0x7f65bc068550 2023-10-06T03:07:13.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.520+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65bc04c3c0 con 0x7f65bc068550 2023-10-06T03:07:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.521+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f65ac014a20 con 0x7f65bc068550 2023-10-06T03:07:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.521+0000 7f65a37fe700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f65a8067200 0x7f65a80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.521+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f65ac084e20 con 0x7f65bc068550 2023-10-06T03:07:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.521+0000 7f65baffd700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f65a8067200 0x7f65a80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.521+0000 7f65baffd700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f65a8067200 0x7f65a80696b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65bc06ebe0 tx=0x7f65b0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.525+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65ac051f30 con 0x7f65bc068550 2023-10-06T03:07:13.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.708+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65bc06e900 con 0x7f65bc068550 2023-10-06T03:07:13.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.709+0000 7f65a37fe700 1 -- 172.21.15.92:0/3904055113 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13651 (secure 0 0 0) 0x7f65ac009a60 con 0x7f65bc068550 2023-10-06T03:07:13.711 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:13.711 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:12.977420+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f65a8067200 msgr2=0x7f65a80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f65a8067200 0x7f65a80696b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f65bc06ebe0 tx=0x7f65b0017000 comp rx=0 tx=0).stop 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 msgr2=0x7f65bc146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 0x7f65bc146fc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00e190 tx=0x7f65ac009920 comp rx=0 tx=0).stop 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 shutdown_connections 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f65a8067200 0x7f65a80696b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f65bc068550 0x7f65bc146fc0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65bc15e010 0x7f65bc1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 --2- 172.21.15.92:0/3904055113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65bc06f640 0x7f65bc154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 >> 172.21.15.92:0/3904055113 conn(0x7f65bc0cd480 msgr2=0x7f65bc156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 shutdown_connections 2023-10-06T03:07:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.711+0000 7f65c1b6e700 1 -- 172.21.15.92:0/3904055113 wait complete. 2023-10-06T03:07:13.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:07:13.725 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:07:13.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.841+0000 7f83c2fde700 1 Processor -- start 2023-10-06T03:07:13.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.841+0000 7f83c2fde700 1 -- start start 2023-10-06T03:07:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c2fde700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc159890 0x7f83bc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c2fde700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83bc159c60 0x7f83bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c2fde700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83bc15db10 0x7f83bc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c2fde700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc15a160 con 0x7f83bc159c60 2023-10-06T03:07:13.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c2fde700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc0d13a0 con 0x7f83bc15db10 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c2fde700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc15e140 con 0x7f83bc159890 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83b3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83bc159c60 0x7f83bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83b3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f83bc159c60 0x7f83bc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50406/0 (socket says 172.21.15.92:50406) 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83b3fff700 1 -- 172.21.15.92:0/704924585 learned_addr learned my addr 172.21.15.92:0/704924585 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c157b700 1 --2- 172.21.15.92:0/704924585 >> v2:172.21.15.136:3300/0 conn(0x7f83bc15db10 0x7f83bc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.842+0000 7f83c0d7a700 1 --2- 172.21.15.92:0/704924585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc159890 0x7f83bc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b3fff700 1 -- 172.21.15.92:0/704924585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc159890 msgr2=0x7f83bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b3fff700 1 --2- 172.21.15.92:0/704924585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc159890 0x7f83bc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b3fff700 1 -- 172.21.15.92:0/704924585 >> v2:172.21.15.136:3300/0 conn(0x7f83bc15db10 msgr2=0x7f83bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b3fff700 1 --2- 172.21.15.92:0/704924585 >> v2:172.21.15.136:3300/0 conn(0x7f83bc15db10 0x7f83bc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b3fff700 1 -- 172.21.15.92:0/704924585 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc15a3d0 con 0x7f83bc159c60 2023-10-06T03:07:13.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b3fff700 1 --2- 172.21.15.92:0/704924585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159c60 0x7f83bc15d5d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f83bc089b50 tx=0x7f83ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7294dfb9cd86978b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b37fe700 1 -- 172.21.15.92:0/704924585 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00b860 con 0x7f83bc159c60 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b37fe700 1 -- 172.21.15.92:0/704924585 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac00b9c0 con 0x7f83bc159c60 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83b37fe700 1 -- 172.21.15.92:0/704924585 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00bd60 con 0x7f83bc159c60 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 -- 172.21.15.92:0/704924585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159c60 msgr2=0x7f83bc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 --2- 172.21.15.92:0/704924585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159c60 0x7f83bc15d5d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f83bc089b50 tx=0x7f83ac009250 comp rx=0 tx=0).stop 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 -- 172.21.15.92:0/704924585 shutdown_connections 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 --2- 172.21.15.92:0/704924585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc159890 0x7f83bc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 --2- 172.21.15.92:0/704924585 >> v2:172.21.15.136:3300/0 conn(0x7f83bc15db10 0x7f83bc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 --2- 172.21.15.92:0/704924585 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159c60 0x7f83bc15d5d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 -- 172.21.15.92:0/704924585 >> 172.21.15.92:0/704924585 conn(0x7f83bc068e20 msgr2=0x7f83bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:13.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 -- 172.21.15.92:0/704924585 shutdown_connections 2023-10-06T03:07:13.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.843+0000 7f83c2fde700 1 -- 172.21.15.92:0/704924585 wait complete. 2023-10-06T03:07:13.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.844+0000 7f83c2fde700 1 Processor -- start 2023-10-06T03:07:13.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 -- start start 2023-10-06T03:07:13.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 0x7f83bc0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83bc159c60 0x7f83bc1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc15db10 0x7f83bc15f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0c79b0 con 0x7f83bc159890 2023-10-06T03:07:13.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0cf790 con 0x7f83bc159c60 2023-10-06T03:07:13.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.845+0000 7f83c2fde700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0ce930 con 0x7f83bc15db10 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 0x7f83bc0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 0x7f83bc0ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50408/0 (socket says 172.21.15.92:50408) 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 -- 172.21.15.92:0/2993803694 learned_addr learned my addr 172.21.15.92:0/2993803694 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c157b700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc15db10 0x7f83bc15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83b3fff700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83bc159c60 0x7f83bc1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 -- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc15db10 msgr2=0x7f83bc15f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc15db10 0x7f83bc15f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 -- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83bc159c60 msgr2=0x7f83bc1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:13.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83bc159c60 0x7f83bc1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 -- 172.21.15.92:0/2993803694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac0094f0 con 0x7f83bc159890 2023-10-06T03:07:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c0d7a700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 0x7f83bc0ca600 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e190 tx=0x7f83b8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800ecd0 con 0x7f83bc159890 2023-10-06T03:07:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b800ee30 con 0x7f83bc159890 2023-10-06T03:07:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc0ceb80 con 0x7f83bc159890 2023-10-06T03:07:13.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8014670 con 0x7f83bc159890 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.846+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc0cf1c0 con 0x7f83bc159890 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.847+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc04c3c0 con 0x7f83bc159890 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.848+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f83b8014a20 con 0x7f83bc159890 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.848+0000 7f83b0ff9700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83a8067200 0x7f83a80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.848+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f83b8084ed0 con 0x7f83bc159890 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.848+0000 7f83b3fff700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83a8067200 0x7f83a80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:13.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.848+0000 7f83b3fff700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83a8067200 0x7f83a80696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83bc089b50 tx=0x7f83ac017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:13.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:13.852+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83b8051fe0 con 0x7f83bc159890 2023-10-06T03:07:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.033+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83bc06a5b0 con 0x7f83bc159890 2023-10-06T03:07:14.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.033+0000 7f83b0ff9700 1 -- 172.21.15.92:0/2993803694 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83b8009a60 con 0x7f83bc159890 2023-10-06T03:07:14.037 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:07:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83a8067200 msgr2=0x7f83a80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83a8067200 0x7f83a80696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f83bc089b50 tx=0x7f83ac017000 comp rx=0 tx=0).stop 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 msgr2=0x7f83bc0ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 0x7f83bc0ca600 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e190 tx=0x7f83b8009920 comp rx=0 tx=0).stop 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 shutdown_connections 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f83a8067200 0x7f83a80696b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f83bc159890 0x7f83bc0ca600 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83bc15db10 0x7f83bc15f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 --2- 172.21.15.92:0/2993803694 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83bc159c60 0x7f83bc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 >> 172.21.15.92:0/2993803694 conn(0x7f83bc068e20 msgr2=0x7f83bc154690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 shutdown_connections 2023-10-06T03:07:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.035+0000 7f83c2fde700 1 -- 172.21.15.92:0/2993803694 wait complete. 2023-10-06T03:07:14.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:07:14.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.165+0000 7fdd21bd7700 1 Processor -- start 2023-10-06T03:07:14.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.165+0000 7fdd21bd7700 1 -- start start 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd21bd7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdd1c159c60 0x7fdd1c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd21bd7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd1c15db10 0x7fdd1c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd21bd7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1c15a160 con 0x7fdd1c159890 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd21bd7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c15df50 con 0x7fdd1c15db10 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd21bd7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0d13a0 con 0x7fdd1c159c60 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd1c15db10 0x7fdd1c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdd1c15db10 0x7fdd1c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50422/0 (socket says 172.21.15.92:50422) 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 -- 172.21.15.92:0/3388677829 learned_addr learned my addr 172.21.15.92:0/3388677829 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 -- 172.21.15.92:0/3388677829 >> v2:172.21.15.136:3300/0 conn(0x7fdd1c159c60 msgr2=0x7fdd1c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1affd700 1 --2- 172.21.15.92:0/3388677829 >> v2:172.21.15.136:3300/0 conn(0x7fdd1c159c60 0x7fdd1c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 --2- 172.21.15.92:0/3388677829 >> v2:172.21.15.136:3300/0 conn(0x7fdd1c159c60 0x7fdd1c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 -- 172.21.15.92:0/3388677829 >> v1:172.21.15.136:6789/0 conn(0x7fdd1c159890 legacy=0x7fdd1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 -- 172.21.15.92:0/3388677829 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c15e140 con 0x7fdd1c15db10 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1bfff700 1 --2- 172.21.15.92:0/3388677829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c15db10 0x7fdd1c069560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0b67f0 tx=0x7fdd14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2246b66b97ca6408 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1a7fc700 1 -- 172.21.15.92:0/3388677829 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400e070 con 0x7fdd1c15db10 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.166+0000 7fdd1a7fc700 1 -- 172.21.15.92:0/3388677829 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1400ba10 con 0x7fdd1c15db10 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd1a7fc700 1 -- 172.21.15.92:0/3388677829 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd140092e0 con 0x7fdd1c15db10 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3388677829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c15db10 msgr2=0x7fdd1c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3388677829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c15db10 0x7fdd1c069560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0b67f0 tx=0x7fdd14009840 comp rx=0 tx=0).stop 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3388677829 shutdown_connections 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3388677829 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c15db10 0x7fdd1c069560 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3388677829 >> v2:172.21.15.136:3300/0 conn(0x7fdd1c159c60 0x7fdd1c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3388677829 >> 172.21.15.92:0/3388677829 conn(0x7fdd1c068e20 msgr2=0x7fdd1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:14.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3388677829 shutdown_connections 2023-10-06T03:07:14.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.167+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3388677829 wait complete. 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.168+0000 7fdd21bd7700 1 Processor -- start 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.168+0000 7fdd21bd7700 1 -- start start 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd21bd7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 0x7fdd1c14a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd21bd7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1c159c60 0x7fdd1c14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd21bd7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c15db10 0x7fdd1c14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd21bd7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c158070 con 0x7fdd1c159890 2023-10-06T03:07:14.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd21bd7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0c6310 con 0x7fdd1c15db10 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd21bd7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0c65f0 con 0x7fdd1c159c60 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 0x7fdd1c14a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 0x7fdd1c14a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50432/0 (socket says 172.21.15.92:50432) 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 -- 172.21.15.92:0/3237274597 learned_addr learned my addr 172.21.15.92:0/3237274597 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1bfff700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c15db10 0x7fdd1c14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1affd700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1c159c60 0x7fdd1c14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 -- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1c159c60 msgr2=0x7fdd1c14ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1c159c60 0x7fdd1c14ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 -- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c15db10 msgr2=0x7fdd1c14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c15db10 0x7fdd1c14b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.169+0000 7fdd1b7fe700 1 -- 172.21.15.92:0/3237274597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009ae0 con 0x7fdd1c159890 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.170+0000 7fdd1b7fe700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 0x7fdd1c14a3b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c5d0 tx=0x7fdd10007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.170+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10011590 con 0x7fdd1c159890 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.170+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd10011710 con 0x7fdd1c159890 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.170+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0c68d0 con 0x7fdd1c159890 2023-10-06T03:07:14.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.170+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10011ad0 con 0x7fdd1c159890 2023-10-06T03:07:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.170+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c0c6e90 con 0x7fdd1c159890 2023-10-06T03:07:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.171+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c04c3c0 con 0x7fdd1c159890 2023-10-06T03:07:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.171+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fdd1001c460 con 0x7fdd1c159890 2023-10-06T03:07:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.172+0000 7fdd037fe700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd08067200 0x7fdd080696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.172+0000 7fdd1affd700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd08067200 0x7fdd080696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.172+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdd10084520 con 0x7fdd1c159890 2023-10-06T03:07:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.172+0000 7fdd1affd700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd08067200 0x7fdd080696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0b2460 tx=0x7fdd0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.176+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd100515f0 con 0x7fdd1c159890 2023-10-06T03:07:14.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.361+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd1c153490 con 0x7fdd1c159890 2023-10-06T03:07:14.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.361+0000 7fdd037fe700 1 -- 172.21.15.92:0/3237274597 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdd10054c10 con 0x7fdd1c159890 2023-10-06T03:07:14.363 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:07:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd08067200 msgr2=0x7fdd080696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd08067200 0x7fdd080696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0b2460 tx=0x7fdd0c008040 comp rx=0 tx=0).stop 2023-10-06T03:07:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 msgr2=0x7fdd1c14a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 0x7fdd1c14a3b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000c5d0 tx=0x7fdd10007f60 comp rx=0 tx=0).stop 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 shutdown_connections 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdd08067200 0x7fdd080696b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdd1c159890 0x7fdd1c14a3b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd1c15db10 0x7fdd1c14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 --2- 172.21.15.92:0/3237274597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd1c159c60 0x7fdd1c14ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 >> 172.21.15.92:0/3237274597 conn(0x7fdd1c068e20 msgr2=0x7fdd1c1584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 shutdown_connections 2023-10-06T03:07:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.363+0000 7fdd21bd7700 1 -- 172.21.15.92:0/3237274597 wait complete. 2023-10-06T03:07:14.377 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:07:14.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.490+0000 7f18a0c10700 1 Processor -- start 2023-10-06T03:07:14.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.490+0000 7f18a0c10700 1 -- start start 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f18a0c10700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f189c159890 0x7f189c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f18a0c10700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f18a0c10700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c15a160 con 0x7f189c15db10 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f18a0c10700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c0d13a0 con 0x7f189c159890 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f18a0c10700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c15e140 con 0x7f189c159c60 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f189c159890 0x7f189c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f189c159890 0x7f189c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50446/0 (socket says 172.21.15.92:50446) 2023-10-06T03:07:14.494 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 -- 172.21.15.92:0/3046609395 learned_addr learned my addr 172.21.15.92:0/3046609395 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f1891d9b700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 -- 172.21.15.92:0/3046609395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 msgr2=0x7f189c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 -- 172.21.15.92:0/3046609395 >> v1:172.21.15.136:6789/0 conn(0x7f189c15db10 legacy=0x7f189c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 -- 172.21.15.92:0/3046609395 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c15a3d0 con 0x7f189c159890 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f1891d9b700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f189a59c700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c159890 0x7f189c15abd0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f188c0099b0 tx=0x7f188c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5b856f87bcff78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f1899d9b700 1 -- 172.21.15.92:0/3046609395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00b860 con 0x7f189c159890 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f1899d9b700 1 -- 172.21.15.92:0/3046609395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c00b9c0 con 0x7f189c159890 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.491+0000 7f1899d9b700 1 -- 172.21.15.92:0/3046609395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00bd60 con 0x7f189c159890 2023-10-06T03:07:14.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 -- 172.21.15.92:0/3046609395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c159890 msgr2=0x7f189c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c159890 0x7f189c15abd0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f188c0099b0 tx=0x7f188c009250 comp rx=0 tx=0).stop 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 -- 172.21.15.92:0/3046609395 shutdown_connections 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c159890 0x7f189c15abd0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 --2- 172.21.15.92:0/3046609395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 -- 172.21.15.92:0/3046609395 >> 172.21.15.92:0/3046609395 conn(0x7f189c068e20 msgr2=0x7f189c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 -- 172.21.15.92:0/3046609395 shutdown_connections 2023-10-06T03:07:14.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.492+0000 7f18a0c10700 1 -- 172.21.15.92:0/3046609395 wait complete. 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.493+0000 7f18a0c10700 1 Processor -- start 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.493+0000 7f18a0c10700 1 -- start start 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f18a0c10700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f189c159890 0x7f189c1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f18a0c10700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f18a0c10700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 0x7f189c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f18a0c10700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c16ae10 con 0x7f189c15db10 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f18a0c10700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c16a200 con 0x7f189c159890 2023-10-06T03:07:14.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f18a0c10700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c16a4e0 con 0x7f189c159c60 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 0x7f189c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 0x7f189c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50448/0 (socket says 172.21.15.92:50448) 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 -- 172.21.15.92:0/2672702930 learned_addr learned my addr 172.21.15.92:0/2672702930 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189a59c700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f189c159890 0x7f189c1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f1891d9b700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 -- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 msgr2=0x7f189c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 -- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f189c159890 msgr2=0x7f189c1523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f189c159890 0x7f189c1523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.494+0000 7f189ad9d700 1 -- 172.21.15.92:0/2672702930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0094f0 con 0x7f189c15db10 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.495+0000 7f189ad9d700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 0x7f189c156650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f189400c840 tx=0x7f1894001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.495+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894012630 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.495+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18940127b0 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.495+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894012b70 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.495+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c16a7c0 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.495+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c169f00 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.496+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c04c3c0 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.496+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f1894003030 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.496+0000 7f1892ffd700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f187c067200 0x7f187c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.496+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1894084700 con 0x7f189c15db10 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.496+0000 7f189a59c700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f187c067200 0x7f187c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.497+0000 7f189a59c700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f187c067200 0x7f187c0696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f188c0099b0 tx=0x7f188c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.501+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1894051810 con 0x7f189c15db10 2023-10-06T03:07:14.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.685+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f189c16af50 con 0x7f189c15db10 2023-10-06T03:07:14.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.686+0000 7f1892ffd700 1 -- 172.21.15.92:0/2672702930 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+126 (secure 0 0 0) 0x7f189c16af50 con 0x7f189c15db10 2023-10-06T03:07:14.688 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:14.688 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:07:14.688 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:14.688 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:07:14.688 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:07:14.688 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:14.689 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:14.690 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:07:14.690 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:07:14.690 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:14.690 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:07:14.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f187c067200 msgr2=0x7f187c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f187c067200 0x7f187c0696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f188c0099b0 tx=0x7f188c017000 comp rx=0 tx=0).stop 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 msgr2=0x7f189c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 0x7f189c156650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f189400c840 tx=0x7f1894001670 comp rx=0 tx=0).stop 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 shutdown_connections 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f187c067200 0x7f187c0696b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f189c15db10 0x7f189c156650 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f189c159890 0x7f189c1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 --2- 172.21.15.92:0/2672702930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f189c159c60 0x7f189c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 >> 172.21.15.92:0/2672702930 conn(0x7f189c068e20 msgr2=0x7f189c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:14.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 shutdown_connections 2023-10-06T03:07:14.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.688+0000 7f18a0c10700 1 -- 172.21.15.92:0/2672702930 wait complete. 2023-10-06T03:07:14.701 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:07:14.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:07:14.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.818+0000 7f8db5c1f700 1 Processor -- start 2023-10-06T03:07:14.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.818+0000 7f8db5c1f700 1 -- start start 2023-10-06T03:07:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8db5c1f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8db00b55a0 0x7f8db0164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8db5c1f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00cfa60 con 0x7f8db0159ec0 2023-10-06T03:07:14.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8db5c1f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db00b5b00 con 0x7f8db00cd910 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8db5c1f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db00b5da0 con 0x7f8db00b55a0 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8daf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8db0159ec0 0x7f8db00b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40728/0 (socket says 172.21.15.92:40728) 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8daf7fe700 1 -- 172.21.15.92:0/331667770 learned_addr learned my addr 172.21.15.92:0/331667770 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8daffff700 1 --2- 172.21.15.92:0/331667770 >> v2:172.21.15.136:3300/0 conn(0x7f8db00b55a0 0x7f8db0164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.819+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879754378 0 0) 0x7f8db00cfa60 con 0x7f8db0159ec0 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d98003570 con 0x7f8db0159ec0 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4011816914 0 0) 0x7f8d98003570 con 0x7f8db0159ec0 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 >> v2:172.21.15.136:3300/0 conn(0x7f8db00b55a0 msgr2=0x7f8db0164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 --2- 172.21.15.92:0/331667770 >> v2:172.21.15.136:3300/0 conn(0x7f8db00b55a0 0x7f8db0164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 >> v1:172.21.15.136:6789/0 conn(0x7f8db00cd910 legacy=0x7f8db00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00b6080 con 0x7f8db0159ec0 2023-10-06T03:07:14.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8da0001970 con 0x7f8db0159ec0 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 >> v1:172.21.15.92:6789/0 conn(0x7f8db0159ec0 legacy=0x7f8db00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d98003a60 0x7f8d98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d98004810 0x7f8d98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98005210 0x7f8d9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b5da0 con 0x7f8d98004810 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9800c130 con 0x7f8d98003a60 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9800c2c0 con 0x7f8d98005210 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d98004810 0x7f8d98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daf7fe700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d98003a60 0x7f8d98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daffff700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98005210 0x7f8d9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 -- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98005210 msgr2=0x7f8d9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98005210 0x7f8d9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 -- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d98003a60 msgr2=0x7f8d98003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d98003a60 0x7f8d98003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 -- 172.21.15.92:0/331667770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98003570 con 0x7f8d98004810 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daffff700 1 -- 172.21.15.92:0/331667770 reap_dead start 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daf7fe700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d98003a60 0x7f8d98003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8daeffd700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d98004810 0x7f8d98004c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8da40018b0 tx=0x7f8da4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62866c9744a2e7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4012420 con 0x7f8d98004810 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da40125a0 con 0x7f8d98004810 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.820+0000 7f8dae7fc700 1 -- 172.21.15.92:0/331667770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4012960 con 0x7f8d98004810 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 -- 172.21.15.92:0/331667770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d98004810 msgr2=0x7f8d98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d98004810 0x7f8d98004c80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8da40018b0 tx=0x7f8da4007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:14.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 -- 172.21.15.92:0/331667770 shutdown_connections 2023-10-06T03:07:14.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/331667770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d98004810 0x7f8d98004c80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 -- 172.21.15.92:0/331667770 >> 172.21.15.92:0/331667770 conn(0x7f8db00999f0 msgr2=0x7f8db00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:14.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 -- 172.21.15.92:0/331667770 shutdown_connections 2023-10-06T03:07:14.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.821+0000 7f8db5c1f700 1 -- 172.21.15.92:0/331667770 wait complete. 2023-10-06T03:07:14.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.822+0000 7f8db5c1f700 1 Processor -- start 2023-10-06T03:07:14.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.822+0000 7f8db5c1f700 1 -- start start 2023-10-06T03:07:14.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8db5c1f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98004810 0x7f8db006ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8db5c1f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 0x7f8db014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8db5c1f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8db0169e90 0x7f8db015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8db5c1f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db015bc60 con 0x7f8db0157440 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8db5c1f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db015c220 con 0x7f8db0169e90 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8db5c1f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db006b2d0 con 0x7f8d98004810 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 0x7f8db014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 0x7f8db014b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50468/0 (socket says 172.21.15.92:50468) 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 -- 172.21.15.92:0/2984334376 learned_addr learned my addr 172.21.15.92:0/2984334376 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daf7fe700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98004810 0x7f8db006ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 -- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98004810 msgr2=0x7f8db006ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daffff700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8db0169e90 0x7f8db015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98004810 0x7f8db006ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 -- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8db0169e90 msgr2=0x7f8db015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8db0169e90 0x7f8db015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.823+0000 7f8daeffd700 1 -- 172.21.15.92:0/2984334376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db014b770 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.824+0000 7f8daeffd700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 0x7f8db014b230 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8da40124a0 tx=0x7f8da4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.824+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4006820 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.824+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da40069a0 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.824+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db015b370 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.824+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4006d60 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.824+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db015b8f0 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.825+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db0063c90 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.825+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8da4007240 con 0x7f8db0157440 2023-10-06T03:07:14.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.825+0000 7f8d977fe700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d9c067200 0x7f8d9c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:14.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.826+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8da4086100 con 0x7f8db0157440 2023-10-06T03:07:14.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.826+0000 7f8daf7fe700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d9c067200 0x7f8d9c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:14.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.826+0000 7f8daf7fe700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d9c067200 0x7f8d9c0696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8da0001270 tx=0x7f8da0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:14.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:14.830+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8da4053210 con 0x7f8db0157440 2023-10-06T03:07:15.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.013+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8db015c3b0 con 0x7f8db0157440 2023-10-06T03:07:15.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.990+0000 7f8d977fe700 1 -- 172.21.15.92:0/2984334376 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8da405ae60 con 0x7f8db0157440 2023-10-06T03:07:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d9c067200 msgr2=0x7f8d9c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d9c067200 0x7f8d9c0696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8da0001270 tx=0x7f8da0009040 comp rx=0 tx=0).stop 2023-10-06T03:07:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 msgr2=0x7f8db014b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:15.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 0x7f8db014b230 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8da40124a0 tx=0x7f8da4015000 comp rx=0 tx=0).stop 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 shutdown_connections 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d9c067200 0x7f8d9c0696b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8db0169e90 0x7f8db015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d98004810 0x7f8db006ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 --2- 172.21.15.92:0/2984334376 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8db0157440 0x7f8db014b230 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.993+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 >> 172.21.15.92:0/2984334376 conn(0x7f8db00999f0 msgr2=0x7f8db0159360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.994+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 shutdown_connections 2023-10-06T03:07:15.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:15.994+0000 7f8db5c1f700 1 -- 172.21.15.92:0/2984334376 wait complete. 2023-10-06T03:07:15.997 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:07:16.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:07:16.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.134+0000 7f7f6b59e700 1 Processor -- start 2023-10-06T03:07:16.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.134+0000 7f7f6b59e700 1 -- start start 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6b59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7f6c06af90 0x7f7f6c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 0x7f7f6c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0c9860 con 0x7f7f6c06b360 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6b59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c0c8cd0 con 0x7f7f6c06af90 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cd200 con 0x7f7f6c06d840 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f69d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7f6c06b360 0x7f7f6c06d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40736/0 (socket says 172.21.15.92:40736) 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f69d9b700 1 -- 172.21.15.92:0/1032590635 learned_addr learned my addr 172.21.15.92:0/1032590635 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126127257 0 0) 0x7f7f6c0c9860 con 0x7f7f6c06b360 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6ad9d700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 0x7f7f6c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/1032590635 >> v2:172.21.15.136:3300/0 conn(0x7f7f6c06af90 0x7f7f6c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.135+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003570 con 0x7f7f6c06b360 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2090264632 0 0) 0x7f7f54003570 con 0x7f7f6c06b360 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 msgr2=0x7f7f6c0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 0x7f7f6c0ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 >> v2:172.21.15.136:3300/0 conn(0x7f7f6c06af90 msgr2=0x7f7f6c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 --2- 172.21.15.92:0/1032590635 >> v2:172.21.15.136:3300/0 conn(0x7f7f6c06af90 0x7f7f6c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0c8f30 con 0x7f7f6c06b360 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7f60001470 con 0x7f7f6c06b360 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 >> v1:172.21.15.92:6789/0 conn(0x7f7f6c06b360 legacy=0x7f7f6c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54003a60 0x7f7f54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 0x7f7f54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54004d50 0x7f7f540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0c8cd0 con 0x7f7f54003a60 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0cd200 con 0x7f7f54004d50 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400ba60 con 0x7f7f6c06d840 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54003a60 0x7f7f54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f69d9b700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 0x7f7f54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6ad9d700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54004d50 0x7f7f540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 -- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 msgr2=0x7f7f54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f6c06d840 0x7f7f54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 -- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54004d50 msgr2=0x7f7f540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54004d50 0x7f7f540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 -- 172.21.15.92:0/1032590635 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54003570 con 0x7f7f54003a60 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6ad9d700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f54004d50 0x7f7f540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54003a60 0x7f7f54003e70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00e210 tx=0x7f7f5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c265bc1042ef0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00ea60 con 0x7f7f54003a60 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00f040 con 0x7f7f54003a60 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.136+0000 7f7f6959a700 1 -- 172.21.15.92:0/1032590635 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00ee10 con 0x7f7f54003a60 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6b59e700 1 -- 172.21.15.92:0/1032590635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54003a60 msgr2=0x7f7f54003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/1032590635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f54003a60 0x7f7f54003e70 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00e210 tx=0x7f7f5c00b040 comp rx=0 tx=0).stop 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6ad9d700 1 -- 172.21.15.92:0/1032590635 reap_dead start 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6b59e700 1 -- 172.21.15.92:0/1032590635 shutdown_connections 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6b59e700 1 -- 172.21.15.92:0/1032590635 >> 172.21.15.92:0/1032590635 conn(0x7f7f6c06baa0 msgr2=0x7f7f6c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6b59e700 1 -- 172.21.15.92:0/1032590635 shutdown_connections 2023-10-06T03:07:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.137+0000 7f7f6b59e700 1 -- 172.21.15.92:0/1032590635 wait complete. 2023-10-06T03:07:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.138+0000 7f7f6b59e700 1 Processor -- start 2023-10-06T03:07:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.138+0000 7f7f6b59e700 1 -- start start 2023-10-06T03:07:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f54003a60 0x7f7f6c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f6c159fb0 0x7f7f6c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 0x7f7f6c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0b29c0 con 0x7f7f6c0b1430 2023-10-06T03:07:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c15a960 con 0x7f7f6c159fb0 2023-10-06T03:07:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c169d50 con 0x7f7f54003a60 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 0x7f7f6c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 0x7f7f6c0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50482/0 (socket says 172.21.15.92:50482) 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 -- 172.21.15.92:0/355433680 learned_addr learned my addr 172.21.15.92:0/355433680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f69d9b700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f6c159fb0 0x7f7f6c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f54003a60 0x7f7f6c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 -- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f54003a60 msgr2=0x7f7f6c0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f54003a60 0x7f7f6c0b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 -- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f6c159fb0 msgr2=0x7f7f6c15a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f6c159fb0 0x7f7f6c15a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f6ad9d700 1 -- 172.21.15.92:0/355433680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0b2120 con 0x7f7f6c0b1430 2023-10-06T03:07:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.139+0000 7f7f69d9b700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f6c159fb0 0x7f7f6c15a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.140+0000 7f7f6ad9d700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 0x7f7f6c0b18b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0cc4f0 tx=0x7f7f64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.140+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64015670 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.140+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f640157d0 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.140+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c0b2400 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.140+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64015b90 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.140+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c0b1df0 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.141+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c169b20 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.141+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7f64020460 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.141+0000 7f7f5a7fc700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f4c067200 0x7f7f4c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.142+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f7f64084b00 con 0x7f7f6c0b1430 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.142+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f4c067200 0x7f7f4c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.142+0000 7f7f6a59c700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f4c067200 0x7f7f4c0696b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00afd0 tx=0x7f7f5c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.146+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f6c169b20 con 0x7f7f6c0b1430 2023-10-06T03:07:16.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.329+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7f6c169b20 con 0x7f7f6c0b1430 2023-10-06T03:07:16.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:16.996+0000 7f7f5a7fc700 1 -- 172.21.15.92:0/355433680 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7f6c169b20 con 0x7f7f6c0b1430 2023-10-06T03:07:17.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.001+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f4c067200 msgr2=0x7f7f4c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:17.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.001+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f4c067200 0x7f7f4c0696b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00afd0 tx=0x7f7f5c01a000 comp rx=0 tx=0).stop 2023-10-06T03:07:17.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.001+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 msgr2=0x7f7f6c0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:17.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.001+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 0x7f7f6c0b18b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0cc4f0 tx=0x7f7f64008040 comp rx=0 tx=0).stop 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.003+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 shutdown_connections 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.003+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7f4c067200 0x7f7f4c0696b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.003+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f6c0b1430 0x7f7f6c0b18b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.003+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7f54003a60 0x7f7f6c0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.003+0000 7f7f6b59e700 1 --2- 172.21.15.92:0/355433680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f6c159fb0 0x7f7f6c15a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.003+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 >> 172.21.15.92:0/355433680 conn(0x7f7f6c06baa0 msgr2=0x7f7f6c0c3480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:17.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.004+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 shutdown_connections 2023-10-06T03:07:17.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.004+0000 7f7f6b59e700 1 -- 172.21.15.92:0/355433680 wait complete. 2023-10-06T03:07:17.010 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:07:17.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:07:17.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.141+0000 7fe7409ed700 1 Processor -- start 2023-10-06T03:07:17.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.141+0000 7fe7409ed700 1 -- start start 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe7409ed700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe73c159c60 0x7fe73c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe7409ed700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe7409ed700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c15a160 con 0x7fe73c159890 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe7409ed700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c15df50 con 0x7fe73c159c60 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe7409ed700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0d13a0 con 0x7fe73c15db10 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.142+0000 7fe731d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe73c159c60 0x7fe73c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe73c159c60 0x7fe73c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:35436/0 (socket says 172.21.15.92:35436) 2023-10-06T03:07:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:43944/0 (socket says 172.21.15.92:43944) 2023-10-06T03:07:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 -- 172.21.15.92:0/3306846570 learned_addr learned my addr 172.21.15.92:0/3306846570 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:17.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 -- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 msgr2=0x7fe73c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:17.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 --2- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 -- 172.21.15.92:0/3306846570 >> v1:172.21.15.136:6789/0 conn(0x7fe73c159890 legacy=0x7fe73c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 -- 172.21.15.92:0/3306846570 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c15e140 con 0x7fe73c159c60 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe739d9b700 1 -- 172.21.15.92:0/3306846570 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124881552 0 0) 0x7fe73c15a160 con 0x7fe73c159890 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe731d9b700 1 --2- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159c60 0x7fe73c15d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe7240099b0 tx=0x7fe724009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e57b526270d7168 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe739d9b700 1 -- 172.21.15.92:0/3306846570 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724009ea0 con 0x7fe73c159c60 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe739d9b700 1 -- 172.21.15.92:0/3306846570 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72400b860 con 0x7fe73c159c60 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.143+0000 7fe739d9b700 1 -- 172.21.15.92:0/3306846570 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72400bc00 con 0x7fe73c159c60 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 -- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159c60 msgr2=0x7fe73c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159c60 0x7fe73c15d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe7240099b0 tx=0x7fe724009250 comp rx=0 tx=0).stop 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 -- 172.21.15.92:0/3306846570 shutdown_connections 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3306846570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159c60 0x7fe73c15d5d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 -- 172.21.15.92:0/3306846570 >> 172.21.15.92:0/3306846570 conn(0x7fe73c068e20 msgr2=0x7fe73c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:17.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 -- 172.21.15.92:0/3306846570 shutdown_connections 2023-10-06T03:07:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.144+0000 7fe7409ed700 1 -- 172.21.15.92:0/3306846570 wait complete. 2023-10-06T03:07:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.145+0000 7fe7409ed700 1 Processor -- start 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.145+0000 7fe7409ed700 1 -- start start 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe7409ed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159890 0x7fe73c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe7409ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 0x7fe73c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe7409ed700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe7409ed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0b0a80 con 0x7fe73c159c60 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe7409ed700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c154590 con 0x7fe73c159890 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe7409ed700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c153980 con 0x7fe73c15db10 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 0x7fe73c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 0x7fe73c150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50486/0 (socket says 172.21.15.92:50486) 2023-10-06T03:07:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 -- 172.21.15.92:0/3666988391 learned_addr learned my addr 172.21.15.92:0/3666988391 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe73ad9d700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe73a59c700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159890 0x7fe73c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 -- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 msgr2=0x7fe73c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 -- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159890 msgr2=0x7fe73c0b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159890 0x7fe73c0b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe731d9b700 1 -- 172.21.15.92:0/3666988391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240094f0 con 0x7fe73c159c60 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe73ad9d700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.146+0000 7fe73a59c700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159890 0x7fe73c0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.147+0000 7fe731d9b700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 0x7fe73c150f30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0899c0 tx=0x7fe724017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.147+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72400bbe0 con 0x7fe73c159c60 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.147+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724011420 con 0x7fe73c159c60 2023-10-06T03:07:17.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.147+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7240117e0 con 0x7fe73c159c60 2023-10-06T03:07:17.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.147+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c153c60 con 0x7fe73c159c60 2023-10-06T03:07:17.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.147+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c154220 con 0x7fe73c159c60 2023-10-06T03:07:17.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.148+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c154720 con 0x7fe73c159c60 2023-10-06T03:07:17.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.148+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe724011c10 con 0x7fe73c159c60 2023-10-06T03:07:17.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.149+0000 7fe732ffd700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe71c067200 0x7fe71c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:17.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.149+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fe724086480 con 0x7fe73c159c60 2023-10-06T03:07:17.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.149+0000 7fe73a59c700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe71c067200 0x7fe71c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:17.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.149+0000 7fe73a59c700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe71c067200 0x7fe71c0696b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0011e0 tx=0x7fe72c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:17.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.153+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7240532a0 con 0x7fe73c159c60 2023-10-06T03:07:17.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:17.379+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe73c04c3c0 con 0x7fe73c159c60 2023-10-06T03:07:18.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.015+0000 7fe732ffd700 1 -- 172.21.15.92:0/3666988391 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe7240568c0 con 0x7fe73c159c60 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe71c067200 msgr2=0x7fe71c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe71c067200 0x7fe71c0696b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0011e0 tx=0x7fe72c009040 comp rx=0 tx=0).stop 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 msgr2=0x7fe73c150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 0x7fe73c150f30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0899c0 tx=0x7fe724017000 comp rx=0 tx=0).stop 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 shutdown_connections 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe71c067200 0x7fe71c0696b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe73c15db10 0x7fe73c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe73c159890 0x7fe73c0b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 --2- 172.21.15.92:0/3666988391 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe73c159c60 0x7fe73c150f30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 >> 172.21.15.92:0/3666988391 conn(0x7fe73c068e20 msgr2=0x7fe73c152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 shutdown_connections 2023-10-06T03:07:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.019+0000 7fe7409ed700 1 -- 172.21.15.92:0/3666988391 wait complete. 2023-10-06T03:07:18.022 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 15 and data pool 16 2023-10-06T03:07:18.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:18.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.148+0000 7fd128160700 1 Processor -- start 2023-10-06T03:07:18.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.148+0000 7fd128160700 1 -- start start 2023-10-06T03:07:18.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd128160700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd120159890 0x7fd12015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd128160700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd120069560 0x7fd1200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd128160700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd120069ee0 con 0x7fd120159c60 2023-10-06T03:07:18.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd128160700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12015a160 con 0x7fd120159890 2023-10-06T03:07:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd128160700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200d13a0 con 0x7fd120069560 2023-10-06T03:07:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd125efc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd120159890 0x7fd12015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd1256fb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd120159c60 0x7fd12015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40738/0 (socket says 172.21.15.92:40738) 2023-10-06T03:07:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd1256fb700 1 -- 172.21.15.92:0/4012227912 learned_addr learned my addr 172.21.15.92:0/4012227912 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd1266fd700 1 --2- 172.21.15.92:0/4012227912 >> v2:172.21.15.136:3300/0 conn(0x7fd120069560 0x7fd1200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd125efc700 1 -- 172.21.15.92:0/4012227912 >> v2:172.21.15.136:3300/0 conn(0x7fd120069560 msgr2=0x7fd1200699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd125efc700 1 --2- 172.21.15.92:0/4012227912 >> v2:172.21.15.136:3300/0 conn(0x7fd120069560 0x7fd1200699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd125efc700 1 -- 172.21.15.92:0/4012227912 >> v1:172.21.15.92:6789/0 conn(0x7fd120159c60 legacy=0x7fd12015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd124efa700 1 -- 172.21.15.92:0/4012227912 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434714146 0 0) 0x7fd120069ee0 con 0x7fd120159c60 2023-10-06T03:07:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd125efc700 1 -- 172.21.15.92:0/4012227912 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12015e140 con 0x7fd120159890 2023-10-06T03:07:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd125efc700 1 --2- 172.21.15.92:0/4012227912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120159890 0x7fd12015abd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd12006a890 tx=0x7fd118009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4138acbd5ac7ef21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd124efa700 1 -- 172.21.15.92:0/4012227912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800e070 con 0x7fd120159890 2023-10-06T03:07:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd124efa700 1 -- 172.21.15.92:0/4012227912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800bc30 con 0x7fd120159890 2023-10-06T03:07:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.149+0000 7fd124efa700 1 -- 172.21.15.92:0/4012227912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118012430 con 0x7fd120159890 2023-10-06T03:07:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 -- 172.21.15.92:0/4012227912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120159890 msgr2=0x7fd12015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 --2- 172.21.15.92:0/4012227912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120159890 0x7fd12015abd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd12006a890 tx=0x7fd118009780 comp rx=0 tx=0).stop 2023-10-06T03:07:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 -- 172.21.15.92:0/4012227912 shutdown_connections 2023-10-06T03:07:18.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 --2- 172.21.15.92:0/4012227912 >> v2:172.21.15.136:3300/0 conn(0x7fd120069560 0x7fd1200699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 --2- 172.21.15.92:0/4012227912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120159890 0x7fd12015abd0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 -- 172.21.15.92:0/4012227912 >> 172.21.15.92:0/4012227912 conn(0x7fd120068e20 msgr2=0x7fd1200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:18.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 -- 172.21.15.92:0/4012227912 shutdown_connections 2023-10-06T03:07:18.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.150+0000 7fd128160700 1 -- 172.21.15.92:0/4012227912 wait complete. 2023-10-06T03:07:18.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.151+0000 7fd128160700 1 Processor -- start 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.151+0000 7fd128160700 1 -- start start 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 0x7fd12014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd120159890 0x7fd1200cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd120159c60 0x7fd12015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200ce590 con 0x7fd120069560 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200ce870 con 0x7fd120159c60 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200ceb50 con 0x7fd120159890 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 0x7fd12014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 0x7fd12014b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50500/0 (socket says 172.21.15.92:50500) 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 -- 172.21.15.92:0/3334456784 learned_addr learned my addr 172.21.15.92:0/3334456784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 -- 172.21.15.92:0/3334456784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd120159890 msgr2=0x7fd1200cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd120159890 0x7fd1200cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 -- 172.21.15.92:0/3334456784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd120159c60 msgr2=0x7fd12015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd120159c60 0x7fd12015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 -- 172.21.15.92:0/3334456784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd118009a20 con 0x7fd120069560 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd125efc700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 0x7fd12014b790 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd1180099f0 tx=0x7fd118019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118001090 con 0x7fd120069560 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11801d630 con 0x7fd120069560 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.152+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1200cee30 con 0x7fd120069560 2023-10-06T03:07:18.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.153+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11801d9f0 con 0x7fd120069560 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.153+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd120152cd0 con 0x7fd120069560 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.154+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1200cf330 con 0x7fd120069560 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.154+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd118024070 con 0x7fd120069560 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.154+0000 7fd11dffb700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd10c067200 0x7fd10c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.155+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd1180873d0 con 0x7fd120069560 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.155+0000 7fd1256fb700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd10c067200 0x7fd10c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.155+0000 7fd1256fb700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd10c067200 0x7fd10c0696b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd12015cf30 tx=0x7fd114007bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.159+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd118054070 con 0x7fd120069560 2023-10-06T03:07:18.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.343+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd12004c3c0 con 0x7fd120069560 2023-10-06T03:07:18.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.344+0000 7fd11dffb700 1 -- 172.21.15.92:0/3334456784 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16882 (secure 0 0 0) 0x7fd118057690 con 0x7fd120069560 2023-10-06T03:07:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:18.346 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":89,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:17.381271+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:15.031911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:16.347808+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd10c067200 msgr2=0x7fd10c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd10c067200 0x7fd10c0696b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd12015cf30 tx=0x7fd114007bd0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 msgr2=0x7fd12014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 0x7fd12014b790 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd1180099f0 tx=0x7fd118019000 comp rx=0 tx=0).stop 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 shutdown_connections 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd10c067200 0x7fd10c0696b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd120069560 0x7fd12014b790 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd120159c60 0x7fd12015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 --2- 172.21.15.92:0/3334456784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd120159890 0x7fd1200cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 >> 172.21.15.92:0/3334456784 conn(0x7fd120068e20 msgr2=0x7fd1200b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:18.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 shutdown_connections 2023-10-06T03:07:18.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.346+0000 7fd128160700 1 -- 172.21.15.92:0/3334456784 wait complete. 2023-10-06T03:07:18.360 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.473+0000 7f9756eef700 1 Processor -- start 2023-10-06T03:07:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.473+0000 7f9756eef700 1 -- start start 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9756eef700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9750159890 0x7f975015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9756eef700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9750159c60 0x7f975015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9756eef700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f975015db10 0x7f9750069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9756eef700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975015df50 con 0x7f9750159c60 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9756eef700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975015a160 con 0x7f9750159890 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9756eef700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97500d13a0 con 0x7f975015db10 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9750159c60 0x7f975015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9750159c60 0x7f975015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50514/0 (socket says 172.21.15.92:50514) 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 -- 172.21.15.92:0/2315547737 learned_addr learned my addr 172.21.15.92:0/2315547737 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9754c8b700 1 --2- 172.21.15.92:0/2315547737 >> v2:172.21.15.136:3300/0 conn(0x7f9750159890 0x7f975015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f975548c700 1 --2- 172.21.15.92:0/2315547737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f975015db10 0x7f9750069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 -- 172.21.15.92:0/2315547737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f975015db10 msgr2=0x7f9750069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 --2- 172.21.15.92:0/2315547737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f975015db10 0x7f9750069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 -- 172.21.15.92:0/2315547737 >> v2:172.21.15.136:3300/0 conn(0x7f9750159890 msgr2=0x7f975015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 --2- 172.21.15.92:0/2315547737 >> v2:172.21.15.136:3300/0 conn(0x7f9750159890 0x7f975015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 -- 172.21.15.92:0/2315547737 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975015e140 con 0x7f9750159c60 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f9747fff700 1 --2- 172.21.15.92:0/2315547737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750159c60 0x7f975015d5d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f97500d1090 tx=0x7f9740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62defccfafd4f024 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.474+0000 7f97477fe700 1 -- 172.21.15.92:0/2315547737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000b860 con 0x7f9750159c60 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f97477fe700 1 -- 172.21.15.92:0/2315547737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974000b9c0 con 0x7f9750159c60 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f97477fe700 1 -- 172.21.15.92:0/2315547737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000bd60 con 0x7f9750159c60 2023-10-06T03:07:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 -- 172.21.15.92:0/2315547737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750159c60 msgr2=0x7f975015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 --2- 172.21.15.92:0/2315547737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750159c60 0x7f975015d5d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f97500d1090 tx=0x7f9740009250 comp rx=0 tx=0).stop 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 -- 172.21.15.92:0/2315547737 shutdown_connections 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 --2- 172.21.15.92:0/2315547737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f975015db10 0x7f9750069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 --2- 172.21.15.92:0/2315547737 >> v2:172.21.15.136:3300/0 conn(0x7f9750159890 0x7f975015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 --2- 172.21.15.92:0/2315547737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9750159c60 0x7f975015d5d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 -- 172.21.15.92:0/2315547737 >> 172.21.15.92:0/2315547737 conn(0x7f9750068e20 msgr2=0x7f97500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 -- 172.21.15.92:0/2315547737 shutdown_connections 2023-10-06T03:07:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.475+0000 7f9756eef700 1 -- 172.21.15.92:0/2315547737 wait complete. 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.476+0000 7f9756eef700 1 Processor -- start 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.476+0000 7f9756eef700 1 -- start start 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f9756eef700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9750159890 0x7f97500b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f9756eef700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9750159c60 0x7f97500b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f9756eef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 0x7f97500b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f9756eef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500b05d0 con 0x7f975015db10 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f9756eef700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500b46a0 con 0x7f9750159890 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f9756eef700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97500b4980 con 0x7f9750159c60 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 0x7f97500b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 0x7f97500b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50524/0 (socket says 172.21.15.92:50524) 2023-10-06T03:07:18.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 -- 172.21.15.92:0/1159179965 learned_addr learned my addr 172.21.15.92:0/1159179965 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 -- 172.21.15.92:0/1159179965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9750159c60 msgr2=0x7f97500b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9750159c60 0x7f97500b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 -- 172.21.15.92:0/1159179965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9750159890 msgr2=0x7f97500b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9750159890 0x7f97500b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 -- 172.21.15.92:0/1159179965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400094f0 con 0x7f975015db10 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.477+0000 7f975548c700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 0x7f97500b5550 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f974800d890 tx=0x7f9748008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.478+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97480125f0 con 0x7f975015db10 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.478+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9748012770 con 0x7f975015db10 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.478+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97500b07e0 con 0x7f975015db10 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.478+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748012b30 con 0x7f975015db10 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.478+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97500c8560 con 0x7f975015db10 2023-10-06T03:07:18.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.479+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97500b4ba0 con 0x7f975015db10 2023-10-06T03:07:18.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.479+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f974801e460 con 0x7f975015db10 2023-10-06T03:07:18.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.479+0000 7f9744ff9700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f973c067200 0x7f973c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.479+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9748085240 con 0x7f975015db10 2023-10-06T03:07:18.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.480+0000 7f9754c8b700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f973c067200 0x7f973c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.480+0000 7f9754c8b700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f973c067200 0x7f973c0696b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f974c009fd0 tx=0x7f974c009970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.484+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9748052000 con 0x7f975015db10 2023-10-06T03:07:18.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.667+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97500b0d00 con 0x7f975015db10 2023-10-06T03:07:18.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.668+0000 7f9744ff9700 1 -- 172.21.15.92:0/1159179965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f97500b0d00 con 0x7f975015db10 2023-10-06T03:07:18.670 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:18.670 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:18.512185+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:15.031911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:16.347808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f973c067200 msgr2=0x7f973c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f973c067200 0x7f973c0696b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f974c009fd0 tx=0x7f974c009970 comp rx=0 tx=0).stop 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 msgr2=0x7f97500b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 0x7f97500b5550 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f974800d890 tx=0x7f9748008040 comp rx=0 tx=0).stop 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 shutdown_connections 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f973c067200 0x7f973c0696b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f975015db10 0x7f97500b5550 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9750159890 0x7f97500b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 --2- 172.21.15.92:0/1159179965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9750159c60 0x7f97500b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 >> 172.21.15.92:0/1159179965 conn(0x7f9750068e20 msgr2=0x7f97500cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 shutdown_connections 2023-10-06T03:07:18.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.671+0000 7f9756eef700 1 -- 172.21.15.92:0/1159179965 wait complete. 2023-10-06T03:07:18.685 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:07:18.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.800+0000 7f2df89d1700 1 Processor -- start 2023-10-06T03:07:18.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.800+0000 7f2df89d1700 1 -- start start 2023-10-06T03:07:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df89d1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2df4159c60 0x7f2df415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df89d1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df415a160 con 0x7f2df415db10 2023-10-06T03:07:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df89d1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40d13a0 con 0x7f2df4159890 2023-10-06T03:07:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df89d1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df415df50 con 0x7f2df4159c60 2023-10-06T03:07:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df2d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2df415db10 0x7f2df4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40748/0 (socket says 172.21.15.92:40748) 2023-10-06T03:07:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df2d9d700 1 -- 172.21.15.92:0/1424824907 learned_addr learned my addr 172.21.15.92:0/1424824907 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2de9d9b700 1 --2- 172.21.15.92:0/1424824907 >> v2:172.21.15.136:3300/0 conn(0x7f2df4159c60 0x7f2df415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445642023 0 0) 0x7f2df415a160 con 0x7f2df415db10 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.801+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de0003570 con 0x7f2df415db10 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138066821 0 0) 0x7f2df40d13a0 con 0x7f2df4159890 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df415a160 con 0x7f2df4159890 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926667963 0 0) 0x7f2de0003570 con 0x7f2df415db10 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 >> v2:172.21.15.136:3300/0 conn(0x7f2df4159c60 msgr2=0x7f2df415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 --2- 172.21.15.92:0/1424824907 >> v2:172.21.15.136:3300/0 conn(0x7f2df4159c60 0x7f2df415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 >> v1:172.21.15.136:6789/0 conn(0x7f2df4159890 legacy=0x7f2df415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df415e140 con 0x7f2df415db10 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2dec0017f0 con 0x7f2df415db10 2023-10-06T03:07:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2de9d9b700 1 --2- 172.21.15.92:0/1424824907 >> v2:172.21.15.136:3300/0 conn(0x7f2df4159c60 0x7f2df415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 >> v1:172.21.15.92:6789/0 conn(0x7f2df415db10 legacy=0x7f2df4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2de0003c40 0x7f2de0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de0004810 0x7f2de0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0005210 0x7f2de000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de000c130 con 0x7f2de0003c40 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de000c270 con 0x7f2de0005210 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de000c400 con 0x7f2de0004810 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2de0003c40 0x7f2de0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df2d9d700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0005210 0x7f2de000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2de9d9b700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de0004810 0x7f2de0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 -- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de0004810 msgr2=0x7f2de0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de0004810 0x7f2de0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 -- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0005210 msgr2=0x7f2de000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0005210 0x7f2de000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 -- 172.21.15.92:0/1424824907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de0003570 con 0x7f2de0003c40 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df2d9d700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0005210 0x7f2de000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df2d9d700 1 -- 172.21.15.92:0/1424824907 reap_dead start 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df259c700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2de0003c40 0x7f2de0004030 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001a40 tx=0x7f2de4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543a448a29ebcae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de400c590 con 0x7f2de0003c40 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de400c6f0 con 0x7f2de0003c40 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.802+0000 7f2df1d9b700 1 -- 172.21.15.92:0/1424824907 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de400ca90 con 0x7f2de0003c40 2023-10-06T03:07:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 -- 172.21.15.92:0/1424824907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2de0003c40 msgr2=0x7f2de0004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2de0003c40 0x7f2de0004030 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2de4001a40 tx=0x7f2de4007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 -- 172.21.15.92:0/1424824907 shutdown_connections 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 --2- 172.21.15.92:0/1424824907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2de0003c40 0x7f2de0004030 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 -- 172.21.15.92:0/1424824907 >> 172.21.15.92:0/1424824907 conn(0x7f2df4068e20 msgr2=0x7f2df40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 -- 172.21.15.92:0/1424824907 shutdown_connections 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.803+0000 7f2df89d1700 1 -- 172.21.15.92:0/1424824907 wait complete. 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.804+0000 7f2df89d1700 1 Processor -- start 2023-10-06T03:07:18.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.804+0000 7f2df89d1700 1 -- start start 2023-10-06T03:07:18.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df89d1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0003c40 0x7f2df40ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df89d1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2df40cf050 0x7f2df40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df89d1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 0x7f2df40b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df89d1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4151b40 con 0x7f2df40b3ad0 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df89d1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40cf620 con 0x7f2de0003c40 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df89d1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40cf900 con 0x7f2df40cf050 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 0x7f2df40b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 0x7f2df40b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50534/0 (socket says 172.21.15.92:50534) 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 -- 172.21.15.92:0/3561445490 learned_addr learned my addr 172.21.15.92:0/3561445490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2de9d9b700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2df40cf050 0x7f2df40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df259c700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0003c40 0x7f2df40ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 -- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2df40cf050 msgr2=0x7f2df40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2df40cf050 0x7f2df40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 -- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0003c40 msgr2=0x7f2df40ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0003c40 0x7f2df40ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 -- 172.21.15.92:0/3561445490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4151480 con 0x7f2df40b3ad0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.805+0000 7f2df2d9d700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 0x7f2df40b3f40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2df406a300 tx=0x7f2dec0018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.806+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec00ebc0 con 0x7f2df40b3ad0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.806+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec00ed40 con 0x7f2df40b3ad0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.806+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df4151760 con 0x7f2df40b3ad0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.806+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec013670 con 0x7f2df40b3ad0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.806+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df40b6390 con 0x7f2df40b3ad0 2023-10-06T03:07:18.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.806+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4151c80 con 0x7f2df40b3ad0 2023-10-06T03:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.807+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2dec013a20 con 0x7f2df40b3ad0 2023-10-06T03:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.807+0000 7f2deaffd700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2dd4067200 0x7f2dd40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.807+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2dec084f00 con 0x7f2df40b3ad0 2023-10-06T03:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.807+0000 7f2df259c700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2dd4067200 0x7f2dd40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.808+0000 7f2df259c700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2dd4067200 0x7f2dd40696b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2de40019e0 tx=0x7f2de4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:18.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:18.811+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2df4151c80 con 0x7f2df40b3ad0 2023-10-06T03:07:19.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:19.040+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2df4151c80 con 0x7f2df40b3ad0 2023-10-06T03:07:20.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.036+0000 7f2deaffd700 1 -- 172.21.15.92:0/3561445490 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2df4151c80 con 0x7f2df40b3ad0 2023-10-06T03:07:20.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2dd4067200 msgr2=0x7f2dd40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:20.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2dd4067200 0x7f2dd40696b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2de40019e0 tx=0x7f2de4015000 comp rx=0 tx=0).stop 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 msgr2=0x7f2df40b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 0x7f2df40b3f40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2df406a300 tx=0x7f2dec0018a0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 shutdown_connections 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2dd4067200 0x7f2dd40696b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.038+0000 7f2df89d1700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2df40b3ad0 0x7f2df40b3f40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.039+0000 7f2df89d1700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de0003c40 0x7f2df40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.039+0000 7f2df89d1700 1 --2- 172.21.15.92:0/3561445490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2df40cf050 0x7f2df40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.039+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 >> 172.21.15.92:0/3561445490 conn(0x7f2df4068e20 msgr2=0x7f2df40d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.039+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 shutdown_connections 2023-10-06T03:07:20.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.039+0000 7f2df89d1700 1 -- 172.21.15.92:0/3561445490 wait complete. 2023-10-06T03:07:20.053 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:07:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.170+0000 7fd97a52e700 1 Processor -- start 2023-10-06T03:07:20.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.170+0000 7fd97a52e700 1 -- start start 2023-10-06T03:07:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd97a52e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd974159890 0x7fd97415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd97a52e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd974069560 0x7fd9740699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd97a52e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd974069ee0 con 0x7fd974159c60 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd97a52e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97415a160 con 0x7fd974159890 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd97a52e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9740d13a0 con 0x7fd974069560 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd973fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd974159890 0x7fd97415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd9737fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd974159c60 0x7fd97415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46882/0 (socket says 172.21.15.92:46882) 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd9737fe700 1 -- 172.21.15.92:0/2506861447 learned_addr learned my addr 172.21.15.92:0/2506861447 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.171+0000 7fd978acb700 1 --2- 172.21.15.92:0/2506861447 >> v2:172.21.15.136:3300/0 conn(0x7fd974069560 0x7fd9740699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd973fff700 1 -- 172.21.15.92:0/2506861447 >> v2:172.21.15.136:3300/0 conn(0x7fd974069560 msgr2=0x7fd9740699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd973fff700 1 --2- 172.21.15.92:0/2506861447 >> v2:172.21.15.136:3300/0 conn(0x7fd974069560 0x7fd9740699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd973fff700 1 -- 172.21.15.92:0/2506861447 >> v1:172.21.15.92:6789/0 conn(0x7fd974159c60 legacy=0x7fd97415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd973fff700 1 -- 172.21.15.92:0/2506861447 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97415e140 con 0x7fd974159890 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd972ffd700 1 -- 172.21.15.92:0/2506861447 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870974082 0 0) 0x7fd974069ee0 con 0x7fd974159c60 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd973fff700 1 --2- 172.21.15.92:0/2506861447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd97415abd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd97406a890 tx=0x7fd9640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3218dd2d345eacb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd972ffd700 1 -- 172.21.15.92:0/2506861447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964011630 con 0x7fd974159890 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd972ffd700 1 -- 172.21.15.92:0/2506861447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964011790 con 0x7fd974159890 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd972ffd700 1 -- 172.21.15.92:0/2506861447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964011b30 con 0x7fd974159890 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd97a52e700 1 -- 172.21.15.92:0/2506861447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 msgr2=0x7fd97415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd97a52e700 1 --2- 172.21.15.92:0/2506861447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd97415abd0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd97406a890 tx=0x7fd9640091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd97a52e700 1 -- 172.21.15.92:0/2506861447 shutdown_connections 2023-10-06T03:07:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd97a52e700 1 --2- 172.21.15.92:0/2506861447 >> v2:172.21.15.136:3300/0 conn(0x7fd974069560 0x7fd9740699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd97a52e700 1 --2- 172.21.15.92:0/2506861447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd97415abd0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.172+0000 7fd97a52e700 1 -- 172.21.15.92:0/2506861447 >> 172.21.15.92:0/2506861447 conn(0x7fd974068e20 msgr2=0x7fd9740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.173+0000 7fd97a52e700 1 -- 172.21.15.92:0/2506861447 shutdown_connections 2023-10-06T03:07:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.173+0000 7fd97a52e700 1 -- 172.21.15.92:0/2506861447 wait complete. 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.174+0000 7fd97a52e700 1 Processor -- start 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.174+0000 7fd97a52e700 1 -- start start 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.174+0000 7fd97a52e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd974069560 0x7fd97415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.174+0000 7fd97a52e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd974149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.174+0000 7fd97a52e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd974159c60 0x7fd97415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd97a52e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd974153180 con 0x7fd974159890 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd97a52e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740cd380 con 0x7fd974159c60 2023-10-06T03:07:20.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd97a52e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97414a160 con 0x7fd974069560 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd974149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd974149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36934/0 (socket says 172.21.15.92:36934) 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 -- 172.21.15.92:0/3919394489 learned_addr learned my addr 172.21.15.92:0/3919394489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd973fff700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd974069560 0x7fd97415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd978acb700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd974159c60 0x7fd97415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 -- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd974069560 msgr2=0x7fd97415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd974069560 0x7fd97415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 -- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd974159c60 msgr2=0x7fd97415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd974159c60 0x7fd97415ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 -- 172.21.15.92:0/3919394489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964009420 con 0x7fd974159890 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd973fff700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd974069560 0x7fd97415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd9737fe700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd974149c20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c5b0 tx=0x7fd968001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968005b30 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968005c90 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968007300 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97415cfb0 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.175+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97415d570 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.176+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd974153330 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.177+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd96801b3b0 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.177+0000 7fd95bfff700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd960067200 0x7fd9600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.177+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd968084010 con 0x7fd974159890 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.177+0000 7fd973fff700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd960067200 0x7fd9600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.177+0000 7fd973fff700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd960067200 0x7fd9600696b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd9640093f0 tx=0x7fd964017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.181+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd974153330 con 0x7fd974159890 2023-10-06T03:07:20.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:20.406+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd974153330 con 0x7fd974159890 2023-10-06T03:07:21.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.037+0000 7fd95bfff700 1 -- 172.21.15.92:0/3919394489 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd974153330 con 0x7fd974159890 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd960067200 msgr2=0x7fd9600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd960067200 0x7fd9600696b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd9640093f0 tx=0x7fd964017000 comp rx=0 tx=0).stop 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 msgr2=0x7fd974149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd974149c20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c5b0 tx=0x7fd968001070 comp rx=0 tx=0).stop 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 shutdown_connections 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd960067200 0x7fd9600696b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd974159c60 0x7fd97415ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd974069560 0x7fd97415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 --2- 172.21.15.92:0/3919394489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd974159890 0x7fd974149c20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 >> 172.21.15.92:0/3919394489 conn(0x7fd974068e20 msgr2=0x7fd9740b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:21.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 shutdown_connections 2023-10-06T03:07:21.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.040+0000 7fd97a52e700 1 -- 172.21.15.92:0/3919394489 wait complete. 2023-10-06T03:07:21.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:21.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.168+0000 7f38d5bc8700 1 Processor -- start 2023-10-06T03:07:21.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.168+0000 7f38d5bc8700 1 -- start start 2023-10-06T03:07:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38d5bc8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f38d0159890 0x7f38d015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38d5bc8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38d0069560 0x7f38d00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38d5bc8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d0069ee0 con 0x7f38d0159c60 2023-10-06T03:07:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38d5bc8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d015a160 con 0x7f38d0069560 2023-10-06T03:07:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38d5bc8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00d13a0 con 0x7f38d0159890 2023-10-06T03:07:21.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38cffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38d0069560 0x7f38d00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38ceffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f38d0159c60 0x7f38d015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46894/0 (socket says 172.21.15.92:46894) 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38ceffd700 1 -- 172.21.15.92:0/2113056164 learned_addr learned my addr 172.21.15.92:0/2113056164 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.169+0000 7f38cf7fe700 1 --2- 172.21.15.92:0/2113056164 >> v2:172.21.15.136:3300/0 conn(0x7f38d0159890 0x7f38d015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38cffff700 1 -- 172.21.15.92:0/2113056164 >> v2:172.21.15.136:3300/0 conn(0x7f38d0159890 msgr2=0x7f38d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38cffff700 1 --2- 172.21.15.92:0/2113056164 >> v2:172.21.15.136:3300/0 conn(0x7f38d0159890 0x7f38d015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38cffff700 1 -- 172.21.15.92:0/2113056164 >> v1:172.21.15.92:6789/0 conn(0x7f38d0159c60 legacy=0x7f38d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38ce7fc700 1 -- 172.21.15.92:0/2113056164 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748314695 0 0) 0x7f38d0069ee0 con 0x7f38d0159c60 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38cffff700 1 -- 172.21.15.92:0/2113056164 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d015e140 con 0x7f38d0069560 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38cffff700 1 --2- 172.21.15.92:0/2113056164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0069560 0x7f38d00699a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f38d00d1090 tx=0x7f38c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f93e2bfcdca3abb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38ce7fc700 1 -- 172.21.15.92:0/2113056164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800b6b0 con 0x7f38d0069560 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38ce7fc700 1 -- 172.21.15.92:0/2113056164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c800b810 con 0x7f38d0069560 2023-10-06T03:07:21.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38ce7fc700 1 -- 172.21.15.92:0/2113056164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c800bbb0 con 0x7f38d0069560 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 -- 172.21.15.92:0/2113056164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0069560 msgr2=0x7f38d00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/2113056164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0069560 0x7f38d00699a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f38d00d1090 tx=0x7f38c80091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 -- 172.21.15.92:0/2113056164 shutdown_connections 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/2113056164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0069560 0x7f38d00699a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/2113056164 >> v2:172.21.15.136:3300/0 conn(0x7f38d0159890 0x7f38d015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 -- 172.21.15.92:0/2113056164 >> 172.21.15.92:0/2113056164 conn(0x7f38d0068e20 msgr2=0x7f38d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 -- 172.21.15.92:0/2113056164 shutdown_connections 2023-10-06T03:07:21.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.170+0000 7f38d5bc8700 1 -- 172.21.15.92:0/2113056164 wait complete. 2023-10-06T03:07:21.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 Processor -- start 2023-10-06T03:07:21.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 -- start start 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d0069560 0x7f38d014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0159890 0x7f38d00ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 0x7f38d00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d014bee0 con 0x7f38d0159c60 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d0152f10 con 0x7f38d0069560 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38d5bc8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00b5cf0 con 0x7f38d0159890 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 0x7f38d00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 0x7f38d00cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36950/0 (socket says 172.21.15.92:36950) 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.172+0000 7f38cffff700 1 -- 172.21.15.92:0/4134982142 learned_addr learned my addr 172.21.15.92:0/4134982142 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38cffff700 1 -- 172.21.15.92:0/4134982142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0159890 msgr2=0x7f38d00ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38cffff700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0159890 0x7f38d00ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38cffff700 1 -- 172.21.15.92:0/4134982142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d0069560 msgr2=0x7f38d014b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38cffff700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d0069560 0x7f38d014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38cffff700 1 -- 172.21.15.92:0/4134982142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009420 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38cffff700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 0x7f38d00cf2c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38d0070e20 tx=0x7f38c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8025420 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8025580 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8025920 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00b5fd0 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.173+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00b3600 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.174+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d0152a00 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.174+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f38c8023070 con 0x7f38d0159c60 2023-10-06T03:07:21.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.175+0000 7f38b77fe700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38bc067200 0x7f38bc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.175+0000 7f38cf7fe700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38bc067200 0x7f38bc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.175+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f38c80862c0 con 0x7f38d0159c60 2023-10-06T03:07:21.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.175+0000 7f38cf7fe700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38bc067200 0x7f38bc0696b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f38c0007f50 tx=0x7f38c0007440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.179+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38d0152a00 con 0x7f38d0159c60 2023-10-06T03:07:21.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.406+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38d0152a00 con 0x7f38d0159c60 2023-10-06T03:07:21.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.406+0000 7f38b77fe700 1 -- 172.21.15.92:0/4134982142 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f38d0152a00 con 0x7f38d0159c60 2023-10-06T03:07:21.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:21.408 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:17.381585+0000","modified":"2023-10-06T03:07:21.036747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6480":{"gid":6480,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/428935748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":428935748},{"type":"v1","addr":"172.21.15.92:6837","nonce":428935748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:07:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38bc067200 msgr2=0x7f38bc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38bc067200 0x7f38bc0696b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f38c0007f50 tx=0x7f38c0007440 comp rx=0 tx=0).stop 2023-10-06T03:07:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 msgr2=0x7f38d00cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 0x7f38d00cf2c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f38d0070e20 tx=0x7f38c8017000 comp rx=0 tx=0).stop 2023-10-06T03:07:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 shutdown_connections 2023-10-06T03:07:21.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38bc067200 0x7f38bc0696b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d0159c60 0x7f38d00cf2c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d0069560 0x7f38d014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 --2- 172.21.15.92:0/4134982142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0159890 0x7f38d00ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 >> 172.21.15.92:0/4134982142 conn(0x7f38d0068e20 msgr2=0x7f38d0158770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 shutdown_connections 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.408+0000 7f38d5bc8700 1 -- 172.21.15.92:0/4134982142 wait complete. 2023-10-06T03:07:21.411 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 56 2023-10-06T03:07:21.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.535+0000 7f7e1b59e700 1 Processor -- start 2023-10-06T03:07:21.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.535+0000 7f7e1b59e700 1 -- start start 2023-10-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e1c159420 0x7f7e1c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1b59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7e1c15d230 0x7f7e1c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1c15dbd0 con 0x7f7e1c159420 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1b59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1c0b11d0 con 0x7f7e1c15d230 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0b14b0 con 0x7f7e1c1597f0 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e1c159420 0x7f7e1c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e1c159420 0x7f7e1c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36966/0 (socket says 172.21.15.92:36966) 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 -- 172.21.15.92:0/1796758337 learned_addr learned my addr 172.21.15.92:0/1796758337 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1ad9d700 1 --2- 172.21.15.92:0/1796758337 >> v2:172.21.15.136:3300/0 conn(0x7f7e1c15d230 0x7f7e1c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e11d9b700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 -- 172.21.15.92:0/1796758337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 msgr2=0x7f7e1c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 -- 172.21.15.92:0/1796758337 >> v2:172.21.15.136:3300/0 conn(0x7f7e1c15d230 msgr2=0x7f7e1c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 --2- 172.21.15.92:0/1796758337 >> v2:172.21.15.136:3300/0 conn(0x7f7e1c15d230 0x7f7e1c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 -- 172.21.15.92:0/1796758337 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1c0b1790 con 0x7f7e1c159420 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1ad9d700 1 --2- 172.21.15.92:0/1796758337 >> v2:172.21.15.136:3300/0 conn(0x7f7e1c15d230 0x7f7e1c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e11d9b700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e1a59c700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c0b5f40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0d1720 tx=0x7f7e0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ca6ac479272744 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.536+0000 7f7e19d9b700 1 -- 172.21.15.92:0/1796758337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00b860 con 0x7f7e1c159420 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e19d9b700 1 -- 172.21.15.92:0/1796758337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0c00b9c0 con 0x7f7e1c159420 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e19d9b700 1 -- 172.21.15.92:0/1796758337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00bd60 con 0x7f7e1c159420 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 -- 172.21.15.92:0/1796758337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 msgr2=0x7f7e1c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c0b5f40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0d1720 tx=0x7f7e0c009250 comp rx=0 tx=0).stop 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 -- 172.21.15.92:0/1796758337 shutdown_connections 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/1796758337 >> v2:172.21.15.136:3300/0 conn(0x7f7e1c15d230 0x7f7e1c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c0b5f40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/1796758337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 -- 172.21.15.92:0/1796758337 >> 172.21.15.92:0/1796758337 conn(0x7f7e1c0d0660 msgr2=0x7f7e1c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 -- 172.21.15.92:0/1796758337 shutdown_connections 2023-10-06T03:07:21.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.537+0000 7f7e1b59e700 1 -- 172.21.15.92:0/1796758337 wait complete. 2023-10-06T03:07:21.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.538+0000 7f7e1b59e700 1 Processor -- start 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.538+0000 7f7e1b59e700 1 -- start start 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c15d230 0x7f7e1c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c1415d0 con 0x7f7e1c159420 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c15aa80 con 0x7f7e1c15d230 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c15ad60 con 0x7f7e1c1597f0 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36970/0 (socket says 172.21.15.92:36970) 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 -- 172.21.15.92:0/2078777305 learned_addr learned my addr 172.21.15.92:0/2078777305 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:21.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e11d9b700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1ad9d700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c15d230 0x7f7e1c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 -- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 msgr2=0x7f7e1c169af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c169af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 -- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c15d230 msgr2=0x7f7e1c1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.539+0000 7f7e1a59c700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c15d230 0x7f7e1c1521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e1a59c700 1 -- 172.21.15.92:0/2078777305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0094f0 con 0x7f7e1c159420 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e1ad9d700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c15d230 0x7f7e1c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e1a59c700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c141090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c089340 tx=0x7f7e0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c011e70 con 0x7f7e1c159420 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0c024440 con 0x7f7e1c159420 2023-10-06T03:07:21.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c15b040 con 0x7f7e1c159420 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c024800 con 0x7f7e1c159420 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.540+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c15b600 con 0x7f7e1c159420 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.541+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1c15b7f0 con 0x7f7e1c159420 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.541+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7e0c01a420 con 0x7f7e1c159420 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.542+0000 7f7e12ffd700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7dfc067200 0x7f7dfc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.542+0000 7f7e11d9b700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7dfc067200 0x7f7dfc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.542+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e0c086ab0 con 0x7f7e1c159420 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.542+0000 7f7e11d9b700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7dfc067200 0x7f7dfc0696b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c15a630 tx=0x7f7e04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.546+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e0c053870 con 0x7f7e1c159420 2023-10-06T03:07:21.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.729+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e1c04c3c0 con 0x7f7e1c159420 2023-10-06T03:07:21.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.730+0000 7f7e12ffd700 1 -- 172.21.15.92:0/2078777305 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16869 (secure 0 0 0) 0x7f7e0c015090 con 0x7f7e1c159420 2023-10-06T03:07:21.732 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:21.732 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":90,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:18.512185+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:15.031911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:16.347808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7dfc067200 msgr2=0x7f7dfc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7dfc067200 0x7f7dfc0696b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c15a630 tx=0x7f7e04008040 comp rx=0 tx=0).stop 2023-10-06T03:07:21.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 msgr2=0x7f7e1c141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c141090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c089340 tx=0x7f7e0c017000 comp rx=0 tx=0).stop 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 shutdown_connections 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7dfc067200 0x7f7dfc0696b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e1c159420 0x7f7e1c141090 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e1c15d230 0x7f7e1c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 --2- 172.21.15.92:0/2078777305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e1c1597f0 0x7f7e1c169af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 >> 172.21.15.92:0/2078777305 conn(0x7f7e1c0d0660 msgr2=0x7f7e1c0698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 shutdown_connections 2023-10-06T03:07:21.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.732+0000 7f7e1b59e700 1 -- 172.21.15.92:0/2078777305 wait complete. 2023-10-06T03:07:21.746 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:07:21.746 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.859+0000 7f72c735b700 1 Processor -- start 2023-10-06T03:07:21.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.859+0000 7f72c735b700 1 -- start start 2023-10-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c735b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72c0159420 0x7f72c00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c735b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c735b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c00b2980 con 0x7f72c01597f0 2023-10-06T03:07:21.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c735b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c015dbb0 con 0x7f72c0159420 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c735b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c00b11d0 con 0x7f72c015d230 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c6359700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72c0159420 0x7f72c00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c5b58700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f72c01597f0 0x7f72c00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46896/0 (socket says 172.21.15.92:46896) 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c5b58700 1 -- 172.21.15.92:0/3435279421 learned_addr learned my addr 172.21.15.92:0/3435279421 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c6359700 1 -- 172.21.15.92:0/3435279421 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 msgr2=0x7f72c015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c6b5a700 1 --2- 172.21.15.92:0/3435279421 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c6359700 1 --2- 172.21.15.92:0/3435279421 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c015d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c5357700 1 -- 172.21.15.92:0/3435279421 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504542896 0 0) 0x7f72c00b2980 con 0x7f72c01597f0 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c6359700 1 -- 172.21.15.92:0/3435279421 >> v1:172.21.15.92:6789/0 conn(0x7f72c01597f0 legacy=0x7f72c00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.860+0000 7f72c6359700 1 -- 172.21.15.92:0/3435279421 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00b14b0 con 0x7f72c0159420 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c6359700 1 --2- 172.21.15.92:0/3435279421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c0159420 0x7f72c00b5f40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0098e0 tx=0x7f72bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50bab5dce1470aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c5357700 1 -- 172.21.15.92:0/3435279421 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc009e80 con 0x7f72c0159420 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c5357700 1 -- 172.21.15.92:0/3435279421 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc00b860 con 0x7f72c0159420 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c5357700 1 -- 172.21.15.92:0/3435279421 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00bbe0 con 0x7f72c0159420 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 -- 172.21.15.92:0/3435279421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c0159420 msgr2=0x7f72c00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 --2- 172.21.15.92:0/3435279421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c0159420 0x7f72c00b5f40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0098e0 tx=0x7f72bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 -- 172.21.15.92:0/3435279421 shutdown_connections 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 --2- 172.21.15.92:0/3435279421 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 --2- 172.21.15.92:0/3435279421 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c0159420 0x7f72c00b5f40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 -- 172.21.15.92:0/3435279421 >> 172.21.15.92:0/3435279421 conn(0x7f72c00d0660 msgr2=0x7f72c00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 -- 172.21.15.92:0/3435279421 shutdown_connections 2023-10-06T03:07:21.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.861+0000 7f72c735b700 1 -- 172.21.15.92:0/3435279421 wait complete. 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.862+0000 7f72c735b700 1 Processor -- start 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 -- start start 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72c0159420 0x7f72c01428d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 0x7f72c00c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c00c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c00c6510 con 0x7f72c01597f0 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c00c5900 con 0x7f72c0159420 2023-10-06T03:07:21.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c735b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c00c5be0 con 0x7f72c015d230 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c5b58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 0x7f72c00c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c5b58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 0x7f72c00c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36976/0 (socket says 172.21.15.92:36976) 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c5b58700 1 -- 172.21.15.92:0/3677030821 learned_addr learned my addr 172.21.15.92:0/3677030821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.863+0000 7f72c6359700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72c0159420 0x7f72c01428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c6b5a700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c00c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c5b58700 1 -- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 msgr2=0x7f72c00c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c5b58700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c00c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c5b58700 1 -- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72c0159420 msgr2=0x7f72c01428d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c5b58700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72c0159420 0x7f72c01428d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c5b58700 1 -- 172.21.15.92:0/3677030821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc009420 con 0x7f72c01597f0 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c5b58700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 0x7f72c00c69a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c810 tx=0x7f72b0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0005c00 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0005d60 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c00c5ec0 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007320 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.864+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c00c5400 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.865+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c00c5680 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.865+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f72b001b400 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.866+0000 7f72b67fc700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a4067280 0x7f72a4069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.866+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72b0084600 con 0x7f72c01597f0 2023-10-06T03:07:21.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.866+0000 7f72c6359700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a4067280 0x7f72a4069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.866+0000 7f72c6359700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a4067280 0x7f72a4069730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0091e0 tx=0x7f72bc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:21.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:21.870+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72b0051400 con 0x7f72c01597f0 2023-10-06T03:07:22.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.049+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72c004c3c0 con 0x7f72a4067280 2023-10-06T03:07:22.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.058+0000 7f72b67fc700 1 -- 172.21.15.92:0/3677030821 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311762 (secure 0 0 0) 0x7f72c004c3c0 con 0x7f72a4067280 2023-10-06T03:07:22.063 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a4067280 msgr2=0x7f72a4069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a4067280 0x7f72a4069730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0091e0 tx=0x7f72bc017000 comp rx=0 tx=0).stop 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 msgr2=0x7f72c00c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 0x7f72c00c69a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c810 tx=0x7f72b0007db0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 shutdown_connections 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72a4067280 0x7f72a4069730 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72c015d230 0x7f72c00c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72c0159420 0x7f72c01428d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 --2- 172.21.15.92:0/3677030821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72c01597f0 0x7f72c00c69a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 >> 172.21.15.92:0/3677030821 conn(0x7f72c00d0660 msgr2=0x7f72c00cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 shutdown_connections 2023-10-06T03:07:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.061+0000 7f72c735b700 1 -- 172.21.15.92:0/3677030821 wait complete. 2023-10-06T03:07:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:22.076 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-10-06T03:07:21.252029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400448,"kb_used_data":9856,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752574272,"statfs":{"total":773094113280,"available":770636054528,"internally_reserved":0,"allocated":10092544,"data_stored":5195686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.261227"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527339+0000","last_change":"2023-10-06T03:07:17.527339+0000","last_active":"2023-10-06T03:07:17.527339+0000","last_peered":"2023-10-06T03:07:17.527339+0000","last_clean":"2023-10-06T03:07:17.527339+0000","last_became_active":"2023-10-06T03:07:17.527057+0000","last_became_peered":"2023-10-06T03:07:17.527057+0000","last_unstale":"2023-10-06T03:07:17.527339+0000","last_undegraded":"2023-10-06T03:07:17.527339+0000","last_fullsized":"2023-10-06T03:07:17.527339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:26.929634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.544153+0000","last_change":"2023-10-06T03:07:17.538819+0000","last_active":"2023-10-06T03:07:17.544153+0000","last_peered":"2023-10-06T03:07:17.544153+0000","last_clean":"2023-10-06T03:07:17.544153+0000","last_became_active":"2023-10-06T03:07:17.538545+0000","last_became_peered":"2023-10-06T03:07:17.538545+0000","last_unstale":"2023-10-06T03:07:17.544153+0000","last_undegraded":"2023-10-06T03:07:17.544153+0000","last_fullsized":"2023-10-06T03:07:17.544153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:10.728835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.529417+0000","last_change":"2023-10-06T03:07:17.529417+0000","last_active":"2023-10-06T03:07:17.529417+0000","last_peered":"2023-10-06T03:07:17.529417+0000","last_clean":"2023-10-06T03:07:17.529417+0000","last_became_active":"2023-10-06T03:07:17.529154+0000","last_became_peered":"2023-10-06T03:07:17.529154+0000","last_unstale":"2023-10-06T03:07:17.529417+0000","last_undegraded":"2023-10-06T03:07:17.529417+0000","last_fullsized":"2023-10-06T03:07:17.529417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:46.786714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534521+0000","last_change":"2023-10-06T03:07:17.530089+0000","last_active":"2023-10-06T03:07:17.534521+0000","last_peered":"2023-10-06T03:07:17.534521+0000","last_clean":"2023-10-06T03:07:17.534521+0000","last_became_active":"2023-10-06T03:07:17.529839+0000","last_became_peered":"2023-10-06T03:07:17.529839+0000","last_unstale":"2023-10-06T03:07:17.534521+0000","last_undegraded":"2023-10-06T03:07:17.534521+0000","last_fullsized":"2023-10-06T03:07:17.534521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:38.597127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534743+0000","last_change":"2023-10-06T03:07:17.528293+0000","last_active":"2023-10-06T03:07:17.534743+0000","last_peered":"2023-10-06T03:07:17.534743+0000","last_clean":"2023-10-06T03:07:17.534743+0000","last_became_active":"2023-10-06T03:07:17.527792+0000","last_became_peered":"2023-10-06T03:07:17.527792+0000","last_unstale":"2023-10-06T03:07:17.534743+0000","last_undegraded":"2023-10-06T03:07:17.534743+0000","last_fullsized":"2023-10-06T03:07:17.534743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:48.414529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.543618+0000","last_change":"2023-10-06T03:07:17.537962+0000","last_active":"2023-10-06T03:07:17.543618+0000","last_peered":"2023-10-06T03:07:17.543618+0000","last_clean":"2023-10-06T03:07:17.543618+0000","last_became_active":"2023-10-06T03:07:17.537630+0000","last_became_peered":"2023-10-06T03:07:17.537630+0000","last_unstale":"2023-10-06T03:07:17.543618+0000","last_undegraded":"2023-10-06T03:07:17.543618+0000","last_fullsized":"2023-10-06T03:07:17.543618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:22.771445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539300+0000","last_change":"2023-10-06T03:07:17.536823+0000","last_active":"2023-10-06T03:07:17.539300+0000","last_peered":"2023-10-06T03:07:17.539300+0000","last_clean":"2023-10-06T03:07:17.539300+0000","last_became_active":"2023-10-06T03:07:17.536568+0000","last_became_peered":"2023-10-06T03:07:17.536568+0000","last_unstale":"2023-10-06T03:07:17.539300+0000","last_undegraded":"2023-10-06T03:07:17.539300+0000","last_fullsized":"2023-10-06T03:07:17.539300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:25.254466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.532209+0000","last_change":"2023-10-06T03:07:17.532209+0000","last_active":"2023-10-06T03:07:17.532209+0000","last_peered":"2023-10-06T03:07:17.532209+0000","last_clean":"2023-10-06T03:07:17.532209+0000","last_became_active":"2023-10-06T03:07:17.531855+0000","last_became_peered":"2023-10-06T03:07:17.531855+0000","last_unstale":"2023-10-06T03:07:17.532209+0000","last_undegraded":"2023-10-06T03:07:17.532209+0000","last_fullsized":"2023-10-06T03:07:17.532209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:06.974401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539198+0000","last_change":"2023-10-06T03:07:17.536094+0000","last_active":"2023-10-06T03:07:17.539198+0000","last_peered":"2023-10-06T03:07:17.539198+0000","last_clean":"2023-10-06T03:07:17.539198+0000","last_became_active":"2023-10-06T03:07:17.535829+0000","last_became_peered":"2023-10-06T03:07:17.535829+0000","last_unstale":"2023-10-06T03:07:17.539198+0000","last_undegraded":"2023-10-06T03:07:17.539198+0000","last_fullsized":"2023-10-06T03:07:17.539198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:52.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535680+0000","last_change":"2023-10-06T03:07:17.532834+0000","last_active":"2023-10-06T03:07:17.535680+0000","last_peered":"2023-10-06T03:07:17.535680+0000","last_clean":"2023-10-06T03:07:17.535680+0000","last_became_active":"2023-10-06T03:07:17.532577+0000","last_became_peered":"2023-10-06T03:07:17.532577+0000","last_unstale":"2023-10-06T03:07:17.535680+0000","last_undegraded":"2023-10-06T03:07:17.535680+0000","last_fullsized":"2023-10-06T03:07:17.535680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:25.254466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536035+0000","last_change":"2023-10-06T03:07:17.533123+0000","last_active":"2023-10-06T03:07:17.536035+0000","last_peered":"2023-10-06T03:07:17.536035+0000","last_clean":"2023-10-06T03:07:17.536035+0000","last_became_active":"2023-10-06T03:07:17.532876+0000","last_became_peered":"2023-10-06T03:07:17.532876+0000","last_unstale":"2023-10-06T03:07:17.536035+0000","last_undegraded":"2023-10-06T03:07:17.536035+0000","last_fullsized":"2023-10-06T03:07:17.536035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:46.186781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531328+0000","last_change":"2023-10-06T03:07:17.527206+0000","last_active":"2023-10-06T03:07:17.531328+0000","last_peered":"2023-10-06T03:07:17.531328+0000","last_clean":"2023-10-06T03:07:17.531328+0000","last_became_active":"2023-10-06T03:07:17.526943+0000","last_became_peered":"2023-10-06T03:07:17.526943+0000","last_unstale":"2023-10-06T03:07:17.531328+0000","last_undegraded":"2023-10-06T03:07:17.531328+0000","last_fullsized":"2023-10-06T03:07:17.531328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:43.965385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535072+0000","last_change":"2023-10-06T03:07:17.533145+0000","last_active":"2023-10-06T03:07:17.535072+0000","last_peered":"2023-10-06T03:07:17.535072+0000","last_clean":"2023-10-06T03:07:17.535072+0000","last_became_active":"2023-10-06T03:07:17.532910+0000","last_became_peered":"2023-10-06T03:07:17.532910+0000","last_unstale":"2023-10-06T03:07:17.535072+0000","last_undegraded":"2023-10-06T03:07:17.535072+0000","last_fullsized":"2023-10-06T03:07:17.535072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:50.035401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534325+0000","last_change":"2023-10-06T03:07:17.534325+0000","last_active":"2023-10-06T03:07:17.534325+0000","last_peered":"2023-10-06T03:07:17.534325+0000","last_clean":"2023-10-06T03:07:17.534325+0000","last_became_active":"2023-10-06T03:07:17.533938+0000","last_became_peered":"2023-10-06T03:07:17.533938+0000","last_unstale":"2023-10-06T03:07:17.534325+0000","last_undegraded":"2023-10-06T03:07:17.534325+0000","last_fullsized":"2023-10-06T03:07:17.534325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:41.311041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.532754+0000","last_change":"2023-10-06T03:07:17.532754+0000","last_active":"2023-10-06T03:07:17.532754+0000","last_peered":"2023-10-06T03:07:17.532754+0000","last_clean":"2023-10-06T03:07:17.532754+0000","last_became_active":"2023-10-06T03:07:17.532506+0000","last_became_peered":"2023-10-06T03:07:17.532506+0000","last_unstale":"2023-10-06T03:07:17.532754+0000","last_undegraded":"2023-10-06T03:07:17.532754+0000","last_fullsized":"2023-10-06T03:07:17.532754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:06.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534386+0000","last_change":"2023-10-06T03:07:17.534386+0000","last_active":"2023-10-06T03:07:17.534386+0000","last_peered":"2023-10-06T03:07:17.534386+0000","last_clean":"2023-10-06T03:07:17.534386+0000","last_became_active":"2023-10-06T03:07:17.534072+0000","last_became_peered":"2023-10-06T03:07:17.534072+0000","last_unstale":"2023-10-06T03:07:17.534386+0000","last_undegraded":"2023-10-06T03:07:17.534386+0000","last_fullsized":"2023-10-06T03:07:17.534386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:20.474907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.529342+0000","last_change":"2023-10-06T03:07:17.529342+0000","last_active":"2023-10-06T03:07:17.529342+0000","last_peered":"2023-10-06T03:07:17.529342+0000","last_clean":"2023-10-06T03:07:17.529342+0000","last_became_active":"2023-10-06T03:07:17.529132+0000","last_became_peered":"2023-10-06T03:07:17.529132+0000","last_unstale":"2023-10-06T03:07:17.529342+0000","last_undegraded":"2023-10-06T03:07:17.529342+0000","last_fullsized":"2023-10-06T03:07:17.529342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:24.805888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.543599+0000","last_change":"2023-10-06T03:07:17.537578+0000","last_active":"2023-10-06T03:07:17.543599+0000","last_peered":"2023-10-06T03:07:17.543599+0000","last_clean":"2023-10-06T03:07:17.543599+0000","last_became_active":"2023-10-06T03:07:17.537303+0000","last_became_peered":"2023-10-06T03:07:17.537303+0000","last_unstale":"2023-10-06T03:07:17.543599+0000","last_undegraded":"2023-10-06T03:07:17.543599+0000","last_fullsized":"2023-10-06T03:07:17.543599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:40.254108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534224+0000","last_change":"2023-10-06T03:07:17.534224+0000","last_active":"2023-10-06T03:07:17.534224+0000","last_peered":"2023-10-06T03:07:17.534224+0000","last_clean":"2023-10-06T03:07:17.534224+0000","last_became_active":"2023-10-06T03:07:17.533869+0000","last_became_peered":"2023-10-06T03:07:17.533869+0000","last_unstale":"2023-10-06T03:07:17.534224+0000","last_undegraded":"2023-10-06T03:07:17.534224+0000","last_fullsized":"2023-10-06T03:07:17.534224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:44.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537941+0000","last_change":"2023-10-06T03:07:17.534432+0000","last_active":"2023-10-06T03:07:17.537941+0000","last_peered":"2023-10-06T03:07:17.537941+0000","last_clean":"2023-10-06T03:07:17.537941+0000","last_became_active":"2023-10-06T03:07:17.534177+0000","last_became_peered":"2023-10-06T03:07:17.534177+0000","last_unstale":"2023-10-06T03:07:17.537941+0000","last_undegraded":"2023-10-06T03:07:17.537941+0000","last_fullsized":"2023-10-06T03:07:17.537941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:37:40.361815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534227+0000","last_change":"2023-10-06T03:07:17.534227+0000","last_active":"2023-10-06T03:07:17.534227+0000","last_peered":"2023-10-06T03:07:17.534227+0000","last_clean":"2023-10-06T03:07:17.534227+0000","last_became_active":"2023-10-06T03:07:17.533984+0000","last_became_peered":"2023-10-06T03:07:17.533984+0000","last_unstale":"2023-10-06T03:07:17.534227+0000","last_undegraded":"2023-10-06T03:07:17.534227+0000","last_fullsized":"2023-10-06T03:07:17.534227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:34.596239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534491+0000","last_change":"2023-10-06T03:07:17.526376+0000","last_active":"2023-10-06T03:07:17.534491+0000","last_peered":"2023-10-06T03:07:17.534491+0000","last_clean":"2023-10-06T03:07:17.534491+0000","last_became_active":"2023-10-06T03:07:17.526134+0000","last_became_peered":"2023-10-06T03:07:17.526134+0000","last_unstale":"2023-10-06T03:07:17.534491+0000","last_undegraded":"2023-10-06T03:07:17.534491+0000","last_fullsized":"2023-10-06T03:07:17.534491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:58.202326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.523919+0000","last_change":"2023-10-06T03:07:17.523919+0000","last_active":"2023-10-06T03:07:17.523919+0000","last_peered":"2023-10-06T03:07:17.523919+0000","last_clean":"2023-10-06T03:07:17.523919+0000","last_became_active":"2023-10-06T03:07:17.523629+0000","last_became_peered":"2023-10-06T03:07:17.523629+0000","last_unstale":"2023-10-06T03:07:17.523919+0000","last_undegraded":"2023-10-06T03:07:17.523919+0000","last_fullsized":"2023-10-06T03:07:17.523919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:02.609086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530345+0000","last_change":"2023-10-06T03:07:17.524163+0000","last_active":"2023-10-06T03:07:17.530345+0000","last_peered":"2023-10-06T03:07:17.530345+0000","last_clean":"2023-10-06T03:07:17.530345+0000","last_became_active":"2023-10-06T03:07:17.523931+0000","last_became_peered":"2023-10-06T03:07:17.523931+0000","last_unstale":"2023-10-06T03:07:17.530345+0000","last_undegraded":"2023-10-06T03:07:17.530345+0000","last_fullsized":"2023-10-06T03:07:17.530345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:22.771445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536791+0000","last_change":"2023-10-06T03:07:17.531005+0000","last_active":"2023-10-06T03:07:17.536791+0000","last_peered":"2023-10-06T03:07:17.536791+0000","last_clean":"2023-10-06T03:07:17.536791+0000","last_became_active":"2023-10-06T03:07:17.530778+0000","last_became_peered":"2023-10-06T03:07:17.530778+0000","last_unstale":"2023-10-06T03:07:17.536791+0000","last_undegraded":"2023-10-06T03:07:17.536791+0000","last_fullsized":"2023-10-06T03:07:17.536791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:52.229338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537508+0000","last_change":"2023-10-06T03:07:17.532858+0000","last_active":"2023-10-06T03:07:17.537508+0000","last_peered":"2023-10-06T03:07:17.537508+0000","last_clean":"2023-10-06T03:07:17.537508+0000","last_became_active":"2023-10-06T03:07:17.532599+0000","last_became_peered":"2023-10-06T03:07:17.532599+0000","last_unstale":"2023-10-06T03:07:17.537508+0000","last_undegraded":"2023-10-06T03:07:17.537508+0000","last_fullsized":"2023-10-06T03:07:17.537508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:16.905978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.538071+0000","last_change":"2023-10-06T03:07:17.535052+0000","last_active":"2023-10-06T03:07:17.538071+0000","last_peered":"2023-10-06T03:07:17.538071+0000","last_clean":"2023-10-06T03:07:17.538071+0000","last_became_active":"2023-10-06T03:07:17.534748+0000","last_became_peered":"2023-10-06T03:07:17.534748+0000","last_unstale":"2023-10-06T03:07:17.538071+0000","last_undegraded":"2023-10-06T03:07:17.538071+0000","last_fullsized":"2023-10-06T03:07:17.538071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537994+0000","last_change":"2023-10-06T03:07:17.533302+0000","last_active":"2023-10-06T03:07:17.537994+0000","last_peered":"2023-10-06T03:07:17.537994+0000","last_clean":"2023-10-06T03:07:17.537994+0000","last_became_active":"2023-10-06T03:07:17.532954+0000","last_became_peered":"2023-10-06T03:07:17.532954+0000","last_unstale":"2023-10-06T03:07:17.537994+0000","last_undegraded":"2023-10-06T03:07:17.537994+0000","last_fullsized":"2023-10-06T03:07:17.537994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:07.990891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536183+0000","last_change":"2023-10-06T03:07:17.536183+0000","last_active":"2023-10-06T03:07:17.536183+0000","last_peered":"2023-10-06T03:07:17.536183+0000","last_clean":"2023-10-06T03:07:17.536183+0000","last_became_active":"2023-10-06T03:07:17.535934+0000","last_became_peered":"2023-10-06T03:07:17.535934+0000","last_unstale":"2023-10-06T03:07:17.536183+0000","last_undegraded":"2023-10-06T03:07:17.536183+0000","last_fullsized":"2023-10-06T03:07:17.536183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:09.679192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537281+0000","last_change":"2023-10-06T03:07:17.531702+0000","last_active":"2023-10-06T03:07:17.537281+0000","last_peered":"2023-10-06T03:07:17.537281+0000","last_clean":"2023-10-06T03:07:17.537281+0000","last_became_active":"2023-10-06T03:07:17.531445+0000","last_became_peered":"2023-10-06T03:07:17.531445+0000","last_unstale":"2023-10-06T03:07:17.537281+0000","last_undegraded":"2023-10-06T03:07:17.537281+0000","last_fullsized":"2023-10-06T03:07:17.537281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:04.098459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539280+0000","last_change":"2023-10-06T03:07:17.536702+0000","last_active":"2023-10-06T03:07:17.539280+0000","last_peered":"2023-10-06T03:07:17.539280+0000","last_clean":"2023-10-06T03:07:17.539280+0000","last_became_active":"2023-10-06T03:07:17.536292+0000","last_became_peered":"2023-10-06T03:07:17.536292+0000","last_unstale":"2023-10-06T03:07:17.539280+0000","last_undegraded":"2023-10-06T03:07:17.539280+0000","last_fullsized":"2023-10-06T03:07:17.539280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:13.497707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.533117+0000","last_change":"2023-10-06T03:07:17.533117+0000","last_active":"2023-10-06T03:07:17.533117+0000","last_peered":"2023-10-06T03:07:17.533117+0000","last_clean":"2023-10-06T03:07:17.533117+0000","last_became_active":"2023-10-06T03:07:17.532175+0000","last_became_peered":"2023-10-06T03:07:17.532175+0000","last_unstale":"2023-10-06T03:07:17.533117+0000","last_undegraded":"2023-10-06T03:07:17.533117+0000","last_fullsized":"2023-10-06T03:07:17.533117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:52.115167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530334+0000","last_change":"2023-10-06T03:07:17.530334+0000","last_active":"2023-10-06T03:07:17.530334+0000","last_peered":"2023-10-06T03:07:17.530334+0000","last_clean":"2023-10-06T03:07:17.530334+0000","last_became_active":"2023-10-06T03:07:17.529959+0000","last_became_peered":"2023-10-06T03:07:17.529959+0000","last_unstale":"2023-10-06T03:07:17.530334+0000","last_undegraded":"2023-10-06T03:07:17.530334+0000","last_fullsized":"2023-10-06T03:07:17.530334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:52.977924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.025640+0000","last_change":"2023-10-06T03:07:17.017667+0000","last_active":"2023-10-06T03:07:18.025640+0000","last_peered":"2023-10-06T03:07:18.025640+0000","last_clean":"2023-10-06T03:07:18.025640+0000","last_became_active":"2023-10-06T03:07:17.017368+0000","last_became_peered":"2023-10-06T03:07:17.017368+0000","last_unstale":"2023-10-06T03:07:18.025640+0000","last_undegraded":"2023-10-06T03:07:18.025640+0000","last_fullsized":"2023-10-06T03:07:18.025640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:57.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531286+0000","last_change":"2023-10-06T03:07:17.531286+0000","last_active":"2023-10-06T03:07:17.531286+0000","last_peered":"2023-10-06T03:07:17.531286+0000","last_clean":"2023-10-06T03:07:17.531286+0000","last_became_active":"2023-10-06T03:07:17.530863+0000","last_became_peered":"2023-10-06T03:07:17.530863+0000","last_unstale":"2023-10-06T03:07:17.531286+0000","last_undegraded":"2023-10-06T03:07:17.531286+0000","last_fullsized":"2023-10-06T03:07:17.531286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:15:06.534375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.522636+0000","last_change":"2023-10-06T03:07:17.009505+0000","last_active":"2023-10-06T03:07:17.522636+0000","last_peered":"2023-10-06T03:07:17.522636+0000","last_clean":"2023-10-06T03:07:17.522636+0000","last_became_active":"2023-10-06T03:07:17.009266+0000","last_became_peered":"2023-10-06T03:07:17.009266+0000","last_unstale":"2023-10-06T03:07:17.522636+0000","last_undegraded":"2023-10-06T03:07:17.522636+0000","last_fullsized":"2023-10-06T03:07:17.522636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:49.449105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536868+0000","last_change":"2023-10-06T03:07:17.530386+0000","last_active":"2023-10-06T03:07:17.536868+0000","last_peered":"2023-10-06T03:07:17.536868+0000","last_clean":"2023-10-06T03:07:17.536868+0000","last_became_active":"2023-10-06T03:07:17.530077+0000","last_became_peered":"2023-10-06T03:07:17.530077+0000","last_unstale":"2023-10-06T03:07:17.536868+0000","last_undegraded":"2023-10-06T03:07:17.536868+0000","last_fullsized":"2023-10-06T03:07:17.536868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:04.548561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.025611+0000","last_change":"2023-10-06T03:07:17.006708+0000","last_active":"2023-10-06T03:07:18.025611+0000","last_peered":"2023-10-06T03:07:18.025611+0000","last_clean":"2023-10-06T03:07:18.025611+0000","last_became_active":"2023-10-06T03:07:17.006317+0000","last_became_peered":"2023-10-06T03:07:17.006317+0000","last_unstale":"2023-10-06T03:07:18.025611+0000","last_undegraded":"2023-10-06T03:07:18.025611+0000","last_fullsized":"2023-10-06T03:07:18.025611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:42:19.997214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531908+0000","last_change":"2023-10-06T03:07:17.526243+0000","last_active":"2023-10-06T03:07:17.531908+0000","last_peered":"2023-10-06T03:07:17.531908+0000","last_clean":"2023-10-06T03:07:17.531908+0000","last_became_active":"2023-10-06T03:07:17.525987+0000","last_became_peered":"2023-10-06T03:07:17.525987+0000","last_unstale":"2023-10-06T03:07:17.531908+0000","last_undegraded":"2023-10-06T03:07:17.531908+0000","last_fullsized":"2023-10-06T03:07:17.531908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:38.597127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.025342+0000","last_change":"2023-10-06T03:07:17.006940+0000","last_active":"2023-10-06T03:07:18.025342+0000","last_peered":"2023-10-06T03:07:18.025342+0000","last_clean":"2023-10-06T03:07:18.025342+0000","last_became_active":"2023-10-06T03:07:17.006496+0000","last_became_peered":"2023-10-06T03:07:17.006496+0000","last_unstale":"2023-10-06T03:07:18.025342+0000","last_undegraded":"2023-10-06T03:07:18.025342+0000","last_fullsized":"2023-10-06T03:07:18.025342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:01:17.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534539+0000","last_change":"2023-10-06T03:07:17.531004+0000","last_active":"2023-10-06T03:07:17.534539+0000","last_peered":"2023-10-06T03:07:17.534539+0000","last_clean":"2023-10-06T03:07:17.534539+0000","last_became_active":"2023-10-06T03:07:17.530750+0000","last_became_peered":"2023-10-06T03:07:17.530750+0000","last_unstale":"2023-10-06T03:07:17.534539+0000","last_undegraded":"2023-10-06T03:07:17.534539+0000","last_fullsized":"2023-10-06T03:07:17.534539+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:13.497707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.525182+0000","last_change":"2023-10-06T03:07:17.014936+0000","last_active":"2023-10-06T03:07:17.525182+0000","last_peered":"2023-10-06T03:07:17.525182+0000","last_clean":"2023-10-06T03:07:17.525182+0000","last_became_active":"2023-10-06T03:07:17.014624+0000","last_became_peered":"2023-10-06T03:07:17.014624+0000","last_unstale":"2023-10-06T03:07:17.525182+0000","last_undegraded":"2023-10-06T03:07:17.525182+0000","last_fullsized":"2023-10-06T03:07:17.525182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:09.724175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534267+0000","last_change":"2023-10-06T03:07:17.534267+0000","last_active":"2023-10-06T03:07:17.534267+0000","last_peered":"2023-10-06T03:07:17.534267+0000","last_clean":"2023-10-06T03:07:17.534267+0000","last_became_active":"2023-10-06T03:07:17.534002+0000","last_became_peered":"2023-10-06T03:07:17.534002+0000","last_unstale":"2023-10-06T03:07:17.534267+0000","last_undegraded":"2023-10-06T03:07:17.534267+0000","last_fullsized":"2023-10-06T03:07:17.534267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:15.983003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.023074+0000","last_change":"2023-10-06T03:07:17.007342+0000","last_active":"2023-10-06T03:07:18.023074+0000","last_peered":"2023-10-06T03:07:18.023074+0000","last_clean":"2023-10-06T03:07:18.023074+0000","last_became_active":"2023-10-06T03:07:17.007090+0000","last_became_peered":"2023-10-06T03:07:17.007090+0000","last_unstale":"2023-10-06T03:07:18.023074+0000","last_undegraded":"2023-10-06T03:07:18.023074+0000","last_fullsized":"2023-10-06T03:07:18.023074+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:49.057674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539343+0000","last_change":"2023-10-06T03:07:17.537040+0000","last_active":"2023-10-06T03:07:17.539343+0000","last_peered":"2023-10-06T03:07:17.539343+0000","last_clean":"2023-10-06T03:07:17.539343+0000","last_became_active":"2023-10-06T03:07:17.535498+0000","last_became_peered":"2023-10-06T03:07:17.535498+0000","last_unstale":"2023-10-06T03:07:17.539343+0000","last_undegraded":"2023-10-06T03:07:17.539343+0000","last_fullsized":"2023-10-06T03:07:17.539343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:29.797991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.520020+0000","last_change":"2023-10-06T03:07:17.006621+0000","last_active":"2023-10-06T03:07:17.520020+0000","last_peered":"2023-10-06T03:07:17.520020+0000","last_clean":"2023-10-06T03:07:17.520020+0000","last_became_active":"2023-10-06T03:07:17.006249+0000","last_became_peered":"2023-10-06T03:07:17.006249+0000","last_unstale":"2023-10-06T03:07:17.520020+0000","last_undegraded":"2023-10-06T03:07:17.520020+0000","last_fullsized":"2023-10-06T03:07:17.520020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:17.225175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535017+0000","last_change":"2023-10-06T03:07:17.531718+0000","last_active":"2023-10-06T03:07:17.535017+0000","last_peered":"2023-10-06T03:07:17.535017+0000","last_clean":"2023-10-06T03:07:17.535017+0000","last_became_active":"2023-10-06T03:07:17.531454+0000","last_became_peered":"2023-10-06T03:07:17.531454+0000","last_unstale":"2023-10-06T03:07:17.535017+0000","last_undegraded":"2023-10-06T03:07:17.535017+0000","last_fullsized":"2023-10-06T03:07:17.535017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:28.389638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.522649+0000","last_change":"2023-10-06T03:07:17.010059+0000","last_active":"2023-10-06T03:07:17.522649+0000","last_peered":"2023-10-06T03:07:17.522649+0000","last_clean":"2023-10-06T03:07:17.522649+0000","last_became_active":"2023-10-06T03:07:17.009662+0000","last_became_peered":"2023-10-06T03:07:17.009662+0000","last_unstale":"2023-10-06T03:07:17.522649+0000","last_undegraded":"2023-10-06T03:07:17.522649+0000","last_fullsized":"2023-10-06T03:07:17.522649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:47.409869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530591+0000","last_change":"2023-10-06T03:07:17.530591+0000","last_active":"2023-10-06T03:07:17.530591+0000","last_peered":"2023-10-06T03:07:17.530591+0000","last_clean":"2023-10-06T03:07:17.530591+0000","last_became_active":"2023-10-06T03:07:17.530301+0000","last_became_peered":"2023-10-06T03:07:17.530301+0000","last_unstale":"2023-10-06T03:07:17.530591+0000","last_undegraded":"2023-10-06T03:07:17.530591+0000","last_fullsized":"2023-10-06T03:07:17.530591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:28.312569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.520259+0000","last_change":"2023-10-06T03:07:17.007059+0000","last_active":"2023-10-06T03:07:17.520259+0000","last_peered":"2023-10-06T03:07:17.520259+0000","last_clean":"2023-10-06T03:07:17.520259+0000","last_became_active":"2023-10-06T03:07:17.006782+0000","last_became_peered":"2023-10-06T03:07:17.006782+0000","last_unstale":"2023-10-06T03:07:17.520259+0000","last_undegraded":"2023-10-06T03:07:17.520259+0000","last_fullsized":"2023-10-06T03:07:17.520259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:42.034142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534172+0000","last_change":"2023-10-06T03:07:17.527518+0000","last_active":"2023-10-06T03:07:17.534172+0000","last_peered":"2023-10-06T03:07:17.534172+0000","last_clean":"2023-10-06T03:07:17.534172+0000","last_became_active":"2023-10-06T03:07:17.526872+0000","last_became_peered":"2023-10-06T03:07:17.526872+0000","last_unstale":"2023-10-06T03:07:17.534172+0000","last_undegraded":"2023-10-06T03:07:17.534172+0000","last_fullsized":"2023-10-06T03:07:17.534172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:50.742831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.014923+0000","last_change":"2023-10-06T03:07:17.011807+0000","last_active":"2023-10-06T03:07:17.014923+0000","last_peered":"2023-10-06T03:07:17.014923+0000","last_clean":"2023-10-06T03:07:17.014923+0000","last_became_active":"2023-10-06T03:07:17.011540+0000","last_became_peered":"2023-10-06T03:07:17.011540+0000","last_unstale":"2023-10-06T03:07:17.014923+0000","last_undegraded":"2023-10-06T03:07:17.014923+0000","last_fullsized":"2023-10-06T03:07:17.014923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:56.364506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531051+0000","last_change":"2023-10-06T03:07:17.531051+0000","last_active":"2023-10-06T03:07:17.531051+0000","last_peered":"2023-10-06T03:07:17.531051+0000","last_clean":"2023-10-06T03:07:17.531051+0000","last_became_active":"2023-10-06T03:07:17.530754+0000","last_became_peered":"2023-10-06T03:07:17.530754+0000","last_unstale":"2023-10-06T03:07:17.531051+0000","last_undegraded":"2023-10-06T03:07:17.531051+0000","last_fullsized":"2023-10-06T03:07:17.531051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:07.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"89'3","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.027586+0000","last_change":"2023-10-06T03:07:17.015454+0000","last_active":"2023-10-06T03:07:18.027586+0000","last_peered":"2023-10-06T03:07:18.027586+0000","last_clean":"2023-10-06T03:07:18.027586+0000","last_became_active":"2023-10-06T03:07:17.015175+0000","last_became_peered":"2023-10-06T03:07:17.015175+0000","last_unstale":"2023-10-06T03:07:18.027586+0000","last_undegraded":"2023-10-06T03:07:18.027586+0000","last_fullsized":"2023-10-06T03:07:18.027586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:49.030741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534761+0000","last_change":"2023-10-06T03:07:17.528416+0000","last_active":"2023-10-06T03:07:17.534761+0000","last_peered":"2023-10-06T03:07:17.534761+0000","last_clean":"2023-10-06T03:07:17.534761+0000","last_became_active":"2023-10-06T03:07:17.528041+0000","last_became_peered":"2023-10-06T03:07:17.528041+0000","last_unstale":"2023-10-06T03:07:17.534761+0000","last_undegraded":"2023-10-06T03:07:17.534761+0000","last_fullsized":"2023-10-06T03:07:17.534761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:40.254108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.029420+0000","last_change":"2023-10-06T03:07:17.017273+0000","last_active":"2023-10-06T03:07:18.029420+0000","last_peered":"2023-10-06T03:07:18.029420+0000","last_clean":"2023-10-06T03:07:18.029420+0000","last_became_active":"2023-10-06T03:07:17.017027+0000","last_became_peered":"2023-10-06T03:07:17.017027+0000","last_unstale":"2023-10-06T03:07:18.029420+0000","last_undegraded":"2023-10-06T03:07:18.029420+0000","last_fullsized":"2023-10-06T03:07:18.029420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:01:17.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534475+0000","last_change":"2023-10-06T03:07:17.529212+0000","last_active":"2023-10-06T03:07:17.534475+0000","last_peered":"2023-10-06T03:07:17.534475+0000","last_clean":"2023-10-06T03:07:17.534475+0000","last_became_active":"2023-10-06T03:07:17.529001+0000","last_became_peered":"2023-10-06T03:07:17.529001+0000","last_unstale":"2023-10-06T03:07:17.534475+0000","last_undegraded":"2023-10-06T03:07:17.534475+0000","last_fullsized":"2023-10-06T03:07:17.534475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:33.419255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.031294+0000","last_change":"2023-10-06T03:07:17.013452+0000","last_active":"2023-10-06T03:07:18.031294+0000","last_peered":"2023-10-06T03:07:18.031294+0000","last_clean":"2023-10-06T03:07:18.031294+0000","last_became_active":"2023-10-06T03:07:17.013201+0000","last_became_peered":"2023-10-06T03:07:17.013201+0000","last_unstale":"2023-10-06T03:07:18.031294+0000","last_undegraded":"2023-10-06T03:07:18.031294+0000","last_fullsized":"2023-10-06T03:07:18.031294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:16.512800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.538036+0000","last_change":"2023-10-06T03:07:17.532543+0000","last_active":"2023-10-06T03:07:17.538036+0000","last_peered":"2023-10-06T03:07:17.538036+0000","last_clean":"2023-10-06T03:07:17.538036+0000","last_became_active":"2023-10-06T03:07:17.532240+0000","last_became_peered":"2023-10-06T03:07:17.532240+0000","last_unstale":"2023-10-06T03:07:17.538036+0000","last_undegraded":"2023-10-06T03:07:17.538036+0000","last_fullsized":"2023-10-06T03:07:17.538036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:58.202326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.521970+0000","last_change":"2023-10-06T03:07:17.009383+0000","last_active":"2023-10-06T03:07:17.521970+0000","last_peered":"2023-10-06T03:07:17.521970+0000","last_clean":"2023-10-06T03:07:17.521970+0000","last_became_active":"2023-10-06T03:07:17.009117+0000","last_became_peered":"2023-10-06T03:07:17.009117+0000","last_unstale":"2023-10-06T03:07:17.521970+0000","last_undegraded":"2023-10-06T03:07:17.521970+0000","last_fullsized":"2023-10-06T03:07:17.521970+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:26.379689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.526589+0000","last_change":"2023-10-06T03:07:17.526589+0000","last_active":"2023-10-06T03:07:17.526589+0000","last_peered":"2023-10-06T03:07:17.526589+0000","last_clean":"2023-10-06T03:07:17.526589+0000","last_became_active":"2023-10-06T03:07:17.526325+0000","last_became_peered":"2023-10-06T03:07:17.526325+0000","last_unstale":"2023-10-06T03:07:17.526589+0000","last_undegraded":"2023-10-06T03:07:17.526589+0000","last_fullsized":"2023-10-06T03:07:17.526589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:44.858785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015828+0000","last_change":"2023-10-06T03:07:17.008343+0000","last_active":"2023-10-06T03:07:17.015828+0000","last_peered":"2023-10-06T03:07:17.015828+0000","last_clean":"2023-10-06T03:07:17.015828+0000","last_became_active":"2023-10-06T03:07:17.008091+0000","last_became_peered":"2023-10-06T03:07:17.008091+0000","last_unstale":"2023-10-06T03:07:17.015828+0000","last_undegraded":"2023-10-06T03:07:17.015828+0000","last_fullsized":"2023-10-06T03:07:17.015828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:42.588920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.524415+0000","last_change":"2023-10-06T03:07:17.524415+0000","last_active":"2023-10-06T03:07:17.524415+0000","last_peered":"2023-10-06T03:07:17.524415+0000","last_clean":"2023-10-06T03:07:17.524415+0000","last_became_active":"2023-10-06T03:07:17.523666+0000","last_became_peered":"2023-10-06T03:07:17.523666+0000","last_unstale":"2023-10-06T03:07:17.524415+0000","last_undegraded":"2023-10-06T03:07:17.524415+0000","last_fullsized":"2023-10-06T03:07:17.524415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:07.574334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"89'8","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.028512+0000","last_change":"2023-10-06T03:07:17.006546+0000","last_active":"2023-10-06T03:07:18.028512+0000","last_peered":"2023-10-06T03:07:18.028512+0000","last_clean":"2023-10-06T03:07:18.028512+0000","last_became_active":"2023-10-06T03:07:17.006217+0000","last_became_peered":"2023-10-06T03:07:17.006217+0000","last_unstale":"2023-10-06T03:07:18.028512+0000","last_undegraded":"2023-10-06T03:07:18.028512+0000","last_fullsized":"2023-10-06T03:07:18.028512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:28.315419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527585+0000","last_change":"2023-10-06T03:07:17.527585+0000","last_active":"2023-10-06T03:07:17.527585+0000","last_peered":"2023-10-06T03:07:17.527585+0000","last_clean":"2023-10-06T03:07:17.527585+0000","last_became_active":"2023-10-06T03:07:17.526751+0000","last_became_peered":"2023-10-06T03:07:17.526751+0000","last_unstale":"2023-10-06T03:07:17.527585+0000","last_undegraded":"2023-10-06T03:07:17.527585+0000","last_fullsized":"2023-10-06T03:07:17.527585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:13.406743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.031208+0000","last_change":"2023-10-06T03:07:17.013091+0000","last_active":"2023-10-06T03:07:18.031208+0000","last_peered":"2023-10-06T03:07:18.031208+0000","last_clean":"2023-10-06T03:07:18.031208+0000","last_became_active":"2023-10-06T03:07:17.012742+0000","last_became_peered":"2023-10-06T03:07:17.012742+0000","last_unstale":"2023-10-06T03:07:18.031208+0000","last_undegraded":"2023-10-06T03:07:18.031208+0000","last_fullsized":"2023-10-06T03:07:18.031208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:59.119110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531358+0000","last_change":"2023-10-06T03:07:17.526556+0000","last_active":"2023-10-06T03:07:17.531358+0000","last_peered":"2023-10-06T03:07:17.531358+0000","last_clean":"2023-10-06T03:07:17.531358+0000","last_became_active":"2023-10-06T03:07:17.526238+0000","last_became_peered":"2023-10-06T03:07:17.526238+0000","last_unstale":"2023-10-06T03:07:17.531358+0000","last_undegraded":"2023-10-06T03:07:17.531358+0000","last_fullsized":"2023-10-06T03:07:17.531358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:40.834864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.016530+0000","last_change":"2023-10-06T03:07:17.015654+0000","last_active":"2023-10-06T03:07:17.016530+0000","last_peered":"2023-10-06T03:07:17.016530+0000","last_clean":"2023-10-06T03:07:17.016530+0000","last_became_active":"2023-10-06T03:07:17.015404+0000","last_became_peered":"2023-10-06T03:07:17.015404+0000","last_unstale":"2023-10-06T03:07:17.016530+0000","last_undegraded":"2023-10-06T03:07:17.016530+0000","last_fullsized":"2023-10-06T03:07:17.016530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:08.225554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535154+0000","last_change":"2023-10-06T03:07:17.532464+0000","last_active":"2023-10-06T03:07:17.535154+0000","last_peered":"2023-10-06T03:07:17.535154+0000","last_clean":"2023-10-06T03:07:17.535154+0000","last_became_active":"2023-10-06T03:07:17.532159+0000","last_became_peered":"2023-10-06T03:07:17.532159+0000","last_unstale":"2023-10-06T03:07:17.535154+0000","last_undegraded":"2023-10-06T03:07:17.535154+0000","last_fullsized":"2023-10-06T03:07:17.535154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:06.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531720+0000","last_change":"2023-10-06T03:07:17.013693+0000","last_active":"2023-10-06T03:07:17.531720+0000","last_peered":"2023-10-06T03:07:17.531720+0000","last_clean":"2023-10-06T03:07:17.531720+0000","last_became_active":"2023-10-06T03:07:17.013475+0000","last_became_peered":"2023-10-06T03:07:17.013475+0000","last_unstale":"2023-10-06T03:07:17.531720+0000","last_undegraded":"2023-10-06T03:07:17.531720+0000","last_fullsized":"2023-10-06T03:07:17.531720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:05.969741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535802+0000","last_change":"2023-10-06T03:07:17.531933+0000","last_active":"2023-10-06T03:07:17.535802+0000","last_peered":"2023-10-06T03:07:17.535802+0000","last_clean":"2023-10-06T03:07:17.535802+0000","last_became_active":"2023-10-06T03:07:17.530965+0000","last_became_peered":"2023-10-06T03:07:17.530965+0000","last_unstale":"2023-10-06T03:07:17.535802+0000","last_undegraded":"2023-10-06T03:07:17.535802+0000","last_fullsized":"2023-10-06T03:07:17.535802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:06.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.518778+0000","last_change":"2023-10-06T03:07:17.014448+0000","last_active":"2023-10-06T03:07:17.518778+0000","last_peered":"2023-10-06T03:07:17.518778+0000","last_clean":"2023-10-06T03:07:17.518778+0000","last_became_active":"2023-10-06T03:07:17.014192+0000","last_became_peered":"2023-10-06T03:07:17.014192+0000","last_unstale":"2023-10-06T03:07:17.518778+0000","last_undegraded":"2023-10-06T03:07:17.518778+0000","last_fullsized":"2023-10-06T03:07:17.518778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:03.041485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530926+0000","last_change":"2023-10-06T03:07:17.530926+0000","last_active":"2023-10-06T03:07:17.530926+0000","last_peered":"2023-10-06T03:07:17.530926+0000","last_clean":"2023-10-06T03:07:17.530926+0000","last_became_active":"2023-10-06T03:07:17.530650+0000","last_became_peered":"2023-10-06T03:07:17.530650+0000","last_unstale":"2023-10-06T03:07:17.530926+0000","last_undegraded":"2023-10-06T03:07:17.530926+0000","last_fullsized":"2023-10-06T03:07:17.530926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:00.690209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.519233+0000","last_change":"2023-10-06T03:07:17.014863+0000","last_active":"2023-10-06T03:07:17.519233+0000","last_peered":"2023-10-06T03:07:17.519233+0000","last_clean":"2023-10-06T03:07:17.519233+0000","last_became_active":"2023-10-06T03:07:17.014617+0000","last_became_peered":"2023-10-06T03:07:17.014617+0000","last_unstale":"2023-10-06T03:07:17.519233+0000","last_undegraded":"2023-10-06T03:07:17.519233+0000","last_fullsized":"2023-10-06T03:07:17.519233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:19.470247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531484+0000","last_change":"2023-10-06T03:07:17.531484+0000","last_active":"2023-10-06T03:07:17.531484+0000","last_peered":"2023-10-06T03:07:17.531484+0000","last_clean":"2023-10-06T03:07:17.531484+0000","last_became_active":"2023-10-06T03:07:17.530964+0000","last_became_peered":"2023-10-06T03:07:17.530964+0000","last_unstale":"2023-10-06T03:07:17.531484+0000","last_undegraded":"2023-10-06T03:07:17.531484+0000","last_fullsized":"2023-10-06T03:07:17.531484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:51.502108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015878+0000","last_change":"2023-10-06T03:07:17.008960+0000","last_active":"2023-10-06T03:07:17.015878+0000","last_peered":"2023-10-06T03:07:17.015878+0000","last_clean":"2023-10-06T03:07:17.015878+0000","last_became_active":"2023-10-06T03:07:17.008708+0000","last_became_peered":"2023-10-06T03:07:17.008708+0000","last_unstale":"2023-10-06T03:07:17.015878+0000","last_undegraded":"2023-10-06T03:07:17.015878+0000","last_fullsized":"2023-10-06T03:07:17.015878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.011847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535039+0000","last_change":"2023-10-06T03:07:17.535039+0000","last_active":"2023-10-06T03:07:17.535039+0000","last_peered":"2023-10-06T03:07:17.535039+0000","last_clean":"2023-10-06T03:07:17.535039+0000","last_became_active":"2023-10-06T03:07:17.534772+0000","last_became_peered":"2023-10-06T03:07:17.534772+0000","last_unstale":"2023-10-06T03:07:17.535039+0000","last_undegraded":"2023-10-06T03:07:17.535039+0000","last_fullsized":"2023-10-06T03:07:17.535039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:46.186781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531086+0000","last_change":"2023-10-06T03:07:17.017624+0000","last_active":"2023-10-06T03:07:17.531086+0000","last_peered":"2023-10-06T03:07:17.531086+0000","last_clean":"2023-10-06T03:07:17.531086+0000","last_became_active":"2023-10-06T03:07:17.017381+0000","last_became_peered":"2023-10-06T03:07:17.017381+0000","last_unstale":"2023-10-06T03:07:17.531086+0000","last_undegraded":"2023-10-06T03:07:17.531086+0000","last_fullsized":"2023-10-06T03:07:17.531086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:03.041485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.541524+0000","last_change":"2023-10-06T03:07:17.535890+0000","last_active":"2023-10-06T03:07:17.541524+0000","last_peered":"2023-10-06T03:07:17.541524+0000","last_clean":"2023-10-06T03:07:17.541524+0000","last_became_active":"2023-10-06T03:07:17.535580+0000","last_became_peered":"2023-10-06T03:07:17.535580+0000","last_unstale":"2023-10-06T03:07:17.541524+0000","last_undegraded":"2023-10-06T03:07:17.541524+0000","last_fullsized":"2023-10-06T03:07:17.541524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:28.389638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.012466+0000","last_change":"2023-10-06T03:07:17.007302+0000","last_active":"2023-10-06T03:07:17.012466+0000","last_peered":"2023-10-06T03:07:17.012466+0000","last_clean":"2023-10-06T03:07:17.012466+0000","last_became_active":"2023-10-06T03:07:17.007042+0000","last_became_peered":"2023-10-06T03:07:17.007042+0000","last_unstale":"2023-10-06T03:07:17.012466+0000","last_undegraded":"2023-10-06T03:07:17.012466+0000","last_fullsized":"2023-10-06T03:07:17.012466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:29.457148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.528032+0000","last_change":"2023-10-06T03:07:17.528032+0000","last_active":"2023-10-06T03:07:17.528032+0000","last_peered":"2023-10-06T03:07:17.528032+0000","last_clean":"2023-10-06T03:07:17.528032+0000","last_became_active":"2023-10-06T03:07:17.527783+0000","last_became_peered":"2023-10-06T03:07:17.527783+0000","last_unstale":"2023-10-06T03:07:17.528032+0000","last_undegraded":"2023-10-06T03:07:17.528032+0000","last_fullsized":"2023-10-06T03:07:17.528032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:13.413172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015954+0000","last_change":"2023-10-06T03:07:17.010923+0000","last_active":"2023-10-06T03:07:17.015954+0000","last_peered":"2023-10-06T03:07:17.015954+0000","last_clean":"2023-10-06T03:07:17.015954+0000","last_became_active":"2023-10-06T03:07:17.010649+0000","last_became_peered":"2023-10-06T03:07:17.010649+0000","last_unstale":"2023-10-06T03:07:17.015954+0000","last_undegraded":"2023-10-06T03:07:17.015954+0000","last_fullsized":"2023-10-06T03:07:17.015954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:31.795198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531982+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:17.531982+0000","last_peered":"2023-10-06T03:07:17.531982+0000","last_clean":"2023-10-06T03:07:17.531982+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:17.531982+0000","last_undegraded":"2023-10-06T03:07:17.531982+0000","last_fullsized":"2023-10-06T03:07:17.531982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527116+0000","last_change":"2023-10-06T03:07:17.527116+0000","last_active":"2023-10-06T03:07:17.527116+0000","last_peered":"2023-10-06T03:07:17.527116+0000","last_clean":"2023-10-06T03:07:17.527116+0000","last_became_active":"2023-10-06T03:07:17.526865+0000","last_became_peered":"2023-10-06T03:07:17.526865+0000","last_unstale":"2023-10-06T03:07:17.527116+0000","last_undegraded":"2023-10-06T03:07:17.527116+0000","last_fullsized":"2023-10-06T03:07:17.527116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:41.234630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.030658+0000","last_change":"2023-10-06T03:07:17.008320+0000","last_active":"2023-10-06T03:07:18.030658+0000","last_peered":"2023-10-06T03:07:18.030658+0000","last_clean":"2023-10-06T03:07:18.030658+0000","last_became_active":"2023-10-06T03:07:17.008070+0000","last_became_peered":"2023-10-06T03:07:17.008070+0000","last_unstale":"2023-10-06T03:07:18.030658+0000","last_undegraded":"2023-10-06T03:07:18.030658+0000","last_fullsized":"2023-10-06T03:07:18.030658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:09.135222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.518994+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:17.518994+0000","last_peered":"2023-10-06T03:07:17.518994+0000","last_clean":"2023-10-06T03:07:17.518994+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:17.518994+0000","last_undegraded":"2023-10-06T03:07:17.518994+0000","last_fullsized":"2023-10-06T03:07:17.518994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531310+0000","last_change":"2023-10-06T03:07:17.531310+0000","last_active":"2023-10-06T03:07:17.531310+0000","last_peered":"2023-10-06T03:07:17.531310+0000","last_clean":"2023-10-06T03:07:17.531310+0000","last_became_active":"2023-10-06T03:07:17.531078+0000","last_became_peered":"2023-10-06T03:07:17.531078+0000","last_unstale":"2023-10-06T03:07:17.531310+0000","last_undegraded":"2023-10-06T03:07:17.531310+0000","last_fullsized":"2023-10-06T03:07:17.531310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:55.344710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015925+0000","last_change":"2023-10-06T03:07:17.010164+0000","last_active":"2023-10-06T03:07:17.015925+0000","last_peered":"2023-10-06T03:07:17.015925+0000","last_clean":"2023-10-06T03:07:17.015925+0000","last_became_active":"2023-10-06T03:07:17.009912+0000","last_became_peered":"2023-10-06T03:07:17.009912+0000","last_unstale":"2023-10-06T03:07:17.015925+0000","last_undegraded":"2023-10-06T03:07:17.015925+0000","last_fullsized":"2023-10-06T03:07:17.015925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:40.035439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.004554+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:17.004554+0000","last_peered":"2023-10-06T03:07:17.004554+0000","last_clean":"2023-10-06T03:07:17.004554+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:17.004554+0000","last_undegraded":"2023-10-06T03:07:17.004554+0000","last_fullsized":"2023-10-06T03:07:17.004554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539039+0000","last_change":"2023-10-06T03:07:17.534614+0000","last_active":"2023-10-06T03:07:17.539039+0000","last_peered":"2023-10-06T03:07:17.539039+0000","last_clean":"2023-10-06T03:07:17.539039+0000","last_became_active":"2023-10-06T03:07:17.534330+0000","last_became_peered":"2023-10-06T03:07:17.534330+0000","last_unstale":"2023-10-06T03:07:17.539039+0000","last_undegraded":"2023-10-06T03:07:17.539039+0000","last_fullsized":"2023-10-06T03:07:17.539039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:50.035401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527993+0000","last_change":"2023-10-06T03:07:17.013358+0000","last_active":"2023-10-06T03:07:17.527993+0000","last_peered":"2023-10-06T03:07:17.527993+0000","last_clean":"2023-10-06T03:07:17.527993+0000","last_became_active":"2023-10-06T03:07:17.013004+0000","last_became_peered":"2023-10-06T03:07:17.013004+0000","last_unstale":"2023-10-06T03:07:17.527993+0000","last_undegraded":"2023-10-06T03:07:17.527993+0000","last_fullsized":"2023-10-06T03:07:17.527993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:28.793331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.002230+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:17.002230+0000","last_peered":"2023-10-06T03:07:17.002230+0000","last_clean":"2023-10-06T03:07:17.002230+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:17.002230+0000","last_undegraded":"2023-10-06T03:07:17.002230+0000","last_fullsized":"2023-10-06T03:07:17.002230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531989+0000","last_change":"2023-10-06T03:07:17.525296+0000","last_active":"2023-10-06T03:07:17.531989+0000","last_peered":"2023-10-06T03:07:17.531989+0000","last_clean":"2023-10-06T03:07:17.531989+0000","last_became_active":"2023-10-06T03:07:17.524893+0000","last_became_peered":"2023-10-06T03:07:17.524893+0000","last_unstale":"2023-10-06T03:07:17.531989+0000","last_undegraded":"2023-10-06T03:07:17.531989+0000","last_fullsized":"2023-10-06T03:07:17.531989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:41.311041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.030636+0000","last_change":"2023-10-06T03:07:17.010071+0000","last_active":"2023-10-06T03:07:18.030636+0000","last_peered":"2023-10-06T03:07:18.030636+0000","last_clean":"2023-10-06T03:07:18.030636+0000","last_became_active":"2023-10-06T03:07:17.009831+0000","last_became_peered":"2023-10-06T03:07:17.009831+0000","last_unstale":"2023-10-06T03:07:18.030636+0000","last_undegraded":"2023-10-06T03:07:18.030636+0000","last_fullsized":"2023-10-06T03:07:18.030636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:40.175153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":223,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.002301+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:17.002301+0000","last_peered":"2023-10-06T03:07:17.002301+0000","last_clean":"2023-10-06T03:07:17.002301+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:17.002301+0000","last_undegraded":"2023-10-06T03:07:17.002301+0000","last_fullsized":"2023-10-06T03:07:17.002301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.521853+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:17.521853+0000","last_peered":"2023-10-06T03:07:17.521853+0000","last_clean":"2023-10-06T03:07:17.521853+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:17.521853+0000","last_undegraded":"2023-10-06T03:07:17.521853+0000","last_fullsized":"2023-10-06T03:07:17.521853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539724+0000","last_change":"2023-10-06T03:07:17.535951+0000","last_active":"2023-10-06T03:07:17.539724+0000","last_peered":"2023-10-06T03:07:17.539724+0000","last_clean":"2023-10-06T03:07:17.539724+0000","last_became_active":"2023-10-06T03:07:17.535676+0000","last_became_peered":"2023-10-06T03:07:17.535676+0000","last_unstale":"2023-10-06T03:07:17.539724+0000","last_undegraded":"2023-10-06T03:07:17.539724+0000","last_fullsized":"2023-10-06T03:07:17.539724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:50.062334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.521669+0000","last_change":"2023-10-06T03:07:17.007812+0000","last_active":"2023-10-06T03:07:17.521669+0000","last_peered":"2023-10-06T03:07:17.521669+0000","last_clean":"2023-10-06T03:07:17.521669+0000","last_became_active":"2023-10-06T03:07:17.007525+0000","last_became_peered":"2023-10-06T03:07:17.007525+0000","last_unstale":"2023-10-06T03:07:17.521669+0000","last_undegraded":"2023-10-06T03:07:17.521669+0000","last_fullsized":"2023-10-06T03:07:17.521669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:49.738171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:16.996796+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:16.996796+0000","last_peered":"2023-10-06T03:07:16.996796+0000","last_clean":"2023-10-06T03:07:16.996796+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:16.996796+0000","last_undegraded":"2023-10-06T03:07:16.996796+0000","last_fullsized":"2023-10-06T03:07:16.996796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527079+0000","last_change":"2023-10-06T03:07:17.527079+0000","last_active":"2023-10-06T03:07:17.527079+0000","last_peered":"2023-10-06T03:07:17.527079+0000","last_clean":"2023-10-06T03:07:17.527079+0000","last_became_active":"2023-10-06T03:07:17.526821+0000","last_became_peered":"2023-10-06T03:07:17.526821+0000","last_unstale":"2023-10-06T03:07:17.527079+0000","last_undegraded":"2023-10-06T03:07:17.527079+0000","last_fullsized":"2023-10-06T03:07:17.527079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:22.020130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.028238+0000","last_change":"2023-10-06T03:07:17.016948+0000","last_active":"2023-10-06T03:07:18.028238+0000","last_peered":"2023-10-06T03:07:18.028238+0000","last_clean":"2023-10-06T03:07:18.028238+0000","last_became_active":"2023-10-06T03:07:17.016651+0000","last_became_peered":"2023-10-06T03:07:17.016651+0000","last_unstale":"2023-10-06T03:07:18.028238+0000","last_undegraded":"2023-10-06T03:07:18.028238+0000","last_fullsized":"2023-10-06T03:07:18.028238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:21.015470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.520783+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:17.520783+0000","last_peered":"2023-10-06T03:07:17.520783+0000","last_clean":"2023-10-06T03:07:17.520783+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:17.520783+0000","last_undegraded":"2023-10-06T03:07:17.520783+0000","last_fullsized":"2023-10-06T03:07:17.520783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539324+0000","last_change":"2023-10-06T03:07:17.535352+0000","last_active":"2023-10-06T03:07:17.539324+0000","last_peered":"2023-10-06T03:07:17.539324+0000","last_clean":"2023-10-06T03:07:17.539324+0000","last_became_active":"2023-10-06T03:07:17.535096+0000","last_became_peered":"2023-10-06T03:07:17.535096+0000","last_unstale":"2023-10-06T03:07:17.539324+0000","last_undegraded":"2023-10-06T03:07:17.539324+0000","last_fullsized":"2023-10-06T03:07:17.539324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531140+0000","last_change":"2023-10-06T03:07:17.017093+0000","last_active":"2023-10-06T03:07:17.531140+0000","last_peered":"2023-10-06T03:07:17.531140+0000","last_clean":"2023-10-06T03:07:17.531140+0000","last_became_active":"2023-10-06T03:07:17.016695+0000","last_became_peered":"2023-10-06T03:07:17.016695+0000","last_unstale":"2023-10-06T03:07:17.531140+0000","last_undegraded":"2023-10-06T03:07:17.531140+0000","last_fullsized":"2023-10-06T03:07:17.531140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:59.119110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.518856+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:17.518856+0000","last_peered":"2023-10-06T03:07:17.518856+0000","last_clean":"2023-10-06T03:07:17.518856+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:17.518856+0000","last_undegraded":"2023-10-06T03:07:17.518856+0000","last_fullsized":"2023-10-06T03:07:17.518856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1090160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55500000000000005,"15min":0.72799999999999998},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76200000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.38,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.501,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71599999999999997,"5min":0.875,"15min":0.875},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81299999999999994,"5min":0.999,"15min":0.999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":1.026,"15min":1.026},"last":0.503}]},{"osd":5,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298396,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073444,"statfs":{"total":96636764160,"available":96331206656,"internally_reserved":0,"allocated":1134592,"data_stored":508385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.626},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.55100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":499666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.25},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.372},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55900000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.307}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32300000000000001,"15min":0.33100000000000002},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48599999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.23300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.50800000000000001,"5min":0.628,"15min":0.628},"last":0.436},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.29399999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":1.115,"15min":1.115},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84699999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82699999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1712128,"data_stored":1091626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.307,"15min":0.31},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.31900000000000001,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40300000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.26600000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.433,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.33300000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.48499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63500000000000001,"5min":0.871,"15min":0.871},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68000000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.754,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71099999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76300000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":499666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.32200000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.498,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.315},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.219}]},{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.307},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34599999999999997,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.504,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.29099999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.42599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.87,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":1.903,"15min":1.903},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":500186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.30499999999999999},"min":{"1min":0.215,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.503,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.25600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.49299999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.317}]},{"osd":1,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.248},{"interface":"front","average":{"1min":0.314,"5min":0.33300000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.308},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33900000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.625,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77000000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.997,"15min":0.997},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73799999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.441},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.40300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.501}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302480,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069360,"statfs":{"total":96636764160,"available":96327024640,"internally_reserved":0,"allocated":1122304,"data_stored":506365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52400000000000002,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64300000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.622}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54800000000000004,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70799999999999996,"5min":3,"15min":3},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.60999999999999999,"15min":0.81799999999999995},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69199999999999995,"5min":10.084,"15min":10.084},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59199999999999997,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77600000000000002,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.58999999999999997},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":1.54,"15min":1.54},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.40999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.35399999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.59199999999999997},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76800000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.61599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.40699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":499632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.505},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":2.371,"15min":2.371},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.505,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79200000000000004,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78200000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67700000000000005,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.72599999999999998},"min":{"1min":0.373,"5min":0.315,"15min":0.315},"max":{"1min":0.72099999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68400000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.437}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72999999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.63400000000000001},"min":{"1min":0.45400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.746,"5min":1.052,"15min":1.052},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.63700000000000001},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":1.175,"15min":1.175},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":1.145,"15min":1.145},"last":0.49199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:22.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:22.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.228+0000 7fd2c679a700 1 Processor -- start 2023-10-06T03:07:22.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.228+0000 7fd2c679a700 1 -- start start 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c679a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd2c01597f0 0x7fd2c00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c679a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c00b11d0 con 0x7fd2c0159420 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c679a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c00b14b0 con 0x7fd2c015d230 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c679a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c00b1790 con 0x7fd2c01597f0 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c5798700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd2c0159420 0x7fd2c00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46900/0 (socket says 172.21.15.92:46900) 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c5798700 1 -- 172.21.15.92:0/288606076 learned_addr learned my addr 172.21.15.92:0/288606076 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2c4f97700 1 --2- 172.21.15.92:0/288606076 >> v2:172.21.15.136:3300/0 conn(0x7fd2c01597f0 0x7fd2c00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945514305 0 0) 0x7fd2c00b11d0 con 0x7fd2c0159420 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.229+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2ac003570 con 0x7fd2c0159420 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1627525536 0 0) 0x7fd2ac003570 con 0x7fd2c0159420 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 >> v2:172.21.15.136:3300/0 conn(0x7fd2c01597f0 msgr2=0x7fd2c00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 --2- 172.21.15.92:0/288606076 >> v2:172.21.15.136:3300/0 conn(0x7fd2c01597f0 0x7fd2c00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 >> v1:172.21.15.136:6789/0 conn(0x7fd2c015d230 legacy=0x7fd2c015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00b1a70 con 0x7fd2c0159420 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd2bc0017f0 con 0x7fd2c0159420 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 >> v1:172.21.15.92:6789/0 conn(0x7fd2c0159420 legacy=0x7fd2c00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ac003a60 0x7fd2ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2ac004810 0x7fd2ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac0051c0 0x7fd2ac00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00b1790 con 0x7fd2ac003a60 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00c0d0 con 0x7fd2ac0051c0 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac00c290 con 0x7fd2ac004810 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ac003a60 0x7fd2ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c4f97700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2ac004810 0x7fd2ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 -- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2ac004810 msgr2=0x7fd2ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2ac004810 0x7fd2ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 -- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac0051c0 msgr2=0x7fd2ac00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5f99700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac0051c0 0x7fd2ac00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac0051c0 0x7fd2ac00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 -- 172.21.15.92:0/288606076 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac003570 con 0x7fd2ac003a60 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5f99700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac0051c0 0x7fd2ac00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5f99700 1 -- 172.21.15.92:0/288606076 reap_dead start 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2c5798700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ac003a60 0x7fd2ac003e70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00a930 tx=0x7fd2bc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6fe54be6b095ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0070d0 con 0x7fd2ac003a60 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc007230 con 0x7fd2ac003a60 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.230+0000 7fd2b7fff700 1 -- 172.21.15.92:0/288606076 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0075d0 con 0x7fd2ac003a60 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 -- 172.21.15.92:0/288606076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ac003a60 msgr2=0x7fd2ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ac003a60 0x7fd2ac003e70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00a930 tx=0x7fd2bc0021d0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 -- 172.21.15.92:0/288606076 shutdown_connections 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 --2- 172.21.15.92:0/288606076 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2ac003a60 0x7fd2ac003e70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 -- 172.21.15.92:0/288606076 >> 172.21.15.92:0/288606076 conn(0x7fd2c00d0660 msgr2=0x7fd2c00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:22.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 -- 172.21.15.92:0/288606076 shutdown_connections 2023-10-06T03:07:22.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.231+0000 7fd2c679a700 1 -- 172.21.15.92:0/288606076 wait complete. 2023-10-06T03:07:22.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.232+0000 7fd2c679a700 1 Processor -- start 2023-10-06T03:07:22.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.232+0000 7fd2c679a700 1 -- start start 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c679a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac003a60 0x7fd2c0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c679a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 0x7fd2c014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c679a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c016d610 0x7fd2c016da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c679a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c016d180 con 0x7fd2c0169940 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c679a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00b6520 con 0x7fd2ac003a60 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c679a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0169db0 con 0x7fd2c016d610 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 0x7fd2c014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 0x7fd2c014a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36996/0 (socket says 172.21.15.92:36996) 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 -- 172.21.15.92:0/1046621315 learned_addr learned my addr 172.21.15.92:0/1046621315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c5798700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac003a60 0x7fd2c0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c5f99700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c016d610 0x7fd2c016da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 -- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c016d610 msgr2=0x7fd2c016da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c016d610 0x7fd2c016da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 -- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac003a60 msgr2=0x7fd2c0155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac003a60 0x7fd2c0155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 -- 172.21.15.92:0/1046621315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c016a070 con 0x7fd2c0169940 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c5798700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac003a60 0x7fd2c0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c5f99700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c016d610 0x7fd2c016da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.233+0000 7fd2c4f97700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 0x7fd2c014a0b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011990 tx=0x7fd2b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:22.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.234+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000f6d0 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.234+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000f830 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.234+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000fbf0 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.234+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c016a350 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.234+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c014a5f0 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.235+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c016a870 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.235+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd2b0015460 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.235+0000 7fd2b57fa700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2a4067230 0x7fd2a40696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.236+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd2b0084d70 con 0x7fd2c0169940 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.236+0000 7fd2c5798700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2a4067230 0x7fd2a40696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.236+0000 7fd2c5798700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2a4067230 0x7fd2a40696e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc002250 tx=0x7fd2bc016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:22.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.240+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2c016a870 con 0x7fd2c0169940 2023-10-06T03:07:22.306 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T03:07:22.304+0000 7f484b5ab700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:07:22.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.418+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2c016a870 con 0x7fd2a4067230 2023-10-06T03:07:22.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.426+0000 7fd2b57fa700 1 -- 172.21.15.92:0/1046621315 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311762 (secure 0 0 0) 0x7fd2c016a870 con 0x7fd2a4067230 2023-10-06T03:07:22.432 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2a4067230 msgr2=0x7fd2a40696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2a4067230 0x7fd2a40696e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc002250 tx=0x7fd2bc016000 comp rx=0 tx=0).stop 2023-10-06T03:07:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 msgr2=0x7fd2c014a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 0x7fd2c014a0b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011990 tx=0x7fd2b0008040 comp rx=0 tx=0).stop 2023-10-06T03:07:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 shutdown_connections 2023-10-06T03:07:22.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2a4067230 0x7fd2a40696e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c016d610 0x7fd2c016da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2ac003a60 0x7fd2c0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 --2- 172.21.15.92:0/1046621315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2c0169940 0x7fd2c014a0b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 >> 172.21.15.92:0/1046621315 conn(0x7fd2c00d0660 msgr2=0x7fd2c00b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 shutdown_connections 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.430+0000 7fd2c679a700 1 -- 172.21.15.92:0/1046621315 wait complete. 2023-10-06T03:07:22.435 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:22.444 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-10-06T03:07:21.252029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2400448,"kb_used_data":9856,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752574272,"statfs":{"total":773094113280,"available":770636054528,"internally_reserved":0,"allocated":10092544,"data_stored":5195686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.261227"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527339+0000","last_change":"2023-10-06T03:07:17.527339+0000","last_active":"2023-10-06T03:07:17.527339+0000","last_peered":"2023-10-06T03:07:17.527339+0000","last_clean":"2023-10-06T03:07:17.527339+0000","last_became_active":"2023-10-06T03:07:17.527057+0000","last_became_peered":"2023-10-06T03:07:17.527057+0000","last_unstale":"2023-10-06T03:07:17.527339+0000","last_undegraded":"2023-10-06T03:07:17.527339+0000","last_fullsized":"2023-10-06T03:07:17.527339+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:26.929634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.544153+0000","last_change":"2023-10-06T03:07:17.538819+0000","last_active":"2023-10-06T03:07:17.544153+0000","last_peered":"2023-10-06T03:07:17.544153+0000","last_clean":"2023-10-06T03:07:17.544153+0000","last_became_active":"2023-10-06T03:07:17.538545+0000","last_became_peered":"2023-10-06T03:07:17.538545+0000","last_unstale":"2023-10-06T03:07:17.544153+0000","last_undegraded":"2023-10-06T03:07:17.544153+0000","last_fullsized":"2023-10-06T03:07:17.544153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:10.728835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.529417+0000","last_change":"2023-10-06T03:07:17.529417+0000","last_active":"2023-10-06T03:07:17.529417+0000","last_peered":"2023-10-06T03:07:17.529417+0000","last_clean":"2023-10-06T03:07:17.529417+0000","last_became_active":"2023-10-06T03:07:17.529154+0000","last_became_peered":"2023-10-06T03:07:17.529154+0000","last_unstale":"2023-10-06T03:07:17.529417+0000","last_undegraded":"2023-10-06T03:07:17.529417+0000","last_fullsized":"2023-10-06T03:07:17.529417+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:46.786714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534521+0000","last_change":"2023-10-06T03:07:17.530089+0000","last_active":"2023-10-06T03:07:17.534521+0000","last_peered":"2023-10-06T03:07:17.534521+0000","last_clean":"2023-10-06T03:07:17.534521+0000","last_became_active":"2023-10-06T03:07:17.529839+0000","last_became_peered":"2023-10-06T03:07:17.529839+0000","last_unstale":"2023-10-06T03:07:17.534521+0000","last_undegraded":"2023-10-06T03:07:17.534521+0000","last_fullsized":"2023-10-06T03:07:17.534521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:38.597127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534743+0000","last_change":"2023-10-06T03:07:17.528293+0000","last_active":"2023-10-06T03:07:17.534743+0000","last_peered":"2023-10-06T03:07:17.534743+0000","last_clean":"2023-10-06T03:07:17.534743+0000","last_became_active":"2023-10-06T03:07:17.527792+0000","last_became_peered":"2023-10-06T03:07:17.527792+0000","last_unstale":"2023-10-06T03:07:17.534743+0000","last_undegraded":"2023-10-06T03:07:17.534743+0000","last_fullsized":"2023-10-06T03:07:17.534743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:48.414529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.543618+0000","last_change":"2023-10-06T03:07:17.537962+0000","last_active":"2023-10-06T03:07:17.543618+0000","last_peered":"2023-10-06T03:07:17.543618+0000","last_clean":"2023-10-06T03:07:17.543618+0000","last_became_active":"2023-10-06T03:07:17.537630+0000","last_became_peered":"2023-10-06T03:07:17.537630+0000","last_unstale":"2023-10-06T03:07:17.543618+0000","last_undegraded":"2023-10-06T03:07:17.543618+0000","last_fullsized":"2023-10-06T03:07:17.543618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:22.771445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539300+0000","last_change":"2023-10-06T03:07:17.536823+0000","last_active":"2023-10-06T03:07:17.539300+0000","last_peered":"2023-10-06T03:07:17.539300+0000","last_clean":"2023-10-06T03:07:17.539300+0000","last_became_active":"2023-10-06T03:07:17.536568+0000","last_became_peered":"2023-10-06T03:07:17.536568+0000","last_unstale":"2023-10-06T03:07:17.539300+0000","last_undegraded":"2023-10-06T03:07:17.539300+0000","last_fullsized":"2023-10-06T03:07:17.539300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:25.254466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.532209+0000","last_change":"2023-10-06T03:07:17.532209+0000","last_active":"2023-10-06T03:07:17.532209+0000","last_peered":"2023-10-06T03:07:17.532209+0000","last_clean":"2023-10-06T03:07:17.532209+0000","last_became_active":"2023-10-06T03:07:17.531855+0000","last_became_peered":"2023-10-06T03:07:17.531855+0000","last_unstale":"2023-10-06T03:07:17.532209+0000","last_undegraded":"2023-10-06T03:07:17.532209+0000","last_fullsized":"2023-10-06T03:07:17.532209+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:06.974401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539198+0000","last_change":"2023-10-06T03:07:17.536094+0000","last_active":"2023-10-06T03:07:17.539198+0000","last_peered":"2023-10-06T03:07:17.539198+0000","last_clean":"2023-10-06T03:07:17.539198+0000","last_became_active":"2023-10-06T03:07:17.535829+0000","last_became_peered":"2023-10-06T03:07:17.535829+0000","last_unstale":"2023-10-06T03:07:17.539198+0000","last_undegraded":"2023-10-06T03:07:17.539198+0000","last_fullsized":"2023-10-06T03:07:17.539198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:52.803767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535680+0000","last_change":"2023-10-06T03:07:17.532834+0000","last_active":"2023-10-06T03:07:17.535680+0000","last_peered":"2023-10-06T03:07:17.535680+0000","last_clean":"2023-10-06T03:07:17.535680+0000","last_became_active":"2023-10-06T03:07:17.532577+0000","last_became_peered":"2023-10-06T03:07:17.532577+0000","last_unstale":"2023-10-06T03:07:17.535680+0000","last_undegraded":"2023-10-06T03:07:17.535680+0000","last_fullsized":"2023-10-06T03:07:17.535680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:25.254466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536035+0000","last_change":"2023-10-06T03:07:17.533123+0000","last_active":"2023-10-06T03:07:17.536035+0000","last_peered":"2023-10-06T03:07:17.536035+0000","last_clean":"2023-10-06T03:07:17.536035+0000","last_became_active":"2023-10-06T03:07:17.532876+0000","last_became_peered":"2023-10-06T03:07:17.532876+0000","last_unstale":"2023-10-06T03:07:17.536035+0000","last_undegraded":"2023-10-06T03:07:17.536035+0000","last_fullsized":"2023-10-06T03:07:17.536035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:46.186781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531328+0000","last_change":"2023-10-06T03:07:17.527206+0000","last_active":"2023-10-06T03:07:17.531328+0000","last_peered":"2023-10-06T03:07:17.531328+0000","last_clean":"2023-10-06T03:07:17.531328+0000","last_became_active":"2023-10-06T03:07:17.526943+0000","last_became_peered":"2023-10-06T03:07:17.526943+0000","last_unstale":"2023-10-06T03:07:17.531328+0000","last_undegraded":"2023-10-06T03:07:17.531328+0000","last_fullsized":"2023-10-06T03:07:17.531328+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:43.965385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535072+0000","last_change":"2023-10-06T03:07:17.533145+0000","last_active":"2023-10-06T03:07:17.535072+0000","last_peered":"2023-10-06T03:07:17.535072+0000","last_clean":"2023-10-06T03:07:17.535072+0000","last_became_active":"2023-10-06T03:07:17.532910+0000","last_became_peered":"2023-10-06T03:07:17.532910+0000","last_unstale":"2023-10-06T03:07:17.535072+0000","last_undegraded":"2023-10-06T03:07:17.535072+0000","last_fullsized":"2023-10-06T03:07:17.535072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:50.035401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534325+0000","last_change":"2023-10-06T03:07:17.534325+0000","last_active":"2023-10-06T03:07:17.534325+0000","last_peered":"2023-10-06T03:07:17.534325+0000","last_clean":"2023-10-06T03:07:17.534325+0000","last_became_active":"2023-10-06T03:07:17.533938+0000","last_became_peered":"2023-10-06T03:07:17.533938+0000","last_unstale":"2023-10-06T03:07:17.534325+0000","last_undegraded":"2023-10-06T03:07:17.534325+0000","last_fullsized":"2023-10-06T03:07:17.534325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:41.311041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.532754+0000","last_change":"2023-10-06T03:07:17.532754+0000","last_active":"2023-10-06T03:07:17.532754+0000","last_peered":"2023-10-06T03:07:17.532754+0000","last_clean":"2023-10-06T03:07:17.532754+0000","last_became_active":"2023-10-06T03:07:17.532506+0000","last_became_peered":"2023-10-06T03:07:17.532506+0000","last_unstale":"2023-10-06T03:07:17.532754+0000","last_undegraded":"2023-10-06T03:07:17.532754+0000","last_fullsized":"2023-10-06T03:07:17.532754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:06.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534386+0000","last_change":"2023-10-06T03:07:17.534386+0000","last_active":"2023-10-06T03:07:17.534386+0000","last_peered":"2023-10-06T03:07:17.534386+0000","last_clean":"2023-10-06T03:07:17.534386+0000","last_became_active":"2023-10-06T03:07:17.534072+0000","last_became_peered":"2023-10-06T03:07:17.534072+0000","last_unstale":"2023-10-06T03:07:17.534386+0000","last_undegraded":"2023-10-06T03:07:17.534386+0000","last_fullsized":"2023-10-06T03:07:17.534386+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:20.474907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.529342+0000","last_change":"2023-10-06T03:07:17.529342+0000","last_active":"2023-10-06T03:07:17.529342+0000","last_peered":"2023-10-06T03:07:17.529342+0000","last_clean":"2023-10-06T03:07:17.529342+0000","last_became_active":"2023-10-06T03:07:17.529132+0000","last_became_peered":"2023-10-06T03:07:17.529132+0000","last_unstale":"2023-10-06T03:07:17.529342+0000","last_undegraded":"2023-10-06T03:07:17.529342+0000","last_fullsized":"2023-10-06T03:07:17.529342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:24.805888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.543599+0000","last_change":"2023-10-06T03:07:17.537578+0000","last_active":"2023-10-06T03:07:17.543599+0000","last_peered":"2023-10-06T03:07:17.543599+0000","last_clean":"2023-10-06T03:07:17.543599+0000","last_became_active":"2023-10-06T03:07:17.537303+0000","last_became_peered":"2023-10-06T03:07:17.537303+0000","last_unstale":"2023-10-06T03:07:17.543599+0000","last_undegraded":"2023-10-06T03:07:17.543599+0000","last_fullsized":"2023-10-06T03:07:17.543599+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:40.254108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534224+0000","last_change":"2023-10-06T03:07:17.534224+0000","last_active":"2023-10-06T03:07:17.534224+0000","last_peered":"2023-10-06T03:07:17.534224+0000","last_clean":"2023-10-06T03:07:17.534224+0000","last_became_active":"2023-10-06T03:07:17.533869+0000","last_became_peered":"2023-10-06T03:07:17.533869+0000","last_unstale":"2023-10-06T03:07:17.534224+0000","last_undegraded":"2023-10-06T03:07:17.534224+0000","last_fullsized":"2023-10-06T03:07:17.534224+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:44.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537941+0000","last_change":"2023-10-06T03:07:17.534432+0000","last_active":"2023-10-06T03:07:17.537941+0000","last_peered":"2023-10-06T03:07:17.537941+0000","last_clean":"2023-10-06T03:07:17.537941+0000","last_became_active":"2023-10-06T03:07:17.534177+0000","last_became_peered":"2023-10-06T03:07:17.534177+0000","last_unstale":"2023-10-06T03:07:17.537941+0000","last_undegraded":"2023-10-06T03:07:17.537941+0000","last_fullsized":"2023-10-06T03:07:17.537941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:37:40.361815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534227+0000","last_change":"2023-10-06T03:07:17.534227+0000","last_active":"2023-10-06T03:07:17.534227+0000","last_peered":"2023-10-06T03:07:17.534227+0000","last_clean":"2023-10-06T03:07:17.534227+0000","last_became_active":"2023-10-06T03:07:17.533984+0000","last_became_peered":"2023-10-06T03:07:17.533984+0000","last_unstale":"2023-10-06T03:07:17.534227+0000","last_undegraded":"2023-10-06T03:07:17.534227+0000","last_fullsized":"2023-10-06T03:07:17.534227+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:34.596239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534491+0000","last_change":"2023-10-06T03:07:17.526376+0000","last_active":"2023-10-06T03:07:17.534491+0000","last_peered":"2023-10-06T03:07:17.534491+0000","last_clean":"2023-10-06T03:07:17.534491+0000","last_became_active":"2023-10-06T03:07:17.526134+0000","last_became_peered":"2023-10-06T03:07:17.526134+0000","last_unstale":"2023-10-06T03:07:17.534491+0000","last_undegraded":"2023-10-06T03:07:17.534491+0000","last_fullsized":"2023-10-06T03:07:17.534491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:58.202326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.523919+0000","last_change":"2023-10-06T03:07:17.523919+0000","last_active":"2023-10-06T03:07:17.523919+0000","last_peered":"2023-10-06T03:07:17.523919+0000","last_clean":"2023-10-06T03:07:17.523919+0000","last_became_active":"2023-10-06T03:07:17.523629+0000","last_became_peered":"2023-10-06T03:07:17.523629+0000","last_unstale":"2023-10-06T03:07:17.523919+0000","last_undegraded":"2023-10-06T03:07:17.523919+0000","last_fullsized":"2023-10-06T03:07:17.523919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:02.609086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530345+0000","last_change":"2023-10-06T03:07:17.524163+0000","last_active":"2023-10-06T03:07:17.530345+0000","last_peered":"2023-10-06T03:07:17.530345+0000","last_clean":"2023-10-06T03:07:17.530345+0000","last_became_active":"2023-10-06T03:07:17.523931+0000","last_became_peered":"2023-10-06T03:07:17.523931+0000","last_unstale":"2023-10-06T03:07:17.530345+0000","last_undegraded":"2023-10-06T03:07:17.530345+0000","last_fullsized":"2023-10-06T03:07:17.530345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:22.771445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536791+0000","last_change":"2023-10-06T03:07:17.531005+0000","last_active":"2023-10-06T03:07:17.536791+0000","last_peered":"2023-10-06T03:07:17.536791+0000","last_clean":"2023-10-06T03:07:17.536791+0000","last_became_active":"2023-10-06T03:07:17.530778+0000","last_became_peered":"2023-10-06T03:07:17.530778+0000","last_unstale":"2023-10-06T03:07:17.536791+0000","last_undegraded":"2023-10-06T03:07:17.536791+0000","last_fullsized":"2023-10-06T03:07:17.536791+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:52.229338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537508+0000","last_change":"2023-10-06T03:07:17.532858+0000","last_active":"2023-10-06T03:07:17.537508+0000","last_peered":"2023-10-06T03:07:17.537508+0000","last_clean":"2023-10-06T03:07:17.537508+0000","last_became_active":"2023-10-06T03:07:17.532599+0000","last_became_peered":"2023-10-06T03:07:17.532599+0000","last_unstale":"2023-10-06T03:07:17.537508+0000","last_undegraded":"2023-10-06T03:07:17.537508+0000","last_fullsized":"2023-10-06T03:07:17.537508+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:16.905978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.538071+0000","last_change":"2023-10-06T03:07:17.535052+0000","last_active":"2023-10-06T03:07:17.538071+0000","last_peered":"2023-10-06T03:07:17.538071+0000","last_clean":"2023-10-06T03:07:17.538071+0000","last_became_active":"2023-10-06T03:07:17.534748+0000","last_became_peered":"2023-10-06T03:07:17.534748+0000","last_unstale":"2023-10-06T03:07:17.538071+0000","last_undegraded":"2023-10-06T03:07:17.538071+0000","last_fullsized":"2023-10-06T03:07:17.538071+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537994+0000","last_change":"2023-10-06T03:07:17.533302+0000","last_active":"2023-10-06T03:07:17.537994+0000","last_peered":"2023-10-06T03:07:17.537994+0000","last_clean":"2023-10-06T03:07:17.537994+0000","last_became_active":"2023-10-06T03:07:17.532954+0000","last_became_peered":"2023-10-06T03:07:17.532954+0000","last_unstale":"2023-10-06T03:07:17.537994+0000","last_undegraded":"2023-10-06T03:07:17.537994+0000","last_fullsized":"2023-10-06T03:07:17.537994+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:07.990891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536183+0000","last_change":"2023-10-06T03:07:17.536183+0000","last_active":"2023-10-06T03:07:17.536183+0000","last_peered":"2023-10-06T03:07:17.536183+0000","last_clean":"2023-10-06T03:07:17.536183+0000","last_became_active":"2023-10-06T03:07:17.535934+0000","last_became_peered":"2023-10-06T03:07:17.535934+0000","last_unstale":"2023-10-06T03:07:17.536183+0000","last_undegraded":"2023-10-06T03:07:17.536183+0000","last_fullsized":"2023-10-06T03:07:17.536183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:09.679192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.537281+0000","last_change":"2023-10-06T03:07:17.531702+0000","last_active":"2023-10-06T03:07:17.537281+0000","last_peered":"2023-10-06T03:07:17.537281+0000","last_clean":"2023-10-06T03:07:17.537281+0000","last_became_active":"2023-10-06T03:07:17.531445+0000","last_became_peered":"2023-10-06T03:07:17.531445+0000","last_unstale":"2023-10-06T03:07:17.537281+0000","last_undegraded":"2023-10-06T03:07:17.537281+0000","last_fullsized":"2023-10-06T03:07:17.537281+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:04.098459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539280+0000","last_change":"2023-10-06T03:07:17.536702+0000","last_active":"2023-10-06T03:07:17.539280+0000","last_peered":"2023-10-06T03:07:17.539280+0000","last_clean":"2023-10-06T03:07:17.539280+0000","last_became_active":"2023-10-06T03:07:17.536292+0000","last_became_peered":"2023-10-06T03:07:17.536292+0000","last_unstale":"2023-10-06T03:07:17.539280+0000","last_undegraded":"2023-10-06T03:07:17.539280+0000","last_fullsized":"2023-10-06T03:07:17.539280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:13.497707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.533117+0000","last_change":"2023-10-06T03:07:17.533117+0000","last_active":"2023-10-06T03:07:17.533117+0000","last_peered":"2023-10-06T03:07:17.533117+0000","last_clean":"2023-10-06T03:07:17.533117+0000","last_became_active":"2023-10-06T03:07:17.532175+0000","last_became_peered":"2023-10-06T03:07:17.532175+0000","last_unstale":"2023-10-06T03:07:17.533117+0000","last_undegraded":"2023-10-06T03:07:17.533117+0000","last_fullsized":"2023-10-06T03:07:17.533117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:52.115167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530334+0000","last_change":"2023-10-06T03:07:17.530334+0000","last_active":"2023-10-06T03:07:17.530334+0000","last_peered":"2023-10-06T03:07:17.530334+0000","last_clean":"2023-10-06T03:07:17.530334+0000","last_became_active":"2023-10-06T03:07:17.529959+0000","last_became_peered":"2023-10-06T03:07:17.529959+0000","last_unstale":"2023-10-06T03:07:17.530334+0000","last_undegraded":"2023-10-06T03:07:17.530334+0000","last_fullsized":"2023-10-06T03:07:17.530334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:52.977924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"89'4","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.025640+0000","last_change":"2023-10-06T03:07:17.017667+0000","last_active":"2023-10-06T03:07:18.025640+0000","last_peered":"2023-10-06T03:07:18.025640+0000","last_clean":"2023-10-06T03:07:18.025640+0000","last_became_active":"2023-10-06T03:07:17.017368+0000","last_became_peered":"2023-10-06T03:07:17.017368+0000","last_unstale":"2023-10-06T03:07:18.025640+0000","last_undegraded":"2023-10-06T03:07:18.025640+0000","last_fullsized":"2023-10-06T03:07:18.025640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:57.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531286+0000","last_change":"2023-10-06T03:07:17.531286+0000","last_active":"2023-10-06T03:07:17.531286+0000","last_peered":"2023-10-06T03:07:17.531286+0000","last_clean":"2023-10-06T03:07:17.531286+0000","last_became_active":"2023-10-06T03:07:17.530863+0000","last_became_peered":"2023-10-06T03:07:17.530863+0000","last_unstale":"2023-10-06T03:07:17.531286+0000","last_undegraded":"2023-10-06T03:07:17.531286+0000","last_fullsized":"2023-10-06T03:07:17.531286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:15:06.534375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.522636+0000","last_change":"2023-10-06T03:07:17.009505+0000","last_active":"2023-10-06T03:07:17.522636+0000","last_peered":"2023-10-06T03:07:17.522636+0000","last_clean":"2023-10-06T03:07:17.522636+0000","last_became_active":"2023-10-06T03:07:17.009266+0000","last_became_peered":"2023-10-06T03:07:17.009266+0000","last_unstale":"2023-10-06T03:07:17.522636+0000","last_undegraded":"2023-10-06T03:07:17.522636+0000","last_fullsized":"2023-10-06T03:07:17.522636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:49.449105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.536868+0000","last_change":"2023-10-06T03:07:17.530386+0000","last_active":"2023-10-06T03:07:17.536868+0000","last_peered":"2023-10-06T03:07:17.536868+0000","last_clean":"2023-10-06T03:07:17.536868+0000","last_became_active":"2023-10-06T03:07:17.530077+0000","last_became_peered":"2023-10-06T03:07:17.530077+0000","last_unstale":"2023-10-06T03:07:17.536868+0000","last_undegraded":"2023-10-06T03:07:17.536868+0000","last_fullsized":"2023-10-06T03:07:17.536868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:04.548561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.025611+0000","last_change":"2023-10-06T03:07:17.006708+0000","last_active":"2023-10-06T03:07:18.025611+0000","last_peered":"2023-10-06T03:07:18.025611+0000","last_clean":"2023-10-06T03:07:18.025611+0000","last_became_active":"2023-10-06T03:07:17.006317+0000","last_became_peered":"2023-10-06T03:07:17.006317+0000","last_unstale":"2023-10-06T03:07:18.025611+0000","last_undegraded":"2023-10-06T03:07:18.025611+0000","last_fullsized":"2023-10-06T03:07:18.025611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:42:19.997214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531908+0000","last_change":"2023-10-06T03:07:17.526243+0000","last_active":"2023-10-06T03:07:17.531908+0000","last_peered":"2023-10-06T03:07:17.531908+0000","last_clean":"2023-10-06T03:07:17.531908+0000","last_became_active":"2023-10-06T03:07:17.525987+0000","last_became_peered":"2023-10-06T03:07:17.525987+0000","last_unstale":"2023-10-06T03:07:17.531908+0000","last_undegraded":"2023-10-06T03:07:17.531908+0000","last_fullsized":"2023-10-06T03:07:17.531908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:38.597127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.025342+0000","last_change":"2023-10-06T03:07:17.006940+0000","last_active":"2023-10-06T03:07:18.025342+0000","last_peered":"2023-10-06T03:07:18.025342+0000","last_clean":"2023-10-06T03:07:18.025342+0000","last_became_active":"2023-10-06T03:07:17.006496+0000","last_became_peered":"2023-10-06T03:07:17.006496+0000","last_unstale":"2023-10-06T03:07:18.025342+0000","last_undegraded":"2023-10-06T03:07:18.025342+0000","last_fullsized":"2023-10-06T03:07:18.025342+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:01:17.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534539+0000","last_change":"2023-10-06T03:07:17.531004+0000","last_active":"2023-10-06T03:07:17.534539+0000","last_peered":"2023-10-06T03:07:17.534539+0000","last_clean":"2023-10-06T03:07:17.534539+0000","last_became_active":"2023-10-06T03:07:17.530750+0000","last_became_peered":"2023-10-06T03:07:17.530750+0000","last_unstale":"2023-10-06T03:07:17.534539+0000","last_undegraded":"2023-10-06T03:07:17.534539+0000","last_fullsized":"2023-10-06T03:07:17.534539+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:13.497707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.525182+0000","last_change":"2023-10-06T03:07:17.014936+0000","last_active":"2023-10-06T03:07:17.525182+0000","last_peered":"2023-10-06T03:07:17.525182+0000","last_clean":"2023-10-06T03:07:17.525182+0000","last_became_active":"2023-10-06T03:07:17.014624+0000","last_became_peered":"2023-10-06T03:07:17.014624+0000","last_unstale":"2023-10-06T03:07:17.525182+0000","last_undegraded":"2023-10-06T03:07:17.525182+0000","last_fullsized":"2023-10-06T03:07:17.525182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:02:09.724175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534267+0000","last_change":"2023-10-06T03:07:17.534267+0000","last_active":"2023-10-06T03:07:17.534267+0000","last_peered":"2023-10-06T03:07:17.534267+0000","last_clean":"2023-10-06T03:07:17.534267+0000","last_became_active":"2023-10-06T03:07:17.534002+0000","last_became_peered":"2023-10-06T03:07:17.534002+0000","last_unstale":"2023-10-06T03:07:17.534267+0000","last_undegraded":"2023-10-06T03:07:17.534267+0000","last_fullsized":"2023-10-06T03:07:17.534267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:15.983003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.023074+0000","last_change":"2023-10-06T03:07:17.007342+0000","last_active":"2023-10-06T03:07:18.023074+0000","last_peered":"2023-10-06T03:07:18.023074+0000","last_clean":"2023-10-06T03:07:18.023074+0000","last_became_active":"2023-10-06T03:07:17.007090+0000","last_became_peered":"2023-10-06T03:07:17.007090+0000","last_unstale":"2023-10-06T03:07:18.023074+0000","last_undegraded":"2023-10-06T03:07:18.023074+0000","last_fullsized":"2023-10-06T03:07:18.023074+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:49.057674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539343+0000","last_change":"2023-10-06T03:07:17.537040+0000","last_active":"2023-10-06T03:07:17.539343+0000","last_peered":"2023-10-06T03:07:17.539343+0000","last_clean":"2023-10-06T03:07:17.539343+0000","last_became_active":"2023-10-06T03:07:17.535498+0000","last_became_peered":"2023-10-06T03:07:17.535498+0000","last_unstale":"2023-10-06T03:07:17.539343+0000","last_undegraded":"2023-10-06T03:07:17.539343+0000","last_fullsized":"2023-10-06T03:07:17.539343+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:29.797991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.520020+0000","last_change":"2023-10-06T03:07:17.006621+0000","last_active":"2023-10-06T03:07:17.520020+0000","last_peered":"2023-10-06T03:07:17.520020+0000","last_clean":"2023-10-06T03:07:17.520020+0000","last_became_active":"2023-10-06T03:07:17.006249+0000","last_became_peered":"2023-10-06T03:07:17.006249+0000","last_unstale":"2023-10-06T03:07:17.520020+0000","last_undegraded":"2023-10-06T03:07:17.520020+0000","last_fullsized":"2023-10-06T03:07:17.520020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:17.225175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535017+0000","last_change":"2023-10-06T03:07:17.531718+0000","last_active":"2023-10-06T03:07:17.535017+0000","last_peered":"2023-10-06T03:07:17.535017+0000","last_clean":"2023-10-06T03:07:17.535017+0000","last_became_active":"2023-10-06T03:07:17.531454+0000","last_became_peered":"2023-10-06T03:07:17.531454+0000","last_unstale":"2023-10-06T03:07:17.535017+0000","last_undegraded":"2023-10-06T03:07:17.535017+0000","last_fullsized":"2023-10-06T03:07:17.535017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:28.389638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.522649+0000","last_change":"2023-10-06T03:07:17.010059+0000","last_active":"2023-10-06T03:07:17.522649+0000","last_peered":"2023-10-06T03:07:17.522649+0000","last_clean":"2023-10-06T03:07:17.522649+0000","last_became_active":"2023-10-06T03:07:17.009662+0000","last_became_peered":"2023-10-06T03:07:17.009662+0000","last_unstale":"2023-10-06T03:07:17.522649+0000","last_undegraded":"2023-10-06T03:07:17.522649+0000","last_fullsized":"2023-10-06T03:07:17.522649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:47.409869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530591+0000","last_change":"2023-10-06T03:07:17.530591+0000","last_active":"2023-10-06T03:07:17.530591+0000","last_peered":"2023-10-06T03:07:17.530591+0000","last_clean":"2023-10-06T03:07:17.530591+0000","last_became_active":"2023-10-06T03:07:17.530301+0000","last_became_peered":"2023-10-06T03:07:17.530301+0000","last_unstale":"2023-10-06T03:07:17.530591+0000","last_undegraded":"2023-10-06T03:07:17.530591+0000","last_fullsized":"2023-10-06T03:07:17.530591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:28.312569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.520259+0000","last_change":"2023-10-06T03:07:17.007059+0000","last_active":"2023-10-06T03:07:17.520259+0000","last_peered":"2023-10-06T03:07:17.520259+0000","last_clean":"2023-10-06T03:07:17.520259+0000","last_became_active":"2023-10-06T03:07:17.006782+0000","last_became_peered":"2023-10-06T03:07:17.006782+0000","last_unstale":"2023-10-06T03:07:17.520259+0000","last_undegraded":"2023-10-06T03:07:17.520259+0000","last_fullsized":"2023-10-06T03:07:17.520259+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:42.034142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534172+0000","last_change":"2023-10-06T03:07:17.527518+0000","last_active":"2023-10-06T03:07:17.534172+0000","last_peered":"2023-10-06T03:07:17.534172+0000","last_clean":"2023-10-06T03:07:17.534172+0000","last_became_active":"2023-10-06T03:07:17.526872+0000","last_became_peered":"2023-10-06T03:07:17.526872+0000","last_unstale":"2023-10-06T03:07:17.534172+0000","last_undegraded":"2023-10-06T03:07:17.534172+0000","last_fullsized":"2023-10-06T03:07:17.534172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:50.742831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.014923+0000","last_change":"2023-10-06T03:07:17.011807+0000","last_active":"2023-10-06T03:07:17.014923+0000","last_peered":"2023-10-06T03:07:17.014923+0000","last_clean":"2023-10-06T03:07:17.014923+0000","last_became_active":"2023-10-06T03:07:17.011540+0000","last_became_peered":"2023-10-06T03:07:17.011540+0000","last_unstale":"2023-10-06T03:07:17.014923+0000","last_undegraded":"2023-10-06T03:07:17.014923+0000","last_fullsized":"2023-10-06T03:07:17.014923+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:56.364506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531051+0000","last_change":"2023-10-06T03:07:17.531051+0000","last_active":"2023-10-06T03:07:17.531051+0000","last_peered":"2023-10-06T03:07:17.531051+0000","last_clean":"2023-10-06T03:07:17.531051+0000","last_became_active":"2023-10-06T03:07:17.530754+0000","last_became_peered":"2023-10-06T03:07:17.530754+0000","last_unstale":"2023-10-06T03:07:17.531051+0000","last_undegraded":"2023-10-06T03:07:17.531051+0000","last_fullsized":"2023-10-06T03:07:17.531051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:07.182287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"89'3","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.027586+0000","last_change":"2023-10-06T03:07:17.015454+0000","last_active":"2023-10-06T03:07:18.027586+0000","last_peered":"2023-10-06T03:07:18.027586+0000","last_clean":"2023-10-06T03:07:18.027586+0000","last_became_active":"2023-10-06T03:07:17.015175+0000","last_became_peered":"2023-10-06T03:07:17.015175+0000","last_unstale":"2023-10-06T03:07:18.027586+0000","last_undegraded":"2023-10-06T03:07:18.027586+0000","last_fullsized":"2023-10-06T03:07:18.027586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:49.030741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534761+0000","last_change":"2023-10-06T03:07:17.528416+0000","last_active":"2023-10-06T03:07:17.534761+0000","last_peered":"2023-10-06T03:07:17.534761+0000","last_clean":"2023-10-06T03:07:17.534761+0000","last_became_active":"2023-10-06T03:07:17.528041+0000","last_became_peered":"2023-10-06T03:07:17.528041+0000","last_unstale":"2023-10-06T03:07:17.534761+0000","last_undegraded":"2023-10-06T03:07:17.534761+0000","last_fullsized":"2023-10-06T03:07:17.534761+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:40.254108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"89'6","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.029420+0000","last_change":"2023-10-06T03:07:17.017273+0000","last_active":"2023-10-06T03:07:18.029420+0000","last_peered":"2023-10-06T03:07:18.029420+0000","last_clean":"2023-10-06T03:07:18.029420+0000","last_became_active":"2023-10-06T03:07:17.017027+0000","last_became_peered":"2023-10-06T03:07:17.017027+0000","last_unstale":"2023-10-06T03:07:18.029420+0000","last_undegraded":"2023-10-06T03:07:18.029420+0000","last_fullsized":"2023-10-06T03:07:18.029420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:01:17.713574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.534475+0000","last_change":"2023-10-06T03:07:17.529212+0000","last_active":"2023-10-06T03:07:17.534475+0000","last_peered":"2023-10-06T03:07:17.534475+0000","last_clean":"2023-10-06T03:07:17.534475+0000","last_became_active":"2023-10-06T03:07:17.529001+0000","last_became_peered":"2023-10-06T03:07:17.529001+0000","last_unstale":"2023-10-06T03:07:17.534475+0000","last_undegraded":"2023-10-06T03:07:17.534475+0000","last_fullsized":"2023-10-06T03:07:17.534475+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:39:33.419255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.031294+0000","last_change":"2023-10-06T03:07:17.013452+0000","last_active":"2023-10-06T03:07:18.031294+0000","last_peered":"2023-10-06T03:07:18.031294+0000","last_clean":"2023-10-06T03:07:18.031294+0000","last_became_active":"2023-10-06T03:07:17.013201+0000","last_became_peered":"2023-10-06T03:07:17.013201+0000","last_unstale":"2023-10-06T03:07:18.031294+0000","last_undegraded":"2023-10-06T03:07:18.031294+0000","last_fullsized":"2023-10-06T03:07:18.031294+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:16.512800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.538036+0000","last_change":"2023-10-06T03:07:17.532543+0000","last_active":"2023-10-06T03:07:17.538036+0000","last_peered":"2023-10-06T03:07:17.538036+0000","last_clean":"2023-10-06T03:07:17.538036+0000","last_became_active":"2023-10-06T03:07:17.532240+0000","last_became_peered":"2023-10-06T03:07:17.532240+0000","last_unstale":"2023-10-06T03:07:17.538036+0000","last_undegraded":"2023-10-06T03:07:17.538036+0000","last_fullsized":"2023-10-06T03:07:17.538036+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:58.202326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.521970+0000","last_change":"2023-10-06T03:07:17.009383+0000","last_active":"2023-10-06T03:07:17.521970+0000","last_peered":"2023-10-06T03:07:17.521970+0000","last_clean":"2023-10-06T03:07:17.521970+0000","last_became_active":"2023-10-06T03:07:17.009117+0000","last_became_peered":"2023-10-06T03:07:17.009117+0000","last_unstale":"2023-10-06T03:07:17.521970+0000","last_undegraded":"2023-10-06T03:07:17.521970+0000","last_fullsized":"2023-10-06T03:07:17.521970+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:26.379689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.526589+0000","last_change":"2023-10-06T03:07:17.526589+0000","last_active":"2023-10-06T03:07:17.526589+0000","last_peered":"2023-10-06T03:07:17.526589+0000","last_clean":"2023-10-06T03:07:17.526589+0000","last_became_active":"2023-10-06T03:07:17.526325+0000","last_became_peered":"2023-10-06T03:07:17.526325+0000","last_unstale":"2023-10-06T03:07:17.526589+0000","last_undegraded":"2023-10-06T03:07:17.526589+0000","last_fullsized":"2023-10-06T03:07:17.526589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:44.858785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015828+0000","last_change":"2023-10-06T03:07:17.008343+0000","last_active":"2023-10-06T03:07:17.015828+0000","last_peered":"2023-10-06T03:07:17.015828+0000","last_clean":"2023-10-06T03:07:17.015828+0000","last_became_active":"2023-10-06T03:07:17.008091+0000","last_became_peered":"2023-10-06T03:07:17.008091+0000","last_unstale":"2023-10-06T03:07:17.015828+0000","last_undegraded":"2023-10-06T03:07:17.015828+0000","last_fullsized":"2023-10-06T03:07:17.015828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:42.588920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.524415+0000","last_change":"2023-10-06T03:07:17.524415+0000","last_active":"2023-10-06T03:07:17.524415+0000","last_peered":"2023-10-06T03:07:17.524415+0000","last_clean":"2023-10-06T03:07:17.524415+0000","last_became_active":"2023-10-06T03:07:17.523666+0000","last_became_peered":"2023-10-06T03:07:17.523666+0000","last_unstale":"2023-10-06T03:07:17.524415+0000","last_undegraded":"2023-10-06T03:07:17.524415+0000","last_fullsized":"2023-10-06T03:07:17.524415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:07.574334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"89'8","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.028512+0000","last_change":"2023-10-06T03:07:17.006546+0000","last_active":"2023-10-06T03:07:18.028512+0000","last_peered":"2023-10-06T03:07:18.028512+0000","last_clean":"2023-10-06T03:07:18.028512+0000","last_became_active":"2023-10-06T03:07:17.006217+0000","last_became_peered":"2023-10-06T03:07:17.006217+0000","last_unstale":"2023-10-06T03:07:18.028512+0000","last_undegraded":"2023-10-06T03:07:18.028512+0000","last_fullsized":"2023-10-06T03:07:18.028512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:28.315419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527585+0000","last_change":"2023-10-06T03:07:17.527585+0000","last_active":"2023-10-06T03:07:17.527585+0000","last_peered":"2023-10-06T03:07:17.527585+0000","last_clean":"2023-10-06T03:07:17.527585+0000","last_became_active":"2023-10-06T03:07:17.526751+0000","last_became_peered":"2023-10-06T03:07:17.526751+0000","last_unstale":"2023-10-06T03:07:17.527585+0000","last_undegraded":"2023-10-06T03:07:17.527585+0000","last_fullsized":"2023-10-06T03:07:17.527585+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:13.406743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"89'2","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.031208+0000","last_change":"2023-10-06T03:07:17.013091+0000","last_active":"2023-10-06T03:07:18.031208+0000","last_peered":"2023-10-06T03:07:18.031208+0000","last_clean":"2023-10-06T03:07:18.031208+0000","last_became_active":"2023-10-06T03:07:17.012742+0000","last_became_peered":"2023-10-06T03:07:17.012742+0000","last_unstale":"2023-10-06T03:07:18.031208+0000","last_undegraded":"2023-10-06T03:07:18.031208+0000","last_fullsized":"2023-10-06T03:07:18.031208+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:59.119110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531358+0000","last_change":"2023-10-06T03:07:17.526556+0000","last_active":"2023-10-06T03:07:17.531358+0000","last_peered":"2023-10-06T03:07:17.531358+0000","last_clean":"2023-10-06T03:07:17.531358+0000","last_became_active":"2023-10-06T03:07:17.526238+0000","last_became_peered":"2023-10-06T03:07:17.526238+0000","last_unstale":"2023-10-06T03:07:17.531358+0000","last_undegraded":"2023-10-06T03:07:17.531358+0000","last_fullsized":"2023-10-06T03:07:17.531358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:40.834864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.016530+0000","last_change":"2023-10-06T03:07:17.015654+0000","last_active":"2023-10-06T03:07:17.016530+0000","last_peered":"2023-10-06T03:07:17.016530+0000","last_clean":"2023-10-06T03:07:17.016530+0000","last_became_active":"2023-10-06T03:07:17.015404+0000","last_became_peered":"2023-10-06T03:07:17.015404+0000","last_unstale":"2023-10-06T03:07:17.016530+0000","last_undegraded":"2023-10-06T03:07:17.016530+0000","last_fullsized":"2023-10-06T03:07:17.016530+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:08.225554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535154+0000","last_change":"2023-10-06T03:07:17.532464+0000","last_active":"2023-10-06T03:07:17.535154+0000","last_peered":"2023-10-06T03:07:17.535154+0000","last_clean":"2023-10-06T03:07:17.535154+0000","last_became_active":"2023-10-06T03:07:17.532159+0000","last_became_peered":"2023-10-06T03:07:17.532159+0000","last_unstale":"2023-10-06T03:07:17.535154+0000","last_undegraded":"2023-10-06T03:07:17.535154+0000","last_fullsized":"2023-10-06T03:07:17.535154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:06.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531720+0000","last_change":"2023-10-06T03:07:17.013693+0000","last_active":"2023-10-06T03:07:17.531720+0000","last_peered":"2023-10-06T03:07:17.531720+0000","last_clean":"2023-10-06T03:07:17.531720+0000","last_became_active":"2023-10-06T03:07:17.013475+0000","last_became_peered":"2023-10-06T03:07:17.013475+0000","last_unstale":"2023-10-06T03:07:17.531720+0000","last_undegraded":"2023-10-06T03:07:17.531720+0000","last_fullsized":"2023-10-06T03:07:17.531720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:05.969741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535802+0000","last_change":"2023-10-06T03:07:17.531933+0000","last_active":"2023-10-06T03:07:17.535802+0000","last_peered":"2023-10-06T03:07:17.535802+0000","last_clean":"2023-10-06T03:07:17.535802+0000","last_became_active":"2023-10-06T03:07:17.530965+0000","last_became_peered":"2023-10-06T03:07:17.530965+0000","last_unstale":"2023-10-06T03:07:17.535802+0000","last_undegraded":"2023-10-06T03:07:17.535802+0000","last_fullsized":"2023-10-06T03:07:17.535802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:06.186387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.518778+0000","last_change":"2023-10-06T03:07:17.014448+0000","last_active":"2023-10-06T03:07:17.518778+0000","last_peered":"2023-10-06T03:07:17.518778+0000","last_clean":"2023-10-06T03:07:17.518778+0000","last_became_active":"2023-10-06T03:07:17.014192+0000","last_became_peered":"2023-10-06T03:07:17.014192+0000","last_unstale":"2023-10-06T03:07:17.518778+0000","last_undegraded":"2023-10-06T03:07:17.518778+0000","last_fullsized":"2023-10-06T03:07:17.518778+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:03.041485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.530926+0000","last_change":"2023-10-06T03:07:17.530926+0000","last_active":"2023-10-06T03:07:17.530926+0000","last_peered":"2023-10-06T03:07:17.530926+0000","last_clean":"2023-10-06T03:07:17.530926+0000","last_became_active":"2023-10-06T03:07:17.530650+0000","last_became_peered":"2023-10-06T03:07:17.530650+0000","last_unstale":"2023-10-06T03:07:17.530926+0000","last_undegraded":"2023-10-06T03:07:17.530926+0000","last_fullsized":"2023-10-06T03:07:17.530926+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:00.690209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.519233+0000","last_change":"2023-10-06T03:07:17.014863+0000","last_active":"2023-10-06T03:07:17.519233+0000","last_peered":"2023-10-06T03:07:17.519233+0000","last_clean":"2023-10-06T03:07:17.519233+0000","last_became_active":"2023-10-06T03:07:17.014617+0000","last_became_peered":"2023-10-06T03:07:17.014617+0000","last_unstale":"2023-10-06T03:07:17.519233+0000","last_undegraded":"2023-10-06T03:07:17.519233+0000","last_fullsized":"2023-10-06T03:07:17.519233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:19.470247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531484+0000","last_change":"2023-10-06T03:07:17.531484+0000","last_active":"2023-10-06T03:07:17.531484+0000","last_peered":"2023-10-06T03:07:17.531484+0000","last_clean":"2023-10-06T03:07:17.531484+0000","last_became_active":"2023-10-06T03:07:17.530964+0000","last_became_peered":"2023-10-06T03:07:17.530964+0000","last_unstale":"2023-10-06T03:07:17.531484+0000","last_undegraded":"2023-10-06T03:07:17.531484+0000","last_fullsized":"2023-10-06T03:07:17.531484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:51.502108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015878+0000","last_change":"2023-10-06T03:07:17.008960+0000","last_active":"2023-10-06T03:07:17.015878+0000","last_peered":"2023-10-06T03:07:17.015878+0000","last_clean":"2023-10-06T03:07:17.015878+0000","last_became_active":"2023-10-06T03:07:17.008708+0000","last_became_peered":"2023-10-06T03:07:17.008708+0000","last_unstale":"2023-10-06T03:07:17.015878+0000","last_undegraded":"2023-10-06T03:07:17.015878+0000","last_fullsized":"2023-10-06T03:07:17.015878+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:58.011847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.535039+0000","last_change":"2023-10-06T03:07:17.535039+0000","last_active":"2023-10-06T03:07:17.535039+0000","last_peered":"2023-10-06T03:07:17.535039+0000","last_clean":"2023-10-06T03:07:17.535039+0000","last_became_active":"2023-10-06T03:07:17.534772+0000","last_became_peered":"2023-10-06T03:07:17.534772+0000","last_unstale":"2023-10-06T03:07:17.535039+0000","last_undegraded":"2023-10-06T03:07:17.535039+0000","last_fullsized":"2023-10-06T03:07:17.535039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:46.186781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531086+0000","last_change":"2023-10-06T03:07:17.017624+0000","last_active":"2023-10-06T03:07:17.531086+0000","last_peered":"2023-10-06T03:07:17.531086+0000","last_clean":"2023-10-06T03:07:17.531086+0000","last_became_active":"2023-10-06T03:07:17.017381+0000","last_became_peered":"2023-10-06T03:07:17.017381+0000","last_unstale":"2023-10-06T03:07:17.531086+0000","last_undegraded":"2023-10-06T03:07:17.531086+0000","last_fullsized":"2023-10-06T03:07:17.531086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:03.041485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.541524+0000","last_change":"2023-10-06T03:07:17.535890+0000","last_active":"2023-10-06T03:07:17.541524+0000","last_peered":"2023-10-06T03:07:17.541524+0000","last_clean":"2023-10-06T03:07:17.541524+0000","last_became_active":"2023-10-06T03:07:17.535580+0000","last_became_peered":"2023-10-06T03:07:17.535580+0000","last_unstale":"2023-10-06T03:07:17.541524+0000","last_undegraded":"2023-10-06T03:07:17.541524+0000","last_fullsized":"2023-10-06T03:07:17.541524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:28.389638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.012466+0000","last_change":"2023-10-06T03:07:17.007302+0000","last_active":"2023-10-06T03:07:17.012466+0000","last_peered":"2023-10-06T03:07:17.012466+0000","last_clean":"2023-10-06T03:07:17.012466+0000","last_became_active":"2023-10-06T03:07:17.007042+0000","last_became_peered":"2023-10-06T03:07:17.007042+0000","last_unstale":"2023-10-06T03:07:17.012466+0000","last_undegraded":"2023-10-06T03:07:17.012466+0000","last_fullsized":"2023-10-06T03:07:17.012466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:18:29.457148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.528032+0000","last_change":"2023-10-06T03:07:17.528032+0000","last_active":"2023-10-06T03:07:17.528032+0000","last_peered":"2023-10-06T03:07:17.528032+0000","last_clean":"2023-10-06T03:07:17.528032+0000","last_became_active":"2023-10-06T03:07:17.527783+0000","last_became_peered":"2023-10-06T03:07:17.527783+0000","last_unstale":"2023-10-06T03:07:17.528032+0000","last_undegraded":"2023-10-06T03:07:17.528032+0000","last_fullsized":"2023-10-06T03:07:17.528032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:13.413172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015954+0000","last_change":"2023-10-06T03:07:17.010923+0000","last_active":"2023-10-06T03:07:17.015954+0000","last_peered":"2023-10-06T03:07:17.015954+0000","last_clean":"2023-10-06T03:07:17.015954+0000","last_became_active":"2023-10-06T03:07:17.010649+0000","last_became_peered":"2023-10-06T03:07:17.010649+0000","last_unstale":"2023-10-06T03:07:17.015954+0000","last_undegraded":"2023-10-06T03:07:17.015954+0000","last_fullsized":"2023-10-06T03:07:17.015954+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:31.795198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531982+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:17.531982+0000","last_peered":"2023-10-06T03:07:17.531982+0000","last_clean":"2023-10-06T03:07:17.531982+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:17.531982+0000","last_undegraded":"2023-10-06T03:07:17.531982+0000","last_fullsized":"2023-10-06T03:07:17.531982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527116+0000","last_change":"2023-10-06T03:07:17.527116+0000","last_active":"2023-10-06T03:07:17.527116+0000","last_peered":"2023-10-06T03:07:17.527116+0000","last_clean":"2023-10-06T03:07:17.527116+0000","last_became_active":"2023-10-06T03:07:17.526865+0000","last_became_peered":"2023-10-06T03:07:17.526865+0000","last_unstale":"2023-10-06T03:07:17.527116+0000","last_undegraded":"2023-10-06T03:07:17.527116+0000","last_fullsized":"2023-10-06T03:07:17.527116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:41.234630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.030658+0000","last_change":"2023-10-06T03:07:17.008320+0000","last_active":"2023-10-06T03:07:18.030658+0000","last_peered":"2023-10-06T03:07:18.030658+0000","last_clean":"2023-10-06T03:07:18.030658+0000","last_became_active":"2023-10-06T03:07:17.008070+0000","last_became_peered":"2023-10-06T03:07:17.008070+0000","last_unstale":"2023-10-06T03:07:18.030658+0000","last_undegraded":"2023-10-06T03:07:18.030658+0000","last_fullsized":"2023-10-06T03:07:18.030658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:09.135222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.518994+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:17.518994+0000","last_peered":"2023-10-06T03:07:17.518994+0000","last_clean":"2023-10-06T03:07:17.518994+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:17.518994+0000","last_undegraded":"2023-10-06T03:07:17.518994+0000","last_fullsized":"2023-10-06T03:07:17.518994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531310+0000","last_change":"2023-10-06T03:07:17.531310+0000","last_active":"2023-10-06T03:07:17.531310+0000","last_peered":"2023-10-06T03:07:17.531310+0000","last_clean":"2023-10-06T03:07:17.531310+0000","last_became_active":"2023-10-06T03:07:17.531078+0000","last_became_peered":"2023-10-06T03:07:17.531078+0000","last_unstale":"2023-10-06T03:07:17.531310+0000","last_undegraded":"2023-10-06T03:07:17.531310+0000","last_fullsized":"2023-10-06T03:07:17.531310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:46:55.344710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.015925+0000","last_change":"2023-10-06T03:07:17.010164+0000","last_active":"2023-10-06T03:07:17.015925+0000","last_peered":"2023-10-06T03:07:17.015925+0000","last_clean":"2023-10-06T03:07:17.015925+0000","last_became_active":"2023-10-06T03:07:17.009912+0000","last_became_peered":"2023-10-06T03:07:17.009912+0000","last_unstale":"2023-10-06T03:07:17.015925+0000","last_undegraded":"2023-10-06T03:07:17.015925+0000","last_fullsized":"2023-10-06T03:07:17.015925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:40.035439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.004554+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:17.004554+0000","last_peered":"2023-10-06T03:07:17.004554+0000","last_clean":"2023-10-06T03:07:17.004554+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:17.004554+0000","last_undegraded":"2023-10-06T03:07:17.004554+0000","last_fullsized":"2023-10-06T03:07:17.004554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539039+0000","last_change":"2023-10-06T03:07:17.534614+0000","last_active":"2023-10-06T03:07:17.539039+0000","last_peered":"2023-10-06T03:07:17.539039+0000","last_clean":"2023-10-06T03:07:17.539039+0000","last_became_active":"2023-10-06T03:07:17.534330+0000","last_became_peered":"2023-10-06T03:07:17.534330+0000","last_unstale":"2023-10-06T03:07:17.539039+0000","last_undegraded":"2023-10-06T03:07:17.539039+0000","last_fullsized":"2023-10-06T03:07:17.539039+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:45:50.035401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"89'1","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527993+0000","last_change":"2023-10-06T03:07:17.013358+0000","last_active":"2023-10-06T03:07:17.527993+0000","last_peered":"2023-10-06T03:07:17.527993+0000","last_clean":"2023-10-06T03:07:17.527993+0000","last_became_active":"2023-10-06T03:07:17.013004+0000","last_became_peered":"2023-10-06T03:07:17.013004+0000","last_unstale":"2023-10-06T03:07:17.527993+0000","last_undegraded":"2023-10-06T03:07:17.527993+0000","last_fullsized":"2023-10-06T03:07:17.527993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:28.793331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.002230+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:17.002230+0000","last_peered":"2023-10-06T03:07:17.002230+0000","last_clean":"2023-10-06T03:07:17.002230+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:17.002230+0000","last_undegraded":"2023-10-06T03:07:17.002230+0000","last_fullsized":"2023-10-06T03:07:17.002230+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531989+0000","last_change":"2023-10-06T03:07:17.525296+0000","last_active":"2023-10-06T03:07:17.531989+0000","last_peered":"2023-10-06T03:07:17.531989+0000","last_clean":"2023-10-06T03:07:17.531989+0000","last_became_active":"2023-10-06T03:07:17.524893+0000","last_became_peered":"2023-10-06T03:07:17.524893+0000","last_unstale":"2023-10-06T03:07:17.531989+0000","last_undegraded":"2023-10-06T03:07:17.531989+0000","last_fullsized":"2023-10-06T03:07:17.531989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:41.311041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"89'2","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.030636+0000","last_change":"2023-10-06T03:07:17.010071+0000","last_active":"2023-10-06T03:07:18.030636+0000","last_peered":"2023-10-06T03:07:18.030636+0000","last_clean":"2023-10-06T03:07:18.030636+0000","last_became_active":"2023-10-06T03:07:17.009831+0000","last_became_peered":"2023-10-06T03:07:17.009831+0000","last_unstale":"2023-10-06T03:07:18.030636+0000","last_undegraded":"2023-10-06T03:07:18.030636+0000","last_fullsized":"2023-10-06T03:07:18.030636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:40.175153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":223,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.002301+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:17.002301+0000","last_peered":"2023-10-06T03:07:17.002301+0000","last_clean":"2023-10-06T03:07:17.002301+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:17.002301+0000","last_undegraded":"2023-10-06T03:07:17.002301+0000","last_fullsized":"2023-10-06T03:07:17.002301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.521853+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:17.521853+0000","last_peered":"2023-10-06T03:07:17.521853+0000","last_clean":"2023-10-06T03:07:17.521853+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:17.521853+0000","last_undegraded":"2023-10-06T03:07:17.521853+0000","last_fullsized":"2023-10-06T03:07:17.521853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539724+0000","last_change":"2023-10-06T03:07:17.535951+0000","last_active":"2023-10-06T03:07:17.539724+0000","last_peered":"2023-10-06T03:07:17.539724+0000","last_clean":"2023-10-06T03:07:17.539724+0000","last_became_active":"2023-10-06T03:07:17.535676+0000","last_became_peered":"2023-10-06T03:07:17.535676+0000","last_unstale":"2023-10-06T03:07:17.539724+0000","last_undegraded":"2023-10-06T03:07:17.539724+0000","last_fullsized":"2023-10-06T03:07:17.539724+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:50.062334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.521669+0000","last_change":"2023-10-06T03:07:17.007812+0000","last_active":"2023-10-06T03:07:17.521669+0000","last_peered":"2023-10-06T03:07:17.521669+0000","last_clean":"2023-10-06T03:07:17.521669+0000","last_became_active":"2023-10-06T03:07:17.007525+0000","last_became_peered":"2023-10-06T03:07:17.007525+0000","last_unstale":"2023-10-06T03:07:17.521669+0000","last_undegraded":"2023-10-06T03:07:17.521669+0000","last_fullsized":"2023-10-06T03:07:17.521669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:49.738171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-06T03:07:16.996796+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:16.996796+0000","last_peered":"2023-10-06T03:07:16.996796+0000","last_clean":"2023-10-06T03:07:16.996796+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:16.996796+0000","last_undegraded":"2023-10-06T03:07:16.996796+0000","last_fullsized":"2023-10-06T03:07:16.996796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.527079+0000","last_change":"2023-10-06T03:07:17.527079+0000","last_active":"2023-10-06T03:07:17.527079+0000","last_peered":"2023-10-06T03:07:17.527079+0000","last_clean":"2023-10-06T03:07:17.527079+0000","last_became_active":"2023-10-06T03:07:17.526821+0000","last_became_peered":"2023-10-06T03:07:17.526821+0000","last_unstale":"2023-10-06T03:07:17.527079+0000","last_undegraded":"2023-10-06T03:07:17.527079+0000","last_fullsized":"2023-10-06T03:07:17.527079+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:22.020130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"89'1","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:18.028238+0000","last_change":"2023-10-06T03:07:17.016948+0000","last_active":"2023-10-06T03:07:18.028238+0000","last_peered":"2023-10-06T03:07:18.028238+0000","last_clean":"2023-10-06T03:07:18.028238+0000","last_became_active":"2023-10-06T03:07:17.016651+0000","last_became_peered":"2023-10-06T03:07:17.016651+0000","last_unstale":"2023-10-06T03:07:18.028238+0000","last_undegraded":"2023-10-06T03:07:18.028238+0000","last_fullsized":"2023-10-06T03:07:18.028238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:21.015470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.520783+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:17.520783+0000","last_peered":"2023-10-06T03:07:17.520783+0000","last_clean":"2023-10-06T03:07:17.520783+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:17.520783+0000","last_undegraded":"2023-10-06T03:07:17.520783+0000","last_fullsized":"2023-10-06T03:07:17.520783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.539324+0000","last_change":"2023-10-06T03:07:17.535352+0000","last_active":"2023-10-06T03:07:17.539324+0000","last_peered":"2023-10-06T03:07:17.539324+0000","last_clean":"2023-10-06T03:07:17.539324+0000","last_became_active":"2023-10-06T03:07:17.535096+0000","last_became_peered":"2023-10-06T03:07:17.535096+0000","last_unstale":"2023-10-06T03:07:17.539324+0000","last_undegraded":"2023-10-06T03:07:17.539324+0000","last_fullsized":"2023-10-06T03:07:17.539324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:16.990493+0000","last_clean_scrub_stamp":"2023-10-06T03:07:16.990493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:42:56.202893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.531140+0000","last_change":"2023-10-06T03:07:17.017093+0000","last_active":"2023-10-06T03:07:17.531140+0000","last_peered":"2023-10-06T03:07:17.531140+0000","last_clean":"2023-10-06T03:07:17.531140+0000","last_became_active":"2023-10-06T03:07:17.016695+0000","last_became_peered":"2023-10-06T03:07:17.016695+0000","last_unstale":"2023-10-06T03:07:17.531140+0000","last_undegraded":"2023-10-06T03:07:17.531140+0000","last_fullsized":"2023-10-06T03:07:17.531140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:15.985833+0000","last_clean_scrub_stamp":"2023-10-06T03:07:15.985833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:40:59.119110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-06T03:07:17.518856+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:17.518856+0000","last_peered":"2023-10-06T03:07:17.518856+0000","last_clean":"2023-10-06T03:07:17.518856+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:17.518856+0000","last_undegraded":"2023-10-06T03:07:17.518856+0000","last_fullsized":"2023-10-06T03:07:17.518856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1090160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55500000000000005,"15min":0.72799999999999998},"min":{"1min":0.34200000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76200000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.505,"15min":0.51500000000000001},"min":{"1min":0.38,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.501,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71599999999999997,"5min":0.875,"15min":0.875},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81299999999999994,"5min":0.999,"15min":0.999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.52800000000000002},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72199999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.755,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67600000000000005,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.78800000000000003,"5min":1.026,"15min":1.026},"last":0.503}]},{"osd":5,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56599999999999995,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75800000000000001,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73199999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.58799999999999997},"min":{"1min":0.36099999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298396,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073444,"statfs":{"total":96636764160,"available":96331206656,"internally_reserved":0,"allocated":1134592,"data_stored":508385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.751,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.434}]},{"osd":2,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.40100000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69499999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.33100000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72699999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82799999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.626},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51200000000000001,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52400000000000002,"15min":0.55100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.59099999999999997},"min":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57299999999999995,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77000000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79900000000000004,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":499666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.215,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53500000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.25},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.372},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55900000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.307}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32300000000000001,"15min":0.33100000000000002},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.48599999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54200000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.23300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.50800000000000001,"5min":0.628,"15min":0.628},"last":0.436},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.29399999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71999999999999997,"5min":1.115,"15min":1.115},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.55900000000000005},"min":{"1min":0.40200000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54100000000000004,"15min":0.56699999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84699999999999998,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.41599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55600000000000005,"15min":0.57399999999999995},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.435,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82699999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.44700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574921,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1712128,"data_stored":1091626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30199999999999999,"5min":0.307,"15min":0.31},"min":{"1min":0.219,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.30499999999999999,"5min":0.31900000000000001,"15min":0.32700000000000001},"min":{"1min":0.222,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.40300000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.33400000000000002,"15min":0.33100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.48099999999999998,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.26600000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.433,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.33300000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.33400000000000002,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.41999999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.372,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.48499999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.63500000000000001,"5min":0.871,"15min":0.871},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.60499999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68000000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.59599999999999997},"min":{"1min":0.42199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.754,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58899999999999997,"15min":0.61199999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.74299999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71099999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76300000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.68899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574921,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":499666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.31,"15min":0.32200000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.498,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.32500000000000001,"15min":0.315},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.219}]},{"osd":1,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.307},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.34599999999999997,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.504,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.29099999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32700000000000001,"15min":0.32100000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.48499999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.371,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.52300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.68000000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.53400000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.51400000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.54000000000000004},"min":{"1min":0.42599999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.82399999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77400000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.40699999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.87,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.59599999999999997,"15min":0.62},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83499999999999996,"5min":1.903,"15min":1.903},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.54500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.748,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59399999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":500186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.31,"15min":0.30499999999999999},"min":{"1min":0.215,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.503,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.25600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.49299999999999999,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.317}]},{"osd":1,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.317,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.248},{"interface":"front","average":{"1min":0.314,"5min":0.33300000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.249,"15min":0.249},"max":{"1min":0.59599999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36199999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.53800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.308},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.35699999999999998},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33900000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.41199999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.625,"15min":0.58499999999999996},"min":{"1min":0.38900000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.77000000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.997,"15min":0.997},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73799999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.441},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.40300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79600000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.56100000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78900000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.501}]}]},{"osd":4,"up_from":13,"seq":55834574921,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302480,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069360,"statfs":{"total":96636764160,"available":96327024640,"internally_reserved":0,"allocated":1122304,"data_stored":506365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52700000000000002,"15min":0.499},"min":{"1min":0.32500000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67500000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52400000000000002,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64300000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.622}]},{"osd":2,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54800000000000004,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70799999999999996,"5min":3,"15min":3},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.60999999999999999,"15min":0.81799999999999995},"min":{"1min":0.34999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69199999999999995,"5min":10.084,"15min":10.084},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54600000000000004,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59199999999999997,"15min":0.69899999999999995},"min":{"1min":0.36299999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77600000000000002,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.51700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72199999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53800000000000003,"15min":0.58999999999999997},"min":{"1min":0.30499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72999999999999998,"5min":1.54,"15min":1.54},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53000000000000003,"15min":0.54600000000000004},"min":{"1min":0.40999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71099999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.57399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.35399999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.59199999999999997},"min":{"1min":0.34699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76800000000000002,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.61599999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.40699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298364,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073476,"statfs":{"total":96636764160,"available":96331239424,"internally_reserved":0,"allocated":1101824,"data_stored":499632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75600000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.505},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56899999999999995,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.76600000000000001,"5min":2.371,"15min":2.371},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.505,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79200000000000004,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.35899999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.69799999999999995,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.59299999999999997},"min":{"1min":0.35099999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78200000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59099999999999997,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67700000000000005,"5min":5.3869999999999996,"15min":5.3869999999999996},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.57999999999999996,"15min":0.72599999999999998},"min":{"1min":0.373,"5min":0.315,"15min":0.315},"max":{"1min":0.72099999999999997,"5min":5.4630000000000001,"15min":5.4630000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.68400000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.437}]},{"osd":4,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.59199999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.72999999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60399999999999998,"15min":0.63400000000000001},"min":{"1min":0.45400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.746,"5min":1.052,"15min":1.052},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:06:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.63700000000000001},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86799999999999999,"5min":1.175,"15min":1.175},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57899999999999996,"15min":0.61899999999999999},"min":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73299999999999998,"5min":1.145,"15min":1.145},"last":0.49199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:22.451 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:07:22.451 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:07:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.604+0000 7f56430fc700 1 Processor -- start 2023-10-06T03:07:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.604+0000 7f56430fc700 1 -- start start 2023-10-06T03:07:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f56430fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f563c159950 0x7f563c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f56430fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f56430fc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0ce380 con 0x7f563c069aa0 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f56430fc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c069f20 con 0x7f563c159950 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f56430fc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0cec20 con 0x7f563c0af320 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f5640e98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f563c159950 0x7f563c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f5640e98700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f563c159950 0x7f563c0cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37000/0 (socket says 172.21.15.92:37000) 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f5640e98700 1 -- 172.21.15.92:0/1421054242 learned_addr learned my addr 172.21.15.92:0/1421054242 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.605+0000 7f5633fff700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f5640e98700 1 -- 172.21.15.92:0/1421054242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 msgr2=0x7f563c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f5640e98700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f5640e98700 1 -- 172.21.15.92:0/1421054242 >> v1:172.21.15.136:6789/0 conn(0x7f563c069aa0 legacy=0x7f563c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f5640e98700 1 -- 172.21.15.92:0/1421054242 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0afa90 con 0x7f563c159950 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f5633fff700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f5640e98700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0cbb90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5638009f20 tx=0x7f5638009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6773cbc1a8be9ccb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56337fe700 1 -- 172.21.15.92:0/1421054242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800e070 con 0x7f563c159950 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56337fe700 1 -- 172.21.15.92:0/1421054242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563800ba80 con 0x7f563c159950 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56337fe700 1 -- 172.21.15.92:0/1421054242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56380092e0 con 0x7f563c159950 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 -- 172.21.15.92:0/1421054242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 msgr2=0x7f563c0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0cbb90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5638009f20 tx=0x7f5638009840 comp rx=0 tx=0).stop 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 -- 172.21.15.92:0/1421054242 shutdown_connections 2023-10-06T03:07:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0cbb90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 --2- 172.21.15.92:0/1421054242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 -- 172.21.15.92:0/1421054242 >> 172.21.15.92:0/1421054242 conn(0x7f563c068e00 msgr2=0x7f563c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 -- 172.21.15.92:0/1421054242 shutdown_connections 2023-10-06T03:07:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.606+0000 7f56430fc700 1 -- 172.21.15.92:0/1421054242 wait complete. 2023-10-06T03:07:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 Processor -- start 2023-10-06T03:07:22.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 -- start start 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f563c069aa0 0x7f563c150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1516f0 con 0x7f563c159950 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1519d0 con 0x7f563c069aa0 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.608+0000 7f56430fc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c151cb0 con 0x7f563c0af320 2023-10-06T03:07:22.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5640e98700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f563c069aa0 0x7f563c150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37016/0 (socket says 172.21.15.92:37016) 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 -- 172.21.15.92:0/3923400007 learned_addr learned my addr 172.21.15.92:0/3923400007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5633fff700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 -- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 msgr2=0x7f563c0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c0b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 -- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f563c069aa0 msgr2=0x7f563c150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f563c069aa0 0x7f563c150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 -- 172.21.15.92:0/3923400007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638009ae0 con 0x7f563c159950 2023-10-06T03:07:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5641699700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0b6070 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f563400c560 tx=0x7f5634007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634011590 con 0x7f563c159950 2023-10-06T03:07:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56340116f0 con 0x7f563c159950 2023-10-06T03:07:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c151f90 con 0x7f563c159950 2023-10-06T03:07:22.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634011ab0 con 0x7f563c159950 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.609+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c151320 con 0x7f563c159950 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.610+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c1524b0 con 0x7f563c159950 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.611+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f563401b3b0 con 0x7f563c159950 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.611+0000 7f5630ff9700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5628067200 0x7f56280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.611+0000 7f5640e98700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5628067200 0x7f56280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.611+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5634083ea0 con 0x7f563c159950 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.611+0000 7f5640e98700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5628067200 0x7f56280696b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5638009880 tx=0x7f5638018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:22.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.615+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f563c1524b0 con 0x7f563c159950 2023-10-06T03:07:22.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.872+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f563c1524b0 con 0x7f563c159950 2023-10-06T03:07:22.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.876+0000 7f5630ff9700 1 -- 172.21.15.92:0/3923400007 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f563c1524b0 con 0x7f563c159950 2023-10-06T03:07:22.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.877+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5628067200 msgr2=0x7f56280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.877+0000 7f56430fc700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5628067200 0x7f56280696b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5638009880 tx=0x7f5638018000 comp rx=0 tx=0).stop 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.877+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 msgr2=0x7f563c0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0b6070 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f563400c560 tx=0x7f5634007ef0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 shutdown_connections 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5628067200 0x7f56280696b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f563c159950 0x7f563c0b6070 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f563c069aa0 0x7f563c150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 --2- 172.21.15.92:0/3923400007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f563c0af320 0x7f563c0b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 >> 172.21.15.92:0/3923400007 conn(0x7f563c068e00 msgr2=0x7f563c0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:22.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 shutdown_connections 2023-10-06T03:07:22.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:22.878+0000 7f56430fc700 1 -- 172.21.15.92:0/3923400007 wait complete. 2023-10-06T03:07:22.881 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:07:22.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.004+0000 7f2914a82700 1 Processor -- start 2023-10-06T03:07:23.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.004+0000 7f2914a82700 1 -- start start 2023-10-06T03:07:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f2914a82700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2910158d30 0x7f291006d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f2914a82700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100acc80 con 0x7f291006de60 2023-10-06T03:07:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f2914a82700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291006e270 con 0x7f2910155070 2023-10-06T03:07:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f2914a82700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291006bee0 con 0x7f2910158d30 2023-10-06T03:07:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f290ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f291006de60 0x7f29100ae560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46914/0 (socket says 172.21.15.92:46914) 2023-10-06T03:07:23.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f290ed9d700 1 -- 172.21.15.92:0/4008069167 learned_addr learned my addr 172.21.15.92:0/4008069167 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f2905d9b700 1 --2- 172.21.15.92:0/4008069167 >> v2:172.21.15.136:3300/0 conn(0x7f2910158d30 0x7f291006d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.005+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029396641 0 0) 0x7f29100acc80 con 0x7f291006de60 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28fc003570 con 0x7f291006de60 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768402688 0 0) 0x7f291006e270 con 0x7f2910155070 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29100acc80 con 0x7f2910155070 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872843074 0 0) 0x7f28fc003570 con 0x7f291006de60 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 >> v2:172.21.15.136:3300/0 conn(0x7f2910158d30 msgr2=0x7f291006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 --2- 172.21.15.92:0/4008069167 >> v2:172.21.15.136:3300/0 conn(0x7f2910158d30 0x7f291006d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 >> v1:172.21.15.136:6789/0 conn(0x7f2910155070 legacy=0x7f29101585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100ac070 con 0x7f291006de60 2023-10-06T03:07:23.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f29080017f0 con 0x7f291006de60 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 >> v1:172.21.15.92:6789/0 conn(0x7f291006de60 legacy=0x7f29100ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc003bf0 0x7f28fc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28fc004810 0x7f28fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28fc0051c0 0x7f28fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291006bee0 con 0x7f28fc0051c0 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc00c0b0 con 0x7f28fc004810 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc00c270 con 0x7f28fc003bf0 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28fc0051c0 0x7f28fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290e59c700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc003bf0 0x7f28fc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f2905d9b700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28fc004810 0x7f28fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 -- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc003bf0 msgr2=0x7f28fc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc003bf0 0x7f28fc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 -- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28fc004810 msgr2=0x7f28fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28fc004810 0x7f28fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 -- 172.21.15.92:0/4008069167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc003570 con 0x7f28fc0051c0 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f2905d9b700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28fc004810 0x7f28fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 -- 172.21.15.92:0/4008069167 reap_dead start 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.006+0000 7f290ed9d700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28fc0051c0 0x7f28fc00ba60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f290800a8f0 tx=0x7f29100acbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04c97d087d74bfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080070d0 con 0x7f28fc0051c0 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908007270 con 0x7f28fc0051c0 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f290dd9b700 1 -- 172.21.15.92:0/4008069167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908007630 con 0x7f28fc0051c0 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 -- 172.21.15.92:0/4008069167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28fc0051c0 msgr2=0x7f28fc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28fc0051c0 0x7f28fc00ba60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f290800a8f0 tx=0x7f29100acbf0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 -- 172.21.15.92:0/4008069167 shutdown_connections 2023-10-06T03:07:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 --2- 172.21.15.92:0/4008069167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28fc0051c0 0x7f28fc00ba60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 -- 172.21.15.92:0/4008069167 >> 172.21.15.92:0/4008069167 conn(0x7f29100cc780 msgr2=0x7f29100ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 -- 172.21.15.92:0/4008069167 shutdown_connections 2023-10-06T03:07:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.007+0000 7f2914a82700 1 -- 172.21.15.92:0/4008069167 wait complete. 2023-10-06T03:07:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.008+0000 7f2914a82700 1 Processor -- start 2023-10-06T03:07:23.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.008+0000 7f2914a82700 1 -- start start 2023-10-06T03:07:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2914a82700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc0051c0 0x7f29101465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2914a82700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 0x7f2910143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2914a82700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2910144030 0x7f291015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2914a82700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101444d0 con 0x7f2910152e90 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2914a82700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29101447b0 con 0x7f2910144030 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2914a82700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2910144a90 con 0x7f28fc0051c0 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 0x7f2910143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 0x7f2910143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37038/0 (socket says 172.21.15.92:37038) 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 -- 172.21.15.92:0/46764003 learned_addr learned my addr 172.21.15.92:0/46764003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f290ed9d700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2910144030 0x7f291015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f290e59c700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc0051c0 0x7f29101465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 -- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc0051c0 msgr2=0x7f29101465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc0051c0 0x7f29101465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 -- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2910144030 msgr2=0x7f291015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2910144030 0x7f291015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.009+0000 7f2905d9b700 1 -- 172.21.15.92:0/46764003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2910144d70 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.010+0000 7f2905d9b700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 0x7f2910143af0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fd0 tx=0x7f28f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.010+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8015740 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.010+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f80158a0 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.010+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2910145050 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.010+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8015c40 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.010+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29100ab4b0 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.011+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291004c3c0 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.011+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f28f8013410 con 0x7f2910152e90 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.011+0000 7f2906ffd700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28f0067200 0x7f28f00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.012+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f28f8085240 con 0x7f2910152e90 2023-10-06T03:07:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.012+0000 7f290e59c700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28f0067200 0x7f28f00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.012+0000 7f290e59c700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28f0067200 0x7f28f00696b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2900000c00 tx=0x7f2900009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.016+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28f8052000 con 0x7f2910152e90 2023-10-06T03:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.242+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29100ac350 con 0x7f2910152e90 2023-10-06T03:07:23.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.242+0000 7f2906ffd700 1 -- 172.21.15.92:0/46764003 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f28f8055620 con 0x7f2910152e90 2023-10-06T03:07:23.244 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:23.245 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:17.381585+0000","modified":"2023-10-06T03:07:21.036747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6480":{"gid":6480,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/428935748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":428935748},{"type":"v1","addr":"172.21.15.92:6837","nonce":428935748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:07:23.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28f0067200 msgr2=0x7f28f00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28f0067200 0x7f28f00696b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2900000c00 tx=0x7f2900009040 comp rx=0 tx=0).stop 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 msgr2=0x7f2910143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 0x7f2910143af0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fd0 tx=0x7f28f8008040 comp rx=0 tx=0).stop 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 shutdown_connections 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28f0067200 0x7f28f00696b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2910144030 0x7f291015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28fc0051c0 0x7f29101465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 --2- 172.21.15.92:0/46764003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2910152e90 0x7f2910143af0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 >> 172.21.15.92:0/46764003 conn(0x7f29100cc780 msgr2=0x7f2910154150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 shutdown_connections 2023-10-06T03:07:23.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.245+0000 7f2914a82700 1 -- 172.21.15.92:0/46764003 wait complete. 2023-10-06T03:07:23.248 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 56 2023-10-06T03:07:23.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:07:17.381585+0000', 'modified': '2023-10-06T03:07:21.036747+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6480}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6480': {'gid': 6480, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/428935748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 428935748}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 428935748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:07:23.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:07:23.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:07:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.776+0000 7f4ac03a4700 1 Processor -- start 2023-10-06T03:07:23.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.776+0000 7f4ac03a4700 1 -- start start 2023-10-06T03:07:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.777+0000 7f4ac03a4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4ab8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.777+0000 7f4ac03a4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab815c2a0 con 0x7f4ab80c9510 2023-10-06T03:07:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.777+0000 7f4ac03a4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab815b690 con 0x7f4ab806c0a0 2023-10-06T03:07:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.777+0000 7f4ac03a4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815b970 con 0x7f4ab806bcd0 2023-10-06T03:07:23.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.777+0000 7f4abe941700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4ab80c9510 0x7f4ab80cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46926/0 (socket says 172.21.15.92:46926) 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.777+0000 7f4abe941700 1 -- 172.21.15.92:0/1832297031 learned_addr learned my addr 172.21.15.92:0/1832297031 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abe140700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4ab8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156642884 0 0) 0x7f4ab815c2a0 con 0x7f4ab80c9510 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa4003570 con 0x7f4ab80c9510 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724300965 0 0) 0x7f4ab815b690 con 0x7f4ab806c0a0 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab815c2a0 con 0x7f4ab806c0a0 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037602921 0 0) 0x7f4aa4003570 con 0x7f4ab80c9510 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 msgr2=0x7f4ab8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4ab8140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 >> v1:172.21.15.136:6789/0 conn(0x7f4ab806c0a0 legacy=0x7f4ab80c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab815bc50 con 0x7f4ab80c9510 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f4ab00017f0 con 0x7f4ab80c9510 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 >> v1:172.21.15.92:6789/0 conn(0x7f4ab80c9510 legacy=0x7f4ab80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4003c20 0x7f4aa4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4aa4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815b970 con 0x7f4aa4004810 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa400bbf0 con 0x7f4aa4003c20 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa400bdb0 con 0x7f4ab806bcd0 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd93f700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abe140700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4003c20 0x7f4aa4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abe941700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4aa4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd93f700 1 -- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 msgr2=0x7f4aa4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd93f700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4aa4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd93f700 1 -- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4003c20 msgr2=0x7f4aa4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd93f700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4aa4003c20 0x7f4aa4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abd93f700 1 -- 172.21.15.92:0/1832297031 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4003570 con 0x7f4aa4004810 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.778+0000 7f4abe941700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab806bcd0 0x7f4aa4005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4abd93f700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8001a10 tx=0x7f4aa8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b1c879c8fe3073 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800c5d0 con 0x7f4aa4004810 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa800c730 con 0x7f4aa4004810 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4abd13e700 1 -- 172.21.15.92:0/1832297031 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa800cad0 con 0x7f4aa4004810 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4ac03a4700 1 -- 172.21.15.92:0/1832297031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 msgr2=0x7f4aa4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/1832297031 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4aa4004c80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8001a10 tx=0x7f4aa8007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4abe941700 1 -- 172.21.15.92:0/1832297031 reap_dead start 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4ac03a4700 1 -- 172.21.15.92:0/1832297031 shutdown_connections 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4ac03a4700 1 -- 172.21.15.92:0/1832297031 >> 172.21.15.92:0/1832297031 conn(0x7f4ab80ce600 msgr2=0x7f4ab80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4ac03a4700 1 -- 172.21.15.92:0/1832297031 shutdown_connections 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.779+0000 7f4ac03a4700 1 -- 172.21.15.92:0/1832297031 wait complete. 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.780+0000 7f4ac03a4700 1 Processor -- start 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.780+0000 7f4ac03a4700 1 -- start start 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4ac03a4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4ab813db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4ac03a4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ab80c6280 0x7f4ab81575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4ac03a4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab80c7cc0 0x7f4ab80c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4ac03a4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab8158e30 con 0x7f4aa4004810 2023-10-06T03:07:23.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4ac03a4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab80c66f0 con 0x7f4ab80c6280 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4ac03a4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab813e150 con 0x7f4ab80c7cc0 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4abe140700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4ab813db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4abe140700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4ab813db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37066/0 (socket says 172.21.15.92:37066) 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4abe140700 1 -- 172.21.15.92:0/2049874385 learned_addr learned my addr 172.21.15.92:0/2049874385 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ab80c6280 0x7f4ab81575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.781+0000 7f4abe941700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab80c7cc0 0x7f4ab80c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4abe140700 1 -- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab80c7cc0 msgr2=0x7f4ab80c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4abe140700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab80c7cc0 0x7f4ab80c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4abe140700 1 -- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ab80c6280 msgr2=0x7f4ab81575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4abe140700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ab80c6280 0x7f4ab81575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4abe140700 1 -- 172.21.15.92:0/2049874385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab813e430 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4abe140700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4ab813db20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4011d40 tx=0x7f4ab40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4aae7fc700 1 -- 172.21.15.92:0/2049874385 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400fac0 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4aae7fc700 1 -- 172.21.15.92:0/2049874385 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab400fc20 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4aae7fc700 1 -- 172.21.15.92:0/2049874385 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4014440 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab813e710 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.782+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab80c86a0 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.783+0000 7f4aae7fc700 1 -- 172.21.15.92:0/2049874385 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4ab4014870 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.783+0000 7f4aae7fc700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a9c067200 0x7f4a9c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.784+0000 7f4aae7fc700 1 -- 172.21.15.92:0/2049874385 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ab4018030 con 0x7f4aa4004810 2023-10-06T03:07:23.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.784+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a9c067200 0x7f4a9c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.784+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a9c067200 0x7f4a9c0696b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c8ba0 tx=0x7f4aa8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.785+0000 7f4ac969cb80 1 Processor -- start 2023-10-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 -- start start 2023-10-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c57220 0x559455c575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c2a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c71130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455b81a70 con 0x559455c74af0 2023-10-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455b7f270 con 0x559455c2abb0 2023-10-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559455b7b060 con 0x559455c57220 2023-10-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4abd93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c2a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4abd93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c2a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37080/0 (socket says 172.21.15.92:37080) 2023-10-06T03:07:23.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.786+0000 7f4abd93f700 1 -- 172.21.15.92:0/2460898187 learned_addr learned my addr 172.21.15.92:0/2460898187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abe140700 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c57220 0x559455c575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abe941700 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c71130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abd93f700 1 -- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c57220 msgr2=0x559455c575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c57220 0x559455c575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abd93f700 1 -- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 msgr2=0x559455c71130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c71130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abd93f700 1 -- 172.21.15.92:0/2460898187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559455a15270 con 0x559455c74af0 2023-10-06T03:07:23.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c2a670 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c3c0 tx=0x7f4aa8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a2b7ad1907ff7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4a9b7fe700 1 -- 172.21.15.92:0/2460898187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80128b0 con 0x559455c74af0 2023-10-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4a9b7fe700 1 -- 172.21.15.92:0/2460898187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8012a10 con 0x559455c74af0 2023-10-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4a9b7fe700 1 -- 172.21.15.92:0/2460898187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8012b80 con 0x559455c74af0 2023-10-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2460898187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 msgr2=0x559455c2a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.787+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c2a670 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c3c0 tx=0x7f4aa8021040 comp rx=0 tx=0).stop 2023-10-06T03:07:23.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2460898187 shutdown_connections 2023-10-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c71130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c57220 0x559455c575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2460898187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c2a670 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2460898187 >> 172.21.15.92:0/2460898187 conn(0x559455c2c160 msgr2=0x559455c2c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2460898187 shutdown_connections 2023-10-06T03:07:23.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2460898187 wait complete. 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 -- start start 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.788+0000 7f4ac969cb80 1 Processor -- start 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455b26890 con 0x559455c57220 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455b41ea0 con 0x559455c2abb0 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x5594559fde30 con 0x559455c74af0 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4ac969cb80 10 client.0 _ll_register_callbacks cb 0x559455bede70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:07:23.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4abd93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4abd93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37082/0 (socket says 172.21.15.92:37082) 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4abd93f700 1 -- 172.21.15.92:0/4125863503 learned_addr learned my addr 172.21.15.92:0/4125863503 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4abe140700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.789+0000 7f4abd93f700 1 -- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 msgr2=0x559455c26120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abe941700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abd93f700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c26120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abd93f700 1 -- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 msgr2=0x559455c74150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abd93f700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c74150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abd93f700 1 -- 172.21.15.92:0/4125863503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa4003570 con 0x559455c57220 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abe140700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c74150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abe941700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c26120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:23.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abd93f700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c564a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c700 tx=0x7f4aa802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd4d04c9c1db79d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4a98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4a98ff9700 1 -- 172.21.15.92:0/4125863503 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa802b630 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4a98ff9700 1 -- 172.21.15.92:0/4125863503 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4aa8021860 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559455c1f940 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4a98ff9700 1 -- 172.21.15.92:0/4125863503 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4aa8021c30 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 10 client.6630 fetch_fsmap learned FSMap version 56 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x5594559b7fe0 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4a98ff9700 1 -- 172.21.15.92:0/4125863503 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559455c1f940 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4a98ff9700 1 -- 172.21.15.92:0/4125863503 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4aa8021d90 con 0x559455c57220 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 10 client.6630 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 10 client.6630 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:07:23.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 20 client.6630 populate_metadata read hostname 'smithi092' 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c57c30 0x559455c27320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 4 client.6630 mds_command: new command op to 6480 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 --> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559455bfc870 con 0x559455c57c30 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.790+0000 7f4abe140700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c57c30 0x559455c27320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4a98ff9700 10 client.6630 ms_handle_connect on v2:172.21.15.92:6835/428935748 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4abe140700 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c57c30 0x559455c27320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb630771d783ba8 server_cookie=67130ab489fdab0 in_seq=0 out_seq=0 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4a98ff9700 1 -- 172.21.15.92:0/4125863503 <== mds.0 v2:172.21.15.92:6835/428935748 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559455bfc870 con 0x559455c57c30 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4a98ff9700 10 client.6630 handle_command_reply: tid=0 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 client.6630 shutdown 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 msgr2=0x559455c564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c564a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800c700 tx=0x7f4aa802d070 comp rx=0 tx=0).stop 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 shutdown_connections 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c57c30 0x559455c27320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c26120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c74150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4a98ff9700 0 client.6630 ms_handle_reset on v2:172.21.15.92:6835/428935748 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/4125863503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c564a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 >> 172.21.15.92:0/4125863503 conn(0x559455c2c160 msgr2=0x559455c2b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 shutdown_connections 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.791+0000 7f4ac969cb80 1 -- 172.21.15.92:0/4125863503 wait complete. 2023-10-06T03:07:23.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.792+0000 7f4ac969cb80 20 client.6630 trim_cache size 0 max 16384 2023-10-06T03:07:23.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.808+0000 7f4ac969cb80 1 Processor -- start 2023-10-06T03:07:23.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.808+0000 7f4ac969cb80 1 -- start start 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c27320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c27860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455c72790 con 0x559455c57220 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455c748c0 con 0x559455c2abb0 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559455c0a2e0 con 0x559455c74af0 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c27320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c27320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37096/0 (socket says 172.21.15.92:37096) 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 -- 172.21.15.92:0/3161527280 learned_addr learned my addr 172.21.15.92:0/3161527280 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 -- 172.21.15.92:0/3161527280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 msgr2=0x559455c27860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c27860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 -- 172.21.15.92:0/3161527280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 msgr2=0x559455c705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 -- 172.21.15.92:0/3161527280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559455b49780 con 0x559455c57220 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.809+0000 7f4abd93f700 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c27320 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8012700 tx=0x7f4aa800c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405cf6bfad249a79 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4a9bfff700 1 -- 172.21.15.92:0/3161527280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8000c30 con 0x559455c57220 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4a9bfff700 1 -- 172.21.15.92:0/3161527280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8021080 con 0x559455c57220 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4a9bfff700 1 -- 172.21.15.92:0/3161527280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8021e60 con 0x559455c57220 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 -- 172.21.15.92:0/3161527280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 msgr2=0x559455c27320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c27320 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8012700 tx=0x7f4aa800c050 comp rx=0 tx=0).stop 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 -- 172.21.15.92:0/3161527280 shutdown_connections 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c74af0 0x559455c27860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c2abb0 0x559455c705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/3161527280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c57220 0x559455c27320 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 -- 172.21.15.92:0/3161527280 >> 172.21.15.92:0/3161527280 conn(0x559455c2c160 msgr2=0x559455c23280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 -- 172.21.15.92:0/3161527280 shutdown_connections 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.810+0000 7f4ac969cb80 1 -- 172.21.15.92:0/3161527280 wait complete. 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.811+0000 7f4ac969cb80 1 -- start start 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.811+0000 7f4ac969cb80 1 Processor -- start 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c2abb0 0x559455c6b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c57220 0x559455c6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c74ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455c280c0 con 0x559455c74af0 2023-10-06T03:07:23.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559455cdb7d0 con 0x559455c57220 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559455c70d20 con 0x559455c2abb0 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 10 client.0 _ll_register_callbacks cb 0x559455bede70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c74ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c74ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37104/0 (socket says 172.21.15.92:37104) 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 -- 172.21.15.92:0/2943143277 learned_addr learned my addr 172.21.15.92:0/2943143277 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abd93f700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c57220 0x559455c6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 -- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c2abb0 msgr2=0x559455c6b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe140700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c2abb0 0x559455c6b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c2abb0 0x559455c6b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 -- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c57220 msgr2=0x559455c6b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c57220 0x559455c6b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 -- 172.21.15.92:0/2943143277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594559b7fe0 con 0x559455c74af0 2023-10-06T03:07:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4abe941700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c74ec0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80ce190 tx=0x7f4ab0006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff45c574fe39cacd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4a98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:07:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4a98ff9700 1 -- 172.21.15.92:0/2943143277 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0007b40 con 0x559455c74af0 2023-10-06T03:07:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4a98ff9700 1 -- 172.21.15.92:0/2943143277 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ab0012900 con 0x559455c74af0 2023-10-06T03:07:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4aa8012a10 con 0x559455c74af0 2023-10-06T03:07:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.812+0000 7f4a98ff9700 1 -- 172.21.15.92:0/2943143277 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ab0012cd0 con 0x559455c74af0 2023-10-06T03:07:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 10 client.6636 fetch_fsmap learned FSMap version 56 2023-10-06T03:07:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x559455b30470 con 0x559455c74af0 2023-10-06T03:07:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4a98ff9700 1 -- 172.21.15.92:0/2943143277 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8012a10 con 0x559455c74af0 2023-10-06T03:07:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4a98ff9700 1 -- 172.21.15.92:0/2943143277 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4ab0016480 con 0x559455c74af0 2023-10-06T03:07:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 10 client.6636 fetch_fsmap finished waiting for FSMap version 56 2023-10-06T03:07:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 10 client.6636 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:07:23.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 20 client.6636 populate_metadata read hostname 'smithi092' 2023-10-06T03:07:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c75660 0x559455cda440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 4 client.6636 mds_command: new command op to 6480 tid=0[{"prefix": "status"}] 2023-10-06T03:07:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 --> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559455b1de00 con 0x559455c75660 2023-10-06T03:07:23.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4abe140700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c75660 0x559455cda440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4a98ff9700 10 client.6636 ms_handle_connect on v2:172.21.15.92:6835/428935748 2023-10-06T03:07:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4abe140700 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c75660 0x559455cda440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b4ab9bba0ddacb server_cookie=a21c1e61bfaf6028 in_seq=0 out_seq=0 2023-10-06T03:07:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4a98ff9700 1 -- 172.21.15.92:0/2943143277 <== mds.0 v2:172.21.15.92:6835/428935748 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x559455b1de00 con 0x559455c75660 2023-10-06T03:07:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4a98ff9700 10 client.6636 handle_command_reply: tid=0 2023-10-06T03:07:23.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.813+0000 7f4ac969cb80 1 client.6636 shutdown 2023-10-06T03:07:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 msgr2=0x559455c74ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c74ec0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80ce190 tx=0x7f4ab0006040 comp rx=0 tx=0).stop 2023-10-06T03:07:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 shutdown_connections 2023-10-06T03:07:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:6835/428935748,v1:172.21.15.92:6837/428935748] conn(0x559455c75660 0x559455cda440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559455c74af0 0x559455c74ec0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559455c2abb0 0x559455c6b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 --2- 172.21.15.92:0/2943143277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559455c57220 0x559455c6b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4a98ff9700 0 client.6636 ms_handle_reset on v2:172.21.15.92:6835/428935748 2023-10-06T03:07:23.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 >> 172.21.15.92:0/2943143277 conn(0x559455c2c160 msgr2=0x559455c6eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 shutdown_connections 2023-10-06T03:07:23.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 1 -- 172.21.15.92:0/2943143277 wait complete. 2023-10-06T03:07:23.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac969cb80 20 client.6636 trim_cache size 0 max 16384 2023-10-06T03:07:23.828 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:07:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:07:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:07:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6480, 2023-10-06T03:07:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:07:23.829 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:07:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:07:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.8003547150000001, 2023-10-06T03:07:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 56, 2023-10-06T03:07:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 90, 2023-10-06T03:07:23.830 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 90, 2023-10-06T03:07:23.831 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.92475909 2023-10-06T03:07:23.831 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:07:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a9c067200 msgr2=0x7f4a9c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a9c067200 0x7f4a9c0696b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c8ba0 tx=0x7f4aa8015000 comp rx=0 tx=0).stop 2023-10-06T03:07:23.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 msgr2=0x7f4ab813db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4ab813db20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4011d40 tx=0x7f4ab40091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 shutdown_connections 2023-10-06T03:07:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a9c067200 0x7f4a9c0696b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4aa4004810 0x7f4ab813db20 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ab80c7cc0 0x7f4ab80c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 --2- 172.21.15.92:0/2049874385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ab80c6280 0x7f4ab81575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 >> 172.21.15.92:0/2049874385 conn(0x7f4ab80ce600 msgr2=0x7f4ab80cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 shutdown_connections 2023-10-06T03:07:23.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.814+0000 7f4ac03a4700 1 -- 172.21.15.92:0/2049874385 wait complete. 2023-10-06T03:07:23.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:07:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.951+0000 7f7d9cf5c700 1 Processor -- start 2023-10-06T03:07:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.951+0000 7f7d9cf5c700 1 -- start start 2023-10-06T03:07:23.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9cf5c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d98159890 0x7f7d9815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9cf5c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7d98159c60 0x7f7d9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9cf5c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9815a160 con 0x7f7d9815db10 2023-10-06T03:07:23.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9cf5c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9815df50 con 0x7f7d98159890 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9cf5c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980d13a0 con 0x7f7d98159c60 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d98159890 0x7f7d9815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d98159890 0x7f7d9815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37106/0 (socket says 172.21.15.92:37106) 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 -- 172.21.15.92:0/987521675 learned_addr learned my addr 172.21.15.92:0/987521675 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/987521675 >> v2:172.21.15.136:3300/0 conn(0x7f7d98159c60 0x7f7d9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 -- 172.21.15.92:0/987521675 >> v2:172.21.15.136:3300/0 conn(0x7f7d98159c60 msgr2=0x7f7d9815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 --2- 172.21.15.92:0/987521675 >> v2:172.21.15.136:3300/0 conn(0x7f7d98159c60 0x7f7d9815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 -- 172.21.15.92:0/987521675 >> v1:172.21.15.136:6789/0 conn(0x7f7d9815db10 legacy=0x7f7d98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 -- 172.21.15.92:0/987521675 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9815e140 con 0x7f7d98159890 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/987521675 >> v2:172.21.15.136:3300/0 conn(0x7f7d98159c60 0x7f7d9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.952+0000 7f7d9659c700 1 --2- 172.21.15.92:0/987521675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9815abd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7d980b67f0 tx=0x7f7d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7052bc974e6dc5c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d95d9b700 1 -- 172.21.15.92:0/987521675 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800b820 con 0x7f7d98159890 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d95d9b700 1 -- 172.21.15.92:0/987521675 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8800b980 con 0x7f7d98159890 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d95d9b700 1 -- 172.21.15.92:0/987521675 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800bd20 con 0x7f7d98159890 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/987521675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 msgr2=0x7f7d9815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/987521675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9815abd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7d980b67f0 tx=0x7f7d88009250 comp rx=0 tx=0).stop 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/987521675 shutdown_connections 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/987521675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9815abd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/987521675 >> v2:172.21.15.136:3300/0 conn(0x7f7d98159c60 0x7f7d9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/987521675 >> 172.21.15.92:0/987521675 conn(0x7f7d98068e20 msgr2=0x7f7d980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/987521675 shutdown_connections 2023-10-06T03:07:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.953+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/987521675 wait complete. 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.954+0000 7f7d9cf5c700 1 Processor -- start 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.954+0000 7f7d9cf5c700 1 -- start start 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9cf5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9816a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9cf5c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d98159c60 0x7f7d980b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9cf5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d9815db10 0x7f7d980c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9cf5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9816a4b0 con 0x7f7d98159890 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9cf5c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98158b40 con 0x7f7d98159c60 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9cf5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98158860 con 0x7f7d9815db10 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9816a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9816a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37122/0 (socket says 172.21.15.92:37122) 2023-10-06T03:07:23.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d9659c700 1 -- 172.21.15.92:0/1936478875 learned_addr learned my addr 172.21.15.92:0/1936478875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d98159c60 0x7f7d980b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.955+0000 7f7d96d9d700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d9815db10 0x7f7d980c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9659c700 1 -- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d9815db10 msgr2=0x7f7d980c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9659c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d9815db10 0x7f7d980c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9659c700 1 -- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d98159c60 msgr2=0x7f7d980b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9659c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d98159c60 0x7f7d980b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9659c700 1 -- 172.21.15.92:0/1936478875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880094f0 con 0x7f7d98159890 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d98159c60 0x7f7d980b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9659c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9816a940 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7d880030c0 tx=0x7f7d88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88011e60 con 0x7f7d98159890 2023-10-06T03:07:23.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d88020070 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8801b6a0 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d98169e60 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.956+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d980b6270 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.957+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d980b6460 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.957+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7d88022030 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.957+0000 7f7d8effd700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067200 0x7f7d780696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.958+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7d88086ad0 con 0x7f7d98159890 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.958+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067200 0x7f7d780696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:23.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.958+0000 7f7d8dd9b700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067200 0x7f7d780696b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806a530 tx=0x7f7d80006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:23.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:23.962+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d88053870 con 0x7f7d98159890 2023-10-06T03:07:24.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.192+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7d980b6460 con 0x7f7d98159890 2023-10-06T03:07:24.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.192+0000 7f7d8effd700 1 -- 172.21.15.92:0/1936478875 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7d88056e90 con 0x7f7d98159890 2023-10-06T03:07:24.194 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:07:24.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067200 msgr2=0x7f7d780696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:24.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067200 0x7f7d780696b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806a530 tx=0x7f7d80006f90 comp rx=0 tx=0).stop 2023-10-06T03:07:24.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 msgr2=0x7f7d9816a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9816a940 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7d880030c0 tx=0x7f7d88016000 comp rx=0 tx=0).stop 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 shutdown_connections 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067200 0x7f7d780696b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d98159890 0x7f7d9816a940 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d9815db10 0x7f7d980c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 --2- 172.21.15.92:0/1936478875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d98159c60 0x7f7d980b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 >> 172.21.15.92:0/1936478875 conn(0x7f7d98068e20 msgr2=0x7f7d980c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 shutdown_connections 2023-10-06T03:07:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.194+0000 7f7d9cf5c700 1 -- 172.21.15.92:0/1936478875 wait complete. 2023-10-06T03:07:24.208 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:07:24.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:07:24.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:07:24.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:07:24.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:07:24.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:07:24.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:07:24.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:07:24.209 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:07:24.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.209 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:07:24.224 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:07:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:07:24.224 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:07:24.225 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42162sec preferred_lft 42162sec 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:07:24.226 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.227 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:07:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:07:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:07:24.228 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:24.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.228 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:07:24.288 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:07:24.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.289 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:07:24.345 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:07:24.346 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:07:24.346 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:07:24.346 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:07:24.346 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:07:24.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.346 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:07:24.346 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:07:24.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:07:24.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:07:24.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:07:24.347 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:07:24.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.446 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:07:24.460 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:07:24.460 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:07:24.460 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:07:24.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.461 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:07:24.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:07:24.526 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:07:24.580 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:07:24.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:07:24.581 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:07:24.634 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:07:24.635 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:07:24.635 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:24.714 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:24.714 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:07:24.732 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:07:24.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:07:24.755 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:24.755 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:07:24.755 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:07:24.755 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:07:24.755 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:07:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:07:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:07:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:07:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:07:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:07:24.756 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:07:24.757 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:07:24.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:24.757 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:07:24.822 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:07:24.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.976+0000 7f1b3c834700 1 Processor -- start 2023-10-06T03:07:24.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.976+0000 7f1b3c834700 1 -- start start 2023-10-06T03:07:24.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b3c834700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1b381574d0 0x7f1b380cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b3c834700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 0x7f1b380b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b3c834700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b380b16a0 con 0x7f1b380cc3c0 2023-10-06T03:07:24.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b3c834700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380b1c60 con 0x7f1b381574d0 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b3c834700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381578a0 con 0x7f1b3806c0f0 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b36ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1b380cc3c0 0x7f1b3806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46928/0 (socket says 172.21.15.92:46928) 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b36ffd700 1 -- 172.21.15.92:0/583213167 learned_addr learned my addr 172.21.15.92:0/583213167 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b377fe700 1 --2- 172.21.15.92:0/583213167 >> v2:172.21.15.136:3300/0 conn(0x7f1b381574d0 0x7f1b380cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b37fff700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 0x7f1b380b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201751543 0 0) 0x7f1b380b16a0 con 0x7f1b380cc3c0 2023-10-06T03:07:24.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.977+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b20003570 con 0x7f1b380cc3c0 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 10461069 0 0) 0x7f1b20003570 con 0x7f1b380cc3c0 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 msgr2=0x7f1b380b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 0x7f1b380b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 >> v2:172.21.15.136:3300/0 conn(0x7f1b381574d0 msgr2=0x7f1b380cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 --2- 172.21.15.92:0/583213167 >> v2:172.21.15.136:3300/0 conn(0x7f1b381574d0 0x7f1b380cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380cdb70 con 0x7f1b380cc3c0 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f1b2c001470 con 0x7f1b380cc3c0 2023-10-06T03:07:24.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 >> v1:172.21.15.92:6789/0 conn(0x7f1b380cc3c0 legacy=0x7f1b3806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 0x7f1b20003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b20003fe0 0x7f1b20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380b1c60 con 0x7f1b20004d50 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b381578a0 con 0x7f1b20003fe0 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2000ba90 con 0x7f1b3806c0f0 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b36ffd700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b20003fe0 0x7f1b20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b377fe700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 0x7f1b20003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 -- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 msgr2=0x7f1b20003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b3806c0f0 0x7f1b20003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 -- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b20003fe0 msgr2=0x7f1b20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b20003fe0 0x7f1b20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 -- 172.21.15.92:0/583213167 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b20003570 con 0x7f1b20004d50 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b37fff700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b200051c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1b3806c4f0 tx=0x7f1b300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1b22966c8ec85a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.978+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000e6b0 con 0x7f1b20004d50 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3000e810 con 0x7f1b20004d50 2023-10-06T03:07:24.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b367fc700 1 -- 172.21.15.92:0/583213167 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000ebd0 con 0x7f1b20004d50 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b3c834700 1 -- 172.21.15.92:0/583213167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 msgr2=0x7f1b200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b3c834700 1 --2- 172.21.15.92:0/583213167 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b200051c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1b3806c4f0 tx=0x7f1b300091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b37fff700 1 -- 172.21.15.92:0/583213167 reap_dead start 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b3c834700 1 -- 172.21.15.92:0/583213167 shutdown_connections 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b3c834700 1 -- 172.21.15.92:0/583213167 >> 172.21.15.92:0/583213167 conn(0x7f1b380ce430 msgr2=0x7f1b380ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b3c834700 1 -- 172.21.15.92:0/583213167 shutdown_connections 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.979+0000 7f1b3c834700 1 -- 172.21.15.92:0/583213167 wait complete. 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.980+0000 7f1b3c834700 1 Processor -- start 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.980+0000 7f1b3c834700 1 -- start start 2023-10-06T03:07:24.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b3c834700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b3816aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b3c834700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b380c83a0 0x7f1b38151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b3c834700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b3815d380 0x7f1b3815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b3c834700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3815dd60 con 0x7f1b20004d50 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b3c834700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c8d80 con 0x7f1b3815d380 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b3c834700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3806a0b0 con 0x7f1b380c83a0 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b3816aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b3816aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37144/0 (socket says 172.21.15.92:37144) 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 -- 172.21.15.92:0/2488875183 learned_addr learned my addr 172.21.15.92:0/2488875183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b37fff700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b3815d380 0x7f1b3815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b36ffd700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b380c83a0 0x7f1b38151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 -- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b380c83a0 msgr2=0x7f1b38151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b380c83a0 0x7f1b38151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 -- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b3815d380 msgr2=0x7f1b3815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b3815d380 0x7f1b3815d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b377fe700 1 -- 172.21.15.92:0/2488875183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b380c8810 con 0x7f1b20004d50 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.981+0000 7f1b37fff700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b3815d380 0x7f1b3815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.982+0000 7f1b377fe700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b3816aa40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1b28011f00 tx=0x7f1b2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.982+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800e040 con 0x7f1b20004d50 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.982+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800f040 con 0x7f1b20004d50 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.982+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28015430 con 0x7f1b20004d50 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.982+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b380c8aa0 con 0x7f1b20004d50 2023-10-06T03:07:24.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.982+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b38069d40 con 0x7f1b20004d50 2023-10-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.983+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3806a390 con 0x7f1b20004d50 2023-10-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.983+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f1b28015860 con 0x7f1b20004d50 2023-10-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.983+0000 7f1b1f7fe700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b24067200 0x7f1b240696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.984+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f1b280858e0 con 0x7f1b20004d50 2023-10-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.984+0000 7f1b36ffd700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b24067200 0x7f1b240696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:24.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.984+0000 7f1b36ffd700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b24067200 0x7f1b240696b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c001c40 tx=0x7f1b2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:24.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:24.988+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b280525b0 con 0x7f1b20004d50 2023-10-06T03:07:25.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.218+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1b380698c0 con 0x7f1b20004d50 2023-10-06T03:07:25.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.218+0000 7f1b1f7fe700 1 -- 172.21.15.92:0/2488875183 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1b28022060 con 0x7f1b20004d50 2023-10-06T03:07:25.220 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:07:25.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b24067200 msgr2=0x7f1b240696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:25.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b24067200 0x7f1b240696b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c001c40 tx=0x7f1b2c009040 comp rx=0 tx=0).stop 2023-10-06T03:07:25.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 msgr2=0x7f1b3816aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b3816aa40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1b28011f00 tx=0x7f1b2800b040 comp rx=0 tx=0).stop 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 shutdown_connections 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1b24067200 0x7f1b240696b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1b20004d50 0x7f1b3816aa40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b3815d380 0x7f1b3815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 --2- 172.21.15.92:0/2488875183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b380c83a0 0x7f1b38151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 >> 172.21.15.92:0/2488875183 conn(0x7f1b380ce430 msgr2=0x7f1b38154e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 shutdown_connections 2023-10-06T03:07:25.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.220+0000 7f1b3c834700 1 -- 172.21.15.92:0/2488875183 wait complete. 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:07:25.234 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:07:25.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.235 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:07:25.253 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:07:25.253 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:07:25.254 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42162sec preferred_lft 42162sec 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.255 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:07:25.256 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:07:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:07:25.257 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:07:25.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.257 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:07:25.322 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:07:25.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.323 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:07:25.383 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:07:25.383 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:07:25.383 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:07:25.384 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:07:25.384 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:07:25.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.384 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:07:25.384 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:07:25.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:07:25.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:07:25.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:07:25.384 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:07:25.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.490 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:07:25.505 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:07:25.505 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:07:25.505 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:07:25.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:07:25.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:07:25.571 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:07:25.626 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:07:25.626 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:07:25.626 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:07:25.681 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:07:25.681 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:07:25.681 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:25.765 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:25.765 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:07:25.783 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:25.783 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:07:25.783 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:07:25.783 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:07:25.784 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:07:25.784 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:07:25.784 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:07:25.784 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:07:25.806 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:07:25.807 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:07:25.807 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:07:25.807 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:07:25.807 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:07:25.808 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:07:25.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:25.808 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:07:25.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:07:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.991+0000 7f47092e8700 1 Processor -- start 2023-10-06T03:07:25.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.991+0000 7f47092e8700 1 -- start start 2023-10-06T03:07:25.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47092e8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47040cd910 0x7f47040ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:25.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47092e8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040b5b00 con 0x7f47040b55a0 2023-10-06T03:07:25.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47092e8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040b5da0 con 0x7f4704159ec0 2023-10-06T03:07:25.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47092e8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47040b6080 con 0x7f47040cd910 2023-10-06T03:07:25.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47037fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f47040b55a0 0x7f47040ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46934/0 (socket says 172.21.15.92:46934) 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47037fe700 1 -- 172.21.15.92:0/4219867622 learned_addr learned my addr 172.21.15.92:0/4219867622 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47040cd910 0x7f47040ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586684801 0 0) 0x7f47040b5b00 con 0x7f47040b55a0 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f0003570 con 0x7f47040b55a0 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172873641 0 0) 0x7f47040b5da0 con 0x7f4704159ec0 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47040b5b00 con 0x7f4704159ec0 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298223276 0 0) 0x7f46f0003570 con 0x7f47040b55a0 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.992+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47040cd910 msgr2=0x7f47040ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47040cd910 0x7f47040ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 >> v1:172.21.15.136:6789/0 conn(0x7f4704159ec0 legacy=0x7f47040b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:25.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040cc100 con 0x7f47040b55a0 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f46fc0017f0 con 0x7f47040b55a0 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47040cd910 0x7f47040ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 >> v1:172.21.15.92:6789/0 conn(0x7f47040b55a0 legacy=0x7f47040ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f0003c20 0x7f46f0003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46f0004810 0x7f46f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f00051c0 0x7f46f000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040b6080 con 0x7f46f0004810 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f000bfa0 con 0x7f46f00051c0 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f000c160 con 0x7f46f0003c20 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46f0004810 0x7f46f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f4702ffd700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f0003c20 0x7f46f0003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:25.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47037fe700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f00051c0 0x7f46f000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 -- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f0003c20 msgr2=0x7f46f0003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f0003c20 0x7f46f0003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 -- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f00051c0 msgr2=0x7f46f000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f00051c0 0x7f46f000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 -- 172.21.15.92:0/4219867622 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009420 con 0x7f46f0004810 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46f0004810 0x7f46f0004c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009db0 tx=0x7f46ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78646b77b0073c6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec009740 con 0x7f46f0004810 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00e920 con 0x7f46f0004810 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.993+0000 7f47027fc700 1 -- 172.21.15.92:0/4219867622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00ece0 con 0x7f46f0004810 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47092e8700 1 -- 172.21.15.92:0/4219867622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46f0004810 msgr2=0x7f46f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:26.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47092e8700 1 --2- 172.21.15.92:0/4219867622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46f0004810 0x7f46f0004c80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009db0 tx=0x7f46ec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47037fe700 1 -- 172.21.15.92:0/4219867622 reap_dead start 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47092e8700 1 -- 172.21.15.92:0/4219867622 shutdown_connections 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47092e8700 1 -- 172.21.15.92:0/4219867622 >> 172.21.15.92:0/4219867622 conn(0x7f47040999f0 msgr2=0x7f47040b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47092e8700 1 -- 172.21.15.92:0/4219867622 shutdown_connections 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.994+0000 7f47092e8700 1 -- 172.21.15.92:0/4219867622 wait complete. 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.995+0000 7f47092e8700 1 Processor -- start 2023-10-06T03:07:26.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.995+0000 7f47092e8700 1 -- start start 2023-10-06T03:07:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47092e8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f0004810 0x7f4704153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47092e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 0x7f4704069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:26.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47092e8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470406a260 0x7f470414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47092e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040cd0c0 con 0x7f47040cc800 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47092e8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040cd3a0 con 0x7f46f0004810 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47092e8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47040cd680 con 0x7f470406a260 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 0x7f4704069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 0x7f4704069d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37172/0 (socket says 172.21.15.92:37172) 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 -- 172.21.15.92:0/3881831932 learned_addr learned my addr 172.21.15.92:0/3881831932 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f4702ffd700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f0004810 0x7f4704153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f47037fe700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470406a260 0x7f470414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 -- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470406a260 msgr2=0x7f470414e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:26.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470406a260 0x7f470414e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 -- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f0004810 msgr2=0x7f4704153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f0004810 0x7f4704153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 -- 172.21.15.92:0/3881831932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040cde90 con 0x7f47040cc800 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f4702ffd700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f0004810 0x7f4704153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.996+0000 7f46fa7fc700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 0x7f4704069d20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f46ec006eb0 tx=0x7f46ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.997+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01b400 con 0x7f47040cc800 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.997+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec021040 con 0x7f47040cc800 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.997+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47040ce170 con 0x7f47040cc800 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.997+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01b7d0 con 0x7f47040cc800 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.997+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470414ed50 con 0x7f47040cc800 2023-10-06T03:07:26.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.998+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47040718b0 con 0x7f47040cc800 2023-10-06T03:07:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.998+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f46ec01bbc0 con 0x7f47040cc800 2023-10-06T03:07:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.998+0000 7f46fb7fe700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46e4067280 0x7f46e4069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.999+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f46ec086b60 con 0x7f47040cc800 2023-10-06T03:07:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.999+0000 7f4702ffd700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46e4067280 0x7f46e4069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:25.999+0000 7f4702ffd700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46e4067280 0x7f46e4069730 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f46f4001130 tx=0x7f46f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:26.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.003+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46ec0538c0 con 0x7f47040cc800 2023-10-06T03:07:26.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.229+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4704063c40 con 0x7f47040cc800 2023-10-06T03:07:26.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.895+0000 7f46fb7fe700 1 -- 172.21.15.92:0/3881831932 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 136+0+0 (secure 0 0 0) 0x7f46ec0170c0 con 0x7f47040cc800 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46e4067280 msgr2=0x7f46e4069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46e4067280 0x7f46e4069730 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f46f4001130 tx=0x7f46f4009040 comp rx=0 tx=0).stop 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 msgr2=0x7f4704069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 0x7f4704069d20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f46ec006eb0 tx=0x7f46ec019000 comp rx=0 tx=0).stop 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 shutdown_connections 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f46e4067280 0x7f46e4069730 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f470406a260 0x7f470414e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f0004810 0x7f4704153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 --2- 172.21.15.92:0/3881831932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47040cc800 0x7f4704069d20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:26.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 >> 172.21.15.92:0/3881831932 conn(0x7f47040999f0 msgr2=0x7f47040c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:26.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 shutdown_connections 2023-10-06T03:07:26.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:26.898+0000 7f47092e8700 1 -- 172.21.15.92:0/3881831932 wait complete. 2023-10-06T03:07:26.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:07:27.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.028+0000 7f2562889700 1 Processor -- start 2023-10-06T03:07:27.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.028+0000 7f2562889700 1 -- start start 2023-10-06T03:07:27.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f2562889700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f255c159890 0x7f255c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f2562889700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f255c159c60 0x7f255c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f2562889700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c15a160 con 0x7f255c15db10 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f2562889700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c15df50 con 0x7f255c159c60 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f2562889700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0d13a0 con 0x7f255c159890 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f255b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f255c159c60 0x7f255c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f255b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f255c159c60 0x7f255c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37176/0 (socket says 172.21.15.92:37176) 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.029+0000 7f255b7fe700 1 -- 172.21.15.92:0/4041732629 learned_addr learned my addr 172.21.15.92:0/4041732629 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255b7fe700 1 -- 172.21.15.92:0/4041732629 >> v2:172.21.15.136:3300/0 conn(0x7f255c159890 msgr2=0x7f255c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255bfff700 1 --2- 172.21.15.92:0/4041732629 >> v2:172.21.15.136:3300/0 conn(0x7f255c159890 0x7f255c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255b7fe700 1 --2- 172.21.15.92:0/4041732629 >> v2:172.21.15.136:3300/0 conn(0x7f255c159890 0x7f255c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255b7fe700 1 -- 172.21.15.92:0/4041732629 >> v1:172.21.15.136:6789/0 conn(0x7f255c15db10 legacy=0x7f255c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255b7fe700 1 -- 172.21.15.92:0/4041732629 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c15e140 con 0x7f255c159c60 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255b7fe700 1 --2- 172.21.15.92:0/4041732629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c159c60 0x7f255c15d5d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f255c0d1090 tx=0x7f2550009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec90f92e95c69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255affd700 1 -- 172.21.15.92:0/4041732629 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255000b820 con 0x7f255c159c60 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255affd700 1 -- 172.21.15.92:0/4041732629 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255000b980 con 0x7f255c159c60 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f255affd700 1 -- 172.21.15.92:0/4041732629 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255000bd20 con 0x7f255c159c60 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f2562889700 1 -- 172.21.15.92:0/4041732629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c159c60 msgr2=0x7f255c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.030+0000 7f2562889700 1 --2- 172.21.15.92:0/4041732629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c159c60 0x7f255c15d5d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f255c0d1090 tx=0x7f2550009250 comp rx=0 tx=0).stop 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.031+0000 7f2562889700 1 -- 172.21.15.92:0/4041732629 shutdown_connections 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.031+0000 7f2562889700 1 --2- 172.21.15.92:0/4041732629 >> v2:172.21.15.136:3300/0 conn(0x7f255c159890 0x7f255c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.031+0000 7f2562889700 1 --2- 172.21.15.92:0/4041732629 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c159c60 0x7f255c15d5d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.031+0000 7f2562889700 1 -- 172.21.15.92:0/4041732629 >> 172.21.15.92:0/4041732629 conn(0x7f255c068e20 msgr2=0x7f255c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:27.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.031+0000 7f2562889700 1 -- 172.21.15.92:0/4041732629 shutdown_connections 2023-10-06T03:07:27.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.031+0000 7f2562889700 1 -- 172.21.15.92:0/4041732629 wait complete. 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.032+0000 7f2562889700 1 Processor -- start 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.032+0000 7f2562889700 1 -- start start 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.032+0000 7f2562889700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c159890 0x7f255c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.032+0000 7f2562889700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c159c60 0x7f255c14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2562889700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 0x7f255c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2562889700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0b1e90 con 0x7f255c15db10 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2562889700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c14c7a0 con 0x7f255c159890 2023-10-06T03:07:27.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2562889700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0b3ad0 con 0x7f255c159c60 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 0x7f255c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 0x7f255c0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37182/0 (socket says 172.21.15.92:37182) 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 -- 172.21.15.92:0/584198332 learned_addr learned my addr 172.21.15.92:0/584198332 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f255bfff700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c159890 0x7f255c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f255b7fe700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c159c60 0x7f255c14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 -- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c159c60 msgr2=0x7f255c14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c159c60 0x7f255c14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 -- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c159890 msgr2=0x7f255c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c159890 0x7f255c0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 -- 172.21.15.92:0/584198332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500094f0 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2560e26700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 0x7f255c0b3590 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f255400bbe0 tx=0x7f2554009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25540092e0 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255400c630 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c0cfd40 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400c9d0 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.033+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0b1bb0 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.034+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c151f30 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.035+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2554012070 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.035+0000 7f2543fff700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2548067200 0x7f25480696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.035+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f2554084540 con 0x7f255c15db10 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.035+0000 7f255bfff700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2548067200 0x7f25480696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.036+0000 7f255bfff700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2548067200 0x7f25480696b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f254c00acb0 tx=0x7f254c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.039+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f255c151f30 con 0x7f255c15db10 2023-10-06T03:07:27.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.246+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f255c151f30 con 0x7f2548067200 2023-10-06T03:07:27.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.247+0000 7f2543fff700 1 -- 172.21.15.92:0/584198332 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f255c151f30 con 0x7f2548067200 2023-10-06T03:07:27.248 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:07:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:07:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:27.248 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:07:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2548067200 msgr2=0x7f25480696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2548067200 0x7f25480696b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f254c00acb0 tx=0x7f254c008040 comp rx=0 tx=0).stop 2023-10-06T03:07:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 msgr2=0x7f255c0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 0x7f255c0b3590 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f255400bbe0 tx=0x7f2554009840 comp rx=0 tx=0).stop 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 shutdown_connections 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2548067200 0x7f25480696b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f255c15db10 0x7f255c0b3590 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c159890 0x7f255c0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 --2- 172.21.15.92:0/584198332 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c159c60 0x7f255c14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 >> 172.21.15.92:0/584198332 conn(0x7f255c068e20 msgr2=0x7f255c1585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 shutdown_connections 2023-10-06T03:07:27.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.249+0000 7f2562889700 1 -- 172.21.15.92:0/584198332 wait complete. 2023-10-06T03:07:27.262 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:07:27.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.380+0000 7f7d7651d700 1 Processor -- start 2023-10-06T03:07:27.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.380+0000 7f7d7651d700 1 -- start start 2023-10-06T03:07:27.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d7651d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 0x7f7d7015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d7651d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7015a160 con 0x7f7d70069560 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d7651d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d700d13a0 con 0x7f7d70159c60 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d7651d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7015e140 con 0x7f7d70159890 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d74aba700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7d70069560 0x7f7d700ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46936/0 (socket says 172.21.15.92:46936) 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d74aba700 1 -- 172.21.15.92:0/2802787606 learned_addr learned my addr 172.21.15.92:0/2802787606 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 0x7f7d7015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002594494 0 0) 0x7f7d7015a160 con 0x7f7d70069560 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d58003570 con 0x7f7d70069560 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508532637 0 0) 0x7f7d700d13a0 con 0x7f7d70159c60 2023-10-06T03:07:27.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d7015a160 con 0x7f7d70159c60 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4289297507 0 0) 0x7f7d58003570 con 0x7f7d70069560 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 msgr2=0x7f7d7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 0x7f7d7015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 >> v1:172.21.15.136:6789/0 conn(0x7f7d70159c60 legacy=0x7f7d7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7015a3d0 con 0x7f7d70069560 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7d680017f0 con 0x7f7d70069560 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.381+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 >> v1:172.21.15.92:6789/0 conn(0x7f7d70069560 legacy=0x7f7d700ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d58003c20 0x7f7d58004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 0x7f7d58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58004d50 0x7f7d580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7015e140 con 0x7f7d58003c20 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5800bbf0 con 0x7f7d58004d50 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5800bd60 con 0x7f7d70159890 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d58003c20 0x7f7d58004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d74aba700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58004d50 0x7f7d580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6f7fe700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 0x7f7d58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 -- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 msgr2=0x7f7d58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d70159890 0x7f7d58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 -- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58004d50 msgr2=0x7f7d580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58004d50 0x7f7d580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 -- 172.21.15.92:0/2802787606 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58003570 con 0x7f7d58003c20 2023-10-06T03:07:27.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d74aba700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58004d50 0x7f7d580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d58003c20 0x7f7d58004010 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7d60006f00 tx=0x7f7d600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550b6ea46eed86e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000e710 con 0x7f7d58003c20 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6000e870 con 0x7f7d58003c20 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.382+0000 7f7d6effd700 1 -- 172.21.15.92:0/2802787606 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000ec30 con 0x7f7d58003c20 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d7651d700 1 -- 172.21.15.92:0/2802787606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d58003c20 msgr2=0x7f7d58004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d7651d700 1 --2- 172.21.15.92:0/2802787606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d58003c20 0x7f7d58004010 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7d60006f00 tx=0x7f7d600091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d74aba700 1 -- 172.21.15.92:0/2802787606 reap_dead start 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d7651d700 1 -- 172.21.15.92:0/2802787606 shutdown_connections 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d7651d700 1 -- 172.21.15.92:0/2802787606 >> 172.21.15.92:0/2802787606 conn(0x7f7d70068e20 msgr2=0x7f7d700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d7651d700 1 -- 172.21.15.92:0/2802787606 shutdown_connections 2023-10-06T03:07:27.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.383+0000 7f7d7651d700 1 -- 172.21.15.92:0/2802787606 wait complete. 2023-10-06T03:07:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.384+0000 7f7d7651d700 1 Processor -- start 2023-10-06T03:07:27.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.384+0000 7f7d7651d700 1 -- start start 2023-10-06T03:07:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58003c20 0x7f7d7014c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 0x7f7d70155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d701561e0 0x7f7d700b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700cdcc0 con 0x7f7d700b43f0 2023-10-06T03:07:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700cdfa0 con 0x7f7d58003c20 2023-10-06T03:07:27.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700ce280 con 0x7f7d701561e0 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 0x7f7d70155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 0x7f7d70155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37208/0 (socket says 172.21.15.92:37208) 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 -- 172.21.15.92:0/1689293520 learned_addr learned my addr 172.21.15.92:0/1689293520 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58003c20 0x7f7d7014c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d74aba700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d701561e0 0x7f7d700b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 -- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d701561e0 msgr2=0x7f7d700b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d701561e0 0x7f7d700b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 -- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58003c20 msgr2=0x7f7d7014c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58003c20 0x7f7d7014c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 -- 172.21.15.92:0/1689293520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700ce560 con 0x7f7d700b43f0 2023-10-06T03:07:27.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d6f7fe700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 0x7f7d70155ca0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e810 tx=0x7f7d64009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d640026a0 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d64002800 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d700ce840 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64002ba0 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.385+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d700b6040 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.386+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d700b6320 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.387+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7d6401d460 con 0x7f7d700b43f0 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.387+0000 7f7d57fff700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d5c067200 0x7f7d5c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.387+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d5c067200 0x7f7d5c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.387+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7d64085200 con 0x7f7d700b43f0 2023-10-06T03:07:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.388+0000 7f7d6ffff700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d5c067200 0x7f7d5c0696b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7d701557c0 tx=0x7f7d6001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.392+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d64051fa0 con 0x7f7d700b43f0 2023-10-06T03:07:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.576+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7d7004c3c0 con 0x7f7d700b43f0 2023-10-06T03:07:27.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.576+0000 7f7d57fff700 1 -- 172.21.15.92:0/1689293520 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7d640555c0 con 0x7f7d700b43f0 2023-10-06T03:07:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d5c067200 msgr2=0x7f7d5c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d5c067200 0x7f7d5c0696b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7d701557c0 tx=0x7f7d6001a000 comp rx=0 tx=0).stop 2023-10-06T03:07:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 msgr2=0x7f7d70155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 0x7f7d70155ca0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e810 tx=0x7f7d64009040 comp rx=0 tx=0).stop 2023-10-06T03:07:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 shutdown_connections 2023-10-06T03:07:27.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d5c067200 0x7f7d5c0696b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d701561e0 0x7f7d700b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d58003c20 0x7f7d7014c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 --2- 172.21.15.92:0/1689293520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d700b43f0 0x7f7d70155ca0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 >> 172.21.15.92:0/1689293520 conn(0x7f7d70068e20 msgr2=0x7f7d70152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 shutdown_connections 2023-10-06T03:07:27.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.578+0000 7f7d7651d700 1 -- 172.21.15.92:0/1689293520 wait complete. 2023-10-06T03:07:27.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-06T03:07:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.708+0000 7f11b66c0700 1 Processor -- start 2023-10-06T03:07:27.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.708+0000 7f11b66c0700 1 -- start start 2023-10-06T03:07:27.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b66c0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11b0069560 0x7f11b00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b66c0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b0069ee0 con 0x7f11b0159c60 2023-10-06T03:07:27.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b66c0700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b015a160 con 0x7f11b0159890 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b66c0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b00d13a0 con 0x7f11b0069560 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b4c5d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f11b0069560 0x7f11b00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11af7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f11b0159c60 0x7f11b015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46948/0 (socket says 172.21.15.92:46948) 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11af7fe700 1 -- 172.21.15.92:0/992791925 learned_addr learned my addr 172.21.15.92:0/992791925 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b4c5d700 1 -- 172.21.15.92:0/992791925 >> v1:172.21.15.136:6789/0 conn(0x7f11b0159890 legacy=0x7f11b015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b4c5d700 1 -- 172.21.15.92:0/992791925 >> v1:172.21.15.92:6789/0 conn(0x7f11b0159c60 legacy=0x7f11b015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b4c5d700 1 -- 172.21.15.92:0/992791925 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b015e140 con 0x7f11b0069560 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11aeffd700 1 -- 172.21.15.92:0/992791925 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217712782 0 0) 0x7f11b0069ee0 con 0x7f11b0159c60 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11b4c5d700 1 --2- 172.21.15.92:0/992791925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0069560 0x7f11b00699a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009e60 tx=0x7f11a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350dea2574d7990f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.709+0000 7f11aeffd700 1 -- 172.21.15.92:0/992791925 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a800e070 con 0x7f11b0069560 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11aeffd700 1 -- 172.21.15.92:0/992791925 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a800bc30 con 0x7f11b0069560 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11aeffd700 1 -- 172.21.15.92:0/992791925 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8012430 con 0x7f11b0069560 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 -- 172.21.15.92:0/992791925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0069560 msgr2=0x7f11b00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 --2- 172.21.15.92:0/992791925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0069560 0x7f11b00699a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009e60 tx=0x7f11a8009780 comp rx=0 tx=0).stop 2023-10-06T03:07:27.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 -- 172.21.15.92:0/992791925 shutdown_connections 2023-10-06T03:07:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 --2- 172.21.15.92:0/992791925 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0069560 0x7f11b00699a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 -- 172.21.15.92:0/992791925 >> 172.21.15.92:0/992791925 conn(0x7f11b0068e20 msgr2=0x7f11b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 -- 172.21.15.92:0/992791925 shutdown_connections 2023-10-06T03:07:27.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.710+0000 7f11b66c0700 1 -- 172.21.15.92:0/992791925 wait complete. 2023-10-06T03:07:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.711+0000 7f11b66c0700 1 Processor -- start 2023-10-06T03:07:27.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.711+0000 7f11b66c0700 1 -- start start 2023-10-06T03:07:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11b66c0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11b0069560 0x7f11b015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11b66c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 0x7f11b00cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11b66c0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b0159c60 0x7f11b01452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11b66c0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b015cb90 con 0x7f11b0159890 2023-10-06T03:07:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11b66c0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b015bf80 con 0x7f11b0159c60 2023-10-06T03:07:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11b66c0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b015c260 con 0x7f11b0069560 2023-10-06T03:07:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 0x7f11b00cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 0x7f11b00cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37234/0 (socket says 172.21.15.92:37234) 2023-10-06T03:07:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 -- 172.21.15.92:0/2207983482 learned_addr learned my addr 172.21.15.92:0/2207983482 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 -- 172.21.15.92:0/2207983482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11b0069560 msgr2=0x7f11b015b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11b0069560 0x7f11b015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 -- 172.21.15.92:0/2207983482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b0159c60 msgr2=0x7f11b01452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b0159c60 0x7f11b01452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.712+0000 7f11af7fe700 1 -- 172.21.15.92:0/2207983482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8009a20 con 0x7f11b0159890 2023-10-06T03:07:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.713+0000 7f11af7fe700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 0x7f11b00cfa40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c5b0 tx=0x7f11a4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.713+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4005b30 con 0x7f11b0159890 2023-10-06T03:07:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.713+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a4005c90 con 0x7f11b0159890 2023-10-06T03:07:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.713+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4007300 con 0x7f11b0159890 2023-10-06T03:07:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.713+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b015c540 con 0x7f11b0159890 2023-10-06T03:07:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.713+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b0145830 con 0x7f11b0159890 2023-10-06T03:07:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.714+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b004c3c0 con 0x7f11b0159890 2023-10-06T03:07:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.714+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f11a401b3b0 con 0x7f11b0159890 2023-10-06T03:07:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.714+0000 7f1197fff700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f119c067200 0x7f119c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.715+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f11a4084110 con 0x7f11b0159890 2023-10-06T03:07:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.715+0000 7f11affff700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f119c067200 0x7f119c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.715+0000 7f11affff700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f119c067200 0x7f119c0696b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f11a0001660 tx=0x7f11a0007b00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:27.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.719+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11a4050dd0 con 0x7f11b0159890 2023-10-06T03:07:27.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.931+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f11b015ccd0 con 0x7f119c067200 2023-10-06T03:07:27.933 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:27.933+0000 7fdd0eb80700 -1 client.6327 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-06T03:07:01.264891+0000 mtime=2023-10-06T03:07:01.264891+0000 ctime=2023-10-06T03:07:01.270540+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000371993"] has_dir_layout 0x555aeb33de00) 2023-10-06T03:07:27.936 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:27.936+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:27.937 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:27.936+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:27.937 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:27.936+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:27.937 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:27.936+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:27.937 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:07:27.936+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.941+0000 7f1197fff700 1 -- 172.21.15.92:0/2207983482 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+470 (secure 0 0 0) 0x7f11a00012d0 con 0x7f119c067200 2023-10-06T03:07:27.941 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_addrs": [ 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6789", 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6789", 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6790" 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": { 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "data": [ 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": " 340G", 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_data", 2023-10-06T03:07:27.942 INFO:teuthology.orchestra.run.smithi092.stdout: "used": " 0 " 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": [ 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": " 340G", 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_metadata", 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: "used": "64.0k" 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:27.943 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f119c067200 msgr2=0x7f119c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f119c067200 0x7f119c0696b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f11a0001660 tx=0x7f11a0007b00 comp rx=0 tx=0).stop 2023-10-06T03:07:27.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 msgr2=0x7f11b00cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 0x7f11b00cfa40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c5b0 tx=0x7f11a4001070 comp rx=0 tx=0).stop 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 shutdown_connections 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f119c067200 0x7f119c0696b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b0159c60 0x7f11b01452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11b0069560 0x7f11b015b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 --2- 172.21.15.92:0/2207983482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11b0159890 0x7f11b00cfa40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 >> 172.21.15.92:0/2207983482 conn(0x7f11b0068e20 msgr2=0x7f11b01553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 shutdown_connections 2023-10-06T03:07:27.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:27.943+0000 7f11b66c0700 1 -- 172.21.15.92:0/2207983482 wait complete. 2023-10-06T03:07:27.955 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:07:27.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:27.991 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:28.040 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:07:28.040 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:07:28.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:07:28.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:07:28.076 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:07:28.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:28.077 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:07:28.131 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:07:28.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:07:28.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:07:28.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:07:28.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:07:28.174 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:07:28.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:28.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.345+0000 7f4808e64700 1 Processor -- start 2023-10-06T03:07:28.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.345+0000 7f4808e64700 1 -- start start 2023-10-06T03:07:28.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f4808e64700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 0x7f48040699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f4808e64700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4804069ee0 con 0x7f4804159890 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f4808e64700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f480415a160 con 0x7f4804159c60 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f4808e64700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040d13a0 con 0x7f4804069560 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f480259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4804159890 0x7f480415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46956/0 (socket says 172.21.15.92:46956) 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f480259c700 1 -- 172.21.15.92:0/779746524 learned_addr learned my addr 172.21.15.92:0/779746524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99172535 0 0) 0x7f4804069ee0 con 0x7f4804159890 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f4802d9d700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 0x7f48040699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.346+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47ec003570 con 0x7f4804159890 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378881434 0 0) 0x7f47ec003570 con 0x7f4804159890 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 msgr2=0x7f48040699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 0x7f48040699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 >> v1:172.21.15.136:6789/0 conn(0x7f4804159c60 legacy=0x7f480415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480415e140 con 0x7f4804159890 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f47f40017f0 con 0x7f4804159890 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 >> v1:172.21.15.92:6789/0 conn(0x7f4804159890 legacy=0x7f480415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec003a90 0x7f47ec003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 0x7f47ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ec004d50 0x7f47ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040d13a0 con 0x7f47ec004d50 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec00bb40 con 0x7f47ec003a90 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec00bd00 con 0x7f4804069560 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ec004d50 0x7f47ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f480259c700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec003a90 0x7f47ec003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4801d9b700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 0x7f47ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 -- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 msgr2=0x7f47ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4804069560 0x7f47ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 -- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec003a90 msgr2=0x7f47ec003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec003a90 0x7f47ec003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 -- 172.21.15.92:0/779746524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec003570 con 0x7f47ec004d50 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.347+0000 7f4802d9d700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ec004d50 0x7f47ec0051c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009be0 tx=0x7f47fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c0b511de5cf17a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00e710 con 0x7f47ec004d50 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00e870 con 0x7f47ec004d50 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f480159a700 1 -- 172.21.15.92:0/779746524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00ec30 con 0x7f47ec004d50 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4808e64700 1 -- 172.21.15.92:0/779746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ec004d50 msgr2=0x7f47ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4808e64700 1 --2- 172.21.15.92:0/779746524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47ec004d50 0x7f47ec0051c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009be0 tx=0x7f47fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4802d9d700 1 -- 172.21.15.92:0/779746524 reap_dead start 2023-10-06T03:07:28.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4808e64700 1 -- 172.21.15.92:0/779746524 shutdown_connections 2023-10-06T03:07:28.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4808e64700 1 -- 172.21.15.92:0/779746524 >> 172.21.15.92:0/779746524 conn(0x7f4804068e20 msgr2=0x7f48040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:28.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4808e64700 1 -- 172.21.15.92:0/779746524 shutdown_connections 2023-10-06T03:07:28.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.348+0000 7f4808e64700 1 -- 172.21.15.92:0/779746524 wait complete. 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.349+0000 7f4808e64700 1 Processor -- start 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.349+0000 7f4808e64700 1 -- start start 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4808e64700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec004d50 0x7f48040b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4808e64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 0x7f48040b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4808e64700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48040b3fe0 0x7f48040b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4808e64700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804154a50 con 0x7f48040cfc80 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4808e64700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040ce4a0 con 0x7f47ec004d50 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4808e64700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040b63c0 con 0x7f48040b3fe0 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4801d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 0x7f48040b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4801d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 0x7f48040b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37272/0 (socket says 172.21.15.92:37272) 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4801d9b700 1 -- 172.21.15.92:0/399042549 learned_addr learned my addr 172.21.15.92:0/399042549 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f480259c700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec004d50 0x7f48040b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4802d9d700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48040b3fe0 0x7f48040b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4801d9b700 1 -- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48040b3fe0 msgr2=0x7f48040b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.350+0000 7f4801d9b700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48040b3fe0 0x7f48040b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4801d9b700 1 -- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec004d50 msgr2=0x7f48040b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4801d9b700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec004d50 0x7f48040b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4801d9b700 1 -- 172.21.15.92:0/399042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040cdb70 con 0x7f48040cfc80 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f480259c700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec004d50 0x7f48040b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4802d9d700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48040b3fe0 0x7f48040b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4801d9b700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 0x7f48040b3aa0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f47f800f8d0 tx=0x7f47f8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800d640 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800d7a0 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800db60 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48040cde50 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.351+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48040ce930 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.352+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48040ce630 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.352+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f47f8014470 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.352+0000 7f47f27fc700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47e4067200 0x7f47e40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.353+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f47f8084f70 con 0x7f48040cfc80 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.353+0000 7f480259c700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47e4067200 0x7f47e40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.353+0000 7f480259c700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47e4067200 0x7f47e40696b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001920 tx=0x7f47f4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:28.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.357+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47f8051d10 con 0x7f48040cfc80 2023-10-06T03:07:28.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.581+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f480404c3c0 con 0x7f48040cfc80 2023-10-06T03:07:28.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.581+0000 7f47f27fc700 1 -- 172.21.15.92:0/399042549 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f47f8055330 con 0x7f48040cfc80 2023-10-06T03:07:28.582 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:28.582 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:17.381585+0000","modified":"2023-10-06T03:07:21.036747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6480":{"gid":6480,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/428935748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":428935748},{"type":"v1","addr":"172.21.15.92:6837","nonce":428935748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:07:28.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47e4067200 msgr2=0x7f47e40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47e4067200 0x7f47e40696b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f47f4001920 tx=0x7f47f4006040 comp rx=0 tx=0).stop 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 msgr2=0x7f48040b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 0x7f48040b3aa0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f47f800f8d0 tx=0x7f47f8007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 shutdown_connections 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47e4067200 0x7f47e40696b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f48040b3fe0 0x7f48040b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47ec004d50 0x7f48040b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 --2- 172.21.15.92:0/399042549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48040cfc80 0x7f48040b3aa0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 >> 172.21.15.92:0/399042549 conn(0x7f4804068e20 msgr2=0x7f48040b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 shutdown_connections 2023-10-06T03:07:28.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.584+0000 7f4808e64700 1 -- 172.21.15.92:0/399042549 wait complete. 2023-10-06T03:07:28.586 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 57 2023-10-06T03:07:28.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.710+0000 7febd641f700 1 Processor -- start 2023-10-06T03:07:28.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.710+0000 7febd641f700 1 -- start start 2023-10-06T03:07:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd641f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 0x7febd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd641f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7febd0159c60 0x7febd015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd641f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd015a160 con 0x7febd015db10 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd641f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd00d13a0 con 0x7febd0159c60 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd641f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd015e140 con 0x7febd0159890 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd49bc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7febd015db10 0x7febd0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46968/0 (socket says 172.21.15.92:46968) 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febd49bc700 1 -- 172.21.15.92:0/927403344 learned_addr learned my addr 172.21.15.92:0/927403344 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febcf7fe700 1 --2- 172.21.15.92:0/927403344 >> v2:172.21.15.136:3300/0 conn(0x7febd0159c60 0x7febd015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.711+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372666023 0 0) 0x7febd015a160 con 0x7febd015db10 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febcffff700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 0x7febd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb8003570 con 0x7febd015db10 2023-10-06T03:07:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233686134 0 0) 0x7febb8003570 con 0x7febd015db10 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 msgr2=0x7febd015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 0x7febd015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 >> v2:172.21.15.136:3300/0 conn(0x7febd0159c60 msgr2=0x7febd015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 --2- 172.21.15.92:0/927403344 >> v2:172.21.15.136:3300/0 conn(0x7febd0159c60 0x7febd015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd015a3d0 con 0x7febd015db10 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7febc80017f0 con 0x7febd015db10 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 >> v1:172.21.15.92:6789/0 conn(0x7febd015db10 legacy=0x7febd0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 0x7febb8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febb8003fe0 0x7febb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb8004d50 0x7febb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd00d13a0 con 0x7febb8004d50 2023-10-06T03:07:28.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd015e140 con 0x7febb8003fe0 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb800ba90 con 0x7febd0159890 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb8004d50 0x7febb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febcffff700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 0x7febb8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febcf7fe700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febb8003fe0 0x7febb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 -- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 msgr2=0x7febb8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd0159890 0x7febb8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 -- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febb8003fe0 msgr2=0x7febb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febb8003fe0 0x7febb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 -- 172.21.15.92:0/927403344 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8003570 con 0x7febb8004d50 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.712+0000 7febd49bc700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb8004d50 0x7febb80051c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7febc800a970 tx=0x7febc8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8772d358ebd8c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8012440 con 0x7febb8004d50 2023-10-06T03:07:28.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc80125c0 con 0x7febb8004d50 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febceffd700 1 -- 172.21.15.92:0/927403344 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8012980 con 0x7febb8004d50 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd641f700 1 -- 172.21.15.92:0/927403344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb8004d50 msgr2=0x7febb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd641f700 1 --2- 172.21.15.92:0/927403344 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febb8004d50 0x7febb80051c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7febc800a970 tx=0x7febc8002210 comp rx=0 tx=0).stop 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd49bc700 1 -- 172.21.15.92:0/927403344 reap_dead start 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd641f700 1 -- 172.21.15.92:0/927403344 shutdown_connections 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd641f700 1 -- 172.21.15.92:0/927403344 >> 172.21.15.92:0/927403344 conn(0x7febd0068e20 msgr2=0x7febd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd641f700 1 -- 172.21.15.92:0/927403344 shutdown_connections 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.713+0000 7febd641f700 1 -- 172.21.15.92:0/927403344 wait complete. 2023-10-06T03:07:28.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.714+0000 7febd641f700 1 Processor -- start 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.714+0000 7febd641f700 1 -- start start 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd641f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd00b1580 0x7febd00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd641f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febd00b1950 0x7febd00cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd641f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 0x7febd0145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd641f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd0154bb0 con 0x7febd0144df0 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd641f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd01457a0 con 0x7febd00b1950 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd641f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd00cf3d0 con 0x7febd00b1580 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd49bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 0x7febd0145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd49bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 0x7febd0145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37300/0 (socket says 172.21.15.92:37300) 2023-10-06T03:07:28.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febd49bc700 1 -- 172.21.15.92:0/3626760180 learned_addr learned my addr 172.21.15.92:0/3626760180 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febcf7fe700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febd00b1950 0x7febd00cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.715+0000 7febcffff700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd00b1580 0x7febd00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd49bc700 1 -- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd00b1580 msgr2=0x7febd00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd49bc700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd00b1580 0x7febd00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd49bc700 1 -- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febd00b1950 msgr2=0x7febd00cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd49bc700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febd00b1950 0x7febd00cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd49bc700 1 -- 172.21.15.92:0/3626760180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00cf6b0 con 0x7febd0144df0 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febcf7fe700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febd00b1950 0x7febd00cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd49bc700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 0x7febd0145260 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7febc800a430 tx=0x7febc8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80070d0 con 0x7febd0144df0 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc8007230 con 0x7febd0144df0 2023-10-06T03:07:28.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd00cf990 con 0x7febd0144df0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80075f0 con 0x7febd0144df0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.716+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd00cd0c0 con 0x7febd0144df0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.717+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd00b3620 con 0x7febd0144df0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.717+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7febc8019030 con 0x7febd0144df0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.717+0000 7febb7fff700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7febbc067230 0x7febbc0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.718+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7febc8086800 con 0x7febd0144df0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.718+0000 7febcffff700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7febbc067230 0x7febbc0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.718+0000 7febcffff700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7febbc067230 0x7febbc0696e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7febc0006f00 tx=0x7febc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:28.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.722+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febd00b3620 con 0x7febd0144df0 2023-10-06T03:07:28.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.905+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febd00b3620 con 0x7febd0144df0 2023-10-06T03:07:28.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.906+0000 7febb7fff700 1 -- 172.21.15.92:0/3626760180 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16919 (secure 0 0 0) 0x7febd00b3620 con 0x7febd0144df0 2023-10-06T03:07:28.907 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:28.907 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:24.879999+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:15.031911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:16.347808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:28.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7febbc067230 msgr2=0x7febbc0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7febbc067230 0x7febbc0696e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7febc0006f00 tx=0x7febc0008040 comp rx=0 tx=0).stop 2023-10-06T03:07:28.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 msgr2=0x7febd0145260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 0x7febd0145260 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7febc800a430 tx=0x7febc8015000 comp rx=0 tx=0).stop 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 shutdown_connections 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7febbc067230 0x7febbc0696e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7febd0144df0 0x7febd0145260 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febd00b1580 0x7febd00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 --2- 172.21.15.92:0/3626760180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febd00b1950 0x7febd00cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.908+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 >> 172.21.15.92:0/3626760180 conn(0x7febd0068e20 msgr2=0x7febd0158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.909+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 shutdown_connections 2023-10-06T03:07:28.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:28.909+0000 7febd641f700 1 -- 172.21.15.92:0/3626760180 wait complete. 2023-10-06T03:07:28.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:07:15.031911+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:07:16.347808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:07:28.922 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:07:29.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.038+0000 7f11f1ccc700 1 Processor -- start 2023-10-06T03:07:29.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.038+0000 7f11f1ccc700 1 -- start start 2023-10-06T03:07:29.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f1ccc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f11ec0b16d0 0x7f11ec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f1ccc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11ec0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f1ccc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec0b2080 con 0x7f11ec0b4e70 2023-10-06T03:07:29.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f1ccc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec0cfb00 con 0x7f11ec0b16d0 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f1ccc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b5c40 con 0x7f11ec0b33b0 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f0cca700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f11ec0b4e70 0x7f11ec0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46970/0 (socket says 172.21.15.92:46970) 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f0cca700 1 -- 172.21.15.92:0/3792062756 learned_addr learned my addr 172.21.15.92:0/3792062756 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11ebfff700 1 --2- 172.21.15.92:0/3792062756 >> v2:172.21.15.136:3300/0 conn(0x7f11ec0b16d0 0x7f11ec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f14cb700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11ec0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544869070 0 0) 0x7f11ec0b2080 con 0x7f11ec0b4e70 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d4003570 con 0x7f11ec0b4e70 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030248089 0 0) 0x7f11d4003570 con 0x7f11ec0b4e70 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 msgr2=0x7f11ec0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11ec0b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 >> v2:172.21.15.136:3300/0 conn(0x7f11ec0b16d0 msgr2=0x7f11ec0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:29.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 --2- 172.21.15.92:0/3792062756 >> v2:172.21.15.136:3300/0 conn(0x7f11ec0b16d0 0x7f11ec0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0b5f20 con 0x7f11ec0b4e70 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f11dc0017f0 con 0x7f11ec0b4e70 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11f14cb700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11ec0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 >> v1:172.21.15.92:6789/0 conn(0x7f11ec0b4e70 legacy=0x7f11ec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11d4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d4003fa0 0x7f11d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4004d50 0x7f11d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0cfb00 con 0x7f11d4003fa0 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec0b5c40 con 0x7f11d4004d50 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d400bae0 con 0x7f11ec0b33b0 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.039+0000 7f11ebfff700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d4003fa0 0x7f11d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f0cca700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11d4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f14cb700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4004d50 0x7f11d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11ebfff700 1 -- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 msgr2=0x7f11d4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11ebfff700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0b33b0 0x7f11d4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11ebfff700 1 -- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4004d50 msgr2=0x7f11d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11ebfff700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4004d50 0x7f11d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11ebfff700 1 -- 172.21.15.92:0/3792062756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4009420 con 0x7f11d4003fa0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11ebfff700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d4003fa0 0x7f11d4004810 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009c00 tx=0x7f11e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234b6b11cb9a614 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000e710 con 0x7f11d4003fa0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e000e870 con 0x7f11d4003fa0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11eb7fe700 1 -- 172.21.15.92:0/3792062756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e000ec30 con 0x7f11d4003fa0 2023-10-06T03:07:29.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f1ccc700 1 -- 172.21.15.92:0/3792062756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d4003fa0 msgr2=0x7f11d4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/3792062756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11d4003fa0 0x7f11d4004810 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009c00 tx=0x7f11e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f14cb700 1 -- 172.21.15.92:0/3792062756 reap_dead start 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f1ccc700 1 -- 172.21.15.92:0/3792062756 shutdown_connections 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f1ccc700 1 -- 172.21.15.92:0/3792062756 >> 172.21.15.92:0/3792062756 conn(0x7f11ec0d10c0 msgr2=0x7f11ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f1ccc700 1 -- 172.21.15.92:0/3792062756 shutdown_connections 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.040+0000 7f11f1ccc700 1 -- 172.21.15.92:0/3792062756 wait complete. 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.041+0000 7f11f1ccc700 1 Processor -- start 2023-10-06T03:07:29.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 -- start start 2023-10-06T03:07:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4003fa0 0x7f11ec16d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0cc940 0x7f11ec16d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 0x7f11ec0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec167870 con 0x7f11ec0c8800 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec16ae00 con 0x7f11d4003fa0 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.042+0000 7f11f1ccc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec16b0e0 con 0x7f11ec0cc940 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 0x7f11ec0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 0x7f11ec0c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37314/0 (socket says 172.21.15.92:37314) 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 -- 172.21.15.92:0/2667020965 learned_addr learned my addr 172.21.15.92:0/2667020965 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11ebfff700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0cc940 0x7f11ec16d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f0cca700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4003fa0 0x7f11ec16d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 -- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0cc940 msgr2=0x7f11ec16d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0cc940 0x7f11ec16d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 -- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4003fa0 msgr2=0x7f11ec16d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4003fa0 0x7f11ec16d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 -- 172.21.15.92:0/2667020965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec16b3c0 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f14cb700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 0x7f11ec0c8c80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f11ec071410 tx=0x7f11e4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4015670 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e40157d0 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4015b70 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec16b6a0 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.043+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec0c91c0 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.044+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec16bbc0 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.044+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f11e4013410 con 0x7f11ec0c8800 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.045+0000 7f11e8ff9700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d8067200 0x7f11d80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.045+0000 7f11f0cca700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d8067200 0x7f11d80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.045+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f11e4084ca0 con 0x7f11ec0c8800 2023-10-06T03:07:29.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.045+0000 7f11f0cca700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d8067200 0x7f11d80696b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0014a0 tx=0x7f11dc006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.049+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11ec16bbc0 con 0x7f11ec0c8800 2023-10-06T03:07:29.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.231+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11ec16bbc0 con 0x7f11ec0c8800 2023-10-06T03:07:29.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.232+0000 7f11e8ff9700 1 -- 172.21.15.92:0/2667020965 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11ec16bbc0 con 0x7f11ec0c8800 2023-10-06T03:07:29.233 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d8067200 msgr2=0x7f11d80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d8067200 0x7f11d80696b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0014a0 tx=0x7f11dc006040 comp rx=0 tx=0).stop 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 msgr2=0x7f11ec0c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 0x7f11ec0c8c80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f11ec071410 tx=0x7f11e4008040 comp rx=0 tx=0).stop 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 shutdown_connections 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11d8067200 0x7f11d80696b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11ec0c8800 0x7f11ec0c8c80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11d4003fa0 0x7f11ec16d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 --2- 172.21.15.92:0/2667020965 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11ec0cc940 0x7f11ec16d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 >> 172.21.15.92:0/2667020965 conn(0x7f11ec0d10c0 msgr2=0x7f11ec068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:29.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 shutdown_connections 2023-10-06T03:07:29.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.234+0000 7f11f1ccc700 1 -- 172.21.15.92:0/2667020965 wait complete. 2023-10-06T03:07:29.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:07:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.364+0000 7fb787fc3700 1 Processor -- start 2023-10-06T03:07:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb787fc3700 1 -- start start 2023-10-06T03:07:29.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb787fc3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780069560 0x7fb7800699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb787fc3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb780069ee0 con 0x7fb780159890 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb787fc3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78015a160 con 0x7fb780159c60 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb787fc3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800d13a0 con 0x7fb780069560 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb785d5f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb780159890 0x7fb78015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32952/0 (socket says 172.21.15.92:32952) 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.365+0000 7fb785d5f700 1 -- 172.21.15.92:0/2243456170 learned_addr learned my addr 172.21.15.92:0/2243456170 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb786560700 1 --2- 172.21.15.92:0/2243456170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780069560 0x7fb7800699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482810185 0 0) 0x7fb780069ee0 con 0x7fb780159890 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb76c003570 con 0x7fb780159890 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847284057 0 0) 0x7fb78015a160 con 0x7fb780159c60 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb780069ee0 con 0x7fb780159c60 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb786560700 1 -- 172.21.15.92:0/2243456170 >> v1:172.21.15.136:6789/0 conn(0x7fb780159c60 legacy=0x7fb78015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb786560700 1 -- 172.21.15.92:0/2243456170 >> v1:172.21.15.92:6789/0 conn(0x7fb780159890 legacy=0x7fb78015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287719763 0 0) 0x7fb76c003570 con 0x7fb780159890 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb786560700 1 -- 172.21.15.92:0/2243456170 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78015e140 con 0x7fb780069560 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb786560700 1 --2- 172.21.15.92:0/2243456170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780069560 0x7fb7800699a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb778009910 tx=0x7fb7780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e888cf02397d9d3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0017f0 con 0x7fb780069560 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778011420 con 0x7fb780069560 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.366+0000 7fb784d5d700 1 -- 172.21.15.92:0/2243456170 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7780117c0 con 0x7fb780069560 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 -- 172.21.15.92:0/2243456170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780069560 msgr2=0x7fb7800699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 --2- 172.21.15.92:0/2243456170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780069560 0x7fb7800699a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb778009910 tx=0x7fb7780091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 -- 172.21.15.92:0/2243456170 shutdown_connections 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 --2- 172.21.15.92:0/2243456170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780069560 0x7fb7800699a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 -- 172.21.15.92:0/2243456170 >> 172.21.15.92:0/2243456170 conn(0x7fb780068e20 msgr2=0x7fb7800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:29.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 -- 172.21.15.92:0/2243456170 shutdown_connections 2023-10-06T03:07:29.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.367+0000 7fb787fc3700 1 -- 172.21.15.92:0/2243456170 wait complete. 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.368+0000 7fb787fc3700 1 Processor -- start 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.368+0000 7fb787fc3700 1 -- start start 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb780069560 0x7fb7801540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780159890 0x7fb7800b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 0x7fb78015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015c510 con 0x7fb780159c60 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015bf50 con 0x7fb780069560 2023-10-06T03:07:29.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78015c230 con 0x7fb780159890 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 0x7fb78015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 0x7fb78015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40920/0 (socket says 172.21.15.92:40920) 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 -- 172.21.15.92:0/3928188708 learned_addr learned my addr 172.21.15.92:0/3928188708 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb785d5f700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb780069560 0x7fb7801540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb78555e700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780159890 0x7fb7800b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 -- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780159890 msgr2=0x7fb7800b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780159890 0x7fb7800b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 -- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb780069560 msgr2=0x7fb7801540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb780069560 0x7fb7801540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 -- 172.21.15.92:0/3928188708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778009420 con 0x7fb780159c60 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb786560700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 0x7fb78015b4d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb778009860 tx=0x7fb778016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77801cd40 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77801a450 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.369+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800ce7e0 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.370+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77801a810 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.370+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78015d460 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.371+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7800ced00 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.371+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb77801ac40 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.371+0000 7fb775ffb700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb764067200 0x7fb7640696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.371+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb778086ab0 con 0x7fb780159c60 2023-10-06T03:07:29.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.371+0000 7fb785d5f700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb764067200 0x7fb7640696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.372+0000 7fb785d5f700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb764067200 0x7fb7640696b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001920 tx=0x7fb77c000f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.376+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7800ced00 con 0x7fb780159c60 2023-10-06T03:07:29.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.559+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb7800ced00 con 0x7fb780159c60 2023-10-06T03:07:29.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.559+0000 7fb775ffb700 1 -- 172.21.15.92:0/3928188708 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb7800ced00 con 0x7fb780159c60 2023-10-06T03:07:29.560 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb764067200 msgr2=0x7fb7640696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb764067200 0x7fb7640696b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb77c001920 tx=0x7fb77c000f40 comp rx=0 tx=0).stop 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 msgr2=0x7fb78015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 0x7fb78015b4d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb778009860 tx=0x7fb778016000 comp rx=0 tx=0).stop 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 shutdown_connections 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb764067200 0x7fb7640696b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb780159c60 0x7fb78015b4d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb780069560 0x7fb7801540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 --2- 172.21.15.92:0/3928188708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb780159890 0x7fb7800b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 >> 172.21.15.92:0/3928188708 conn(0x7fb780068e20 msgr2=0x7fb7800c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 shutdown_connections 2023-10-06T03:07:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.562+0000 7fb787fc3700 1 -- 172.21.15.92:0/3928188708 wait complete. 2023-10-06T03:07:29.574 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:07:29.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.691+0000 7f306f36c700 1 Processor -- start 2023-10-06T03:07:29.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.691+0000 7f306f36c700 1 -- start start 2023-10-06T03:07:29.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306f36c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c93a0 0x7f306806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306f36c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680cb5d0 con 0x7f30680c7fd0 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306f36c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680af490 con 0x7f30680c9770 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306f36c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680af770 con 0x7f30680c93a0 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306eb6b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f30680c7fd0 0x7f30680c8410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32966/0 (socket says 172.21.15.92:32966) 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306eb6b700 1 -- 172.21.15.92:0/2471427669 learned_addr learned my addr 172.21.15.92:0/2471427669 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.692+0000 7f306e36a700 1 --2- 172.21.15.92:0/2471427669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c93a0 0x7f306806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315233592 0 0) 0x7f30680cb5d0 con 0x7f30680c7fd0 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3054003570 con 0x7f30680c7fd0 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861688322 0 0) 0x7f30680af490 con 0x7f30680c9770 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30680cb5d0 con 0x7f30680c9770 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633991814 0 0) 0x7f3054003570 con 0x7f30680c7fd0 2023-10-06T03:07:29.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306e36a700 1 -- 172.21.15.92:0/2471427669 >> v1:172.21.15.136:6789/0 conn(0x7f30680c9770 legacy=0x7f30680caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306e36a700 1 -- 172.21.15.92:0/2471427669 >> v1:172.21.15.92:6789/0 conn(0x7f30680c7fd0 legacy=0x7f30680c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306e36a700 1 -- 172.21.15.92:0/2471427669 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30680afa50 con 0x7f30680c93a0 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306e36a700 1 --2- 172.21.15.92:0/2471427669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c93a0 0x7f306806c980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3064009910 tx=0x7f30640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c997d4c2f7c6a8a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30600017f0 con 0x7f30680c93a0 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3064011420 con 0x7f30680c93a0 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.693+0000 7f306d368700 1 -- 172.21.15.92:0/2471427669 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640117c0 con 0x7f30680c93a0 2023-10-06T03:07:29.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 -- 172.21.15.92:0/2471427669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c93a0 msgr2=0x7f306806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 --2- 172.21.15.92:0/2471427669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c93a0 0x7f306806c980 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3064009910 tx=0x7f30640091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 -- 172.21.15.92:0/2471427669 shutdown_connections 2023-10-06T03:07:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 --2- 172.21.15.92:0/2471427669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c93a0 0x7f306806c980 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 -- 172.21.15.92:0/2471427669 >> 172.21.15.92:0/2471427669 conn(0x7f30680cddc0 msgr2=0x7f30680c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 -- 172.21.15.92:0/2471427669 shutdown_connections 2023-10-06T03:07:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.694+0000 7f306f36c700 1 -- 172.21.15.92:0/2471427669 wait complete. 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.695+0000 7f306f36c700 1 Processor -- start 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.695+0000 7f306f36c700 1 -- start start 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 0x7f3068068550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30680c93a0 0x7f3068144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c9770 0x7f3068145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680c5050 con 0x7f30680c7fd0 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680c4440 con 0x7f30680c93a0 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680c4720 con 0x7f30680c9770 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 0x7f3068068550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 0x7f3068068550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40934/0 (socket says 172.21.15.92:40934) 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 -- 172.21.15.92:0/1753822755 learned_addr learned my addr 172.21.15.92:0/1753822755 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306db69700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30680c93a0 0x7f3068144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306eb6b700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c9770 0x7f3068145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 -- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c9770 msgr2=0x7f3068145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c9770 0x7f3068145060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 -- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30680c93a0 msgr2=0x7f3068144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30680c93a0 0x7f3068144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 -- 172.21.15.92:0/1753822755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009420 con 0x7f30680c7fd0 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306eb6b700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c9770 0x7f3068145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306e36a700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 0x7f3068068550 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f30640030c0 tx=0x7f3064016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306401cd60 con 0x7f30680c7fd0 2023-10-06T03:07:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306401a450 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306401a810 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30680c4a00 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.696+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068068a90 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.697+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30680c5330 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.698+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f306401ac40 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.698+0000 7f305e7fc700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f304c067200 0x7f304c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.698+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f3064086a20 con 0x7f30680c7fd0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.698+0000 7f306db69700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f304c067200 0x7f304c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.699+0000 7f306db69700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f304c067200 0x7f304c0696b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f306816b700 tx=0x7f3058009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.702+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30640537c0 con 0x7f30680c7fd0 2023-10-06T03:07:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.886+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3068068cb0 con 0x7f30680c7fd0 2023-10-06T03:07:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.886+0000 7f305e7fc700 1 -- 172.21.15.92:0/1753822755 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3064056de0 con 0x7f30680c7fd0 2023-10-06T03:07:29.887 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:07:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f304c067200 msgr2=0x7f304c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f304c067200 0x7f304c0696b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f306816b700 tx=0x7f3058009040 comp rx=0 tx=0).stop 2023-10-06T03:07:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 msgr2=0x7f3068068550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 0x7f3068068550 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f30640030c0 tx=0x7f3064016000 comp rx=0 tx=0).stop 2023-10-06T03:07:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 shutdown_connections 2023-10-06T03:07:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f304c067200 0x7f304c0696b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30680c7fd0 0x7f3068068550 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30680c9770 0x7f3068145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 --2- 172.21.15.92:0/1753822755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30680c93a0 0x7f3068144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 >> 172.21.15.92:0/1753822755 conn(0x7f30680cddc0 msgr2=0x7f30681550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 shutdown_connections 2023-10-06T03:07:29.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:29.888+0000 7f306f36c700 1 -- 172.21.15.92:0/1753822755 wait complete. 2023-10-06T03:07:29.902 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:07:30.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.019+0000 7fd493f68700 1 Processor -- start 2023-10-06T03:07:30.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.019+0000 7fd493f68700 1 -- start start 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd493f68700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd48c069560 0x7fd48c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd493f68700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c069ee0 con 0x7fd48c159c60 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd493f68700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c15a160 con 0x7fd48c159890 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd493f68700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0d13a0 con 0x7fd48c069560 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd491503700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd48c159c60 0x7fd48c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32968/0 (socket says 172.21.15.92:32968) 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd491503700 1 -- 172.21.15.92:0/673445296 learned_addr learned my addr 172.21.15.92:0/673445296 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.020+0000 7fd492505700 1 --2- 172.21.15.92:0/673445296 >> v2:172.21.15.136:3300/0 conn(0x7fd48c069560 0x7fd48c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135874528 0 0) 0x7fd48c069ee0 con 0x7fd48c159c60 2023-10-06T03:07:30.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd478003570 con 0x7fd48c159c60 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502893023 0 0) 0x7fd48c15a160 con 0x7fd48c159890 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd48c069ee0 con 0x7fd48c159890 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010560196 0 0) 0x7fd478003570 con 0x7fd48c159c60 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 >> v2:172.21.15.136:3300/0 conn(0x7fd48c069560 msgr2=0x7fd48c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 --2- 172.21.15.92:0/673445296 >> v2:172.21.15.136:3300/0 conn(0x7fd48c069560 0x7fd48c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 >> v1:172.21.15.136:6789/0 conn(0x7fd48c159890 legacy=0x7fd48c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c15e140 con 0x7fd48c159c60 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd47c001470 con 0x7fd48c159c60 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 >> v1:172.21.15.92:6789/0 conn(0x7fd48c159c60 legacy=0x7fd48c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd478003bf0 0x7fd478003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd478004810 0x7fd478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780051c0 0x7fd47800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c0d13a0 con 0x7fd4780051c0 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47800c0b0 con 0x7fd478003bf0 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47800c270 con 0x7fd478004810 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780051c0 0x7fd47800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd491d04700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd478003bf0 0x7fd478003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd491503700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd478004810 0x7fd478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 -- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd478004810 msgr2=0x7fd478004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd478004810 0x7fd478004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 -- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd478003bf0 msgr2=0x7fd478003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd478003bf0 0x7fd478003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 -- 172.21.15.92:0/673445296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478003570 con 0x7fd4780051c0 2023-10-06T03:07:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd491503700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd478004810 0x7fd478004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.021+0000 7fd492505700 1 -- 172.21.15.92:0/673445296 reap_dead start 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd492505700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780051c0 0x7fd47800ba60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd48400e210 tx=0x7fd48400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9cbc7639c1ec6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48400eac0 con 0x7fd4780051c0 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48400f040 con 0x7fd4780051c0 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd490d02700 1 -- 172.21.15.92:0/673445296 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48400ee70 con 0x7fd4780051c0 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 -- 172.21.15.92:0/673445296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780051c0 msgr2=0x7fd47800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780051c0 0x7fd47800ba60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd48400e210 tx=0x7fd48400b040 comp rx=0 tx=0).stop 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 -- 172.21.15.92:0/673445296 shutdown_connections 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 --2- 172.21.15.92:0/673445296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4780051c0 0x7fd47800ba60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 -- 172.21.15.92:0/673445296 >> 172.21.15.92:0/673445296 conn(0x7fd48c068e20 msgr2=0x7fd48c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 -- 172.21.15.92:0/673445296 shutdown_connections 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.022+0000 7fd493f68700 1 -- 172.21.15.92:0/673445296 wait complete. 2023-10-06T03:07:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.023+0000 7fd493f68700 1 Processor -- start 2023-10-06T03:07:30.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.023+0000 7fd493f68700 1 -- start start 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd493f68700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4780051c0 0x7fd48c14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd493f68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 0x7fd48c16a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd493f68700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd48c0cdf40 0x7fd48c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd493f68700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c14f710 con 0x7fd48c0b61b0 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd493f68700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c154740 con 0x7fd4780051c0 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd493f68700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c153b70 con 0x7fd48c0cdf40 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 0x7fd48c16a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 0x7fd48c16a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40950/0 (socket says 172.21.15.92:40950) 2023-10-06T03:07:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 -- 172.21.15.92:0/2762383414 learned_addr learned my addr 172.21.15.92:0/2762383414 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491d04700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4780051c0 0x7fd48c14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd492505700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd48c0cdf40 0x7fd48c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 -- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd48c0cdf40 msgr2=0x7fd48c0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd48c0cdf40 0x7fd48c0b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 -- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4780051c0 msgr2=0x7fd48c14f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4780051c0 0x7fd48c14f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.024+0000 7fd491503700 1 -- 172.21.15.92:0/2762383414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c153e10 con 0x7fd48c0b61b0 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.025+0000 7fd491503700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 0x7fd48c16a9e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e780 tx=0x7fd47c000cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.025+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c010490 con 0x7fd48c0b61b0 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.025+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c010040 con 0x7fd48c0b61b0 2023-10-06T03:07:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.025+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c1540f0 con 0x7fd48c0b61b0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.025+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c010800 con 0x7fd48c0b61b0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.025+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c16af20 con 0x7fd48c0b61b0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.026+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c04c3c0 con 0x7fd48c0b61b0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.026+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd47c010bb0 con 0x7fd48c0b61b0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.027+0000 7fd481ffb700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470067290 0x7fd470069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.027+0000 7fd491d04700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470067290 0x7fd470069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.027+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fd47c083e90 con 0x7fd48c0b61b0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.027+0000 7fd491d04700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470067290 0x7fd470069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd488000c00 tx=0x7fd488007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.031+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd47c04d5a0 con 0x7fd48c0b61b0 2023-10-06T03:07:30.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.215+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd48c154880 con 0x7fd48c0b61b0 2023-10-06T03:07:30.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.215+0000 7fd481ffb700 1 -- 172.21.15.92:0/2762383414 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd47c019070 con 0x7fd48c0b61b0 2023-10-06T03:07:30.216 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:07:30.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.217+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470067290 msgr2=0x7fd470069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.217+0000 7fd493f68700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470067290 0x7fd470069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd488000c00 tx=0x7fd488007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 msgr2=0x7fd48c16a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 0x7fd48c16a9e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e780 tx=0x7fd47c000cf0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 shutdown_connections 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd470067290 0x7fd470069740 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd48c0cdf40 0x7fd48c0b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4780051c0 0x7fd48c14f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 --2- 172.21.15.92:0/2762383414 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd48c0b61b0 0x7fd48c16a9e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 >> 172.21.15.92:0/2762383414 conn(0x7fd48c068e20 msgr2=0x7fd48c0b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 shutdown_connections 2023-10-06T03:07:30.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.218+0000 7fd493f68700 1 -- 172.21.15.92:0/2762383414 wait complete. 2023-10-06T03:07:30.230 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:30.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.344+0000 7f7ec4cdb700 1 Processor -- start 2023-10-06T03:07:30.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.344+0000 7f7ec4cdb700 1 -- start start 2023-10-06T03:07:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.344+0000 7f7ec4cdb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ec0159890 0x7f7ec015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ec4cdb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec0159c60 0x7f7ec015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ec4cdb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7ec015db10 0x7f7ec0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ec4cdb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec015a160 con 0x7f7ec0159890 2023-10-06T03:07:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ec4cdb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec00d13a0 con 0x7f7ec015db10 2023-10-06T03:07:30.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ec4cdb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec015e140 con 0x7f7ec0159c60 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ec0159890 0x7f7ec015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7ec0159890 0x7f7ec015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40958/0 (socket says 172.21.15.92:40958) 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 -- 172.21.15.92:0/3803024578 learned_addr learned my addr 172.21.15.92:0/3803024578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 -- 172.21.15.92:0/3803024578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec0159c60 msgr2=0x7f7ec015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7eb5d9b700 1 --2- 172.21.15.92:0/3803024578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec0159c60 0x7f7ec015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebed9d700 1 --2- 172.21.15.92:0/3803024578 >> v2:172.21.15.136:3300/0 conn(0x7f7ec015db10 0x7f7ec0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 --2- 172.21.15.92:0/3803024578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec0159c60 0x7f7ec015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 -- 172.21.15.92:0/3803024578 >> v2:172.21.15.136:3300/0 conn(0x7f7ec015db10 msgr2=0x7f7ec0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 --2- 172.21.15.92:0/3803024578 >> v2:172.21.15.136:3300/0 conn(0x7f7ec015db10 0x7f7ec0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 -- 172.21.15.92:0/3803024578 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec015a3d0 con 0x7f7ec0159890 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebed9d700 1 --2- 172.21.15.92:0/3803024578 >> v2:172.21.15.136:3300/0 conn(0x7f7ec015db10 0x7f7ec0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:30.348 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebe59c700 1 --2- 172.21.15.92:0/3803024578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec015abd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0089b50 tx=0x7f7eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fe8c08913e2467 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebdd9b700 1 -- 172.21.15.92:0/3803024578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000b860 con 0x7f7ec0159890 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebdd9b700 1 -- 172.21.15.92:0/3803024578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb000b9c0 con 0x7f7ec0159890 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.345+0000 7f7ebdd9b700 1 -- 172.21.15.92:0/3803024578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000bd60 con 0x7f7ec0159890 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/3803024578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 msgr2=0x7f7ec015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/3803024578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec015abd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0089b50 tx=0x7f7eb0009250 comp rx=0 tx=0).stop 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/3803024578 shutdown_connections 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/3803024578 >> v2:172.21.15.136:3300/0 conn(0x7f7ec015db10 0x7f7ec0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/3803024578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec015abd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/3803024578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec0159c60 0x7f7ec015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/3803024578 >> 172.21.15.92:0/3803024578 conn(0x7f7ec0068e20 msgr2=0x7f7ec00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/3803024578 shutdown_connections 2023-10-06T03:07:30.349 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.346+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/3803024578 wait complete. 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.347+0000 7f7ec4cdb700 1 Processor -- start 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.347+0000 7f7ec4cdb700 1 -- start start 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ec4cdb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec0142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ec4cdb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ec0159c60 0x7f7ec00c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ec4cdb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec015db10 0x7f7ec00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ec4cdb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00cdd30 con 0x7f7ec0159890 2023-10-06T03:07:30.352 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ec4cdb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00cd160 con 0x7f7ec0159c60 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ec4cdb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec00cd400 con 0x7f7ec015db10 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec0142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec0142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40972/0 (socket says 172.21.15.92:40972) 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 -- 172.21.15.92:0/662507955 learned_addr learned my addr 172.21.15.92:0/662507955 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7eb5d9b700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ec0159c60 0x7f7ec00c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebed9d700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec015db10 0x7f7ec00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 -- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec015db10 msgr2=0x7f7ec00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec015db10 0x7f7ec00c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 -- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ec0159c60 msgr2=0x7f7ec00c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ec0159c60 0x7f7ec00c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 -- 172.21.15.92:0/662507955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00094f0 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7eb5d9b700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ec0159c60 0x7f7ec00c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.348+0000 7f7ebe59c700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec0142100 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0070e20 tx=0x7f7eb0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.349+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0024440 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.349+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb00245a0 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.349+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec00cd6e0 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.349+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0024960 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.349+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec0142640 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.350+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec00ce010 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.350+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7eb001a420 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.350+0000 7f7eb6ffd700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ea0067290 0x7f7ea0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.350+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7eb0086a10 con 0x7f7ec0159890 2023-10-06T03:07:30.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.350+0000 7f7eb5d9b700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ea0067290 0x7f7ea0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.351+0000 7f7eb5d9b700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ea0067290 0x7f7ea0069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00b0330 tx=0x7f7ea8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.355+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7eb00536f0 con 0x7f7ec0159890 2023-10-06T03:07:30.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.578+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec004c3c0 con 0x7f7ec0159890 2023-10-06T03:07:30.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.578+0000 7f7eb6ffd700 1 -- 172.21.15.92:0/662507955 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7eb0056d10 con 0x7f7ec0159890 2023-10-06T03:07:30.579 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:30.579 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:17.381585+0000","modified":"2023-10-06T03:07:21.036747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6480":{"gid":6480,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/428935748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":428935748},{"type":"v1","addr":"172.21.15.92:6837","nonce":428935748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ea0067290 msgr2=0x7f7ea0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ea0067290 0x7f7ea0069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00b0330 tx=0x7f7ea8008040 comp rx=0 tx=0).stop 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 msgr2=0x7f7ec0142100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec0142100 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0070e20 tx=0x7f7eb0017000 comp rx=0 tx=0).stop 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 shutdown_connections 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7ea0067290 0x7f7ea0069740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ec0159890 0x7f7ec0142100 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ec015db10 0x7f7ec00c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 --2- 172.21.15.92:0/662507955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7ec0159c60 0x7f7ec00c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 >> 172.21.15.92:0/662507955 conn(0x7f7ec0068e20 msgr2=0x7f7ec0153850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 shutdown_connections 2023-10-06T03:07:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.580+0000 7f7ec4cdb700 1 -- 172.21.15.92:0/662507955 wait complete. 2023-10-06T03:07:30.582 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 57 2023-10-06T03:07:30.593 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:30.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.707+0000 7f40eb70e700 1 Processor -- start 2023-10-06T03:07:30.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.707+0000 7f40eb70e700 1 -- start start 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40eb70e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f40e4159890 0x7f40e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40eb70e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40e4159c60 0x7f40e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40eb70e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e415a160 con 0x7f40e415db10 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40eb70e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e415df50 con 0x7f40e4159c60 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40eb70e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40e40d13a0 con 0x7f40e4159890 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e8ca9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f40e4159c60 0x7f40e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e9cab700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f40e415db10 0x7f40e4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32970/0 (socket says 172.21.15.92:32970) 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e9cab700 1 -- 172.21.15.92:0/2965006177 learned_addr learned my addr 172.21.15.92:0/2965006177 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e94aa700 1 --2- 172.21.15.92:0/2965006177 >> v2:172.21.15.136:3300/0 conn(0x7f40e4159890 0x7f40e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e8ca9700 1 -- 172.21.15.92:0/2965006177 >> v2:172.21.15.136:3300/0 conn(0x7f40e4159890 msgr2=0x7f40e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e8ca9700 1 --2- 172.21.15.92:0/2965006177 >> v2:172.21.15.136:3300/0 conn(0x7f40e4159890 0x7f40e415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e8ca9700 1 -- 172.21.15.92:0/2965006177 >> v1:172.21.15.92:6789/0 conn(0x7f40e415db10 legacy=0x7f40e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e8ca9700 1 -- 172.21.15.92:0/2965006177 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e415e140 con 0x7f40e4159c60 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40e8ca9700 1 --2- 172.21.15.92:0/2965006177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e415d5d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d1090 tx=0x7f40d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed1a7bed171ed89 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40dbfff700 1 -- 172.21.15.92:0/2965006177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400b6b0 con 0x7f40e4159c60 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40dbfff700 1 -- 172.21.15.92:0/2965006177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d400b810 con 0x7f40e4159c60 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.708+0000 7f40dbfff700 1 -- 172.21.15.92:0/2965006177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400bbb0 con 0x7f40e4159c60 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 -- 172.21.15.92:0/2965006177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 msgr2=0x7f40e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 --2- 172.21.15.92:0/2965006177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e415d5d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d1090 tx=0x7f40d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 -- 172.21.15.92:0/2965006177 shutdown_connections 2023-10-06T03:07:30.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 --2- 172.21.15.92:0/2965006177 >> v2:172.21.15.136:3300/0 conn(0x7f40e4159890 0x7f40e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 --2- 172.21.15.92:0/2965006177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e415d5d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 -- 172.21.15.92:0/2965006177 >> 172.21.15.92:0/2965006177 conn(0x7f40e4068e20 msgr2=0x7f40e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 -- 172.21.15.92:0/2965006177 shutdown_connections 2023-10-06T03:07:30.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.709+0000 7f40eb70e700 1 -- 172.21.15.92:0/2965006177 wait complete. 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.710+0000 7f40eb70e700 1 Processor -- start 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.710+0000 7f40eb70e700 1 -- start start 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40eb70e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40e4159890 0x7f40e4145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40eb70e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e4153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40eb70e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e415db10 0x7f40e40c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40eb70e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40b4480 con 0x7f40e4159c60 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40eb70e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4153970 con 0x7f40e415db10 2023-10-06T03:07:30.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40eb70e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e40b3590 con 0x7f40e4159890 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e4153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e4153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40994/0 (socket says 172.21.15.92:40994) 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 -- 172.21.15.92:0/844200446 learned_addr learned my addr 172.21.15.92:0/844200446 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e94aa700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40e4159890 0x7f40e4145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e9cab700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e415db10 0x7f40e40c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 -- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40e4159890 msgr2=0x7f40e4145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40e4159890 0x7f40e4145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 -- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e415db10 msgr2=0x7f40e40c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.711+0000 7f40e8ca9700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e415db10 0x7f40e40c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40e8ca9700 1 -- 172.21.15.92:0/844200446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4009420 con 0x7f40e4159c60 2023-10-06T03:07:30.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40e8ca9700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e4153430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f40e4070e20 tx=0x7f40d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d400bb90 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d4011420 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40117a0 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e40b3870 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.712+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e40b3e30 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.713+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e40b45f0 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.713+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f40d4011b90 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.713+0000 7f40d97fa700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40c8067290 0x7f40c8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.714+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f40d4086580 con 0x7f40e4159c60 2023-10-06T03:07:30.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.714+0000 7f40e94aa700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40c8067290 0x7f40c8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:30.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.714+0000 7f40e94aa700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40c8067290 0x7f40c8069740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00ac20 tx=0x7f40dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:30.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.718+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40d4053260 con 0x7f40e4159c60 2023-10-06T03:07:30.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.899+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40e404c3c0 con 0x7f40e4159c60 2023-10-06T03:07:30.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.900+0000 7f40d97fa700 1 -- 172.21.15.92:0/844200446 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16919 (secure 0 0 0) 0x7f40d4056880 con 0x7f40e4159c60 2023-10-06T03:07:30.901 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:30.901 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:24.879999+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:15.031911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:16.347808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40c8067290 msgr2=0x7f40c8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40c8067290 0x7f40c8069740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00ac20 tx=0x7f40dc008040 comp rx=0 tx=0).stop 2023-10-06T03:07:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 msgr2=0x7f40e4153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e4153430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f40e4070e20 tx=0x7f40d4017000 comp rx=0 tx=0).stop 2023-10-06T03:07:30.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 shutdown_connections 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f40c8067290 0x7f40c8069740 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40e415db10 0x7f40e40c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40e4159890 0x7f40e4145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 --2- 172.21.15.92:0/844200446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f40e4159c60 0x7f40e4153430 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 >> 172.21.15.92:0/844200446 conn(0x7f40e4068e20 msgr2=0x7f40e41543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 shutdown_connections 2023-10-06T03:07:30.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:30.902+0000 7f40eb70e700 1 -- 172.21.15.92:0/844200446 wait complete. 2023-10-06T03:07:30.915 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:07:30.916 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.029+0000 7f726ef55700 1 Processor -- start 2023-10-06T03:07:31.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.029+0000 7f726ef55700 1 -- start start 2023-10-06T03:07:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f726ef55700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7268159c60 0x7f726815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f726ef55700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726815a160 con 0x7f7268159890 2023-10-06T03:07:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f726ef55700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680d13a0 con 0x7f726815db10 2023-10-06T03:07:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f726ef55700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726815df50 con 0x7f7268159c60 2023-10-06T03:07:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f726ccf1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7268159890 0x7f726815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32982/0 (socket says 172.21.15.92:32982) 2023-10-06T03:07:31.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f726ccf1700 1 -- 172.21.15.92:0/3374115736 learned_addr learned my addr 172.21.15.92:0/3374115736 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f7267fff700 1 --2- 172.21.15.92:0/3374115736 >> v2:172.21.15.136:3300/0 conn(0x7f7268159c60 0x7f726815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487859638 0 0) 0x7f726815a160 con 0x7f7268159890 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.030+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7258003570 con 0x7f7268159890 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 228365858 0 0) 0x7f7258003570 con 0x7f7268159890 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 >> v2:172.21.15.136:3300/0 conn(0x7f7268159c60 msgr2=0x7f726815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 --2- 172.21.15.92:0/3374115736 >> v2:172.21.15.136:3300/0 conn(0x7f7268159c60 0x7f726815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 >> v1:172.21.15.136:6789/0 conn(0x7f726815db10 legacy=0x7f7268069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726815e140 con 0x7f7268159890 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f725c001470 con 0x7f7268159890 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 >> v1:172.21.15.92:6789/0 conn(0x7f7268159890 legacy=0x7f726815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7258003a60 0x7f7258003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7258004810 0x7f7258004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72580051c0 0x7f725800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725800c0a0 con 0x7f72580051c0 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725800c210 con 0x7f7258004810 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725800c3d0 con 0x7f7258003a60 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72580051c0 0x7f725800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726ccf1700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7258003a60 0x7f7258003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f7267fff700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7258004810 0x7f7258004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 -- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7258003a60 msgr2=0x7f7258003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7258003a60 0x7f7258003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 -- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7258004810 msgr2=0x7f7258004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:31.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7258004810 0x7f7258004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 -- 172.21.15.92:0/3374115736 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258003570 con 0x7f72580051c0 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 -- 172.21.15.92:0/3374115736 reap_dead start 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f726d4f2700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72580051c0 0x7f725800ba60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7260001fc0 tx=0x7f7260001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13939a1030bd51c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260001950 con 0x7f72580051c0 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726000c8b0 con 0x7f72580051c0 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.031+0000 7f72677fe700 1 -- 172.21.15.92:0/3374115736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726000cc50 con 0x7f72580051c0 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 -- 172.21.15.92:0/3374115736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72580051c0 msgr2=0x7f725800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72580051c0 0x7f725800ba60 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7260001fc0 tx=0x7f7260001640 comp rx=0 tx=0).stop 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 -- 172.21.15.92:0/3374115736 shutdown_connections 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 --2- 172.21.15.92:0/3374115736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72580051c0 0x7f725800ba60 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 -- 172.21.15.92:0/3374115736 >> 172.21.15.92:0/3374115736 conn(0x7f7268068e20 msgr2=0x7f72680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:31.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 -- 172.21.15.92:0/3374115736 shutdown_connections 2023-10-06T03:07:31.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.032+0000 7f726ef55700 1 -- 172.21.15.92:0/3374115736 wait complete. 2023-10-06T03:07:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.033+0000 7f726ef55700 1 Processor -- start 2023-10-06T03:07:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.033+0000 7f726ef55700 1 -- start start 2023-10-06T03:07:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ef55700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72580051c0 0x7f726815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ef55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 0x7f72680cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ef55700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72680cd810 0x7f726816d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ef55700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726814cba0 con 0x7f72681554b0 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ef55700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680cdc90 con 0x7f72580051c0 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ef55700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726815cf50 con 0x7f72680cd810 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 0x7f72680cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 0x7f72680cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41004/0 (socket says 172.21.15.92:41004) 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 -- 172.21.15.92:0/574577412 learned_addr learned my addr 172.21.15.92:0/574577412 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ccf1700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72580051c0 0x7f726815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726d4f2700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72680cd810 0x7f726816d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 -- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72680cd810 msgr2=0x7f726816d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72680cd810 0x7f726816d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 -- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72580051c0 msgr2=0x7f726815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72580051c0 0x7f726815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 -- 172.21.15.92:0/574577412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726815bc50 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f726ccf1700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72580051c0 0x7f726815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.034+0000 7f7267fff700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 0x7f72680cd2d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7254011b10 tx=0x7f72540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.035+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400f990 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.035+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725400faf0 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.035+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726815c340 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.035+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400fe70 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.035+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726815c900 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.036+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726804c3c0 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.036+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7254014650 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.036+0000 7f7264ff9700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f724c067290 0x7f724c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.036+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f7254018030 con 0x7f72681554b0 2023-10-06T03:07:31.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.036+0000 7f726ccf1700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f724c067290 0x7f724c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.037+0000 7f726ccf1700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f724c067290 0x7f724c069740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f725c001ce0 tx=0x7f725c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.040+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7254055d80 con 0x7f72681554b0 2023-10-06T03:07:31.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.268+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f726806a5b0 con 0x7f72681554b0 2023-10-06T03:07:31.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.269+0000 7f7264ff9700 1 -- 172.21.15.92:0/574577412 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f7254055910 con 0x7f72681554b0 2023-10-06T03:07:31.270 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:31.270 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:17.381585+0000","modified":"2023-10-06T03:07:21.036747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6480":{"gid":6480,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/428935748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":428935748},{"type":"v1","addr":"172.21.15.92:6837","nonce":428935748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:07:31.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f724c067290 msgr2=0x7f724c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f724c067290 0x7f724c069740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f725c001ce0 tx=0x7f725c009040 comp rx=0 tx=0).stop 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 msgr2=0x7f72680cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 0x7f72680cd2d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7254011b10 tx=0x7f72540091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 shutdown_connections 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f724c067290 0x7f724c069740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72680cd810 0x7f726816d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72580051c0 0x7f726815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 --2- 172.21.15.92:0/574577412 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72681554b0 0x7f72680cd2d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 >> 172.21.15.92:0/574577412 conn(0x7f7268068e20 msgr2=0x7f72680c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:31.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 shutdown_connections 2023-10-06T03:07:31.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.271+0000 7f726ef55700 1 -- 172.21.15.92:0/574577412 wait complete. 2023-10-06T03:07:31.273 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 57 2023-10-06T03:07:31.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:31.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.398+0000 7f338a6a4700 1 Processor -- start 2023-10-06T03:07:31.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.398+0000 7f338a6a4700 1 -- start start 2023-10-06T03:07:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f338a6a4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3384159420 0x7f33840b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f338a6a4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f338415d230 0x7f338415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f338a6a4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840b2980 con 0x7f33841597f0 2023-10-06T03:07:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f338a6a4700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338415dbb0 con 0x7f3384159420 2023-10-06T03:07:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f338a6a4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33840b11d0 con 0x7f338415d230 2023-10-06T03:07:31.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f33896a2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3384159420 0x7f33840b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f3389ea3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f338415d230 0x7f338415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f3388ea1700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f33841597f0 0x7f33840b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:38738/0 (socket says 172.21.15.92:38738) 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.399+0000 7f3388ea1700 1 -- 172.21.15.92:0/1229988311 learned_addr learned my addr 172.21.15.92:0/1229988311 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f3389ea3700 1 -- 172.21.15.92:0/1229988311 >> v2:172.21.15.136:3300/0 conn(0x7f3384159420 msgr2=0x7f33840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f3389ea3700 1 --2- 172.21.15.92:0/1229988311 >> v2:172.21.15.136:3300/0 conn(0x7f3384159420 0x7f33840b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f3389ea3700 1 -- 172.21.15.92:0/1229988311 >> v1:172.21.15.136:6789/0 conn(0x7f33841597f0 legacy=0x7f33840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f3389ea3700 1 -- 172.21.15.92:0/1229988311 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33840b14b0 con 0x7f338415d230 2023-10-06T03:07:31.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f3389ea3700 1 --2- 172.21.15.92:0/1229988311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f338415d230 0x7f338415d670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f337c009e60 tx=0x7f337c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66afd516a8dfcd94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f337bfff700 1 -- 172.21.15.92:0/1229988311 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c00e070 con 0x7f338415d230 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f337bfff700 1 -- 172.21.15.92:0/1229988311 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c00bb60 con 0x7f338415d230 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f337bfff700 1 -- 172.21.15.92:0/1229988311 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c012430 con 0x7f338415d230 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f338a6a4700 1 -- 172.21.15.92:0/1229988311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f338415d230 msgr2=0x7f338415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1229988311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f338415d230 0x7f338415d670 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f337c009e60 tx=0x7f337c009780 comp rx=0 tx=0).stop 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f338a6a4700 1 -- 172.21.15.92:0/1229988311 shutdown_connections 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.400+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1229988311 >> v2:172.21.15.136:3300/0 conn(0x7f3384159420 0x7f33840b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.401+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1229988311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f338415d230 0x7f338415d670 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.401+0000 7f338a6a4700 1 -- 172.21.15.92:0/1229988311 >> 172.21.15.92:0/1229988311 conn(0x7f33840d0660 msgr2=0x7f33840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.401+0000 7f338a6a4700 1 -- 172.21.15.92:0/1229988311 shutdown_connections 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.401+0000 7f338a6a4700 1 -- 172.21.15.92:0/1229988311 wait complete. 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.402+0000 7f338a6a4700 1 Processor -- start 2023-10-06T03:07:31.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.402+0000 7f338a6a4700 1 -- start start 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.402+0000 7f338a6a4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 0x7f33841543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33841597f0 0x7f3384147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338415d230 0x7f3384159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3384146e00 con 0x7f3384159420 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33841461f0 con 0x7f338415d230 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33841464d0 con 0x7f33841597f0 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 0x7f33841543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 0x7f33841543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41016/0 (socket says 172.21.15.92:41016) 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 -- 172.21.15.92:0/1637378477 learned_addr learned my addr 172.21.15.92:0/1637378477 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f3389ea3700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338415d230 0x7f3384159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f3388ea1700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33841597f0 0x7f3384147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 -- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33841597f0 msgr2=0x7f3384147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33841597f0 0x7f3384147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 -- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338415d230 msgr2=0x7f3384159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338415d230 0x7f3384159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 -- 172.21.15.92:0/1637378477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c009a20 con 0x7f3384159420 2023-10-06T03:07:31.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33896a2700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 0x7f33841543d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f338000ec80 tx=0x7f33800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000c850 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338000c9b0 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33841467b0 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338000cd50 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.403+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3384154910 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.404+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338404c3c0 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.405+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f3380013680 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.405+0000 7f33797fa700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3368067290 0x7f3368069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.405+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7f3380084e10 con 0x7f3384159420 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.405+0000 7f3388ea1700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3368067290 0x7f3368069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.405+0000 7f3388ea1700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3368067290 0x7f3368069740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3384158610 tx=0x7f3374009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.409+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3380051af0 con 0x7f3384159420 2023-10-06T03:07:31.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.638+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33840b1790 con 0x7f3384159420 2023-10-06T03:07:31.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.638+0000 7f33797fa700 1 -- 172.21.15.92:0/1637378477 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4596 (secure 0 0 0) 0x7f3380055110 con 0x7f3384159420 2023-10-06T03:07:31.639 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:31.639 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:17.381585+0000","modified":"2023-10-06T03:07:21.036747+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6480},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6480":{"gid":6480,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/428935748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":428935748},{"type":"v1","addr":"172.21.15.92:6837","nonce":428935748}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:07:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3368067290 msgr2=0x7f3368069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3368067290 0x7f3368069740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3384158610 tx=0x7f3374009040 comp rx=0 tx=0).stop 2023-10-06T03:07:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 msgr2=0x7f33841543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 0x7f33841543d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f338000ec80 tx=0x7f33800091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 shutdown_connections 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3368067290 0x7f3368069740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3384159420 0x7f33841543d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f338415d230 0x7f3384159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 --2- 172.21.15.92:0/1637378477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33841597f0 0x7f3384147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 >> 172.21.15.92:0/1637378477 conn(0x7f33840d0660 msgr2=0x7f3384156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 shutdown_connections 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.640+0000 7f338a6a4700 1 -- 172.21.15.92:0/1637378477 wait complete. 2023-10-06T03:07:31.642 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 57 2023-10-06T03:07:31.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:31.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.767+0000 7fb0d5df7700 1 Processor -- start 2023-10-06T03:07:31.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.767+0000 7fb0d5df7700 1 -- start start 2023-10-06T03:07:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0d5df7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d0069560 0x7fb0d00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0d5df7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d0069ee0 con 0x7fb0d0159890 2023-10-06T03:07:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0d5df7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d015a160 con 0x7fb0d0159c60 2023-10-06T03:07:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0d5df7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00d13a0 con 0x7fb0d0069560 2023-10-06T03:07:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d0069560 0x7fb0d00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb0d0159890 0x7fb0d015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:32994/0 (socket says 172.21.15.92:32994) 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cf7fe700 1 -- 172.21.15.92:0/2407670154 learned_addr learned my addr 172.21.15.92:0/2407670154 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cffff700 1 -- 172.21.15.92:0/2407670154 >> v1:172.21.15.136:6789/0 conn(0x7fb0d0159c60 legacy=0x7fb0d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cffff700 1 -- 172.21.15.92:0/2407670154 >> v1:172.21.15.92:6789/0 conn(0x7fb0d0159890 legacy=0x7fb0d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0ce7fc700 1 -- 172.21.15.92:0/2407670154 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152284909 0 0) 0x7fb0d0069ee0 con 0x7fb0d0159890 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cffff700 1 -- 172.21.15.92:0/2407670154 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d015e140 con 0x7fb0d0069560 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.768+0000 7fb0cffff700 1 --2- 172.21.15.92:0/2407670154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0069560 0x7fb0d00699a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009e60 tx=0x7fb0c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce97be775e6ff61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0ce7fc700 1 -- 172.21.15.92:0/2407670154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800e070 con 0x7fb0d0069560 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0ce7fc700 1 -- 172.21.15.92:0/2407670154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c800bbf0 con 0x7fb0d0069560 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0ce7fc700 1 -- 172.21.15.92:0/2407670154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8012430 con 0x7fb0d0069560 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2407670154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0069560 msgr2=0x7fb0d00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2407670154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0069560 0x7fb0d00699a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009e60 tx=0x7fb0c8009780 comp rx=0 tx=0).stop 2023-10-06T03:07:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2407670154 shutdown_connections 2023-10-06T03:07:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2407670154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0069560 0x7fb0d00699a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2407670154 >> 172.21.15.92:0/2407670154 conn(0x7fb0d0068e20 msgr2=0x7fb0d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2407670154 shutdown_connections 2023-10-06T03:07:31.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.769+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2407670154 wait complete. 2023-10-06T03:07:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.770+0000 7fb0d5df7700 1 Processor -- start 2023-10-06T03:07:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.770+0000 7fb0d5df7700 1 -- start start 2023-10-06T03:07:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0d5df7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d0069560 0x7fb0d01534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0d5df7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 0x7fb0d00b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0d5df7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159c60 0x7fb0d0154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0d5df7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00cd0c0 con 0x7fb0d0159890 2023-10-06T03:07:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0d5df7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00cd960 con 0x7fb0d0159c60 2023-10-06T03:07:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0d5df7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0153a30 con 0x7fb0d0069560 2023-10-06T03:07:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 0x7fb0d00b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 0x7fb0d00b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41040/0 (socket says 172.21.15.92:41040) 2023-10-06T03:07:31.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 -- 172.21.15.92:0/2498163470 learned_addr learned my addr 172.21.15.92:0/2498163470 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:31.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 -- 172.21.15.92:0/2498163470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d0069560 msgr2=0x7fb0d01534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d0069560 0x7fb0d01534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 -- 172.21.15.92:0/2498163470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159c60 msgr2=0x7fb0d0154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159c60 0x7fb0d0154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.771+0000 7fb0ceffd700 1 -- 172.21.15.92:0/2498163470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c8009a20 con 0x7fb0d0159890 2023-10-06T03:07:31.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.772+0000 7fb0ceffd700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 0x7fb0d00b5980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b610 tx=0x7fb0c4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.772+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400f510 con 0x7fb0d0159890 2023-10-06T03:07:31.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.772+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c400f670 con 0x7fb0d0159890 2023-10-06T03:07:31.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.772+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d00b62c0 con 0x7fb0d0159890 2023-10-06T03:07:31.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.772+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400fa30 con 0x7fb0d0159890 2023-10-06T03:07:31.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.772+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d00cd5f0 con 0x7fb0d0159890 2023-10-06T03:07:31.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.773+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d004c3c0 con 0x7fb0d0159890 2023-10-06T03:07:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.773+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb0c401a400 con 0x7fb0d0159890 2023-10-06T03:07:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.773+0000 7fb0b77fe700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0bc067290 0x7fb0bc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.773+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fb0c4084820 con 0x7fb0d0159890 2023-10-06T03:07:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.774+0000 7fb0cf7fe700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0bc067290 0x7fb0bc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:31.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.774+0000 7fb0cf7fe700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0bc067290 0x7fb0bc069740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00011e0 tx=0x7fb0c00079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:31.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.778+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0c4051480 con 0x7fb0d0159890 2023-10-06T03:07:31.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.964+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0d00cdad0 con 0x7fb0d0159890 2023-10-06T03:07:31.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.965+0000 7fb0b77fe700 1 -- 172.21.15.92:0/2498163470 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16919 (secure 0 0 0) 0x7fb0c4054aa0 con 0x7fb0d0159890 2023-10-06T03:07:31.965 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:31.966 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":92,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:24.879999+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:15.031911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:16.347808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.35","mappings":[{"from":5,"to":7}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0bc067290 msgr2=0x7fb0bc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0bc067290 0x7fb0bc069740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00011e0 tx=0x7fb0c00079d0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 msgr2=0x7fb0d00b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 0x7fb0d00b5980 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b610 tx=0x7fb0c4009120 comp rx=0 tx=0).stop 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 shutdown_connections 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0bc067290 0x7fb0bc069740 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159c60 0x7fb0d0154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d0069560 0x7fb0d01534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 --2- 172.21.15.92:0/2498163470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159890 0x7fb0d00b5980 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:31.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 >> 172.21.15.92:0/2498163470 conn(0x7fb0d0068e20 msgr2=0x7fb0d00d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:31.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 shutdown_connections 2023-10-06T03:07:31.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:31.967+0000 7fb0d5df7700 1 -- 172.21.15.92:0/2498163470 wait complete. 2023-10-06T03:07:31.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:07:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.095+0000 7fe2b9441700 1 Processor -- start 2023-10-06T03:07:32.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.095+0000 7fe2b9441700 1 -- start start 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b9441700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b9441700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2b4159c60 0x7fe2b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b9441700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b415a160 con 0x7fe2b415db10 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b9441700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b40d13a0 con 0x7fe2b4159c60 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b9441700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b415e140 con 0x7fe2b4159890 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2aa7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe2b4159c60 0x7fe2b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe2b415db10 0x7fe2b4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33008/0 (socket says 172.21.15.92:33008) 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b37fe700 1 -- 172.21.15.92:0/565310955 learned_addr learned my addr 172.21.15.92:0/565310955 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:32.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.096+0000 7fe2b2ffd700 1 --2- 172.21.15.92:0/565310955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2aa7fc700 1 -- 172.21.15.92:0/565310955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 msgr2=0x7fe2b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2aa7fc700 1 --2- 172.21.15.92:0/565310955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2aa7fc700 1 -- 172.21.15.92:0/565310955 >> v1:172.21.15.92:6789/0 conn(0x7fe2b415db10 legacy=0x7fe2b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b27fc700 1 -- 172.21.15.92:0/565310955 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156425019 0 0) 0x7fe2b415a160 con 0x7fe2b415db10 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2aa7fc700 1 -- 172.21.15.92:0/565310955 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b415a3d0 con 0x7fe2b4159c60 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2aa7fc700 1 --2- 172.21.15.92:0/565310955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159c60 0x7fe2b415d5d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0098e0 tx=0x7fe29c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58411a7cd397e11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b27fc700 1 -- 172.21.15.92:0/565310955 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00b640 con 0x7fe2b4159c60 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b27fc700 1 -- 172.21.15.92:0/565310955 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c00b7a0 con 0x7fe2b4159c60 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b27fc700 1 -- 172.21.15.92:0/565310955 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00bb40 con 0x7fe2b4159c60 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b9441700 1 -- 172.21.15.92:0/565310955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159c60 msgr2=0x7fe2b415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b9441700 1 --2- 172.21.15.92:0/565310955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159c60 0x7fe2b415d5d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0098e0 tx=0x7fe29c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b9441700 1 -- 172.21.15.92:0/565310955 shutdown_connections 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b9441700 1 --2- 172.21.15.92:0/565310955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b9441700 1 --2- 172.21.15.92:0/565310955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b4159c60 0x7fe2b415d5d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.097+0000 7fe2b9441700 1 -- 172.21.15.92:0/565310955 >> 172.21.15.92:0/565310955 conn(0x7fe2b4068e20 msgr2=0x7fe2b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.098+0000 7fe2b9441700 1 -- 172.21.15.92:0/565310955 shutdown_connections 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.098+0000 7fe2b9441700 1 -- 172.21.15.92:0/565310955 wait complete. 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.099+0000 7fe2b9441700 1 Processor -- start 2023-10-06T03:07:32.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.099+0000 7fe2b9441700 1 -- start start 2023-10-06T03:07:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.099+0000 7fe2b9441700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b40c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.099+0000 7fe2b9441700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159c60 0x7fe2b4149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.099+0000 7fe2b9441700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 0x7fe2b4148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b9441700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c6160 con 0x7fe2b415db10 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b9441700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c6440 con 0x7fe2b4159c60 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b9441700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b40c6720 con 0x7fe2b4159890 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 0x7fe2b4148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 0x7fe2b4148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41056/0 (socket says 172.21.15.92:41056) 2023-10-06T03:07:32.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 -- 172.21.15.92:0/3669299306 learned_addr learned my addr 172.21.15.92:0/3669299306 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b2ffd700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b40c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2aa7fc700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159c60 0x7fe2b4149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 -- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 msgr2=0x7fe2b40c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b40c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 -- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159c60 msgr2=0x7fe2b4149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159c60 0x7fe2b4149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 -- 172.21.15.92:0/3669299306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c009420 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b37fe700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 0x7fe2b4148fe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00cbf0 tx=0x7fe2ac0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00a8b0 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac00aa10 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b40c6a00 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00add0 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.100+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b40c5d70 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.101+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b40c6f20 con 0x7fe2b415db10 2023-10-06T03:07:32.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.101+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe2ac007300 con 0x7fe2b415db10 2023-10-06T03:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.102+0000 7fe2ab7fe700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294067290 0x7fe294069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.102+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fe2ac084f60 con 0x7fe2b415db10 2023-10-06T03:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.102+0000 7fe2b2ffd700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294067290 0x7fe294069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.102+0000 7fe2b2ffd700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294067290 0x7fe294069740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009db0 tx=0x7fe2a40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:32.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.106+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2b40c6f20 con 0x7fe2b415db10 2023-10-06T03:07:32.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.332+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe2b40c6f20 con 0x7fe2b415db10 2023-10-06T03:07:32.333 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:07:32.333+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:07:32.337 INFO:tasks.ceph.mds.c.smithi092.stderr: -836> 2023-10-06T03:07:22.304+0000 7f484b5ab700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:07:32.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.339+0000 7fe2ab7fe700 1 -- 172.21.15.92:0/3669299306 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe2b40c6f20 con 0x7fe2b415db10 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294067290 msgr2=0x7fe294069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294067290 0x7fe294069740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009db0 tx=0x7fe2a40091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 msgr2=0x7fe2b4148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 0x7fe2b4148fe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00cbf0 tx=0x7fe2ac0019b0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 shutdown_connections 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe294067290 0x7fe294069740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2b415db10 0x7fe2b4148fe0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2b4159890 0x7fe2b40c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 --2- 172.21.15.92:0/3669299306 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2b4159c60 0x7fe2b4149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 >> 172.21.15.92:0/3669299306 conn(0x7fe2b4068e20 msgr2=0x7fe2b415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 shutdown_connections 2023-10-06T03:07:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.342+0000 7fe2b9441700 1 -- 172.21.15.92:0/3669299306 wait complete. 2023-10-06T03:07:32.344 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:07:32.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:07:32.359 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:07:32.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.471+0000 7f94a7282700 1 Processor -- start 2023-10-06T03:07:32.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.471+0000 7f94a7282700 1 -- start start 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a7282700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a7282700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f94a0069560 0x7f94a00699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a7282700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a015a160 con 0x7f94a0159c60 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a7282700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a0069f20 con 0x7f94a0069560 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a7282700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00d13a0 con 0x7f94a0159890 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a481d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f94a0159c60 0x7f94a015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33032/0 (socket says 172.21.15.92:33032) 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.472+0000 7f94a481d700 1 -- 172.21.15.92:0/3182249605 learned_addr learned my addr 172.21.15.92:0/3182249605 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:32.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a581f700 1 --2- 172.21.15.92:0/3182249605 >> v2:172.21.15.136:3300/0 conn(0x7f94a0069560 0x7f94a00699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a501e700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f9497fff700 1 -- 172.21.15.92:0/3182249605 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825994110 0 0) 0x7f94a015a160 con 0x7f94a0159c60 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f9497fff700 1 -- 172.21.15.92:0/3182249605 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f948c003570 con 0x7f94a0159c60 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a581f700 1 -- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 msgr2=0x7f94a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a581f700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a581f700 1 -- 172.21.15.92:0/3182249605 >> v1:172.21.15.92:6789/0 conn(0x7f94a0159c60 legacy=0x7f94a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f9497fff700 1 -- 172.21.15.92:0/3182249605 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1232692856 0 0) 0x7f948c003570 con 0x7f94a0159c60 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a581f700 1 -- 172.21.15.92:0/3182249605 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a015e140 con 0x7f94a0069560 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a501e700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:32.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f94a581f700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00699e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f94980098e0 tx=0x7f94980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20795bfc7d272f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f9497fff700 1 -- 172.21.15.92:0/3182249605 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498011630 con 0x7f94a0069560 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f9497fff700 1 -- 172.21.15.92:0/3182249605 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498011790 con 0x7f94a0069560 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.473+0000 7f9497fff700 1 -- 172.21.15.92:0/3182249605 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498011b10 con 0x7f94a0069560 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 -- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 msgr2=0x7f94a00699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00699e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f94980098e0 tx=0x7f94980091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 -- 172.21.15.92:0/3182249605 shutdown_connections 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 --2- 172.21.15.92:0/3182249605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00699e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 -- 172.21.15.92:0/3182249605 >> 172.21.15.92:0/3182249605 conn(0x7f94a0068e20 msgr2=0x7f94a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 -- 172.21.15.92:0/3182249605 shutdown_connections 2023-10-06T03:07:32.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.474+0000 7f94a7282700 1 -- 172.21.15.92:0/3182249605 wait complete. 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.475+0000 7f94a7282700 1 Processor -- start 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.475+0000 7f94a7282700 1 -- start start 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a7282700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a7282700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a00cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a7282700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 0x7f94a00cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a7282700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cff20 con 0x7f94a0159c60 2023-10-06T03:07:32.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a7282700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a015c770 con 0x7f94a0069560 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a7282700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a015d520 con 0x7f94a0159890 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 0x7f94a00cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 0x7f94a00cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41092/0 (socket says 172.21.15.92:41092) 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 -- 172.21.15.92:0/1721863057 learned_addr learned my addr 172.21.15.92:0/1721863057 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a481d700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a00cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a501e700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 -- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 msgr2=0x7f94a00cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a00cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 -- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 msgr2=0x7f94a00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:32.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.476+0000 7f94a581f700 1 -- 172.21.15.92:0/1721863057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c009a20 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.477+0000 7f94a581f700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 0x7f94a00cdd20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ed0 tx=0x7f9498017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.477+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949801a430 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.477+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949801a590 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.477+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9498009420 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.477+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949801a910 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.477+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a015c490 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.478+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a004c3c0 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.478+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f949801acc0 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.478+0000 7f94957fa700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9484067290 0x7f9484069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.479+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f9498086b70 con 0x7f94a0159c60 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.479+0000 7f94a501e700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9484067290 0x7f9484069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:32.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.479+0000 7f94a501e700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9484067290 0x7f9484069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f949c0099f0 tx=0x7f949c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:32.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.483+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9498053800 con 0x7f94a0159c60 2023-10-06T03:07:32.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:32.706+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f94a0159020 con 0x7f94a0159c60 2023-10-06T03:07:33.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.340+0000 7f94957fa700 1 -- 172.21.15.92:0/1721863057 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9498056e20 con 0x7f94a0159c60 2023-10-06T03:07:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9484067290 msgr2=0x7f9484069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9484067290 0x7f9484069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f949c0099f0 tx=0x7f949c00b040 comp rx=0 tx=0).stop 2023-10-06T03:07:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 msgr2=0x7f94a00cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 0x7f94a00cdd20 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9498009ed0 tx=0x7f9498017000 comp rx=0 tx=0).stop 2023-10-06T03:07:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 shutdown_connections 2023-10-06T03:07:33.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9484067290 0x7f9484069740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94a0159c60 0x7f94a00cdd20 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94a0069560 0x7f94a00cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 --2- 172.21.15.92:0/1721863057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94a0159890 0x7f94a00cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 >> 172.21.15.92:0/1721863057 conn(0x7f94a0068e20 msgr2=0x7f94a0152480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:33.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 shutdown_connections 2023-10-06T03:07:33.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.342+0000 7f94a7282700 1 -- 172.21.15.92:0/1721863057 wait complete. 2023-10-06T03:07:33.355 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:07:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.514+0000 7f16ef912700 1 Processor -- start 2023-10-06T03:07:33.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.514+0000 7f16ef912700 1 -- start start 2023-10-06T03:07:33.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ef912700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e8159ec0 0x7f16e80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ef912700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16e80b53d0 0x7f16e80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ef912700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80b5b00 con 0x7f16e80cce40 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ef912700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e80b5da0 con 0x7f16e80b53d0 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ef912700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80b6080 con 0x7f16e8159ec0 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ecead700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16e80b53d0 0x7f16e80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ecead700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f16e80b53d0 0x7f16e80cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41104/0 (socket says 172.21.15.92:41104) 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.515+0000 7f16ecead700 1 -- 172.21.15.92:0/747477953 learned_addr learned my addr 172.21.15.92:0/747477953 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ecead700 1 -- 172.21.15.92:0/747477953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e8159ec0 msgr2=0x7f16e80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ecead700 1 --2- 172.21.15.92:0/747477953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e8159ec0 0x7f16e80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ecead700 1 -- 172.21.15.92:0/747477953 >> v1:172.21.15.136:6789/0 conn(0x7f16e80cce40 legacy=0x7f16e80cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ecead700 1 -- 172.21.15.92:0/747477953 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80cc100 con 0x7f16e80b53d0 2023-10-06T03:07:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ecead700 1 --2- 172.21.15.92:0/747477953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e80b53d0 0x7f16e80cc900 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0099b0 tx=0x7f16dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24bb1d845e6fcc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16e7fff700 1 -- 172.21.15.92:0/747477953 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00b820 con 0x7f16e80b53d0 2023-10-06T03:07:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16e7fff700 1 -- 172.21.15.92:0/747477953 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc00b980 con 0x7f16e80b53d0 2023-10-06T03:07:33.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16e7fff700 1 -- 172.21.15.92:0/747477953 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00bd20 con 0x7f16e80b53d0 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ef912700 1 -- 172.21.15.92:0/747477953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e80b53d0 msgr2=0x7f16e80cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.516+0000 7f16ef912700 1 --2- 172.21.15.92:0/747477953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e80b53d0 0x7f16e80cc900 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0099b0 tx=0x7f16dc009250 comp rx=0 tx=0).stop 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.517+0000 7f16ef912700 1 -- 172.21.15.92:0/747477953 shutdown_connections 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.517+0000 7f16ef912700 1 --2- 172.21.15.92:0/747477953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e8159ec0 0x7f16e80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.517+0000 7f16ef912700 1 --2- 172.21.15.92:0/747477953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e80b53d0 0x7f16e80cc900 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.517+0000 7f16ef912700 1 -- 172.21.15.92:0/747477953 >> 172.21.15.92:0/747477953 conn(0x7f16e80999f0 msgr2=0x7f16e80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.517+0000 7f16ef912700 1 -- 172.21.15.92:0/747477953 shutdown_connections 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.517+0000 7f16ef912700 1 -- 172.21.15.92:0/747477953 wait complete. 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.518+0000 7f16ef912700 1 Processor -- start 2023-10-06T03:07:33.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.518+0000 7f16ef912700 1 -- start start 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e80b53d0 0x7f16e8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e80cce40 0x7f16e80c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 0x7f16e814ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8068e70 con 0x7f16e8159ec0 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80c9ee0 con 0x7f16e80cce40 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80ca1c0 con 0x7f16e80b53d0 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 0x7f16e814ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 0x7f16e814ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41114/0 (socket says 172.21.15.92:41114) 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 -- 172.21.15.92:0/522768253 learned_addr learned my addr 172.21.15.92:0/522768253 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ed6ae700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e80b53d0 0x7f16e8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ecead700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e80cce40 0x7f16e80c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:33.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 -- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e80b53d0 msgr2=0x7f16e8152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e80b53d0 0x7f16e8152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 -- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e80cce40 msgr2=0x7f16e80c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e80cce40 0x7f16e80c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 -- 172.21.15.92:0/522768253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0094f0 con 0x7f16e8159ec0 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ecead700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e80cce40 0x7f16e80c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16edeaf700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 0x7f16e814ea10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c5b0 tx=0x7f16e0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0011590 con 0x7f16e8159ec0 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e0011710 con 0x7f16e8159ec0 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e80690c0 con 0x7f16e8159ec0 2023-10-06T03:07:33.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0011ad0 con 0x7f16e8159ec0 2023-10-06T03:07:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.519+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e814f7f0 con 0x7f16e8159ec0 2023-10-06T03:07:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.520+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e8063c90 con 0x7f16e8159ec0 2023-10-06T03:07:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.521+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f16e001c460 con 0x7f16e8159ec0 2023-10-06T03:07:33.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.521+0000 7f16e57fa700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16d4067290 0x7f16d4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.521+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6173+0+0 (secure 0 0 0) 0x7f16e00848d0 con 0x7f16e8159ec0 2023-10-06T03:07:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.521+0000 7f16ed6ae700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16d4067290 0x7f16d4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:33.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.522+0000 7f16ed6ae700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16d4067290 0x7f16d4069740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f16e80cfae0 tx=0x7f16d8009620 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:33.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.525+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16e00515a0 con 0x7f16e8159ec0 2023-10-06T03:07:33.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:33.709+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16e80718b0 con 0x7f16e8159ec0 2023-10-06T03:07:34.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.346+0000 7f16e57fa700 1 -- 172.21.15.92:0/522768253 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f16e0054bc0 con 0x7f16e8159ec0 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16d4067290 msgr2=0x7f16d4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16d4067290 0x7f16d4069740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f16e80cfae0 tx=0x7f16d8009620 comp rx=0 tx=0).stop 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 msgr2=0x7f16e814ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 0x7f16e814ea10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f16e000c5b0 tx=0x7f16e0007f60 comp rx=0 tx=0).stop 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 shutdown_connections 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16d4067290 0x7f16d4069740 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16e8159ec0 0x7f16e814ea10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16e80b53d0 0x7f16e8152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 --2- 172.21.15.92:0/522768253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16e80cce40 0x7f16e80c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 >> 172.21.15.92:0/522768253 conn(0x7f16e80999f0 msgr2=0x7f16e81522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 shutdown_connections 2023-10-06T03:07:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.349+0000 7f16ef912700 1 -- 172.21.15.92:0/522768253 wait complete. 2023-10-06T03:07:34.351 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:07:34.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:07:34.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.481+0000 7fdade1ec700 1 Processor -- start 2023-10-06T03:07:34.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.481+0000 7fdade1ec700 1 -- start start 2023-10-06T03:07:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdade1ec700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdad80cd910 0x7fdad80ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:34.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdade1ec700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdad80b55a0 0x7fdad80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdade1ec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad80b5b00 con 0x7fdad8159ec0 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdade1ec700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad80b5da0 con 0x7fdad80b55a0 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdade1ec700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad80b6080 con 0x7fdad80cd910 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdad7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdad80b55a0 0x7fdad80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdad7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdad80b55a0 0x7fdad80ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41122/0 (socket says 172.21.15.92:41122) 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdad7fff700 1 -- 172.21.15.92:0/656361493 learned_addr learned my addr 172.21.15.92:0/656361493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdad77fe700 1 --1- 172.21.15.92:0/656361493 >> v1:172.21.15.92:6789/0 conn(0x7fdad8159ec0 0x7fdad80b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33048/0 (socket says 172.21.15.92:33048) 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.482+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/656361493 >> v2:172.21.15.136:3300/0 conn(0x7fdad80cd910 0x7fdad80ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad7fff700 1 -- 172.21.15.92:0/656361493 >> v2:172.21.15.136:3300/0 conn(0x7fdad80cd910 msgr2=0x7fdad80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad7fff700 1 --2- 172.21.15.92:0/656361493 >> v2:172.21.15.136:3300/0 conn(0x7fdad80cd910 0x7fdad80ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad7fff700 1 -- 172.21.15.92:0/656361493 >> v1:172.21.15.92:6789/0 conn(0x7fdad8159ec0 legacy=0x7fdad80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:34.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad7fff700 1 -- 172.21.15.92:0/656361493 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad80cc100 con 0x7fdad80b55a0 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad67fc700 1 -- 172.21.15.92:0/656361493 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795997728 0 0) 0x7fdad80b5b00 con 0x7fdad8159ec0 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/656361493 >> v2:172.21.15.136:3300/0 conn(0x7fdad80cd910 0x7fdad80ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad7fff700 1 --2- 172.21.15.92:0/656361493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b55a0 0x7fdad80ce880 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdac80099b0 tx=0x7fdac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3424fc157fb09780 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad67fc700 1 -- 172.21.15.92:0/656361493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800b820 con 0x7fdad80b55a0 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad67fc700 1 -- 172.21.15.92:0/656361493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac800b980 con 0x7fdad80b55a0 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdad67fc700 1 -- 172.21.15.92:0/656361493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800bd20 con 0x7fdad80b55a0 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 -- 172.21.15.92:0/656361493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b55a0 msgr2=0x7fdad80ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 --2- 172.21.15.92:0/656361493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b55a0 0x7fdad80ce880 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fdac80099b0 tx=0x7fdac8009250 comp rx=0 tx=0).stop 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 -- 172.21.15.92:0/656361493 shutdown_connections 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 --2- 172.21.15.92:0/656361493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b55a0 0x7fdad80ce880 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 --2- 172.21.15.92:0/656361493 >> v2:172.21.15.136:3300/0 conn(0x7fdad80cd910 0x7fdad80ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 -- 172.21.15.92:0/656361493 >> 172.21.15.92:0/656361493 conn(0x7fdad80999f0 msgr2=0x7fdad80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 -- 172.21.15.92:0/656361493 shutdown_connections 2023-10-06T03:07:34.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.483+0000 7fdade1ec700 1 -- 172.21.15.92:0/656361493 wait complete. 2023-10-06T03:07:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 Processor -- start 2023-10-06T03:07:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 -- start start 2023-10-06T03:07:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80b55a0 0x7fdad8153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:34.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 0x7fdad8156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81513c0 0x7fdad8169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8156a80 con 0x7fdad8159ec0 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8154280 con 0x7fdad81513c0 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.485+0000 7fdade1ec700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8153670 con 0x7fdad80b55a0 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 0x7fdad8156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 0x7fdad8156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41130/0 (socket says 172.21.15.92:41130) 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 -- 172.21.15.92:0/1374316917 learned_addr learned my addr 172.21.15.92:0/1374316917 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad7fff700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81513c0 0x7fdad8169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad77fe700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80b55a0 0x7fdad8153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:34.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 -- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80b55a0 msgr2=0x7fdad8153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80b55a0 0x7fdad8153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 -- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81513c0 msgr2=0x7fdad8169b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81513c0 0x7fdad8169b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 -- 172.21.15.92:0/1374316917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80094f0 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 0x7fdad8156540 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c8b0 tx=0x7fdacc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc011600 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc011760 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad8153950 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc011b00 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.486+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad8153f10 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.487+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad8063730 con 0x7fdad8159ec0 2023-10-06T03:07:34.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.487+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fdacc0073d0 con 0x7fdad8159ec0 2023-10-06T03:07:34.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.488+0000 7fdabf7fe700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdac4067290 0x7fdac4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:34.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.488+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5735+0+0 (secure 0 0 0) 0x7fdacc084c30 con 0x7fdad8159ec0 2023-10-06T03:07:34.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.488+0000 7fdad77fe700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdac4067290 0x7fdac4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:34.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.488+0000 7fdad77fe700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdac4067290 0x7fdac4069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdad0007fd0 tx=0x7fdad0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:34.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.492+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdacc051960 con 0x7fdad8159ec0 2023-10-06T03:07:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:34.675+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdad8063c90 con 0x7fdad8159ec0 2023-10-06T03:07:35.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.349+0000 7fdabf7fe700 1 -- 172.21.15.92:0/1374316917 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdacc0150d0 con 0x7fdad8159ec0 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdac4067290 msgr2=0x7fdac4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdac4067290 0x7fdac4069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdad0007fd0 tx=0x7fdad0009040 comp rx=0 tx=0).stop 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 msgr2=0x7fdad8156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 0x7fdad8156540 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00c8b0 tx=0x7fdacc008040 comp rx=0 tx=0).stop 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 shutdown_connections 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdac4067290 0x7fdac4069740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad81513c0 0x7fdad8169b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80b55a0 0x7fdad8153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 --2- 172.21.15.92:0/1374316917 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad8159ec0 0x7fdad8156540 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 >> 172.21.15.92:0/1374316917 conn(0x7fdad80999f0 msgr2=0x7fdad80cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 shutdown_connections 2023-10-06T03:07:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.352+0000 7fdade1ec700 1 -- 172.21.15.92:0/1374316917 wait complete. 2023-10-06T03:07:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:07:35.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:07:35.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.487+0000 7f13d8d91700 1 Processor -- start 2023-10-06T03:07:35.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.487+0000 7f13d8d91700 1 -- start start 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13d8d91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f13d40c70b0 0x7f13d4069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13d8d91700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d406e950 con 0x7f13d4155eb0 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13d8d91700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d406eb90 con 0x7f13d4069cb0 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13d8d91700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d406a130 con 0x7f13d40c70b0 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13c9d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f13d40c70b0 0x7f13d4069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13d259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f13d4155eb0 0x7f13d406e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33064/0 (socket says 172.21.15.92:33064) 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.488+0000 7f13d259c700 1 -- 172.21.15.92:0/2679047452 learned_addr learned my addr 172.21.15.92:0/2679047452 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13c9d9b700 1 -- 172.21.15.92:0/2679047452 >> v1:172.21.15.136:6789/0 conn(0x7f13d4069cb0 legacy=0x7f13d40ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:35.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13c9d9b700 1 -- 172.21.15.92:0/2679047452 >> v1:172.21.15.92:6789/0 conn(0x7f13d4155eb0 legacy=0x7f13d406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13c9d9b700 1 -- 172.21.15.92:0/2679047452 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d406ee70 con 0x7f13d40c70b0 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d1d9b700 1 -- 172.21.15.92:0/2679047452 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842948397 0 0) 0x7f13c4001110 con 0x7f13d4155eb0 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13c9d9b700 1 --2- 172.21.15.92:0/2679047452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d40c70b0 0x7f13d4069770 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0098e0 tx=0x7f13bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e4635bd4e03568 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d1d9b700 1 -- 172.21.15.92:0/2679047452 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc009e80 con 0x7f13d40c70b0 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d1d9b700 1 -- 172.21.15.92:0/2679047452 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc00b820 con 0x7f13d40c70b0 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d1d9b700 1 -- 172.21.15.92:0/2679047452 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00bba0 con 0x7f13d40c70b0 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 -- 172.21.15.92:0/2679047452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d40c70b0 msgr2=0x7f13d4069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 --2- 172.21.15.92:0/2679047452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d40c70b0 0x7f13d4069770 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0098e0 tx=0x7f13bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 -- 172.21.15.92:0/2679047452 shutdown_connections 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 --2- 172.21.15.92:0/2679047452 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d40c70b0 0x7f13d4069770 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 -- 172.21.15.92:0/2679047452 >> 172.21.15.92:0/2679047452 conn(0x7f13d4068e10 msgr2=0x7f13d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:35.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 -- 172.21.15.92:0/2679047452 shutdown_connections 2023-10-06T03:07:35.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.489+0000 7f13d8d91700 1 -- 172.21.15.92:0/2679047452 wait complete. 2023-10-06T03:07:35.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 Processor -- start 2023-10-06T03:07:35.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 -- start start 2023-10-06T03:07:35.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 0x7f13d40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f13d40c70b0 0x7f13d40acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13d4155eb0 0x7f13d40ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c6500 con 0x7f13d4069cb0 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d414e210 con 0x7f13d4155eb0 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d8d91700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d414e4f0 con 0x7f13d40c70b0 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 0x7f13d40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 0x7f13d40c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41162/0 (socket says 172.21.15.92:41162) 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.491+0000 7f13d259c700 1 -- 172.21.15.92:0/3935444767 learned_addr learned my addr 172.21.15.92:0/3935444767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13c9d9b700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f13d40c70b0 0x7f13d40acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d2d9d700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13d4155eb0 0x7f13d40ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d259c700 1 -- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f13d40c70b0 msgr2=0x7f13d40acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d259c700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f13d40c70b0 0x7f13d40acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d259c700 1 -- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13d4155eb0 msgr2=0x7f13d40ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d259c700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13d4155eb0 0x7f13d40ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d259c700 1 -- 172.21.15.92:0/3935444767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13bc009420 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d259c700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 0x7f13d40c5b40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f13c400c5b0 tx=0x7f13c4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4005b30 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4005c90 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d40c5060 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4007300 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.492+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d40c5620 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.493+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d40c5840 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.493+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f13c401b3b0 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.494+0000 7f13caffd700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f13b4067290 0x7f13b4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.494+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f13c4083c90 con 0x7f13d4069cb0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.494+0000 7f13c9d9b700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f13b4067290 0x7f13b4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.494+0000 7f13c9d9b700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f13b4067290 0x7f13b4069740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0030c0 tx=0x7f13bc016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:35.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.498+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13c4050c90 con 0x7f13d4069cb0 2023-10-06T03:07:35.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.685+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f13d406e950 con 0x7f13d4069cb0 2023-10-06T03:07:35.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.689+0000 7f13caffd700 1 -- 172.21.15.92:0/3935444767 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f13d406e950 con 0x7f13d4069cb0 2023-10-06T03:07:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f13b4067290 msgr2=0x7f13b4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f13b4067290 0x7f13b4069740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0030c0 tx=0x7f13bc016000 comp rx=0 tx=0).stop 2023-10-06T03:07:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 msgr2=0x7f13d40c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 0x7f13d40c5b40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f13c400c5b0 tx=0x7f13c4001070 comp rx=0 tx=0).stop 2023-10-06T03:07:35.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 shutdown_connections 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f13b4067290 0x7f13b4069740 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f13d4069cb0 0x7f13d40c5b40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f13d4155eb0 0x7f13d40ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 --2- 172.21.15.92:0/3935444767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f13d40c70b0 0x7f13d40acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 >> 172.21.15.92:0/3935444767 conn(0x7f13d4068e10 msgr2=0x7f13d40af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 shutdown_connections 2023-10-06T03:07:35.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.691+0000 7f13d8d91700 1 -- 172.21.15.92:0/3935444767 wait complete. 2023-10-06T03:07:35.704 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-06T03:07:35.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.817+0000 7f7735e61700 1 Processor -- start 2023-10-06T03:07:35.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.817+0000 7f7735e61700 1 -- start start 2023-10-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f7735e61700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7730140350 0x7f7730069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f7735e61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 0x7f77300c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f7735e61700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300c7de0 con 0x7f773006a1c0 2023-10-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f7735e61700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7730140c10 con 0x7f7730140350 2023-10-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f7735e61700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7730141490 con 0x7f7730068aa0 2023-10-06T03:07:35.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f773006a1c0 0x7f7730068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33070/0 (socket says 172.21.15.92:33070) 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772effd700 1 -- 172.21.15.92:0/2029984713 learned_addr learned my addr 172.21.15.92:0/2029984713 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772f7fe700 1 --2- 172.21.15.92:0/2029984713 >> v2:172.21.15.136:3300/0 conn(0x7f7730140350 0x7f7730069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150409439 0 0) 0x7f77300c7de0 con 0x7f773006a1c0 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772ffff700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 0x7f77300c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7718003570 con 0x7f773006a1c0 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 513042182 0 0) 0x7f7718003570 con 0x7f773006a1c0 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 msgr2=0x7f77300c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 0x7f77300c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 >> v2:172.21.15.136:3300/0 conn(0x7f7730140350 msgr2=0x7f7730069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 --2- 172.21.15.92:0/2029984713 >> v2:172.21.15.136:3300/0 conn(0x7f7730140350 0x7f7730069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7730140ed0 con 0x7f773006a1c0 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.818+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7724001470 con 0x7f773006a1c0 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 >> v1:172.21.15.92:6789/0 conn(0x7f773006a1c0 legacy=0x7f7730068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718003a60 0x7f7718003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7718004810 0x7f7718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 0x7f77180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730140c10 con 0x7f7718003a60 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7730141490 con 0x7f7718004810 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771800ba60 con 0x7f7730068aa0 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718003a60 0x7f7718003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772effd700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7718004810 0x7f7718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772ffff700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 0x7f77180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 -- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 msgr2=0x7f77180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7730068aa0 0x7f77180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 -- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7718004810 msgr2=0x7f7718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7718004810 0x7f7718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 -- 172.21.15.92:0/2029984713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718003570 con 0x7f7718003a60 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772effd700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7718004810 0x7f7718004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772f7fe700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718003a60 0x7f7718003e70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f772000e300 tx=0x7f772000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7763d3ae9859d05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772000eb50 con 0x7f7718003a60 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772000f040 con 0x7f7718003a60 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.819+0000 7f772e7fc700 1 -- 172.21.15.92:0/2029984713 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720014480 con 0x7f7718003a60 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f7735e61700 1 -- 172.21.15.92:0/2029984713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718003a60 msgr2=0x7f7718003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f7735e61700 1 --2- 172.21.15.92:0/2029984713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7718003a60 0x7f7718003e70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f772000e300 tx=0x7f772000b040 comp rx=0 tx=0).stop 2023-10-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f772ffff700 1 -- 172.21.15.92:0/2029984713 reap_dead start 2023-10-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f7735e61700 1 -- 172.21.15.92:0/2029984713 shutdown_connections 2023-10-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f7735e61700 1 -- 172.21.15.92:0/2029984713 >> 172.21.15.92:0/2029984713 conn(0x7f77300cd670 msgr2=0x7f77300cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f7735e61700 1 -- 172.21.15.92:0/2029984713 shutdown_connections 2023-10-06T03:07:35.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.820+0000 7f7735e61700 1 -- 172.21.15.92:0/2029984713 wait complete. 2023-10-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.821+0000 7f7735e61700 1 Processor -- start 2023-10-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.821+0000 7f7735e61700 1 -- start start 2023-10-06T03:07:35.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7718003a60 0x7f773013c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 0x7f77300ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77300b0410 0x7f77300af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300cd070 con 0x7f77300cc4b0 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300ae710 con 0x7f77300b0410 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77300ae9f0 con 0x7f7718003a60 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 0x7f77300ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 0x7f77300ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41172/0 (socket says 172.21.15.92:41172) 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 -- 172.21.15.92:0/1730224381 learned_addr learned my addr 172.21.15.92:0/1730224381 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 -- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7718003a60 msgr2=0x7f773013c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7718003a60 0x7f773013c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 -- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77300b0410 msgr2=0x7f77300af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772ffff700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77300b0410 0x7f77300af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77300b0410 0x7f77300af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 -- 172.21.15.92:0/1730224381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300aecd0 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772ffff700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77300b0410 0x7f77300af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f772effd700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 0x7f77300ccb30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7724000fd0 tx=0x7f77300c7d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400c7b0 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772400c910 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77300aefb0 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400cc90 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.822+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77300b0a30 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.823+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77300af1d0 con 0x7f77300cc4b0 2023-10-06T03:07:35.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.824+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f772401d460 con 0x7f77300cc4b0 2023-10-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.824+0000 7f77177fe700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f771c067290 0x7f771c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.824+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f77240848a0 con 0x7f77300cc4b0 2023-10-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.824+0000 7f772f7fe700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f771c067290 0x7f771c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.825+0000 7f772f7fe700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f771c067290 0x7f771c069740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f772000bfd0 tx=0x7f772001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:35.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:35.828+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77240518a0 con 0x7f77300cc4b0 2023-10-06T03:07:36.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.089+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7f773004c3c0 con 0x7f77300cc4b0 2023-10-06T03:07:36.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.356+0000 7f77177fe700 1 -- 172.21.15.92:0/1730224381 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v286) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7724054ec0 con 0x7f77300cc4b0 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f771c067290 msgr2=0x7f771c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f771c067290 0x7f771c069740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f772000bfd0 tx=0x7f772001a000 comp rx=0 tx=0).stop 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 msgr2=0x7f77300ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 0x7f77300ccb30 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7724000fd0 tx=0x7f77300c7d50 comp rx=0 tx=0).stop 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 shutdown_connections 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f771c067290 0x7f771c069740 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77300b0410 0x7f77300af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7718003a60 0x7f773013c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 --2- 172.21.15.92:0/1730224381 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f77300cc4b0 0x7f77300ccb30 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 >> 172.21.15.92:0/1730224381 conn(0x7f77300cd670 msgr2=0x7f7730068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:36.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 shutdown_connections 2023-10-06T03:07:36.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.359+0000 7f7735e61700 1 -- 172.21.15.92:0/1730224381 wait complete. 2023-10-06T03:07:36.373 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command without subvolume group ... ok 2023-10-06T03:07:36.373 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command without subvolume group 2023-10-06T03:07:36.373 INFO:tasks.cephfs_test_runner:test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:07:36.374 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-06T03:07:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.487+0000 7fc71ffff700 1 Processor -- start 2023-10-06T03:07:36.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.487+0000 7fc71ffff700 1 -- start start 2023-10-06T03:07:36.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71ffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc720143050 0x7fc720159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:36.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc72006b9e0 0x7fc72006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:36.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71ffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc72006c390 con 0x7fc7200b3810 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71ffff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc720143420 con 0x7fc72006b9e0 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71ffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72006a1e0 con 0x7fc720143050 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc72006b9e0 0x7fc72006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc7200b3810 0x7fc7200b3c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:33080/0 (socket says 172.21.15.92:33080) 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71f7fe700 1 -- 172.21.15.92:0/1782812683 learned_addr learned my addr 172.21.15.92:0/1782812683 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.488+0000 7fc71effd700 1 --2- 172.21.15.92:0/1782812683 >> v2:172.21.15.136:3300/0 conn(0x7fc720143050 0x7fc720159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71e7fc700 1 -- 172.21.15.92:0/1782812683 >> v2:172.21.15.136:3300/0 conn(0x7fc720143050 msgr2=0x7fc720159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71e7fc700 1 --2- 172.21.15.92:0/1782812683 >> v2:172.21.15.136:3300/0 conn(0x7fc720143050 0x7fc720159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71e7fc700 1 -- 172.21.15.92:0/1782812683 >> v1:172.21.15.92:6789/0 conn(0x7fc7200b3810 legacy=0x7fc7200b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71dffb700 1 -- 172.21.15.92:0/1782812683 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487027988 0 0) 0x7fc72006c390 con 0x7fc7200b3810 2023-10-06T03:07:36.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71e7fc700 1 -- 172.21.15.92:0/1782812683 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7200695d0 con 0x7fc72006b9e0 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71e7fc700 1 --2- 172.21.15.92:0/1782812683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72006be50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc72006eec0 tx=0x7fc7140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6533c511bed09b9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71dffb700 1 -- 172.21.15.92:0/1782812683 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400b6b0 con 0x7fc72006b9e0 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71dffb700 1 -- 172.21.15.92:0/1782812683 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71400b810 con 0x7fc72006b9e0 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71dffb700 1 -- 172.21.15.92:0/1782812683 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400bbb0 con 0x7fc72006b9e0 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71ffff700 1 -- 172.21.15.92:0/1782812683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 msgr2=0x7fc72006be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71ffff700 1 --2- 172.21.15.92:0/1782812683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72006be50 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc72006eec0 tx=0x7fc7140091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71ffff700 1 -- 172.21.15.92:0/1782812683 shutdown_connections 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71ffff700 1 --2- 172.21.15.92:0/1782812683 >> v2:172.21.15.136:3300/0 conn(0x7fc720143050 0x7fc720159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71ffff700 1 --2- 172.21.15.92:0/1782812683 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72006be50 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.489+0000 7fc71ffff700 1 -- 172.21.15.92:0/1782812683 >> 172.21.15.92:0/1782812683 conn(0x7fc7200ce430 msgr2=0x7fc7200ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.490+0000 7fc71ffff700 1 -- 172.21.15.92:0/1782812683 shutdown_connections 2023-10-06T03:07:36.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.490+0000 7fc71ffff700 1 -- 172.21.15.92:0/1782812683 wait complete. 2023-10-06T03:07:36.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 Processor -- start 2023-10-06T03:07:36.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 -- start start 2023-10-06T03:07:36.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72015c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:36.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7200b3810 0x7fc72013dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:36.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc720143050 0x7fc7201582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72013ff20 con 0x7fc72006b9e0 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72015ba00 con 0x7fc720143050 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.491+0000 7fc71ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72015bce0 con 0x7fc7200b3810 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72015c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72015c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41194/0 (socket says 172.21.15.92:41194) 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 -- 172.21.15.92:0/2615580766 learned_addr learned my addr 172.21.15.92:0/2615580766 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71f7fe700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc720143050 0x7fc7201582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71e7fc700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7200b3810 0x7fc72013dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:36.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 -- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7200b3810 msgr2=0x7fc72013dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7200b3810 0x7fc72013dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 -- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc720143050 msgr2=0x7fc7201582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc720143050 0x7fc7201582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 -- 172.21.15.92:0/2615580766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714009420 con 0x7fc72006b9e0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71effd700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72015c7c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc710009fd0 tx=0x7fc710009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000ec70 con 0x7fc72006b9e0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71000edd0 con 0x7fc72006b9e0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72015bfc0 con 0x7fc72006b9e0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710014650 con 0x7fc72006b9e0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.492+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc720158800 con 0x7fc72006b9e0 2023-10-06T03:07:36.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.493+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72004c3c0 con 0x7fc72006b9e0 2023-10-06T03:07:36.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.494+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc710014a40 con 0x7fc72006b9e0 2023-10-06T03:07:36.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.494+0000 7fc706ffd700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc70c067290 0x7fc70c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:36.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.494+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc710084f40 con 0x7fc72006b9e0 2023-10-06T03:07:36.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.494+0000 7fc71e7fc700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc70c067290 0x7fc70c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:36.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.494+0000 7fc71e7fc700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc70c067290 0x7fc70c069740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7140093f0 tx=0x7fc714017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:36.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.498+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc710051f40 con 0x7fc72006b9e0 2023-10-06T03:07:36.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:36.758+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7fc7200adad0 con 0x7fc72006b9e0 2023-10-06T03:07:37.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.353+0000 7fc706ffd700 1 -- 172.21.15.92:0/2615580766 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v287) v1 ==== 159+0+0 (secure 0 0 0) 0x7fc710009a00 con 0x7fc72006b9e0 2023-10-06T03:07:37.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc70c067290 msgr2=0x7fc70c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc70c067290 0x7fc70c069740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc7140093f0 tx=0x7fc714017000 comp rx=0 tx=0).stop 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 msgr2=0x7fc72015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72015c7c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc710009fd0 tx=0x7fc710009970 comp rx=0 tx=0).stop 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 shutdown_connections 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc70c067290 0x7fc70c069740 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc72006b9e0 0x7fc72015c7c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc720143050 0x7fc7201582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 --2- 172.21.15.92:0/2615580766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7200b3810 0x7fc72013dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 >> 172.21.15.92:0/2615580766 conn(0x7fc7200ce430 msgr2=0x7fc72015f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:37.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 shutdown_connections 2023-10-06T03:07:37.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.355+0000 7fc71ffff700 1 -- 172.21.15.92:0/2615580766 wait complete. 2023-10-06T03:07:37.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:07:37.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.486+0000 7f18d6de4700 1 Processor -- start 2023-10-06T03:07:37.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.486+0000 7f18d6de4700 1 -- start start 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.486+0000 7f18d6de4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18d0159890 0x7f18d015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d6de4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159c60 0x7f18d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d6de4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d015a160 con 0x7f18d015db10 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d6de4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00d13a0 con 0x7f18d0159890 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d6de4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015e140 con 0x7f18d0159c60 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18d0159890 0x7f18d015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18d0159890 0x7f18d015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41202/0 (socket says 172.21.15.92:41202) 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 -- 172.21.15.92:0/1659726774 learned_addr learned my addr 172.21.15.92:0/1659726774 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18cffff700 1 --2- 172.21.15.92:0/1659726774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159c60 0x7f18d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 -- 172.21.15.92:0/1659726774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159c60 msgr2=0x7f18d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 --2- 172.21.15.92:0/1659726774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159c60 0x7f18d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 -- 172.21.15.92:0/1659726774 >> v1:172.21.15.136:6789/0 conn(0x7f18d015db10 legacy=0x7f18d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 -- 172.21.15.92:0/1659726774 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d015a3d0 con 0x7f18d0159890 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18d4b80700 1 --2- 172.21.15.92:0/1659726774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159890 0x7f18d015abd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009f20 tx=0x7f18c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409d93e9504803e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18cf7fe700 1 -- 172.21.15.92:0/1659726774 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000e070 con 0x7f18d0159890 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18cf7fe700 1 -- 172.21.15.92:0/1659726774 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000ba80 con 0x7f18d0159890 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.487+0000 7f18cf7fe700 1 -- 172.21.15.92:0/1659726774 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c00092e0 con 0x7f18d0159890 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 -- 172.21.15.92:0/1659726774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159890 msgr2=0x7f18d015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 --2- 172.21.15.92:0/1659726774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159890 0x7f18d015abd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009f20 tx=0x7f18c0009840 comp rx=0 tx=0).stop 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 -- 172.21.15.92:0/1659726774 shutdown_connections 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 --2- 172.21.15.92:0/1659726774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159890 0x7f18d015abd0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 --2- 172.21.15.92:0/1659726774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159c60 0x7f18d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 -- 172.21.15.92:0/1659726774 >> 172.21.15.92:0/1659726774 conn(0x7f18d0068e20 msgr2=0x7f18d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 -- 172.21.15.92:0/1659726774 shutdown_connections 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.488+0000 7f18d6de4700 1 -- 172.21.15.92:0/1659726774 wait complete. 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.489+0000 7f18d6de4700 1 Processor -- start 2023-10-06T03:07:37.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.489+0000 7f18d6de4700 1 -- start start 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d6de4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159890 0x7f18d00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d6de4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 0x7f18d014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d6de4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d015db10 0x7f18d015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d6de4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00cfd40 con 0x7f18d0159c60 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d6de4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b3ad0 con 0x7f18d015db10 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d6de4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00b3d00 con 0x7f18d0159890 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 0x7f18d014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 0x7f18d014c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41212/0 (socket says 172.21.15.92:41212) 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 -- 172.21.15.92:0/50562931 learned_addr learned my addr 172.21.15.92:0/50562931 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18d4b80700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159890 0x7f18d00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 -- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159890 msgr2=0x7f18d00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159890 0x7f18d00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 -- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d015db10 msgr2=0x7f18d015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d015db10 0x7f18d015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 -- 172.21.15.92:0/50562931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009ae0 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.490+0000 7f18cffff700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 0x7f18d014c8a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d890 tx=0x7f18b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.491+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80125f0 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.491+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8012770 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.491+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d00b3fe0 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.491+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8012b30 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.491+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d00b45a0 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.492+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d004c3c0 con 0x7f18d0159c60 2023-10-06T03:07:37.496 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.492+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f18b801e460 con 0x7f18d0159c60 2023-10-06T03:07:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.492+0000 7f18ccff9700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18b0067290 0x7f18b0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.492+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f18b8084fe0 con 0x7f18d0159c60 2023-10-06T03:07:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.492+0000 7f18d4b80700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18b0067290 0x7f18b0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.493+0000 7f18d4b80700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18b0067290 0x7f18b0069740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009880 tx=0x7f18c0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:37.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.496+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18b8051fe0 con 0x7f18d0159c60 2023-10-06T03:07:37.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.680+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f18d006a5b0 con 0x7f18d0159c60 2023-10-06T03:07:37.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.686+0000 7f18ccff9700 1 -- 172.21.15.92:0/50562931 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f18b8055600 con 0x7f18d0159c60 2023-10-06T03:07:37.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18b0067290 msgr2=0x7f18b0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18b0067290 0x7f18b0069740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009880 tx=0x7f18c0018000 comp rx=0 tx=0).stop 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 msgr2=0x7f18d014c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 0x7f18d014c8a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d890 tx=0x7f18b8008040 comp rx=0 tx=0).stop 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 shutdown_connections 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f18b0067290 0x7f18b0069740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d015db10 0x7f18d015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d0159890 0x7f18d00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 --2- 172.21.15.92:0/50562931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18d0159c60 0x7f18d014c8a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 >> 172.21.15.92:0/50562931 conn(0x7f18d0068e20 msgr2=0x7f18d014d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:37.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 shutdown_connections 2023-10-06T03:07:37.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.688+0000 7f18d6de4700 1 -- 172.21.15.92:0/50562931 wait complete. 2023-10-06T03:07:37.700 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:07:37.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:37.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:07:37.798 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:07:37.798 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:07:37.798 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:37.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.912+0000 7f18ad79f700 1 Processor -- start 2023-10-06T03:07:37.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.913+0000 7f18ad79f700 1 -- start start 2023-10-06T03:07:37.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.913+0000 7f18ad79f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18a8159c60 0x7f18a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.913+0000 7f18ad79f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f18a815db10 0x7f18a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.913+0000 7f18ad79f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a815a160 con 0x7f18a8159890 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.913+0000 7f18ad79f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a815df50 con 0x7f18a8159c60 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.913+0000 7f18ad79f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a80d13a0 con 0x7f18a815db10 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18a8159c60 0x7f18a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f18a8159c60 0x7f18a815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41226/0 (socket says 172.21.15.92:41226) 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 -- 172.21.15.92:0/2309566037 learned_addr learned my addr 172.21.15.92:0/2309566037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f18a77fe700 1 --2- 172.21.15.92:0/2309566037 >> v2:172.21.15.136:3300/0 conn(0x7f18a815db10 0x7f18a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 -- 172.21.15.92:0/2309566037 >> v2:172.21.15.136:3300/0 conn(0x7f18a815db10 msgr2=0x7f18a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 --2- 172.21.15.92:0/2309566037 >> v2:172.21.15.136:3300/0 conn(0x7f18a815db10 0x7f18a8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 -- 172.21.15.92:0/2309566037 >> v1:172.21.15.136:6789/0 conn(0x7f18a8159890 legacy=0x7f18a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:37.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 -- 172.21.15.92:0/2309566037 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a815e140 con 0x7f18a8159c60 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f189e7fc700 1 --2- 172.21.15.92:0/2309566037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159c60 0x7f18a815d5d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f18a806a890 tx=0x7f1890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25d14fdb0a99484 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f18a67fc700 1 -- 172.21.15.92:0/2309566037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000b860 con 0x7f18a8159c60 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f18a67fc700 1 -- 172.21.15.92:0/2309566037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189000b9c0 con 0x7f18a8159c60 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.914+0000 7f18a67fc700 1 -- 172.21.15.92:0/2309566037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000bd60 con 0x7f18a8159c60 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 -- 172.21.15.92:0/2309566037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159c60 msgr2=0x7f18a815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 --2- 172.21.15.92:0/2309566037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159c60 0x7f18a815d5d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f18a806a890 tx=0x7f1890009250 comp rx=0 tx=0).stop 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 -- 172.21.15.92:0/2309566037 shutdown_connections 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 --2- 172.21.15.92:0/2309566037 >> v2:172.21.15.136:3300/0 conn(0x7f18a815db10 0x7f18a8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 --2- 172.21.15.92:0/2309566037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159c60 0x7f18a815d5d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 -- 172.21.15.92:0/2309566037 >> 172.21.15.92:0/2309566037 conn(0x7f18a8068e20 msgr2=0x7f18a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 -- 172.21.15.92:0/2309566037 shutdown_connections 2023-10-06T03:07:37.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.915+0000 7f18ad79f700 1 -- 172.21.15.92:0/2309566037 wait complete. 2023-10-06T03:07:37.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.916+0000 7f18ad79f700 1 Processor -- start 2023-10-06T03:07:37.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.916+0000 7f18ad79f700 1 -- start start 2023-10-06T03:07:37.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 0x7f18a8140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a8159c60 0x7f18a80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18a815db10 0x7f18a80b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8141300 con 0x7f18a8159890 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cd6d0 con 0x7f18a8159c60 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cd9b0 con 0x7f18a815db10 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 0x7f18a8140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 0x7f18a8140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41242/0 (socket says 172.21.15.92:41242) 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 -- 172.21.15.92:0/4213444126 learned_addr learned my addr 172.21.15.92:0/4213444126 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f189e7fc700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a8159c60 0x7f18a80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 -- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18a815db10 msgr2=0x7f18a80b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18a815db10 0x7f18a80b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 -- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a8159c60 msgr2=0x7f18a80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:37.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a8159c60 0x7f18a80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 -- 172.21.15.92:0/4213444126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900094f0 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f189e7fc700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a8159c60 0x7f18a80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18a6ffd700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 0x7f18a8140dc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f189800c590 tx=0x7f189800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898011590 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18980116f0 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a80cdc90 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898011ab0 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.917+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a80ce250 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.918+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a80ce440 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.919+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f189801b3b0 con 0x7f18a8159890 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.919+0000 7f189f7fe700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1888067290 0x7f1888069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:37.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.919+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1898083e80 con 0x7f18a8159890 2023-10-06T03:07:37.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.919+0000 7f189e7fc700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1888067290 0x7f1888069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:37.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.919+0000 7f189e7fc700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1888067290 0x7f1888069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f18a806a890 tx=0x7f1890017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:37.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:37.923+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1898050e80 con 0x7f18a8159890 2023-10-06T03:07:38.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.148+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a804c3c0 con 0x7f18a8159890 2023-10-06T03:07:38.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.149+0000 7f189f7fe700 1 -- 172.21.15.92:0/4213444126 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f1898017070 con 0x7f18a8159890 2023-10-06T03:07:38.150 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:38.150 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5561,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/75515509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":75515509},{"type":"v1","addr":"172.21.15.136:6835","nonce":75515509}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6100,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2120244844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2120244844},{"type":"v1","addr":"172.21.15.136:6834","nonce":2120244844}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6477,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3995812341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3995812341},{"type":"v1","addr":"172.21.15.92:6836","nonce":3995812341}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6768,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/4172929230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":4172929230},{"type":"v1","addr":"172.21.15.92:6837","nonce":4172929230}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[]} 2023-10-06T03:07:38.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.150+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1888067290 msgr2=0x7f1888069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:38.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.150+0000 7f18ad79f700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1888067290 0x7f1888069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f18a806a890 tx=0x7f1890017000 comp rx=0 tx=0).stop 2023-10-06T03:07:38.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.150+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 msgr2=0x7f18a8140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 0x7f18a8140dc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f189800c590 tx=0x7f189800c950 comp rx=0 tx=0).stop 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 shutdown_connections 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1888067290 0x7f1888069740 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f18a8159890 0x7f18a8140dc0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18a815db10 0x7f18a80b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 --2- 172.21.15.92:0/4213444126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18a8159c60 0x7f18a80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 >> 172.21.15.92:0/4213444126 conn(0x7f18a8068e20 msgr2=0x7f18a8157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 shutdown_connections 2023-10-06T03:07:38.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:38.151+0000 7f18ad79f700 1 -- 172.21.15.92:0/4213444126 wait complete. 2023-10-06T03:07:38.153 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 59 2023-10-06T03:07:38.163 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:07:38.164 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:07:38.164 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:07:38.164 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:38.164 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:07:38.164 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:07:38.164 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:07:38.165 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:38.165 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:07:38.165 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:07:38.165 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:07:38.165 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:38.165 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:07:38.165 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:07:38.166 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:07:38.166 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:07:38.172 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:07:38.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:07:38.417 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:07:38.417 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:07:38.459 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:07:38.460 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:38.461 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:07:38.461 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:38.531 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:07:38.532 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:07:38.532 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:07:38.531+0000 7f1904a98700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T03:07:38.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-06T03:07:38.620 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T03:07:38.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-06T03:07:42.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:07:42.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:07:44.419 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:07:44.419 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:07:44.419 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:07:44.419 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:07:44.422 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:07:44.422 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:44.423 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:07:44.423 DEBUG:teuthology.parallel:result is None 2023-10-06T03:07:44.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:07:44.505 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:07:44.505 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:07:44.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.548+0000 7f286a6eb700 1 Processor -- start 2023-10-06T03:07:44.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.549+0000 7f286a6eb700 1 -- start start 2023-10-06T03:07:44.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.549+0000 7f286a6eb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2864140720 0x7f2864068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.549+0000 7f286a6eb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 0x7f28640c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.549+0000 7f286a6eb700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28640c7de0 con 0x7f2864140350 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.549+0000 7f286a6eb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2864068d50 con 0x7f2864140720 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.549+0000 7f286a6eb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2864140c10 con 0x7f28640688d0 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2868c88700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 0x7f28640c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2864140720 0x7f2864068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2864140720 0x7f2864068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:47930/0 (socket says 172.21.15.92:47930) 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2868c88700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 0x7f28640c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:56296/0 (socket says 172.21.15.92:56296) 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2863fff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f2864140350 0x7f2864069c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:51152/0 (socket says 172.21.15.92:51152) 2023-10-06T03:07:44.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 -- 172.21.15.92:0/3452087266 learned_addr learned my addr 172.21.15.92:0/3452087266 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 -- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 msgr2=0x7f28640c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 --2- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 0x7f28640c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 -- 172.21.15.92:0/3452087266 >> v1:172.21.15.136:6789/0 conn(0x7f2864140350 legacy=0x7f2864069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 -- 172.21.15.92:0/3452087266 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864141490 con 0x7f2864140720 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2868c88700 1 --2- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 0x7f28640c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2862ffd700 1 -- 172.21.15.92:0/3452087266 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936667776 0 0) 0x7f28640c7de0 con 0x7f2864140350 2023-10-06T03:07:44.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f28637fe700 1 --2- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2864140720 0x7f2864068390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28580099b0 tx=0x7f2858009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86ae52a622b12bad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:44.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2862ffd700 1 -- 172.21.15.92:0/3452087266 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858009ea0 con 0x7f2864140720 2023-10-06T03:07:44.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2862ffd700 1 -- 172.21.15.92:0/3452087266 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800b860 con 0x7f2864140720 2023-10-06T03:07:44.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.550+0000 7f2862ffd700 1 -- 172.21.15.92:0/3452087266 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800bc00 con 0x7f2864140720 2023-10-06T03:07:44.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 -- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2864140720 msgr2=0x7f2864068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:44.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2864140720 0x7f2864068390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28580099b0 tx=0x7f2858009250 comp rx=0 tx=0).stop 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 -- 172.21.15.92:0/3452087266 shutdown_connections 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28640688d0 0x7f28640c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3452087266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2864140720 0x7f2864068390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 -- 172.21.15.92:0/3452087266 >> 172.21.15.92:0/3452087266 conn(0x7f28640cd670 msgr2=0x7f28640cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 -- 172.21.15.92:0/3452087266 shutdown_connections 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.551+0000 7f286a6eb700 1 -- 172.21.15.92:0/3452087266 wait complete. 2023-10-06T03:07:44.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.552+0000 7f286a6eb700 1 Processor -- start 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.552+0000 7f286a6eb700 1 -- start start 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f286a6eb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28640688d0 0x7f28640b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f286a6eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 0x7f2864155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f286a6eb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2864140720 0x7f28640c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f286a6eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640b1d90 con 0x7f2864140350 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f286a6eb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28640b2050 con 0x7f28640688d0 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f286a6eb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28640b2330 con 0x7f2864140720 2023-10-06T03:07:44.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 0x7f2864155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 0x7f2864155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56650/0 (socket says 172.21.15.92:56650) 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 -- 172.21.15.92:0/3445606185 learned_addr learned my addr 172.21.15.92:0/3445606185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f2863fff700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28640688d0 0x7f28640b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f2868c88700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2864140720 0x7f28640c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 -- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2864140720 msgr2=0x7f28640c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2864140720 0x7f28640c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 -- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28640688d0 msgr2=0x7f28640b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28640688d0 0x7f28640b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f28637fe700 1 -- 172.21.15.92:0/3445606185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c009a20 con 0x7f2864140350 2023-10-06T03:07:44.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.553+0000 7f2868c88700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2864140720 0x7f28640c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.554+0000 7f28637fe700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 0x7f2864155fe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2858006fd0 tx=0x7f2858017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.554+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800b9d0 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.554+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800bb50 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.554+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28580094f0 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.554+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858025670 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.554+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28640b28f0 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.555+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28640cd450 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.555+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2858023070 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.556+0000 7f284bfff700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2850067290 0x7f2850069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.556+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2858085e00 con 0x7f2864140350 2023-10-06T03:07:44.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.556+0000 7f2863fff700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2850067290 0x7f2850069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:44.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.556+0000 7f2863fff700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2850067290 0x7f2850069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f28640cb400 tx=0x7f2854007d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:44.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.560+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2858052e00 con 0x7f2864140350 2023-10-06T03:07:44.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:44.758+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f28640b2b10 con 0x7f2864140350 2023-10-06T03:07:45.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-10-06T03:07:45.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T03:07:45.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-06T03:07:45.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.537+0000 7f284bfff700 1 -- 172.21.15.92:0/3445606185 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2858056420 con 0x7f2864140350 2023-10-06T03:07:45.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2850067290 msgr2=0x7f2850069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2850067290 0x7f2850069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f28640cb400 tx=0x7f2854007d20 comp rx=0 tx=0).stop 2023-10-06T03:07:45.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 msgr2=0x7f2864155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 0x7f2864155fe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2858006fd0 tx=0x7f2858017000 comp rx=0 tx=0).stop 2023-10-06T03:07:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 shutdown_connections 2023-10-06T03:07:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2850067290 0x7f2850069740 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2864140720 0x7f28640c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28640688d0 0x7f28640b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 --2- 172.21.15.92:0/3445606185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2864140350 0x7f2864155fe0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 >> 172.21.15.92:0/3445606185 conn(0x7f28640cd670 msgr2=0x7f286413bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:45.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 shutdown_connections 2023-10-06T03:07:45.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.539+0000 7f286a6eb700 1 -- 172.21.15.92:0/3445606185 wait complete. 2023-10-06T03:07:45.543 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:07:45.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:07:45.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.676+0000 7fad14996700 1 Processor -- start 2023-10-06T03:07:45.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.676+0000 7fad14996700 1 -- start start 2023-10-06T03:07:45.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad14996700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fad1015d230 0x7fad1015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad14996700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad100b2980 con 0x7fad101597f0 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad14996700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1015db70 con 0x7fad10159420 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad14996700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad100b11d0 con 0x7fad1015d230 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad0effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad101597f0 0x7fad100b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36322/0 (socket says 172.21.15.92:36322) 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad0effd700 1 -- 172.21.15.92:0/1459492652 learned_addr learned my addr 172.21.15.92:0/1459492652 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588010461 0 0) 0x7fad100b2980 con 0x7fad101597f0 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf8003570 con 0x7fad101597f0 2023-10-06T03:07:45.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.677+0000 7fad0ffff700 1 --2- 172.21.15.92:0/1459492652 >> v2:172.21.15.136:3300/0 conn(0x7fad1015d230 0x7fad1015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627502882 0 0) 0x7facf8003570 con 0x7fad101597f0 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 >> v2:172.21.15.136:3300/0 conn(0x7fad1015d230 msgr2=0x7fad1015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 --2- 172.21.15.92:0/1459492652 >> v2:172.21.15.136:3300/0 conn(0x7fad1015d230 0x7fad1015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 >> v1:172.21.15.136:6789/0 conn(0x7fad10159420 legacy=0x7fad100b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100b14b0 con 0x7fad101597f0 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fad04001470 con 0x7fad101597f0 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 >> v1:172.21.15.92:6789/0 conn(0x7fad101597f0 legacy=0x7fad100b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8003a60 0x7facf8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7facf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf80051c0 0x7facf800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100b11d0 con 0x7facf8004810 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf800c0d0 con 0x7facf8003a60 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf800c290 con 0x7facf80051c0 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7facf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0f7fe700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8003a60 0x7facf8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0ffff700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf80051c0 0x7facf800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 -- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf80051c0 msgr2=0x7facf800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:45.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf80051c0 0x7facf800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 -- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8003a60 msgr2=0x7facf8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf8003a60 0x7facf8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 -- 172.21.15.92:0/1459492652 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8003570 con 0x7facf8004810 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0ffff700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf80051c0 0x7facf800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0ffff700 1 -- 172.21.15.92:0/1459492652 reap_dead start 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0effd700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7facf8004c80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fad10159c70 tx=0x7fad1015dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1c92152906be6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.678+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400cc30 con 0x7facf8004810 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0400cd90 con 0x7facf8004810 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad0e7fc700 1 -- 172.21.15.92:0/1459492652 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad04005b70 con 0x7facf8004810 2023-10-06T03:07:45.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 -- 172.21.15.92:0/1459492652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 msgr2=0x7facf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7facf8004c80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fad10159c70 tx=0x7fad1015dd00 comp rx=0 tx=0).stop 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 -- 172.21.15.92:0/1459492652 shutdown_connections 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 --2- 172.21.15.92:0/1459492652 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7facf8004c80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 -- 172.21.15.92:0/1459492652 >> 172.21.15.92:0/1459492652 conn(0x7fad100d0660 msgr2=0x7fad100d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 -- 172.21.15.92:0/1459492652 shutdown_connections 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.679+0000 7fad14996700 1 -- 172.21.15.92:0/1459492652 wait complete. 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.680+0000 7fad14996700 1 Processor -- start 2023-10-06T03:07:45.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.680+0000 7fad14996700 1 -- start start 2023-10-06T03:07:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad14996700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7fad1016c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad14996700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad1016a5d0 0x7fad1015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad14996700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1015caf0 0x7fad101693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad14996700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad100c6d70 con 0x7facf8004810 2023-10-06T03:07:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad14996700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1015a350 con 0x7fad1016a5d0 2023-10-06T03:07:45.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad14996700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1015cf40 con 0x7fad1015caf0 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7fad1016c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7fad1016c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56676/0 (socket says 172.21.15.92:56676) 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 -- 172.21.15.92:0/4246152849 learned_addr learned my addr 172.21.15.92:0/4246152849 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0effd700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad1016a5d0 0x7fad1015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0ffff700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1015caf0 0x7fad101693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 -- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1015caf0 msgr2=0x7fad101693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1015caf0 0x7fad101693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 -- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad1016a5d0 msgr2=0x7fad1015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad1016a5d0 0x7fad1015c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0f7fe700 1 -- 172.21.15.92:0/4246152849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10159d40 con 0x7facf8004810 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.681+0000 7fad0ffff700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1015caf0 0x7fad101693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.682+0000 7fad0f7fe700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7fad1016c150 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fad0000eb70 tx=0x7fad00001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.682+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000c910 con 0x7facf8004810 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.682+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0000ca70 con 0x7facf8004810 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.682+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0000ce10 con 0x7facf8004810 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.682+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1016a050 con 0x7facf8004810 2023-10-06T03:07:45.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.682+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1016a330 con 0x7facf8004810 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.683+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1015a1a0 con 0x7facf8004810 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.683+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fad00013630 con 0x7facf8004810 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.683+0000 7facf77fe700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facfc067290 0x7facfc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.683+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fad00084aa0 con 0x7facf8004810 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.683+0000 7fad0effd700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facfc067290 0x7facfc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.684+0000 7fad0effd700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facfc067290 0x7facfc069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fad04007cd0 tx=0x7fad04015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:45.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.688+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad00051b70 con 0x7facf8004810 2023-10-06T03:07:45.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.948+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fad1004c3c0 con 0x7facf8004810 2023-10-06T03:07:45.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.948+0000 7facf77fe700 1 -- 172.21.15.92:0/4246152849 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fad00055190 con 0x7facf8004810 2023-10-06T03:07:45.951 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:45.952 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:07:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:07:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.952 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:07:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:07:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.953 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.955 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.956 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:45.957 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.958 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:07:45.959 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.960 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.961 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:07:45.962 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:07:45.963 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.964 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:07:45.965 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.966 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:07:45.967 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:07:45.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:07:45.969 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:07:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:45.970 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:07:45.970 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facfc067290 msgr2=0x7facfc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facfc067290 0x7facfc069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fad04007cd0 tx=0x7fad04015000 comp rx=0 tx=0).stop 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 msgr2=0x7fad1016c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7fad1016c150 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fad0000eb70 tx=0x7fad00001640 comp rx=0 tx=0).stop 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 shutdown_connections 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facfc067290 0x7facfc069740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf8004810 0x7fad1016c150 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1015caf0 0x7fad101693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 --2- 172.21.15.92:0/4246152849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad1016a5d0 0x7fad1015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.950+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 >> 172.21.15.92:0/4246152849 conn(0x7fad100d0660 msgr2=0x7fad10068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.951+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 shutdown_connections 2023-10-06T03:07:45.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:45.951+0000 7fad14996700 1 -- 172.21.15.92:0/4246152849 wait complete. 2023-10-06T03:07:45.973 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.086+0000 7f05e7866700 1 Processor -- start 2023-10-06T03:07:46.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.086+0000 7f05e7866700 1 -- start start 2023-10-06T03:07:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e7866700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e7866700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05e015db10 0x7f05e0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e7866700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e015a160 con 0x7f05e0159890 2023-10-06T03:07:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e7866700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00d13a0 con 0x7f05e015db10 2023-10-06T03:07:46.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e7866700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015e140 con 0x7f05e0159c60 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e5e03700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f05e015db10 0x7f05e0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e5602700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f05e0159890 0x7f05e015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:51156/0 (socket says 172.21.15.92:51156) 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e5602700 1 -- 172.21.15.92:0/3310746582 learned_addr learned my addr 172.21.15.92:0/3310746582 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.087+0000 7f05e4e01700 1 --2- 172.21.15.92:0/3310746582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05e5e03700 1 -- 172.21.15.92:0/3310746582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 msgr2=0x7f05e015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05e5e03700 1 --2- 172.21.15.92:0/3310746582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05e5e03700 1 -- 172.21.15.92:0/3310746582 >> v1:172.21.15.136:6789/0 conn(0x7f05e0159890 legacy=0x7f05e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05e5e03700 1 -- 172.21.15.92:0/3310746582 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e015a3d0 con 0x7f05e015db10 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05e5e03700 1 --2- 172.21.15.92:0/3310746582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e015db10 0x7f05e0069560 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098e0 tx=0x7f05d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54bb050231db4df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05d7fff700 1 -- 172.21.15.92:0/3310746582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800b640 con 0x7f05e015db10 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05d7fff700 1 -- 172.21.15.92:0/3310746582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d800b7a0 con 0x7f05e015db10 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.088+0000 7f05d7fff700 1 -- 172.21.15.92:0/3310746582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d800bb40 con 0x7f05e015db10 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 -- 172.21.15.92:0/3310746582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e015db10 msgr2=0x7f05e0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 --2- 172.21.15.92:0/3310746582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e015db10 0x7f05e0069560 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f05d80098e0 tx=0x7f05d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 -- 172.21.15.92:0/3310746582 shutdown_connections 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 --2- 172.21.15.92:0/3310746582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e015db10 0x7f05e0069560 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 --2- 172.21.15.92:0/3310746582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 -- 172.21.15.92:0/3310746582 >> 172.21.15.92:0/3310746582 conn(0x7f05e0068e20 msgr2=0x7f05e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 -- 172.21.15.92:0/3310746582 shutdown_connections 2023-10-06T03:07:46.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.089+0000 7f05e7866700 1 -- 172.21.15.92:0/3310746582 wait complete. 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.090+0000 7f05e7866700 1 Processor -- start 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.090+0000 7f05e7866700 1 -- start start 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e7866700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 0x7f05e014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e7866700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e7866700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e015db10 0x7f05e014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e7866700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b2240 con 0x7f05e0159890 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e7866700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00cd2d0 con 0x7f05e015db10 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e7866700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b16c0 con 0x7f05e0159c60 2023-10-06T03:07:46.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 0x7f05e014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 0x7f05e014a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56686/0 (socket says 172.21.15.92:56686) 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 -- 172.21.15.92:0/1044872026 learned_addr learned my addr 172.21.15.92:0/1044872026 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5e03700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e015db10 0x7f05e014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e4e01700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 -- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 msgr2=0x7f05e014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e014afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 -- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e015db10 msgr2=0x7f05e014b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e015db10 0x7f05e014b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 -- 172.21.15.92:0/1044872026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8009420 con 0x7f05e0159890 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e4e01700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:46.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.091+0000 7f05e5602700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 0x7f05e014a550 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00cd30 tx=0x7f05dc001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.092+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc00ac00 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.092+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc00ad60 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.092+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e00b19a0 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.092+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc004e00 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.092+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00b1f60 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.093+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e004c3c0 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.093+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f05dc007300 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.093+0000 7f05d57fa700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f05c4067200 0x7f05c40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.093+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f05dc084400 con 0x7f05e0159890 2023-10-06T03:07:46.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.093+0000 7f05e4e01700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f05c4067200 0x7f05c40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.094+0000 7f05e4e01700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f05c4067200 0x7f05c40696b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f05e015d7b0 tx=0x7f05d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.098+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05dc051430 con 0x7f05e0159890 2023-10-06T03:07:46.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.282+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05e006a5b0 con 0x7f05e0159890 2023-10-06T03:07:46.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.283+0000 7f05d57fa700 1 -- 172.21.15.92:0/1044872026 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13651 (secure 0 0 0) 0x7f05dc054a50 con 0x7f05e0159890 2023-10-06T03:07:46.285 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:46.285 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":99,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:45.532378+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:46.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f05c4067200 msgr2=0x7f05c40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f05c4067200 0x7f05c40696b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f05e015d7b0 tx=0x7f05d0008040 comp rx=0 tx=0).stop 2023-10-06T03:07:46.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 msgr2=0x7f05e014a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 0x7f05e014a550 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00cd30 tx=0x7f05dc001ad0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 shutdown_connections 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f05c4067200 0x7f05c40696b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05e0159890 0x7f05e014a550 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05e015db10 0x7f05e014b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 --2- 172.21.15.92:0/1044872026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05e0159c60 0x7f05e014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 >> 172.21.15.92:0/1044872026 conn(0x7f05e0068e20 msgr2=0x7f05e0153de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 shutdown_connections 2023-10-06T03:07:46.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.285+0000 7f05e7866700 1 -- 172.21.15.92:0/1044872026 wait complete. 2023-10-06T03:07:46.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:07:46.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:07:46.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.415+0000 7f53256c8700 1 Processor -- start 2023-10-06T03:07:46.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.415+0000 7f53256c8700 1 -- start start 2023-10-06T03:07:46.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f53256c8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f53200b16d0 0x7f53200b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f53256c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b33b0 0x7f53200b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f53256c8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53200b2080 con 0x7f53200b4e70 2023-10-06T03:07:46.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f53256c8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53200cfb00 con 0x7f53200b16d0 2023-10-06T03:07:46.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f53256c8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200b5c40 con 0x7f53200b33b0 2023-10-06T03:07:46.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f531f7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f53200b16d0 0x7f53200b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f5324ec7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b33b0 0x7f53200b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f531ffff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f53200b4e70 0x7f53200ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:51164/0 (socket says 172.21.15.92:51164) 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f531ffff700 1 -- 172.21.15.92:0/203591608 learned_addr learned my addr 172.21.15.92:0/203591608 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f531effd700 1 -- 172.21.15.92:0/203591608 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426723531 0 0) 0x7f53200b2080 con 0x7f53200b4e70 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.416+0000 7f531effd700 1 -- 172.21.15.92:0/203591608 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5308003570 con 0x7f53200b4e70 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531f7fe700 1 -- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b33b0 msgr2=0x7f53200b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531f7fe700 1 --2- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b33b0 0x7f53200b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531f7fe700 1 -- 172.21.15.92:0/203591608 >> v1:172.21.15.136:6789/0 conn(0x7f53200b4e70 legacy=0x7f53200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531f7fe700 1 -- 172.21.15.92:0/203591608 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200b5f20 con 0x7f53200b16d0 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f5324ec7700 1 --2- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b33b0 0x7f53200b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531f7fe700 1 --2- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200b1b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f53140098e0 tx=0x7f53140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a3345e98fb41c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531effd700 1 -- 172.21.15.92:0/203591608 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314011630 con 0x7f53200b16d0 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531effd700 1 -- 172.21.15.92:0/203591608 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314011790 con 0x7f53200b16d0 2023-10-06T03:07:46.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f531effd700 1 -- 172.21.15.92:0/203591608 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314011b30 con 0x7f53200b16d0 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 -- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 msgr2=0x7f53200b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 --2- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200b1b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f53140098e0 tx=0x7f53140091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 -- 172.21.15.92:0/203591608 shutdown_connections 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 --2- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b33b0 0x7f53200b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 --2- 172.21.15.92:0/203591608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200b1b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 -- 172.21.15.92:0/203591608 >> 172.21.15.92:0/203591608 conn(0x7f53200d10c0 msgr2=0x7f53200affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 -- 172.21.15.92:0/203591608 shutdown_connections 2023-10-06T03:07:46.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.417+0000 7f53256c8700 1 -- 172.21.15.92:0/203591608 wait complete. 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 Processor -- start 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 -- start start 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 0x7f53200a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b4e70 0x7f5320169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200cb7d0 con 0x7f53200b33b0 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200a76e0 con 0x7f53200b16d0 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.419+0000 7f53256c8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200cb9c0 con 0x7f53200b4e70 2023-10-06T03:07:46.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 0x7f53200a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 0x7f53200a71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56688/0 (socket says 172.21.15.92:56688) 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 -- 172.21.15.92:0/6424594 learned_addr learned my addr 172.21.15.92:0/6424594 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531ffff700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f5324ec7700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b4e70 0x7f5320169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 -- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b4e70 msgr2=0x7f5320169e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b4e70 0x7f5320169e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 -- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 msgr2=0x7f53200cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200cb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 -- 172.21.15.92:0/6424594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009a20 con 0x7f53200b33b0 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f531f7fe700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 0x7f53200a71a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5314009ed0 tx=0x7f5314017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.425 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314011900 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314011aa0 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5314009420 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314025640 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.420+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53200ca360 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.421+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53200ca620 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.422+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5314025a30 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.422+0000 7f5307fff700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f530c067290 0x7f530c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.422+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5314085e30 con 0x7f53200b33b0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.422+0000 7f531ffff700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f530c067290 0x7f530c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.426 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.422+0000 7f531ffff700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f530c067290 0x7f530c069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5310007fd0 tx=0x7f5310009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.427 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.426+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5314052f40 con 0x7f53200b33b0 2023-10-06T03:07:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.608+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53200889e0 con 0x7f53200b33b0 2023-10-06T03:07:46.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.609+0000 7f5307fff700 1 -- 172.21.15.92:0/6424594 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53140150a0 con 0x7f53200b33b0 2023-10-06T03:07:46.611 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f530c067290 msgr2=0x7f530c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f530c067290 0x7f530c069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5310007fd0 tx=0x7f5310009040 comp rx=0 tx=0).stop 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 msgr2=0x7f53200a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 0x7f53200a71a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5314009ed0 tx=0x7f5314017000 comp rx=0 tx=0).stop 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 shutdown_connections 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f530c067290 0x7f530c069740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53200b4e70 0x7f5320169e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53200b16d0 0x7f53200cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 --2- 172.21.15.92:0/6424594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f53200b33b0 0x7f53200a71a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 >> 172.21.15.92:0/6424594 conn(0x7f53200d10c0 msgr2=0x7f53200ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:46.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 shutdown_connections 2023-10-06T03:07:46.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.611+0000 7f53256c8700 1 -- 172.21.15.92:0/6424594 wait complete. 2023-10-06T03:07:46.625 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:07:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.740+0000 7f44035fe700 1 Processor -- start 2023-10-06T03:07:46.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.741+0000 7f44035fe700 1 -- start start 2023-10-06T03:07:46.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.741+0000 7f44035fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43fc069d60 0x7f43fc0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.741+0000 7f44035fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.741+0000 7f44035fe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc0c8650 con 0x7f43fc0aca80 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.741+0000 7f44035fe700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc06bdb0 con 0x7f43fc069d60 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.741+0000 7f44035fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0cb370 con 0x7f43fc0c81f0 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43fc069d60 0x7f43fc0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f440139a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f43fc0aca80 0x7f43fc06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36326/0 (socket says 172.21.15.92:36326) 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f440139a700 1 -- 172.21.15.92:0/819148817 learned_addr learned my addr 172.21.15.92:0/819148817 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f43fc069d60 0x7f43fc0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56692/0 (socket says 172.21.15.92:56692) 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4401b9b700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 -- 172.21.15.92:0/819148817 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 msgr2=0x7f43fc0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:46.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 -- 172.21.15.92:0/819148817 >> v1:172.21.15.92:6789/0 conn(0x7f43fc0aca80 legacy=0x7f43fc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 -- 172.21.15.92:0/819148817 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc159f60 con 0x7f43fc069d60 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f43f3fff700 1 -- 172.21.15.92:0/819148817 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787065627 0 0) 0x7f43f8001290 con 0x7f43fc0aca80 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4401b9b700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f4400b99700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc069d60 0x7f43fc0c7cb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0098e0 tx=0x7f43ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df462eb574ab99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f43f3fff700 1 -- 172.21.15.92:0/819148817 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec009e80 con 0x7f43fc069d60 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f43f3fff700 1 -- 172.21.15.92:0/819148817 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec00b860 con 0x7f43fc069d60 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.742+0000 7f43f3fff700 1 -- 172.21.15.92:0/819148817 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00bbe0 con 0x7f43fc069d60 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 -- 172.21.15.92:0/819148817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc069d60 msgr2=0x7f43fc0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc069d60 0x7f43fc0c7cb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0098e0 tx=0x7f43ec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 -- 172.21.15.92:0/819148817 shutdown_connections 2023-10-06T03:07:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 --2- 172.21.15.92:0/819148817 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc069d60 0x7f43fc0c7cb0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 -- 172.21.15.92:0/819148817 >> 172.21.15.92:0/819148817 conn(0x7f43fc0cd670 msgr2=0x7f43fc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 -- 172.21.15.92:0/819148817 shutdown_connections 2023-10-06T03:07:46.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.743+0000 7f44035fe700 1 -- 172.21.15.92:0/819148817 wait complete. 2023-10-06T03:07:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.744+0000 7f44035fe700 1 Processor -- start 2023-10-06T03:07:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.744+0000 7f44035fe700 1 -- start start 2023-10-06T03:07:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f44035fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43fc069d60 0x7f43fc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f44035fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 0x7f43fc157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f44035fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f44035fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc157ed0 con 0x7f43fc0aca80 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f44035fe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0b1290 con 0x7f43fc069d60 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f44035fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0b09f0 con 0x7f43fc0c81f0 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 0x7f43fc157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 0x7f43fc157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56702/0 (socket says 172.21.15.92:56702) 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 -- 172.21.15.92:0/3185377028 learned_addr learned my addr 172.21.15.92:0/3185377028 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f440139a700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43fc069d60 0x7f43fc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4401b9b700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 -- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 msgr2=0x7f43fc0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 -- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43fc069d60 msgr2=0x7f43fc0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43fc069d60 0x7f43fc0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 -- 172.21.15.92:0/3185377028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec009420 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f4400b99700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 0x7f43fc157990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f43ec013000 tx=0x7f43ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00bbe0 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.745+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec011420 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.746+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc0b0cd0 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.746+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0117e0 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.746+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc16d690 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.746+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc16d880 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.747+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f43ec011bd0 con 0x7f43fc0aca80 2023-10-06T03:07:46.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.747+0000 7f43f17fa700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43e0067290 0x7f43e0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.747+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f43ec0863d0 con 0x7f43fc0aca80 2023-10-06T03:07:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.747+0000 7f440139a700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43e0067290 0x7f43e0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.748+0000 7f440139a700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43e0067290 0x7f43e0069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0aee30 tx=0x7f43f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:46.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.752+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43ec053420 con 0x7f43fc0aca80 2023-10-06T03:07:46.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.935+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43fc04c3c0 con 0x7f43fc0aca80 2023-10-06T03:07:46.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.935+0000 7f43f17fa700 1 -- 172.21.15.92:0/3185377028 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f43ec056a40 con 0x7f43fc0aca80 2023-10-06T03:07:46.937 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:07:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.937+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43e0067290 msgr2=0x7f43e0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.937+0000 7f44035fe700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43e0067290 0x7f43e0069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0aee30 tx=0x7f43f8009040 comp rx=0 tx=0).stop 2023-10-06T03:07:46.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.937+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 msgr2=0x7f43fc157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.937+0000 7f44035fe700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 0x7f43fc157990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f43ec013000 tx=0x7f43ec017000 comp rx=0 tx=0).stop 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.937+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 shutdown_connections 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43e0067290 0x7f43e0069740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43fc0c81f0 0x7f43fc0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43fc069d60 0x7f43fc0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 --2- 172.21.15.92:0/3185377028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f43fc0aca80 0x7f43fc157990 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 >> 172.21.15.92:0/3185377028 conn(0x7f43fc0cd670 msgr2=0x7f43fc158180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 shutdown_connections 2023-10-06T03:07:46.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:46.938+0000 7f44035fe700 1 -- 172.21.15.92:0/3185377028 wait complete. 2023-10-06T03:07:46.951 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:07:47.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.064+0000 7f5ccec1a700 1 Processor -- start 2023-10-06T03:07:47.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.064+0000 7f5ccec1a700 1 -- start start 2023-10-06T03:07:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.065+0000 7f5ccec1a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5cc80b1420 0x7f5cc80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.065+0000 7f5ccec1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80cc300 con 0x7f5cc80cbf30 2023-10-06T03:07:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.065+0000 7f5ccec1a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80ad370 con 0x7f5cc80ccf90 2023-10-06T03:07:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.065+0000 7f5ccec1a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc8156bf0 con 0x7f5cc80b1420 2023-10-06T03:07:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.065+0000 7f5ccc9b6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5cc80cbf30 0x7f5cc80b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:36338/0 (socket says 172.21.15.92:36338) 2023-10-06T03:07:47.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.065+0000 7f5ccc9b6700 1 -- 172.21.15.92:0/1101717660 learned_addr learned my addr 172.21.15.92:0/1101717660 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 --2- 172.21.15.92:0/1101717660 >> v2:172.21.15.136:3300/0 conn(0x7f5cc80b1420 0x7f5cc80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609713279 0 0) 0x7f5cc80cc300 con 0x7f5cc80cbf30 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb0003570 con 0x7f5cc80cbf30 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530228015 0 0) 0x7f5cc80ad370 con 0x7f5cc80ccf90 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc80cc300 con 0x7f5cc80ccf90 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2611617033 0 0) 0x7f5cb0003570 con 0x7f5cc80cbf30 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 >> v2:172.21.15.136:3300/0 conn(0x7f5cc80b1420 msgr2=0x7f5cc80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 --2- 172.21.15.92:0/1101717660 >> v2:172.21.15.136:3300/0 conn(0x7f5cc80b1420 0x7f5cc80ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 >> v1:172.21.15.136:6789/0 conn(0x7f5cc80ccf90 legacy=0x7f5cc80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80ad650 con 0x7f5cc80cbf30 2023-10-06T03:07:47.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5cc40017f0 con 0x7f5cc80cbf30 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 >> v1:172.21.15.92:6789/0 conn(0x7f5cc80cbf30 legacy=0x7f5cc80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0003c40 0x7f5cb0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb0004810 0x7f5cb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cb0005210 0x7f5cb000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8156bf0 con 0x7f5cb0004810 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb000c130 con 0x7f5cb0005210 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb000c2c0 con 0x7f5cb0003c40 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb0004810 0x7f5cb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5ccc9b6700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0003c40 0x7f5cb0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5ccd1b7700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cb0005210 0x7f5cb000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 -- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0003c40 msgr2=0x7f5cb0004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:47.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0003c40 0x7f5cb0004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 -- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cb0005210 msgr2=0x7f5cb000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cb0005210 0x7f5cb000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5ccd1b7700 1 -- 172.21.15.92:0/1101717660 reap_dead start 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 -- 172.21.15.92:0/1101717660 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb0003570 con 0x7f5cb0004810 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5ccc9b6700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0003c40 0x7f5cb0004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.066+0000 7f5cbffff700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb0004810 0x7f5cb0004c80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80b1820 tx=0x7f5cb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35921ce9410b647 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb800e6e0 con 0x7f5cb0004810 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb800e840 con 0x7f5cb0004810 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5cbf7fe700 1 -- 172.21.15.92:0/1101717660 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb800ec00 con 0x7f5cb0004810 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 -- 172.21.15.92:0/1101717660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb0004810 msgr2=0x7f5cb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb0004810 0x7f5cb0004c80 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80b1820 tx=0x7f5cb80091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 -- 172.21.15.92:0/1101717660 shutdown_connections 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/1101717660 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cb0004810 0x7f5cb0004c80 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 -- 172.21.15.92:0/1101717660 >> 172.21.15.92:0/1101717660 conn(0x7f5cc806a320 msgr2=0x7f5cc80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 -- 172.21.15.92:0/1101717660 shutdown_connections 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.067+0000 7f5ccec1a700 1 -- 172.21.15.92:0/1101717660 wait complete. 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.068+0000 7f5ccec1a700 1 Processor -- start 2023-10-06T03:07:47.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.068+0000 7f5ccec1a700 1 -- start start 2023-10-06T03:07:47.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccec1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0004810 0x7f5cc8157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccec1a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc8159500 0x7f5cc8148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccec1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc80abef0 0x7f5cc80ac340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccec1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80a7760 con 0x7f5cc80abef0 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccec1a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc80ac880 con 0x7f5cc8159500 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccec1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc81582e0 con 0x7f5cb0004810 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccc9b6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0004810 0x7f5cc8157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccd1b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc80abef0 0x7f5cc80ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.069+0000 7f5ccc9b6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0004810 0x7f5cc8157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:56418/0 (socket says 172.21.15.92:56418) 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccc9b6700 1 -- 172.21.15.92:0/4023919564 learned_addr learned my addr 172.21.15.92:0/4023919564 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:47.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5cbffff700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc8159500 0x7f5cc8148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccd1b7700 1 -- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0004810 msgr2=0x7f5cc8157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccd1b7700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0004810 0x7f5cc8157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccd1b7700 1 -- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc8159500 msgr2=0x7f5cc8148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccd1b7700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc8159500 0x7f5cc8148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccd1b7700 1 -- 172.21.15.92:0/4023919564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc81585c0 con 0x7f5cc80abef0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccd1b7700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc80abef0 0x7f5cc80ac340 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cd410 tx=0x7f5cc0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0013420 con 0x7f5cc80abef0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc81588a0 con 0x7f5cc80abef0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc0013580 con 0x7f5cc80abef0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0013940 con 0x7f5cc80abef0 2023-10-06T03:07:47.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.070+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8167a90 con 0x7f5cc80abef0 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.071+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc80a78f0 con 0x7f5cc80abef0 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.071+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5cc0013cb0 con 0x7f5cc80abef0 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.072+0000 7f5cbcff9700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cb4067290 0x7f5cb4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.072+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5cc0084820 con 0x7f5cc80abef0 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.072+0000 7f5ccc9b6700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cb4067290 0x7f5cb4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.072+0000 7f5ccc9b6700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cb4067290 0x7f5cb4069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4002250 tx=0x7f5cc40018e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:47.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.076+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5cc0051870 con 0x7f5cc80abef0 2023-10-06T03:07:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.261+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5cc804c3c0 con 0x7f5cc80abef0 2023-10-06T03:07:47.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.261+0000 7f5cbcff9700 1 -- 172.21.15.92:0/4023919564 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5cc0054e90 con 0x7f5cc80abef0 2023-10-06T03:07:47.263 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:47.263 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:07:47.264 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:07:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cb4067290 msgr2=0x7f5cb4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cb4067290 0x7f5cb4069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4002250 tx=0x7f5cc40018e0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc80abef0 msgr2=0x7f5cc80ac340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc80abef0 0x7f5cc80ac340 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80cd410 tx=0x7f5cc0009040 comp rx=0 tx=0).stop 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 shutdown_connections 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cb4067290 0x7f5cb4069740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5cc80abef0 0x7f5cc80ac340 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cb0004810 0x7f5cc8157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 --2- 172.21.15.92:0/4023919564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc8159500 0x7f5cc8148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 >> 172.21.15.92:0/4023919564 conn(0x7f5cc806a320 msgr2=0x7f5cc80ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.263+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 shutdown_connections 2023-10-06T03:07:47.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.264+0000 7f5ccec1a700 1 -- 172.21.15.92:0/4023919564 wait complete. 2023-10-06T03:07:47.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:07:47.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:07:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.394+0000 7fd592b81700 1 Processor -- start 2023-10-06T03:07:47.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.394+0000 7fd592b81700 1 -- start start 2023-10-06T03:07:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd592b81700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd592b81700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd58c0cf350 0x7fd58c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd592b81700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c0b5b00 con 0x7fd58c0b53d0 2023-10-06T03:07:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd592b81700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c0b5da0 con 0x7fd58c0cf350 2023-10-06T03:07:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd592b81700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c0b6080 con 0x7fd58c159ec0 2023-10-06T03:07:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd583fff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd58c0b53d0 0x7fd58c0cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:51182/0 (socket says 172.21.15.92:51182) 2023-10-06T03:07:47.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd59111e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd58c0cf350 0x7fd58c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd59091d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.395+0000 7fd583fff700 1 -- 172.21.15.92:0/1789020478 learned_addr learned my addr 172.21.15.92:0/1789020478 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd5837fe700 1 -- 172.21.15.92:0/1789020478 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968279641 0 0) 0x7fd58c0b5b00 con 0x7fd58c0b53d0 2023-10-06T03:07:47.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd5837fe700 1 -- 172.21.15.92:0/1789020478 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd574003570 con 0x7fd58c0b53d0 2023-10-06T03:07:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd59091d700 1 -- 172.21.15.92:0/1789020478 >> v2:172.21.15.136:3300/0 conn(0x7fd58c0cf350 msgr2=0x7fd58c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd59091d700 1 --2- 172.21.15.92:0/1789020478 >> v2:172.21.15.136:3300/0 conn(0x7fd58c0cf350 0x7fd58c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd59091d700 1 -- 172.21.15.92:0/1789020478 >> v1:172.21.15.136:6789/0 conn(0x7fd58c0b53d0 legacy=0x7fd58c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd59091d700 1 -- 172.21.15.92:0/1789020478 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0cc100 con 0x7fd58c159ec0 2023-10-06T03:07:47.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd59091d700 1 --2- 172.21.15.92:0/1789020478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c0b4e90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd588009eb0 tx=0x7fd588009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5445fdd02d213f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd5837fe700 1 -- 172.21.15.92:0/1789020478 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800e070 con 0x7fd58c159ec0 2023-10-06T03:07:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd5837fe700 1 -- 172.21.15.92:0/1789020478 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58800bb80 con 0x7fd58c159ec0 2023-10-06T03:07:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.396+0000 7fd5837fe700 1 -- 172.21.15.92:0/1789020478 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588012430 con 0x7fd58c159ec0 2023-10-06T03:07:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 -- 172.21.15.92:0/1789020478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 msgr2=0x7fd58c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 --2- 172.21.15.92:0/1789020478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c0b4e90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd588009eb0 tx=0x7fd588009780 comp rx=0 tx=0).stop 2023-10-06T03:07:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 -- 172.21.15.92:0/1789020478 shutdown_connections 2023-10-06T03:07:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 --2- 172.21.15.92:0/1789020478 >> v2:172.21.15.136:3300/0 conn(0x7fd58c0cf350 0x7fd58c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 --2- 172.21.15.92:0/1789020478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c0b4e90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 -- 172.21.15.92:0/1789020478 >> 172.21.15.92:0/1789020478 conn(0x7fd58c0999f0 msgr2=0x7fd58c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:47.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 -- 172.21.15.92:0/1789020478 shutdown_connections 2023-10-06T03:07:47.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.397+0000 7fd592b81700 1 -- 172.21.15.92:0/1789020478 wait complete. 2023-10-06T03:07:47.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.398+0000 7fd592b81700 1 Processor -- start 2023-10-06T03:07:47.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.398+0000 7fd592b81700 1 -- start start 2023-10-06T03:07:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd592b81700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 0x7fd58c068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd592b81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd58c0cf350 0x7fd58c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd592b81700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd592b81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c068600 con 0x7fd58c0cf350 2023-10-06T03:07:47.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd592b81700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c069c40 con 0x7fd58c0b53d0 2023-10-06T03:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd592b81700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c09a460 con 0x7fd58c159ec0 2023-10-06T03:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 0x7fd58c068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 0x7fd58c068ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:48042/0 (socket says 172.21.15.92:48042) 2023-10-06T03:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 -- 172.21.15.92:0/3591288425 learned_addr learned my addr 172.21.15.92:0/3591288425 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:47.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59111e700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd583fff700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd58c0cf350 0x7fd58c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 -- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 msgr2=0x7fd58c1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 -- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd58c0cf350 msgr2=0x7fd58c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd58c0cf350 0x7fd58c153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd59091d700 1 -- 172.21.15.92:0/3591288425 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588009a20 con 0x7fd58c0b53d0 2023-10-06T03:07:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.399+0000 7fd583fff700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd58c0cf350 0x7fd58c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.400+0000 7fd59091d700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 0x7fd58c068ff0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b610 tx=0x7fd588019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:47.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.400+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588012c00 con 0x7fd58c0b53d0 2023-10-06T03:07:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.400+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd588012d80 con 0x7fd58c0b53d0 2023-10-06T03:07:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.400+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5880276c0 con 0x7fd58c0b53d0 2023-10-06T03:07:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.400+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c069680 con 0x7fd58c0b53d0 2023-10-06T03:07:47.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.400+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c0c8a60 con 0x7fd58c0b53d0 2023-10-06T03:07:47.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.401+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c063c90 con 0x7fd58c0b53d0 2023-10-06T03:07:47.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.401+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd588024070 con 0x7fd58c0b53d0 2023-10-06T03:07:47.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.402+0000 7fd580ff9700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd578067290 0x7fd578069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:47.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.402+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd588086f10 con 0x7fd58c0b53d0 2023-10-06T03:07:47.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.402+0000 7fd583fff700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd578067290 0x7fd578069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.402+0000 7fd583fff700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd578067290 0x7fd578069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd58c06adc0 tx=0x7fd57c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:47.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.406+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd588053fe0 con 0x7fd58c0b53d0 2023-10-06T03:07:47.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:47.589+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd58c0718b0 con 0x7fd58c0b53d0 2023-10-06T03:07:48.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.550+0000 7fd580ff9700 1 -- 172.21.15.92:0/3591288425 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd58802c0b0 con 0x7fd58c0b53d0 2023-10-06T03:07:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd578067290 msgr2=0x7fd578069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd578067290 0x7fd578069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd58c06adc0 tx=0x7fd57c009040 comp rx=0 tx=0).stop 2023-10-06T03:07:48.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 msgr2=0x7fd58c068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 0x7fd58c068ff0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd58800b610 tx=0x7fd588019000 comp rx=0 tx=0).stop 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 shutdown_connections 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd578067290 0x7fd578069740 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd58c159ec0 0x7fd58c1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd58c0b53d0 0x7fd58c068ff0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 --2- 172.21.15.92:0/3591288425 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd58c0cf350 0x7fd58c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 >> 172.21.15.92:0/3591288425 conn(0x7fd58c0999f0 msgr2=0x7fd58c158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 shutdown_connections 2023-10-06T03:07:48.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.554+0000 7fd592b81700 1 -- 172.21.15.92:0/3591288425 wait complete. 2023-10-06T03:07:48.557 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:07:48.569 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:07:48.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.692+0000 7fb319eb8700 1 Processor -- start 2023-10-06T03:07:48.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.692+0000 7fb319eb8700 1 -- start start 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb319eb8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31415d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb319eb8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb31415d760 0x7fb3140b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb319eb8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140b10f0 con 0x7fb3141595d0 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb319eb8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31415dbe0 con 0x7fb31415d760 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb319eb8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415e000 con 0x7fb3140cf480 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb3196b7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb31415d760 0x7fb3140b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb318eb6700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb3141595d0 0x7fb3140ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:51186/0 (socket says 172.21.15.92:51186) 2023-10-06T03:07:48.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb318eb6700 1 -- 172.21.15.92:0/1364175015 learned_addr learned my addr 172.21.15.92:0/1364175015 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb313fff700 1 --2- 172.21.15.92:0/1364175015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb3137fe700 1 -- 172.21.15.92:0/1364175015 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481093175 0 0) 0x7fb3140b10f0 con 0x7fb3141595d0 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb3137fe700 1 -- 172.21.15.92:0/1364175015 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb300003570 con 0x7fb3141595d0 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb313fff700 1 -- 172.21.15.92:0/1364175015 >> v2:172.21.15.136:3300/0 conn(0x7fb31415d760 msgr2=0x7fb3140b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb313fff700 1 --2- 172.21.15.92:0/1364175015 >> v2:172.21.15.136:3300/0 conn(0x7fb31415d760 0x7fb3140b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb313fff700 1 -- 172.21.15.92:0/1364175015 >> v1:172.21.15.136:6789/0 conn(0x7fb3141595d0 legacy=0x7fb3140ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb313fff700 1 -- 172.21.15.92:0/1364175015 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140cf990 con 0x7fb3140cf480 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb3196b7700 1 --2- 172.21.15.92:0/1364175015 >> v2:172.21.15.136:3300/0 conn(0x7fb31415d760 0x7fb3140b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.693+0000 7fb313fff700 1 --2- 172.21.15.92:0/1364175015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31415d220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0098e0 tx=0x7fb2fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a33eb66e4787fc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb3137fe700 1 -- 172.21.15.92:0/1364175015 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011630 con 0x7fb3140cf480 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb3137fe700 1 -- 172.21.15.92:0/1364175015 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc011790 con 0x7fb3140cf480 2023-10-06T03:07:48.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb3137fe700 1 -- 172.21.15.92:0/1364175015 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011b30 con 0x7fb3140cf480 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 -- 172.21.15.92:0/1364175015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 msgr2=0x7fb31415d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1364175015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31415d220 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0098e0 tx=0x7fb2fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 -- 172.21.15.92:0/1364175015 shutdown_connections 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1364175015 >> v2:172.21.15.136:3300/0 conn(0x7fb31415d760 0x7fb3140b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1364175015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31415d220 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 -- 172.21.15.92:0/1364175015 >> 172.21.15.92:0/1364175015 conn(0x7fb3140d05f0 msgr2=0x7fb3140d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 -- 172.21.15.92:0/1364175015 shutdown_connections 2023-10-06T03:07:48.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.694+0000 7fb319eb8700 1 -- 172.21.15.92:0/1364175015 wait complete. 2023-10-06T03:07:48.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.695+0000 7fb319eb8700 1 Processor -- start 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 -- start start 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 0x7fb3140b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb31415d760 0x7fb31414e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31416b200 con 0x7fb3141595d0 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b59b0 con 0x7fb31415d760 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb319eb8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140b5110 con 0x7fb3140cf480 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb313fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 0x7fb3140b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb313fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 0x7fb3140b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56722/0 (socket says 172.21.15.92:56722) 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb313fff700 1 -- 172.21.15.92:0/1015058226 learned_addr learned my addr 172.21.15.92:0/1015058226 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:48.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb318eb6700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.696+0000 7fb3196b7700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb31415d760 0x7fb31414e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb313fff700 1 -- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 msgr2=0x7fb31414d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb313fff700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31414d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb313fff700 1 -- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb31415d760 msgr2=0x7fb31414e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb313fff700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb31415d760 0x7fb31414e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb313fff700 1 -- 172.21.15.92:0/1015058226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c009420 con 0x7fb3141595d0 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb318eb6700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb313fff700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 0x7fb3140b4700 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb31415e140 tx=0x7fb2fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011b10 con 0x7fb3141595d0 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc000cc0 con 0x7fb3141595d0 2023-10-06T03:07:48.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc009420 con 0x7fb3141595d0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc025620 con 0x7fb3141595d0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.697+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3140b56d0 con 0x7fb3141595d0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.698+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31416b370 con 0x7fb3141595d0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.698+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb2fc025a10 con 0x7fb3141595d0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.699+0000 7fb310ff9700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2f4067290 0x7fb2f4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.699+0000 7fb318eb6700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2f4067290 0x7fb2f4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.699+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fb2fc0860e0 con 0x7fb3141595d0 2023-10-06T03:07:48.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.699+0000 7fb318eb6700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2f4067290 0x7fb2f4069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb304005ca0 tx=0x7fb304009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:48.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.703+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb31416b370 con 0x7fb3141595d0 2023-10-06T03:07:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:48.886+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb31416b340 con 0x7fb3141595d0 2023-10-06T03:07:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.549+0000 7fb310ff9700 1 -- 172.21.15.92:0/1015058226 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb2fc053040 con 0x7fb3141595d0 2023-10-06T03:07:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.554+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2f4067290 msgr2=0x7fb2f4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.554+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2f4067290 0x7fb2f4069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb304005ca0 tx=0x7fb304009040 comp rx=0 tx=0).stop 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.554+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 msgr2=0x7fb3140b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.554+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 0x7fb3140b4700 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb31415e140 tx=0x7fb2fc017000 comp rx=0 tx=0).stop 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 shutdown_connections 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb2f4067290 0x7fb2f4069740 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb31415d760 0x7fb31414e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3140cf480 0x7fb31414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 --2- 172.21.15.92:0/1015058226 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3141595d0 0x7fb3140b4700 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 >> 172.21.15.92:0/1015058226 conn(0x7fb3140d05f0 msgr2=0x7fb314068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:49.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.557+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 shutdown_connections 2023-10-06T03:07:49.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.558+0000 7fb319eb8700 1 -- 172.21.15.92:0/1015058226 wait complete. 2023-10-06T03:07:49.563 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:07:49.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:07:49.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.695+0000 7f120e378700 1 Processor -- start 2023-10-06T03:07:49.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.695+0000 7f120e378700 1 -- start start 2023-10-06T03:07:49.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f120e378700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f120806c0a0 0x7f12080caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:49.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f120e378700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120815df50 0x7f120815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:49.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f120e378700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120815d7e0 con 0x7f120806bcd0 2023-10-06T03:07:49.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f120e378700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120815dac0 con 0x7f120806c0a0 2023-10-06T03:07:49.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f120e378700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120815aa30 con 0x7f120815df50 2023-10-06T03:07:49.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f12077fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f120806c0a0 0x7f12080caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:49.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f1207fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f120806bcd0 0x7f120815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46944/0 (socket says 172.21.15.92:46944) 2023-10-06T03:07:49.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f1207fff700 1 -- 172.21.15.92:0/3897230272 learned_addr learned my addr 172.21.15.92:0/3897230272 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f120c915700 1 --2- 172.21.15.92:0/3897230272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120815df50 0x7f120815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.696+0000 7f12077fe700 1 -- 172.21.15.92:0/3897230272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120815df50 msgr2=0x7f120815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f12077fe700 1 --2- 172.21.15.92:0/3897230272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120815df50 0x7f120815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f12077fe700 1 -- 172.21.15.92:0/3897230272 >> v1:172.21.15.92:6789/0 conn(0x7f120806bcd0 legacy=0x7f120815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f12077fe700 1 -- 172.21.15.92:0/3897230272 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120815ad10 con 0x7f120806c0a0 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f1206ffd700 1 -- 172.21.15.92:0/3897230272 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316433293 0 0) 0x7f120815d7e0 con 0x7f120806bcd0 2023-10-06T03:07:49.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f12077fe700 1 --2- 172.21.15.92:0/3897230272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806c0a0 0x7f12080caa90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0098e0 tx=0x7f11fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e0d928aa20340e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f1206ffd700 1 -- 172.21.15.92:0/3897230272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc009e80 con 0x7f120806c0a0 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f1206ffd700 1 -- 172.21.15.92:0/3897230272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc00b860 con 0x7f120806c0a0 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f1206ffd700 1 -- 172.21.15.92:0/3897230272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00bbe0 con 0x7f120806c0a0 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 -- 172.21.15.92:0/3897230272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806c0a0 msgr2=0x7f12080caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 --2- 172.21.15.92:0/3897230272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806c0a0 0x7f12080caa90 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f11fc0098e0 tx=0x7f11fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 -- 172.21.15.92:0/3897230272 shutdown_connections 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 --2- 172.21.15.92:0/3897230272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120815df50 0x7f120815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 --2- 172.21.15.92:0/3897230272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806c0a0 0x7f12080caa90 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 -- 172.21.15.92:0/3897230272 >> 172.21.15.92:0/3897230272 conn(0x7f12080ce600 msgr2=0x7f12080cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:49.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 -- 172.21.15.92:0/3897230272 shutdown_connections 2023-10-06T03:07:49.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.697+0000 7f120e378700 1 -- 172.21.15.92:0/3897230272 wait complete. 2023-10-06T03:07:49.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.698+0000 7f120e378700 1 Processor -- start 2023-10-06T03:07:49.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 -- start start 2023-10-06T03:07:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 0x7f12080c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120806c0a0 0x7f12080c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f120815df50 0x7f12080c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:49.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208154db0 con 0x7f120806bcd0 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12081587c0 con 0x7f120815df50 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f120e378700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1208158a40 con 0x7f120806c0a0 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.699+0000 7f1207fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 0x7f12080c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 0x7f12080c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50574/0 (socket says 172.21.15.92:50574) 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 -- 172.21.15.92:0/591702867 learned_addr learned my addr 172.21.15.92:0/591702867 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f12077fe700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120806c0a0 0x7f12080c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f120c915700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f120815df50 0x7f12080c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 -- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120806c0a0 msgr2=0x7f12080c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:49.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120806c0a0 0x7f12080c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 -- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f120815df50 msgr2=0x7f12080c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f120815df50 0x7f12080c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 -- 172.21.15.92:0/591702867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc009420 con 0x7f120806bcd0 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f120c915700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f120815df50 0x7f12080c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f1207fff700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 0x7f12080c56b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c5b0 tx=0x7f11f8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8005b30 con 0x7f120806bcd0 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8005c90 con 0x7f120806bcd0 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1208158d20 con 0x7f120806bcd0 2023-10-06T03:07:49.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8007300 con 0x7f120806bcd0 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.700+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1208159360 con 0x7f120806bcd0 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.701+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120804c3c0 con 0x7f120806bcd0 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.702+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f11f801b3b0 con 0x7f120806bcd0 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.702+0000 7f11effff700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11f4067290 0x7f11f4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.702+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f11f8084070 con 0x7f120806bcd0 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.702+0000 7f12077fe700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11f4067290 0x7f11f4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:49.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.702+0000 7f12077fe700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11f4067290 0x7f11f4069740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f12080b43a0 tx=0x7f11fc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:49.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.706+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f8050dd0 con 0x7f120806bcd0 2023-10-06T03:07:49.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:49.930+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f120815aff0 con 0x7f120806bcd0 2023-10-06T03:07:50.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.556+0000 7f11effff700 1 -- 172.21.15.92:0/591702867 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7f11f80170c0 con 0x7f120806bcd0 2023-10-06T03:07:50.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11f4067290 msgr2=0x7f11f4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11f4067290 0x7f11f4069740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f12080b43a0 tx=0x7f11fc017000 comp rx=0 tx=0).stop 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 msgr2=0x7f12080c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 0x7f12080c56b0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c5b0 tx=0x7f11f8001070 comp rx=0 tx=0).stop 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 shutdown_connections 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f11f4067290 0x7f11f4069740 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f120806bcd0 0x7f12080c56b0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f120815df50 0x7f12080c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 --2- 172.21.15.92:0/591702867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f120806c0a0 0x7f12080c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.562+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 >> 172.21.15.92:0/591702867 conn(0x7f12080ce600 msgr2=0x7f120815b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:50.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.564+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 shutdown_connections 2023-10-06T03:07:50.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.564+0000 7f120e378700 1 -- 172.21.15.92:0/591702867 wait complete. 2023-10-06T03:07:50.573 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T03:07:50.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.702+0000 7f2831142700 1 Processor -- start 2023-10-06T03:07:50.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.702+0000 7f2831142700 1 -- start start 2023-10-06T03:07:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f2831142700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f282c159420 0x7f282c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f2831142700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c0b2980 con 0x7f282c15d230 2023-10-06T03:07:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f2831142700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282c0b11d0 con 0x7f282c1597f0 2023-10-06T03:07:50.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f2831142700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282c0b14b0 con 0x7f282c159420 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f282bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f282c159420 0x7f282c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f2830941700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f282c15d230 0x7f282c15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46958/0 (socket says 172.21.15.92:46958) 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.703+0000 7f2830941700 1 -- 172.21.15.92:0/2096164057 learned_addr learned my addr 172.21.15.92:0/2096164057 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282bfff700 1 -- 172.21.15.92:0/2096164057 >> v1:172.21.15.136:6789/0 conn(0x7f282c1597f0 legacy=0x7f282c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282bfff700 1 -- 172.21.15.92:0/2096164057 >> v1:172.21.15.92:6789/0 conn(0x7f282c15d230 legacy=0x7f282c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282bfff700 1 -- 172.21.15.92:0/2096164057 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c0b1790 con 0x7f282c159420 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282bfff700 1 --2- 172.21.15.92:0/2096164057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c159420 0x7f282c0b5f40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098e0 tx=0x7f281c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b32fd1c34659b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282affd700 1 -- 172.21.15.92:0/2096164057 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c009e80 con 0x7f282c159420 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282affd700 1 -- 172.21.15.92:0/2096164057 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c00b820 con 0x7f282c159420 2023-10-06T03:07:50.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f282affd700 1 -- 172.21.15.92:0/2096164057 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c00bba0 con 0x7f282c159420 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 -- 172.21.15.92:0/2096164057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c159420 msgr2=0x7f282c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 --2- 172.21.15.92:0/2096164057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c159420 0x7f282c0b5f40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098e0 tx=0x7f281c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 -- 172.21.15.92:0/2096164057 shutdown_connections 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 --2- 172.21.15.92:0/2096164057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c159420 0x7f282c0b5f40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 -- 172.21.15.92:0/2096164057 >> 172.21.15.92:0/2096164057 conn(0x7f282c0d0660 msgr2=0x7f282c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 -- 172.21.15.92:0/2096164057 shutdown_connections 2023-10-06T03:07:50.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.704+0000 7f2831142700 1 -- 172.21.15.92:0/2096164057 wait complete. 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.705+0000 7f2831142700 1 Processor -- start 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 -- start start 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f282c159420 0x7f282c171c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 0x7f282c172190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f282c15d230 0x7f282c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c148c90 con 0x7f282c1597f0 2023-10-06T03:07:50.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c069cd0 con 0x7f282c159420 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f2831142700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282c069150 con 0x7f282c15d230 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f282b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 0x7f282c172190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f282b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 0x7f282c172190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50604/0 (socket says 172.21.15.92:50604) 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.706+0000 7f282b7fe700 1 -- 172.21.15.92:0/455853026 learned_addr learned my addr 172.21.15.92:0/455853026 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282bfff700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f282c159420 0x7f282c171c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2830941700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f282c15d230 0x7f282c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282b7fe700 1 -- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f282c15d230 msgr2=0x7f282c0cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282b7fe700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f282c15d230 0x7f282c0cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282b7fe700 1 -- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f282c159420 msgr2=0x7f282c171c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282b7fe700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f282c159420 0x7f282c171c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282b7fe700 1 -- 172.21.15.92:0/455853026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009420 con 0x7f282c1597f0 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2830941700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f282c15d230 0x7f282c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f282b7fe700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 0x7f282c172190 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f282000b7e0 tx=0x7f28200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282000f510 con 0x7f282c1597f0 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282000f670 con 0x7f282c1597f0 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c069430 con 0x7f282c1597f0 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282000fa10 con 0x7f282c1597f0 2023-10-06T03:07:50.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.707+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c0699f0 con 0x7f282c1597f0 2023-10-06T03:07:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.708+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c04c3c0 con 0x7f282c1597f0 2023-10-06T03:07:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.708+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f282001a400 con 0x7f282c1597f0 2023-10-06T03:07:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.709+0000 7f2813fff700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2818067290 0x7f2818069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.709+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2820084730 con 0x7f282c1597f0 2023-10-06T03:07:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.709+0000 7f282bfff700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2818067290 0x7f2818069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:50.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.709+0000 7f282bfff700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2818067290 0x7f2818069740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f281c0030c0 tx=0x7f281c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:50.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.713+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28200513b0 con 0x7f282c1597f0 2023-10-06T03:07:50.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.895+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f282c0b1a70 con 0x7f282c1597f0 2023-10-06T03:07:50.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.896+0000 7f2813fff700 1 -- 172.21.15.92:0/455853026 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f28200549d0 con 0x7f282c1597f0 2023-10-06T03:07:50.898 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:50.898 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":102,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:49.932199+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:47.608728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:48.905445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2818067290 msgr2=0x7f2818069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2818067290 0x7f2818069740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f281c0030c0 tx=0x7f281c016000 comp rx=0 tx=0).stop 2023-10-06T03:07:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 msgr2=0x7f282c172190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:50.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 0x7f282c172190 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f282000b7e0 tx=0x7f28200091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 shutdown_connections 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2818067290 0x7f2818069740 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f282c15d230 0x7f282c0cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f282c159420 0x7f282c171c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 --2- 172.21.15.92:0/455853026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f282c1597f0 0x7f282c172190 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.898+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 >> 172.21.15.92:0/455853026 conn(0x7f282c0d0660 msgr2=0x7f282c156330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.899+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 shutdown_connections 2023-10-06T03:07:50.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:50.899+0000 7f2831142700 1 -- 172.21.15.92:0/455853026 wait complete. 2023-10-06T03:07:50.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.026+0000 7f0692ea8700 1 Processor -- start 2023-10-06T03:07:51.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.026+0000 7f0692ea8700 1 -- start start 2023-10-06T03:07:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0692ea8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f068c1597f0 0x7f068c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0692ea8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068c0b11d0 con 0x7f068c159420 2023-10-06T03:07:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0692ea8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068c0b14b0 con 0x7f068c15d230 2023-10-06T03:07:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0692ea8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c0b1790 con 0x7f068c1597f0 2023-10-06T03:07:51.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0691ea6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f068c159420 0x7f068c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46970/0 (socket says 172.21.15.92:46970) 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0691ea6700 1 -- 172.21.15.92:0/3576503095 learned_addr learned my addr 172.21.15.92:0/3576503095 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f06916a5700 1 --2- 172.21.15.92:0/3576503095 >> v2:172.21.15.92:3300/0 conn(0x7f068c1597f0 0x7f068c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0690ea4700 1 -- 172.21.15.92:0/3576503095 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984899600 0 0) 0x7f068c0b11d0 con 0x7f068c159420 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f0690ea4700 1 -- 172.21.15.92:0/3576503095 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0678003570 con 0x7f068c159420 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f06916a5700 1 -- 172.21.15.92:0/3576503095 >> v1:172.21.15.136:6789/0 conn(0x7f068c15d230 legacy=0x7f068c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.027+0000 7f06916a5700 1 -- 172.21.15.92:0/3576503095 >> v1:172.21.15.92:6789/0 conn(0x7f068c159420 legacy=0x7f068c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f06916a5700 1 -- 172.21.15.92:0/3576503095 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0b1a70 con 0x7f068c1597f0 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f06916a5700 1 --2- 172.21.15.92:0/3576503095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c1597f0 0x7f068c0b2270 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f067c0098e0 tx=0x7f067c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d980c564443eec25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0690ea4700 1 -- 172.21.15.92:0/3576503095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c011630 con 0x7f068c1597f0 2023-10-06T03:07:51.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0690ea4700 1 -- 172.21.15.92:0/3576503095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c011790 con 0x7f068c1597f0 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0690ea4700 1 -- 172.21.15.92:0/3576503095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c011b30 con 0x7f068c1597f0 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 -- 172.21.15.92:0/3576503095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c1597f0 msgr2=0x7f068c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 --2- 172.21.15.92:0/3576503095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c1597f0 0x7f068c0b2270 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f067c0098e0 tx=0x7f067c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 -- 172.21.15.92:0/3576503095 shutdown_connections 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 --2- 172.21.15.92:0/3576503095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c1597f0 0x7f068c0b2270 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 -- 172.21.15.92:0/3576503095 >> 172.21.15.92:0/3576503095 conn(0x7f068c0d0660 msgr2=0x7f068c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 -- 172.21.15.92:0/3576503095 shutdown_connections 2023-10-06T03:07:51.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.028+0000 7f0692ea8700 1 -- 172.21.15.92:0/3576503095 wait complete. 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 Processor -- start 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 -- start start 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 0x7f068c068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f068c1597f0 0x7f068c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f068c15d230 0x7f068c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c169630 con 0x7f068c159420 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c15b620 con 0x7f068c1597f0 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0692ea8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c15b8d0 con 0x7f068c15d230 2023-10-06T03:07:51.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0691ea6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 0x7f068c068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0691ea6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 0x7f068c068d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50616/0 (socket says 172.21.15.92:50616) 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.030+0000 7f0691ea6700 1 -- 172.21.15.92:0/2876175050 learned_addr learned my addr 172.21.15.92:0/2876175050 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0691ea6700 1 -- 172.21.15.92:0/2876175050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f068c15d230 msgr2=0x7f068c0c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0691ea6700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f068c15d230 0x7f068c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0691ea6700 1 -- 172.21.15.92:0/2876175050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f068c1597f0 msgr2=0x7f068c0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0691ea6700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f068c1597f0 0x7f068c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0691ea6700 1 -- 172.21.15.92:0/2876175050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c009420 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0691ea6700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 0x7f068c068d00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f068800c7c0 tx=0x7f0688007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688005b30 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688005c90 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c15bbb0 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688007340 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.031+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c15c170 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.032+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c04c3c0 con 0x7f068c159420 2023-10-06T03:07:51.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.032+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f068801b400 con 0x7f068c159420 2023-10-06T03:07:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.033+0000 7f0681ffb700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0670067280 0x7f0670069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.033+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0688084740 con 0x7f068c159420 2023-10-06T03:07:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.033+0000 7f06916a5700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0670067280 0x7f0670069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.033+0000 7f06916a5700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0670067280 0x7f0670069730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f068c159c70 tx=0x7f067c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:51.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.037+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06880514a0 con 0x7f068c159420 2023-10-06T03:07:51.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.218+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f068c0b1d50 con 0x7f068c159420 2023-10-06T03:07:51.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.219+0000 7f0681ffb700 1 -- 172.21.15.92:0/2876175050 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f0688054ac0 con 0x7f068c159420 2023-10-06T03:07:51.221 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:51.221 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":102,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:49.932199+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:47.608728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:48.905445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0670067280 msgr2=0x7f0670069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0670067280 0x7f0670069730 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f068c159c70 tx=0x7f067c016000 comp rx=0 tx=0).stop 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 msgr2=0x7f068c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 0x7f068c068d00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f068800c7c0 tx=0x7f0688007e00 comp rx=0 tx=0).stop 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 shutdown_connections 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0670067280 0x7f0670069730 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f068c159420 0x7f068c068d00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f068c15d230 0x7f068c0c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 --2- 172.21.15.92:0/2876175050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f068c1597f0 0x7f068c0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 >> 172.21.15.92:0/2876175050 conn(0x7f068c0d0660 msgr2=0x7f068c155240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 shutdown_connections 2023-10-06T03:07:51.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.221+0000 7f0692ea8700 1 -- 172.21.15.92:0/2876175050 wait complete. 2023-10-06T03:07:51.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:07:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.351+0000 7fa124a93700 1 Processor -- start 2023-10-06T03:07:51.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.351+0000 7fa124a93700 1 -- start start 2023-10-06T03:07:51.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa124a93700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa120069d60 0x7fa1200c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa124a93700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 0x7fa1200cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa124a93700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1200c8650 con 0x7fa1200aca80 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa124a93700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12006bdb0 con 0x7fa120069d60 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa124a93700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200cb370 con 0x7fa1200c81f0 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa11e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa1200aca80 0x7fa12006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46982/0 (socket says 172.21.15.92:46982) 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa11e59c700 1 -- 172.21.15.92:0/308503284 learned_addr learned my addr 172.21.15.92:0/308503284 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa117fff700 1 --2- 172.21.15.92:0/308503284 >> v2:172.21.15.136:3300/0 conn(0x7fa120069d60 0x7fa1200c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa11ed9d700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 0x7fa1200cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915799624 0 0) 0x7fa110001290 con 0x7fa1200aca80 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.352+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa10c003570 con 0x7fa1200aca80 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749447365 0 0) 0x7fa10c003570 con 0x7fa1200aca80 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 msgr2=0x7fa1200cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:51.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 0x7fa1200cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 >> v2:172.21.15.136:3300/0 conn(0x7fa120069d60 msgr2=0x7fa1200c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 --2- 172.21.15.92:0/308503284 >> v2:172.21.15.136:3300/0 conn(0x7fa120069d60 0x7fa1200c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120159f60 con 0x7fa1200aca80 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa110001780 con 0x7fa1200aca80 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 >> v1:172.21.15.92:6789/0 conn(0x7fa1200aca80 legacy=0x7fa12006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c003a60 0x7fa10c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c004810 0x7fa10c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 0x7fa10c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12006bdb0 con 0x7fa10c003a60 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200cb370 con 0x7fa10c004810 2023-10-06T03:07:51.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c00ba60 con 0x7fa1200c81f0 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c003a60 0x7fa10c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa117fff700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c004810 0x7fa10c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11ed9d700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 0x7fa10c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 -- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 msgr2=0x7fa10c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200c81f0 0x7fa10c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 -- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c004810 msgr2=0x7fa10c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c004810 0x7fa10c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 -- 172.21.15.92:0/308503284 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c003570 con 0x7fa10c003a60 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11e59c700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c003a60 0x7fa10c003e70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c620 tx=0x7fa110007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e36f06ed8ebb453 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11000cde0 con 0x7fa10c003a60 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1100070d0 con 0x7fa10c003a60 2023-10-06T03:07:51.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.353+0000 7fa11dd9b700 1 -- 172.21.15.92:0/308503284 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110007470 con 0x7fa10c003a60 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa124a93700 1 -- 172.21.15.92:0/308503284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c003a60 msgr2=0x7fa10c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa124a93700 1 --2- 172.21.15.92:0/308503284 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa10c003a60 0x7fa10c003e70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c620 tx=0x7fa110007ab0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa11ed9d700 1 -- 172.21.15.92:0/308503284 reap_dead start 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa124a93700 1 -- 172.21.15.92:0/308503284 shutdown_connections 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa124a93700 1 -- 172.21.15.92:0/308503284 >> 172.21.15.92:0/308503284 conn(0x7fa1200cd670 msgr2=0x7fa1200cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa124a93700 1 -- 172.21.15.92:0/308503284 shutdown_connections 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.354+0000 7fa124a93700 1 -- 172.21.15.92:0/308503284 wait complete. 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.355+0000 7fa124a93700 1 Processor -- start 2023-10-06T03:07:51.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.355+0000 7fa124a93700 1 -- start start 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c003a60 0x7fa1201504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 0x7fa1200aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200aa6c0 0x7fa1200aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200af3e0 con 0x7fa120158330 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200af680 con 0x7fa10c003a60 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200af960 con 0x7fa1200aa6c0 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 0x7fa1200aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 0x7fa1200aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50628/0 (socket says 172.21.15.92:50628) 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 -- 172.21.15.92:0/3804082409 learned_addr learned my addr 172.21.15.92:0/3804082409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa11e59c700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c003a60 0x7fa1201504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa11ed9d700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200aa6c0 0x7fa1200aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 -- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200aa6c0 msgr2=0x7fa1200aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200aa6c0 0x7fa1200aab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 -- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c003a60 msgr2=0x7fa1201504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c003a60 0x7fa1201504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 -- 172.21.15.92:0/3804082409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200afc40 con 0x7fa120158330 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa11e59c700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c003a60 0x7fa1201504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa117fff700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 0x7fa1200aa180 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa108011910 tx=0x7fa108008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800f6d0 con 0x7fa120158330 2023-10-06T03:07:51.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10800f830 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1201564c0 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10800fbf0 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.356+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa120150a30 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.357+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12004c3c0 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.358+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa108015460 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.358+0000 7fa1167fc700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa100067290 0x7fa100069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.358+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa108084d60 con 0x7fa120158330 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.358+0000 7fa11e59c700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa100067290 0x7fa100069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:51.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.358+0000 7fa11e59c700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa100067290 0x7fa100069740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa110000ef0 tx=0x7fa110015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:51.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.362+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa108051a60 con 0x7fa120158330 2023-10-06T03:07:51.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:51.592+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa1200678d0 con 0x7fa120158330 2023-10-06T03:07:52.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.568+0000 7fa1167fc700 1 -- 172.21.15.92:0/3804082409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa1080220b0 con 0x7fa120158330 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa100067290 msgr2=0x7fa100069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa100067290 0x7fa100069740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa110000ef0 tx=0x7fa110015000 comp rx=0 tx=0).stop 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 msgr2=0x7fa1200aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 0x7fa1200aa180 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa108011910 tx=0x7fa108008040 comp rx=0 tx=0).stop 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 shutdown_connections 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa100067290 0x7fa100069740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1200aa6c0 0x7fa1200aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa10c003a60 0x7fa1201504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 --2- 172.21.15.92:0/3804082409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa120158330 0x7fa1200aa180 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 >> 172.21.15.92:0/3804082409 conn(0x7fa1200cd670 msgr2=0x7fa120152450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 shutdown_connections 2023-10-06T03:07:52.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.571+0000 7fa124a93700 1 -- 172.21.15.92:0/3804082409 wait complete. 2023-10-06T03:07:52.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:07:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.702+0000 7f51f4923700 1 Processor -- start 2023-10-06T03:07:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.702+0000 7f51f4923700 1 -- start start 2023-10-06T03:07:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.703+0000 7f51f4923700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 0x7f51f00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.703+0000 7f51f4923700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f0069ee0 con 0x7f51f0159890 2023-10-06T03:07:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.703+0000 7f51f4923700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f015a160 con 0x7f51f0159c60 2023-10-06T03:07:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.703+0000 7f51f4923700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00d13a0 con 0x7f51f0069560 2023-10-06T03:07:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.703+0000 7f51ee59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51f0159890 0x7f51f015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46988/0 (socket says 172.21.15.92:46988) 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.703+0000 7f51ee59c700 1 -- 172.21.15.92:0/2746175550 learned_addr learned my addr 172.21.15.92:0/2746175550 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233486252 0 0) 0x7f51f0069ee0 con 0x7f51f0159890 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51dc003570 con 0x7f51f0159890 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51eed9d700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 0x7f51f00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889319819 0 0) 0x7f51dc003570 con 0x7f51f0159890 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 msgr2=0x7f51f00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 0x7f51f00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 >> v1:172.21.15.136:6789/0 conn(0x7f51f0159c60 legacy=0x7f51f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f015e140 con 0x7f51f0159890 2023-10-06T03:07:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f51e0001470 con 0x7f51f0159890 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 >> v1:172.21.15.92:6789/0 conn(0x7f51f0159890 legacy=0x7f51f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 0x7f51dc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51dc004060 0x7f51dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51dc004d50 0x7f51dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00d13a0 con 0x7f51dc004060 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc00bc20 con 0x7f51dc004d50 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc00bde0 con 0x7f51f0069560 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51e5d9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51dc004060 0x7f51dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51ee59c700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 0x7f51dc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51eed9d700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51dc004d50 0x7f51dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51e5d9b700 1 -- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 msgr2=0x7f51dc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51e5d9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51f0069560 0x7f51dc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51e5d9b700 1 -- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51dc004d50 msgr2=0x7f51dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51e5d9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51dc004d50 0x7f51dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.704+0000 7f51e5d9b700 1 -- 172.21.15.92:0/2746175550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc003570 con 0x7f51dc004060 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51e5d9b700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51dc004060 0x7f51dc004810 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f51d8001870 tx=0x7f51d8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d12c7052346ce92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8012400 con 0x7f51dc004060 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8012580 con 0x7f51dc004060 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51edd9b700 1 -- 172.21.15.92:0/2746175550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8012940 con 0x7f51dc004060 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51f4923700 1 -- 172.21.15.92:0/2746175550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51dc004060 msgr2=0x7f51dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51f4923700 1 --2- 172.21.15.92:0/2746175550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51dc004060 0x7f51dc004810 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f51d8001870 tx=0x7f51d8007d10 comp rx=0 tx=0).stop 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51eed9d700 1 -- 172.21.15.92:0/2746175550 reap_dead start 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51f4923700 1 -- 172.21.15.92:0/2746175550 shutdown_connections 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51f4923700 1 -- 172.21.15.92:0/2746175550 >> 172.21.15.92:0/2746175550 conn(0x7f51f0068e20 msgr2=0x7f51f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51f4923700 1 -- 172.21.15.92:0/2746175550 shutdown_connections 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.705+0000 7f51f4923700 1 -- 172.21.15.92:0/2746175550 wait complete. 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.706+0000 7f51f4923700 1 Processor -- start 2023-10-06T03:07:52.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 -- start start 2023-10-06T03:07:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51dc004060 0x7f51f00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51f00ced50 0x7f51f01520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 0x7f51f00c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00ce490 con 0x7f51f00c6710 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00c70d0 con 0x7f51f00ced50 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51f4923700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00b47f0 con 0x7f51dc004060 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51eed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 0x7f51f00c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51eed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 0x7f51f00c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50656/0 (socket says 172.21.15.92:50656) 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51eed9d700 1 -- 172.21.15.92:0/1467586187 learned_addr learned my addr 172.21.15.92:0/1467586187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51ee59c700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51dc004060 0x7f51f00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.707+0000 7f51e5d9b700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51f00ced50 0x7f51f01520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51eed9d700 1 -- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51dc004060 msgr2=0x7f51f00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51eed9d700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51dc004060 0x7f51f00cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51eed9d700 1 -- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51f00ced50 msgr2=0x7f51f01520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51eed9d700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51f00ced50 0x7f51f01520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51eed9d700 1 -- 172.21.15.92:0/1467586187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00cd600 con 0x7f51f00c6710 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51e5d9b700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51f00ced50 0x7f51f01520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51eed9d700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 0x7f51f00c6b90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f51e80094d0 tx=0x7f51e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800e040 con 0x7f51f00c6710 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e800f040 con 0x7f51f00c6710 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800fdf0 con 0x7f51f00c6710 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f00cd880 con 0x7f51f00c6710 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.708+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f00cde40 con 0x7f51f00c6710 2023-10-06T03:07:52.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.709+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f00b4980 con 0x7f51f00c6710 2023-10-06T03:07:52.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.709+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f51e8015640 con 0x7f51f00c6710 2023-10-06T03:07:52.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.710+0000 7f51e6ffd700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f51d0067290 0x7f51d0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:52.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.710+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51e8085560 con 0x7f51f00c6710 2023-10-06T03:07:52.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.710+0000 7f51ee59c700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f51d0067290 0x7f51d0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:52.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.710+0000 7f51ee59c700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f51d0067290 0x7f51d0069740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f51f00c73b0 tx=0x7f51e0001d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:52.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.714+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51f00b4980 con 0x7f51f00c6710 2023-10-06T03:07:52.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:52.940+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51f00b4980 con 0x7f51f00c6710 2023-10-06T03:07:53.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.575+0000 7f51e6ffd700 1 -- 172.21.15.92:0/1467586187 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51f00b4980 con 0x7f51f00c6710 2023-10-06T03:07:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.577+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f51d0067290 msgr2=0x7f51d0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.577+0000 7f51f4923700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f51d0067290 0x7f51d0069740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f51f00c73b0 tx=0x7f51e0001d80 comp rx=0 tx=0).stop 2023-10-06T03:07:53.579 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.577+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 msgr2=0x7f51f00c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.577+0000 7f51f4923700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 0x7f51f00c6b90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f51e80094d0 tx=0x7f51e800b040 comp rx=0 tx=0).stop 2023-10-06T03:07:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 shutdown_connections 2023-10-06T03:07:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f51d0067290 0x7f51d0069740 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51f00c6710 0x7f51f00c6b90 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51dc004060 0x7f51f00cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 --2- 172.21.15.92:0/1467586187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51f00ced50 0x7f51f01520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 >> 172.21.15.92:0/1467586187 conn(0x7f51f0068e20 msgr2=0x7f51f0069960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 shutdown_connections 2023-10-06T03:07:53.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.578+0000 7f51f4923700 1 -- 172.21.15.92:0/1467586187 wait complete. 2023-10-06T03:07:53.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:07:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.706+0000 7fc9b2379700 1 Processor -- start 2023-10-06T03:07:53.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.706+0000 7fc9b2379700 1 -- start start 2023-10-06T03:07:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9b2379700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc9ac0cd1e0 0x7fc9ac0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9b2379700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac15c2a0 con 0x7fc9ac06bcd0 2023-10-06T03:07:53.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9b2379700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac15b690 con 0x7fc9ac141500 2023-10-06T03:07:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9b2379700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac15b970 con 0x7fc9ac0cd1e0 2023-10-06T03:07:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9abfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc9ac06bcd0 0x7fc9ac140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:46992/0 (socket says 172.21.15.92:46992) 2023-10-06T03:07:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9abfff700 1 -- 172.21.15.92:0/978964595 learned_addr learned my addr 172.21.15.92:0/978964595 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9b0916700 1 --2- 172.21.15.92:0/978964595 >> v2:172.21.15.136:3300/0 conn(0x7fc9ac0cd1e0 0x7fc9ac0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.707+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953864437 0 0) 0x7fc9ac15c2a0 con 0x7fc9ac06bcd0 2023-10-06T03:07:53.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc994003570 con 0x7fc9ac06bcd0 2023-10-06T03:07:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1637528161 0 0) 0x7fc994003570 con 0x7fc9ac06bcd0 2023-10-06T03:07:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 >> v2:172.21.15.136:3300/0 conn(0x7fc9ac0cd1e0 msgr2=0x7fc9ac0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 --2- 172.21.15.92:0/978964595 >> v2:172.21.15.136:3300/0 conn(0x7fc9ac0cd1e0 0x7fc9ac0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 >> v1:172.21.15.136:6789/0 conn(0x7fc9ac141500 legacy=0x7fc9ac0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac15bc50 con 0x7fc9ac06bcd0 2023-10-06T03:07:53.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc99c001470 con 0x7fc9ac06bcd0 2023-10-06T03:07:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 >> v1:172.21.15.92:6789/0 conn(0x7fc9ac06bcd0 legacy=0x7fc9ac140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994003a60 0x7fc994003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc994004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9940051c0 0x7fc99400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac15b970 con 0x7fc994004810 2023-10-06T03:07:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400c0d0 con 0x7fc994003a60 2023-10-06T03:07:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99400c290 con 0x7fc9940051c0 2023-10-06T03:07:53.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc994004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9abfff700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994003a60 0x7fc994003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9b0916700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9940051c0 0x7fc99400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 -- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9940051c0 msgr2=0x7fc99400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9940051c0 0x7fc99400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 -- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994003a60 msgr2=0x7fc994003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994003a60 0x7fc994003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 -- 172.21.15.92:0/978964595 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994003570 con 0x7fc994004810 2023-10-06T03:07:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9b0916700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9940051c0 0x7fc99400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9b0916700 1 -- 172.21.15.92:0/978964595 reap_dead start 2023-10-06T03:07:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.708+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc994004c80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00018b0 tx=0x7fc9a0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b66ff0ea0769f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0012420 con 0x7fc994004810 2023-10-06T03:07:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a00125a0 con 0x7fc994004810 2023-10-06T03:07:53.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9aaffd700 1 -- 172.21.15.92:0/978964595 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0012960 con 0x7fc994004810 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 -- 172.21.15.92:0/978964595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 msgr2=0x7fc994004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc994004c80 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00018b0 tx=0x7fc9a0007d30 comp rx=0 tx=0).stop 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 -- 172.21.15.92:0/978964595 shutdown_connections 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 --2- 172.21.15.92:0/978964595 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc994004c80 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 -- 172.21.15.92:0/978964595 >> 172.21.15.92:0/978964595 conn(0x7fc9ac0ce600 msgr2=0x7fc9ac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 -- 172.21.15.92:0/978964595 shutdown_connections 2023-10-06T03:07:53.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.709+0000 7fc9b2379700 1 -- 172.21.15.92:0/978964595 wait complete. 2023-10-06T03:07:53.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.710+0000 7fc9b2379700 1 Processor -- start 2023-10-06T03:07:53.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.710+0000 7fc9b2379700 1 -- start start 2023-10-06T03:07:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b2379700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc9ac0c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b2379700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ac0b0110 0x7fc9ac13c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b2379700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9ac0c4570 0x7fc9ac0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b2379700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0b0580 con 0x7fc994004810 2023-10-06T03:07:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b2379700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0c4f20 con 0x7fc9ac0b0110 2023-10-06T03:07:53.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b2379700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac13cb80 con 0x7fc9ac0c4570 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc9ac0c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc9ac0c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50684/0 (socket says 172.21.15.92:50684) 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 -- 172.21.15.92:0/1019626777 learned_addr learned my addr 172.21.15.92:0/1019626777 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9b0916700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9ac0c4570 0x7fc9ac0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ac0b0110 0x7fc9ac13c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 -- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9ac0c4570 msgr2=0x7fc9ac0c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9ac0c4570 0x7fc9ac0c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 -- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ac0b0110 msgr2=0x7fc9ac13c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ac0b0110 0x7fc9ac13c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9abfff700 1 -- 172.21.15.92:0/1019626777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac13fff0 con 0x7fc994004810 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.711+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ac0b0110 0x7fc9ac13c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:53.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.712+0000 7fc9abfff700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc9ac0c3cf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc99c007390 tx=0x7fc9ac15c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.712+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c018030 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.712+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c00f7f0 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.712+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00fb90 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.712+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac13d070 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.712+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac13f6c0 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.713+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac1402d0 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.713+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc99c01b070 con 0x7fc994004810 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.713+0000 7fc993fff700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc998067290 0x7fc998069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:53.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.714+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc99c019070 con 0x7fc994004810 2023-10-06T03:07:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.714+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc998067290 0x7fc998069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.714+0000 7fc9ab7fe700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc998067290 0x7fc998069740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac1409d0 tx=0x7fc9a0015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:53.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.718+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc99c051870 con 0x7fc994004810 2023-10-06T03:07:53.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.943+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9ac04c3c0 con 0x7fc994004810 2023-10-06T03:07:53.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.943+0000 7fc993fff700 1 -- 172.21.15.92:0/1019626777 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fc99c054e90 con 0x7fc994004810 2023-10-06T03:07:53.945 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:53.945 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:49.932559+0000","modified":"2023-10-06T03:07:53.569016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/1523868194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1523868194},{"type":"v1","addr":"172.21.15.92:6837","nonce":1523868194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:07:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc998067290 msgr2=0x7fc998069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc998067290 0x7fc998069740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac1409d0 tx=0x7fc9a0015000 comp rx=0 tx=0).stop 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 msgr2=0x7fc9ac0c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc9ac0c3cf0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc99c007390 tx=0x7fc9ac15c210 comp rx=0 tx=0).stop 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 shutdown_connections 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc998067290 0x7fc998069740 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc994004810 0x7fc9ac0c3cf0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9ac0c4570 0x7fc9ac0c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 --2- 172.21.15.92:0/1019626777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9ac0b0110 0x7fc9ac13c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.945+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 >> 172.21.15.92:0/1019626777 conn(0x7fc9ac0ce600 msgr2=0x7fc9ac157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.946+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 shutdown_connections 2023-10-06T03:07:53.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:53.946+0000 7fc9b2379700 1 -- 172.21.15.92:0/1019626777 wait complete. 2023-10-06T03:07:53.949 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 68 2023-10-06T03:07:53.959 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:07:54.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.072+0000 7f14747a1700 1 Processor -- start 2023-10-06T03:07:54.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.072+0000 7f14747a1700 1 -- start start 2023-10-06T03:07:54.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f14747a1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140350 0x7f146c069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f14747a1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f146c06a1c0 0x7f146c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f14747a1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0c7de0 con 0x7f146c140720 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f14747a1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c140c10 con 0x7f146c06a1c0 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f14747a1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c141490 con 0x7f146c140350 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1471d3c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f146c06a1c0 0x7f146c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1472d3e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f146c140720 0x7f146c15b480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47000/0 (socket says 172.21.15.92:47000) 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1472d3e700 1 -- 172.21.15.92:0/227167937 learned_addr learned my addr 172.21.15.92:0/227167937 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f147253d700 1 --2- 172.21.15.92:0/227167937 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140350 0x7f146c069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1471d3c700 1 -- 172.21.15.92:0/227167937 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140350 msgr2=0x7f146c069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1471d3c700 1 --2- 172.21.15.92:0/227167937 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140350 0x7f146c069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1471d3c700 1 -- 172.21.15.92:0/227167937 >> v1:172.21.15.92:6789/0 conn(0x7f146c140720 legacy=0x7f146c15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f1471d3c700 1 -- 172.21.15.92:0/227167937 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c140ed0 con 0x7f146c06a1c0 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.073+0000 7f147153b700 1 -- 172.21.15.92:0/227167937 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079301201 0 0) 0x7f146c0c7de0 con 0x7f146c140720 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f1471d3c700 1 --2- 172.21.15.92:0/227167937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c06a1c0 0x7f146c068390 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f14680098e0 tx=0x7f14680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c921258c99cf1ee9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f147153b700 1 -- 172.21.15.92:0/227167937 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800b640 con 0x7f146c06a1c0 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f147153b700 1 -- 172.21.15.92:0/227167937 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146800b7a0 con 0x7f146c06a1c0 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f147153b700 1 -- 172.21.15.92:0/227167937 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800bb40 con 0x7f146c06a1c0 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 -- 172.21.15.92:0/227167937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c06a1c0 msgr2=0x7f146c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 --2- 172.21.15.92:0/227167937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c06a1c0 0x7f146c068390 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f14680098e0 tx=0x7f14680091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 -- 172.21.15.92:0/227167937 shutdown_connections 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 --2- 172.21.15.92:0/227167937 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140350 0x7f146c069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 --2- 172.21.15.92:0/227167937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c06a1c0 0x7f146c068390 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 -- 172.21.15.92:0/227167937 >> 172.21.15.92:0/227167937 conn(0x7f146c0cd670 msgr2=0x7f146c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 -- 172.21.15.92:0/227167937 shutdown_connections 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.074+0000 7f14747a1700 1 -- 172.21.15.92:0/227167937 wait complete. 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.075+0000 7f14747a1700 1 Processor -- start 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.075+0000 7f14747a1700 1 -- start start 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f14747a1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f146c06a1c0 0x7f146c0c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f14747a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 0x7f146c0c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f14747a1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140720 0x7f146c0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f14747a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0b30d0 con 0x7f146c140350 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f14747a1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0c7220 con 0x7f146c06a1c0 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f14747a1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0c7500 con 0x7f146c140720 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f1471d3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 0x7f146c0c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f1471d3c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 0x7f146c0c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50704/0 (socket says 172.21.15.92:50704) 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f1471d3c700 1 -- 172.21.15.92:0/3020214183 learned_addr learned my addr 172.21.15.92:0/3020214183 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f147253d700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f146c06a1c0 0x7f146c0c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f1472d3e700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140720 0x7f146c0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f1471d3c700 1 -- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140720 msgr2=0x7f146c0c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.076+0000 7f1471d3c700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140720 0x7f146c0c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f1471d3c700 1 -- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f146c06a1c0 msgr2=0x7f146c0c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f1471d3c700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f146c06a1c0 0x7f146c0c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f1471d3c700 1 -- 172.21.15.92:0/3020214183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1468009420 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f147253d700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f146c06a1c0 0x7f146c0c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f1471d3c700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 0x7f146c0c6660 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f146c06c6a0 tx=0x7f1468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146801f070 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1468000bc0 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0c77e0 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468025420 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.077+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c169bf0 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.078+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0cb240 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.078+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1468023070 con 0x7f146c140350 2023-10-06T03:07:54.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.078+0000 7f14627fc700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1450067290 0x7f1450069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.079+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1468086150 con 0x7f146c140350 2023-10-06T03:07:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.079+0000 7f147253d700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1450067290 0x7f1450069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.079+0000 7f147253d700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1450067290 0x7f1450069740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f145c009d70 tx=0x7f145c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.083+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f146c0cb240 con 0x7f146c140350 2023-10-06T03:07:54.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.270+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f146c0cb240 con 0x7f146c140350 2023-10-06T03:07:54.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.271+0000 7f14627fc700 1 -- 172.21.15.92:0/3020214183 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f146c0cb240 con 0x7f146c140350 2023-10-06T03:07:54.273 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:54.273 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":103,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:51.555867+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:47.608728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:48.905445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:07:54.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1450067290 msgr2=0x7f1450069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1450067290 0x7f1450069740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f145c009d70 tx=0x7f145c008040 comp rx=0 tx=0).stop 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 msgr2=0x7f146c0c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 0x7f146c0c6660 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f146c06c6a0 tx=0x7f1468017000 comp rx=0 tx=0).stop 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 shutdown_connections 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1450067290 0x7f1450069740 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f146c140720 0x7f146c0c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f146c06a1c0 0x7f146c0c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 --2- 172.21.15.92:0/3020214183 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f146c140350 0x7f146c0c6660 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 >> 172.21.15.92:0/3020214183 conn(0x7f146c0cd670 msgr2=0x7f146c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 shutdown_connections 2023-10-06T03:07:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.274+0000 7f14747a1700 1 -- 172.21.15.92:0/3020214183 wait complete. 2023-10-06T03:07:54.288 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:07:54.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:54.362 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:07:54.361+0000 7f18fea8c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:07:54.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.400+0000 7fb0d8193700 1 Processor -- start 2023-10-06T03:07:54.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.400+0000 7fb0d8193700 1 -- start start 2023-10-06T03:07:54.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d8193700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d0159c60 0x7fb0d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d8193700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb0d015db10 0x7fb0d0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d8193700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d015a160 con 0x7fb0d0159890 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d8193700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d015df50 con 0x7fb0d0159c60 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d8193700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00d13a0 con 0x7fb0d015db10 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d572e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d0159c60 0x7fb0d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d572e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d0159c60 0x7fb0d015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50706/0 (socket says 172.21.15.92:50706) 2023-10-06T03:07:54.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d572e700 1 -- 172.21.15.92:0/137210374 learned_addr learned my addr 172.21.15.92:0/137210374 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.401+0000 7fb0d6730700 1 --2- 172.21.15.92:0/137210374 >> v2:172.21.15.136:3300/0 conn(0x7fb0d015db10 0x7fb0d0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d572e700 1 -- 172.21.15.92:0/137210374 >> v2:172.21.15.136:3300/0 conn(0x7fb0d015db10 msgr2=0x7fb0d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d572e700 1 --2- 172.21.15.92:0/137210374 >> v2:172.21.15.136:3300/0 conn(0x7fb0d015db10 0x7fb0d0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d572e700 1 -- 172.21.15.92:0/137210374 >> v1:172.21.15.136:6789/0 conn(0x7fb0d0159890 legacy=0x7fb0d015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d572e700 1 -- 172.21.15.92:0/137210374 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d015e140 con 0x7fb0d0159c60 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d6730700 1 --2- 172.21.15.92:0/137210374 >> v2:172.21.15.136:3300/0 conn(0x7fb0d015db10 0x7fb0d0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:54.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d572e700 1 --2- 172.21.15.92:0/137210374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d015d5d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb0d006a890 tx=0x7fb0c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6747317448e459b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d4f2d700 1 -- 172.21.15.92:0/137210374 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000b860 con 0x7fb0d0159c60 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d4f2d700 1 -- 172.21.15.92:0/137210374 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c000b9c0 con 0x7fb0d0159c60 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d4f2d700 1 -- 172.21.15.92:0/137210374 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000bd60 con 0x7fb0d0159c60 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d8193700 1 -- 172.21.15.92:0/137210374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 msgr2=0x7fb0d015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d8193700 1 --2- 172.21.15.92:0/137210374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d015d5d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb0d006a890 tx=0x7fb0c0009250 comp rx=0 tx=0).stop 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d8193700 1 -- 172.21.15.92:0/137210374 shutdown_connections 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d8193700 1 --2- 172.21.15.92:0/137210374 >> v2:172.21.15.136:3300/0 conn(0x7fb0d015db10 0x7fb0d0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d8193700 1 --2- 172.21.15.92:0/137210374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d015d5d0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.402+0000 7fb0d8193700 1 -- 172.21.15.92:0/137210374 >> 172.21.15.92:0/137210374 conn(0x7fb0d0068e20 msgr2=0x7fb0d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.403+0000 7fb0d8193700 1 -- 172.21.15.92:0/137210374 shutdown_connections 2023-10-06T03:07:54.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.403+0000 7fb0d8193700 1 -- 172.21.15.92:0/137210374 wait complete. 2023-10-06T03:07:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 Processor -- start 2023-10-06T03:07:54.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 -- start start 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159890 0x7fb0d014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d015db10 0x7fb0d015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d016bde0 con 0x7fb0d0159c60 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d016b1d0 con 0x7fb0d0159890 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d8193700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d016b4b0 con 0x7fb0d015db10 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d572e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d572e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d00cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50720/0 (socket says 172.21.15.92:50720) 2023-10-06T03:07:54.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d572e700 1 -- 172.21.15.92:0/1511113611 learned_addr learned my addr 172.21.15.92:0/1511113611 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.404+0000 7fb0d5f2f700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159890 0x7fb0d014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d6730700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d015db10 0x7fb0d015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d572e700 1 -- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d015db10 msgr2=0x7fb0d015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d572e700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d015db10 0x7fb0d015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d572e700 1 -- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159890 msgr2=0x7fb0d014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d572e700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159890 0x7fb0d014caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d572e700 1 -- 172.21.15.92:0/1511113611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c00094f0 con 0x7fb0d0159c60 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d5f2f700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159890 0x7fb0d014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d6730700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d015db10 0x7fb0d015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d572e700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d00cf030 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009290 tx=0x7fb0c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0011e60 con 0x7fb0d0159c60 2023-10-06T03:07:54.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c001f070 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d016b790 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c001a870 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.405+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d00b3590 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.406+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d004c3c0 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.406+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb0c001abe0 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.407+0000 7fb0c5ffb700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0b4067290 0x7fb0b4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.407+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb0c0021070 con 0x7fb0d0159c60 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.407+0000 7fb0d5f2f700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0b4067290 0x7fb0b4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.407+0000 7fb0d5f2f700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0b4067290 0x7fb0b4069740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007fd0 tx=0x7fb0cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.411+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0c00538d0 con 0x7fb0d0159c60 2023-10-06T03:07:54.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.590+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0d00b1aa0 con 0x7fb0b4067290 2023-10-06T03:07:54.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.598+0000 7fb0c5ffb700 1 -- 172.21.15.92:0/1511113611 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301100 (secure 0 0 0) 0x7fb0d00b1aa0 con 0x7fb0b4067290 2023-10-06T03:07:54.603 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0b4067290 msgr2=0x7fb0b4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0b4067290 0x7fb0b4069740 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007fd0 tx=0x7fb0cc009040 comp rx=0 tx=0).stop 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 msgr2=0x7fb0d00cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d00cf030 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009290 tx=0x7fb0c0017000 comp rx=0 tx=0).stop 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 shutdown_connections 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb0b4067290 0x7fb0b4069740 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0d015db10 0x7fb0d015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0d0159890 0x7fb0d014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 --2- 172.21.15.92:0/1511113611 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0d0159c60 0x7fb0d00cf030 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 >> 172.21.15.92:0/1511113611 conn(0x7fb0d0068e20 msgr2=0x7fb0d00c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:54.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 shutdown_connections 2023-10-06T03:07:54.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.601+0000 7fb0d8193700 1 -- 172.21.15.92:0/1511113611 wait complete. 2023-10-06T03:07:54.607 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:54.615 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2023-10-06T03:07:53.257820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405528,"kb_used_data":10840,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752569192,"statfs":{"total":773094113280,"available":770630852608,"internally_reserved":0,"allocated":11100160,"data_stored":5681648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.714201"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.553902+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:19.513141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550095+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:47.531411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551341+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:17.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550113+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:16.267472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.552255+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:29.499952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547683+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:46.764118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550427+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547640+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:37.609401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550261+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.526332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.545710+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:39.311411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550785+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:01.532107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550666+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:21.064960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547697+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:39.772886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551074+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:25.290984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550154+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:18.005375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551642+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:57.954806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547992+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:47.547782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.554475+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:00.877507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.554395+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:18.005375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547744+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:19.485603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551734+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:17.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.552606+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:55.985613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547683+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:06.407882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550660+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:56.313797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551172+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:00.877507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.708522+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:45.708522+0000","last_peered":"2023-10-06T03:07:45.708522+0000","last_clean":"2023-10-06T03:07:45.708522+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:45.708522+0000","last_undegraded":"2023-10-06T03:07:45.708522+0000","last_fullsized":"2023-10-06T03:07:45.708522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550423+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937387+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:45.937387+0000","last_peered":"2023-10-06T03:07:45.937387+0000","last_clean":"2023-10-06T03:07:45.937387+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:45.937387+0000","last_undegraded":"2023-10-06T03:07:45.937387+0000","last_fullsized":"2023-10-06T03:07:45.937387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550501+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.100040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937687+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:45.937687+0000","last_peered":"2023-10-06T03:07:45.937687+0000","last_clean":"2023-10-06T03:07:45.937687+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:45.937687+0000","last_undegraded":"2023-10-06T03:07:45.937687+0000","last_fullsized":"2023-10-06T03:07:45.937687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551746+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937626+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:45.937626+0000","last_peered":"2023-10-06T03:07:45.937626+0000","last_clean":"2023-10-06T03:07:45.937626+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:45.937626+0000","last_undegraded":"2023-10-06T03:07:45.937626+0000","last_fullsized":"2023-10-06T03:07:45.937626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.552182+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.100040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937693+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:45.937693+0000","last_peered":"2023-10-06T03:07:45.937693+0000","last_clean":"2023-10-06T03:07:45.937693+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:45.937693+0000","last_undegraded":"2023-10-06T03:07:45.937693+0000","last_fullsized":"2023-10-06T03:07:45.937693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.544096+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:45.544096+0000","last_peered":"2023-10-06T03:07:45.544096+0000","last_clean":"2023-10-06T03:07:45.544096+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:45.544096+0000","last_undegraded":"2023-10-06T03:07:45.544096+0000","last_fullsized":"2023-10-06T03:07:45.544096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551774+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937144+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:45.937144+0000","last_peered":"2023-10-06T03:07:45.937144+0000","last_clean":"2023-10-06T03:07:45.937144+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:45.937144+0000","last_undegraded":"2023-10-06T03:07:45.937144+0000","last_fullsized":"2023-10-06T03:07:45.937144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550654+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.544094+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:45.544094+0000","last_peered":"2023-10-06T03:07:45.544094+0000","last_clean":"2023-10-06T03:07:45.544094+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:45.544094+0000","last_undegraded":"2023-10-06T03:07:45.544094+0000","last_fullsized":"2023-10-06T03:07:45.544094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.553441+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:25.765674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937350+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:45.937350+0000","last_peered":"2023-10-06T03:07:45.937350+0000","last_clean":"2023-10-06T03:07:45.937350+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:45.937350+0000","last_undegraded":"2023-10-06T03:07:45.937350+0000","last_fullsized":"2023-10-06T03:07:45.937350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303180,"kb_used_data":1796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068660,"statfs":{"total":96636764160,"available":96326307840,"internally_reserved":0,"allocated":1839104,"data_stored":1156061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":0.498}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.312},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.376},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302600,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069240,"statfs":{"total":96636764160,"available":96326901760,"internally_reserved":0,"allocated":1245184,"data_stored":565693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.503},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.248},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.26400000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.439},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.38}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.41799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303180,"kb_used_data":1796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068660,"statfs":{"total":96636764160,"available":96326307840,"internally_reserved":0,"allocated":1839104,"data_stored":1156061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.314}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.27500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.217},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.33300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.28899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.316},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.502},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.626},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302600,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069240,"statfs":{"total":96636764160,"available":96326901760,"internally_reserved":0,"allocated":1245184,"data_stored":565693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.373}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.436}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:54.623 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.768+0000 7fd080230700 1 Processor -- start 2023-10-06T03:07:54.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.768+0000 7fd080230700 1 -- start start 2023-10-06T03:07:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd080230700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd078159890 0x7fd07815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd080230700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd078159c60 0x7fd07815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd080230700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd080230700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07815df50 con 0x7fd078159890 2023-10-06T03:07:54.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd080230700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07815a160 con 0x7fd078159c60 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd080230700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780d13a0 con 0x7fd07815db10 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd07dfcc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd078159890 0x7fd07815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd07dfcc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd078159890 0x7fd07815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50726/0 (socket says 172.21.15.92:50726) 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd07dfcc700 1 -- 172.21.15.92:0/2003158046 learned_addr learned my addr 172.21.15.92:0/2003158046 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.769+0000 7fd07dfcc700 1 -- 172.21.15.92:0/2003158046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 msgr2=0x7fd078069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07dfcc700 1 --2- 172.21.15.92:0/2003158046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07dfcc700 1 -- 172.21.15.92:0/2003158046 >> v2:172.21.15.136:3300/0 conn(0x7fd078159c60 msgr2=0x7fd07815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07d7cb700 1 --2- 172.21.15.92:0/2003158046 >> v2:172.21.15.136:3300/0 conn(0x7fd078159c60 0x7fd07815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07dfcc700 1 --2- 172.21.15.92:0/2003158046 >> v2:172.21.15.136:3300/0 conn(0x7fd078159c60 0x7fd07815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07dfcc700 1 -- 172.21.15.92:0/2003158046 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07815e140 con 0x7fd078159890 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07d7cb700 1 --2- 172.21.15.92:0/2003158046 >> v2:172.21.15.136:3300/0 conn(0x7fd078159c60 0x7fd07815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:54.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07dfcc700 1 --2- 172.21.15.92:0/2003158046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd07815abd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd0780b67f0 tx=0x7fd074009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b800343d4daf23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07cfca700 1 -- 172.21.15.92:0/2003158046 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07400e070 con 0x7fd078159890 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07cfca700 1 -- 172.21.15.92:0/2003158046 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07400ba50 con 0x7fd078159890 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd07cfca700 1 -- 172.21.15.92:0/2003158046 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740092e0 con 0x7fd078159890 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 -- 172.21.15.92:0/2003158046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 msgr2=0x7fd07815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 --2- 172.21.15.92:0/2003158046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd07815abd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd0780b67f0 tx=0x7fd074009840 comp rx=0 tx=0).stop 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 -- 172.21.15.92:0/2003158046 shutdown_connections 2023-10-06T03:07:54.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 --2- 172.21.15.92:0/2003158046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 --2- 172.21.15.92:0/2003158046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd07815abd0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 --2- 172.21.15.92:0/2003158046 >> v2:172.21.15.136:3300/0 conn(0x7fd078159c60 0x7fd07815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 -- 172.21.15.92:0/2003158046 >> 172.21.15.92:0/2003158046 conn(0x7fd078068e20 msgr2=0x7fd0780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:54.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 -- 172.21.15.92:0/2003158046 shutdown_connections 2023-10-06T03:07:54.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.770+0000 7fd080230700 1 -- 172.21.15.92:0/2003158046 wait complete. 2023-10-06T03:07:54.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.771+0000 7fd080230700 1 Processor -- start 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 -- start start 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd0780c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078159c60 0x7fd0780b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780c7a20 con 0x7fd078159890 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b3ad0 con 0x7fd078159c60 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd080230700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b3da0 con 0x7fd07815db10 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd07dfcc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd0780c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd07dfcc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd0780c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50736/0 (socket says 172.21.15.92:50736) 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd07dfcc700 1 -- 172.21.15.92:0/3100418029 learned_addr learned my addr 172.21.15.92:0/3100418029 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:54.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd07d7cb700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078159c60 0x7fd0780b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.772+0000 7fd07e7cd700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07dfcc700 1 -- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 msgr2=0x7fd078153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07dfcc700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07dfcc700 1 -- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078159c60 msgr2=0x7fd0780b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07dfcc700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078159c60 0x7fd0780b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07dfcc700 1 -- 172.21.15.92:0/3100418029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd074009ae0 con 0x7fd078159890 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07e7cd700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd07dfcc700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd0780c9530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd074009430 tx=0x7fd074018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074008a80 con 0x7fd078159890 2023-10-06T03:07:54.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074008c00 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780b4080 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07401c410 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.773+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd078153760 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.774+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0780c7880 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.774+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd074003040 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.775+0000 7fd06dffb700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd05c067290 0x7fd05c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.775+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0740863c0 con 0x7fd078159890 2023-10-06T03:07:54.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.775+0000 7fd07d7cb700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd05c067290 0x7fd05c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:54.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.775+0000 7fd07d7cb700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd05c067290 0x7fd05c069740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0780b64e0 tx=0x7fd068006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:54.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.779+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0780c7880 con 0x7fd078159890 2023-10-06T03:07:54.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.960+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0780c7880 con 0x7fd05c067290 2023-10-06T03:07:54.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.968+0000 7fd06dffb700 1 -- 172.21.15.92:0/3100418029 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+301100 (secure 0 0 0) 0x7fd0780c7880 con 0x7fd05c067290 2023-10-06T03:07:54.973 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd05c067290 msgr2=0x7fd05c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd05c067290 0x7fd05c069740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0780b64e0 tx=0x7fd068006f90 comp rx=0 tx=0).stop 2023-10-06T03:07:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 msgr2=0x7fd0780c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd0780c9530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd074009430 tx=0x7fd074018000 comp rx=0 tx=0).stop 2023-10-06T03:07:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 shutdown_connections 2023-10-06T03:07:54.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd05c067290 0x7fd05c069740 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd078159890 0x7fd0780c9530 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd07815db10 0x7fd078153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 --2- 172.21.15.92:0/3100418029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078159c60 0x7fd0780b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 >> 172.21.15.92:0/3100418029 conn(0x7fd078068e20 msgr2=0x7fd078154170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 shutdown_connections 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:54.971+0000 7fd080230700 1 -- 172.21.15.92:0/3100418029 wait complete. 2023-10-06T03:07:54.976 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:54.985 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2023-10-06T03:07:53.257820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405528,"kb_used_data":10840,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752569192,"statfs":{"total":773094113280,"available":770630852608,"internally_reserved":0,"allocated":11100160,"data_stored":5681648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.714201"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.553902+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:19.513141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550095+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:47.531411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551341+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:17.481210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550113+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:16.267472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.552255+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:29.499952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547683+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:46.764118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550427+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547640+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:37.609401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550261+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:24.526332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.545710+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:39.311411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550785+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:01.532107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550666+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:21.064960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547697+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:39.772886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551074+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:25.290984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550154+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:18.005375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551642+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:57.954806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547992+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:47.547782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.554475+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:00.877507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.554395+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:14:18.005375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547744+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:01:19.485603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551734+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:17.269293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.552606+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:55.985613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.547683+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:06.407882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550660+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:56.313797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551172+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:00.877507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":77,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.708522+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:45.708522+0000","last_peered":"2023-10-06T03:07:45.708522+0000","last_clean":"2023-10-06T03:07:45.708522+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:45.708522+0000","last_undegraded":"2023-10-06T03:07:45.708522+0000","last_fullsized":"2023-10-06T03:07:45.708522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550423+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937387+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:45.937387+0000","last_peered":"2023-10-06T03:07:45.937387+0000","last_clean":"2023-10-06T03:07:45.937387+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:45.937387+0000","last_undegraded":"2023-10-06T03:07:45.937387+0000","last_fullsized":"2023-10-06T03:07:45.937387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550501+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.100040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937687+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:45.937687+0000","last_peered":"2023-10-06T03:07:45.937687+0000","last_clean":"2023-10-06T03:07:45.937687+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:45.937687+0000","last_undegraded":"2023-10-06T03:07:45.937687+0000","last_fullsized":"2023-10-06T03:07:45.937687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551746+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937626+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:45.937626+0000","last_peered":"2023-10-06T03:07:45.937626+0000","last_clean":"2023-10-06T03:07:45.937626+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:45.937626+0000","last_undegraded":"2023-10-06T03:07:45.937626+0000","last_fullsized":"2023-10-06T03:07:45.937626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.552182+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:54.100040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937693+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:45.937693+0000","last_peered":"2023-10-06T03:07:45.937693+0000","last_clean":"2023-10-06T03:07:45.937693+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:45.937693+0000","last_undegraded":"2023-10-06T03:07:45.937693+0000","last_fullsized":"2023-10-06T03:07:45.937693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.544096+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:45.544096+0000","last_peered":"2023-10-06T03:07:45.544096+0000","last_clean":"2023-10-06T03:07:45.544096+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:45.544096+0000","last_undegraded":"2023-10-06T03:07:45.544096+0000","last_fullsized":"2023-10-06T03:07:45.544096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.551774+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937144+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:45.937144+0000","last_peered":"2023-10-06T03:07:45.937144+0000","last_clean":"2023-10-06T03:07:45.937144+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:45.937144+0000","last_undegraded":"2023-10-06T03:07:45.937144+0000","last_fullsized":"2023-10-06T03:07:45.937144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.550654+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":85,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.544094+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:45.544094+0000","last_peered":"2023-10-06T03:07:45.544094+0000","last_clean":"2023-10-06T03:07:45.544094+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:45.544094+0000","last_undegraded":"2023-10-06T03:07:45.544094+0000","last_fullsized":"2023-10-06T03:07:45.544094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:07:49.543624+0000","last_change":"2023-10-06T03:07:49.543624+0000","last_active":"2023-10-06T03:07:49.543624+0000","last_peered":"2023-10-06T03:07:49.543624+0000","last_clean":"2023-10-06T03:07:49.543624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:49.543624+0000","last_undegraded":"2023-10-06T03:07:49.543624+0000","last_fullsized":"2023-10-06T03:07:49.543624+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:07:48.538901+0000","last_change":"2023-10-06T03:07:48.553441+0000","last_active":"2023-10-06T03:07:48.538901+0000","last_peered":"2023-10-06T03:07:48.538901+0000","last_clean":"2023-10-06T03:07:48.538901+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:07:48.538901+0000","last_undegraded":"2023-10-06T03:07:48.538901+0000","last_fullsized":"2023-10-06T03:07:48.538901+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:25.765674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-06T03:07:45.937350+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:45.937350+0000","last_peered":"2023-10-06T03:07:45.937350+0000","last_clean":"2023-10-06T03:07:45.937350+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:45.937350+0000","last_undegraded":"2023-10-06T03:07:45.937350+0000","last_fullsized":"2023-10-06T03:07:45.937350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303180,"kb_used_data":1796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068660,"statfs":{"total":96636764160,"available":96326307840,"internally_reserved":0,"allocated":1839104,"data_stored":1156061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.44}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":0.498}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.312},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.376},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302600,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069240,"statfs":{"total":96636764160,"available":96326901760,"internally_reserved":0,"allocated":1245184,"data_stored":565693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.503},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.248},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.26400000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.439},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.38}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.45300000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.41799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303180,"kb_used_data":1796,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068660,"statfs":{"total":96636764160,"available":96326307840,"internally_reserved":0,"allocated":1839104,"data_stored":1156061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.314}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.34100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.27500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.47799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.217},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.33300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.28899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.316},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.502},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.626},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302600,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069240,"statfs":{"total":96636764160,"available":96326901760,"internally_reserved":0,"allocated":1245184,"data_stored":565693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.439},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.373}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.50800000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.58599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":559535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.436}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.55900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:54.993 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:55.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.138+0000 7f0bbc94a700 1 Processor -- start 2023-10-06T03:07:55.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.138+0000 7f0bbc94a700 1 -- start start 2023-10-06T03:07:55.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bbc94a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bb8159420 0x7f0bb80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bbc94a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb81597f0 0x7f0bb80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bbc94a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb80b11d0 con 0x7f0bb815d230 2023-10-06T03:07:55.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bbc94a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb80b14b0 con 0x7f0bb8159420 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bbc94a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80b1790 con 0x7f0bb81597f0 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb6d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0bb8159420 0x7f0bb80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb759e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0bb815d230 0x7f0bb80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47008/0 (socket says 172.21.15.92:47008) 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb759e700 1 -- 172.21.15.92:0/868757219 learned_addr learned my addr 172.21.15.92:0/868757219 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb659c700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb81597f0 0x7f0bb80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/868757219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb81597f0 msgr2=0x7f0bb80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb6d9d700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb81597f0 0x7f0bb80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/868757219 >> v1:172.21.15.92:6789/0 conn(0x7f0bb815d230 legacy=0x7f0bb80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb5d9b700 1 -- 172.21.15.92:0/868757219 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017991556 0 0) 0x7f0bb80b11d0 con 0x7f0bb815d230 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/868757219 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb80b1a70 con 0x7f0bb8159420 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb659c700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb81597f0 0x7f0bb80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:55.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.139+0000 7f0bb6d9d700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb80b5f40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098e0 tx=0x7f0ba80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23b763572b04ffa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bb5d9b700 1 -- 172.21.15.92:0/868757219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8009e80 con 0x7f0bb8159420 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bb5d9b700 1 -- 172.21.15.92:0/868757219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba800b860 con 0x7f0bb8159420 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bb5d9b700 1 -- 172.21.15.92:0/868757219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800bbe0 con 0x7f0bb8159420 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 -- 172.21.15.92:0/868757219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 msgr2=0x7f0bb80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb80b5f40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098e0 tx=0x7f0ba80091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 -- 172.21.15.92:0/868757219 shutdown_connections 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb80b5f40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/868757219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb81597f0 0x7f0bb80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 -- 172.21.15.92:0/868757219 >> 172.21.15.92:0/868757219 conn(0x7f0bb80d0660 msgr2=0x7f0bb80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:55.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 -- 172.21.15.92:0/868757219 shutdown_connections 2023-10-06T03:07:55.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.140+0000 7f0bbc94a700 1 -- 172.21.15.92:0/868757219 wait complete. 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.141+0000 7f0bbc94a700 1 Processor -- start 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 -- start start 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb8156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bb81597f0 0x7f0bb8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb815d230 0x7f0bb815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8143a50 con 0x7f0bb8159420 2023-10-06T03:07:55.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80c9cf0 con 0x7f0bb81597f0 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bbc94a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb80c9fd0 con 0x7f0bb815d230 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bb6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb8156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bb6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb8156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50750/0 (socket says 172.21.15.92:50750) 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.142+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/3995490831 learned_addr learned my addr 172.21.15.92:0/3995490831 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb759e700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb815d230 0x7f0bb815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb659c700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bb81597f0 0x7f0bb8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb815d230 msgr2=0x7f0bb815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb6d9d700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb815d230 0x7f0bb815a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bb81597f0 msgr2=0x7f0bb8143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb6d9d700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bb81597f0 0x7f0bb8143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb6d9d700 1 -- 172.21.15.92:0/3995490831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba8009420 con 0x7f0bb8159420 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb759e700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb815d230 0x7f0bb815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb659c700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bb81597f0 0x7f0bb8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bb6d9d700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb8156eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8089340 tx=0x7f0ba8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800bbe0 con 0x7f0bb8159420 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8011420 con 0x7f0bb8159420 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb80ca2b0 con 0x7f0bb8159420 2023-10-06T03:07:55.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80117e0 con 0x7f0bb8159420 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.143+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb80ca870 con 0x7f0bb8159420 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.144+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb804c3c0 con 0x7f0bb8159420 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.144+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0ba8011bd0 con 0x7f0bb8159420 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.145+0000 7f0b9effd700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ba4067290 0x7f0ba4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.145+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0ba8086740 con 0x7f0bb8159420 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.145+0000 7f0bb659c700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ba4067290 0x7f0ba4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.145+0000 7f0bb659c700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ba4067290 0x7f0ba4069740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806a310 tx=0x7f0bac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.149+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ba8053440 con 0x7f0bb8159420 2023-10-06T03:07:55.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.328+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bb80b1d50 con 0x7f0ba4067290 2023-10-06T03:07:55.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.337+0000 7f0b9effd700 1 -- 172.21.15.92:0/3995490831 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312149 (secure 0 0 0) 0x7f0bb80b1d50 con 0x7f0ba4067290 2023-10-06T03:07:55.340 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:55.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ba4067290 msgr2=0x7f0ba4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ba4067290 0x7f0ba4069740 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0bb806a310 tx=0x7f0bac008040 comp rx=0 tx=0).stop 2023-10-06T03:07:55.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 msgr2=0x7f0bb8156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb8156eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8089340 tx=0x7f0ba8017000 comp rx=0 tx=0).stop 2023-10-06T03:07:55.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 shutdown_connections 2023-10-06T03:07:55.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0ba4067290 0x7f0ba4069740 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bb8159420 0x7f0bb8156eb0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bb815d230 0x7f0bb815a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 --2- 172.21.15.92:0/3995490831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bb81597f0 0x7f0bb8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 >> 172.21.15.92:0/3995490831 conn(0x7f0bb80d0660 msgr2=0x7f0bb80696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 shutdown_connections 2023-10-06T03:07:55.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.340+0000 7f0bbc94a700 1 -- 172.21.15.92:0/3995490831 wait complete. 2023-10-06T03:07:55.359 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:55.359 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2023-10-06T03:07:55.258129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405928,"kb_used_data":11240,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752568792,"statfs":{"total":773094113280,"available":770630443008,"internally_reserved":0,"allocated":11509760,"data_stored":5873876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001037"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.578037+0000","last_change":"2023-10-06T03:07:50.572362+0000","last_active":"2023-10-06T03:07:50.578037+0000","last_peered":"2023-10-06T03:07:50.578037+0000","last_clean":"2023-10-06T03:07:50.578037+0000","last_became_active":"2023-10-06T03:07:50.571731+0000","last_became_peered":"2023-10-06T03:07:50.571731+0000","last_unstale":"2023-10-06T03:07:50.578037+0000","last_undegraded":"2023-10-06T03:07:50.578037+0000","last_fullsized":"2023-10-06T03:07:50.578037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:40.777609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584493+0000","last_change":"2023-10-06T03:07:50.576225+0000","last_active":"2023-10-06T03:07:50.584493+0000","last_peered":"2023-10-06T03:07:50.584493+0000","last_clean":"2023-10-06T03:07:50.584493+0000","last_became_active":"2023-10-06T03:07:50.575924+0000","last_became_peered":"2023-10-06T03:07:50.575924+0000","last_unstale":"2023-10-06T03:07:50.584493+0000","last_undegraded":"2023-10-06T03:07:50.584493+0000","last_fullsized":"2023-10-06T03:07:50.584493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:56.990336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584610+0000","last_change":"2023-10-06T03:07:50.578507+0000","last_active":"2023-10-06T03:07:50.584610+0000","last_peered":"2023-10-06T03:07:50.584610+0000","last_clean":"2023-10-06T03:07:50.584610+0000","last_became_active":"2023-10-06T03:07:50.578265+0000","last_became_peered":"2023-10-06T03:07:50.578265+0000","last_unstale":"2023-10-06T03:07:50.584610+0000","last_undegraded":"2023-10-06T03:07:50.584610+0000","last_fullsized":"2023-10-06T03:07:50.584610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573221+0000","last_change":"2023-10-06T03:07:50.571328+0000","last_active":"2023-10-06T03:07:50.573221+0000","last_peered":"2023-10-06T03:07:50.573221+0000","last_clean":"2023-10-06T03:07:50.573221+0000","last_became_active":"2023-10-06T03:07:50.571025+0000","last_became_peered":"2023-10-06T03:07:50.571025+0000","last_unstale":"2023-10-06T03:07:50.573221+0000","last_undegraded":"2023-10-06T03:07:50.573221+0000","last_fullsized":"2023-10-06T03:07:50.573221+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:38.614125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.564726+0000","last_change":"2023-10-06T03:07:50.559179+0000","last_active":"2023-10-06T03:07:50.564726+0000","last_peered":"2023-10-06T03:07:50.564726+0000","last_clean":"2023-10-06T03:07:50.564726+0000","last_became_active":"2023-10-06T03:07:50.558881+0000","last_became_peered":"2023-10-06T03:07:50.558881+0000","last_unstale":"2023-10-06T03:07:50.564726+0000","last_undegraded":"2023-10-06T03:07:50.564726+0000","last_fullsized":"2023-10-06T03:07:50.564726+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:47.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579171+0000","last_change":"2023-10-06T03:07:50.570727+0000","last_active":"2023-10-06T03:07:50.579171+0000","last_peered":"2023-10-06T03:07:50.579171+0000","last_clean":"2023-10-06T03:07:50.579171+0000","last_became_active":"2023-10-06T03:07:50.570484+0000","last_became_peered":"2023-10-06T03:07:50.570484+0000","last_unstale":"2023-10-06T03:07:50.579171+0000","last_undegraded":"2023-10-06T03:07:50.579171+0000","last_fullsized":"2023-10-06T03:07:50.579171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:57.184437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569950+0000","last_change":"2023-10-06T03:07:50.563316+0000","last_active":"2023-10-06T03:07:50.569950+0000","last_peered":"2023-10-06T03:07:50.569950+0000","last_clean":"2023-10-06T03:07:50.569950+0000","last_became_active":"2023-10-06T03:07:50.563031+0000","last_became_peered":"2023-10-06T03:07:50.563031+0000","last_unstale":"2023-10-06T03:07:50.569950+0000","last_undegraded":"2023-10-06T03:07:50.569950+0000","last_fullsized":"2023-10-06T03:07:50.569950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:28.830391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584901+0000","last_change":"2023-10-06T03:07:50.580182+0000","last_active":"2023-10-06T03:07:50.584901+0000","last_peered":"2023-10-06T03:07:50.584901+0000","last_clean":"2023-10-06T03:07:50.584901+0000","last_became_active":"2023-10-06T03:07:50.580001+0000","last_became_peered":"2023-10-06T03:07:50.580001+0000","last_unstale":"2023-10-06T03:07:50.584901+0000","last_undegraded":"2023-10-06T03:07:50.584901+0000","last_fullsized":"2023-10-06T03:07:50.584901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:27.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582970+0000","last_change":"2023-10-06T03:07:50.577719+0000","last_active":"2023-10-06T03:07:50.582970+0000","last_peered":"2023-10-06T03:07:50.582970+0000","last_clean":"2023-10-06T03:07:50.582970+0000","last_became_active":"2023-10-06T03:07:50.577452+0000","last_became_peered":"2023-10-06T03:07:50.577452+0000","last_unstale":"2023-10-06T03:07:50.582970+0000","last_undegraded":"2023-10-06T03:07:50.582970+0000","last_fullsized":"2023-10-06T03:07:50.582970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:08.241579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582530+0000","last_change":"2023-10-06T03:07:50.571565+0000","last_active":"2023-10-06T03:07:50.582530+0000","last_peered":"2023-10-06T03:07:50.582530+0000","last_clean":"2023-10-06T03:07:50.582530+0000","last_became_active":"2023-10-06T03:07:50.570520+0000","last_became_peered":"2023-10-06T03:07:50.570520+0000","last_unstale":"2023-10-06T03:07:50.582530+0000","last_undegraded":"2023-10-06T03:07:50.582530+0000","last_fullsized":"2023-10-06T03:07:50.582530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:26.295707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567219+0000","last_change":"2023-10-06T03:07:50.562739+0000","last_active":"2023-10-06T03:07:50.567219+0000","last_peered":"2023-10-06T03:07:50.567219+0000","last_clean":"2023-10-06T03:07:50.567219+0000","last_became_active":"2023-10-06T03:07:50.561529+0000","last_became_peered":"2023-10-06T03:07:50.561529+0000","last_unstale":"2023-10-06T03:07:50.567219+0000","last_undegraded":"2023-10-06T03:07:50.567219+0000","last_fullsized":"2023-10-06T03:07:50.567219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:33.483532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579373+0000","last_change":"2023-10-06T03:07:50.570788+0000","last_active":"2023-10-06T03:07:50.579373+0000","last_peered":"2023-10-06T03:07:50.579373+0000","last_clean":"2023-10-06T03:07:50.579373+0000","last_became_active":"2023-10-06T03:07:50.570296+0000","last_became_peered":"2023-10-06T03:07:50.570296+0000","last_unstale":"2023-10-06T03:07:50.579373+0000","last_undegraded":"2023-10-06T03:07:50.579373+0000","last_fullsized":"2023-10-06T03:07:50.579373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:56.995621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581561+0000","last_change":"2023-10-06T03:07:50.572906+0000","last_active":"2023-10-06T03:07:50.581561+0000","last_peered":"2023-10-06T03:07:50.581561+0000","last_clean":"2023-10-06T03:07:50.581561+0000","last_became_active":"2023-10-06T03:07:50.572656+0000","last_became_peered":"2023-10-06T03:07:50.572656+0000","last_unstale":"2023-10-06T03:07:50.581561+0000","last_undegraded":"2023-10-06T03:07:50.581561+0000","last_fullsized":"2023-10-06T03:07:50.581561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:16.879548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584804+0000","last_change":"2023-10-06T03:07:50.580252+0000","last_active":"2023-10-06T03:07:50.584804+0000","last_peered":"2023-10-06T03:07:50.584804+0000","last_clean":"2023-10-06T03:07:50.584804+0000","last_became_active":"2023-10-06T03:07:50.578591+0000","last_became_peered":"2023-10-06T03:07:50.578591+0000","last_unstale":"2023-10-06T03:07:50.584804+0000","last_undegraded":"2023-10-06T03:07:50.584804+0000","last_fullsized":"2023-10-06T03:07:50.584804+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567088+0000","last_change":"2023-10-06T03:07:50.562853+0000","last_active":"2023-10-06T03:07:50.567088+0000","last_peered":"2023-10-06T03:07:50.567088+0000","last_clean":"2023-10-06T03:07:50.567088+0000","last_became_active":"2023-10-06T03:07:50.561839+0000","last_became_peered":"2023-10-06T03:07:50.561839+0000","last_unstale":"2023-10-06T03:07:50.567088+0000","last_undegraded":"2023-10-06T03:07:50.567088+0000","last_fullsized":"2023-10-06T03:07:50.567088+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:25.304588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573661+0000","last_change":"2023-10-06T03:07:50.567005+0000","last_active":"2023-10-06T03:07:50.573661+0000","last_peered":"2023-10-06T03:07:50.573661+0000","last_clean":"2023-10-06T03:07:50.573661+0000","last_became_active":"2023-10-06T03:07:50.566760+0000","last_became_peered":"2023-10-06T03:07:50.566760+0000","last_unstale":"2023-10-06T03:07:50.573661+0000","last_undegraded":"2023-10-06T03:07:50.573661+0000","last_fullsized":"2023-10-06T03:07:50.573661+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:21.538700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581519+0000","last_change":"2023-10-06T03:07:50.573244+0000","last_active":"2023-10-06T03:07:50.581519+0000","last_peered":"2023-10-06T03:07:50.581519+0000","last_clean":"2023-10-06T03:07:50.581519+0000","last_became_active":"2023-10-06T03:07:50.573001+0000","last_became_peered":"2023-10-06T03:07:50.573001+0000","last_unstale":"2023-10-06T03:07:50.581519+0000","last_undegraded":"2023-10-06T03:07:50.581519+0000","last_fullsized":"2023-10-06T03:07:50.581519+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:58.959529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584666+0000","last_change":"2023-10-06T03:07:50.579168+0000","last_active":"2023-10-06T03:07:50.584666+0000","last_peered":"2023-10-06T03:07:50.584666+0000","last_clean":"2023-10-06T03:07:50.584666+0000","last_became_active":"2023-10-06T03:07:50.577986+0000","last_became_peered":"2023-10-06T03:07:50.577986+0000","last_unstale":"2023-10-06T03:07:50.584666+0000","last_undegraded":"2023-10-06T03:07:50.584666+0000","last_fullsized":"2023-10-06T03:07:50.584666+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:30.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570414+0000","last_change":"2023-10-06T03:07:50.565479+0000","last_active":"2023-10-06T03:07:50.570414+0000","last_peered":"2023-10-06T03:07:50.570414+0000","last_clean":"2023-10-06T03:07:50.570414+0000","last_became_active":"2023-10-06T03:07:50.565172+0000","last_became_peered":"2023-10-06T03:07:50.565172+0000","last_unstale":"2023-10-06T03:07:50.570414+0000","last_undegraded":"2023-10-06T03:07:50.570414+0000","last_fullsized":"2023-10-06T03:07:50.570414+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:26.295707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576083+0000","last_change":"2023-10-06T03:07:50.573007+0000","last_active":"2023-10-06T03:07:50.576083+0000","last_peered":"2023-10-06T03:07:50.576083+0000","last_clean":"2023-10-06T03:07:50.576083+0000","last_became_active":"2023-10-06T03:07:50.572729+0000","last_became_peered":"2023-10-06T03:07:50.572729+0000","last_unstale":"2023-10-06T03:07:50.576083+0000","last_undegraded":"2023-10-06T03:07:50.576083+0000","last_fullsized":"2023-10-06T03:07:50.576083+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:45.959875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577990+0000","last_change":"2023-10-06T03:07:50.574602+0000","last_active":"2023-10-06T03:07:50.577990+0000","last_peered":"2023-10-06T03:07:50.577990+0000","last_clean":"2023-10-06T03:07:50.577990+0000","last_became_active":"2023-10-06T03:07:50.573933+0000","last_became_peered":"2023-10-06T03:07:50.573933+0000","last_unstale":"2023-10-06T03:07:50.577990+0000","last_undegraded":"2023-10-06T03:07:50.577990+0000","last_fullsized":"2023-10-06T03:07:50.577990+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.580385+0000","last_change":"2023-10-06T03:07:50.579726+0000","last_active":"2023-10-06T03:07:50.580385+0000","last_peered":"2023-10-06T03:07:50.580385+0000","last_clean":"2023-10-06T03:07:50.580385+0000","last_became_active":"2023-10-06T03:07:50.579482+0000","last_became_peered":"2023-10-06T03:07:50.579482+0000","last_unstale":"2023-10-06T03:07:50.580385+0000","last_undegraded":"2023-10-06T03:07:50.580385+0000","last_fullsized":"2023-10-06T03:07:50.580385+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:48.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587676+0000","last_change":"2023-10-06T03:07:50.581210+0000","last_active":"2023-10-06T03:07:50.587676+0000","last_peered":"2023-10-06T03:07:50.587676+0000","last_clean":"2023-10-06T03:07:50.587676+0000","last_became_active":"2023-10-06T03:07:50.580986+0000","last_became_peered":"2023-10-06T03:07:50.580986+0000","last_unstale":"2023-10-06T03:07:50.587676+0000","last_undegraded":"2023-10-06T03:07:50.587676+0000","last_fullsized":"2023-10-06T03:07:50.587676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:41.442852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569161+0000","last_change":"2023-10-06T03:07:50.561654+0000","last_active":"2023-10-06T03:07:50.569161+0000","last_peered":"2023-10-06T03:07:50.569161+0000","last_clean":"2023-10-06T03:07:50.569161+0000","last_became_active":"2023-10-06T03:07:50.561023+0000","last_became_peered":"2023-10-06T03:07:50.561023+0000","last_unstale":"2023-10-06T03:07:50.569161+0000","last_undegraded":"2023-10-06T03:07:50.569161+0000","last_fullsized":"2023-10-06T03:07:50.569161+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:44.578103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569197+0000","last_change":"2023-10-06T03:07:50.562602+0000","last_active":"2023-10-06T03:07:50.569197+0000","last_peered":"2023-10-06T03:07:50.569197+0000","last_clean":"2023-10-06T03:07:50.569197+0000","last_became_active":"2023-10-06T03:07:50.562303+0000","last_became_peered":"2023-10-06T03:07:50.562303+0000","last_unstale":"2023-10-06T03:07:50.569197+0000","last_undegraded":"2023-10-06T03:07:50.569197+0000","last_fullsized":"2023-10-06T03:07:50.569197+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:56.990336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575245+0000","last_change":"2023-10-06T03:07:50.569444+0000","last_active":"2023-10-06T03:07:50.575245+0000","last_peered":"2023-10-06T03:07:50.575245+0000","last_clean":"2023-10-06T03:07:50.575245+0000","last_became_active":"2023-10-06T03:07:50.569142+0000","last_became_peered":"2023-10-06T03:07:50.569142+0000","last_unstale":"2023-10-06T03:07:50.575245+0000","last_undegraded":"2023-10-06T03:07:50.575245+0000","last_fullsized":"2023-10-06T03:07:50.575245+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:18.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577568+0000","last_change":"2023-10-06T03:07:50.570913+0000","last_active":"2023-10-06T03:07:50.577568+0000","last_peered":"2023-10-06T03:07:50.577568+0000","last_clean":"2023-10-06T03:07:50.577568+0000","last_became_active":"2023-10-06T03:07:50.570641+0000","last_became_peered":"2023-10-06T03:07:50.570641+0000","last_unstale":"2023-10-06T03:07:50.577568+0000","last_undegraded":"2023-10-06T03:07:50.577568+0000","last_fullsized":"2023-10-06T03:07:50.577568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:22.069683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579352+0000","last_change":"2023-10-06T03:07:50.570632+0000","last_active":"2023-10-06T03:07:50.579352+0000","last_peered":"2023-10-06T03:07:50.579352+0000","last_clean":"2023-10-06T03:07:50.579352+0000","last_became_active":"2023-10-06T03:07:50.569983+0000","last_became_peered":"2023-10-06T03:07:50.569983+0000","last_unstale":"2023-10-06T03:07:50.579352+0000","last_undegraded":"2023-10-06T03:07:50.579352+0000","last_fullsized":"2023-10-06T03:07:50.579352+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.064356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582225+0000","last_change":"2023-10-06T03:07:50.577544+0000","last_active":"2023-10-06T03:07:50.582225+0000","last_peered":"2023-10-06T03:07:50.582225+0000","last_clean":"2023-10-06T03:07:50.582225+0000","last_became_active":"2023-10-06T03:07:50.577322+0000","last_became_peered":"2023-10-06T03:07:50.577322+0000","last_unstale":"2023-10-06T03:07:50.582225+0000","last_undegraded":"2023-10-06T03:07:50.582225+0000","last_fullsized":"2023-10-06T03:07:50.582225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:12.914946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585038+0000","last_change":"2023-10-06T03:07:50.581285+0000","last_active":"2023-10-06T03:07:50.585038+0000","last_peered":"2023-10-06T03:07:50.585038+0000","last_clean":"2023-10-06T03:07:50.585038+0000","last_became_active":"2023-10-06T03:07:50.581039+0000","last_became_peered":"2023-10-06T03:07:50.581039+0000","last_unstale":"2023-10-06T03:07:50.585038+0000","last_undegraded":"2023-10-06T03:07:50.585038+0000","last_fullsized":"2023-10-06T03:07:50.585038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:02.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574262+0000","last_change":"2023-10-06T03:07:50.566299+0000","last_active":"2023-10-06T03:07:50.574262+0000","last_peered":"2023-10-06T03:07:50.574262+0000","last_clean":"2023-10-06T03:07:50.574262+0000","last_became_active":"2023-10-06T03:07:50.566026+0000","last_became_peered":"2023-10-06T03:07:50.566026+0000","last_unstale":"2023-10-06T03:07:50.574262+0000","last_undegraded":"2023-10-06T03:07:50.574262+0000","last_fullsized":"2023-10-06T03:07:50.574262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:30.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569182+0000","last_change":"2023-10-06T03:07:50.561817+0000","last_active":"2023-10-06T03:07:50.569182+0000","last_peered":"2023-10-06T03:07:50.569182+0000","last_clean":"2023-10-06T03:07:50.569182+0000","last_became_active":"2023-10-06T03:07:50.561357+0000","last_became_peered":"2023-10-06T03:07:50.561357+0000","last_unstale":"2023-10-06T03:07:50.569182+0000","last_undegraded":"2023-10-06T03:07:50.569182+0000","last_fullsized":"2023-10-06T03:07:50.569182+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:26.301309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581912+0000","last_change":"2023-10-06T03:07:50.577842+0000","last_active":"2023-10-06T03:07:50.581912+0000","last_peered":"2023-10-06T03:07:50.581912+0000","last_clean":"2023-10-06T03:07:50.581912+0000","last_became_active":"2023-10-06T03:07:50.577581+0000","last_became_peered":"2023-10-06T03:07:50.577581+0000","last_unstale":"2023-10-06T03:07:50.581912+0000","last_undegraded":"2023-10-06T03:07:50.581912+0000","last_fullsized":"2023-10-06T03:07:50.581912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:18.485934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"102'4","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572923+0000","last_change":"2023-10-06T03:07:49.568419+0000","last_active":"2023-10-06T03:07:50.572923+0000","last_peered":"2023-10-06T03:07:50.572923+0000","last_clean":"2023-10-06T03:07:50.572923+0000","last_became_active":"2023-10-06T03:07:49.568113+0000","last_became_peered":"2023-10-06T03:07:49.568113+0000","last_unstale":"2023-10-06T03:07:50.572923+0000","last_undegraded":"2023-10-06T03:07:50.572923+0000","last_fullsized":"2023-10-06T03:07:50.572923+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584422+0000","last_change":"2023-10-06T03:07:50.578986+0000","last_active":"2023-10-06T03:07:50.584422+0000","last_peered":"2023-10-06T03:07:50.584422+0000","last_clean":"2023-10-06T03:07:50.584422+0000","last_became_active":"2023-10-06T03:07:50.578662+0000","last_became_peered":"2023-10-06T03:07:50.578662+0000","last_unstale":"2023-10-06T03:07:50.584422+0000","last_undegraded":"2023-10-06T03:07:50.584422+0000","last_fullsized":"2023-10-06T03:07:50.584422+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:13.437927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573465+0000","last_change":"2023-10-06T03:07:49.568589+0000","last_active":"2023-10-06T03:07:50.573465+0000","last_peered":"2023-10-06T03:07:50.573465+0000","last_clean":"2023-10-06T03:07:50.573465+0000","last_became_active":"2023-10-06T03:07:49.568313+0000","last_became_peered":"2023-10-06T03:07:49.568313+0000","last_unstale":"2023-10-06T03:07:50.573465+0000","last_undegraded":"2023-10-06T03:07:50.573465+0000","last_fullsized":"2023-10-06T03:07:50.573465+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:20.278853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572529+0000","last_change":"2023-10-06T03:07:50.567007+0000","last_active":"2023-10-06T03:07:50.572529+0000","last_peered":"2023-10-06T03:07:50.572529+0000","last_clean":"2023-10-06T03:07:50.572529+0000","last_became_active":"2023-10-06T03:07:50.566733+0000","last_became_peered":"2023-10-06T03:07:50.566733+0000","last_unstale":"2023-10-06T03:07:50.572529+0000","last_undegraded":"2023-10-06T03:07:50.572529+0000","last_fullsized":"2023-10-06T03:07:50.572529+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:54.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573062+0000","last_change":"2023-10-06T03:07:49.568215+0000","last_active":"2023-10-06T03:07:50.573062+0000","last_peered":"2023-10-06T03:07:50.573062+0000","last_clean":"2023-10-06T03:07:50.573062+0000","last_became_active":"2023-10-06T03:07:49.567900+0000","last_became_peered":"2023-10-06T03:07:49.567900+0000","last_unstale":"2023-10-06T03:07:50.573062+0000","last_undegraded":"2023-10-06T03:07:50.573062+0000","last_fullsized":"2023-10-06T03:07:50.573062+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:25.296585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577913+0000","last_change":"2023-10-06T03:07:50.573487+0000","last_active":"2023-10-06T03:07:50.577913+0000","last_peered":"2023-10-06T03:07:50.577913+0000","last_clean":"2023-10-06T03:07:50.577913+0000","last_became_active":"2023-10-06T03:07:50.573233+0000","last_became_peered":"2023-10-06T03:07:50.573233+0000","last_unstale":"2023-10-06T03:07:50.577913+0000","last_undegraded":"2023-10-06T03:07:50.577913+0000","last_fullsized":"2023-10-06T03:07:50.577913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:18:39.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582880+0000","last_change":"2023-10-06T03:07:49.561326+0000","last_active":"2023-10-06T03:07:50.582880+0000","last_peered":"2023-10-06T03:07:50.582880+0000","last_clean":"2023-10-06T03:07:50.582880+0000","last_became_active":"2023-10-06T03:07:49.561103+0000","last_became_peered":"2023-10-06T03:07:49.561103+0000","last_unstale":"2023-10-06T03:07:50.582880+0000","last_undegraded":"2023-10-06T03:07:50.582880+0000","last_fullsized":"2023-10-06T03:07:50.582880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:25.765674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576162+0000","last_change":"2023-10-06T03:07:50.571130+0000","last_active":"2023-10-06T03:07:50.576162+0000","last_peered":"2023-10-06T03:07:50.576162+0000","last_clean":"2023-10-06T03:07:50.576162+0000","last_became_active":"2023-10-06T03:07:50.570902+0000","last_became_peered":"2023-10-06T03:07:50.570902+0000","last_unstale":"2023-10-06T03:07:50.576162+0000","last_undegraded":"2023-10-06T03:07:50.576162+0000","last_fullsized":"2023-10-06T03:07:50.576162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:22.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572082+0000","last_change":"2023-10-06T03:07:49.564406+0000","last_active":"2023-10-06T03:07:50.572082+0000","last_peered":"2023-10-06T03:07:50.572082+0000","last_clean":"2023-10-06T03:07:50.572082+0000","last_became_active":"2023-10-06T03:07:49.564128+0000","last_became_peered":"2023-10-06T03:07:49.564128+0000","last_unstale":"2023-10-06T03:07:50.572082+0000","last_undegraded":"2023-10-06T03:07:50.572082+0000","last_fullsized":"2023-10-06T03:07:50.572082+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:49.009245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.568929+0000","last_change":"2023-10-06T03:07:50.565122+0000","last_active":"2023-10-06T03:07:50.568929+0000","last_peered":"2023-10-06T03:07:50.568929+0000","last_clean":"2023-10-06T03:07:50.568929+0000","last_became_active":"2023-10-06T03:07:50.564842+0000","last_became_peered":"2023-10-06T03:07:50.564842+0000","last_unstale":"2023-10-06T03:07:50.568929+0000","last_undegraded":"2023-10-06T03:07:50.568929+0000","last_fullsized":"2023-10-06T03:07:50.568929+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:20.294921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579362+0000","last_change":"2023-10-06T03:07:49.564123+0000","last_active":"2023-10-06T03:07:50.579362+0000","last_peered":"2023-10-06T03:07:50.579362+0000","last_clean":"2023-10-06T03:07:50.579362+0000","last_became_active":"2023-10-06T03:07:49.563157+0000","last_became_peered":"2023-10-06T03:07:49.563157+0000","last_unstale":"2023-10-06T03:07:50.579362+0000","last_undegraded":"2023-10-06T03:07:50.579362+0000","last_fullsized":"2023-10-06T03:07:50.579362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:51.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584725+0000","last_change":"2023-10-06T03:07:50.578869+0000","last_active":"2023-10-06T03:07:50.584725+0000","last_peered":"2023-10-06T03:07:50.584725+0000","last_clean":"2023-10-06T03:07:50.584725+0000","last_became_active":"2023-10-06T03:07:50.578111+0000","last_became_peered":"2023-10-06T03:07:50.578111+0000","last_unstale":"2023-10-06T03:07:50.584725+0000","last_undegraded":"2023-10-06T03:07:50.584725+0000","last_fullsized":"2023-10-06T03:07:50.584725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:18:39.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.554122+0000","last_change":"2023-10-06T03:07:49.570378+0000","last_active":"2023-10-06T03:07:50.554122+0000","last_peered":"2023-10-06T03:07:50.554122+0000","last_clean":"2023-10-06T03:07:50.554122+0000","last_became_active":"2023-10-06T03:07:49.570112+0000","last_became_peered":"2023-10-06T03:07:49.570112+0000","last_unstale":"2023-10-06T03:07:50.554122+0000","last_undegraded":"2023-10-06T03:07:50.554122+0000","last_fullsized":"2023-10-06T03:07:50.554122+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:21.074952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581873+0000","last_change":"2023-10-06T03:07:50.577006+0000","last_active":"2023-10-06T03:07:50.581873+0000","last_peered":"2023-10-06T03:07:50.581873+0000","last_clean":"2023-10-06T03:07:50.581873+0000","last_became_active":"2023-10-06T03:07:50.576746+0000","last_became_peered":"2023-10-06T03:07:50.576746+0000","last_unstale":"2023-10-06T03:07:50.581873+0000","last_undegraded":"2023-10-06T03:07:50.581873+0000","last_fullsized":"2023-10-06T03:07:50.581873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:58.698362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552740+0000","last_change":"2023-10-06T03:07:49.563228+0000","last_active":"2023-10-06T03:07:50.552740+0000","last_peered":"2023-10-06T03:07:50.552740+0000","last_clean":"2023-10-06T03:07:50.552740+0000","last_became_active":"2023-10-06T03:07:49.562933+0000","last_became_peered":"2023-10-06T03:07:49.562933+0000","last_unstale":"2023-10-06T03:07:50.552740+0000","last_undegraded":"2023-10-06T03:07:50.552740+0000","last_fullsized":"2023-10-06T03:07:50.552740+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:19.290197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577687+0000","last_change":"2023-10-06T03:07:50.571814+0000","last_active":"2023-10-06T03:07:50.577687+0000","last_peered":"2023-10-06T03:07:50.577687+0000","last_clean":"2023-10-06T03:07:50.577687+0000","last_became_active":"2023-10-06T03:07:50.571432+0000","last_became_peered":"2023-10-06T03:07:50.571432+0000","last_unstale":"2023-10-06T03:07:50.577687+0000","last_undegraded":"2023-10-06T03:07:50.577687+0000","last_fullsized":"2023-10-06T03:07:50.577687+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:50.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574836+0000","last_change":"2023-10-06T03:07:49.568577+0000","last_active":"2023-10-06T03:07:50.574836+0000","last_peered":"2023-10-06T03:07:50.574836+0000","last_clean":"2023-10-06T03:07:50.574836+0000","last_became_active":"2023-10-06T03:07:49.568351+0000","last_became_peered":"2023-10-06T03:07:49.568351+0000","last_unstale":"2023-10-06T03:07:50.574836+0000","last_undegraded":"2023-10-06T03:07:50.574836+0000","last_fullsized":"2023-10-06T03:07:50.574836+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:59.860977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587789+0000","last_change":"2023-10-06T03:07:50.583381+0000","last_active":"2023-10-06T03:07:50.587789+0000","last_peered":"2023-10-06T03:07:50.587789+0000","last_clean":"2023-10-06T03:07:50.587789+0000","last_became_active":"2023-10-06T03:07:50.583119+0000","last_became_peered":"2023-10-06T03:07:50.583119+0000","last_unstale":"2023-10-06T03:07:50.587789+0000","last_undegraded":"2023-10-06T03:07:50.587789+0000","last_fullsized":"2023-10-06T03:07:50.587789+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:07.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575307+0000","last_change":"2023-10-06T03:07:49.567393+0000","last_active":"2023-10-06T03:07:50.575307+0000","last_peered":"2023-10-06T03:07:50.575307+0000","last_clean":"2023-10-06T03:07:50.575307+0000","last_became_active":"2023-10-06T03:07:49.567129+0000","last_became_peered":"2023-10-06T03:07:49.567129+0000","last_unstale":"2023-10-06T03:07:50.575307+0000","last_undegraded":"2023-10-06T03:07:50.575307+0000","last_fullsized":"2023-10-06T03:07:50.575307+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:41.761455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573801+0000","last_change":"2023-10-06T03:07:50.565621+0000","last_active":"2023-10-06T03:07:50.573801+0000","last_peered":"2023-10-06T03:07:50.573801+0000","last_clean":"2023-10-06T03:07:50.573801+0000","last_became_active":"2023-10-06T03:07:50.565348+0000","last_became_peered":"2023-10-06T03:07:50.565348+0000","last_unstale":"2023-10-06T03:07:50.573801+0000","last_undegraded":"2023-10-06T03:07:50.573801+0000","last_fullsized":"2023-10-06T03:07:50.573801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573173+0000","last_change":"2023-10-06T03:07:49.574956+0000","last_active":"2023-10-06T03:07:50.573173+0000","last_peered":"2023-10-06T03:07:50.573173+0000","last_clean":"2023-10-06T03:07:50.573173+0000","last_became_active":"2023-10-06T03:07:49.574691+0000","last_became_peered":"2023-10-06T03:07:49.574691+0000","last_unstale":"2023-10-06T03:07:50.573173+0000","last_undegraded":"2023-10-06T03:07:50.573173+0000","last_fullsized":"2023-10-06T03:07:50.573173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:15.874825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571996+0000","last_change":"2023-10-06T03:07:50.565193+0000","last_active":"2023-10-06T03:07:50.571996+0000","last_peered":"2023-10-06T03:07:50.571996+0000","last_clean":"2023-10-06T03:07:50.571996+0000","last_became_active":"2023-10-06T03:07:50.564949+0000","last_became_peered":"2023-10-06T03:07:50.564949+0000","last_unstale":"2023-10-06T03:07:50.571996+0000","last_undegraded":"2023-10-06T03:07:50.571996+0000","last_fullsized":"2023-10-06T03:07:50.571996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:07.731428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562713+0000","last_change":"2023-10-06T03:07:49.571624+0000","last_active":"2023-10-06T03:07:50.562713+0000","last_peered":"2023-10-06T03:07:50.562713+0000","last_clean":"2023-10-06T03:07:50.562713+0000","last_became_active":"2023-10-06T03:07:49.571371+0000","last_became_peered":"2023-10-06T03:07:49.571371+0000","last_unstale":"2023-10-06T03:07:50.562713+0000","last_undegraded":"2023-10-06T03:07:50.562713+0000","last_fullsized":"2023-10-06T03:07:50.562713+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:43.573379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570121+0000","last_change":"2023-10-06T03:07:50.559354+0000","last_active":"2023-10-06T03:07:50.570121+0000","last_peered":"2023-10-06T03:07:50.570121+0000","last_clean":"2023-10-06T03:07:50.570121+0000","last_became_active":"2023-10-06T03:07:50.559074+0000","last_became_peered":"2023-10-06T03:07:50.559074+0000","last_unstale":"2023-10-06T03:07:50.570121+0000","last_undegraded":"2023-10-06T03:07:50.570121+0000","last_fullsized":"2023-10-06T03:07:50.570121+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:31.306909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574667+0000","last_change":"2023-10-06T03:07:49.567927+0000","last_active":"2023-10-06T03:07:50.574667+0000","last_peered":"2023-10-06T03:07:50.574667+0000","last_clean":"2023-10-06T03:07:50.574667+0000","last_became_active":"2023-10-06T03:07:49.567651+0000","last_became_peered":"2023-10-06T03:07:49.567651+0000","last_unstale":"2023-10-06T03:07:50.574667+0000","last_undegraded":"2023-10-06T03:07:50.574667+0000","last_fullsized":"2023-10-06T03:07:50.574667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:41.894694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584764+0000","last_change":"2023-10-06T03:07:50.579620+0000","last_active":"2023-10-06T03:07:50.584764+0000","last_peered":"2023-10-06T03:07:50.584764+0000","last_clean":"2023-10-06T03:07:50.584764+0000","last_became_active":"2023-10-06T03:07:50.579374+0000","last_became_peered":"2023-10-06T03:07:50.579374+0000","last_unstale":"2023-10-06T03:07:50.584764+0000","last_undegraded":"2023-10-06T03:07:50.584764+0000","last_fullsized":"2023-10-06T03:07:50.584764+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.561899+0000","last_change":"2023-10-06T03:07:49.566472+0000","last_active":"2023-10-06T03:07:50.561899+0000","last_peered":"2023-10-06T03:07:50.561899+0000","last_clean":"2023-10-06T03:07:50.561899+0000","last_became_active":"2023-10-06T03:07:49.566231+0000","last_became_peered":"2023-10-06T03:07:49.566231+0000","last_unstale":"2023-10-06T03:07:50.561899+0000","last_undegraded":"2023-10-06T03:07:50.561899+0000","last_fullsized":"2023-10-06T03:07:50.561899+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:49.540743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584709+0000","last_change":"2023-10-06T03:07:50.578349+0000","last_active":"2023-10-06T03:07:50.584709+0000","last_peered":"2023-10-06T03:07:50.584709+0000","last_clean":"2023-10-06T03:07:50.584709+0000","last_became_active":"2023-10-06T03:07:50.576833+0000","last_became_peered":"2023-10-06T03:07:50.576833+0000","last_unstale":"2023-10-06T03:07:50.584709+0000","last_undegraded":"2023-10-06T03:07:50.584709+0000","last_fullsized":"2023-10-06T03:07:50.584709+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:07.731428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570946+0000","last_change":"2023-10-06T03:07:49.565372+0000","last_active":"2023-10-06T03:07:50.570946+0000","last_peered":"2023-10-06T03:07:50.570946+0000","last_clean":"2023-10-06T03:07:50.570946+0000","last_became_active":"2023-10-06T03:07:49.565082+0000","last_became_peered":"2023-10-06T03:07:49.565082+0000","last_unstale":"2023-10-06T03:07:50.570946+0000","last_undegraded":"2023-10-06T03:07:50.570946+0000","last_fullsized":"2023-10-06T03:07:50.570946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:21.074952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572382+0000","last_change":"2023-10-06T03:07:50.566374+0000","last_active":"2023-10-06T03:07:50.572382+0000","last_peered":"2023-10-06T03:07:50.572382+0000","last_clean":"2023-10-06T03:07:50.572382+0000","last_became_active":"2023-10-06T03:07:50.565949+0000","last_became_peered":"2023-10-06T03:07:50.565949+0000","last_unstale":"2023-10-06T03:07:50.572382+0000","last_undegraded":"2023-10-06T03:07:50.572382+0000","last_fullsized":"2023-10-06T03:07:50.572382+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:18.274017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562047+0000","last_change":"2023-10-06T03:07:49.567446+0000","last_active":"2023-10-06T03:07:50.562047+0000","last_peered":"2023-10-06T03:07:50.562047+0000","last_clean":"2023-10-06T03:07:50.562047+0000","last_became_active":"2023-10-06T03:07:49.567093+0000","last_became_peered":"2023-10-06T03:07:49.567093+0000","last_unstale":"2023-10-06T03:07:50.562047+0000","last_undegraded":"2023-10-06T03:07:50.562047+0000","last_fullsized":"2023-10-06T03:07:50.562047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:45.671026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574000+0000","last_change":"2023-10-06T03:07:50.568922+0000","last_active":"2023-10-06T03:07:50.574000+0000","last_peered":"2023-10-06T03:07:50.574000+0000","last_clean":"2023-10-06T03:07:50.574000+0000","last_became_active":"2023-10-06T03:07:50.568709+0000","last_became_peered":"2023-10-06T03:07:50.568709+0000","last_unstale":"2023-10-06T03:07:50.574000+0000","last_undegraded":"2023-10-06T03:07:50.574000+0000","last_fullsized":"2023-10-06T03:07:50.574000+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:10.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573350+0000","last_change":"2023-10-06T03:07:49.567779+0000","last_active":"2023-10-06T03:07:50.573350+0000","last_peered":"2023-10-06T03:07:50.573350+0000","last_clean":"2023-10-06T03:07:50.573350+0000","last_became_active":"2023-10-06T03:07:49.567579+0000","last_became_peered":"2023-10-06T03:07:49.567579+0000","last_unstale":"2023-10-06T03:07:50.573350+0000","last_undegraded":"2023-10-06T03:07:50.573350+0000","last_fullsized":"2023-10-06T03:07:50.573350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:48.232486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576201+0000","last_change":"2023-10-06T03:07:50.570770+0000","last_active":"2023-10-06T03:07:50.576201+0000","last_peered":"2023-10-06T03:07:50.576201+0000","last_clean":"2023-10-06T03:07:50.576201+0000","last_became_active":"2023-10-06T03:07:50.570500+0000","last_became_peered":"2023-10-06T03:07:50.570500+0000","last_unstale":"2023-10-06T03:07:50.576201+0000","last_undegraded":"2023-10-06T03:07:50.576201+0000","last_fullsized":"2023-10-06T03:07:50.576201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:02.621345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-06T03:07:52.739762+0000","last_change":"2023-10-06T03:07:49.567546+0000","last_active":"2023-10-06T03:07:52.739762+0000","last_peered":"2023-10-06T03:07:52.739762+0000","last_clean":"2023-10-06T03:07:52.739762+0000","last_became_active":"2023-10-06T03:07:49.567274+0000","last_became_peered":"2023-10-06T03:07:49.567274+0000","last_unstale":"2023-10-06T03:07:52.739762+0000","last_undegraded":"2023-10-06T03:07:52.739762+0000","last_fullsized":"2023-10-06T03:07:52.739762+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573164+0000","last_change":"2023-10-06T03:07:50.570896+0000","last_active":"2023-10-06T03:07:50.573164+0000","last_peered":"2023-10-06T03:07:50.573164+0000","last_clean":"2023-10-06T03:07:50.573164+0000","last_became_active":"2023-10-06T03:07:50.570605+0000","last_became_peered":"2023-10-06T03:07:50.570605+0000","last_unstale":"2023-10-06T03:07:50.573164+0000","last_undegraded":"2023-10-06T03:07:50.573164+0000","last_fullsized":"2023-10-06T03:07:50.573164+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:07.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.558154+0000","last_change":"2023-10-06T03:07:49.571191+0000","last_active":"2023-10-06T03:07:50.558154+0000","last_peered":"2023-10-06T03:07:50.558154+0000","last_clean":"2023-10-06T03:07:50.558154+0000","last_became_active":"2023-10-06T03:07:49.570908+0000","last_became_peered":"2023-10-06T03:07:49.570908+0000","last_unstale":"2023-10-06T03:07:50.558154+0000","last_undegraded":"2023-10-06T03:07:50.558154+0000","last_fullsized":"2023-10-06T03:07:50.558154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581923+0000","last_change":"2023-10-06T03:07:50.573868+0000","last_active":"2023-10-06T03:07:50.581923+0000","last_peered":"2023-10-06T03:07:50.581923+0000","last_clean":"2023-10-06T03:07:50.581923+0000","last_became_active":"2023-10-06T03:07:50.572772+0000","last_became_peered":"2023-10-06T03:07:50.572772+0000","last_unstale":"2023-10-06T03:07:50.581923+0000","last_undegraded":"2023-10-06T03:07:50.581923+0000","last_fullsized":"2023-10-06T03:07:50.581923+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:45.966304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552674+0000","last_change":"2023-10-06T03:07:49.567138+0000","last_active":"2023-10-06T03:07:50.552674+0000","last_peered":"2023-10-06T03:07:50.552674+0000","last_clean":"2023-10-06T03:07:50.552674+0000","last_became_active":"2023-10-06T03:07:49.566884+0000","last_became_peered":"2023-10-06T03:07:49.566884+0000","last_unstale":"2023-10-06T03:07:50.552674+0000","last_undegraded":"2023-10-06T03:07:50.552674+0000","last_fullsized":"2023-10-06T03:07:50.552674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.299865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573641+0000","last_change":"2023-10-06T03:07:50.565737+0000","last_active":"2023-10-06T03:07:50.573641+0000","last_peered":"2023-10-06T03:07:50.573641+0000","last_clean":"2023-10-06T03:07:50.573641+0000","last_became_active":"2023-10-06T03:07:50.565477+0000","last_became_peered":"2023-10-06T03:07:50.565477+0000","last_unstale":"2023-10-06T03:07:50.573641+0000","last_undegraded":"2023-10-06T03:07:50.573641+0000","last_fullsized":"2023-10-06T03:07:50.573641+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:29.710606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.553258+0000","last_change":"2023-10-06T03:07:49.565800+0000","last_active":"2023-10-06T03:07:50.553258+0000","last_peered":"2023-10-06T03:07:50.553258+0000","last_clean":"2023-10-06T03:07:50.553258+0000","last_became_active":"2023-10-06T03:07:49.565480+0000","last_became_peered":"2023-10-06T03:07:49.565480+0000","last_unstale":"2023-10-06T03:07:50.553258+0000","last_undegraded":"2023-10-06T03:07:50.553258+0000","last_fullsized":"2023-10-06T03:07:50.553258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:06.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584782+0000","last_change":"2023-10-06T03:07:50.579751+0000","last_active":"2023-10-06T03:07:50.584782+0000","last_peered":"2023-10-06T03:07:50.584782+0000","last_clean":"2023-10-06T03:07:50.584782+0000","last_became_active":"2023-10-06T03:07:50.579393+0000","last_became_peered":"2023-10-06T03:07:50.579393+0000","last_unstale":"2023-10-06T03:07:50.584782+0000","last_undegraded":"2023-10-06T03:07:50.584782+0000","last_fullsized":"2023-10-06T03:07:50.584782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:18.274017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552787+0000","last_change":"2023-10-06T03:07:49.574874+0000","last_active":"2023-10-06T03:07:50.552787+0000","last_peered":"2023-10-06T03:07:50.552787+0000","last_clean":"2023-10-06T03:07:50.552787+0000","last_became_active":"2023-10-06T03:07:49.574539+0000","last_became_peered":"2023-10-06T03:07:49.574539+0000","last_unstale":"2023-10-06T03:07:50.552787+0000","last_undegraded":"2023-10-06T03:07:50.552787+0000","last_fullsized":"2023-10-06T03:07:50.552787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:26.959832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577939+0000","last_change":"2023-10-06T03:07:50.572981+0000","last_active":"2023-10-06T03:07:50.577939+0000","last_peered":"2023-10-06T03:07:50.577939+0000","last_clean":"2023-10-06T03:07:50.577939+0000","last_became_active":"2023-10-06T03:07:50.572653+0000","last_became_peered":"2023-10-06T03:07:50.572653+0000","last_unstale":"2023-10-06T03:07:50.577939+0000","last_undegraded":"2023-10-06T03:07:50.577939+0000","last_fullsized":"2023-10-06T03:07:50.577939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:40.777609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552118+0000","last_change":"2023-10-06T03:07:49.563073+0000","last_active":"2023-10-06T03:07:50.552118+0000","last_peered":"2023-10-06T03:07:50.552118+0000","last_clean":"2023-10-06T03:07:50.552118+0000","last_became_active":"2023-10-06T03:07:49.562796+0000","last_became_peered":"2023-10-06T03:07:49.562796+0000","last_unstale":"2023-10-06T03:07:50.552118+0000","last_undegraded":"2023-10-06T03:07:50.552118+0000","last_fullsized":"2023-10-06T03:07:50.552118+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:32.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581581+0000","last_change":"2023-10-06T03:07:50.574361+0000","last_active":"2023-10-06T03:07:50.581581+0000","last_peered":"2023-10-06T03:07:50.581581+0000","last_clean":"2023-10-06T03:07:50.581581+0000","last_became_active":"2023-10-06T03:07:50.574196+0000","last_became_peered":"2023-10-06T03:07:50.574196+0000","last_unstale":"2023-10-06T03:07:50.581581+0000","last_undegraded":"2023-10-06T03:07:50.581581+0000","last_fullsized":"2023-10-06T03:07:50.581581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:02.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562144+0000","last_change":"2023-10-06T03:07:49.566063+0000","last_active":"2023-10-06T03:07:50.562144+0000","last_peered":"2023-10-06T03:07:50.562144+0000","last_clean":"2023-10-06T03:07:50.562144+0000","last_became_active":"2023-10-06T03:07:49.565801+0000","last_became_peered":"2023-10-06T03:07:49.565801+0000","last_unstale":"2023-10-06T03:07:50.562144+0000","last_undegraded":"2023-10-06T03:07:50.562144+0000","last_fullsized":"2023-10-06T03:07:50.562144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:53.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572719+0000","last_change":"2023-10-06T03:07:50.566817+0000","last_active":"2023-10-06T03:07:50.572719+0000","last_peered":"2023-10-06T03:07:50.572719+0000","last_clean":"2023-10-06T03:07:50.572719+0000","last_became_active":"2023-10-06T03:07:50.566535+0000","last_became_peered":"2023-10-06T03:07:50.566535+0000","last_unstale":"2023-10-06T03:07:50.572719+0000","last_undegraded":"2023-10-06T03:07:50.572719+0000","last_fullsized":"2023-10-06T03:07:50.572719+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:27.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"102'6","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.589060+0000","last_change":"2023-10-06T03:07:49.562916+0000","last_active":"2023-10-06T03:07:50.589060+0000","last_peered":"2023-10-06T03:07:50.589060+0000","last_clean":"2023-10-06T03:07:50.589060+0000","last_became_active":"2023-10-06T03:07:49.562661+0000","last_became_peered":"2023-10-06T03:07:49.562661+0000","last_unstale":"2023-10-06T03:07:50.589060+0000","last_undegraded":"2023-10-06T03:07:50.589060+0000","last_fullsized":"2023-10-06T03:07:50.589060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:14.508655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.556557+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:50.556557+0000","last_peered":"2023-10-06T03:07:50.556557+0000","last_clean":"2023-10-06T03:07:50.556557+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:50.556557+0000","last_undegraded":"2023-10-06T03:07:50.556557+0000","last_fullsized":"2023-10-06T03:07:50.556557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584647+0000","last_change":"2023-10-06T03:07:50.578854+0000","last_active":"2023-10-06T03:07:50.584647+0000","last_peered":"2023-10-06T03:07:50.584647+0000","last_clean":"2023-10-06T03:07:50.584647+0000","last_became_active":"2023-10-06T03:07:50.578290+0000","last_became_peered":"2023-10-06T03:07:50.578290+0000","last_unstale":"2023-10-06T03:07:50.584647+0000","last_undegraded":"2023-10-06T03:07:50.584647+0000","last_fullsized":"2023-10-06T03:07:50.584647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:50.013969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"102'3","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575195+0000","last_change":"2023-10-06T03:07:49.570381+0000","last_active":"2023-10-06T03:07:50.575195+0000","last_peered":"2023-10-06T03:07:50.575195+0000","last_clean":"2023-10-06T03:07:50.575195+0000","last_became_active":"2023-10-06T03:07:49.570123+0000","last_became_peered":"2023-10-06T03:07:49.570123+0000","last_unstale":"2023-10-06T03:07:50.575195+0000","last_undegraded":"2023-10-06T03:07:50.575195+0000","last_fullsized":"2023-10-06T03:07:50.575195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.564578+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:50.564578+0000","last_peered":"2023-10-06T03:07:50.564578+0000","last_clean":"2023-10-06T03:07:50.564578+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:50.564578+0000","last_undegraded":"2023-10-06T03:07:50.564578+0000","last_fullsized":"2023-10-06T03:07:50.564578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585231+0000","last_change":"2023-10-06T03:07:50.582099+0000","last_active":"2023-10-06T03:07:50.585231+0000","last_peered":"2023-10-06T03:07:50.585231+0000","last_clean":"2023-10-06T03:07:50.585231+0000","last_became_active":"2023-10-06T03:07:50.581860+0000","last_became_peered":"2023-10-06T03:07:50.581860+0000","last_unstale":"2023-10-06T03:07:50.585231+0000","last_undegraded":"2023-10-06T03:07:50.585231+0000","last_fullsized":"2023-10-06T03:07:50.585231+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:50.013969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.553486+0000","last_change":"2023-10-06T03:07:49.574358+0000","last_active":"2023-10-06T03:07:50.553486+0000","last_peered":"2023-10-06T03:07:50.553486+0000","last_clean":"2023-10-06T03:07:50.553486+0000","last_became_active":"2023-10-06T03:07:49.574001+0000","last_became_peered":"2023-10-06T03:07:49.574001+0000","last_unstale":"2023-10-06T03:07:50.553486+0000","last_undegraded":"2023-10-06T03:07:50.553486+0000","last_fullsized":"2023-10-06T03:07:50.553486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:14.508655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567674+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:50.567674+0000","last_peered":"2023-10-06T03:07:50.567674+0000","last_clean":"2023-10-06T03:07:50.567674+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:50.567674+0000","last_undegraded":"2023-10-06T03:07:50.567674+0000","last_fullsized":"2023-10-06T03:07:50.567674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585304+0000","last_change":"2023-10-06T03:07:50.581005+0000","last_active":"2023-10-06T03:07:50.585304+0000","last_peered":"2023-10-06T03:07:50.585304+0000","last_clean":"2023-10-06T03:07:50.585304+0000","last_became_active":"2023-10-06T03:07:50.580494+0000","last_became_peered":"2023-10-06T03:07:50.580494+0000","last_unstale":"2023-10-06T03:07:50.585304+0000","last_undegraded":"2023-10-06T03:07:50.585304+0000","last_fullsized":"2023-10-06T03:07:50.585304+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:57.318520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584485+0000","last_change":"2023-10-06T03:07:49.566406+0000","last_active":"2023-10-06T03:07:50.584485+0000","last_peered":"2023-10-06T03:07:50.584485+0000","last_clean":"2023-10-06T03:07:50.584485+0000","last_became_active":"2023-10-06T03:07:49.566148+0000","last_became_peered":"2023-10-06T03:07:49.566148+0000","last_unstale":"2023-10-06T03:07:50.584485+0000","last_undegraded":"2023-10-06T03:07:50.584485+0000","last_fullsized":"2023-10-06T03:07:50.584485+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:57.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562238+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:50.562238+0000","last_peered":"2023-10-06T03:07:50.562238+0000","last_clean":"2023-10-06T03:07:50.562238+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:50.562238+0000","last_undegraded":"2023-10-06T03:07:50.562238+0000","last_fullsized":"2023-10-06T03:07:50.562238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.578008+0000","last_change":"2023-10-06T03:07:50.574780+0000","last_active":"2023-10-06T03:07:50.578008+0000","last_peered":"2023-10-06T03:07:50.578008+0000","last_clean":"2023-10-06T03:07:50.578008+0000","last_became_active":"2023-10-06T03:07:50.574322+0000","last_became_peered":"2023-10-06T03:07:50.574322+0000","last_unstale":"2023-10-06T03:07:50.578008+0000","last_undegraded":"2023-10-06T03:07:50.578008+0000","last_fullsized":"2023-10-06T03:07:50.578008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:02.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"102'8","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587760+0000","last_change":"2023-10-06T03:07:49.562128+0000","last_active":"2023-10-06T03:07:50.587760+0000","last_peered":"2023-10-06T03:07:50.587760+0000","last_clean":"2023-10-06T03:07:50.587760+0000","last_became_active":"2023-10-06T03:07:49.561879+0000","last_became_peered":"2023-10-06T03:07:49.561879+0000","last_unstale":"2023-10-06T03:07:50.587760+0000","last_undegraded":"2023-10-06T03:07:50.587760+0000","last_fullsized":"2023-10-06T03:07:50.587760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:56.313797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562418+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:50.562418+0000","last_peered":"2023-10-06T03:07:50.562418+0000","last_clean":"2023-10-06T03:07:50.562418+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:50.562418+0000","last_undegraded":"2023-10-06T03:07:50.562418+0000","last_fullsized":"2023-10-06T03:07:50.562418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552654+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:50.552654+0000","last_peered":"2023-10-06T03:07:50.552654+0000","last_clean":"2023-10-06T03:07:50.552654+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:50.552654+0000","last_undegraded":"2023-10-06T03:07:50.552654+0000","last_fullsized":"2023-10-06T03:07:50.552654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581699+0000","last_change":"2023-10-06T03:07:50.573994+0000","last_active":"2023-10-06T03:07:50.581699+0000","last_peered":"2023-10-06T03:07:50.581699+0000","last_clean":"2023-10-06T03:07:50.581699+0000","last_became_active":"2023-10-06T03:07:50.573801+0000","last_became_peered":"2023-10-06T03:07:50.573801+0000","last_unstale":"2023-10-06T03:07:50.581699+0000","last_undegraded":"2023-10-06T03:07:50.581699+0000","last_fullsized":"2023-10-06T03:07:50.581699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:22.069683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571236+0000","last_change":"2023-10-06T03:07:49.574564+0000","last_active":"2023-10-06T03:07:50.571236+0000","last_peered":"2023-10-06T03:07:50.571236+0000","last_clean":"2023-10-06T03:07:50.571236+0000","last_became_active":"2023-10-06T03:07:49.574290+0000","last_became_peered":"2023-10-06T03:07:49.574290+0000","last_unstale":"2023-10-06T03:07:50.571236+0000","last_undegraded":"2023-10-06T03:07:50.571236+0000","last_fullsized":"2023-10-06T03:07:50.571236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562557+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:50.562557+0000","last_peered":"2023-10-06T03:07:50.562557+0000","last_clean":"2023-10-06T03:07:50.562557+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:50.562557+0000","last_undegraded":"2023-10-06T03:07:50.562557+0000","last_fullsized":"2023-10-06T03:07:50.562557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573132+0000","last_change":"2023-10-06T03:07:50.568705+0000","last_active":"2023-10-06T03:07:50.573132+0000","last_peered":"2023-10-06T03:07:50.573132+0000","last_clean":"2023-10-06T03:07:50.573132+0000","last_became_active":"2023-10-06T03:07:50.568454+0000","last_became_peered":"2023-10-06T03:07:50.568454+0000","last_unstale":"2023-10-06T03:07:50.573132+0000","last_undegraded":"2023-10-06T03:07:50.573132+0000","last_fullsized":"2023-10-06T03:07:50.573132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:49.459109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570260+0000","last_change":"2023-10-06T03:07:49.563167+0000","last_active":"2023-10-06T03:07:50.570260+0000","last_peered":"2023-10-06T03:07:50.570260+0000","last_clean":"2023-10-06T03:07:50.570260+0000","last_became_active":"2023-10-06T03:07:49.562680+0000","last_became_peered":"2023-10-06T03:07:49.562680+0000","last_unstale":"2023-10-06T03:07:50.570260+0000","last_undegraded":"2023-10-06T03:07:50.570260+0000","last_fullsized":"2023-10-06T03:07:50.570260+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:01.616622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.554084+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:50.554084+0000","last_peered":"2023-10-06T03:07:50.554084+0000","last_clean":"2023-10-06T03:07:50.554084+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:50.554084+0000","last_undegraded":"2023-10-06T03:07:50.554084+0000","last_fullsized":"2023-10-06T03:07:50.554084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571441+0000","last_change":"2023-10-06T03:07:50.565348+0000","last_active":"2023-10-06T03:07:50.571441+0000","last_peered":"2023-10-06T03:07:50.571441+0000","last_clean":"2023-10-06T03:07:50.571441+0000","last_became_active":"2023-10-06T03:07:50.565091+0000","last_became_peered":"2023-10-06T03:07:50.565091+0000","last_unstale":"2023-10-06T03:07:50.571441+0000","last_undegraded":"2023-10-06T03:07:50.571441+0000","last_fullsized":"2023-10-06T03:07:50.571441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:02.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.556502+0000","last_change":"2023-10-06T03:07:49.568754+0000","last_active":"2023-10-06T03:07:50.556502+0000","last_peered":"2023-10-06T03:07:50.556502+0000","last_clean":"2023-10-06T03:07:50.556502+0000","last_became_active":"2023-10-06T03:07:49.568471+0000","last_became_peered":"2023-10-06T03:07:49.568471+0000","last_unstale":"2023-10-06T03:07:50.556502+0000","last_undegraded":"2023-10-06T03:07:50.556502+0000","last_fullsized":"2023-10-06T03:07:50.556502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.565111+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:50.565111+0000","last_peered":"2023-10-06T03:07:50.565111+0000","last_clean":"2023-10-06T03:07:50.565111+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:50.565111+0000","last_undegraded":"2023-10-06T03:07:50.565111+0000","last_fullsized":"2023-10-06T03:07:50.565111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303232,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068608,"statfs":{"total":96636764160,"available":96326254592,"internally_reserved":0,"allocated":1892352,"data_stored":1177182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302640,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069200,"statfs":{"total":96636764160,"available":96326860800,"internally_reserved":0,"allocated":1286144,"data_stored":586620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.499},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.441}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.59099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":586088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.308}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.72899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303220,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068620,"statfs":{"total":96636764160,"available":96326266880,"internally_reserved":0,"allocated":1880064,"data_stored":1176456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.371},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.308},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.44},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.42799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.52900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1282048,"data_stored":586066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28999999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.309},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.628}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298556,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073284,"statfs":{"total":96636764160,"available":96331042816,"internally_reserved":0,"allocated":1298432,"data_stored":587636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.307}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.621},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.5}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.60999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302648,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069192,"statfs":{"total":96636764160,"available":96326852608,"internally_reserved":0,"allocated":1294336,"data_stored":587602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.80600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073292,"statfs":{"total":96636764160,"available":96331051008,"internally_reserved":0,"allocated":1290240,"data_stored":586226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.435},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:55.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T03:07:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.479+0000 7f015b029700 1 Processor -- start 2023-10-06T03:07:55.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.479+0000 7f015b029700 1 -- start start 2023-10-06T03:07:55.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f015b029700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0154159ec0 0x7f01540b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f015b029700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01540cf350 0x7f01540cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f015b029700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01540b5b00 con 0x7f01540b53d0 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f015b029700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01540b5850 con 0x7f0154159ec0 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f015b029700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01540b5da0 con 0x7f01540cf350 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f0158dc5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0154159ec0 0x7f01540b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f0158dc5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0154159ec0 0x7f01540b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50760/0 (socket says 172.21.15.92:50760) 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.480+0000 7f0158dc5700 1 -- 172.21.15.92:0/383186608 learned_addr learned my addr 172.21.15.92:0/383186608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f01595c6700 1 --2- 172.21.15.92:0/383186608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01540cf350 0x7f01540cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f0158dc5700 1 -- 172.21.15.92:0/383186608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01540cf350 msgr2=0x7f01540cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f0158dc5700 1 --2- 172.21.15.92:0/383186608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01540cf350 0x7f01540cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f0158dc5700 1 -- 172.21.15.92:0/383186608 >> v1:172.21.15.136:6789/0 conn(0x7f01540b53d0 legacy=0x7f01540cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f0158dc5700 1 -- 172.21.15.92:0/383186608 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540b6080 con 0x7f0154159ec0 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f0158dc5700 1 --2- 172.21.15.92:0/383186608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0154159ec0 0x7f01540b4e90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0150009f70 tx=0x7f0150009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4609ea4ed1c534a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f014b7fe700 1 -- 172.21.15.92:0/383186608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015000e070 con 0x7f0154159ec0 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f014b7fe700 1 -- 172.21.15.92:0/383186608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015000ba90 con 0x7f0154159ec0 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f014b7fe700 1 -- 172.21.15.92:0/383186608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500092e0 con 0x7f0154159ec0 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 -- 172.21.15.92:0/383186608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0154159ec0 msgr2=0x7f01540b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 --2- 172.21.15.92:0/383186608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0154159ec0 0x7f01540b4e90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0150009f70 tx=0x7f0150009840 comp rx=0 tx=0).stop 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 -- 172.21.15.92:0/383186608 shutdown_connections 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 --2- 172.21.15.92:0/383186608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0154159ec0 0x7f01540b4e90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 --2- 172.21.15.92:0/383186608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01540cf350 0x7f01540cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 -- 172.21.15.92:0/383186608 >> 172.21.15.92:0/383186608 conn(0x7f01540999f0 msgr2=0x7f01540b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:55.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 -- 172.21.15.92:0/383186608 shutdown_connections 2023-10-06T03:07:55.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.481+0000 7f015b029700 1 -- 172.21.15.92:0/383186608 wait complete. 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.482+0000 7f015b029700 1 Processor -- start 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.483+0000 7f015b029700 1 -- start start 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.483+0000 7f015b029700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 0x7f015406a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.483+0000 7f015b029700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01540cf350 0x7f01540c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.483+0000 7f0158dc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 0x7f015406a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 0x7f015406a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50770/0 (socket says 172.21.15.92:50770) 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f015b029700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0154159ec0 0x7f015416b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 -- 172.21.15.92:0/3528437140 learned_addr learned my addr 172.21.15.92:0/3528437140 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015406a8c0 con 0x7f01540b53d0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540c87a0 con 0x7f01540cf350 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01540c9ea0 con 0x7f0154159ec0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f014bfff700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01540cf350 0x7f01540c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f01595c6700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0154159ec0 0x7f015416b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 -- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0154159ec0 msgr2=0x7f015416b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0154159ec0 0x7f015416b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 -- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01540cf350 msgr2=0x7f01540c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01540cf350 0x7f01540c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150009ae0 con 0x7f01540b53d0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0158dc5700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 0x7f015406a380 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0154089390 tx=0x7f0150017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500129d0 con 0x7f01540b53d0 2023-10-06T03:07:55.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150012b30 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01540c7c20 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015001d420 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.484+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01540c81e0 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.485+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0154063c90 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.486+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f015001d850 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.486+0000 7f0148ff9700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0140067290 0x7f0140069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.486+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0150086500 con 0x7f01540b53d0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.486+0000 7f014bfff700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0140067290 0x7f0140069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.486+0000 7f014bfff700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0140067290 0x7f0140069740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01540b28e0 tx=0x7f0144009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.490+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0150052ff0 con 0x7f01540b53d0 2023-10-06T03:07:55.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.754+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f015404c3c0 con 0x7f01540b53d0 2023-10-06T03:07:55.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.755+0000 7f0148ff9700 1 -- 172.21.15.92:0/3528437140 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1437 (secure 0 0 0) 0x7f0150056610 con 0x7f01540b53d0 2023-10-06T03:07:55.757 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:55.757 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":398,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1696561288,"num_in_osds":8,"osd_in_since":1696561280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592674,"bytes_used":2463670272,"bytes_avail":770630443008,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":68,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":6840}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-10-06T03:07:33.254260+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T03:07:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0140067290 msgr2=0x7f0140069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0140067290 0x7f0140069740 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01540b28e0 tx=0x7f0144009060 comp rx=0 tx=0).stop 2023-10-06T03:07:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 msgr2=0x7f015406a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 0x7f015406a380 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0154089390 tx=0x7f0150017000 comp rx=0 tx=0).stop 2023-10-06T03:07:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 shutdown_connections 2023-10-06T03:07:55.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0140067290 0x7f0140069740 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01540b53d0 0x7f015406a380 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0154159ec0 0x7f015416b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 --2- 172.21.15.92:0/3528437140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01540cf350 0x7f01540c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 >> 172.21.15.92:0/3528437140 conn(0x7f01540999f0 msgr2=0x7f01541559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 shutdown_connections 2023-10-06T03:07:55.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.757+0000 7f015b029700 1 -- 172.21.15.92:0/3528437140 wait complete. 2023-10-06T03:07:55.771 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T03:07:55.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:55.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.884+0000 7f569d365700 1 Processor -- start 2023-10-06T03:07:55.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.884+0000 7f569d365700 1 -- start start 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f569d365700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5698159c60 0x7f569815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f569d365700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f569815db10 0x7f5698069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f569d365700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569815a160 con 0x7f5698159890 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f569d365700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569815df50 con 0x7f569815db10 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f569d365700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56980d13a0 con 0x7f5698159c60 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56977fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f569815db10 0x7f5698069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f5696ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5698159890 0x7f569815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47022/0 (socket says 172.21.15.92:47022) 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f5696ffd700 1 -- 172.21.15.92:0/2886510445 learned_addr learned my addr 172.21.15.92:0/2886510445 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:55.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56977fe700 1 -- 172.21.15.92:0/2886510445 >> v2:172.21.15.136:3300/0 conn(0x7f5698159c60 msgr2=0x7f569815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56967fc700 1 --2- 172.21.15.92:0/2886510445 >> v2:172.21.15.136:3300/0 conn(0x7f5698159c60 0x7f569815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56977fe700 1 --2- 172.21.15.92:0/2886510445 >> v2:172.21.15.136:3300/0 conn(0x7f5698159c60 0x7f569815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f5695ffb700 1 -- 172.21.15.92:0/2886510445 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607907418 0 0) 0x7f569815a160 con 0x7f5698159890 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56977fe700 1 -- 172.21.15.92:0/2886510445 >> v1:172.21.15.92:6789/0 conn(0x7f5698159890 legacy=0x7f569815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56977fe700 1 -- 172.21.15.92:0/2886510445 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569815e140 con 0x7f569815db10 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f56977fe700 1 --2- 172.21.15.92:0/2886510445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698069560 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f56980b67f0 tx=0x7f5690009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e522636cd73c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f5695ffb700 1 -- 172.21.15.92:0/2886510445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569000e070 con 0x7f569815db10 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f5695ffb700 1 -- 172.21.15.92:0/2886510445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569000bc30 con 0x7f569815db10 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.885+0000 7f5695ffb700 1 -- 172.21.15.92:0/2886510445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690012430 con 0x7f569815db10 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 -- 172.21.15.92:0/2886510445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 msgr2=0x7f5698069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 --2- 172.21.15.92:0/2886510445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698069560 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f56980b67f0 tx=0x7f5690009780 comp rx=0 tx=0).stop 2023-10-06T03:07:55.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 -- 172.21.15.92:0/2886510445 shutdown_connections 2023-10-06T03:07:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 --2- 172.21.15.92:0/2886510445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698069560 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 --2- 172.21.15.92:0/2886510445 >> v2:172.21.15.136:3300/0 conn(0x7f5698159c60 0x7f569815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 -- 172.21.15.92:0/2886510445 >> 172.21.15.92:0/2886510445 conn(0x7f5698068e20 msgr2=0x7f56980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 -- 172.21.15.92:0/2886510445 shutdown_connections 2023-10-06T03:07:55.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.886+0000 7f569d365700 1 -- 172.21.15.92:0/2886510445 wait complete. 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.887+0000 7f569d365700 1 Processor -- start 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.887+0000 7f569d365700 1 -- start start 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5698159890 0x7f56980cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698159c60 0x7f56980cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56980b0540 con 0x7f569815db10 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569816d490 con 0x7f5698159c60 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569816d770 con 0x7f5698159890 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50792/0 (socket says 172.21.15.92:50792) 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 -- 172.21.15.92:0/1554495945 learned_addr learned my addr 172.21.15.92:0/1554495945 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56967fc700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698159c60 0x7f56980cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 -- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5698159890 msgr2=0x7f56980cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5698159890 0x7f56980cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 -- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698159c60 msgr2=0x7f56980cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698159c60 0x7f56980cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 -- 172.21.15.92:0/1554495945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690009a20 con 0x7f569815db10 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f56977fe700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698148e00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5698070e20 tx=0x7f5690019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569001d630 con 0x7f569815db10 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569001d790 con 0x7f569815db10 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569001db50 con 0x7f569815db10 2023-10-06T03:07:55.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.888+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569816da50 con 0x7f569815db10 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.889+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56980cfce0 con 0x7f569815db10 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.889+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56980b06f0 con 0x7f569815db10 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.890+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5690024070 con 0x7f569815db10 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.890+0000 7f567effd700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5684067290 0x7f5684069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.890+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5690087500 con 0x7f569815db10 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.890+0000 7f5696ffd700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5684067290 0x7f5684069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:55.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.891+0000 7f5696ffd700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5684067290 0x7f5684069740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f56880011e0 tx=0x7f56880079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:55.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:55.895+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5690054200 con 0x7f569815db10 2023-10-06T03:07:56.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.073+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56980cff00 con 0x7f5684067290 2023-10-06T03:07:56.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.081+0000 7f567effd700 1 -- 172.21.15.92:0/1554495945 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312149 (secure 0 0 0) 0x7f56980cff00 con 0x7f5684067290 2023-10-06T03:07:56.087 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:56.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5684067290 msgr2=0x7f5684069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:56.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5684067290 0x7f5684069740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f56880011e0 tx=0x7f56880079d0 comp rx=0 tx=0).stop 2023-10-06T03:07:56.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 msgr2=0x7f5698148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:56.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698148e00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f5698070e20 tx=0x7f5690019000 comp rx=0 tx=0).stop 2023-10-06T03:07:56.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 shutdown_connections 2023-10-06T03:07:56.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5684067290 0x7f5684069740 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f569815db10 0x7f5698148e00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5698159890 0x7f56980cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 --2- 172.21.15.92:0/1554495945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698159c60 0x7f56980cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 >> 172.21.15.92:0/1554495945 conn(0x7f5698068e20 msgr2=0x7f56981552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 shutdown_connections 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:56.085+0000 7f569d365700 1 -- 172.21.15.92:0/1554495945 wait complete. 2023-10-06T03:07:56.090 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:56.099 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2023-10-06T03:07:55.258129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405928,"kb_used_data":11240,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752568792,"statfs":{"total":773094113280,"available":770630443008,"internally_reserved":0,"allocated":11509760,"data_stored":5873876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001037"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.578037+0000","last_change":"2023-10-06T03:07:50.572362+0000","last_active":"2023-10-06T03:07:50.578037+0000","last_peered":"2023-10-06T03:07:50.578037+0000","last_clean":"2023-10-06T03:07:50.578037+0000","last_became_active":"2023-10-06T03:07:50.571731+0000","last_became_peered":"2023-10-06T03:07:50.571731+0000","last_unstale":"2023-10-06T03:07:50.578037+0000","last_undegraded":"2023-10-06T03:07:50.578037+0000","last_fullsized":"2023-10-06T03:07:50.578037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:40.777609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584493+0000","last_change":"2023-10-06T03:07:50.576225+0000","last_active":"2023-10-06T03:07:50.584493+0000","last_peered":"2023-10-06T03:07:50.584493+0000","last_clean":"2023-10-06T03:07:50.584493+0000","last_became_active":"2023-10-06T03:07:50.575924+0000","last_became_peered":"2023-10-06T03:07:50.575924+0000","last_unstale":"2023-10-06T03:07:50.584493+0000","last_undegraded":"2023-10-06T03:07:50.584493+0000","last_fullsized":"2023-10-06T03:07:50.584493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:56.990336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584610+0000","last_change":"2023-10-06T03:07:50.578507+0000","last_active":"2023-10-06T03:07:50.584610+0000","last_peered":"2023-10-06T03:07:50.584610+0000","last_clean":"2023-10-06T03:07:50.584610+0000","last_became_active":"2023-10-06T03:07:50.578265+0000","last_became_peered":"2023-10-06T03:07:50.578265+0000","last_unstale":"2023-10-06T03:07:50.584610+0000","last_undegraded":"2023-10-06T03:07:50.584610+0000","last_fullsized":"2023-10-06T03:07:50.584610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573221+0000","last_change":"2023-10-06T03:07:50.571328+0000","last_active":"2023-10-06T03:07:50.573221+0000","last_peered":"2023-10-06T03:07:50.573221+0000","last_clean":"2023-10-06T03:07:50.573221+0000","last_became_active":"2023-10-06T03:07:50.571025+0000","last_became_peered":"2023-10-06T03:07:50.571025+0000","last_unstale":"2023-10-06T03:07:50.573221+0000","last_undegraded":"2023-10-06T03:07:50.573221+0000","last_fullsized":"2023-10-06T03:07:50.573221+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:38.614125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.564726+0000","last_change":"2023-10-06T03:07:50.559179+0000","last_active":"2023-10-06T03:07:50.564726+0000","last_peered":"2023-10-06T03:07:50.564726+0000","last_clean":"2023-10-06T03:07:50.564726+0000","last_became_active":"2023-10-06T03:07:50.558881+0000","last_became_peered":"2023-10-06T03:07:50.558881+0000","last_unstale":"2023-10-06T03:07:50.564726+0000","last_undegraded":"2023-10-06T03:07:50.564726+0000","last_fullsized":"2023-10-06T03:07:50.564726+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:47.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579171+0000","last_change":"2023-10-06T03:07:50.570727+0000","last_active":"2023-10-06T03:07:50.579171+0000","last_peered":"2023-10-06T03:07:50.579171+0000","last_clean":"2023-10-06T03:07:50.579171+0000","last_became_active":"2023-10-06T03:07:50.570484+0000","last_became_peered":"2023-10-06T03:07:50.570484+0000","last_unstale":"2023-10-06T03:07:50.579171+0000","last_undegraded":"2023-10-06T03:07:50.579171+0000","last_fullsized":"2023-10-06T03:07:50.579171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:57.184437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569950+0000","last_change":"2023-10-06T03:07:50.563316+0000","last_active":"2023-10-06T03:07:50.569950+0000","last_peered":"2023-10-06T03:07:50.569950+0000","last_clean":"2023-10-06T03:07:50.569950+0000","last_became_active":"2023-10-06T03:07:50.563031+0000","last_became_peered":"2023-10-06T03:07:50.563031+0000","last_unstale":"2023-10-06T03:07:50.569950+0000","last_undegraded":"2023-10-06T03:07:50.569950+0000","last_fullsized":"2023-10-06T03:07:50.569950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:28.830391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584901+0000","last_change":"2023-10-06T03:07:50.580182+0000","last_active":"2023-10-06T03:07:50.584901+0000","last_peered":"2023-10-06T03:07:50.584901+0000","last_clean":"2023-10-06T03:07:50.584901+0000","last_became_active":"2023-10-06T03:07:50.580001+0000","last_became_peered":"2023-10-06T03:07:50.580001+0000","last_unstale":"2023-10-06T03:07:50.584901+0000","last_undegraded":"2023-10-06T03:07:50.584901+0000","last_fullsized":"2023-10-06T03:07:50.584901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:27.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582970+0000","last_change":"2023-10-06T03:07:50.577719+0000","last_active":"2023-10-06T03:07:50.582970+0000","last_peered":"2023-10-06T03:07:50.582970+0000","last_clean":"2023-10-06T03:07:50.582970+0000","last_became_active":"2023-10-06T03:07:50.577452+0000","last_became_peered":"2023-10-06T03:07:50.577452+0000","last_unstale":"2023-10-06T03:07:50.582970+0000","last_undegraded":"2023-10-06T03:07:50.582970+0000","last_fullsized":"2023-10-06T03:07:50.582970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:08.241579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582530+0000","last_change":"2023-10-06T03:07:50.571565+0000","last_active":"2023-10-06T03:07:50.582530+0000","last_peered":"2023-10-06T03:07:50.582530+0000","last_clean":"2023-10-06T03:07:50.582530+0000","last_became_active":"2023-10-06T03:07:50.570520+0000","last_became_peered":"2023-10-06T03:07:50.570520+0000","last_unstale":"2023-10-06T03:07:50.582530+0000","last_undegraded":"2023-10-06T03:07:50.582530+0000","last_fullsized":"2023-10-06T03:07:50.582530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:26.295707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567219+0000","last_change":"2023-10-06T03:07:50.562739+0000","last_active":"2023-10-06T03:07:50.567219+0000","last_peered":"2023-10-06T03:07:50.567219+0000","last_clean":"2023-10-06T03:07:50.567219+0000","last_became_active":"2023-10-06T03:07:50.561529+0000","last_became_peered":"2023-10-06T03:07:50.561529+0000","last_unstale":"2023-10-06T03:07:50.567219+0000","last_undegraded":"2023-10-06T03:07:50.567219+0000","last_fullsized":"2023-10-06T03:07:50.567219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:33.483532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579373+0000","last_change":"2023-10-06T03:07:50.570788+0000","last_active":"2023-10-06T03:07:50.579373+0000","last_peered":"2023-10-06T03:07:50.579373+0000","last_clean":"2023-10-06T03:07:50.579373+0000","last_became_active":"2023-10-06T03:07:50.570296+0000","last_became_peered":"2023-10-06T03:07:50.570296+0000","last_unstale":"2023-10-06T03:07:50.579373+0000","last_undegraded":"2023-10-06T03:07:50.579373+0000","last_fullsized":"2023-10-06T03:07:50.579373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:56.995621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581561+0000","last_change":"2023-10-06T03:07:50.572906+0000","last_active":"2023-10-06T03:07:50.581561+0000","last_peered":"2023-10-06T03:07:50.581561+0000","last_clean":"2023-10-06T03:07:50.581561+0000","last_became_active":"2023-10-06T03:07:50.572656+0000","last_became_peered":"2023-10-06T03:07:50.572656+0000","last_unstale":"2023-10-06T03:07:50.581561+0000","last_undegraded":"2023-10-06T03:07:50.581561+0000","last_fullsized":"2023-10-06T03:07:50.581561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:16.879548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584804+0000","last_change":"2023-10-06T03:07:50.580252+0000","last_active":"2023-10-06T03:07:50.584804+0000","last_peered":"2023-10-06T03:07:50.584804+0000","last_clean":"2023-10-06T03:07:50.584804+0000","last_became_active":"2023-10-06T03:07:50.578591+0000","last_became_peered":"2023-10-06T03:07:50.578591+0000","last_unstale":"2023-10-06T03:07:50.584804+0000","last_undegraded":"2023-10-06T03:07:50.584804+0000","last_fullsized":"2023-10-06T03:07:50.584804+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567088+0000","last_change":"2023-10-06T03:07:50.562853+0000","last_active":"2023-10-06T03:07:50.567088+0000","last_peered":"2023-10-06T03:07:50.567088+0000","last_clean":"2023-10-06T03:07:50.567088+0000","last_became_active":"2023-10-06T03:07:50.561839+0000","last_became_peered":"2023-10-06T03:07:50.561839+0000","last_unstale":"2023-10-06T03:07:50.567088+0000","last_undegraded":"2023-10-06T03:07:50.567088+0000","last_fullsized":"2023-10-06T03:07:50.567088+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:25.304588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573661+0000","last_change":"2023-10-06T03:07:50.567005+0000","last_active":"2023-10-06T03:07:50.573661+0000","last_peered":"2023-10-06T03:07:50.573661+0000","last_clean":"2023-10-06T03:07:50.573661+0000","last_became_active":"2023-10-06T03:07:50.566760+0000","last_became_peered":"2023-10-06T03:07:50.566760+0000","last_unstale":"2023-10-06T03:07:50.573661+0000","last_undegraded":"2023-10-06T03:07:50.573661+0000","last_fullsized":"2023-10-06T03:07:50.573661+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:21.538700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581519+0000","last_change":"2023-10-06T03:07:50.573244+0000","last_active":"2023-10-06T03:07:50.581519+0000","last_peered":"2023-10-06T03:07:50.581519+0000","last_clean":"2023-10-06T03:07:50.581519+0000","last_became_active":"2023-10-06T03:07:50.573001+0000","last_became_peered":"2023-10-06T03:07:50.573001+0000","last_unstale":"2023-10-06T03:07:50.581519+0000","last_undegraded":"2023-10-06T03:07:50.581519+0000","last_fullsized":"2023-10-06T03:07:50.581519+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:58.959529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584666+0000","last_change":"2023-10-06T03:07:50.579168+0000","last_active":"2023-10-06T03:07:50.584666+0000","last_peered":"2023-10-06T03:07:50.584666+0000","last_clean":"2023-10-06T03:07:50.584666+0000","last_became_active":"2023-10-06T03:07:50.577986+0000","last_became_peered":"2023-10-06T03:07:50.577986+0000","last_unstale":"2023-10-06T03:07:50.584666+0000","last_undegraded":"2023-10-06T03:07:50.584666+0000","last_fullsized":"2023-10-06T03:07:50.584666+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:30.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570414+0000","last_change":"2023-10-06T03:07:50.565479+0000","last_active":"2023-10-06T03:07:50.570414+0000","last_peered":"2023-10-06T03:07:50.570414+0000","last_clean":"2023-10-06T03:07:50.570414+0000","last_became_active":"2023-10-06T03:07:50.565172+0000","last_became_peered":"2023-10-06T03:07:50.565172+0000","last_unstale":"2023-10-06T03:07:50.570414+0000","last_undegraded":"2023-10-06T03:07:50.570414+0000","last_fullsized":"2023-10-06T03:07:50.570414+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:26.295707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576083+0000","last_change":"2023-10-06T03:07:50.573007+0000","last_active":"2023-10-06T03:07:50.576083+0000","last_peered":"2023-10-06T03:07:50.576083+0000","last_clean":"2023-10-06T03:07:50.576083+0000","last_became_active":"2023-10-06T03:07:50.572729+0000","last_became_peered":"2023-10-06T03:07:50.572729+0000","last_unstale":"2023-10-06T03:07:50.576083+0000","last_undegraded":"2023-10-06T03:07:50.576083+0000","last_fullsized":"2023-10-06T03:07:50.576083+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:45.959875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577990+0000","last_change":"2023-10-06T03:07:50.574602+0000","last_active":"2023-10-06T03:07:50.577990+0000","last_peered":"2023-10-06T03:07:50.577990+0000","last_clean":"2023-10-06T03:07:50.577990+0000","last_became_active":"2023-10-06T03:07:50.573933+0000","last_became_peered":"2023-10-06T03:07:50.573933+0000","last_unstale":"2023-10-06T03:07:50.577990+0000","last_undegraded":"2023-10-06T03:07:50.577990+0000","last_fullsized":"2023-10-06T03:07:50.577990+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.580385+0000","last_change":"2023-10-06T03:07:50.579726+0000","last_active":"2023-10-06T03:07:50.580385+0000","last_peered":"2023-10-06T03:07:50.580385+0000","last_clean":"2023-10-06T03:07:50.580385+0000","last_became_active":"2023-10-06T03:07:50.579482+0000","last_became_peered":"2023-10-06T03:07:50.579482+0000","last_unstale":"2023-10-06T03:07:50.580385+0000","last_undegraded":"2023-10-06T03:07:50.580385+0000","last_fullsized":"2023-10-06T03:07:50.580385+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:48.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587676+0000","last_change":"2023-10-06T03:07:50.581210+0000","last_active":"2023-10-06T03:07:50.587676+0000","last_peered":"2023-10-06T03:07:50.587676+0000","last_clean":"2023-10-06T03:07:50.587676+0000","last_became_active":"2023-10-06T03:07:50.580986+0000","last_became_peered":"2023-10-06T03:07:50.580986+0000","last_unstale":"2023-10-06T03:07:50.587676+0000","last_undegraded":"2023-10-06T03:07:50.587676+0000","last_fullsized":"2023-10-06T03:07:50.587676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:41.442852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569161+0000","last_change":"2023-10-06T03:07:50.561654+0000","last_active":"2023-10-06T03:07:50.569161+0000","last_peered":"2023-10-06T03:07:50.569161+0000","last_clean":"2023-10-06T03:07:50.569161+0000","last_became_active":"2023-10-06T03:07:50.561023+0000","last_became_peered":"2023-10-06T03:07:50.561023+0000","last_unstale":"2023-10-06T03:07:50.569161+0000","last_undegraded":"2023-10-06T03:07:50.569161+0000","last_fullsized":"2023-10-06T03:07:50.569161+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:44.578103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569197+0000","last_change":"2023-10-06T03:07:50.562602+0000","last_active":"2023-10-06T03:07:50.569197+0000","last_peered":"2023-10-06T03:07:50.569197+0000","last_clean":"2023-10-06T03:07:50.569197+0000","last_became_active":"2023-10-06T03:07:50.562303+0000","last_became_peered":"2023-10-06T03:07:50.562303+0000","last_unstale":"2023-10-06T03:07:50.569197+0000","last_undegraded":"2023-10-06T03:07:50.569197+0000","last_fullsized":"2023-10-06T03:07:50.569197+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:56.990336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575245+0000","last_change":"2023-10-06T03:07:50.569444+0000","last_active":"2023-10-06T03:07:50.575245+0000","last_peered":"2023-10-06T03:07:50.575245+0000","last_clean":"2023-10-06T03:07:50.575245+0000","last_became_active":"2023-10-06T03:07:50.569142+0000","last_became_peered":"2023-10-06T03:07:50.569142+0000","last_unstale":"2023-10-06T03:07:50.575245+0000","last_undegraded":"2023-10-06T03:07:50.575245+0000","last_fullsized":"2023-10-06T03:07:50.575245+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:18.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577568+0000","last_change":"2023-10-06T03:07:50.570913+0000","last_active":"2023-10-06T03:07:50.577568+0000","last_peered":"2023-10-06T03:07:50.577568+0000","last_clean":"2023-10-06T03:07:50.577568+0000","last_became_active":"2023-10-06T03:07:50.570641+0000","last_became_peered":"2023-10-06T03:07:50.570641+0000","last_unstale":"2023-10-06T03:07:50.577568+0000","last_undegraded":"2023-10-06T03:07:50.577568+0000","last_fullsized":"2023-10-06T03:07:50.577568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:22.069683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579352+0000","last_change":"2023-10-06T03:07:50.570632+0000","last_active":"2023-10-06T03:07:50.579352+0000","last_peered":"2023-10-06T03:07:50.579352+0000","last_clean":"2023-10-06T03:07:50.579352+0000","last_became_active":"2023-10-06T03:07:50.569983+0000","last_became_peered":"2023-10-06T03:07:50.569983+0000","last_unstale":"2023-10-06T03:07:50.579352+0000","last_undegraded":"2023-10-06T03:07:50.579352+0000","last_fullsized":"2023-10-06T03:07:50.579352+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.064356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582225+0000","last_change":"2023-10-06T03:07:50.577544+0000","last_active":"2023-10-06T03:07:50.582225+0000","last_peered":"2023-10-06T03:07:50.582225+0000","last_clean":"2023-10-06T03:07:50.582225+0000","last_became_active":"2023-10-06T03:07:50.577322+0000","last_became_peered":"2023-10-06T03:07:50.577322+0000","last_unstale":"2023-10-06T03:07:50.582225+0000","last_undegraded":"2023-10-06T03:07:50.582225+0000","last_fullsized":"2023-10-06T03:07:50.582225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:12.914946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585038+0000","last_change":"2023-10-06T03:07:50.581285+0000","last_active":"2023-10-06T03:07:50.585038+0000","last_peered":"2023-10-06T03:07:50.585038+0000","last_clean":"2023-10-06T03:07:50.585038+0000","last_became_active":"2023-10-06T03:07:50.581039+0000","last_became_peered":"2023-10-06T03:07:50.581039+0000","last_unstale":"2023-10-06T03:07:50.585038+0000","last_undegraded":"2023-10-06T03:07:50.585038+0000","last_fullsized":"2023-10-06T03:07:50.585038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:02.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574262+0000","last_change":"2023-10-06T03:07:50.566299+0000","last_active":"2023-10-06T03:07:50.574262+0000","last_peered":"2023-10-06T03:07:50.574262+0000","last_clean":"2023-10-06T03:07:50.574262+0000","last_became_active":"2023-10-06T03:07:50.566026+0000","last_became_peered":"2023-10-06T03:07:50.566026+0000","last_unstale":"2023-10-06T03:07:50.574262+0000","last_undegraded":"2023-10-06T03:07:50.574262+0000","last_fullsized":"2023-10-06T03:07:50.574262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:30.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569182+0000","last_change":"2023-10-06T03:07:50.561817+0000","last_active":"2023-10-06T03:07:50.569182+0000","last_peered":"2023-10-06T03:07:50.569182+0000","last_clean":"2023-10-06T03:07:50.569182+0000","last_became_active":"2023-10-06T03:07:50.561357+0000","last_became_peered":"2023-10-06T03:07:50.561357+0000","last_unstale":"2023-10-06T03:07:50.569182+0000","last_undegraded":"2023-10-06T03:07:50.569182+0000","last_fullsized":"2023-10-06T03:07:50.569182+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:26.301309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581912+0000","last_change":"2023-10-06T03:07:50.577842+0000","last_active":"2023-10-06T03:07:50.581912+0000","last_peered":"2023-10-06T03:07:50.581912+0000","last_clean":"2023-10-06T03:07:50.581912+0000","last_became_active":"2023-10-06T03:07:50.577581+0000","last_became_peered":"2023-10-06T03:07:50.577581+0000","last_unstale":"2023-10-06T03:07:50.581912+0000","last_undegraded":"2023-10-06T03:07:50.581912+0000","last_fullsized":"2023-10-06T03:07:50.581912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:18.485934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"102'4","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572923+0000","last_change":"2023-10-06T03:07:49.568419+0000","last_active":"2023-10-06T03:07:50.572923+0000","last_peered":"2023-10-06T03:07:50.572923+0000","last_clean":"2023-10-06T03:07:50.572923+0000","last_became_active":"2023-10-06T03:07:49.568113+0000","last_became_peered":"2023-10-06T03:07:49.568113+0000","last_unstale":"2023-10-06T03:07:50.572923+0000","last_undegraded":"2023-10-06T03:07:50.572923+0000","last_fullsized":"2023-10-06T03:07:50.572923+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584422+0000","last_change":"2023-10-06T03:07:50.578986+0000","last_active":"2023-10-06T03:07:50.584422+0000","last_peered":"2023-10-06T03:07:50.584422+0000","last_clean":"2023-10-06T03:07:50.584422+0000","last_became_active":"2023-10-06T03:07:50.578662+0000","last_became_peered":"2023-10-06T03:07:50.578662+0000","last_unstale":"2023-10-06T03:07:50.584422+0000","last_undegraded":"2023-10-06T03:07:50.584422+0000","last_fullsized":"2023-10-06T03:07:50.584422+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:13.437927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573465+0000","last_change":"2023-10-06T03:07:49.568589+0000","last_active":"2023-10-06T03:07:50.573465+0000","last_peered":"2023-10-06T03:07:50.573465+0000","last_clean":"2023-10-06T03:07:50.573465+0000","last_became_active":"2023-10-06T03:07:49.568313+0000","last_became_peered":"2023-10-06T03:07:49.568313+0000","last_unstale":"2023-10-06T03:07:50.573465+0000","last_undegraded":"2023-10-06T03:07:50.573465+0000","last_fullsized":"2023-10-06T03:07:50.573465+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:20.278853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572529+0000","last_change":"2023-10-06T03:07:50.567007+0000","last_active":"2023-10-06T03:07:50.572529+0000","last_peered":"2023-10-06T03:07:50.572529+0000","last_clean":"2023-10-06T03:07:50.572529+0000","last_became_active":"2023-10-06T03:07:50.566733+0000","last_became_peered":"2023-10-06T03:07:50.566733+0000","last_unstale":"2023-10-06T03:07:50.572529+0000","last_undegraded":"2023-10-06T03:07:50.572529+0000","last_fullsized":"2023-10-06T03:07:50.572529+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:54.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573062+0000","last_change":"2023-10-06T03:07:49.568215+0000","last_active":"2023-10-06T03:07:50.573062+0000","last_peered":"2023-10-06T03:07:50.573062+0000","last_clean":"2023-10-06T03:07:50.573062+0000","last_became_active":"2023-10-06T03:07:49.567900+0000","last_became_peered":"2023-10-06T03:07:49.567900+0000","last_unstale":"2023-10-06T03:07:50.573062+0000","last_undegraded":"2023-10-06T03:07:50.573062+0000","last_fullsized":"2023-10-06T03:07:50.573062+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:25.296585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577913+0000","last_change":"2023-10-06T03:07:50.573487+0000","last_active":"2023-10-06T03:07:50.577913+0000","last_peered":"2023-10-06T03:07:50.577913+0000","last_clean":"2023-10-06T03:07:50.577913+0000","last_became_active":"2023-10-06T03:07:50.573233+0000","last_became_peered":"2023-10-06T03:07:50.573233+0000","last_unstale":"2023-10-06T03:07:50.577913+0000","last_undegraded":"2023-10-06T03:07:50.577913+0000","last_fullsized":"2023-10-06T03:07:50.577913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:18:39.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582880+0000","last_change":"2023-10-06T03:07:49.561326+0000","last_active":"2023-10-06T03:07:50.582880+0000","last_peered":"2023-10-06T03:07:50.582880+0000","last_clean":"2023-10-06T03:07:50.582880+0000","last_became_active":"2023-10-06T03:07:49.561103+0000","last_became_peered":"2023-10-06T03:07:49.561103+0000","last_unstale":"2023-10-06T03:07:50.582880+0000","last_undegraded":"2023-10-06T03:07:50.582880+0000","last_fullsized":"2023-10-06T03:07:50.582880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:25.765674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576162+0000","last_change":"2023-10-06T03:07:50.571130+0000","last_active":"2023-10-06T03:07:50.576162+0000","last_peered":"2023-10-06T03:07:50.576162+0000","last_clean":"2023-10-06T03:07:50.576162+0000","last_became_active":"2023-10-06T03:07:50.570902+0000","last_became_peered":"2023-10-06T03:07:50.570902+0000","last_unstale":"2023-10-06T03:07:50.576162+0000","last_undegraded":"2023-10-06T03:07:50.576162+0000","last_fullsized":"2023-10-06T03:07:50.576162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:22.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572082+0000","last_change":"2023-10-06T03:07:49.564406+0000","last_active":"2023-10-06T03:07:50.572082+0000","last_peered":"2023-10-06T03:07:50.572082+0000","last_clean":"2023-10-06T03:07:50.572082+0000","last_became_active":"2023-10-06T03:07:49.564128+0000","last_became_peered":"2023-10-06T03:07:49.564128+0000","last_unstale":"2023-10-06T03:07:50.572082+0000","last_undegraded":"2023-10-06T03:07:50.572082+0000","last_fullsized":"2023-10-06T03:07:50.572082+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:49.009245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.568929+0000","last_change":"2023-10-06T03:07:50.565122+0000","last_active":"2023-10-06T03:07:50.568929+0000","last_peered":"2023-10-06T03:07:50.568929+0000","last_clean":"2023-10-06T03:07:50.568929+0000","last_became_active":"2023-10-06T03:07:50.564842+0000","last_became_peered":"2023-10-06T03:07:50.564842+0000","last_unstale":"2023-10-06T03:07:50.568929+0000","last_undegraded":"2023-10-06T03:07:50.568929+0000","last_fullsized":"2023-10-06T03:07:50.568929+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:20.294921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579362+0000","last_change":"2023-10-06T03:07:49.564123+0000","last_active":"2023-10-06T03:07:50.579362+0000","last_peered":"2023-10-06T03:07:50.579362+0000","last_clean":"2023-10-06T03:07:50.579362+0000","last_became_active":"2023-10-06T03:07:49.563157+0000","last_became_peered":"2023-10-06T03:07:49.563157+0000","last_unstale":"2023-10-06T03:07:50.579362+0000","last_undegraded":"2023-10-06T03:07:50.579362+0000","last_fullsized":"2023-10-06T03:07:50.579362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:51.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584725+0000","last_change":"2023-10-06T03:07:50.578869+0000","last_active":"2023-10-06T03:07:50.584725+0000","last_peered":"2023-10-06T03:07:50.584725+0000","last_clean":"2023-10-06T03:07:50.584725+0000","last_became_active":"2023-10-06T03:07:50.578111+0000","last_became_peered":"2023-10-06T03:07:50.578111+0000","last_unstale":"2023-10-06T03:07:50.584725+0000","last_undegraded":"2023-10-06T03:07:50.584725+0000","last_fullsized":"2023-10-06T03:07:50.584725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:18:39.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.554122+0000","last_change":"2023-10-06T03:07:49.570378+0000","last_active":"2023-10-06T03:07:50.554122+0000","last_peered":"2023-10-06T03:07:50.554122+0000","last_clean":"2023-10-06T03:07:50.554122+0000","last_became_active":"2023-10-06T03:07:49.570112+0000","last_became_peered":"2023-10-06T03:07:49.570112+0000","last_unstale":"2023-10-06T03:07:50.554122+0000","last_undegraded":"2023-10-06T03:07:50.554122+0000","last_fullsized":"2023-10-06T03:07:50.554122+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:21.074952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581873+0000","last_change":"2023-10-06T03:07:50.577006+0000","last_active":"2023-10-06T03:07:50.581873+0000","last_peered":"2023-10-06T03:07:50.581873+0000","last_clean":"2023-10-06T03:07:50.581873+0000","last_became_active":"2023-10-06T03:07:50.576746+0000","last_became_peered":"2023-10-06T03:07:50.576746+0000","last_unstale":"2023-10-06T03:07:50.581873+0000","last_undegraded":"2023-10-06T03:07:50.581873+0000","last_fullsized":"2023-10-06T03:07:50.581873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:58.698362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552740+0000","last_change":"2023-10-06T03:07:49.563228+0000","last_active":"2023-10-06T03:07:50.552740+0000","last_peered":"2023-10-06T03:07:50.552740+0000","last_clean":"2023-10-06T03:07:50.552740+0000","last_became_active":"2023-10-06T03:07:49.562933+0000","last_became_peered":"2023-10-06T03:07:49.562933+0000","last_unstale":"2023-10-06T03:07:50.552740+0000","last_undegraded":"2023-10-06T03:07:50.552740+0000","last_fullsized":"2023-10-06T03:07:50.552740+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:19.290197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577687+0000","last_change":"2023-10-06T03:07:50.571814+0000","last_active":"2023-10-06T03:07:50.577687+0000","last_peered":"2023-10-06T03:07:50.577687+0000","last_clean":"2023-10-06T03:07:50.577687+0000","last_became_active":"2023-10-06T03:07:50.571432+0000","last_became_peered":"2023-10-06T03:07:50.571432+0000","last_unstale":"2023-10-06T03:07:50.577687+0000","last_undegraded":"2023-10-06T03:07:50.577687+0000","last_fullsized":"2023-10-06T03:07:50.577687+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:50.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574836+0000","last_change":"2023-10-06T03:07:49.568577+0000","last_active":"2023-10-06T03:07:50.574836+0000","last_peered":"2023-10-06T03:07:50.574836+0000","last_clean":"2023-10-06T03:07:50.574836+0000","last_became_active":"2023-10-06T03:07:49.568351+0000","last_became_peered":"2023-10-06T03:07:49.568351+0000","last_unstale":"2023-10-06T03:07:50.574836+0000","last_undegraded":"2023-10-06T03:07:50.574836+0000","last_fullsized":"2023-10-06T03:07:50.574836+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:59.860977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587789+0000","last_change":"2023-10-06T03:07:50.583381+0000","last_active":"2023-10-06T03:07:50.587789+0000","last_peered":"2023-10-06T03:07:50.587789+0000","last_clean":"2023-10-06T03:07:50.587789+0000","last_became_active":"2023-10-06T03:07:50.583119+0000","last_became_peered":"2023-10-06T03:07:50.583119+0000","last_unstale":"2023-10-06T03:07:50.587789+0000","last_undegraded":"2023-10-06T03:07:50.587789+0000","last_fullsized":"2023-10-06T03:07:50.587789+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:07.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575307+0000","last_change":"2023-10-06T03:07:49.567393+0000","last_active":"2023-10-06T03:07:50.575307+0000","last_peered":"2023-10-06T03:07:50.575307+0000","last_clean":"2023-10-06T03:07:50.575307+0000","last_became_active":"2023-10-06T03:07:49.567129+0000","last_became_peered":"2023-10-06T03:07:49.567129+0000","last_unstale":"2023-10-06T03:07:50.575307+0000","last_undegraded":"2023-10-06T03:07:50.575307+0000","last_fullsized":"2023-10-06T03:07:50.575307+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:41.761455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573801+0000","last_change":"2023-10-06T03:07:50.565621+0000","last_active":"2023-10-06T03:07:50.573801+0000","last_peered":"2023-10-06T03:07:50.573801+0000","last_clean":"2023-10-06T03:07:50.573801+0000","last_became_active":"2023-10-06T03:07:50.565348+0000","last_became_peered":"2023-10-06T03:07:50.565348+0000","last_unstale":"2023-10-06T03:07:50.573801+0000","last_undegraded":"2023-10-06T03:07:50.573801+0000","last_fullsized":"2023-10-06T03:07:50.573801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573173+0000","last_change":"2023-10-06T03:07:49.574956+0000","last_active":"2023-10-06T03:07:50.573173+0000","last_peered":"2023-10-06T03:07:50.573173+0000","last_clean":"2023-10-06T03:07:50.573173+0000","last_became_active":"2023-10-06T03:07:49.574691+0000","last_became_peered":"2023-10-06T03:07:49.574691+0000","last_unstale":"2023-10-06T03:07:50.573173+0000","last_undegraded":"2023-10-06T03:07:50.573173+0000","last_fullsized":"2023-10-06T03:07:50.573173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:15.874825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571996+0000","last_change":"2023-10-06T03:07:50.565193+0000","last_active":"2023-10-06T03:07:50.571996+0000","last_peered":"2023-10-06T03:07:50.571996+0000","last_clean":"2023-10-06T03:07:50.571996+0000","last_became_active":"2023-10-06T03:07:50.564949+0000","last_became_peered":"2023-10-06T03:07:50.564949+0000","last_unstale":"2023-10-06T03:07:50.571996+0000","last_undegraded":"2023-10-06T03:07:50.571996+0000","last_fullsized":"2023-10-06T03:07:50.571996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:07.731428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562713+0000","last_change":"2023-10-06T03:07:49.571624+0000","last_active":"2023-10-06T03:07:50.562713+0000","last_peered":"2023-10-06T03:07:50.562713+0000","last_clean":"2023-10-06T03:07:50.562713+0000","last_became_active":"2023-10-06T03:07:49.571371+0000","last_became_peered":"2023-10-06T03:07:49.571371+0000","last_unstale":"2023-10-06T03:07:50.562713+0000","last_undegraded":"2023-10-06T03:07:50.562713+0000","last_fullsized":"2023-10-06T03:07:50.562713+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:43.573379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570121+0000","last_change":"2023-10-06T03:07:50.559354+0000","last_active":"2023-10-06T03:07:50.570121+0000","last_peered":"2023-10-06T03:07:50.570121+0000","last_clean":"2023-10-06T03:07:50.570121+0000","last_became_active":"2023-10-06T03:07:50.559074+0000","last_became_peered":"2023-10-06T03:07:50.559074+0000","last_unstale":"2023-10-06T03:07:50.570121+0000","last_undegraded":"2023-10-06T03:07:50.570121+0000","last_fullsized":"2023-10-06T03:07:50.570121+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:31.306909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574667+0000","last_change":"2023-10-06T03:07:49.567927+0000","last_active":"2023-10-06T03:07:50.574667+0000","last_peered":"2023-10-06T03:07:50.574667+0000","last_clean":"2023-10-06T03:07:50.574667+0000","last_became_active":"2023-10-06T03:07:49.567651+0000","last_became_peered":"2023-10-06T03:07:49.567651+0000","last_unstale":"2023-10-06T03:07:50.574667+0000","last_undegraded":"2023-10-06T03:07:50.574667+0000","last_fullsized":"2023-10-06T03:07:50.574667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:41.894694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584764+0000","last_change":"2023-10-06T03:07:50.579620+0000","last_active":"2023-10-06T03:07:50.584764+0000","last_peered":"2023-10-06T03:07:50.584764+0000","last_clean":"2023-10-06T03:07:50.584764+0000","last_became_active":"2023-10-06T03:07:50.579374+0000","last_became_peered":"2023-10-06T03:07:50.579374+0000","last_unstale":"2023-10-06T03:07:50.584764+0000","last_undegraded":"2023-10-06T03:07:50.584764+0000","last_fullsized":"2023-10-06T03:07:50.584764+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.561899+0000","last_change":"2023-10-06T03:07:49.566472+0000","last_active":"2023-10-06T03:07:50.561899+0000","last_peered":"2023-10-06T03:07:50.561899+0000","last_clean":"2023-10-06T03:07:50.561899+0000","last_became_active":"2023-10-06T03:07:49.566231+0000","last_became_peered":"2023-10-06T03:07:49.566231+0000","last_unstale":"2023-10-06T03:07:50.561899+0000","last_undegraded":"2023-10-06T03:07:50.561899+0000","last_fullsized":"2023-10-06T03:07:50.561899+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:49.540743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584709+0000","last_change":"2023-10-06T03:07:50.578349+0000","last_active":"2023-10-06T03:07:50.584709+0000","last_peered":"2023-10-06T03:07:50.584709+0000","last_clean":"2023-10-06T03:07:50.584709+0000","last_became_active":"2023-10-06T03:07:50.576833+0000","last_became_peered":"2023-10-06T03:07:50.576833+0000","last_unstale":"2023-10-06T03:07:50.584709+0000","last_undegraded":"2023-10-06T03:07:50.584709+0000","last_fullsized":"2023-10-06T03:07:50.584709+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:07.731428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570946+0000","last_change":"2023-10-06T03:07:49.565372+0000","last_active":"2023-10-06T03:07:50.570946+0000","last_peered":"2023-10-06T03:07:50.570946+0000","last_clean":"2023-10-06T03:07:50.570946+0000","last_became_active":"2023-10-06T03:07:49.565082+0000","last_became_peered":"2023-10-06T03:07:49.565082+0000","last_unstale":"2023-10-06T03:07:50.570946+0000","last_undegraded":"2023-10-06T03:07:50.570946+0000","last_fullsized":"2023-10-06T03:07:50.570946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:21.074952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572382+0000","last_change":"2023-10-06T03:07:50.566374+0000","last_active":"2023-10-06T03:07:50.572382+0000","last_peered":"2023-10-06T03:07:50.572382+0000","last_clean":"2023-10-06T03:07:50.572382+0000","last_became_active":"2023-10-06T03:07:50.565949+0000","last_became_peered":"2023-10-06T03:07:50.565949+0000","last_unstale":"2023-10-06T03:07:50.572382+0000","last_undegraded":"2023-10-06T03:07:50.572382+0000","last_fullsized":"2023-10-06T03:07:50.572382+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:18.274017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562047+0000","last_change":"2023-10-06T03:07:49.567446+0000","last_active":"2023-10-06T03:07:50.562047+0000","last_peered":"2023-10-06T03:07:50.562047+0000","last_clean":"2023-10-06T03:07:50.562047+0000","last_became_active":"2023-10-06T03:07:49.567093+0000","last_became_peered":"2023-10-06T03:07:49.567093+0000","last_unstale":"2023-10-06T03:07:50.562047+0000","last_undegraded":"2023-10-06T03:07:50.562047+0000","last_fullsized":"2023-10-06T03:07:50.562047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:45.671026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574000+0000","last_change":"2023-10-06T03:07:50.568922+0000","last_active":"2023-10-06T03:07:50.574000+0000","last_peered":"2023-10-06T03:07:50.574000+0000","last_clean":"2023-10-06T03:07:50.574000+0000","last_became_active":"2023-10-06T03:07:50.568709+0000","last_became_peered":"2023-10-06T03:07:50.568709+0000","last_unstale":"2023-10-06T03:07:50.574000+0000","last_undegraded":"2023-10-06T03:07:50.574000+0000","last_fullsized":"2023-10-06T03:07:50.574000+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:10.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573350+0000","last_change":"2023-10-06T03:07:49.567779+0000","last_active":"2023-10-06T03:07:50.573350+0000","last_peered":"2023-10-06T03:07:50.573350+0000","last_clean":"2023-10-06T03:07:50.573350+0000","last_became_active":"2023-10-06T03:07:49.567579+0000","last_became_peered":"2023-10-06T03:07:49.567579+0000","last_unstale":"2023-10-06T03:07:50.573350+0000","last_undegraded":"2023-10-06T03:07:50.573350+0000","last_fullsized":"2023-10-06T03:07:50.573350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:48.232486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576201+0000","last_change":"2023-10-06T03:07:50.570770+0000","last_active":"2023-10-06T03:07:50.576201+0000","last_peered":"2023-10-06T03:07:50.576201+0000","last_clean":"2023-10-06T03:07:50.576201+0000","last_became_active":"2023-10-06T03:07:50.570500+0000","last_became_peered":"2023-10-06T03:07:50.570500+0000","last_unstale":"2023-10-06T03:07:50.576201+0000","last_undegraded":"2023-10-06T03:07:50.576201+0000","last_fullsized":"2023-10-06T03:07:50.576201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:02.621345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-06T03:07:52.739762+0000","last_change":"2023-10-06T03:07:49.567546+0000","last_active":"2023-10-06T03:07:52.739762+0000","last_peered":"2023-10-06T03:07:52.739762+0000","last_clean":"2023-10-06T03:07:52.739762+0000","last_became_active":"2023-10-06T03:07:49.567274+0000","last_became_peered":"2023-10-06T03:07:49.567274+0000","last_unstale":"2023-10-06T03:07:52.739762+0000","last_undegraded":"2023-10-06T03:07:52.739762+0000","last_fullsized":"2023-10-06T03:07:52.739762+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573164+0000","last_change":"2023-10-06T03:07:50.570896+0000","last_active":"2023-10-06T03:07:50.573164+0000","last_peered":"2023-10-06T03:07:50.573164+0000","last_clean":"2023-10-06T03:07:50.573164+0000","last_became_active":"2023-10-06T03:07:50.570605+0000","last_became_peered":"2023-10-06T03:07:50.570605+0000","last_unstale":"2023-10-06T03:07:50.573164+0000","last_undegraded":"2023-10-06T03:07:50.573164+0000","last_fullsized":"2023-10-06T03:07:50.573164+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:07.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.558154+0000","last_change":"2023-10-06T03:07:49.571191+0000","last_active":"2023-10-06T03:07:50.558154+0000","last_peered":"2023-10-06T03:07:50.558154+0000","last_clean":"2023-10-06T03:07:50.558154+0000","last_became_active":"2023-10-06T03:07:49.570908+0000","last_became_peered":"2023-10-06T03:07:49.570908+0000","last_unstale":"2023-10-06T03:07:50.558154+0000","last_undegraded":"2023-10-06T03:07:50.558154+0000","last_fullsized":"2023-10-06T03:07:50.558154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581923+0000","last_change":"2023-10-06T03:07:50.573868+0000","last_active":"2023-10-06T03:07:50.581923+0000","last_peered":"2023-10-06T03:07:50.581923+0000","last_clean":"2023-10-06T03:07:50.581923+0000","last_became_active":"2023-10-06T03:07:50.572772+0000","last_became_peered":"2023-10-06T03:07:50.572772+0000","last_unstale":"2023-10-06T03:07:50.581923+0000","last_undegraded":"2023-10-06T03:07:50.581923+0000","last_fullsized":"2023-10-06T03:07:50.581923+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:45.966304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552674+0000","last_change":"2023-10-06T03:07:49.567138+0000","last_active":"2023-10-06T03:07:50.552674+0000","last_peered":"2023-10-06T03:07:50.552674+0000","last_clean":"2023-10-06T03:07:50.552674+0000","last_became_active":"2023-10-06T03:07:49.566884+0000","last_became_peered":"2023-10-06T03:07:49.566884+0000","last_unstale":"2023-10-06T03:07:50.552674+0000","last_undegraded":"2023-10-06T03:07:50.552674+0000","last_fullsized":"2023-10-06T03:07:50.552674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.299865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573641+0000","last_change":"2023-10-06T03:07:50.565737+0000","last_active":"2023-10-06T03:07:50.573641+0000","last_peered":"2023-10-06T03:07:50.573641+0000","last_clean":"2023-10-06T03:07:50.573641+0000","last_became_active":"2023-10-06T03:07:50.565477+0000","last_became_peered":"2023-10-06T03:07:50.565477+0000","last_unstale":"2023-10-06T03:07:50.573641+0000","last_undegraded":"2023-10-06T03:07:50.573641+0000","last_fullsized":"2023-10-06T03:07:50.573641+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:29.710606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.553258+0000","last_change":"2023-10-06T03:07:49.565800+0000","last_active":"2023-10-06T03:07:50.553258+0000","last_peered":"2023-10-06T03:07:50.553258+0000","last_clean":"2023-10-06T03:07:50.553258+0000","last_became_active":"2023-10-06T03:07:49.565480+0000","last_became_peered":"2023-10-06T03:07:49.565480+0000","last_unstale":"2023-10-06T03:07:50.553258+0000","last_undegraded":"2023-10-06T03:07:50.553258+0000","last_fullsized":"2023-10-06T03:07:50.553258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:06.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584782+0000","last_change":"2023-10-06T03:07:50.579751+0000","last_active":"2023-10-06T03:07:50.584782+0000","last_peered":"2023-10-06T03:07:50.584782+0000","last_clean":"2023-10-06T03:07:50.584782+0000","last_became_active":"2023-10-06T03:07:50.579393+0000","last_became_peered":"2023-10-06T03:07:50.579393+0000","last_unstale":"2023-10-06T03:07:50.584782+0000","last_undegraded":"2023-10-06T03:07:50.584782+0000","last_fullsized":"2023-10-06T03:07:50.584782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:18.274017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552787+0000","last_change":"2023-10-06T03:07:49.574874+0000","last_active":"2023-10-06T03:07:50.552787+0000","last_peered":"2023-10-06T03:07:50.552787+0000","last_clean":"2023-10-06T03:07:50.552787+0000","last_became_active":"2023-10-06T03:07:49.574539+0000","last_became_peered":"2023-10-06T03:07:49.574539+0000","last_unstale":"2023-10-06T03:07:50.552787+0000","last_undegraded":"2023-10-06T03:07:50.552787+0000","last_fullsized":"2023-10-06T03:07:50.552787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:26.959832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577939+0000","last_change":"2023-10-06T03:07:50.572981+0000","last_active":"2023-10-06T03:07:50.577939+0000","last_peered":"2023-10-06T03:07:50.577939+0000","last_clean":"2023-10-06T03:07:50.577939+0000","last_became_active":"2023-10-06T03:07:50.572653+0000","last_became_peered":"2023-10-06T03:07:50.572653+0000","last_unstale":"2023-10-06T03:07:50.577939+0000","last_undegraded":"2023-10-06T03:07:50.577939+0000","last_fullsized":"2023-10-06T03:07:50.577939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:40.777609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552118+0000","last_change":"2023-10-06T03:07:49.563073+0000","last_active":"2023-10-06T03:07:50.552118+0000","last_peered":"2023-10-06T03:07:50.552118+0000","last_clean":"2023-10-06T03:07:50.552118+0000","last_became_active":"2023-10-06T03:07:49.562796+0000","last_became_peered":"2023-10-06T03:07:49.562796+0000","last_unstale":"2023-10-06T03:07:50.552118+0000","last_undegraded":"2023-10-06T03:07:50.552118+0000","last_fullsized":"2023-10-06T03:07:50.552118+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:32.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581581+0000","last_change":"2023-10-06T03:07:50.574361+0000","last_active":"2023-10-06T03:07:50.581581+0000","last_peered":"2023-10-06T03:07:50.581581+0000","last_clean":"2023-10-06T03:07:50.581581+0000","last_became_active":"2023-10-06T03:07:50.574196+0000","last_became_peered":"2023-10-06T03:07:50.574196+0000","last_unstale":"2023-10-06T03:07:50.581581+0000","last_undegraded":"2023-10-06T03:07:50.581581+0000","last_fullsized":"2023-10-06T03:07:50.581581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:02.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562144+0000","last_change":"2023-10-06T03:07:49.566063+0000","last_active":"2023-10-06T03:07:50.562144+0000","last_peered":"2023-10-06T03:07:50.562144+0000","last_clean":"2023-10-06T03:07:50.562144+0000","last_became_active":"2023-10-06T03:07:49.565801+0000","last_became_peered":"2023-10-06T03:07:49.565801+0000","last_unstale":"2023-10-06T03:07:50.562144+0000","last_undegraded":"2023-10-06T03:07:50.562144+0000","last_fullsized":"2023-10-06T03:07:50.562144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:53.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572719+0000","last_change":"2023-10-06T03:07:50.566817+0000","last_active":"2023-10-06T03:07:50.572719+0000","last_peered":"2023-10-06T03:07:50.572719+0000","last_clean":"2023-10-06T03:07:50.572719+0000","last_became_active":"2023-10-06T03:07:50.566535+0000","last_became_peered":"2023-10-06T03:07:50.566535+0000","last_unstale":"2023-10-06T03:07:50.572719+0000","last_undegraded":"2023-10-06T03:07:50.572719+0000","last_fullsized":"2023-10-06T03:07:50.572719+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:27.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"102'6","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.589060+0000","last_change":"2023-10-06T03:07:49.562916+0000","last_active":"2023-10-06T03:07:50.589060+0000","last_peered":"2023-10-06T03:07:50.589060+0000","last_clean":"2023-10-06T03:07:50.589060+0000","last_became_active":"2023-10-06T03:07:49.562661+0000","last_became_peered":"2023-10-06T03:07:49.562661+0000","last_unstale":"2023-10-06T03:07:50.589060+0000","last_undegraded":"2023-10-06T03:07:50.589060+0000","last_fullsized":"2023-10-06T03:07:50.589060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:14.508655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.556557+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:50.556557+0000","last_peered":"2023-10-06T03:07:50.556557+0000","last_clean":"2023-10-06T03:07:50.556557+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:50.556557+0000","last_undegraded":"2023-10-06T03:07:50.556557+0000","last_fullsized":"2023-10-06T03:07:50.556557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584647+0000","last_change":"2023-10-06T03:07:50.578854+0000","last_active":"2023-10-06T03:07:50.584647+0000","last_peered":"2023-10-06T03:07:50.584647+0000","last_clean":"2023-10-06T03:07:50.584647+0000","last_became_active":"2023-10-06T03:07:50.578290+0000","last_became_peered":"2023-10-06T03:07:50.578290+0000","last_unstale":"2023-10-06T03:07:50.584647+0000","last_undegraded":"2023-10-06T03:07:50.584647+0000","last_fullsized":"2023-10-06T03:07:50.584647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:50.013969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"102'3","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575195+0000","last_change":"2023-10-06T03:07:49.570381+0000","last_active":"2023-10-06T03:07:50.575195+0000","last_peered":"2023-10-06T03:07:50.575195+0000","last_clean":"2023-10-06T03:07:50.575195+0000","last_became_active":"2023-10-06T03:07:49.570123+0000","last_became_peered":"2023-10-06T03:07:49.570123+0000","last_unstale":"2023-10-06T03:07:50.575195+0000","last_undegraded":"2023-10-06T03:07:50.575195+0000","last_fullsized":"2023-10-06T03:07:50.575195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.564578+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:50.564578+0000","last_peered":"2023-10-06T03:07:50.564578+0000","last_clean":"2023-10-06T03:07:50.564578+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:50.564578+0000","last_undegraded":"2023-10-06T03:07:50.564578+0000","last_fullsized":"2023-10-06T03:07:50.564578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585231+0000","last_change":"2023-10-06T03:07:50.582099+0000","last_active":"2023-10-06T03:07:50.585231+0000","last_peered":"2023-10-06T03:07:50.585231+0000","last_clean":"2023-10-06T03:07:50.585231+0000","last_became_active":"2023-10-06T03:07:50.581860+0000","last_became_peered":"2023-10-06T03:07:50.581860+0000","last_unstale":"2023-10-06T03:07:50.585231+0000","last_undegraded":"2023-10-06T03:07:50.585231+0000","last_fullsized":"2023-10-06T03:07:50.585231+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:50.013969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.553486+0000","last_change":"2023-10-06T03:07:49.574358+0000","last_active":"2023-10-06T03:07:50.553486+0000","last_peered":"2023-10-06T03:07:50.553486+0000","last_clean":"2023-10-06T03:07:50.553486+0000","last_became_active":"2023-10-06T03:07:49.574001+0000","last_became_peered":"2023-10-06T03:07:49.574001+0000","last_unstale":"2023-10-06T03:07:50.553486+0000","last_undegraded":"2023-10-06T03:07:50.553486+0000","last_fullsized":"2023-10-06T03:07:50.553486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:14.508655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567674+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:50.567674+0000","last_peered":"2023-10-06T03:07:50.567674+0000","last_clean":"2023-10-06T03:07:50.567674+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:50.567674+0000","last_undegraded":"2023-10-06T03:07:50.567674+0000","last_fullsized":"2023-10-06T03:07:50.567674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585304+0000","last_change":"2023-10-06T03:07:50.581005+0000","last_active":"2023-10-06T03:07:50.585304+0000","last_peered":"2023-10-06T03:07:50.585304+0000","last_clean":"2023-10-06T03:07:50.585304+0000","last_became_active":"2023-10-06T03:07:50.580494+0000","last_became_peered":"2023-10-06T03:07:50.580494+0000","last_unstale":"2023-10-06T03:07:50.585304+0000","last_undegraded":"2023-10-06T03:07:50.585304+0000","last_fullsized":"2023-10-06T03:07:50.585304+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:57.318520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584485+0000","last_change":"2023-10-06T03:07:49.566406+0000","last_active":"2023-10-06T03:07:50.584485+0000","last_peered":"2023-10-06T03:07:50.584485+0000","last_clean":"2023-10-06T03:07:50.584485+0000","last_became_active":"2023-10-06T03:07:49.566148+0000","last_became_peered":"2023-10-06T03:07:49.566148+0000","last_unstale":"2023-10-06T03:07:50.584485+0000","last_undegraded":"2023-10-06T03:07:50.584485+0000","last_fullsized":"2023-10-06T03:07:50.584485+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:57.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562238+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:50.562238+0000","last_peered":"2023-10-06T03:07:50.562238+0000","last_clean":"2023-10-06T03:07:50.562238+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:50.562238+0000","last_undegraded":"2023-10-06T03:07:50.562238+0000","last_fullsized":"2023-10-06T03:07:50.562238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.578008+0000","last_change":"2023-10-06T03:07:50.574780+0000","last_active":"2023-10-06T03:07:50.578008+0000","last_peered":"2023-10-06T03:07:50.578008+0000","last_clean":"2023-10-06T03:07:50.578008+0000","last_became_active":"2023-10-06T03:07:50.574322+0000","last_became_peered":"2023-10-06T03:07:50.574322+0000","last_unstale":"2023-10-06T03:07:50.578008+0000","last_undegraded":"2023-10-06T03:07:50.578008+0000","last_fullsized":"2023-10-06T03:07:50.578008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:02.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"102'8","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587760+0000","last_change":"2023-10-06T03:07:49.562128+0000","last_active":"2023-10-06T03:07:50.587760+0000","last_peered":"2023-10-06T03:07:50.587760+0000","last_clean":"2023-10-06T03:07:50.587760+0000","last_became_active":"2023-10-06T03:07:49.561879+0000","last_became_peered":"2023-10-06T03:07:49.561879+0000","last_unstale":"2023-10-06T03:07:50.587760+0000","last_undegraded":"2023-10-06T03:07:50.587760+0000","last_fullsized":"2023-10-06T03:07:50.587760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:56.313797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562418+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:50.562418+0000","last_peered":"2023-10-06T03:07:50.562418+0000","last_clean":"2023-10-06T03:07:50.562418+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:50.562418+0000","last_undegraded":"2023-10-06T03:07:50.562418+0000","last_fullsized":"2023-10-06T03:07:50.562418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552654+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:50.552654+0000","last_peered":"2023-10-06T03:07:50.552654+0000","last_clean":"2023-10-06T03:07:50.552654+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:50.552654+0000","last_undegraded":"2023-10-06T03:07:50.552654+0000","last_fullsized":"2023-10-06T03:07:50.552654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581699+0000","last_change":"2023-10-06T03:07:50.573994+0000","last_active":"2023-10-06T03:07:50.581699+0000","last_peered":"2023-10-06T03:07:50.581699+0000","last_clean":"2023-10-06T03:07:50.581699+0000","last_became_active":"2023-10-06T03:07:50.573801+0000","last_became_peered":"2023-10-06T03:07:50.573801+0000","last_unstale":"2023-10-06T03:07:50.581699+0000","last_undegraded":"2023-10-06T03:07:50.581699+0000","last_fullsized":"2023-10-06T03:07:50.581699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:22.069683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571236+0000","last_change":"2023-10-06T03:07:49.574564+0000","last_active":"2023-10-06T03:07:50.571236+0000","last_peered":"2023-10-06T03:07:50.571236+0000","last_clean":"2023-10-06T03:07:50.571236+0000","last_became_active":"2023-10-06T03:07:49.574290+0000","last_became_peered":"2023-10-06T03:07:49.574290+0000","last_unstale":"2023-10-06T03:07:50.571236+0000","last_undegraded":"2023-10-06T03:07:50.571236+0000","last_fullsized":"2023-10-06T03:07:50.571236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562557+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:50.562557+0000","last_peered":"2023-10-06T03:07:50.562557+0000","last_clean":"2023-10-06T03:07:50.562557+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:50.562557+0000","last_undegraded":"2023-10-06T03:07:50.562557+0000","last_fullsized":"2023-10-06T03:07:50.562557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573132+0000","last_change":"2023-10-06T03:07:50.568705+0000","last_active":"2023-10-06T03:07:50.573132+0000","last_peered":"2023-10-06T03:07:50.573132+0000","last_clean":"2023-10-06T03:07:50.573132+0000","last_became_active":"2023-10-06T03:07:50.568454+0000","last_became_peered":"2023-10-06T03:07:50.568454+0000","last_unstale":"2023-10-06T03:07:50.573132+0000","last_undegraded":"2023-10-06T03:07:50.573132+0000","last_fullsized":"2023-10-06T03:07:50.573132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:49.459109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570260+0000","last_change":"2023-10-06T03:07:49.563167+0000","last_active":"2023-10-06T03:07:50.570260+0000","last_peered":"2023-10-06T03:07:50.570260+0000","last_clean":"2023-10-06T03:07:50.570260+0000","last_became_active":"2023-10-06T03:07:49.562680+0000","last_became_peered":"2023-10-06T03:07:49.562680+0000","last_unstale":"2023-10-06T03:07:50.570260+0000","last_undegraded":"2023-10-06T03:07:50.570260+0000","last_fullsized":"2023-10-06T03:07:50.570260+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:01.616622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.554084+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:50.554084+0000","last_peered":"2023-10-06T03:07:50.554084+0000","last_clean":"2023-10-06T03:07:50.554084+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:50.554084+0000","last_undegraded":"2023-10-06T03:07:50.554084+0000","last_fullsized":"2023-10-06T03:07:50.554084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571441+0000","last_change":"2023-10-06T03:07:50.565348+0000","last_active":"2023-10-06T03:07:50.571441+0000","last_peered":"2023-10-06T03:07:50.571441+0000","last_clean":"2023-10-06T03:07:50.571441+0000","last_became_active":"2023-10-06T03:07:50.565091+0000","last_became_peered":"2023-10-06T03:07:50.565091+0000","last_unstale":"2023-10-06T03:07:50.571441+0000","last_undegraded":"2023-10-06T03:07:50.571441+0000","last_fullsized":"2023-10-06T03:07:50.571441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:02.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.556502+0000","last_change":"2023-10-06T03:07:49.568754+0000","last_active":"2023-10-06T03:07:50.556502+0000","last_peered":"2023-10-06T03:07:50.556502+0000","last_clean":"2023-10-06T03:07:50.556502+0000","last_became_active":"2023-10-06T03:07:49.568471+0000","last_became_peered":"2023-10-06T03:07:49.568471+0000","last_unstale":"2023-10-06T03:07:50.556502+0000","last_undegraded":"2023-10-06T03:07:50.556502+0000","last_fullsized":"2023-10-06T03:07:50.556502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.565111+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:50.565111+0000","last_peered":"2023-10-06T03:07:50.565111+0000","last_clean":"2023-10-06T03:07:50.565111+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:50.565111+0000","last_undegraded":"2023-10-06T03:07:50.565111+0000","last_fullsized":"2023-10-06T03:07:50.565111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303232,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068608,"statfs":{"total":96636764160,"available":96326254592,"internally_reserved":0,"allocated":1892352,"data_stored":1177182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.78000000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":0.73199999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302640,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069200,"statfs":{"total":96636764160,"available":96326860800,"internally_reserved":0,"allocated":1286144,"data_stored":586620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.499},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.441}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.59099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":586088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.377}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.308}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.72899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303220,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068620,"statfs":{"total":96636764160,"available":96326266880,"internally_reserved":0,"allocated":1880064,"data_stored":1176456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.371},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.308},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.44},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.497}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.42799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.52900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1282048,"data_stored":586066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28999999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.309},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.628}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298556,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073284,"statfs":{"total":96636764160,"available":96331042816,"internally_reserved":0,"allocated":1298432,"data_stored":587636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.307}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.621},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.5}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54100000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.60999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302648,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069192,"statfs":{"total":96636764160,"available":96326852608,"internally_reserved":0,"allocated":1294336,"data_stored":587602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.80600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073292,"statfs":{"total":96636764160,"available":96331051008,"internally_reserved":0,"allocated":1290240,"data_stored":586226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.435},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.627}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:59.107 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:07:59.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.672+0000 7fe6502bf700 1 Processor -- start 2023-10-06T03:07:59.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.672+0000 7fe6502bf700 1 -- start start 2023-10-06T03:07:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.673+0000 7fe6502bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159890 0x7fe64815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.673+0000 7fe6502bf700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe648069560 0x7fe6480699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.673+0000 7fe6502bf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64815a160 con 0x7fe648159c60 2023-10-06T03:07:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.673+0000 7fe6502bf700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe648069f20 con 0x7fe648069560 2023-10-06T03:07:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.673+0000 7fe6502bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480d13a0 con 0x7fe648159890 2023-10-06T03:07:59.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e85c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe648069560 0x7fe6480699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64d85a700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fe648159c60 0x7fe64815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:54784/0 (socket says 172.21.15.92:54784) 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e05b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159890 0x7fe64815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64d85a700 1 -- 172.21.15.92:0/2004827536 learned_addr learned my addr 172.21.15.92:0/2004827536 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e85c700 1 -- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159890 msgr2=0x7fe64815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e85c700 1 --2- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159890 0x7fe64815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e85c700 1 -- 172.21.15.92:0/2004827536 >> v1:172.21.15.136:6789/0 conn(0x7fe648159c60 legacy=0x7fe64815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e85c700 1 -- 172.21.15.92:0/2004827536 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64815e140 con 0x7fe648069560 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e05b700 1 --2- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159890 0x7fe64815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:07:59.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64e85c700 1 --2- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648069560 0x7fe6480699e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6400098e0 tx=0x7fe6400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25fbb37e479c06ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64d059700 1 -- 172.21.15.92:0/2004827536 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000b640 con 0x7fe648069560 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64d059700 1 -- 172.21.15.92:0/2004827536 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64000b7a0 con 0x7fe648069560 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.674+0000 7fe64d059700 1 -- 172.21.15.92:0/2004827536 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64000bb40 con 0x7fe648069560 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 -- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648069560 msgr2=0x7fe6480699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 --2- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648069560 0x7fe6480699e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6400098e0 tx=0x7fe6400091a0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 -- 172.21.15.92:0/2004827536 shutdown_connections 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 --2- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648069560 0x7fe6480699e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 --2- 172.21.15.92:0/2004827536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159890 0x7fe64815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 -- 172.21.15.92:0/2004827536 >> 172.21.15.92:0/2004827536 conn(0x7fe648068e20 msgr2=0x7fe6480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:59.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 -- 172.21.15.92:0/2004827536 shutdown_connections 2023-10-06T03:07:59.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.675+0000 7fe6502bf700 1 -- 172.21.15.92:0/2004827536 wait complete. 2023-10-06T03:07:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.676+0000 7fe6502bf700 1 Processor -- start 2023-10-06T03:07:59.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.676+0000 7fe6502bf700 1 -- start start 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe6502bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 0x7fe648149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe6502bf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648159890 0x7fe6480d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe6502bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159c60 0x7fe6480d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe6502bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64816a840 con 0x7fe648069560 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe6502bf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe648169c00 con 0x7fe648159890 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe6502bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481498c0 con 0x7fe648159c60 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 0x7fe648149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 0x7fe648149380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38282/0 (socket says 172.21.15.92:38282) 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 -- 172.21.15.92:0/1387194582 learned_addr learned my addr 172.21.15.92:0/1387194582 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:07:59.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64d85a700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648159890 0x7fe6480d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e85c700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159c60 0x7fe6480d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 -- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159c60 msgr2=0x7fe6480d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159c60 0x7fe6480d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 -- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648159890 msgr2=0x7fe6480d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648159890 0x7fe6480d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e05b700 1 -- 172.21.15.92:0/1387194582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009a20 con 0x7fe648069560 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.677+0000 7fe64e85c700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159c60 0x7fe6480d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.678+0000 7fe64e05b700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 0x7fe648149380 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c170 tx=0x7fe63800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.678+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800cd60 con 0x7fe648069560 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.678+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63800f040 con 0x7fe648069560 2023-10-06T03:07:59.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.678+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe640009420 con 0x7fe648069560 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.678+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638014690 con 0x7fe648069560 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.678+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64816a180 con 0x7fe648069560 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.679+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64804c3c0 con 0x7fe648069560 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.679+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe638014a00 con 0x7fe648069560 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.679+0000 7fe6467fc700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe634067290 0x7fe634069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.680+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe638085b50 con 0x7fe648069560 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.680+0000 7fe64d85a700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe634067290 0x7fe634069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:07:59.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.680+0000 7fe64d85a700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe634067290 0x7fe634069740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe64816d030 tx=0x7fe63c007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:07:59.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.684+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe638052740 con 0x7fe648069560 2023-10-06T03:07:59.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.863+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6480b1f90 con 0x7fe634067290 2023-10-06T03:07:59.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.871+0000 7fe6467fc700 1 -- 172.21.15.92:0/1387194582 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312095 (secure 0 0 0) 0x7fe6480b1f90 con 0x7fe634067290 2023-10-06T03:07:59.877 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:07:59.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe634067290 msgr2=0x7fe634069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:59.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe634067290 0x7fe634069740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe64816d030 tx=0x7fe63c007eb0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 msgr2=0x7fe648149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:07:59.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 0x7fe648149380 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c170 tx=0x7fe63800b040 comp rx=0 tx=0).stop 2023-10-06T03:07:59.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 shutdown_connections 2023-10-06T03:07:59.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe634067290 0x7fe634069740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe648069560 0x7fe648149380 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe648159c60 0x7fe6480d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 --2- 172.21.15.92:0/1387194582 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe648159890 0x7fe6480d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:07:59.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 >> 172.21.15.92:0/1387194582 conn(0x7fe648068e20 msgr2=0x7fe648157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:07:59.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 shutdown_connections 2023-10-06T03:07:59.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:07:59.874+0000 7fe6502bf700 1 -- 172.21.15.92:0/1387194582 wait complete. 2023-10-06T03:07:59.883 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:07:59.888 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-10-06T03:07:59.259003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405928,"kb_used_data":11240,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752568792,"statfs":{"total":773094113280,"available":770630443008,"internally_reserved":0,"allocated":11509760,"data_stored":5873876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.708336"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.578037+0000","last_change":"2023-10-06T03:07:50.572362+0000","last_active":"2023-10-06T03:07:50.578037+0000","last_peered":"2023-10-06T03:07:50.578037+0000","last_clean":"2023-10-06T03:07:50.578037+0000","last_became_active":"2023-10-06T03:07:50.571731+0000","last_became_peered":"2023-10-06T03:07:50.571731+0000","last_unstale":"2023-10-06T03:07:50.578037+0000","last_undegraded":"2023-10-06T03:07:50.578037+0000","last_fullsized":"2023-10-06T03:07:50.578037+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:40.777609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584493+0000","last_change":"2023-10-06T03:07:50.576225+0000","last_active":"2023-10-06T03:07:50.584493+0000","last_peered":"2023-10-06T03:07:50.584493+0000","last_clean":"2023-10-06T03:07:50.584493+0000","last_became_active":"2023-10-06T03:07:50.575924+0000","last_became_peered":"2023-10-06T03:07:50.575924+0000","last_unstale":"2023-10-06T03:07:50.584493+0000","last_undegraded":"2023-10-06T03:07:50.584493+0000","last_fullsized":"2023-10-06T03:07:50.584493+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:56.990336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584610+0000","last_change":"2023-10-06T03:07:50.578507+0000","last_active":"2023-10-06T03:07:50.584610+0000","last_peered":"2023-10-06T03:07:50.584610+0000","last_clean":"2023-10-06T03:07:50.584610+0000","last_became_active":"2023-10-06T03:07:50.578265+0000","last_became_peered":"2023-10-06T03:07:50.578265+0000","last_unstale":"2023-10-06T03:07:50.584610+0000","last_undegraded":"2023-10-06T03:07:50.584610+0000","last_fullsized":"2023-10-06T03:07:50.584610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:37.197922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573221+0000","last_change":"2023-10-06T03:07:50.571328+0000","last_active":"2023-10-06T03:07:50.573221+0000","last_peered":"2023-10-06T03:07:50.573221+0000","last_clean":"2023-10-06T03:07:50.573221+0000","last_became_active":"2023-10-06T03:07:50.571025+0000","last_became_peered":"2023-10-06T03:07:50.571025+0000","last_unstale":"2023-10-06T03:07:50.573221+0000","last_undegraded":"2023-10-06T03:07:50.573221+0000","last_fullsized":"2023-10-06T03:07:50.573221+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:38.614125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.564726+0000","last_change":"2023-10-06T03:07:50.559179+0000","last_active":"2023-10-06T03:07:50.564726+0000","last_peered":"2023-10-06T03:07:50.564726+0000","last_clean":"2023-10-06T03:07:50.564726+0000","last_became_active":"2023-10-06T03:07:50.558881+0000","last_became_peered":"2023-10-06T03:07:50.558881+0000","last_unstale":"2023-10-06T03:07:50.564726+0000","last_undegraded":"2023-10-06T03:07:50.564726+0000","last_fullsized":"2023-10-06T03:07:50.564726+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:47.768842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579171+0000","last_change":"2023-10-06T03:07:50.570727+0000","last_active":"2023-10-06T03:07:50.579171+0000","last_peered":"2023-10-06T03:07:50.579171+0000","last_clean":"2023-10-06T03:07:50.579171+0000","last_became_active":"2023-10-06T03:07:50.570484+0000","last_became_peered":"2023-10-06T03:07:50.570484+0000","last_unstale":"2023-10-06T03:07:50.579171+0000","last_undegraded":"2023-10-06T03:07:50.579171+0000","last_fullsized":"2023-10-06T03:07:50.579171+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:57.184437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569950+0000","last_change":"2023-10-06T03:07:50.563316+0000","last_active":"2023-10-06T03:07:50.569950+0000","last_peered":"2023-10-06T03:07:50.569950+0000","last_clean":"2023-10-06T03:07:50.569950+0000","last_became_active":"2023-10-06T03:07:50.563031+0000","last_became_peered":"2023-10-06T03:07:50.563031+0000","last_unstale":"2023-10-06T03:07:50.569950+0000","last_undegraded":"2023-10-06T03:07:50.569950+0000","last_fullsized":"2023-10-06T03:07:50.569950+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:28.830391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584901+0000","last_change":"2023-10-06T03:07:50.580182+0000","last_active":"2023-10-06T03:07:50.584901+0000","last_peered":"2023-10-06T03:07:50.584901+0000","last_clean":"2023-10-06T03:07:50.584901+0000","last_became_active":"2023-10-06T03:07:50.580001+0000","last_became_peered":"2023-10-06T03:07:50.580001+0000","last_unstale":"2023-10-06T03:07:50.584901+0000","last_undegraded":"2023-10-06T03:07:50.584901+0000","last_fullsized":"2023-10-06T03:07:50.584901+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:27.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582970+0000","last_change":"2023-10-06T03:07:50.577719+0000","last_active":"2023-10-06T03:07:50.582970+0000","last_peered":"2023-10-06T03:07:50.582970+0000","last_clean":"2023-10-06T03:07:50.582970+0000","last_became_active":"2023-10-06T03:07:50.577452+0000","last_became_peered":"2023-10-06T03:07:50.577452+0000","last_unstale":"2023-10-06T03:07:50.582970+0000","last_undegraded":"2023-10-06T03:07:50.582970+0000","last_fullsized":"2023-10-06T03:07:50.582970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:08.241579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582530+0000","last_change":"2023-10-06T03:07:50.571565+0000","last_active":"2023-10-06T03:07:50.582530+0000","last_peered":"2023-10-06T03:07:50.582530+0000","last_clean":"2023-10-06T03:07:50.582530+0000","last_became_active":"2023-10-06T03:07:50.570520+0000","last_became_peered":"2023-10-06T03:07:50.570520+0000","last_unstale":"2023-10-06T03:07:50.582530+0000","last_undegraded":"2023-10-06T03:07:50.582530+0000","last_fullsized":"2023-10-06T03:07:50.582530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:26.295707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567219+0000","last_change":"2023-10-06T03:07:50.562739+0000","last_active":"2023-10-06T03:07:50.567219+0000","last_peered":"2023-10-06T03:07:50.567219+0000","last_clean":"2023-10-06T03:07:50.567219+0000","last_became_active":"2023-10-06T03:07:50.561529+0000","last_became_peered":"2023-10-06T03:07:50.561529+0000","last_unstale":"2023-10-06T03:07:50.567219+0000","last_undegraded":"2023-10-06T03:07:50.567219+0000","last_fullsized":"2023-10-06T03:07:50.567219+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:33.483532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579373+0000","last_change":"2023-10-06T03:07:50.570788+0000","last_active":"2023-10-06T03:07:50.579373+0000","last_peered":"2023-10-06T03:07:50.579373+0000","last_clean":"2023-10-06T03:07:50.579373+0000","last_became_active":"2023-10-06T03:07:50.570296+0000","last_became_peered":"2023-10-06T03:07:50.570296+0000","last_unstale":"2023-10-06T03:07:50.579373+0000","last_undegraded":"2023-10-06T03:07:50.579373+0000","last_fullsized":"2023-10-06T03:07:50.579373+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:13:56.995621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581561+0000","last_change":"2023-10-06T03:07:50.572906+0000","last_active":"2023-10-06T03:07:50.581561+0000","last_peered":"2023-10-06T03:07:50.581561+0000","last_clean":"2023-10-06T03:07:50.581561+0000","last_became_active":"2023-10-06T03:07:50.572656+0000","last_became_peered":"2023-10-06T03:07:50.572656+0000","last_unstale":"2023-10-06T03:07:50.581561+0000","last_undegraded":"2023-10-06T03:07:50.581561+0000","last_fullsized":"2023-10-06T03:07:50.581561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:16.879548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584804+0000","last_change":"2023-10-06T03:07:50.580252+0000","last_active":"2023-10-06T03:07:50.584804+0000","last_peered":"2023-10-06T03:07:50.584804+0000","last_clean":"2023-10-06T03:07:50.584804+0000","last_became_active":"2023-10-06T03:07:50.578591+0000","last_became_peered":"2023-10-06T03:07:50.578591+0000","last_unstale":"2023-10-06T03:07:50.584804+0000","last_undegraded":"2023-10-06T03:07:50.584804+0000","last_fullsized":"2023-10-06T03:07:50.584804+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:50.341834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567088+0000","last_change":"2023-10-06T03:07:50.562853+0000","last_active":"2023-10-06T03:07:50.567088+0000","last_peered":"2023-10-06T03:07:50.567088+0000","last_clean":"2023-10-06T03:07:50.567088+0000","last_became_active":"2023-10-06T03:07:50.561839+0000","last_became_peered":"2023-10-06T03:07:50.561839+0000","last_unstale":"2023-10-06T03:07:50.567088+0000","last_undegraded":"2023-10-06T03:07:50.567088+0000","last_fullsized":"2023-10-06T03:07:50.567088+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:25.304588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573661+0000","last_change":"2023-10-06T03:07:50.567005+0000","last_active":"2023-10-06T03:07:50.573661+0000","last_peered":"2023-10-06T03:07:50.573661+0000","last_clean":"2023-10-06T03:07:50.573661+0000","last_became_active":"2023-10-06T03:07:50.566760+0000","last_became_peered":"2023-10-06T03:07:50.566760+0000","last_unstale":"2023-10-06T03:07:50.573661+0000","last_undegraded":"2023-10-06T03:07:50.573661+0000","last_fullsized":"2023-10-06T03:07:50.573661+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:21.538700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581519+0000","last_change":"2023-10-06T03:07:50.573244+0000","last_active":"2023-10-06T03:07:50.581519+0000","last_peered":"2023-10-06T03:07:50.581519+0000","last_clean":"2023-10-06T03:07:50.581519+0000","last_became_active":"2023-10-06T03:07:50.573001+0000","last_became_peered":"2023-10-06T03:07:50.573001+0000","last_unstale":"2023-10-06T03:07:50.581519+0000","last_undegraded":"2023-10-06T03:07:50.581519+0000","last_fullsized":"2023-10-06T03:07:50.581519+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:06:58.959529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584666+0000","last_change":"2023-10-06T03:07:50.579168+0000","last_active":"2023-10-06T03:07:50.584666+0000","last_peered":"2023-10-06T03:07:50.584666+0000","last_clean":"2023-10-06T03:07:50.584666+0000","last_became_active":"2023-10-06T03:07:50.577986+0000","last_became_peered":"2023-10-06T03:07:50.577986+0000","last_unstale":"2023-10-06T03:07:50.584666+0000","last_undegraded":"2023-10-06T03:07:50.584666+0000","last_fullsized":"2023-10-06T03:07:50.584666+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:30.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570414+0000","last_change":"2023-10-06T03:07:50.565479+0000","last_active":"2023-10-06T03:07:50.570414+0000","last_peered":"2023-10-06T03:07:50.570414+0000","last_clean":"2023-10-06T03:07:50.570414+0000","last_became_active":"2023-10-06T03:07:50.565172+0000","last_became_peered":"2023-10-06T03:07:50.565172+0000","last_unstale":"2023-10-06T03:07:50.570414+0000","last_undegraded":"2023-10-06T03:07:50.570414+0000","last_fullsized":"2023-10-06T03:07:50.570414+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:48:26.295707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576083+0000","last_change":"2023-10-06T03:07:50.573007+0000","last_active":"2023-10-06T03:07:50.576083+0000","last_peered":"2023-10-06T03:07:50.576083+0000","last_clean":"2023-10-06T03:07:50.576083+0000","last_became_active":"2023-10-06T03:07:50.572729+0000","last_became_peered":"2023-10-06T03:07:50.572729+0000","last_unstale":"2023-10-06T03:07:50.576083+0000","last_undegraded":"2023-10-06T03:07:50.576083+0000","last_fullsized":"2023-10-06T03:07:50.576083+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:45.959875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577990+0000","last_change":"2023-10-06T03:07:50.574602+0000","last_active":"2023-10-06T03:07:50.577990+0000","last_peered":"2023-10-06T03:07:50.577990+0000","last_clean":"2023-10-06T03:07:50.577990+0000","last_became_active":"2023-10-06T03:07:50.573933+0000","last_became_peered":"2023-10-06T03:07:50.573933+0000","last_unstale":"2023-10-06T03:07:50.577990+0000","last_undegraded":"2023-10-06T03:07:50.577990+0000","last_fullsized":"2023-10-06T03:07:50.577990+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.580385+0000","last_change":"2023-10-06T03:07:50.579726+0000","last_active":"2023-10-06T03:07:50.580385+0000","last_peered":"2023-10-06T03:07:50.580385+0000","last_clean":"2023-10-06T03:07:50.580385+0000","last_became_active":"2023-10-06T03:07:50.579482+0000","last_became_peered":"2023-10-06T03:07:50.579482+0000","last_unstale":"2023-10-06T03:07:50.580385+0000","last_undegraded":"2023-10-06T03:07:50.580385+0000","last_fullsized":"2023-10-06T03:07:50.580385+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:04:48.552505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587676+0000","last_change":"2023-10-06T03:07:50.581210+0000","last_active":"2023-10-06T03:07:50.587676+0000","last_peered":"2023-10-06T03:07:50.587676+0000","last_clean":"2023-10-06T03:07:50.587676+0000","last_became_active":"2023-10-06T03:07:50.580986+0000","last_became_peered":"2023-10-06T03:07:50.580986+0000","last_unstale":"2023-10-06T03:07:50.587676+0000","last_undegraded":"2023-10-06T03:07:50.587676+0000","last_fullsized":"2023-10-06T03:07:50.587676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:50:41.442852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569161+0000","last_change":"2023-10-06T03:07:50.561654+0000","last_active":"2023-10-06T03:07:50.569161+0000","last_peered":"2023-10-06T03:07:50.569161+0000","last_clean":"2023-10-06T03:07:50.569161+0000","last_became_active":"2023-10-06T03:07:50.561023+0000","last_became_peered":"2023-10-06T03:07:50.561023+0000","last_unstale":"2023-10-06T03:07:50.569161+0000","last_undegraded":"2023-10-06T03:07:50.569161+0000","last_fullsized":"2023-10-06T03:07:50.569161+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:44.578103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569197+0000","last_change":"2023-10-06T03:07:50.562602+0000","last_active":"2023-10-06T03:07:50.569197+0000","last_peered":"2023-10-06T03:07:50.569197+0000","last_clean":"2023-10-06T03:07:50.569197+0000","last_became_active":"2023-10-06T03:07:50.562303+0000","last_became_peered":"2023-10-06T03:07:50.562303+0000","last_unstale":"2023-10-06T03:07:50.569197+0000","last_undegraded":"2023-10-06T03:07:50.569197+0000","last_fullsized":"2023-10-06T03:07:50.569197+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:56.990336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575245+0000","last_change":"2023-10-06T03:07:50.569444+0000","last_active":"2023-10-06T03:07:50.575245+0000","last_peered":"2023-10-06T03:07:50.575245+0000","last_clean":"2023-10-06T03:07:50.575245+0000","last_became_active":"2023-10-06T03:07:50.569142+0000","last_became_peered":"2023-10-06T03:07:50.569142+0000","last_unstale":"2023-10-06T03:07:50.575245+0000","last_undegraded":"2023-10-06T03:07:50.575245+0000","last_fullsized":"2023-10-06T03:07:50.575245+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:18.520811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577568+0000","last_change":"2023-10-06T03:07:50.570913+0000","last_active":"2023-10-06T03:07:50.577568+0000","last_peered":"2023-10-06T03:07:50.577568+0000","last_clean":"2023-10-06T03:07:50.577568+0000","last_became_active":"2023-10-06T03:07:50.570641+0000","last_became_peered":"2023-10-06T03:07:50.570641+0000","last_unstale":"2023-10-06T03:07:50.577568+0000","last_undegraded":"2023-10-06T03:07:50.577568+0000","last_fullsized":"2023-10-06T03:07:50.577568+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:22.069683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579352+0000","last_change":"2023-10-06T03:07:50.570632+0000","last_active":"2023-10-06T03:07:50.579352+0000","last_peered":"2023-10-06T03:07:50.579352+0000","last_clean":"2023-10-06T03:07:50.579352+0000","last_became_active":"2023-10-06T03:07:50.569983+0000","last_became_peered":"2023-10-06T03:07:50.569983+0000","last_unstale":"2023-10-06T03:07:50.579352+0000","last_undegraded":"2023-10-06T03:07:50.579352+0000","last_fullsized":"2023-10-06T03:07:50.579352+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:20.064356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582225+0000","last_change":"2023-10-06T03:07:50.577544+0000","last_active":"2023-10-06T03:07:50.582225+0000","last_peered":"2023-10-06T03:07:50.582225+0000","last_clean":"2023-10-06T03:07:50.582225+0000","last_became_active":"2023-10-06T03:07:50.577322+0000","last_became_peered":"2023-10-06T03:07:50.577322+0000","last_unstale":"2023-10-06T03:07:50.582225+0000","last_undegraded":"2023-10-06T03:07:50.582225+0000","last_fullsized":"2023-10-06T03:07:50.582225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:12.914946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585038+0000","last_change":"2023-10-06T03:07:50.581285+0000","last_active":"2023-10-06T03:07:50.585038+0000","last_peered":"2023-10-06T03:07:50.585038+0000","last_clean":"2023-10-06T03:07:50.585038+0000","last_became_active":"2023-10-06T03:07:50.581039+0000","last_became_peered":"2023-10-06T03:07:50.581039+0000","last_unstale":"2023-10-06T03:07:50.585038+0000","last_undegraded":"2023-10-06T03:07:50.585038+0000","last_fullsized":"2023-10-06T03:07:50.585038+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:02.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574262+0000","last_change":"2023-10-06T03:07:50.566299+0000","last_active":"2023-10-06T03:07:50.574262+0000","last_peered":"2023-10-06T03:07:50.574262+0000","last_clean":"2023-10-06T03:07:50.574262+0000","last_became_active":"2023-10-06T03:07:50.566026+0000","last_became_peered":"2023-10-06T03:07:50.566026+0000","last_unstale":"2023-10-06T03:07:50.574262+0000","last_undegraded":"2023-10-06T03:07:50.574262+0000","last_fullsized":"2023-10-06T03:07:50.574262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:13:30.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.569182+0000","last_change":"2023-10-06T03:07:50.561817+0000","last_active":"2023-10-06T03:07:50.569182+0000","last_peered":"2023-10-06T03:07:50.569182+0000","last_clean":"2023-10-06T03:07:50.569182+0000","last_became_active":"2023-10-06T03:07:50.561357+0000","last_became_peered":"2023-10-06T03:07:50.561357+0000","last_unstale":"2023-10-06T03:07:50.569182+0000","last_undegraded":"2023-10-06T03:07:50.569182+0000","last_fullsized":"2023-10-06T03:07:50.569182+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:26.301309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581912+0000","last_change":"2023-10-06T03:07:50.577842+0000","last_active":"2023-10-06T03:07:50.581912+0000","last_peered":"2023-10-06T03:07:50.581912+0000","last_clean":"2023-10-06T03:07:50.581912+0000","last_became_active":"2023-10-06T03:07:50.577581+0000","last_became_peered":"2023-10-06T03:07:50.577581+0000","last_unstale":"2023-10-06T03:07:50.581912+0000","last_undegraded":"2023-10-06T03:07:50.581912+0000","last_fullsized":"2023-10-06T03:07:50.581912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:18.485934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"102'4","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572923+0000","last_change":"2023-10-06T03:07:49.568419+0000","last_active":"2023-10-06T03:07:50.572923+0000","last_peered":"2023-10-06T03:07:50.572923+0000","last_clean":"2023-10-06T03:07:50.572923+0000","last_became_active":"2023-10-06T03:07:49.568113+0000","last_became_peered":"2023-10-06T03:07:49.568113+0000","last_unstale":"2023-10-06T03:07:50.572923+0000","last_undegraded":"2023-10-06T03:07:50.572923+0000","last_fullsized":"2023-10-06T03:07:50.572923+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.891794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584422+0000","last_change":"2023-10-06T03:07:50.578986+0000","last_active":"2023-10-06T03:07:50.584422+0000","last_peered":"2023-10-06T03:07:50.584422+0000","last_clean":"2023-10-06T03:07:50.584422+0000","last_became_active":"2023-10-06T03:07:50.578662+0000","last_became_peered":"2023-10-06T03:07:50.578662+0000","last_unstale":"2023-10-06T03:07:50.584422+0000","last_undegraded":"2023-10-06T03:07:50.584422+0000","last_fullsized":"2023-10-06T03:07:50.584422+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:13.437927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573465+0000","last_change":"2023-10-06T03:07:49.568589+0000","last_active":"2023-10-06T03:07:50.573465+0000","last_peered":"2023-10-06T03:07:50.573465+0000","last_clean":"2023-10-06T03:07:50.573465+0000","last_became_active":"2023-10-06T03:07:49.568313+0000","last_became_peered":"2023-10-06T03:07:49.568313+0000","last_unstale":"2023-10-06T03:07:50.573465+0000","last_undegraded":"2023-10-06T03:07:50.573465+0000","last_fullsized":"2023-10-06T03:07:50.573465+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:20.278853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572529+0000","last_change":"2023-10-06T03:07:50.567007+0000","last_active":"2023-10-06T03:07:50.572529+0000","last_peered":"2023-10-06T03:07:50.572529+0000","last_clean":"2023-10-06T03:07:50.572529+0000","last_became_active":"2023-10-06T03:07:50.566733+0000","last_became_peered":"2023-10-06T03:07:50.566733+0000","last_unstale":"2023-10-06T03:07:50.572529+0000","last_undegraded":"2023-10-06T03:07:50.572529+0000","last_fullsized":"2023-10-06T03:07:50.572529+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:54.517224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573062+0000","last_change":"2023-10-06T03:07:49.568215+0000","last_active":"2023-10-06T03:07:50.573062+0000","last_peered":"2023-10-06T03:07:50.573062+0000","last_clean":"2023-10-06T03:07:50.573062+0000","last_became_active":"2023-10-06T03:07:49.567900+0000","last_became_peered":"2023-10-06T03:07:49.567900+0000","last_unstale":"2023-10-06T03:07:50.573062+0000","last_undegraded":"2023-10-06T03:07:50.573062+0000","last_fullsized":"2023-10-06T03:07:50.573062+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:25.296585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577913+0000","last_change":"2023-10-06T03:07:50.573487+0000","last_active":"2023-10-06T03:07:50.577913+0000","last_peered":"2023-10-06T03:07:50.577913+0000","last_clean":"2023-10-06T03:07:50.577913+0000","last_became_active":"2023-10-06T03:07:50.573233+0000","last_became_peered":"2023-10-06T03:07:50.573233+0000","last_unstale":"2023-10-06T03:07:50.577913+0000","last_undegraded":"2023-10-06T03:07:50.577913+0000","last_fullsized":"2023-10-06T03:07:50.577913+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:18:39.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.582880+0000","last_change":"2023-10-06T03:07:49.561326+0000","last_active":"2023-10-06T03:07:50.582880+0000","last_peered":"2023-10-06T03:07:50.582880+0000","last_clean":"2023-10-06T03:07:50.582880+0000","last_became_active":"2023-10-06T03:07:49.561103+0000","last_became_peered":"2023-10-06T03:07:49.561103+0000","last_unstale":"2023-10-06T03:07:50.582880+0000","last_undegraded":"2023-10-06T03:07:50.582880+0000","last_fullsized":"2023-10-06T03:07:50.582880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:25.765674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576162+0000","last_change":"2023-10-06T03:07:50.571130+0000","last_active":"2023-10-06T03:07:50.576162+0000","last_peered":"2023-10-06T03:07:50.576162+0000","last_clean":"2023-10-06T03:07:50.576162+0000","last_became_active":"2023-10-06T03:07:50.570902+0000","last_became_peered":"2023-10-06T03:07:50.570902+0000","last_unstale":"2023-10-06T03:07:50.576162+0000","last_undegraded":"2023-10-06T03:07:50.576162+0000","last_fullsized":"2023-10-06T03:07:50.576162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:22.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572082+0000","last_change":"2023-10-06T03:07:49.564406+0000","last_active":"2023-10-06T03:07:50.572082+0000","last_peered":"2023-10-06T03:07:50.572082+0000","last_clean":"2023-10-06T03:07:50.572082+0000","last_became_active":"2023-10-06T03:07:49.564128+0000","last_became_peered":"2023-10-06T03:07:49.564128+0000","last_unstale":"2023-10-06T03:07:50.572082+0000","last_undegraded":"2023-10-06T03:07:50.572082+0000","last_fullsized":"2023-10-06T03:07:50.572082+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:49.009245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.568929+0000","last_change":"2023-10-06T03:07:50.565122+0000","last_active":"2023-10-06T03:07:50.568929+0000","last_peered":"2023-10-06T03:07:50.568929+0000","last_clean":"2023-10-06T03:07:50.568929+0000","last_became_active":"2023-10-06T03:07:50.564842+0000","last_became_peered":"2023-10-06T03:07:50.564842+0000","last_unstale":"2023-10-06T03:07:50.568929+0000","last_undegraded":"2023-10-06T03:07:50.568929+0000","last_fullsized":"2023-10-06T03:07:50.568929+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:20.294921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.579362+0000","last_change":"2023-10-06T03:07:49.564123+0000","last_active":"2023-10-06T03:07:50.579362+0000","last_peered":"2023-10-06T03:07:50.579362+0000","last_clean":"2023-10-06T03:07:50.579362+0000","last_became_active":"2023-10-06T03:07:49.563157+0000","last_became_peered":"2023-10-06T03:07:49.563157+0000","last_unstale":"2023-10-06T03:07:50.579362+0000","last_undegraded":"2023-10-06T03:07:50.579362+0000","last_fullsized":"2023-10-06T03:07:50.579362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:51.635789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584725+0000","last_change":"2023-10-06T03:07:50.578869+0000","last_active":"2023-10-06T03:07:50.584725+0000","last_peered":"2023-10-06T03:07:50.584725+0000","last_clean":"2023-10-06T03:07:50.584725+0000","last_became_active":"2023-10-06T03:07:50.578111+0000","last_became_peered":"2023-10-06T03:07:50.578111+0000","last_unstale":"2023-10-06T03:07:50.584725+0000","last_undegraded":"2023-10-06T03:07:50.584725+0000","last_fullsized":"2023-10-06T03:07:50.584725+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:18:39.317449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.554122+0000","last_change":"2023-10-06T03:07:49.570378+0000","last_active":"2023-10-06T03:07:50.554122+0000","last_peered":"2023-10-06T03:07:50.554122+0000","last_clean":"2023-10-06T03:07:50.554122+0000","last_became_active":"2023-10-06T03:07:49.570112+0000","last_became_peered":"2023-10-06T03:07:49.570112+0000","last_unstale":"2023-10-06T03:07:50.554122+0000","last_undegraded":"2023-10-06T03:07:50.554122+0000","last_fullsized":"2023-10-06T03:07:50.554122+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:21.074952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581873+0000","last_change":"2023-10-06T03:07:50.577006+0000","last_active":"2023-10-06T03:07:50.581873+0000","last_peered":"2023-10-06T03:07:50.581873+0000","last_clean":"2023-10-06T03:07:50.581873+0000","last_became_active":"2023-10-06T03:07:50.576746+0000","last_became_peered":"2023-10-06T03:07:50.576746+0000","last_unstale":"2023-10-06T03:07:50.581873+0000","last_undegraded":"2023-10-06T03:07:50.581873+0000","last_fullsized":"2023-10-06T03:07:50.581873+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:58.698362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552740+0000","last_change":"2023-10-06T03:07:49.563228+0000","last_active":"2023-10-06T03:07:50.552740+0000","last_peered":"2023-10-06T03:07:50.552740+0000","last_clean":"2023-10-06T03:07:50.552740+0000","last_became_active":"2023-10-06T03:07:49.562933+0000","last_became_peered":"2023-10-06T03:07:49.562933+0000","last_unstale":"2023-10-06T03:07:50.552740+0000","last_undegraded":"2023-10-06T03:07:50.552740+0000","last_fullsized":"2023-10-06T03:07:50.552740+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:19.290197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577687+0000","last_change":"2023-10-06T03:07:50.571814+0000","last_active":"2023-10-06T03:07:50.577687+0000","last_peered":"2023-10-06T03:07:50.577687+0000","last_clean":"2023-10-06T03:07:50.577687+0000","last_became_active":"2023-10-06T03:07:50.571432+0000","last_became_peered":"2023-10-06T03:07:50.571432+0000","last_unstale":"2023-10-06T03:07:50.577687+0000","last_undegraded":"2023-10-06T03:07:50.577687+0000","last_fullsized":"2023-10-06T03:07:50.577687+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:50.545466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574836+0000","last_change":"2023-10-06T03:07:49.568577+0000","last_active":"2023-10-06T03:07:50.574836+0000","last_peered":"2023-10-06T03:07:50.574836+0000","last_clean":"2023-10-06T03:07:50.574836+0000","last_became_active":"2023-10-06T03:07:49.568351+0000","last_became_peered":"2023-10-06T03:07:49.568351+0000","last_unstale":"2023-10-06T03:07:50.574836+0000","last_undegraded":"2023-10-06T03:07:50.574836+0000","last_fullsized":"2023-10-06T03:07:50.574836+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:59.860977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587789+0000","last_change":"2023-10-06T03:07:50.583381+0000","last_active":"2023-10-06T03:07:50.587789+0000","last_peered":"2023-10-06T03:07:50.587789+0000","last_clean":"2023-10-06T03:07:50.587789+0000","last_became_active":"2023-10-06T03:07:50.583119+0000","last_became_peered":"2023-10-06T03:07:50.583119+0000","last_unstale":"2023-10-06T03:07:50.587789+0000","last_undegraded":"2023-10-06T03:07:50.587789+0000","last_fullsized":"2023-10-06T03:07:50.587789+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:07.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575307+0000","last_change":"2023-10-06T03:07:49.567393+0000","last_active":"2023-10-06T03:07:50.575307+0000","last_peered":"2023-10-06T03:07:50.575307+0000","last_clean":"2023-10-06T03:07:50.575307+0000","last_became_active":"2023-10-06T03:07:49.567129+0000","last_became_peered":"2023-10-06T03:07:49.567129+0000","last_unstale":"2023-10-06T03:07:50.575307+0000","last_undegraded":"2023-10-06T03:07:50.575307+0000","last_fullsized":"2023-10-06T03:07:50.575307+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:41.761455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573801+0000","last_change":"2023-10-06T03:07:50.565621+0000","last_active":"2023-10-06T03:07:50.573801+0000","last_peered":"2023-10-06T03:07:50.573801+0000","last_clean":"2023-10-06T03:07:50.573801+0000","last_became_active":"2023-10-06T03:07:50.565348+0000","last_became_peered":"2023-10-06T03:07:50.565348+0000","last_unstale":"2023-10-06T03:07:50.573801+0000","last_undegraded":"2023-10-06T03:07:50.573801+0000","last_fullsized":"2023-10-06T03:07:50.573801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:46.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"102'1","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573173+0000","last_change":"2023-10-06T03:07:49.574956+0000","last_active":"2023-10-06T03:07:50.573173+0000","last_peered":"2023-10-06T03:07:50.573173+0000","last_clean":"2023-10-06T03:07:50.573173+0000","last_became_active":"2023-10-06T03:07:49.574691+0000","last_became_peered":"2023-10-06T03:07:49.574691+0000","last_unstale":"2023-10-06T03:07:50.573173+0000","last_undegraded":"2023-10-06T03:07:50.573173+0000","last_fullsized":"2023-10-06T03:07:50.573173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:15.874825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571996+0000","last_change":"2023-10-06T03:07:50.565193+0000","last_active":"2023-10-06T03:07:50.571996+0000","last_peered":"2023-10-06T03:07:50.571996+0000","last_clean":"2023-10-06T03:07:50.571996+0000","last_became_active":"2023-10-06T03:07:50.564949+0000","last_became_peered":"2023-10-06T03:07:50.564949+0000","last_unstale":"2023-10-06T03:07:50.571996+0000","last_undegraded":"2023-10-06T03:07:50.571996+0000","last_fullsized":"2023-10-06T03:07:50.571996+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:07.731428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562713+0000","last_change":"2023-10-06T03:07:49.571624+0000","last_active":"2023-10-06T03:07:50.562713+0000","last_peered":"2023-10-06T03:07:50.562713+0000","last_clean":"2023-10-06T03:07:50.562713+0000","last_became_active":"2023-10-06T03:07:49.571371+0000","last_became_peered":"2023-10-06T03:07:49.571371+0000","last_unstale":"2023-10-06T03:07:50.562713+0000","last_undegraded":"2023-10-06T03:07:50.562713+0000","last_fullsized":"2023-10-06T03:07:50.562713+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:43.573379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570121+0000","last_change":"2023-10-06T03:07:50.559354+0000","last_active":"2023-10-06T03:07:50.570121+0000","last_peered":"2023-10-06T03:07:50.570121+0000","last_clean":"2023-10-06T03:07:50.570121+0000","last_became_active":"2023-10-06T03:07:50.559074+0000","last_became_peered":"2023-10-06T03:07:50.559074+0000","last_unstale":"2023-10-06T03:07:50.570121+0000","last_undegraded":"2023-10-06T03:07:50.570121+0000","last_fullsized":"2023-10-06T03:07:50.570121+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:31.306909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574667+0000","last_change":"2023-10-06T03:07:49.567927+0000","last_active":"2023-10-06T03:07:50.574667+0000","last_peered":"2023-10-06T03:07:50.574667+0000","last_clean":"2023-10-06T03:07:50.574667+0000","last_became_active":"2023-10-06T03:07:49.567651+0000","last_became_peered":"2023-10-06T03:07:49.567651+0000","last_unstale":"2023-10-06T03:07:50.574667+0000","last_undegraded":"2023-10-06T03:07:50.574667+0000","last_fullsized":"2023-10-06T03:07:50.574667+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:41.894694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584764+0000","last_change":"2023-10-06T03:07:50.579620+0000","last_active":"2023-10-06T03:07:50.584764+0000","last_peered":"2023-10-06T03:07:50.584764+0000","last_clean":"2023-10-06T03:07:50.584764+0000","last_became_active":"2023-10-06T03:07:50.579374+0000","last_became_peered":"2023-10-06T03:07:50.579374+0000","last_unstale":"2023-10-06T03:07:50.584764+0000","last_undegraded":"2023-10-06T03:07:50.584764+0000","last_fullsized":"2023-10-06T03:07:50.584764+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:17.017455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.561899+0000","last_change":"2023-10-06T03:07:49.566472+0000","last_active":"2023-10-06T03:07:50.561899+0000","last_peered":"2023-10-06T03:07:50.561899+0000","last_clean":"2023-10-06T03:07:50.561899+0000","last_became_active":"2023-10-06T03:07:49.566231+0000","last_became_peered":"2023-10-06T03:07:49.566231+0000","last_unstale":"2023-10-06T03:07:50.561899+0000","last_undegraded":"2023-10-06T03:07:50.561899+0000","last_fullsized":"2023-10-06T03:07:50.561899+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:49.540743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584709+0000","last_change":"2023-10-06T03:07:50.578349+0000","last_active":"2023-10-06T03:07:50.584709+0000","last_peered":"2023-10-06T03:07:50.584709+0000","last_clean":"2023-10-06T03:07:50.584709+0000","last_became_active":"2023-10-06T03:07:50.576833+0000","last_became_peered":"2023-10-06T03:07:50.576833+0000","last_unstale":"2023-10-06T03:07:50.584709+0000","last_undegraded":"2023-10-06T03:07:50.584709+0000","last_fullsized":"2023-10-06T03:07:50.584709+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:07.731428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570946+0000","last_change":"2023-10-06T03:07:49.565372+0000","last_active":"2023-10-06T03:07:50.570946+0000","last_peered":"2023-10-06T03:07:50.570946+0000","last_clean":"2023-10-06T03:07:50.570946+0000","last_became_active":"2023-10-06T03:07:49.565082+0000","last_became_peered":"2023-10-06T03:07:49.565082+0000","last_unstale":"2023-10-06T03:07:50.570946+0000","last_undegraded":"2023-10-06T03:07:50.570946+0000","last_fullsized":"2023-10-06T03:07:50.570946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:21.074952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572382+0000","last_change":"2023-10-06T03:07:50.566374+0000","last_active":"2023-10-06T03:07:50.572382+0000","last_peered":"2023-10-06T03:07:50.572382+0000","last_clean":"2023-10-06T03:07:50.572382+0000","last_became_active":"2023-10-06T03:07:50.565949+0000","last_became_peered":"2023-10-06T03:07:50.565949+0000","last_unstale":"2023-10-06T03:07:50.572382+0000","last_undegraded":"2023-10-06T03:07:50.572382+0000","last_fullsized":"2023-10-06T03:07:50.572382+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:18.274017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562047+0000","last_change":"2023-10-06T03:07:49.567446+0000","last_active":"2023-10-06T03:07:50.562047+0000","last_peered":"2023-10-06T03:07:50.562047+0000","last_clean":"2023-10-06T03:07:50.562047+0000","last_became_active":"2023-10-06T03:07:49.567093+0000","last_became_peered":"2023-10-06T03:07:49.567093+0000","last_unstale":"2023-10-06T03:07:50.562047+0000","last_undegraded":"2023-10-06T03:07:50.562047+0000","last_fullsized":"2023-10-06T03:07:50.562047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:45.671026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.574000+0000","last_change":"2023-10-06T03:07:50.568922+0000","last_active":"2023-10-06T03:07:50.574000+0000","last_peered":"2023-10-06T03:07:50.574000+0000","last_clean":"2023-10-06T03:07:50.574000+0000","last_became_active":"2023-10-06T03:07:50.568709+0000","last_became_peered":"2023-10-06T03:07:50.568709+0000","last_unstale":"2023-10-06T03:07:50.574000+0000","last_undegraded":"2023-10-06T03:07:50.574000+0000","last_fullsized":"2023-10-06T03:07:50.574000+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:10.132436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573350+0000","last_change":"2023-10-06T03:07:49.567779+0000","last_active":"2023-10-06T03:07:50.573350+0000","last_peered":"2023-10-06T03:07:50.573350+0000","last_clean":"2023-10-06T03:07:50.573350+0000","last_became_active":"2023-10-06T03:07:49.567579+0000","last_became_peered":"2023-10-06T03:07:49.567579+0000","last_unstale":"2023-10-06T03:07:50.573350+0000","last_undegraded":"2023-10-06T03:07:50.573350+0000","last_fullsized":"2023-10-06T03:07:50.573350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:48.232486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.576201+0000","last_change":"2023-10-06T03:07:50.570770+0000","last_active":"2023-10-06T03:07:50.576201+0000","last_peered":"2023-10-06T03:07:50.576201+0000","last_clean":"2023-10-06T03:07:50.576201+0000","last_became_active":"2023-10-06T03:07:50.570500+0000","last_became_peered":"2023-10-06T03:07:50.570500+0000","last_unstale":"2023-10-06T03:07:50.576201+0000","last_undegraded":"2023-10-06T03:07:50.576201+0000","last_fullsized":"2023-10-06T03:07:50.576201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:02.621345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-06T03:07:52.739762+0000","last_change":"2023-10-06T03:07:49.567546+0000","last_active":"2023-10-06T03:07:52.739762+0000","last_peered":"2023-10-06T03:07:52.739762+0000","last_clean":"2023-10-06T03:07:52.739762+0000","last_became_active":"2023-10-06T03:07:49.567274+0000","last_became_peered":"2023-10-06T03:07:49.567274+0000","last_unstale":"2023-10-06T03:07:52.739762+0000","last_undegraded":"2023-10-06T03:07:52.739762+0000","last_fullsized":"2023-10-06T03:07:52.739762+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:39:50.664304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573164+0000","last_change":"2023-10-06T03:07:50.570896+0000","last_active":"2023-10-06T03:07:50.573164+0000","last_peered":"2023-10-06T03:07:50.573164+0000","last_clean":"2023-10-06T03:07:50.573164+0000","last_became_active":"2023-10-06T03:07:50.570605+0000","last_became_peered":"2023-10-06T03:07:50.570605+0000","last_unstale":"2023-10-06T03:07:50.573164+0000","last_undegraded":"2023-10-06T03:07:50.573164+0000","last_fullsized":"2023-10-06T03:07:50.573164+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:54:07.412606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.558154+0000","last_change":"2023-10-06T03:07:49.571191+0000","last_active":"2023-10-06T03:07:50.558154+0000","last_peered":"2023-10-06T03:07:50.558154+0000","last_clean":"2023-10-06T03:07:50.558154+0000","last_became_active":"2023-10-06T03:07:49.570908+0000","last_became_peered":"2023-10-06T03:07:49.570908+0000","last_unstale":"2023-10-06T03:07:50.558154+0000","last_undegraded":"2023-10-06T03:07:50.558154+0000","last_fullsized":"2023-10-06T03:07:50.558154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:28.358454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581923+0000","last_change":"2023-10-06T03:07:50.573868+0000","last_active":"2023-10-06T03:07:50.581923+0000","last_peered":"2023-10-06T03:07:50.581923+0000","last_clean":"2023-10-06T03:07:50.581923+0000","last_became_active":"2023-10-06T03:07:50.572772+0000","last_became_peered":"2023-10-06T03:07:50.572772+0000","last_unstale":"2023-10-06T03:07:50.581923+0000","last_undegraded":"2023-10-06T03:07:50.581923+0000","last_fullsized":"2023-10-06T03:07:50.581923+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:45.966304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552674+0000","last_change":"2023-10-06T03:07:49.567138+0000","last_active":"2023-10-06T03:07:50.552674+0000","last_peered":"2023-10-06T03:07:50.552674+0000","last_clean":"2023-10-06T03:07:50.552674+0000","last_became_active":"2023-10-06T03:07:49.566884+0000","last_became_peered":"2023-10-06T03:07:49.566884+0000","last_unstale":"2023-10-06T03:07:50.552674+0000","last_undegraded":"2023-10-06T03:07:50.552674+0000","last_fullsized":"2023-10-06T03:07:50.552674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:24.299865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573641+0000","last_change":"2023-10-06T03:07:50.565737+0000","last_active":"2023-10-06T03:07:50.573641+0000","last_peered":"2023-10-06T03:07:50.573641+0000","last_clean":"2023-10-06T03:07:50.573641+0000","last_became_active":"2023-10-06T03:07:50.565477+0000","last_became_peered":"2023-10-06T03:07:50.565477+0000","last_unstale":"2023-10-06T03:07:50.573641+0000","last_undegraded":"2023-10-06T03:07:50.573641+0000","last_fullsized":"2023-10-06T03:07:50.573641+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:29.710606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.553258+0000","last_change":"2023-10-06T03:07:49.565800+0000","last_active":"2023-10-06T03:07:50.553258+0000","last_peered":"2023-10-06T03:07:50.553258+0000","last_clean":"2023-10-06T03:07:50.553258+0000","last_became_active":"2023-10-06T03:07:49.565480+0000","last_became_peered":"2023-10-06T03:07:49.565480+0000","last_unstale":"2023-10-06T03:07:50.553258+0000","last_undegraded":"2023-10-06T03:07:50.553258+0000","last_fullsized":"2023-10-06T03:07:50.553258+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:06.726705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584782+0000","last_change":"2023-10-06T03:07:50.579751+0000","last_active":"2023-10-06T03:07:50.584782+0000","last_peered":"2023-10-06T03:07:50.584782+0000","last_clean":"2023-10-06T03:07:50.584782+0000","last_became_active":"2023-10-06T03:07:50.579393+0000","last_became_peered":"2023-10-06T03:07:50.579393+0000","last_unstale":"2023-10-06T03:07:50.584782+0000","last_undegraded":"2023-10-06T03:07:50.584782+0000","last_fullsized":"2023-10-06T03:07:50.584782+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:18.274017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552787+0000","last_change":"2023-10-06T03:07:49.574874+0000","last_active":"2023-10-06T03:07:50.552787+0000","last_peered":"2023-10-06T03:07:50.552787+0000","last_clean":"2023-10-06T03:07:50.552787+0000","last_became_active":"2023-10-06T03:07:49.574539+0000","last_became_peered":"2023-10-06T03:07:49.574539+0000","last_unstale":"2023-10-06T03:07:50.552787+0000","last_undegraded":"2023-10-06T03:07:50.552787+0000","last_fullsized":"2023-10-06T03:07:50.552787+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:26.959832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.577939+0000","last_change":"2023-10-06T03:07:50.572981+0000","last_active":"2023-10-06T03:07:50.577939+0000","last_peered":"2023-10-06T03:07:50.577939+0000","last_clean":"2023-10-06T03:07:50.577939+0000","last_became_active":"2023-10-06T03:07:50.572653+0000","last_became_peered":"2023-10-06T03:07:50.572653+0000","last_unstale":"2023-10-06T03:07:50.577939+0000","last_undegraded":"2023-10-06T03:07:50.577939+0000","last_fullsized":"2023-10-06T03:07:50.577939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:21:40.777609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552118+0000","last_change":"2023-10-06T03:07:49.563073+0000","last_active":"2023-10-06T03:07:50.552118+0000","last_peered":"2023-10-06T03:07:50.552118+0000","last_clean":"2023-10-06T03:07:50.552118+0000","last_became_active":"2023-10-06T03:07:49.562796+0000","last_became_peered":"2023-10-06T03:07:49.562796+0000","last_unstale":"2023-10-06T03:07:50.552118+0000","last_undegraded":"2023-10-06T03:07:50.552118+0000","last_fullsized":"2023-10-06T03:07:50.552118+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:32.478809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581581+0000","last_change":"2023-10-06T03:07:50.574361+0000","last_active":"2023-10-06T03:07:50.581581+0000","last_peered":"2023-10-06T03:07:50.581581+0000","last_clean":"2023-10-06T03:07:50.581581+0000","last_became_active":"2023-10-06T03:07:50.574196+0000","last_became_peered":"2023-10-06T03:07:50.574196+0000","last_unstale":"2023-10-06T03:07:50.581581+0000","last_undegraded":"2023-10-06T03:07:50.581581+0000","last_fullsized":"2023-10-06T03:07:50.581581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:02.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562144+0000","last_change":"2023-10-06T03:07:49.566063+0000","last_active":"2023-10-06T03:07:50.562144+0000","last_peered":"2023-10-06T03:07:50.562144+0000","last_clean":"2023-10-06T03:07:50.562144+0000","last_became_active":"2023-10-06T03:07:49.565801+0000","last_became_peered":"2023-10-06T03:07:49.565801+0000","last_unstale":"2023-10-06T03:07:50.562144+0000","last_undegraded":"2023-10-06T03:07:50.562144+0000","last_fullsized":"2023-10-06T03:07:50.562144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:18:53.512501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.572719+0000","last_change":"2023-10-06T03:07:50.566817+0000","last_active":"2023-10-06T03:07:50.572719+0000","last_peered":"2023-10-06T03:07:50.572719+0000","last_clean":"2023-10-06T03:07:50.572719+0000","last_became_active":"2023-10-06T03:07:50.566535+0000","last_became_peered":"2023-10-06T03:07:50.566535+0000","last_unstale":"2023-10-06T03:07:50.572719+0000","last_undegraded":"2023-10-06T03:07:50.572719+0000","last_fullsized":"2023-10-06T03:07:50.572719+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:40:27.964556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"102'6","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.589060+0000","last_change":"2023-10-06T03:07:49.562916+0000","last_active":"2023-10-06T03:07:50.589060+0000","last_peered":"2023-10-06T03:07:50.589060+0000","last_clean":"2023-10-06T03:07:50.589060+0000","last_became_active":"2023-10-06T03:07:49.562661+0000","last_became_peered":"2023-10-06T03:07:49.562661+0000","last_unstale":"2023-10-06T03:07:50.589060+0000","last_undegraded":"2023-10-06T03:07:50.589060+0000","last_fullsized":"2023-10-06T03:07:50.589060+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:14.508655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":80,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.556557+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:07:50.556557+0000","last_peered":"2023-10-06T03:07:50.556557+0000","last_clean":"2023-10-06T03:07:50.556557+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:07:50.556557+0000","last_undegraded":"2023-10-06T03:07:50.556557+0000","last_fullsized":"2023-10-06T03:07:50.556557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584647+0000","last_change":"2023-10-06T03:07:50.578854+0000","last_active":"2023-10-06T03:07:50.584647+0000","last_peered":"2023-10-06T03:07:50.584647+0000","last_clean":"2023-10-06T03:07:50.584647+0000","last_became_active":"2023-10-06T03:07:50.578290+0000","last_became_peered":"2023-10-06T03:07:50.578290+0000","last_unstale":"2023-10-06T03:07:50.584647+0000","last_undegraded":"2023-10-06T03:07:50.584647+0000","last_fullsized":"2023-10-06T03:07:50.584647+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:50.013969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"102'3","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.575195+0000","last_change":"2023-10-06T03:07:49.570381+0000","last_active":"2023-10-06T03:07:50.575195+0000","last_peered":"2023-10-06T03:07:50.575195+0000","last_clean":"2023-10-06T03:07:50.575195+0000","last_became_active":"2023-10-06T03:07:49.570123+0000","last_became_peered":"2023-10-06T03:07:49.570123+0000","last_unstale":"2023-10-06T03:07:50.575195+0000","last_undegraded":"2023-10-06T03:07:50.575195+0000","last_fullsized":"2023-10-06T03:07:50.575195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.564578+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:07:50.564578+0000","last_peered":"2023-10-06T03:07:50.564578+0000","last_clean":"2023-10-06T03:07:50.564578+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:07:50.564578+0000","last_undegraded":"2023-10-06T03:07:50.564578+0000","last_fullsized":"2023-10-06T03:07:50.564578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585231+0000","last_change":"2023-10-06T03:07:50.582099+0000","last_active":"2023-10-06T03:07:50.585231+0000","last_peered":"2023-10-06T03:07:50.585231+0000","last_clean":"2023-10-06T03:07:50.585231+0000","last_became_active":"2023-10-06T03:07:50.581860+0000","last_became_peered":"2023-10-06T03:07:50.581860+0000","last_unstale":"2023-10-06T03:07:50.585231+0000","last_undegraded":"2023-10-06T03:07:50.585231+0000","last_fullsized":"2023-10-06T03:07:50.585231+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:05:50.013969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.553486+0000","last_change":"2023-10-06T03:07:49.574358+0000","last_active":"2023-10-06T03:07:50.553486+0000","last_peered":"2023-10-06T03:07:50.553486+0000","last_clean":"2023-10-06T03:07:50.553486+0000","last_became_active":"2023-10-06T03:07:49.574001+0000","last_became_peered":"2023-10-06T03:07:49.574001+0000","last_unstale":"2023-10-06T03:07:50.553486+0000","last_undegraded":"2023-10-06T03:07:50.553486+0000","last_fullsized":"2023-10-06T03:07:50.553486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:11:14.508655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.567674+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:07:50.567674+0000","last_peered":"2023-10-06T03:07:50.567674+0000","last_clean":"2023-10-06T03:07:50.567674+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:07:50.567674+0000","last_undegraded":"2023-10-06T03:07:50.567674+0000","last_fullsized":"2023-10-06T03:07:50.567674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.585304+0000","last_change":"2023-10-06T03:07:50.581005+0000","last_active":"2023-10-06T03:07:50.585304+0000","last_peered":"2023-10-06T03:07:50.585304+0000","last_clean":"2023-10-06T03:07:50.585304+0000","last_became_active":"2023-10-06T03:07:50.580494+0000","last_became_peered":"2023-10-06T03:07:50.580494+0000","last_unstale":"2023-10-06T03:07:50.585304+0000","last_undegraded":"2023-10-06T03:07:50.585304+0000","last_fullsized":"2023-10-06T03:07:50.585304+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:57.318520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"102'2","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.584485+0000","last_change":"2023-10-06T03:07:49.566406+0000","last_active":"2023-10-06T03:07:50.584485+0000","last_peered":"2023-10-06T03:07:50.584485+0000","last_clean":"2023-10-06T03:07:50.584485+0000","last_became_active":"2023-10-06T03:07:49.566148+0000","last_became_peered":"2023-10-06T03:07:49.566148+0000","last_unstale":"2023-10-06T03:07:50.584485+0000","last_undegraded":"2023-10-06T03:07:50.584485+0000","last_fullsized":"2023-10-06T03:07:50.584485+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:57.693638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562238+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:07:50.562238+0000","last_peered":"2023-10-06T03:07:50.562238+0000","last_clean":"2023-10-06T03:07:50.562238+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:07:50.562238+0000","last_undegraded":"2023-10-06T03:07:50.562238+0000","last_fullsized":"2023-10-06T03:07:50.562238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.578008+0000","last_change":"2023-10-06T03:07:50.574780+0000","last_active":"2023-10-06T03:07:50.578008+0000","last_peered":"2023-10-06T03:07:50.578008+0000","last_clean":"2023-10-06T03:07:50.578008+0000","last_became_active":"2023-10-06T03:07:50.574322+0000","last_became_peered":"2023-10-06T03:07:50.574322+0000","last_unstale":"2023-10-06T03:07:50.578008+0000","last_undegraded":"2023-10-06T03:07:50.578008+0000","last_fullsized":"2023-10-06T03:07:50.578008+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:02.680440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"102'8","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.587760+0000","last_change":"2023-10-06T03:07:49.562128+0000","last_active":"2023-10-06T03:07:50.587760+0000","last_peered":"2023-10-06T03:07:50.587760+0000","last_clean":"2023-10-06T03:07:50.587760+0000","last_became_active":"2023-10-06T03:07:49.561879+0000","last_became_peered":"2023-10-06T03:07:49.561879+0000","last_unstale":"2023-10-06T03:07:50.587760+0000","last_undegraded":"2023-10-06T03:07:50.587760+0000","last_fullsized":"2023-10-06T03:07:50.587760+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:56.313797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562418+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:07:50.562418+0000","last_peered":"2023-10-06T03:07:50.562418+0000","last_clean":"2023-10-06T03:07:50.562418+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:07:50.562418+0000","last_undegraded":"2023-10-06T03:07:50.562418+0000","last_fullsized":"2023-10-06T03:07:50.562418+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.552654+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:07:50.552654+0000","last_peered":"2023-10-06T03:07:50.552654+0000","last_clean":"2023-10-06T03:07:50.552654+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:07:50.552654+0000","last_undegraded":"2023-10-06T03:07:50.552654+0000","last_fullsized":"2023-10-06T03:07:50.552654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.581699+0000","last_change":"2023-10-06T03:07:50.573994+0000","last_active":"2023-10-06T03:07:50.581699+0000","last_peered":"2023-10-06T03:07:50.581699+0000","last_clean":"2023-10-06T03:07:50.581699+0000","last_became_active":"2023-10-06T03:07:50.573801+0000","last_became_peered":"2023-10-06T03:07:50.573801+0000","last_unstale":"2023-10-06T03:07:50.581699+0000","last_undegraded":"2023-10-06T03:07:50.581699+0000","last_fullsized":"2023-10-06T03:07:50.581699+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:22.069683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571236+0000","last_change":"2023-10-06T03:07:49.574564+0000","last_active":"2023-10-06T03:07:50.571236+0000","last_peered":"2023-10-06T03:07:50.571236+0000","last_clean":"2023-10-06T03:07:50.571236+0000","last_became_active":"2023-10-06T03:07:49.574290+0000","last_became_peered":"2023-10-06T03:07:49.574290+0000","last_unstale":"2023-10-06T03:07:50.571236+0000","last_undegraded":"2023-10-06T03:07:50.571236+0000","last_fullsized":"2023-10-06T03:07:50.571236+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:10.135570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.562557+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:07:50.562557+0000","last_peered":"2023-10-06T03:07:50.562557+0000","last_clean":"2023-10-06T03:07:50.562557+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:07:50.562557+0000","last_undegraded":"2023-10-06T03:07:50.562557+0000","last_fullsized":"2023-10-06T03:07:50.562557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.573132+0000","last_change":"2023-10-06T03:07:50.568705+0000","last_active":"2023-10-06T03:07:50.573132+0000","last_peered":"2023-10-06T03:07:50.573132+0000","last_clean":"2023-10-06T03:07:50.573132+0000","last_became_active":"2023-10-06T03:07:50.568454+0000","last_became_peered":"2023-10-06T03:07:50.568454+0000","last_unstale":"2023-10-06T03:07:50.573132+0000","last_undegraded":"2023-10-06T03:07:50.573132+0000","last_fullsized":"2023-10-06T03:07:50.573132+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:51:49.459109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.570260+0000","last_change":"2023-10-06T03:07:49.563167+0000","last_active":"2023-10-06T03:07:50.570260+0000","last_peered":"2023-10-06T03:07:50.570260+0000","last_clean":"2023-10-06T03:07:50.570260+0000","last_became_active":"2023-10-06T03:07:49.562680+0000","last_became_peered":"2023-10-06T03:07:49.562680+0000","last_unstale":"2023-10-06T03:07:50.570260+0000","last_undegraded":"2023-10-06T03:07:50.570260+0000","last_fullsized":"2023-10-06T03:07:50.570260+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:15:01.616622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":88,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.554084+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:07:50.554084+0000","last_peered":"2023-10-06T03:07:50.554084+0000","last_clean":"2023-10-06T03:07:50.554084+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:07:50.554084+0000","last_undegraded":"2023-10-06T03:07:50.554084+0000","last_fullsized":"2023-10-06T03:07:50.554084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.571441+0000","last_change":"2023-10-06T03:07:50.565348+0000","last_active":"2023-10-06T03:07:50.571441+0000","last_peered":"2023-10-06T03:07:50.571441+0000","last_clean":"2023-10-06T03:07:50.571441+0000","last_became_active":"2023-10-06T03:07:50.565091+0000","last_became_peered":"2023-10-06T03:07:50.565091+0000","last_unstale":"2023-10-06T03:07:50.571441+0000","last_undegraded":"2023-10-06T03:07:50.571441+0000","last_fullsized":"2023-10-06T03:07:50.571441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:49.543624+0000","last_clean_scrub_stamp":"2023-10-06T03:07:49.543624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:02.536830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.556502+0000","last_change":"2023-10-06T03:07:49.568754+0000","last_active":"2023-10-06T03:07:50.556502+0000","last_peered":"2023-10-06T03:07:50.556502+0000","last_clean":"2023-10-06T03:07:50.556502+0000","last_became_active":"2023-10-06T03:07:49.568471+0000","last_became_peered":"2023-10-06T03:07:49.568471+0000","last_unstale":"2023-10-06T03:07:50.556502+0000","last_undegraded":"2023-10-06T03:07:50.556502+0000","last_fullsized":"2023-10-06T03:07:50.556502+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:07:48.538901+0000","last_clean_scrub_stamp":"2023-10-06T03:07:48.538901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.866787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-06T03:07:50.565111+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:07:50.565111+0000","last_peered":"2023-10-06T03:07:50.565111+0000","last_clean":"2023-10-06T03:07:50.565111+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:07:50.565111+0000","last_undegraded":"2023-10-06T03:07:50.565111+0000","last_fullsized":"2023-10-06T03:07:50.565111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303232,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068608,"statfs":{"total":96636764160,"available":96326254592,"internally_reserved":0,"allocated":1892352,"data_stored":1177182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.65400000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.629}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302640,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069200,"statfs":{"total":96636764160,"available":96326860800,"internally_reserved":0,"allocated":1286144,"data_stored":586620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.624}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.498},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":586088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.32400000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.26100000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.65100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303220,"kb_used_data":1836,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068620,"statfs":{"total":96636764160,"available":96326266880,"internally_reserved":0,"allocated":1880064,"data_stored":1176456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.252},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.28899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1282048,"data_stored":586066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.249},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27800000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.31}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.624},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53900000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298556,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073284,"statfs":{"total":96636764160,"available":96331042816,"internally_reserved":0,"allocated":1298432,"data_stored":587636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34499999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.309},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.38},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302648,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069192,"statfs":{"total":96636764160,"available":96326852608,"internally_reserved":0,"allocated":1294336,"data_stored":587602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.626}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298548,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073292,"statfs":{"total":96636764160,"available":96331051008,"internally_reserved":0,"allocated":1290240,"data_stored":586226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.433},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.50700000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.496}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:07:59.898 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:07:59.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:08:00.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 Processor -- start 2023-10-06T03:08:00.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 -- start start 2023-10-06T03:08:00.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f507c159950 0x7f507c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c0af320 0x7f507c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c0ce380 con 0x7f507c069aa0 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507c069f20 con 0x7f507c159950 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.050+0000 7f5081637700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0cec20 con 0x7f507c0af320 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f507c159950 0x7f507c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f507c159950 0x7f507c0cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38294/0 (socket says 172.21.15.92:38294) 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 -- 172.21.15.92:0/96324446 learned_addr learned my addr 172.21.15.92:0/96324446 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f50727fc700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c0af320 0x7f507c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 -- 172.21.15.92:0/96324446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c0af320 msgr2=0x7f507c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c0af320 0x7f507c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 -- 172.21.15.92:0/96324446 >> v1:172.21.15.136:6789/0 conn(0x7f507c069aa0 legacy=0x7f507c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 -- 172.21.15.92:0/96324446 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c0afa90 con 0x7f507c159950 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f50727fc700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c0af320 0x7f507c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507affd700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c159950 0x7f507c0cbb90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f506c0099b0 tx=0x7f506c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b2fb5a6a5c4a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507a7fc700 1 -- 172.21.15.92:0/96324446 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00b860 con 0x7f507c159950 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507a7fc700 1 -- 172.21.15.92:0/96324446 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c00b9c0 con 0x7f507c159950 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.051+0000 7f507a7fc700 1 -- 172.21.15.92:0/96324446 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00bd60 con 0x7f507c159950 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 -- 172.21.15.92:0/96324446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c159950 msgr2=0x7f507c0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c159950 0x7f507c0cbb90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f506c0099b0 tx=0x7f506c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 -- 172.21.15.92:0/96324446 shutdown_connections 2023-10-06T03:08:00.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c159950 0x7f507c0cbb90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 --2- 172.21.15.92:0/96324446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c0af320 0x7f507c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 -- 172.21.15.92:0/96324446 >> 172.21.15.92:0/96324446 conn(0x7f507c068e00 msgr2=0x7f507c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:00.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 -- 172.21.15.92:0/96324446 shutdown_connections 2023-10-06T03:08:00.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.052+0000 7f5081637700 1 -- 172.21.15.92:0/96324446 wait complete. 2023-10-06T03:08:00.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.053+0000 7f5081637700 1 Processor -- start 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.053+0000 7f5081637700 1 -- start start 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 0x7f507c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f507c0af320 0x7f507c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c159950 0x7f507c0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0b6530 con 0x7f507c069aa0 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c169ea0 con 0x7f507c0af320 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c16a9d0 con 0x7f507c159950 2023-10-06T03:08:00.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 0x7f507c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 0x7f507c0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38308/0 (socket says 172.21.15.92:38308) 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 -- 172.21.15.92:0/3219152846 learned_addr learned my addr 172.21.15.92:0/3219152846 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f50727fc700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f507c0af320 0x7f507c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507b7fe700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c159950 0x7f507c0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 -- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c159950 msgr2=0x7f507c0b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c159950 0x7f507c0b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 -- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f507c0af320 msgr2=0x7f507c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f507c0af320 0x7f507c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 -- 172.21.15.92:0/3219152846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c0094f0 con 0x7f507c069aa0 2023-10-06T03:08:00.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f50727fc700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f507c0af320 0x7f507c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f507affd700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 0x7f507c0b2040 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f506c009290 tx=0x7f506c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c01a450 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c01a5d0 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c01a990 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c16ae60 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.054+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c16a380 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.055+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c0718b0 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.056+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f506c01ad00 con 0x7f507c069aa0 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.056+0000 7f50737fe700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f505c067290 0x7f505c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.056+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f506c0861d0 con 0x7f507c069aa0 2023-10-06T03:08:00.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.056+0000 7f50727fc700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f505c067290 0x7f505c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.057+0000 7f50727fc700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f505c067290 0x7f505c069740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5064006f00 tx=0x7f5064008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:00.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.060+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f506c052ed0 con 0x7f507c069aa0 2023-10-06T03:08:00.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.319+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f507c16acb0 con 0x7f507c069aa0 2023-10-06T03:08:00.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.322+0000 7f50737fe700 1 -- 172.21.15.92:0/3219152846 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f506c0564f0 con 0x7f507c069aa0 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f505c067290 msgr2=0x7f505c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f505c067290 0x7f505c069740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5064006f00 tx=0x7f5064008040 comp rx=0 tx=0).stop 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 msgr2=0x7f507c0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 0x7f507c0b2040 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f506c009290 tx=0x7f506c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 shutdown_connections 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f505c067290 0x7f505c069740 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f507c069aa0 0x7f507c0b2040 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f507c159950 0x7f507c0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 --2- 172.21.15.92:0/3219152846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f507c0af320 0x7f507c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 >> 172.21.15.92:0/3219152846 conn(0x7f507c068e00 msgr2=0x7f507c0b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:00.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.324+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 shutdown_connections 2023-10-06T03:08:00.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.325+0000 7f5081637700 1 -- 172.21.15.92:0/3219152846 wait complete. 2023-10-06T03:08:00.328 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:08:00.341 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:00.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.454+0000 7f1af40d0700 1 Processor -- start 2023-10-06T03:08:00.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.454+0000 7f1af40d0700 1 -- start start 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af40d0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1aec159890 0x7f1aec15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af40d0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 0x7f1aec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af40d0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec069ee0 con 0x7f1aec159c60 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af40d0700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec15a160 con 0x7f1aec159890 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af40d0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0d13a0 con 0x7f1aec069560 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af166b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1aec159c60 0x7f1aec15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35058/0 (socket says 172.21.15.92:35058) 2023-10-06T03:08:00.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.455+0000 7f1af166b700 1 -- 172.21.15.92:0/1605298992 learned_addr learned my addr 172.21.15.92:0/1605298992 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666589550 0 0) 0x7f1aec069ee0 con 0x7f1aec159c60 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad8003570 con 0x7f1aec159c60 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af1e6c700 1 --2- 172.21.15.92:0/1605298992 >> v2:172.21.15.136:3300/0 conn(0x7f1aec159890 0x7f1aec15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 0x7f1aec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2285525952 0 0) 0x7f1ad8003570 con 0x7f1aec159c60 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 msgr2=0x7f1aec0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 0x7f1aec0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 >> v2:172.21.15.136:3300/0 conn(0x7f1aec159890 msgr2=0x7f1aec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 --2- 172.21.15.92:0/1605298992 >> v2:172.21.15.136:3300/0 conn(0x7f1aec159890 0x7f1aec15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec15e140 con 0x7f1aec159c60 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f1adc001470 con 0x7f1aec159c60 2023-10-06T03:08:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 >> v1:172.21.15.92:6789/0 conn(0x7f1aec159c60 legacy=0x7f1aec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 0x7f1ad8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ad8004090 0x7f1ad8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1ad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15a160 con 0x7f1ad8004d50 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0d13a0 con 0x7f1ad8004090 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad800bbf0 con 0x7f1aec069560 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1ad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af1e6c700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 0x7f1ad8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af166b700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ad8004090 0x7f1ad8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 -- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 msgr2=0x7f1ad8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec069560 0x7f1ad8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 -- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ad8004090 msgr2=0x7f1ad8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ad8004090 0x7f1ad8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 -- 172.21.15.92:0/1605298992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8003570 con 0x7f1ad8004d50 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.456+0000 7f1af266d700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1ad80051c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400bb70 tx=0x7f1ae4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526c3aa8b54e8959 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400d5d0 con 0x7f1ad8004d50 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae400d730 con 0x7f1ad8004d50 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af0e6a700 1 -- 172.21.15.92:0/1605298992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400daf0 con 0x7f1ad8004d50 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af40d0700 1 -- 172.21.15.92:0/1605298992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 msgr2=0x7f1ad80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af40d0700 1 --2- 172.21.15.92:0/1605298992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1ad80051c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400bb70 tx=0x7f1ae4008040 comp rx=0 tx=0).stop 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af266d700 1 -- 172.21.15.92:0/1605298992 reap_dead start 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af40d0700 1 -- 172.21.15.92:0/1605298992 shutdown_connections 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af40d0700 1 -- 172.21.15.92:0/1605298992 >> 172.21.15.92:0/1605298992 conn(0x7f1aec068e20 msgr2=0x7f1aec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:00.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af40d0700 1 -- 172.21.15.92:0/1605298992 shutdown_connections 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.457+0000 7f1af40d0700 1 -- 172.21.15.92:0/1605298992 wait complete. 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.458+0000 7f1af40d0700 1 Processor -- start 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.458+0000 7f1af40d0700 1 -- start start 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af40d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1aec16ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af40d0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec0b6020 0x7f1aec0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af40d0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1aec15c8c0 0x7f1aec15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af40d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec151eb0 con 0x7f1ad8004d50 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af40d0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0b6490 con 0x7f1aec15c8c0 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af40d0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15d280 con 0x7f1aec0b6020 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af1e6c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1aec16ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af1e6c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1aec16ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38328/0 (socket says 172.21.15.92:38328) 2023-10-06T03:08:00.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af1e6c700 1 -- 172.21.15.92:0/3298298948 learned_addr learned my addr 172.21.15.92:0/3298298948 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.459+0000 7f1af1e6c700 1 -- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec0b6020 msgr2=0x7f1aec0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af1e6c700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec0b6020 0x7f1aec0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af1e6c700 1 -- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1aec15c8c0 msgr2=0x7f1aec15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af266d700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1aec15c8c0 0x7f1aec15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af1e6c700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1aec15c8c0 0x7f1aec15cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af1e6c700 1 -- 172.21.15.92:0/3298298948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec156900 con 0x7f1ad8004d50 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af1e6c700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1aec16ae80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80070b0 tx=0x7f1ae8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8015420 con 0x7f1ad8004d50 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae8015580 con 0x7f1ad8004d50 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec169d20 con 0x7f1ad8004d50 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8015920 con 0x7f1ad8004d50 2023-10-06T03:08:00.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.460+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec16a2e0 con 0x7f1ad8004d50 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.461+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec04c3c0 con 0x7f1ad8004d50 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.461+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1ae8003040 con 0x7f1ad8004d50 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.462+0000 7f1ae1ffb700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ad0067290 0x7f1ad0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.462+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1ae8085800 con 0x7f1ad8004d50 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.462+0000 7f1af166b700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ad0067290 0x7f1ad0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.462+0000 7f1af166b700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ad0067290 0x7f1ad0069740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1adc000e10 tx=0x7f1adc001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:00.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.466+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ae8052500 con 0x7f1ad8004d50 2023-10-06T03:08:00.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.692+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1aec069dd0 con 0x7f1ad8004d50 2023-10-06T03:08:00.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.692+0000 7f1ae1ffb700 1 -- 172.21.15.92:0/3298298948 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f1ae8055b20 con 0x7f1ad8004d50 2023-10-06T03:08:00.694 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:00.695 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:49.932559+0000","modified":"2023-10-06T03:07:53.569016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/1523868194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1523868194},{"type":"v1","addr":"172.21.15.92:6837","nonce":1523868194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:08:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ad0067290 msgr2=0x7f1ad0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ad0067290 0x7f1ad0069740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1adc000e10 tx=0x7f1adc001e90 comp rx=0 tx=0).stop 2023-10-06T03:08:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 msgr2=0x7f1aec16ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:00.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1aec16ae80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1ae80070b0 tx=0x7f1ae8009710 comp rx=0 tx=0).stop 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 shutdown_connections 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ad0067290 0x7f1ad0069740 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ad8004d50 0x7f1aec16ae80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1aec15c8c0 0x7f1aec15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 --2- 172.21.15.92:0/3298298948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1aec0b6020 0x7f1aec0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.694+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 >> 172.21.15.92:0/3298298948 conn(0x7f1aec068e20 msgr2=0x7f1aec0c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.695+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 shutdown_connections 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:00.695+0000 7f1af40d0700 1 -- 172.21.15.92:0/3298298948 wait complete. 2023-10-06T03:08:00.697 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 68 2023-10-06T03:08:01.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:07:49.932559+0000', 'modified': '2023-10-06T03:07:53.569016+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6840}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6840': {'gid': 6840, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6837/1523868194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 1523868194}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 1523868194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:08:01.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:08:01.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:08:01.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.230+0000 7f948759e700 1 Processor -- start 2023-10-06T03:08:01.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.230+0000 7f948759e700 1 -- start start 2023-10-06T03:08:01.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9488159420 0x7f94880b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f948815d230 0x7f948815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94880b2980 con 0x7f94881597f0 2023-10-06T03:08:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948815db70 con 0x7f9488159420 2023-10-06T03:08:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94880b11d0 con 0x7f948815d230 2023-10-06T03:08:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9488159420 0x7f94880b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9488159420 0x7f94880b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38338/0 (socket says 172.21.15.92:38338) 2023-10-06T03:08:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 -- 172.21.15.92:0/3697386410 learned_addr learned my addr 172.21.15.92:0/3697386410 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 -- 172.21.15.92:0/3697386410 >> v2:172.21.15.136:3300/0 conn(0x7f948815d230 msgr2=0x7f948815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f9486d9d700 1 --2- 172.21.15.92:0/3697386410 >> v2:172.21.15.136:3300/0 conn(0x7f948815d230 0x7f948815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 --2- 172.21.15.92:0/3697386410 >> v2:172.21.15.136:3300/0 conn(0x7f948815d230 0x7f948815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 -- 172.21.15.92:0/3697386410 >> v1:172.21.15.136:6789/0 conn(0x7f94881597f0 legacy=0x7f94880b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 -- 172.21.15.92:0/3697386410 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94880b14b0 con 0x7f9488159420 2023-10-06T03:08:01.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948659c700 1 --2- 172.21.15.92:0/3697386410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880b5f40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f948815d170 tx=0x7f9478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df44d688c3b1941c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f9485d9b700 1 -- 172.21.15.92:0/3697386410 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947800b820 con 0x7f9488159420 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f9485d9b700 1 -- 172.21.15.92:0/3697386410 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947800b980 con 0x7f9488159420 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f9485d9b700 1 -- 172.21.15.92:0/3697386410 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947800bd20 con 0x7f9488159420 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- 172.21.15.92:0/3697386410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 msgr2=0x7f94880b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 --2- 172.21.15.92:0/3697386410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880b5f40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f948815d170 tx=0x7f9478009250 comp rx=0 tx=0).stop 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- 172.21.15.92:0/3697386410 shutdown_connections 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 --2- 172.21.15.92:0/3697386410 >> v2:172.21.15.136:3300/0 conn(0x7f948815d230 0x7f948815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 --2- 172.21.15.92:0/3697386410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880b5f40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- 172.21.15.92:0/3697386410 >> 172.21.15.92:0/3697386410 conn(0x7f94880d0660 msgr2=0x7f94880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- 172.21.15.92:0/3697386410 shutdown_connections 2023-10-06T03:08:01.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.232+0000 7f948759e700 1 -- 172.21.15.92:0/3697386410 wait complete. 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.233+0000 7f948759e700 1 Processor -- start 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.233+0000 7f948759e700 1 -- start start 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94881597f0 0x7f94880ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948815d230 0x7f94880ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94880cb730 con 0x7f9488159420 2023-10-06T03:08:01.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94880cba10 con 0x7f948815d230 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94880cbcf0 con 0x7f94881597f0 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38354/0 (socket says 172.21.15.92:38354) 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 -- 172.21.15.92:0/3307819232 learned_addr learned my addr 172.21.15.92:0/3307819232 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 -- 172.21.15.92:0/3307819232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94881597f0 msgr2=0x7f94880ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94881597f0 0x7f94880ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 -- 172.21.15.92:0/3307819232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948815d230 msgr2=0x7f94880ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948815d230 0x7f94880ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.234+0000 7f948659c700 1 -- 172.21.15.92:0/3307819232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780094f0 con 0x7f9488159420 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.235+0000 7f948659c700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880cc630 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9478009290 tx=0x7f9478016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.235+0000 7f947effd700 1 -- 172.21.15.92:0/3307819232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478011e20 con 0x7f9488159420 2023-10-06T03:08:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.235+0000 7f947effd700 1 -- 172.21.15.92:0/3307819232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9478020070 con 0x7f9488159420 2023-10-06T03:08:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.235+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94880cbfd0 con 0x7f9488159420 2023-10-06T03:08:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.235+0000 7f947effd700 1 -- 172.21.15.92:0/3307819232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947801b6a0 con 0x7f9488159420 2023-10-06T03:08:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.235+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94880ceec0 con 0x7f9488159420 2023-10-06T03:08:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.236+0000 7f947effd700 1 -- 172.21.15.92:0/3307819232 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f9478022030 con 0x7f9488159420 2023-10-06T03:08:01.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.236+0000 7f947effd700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9468067290 0x7f9468069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.237+0000 7f947effd700 1 -- 172.21.15.92:0/3307819232 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9478086c70 con 0x7f9488159420 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.237+0000 7f947dd9b700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9468067290 0x7f9468069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.237+0000 7f947dd9b700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9468067290 0x7f9468069740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f94880cd220 tx=0x7f9470006ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.238+0000 7f94962f8b80 1 Processor -- start 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.238+0000 7f94962f8b80 1 -- start start 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf3ae50 0x564f2bf15cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 0x564f2bf16740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f94962f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2be70a70 con 0x564f2bf1ca10 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2be6e270 con 0x564f2bf1b2c0 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f2be6a060 con 0x564f2bf3ae50 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f9486d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 0x564f2bf16740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f9486d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 0x564f2bf16740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38366/0 (socket says 172.21.15.92:38366) 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.239+0000 7f9486d9d700 1 -- 172.21.15.92:0/2275340023 learned_addr learned my addr 172.21.15.92:0/2275340023 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f948659c700 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf3ae50 0x564f2bf15cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f947dd9b700 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f9486d9d700 1 -- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf3ae50 msgr2=0x564f2bf15cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f9486d9d700 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf3ae50 0x564f2bf15cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f9486d9d700 1 -- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 msgr2=0x564f2bf16200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f9486d9d700 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf16200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f9486d9d700 1 -- 172.21.15.92:0/2275340023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2bd04270 con 0x564f2bf1ca10 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f9486d9d700 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 0x564f2bf16740 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9480007e30 tx=0x7f94800077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db26e2d43f52fb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f94677fe700 1 -- 172.21.15.92:0/2275340023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948000cc10 con 0x564f2bf1ca10 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f94677fe700 1 -- 172.21.15.92:0/2275340023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948000cd90 con 0x564f2bf1ca10 2023-10-06T03:08:01.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.240+0000 7f94677fe700 1 -- 172.21.15.92:0/2275340023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480018680 con 0x564f2bf1ca10 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 -- 172.21.15.92:0/2275340023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 msgr2=0x564f2bf16740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 0x564f2bf16740 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9480007e30 tx=0x7f94800077a0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 -- 172.21.15.92:0/2275340023 shutdown_connections 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1ca10 0x564f2bf16740 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf3ae50 0x564f2bf15cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2275340023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf16200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 -- 172.21.15.92:0/2275340023 >> 172.21.15.92:0/2275340023 conn(0x564f2bf1ae80 msgr2=0x564f2bf1b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 -- 172.21.15.92:0/2275340023 shutdown_connections 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 -- 172.21.15.92:0/2275340023 wait complete. 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 -- start start 2023-10-06T03:08:01.249 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.241+0000 7f94962f8b80 1 Processor -- start 2023-10-06T03:08:01.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf60080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2be15890 con 0x564f2bf1b2c0 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2be30ea0 con 0x564f2bf3ae50 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f2bcece30 con 0x564f2bf1ca10 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f94962f8b80 10 client.0 _ll_register_callbacks cb 0x564f2bedd020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f948659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf60080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f948659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf60080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38380/0 (socket says 172.21.15.92:38380) 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f948659c700 1 -- 172.21.15.92:0/2934776850 learned_addr learned my addr 172.21.15.92:0/2934776850 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f947dd9b700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.242+0000 7f9486d9d700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f948659c700 1 -- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 msgr2=0x564f2bf64bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f948659c700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf64bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f948659c700 1 -- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 msgr2=0x564f2bf630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f948659c700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f948659c700 1 -- 172.21.15.92:0/2934776850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9480007a60 con 0x564f2bf1b2c0 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9486d9d700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9464ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f948659c700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf60080 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f947800b3c0 tx=0x7f9478009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dca86b65fb0e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9464ff9700 1 -- 172.21.15.92:0/2934776850 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780539e0 con 0x564f2bf1b2c0 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9464ff9700 1 -- 172.21.15.92:0/2934776850 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9478069e40 con 0x564f2bf1b2c0 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f2bf0e940 con 0x564f2bf1b2c0 2023-10-06T03:08:01.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9464ff9700 1 -- 172.21.15.92:0/2934776850 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9478073250 con 0x564f2bf1b2c0 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 10 client.7005 fetch_fsmap learned FSMap version 68 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x564f2bca6fe0 con 0x564f2bf1b2c0 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9464ff9700 1 -- 172.21.15.92:0/2934776850 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f2bf0e940 con 0x564f2bf1b2c0 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f9464ff9700 1 -- 172.21.15.92:0/2934776850 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9478056b90 con 0x564f2bf1b2c0 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 10 client.7005 fetch_fsmap finished waiting for FSMap version 68 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 10 client.7005 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 20 client.7005 populate_metadata read hostname 'smithi092' 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bf27280 0x564f2bf3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 4 client.7005 mds_command: new command op to 6840 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.243+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 --> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f2beeb870 con 0x564f2bf27280 2023-10-06T03:08:01.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f947dd9b700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bf27280 0x564f2bf3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f947dd9b700 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bf27280 0x564f2bf3b220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597f586f0911bf7c server_cookie=d1e29f3fa302eea2 in_seq=0 out_seq=0 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f9464ff9700 10 client.7005 ms_handle_connect on v2:172.21.15.92:6835/1523868194 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f9464ff9700 1 -- 172.21.15.92:0/2934776850 <== mds.0 v2:172.21.15.92:6835/1523868194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564f2beeb870 con 0x564f2bf27280 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f9464ff9700 10 client.7005 handle_command_reply: tid=0 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 client.7005 shutdown 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 msgr2=0x564f2bf60080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf60080 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f947800b3c0 tx=0x7f9478009e80 comp rx=0 tx=0).stop 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 shutdown_connections 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf60080 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bf27280 0x564f2bf3b220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f9464ff9700 0 client.7005 ms_handle_reset on v2:172.21.15.92:6835/1523868194 2023-10-06T03:08:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.244+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.245+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2934776850 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf64bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.245+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 >> 172.21.15.92:0/2934776850 conn(0x564f2bf1ae80 msgr2=0x564f2bf19b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.245+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 shutdown_connections 2023-10-06T03:08:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.245+0000 7f94962f8b80 1 -- 172.21.15.92:0/2934776850 wait complete. 2023-10-06T03:08:01.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.245+0000 7f94962f8b80 20 client.7005 trim_cache size 0 max 16384 2023-10-06T03:08:01.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.262+0000 7f94962f8b80 1 Processor -- start 2023-10-06T03:08:01.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.262+0000 7f94962f8b80 1 -- start start 2023-10-06T03:08:01.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf10800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf10d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f94962f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2bef2280 con 0x564f2bf1b2c0 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2bf1acb0 con 0x564f2bf3ae50 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f2bf12500 con 0x564f2bf1ca10 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf10800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf10800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38384/0 (socket says 172.21.15.92:38384) 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 -- 172.21.15.92:0/2202324898 learned_addr learned my addr 172.21.15.92:0/2202324898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f947dd9b700 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf10d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f9486d9d700 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 -- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 msgr2=0x564f2bf10d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf10d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 -- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 msgr2=0x564f2bf3b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf3b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 -- 172.21.15.92:0/2202324898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2be38780 con 0x564f2bf1b2c0 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f947dd9b700 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf10d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f948659c700 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf10800 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9478053ba0 tx=0x7f947806a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30014ffa0c4ec0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f9467fff700 1 -- 172.21.15.92:0/2202324898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947805e5f0 con 0x564f2bf1b2c0 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f9467fff700 1 -- 172.21.15.92:0/2202324898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9478073250 con 0x564f2bf1b2c0 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.263+0000 7f9467fff700 1 -- 172.21.15.92:0/2202324898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94780539e0 con 0x564f2bf1b2c0 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 -- 172.21.15.92:0/2202324898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 msgr2=0x564f2bf10800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf10800 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9478053ba0 tx=0x7f947806a000 comp rx=0 tx=0).stop 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 -- 172.21.15.92:0/2202324898 shutdown_connections 2023-10-06T03:08:01.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf1b2c0 0x564f2bf10800 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf3ae50 0x564f2bf3b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 --2- 172.21.15.92:0/2202324898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf10d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 -- 172.21.15.92:0/2202324898 >> 172.21.15.92:0/2202324898 conn(0x564f2bf1ae80 msgr2=0x564f2bf6f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 -- 172.21.15.92:0/2202324898 shutdown_connections 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 -- 172.21.15.92:0/2202324898 wait complete. 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 -- start start 2023-10-06T03:08:01.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.264+0000 7f94962f8b80 1 Processor -- start 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 0x564f2bf72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2bf62da0 con 0x564f2bf3ae50 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2bf6f820 con 0x564f2bf1b2c0 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f2bf79550 con 0x564f2bf1ca10 2023-10-06T03:08:01.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f94962f8b80 10 client.0 _ll_register_callbacks cb 0x7f9478057200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f9486d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 0x564f2bf72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.265+0000 7f9486d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 0x564f2bf72560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38386/0 (socket says 172.21.15.92:38386) 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 -- 172.21.15.92:0/1283769507 learned_addr learned my addr 172.21.15.92:0/1283769507 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f947dd9b700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f948659c700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 -- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 msgr2=0x564f2bf15c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf15c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 -- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 msgr2=0x564f2bf622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 -- 172.21.15.92:0/1283769507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94880ceec0 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9486d9d700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 0x564f2bf72560 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9480013000 tx=0x7f9480019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de8cef50733f49f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9464ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9464ff9700 1 -- 172.21.15.92:0/1283769507 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480018430 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9464ff9700 1 -- 172.21.15.92:0/1283769507 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9480020040 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f2bf62140 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9464ff9700 1 -- 172.21.15.92:0/1283769507 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9480018ab0 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 10 client.7011 fetch_fsmap learned FSMap version 68 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x564f2be1f470 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9464ff9700 1 -- 172.21.15.92:0/1283769507 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f2bf62140 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f9464ff9700 1 -- 172.21.15.92:0/1283769507 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 68) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9480018c10 con 0x564f2bf3ae50 2023-10-06T03:08:01.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 10 client.7011 fetch_fsmap finished waiting for FSMap version 68 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 10 client.7011 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 20 client.7011 populate_metadata read hostname 'smithi092' 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bfc8e40 0x564f2bf72aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 4 client.7011 mds_command: new command op to 6840 tid=0[{"prefix": "status"}] 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.266+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 --> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f948000cd90 con 0x564f2bfc8e40 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f948659c700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bfc8e40 0x564f2bf72aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f9464ff9700 10 client.7011 ms_handle_connect on v2:172.21.15.92:6835/1523868194 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f948659c700 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bfc8e40 0x564f2bf72aa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8715ae69275f8a server_cookie=72dd428aed041b3f in_seq=0 out_seq=0 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f9464ff9700 1 -- 172.21.15.92:0/1283769507 <== mds.0 v2:172.21.15.92:6835/1523868194 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f948000cd90 con 0x564f2bfc8e40 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f9464ff9700 10 client.7011 handle_command_reply: tid=0 2023-10-06T03:08:01.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 client.7011 shutdown 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 msgr2=0x564f2bf72560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 0x564f2bf72560 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9480013000 tx=0x7f9480019000 comp rx=0 tx=0).stop 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 shutdown_connections 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564f2bf3ae50 0x564f2bf72560 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.92:6835/1523868194,v1:172.21.15.92:6837/1523868194] conn(0x564f2bfc8e40 0x564f2bf72aa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564f2bf1b2c0 0x564f2bf622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f9464ff9700 0 client.7011 ms_handle_reset on v2:172.21.15.92:6835/1523868194 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 --2- 172.21.15.92:0/1283769507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564f2bf1ca10 0x564f2bf15c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 >> 172.21.15.92:0/1283769507 conn(0x564f2bf1ae80 msgr2=0x564f2bf3b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 shutdown_connections 2023-10-06T03:08:01.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 1 -- 172.21.15.92:0/1283769507 wait complete. 2023-10-06T03:08:01.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.267+0000 7f94962f8b80 20 client.7011 trim_cache size 0 max 16384 2023-10-06T03:08:01.278 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:08:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:08:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:08:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 6840, 2023-10-06T03:08:01.278 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 10.712825029999999, 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 68, 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 103, 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 103, 2023-10-06T03:08:01.279 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 22.735985036999999 2023-10-06T03:08:01.280 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:08:01.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9468067290 msgr2=0x7f9468069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9468067290 0x7f9468069740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f94880cd220 tx=0x7f9470006ce0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 msgr2=0x7f94880cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880cc630 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9478009290 tx=0x7f9478016000 comp rx=0 tx=0).stop 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 shutdown_connections 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9468067290 0x7f9468069740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9488159420 0x7f94880cc630 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f948815d230 0x7f94880ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 --2- 172.21.15.92:0/3307819232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94881597f0 0x7f94880ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 >> 172.21.15.92:0/3307819232 conn(0x7f94880d0660 msgr2=0x7f94880b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 shutdown_connections 2023-10-06T03:08:01.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.268+0000 7f948759e700 1 -- 172.21.15.92:0/3307819232 wait complete. 2023-10-06T03:08:01.282 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:08:01.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.395+0000 7fab8474f700 1 Processor -- start 2023-10-06T03:08:01.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.395+0000 7fab8474f700 1 -- start start 2023-10-06T03:08:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab8474f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab8474f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab7c159c60 0x7fab7c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab8474f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab7c15a160 con 0x7fab7c15db10 2023-10-06T03:08:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab8474f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7c0d13a0 con 0x7fab7c159c60 2023-10-06T03:08:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab8474f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c15e140 con 0x7fab7c159890 2023-10-06T03:08:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab81cea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab7c159c60 0x7fab7c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.396+0000 7fab82cec700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fab7c15db10 0x7fab7c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35074/0 (socket says 172.21.15.92:35074) 2023-10-06T03:08:01.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab81cea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fab7c159c60 0x7fab7c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38388/0 (socket says 172.21.15.92:38388) 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab82cec700 1 -- 172.21.15.92:0/4216487815 learned_addr learned my addr 172.21.15.92:0/4216487815 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab824eb700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab81cea700 1 -- 172.21.15.92:0/4216487815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 msgr2=0x7fab7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab81cea700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab81cea700 1 -- 172.21.15.92:0/4216487815 >> v1:172.21.15.92:6789/0 conn(0x7fab7c15db10 legacy=0x7fab7c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab81cea700 1 -- 172.21.15.92:0/4216487815 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7c15a3d0 con 0x7fab7c159c60 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab824eb700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab814e9700 1 -- 172.21.15.92:0/4216487815 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355605250 0 0) 0x7fab7c15a160 con 0x7fab7c15db10 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab81cea700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c15d5d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0098e0 tx=0x7fab6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b503992d2a8f64a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab814e9700 1 -- 172.21.15.92:0/4216487815 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c009e80 con 0x7fab7c159c60 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab814e9700 1 -- 172.21.15.92:0/4216487815 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c00b860 con 0x7fab7c159c60 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.397+0000 7fab814e9700 1 -- 172.21.15.92:0/4216487815 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00bbe0 con 0x7fab7c159c60 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 -- 172.21.15.92:0/4216487815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 msgr2=0x7fab7c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c15d5d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fab6c0098e0 tx=0x7fab6c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 -- 172.21.15.92:0/4216487815 shutdown_connections 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 --2- 172.21.15.92:0/4216487815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c15d5d0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 -- 172.21.15.92:0/4216487815 >> 172.21.15.92:0/4216487815 conn(0x7fab7c068e20 msgr2=0x7fab7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 -- 172.21.15.92:0/4216487815 shutdown_connections 2023-10-06T03:08:01.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.398+0000 7fab8474f700 1 -- 172.21.15.92:0/4216487815 wait complete. 2023-10-06T03:08:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.399+0000 7fab8474f700 1 Processor -- start 2023-10-06T03:08:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.399+0000 7fab8474f700 1 -- start start 2023-10-06T03:08:01.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab8474f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab8474f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab8474f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7c15db10 0x7fab7c0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab8474f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0cf060 con 0x7fab7c159c60 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab8474f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0ce140 con 0x7fab7c15db10 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab8474f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7c0ce3e0 con 0x7fab7c159890 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c169af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38400/0 (socket says 172.21.15.92:38400) 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 -- 172.21.15.92:0/3544264627 learned_addr learned my addr 172.21.15.92:0/3544264627 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab824eb700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab82cec700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7c15db10 0x7fab7c0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 -- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 msgr2=0x7fab7c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c0cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 -- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7c15db10 msgr2=0x7fab7c0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7c15db10 0x7fab7c0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 -- 172.21.15.92:0/3544264627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c009420 con 0x7fab7c159c60 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab824eb700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab82cec700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7c15db10 0x7fab7c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab81cea700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c169af0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fab6c013000 tx=0x7fab6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.400+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00b9d0 con 0x7fab7c159c60 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.401+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c00bb30 con 0x7fab7c159c60 2023-10-06T03:08:01.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.401+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7c0ce6c0 con 0x7fab7c159c60 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.401+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c025650 con 0x7fab7c159c60 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.401+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7c0cec80 con 0x7fab7c159c60 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.401+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7c0cf1a0 con 0x7fab7c159c60 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.402+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fab6c023070 con 0x7fab7c159c60 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.402+0000 7fab727fc700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab60067310 0x7fab600697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.402+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab6c0861c0 con 0x7fab7c159c60 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.402+0000 7fab824eb700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab60067310 0x7fab600697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:01.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.403+0000 7fab824eb700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab60067310 0x7fab600697c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fab780094d0 tx=0x7fab7800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:01.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.406+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab7c0cf1a0 con 0x7fab7c159c60 2023-10-06T03:08:01.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.635+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab7c0cf1a0 con 0x7fab7c159c60 2023-10-06T03:08:01.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.635+0000 7fab727fc700 1 -- 172.21.15.92:0/3544264627 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7fab7c0cf1a0 con 0x7fab7c159c60 2023-10-06T03:08:01.639 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:08:01.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab60067310 msgr2=0x7fab600697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab60067310 0x7fab600697c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fab780094d0 tx=0x7fab7800b040 comp rx=0 tx=0).stop 2023-10-06T03:08:01.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 msgr2=0x7fab7c169af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:01.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c169af0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fab6c013000 tx=0x7fab6c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:01.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 shutdown_connections 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab60067310 0x7fab600697c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7c15db10 0x7fab7c0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab7c159890 0x7fab7c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 --2- 172.21.15.92:0/3544264627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab7c159c60 0x7fab7c169af0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 >> 172.21.15.92:0/3544264627 conn(0x7fab7c068e20 msgr2=0x7fab7c155170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 shutdown_connections 2023-10-06T03:08:01.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:01.637+0000 7fab8474f700 1 -- 172.21.15.92:0/3544264627 wait complete. 2023-10-06T03:08:01.651 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:08:01.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:08:01.652 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:08:01.652 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:08:01.652 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:01.653 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:08:01.653 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:08:01.653 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:08:01.653 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:08:01.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:01.655 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:08:01.672 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:08:01.672 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:08:01.672 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:08:01.672 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.673 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:08:01.673 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.673 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:01.673 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:01.673 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:08:01.674 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:01.674 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:01.674 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:08:01.674 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:01.674 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:01.675 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:08:01.675 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:01.675 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:08:01.675 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42124sec preferred_lft 42124sec 2023-10-06T03:08:01.675 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:08:01.676 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.676 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:08:01.676 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:01.677 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:08:01.677 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.677 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:08:01.677 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.677 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:01.678 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:08:01.678 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:08:01.678 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.678 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:01.678 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:08:01.678 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:08:01.679 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:01.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:01.679 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:08:01.741 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:08:01.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:01.743 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:08:01.798 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:08:01.799 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:08:01.799 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:08:01.799 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:08:01.799 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:08:01.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:01.799 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:08:01.800 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:08:01.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:08:01.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:08:01.800 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:08:01.800 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:08:01.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:01.899 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:08:01.914 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:08:01.914 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:08:01.914 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:08:01.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:01.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:08:01.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:01.979 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:02.033 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:08:02.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:02.034 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:02.088 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:08:02.088 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:08:02.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:02.167 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:02.167 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:02.186 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:02.208 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:08:02.209 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:08:02.211 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:08:02.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:02.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:08:02.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.430+0000 7f380ccb1700 1 Processor -- start 2023-10-06T03:08:02.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.430+0000 7f380ccb1700 1 -- start start 2023-10-06T03:08:02.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380ccb1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3808159890 0x7f380815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380ccb1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3808159c60 0x7f380815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380ccb1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f3808069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380ccb1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380815df50 con 0x7f3808159890 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380ccb1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380815a160 con 0x7f3808159c60 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380ccb1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080d13a0 con 0x7f380815db10 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3808159890 0x7f380815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3808159890 0x7f380815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38410/0 (socket says 172.21.15.92:38410) 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 -- 172.21.15.92:0/3711812682 learned_addr learned my addr 172.21.15.92:0/3711812682 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f37fdd9b700 1 --2- 172.21.15.92:0/3711812682 >> v2:172.21.15.136:3300/0 conn(0x7f3808159c60 0x7f380815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f3806d9d700 1 --2- 172.21.15.92:0/3711812682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f3808069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 -- 172.21.15.92:0/3711812682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 msgr2=0x7f3808069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:02.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 --2- 172.21.15.92:0/3711812682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f3808069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 -- 172.21.15.92:0/3711812682 >> v2:172.21.15.136:3300/0 conn(0x7f3808159c60 msgr2=0x7f380815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 --2- 172.21.15.92:0/3711812682 >> v2:172.21.15.136:3300/0 conn(0x7f3808159c60 0x7f380815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.431+0000 7f380659c700 1 -- 172.21.15.92:0/3711812682 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380815e140 con 0x7f3808159890 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380659c700 1 --2- 172.21.15.92:0/3711812682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159890 0x7f380815abd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f38080b67f0 tx=0x7f37f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538869dcf3c8ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f3805d9b700 1 -- 172.21.15.92:0/3711812682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800b860 con 0x7f3808159890 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f3805d9b700 1 -- 172.21.15.92:0/3711812682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f800b9c0 con 0x7f3808159890 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f3805d9b700 1 -- 172.21.15.92:0/3711812682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800bd60 con 0x7f3808159890 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 -- 172.21.15.92:0/3711812682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159890 msgr2=0x7f380815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 --2- 172.21.15.92:0/3711812682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159890 0x7f380815abd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f38080b67f0 tx=0x7f37f8009250 comp rx=0 tx=0).stop 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 -- 172.21.15.92:0/3711812682 shutdown_connections 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 --2- 172.21.15.92:0/3711812682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f3808069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 --2- 172.21.15.92:0/3711812682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159890 0x7f380815abd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 --2- 172.21.15.92:0/3711812682 >> v2:172.21.15.136:3300/0 conn(0x7f3808159c60 0x7f380815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 -- 172.21.15.92:0/3711812682 >> 172.21.15.92:0/3711812682 conn(0x7f3808068e20 msgr2=0x7f38080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 -- 172.21.15.92:0/3711812682 shutdown_connections 2023-10-06T03:08:02.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.432+0000 7f380ccb1700 1 -- 172.21.15.92:0/3711812682 wait complete. 2023-10-06T03:08:02.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 Processor -- start 2023-10-06T03:08:02.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 -- start start 2023-10-06T03:08:02.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808159890 0x7f38080b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 0x7f38080c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f38080c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808153650 con 0x7f3808159c60 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080c9dd0 con 0x7f3808159890 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f380ccb1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b3630 con 0x7f380815db10 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f37fdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 0x7f38080c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f37fdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 0x7f38080c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38422/0 (socket says 172.21.15.92:38422) 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f37fdd9b700 1 -- 172.21.15.92:0/2670190720 learned_addr learned my addr 172.21.15.92:0/2670190720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.434+0000 7f3806d9d700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f38080c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f380659c700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808159890 0x7f38080b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37fdd9b700 1 -- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 msgr2=0x7f38080c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37fdd9b700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f38080c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37fdd9b700 1 -- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808159890 msgr2=0x7f38080b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37fdd9b700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808159890 0x7f38080b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37fdd9b700 1 -- 172.21.15.92:0/2670190720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f80094f0 con 0x7f3808159c60 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37fdd9b700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 0x7f38080c9210 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f37f000d890 tx=0x7f37f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00125f0 con 0x7f3808159c60 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0012750 con 0x7f3808159c60 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38080b3910 con 0x7f3808159c60 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0012af0 con 0x7f3808159c60 2023-10-06T03:08:02.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.435+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38080b3ed0 con 0x7f3808159c60 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.436+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380804c3c0 con 0x7f3808159c60 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.436+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f37f001e460 con 0x7f3808159c60 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.437+0000 7f37feffd700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37e8067290 0x7f37e8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.437+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f37f0085140 con 0x7f3808159c60 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.437+0000 7f380659c700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37e8067290 0x7f37e8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.437+0000 7f380659c700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37e8067290 0x7f37e8069740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f38080b67f0 tx=0x7f37f8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:02.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.441+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37f0051e40 con 0x7f3808159c60 2023-10-06T03:08:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.668+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f380806a5b0 con 0x7f3808159c60 2023-10-06T03:08:02.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.668+0000 7f37feffd700 1 -- 172.21.15.92:0/2670190720 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37f0055460 con 0x7f3808159c60 2023-10-06T03:08:02.669 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37e8067290 msgr2=0x7f37e8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37e8067290 0x7f37e8069740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f38080b67f0 tx=0x7f37f8017000 comp rx=0 tx=0).stop 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 msgr2=0x7f38080c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 0x7f38080c9210 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f37f000d890 tx=0x7f37f0008040 comp rx=0 tx=0).stop 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 shutdown_connections 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37e8067290 0x7f37e8069740 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380815db10 0x7f38080c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3808159890 0x7f38080b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 --2- 172.21.15.92:0/2670190720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3808159c60 0x7f38080c9210 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 >> 172.21.15.92:0/2670190720 conn(0x7f3808068e20 msgr2=0x7f38080c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 shutdown_connections 2023-10-06T03:08:02.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:02.671+0000 7f380ccb1700 1 -- 172.21.15.92:0/2670190720 wait complete. 2023-10-06T03:08:02.683 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:08:02.683 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:08:02.684 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:08:02.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.684 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:08:02.703 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:08:02.703 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:08:02.703 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:08:02.703 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.703 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:02.704 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42125sec preferred_lft 42125sec 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:08:02.705 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:08:02.706 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.706 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:02.706 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:08:02.706 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:08:02.706 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:02.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.706 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:08:02.771 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:08:02.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.773 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:08:02.832 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:08:02.832 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:08:02.832 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:08:02.832 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:08:02.832 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:08:02.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.833 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:08:02.833 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:08:02.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:08:02.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:08:02.833 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:08:02.833 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:08:02.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.937 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:08:02.952 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:08:02.952 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:08:02.953 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:08:02.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:02.953 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:08:03.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:03.018 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:03.073 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:08:03.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:03.073 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:03.128 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:08:03.128 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:08:03.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:03.207 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:03.208 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:03.226 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:03.226 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:03.226 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:03.226 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:03.226 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:03.226 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:03.227 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:03.227 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:03.249 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:03.250 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:03.250 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:08:03.250 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:08:03.250 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:08:03.251 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:08:03.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:03.251 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:03.315 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:08:03.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.442+0000 7f2afe92c700 1 Processor -- start 2023-10-06T03:08:03.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.442+0000 7f2afe92c700 1 -- start start 2023-10-06T03:08:03.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afe92c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2af8159ec0 0x7f2af80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:03.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afe92c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:03.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afe92c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af80b5b00 con 0x7f2af80cce40 2023-10-06T03:08:03.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afe92c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80b5850 con 0x7f2af8159ec0 2023-10-06T03:08:03.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afe92c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b5da0 con 0x7f2af80b53d0 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2af8159ec0 0x7f2af80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afcec9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2af80cce40 0x7f2af80cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35084/0 (socket says 172.21.15.92:35084) 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2afcec9700 1 -- 172.21.15.92:0/856478349 learned_addr learned my addr 172.21.15.92:0/856478349 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af77fe700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af7fff700 1 -- 172.21.15.92:0/856478349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 msgr2=0x7f2af80cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af7fff700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af80cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af7fff700 1 -- 172.21.15.92:0/856478349 >> v1:172.21.15.92:6789/0 conn(0x7f2af80cce40 legacy=0x7f2af80cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af7fff700 1 -- 172.21.15.92:0/856478349 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80b6080 con 0x7f2af8159ec0 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af6ffd700 1 -- 172.21.15.92:0/856478349 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213261504 0 0) 0x7f2af80b5b00 con 0x7f2af80cce40 2023-10-06T03:08:03.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af77fe700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af80cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af7fff700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80b4e90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80098d0 tx=0x7f2ae8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979a24c4b361d6ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af6ffd700 1 -- 172.21.15.92:0/856478349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8009e70 con 0x7f2af8159ec0 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af6ffd700 1 -- 172.21.15.92:0/856478349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8005c10 con 0x7f2af8159ec0 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.443+0000 7f2af6ffd700 1 -- 172.21.15.92:0/856478349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800b650 con 0x7f2af8159ec0 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 -- 172.21.15.92:0/856478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 msgr2=0x7f2af80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80b4e90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80098d0 tx=0x7f2ae8009120 comp rx=0 tx=0).stop 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 -- 172.21.15.92:0/856478349 shutdown_connections 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80b4e90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 --2- 172.21.15.92:0/856478349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af80cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 -- 172.21.15.92:0/856478349 >> 172.21.15.92:0/856478349 conn(0x7f2af80999f0 msgr2=0x7f2af80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:03.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 -- 172.21.15.92:0/856478349 shutdown_connections 2023-10-06T03:08:03.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.444+0000 7f2afe92c700 1 -- 172.21.15.92:0/856478349 wait complete. 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.445+0000 7f2afe92c700 1 Processor -- start 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.445+0000 7f2afe92c700 1 -- start start 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afe92c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af8144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afe92c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2af80cce40 0x7f2af80ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afe92c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afe92c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81452b0 con 0x7f2af8159ec0 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afe92c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af809a9d0 con 0x7f2af80cce40 2023-10-06T03:08:03.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afe92c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af809c890 con 0x7f2af80b53d0 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38450/0 (socket says 172.21.15.92:38450) 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 -- 172.21.15.92:0/1250692653 learned_addr learned my addr 172.21.15.92:0/1250692653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2af77fe700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2af80cce40 0x7f2af80ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 -- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 msgr2=0x7f2af8144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2af7fff700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af8144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af8144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 -- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2af80cce40 msgr2=0x7f2af80ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2af80cce40 0x7f2af80ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 -- 172.21.15.92:0/1250692653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae80093c0 con 0x7f2af8159ec0 2023-10-06T03:08:03.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2af77fe700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2af80cce40 0x7f2af80ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2af7fff700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af8144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.446+0000 7f2afcec9700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80cea10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2af000ccc0 tx=0x7f2af0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.447+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ab30 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.447+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000ac90 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.447+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af809bc80 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.447+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0007340 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.447+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af809c240 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.448+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af8063c90 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.448+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2af00076b0 con 0x7f2af8159ec0 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.448+0000 7f2adffff700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2ae4067290 0x7f2ae4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:03.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.449+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2af0085190 con 0x7f2af8159ec0 2023-10-06T03:08:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.449+0000 7f2af7fff700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2ae4067290 0x7f2ae4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.449+0000 7f2af7fff700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2ae4067290 0x7f2ae4069740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009160 tx=0x7f2ae8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:03.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.453+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2af0051e10 con 0x7f2af8159ec0 2023-10-06T03:08:03.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:03.682+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2af80718b0 con 0x7f2af8159ec0 2023-10-06T03:08:04.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.609+0000 7f2adffff700 1 -- 172.21.15.92:0/1250692653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v69) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2af0055430 con 0x7f2af8159ec0 2023-10-06T03:08:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2ae4067290 msgr2=0x7f2ae4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2ae4067290 0x7f2ae4069740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009160 tx=0x7f2ae8016000 comp rx=0 tx=0).stop 2023-10-06T03:08:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 msgr2=0x7f2af80cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80cea10 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2af000ccc0 tx=0x7f2af0001a80 comp rx=0 tx=0).stop 2023-10-06T03:08:04.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 shutdown_connections 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2ae4067290 0x7f2ae4069740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2af8159ec0 0x7f2af80cea10 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2af80b53d0 0x7f2af8144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 --2- 172.21.15.92:0/1250692653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2af80cce40 0x7f2af80ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 >> 172.21.15.92:0/1250692653 conn(0x7f2af80999f0 msgr2=0x7f2af80b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 shutdown_connections 2023-10-06T03:08:04.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.612+0000 7f2afe92c700 1 -- 172.21.15.92:0/1250692653 wait complete. 2023-10-06T03:08:04.629 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:08:04.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.750+0000 7f3b8ff4c700 1 Processor -- start 2023-10-06T03:08:04.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.750+0000 7f3b8ff4c700 1 -- start start 2023-10-06T03:08:04.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8ff4c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b8806c0a0 0x7f3b880caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:04.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8ff4c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8815d7e0 con 0x7f3b8806bcd0 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8ff4c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8815dac0 con 0x7f3b8815df50 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8ff4c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8815aa30 con 0x7f3b8806c0a0 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8d4e7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b8806c0a0 0x7f3b880caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8dce8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3b8806bcd0 0x7f3b8815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35092/0 (socket says 172.21.15.92:35092) 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.751+0000 7f3b8dce8700 1 -- 172.21.15.92:0/3651351014 learned_addr learned my addr 172.21.15.92:0/3651351014 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/3651351014 >> v1:172.21.15.136:6789/0 conn(0x7f3b8815df50 legacy=0x7f3b880c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/3651351014 >> v1:172.21.15.92:6789/0 conn(0x7f3b8806bcd0 legacy=0x7f3b8815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/3651351014 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8815ad10 con 0x7f3b8806c0a0 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8cce6700 1 -- 172.21.15.92:0/3651351014 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240035468 0 0) 0x7f3b8815d7e0 con 0x7f3b8806bcd0 2023-10-06T03:08:04.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8d4e7700 1 --2- 172.21.15.92:0/3651351014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b880caa90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098e0 tx=0x7f3b780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761e26711264b111 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:04.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8cce6700 1 -- 172.21.15.92:0/3651351014 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78009e80 con 0x7f3b8806c0a0 2023-10-06T03:08:04.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8cce6700 1 -- 172.21.15.92:0/3651351014 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7800b860 con 0x7f3b8806c0a0 2023-10-06T03:08:04.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8cce6700 1 -- 172.21.15.92:0/3651351014 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800bbe0 con 0x7f3b8806c0a0 2023-10-06T03:08:04.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/3651351014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 msgr2=0x7f3b880caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/3651351014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b880caa90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098e0 tx=0x7f3b780091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/3651351014 shutdown_connections 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/3651351014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b880caa90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.752+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/3651351014 >> 172.21.15.92:0/3651351014 conn(0x7f3b880ce600 msgr2=0x7f3b880cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.753+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/3651351014 shutdown_connections 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.753+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/3651351014 wait complete. 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 Processor -- start 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 -- start start 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8806bcd0 0x7f3b88153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b88157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b8815df50 0x7f3b880b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:04.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880b3d70 con 0x7f3b8806c0a0 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b881584e0 con 0x7f3b8806bcd0 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8ff4c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880b2650 con 0x7f3b8815df50 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8d4e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b88157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8d4e7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b88157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38466/0 (socket says 172.21.15.92:38466) 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/2204664749 learned_addr learned my addr 172.21.15.92:0/2204664749 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.754+0000 7f3b8dce8700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8806bcd0 0x7f3b88153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8e4e9700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b8815df50 0x7f3b880b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b8815df50 msgr2=0x7f3b880b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8d4e7700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b8815df50 0x7f3b880b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8806bcd0 msgr2=0x7f3b88153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8d4e7700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8806bcd0 0x7f3b88153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8d4e7700 1 -- 172.21.15.92:0/2204664749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78009420 con 0x7f3b8806c0a0 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8e4e9700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b8815df50 0x7f3b880b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8d4e7700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b88157fa0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3b88069220 tx=0x7f3b78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800bbe0 con 0x7f3b8806c0a0 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b78011420 con 0x7f3b8806c0a0 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b780117c0 con 0x7f3b8806c0a0 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b88154020 con 0x7f3b8806c0a0 2023-10-06T03:08:04.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.755+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b880b3990 con 0x7f3b8806c0a0 2023-10-06T03:08:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.756+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b880b3eb0 con 0x7f3b8806c0a0 2023-10-06T03:08:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.757+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f3b78011bf0 con 0x7f3b8806c0a0 2023-10-06T03:08:04.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.757+0000 7f3b7dffb700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b6c067290 0x7f3b6c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:04.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.757+0000 7f3b8dce8700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b6c067290 0x7f3b6c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:04.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.757+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b78086460 con 0x7f3b8806c0a0 2023-10-06T03:08:04.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.758+0000 7f3b8dce8700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b6c067290 0x7f3b6c069740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000c00 tx=0x7f3b84007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:04.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.761+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b880b3eb0 con 0x7f3b8806c0a0 2023-10-06T03:08:04.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.970+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b8804c430 con 0x7f3b6c067290 2023-10-06T03:08:04.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.971+0000 7f3b7dffb700 1 -- 172.21.15.92:0/2204664749 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3b8804c430 con 0x7f3b6c067290 2023-10-06T03:08:04.972 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:08:04.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:04.972 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:08:04.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:04.972 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b6c067290 msgr2=0x7f3b6c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b6c067290 0x7f3b6c069740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3b84000c00 tx=0x7f3b84007f30 comp rx=0 tx=0).stop 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 msgr2=0x7f3b88157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b88157fa0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f3b88069220 tx=0x7f3b78017000 comp rx=0 tx=0).stop 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 shutdown_connections 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b6c067290 0x7f3b6c069740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b8815df50 0x7f3b880b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b8806bcd0 0x7f3b88153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 --2- 172.21.15.92:0/2204664749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b8806c0a0 0x7f3b88157fa0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:04.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 >> 172.21.15.92:0/2204664749 conn(0x7f3b880ce600 msgr2=0x7f3b8815a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:04.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 shutdown_connections 2023-10-06T03:08:04.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:04.974+0000 7f3b8ff4c700 1 -- 172.21.15.92:0/2204664749 wait complete. 2023-10-06T03:08:04.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:08:05.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.104+0000 7f6f756e9700 1 Processor -- start 2023-10-06T03:08:05.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.104+0000 7f6f756e9700 1 -- start start 2023-10-06T03:08:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f756e9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f70069560 0x7f6f700699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f756e9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f70069ee0 con 0x7f6f70159890 2023-10-06T03:08:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f756e9700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7015a160 con 0x7f6f70159c60 2023-10-06T03:08:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f756e9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f700d13a0 con 0x7f6f70069560 2023-10-06T03:08:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6f70069560 0x7f6f700699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6f70159890 0x7f6f7015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35094/0 (socket says 172.21.15.92:35094) 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6effd700 1 -- 172.21.15.92:0/4202708399 learned_addr learned my addr 172.21.15.92:0/4202708399 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6f7fe700 1 -- 172.21.15.92:0/4202708399 >> v1:172.21.15.136:6789/0 conn(0x7f6f70159c60 legacy=0x7f6f7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6f7fe700 1 -- 172.21.15.92:0/4202708399 >> v1:172.21.15.92:6789/0 conn(0x7f6f70159890 legacy=0x7f6f7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6dffb700 1 -- 172.21.15.92:0/4202708399 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576256626 0 0) 0x7f6f70069ee0 con 0x7f6f70159890 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6f7fe700 1 -- 172.21.15.92:0/4202708399 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7015e140 con 0x7f6f70069560 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6f7fe700 1 --2- 172.21.15.92:0/4202708399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f700699a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009e60 tx=0x7f6f68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d804a978d4372fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6dffb700 1 -- 172.21.15.92:0/4202708399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6800e070 con 0x7f6f70069560 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6dffb700 1 -- 172.21.15.92:0/4202708399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6800bc30 con 0x7f6f70069560 2023-10-06T03:08:05.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.105+0000 7f6f6dffb700 1 -- 172.21.15.92:0/4202708399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68012430 con 0x7f6f70069560 2023-10-06T03:08:05.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 -- 172.21.15.92:0/4202708399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 msgr2=0x7f6f700699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 --2- 172.21.15.92:0/4202708399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f700699a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009e60 tx=0x7f6f68009780 comp rx=0 tx=0).stop 2023-10-06T03:08:05.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 -- 172.21.15.92:0/4202708399 shutdown_connections 2023-10-06T03:08:05.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 --2- 172.21.15.92:0/4202708399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f700699a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 -- 172.21.15.92:0/4202708399 >> 172.21.15.92:0/4202708399 conn(0x7f6f70068e20 msgr2=0x7f6f700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 -- 172.21.15.92:0/4202708399 shutdown_connections 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.106+0000 7f6f756e9700 1 -- 172.21.15.92:0/4202708399 wait complete. 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.107+0000 7f6f756e9700 1 Processor -- start 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.107+0000 7f6f756e9700 1 -- start start 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f756e9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f70149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f756e9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f70159890 0x7f6f700b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f756e9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f70159c60 0x7f6f700c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f756e9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f70149880 con 0x7f6f70069560 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f756e9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700cd0c0 con 0x7f6f70159890 2023-10-06T03:08:05.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f756e9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700c8320 con 0x7f6f70159c60 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f70149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f70149340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38480/0 (socket says 172.21.15.92:38480) 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 -- 172.21.15.92:0/1350616909 learned_addr learned my addr 172.21.15.92:0/1350616909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6f7fe700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f70159c60 0x7f6f700c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 -- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f70159c60 msgr2=0x7f6f700c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f70159c60 0x7f6f700c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 -- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f70159890 msgr2=0x7f6f700b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6e7fc700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f70159890 0x7f6f700b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f70159890 0x7f6f700b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6effd700 1 -- 172.21.15.92:0/1350616909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f68009a20 con 0x7f6f70069560 2023-10-06T03:08:05.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.108+0000 7f6f6e7fc700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f70159890 0x7f6f700b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.109+0000 7f6f6effd700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f70149340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000c7e0 tx=0x7f6f60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.109+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60005b30 con 0x7f6f70069560 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.109+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60005c90 con 0x7f6f70069560 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.109+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f700c8600 con 0x7f6f70069560 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.109+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600072e0 con 0x7f6f70069560 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.109+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f700cd310 con 0x7f6f70069560 2023-10-06T03:08:05.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.110+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f700cd530 con 0x7f6f70069560 2023-10-06T03:08:05.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.110+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6f600076d0 con 0x7f6f70069560 2023-10-06T03:08:05.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.110+0000 7f6f56ffd700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f5c067290 0x7f6f5c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.110+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6f60084100 con 0x7f6f70069560 2023-10-06T03:08:05.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.111+0000 7f6f6e7fc700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f5c067290 0x7f6f5c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.111+0000 7f6f6e7fc700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f5c067290 0x7f6f5c069740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6f700b0330 tx=0x7f6f64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:05.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.115+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f6004b5a0 con 0x7f6f70069560 2023-10-06T03:08:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.297+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6f7004c3c0 con 0x7f6f70069560 2023-10-06T03:08:05.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.298+0000 7f6f56ffd700 1 -- 172.21.15.92:0/1350616909 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f60050db0 con 0x7f6f70069560 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f5c067290 msgr2=0x7f6f5c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f5c067290 0x7f6f5c069740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6f700b0330 tx=0x7f6f64009040 comp rx=0 tx=0).stop 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 msgr2=0x7f6f70149340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f70149340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000c7e0 tx=0x7f6f60007df0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 shutdown_connections 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6f5c067290 0x7f6f5c069740 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6f70069560 0x7f6f70149340 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f70159c60 0x7f6f700c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 --2- 172.21.15.92:0/1350616909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f70159890 0x7f6f700b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 >> 172.21.15.92:0/1350616909 conn(0x7f6f70068e20 msgr2=0x7f6f701553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 shutdown_connections 2023-10-06T03:08:05.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.300+0000 7f6f756e9700 1 -- 172.21.15.92:0/1350616909 wait complete. 2023-10-06T03:08:05.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-06T03:08:05.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.428+0000 7f00cac54700 1 Processor -- start 2023-10-06T03:08:05.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.428+0000 7f00cac54700 1 -- start start 2023-10-06T03:08:05.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00cac54700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00c40b1b40 0x7f00c40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00cac54700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c40cfb00 con 0x7f00c40b16d0 2023-10-06T03:08:05.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00cac54700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c40b5c40 con 0x7f00c40b4e70 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00cac54700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c40b5f20 con 0x7f00c40b1b40 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00ca453700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00c40b1b40 0x7f00c40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00c9451700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f00c40b16d0 0x7f00c40b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35100/0 (socket says 172.21.15.92:35100) 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00c9451700 1 -- 172.21.15.92:0/249427297 learned_addr learned my addr 172.21.15.92:0/249427297 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00ca453700 1 -- 172.21.15.92:0/249427297 >> v1:172.21.15.136:6789/0 conn(0x7f00c40b4e70 legacy=0x7f00c40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00ca453700 1 -- 172.21.15.92:0/249427297 >> v1:172.21.15.92:6789/0 conn(0x7f00c40b16d0 legacy=0x7f00c40b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.429+0000 7f00c8c50700 1 -- 172.21.15.92:0/249427297 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092321283 0 0) 0x7f00c40cfb00 con 0x7f00c40b16d0 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00ca453700 1 -- 172.21.15.92:0/249427297 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40b6200 con 0x7f00c40b1b40 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00ca453700 1 --2- 172.21.15.92:0/249427297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b1b40 0x7f00c40b3ac0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098e0 tx=0x7f00bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226f0a564663a57a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00c8c50700 1 -- 172.21.15.92:0/249427297 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc009e80 con 0x7f00c40b1b40 2023-10-06T03:08:05.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00c8c50700 1 -- 172.21.15.92:0/249427297 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc00b820 con 0x7f00c40b1b40 2023-10-06T03:08:05.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00c8c50700 1 -- 172.21.15.92:0/249427297 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00bba0 con 0x7f00c40b1b40 2023-10-06T03:08:05.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 -- 172.21.15.92:0/249427297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b1b40 msgr2=0x7f00c40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 --2- 172.21.15.92:0/249427297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b1b40 0x7f00c40b3ac0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0098e0 tx=0x7f00bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 -- 172.21.15.92:0/249427297 shutdown_connections 2023-10-06T03:08:05.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 --2- 172.21.15.92:0/249427297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b1b40 0x7f00c40b3ac0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 -- 172.21.15.92:0/249427297 >> 172.21.15.92:0/249427297 conn(0x7f00c40d10c0 msgr2=0x7f00c40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:05.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 -- 172.21.15.92:0/249427297 shutdown_connections 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.430+0000 7f00cac54700 1 -- 172.21.15.92:0/249427297 wait complete. 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.431+0000 7f00cac54700 1 Processor -- start 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 -- start start 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c40b16d0 0x7f00c40cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1b40 0x7f00c4144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 0x7f00c40c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40cb370 con 0x7f00c40b4e70 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c5ea0 con 0x7f00c40b1b40 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00cac54700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c6180 con 0x7f00c40b16d0 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00ca453700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 0x7f00c40c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00ca453700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 0x7f00c40c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38504/0 (socket says 172.21.15.92:38504) 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.432+0000 7f00ca453700 1 -- 172.21.15.92:0/1246161821 learned_addr learned my addr 172.21.15.92:0/1246161821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00c9451700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1b40 0x7f00c4144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00ca453700 1 -- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c40b16d0 msgr2=0x7f00c40cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00ca453700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c40b16d0 0x7f00c40cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00ca453700 1 -- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1b40 msgr2=0x7f00c4144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00ca453700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1b40 0x7f00c4144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00ca453700 1 -- 172.21.15.92:0/1246161821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009420 con 0x7f00c40b4e70 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00ca453700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 0x7f00c40c5390 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0093f0 tx=0x7f00bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00bb10 con 0x7f00c40b4e70 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc020070 con 0x7f00c40b4e70 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40cb5f0 con 0x7f00c40b4e70 2023-10-06T03:08:05.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc01b6a0 con 0x7f00c40b4e70 2023-10-06T03:08:05.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.433+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c41577a0 con 0x7f00c40b4e70 2023-10-06T03:08:05.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.434+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c40cbaf0 con 0x7f00c40b4e70 2023-10-06T03:08:05.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.434+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f00bc022030 con 0x7f00c40b4e70 2023-10-06T03:08:05.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.435+0000 7f00b9ffb700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00a8067290 0x7f00a8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:05.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.435+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f00bc086d20 con 0x7f00c40b4e70 2023-10-06T03:08:05.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.435+0000 7f00c9c52700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00a8067290 0x7f00a8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:05.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.435+0000 7f00c9c52700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00a8067290 0x7f00a8069740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00c000ad10 tx=0x7f00c0009370 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:05.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.439+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00bc053a20 con 0x7f00c40b4e70 2023-10-06T03:08:05.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.651+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f00c40cbaf0 con 0x7f00a8067290 2023-10-06T03:08:05.656 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:08:05.656+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:08:05.656 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:08:05.656+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:08:05.657 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:08:05.656+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:08:05.657 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:08:05.656+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:08:05.657 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:08:05.656+0000 7fdd0eb80700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:08:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.661+0000 7f00b9ffb700 1 -- 172.21.15.92:0/1246161821 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+472 (secure 0 0 0) 0x7f00c40cbaf0 con 0x7f00a8067290 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "mon_addrs": [ 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.92:6789", 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6789", 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "172.21.15.136:6790" 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "pools": { 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "data": [ 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 365977665536, 2023-10-06T03:08:05.662 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_data", 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 0 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: ], 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: "metadata": [ 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: "avail": 365977665536, 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs_metadata", 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: "used": 65536 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:05.663 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:08:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00a8067290 msgr2=0x7f00a8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00a8067290 0x7f00a8069740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f00c000ad10 tx=0x7f00c0009370 comp rx=0 tx=0).stop 2023-10-06T03:08:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 msgr2=0x7f00c40c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 0x7f00c40c5390 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0093f0 tx=0x7f00bc016000 comp rx=0 tx=0).stop 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 shutdown_connections 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00a8067290 0x7f00a8069740 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c40b4e70 0x7f00c40c5390 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c40b16d0 0x7f00c40cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 --2- 172.21.15.92:0/1246161821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1b40 0x7f00c4144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 >> 172.21.15.92:0/1246161821 conn(0x7f00c40d10c0 msgr2=0x7f00c4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 shutdown_connections 2023-10-06T03:08:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:05.663+0000 7f00cac54700 1 -- 172.21.15.92:0/1246161821 wait complete. 2023-10-06T03:08:05.676 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:08:05.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:05.719 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:05.761 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:08:05.761 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:08:05.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:08:05.761 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:05.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:08:05.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:05.800 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:08:05.855 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:08:05.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:08:05.856 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:05.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:08:05.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:05.898 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:08:05.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:06.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.069+0000 7fac5e4f5700 1 Processor -- start 2023-10-06T03:08:06.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.069+0000 7fac5e4f5700 1 -- start start 2023-10-06T03:08:06.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac5e4f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac581578b0 0x7fac58155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac5e4f5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fac580af2f0 0x7fac580af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac5e4f5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac580ad930 con 0x7fac58155c10 2023-10-06T03:08:06.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac5e4f5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580b0c80 con 0x7fac581578b0 2023-10-06T03:08:06.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac5e4f5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580acd20 con 0x7fac580af2f0 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac581578b0 0x7fac58155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac581578b0 0x7fac58155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38524/0 (socket says 172.21.15.92:38524) 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.070+0000 7fac577fe700 1 -- 172.21.15.92:0/2554909864 learned_addr learned my addr 172.21.15.92:0/2554909864 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5ca92700 1 --2- 172.21.15.92:0/2554909864 >> v2:172.21.15.136:3300/0 conn(0x7fac580af2f0 0x7fac580af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac577fe700 1 -- 172.21.15.92:0/2554909864 >> v2:172.21.15.136:3300/0 conn(0x7fac580af2f0 msgr2=0x7fac580af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac577fe700 1 --2- 172.21.15.92:0/2554909864 >> v2:172.21.15.136:3300/0 conn(0x7fac580af2f0 0x7fac580af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac577fe700 1 -- 172.21.15.92:0/2554909864 >> v1:172.21.15.136:6789/0 conn(0x7fac58155c10 legacy=0x7fac580ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac577fe700 1 -- 172.21.15.92:0/2554909864 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580ad000 con 0x7fac581578b0 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac577fe700 1 --2- 172.21.15.92:0/2554909864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac581578b0 0x7fac58155fe0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fac58065be0 tx=0x7fac4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb18f4327a35319 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac56ffd700 1 -- 172.21.15.92:0/2554909864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00b820 con 0x7fac581578b0 2023-10-06T03:08:06.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac56ffd700 1 -- 172.21.15.92:0/2554909864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c00b980 con 0x7fac581578b0 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac56ffd700 1 -- 172.21.15.92:0/2554909864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00bd20 con 0x7fac581578b0 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5e4f5700 1 -- 172.21.15.92:0/2554909864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac581578b0 msgr2=0x7fac58155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/2554909864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac581578b0 0x7fac58155fe0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fac58065be0 tx=0x7fac4c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5e4f5700 1 -- 172.21.15.92:0/2554909864 shutdown_connections 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/2554909864 >> v2:172.21.15.136:3300/0 conn(0x7fac580af2f0 0x7fac580af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/2554909864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac581578b0 0x7fac58155fe0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.071+0000 7fac5e4f5700 1 -- 172.21.15.92:0/2554909864 >> 172.21.15.92:0/2554909864 conn(0x7fac5806a320 msgr2=0x7fac580cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.072+0000 7fac5e4f5700 1 -- 172.21.15.92:0/2554909864 shutdown_connections 2023-10-06T03:08:06.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.072+0000 7fac5e4f5700 1 -- 172.21.15.92:0/2554909864 wait complete. 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 Processor -- start 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 -- start start 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 0x7fac580cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac58155c10 0x7fac58140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac581578b0 0x7fac5814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5814a990 con 0x7fac580af2f0 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580cba30 con 0x7fac581578b0 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.073+0000 7fac5e4f5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5814ac00 con 0x7fac58155c10 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 0x7fac580cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 0x7fac580cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38540/0 (socket says 172.21.15.92:38540) 2023-10-06T03:08:06.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 -- 172.21.15.92:0/1883994653 learned_addr learned my addr 172.21.15.92:0/1883994653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac5ca92700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac581578b0 0x7fac5814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac577fe700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac58155c10 0x7fac58140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 -- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac58155c10 msgr2=0x7fac58140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac58155c10 0x7fac58140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 -- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac581578b0 msgr2=0x7fac5814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac581578b0 0x7fac5814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 -- 172.21.15.92:0/1883994653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0094f0 con 0x7fac580af2f0 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac5ca92700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac581578b0 0x7fac5814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac577fe700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac58155c10 0x7fac58140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac57fff700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 0x7fac580cce50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c600 tx=0x7fac48007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48011590 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac48011710 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac58140e60 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48011ad0 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.074+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac580cb750 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.075+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5804c3c0 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.076+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fac4801c460 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.076+0000 7fac3ffff700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac44067290 0x7fac44069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.076+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fac48084960 con 0x7fac580af2f0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.076+0000 7fac577fe700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac44067290 0x7fac44069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.076+0000 7fac577fe700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac44067290 0x7fac44069740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac58065be0 tx=0x7fac4c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.080+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac48051660 con 0x7fac580af2f0 2023-10-06T03:08:06.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.305+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac5814aee0 con 0x7fac580af2f0 2023-10-06T03:08:06.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.306+0000 7fac3ffff700 1 -- 172.21.15.92:0/1883994653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fac48054c80 con 0x7fac580af2f0 2023-10-06T03:08:06.307 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:06.307 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:49.932559+0000","modified":"2023-10-06T03:07:53.569016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/1523868194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1523868194},{"type":"v1","addr":"172.21.15.92:6837","nonce":1523868194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:08:06.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac44067290 msgr2=0x7fac44069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac44067290 0x7fac44069740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac58065be0 tx=0x7fac4c016000 comp rx=0 tx=0).stop 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 msgr2=0x7fac580cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 0x7fac580cce50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c600 tx=0x7fac48007f60 comp rx=0 tx=0).stop 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 shutdown_connections 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac44067290 0x7fac44069740 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac580af2f0 0x7fac580cce50 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac581578b0 0x7fac5814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 --2- 172.21.15.92:0/1883994653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac58155c10 0x7fac58140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 >> 172.21.15.92:0/1883994653 conn(0x7fac5806a320 msgr2=0x7fac5814d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 shutdown_connections 2023-10-06T03:08:06.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.308+0000 7fac5e4f5700 1 -- 172.21.15.92:0/1883994653 wait complete. 2023-10-06T03:08:06.310 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 69 2023-10-06T03:08:06.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.435+0000 7fe53ed39700 1 Processor -- start 2023-10-06T03:08:06.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.435+0000 7fe53ed39700 1 -- start start 2023-10-06T03:08:06.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe53ed39700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe538159890 0x7fe53815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe53ed39700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 0x7fe5380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe53ed39700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe538069ee0 con 0x7fe538159c60 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe53ed39700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53815a160 con 0x7fe538159890 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe53ed39700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380d13a0 con 0x7fe538069560 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe52ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe538159c60 0x7fe53815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35110/0 (socket says 172.21.15.92:35110) 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.436+0000 7fe52ffff700 1 -- 172.21.15.92:0/346876956 learned_addr learned my addr 172.21.15.92:0/346876956 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53cad5700 1 --2- 172.21.15.92:0/346876956 >> v2:172.21.15.136:3300/0 conn(0x7fe538159890 0x7fe53815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696882240 0 0) 0x7fe538069ee0 con 0x7fe538159c60 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 0x7fe5380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe520003570 con 0x7fe538159c60 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 837040506 0 0) 0x7fe520003570 con 0x7fe538159c60 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 msgr2=0x7fe5380699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 0x7fe5380699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 >> v2:172.21.15.136:3300/0 conn(0x7fe538159890 msgr2=0x7fe53815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 --2- 172.21.15.92:0/346876956 >> v2:172.21.15.136:3300/0 conn(0x7fe538159890 0x7fe53815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53815e140 con 0x7fe538159c60 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fe528001470 con 0x7fe538159c60 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 >> v1:172.21.15.92:6789/0 conn(0x7fe538159c60 legacy=0x7fe53815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe520003a60 0x7fe520003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 0x7fe520004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe5200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53815a160 con 0x7fe520004d50 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380d13a0 con 0x7fe520003a60 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52000ba60 con 0x7fe538069560 2023-10-06T03:08:06.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe5200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53cad5700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe520003a60 0x7fe520003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe52ffff700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 0x7fe520004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 -- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 msgr2=0x7fe520004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe538069560 0x7fe520004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 -- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe520003a60 msgr2=0x7fe520003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe520003a60 0x7fe520003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.437+0000 7fe53d2d6700 1 -- 172.21.15.92:0/346876956 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520003570 con 0x7fe520004d50 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53d2d6700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe5200051c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe53000e210 tx=0x7fe53000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718a9730bd6f83e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000ea60 con 0x7fe520004d50 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53000f040 con 0x7fe520004d50 2023-10-06T03:08:06.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe52f7fe700 1 -- 172.21.15.92:0/346876956 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000ee10 con 0x7fe520004d50 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53ed39700 1 -- 172.21.15.92:0/346876956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 msgr2=0x7fe5200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53ed39700 1 --2- 172.21.15.92:0/346876956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe5200051c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe53000e210 tx=0x7fe53000b040 comp rx=0 tx=0).stop 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53d2d6700 1 -- 172.21.15.92:0/346876956 reap_dead start 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53ed39700 1 -- 172.21.15.92:0/346876956 shutdown_connections 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53ed39700 1 -- 172.21.15.92:0/346876956 >> 172.21.15.92:0/346876956 conn(0x7fe538068e20 msgr2=0x7fe5380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53ed39700 1 -- 172.21.15.92:0/346876956 shutdown_connections 2023-10-06T03:08:06.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.438+0000 7fe53ed39700 1 -- 172.21.15.92:0/346876956 wait complete. 2023-10-06T03:08:06.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.439+0000 7fe53ed39700 1 Processor -- start 2023-10-06T03:08:06.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.439+0000 7fe53ed39700 1 -- start start 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53ed39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe538151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53ed39700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe53815f3f0 0x7fe53815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53ed39700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53815d0b0 0x7fe53815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53ed39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53815fda0 con 0x7fe520004d50 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53ed39700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b4060 con 0x7fe53815d0b0 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53ed39700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b3730 con 0x7fe53815f3f0 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53cad5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe538151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53cad5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe538151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38560/0 (socket says 172.21.15.92:38560) 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53cad5700 1 -- 172.21.15.92:0/644168789 learned_addr learned my addr 172.21.15.92:0/644168789 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe52ffff700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe53815f3f0 0x7fe53815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53d2d6700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53815d0b0 0x7fe53815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.440+0000 7fe53cad5700 1 -- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe53815f3f0 msgr2=0x7fe53815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53cad5700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe53815f3f0 0x7fe53815f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53cad5700 1 -- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53815d0b0 msgr2=0x7fe53815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53cad5700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53815d0b0 0x7fe53815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53cad5700 1 -- 172.21.15.92:0/644168789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380b3a10 con 0x7fe520004d50 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53cad5700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe538151910 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe534006ac0 tx=0x7fe5340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400fa10 con 0x7fe520004d50 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53400fb70 con 0x7fe520004d50 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5380b3cf0 con 0x7fe520004d50 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534014440 con 0x7fe520004d50 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.441+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5380b44f0 con 0x7fe520004d50 2023-10-06T03:08:06.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.442+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53804c3c0 con 0x7fe520004d50 2023-10-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.442+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe534014830 con 0x7fe520004d50 2023-10-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.443+0000 7fe52cff9700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe524067290 0x7fe524069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.443+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe534018030 con 0x7fe520004d50 2023-10-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.443+0000 7fe52ffff700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe524067290 0x7fe524069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.443+0000 7fe52ffff700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe524067290 0x7fe524069740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe5381592b0 tx=0x7fe528009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.447+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe534052320 con 0x7fe520004d50 2023-10-06T03:08:06.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.636+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe538069cc0 con 0x7fe520004d50 2023-10-06T03:08:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.636+0000 7fe52cff9700 1 -- 172.21.15.92:0/644168789 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fe534055940 con 0x7fe520004d50 2023-10-06T03:08:06.637 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:06.637 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":103,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:51.555867+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:47.608728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:48.905445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe524067290 msgr2=0x7fe524069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe524067290 0x7fe524069740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe5381592b0 tx=0x7fe528009040 comp rx=0 tx=0).stop 2023-10-06T03:08:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 msgr2=0x7fe538151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe538151910 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fe534006ac0 tx=0x7fe5340091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 shutdown_connections 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe524067290 0x7fe524069740 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe520004d50 0x7fe538151910 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53815d0b0 0x7fe53815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 --2- 172.21.15.92:0/644168789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe53815f3f0 0x7fe53815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.638+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 >> 172.21.15.92:0/644168789 conn(0x7fe538068e20 msgr2=0x7fe5380c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.639+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 shutdown_connections 2023-10-06T03:08:06.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.639+0000 7fe53ed39700 1 -- 172.21.15.92:0/644168789 wait complete. 2023-10-06T03:08:06.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:07:47.608728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:07:48.905445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:08:06.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:08:06.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.768+0000 7fba0b163700 1 Processor -- start 2023-10-06T03:08:06.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.768+0000 7fba0b163700 1 -- start start 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fba0b163700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fba040caa90 0x7fba040c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fba0b163700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0415d7e0 con 0x7fba0406c0a0 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fba0b163700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0415dac0 con 0x7fba0406bcd0 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fba0b163700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040caef0 con 0x7fba040caa90 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fb9fbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fba0406c0a0 0x7fba0415df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35118/0 (socket says 172.21.15.92:35118) 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fb9fbfff700 1 -- 172.21.15.92:0/4073499280 learned_addr learned my addr 172.21.15.92:0/4073499280 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fba09700700 1 --2- 172.21.15.92:0/4073499280 >> v2:172.21.15.136:3300/0 conn(0x7fba040caa90 0x7fba040c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169381948 0 0) 0x7fba0415d7e0 con 0x7fba0406c0a0 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.769+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec003570 con 0x7fba0406c0a0 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744477709 0 0) 0x7fb9ec003570 con 0x7fba0406c0a0 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 >> v2:172.21.15.136:3300/0 conn(0x7fba040caa90 msgr2=0x7fba040c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 --2- 172.21.15.92:0/4073499280 >> v2:172.21.15.136:3300/0 conn(0x7fba040caa90 0x7fba040c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 >> v1:172.21.15.136:6789/0 conn(0x7fba0406bcd0 legacy=0x7fba0415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0415aa30 con 0x7fba0406c0a0 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb9f4001470 con 0x7fba0406c0a0 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 >> v1:172.21.15.92:6789/0 conn(0x7fba0406c0a0 legacy=0x7fba0415df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9ec003a60 0x7fb9ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9ec004810 0x7fb9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fb9ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040caef0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec00c2a0 con 0x7fb9ec004810 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec00c460 con 0x7fb9ec003a60 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fb9ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fbfff700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9ec004810 0x7fb9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba08eff700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9ec003a60 0x7fb9ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 -- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9ec003a60 msgr2=0x7fb9ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9ec003a60 0x7fb9ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 -- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9ec004810 msgr2=0x7fb9ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:06.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9ec004810 0x7fb9ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 -- 172.21.15.92:0/4073499280 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec003570 con 0x7fb9ec0052e0 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 -- 172.21.15.92:0/4073499280 reap_dead start 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fba09700700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fb9ec00ba60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009c00 tx=0x7fb9fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c854dffed72b590b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00e710 con 0x7fb9ec0052e0 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00e870 con 0x7fb9ec0052e0 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.770+0000 7fb9fb7fe700 1 -- 172.21.15.92:0/4073499280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00ec30 con 0x7fb9ec0052e0 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 -- 172.21.15.92:0/4073499280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 msgr2=0x7fb9ec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fb9ec00ba60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009c00 tx=0x7fb9fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 -- 172.21.15.92:0/4073499280 shutdown_connections 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 --2- 172.21.15.92:0/4073499280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fb9ec00ba60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 -- 172.21.15.92:0/4073499280 >> 172.21.15.92:0/4073499280 conn(0x7fba040ce600 msgr2=0x7fba040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:06.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 -- 172.21.15.92:0/4073499280 shutdown_connections 2023-10-06T03:08:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.771+0000 7fba0b163700 1 -- 172.21.15.92:0/4073499280 wait complete. 2023-10-06T03:08:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.772+0000 7fba0b163700 1 Processor -- start 2023-10-06T03:08:06.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.772+0000 7fba0b163700 1 -- start start 2023-10-06T03:08:06.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba0b163700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fba04146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba0b163700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba04069500 0x7fba04069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba0b163700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba040c56b0 0x7fba040c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba0b163700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04069eb0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba0b163700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c8900 con 0x7fba040c56b0 2023-10-06T03:08:06.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba0b163700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c8be0 con 0x7fba04069500 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fba04146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fba04146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38578/0 (socket says 172.21.15.92:38578) 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 -- 172.21.15.92:0/4038972851 learned_addr learned my addr 172.21.15.92:0/4038972851 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba09700700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba040c56b0 0x7fba040c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fb9fbfff700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba04069500 0x7fba04069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 -- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba04069500 msgr2=0x7fba04069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba04069500 0x7fba04069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 -- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba040c56b0 msgr2=0x7fba040c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba040c56b0 0x7fba040c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 -- 172.21.15.92:0/4038972851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040c8ec0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.773+0000 7fba08eff700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fba04146e60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fba0000e8d0 tx=0x7fba00009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.774+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000ee10 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.774+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba00010040 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.774+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba040c91a0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.774+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000a5b0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.774+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba041473a0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.774+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0404c3c0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.775+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fba0000a920 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.775+0000 7fb9f8ff9700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb9f0067290 0x7fb9f0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.775+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba00084800 con 0x7fb9ec0052e0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.775+0000 7fb9fbfff700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb9f0067290 0x7fb9f0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:06.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.776+0000 7fb9fbfff700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb9f0067290 0x7fb9f0069740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000c00 tx=0x7fb9f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:06.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.779+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba000513d0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.960+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba040c97f0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.961+0000 7fb9f8ff9700 1 -- 172.21.15.92:0/4038972851 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba000549f0 con 0x7fb9ec0052e0 2023-10-06T03:08:06.962 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:08:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb9f0067290 msgr2=0x7fb9f0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb9f0067290 0x7fb9f0069740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4000c00 tx=0x7fb9f4009040 comp rx=0 tx=0).stop 2023-10-06T03:08:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 msgr2=0x7fba04146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fba04146e60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fba0000e8d0 tx=0x7fba00009040 comp rx=0 tx=0).stop 2023-10-06T03:08:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 shutdown_connections 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb9f0067290 0x7fb9f0069740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb9ec0052e0 0x7fba04146e60 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba040c56b0 0x7fba040c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 --2- 172.21.15.92:0/4038972851 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba04069500 0x7fba04069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 >> 172.21.15.92:0/4038972851 conn(0x7fba040ce600 msgr2=0x7fba040c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 shutdown_connections 2023-10-06T03:08:06.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:06.963+0000 7fba0b163700 1 -- 172.21.15.92:0/4038972851 wait complete. 2023-10-06T03:08:06.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.914+0000 7fd7913af700 1 Processor -- start 2023-10-06T03:08:07.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd7913af700 1 -- start start 2023-10-06T03:08:07.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd7913af700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd78c141500 0x7fd78c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd7913af700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd78c06c0a0 0x7fd78c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd7913af700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c15c2a0 con 0x7fd78c06bcd0 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd7913af700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c0c9880 con 0x7fd78c06c0a0 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd7913af700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c15b690 con 0x7fd78c141500 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd78b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd78c06c0a0 0x7fd78c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd78b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd78c06c0a0 0x7fd78c0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38584/0 (socket says 172.21.15.92:38584) 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.915+0000 7fd78b7fe700 1 -- 172.21.15.92:0/2902841631 learned_addr learned my addr 172.21.15.92:0/2902841631 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd7827fc700 1 --2- 172.21.15.92:0/2902841631 >> v2:172.21.15.136:3300/0 conn(0x7fd78c141500 0x7fd78c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78b7fe700 1 -- 172.21.15.92:0/2902841631 >> v2:172.21.15.136:3300/0 conn(0x7fd78c141500 msgr2=0x7fd78c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:07.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78b7fe700 1 --2- 172.21.15.92:0/2902841631 >> v2:172.21.15.136:3300/0 conn(0x7fd78c141500 0x7fd78c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78b7fe700 1 -- 172.21.15.92:0/2902841631 >> v1:172.21.15.136:6789/0 conn(0x7fd78c06bcd0 legacy=0x7fd78c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78b7fe700 1 -- 172.21.15.92:0/2902841631 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c15b970 con 0x7fd78c06c0a0 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78b7fe700 1 --2- 172.21.15.92:0/2902841631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c0c9340 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd78c06c6e0 tx=0x7fd784009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd3c0108681ba4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78a7fc700 1 -- 172.21.15.92:0/2902841631 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78400e070 con 0x7fd78c06c0a0 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78a7fc700 1 -- 172.21.15.92:0/2902841631 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400ba10 con 0x7fd78c06c0a0 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd78a7fc700 1 -- 172.21.15.92:0/2902841631 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840092e0 con 0x7fd78c06c0a0 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd7913af700 1 -- 172.21.15.92:0/2902841631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 msgr2=0x7fd78c0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.916+0000 7fd7913af700 1 --2- 172.21.15.92:0/2902841631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c0c9340 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd78c06c6e0 tx=0x7fd784009840 comp rx=0 tx=0).stop 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.917+0000 7fd7913af700 1 -- 172.21.15.92:0/2902841631 shutdown_connections 2023-10-06T03:08:07.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.917+0000 7fd7913af700 1 --2- 172.21.15.92:0/2902841631 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c0c9340 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.917+0000 7fd7913af700 1 --2- 172.21.15.92:0/2902841631 >> v2:172.21.15.136:3300/0 conn(0x7fd78c141500 0x7fd78c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.917+0000 7fd7913af700 1 -- 172.21.15.92:0/2902841631 >> 172.21.15.92:0/2902841631 conn(0x7fd78c0ce600 msgr2=0x7fd78c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.917+0000 7fd7913af700 1 -- 172.21.15.92:0/2902841631 shutdown_connections 2023-10-06T03:08:07.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.917+0000 7fd7913af700 1 -- 172.21.15.92:0/2902841631 wait complete. 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.918+0000 7fd7913af700 1 Processor -- start 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.918+0000 7fd7913af700 1 -- start start 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd78c06bcd0 0x7fd78c16af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd78c141500 0x7fd78c139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c06aa60 con 0x7fd78c06c0a0 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c16b450 con 0x7fd78c06bcd0 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c069980 con 0x7fd78c141500 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c13f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38598/0 (socket says 172.21.15.92:38598) 2023-10-06T03:08:07.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 -- 172.21.15.92:0/1301578657 learned_addr learned my addr 172.21.15.92:0/1301578657 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd78affd700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd78c06bcd0 0x7fd78c16af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd78b7fe700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd78c141500 0x7fd78c139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 -- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd78c141500 msgr2=0x7fd78c139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd78c141500 0x7fd78c139910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 -- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd78c06bcd0 msgr2=0x7fd78c16af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd78c06bcd0 0x7fd78c16af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 -- 172.21.15.92:0/1301578657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784009ae0 con 0x7fd78c06c0a0 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd78affd700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd78c06bcd0 0x7fd78c16af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7827fc700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c13f100 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd77400d860 tx=0x7fd774008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774009610 con 0x7fd78c06c0a0 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd774009770 con 0x7fd78c06c0a0 2023-10-06T03:08:07.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774009b30 con 0x7fd78c06c0a0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c069c60 con 0x7fd78c06c0a0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.919+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c06a220 con 0x7fd78c06c0a0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.920+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c0b2a60 con 0x7fd78c06c0a0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.921+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd77401e460 con 0x7fd78c06c0a0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.921+0000 7fd7837fe700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd76c067290 0x7fd76c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.921+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd774085280 con 0x7fd78c06c0a0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.921+0000 7fd78affd700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd76c067290 0x7fd76c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:07.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.922+0000 7fd78affd700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd76c067290 0x7fd76c069740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd77c002880 tx=0x7fd77c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:07.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:07.926+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd774051f40 con 0x7fd78c06c0a0 2023-10-06T03:08:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.106+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd78c15bc50 con 0x7fd78c06c0a0 2023-10-06T03:08:08.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.107+0000 7fd7837fe700 1 -- 172.21.15.92:0/1301578657 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd774055560 con 0x7fd78c06c0a0 2023-10-06T03:08:08.108 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd76c067290 msgr2=0x7fd76c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd76c067290 0x7fd76c069740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd77c002880 tx=0x7fd77c009040 comp rx=0 tx=0).stop 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 msgr2=0x7fd78c13f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c13f100 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fd77400d860 tx=0x7fd774008040 comp rx=0 tx=0).stop 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 shutdown_connections 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd76c067290 0x7fd76c069740 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd78c141500 0x7fd78c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd78c06bcd0 0x7fd78c16af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 --2- 172.21.15.92:0/1301578657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd78c06c0a0 0x7fd78c13f100 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 >> 172.21.15.92:0/1301578657 conn(0x7fd78c0ce600 msgr2=0x7fd78c06d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 shutdown_connections 2023-10-06T03:08:08.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.109+0000 7fd7913af700 1 -- 172.21.15.92:0/1301578657 wait complete. 2023-10-06T03:08:08.121 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:08:08.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.238+0000 7f8adb016700 1 Processor -- start 2023-10-06T03:08:08.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.238+0000 7f8adb016700 1 -- start start 2023-10-06T03:08:08.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8adb016700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 0x7f8ad4069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8adb016700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad40c9f30 con 0x7f8ad4159a60 2023-10-06T03:08:08.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8adb016700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad40ca470 con 0x7f8ad406a020 2023-10-06T03:08:08.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8adb016700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad406dfc0 con 0x7f8ad40ccc00 2023-10-06T03:08:08.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8ada014700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8ad4159a60 0x7f8ad40c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35134/0 (socket says 172.21.15.92:35134) 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8ada014700 1 -- 172.21.15.92:0/628929540 learned_addr learned my addr 172.21.15.92:0/628929540 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487489798 0 0) 0x7f8ad40c9f30 con 0x7f8ad4159a60 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac0003570 con 0x7f8ad4159a60 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.239+0000 7f8ad9813700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 0x7f8ad4069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335281806 0 0) 0x7f8ac0003570 con 0x7f8ad4159a60 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 msgr2=0x7f8ad4069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 0x7f8ad4069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 >> v1:172.21.15.136:6789/0 conn(0x7f8ad406a020 legacy=0x7f8ad40cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40ca750 con 0x7f8ad4159a60 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8ad00017f0 con 0x7f8ad4159a60 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 >> v1:172.21.15.92:6789/0 conn(0x7f8ad4159a60 legacy=0x7f8ad40c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 0x7f8ac0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0003fd0 0x7f8ac0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ac0004d50 0x7f8ac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad406dfc0 con 0x7f8ac0004d50 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac000bb40 con 0x7f8ac0003fd0 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac000bd00 con 0x7f8ad40ccc00 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ac0004d50 0x7f8ac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada014700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 0x7f8ac0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9813700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0003fd0 0x7f8ac0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 -- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 msgr2=0x7f8ac0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad40ccc00 0x7f8ac0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 -- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0003fd0 msgr2=0x7f8ac0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0003fd0 0x7f8ac0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 -- 172.21.15.92:0/628929540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0003570 con 0x7f8ac0004d50 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ada815700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ac0004d50 0x7f8ac00051c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0019a0 tx=0x7f8acc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddb3a64904466ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00c5d0 con 0x7f8ac0004d50 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00c750 con 0x7f8ac0004d50 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.240+0000 7f8ad9012700 1 -- 172.21.15.92:0/628929540 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00cb10 con 0x7f8ac0004d50 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8adb016700 1 -- 172.21.15.92:0/628929540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ac0004d50 msgr2=0x7f8ac00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8adb016700 1 --2- 172.21.15.92:0/628929540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ac0004d50 0x7f8ac00051c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8acc0019a0 tx=0x7f8acc007d10 comp rx=0 tx=0).stop 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8ada815700 1 -- 172.21.15.92:0/628929540 reap_dead start 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8adb016700 1 -- 172.21.15.92:0/628929540 shutdown_connections 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8adb016700 1 -- 172.21.15.92:0/628929540 >> 172.21.15.92:0/628929540 conn(0x7f8ad406baa0 msgr2=0x7f8ad406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8adb016700 1 -- 172.21.15.92:0/628929540 shutdown_connections 2023-10-06T03:08:08.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.241+0000 7f8adb016700 1 -- 172.21.15.92:0/628929540 wait complete. 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.242+0000 7f8adb016700 1 Processor -- start 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.242+0000 7f8adb016700 1 -- start start 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8adb016700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0004d50 0x7f8ad4152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8adb016700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 0x7f8ad40b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8adb016700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad415d680 0x7f8ad415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8adb016700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40b4120 con 0x7f8ad40b1430 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8adb016700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40b1de0 con 0x7f8ac0004d50 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8adb016700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad4152d10 con 0x7f8ad415d680 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 0x7f8ad40b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 0x7f8ad40b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38614/0 (socket says 172.21.15.92:38614) 2023-10-06T03:08:08.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 -- 172.21.15.92:0/2048846975 learned_addr learned my addr 172.21.15.92:0/2048846975 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ada014700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0004d50 0x7f8ad4152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ada815700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad415d680 0x7f8ad415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 -- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad415d680 msgr2=0x7f8ad415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad415d680 0x7f8ad415dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 -- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0004d50 msgr2=0x7f8ad4152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0004d50 0x7f8ad4152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 -- 172.21.15.92:0/2048846975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad415e390 con 0x7f8ad40b1430 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.243+0000 7f8ad9813700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 0x7f8ad40b18a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4011b10 tx=0x7f8ac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.244+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400f9f0 con 0x7f8ad40b1430 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.244+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac400fb50 con 0x7f8ad40b1430 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.244+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad415e670 con 0x7f8ad40b1430 2023-10-06T03:08:08.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.244+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400fed0 con 0x7f8ad40b1430 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.244+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad415ec30 con 0x7f8ad40b1430 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.245+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad4152ec0 con 0x7f8ad40b1430 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.245+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8ac4014650 con 0x7f8ad40b1430 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.246+0000 7f8aca7fc700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ab8067290 0x7f8ab8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.246+0000 7f8ada014700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ab8067290 0x7f8ab8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.246+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ac4018030 con 0x7f8ad40b1430 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.246+0000 7f8ada014700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ab8067290 0x7f8ab8069740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00011d0 tx=0x7f8ad00021f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.250+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ad4152ec0 con 0x7f8ad40b1430 2023-10-06T03:08:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.431+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ad4152e80 con 0x7f8ad40b1430 2023-10-06T03:08:08.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.432+0000 7f8aca7fc700 1 -- 172.21.15.92:0/2048846975 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v103) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ac4052360 con 0x7f8ad40b1430 2023-10-06T03:08:08.433 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ab8067290 msgr2=0x7f8ab8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ab8067290 0x7f8ab8069740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00011d0 tx=0x7f8ad00021f0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 msgr2=0x7f8ad40b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 0x7f8ad40b18a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4011b10 tx=0x7f8ac40091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 shutdown_connections 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ab8067290 0x7f8ab8069740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ad415d680 0x7f8ad415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ac0004d50 0x7f8ad4152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 --2- 172.21.15.92:0/2048846975 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ad40b1430 0x7f8ad40b18a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 >> 172.21.15.92:0/2048846975 conn(0x7f8ad406baa0 msgr2=0x7f8ad40cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:08.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 shutdown_connections 2023-10-06T03:08:08.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.434+0000 7f8adb016700 1 -- 172.21.15.92:0/2048846975 wait complete. 2023-10-06T03:08:08.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.565+0000 7f013d7a2700 1 Processor -- start 2023-10-06T03:08:08.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.565+0000 7f013d7a2700 1 -- start start 2023-10-06T03:08:08.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013d7a2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01380c93a0 0x7f013806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013d7a2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01380c9770 0x7f01380caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013d7a2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01380af490 con 0x7f01380cb400 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013d7a2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01380af770 con 0x7f01380c93a0 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013d7a2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01380afa50 con 0x7f01380c9770 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f0137fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f01380c93a0 0x7f013806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013cfa1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f01380cb400 0x7f01380c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35142/0 (socket says 172.21.15.92:35142) 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.566+0000 7f013cfa1700 1 -- 172.21.15.92:0/1537108364 learned_addr learned my addr 172.21.15.92:0/1537108364 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:08.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0137fff700 1 -- 172.21.15.92:0/1537108364 >> v2:172.21.15.136:3300/0 conn(0x7f01380c9770 msgr2=0x7f01380caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f01377fe700 1 --2- 172.21.15.92:0/1537108364 >> v2:172.21.15.136:3300/0 conn(0x7f01380c9770 0x7f01380caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0136ffd700 1 -- 172.21.15.92:0/1537108364 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649960671 0 0) 0x7f01380af490 con 0x7f01380cb400 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0137fff700 1 --2- 172.21.15.92:0/1537108364 >> v2:172.21.15.136:3300/0 conn(0x7f01380c9770 0x7f01380caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0137fff700 1 -- 172.21.15.92:0/1537108364 >> v1:172.21.15.92:6789/0 conn(0x7f01380cb400 legacy=0x7f01380c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0137fff700 1 -- 172.21.15.92:0/1537108364 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380c7ab0 con 0x7f01380c93a0 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f01377fe700 1 --2- 172.21.15.92:0/1537108364 >> v2:172.21.15.136:3300/0 conn(0x7f01380c9770 0x7f01380caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0137fff700 1 --2- 172.21.15.92:0/1537108364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c93a0 0x7f013806c980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f01380c7880 tx=0x7f01280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6c5894dea94ba2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0136ffd700 1 -- 172.21.15.92:0/1537108364 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012800b6b0 con 0x7f01380c93a0 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0136ffd700 1 -- 172.21.15.92:0/1537108364 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012800b810 con 0x7f01380c93a0 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f0136ffd700 1 -- 172.21.15.92:0/1537108364 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012800bbb0 con 0x7f01380c93a0 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f013d7a2700 1 -- 172.21.15.92:0/1537108364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c93a0 msgr2=0x7f013806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.567+0000 7f013d7a2700 1 --2- 172.21.15.92:0/1537108364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c93a0 0x7f013806c980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f01380c7880 tx=0x7f01280091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.568+0000 7f013d7a2700 1 -- 172.21.15.92:0/1537108364 shutdown_connections 2023-10-06T03:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.568+0000 7f013d7a2700 1 --2- 172.21.15.92:0/1537108364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c93a0 0x7f013806c980 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.568+0000 7f013d7a2700 1 --2- 172.21.15.92:0/1537108364 >> v2:172.21.15.136:3300/0 conn(0x7f01380c9770 0x7f01380caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.568+0000 7f013d7a2700 1 -- 172.21.15.92:0/1537108364 >> 172.21.15.92:0/1537108364 conn(0x7f01380cddc0 msgr2=0x7f01380c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.568+0000 7f013d7a2700 1 -- 172.21.15.92:0/1537108364 shutdown_connections 2023-10-06T03:08:08.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.568+0000 7f013d7a2700 1 -- 172.21.15.92:0/1537108364 wait complete. 2023-10-06T03:08:08.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.569+0000 7f013d7a2700 1 Processor -- start 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.569+0000 7f013d7a2700 1 -- start start 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.569+0000 7f013d7a2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01380c93a0 0x7f01380c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.569+0000 7f013d7a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 0x7f0138168a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013d7a2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01380cb400 0x7f0138154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013d7a2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380c5e30 con 0x7f01380c9770 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013d7a2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013816a340 con 0x7f01380c93a0 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013d7a2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01381697c0 con 0x7f01380cb400 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 0x7f0138168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 0x7f0138168a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38644/0 (socket says 172.21.15.92:38644) 2023-10-06T03:08:08.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 -- 172.21.15.92:0/499671567 learned_addr learned my addr 172.21.15.92:0/499671567 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f0137fff700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01380c93a0 0x7f01380c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013cfa1700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01380cb400 0x7f0138154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 -- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01380cb400 msgr2=0x7f0138154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01380cb400 0x7f0138154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 -- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01380c93a0 msgr2=0x7f01380c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01380c93a0 0x7f01380c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 -- 172.21.15.92:0/499671567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128009420 con 0x7f01380c9770 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f01377fe700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 0x7f0138168a60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c830 tx=0x7f012c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c005a00 con 0x7f01380c9770 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c005b60 con 0x7f01380c9770 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0138169aa0 con 0x7f01380c9770 2023-10-06T03:08:08.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011600 con 0x7f01380c9770 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.570+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013816a060 con 0x7f01380c9770 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.571+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013806d560 con 0x7f01380c9770 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.572+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f012c0119b0 con 0x7f01380c9770 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.572+0000 7f011ffff700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0124067320 0x7f01240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.572+0000 7f0137fff700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0124067320 0x7f01240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.572+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f012c084b40 con 0x7f01380c9770 2023-10-06T03:08:08.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.573+0000 7f0137fff700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0124067320 0x7f01240697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f01280093f0 tx=0x7f0128017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.576+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f012c051560 con 0x7f01380c9770 2023-10-06T03:08:08.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.761+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f013806b500 con 0x7f01380c9770 2023-10-06T03:08:08.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.761+0000 7f011ffff700 1 -- 172.21.15.92:0/499671567 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v103) v1 ==== 100+0+11 (secure 0 0 0) 0x7f012c01e0a0 con 0x7f01380c9770 2023-10-06T03:08:08.762 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:08:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0124067320 msgr2=0x7f01240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0124067320 0x7f01240697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f01280093f0 tx=0x7f0128017000 comp rx=0 tx=0).stop 2023-10-06T03:08:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 msgr2=0x7f0138168a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 0x7f0138168a60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f012c00c830 tx=0x7f012c008040 comp rx=0 tx=0).stop 2023-10-06T03:08:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 shutdown_connections 2023-10-06T03:08:08.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0124067320 0x7f01240697d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01380cb400 0x7f0138154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01380c93a0 0x7f01380c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 --2- 172.21.15.92:0/499671567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f01380c9770 0x7f0138168a60 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 >> 172.21.15.92:0/499671567 conn(0x7f01380cddc0 msgr2=0x7f01380b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 shutdown_connections 2023-10-06T03:08:08.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.763+0000 7f013d7a2700 1 -- 172.21.15.92:0/499671567 wait complete. 2023-10-06T03:08:08.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:08.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.889+0000 7fe4a5e08700 1 Processor -- start 2023-10-06T03:08:08.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.889+0000 7fe4a5e08700 1 -- start start 2023-10-06T03:08:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.890+0000 7fe4a5e08700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe4a0159c60 0x7fe4a015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.890+0000 7fe4a5e08700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a015a160 con 0x7fe4a015db10 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.890+0000 7fe4a5e08700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a00d13a0 con 0x7fe4a0159890 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.890+0000 7fe4a5e08700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a015df50 con 0x7fe4a0159c60 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.890+0000 7fe49ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4a015db10 0x7fe4a0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:35148/0 (socket says 172.21.15.92:35148) 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49ffff700 1 -- 172.21.15.92:0/1680547475 learned_addr learned my addr 172.21.15.92:0/1680547475 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe496ffd700 1 --2- 172.21.15.92:0/1680547475 >> v2:172.21.15.136:3300/0 conn(0x7fe4a0159c60 0x7fe4a015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490080225 0 0) 0x7fe4a015a160 con 0x7fe4a015db10 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48c003570 con 0x7fe4a015db10 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1314032212 0 0) 0x7fe48c003570 con 0x7fe4a015db10 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 >> v2:172.21.15.136:3300/0 conn(0x7fe4a0159c60 msgr2=0x7fe4a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 --2- 172.21.15.92:0/1680547475 >> v2:172.21.15.136:3300/0 conn(0x7fe4a0159c60 0x7fe4a015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 >> v1:172.21.15.136:6789/0 conn(0x7fe4a0159890 legacy=0x7fe4a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a015e140 con 0x7fe4a015db10 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fe4980017f0 con 0x7fe4a015db10 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 >> v1:172.21.15.92:6789/0 conn(0x7fe4a015db10 legacy=0x7fe4a0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe48c003a60 0x7fe48c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe48c004810 0x7fe48c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c0051c0 0x7fe48c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c00c0a0 con 0x7fe48c003a60 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c00c210 con 0x7fe48c004810 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c00c3d0 con 0x7fe48c0051c0 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe48c003a60 0x7fe48c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe496ffd700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe48c004810 0x7fe48c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.891+0000 7fe49ffff700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c0051c0 0x7fe48c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49f7fe700 1 -- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c0051c0 msgr2=0x7fe48c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c0051c0 0x7fe48c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49f7fe700 1 -- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe48c004810 msgr2=0x7fe48c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe48c004810 0x7fe48c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49f7fe700 1 -- 172.21.15.92:0/1680547475 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c003570 con 0x7fe48c003a60 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49ffff700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c0051c0 0x7fe48c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49ffff700 1 -- 172.21.15.92:0/1680547475 reap_dead start 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe48c003a60 0x7fe48c003e70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe4900018b0 tx=0x7fe490007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990fb0d3c60801d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490012420 con 0x7fe48c003a60 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4900125a0 con 0x7fe48c003a60 2023-10-06T03:08:08.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe49effd700 1 -- 172.21.15.92:0/1680547475 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490012960 con 0x7fe48c003a60 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 -- 172.21.15.92:0/1680547475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe48c003a60 msgr2=0x7fe48c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe48c003a60 0x7fe48c003e70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe4900018b0 tx=0x7fe490007d30 comp rx=0 tx=0).stop 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 -- 172.21.15.92:0/1680547475 shutdown_connections 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/1680547475 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe48c003a60 0x7fe48c003e70 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 -- 172.21.15.92:0/1680547475 >> 172.21.15.92:0/1680547475 conn(0x7fe4a0068e20 msgr2=0x7fe4a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 -- 172.21.15.92:0/1680547475 shutdown_connections 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.892+0000 7fe4a5e08700 1 -- 172.21.15.92:0/1680547475 wait complete. 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.893+0000 7fe4a5e08700 1 Processor -- start 2023-10-06T03:08:08.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 -- start start 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c003a60 0x7fe4a0149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 0x7fe4a00b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4a00cf3f0 0x7fe4a00b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a016b320 con 0x7fe4a00cef80 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a016b600 con 0x7fe4a00cf3f0 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe4a5e08700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a016b8e0 con 0x7fe48c003a60 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe496ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 0x7fe4a00b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe496ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 0x7fe4a00b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38658/0 (socket says 172.21.15.92:38658) 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.894+0000 7fe496ffd700 1 -- 172.21.15.92:0/3769631232 learned_addr learned my addr 172.21.15.92:0/3769631232 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c003a60 0x7fe4a0149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe49ffff700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4a00cf3f0 0x7fe4a00b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe496ffd700 1 -- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c003a60 msgr2=0x7fe4a0149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe496ffd700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c003a60 0x7fe4a0149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe496ffd700 1 -- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4a00cf3f0 msgr2=0x7fe4a00b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:08.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe496ffd700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4a00cf3f0 0x7fe4a00b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe496ffd700 1 -- 172.21.15.92:0/3769631232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a016bbc0 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe496ffd700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 0x7fe4a00b1ed0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe488011d40 tx=0x7fe4880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48800fac0 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48800fc20 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a016bea0 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe488014440 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.895+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a00cfcb0 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.896+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a00b0240 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.896+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe488014a80 con 0x7fe4a00cef80 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.897+0000 7fe497fff700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe480067320 0x7fe4800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:08.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.897+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe488018030 con 0x7fe4a00cef80 2023-10-06T03:08:08.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.897+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe480067320 0x7fe4800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:08.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.897+0000 7fe49f7fe700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe480067320 0x7fe4800697d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00d00d0 tx=0x7fe490015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:08.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:08.901+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4880525c0 con 0x7fe4a00cef80 2023-10-06T03:08:09.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.124+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4a004c3c0 con 0x7fe4a00cef80 2023-10-06T03:08:09.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.125+0000 7fe497fff700 1 -- 172.21.15.92:0/3769631232 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fe488022090 con 0x7fe4a00cef80 2023-10-06T03:08:09.125 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:09.126 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:49.932559+0000","modified":"2023-10-06T03:07:53.569016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/1523868194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1523868194},{"type":"v1","addr":"172.21.15.92:6837","nonce":1523868194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe480067320 msgr2=0x7fe4800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe480067320 0x7fe4800697d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00d00d0 tx=0x7fe490015000 comp rx=0 tx=0).stop 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 msgr2=0x7fe4a00b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 0x7fe4a00b1ed0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe488011d40 tx=0x7fe4880091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 shutdown_connections 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe480067320 0x7fe4800697d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4a00cf3f0 0x7fe4a00b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe48c003a60 0x7fe4a0149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 --2- 172.21.15.92:0/3769631232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4a00cef80 0x7fe4a00b1ed0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 >> 172.21.15.92:0/3769631232 conn(0x7fe4a0068e20 msgr2=0x7fe4a015bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:09.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 shutdown_connections 2023-10-06T03:08:09.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.127+0000 7fe4a5e08700 1 -- 172.21.15.92:0/3769631232 wait complete. 2023-10-06T03:08:09.129 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 69 2023-10-06T03:08:09.140 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:09.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 Processor -- start 2023-10-06T03:08:09.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 -- start start 2023-10-06T03:08:09.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f73f4159420 0x7f73f40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73f415d230 0x7f73f415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f40b2980 con 0x7f73f41597f0 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f415db70 con 0x7f73f415d230 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.255+0000 7f73f887c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f40b11d0 con 0x7f73f4159420 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73f415d230 0x7f73f415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f73f415d230 0x7f73f415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38662/0 (socket says 172.21.15.92:38662) 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 -- 172.21.15.92:0/3518424561 learned_addr learned my addr 172.21.15.92:0/3518424561 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f2d9d700 1 --2- 172.21.15.92:0/3518424561 >> v2:172.21.15.136:3300/0 conn(0x7f73f4159420 0x7f73f40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 -- 172.21.15.92:0/3518424561 >> v2:172.21.15.136:3300/0 conn(0x7f73f4159420 msgr2=0x7f73f40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 --2- 172.21.15.92:0/3518424561 >> v2:172.21.15.136:3300/0 conn(0x7f73f4159420 0x7f73f40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 -- 172.21.15.92:0/3518424561 >> v1:172.21.15.136:6789/0 conn(0x7f73f41597f0 legacy=0x7f73f40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 -- 172.21.15.92:0/3518424561 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f40b14b0 con 0x7f73f415d230 2023-10-06T03:08:09.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f359e700 1 --2- 172.21.15.92:0/3518424561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f415d230 0x7f73f415d630 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f73f40b2f20 tx=0x7f73ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1501c1f0f8c0b9dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f259c700 1 -- 172.21.15.92:0/3518424561 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00e070 con 0x7f73f415d230 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f259c700 1 -- 172.21.15.92:0/3518424561 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec00ba20 con 0x7f73f415d230 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.256+0000 7f73f259c700 1 -- 172.21.15.92:0/3518424561 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0092e0 con 0x7f73f415d230 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 -- 172.21.15.92:0/3518424561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f415d230 msgr2=0x7f73f415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 --2- 172.21.15.92:0/3518424561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f415d230 0x7f73f415d630 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f73f40b2f20 tx=0x7f73ec009840 comp rx=0 tx=0).stop 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 -- 172.21.15.92:0/3518424561 shutdown_connections 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 --2- 172.21.15.92:0/3518424561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f415d230 0x7f73f415d630 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 --2- 172.21.15.92:0/3518424561 >> v2:172.21.15.136:3300/0 conn(0x7f73f4159420 0x7f73f40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 -- 172.21.15.92:0/3518424561 >> 172.21.15.92:0/3518424561 conn(0x7f73f40d0660 msgr2=0x7f73f40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 -- 172.21.15.92:0/3518424561 shutdown_connections 2023-10-06T03:08:09.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.257+0000 7f73f887c700 1 -- 172.21.15.92:0/3518424561 wait complete. 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.258+0000 7f73f887c700 1 Processor -- start 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.258+0000 7f73f887c700 1 -- start start 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73f4159420 0x7f73f4155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 0x7f73f40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73f415d230 0x7f73f40c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40c9740 con 0x7f73f41597f0 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40c5400 con 0x7f73f415d230 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f40c5650 con 0x7f73f4159420 2023-10-06T03:08:09.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 0x7f73f40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 0x7f73f40c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38670/0 (socket says 172.21.15.92:38670) 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 -- 172.21.15.92:0/1121228480 learned_addr learned my addr 172.21.15.92:0/1121228480 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f359e700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73f415d230 0x7f73f40c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 -- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73f4159420 msgr2=0x7f73f4155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f2d9d700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73f4159420 0x7f73f4155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73f4159420 0x7f73f4155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 -- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73f415d230 msgr2=0x7f73f40c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73f415d230 0x7f73f40c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 -- 172.21.15.92:0/1121228480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec009ae0 con 0x7f73f41597f0 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73ea59c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 0x7f73f40c66f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c7c0 tx=0x7f73dc007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc011590 con 0x7f73f41597f0 2023-10-06T03:08:09.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc0116f0 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f40c5930 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc011ab0 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.259+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f40c5ef0 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.260+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f404c3c0 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.261+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f73dc01b3b0 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.261+0000 7f73eb7fe700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f73d4067320 0x7f73d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.261+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f73dc0842d0 con 0x7f73f41597f0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.261+0000 7f73f2d9d700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f73d4067320 0x7f73d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.262+0000 7f73f2d9d700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f73d4067320 0x7f73d40697d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f73e40093f0 tx=0x7f73e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.266+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73dc050e00 con 0x7f73f41597f0 2023-10-06T03:08:09.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.450+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73f40b1790 con 0x7f73f41597f0 2023-10-06T03:08:09.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.451+0000 7f73eb7fe700 1 -- 172.21.15.92:0/1121228480 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f73dc017090 con 0x7f73f41597f0 2023-10-06T03:08:09.452 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:09.452 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":103,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:51.555867+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:47.608728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:48.905445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:09.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f73d4067320 msgr2=0x7f73d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f73d4067320 0x7f73d40697d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f73e40093f0 tx=0x7f73e4008040 comp rx=0 tx=0).stop 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 msgr2=0x7f73f40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 0x7f73f40c66f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c7c0 tx=0x7f73dc007f50 comp rx=0 tx=0).stop 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 shutdown_connections 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f73d4067320 0x7f73d40697d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73f415d230 0x7f73f40c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f73f4159420 0x7f73f4155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 --2- 172.21.15.92:0/1121228480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f73f41597f0 0x7f73f40c66f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 >> 172.21.15.92:0/1121228480 conn(0x7f73f40d0660 msgr2=0x7f73f41586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 shutdown_connections 2023-10-06T03:08:09.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.454+0000 7f73f887c700 1 -- 172.21.15.92:0/1121228480 wait complete. 2023-10-06T03:08:09.466 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:08:09.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.580+0000 7f4fd6907700 1 Processor -- start 2023-10-06T03:08:09.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.580+0000 7f4fd6907700 1 -- start start 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fd6907700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159890 0x7f4fd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fd6907700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fd0069560 0x7f4fd00699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fd6907700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd015a160 con 0x7f4fd0159c60 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fd6907700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd0069f20 con 0x7f4fd0069560 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fd6907700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd00d13a0 con 0x7f4fd0159890 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fd4ea4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fd0069560 0x7f4fd00699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fcf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4fd0159c60 0x7f4fd015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49626/0 (socket says 172.21.15.92:49626) 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fcf7fe700 1 -- 172.21.15.92:0/3009281981 learned_addr learned my addr 172.21.15.92:0/3009281981 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:09.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.581+0000 7f4fcffff700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159890 0x7f4fd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd4ea4700 1 -- 172.21.15.92:0/3009281981 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159890 msgr2=0x7f4fd015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd4ea4700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159890 0x7f4fd015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd4ea4700 1 -- 172.21.15.92:0/3009281981 >> v1:172.21.15.92:6789/0 conn(0x7f4fd0159c60 legacy=0x7f4fd015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd4ea4700 1 -- 172.21.15.92:0/3009281981 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd015e140 con 0x7f4fd0069560 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fceffd700 1 -- 172.21.15.92:0/3009281981 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367317981 0 0) 0x7f4fd015a160 con 0x7f4fd0159c60 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fcffff700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159890 0x7f4fd015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd4ea4700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0069560 0x7f4fd00699e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009e60 tx=0x7f4fc8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c01a3c62f5956f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fceffd700 1 -- 172.21.15.92:0/3009281981 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800e070 con 0x7f4fd0069560 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fceffd700 1 -- 172.21.15.92:0/3009281981 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc800bc30 con 0x7f4fd0069560 2023-10-06T03:08:09.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fceffd700 1 -- 172.21.15.92:0/3009281981 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8012430 con 0x7f4fd0069560 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 -- 172.21.15.92:0/3009281981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0069560 msgr2=0x7f4fd00699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0069560 0x7f4fd00699e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009e60 tx=0x7f4fc8009780 comp rx=0 tx=0).stop 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 -- 172.21.15.92:0/3009281981 shutdown_connections 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159890 0x7f4fd015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3009281981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0069560 0x7f4fd00699e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 -- 172.21.15.92:0/3009281981 >> 172.21.15.92:0/3009281981 conn(0x7f4fd0068e20 msgr2=0x7f4fd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 -- 172.21.15.92:0/3009281981 shutdown_connections 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.582+0000 7f4fd6907700 1 -- 172.21.15.92:0/3009281981 wait complete. 2023-10-06T03:08:09.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.583+0000 7f4fd6907700 1 Processor -- start 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 -- start start 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0069560 0x7f4fd015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 0x7f4fd015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159c60 0x7f4fd0155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd015bc40 con 0x7f4fd0159890 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd015d0e0 con 0x7f4fd0069560 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.584+0000 7f4fd6907700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd015c4d0 con 0x7f4fd0159c60 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 0x7f4fd015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 0x7f4fd015be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39738/0 (socket says 172.21.15.92:39738) 2023-10-06T03:08:09.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 -- 172.21.15.92:0/3258923081 learned_addr learned my addr 172.21.15.92:0/3258923081 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fd4ea4700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159c60 0x7f4fd0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcffff700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0069560 0x7f4fd015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 -- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159c60 msgr2=0x7f4fd0155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159c60 0x7f4fd0155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 -- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0069560 msgr2=0x7f4fd015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0069560 0x7f4fd015dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 -- 172.21.15.92:0/3258923081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8009a20 con 0x7f4fd0159890 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcffff700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0069560 0x7f4fd015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fcf7fe700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 0x7f4fd015be40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c840 tx=0x7f4fc4007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4005c00 con 0x7f4fd0159890 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4005d60 con 0x7f4fd0159890 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd015c7b0 con 0x7f4fd0159890 2023-10-06T03:08:09.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40072e0 con 0x7f4fd0159890 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.585+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd015cd70 con 0x7f4fd0159890 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.586+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd015e380 con 0x7f4fd0159890 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.586+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4fc401b3b0 con 0x7f4fd0159890 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.587+0000 7f4fb7fff700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fbc067320 0x7f4fbc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.587+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4fc40841e0 con 0x7f4fd0159890 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.587+0000 7f4fcffff700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fbc067320 0x7f4fbc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.587+0000 7f4fcffff700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fbc067320 0x7f4fbc0697d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006f00 tx=0x7f4fc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.591+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fc404b5a0 con 0x7f4fd0159890 2023-10-06T03:08:09.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.819+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fd004c3c0 con 0x7f4fd0159890 2023-10-06T03:08:09.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.819+0000 7f4fb7fff700 1 -- 172.21.15.92:0/3258923081 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f4fd015e380 con 0x7f4fd0159890 2023-10-06T03:08:09.820 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:09.820 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:49.932559+0000","modified":"2023-10-06T03:07:53.569016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/1523868194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1523868194},{"type":"v1","addr":"172.21.15.92:6837","nonce":1523868194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fbc067320 msgr2=0x7f4fbc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fbc067320 0x7f4fbc0697d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006f00 tx=0x7f4fc0008040 comp rx=0 tx=0).stop 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 msgr2=0x7f4fd015be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 0x7f4fd015be40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c840 tx=0x7f4fc4007db0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 shutdown_connections 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fbc067320 0x7f4fbc0697d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fd0159c60 0x7f4fd0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd0069560 0x7f4fd015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 --2- 172.21.15.92:0/3258923081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fd0159890 0x7f4fd015be40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 >> 172.21.15.92:0/3258923081 conn(0x7f4fd0068e20 msgr2=0x7f4fd0156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 shutdown_connections 2023-10-06T03:08:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.821+0000 7f4fd6907700 1 -- 172.21.15.92:0/3258923081 wait complete. 2023-10-06T03:08:09.823 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 69 2023-10-06T03:08:09.834 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:09.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.946+0000 7f543a033700 1 Processor -- start 2023-10-06T03:08:09.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.946+0000 7f543a033700 1 -- start start 2023-10-06T03:08:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f543a033700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5434159c60 0x7f543415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f543a033700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f543415db10 0x7f5434069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f543a033700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543415a160 con 0x7f5434159890 2023-10-06T03:08:09.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f543a033700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543415df50 con 0x7f5434159c60 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f543a033700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54340d13a0 con 0x7f543415db10 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5434159c60 0x7f543415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5434159c60 0x7f543415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39748/0 (socket says 172.21.15.92:39748) 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 -- 172.21.15.92:0/1420908164 learned_addr learned my addr 172.21.15.92:0/1420908164 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5433fff700 1 --2- 172.21.15.92:0/1420908164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f543415db10 0x7f5434069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 -- 172.21.15.92:0/1420908164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f543415db10 msgr2=0x7f5434069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 --2- 172.21.15.92:0/1420908164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f543415db10 0x7f5434069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 -- 172.21.15.92:0/1420908164 >> v1:172.21.15.136:6789/0 conn(0x7f5434159890 legacy=0x7f543415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.947+0000 7f5432ffd700 1 -- 172.21.15.92:0/1420908164 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543415e140 con 0x7f5434159c60 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f5432ffd700 1 --2- 172.21.15.92:0/1420908164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543415d5d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f54280099b0 tx=0x7f5428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e367c4d5d5c5a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f54327fc700 1 -- 172.21.15.92:0/1420908164 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800b860 con 0x7f5434159c60 2023-10-06T03:08:09.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f54327fc700 1 -- 172.21.15.92:0/1420908164 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542800b9c0 con 0x7f5434159c60 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f54327fc700 1 -- 172.21.15.92:0/1420908164 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800bd60 con 0x7f5434159c60 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 -- 172.21.15.92:0/1420908164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 msgr2=0x7f543415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 --2- 172.21.15.92:0/1420908164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543415d5d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f54280099b0 tx=0x7f5428009250 comp rx=0 tx=0).stop 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 -- 172.21.15.92:0/1420908164 shutdown_connections 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 --2- 172.21.15.92:0/1420908164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f543415db10 0x7f5434069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 --2- 172.21.15.92:0/1420908164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543415d5d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 -- 172.21.15.92:0/1420908164 >> 172.21.15.92:0/1420908164 conn(0x7f5434068e20 msgr2=0x7f54340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 -- 172.21.15.92:0/1420908164 shutdown_connections 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.948+0000 7f543a033700 1 -- 172.21.15.92:0/1420908164 wait complete. 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 Processor -- start 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 -- start start 2023-10-06T03:08:09.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5434159890 0x7f54340c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543414a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543415db10 0x7f54340ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340c78a0 con 0x7f5434159c60 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340c8140 con 0x7f543415db10 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f543a033700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543414a860 con 0x7f5434159890 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f5432ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543414a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f5432ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543414a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39758/0 (socket says 172.21.15.92:39758) 2023-10-06T03:08:09.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.950+0000 7f5432ffd700 1 -- 172.21.15.92:0/3869673520 learned_addr learned my addr 172.21.15.92:0/3869673520 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5433fff700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543415db10 0x7f54340ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f54337fe700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5434159890 0x7f54340c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5432ffd700 1 -- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5434159890 msgr2=0x7f54340c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5432ffd700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5434159890 0x7f54340c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5432ffd700 1 -- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543415db10 msgr2=0x7f54340ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5432ffd700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543415db10 0x7f54340ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5432ffd700 1 -- 172.21.15.92:0/3869673520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280094f0 con 0x7f5434159c60 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f5432ffd700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543414a320 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5428006fd0 tx=0x7f5428017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428011e00 con 0x7f5434159c60 2023-10-06T03:08:09.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428024440 con 0x7f5434159c60 2023-10-06T03:08:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428024800 con 0x7f5434159c60 2023-10-06T03:08:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54340cf010 con 0x7f5434159c60 2023-10-06T03:08:09.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.951+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340c7dd0 con 0x7f5434159c60 2023-10-06T03:08:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.952+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543404c3c0 con 0x7f5434159c60 2023-10-06T03:08:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.953+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f542801a420 con 0x7f5434159c60 2023-10-06T03:08:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.953+0000 7f541b7fe700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5420067320 0x7f54200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:09.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.953+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5428086b30 con 0x7f5434159c60 2023-10-06T03:08:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.953+0000 7f54337fe700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5420067320 0x7f54200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.953+0000 7f54337fe700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5420067320 0x7f54200697d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5424007fd0 tx=0x7f54240016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:09.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:09.957+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5428053660 con 0x7f5434159c60 2023-10-06T03:08:10.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.182+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54340c8280 con 0x7f5434159c60 2023-10-06T03:08:10.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.183+0000 7f541b7fe700 1 -- 172.21.15.92:0/3869673520 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f5428056c80 con 0x7f5434159c60 2023-10-06T03:08:10.183 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:10.184 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:07:49.932559+0000","modified":"2023-10-06T03:07:53.569016+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6840},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6840":{"gid":6840,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/1523868194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1523868194},{"type":"v1","addr":"172.21.15.92:6837","nonce":1523868194}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5420067320 msgr2=0x7f54200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5420067320 0x7f54200697d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5424007fd0 tx=0x7f54240016f0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 msgr2=0x7f543414a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543414a320 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f5428006fd0 tx=0x7f5428017000 comp rx=0 tx=0).stop 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 shutdown_connections 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5420067320 0x7f54200697d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f543415db10 0x7f54340ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5434159890 0x7f54340c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 --2- 172.21.15.92:0/3869673520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5434159c60 0x7f543414a320 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 >> 172.21.15.92:0/3869673520 conn(0x7f5434068e20 msgr2=0x7f5434158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 shutdown_connections 2023-10-06T03:08:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.185+0000 7f543a033700 1 -- 172.21.15.92:0/3869673520 wait complete. 2023-10-06T03:08:10.187 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 69 2023-10-06T03:08:10.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:10.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.312+0000 7fa1730c2700 1 Processor -- start 2023-10-06T03:08:10.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.312+0000 7fa1730c2700 1 -- start start 2023-10-06T03:08:10.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa1730c2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa16c159890 0x7fa16c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa1730c2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa1730c2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa16c15db10 0x7fa16c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa1730c2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c15a160 con 0x7fa16c15db10 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa1730c2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c0d13a0 con 0x7fa16c159890 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa1730c2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c15e140 con 0x7fa16c159c60 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa16c15db10 0x7fa16c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa16c15db10 0x7fa16c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39774/0 (socket says 172.21.15.92:39774) 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 -- 172.21.15.92:0/3143728940 learned_addr learned my addr 172.21.15.92:0/3143728940 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 -- 172.21.15.92:0/3143728940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 msgr2=0x7fa16c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:10.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 --2- 172.21.15.92:0/3143728940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa170e5e700 1 --2- 172.21.15.92:0/3143728940 >> v2:172.21.15.136:3300/0 conn(0x7fa16c159890 0x7fa16c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 -- 172.21.15.92:0/3143728940 >> v2:172.21.15.136:3300/0 conn(0x7fa16c159890 msgr2=0x7fa16c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 --2- 172.21.15.92:0/3143728940 >> v2:172.21.15.136:3300/0 conn(0x7fa16c159890 0x7fa16c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 -- 172.21.15.92:0/3143728940 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c15a3d0 con 0x7fa16c15db10 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa170e5e700 1 --2- 172.21.15.92:0/3143728940 >> v2:172.21.15.136:3300/0 conn(0x7fa16c159890 0x7fa16c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa17165f700 1 --2- 172.21.15.92:0/3143728940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c15db10 0x7fa16c069560 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0d1090 tx=0x7fa154009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e32586e7dce7c11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa16b7fe700 1 -- 172.21.15.92:0/3143728940 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400b860 con 0x7fa16c15db10 2023-10-06T03:08:10.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa16b7fe700 1 -- 172.21.15.92:0/3143728940 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15400b9c0 con 0x7fa16c15db10 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.313+0000 7fa16b7fe700 1 -- 172.21.15.92:0/3143728940 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400bd60 con 0x7fa16c15db10 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 -- 172.21.15.92:0/3143728940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c15db10 msgr2=0x7fa16c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 --2- 172.21.15.92:0/3143728940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c15db10 0x7fa16c069560 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0d1090 tx=0x7fa154009250 comp rx=0 tx=0).stop 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 -- 172.21.15.92:0/3143728940 shutdown_connections 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 --2- 172.21.15.92:0/3143728940 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c15db10 0x7fa16c069560 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 --2- 172.21.15.92:0/3143728940 >> v2:172.21.15.136:3300/0 conn(0x7fa16c159890 0x7fa16c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 --2- 172.21.15.92:0/3143728940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 -- 172.21.15.92:0/3143728940 >> 172.21.15.92:0/3143728940 conn(0x7fa16c068e20 msgr2=0x7fa16c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 -- 172.21.15.92:0/3143728940 shutdown_connections 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.314+0000 7fa1730c2700 1 -- 172.21.15.92:0/3143728940 wait complete. 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.315+0000 7fa1730c2700 1 Processor -- start 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.315+0000 7fa1730c2700 1 -- start start 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa1730c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 0x7fa16c15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa1730c2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa1730c2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16c15db10 0x7fa16c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa1730c2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0cd0c0 con 0x7fa16c159890 2023-10-06T03:08:10.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa1730c2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c169cf0 con 0x7fa16c15db10 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa1730c2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c169fd0 con 0x7fa16c159c60 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 0x7fa16c15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 0x7fa16c15b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39780/0 (socket says 172.21.15.92:39780) 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 -- 172.21.15.92:0/672834612 learned_addr learned my addr 172.21.15.92:0/672834612 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa17165f700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16c15db10 0x7fa16c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa16bfff700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 -- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 msgr2=0x7fa16c157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 -- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16c15db10 msgr2=0x7fa16c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16c15db10 0x7fa16c157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.316+0000 7fa170e5e700 1 -- 172.21.15.92:0/672834612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540094f0 con 0x7fa16c159890 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.317+0000 7fa170e5e700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 0x7fa16c15b9f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c830 tx=0x7fa164008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.317+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164011600 con 0x7fa16c159890 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.317+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164011760 con 0x7fa16c159890 2023-10-06T03:08:10.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.317+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c16a2b0 con 0x7fa16c159890 2023-10-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.317+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164011b20 con 0x7fa16c159890 2023-10-06T03:08:10.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.317+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c153290 con 0x7fa16c159890 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.318+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c0cd230 con 0x7fa16c159890 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.318+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa16400a3e0 con 0x7fa16c159890 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.318+0000 7fa168ff9700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa14c067320 0x7fa14c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.319+0000 7fa16bfff700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa14c067320 0x7fa14c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.319+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1640853a0 con 0x7fa16c159890 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.319+0000 7fa16bfff700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa14c067320 0x7fa14c0697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa16c16ccb0 tx=0x7fa15c009620 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:10.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.323+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa164051dc0 con 0x7fa16c159890 2023-10-06T03:08:10.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.506+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa16c04c3c0 con 0x7fa16c159890 2023-10-06T03:08:10.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.507+0000 7fa168ff9700 1 -- 172.21.15.92:0/672834612 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fa1640553e0 con 0x7fa16c159890 2023-10-06T03:08:10.508 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:10.509 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":103,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:07:51.555867+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:07:47.608728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:07:48.905445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:10.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa14c067320 msgr2=0x7fa14c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa14c067320 0x7fa14c0697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa16c16ccb0 tx=0x7fa15c009620 comp rx=0 tx=0).stop 2023-10-06T03:08:10.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 msgr2=0x7fa16c15b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 0x7fa16c15b9f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c830 tx=0x7fa164008040 comp rx=0 tx=0).stop 2023-10-06T03:08:10.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 shutdown_connections 2023-10-06T03:08:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa14c067320 0x7fa14c0697d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa16c159890 0x7fa16c15b9f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16c15db10 0x7fa16c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 --2- 172.21.15.92:0/672834612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa16c159c60 0x7fa16c157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 >> 172.21.15.92:0/672834612 conn(0x7fa16c068e20 msgr2=0x7fa16c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:10.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 shutdown_connections 2023-10-06T03:08:10.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.509+0000 7fa1730c2700 1 -- 172.21.15.92:0/672834612 wait complete. 2023-10-06T03:08:10.524 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:08:10.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.639+0000 7fd4e06c8700 1 Processor -- start 2023-10-06T03:08:10.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.640+0000 7fd4e06c8700 1 -- start start 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.640+0000 7fd4e06c8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4d8159890 0x7fd4d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.640+0000 7fd4e06c8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd4d8069560 0x7fd4d80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4e06c8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d8069ee0 con 0x7fd4d8159c60 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4e06c8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d815a160 con 0x7fd4d8159890 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4e06c8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d80d13a0 con 0x7fd4d8069560 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4d8159890 0x7fd4d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4ddc63700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd4d8159c60 0x7fd4d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49640/0 (socket says 172.21.15.92:49640) 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4ddc63700 1 -- 172.21.15.92:0/1884493169 learned_addr learned my addr 172.21.15.92:0/1884493169 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4d8159890 0x7fd4d815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39784/0 (socket says 172.21.15.92:39784) 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4dec65700 1 --2- 172.21.15.92:0/1884493169 >> v2:172.21.15.136:3300/0 conn(0x7fd4d8069560 0x7fd4d80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 -- 172.21.15.92:0/1884493169 >> v2:172.21.15.136:3300/0 conn(0x7fd4d8069560 msgr2=0x7fd4d80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 --2- 172.21.15.92:0/1884493169 >> v2:172.21.15.136:3300/0 conn(0x7fd4d8069560 0x7fd4d80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 -- 172.21.15.92:0/1884493169 >> v1:172.21.15.92:6789/0 conn(0x7fd4d8159c60 legacy=0x7fd4d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 -- 172.21.15.92:0/1884493169 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d815e140 con 0x7fd4d8159890 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4dd462700 1 -- 172.21.15.92:0/1884493169 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288879868 0 0) 0x7fd4d8069ee0 con 0x7fd4d8159c60 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4de464700 1 --2- 172.21.15.92:0/1884493169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d815abd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd4d806a890 tx=0x7fd4d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90db56d81d04529 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4dd462700 1 -- 172.21.15.92:0/1884493169 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400b6b0 con 0x7fd4d8159890 2023-10-06T03:08:10.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4dd462700 1 -- 172.21.15.92:0/1884493169 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d400b810 con 0x7fd4d8159890 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.641+0000 7fd4dd462700 1 -- 172.21.15.92:0/1884493169 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400bbb0 con 0x7fd4d8159890 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 -- 172.21.15.92:0/1884493169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 msgr2=0x7fd4d815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/1884493169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d815abd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd4d806a890 tx=0x7fd4d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 -- 172.21.15.92:0/1884493169 shutdown_connections 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/1884493169 >> v2:172.21.15.136:3300/0 conn(0x7fd4d8069560 0x7fd4d80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/1884493169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d815abd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 -- 172.21.15.92:0/1884493169 >> 172.21.15.92:0/1884493169 conn(0x7fd4d8068e20 msgr2=0x7fd4d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 -- 172.21.15.92:0/1884493169 shutdown_connections 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.642+0000 7fd4e06c8700 1 -- 172.21.15.92:0/1884493169 wait complete. 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.643+0000 7fd4e06c8700 1 Processor -- start 2023-10-06T03:08:10.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.643+0000 7fd4e06c8700 1 -- start start 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4d8069560 0x7fd4d816b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d80af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4d8159c60 0x7fd4d80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d80ce090 con 0x7fd4d8159890 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d80ce930 con 0x7fd4d8159c60 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d816b5b0 con 0x7fd4d8069560 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d80af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39792/0 (socket says 172.21.15.92:39792) 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 -- 172.21.15.92:0/2917480222 learned_addr learned my addr 172.21.15.92:0/2917480222 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:10.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 -- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4d8069560 msgr2=0x7fd4d816b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4d8069560 0x7fd4d816b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 -- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4d8159c60 msgr2=0x7fd4d80c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4dec65700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4d8159c60 0x7fd4d80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4d8159c60 0x7fd4d80c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 -- 172.21.15.92:0/2917480222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4009420 con 0x7fd4d8159890 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ddc63700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d80af830 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800c5c0 tx=0x7fd4c8000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8005b30 con 0x7fd4d8159890 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c8005c90 con 0x7fd4d8159890 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d80caa50 con 0x7fd4d8159890 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8007320 con 0x7fd4d8159890 2023-10-06T03:08:10.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.644+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d80ce5c0 con 0x7fd4d8159890 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.645+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d804c3c0 con 0x7fd4d8159890 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.646+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd4c801b3b0 con 0x7fd4d8159890 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.646+0000 7fd4ce7fc700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4bc067320 0x7fd4bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.646+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4c8084250 con 0x7fd4d8159890 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.646+0000 7fd4de464700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4bc067320 0x7fd4bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.647+0000 7fd4de464700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4bc067320 0x7fd4bc0697d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd4d816d400 tx=0x7fd4d4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:10.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.650+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4c8050d80 con 0x7fd4d8159890 2023-10-06T03:08:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.875+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd4d80cea70 con 0x7fd4d8159890 2023-10-06T03:08:10.875 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:08:10.875+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:08:10.878 INFO:tasks.ceph.mds.a.smithi092.stderr: -2191> 2023-10-06T03:07:38.531+0000 7f1904a98700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T03:08:10.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.879+0000 7fd4ce7fc700 1 -- 172.21.15.92:0/2917480222 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd4c80543a0 con 0x7fd4d8159890 2023-10-06T03:08:10.880 INFO:tasks.ceph.mds.a.smithi092.stderr: -1010> 2023-10-06T03:07:54.361+0000 7f18fea8c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:08:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4bc067320 msgr2=0x7fd4bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4bc067320 0x7fd4bc0697d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd4d816d400 tx=0x7fd4d4017000 comp rx=0 tx=0).stop 2023-10-06T03:08:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 msgr2=0x7fd4d80af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d80af830 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800c5c0 tx=0x7fd4c8000fe0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 shutdown_connections 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4bc067320 0x7fd4bc0697d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4d8159c60 0x7fd4d80c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4d8069560 0x7fd4d816b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 --2- 172.21.15.92:0/2917480222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4d8159890 0x7fd4d80af830 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 >> 172.21.15.92:0/2917480222 conn(0x7fd4d8068e20 msgr2=0x7fd4d80b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 shutdown_connections 2023-10-06T03:08:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:10.881+0000 7fd4e06c8700 1 -- 172.21.15.92:0/2917480222 wait complete. 2023-10-06T03:08:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:08:10.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:08:10.900 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:08:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.010+0000 7fa84db20700 1 Processor -- start 2023-10-06T03:08:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.010+0000 7fa84db20700 1 -- start start 2023-10-06T03:08:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa84db20700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa848159c60 0x7fa84815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa84db20700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa84815db10 0x7fa848069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa84db20700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84815a160 con 0x7fa848159890 2023-10-06T03:08:11.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa84db20700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84815df50 con 0x7fa848159c60 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa84db20700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480d13a0 con 0x7fa84815db10 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa848159c60 0x7fa84815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa848159c60 0x7fa84815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39812/0 (socket says 172.21.15.92:39812) 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 -- 172.21.15.92:0/97205397 learned_addr learned my addr 172.21.15.92:0/97205397 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa847fff700 1 --2- 172.21.15.92:0/97205397 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa84815db10 0x7fa848069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 -- 172.21.15.92:0/97205397 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa84815db10 msgr2=0x7fa848069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 --2- 172.21.15.92:0/97205397 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa84815db10 0x7fa848069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 -- 172.21.15.92:0/97205397 >> v1:172.21.15.136:6789/0 conn(0x7fa848159890 legacy=0x7fa84815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 -- 172.21.15.92:0/97205397 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84815e140 con 0x7fa848159c60 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.011+0000 7fa846ffd700 1 --2- 172.21.15.92:0/97205397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159c60 0x7fa84815d5d0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0099b0 tx=0x7fa83c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9f16247a536dc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa8467fc700 1 -- 172.21.15.92:0/97205397 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00b860 con 0x7fa848159c60 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa8467fc700 1 -- 172.21.15.92:0/97205397 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c00b9c0 con 0x7fa848159c60 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa8467fc700 1 -- 172.21.15.92:0/97205397 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c00bd60 con 0x7fa848159c60 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 -- 172.21.15.92:0/97205397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159c60 msgr2=0x7fa84815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 --2- 172.21.15.92:0/97205397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159c60 0x7fa84815d5d0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0099b0 tx=0x7fa83c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 -- 172.21.15.92:0/97205397 shutdown_connections 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 --2- 172.21.15.92:0/97205397 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa84815db10 0x7fa848069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 --2- 172.21.15.92:0/97205397 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159c60 0x7fa84815d5d0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 -- 172.21.15.92:0/97205397 >> 172.21.15.92:0/97205397 conn(0x7fa848068e20 msgr2=0x7fa8480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 -- 172.21.15.92:0/97205397 shutdown_connections 2023-10-06T03:08:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.012+0000 7fa84db20700 1 -- 172.21.15.92:0/97205397 wait complete. 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.013+0000 7fa84db20700 1 Processor -- start 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.013+0000 7fa84db20700 1 -- start start 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa84db20700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 0x7fa8480b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa84db20700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa848159c60 0x7fa8480cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa84db20700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84815db10 0x7fa8480cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa84db20700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848150730 con 0x7fa848159890 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa84db20700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480cfe90 con 0x7fa84815db10 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa84db20700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480b16c0 con 0x7fa848159c60 2023-10-06T03:08:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 0x7fa8480b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 0x7fa8480b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39824/0 (socket says 172.21.15.92:39824) 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 -- 172.21.15.92:0/1639019427 learned_addr learned my addr 172.21.15.92:0/1639019427 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa847fff700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84815db10 0x7fa8480cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa846ffd700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa848159c60 0x7fa8480cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 -- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa848159c60 msgr2=0x7fa8480cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa848159c60 0x7fa8480cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 -- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84815db10 msgr2=0x7fa8480cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.014+0000 7fa8477fe700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84815db10 0x7fa8480cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa8477fe700 1 -- 172.21.15.92:0/1639019427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0094f0 con 0x7fa848159890 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa8477fe700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 0x7fa8480b1090 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c590 tx=0x7fa83800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:11.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838011590 con 0x7fa848159890 2023-10-06T03:08:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa838011710 con 0x7fa848159890 2023-10-06T03:08:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8480b19a0 con 0x7fa848159890 2023-10-06T03:08:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838011ad0 con 0x7fa848159890 2023-10-06T03:08:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.015+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8480b1f60 con 0x7fa848159890 2023-10-06T03:08:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.016+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84804c3c0 con 0x7fa848159890 2023-10-06T03:08:11.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.016+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa83801b3b0 con 0x7fa848159890 2023-10-06T03:08:11.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.016+0000 7fa82f7fe700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa834067320 0x7fa8340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:11.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.017+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa8380842b0 con 0x7fa848159890 2023-10-06T03:08:11.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.017+0000 7fa846ffd700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa834067320 0x7fa8340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:11.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.017+0000 7fa846ffd700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa834067320 0x7fa8340697d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0099b0 tx=0x7fa83c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:11.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.021+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa838050d90 con 0x7fa848159890 2023-10-06T03:08:11.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.245+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa84806a5b0 con 0x7fa848159890 2023-10-06T03:08:11.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.886+0000 7fa82f7fe700 1 -- 172.21.15.92:0/1639019427 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8380543b0 con 0x7fa848159890 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa834067320 msgr2=0x7fa8340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa834067320 0x7fa8340697d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0099b0 tx=0x7fa83c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 msgr2=0x7fa8480b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 0x7fa8480b1090 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa83800c590 tx=0x7fa83800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 shutdown_connections 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa834067320 0x7fa8340697d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa848159890 0x7fa8480b1090 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84815db10 0x7fa8480cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 --2- 172.21.15.92:0/1639019427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa848159c60 0x7fa8480cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:11.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 >> 172.21.15.92:0/1639019427 conn(0x7fa848068e20 msgr2=0x7fa848153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:11.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 shutdown_connections 2023-10-06T03:08:11.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:11.889+0000 7fa84db20700 1 -- 172.21.15.92:0/1639019427 wait complete. 2023-10-06T03:08:11.903 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:08:12.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.064+0000 7f886ea47700 1 Processor -- start 2023-10-06T03:08:12.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.064+0000 7f886ea47700 1 -- start start 2023-10-06T03:08:12.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f886ea47700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8868164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f886ea47700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680cfa60 con 0x7f88680cd910 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f886ea47700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680b5b00 con 0x7f8868159ec0 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f886ea47700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88680b5da0 con 0x7f88680b55a0 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f88677fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f88680cd910 0x7f88680ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49672/0 (socket says 172.21.15.92:49672) 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f88677fe700 1 -- 172.21.15.92:0/3319766654 learned_addr learned my addr 172.21.15.92:0/3319766654 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.065+0000 7f886cfe4700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8868164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586047213 0 0) 0x7f88680cfa60 con 0x7f88680cd910 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8850003570 con 0x7f88680cd910 2023-10-06T03:08:12.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639300893 0 0) 0x7f88680b5b00 con 0x7f8868159ec0 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88680cfa60 con 0x7f8868159ec0 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1315801580 0 0) 0x7f8850003570 con 0x7f88680cd910 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 msgr2=0x7f8868164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8868164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 >> v1:172.21.15.136:6789/0 conn(0x7f8868159ec0 legacy=0x7f88680b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88680b6080 con 0x7f88680cd910 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f885c001470 con 0x7f88680cd910 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 >> v1:172.21.15.92:6789/0 conn(0x7f88680cd910 legacy=0x7f88680ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8850003c20 0x7f8850004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8850004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8850004d50 0x7f88500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680b5da0 con 0x7f8850004d50 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885000bba0 con 0x7f8850003c20 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885000bd60 con 0x7f88680b55a0 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8850004d50 0x7f88500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f88677fe700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8850004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8867fff700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8850003c20 0x7f8850004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 -- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 msgr2=0x7f8850004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8850004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 -- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8850003c20 msgr2=0x7f8850004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8850003c20 0x7f8850004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 -- 172.21.15.92:0/3319766654 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850003570 con 0x7f8850004d50 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f88677fe700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88680b55a0 0x7f8850004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8867fff700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8850003c20 0x7f8850004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f886cfe4700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8850004d50 0x7f88500051c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f886000e210 tx=0x7f886000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a336511ff93987db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.066+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000eac0 con 0x7f8850004d50 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000f040 con 0x7f8850004d50 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f8866ffd700 1 -- 172.21.15.92:0/3319766654 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000ee70 con 0x7f8850004d50 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886ea47700 1 -- 172.21.15.92:0/3319766654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8850004d50 msgr2=0x7f88500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886ea47700 1 --2- 172.21.15.92:0/3319766654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8850004d50 0x7f88500051c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f886000e210 tx=0x7f886000b040 comp rx=0 tx=0).stop 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886cfe4700 1 -- 172.21.15.92:0/3319766654 reap_dead start 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886ea47700 1 -- 172.21.15.92:0/3319766654 shutdown_connections 2023-10-06T03:08:12.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886ea47700 1 -- 172.21.15.92:0/3319766654 >> 172.21.15.92:0/3319766654 conn(0x7f88680999f0 msgr2=0x7f88680b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:12.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886ea47700 1 -- 172.21.15.92:0/3319766654 shutdown_connections 2023-10-06T03:08:12.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.067+0000 7f886ea47700 1 -- 172.21.15.92:0/3319766654 wait complete. 2023-10-06T03:08:12.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.068+0000 7f886ea47700 1 Processor -- start 2023-10-06T03:08:12.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.068+0000 7f886ea47700 1 -- start start 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f886ea47700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886809b6a0 0x7f8868169b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f886ea47700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 0x7f886809a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f886ea47700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f886809a9a0 0x7f88680c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f886ea47700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681570f0 con 0x7f886809d200 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f886ea47700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868156bd0 con 0x7f886809b6a0 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f886ea47700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886816aa60 con 0x7f886809a9a0 2023-10-06T03:08:12.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 0x7f886809a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 0x7f886809a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39838/0 (socket says 172.21.15.92:39838) 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 -- 172.21.15.92:0/1496338161 learned_addr learned my addr 172.21.15.92:0/1496338161 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f8867fff700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886809b6a0 0x7f8868169b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 -- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f886809a9a0 msgr2=0x7f88680c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f886809a9a0 0x7f88680c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 -- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886809b6a0 msgr2=0x7f8868169b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886809b6a0 0x7f8868169b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.069+0000 7f88677fe700 1 -- 172.21.15.92:0/1496338161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886809d670 con 0x7f886809d200 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.070+0000 7f88677fe700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 0x7f886809a460 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f885c00eb00 tx=0x7f885c001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.070+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00a760 con 0x7f886809d200 2023-10-06T03:08:12.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.070+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c010040 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.070+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886816a1c0 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.070+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c00aad0 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.070+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886816a780 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.071+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8868063c90 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.071+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f885c0143f0 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.071+0000 7f884ffff700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8854067350 0x7f8854069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.071+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f885c00ae40 con 0x7f886809d200 2023-10-06T03:08:12.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.071+0000 7f8867fff700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8854067350 0x7f8854069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:12.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.072+0000 7f8867fff700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8854067350 0x7f8854069800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8858000fb0 tx=0x7f8858009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:12.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.076+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f885c04c5a0 con 0x7f886809d200 2023-10-06T03:08:12.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.257+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f88680718b0 con 0x7f886809d200 2023-10-06T03:08:12.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:08:12.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:08:12.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.889+0000 7f884ffff700 1 -- 172.21.15.92:0/1496338161 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7f885c019020 con 0x7f886809d200 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8854067350 msgr2=0x7f8854069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8854067350 0x7f8854069800 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8858000fb0 tx=0x7f8858009040 comp rx=0 tx=0).stop 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 msgr2=0x7f886809a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 0x7f886809a460 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f885c00eb00 tx=0x7f885c001d50 comp rx=0 tx=0).stop 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 shutdown_connections 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8854067350 0x7f8854069800 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f886809a9a0 0x7f88680c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886809b6a0 0x7f8868169b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 --2- 172.21.15.92:0/1496338161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f886809d200 0x7f886809a460 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 >> 172.21.15.92:0/1496338161 conn(0x7f88680999f0 msgr2=0x7f8868158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 shutdown_connections 2023-10-06T03:08:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:12.893+0000 7f886ea47700 1 -- 172.21.15.92:0/1496338161 wait complete. 2023-10-06T03:08:12.895 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:08:12.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:08:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.025+0000 7fee4f2f0700 1 Processor -- start 2023-10-06T03:08:13.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.025+0000 7fee4f2f0700 1 -- start start 2023-10-06T03:08:13.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4f2f0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee480b03c0 0x7fee480cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:13.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4f2f0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee480b0790 0x7fee480b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:13.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4f2f0700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee480cfa00 con 0x7fee481591e0 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4f2f0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480cb580 con 0x7fee480b0790 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4f2f0700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480b2de0 con 0x7fee480b03c0 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee480b0790 0x7fee480b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee480b0790 0x7fee480b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39846/0 (socket says 172.21.15.92:39846) 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 -- 172.21.15.92:0/4101844191 learned_addr learned my addr 172.21.15.92:0/4101844191 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 -- 172.21.15.92:0/4101844191 >> v2:172.21.15.136:3300/0 conn(0x7fee480b03c0 msgr2=0x7fee480cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4e2ee700 1 --2- 172.21.15.92:0/4101844191 >> v2:172.21.15.136:3300/0 conn(0x7fee480b03c0 0x7fee480cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:13.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 --2- 172.21.15.92:0/4101844191 >> v2:172.21.15.136:3300/0 conn(0x7fee480b03c0 0x7fee480cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 -- 172.21.15.92:0/4101844191 >> v1:172.21.15.136:6789/0 conn(0x7fee481591e0 legacy=0x7fee48159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.026+0000 7fee4daed700 1 -- 172.21.15.92:0/4101844191 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480cf160 con 0x7fee480b0790 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4daed700 1 --2- 172.21.15.92:0/4101844191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee480b0790 0x7fee480b28a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fee380099b0 tx=0x7fee38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d8ae6b87e416ab server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4d2ec700 1 -- 172.21.15.92:0/4101844191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800b7e0 con 0x7fee480b0790 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4d2ec700 1 -- 172.21.15.92:0/4101844191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3800b940 con 0x7fee480b0790 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4d2ec700 1 -- 172.21.15.92:0/4101844191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800bd00 con 0x7fee480b0790 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 -- 172.21.15.92:0/4101844191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee480b0790 msgr2=0x7fee480b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/4101844191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee480b0790 0x7fee480b28a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fee380099b0 tx=0x7fee38009250 comp rx=0 tx=0).stop 2023-10-06T03:08:13.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 -- 172.21.15.92:0/4101844191 shutdown_connections 2023-10-06T03:08:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/4101844191 >> v2:172.21.15.136:3300/0 conn(0x7fee480b03c0 0x7fee480cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/4101844191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee480b0790 0x7fee480b28a0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 -- 172.21.15.92:0/4101844191 >> 172.21.15.92:0/4101844191 conn(0x7fee48063230 msgr2=0x7fee48063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 -- 172.21.15.92:0/4101844191 shutdown_connections 2023-10-06T03:08:13.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.027+0000 7fee4f2f0700 1 -- 172.21.15.92:0/4101844191 wait complete. 2023-10-06T03:08:13.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.028+0000 7fee4f2f0700 1 Processor -- start 2023-10-06T03:08:13.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 -- start start 2023-10-06T03:08:13.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee480b03c0 0x7fee4815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:13.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee480b0790 0x7fee480c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:13.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 0x7fee480c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:13.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c5280 con 0x7fee481591e0 2023-10-06T03:08:13.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c4670 con 0x7fee480b03c0 2023-10-06T03:08:13.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4f2f0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c4950 con 0x7fee480b0790 2023-10-06T03:08:13.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4eaef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 0x7fee480c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:13.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4eaef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 0x7fee480c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39852/0 (socket says 172.21.15.92:39852) 2023-10-06T03:08:13.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.029+0000 7fee4eaef700 1 -- 172.21.15.92:0/2052792301 learned_addr learned my addr 172.21.15.92:0/2052792301 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:13.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4e2ee700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee480b03c0 0x7fee4815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:13.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4daed700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee480b0790 0x7fee480c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:13.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4eaef700 1 -- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee480b0790 msgr2=0x7fee480c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:13.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4eaef700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee480b0790 0x7fee480c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4eaef700 1 -- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee480b03c0 msgr2=0x7fee4815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:13.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4eaef700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee480b03c0 0x7fee4815cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4eaef700 1 -- 172.21.15.92:0/2052792301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380094f0 con 0x7fee481591e0 2023-10-06T03:08:13.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4eaef700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 0x7fee480c5710 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b4e0 tx=0x7fee40009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:13.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4000f5e0 con 0x7fee481591e0 2023-10-06T03:08:13.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4000f760 con 0x7fee481591e0 2023-10-06T03:08:13.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee480c4c30 con 0x7fee481591e0 2023-10-06T03:08:13.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4000fb20 con 0x7fee481591e0 2023-10-06T03:08:13.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.030+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee480c5c50 con 0x7fee481591e0 2023-10-06T03:08:13.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.031+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee480c53f0 con 0x7fee481591e0 2023-10-06T03:08:13.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.031+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fee4001a400 con 0x7fee481591e0 2023-10-06T03:08:13.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.032+0000 7fee3e7fc700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee2c067320 0x7fee2c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:13.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.032+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fee400847e0 con 0x7fee481591e0 2023-10-06T03:08:13.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.032+0000 7fee4e2ee700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee2c067320 0x7fee2c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:13.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.032+0000 7fee4e2ee700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee2c067320 0x7fee2c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e4d0 tx=0x7fee44007680 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:13.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.036+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee400514f0 con 0x7fee481591e0 2023-10-06T03:08:13.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.217+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee4804c3c0 con 0x7fee481591e0 2023-10-06T03:08:13.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.894+0000 7fee3e7fc700 1 -- 172.21.15.92:0/2052792301 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7fee4000fed0 con 0x7fee481591e0 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee2c067320 msgr2=0x7fee2c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee2c067320 0x7fee2c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fee4400e4d0 tx=0x7fee44007680 comp rx=0 tx=0).stop 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 msgr2=0x7fee480c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 0x7fee480c5710 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fee4000b4e0 tx=0x7fee40009060 comp rx=0 tx=0).stop 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 shutdown_connections 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fee2c067320 0x7fee2c0697d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee481591e0 0x7fee480c5710 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee480b03c0 0x7fee4815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 --2- 172.21.15.92:0/2052792301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee480b0790 0x7fee480c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:13.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 >> 172.21.15.92:0/2052792301 conn(0x7fee48063230 msgr2=0x7fee48068b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:13.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 shutdown_connections 2023-10-06T03:08:13.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:13.898+0000 7fee4f2f0700 1 -- 172.21.15.92:0/2052792301 wait complete. 2023-10-06T03:08:13.900 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:08:13.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:08:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.029+0000 7fa868c44700 1 Processor -- start 2023-10-06T03:08:14.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.029+0000 7fa868c44700 1 -- start start 2023-10-06T03:08:14.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa868c44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 0x7fa8640b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa868c44700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8640b33b0 0x7fa8640b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa868c44700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8640b2080 con 0x7fa8640b4e70 2023-10-06T03:08:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa868c44700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8640cfb00 con 0x7fa8640b33b0 2023-10-06T03:08:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa868c44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640b5c40 con 0x7fa8640b16d0 2023-10-06T03:08:14.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa8637fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8640b4e70 0x7fa8640ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49674/0 (socket says 172.21.15.92:49674) 2023-10-06T03:08:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa8637fe700 1 -- 172.21.15.92:0/1043233550 learned_addr learned my addr 172.21.15.92:0/1043233550 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa862ffd700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 0x7fa8640b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa863fff700 1 --2- 172.21.15.92:0/1043233550 >> v2:172.21.15.136:3300/0 conn(0x7fa8640b33b0 0x7fa8640b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818949791 0 0) 0x7fa8640b2080 con 0x7fa8640b4e70 2023-10-06T03:08:14.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.030+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c003570 con 0x7fa8640b4e70 2023-10-06T03:08:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2430311592 0 0) 0x7fa84c003570 con 0x7fa8640b4e70 2023-10-06T03:08:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 msgr2=0x7fa8640b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 0x7fa8640b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 >> v2:172.21.15.136:3300/0 conn(0x7fa8640b33b0 msgr2=0x7fa8640b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 --2- 172.21.15.92:0/1043233550 >> v2:172.21.15.136:3300/0 conn(0x7fa8640b33b0 0x7fa8640b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8640b5f20 con 0x7fa8640b4e70 2023-10-06T03:08:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa8540017f0 con 0x7fa8640b4e70 2023-10-06T03:08:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 >> v1:172.21.15.92:6789/0 conn(0x7fa8640b4e70 legacy=0x7fa8640ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 0x7fa84c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c003fe0 0x7fa84c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84c004d50 0x7fa84c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640cfb00 con 0x7fa84c004d50 2023-10-06T03:08:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640b5c40 con 0x7fa84c003fe0 2023-10-06T03:08:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c00bb40 con 0x7fa8640b16d0 2023-10-06T03:08:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84c004d50 0x7fa84c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8637fe700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 0x7fa84c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa862ffd700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c003fe0 0x7fa84c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 -- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 msgr2=0x7fa84c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8640b16d0 0x7fa84c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 -- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c003fe0 msgr2=0x7fa84c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c003fe0 0x7fa84c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 -- 172.21.15.92:0/1043233550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c003570 con 0x7fa84c004d50 2023-10-06T03:08:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa863fff700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84c004d50 0x7fa84c0051c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa85c006f00 tx=0x7fa85c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf374cb73254b44 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c00e710 con 0x7fa84c004d50 2023-10-06T03:08:14.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.031+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c00e870 con 0x7fa84c004d50 2023-10-06T03:08:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa8627fc700 1 -- 172.21.15.92:0/1043233550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c00ec30 con 0x7fa84c004d50 2023-10-06T03:08:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa868c44700 1 -- 172.21.15.92:0/1043233550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84c004d50 msgr2=0x7fa84c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa868c44700 1 --2- 172.21.15.92:0/1043233550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa84c004d50 0x7fa84c0051c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa85c006f00 tx=0x7fa85c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa863fff700 1 -- 172.21.15.92:0/1043233550 reap_dead start 2023-10-06T03:08:14.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa868c44700 1 -- 172.21.15.92:0/1043233550 shutdown_connections 2023-10-06T03:08:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa868c44700 1 -- 172.21.15.92:0/1043233550 >> 172.21.15.92:0/1043233550 conn(0x7fa8640d10c0 msgr2=0x7fa8640affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa868c44700 1 -- 172.21.15.92:0/1043233550 shutdown_connections 2023-10-06T03:08:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.032+0000 7fa868c44700 1 -- 172.21.15.92:0/1043233550 wait complete. 2023-10-06T03:08:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.033+0000 7fa868c44700 1 Processor -- start 2023-10-06T03:08:14.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.033+0000 7fa868c44700 1 -- start start 2023-10-06T03:08:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa868c44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 0x7fa8640c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa868c44700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8640cbf20 0x7fa86416d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa868c44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa86416dbd0 0x7fa86416a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa868c44700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640c81f0 con 0x7fa8640cbb50 2023-10-06T03:08:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa868c44700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86416e020 con 0x7fa8640cbf20 2023-10-06T03:08:14.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa868c44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86416e490 con 0x7fa86416dbd0 2023-10-06T03:08:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 0x7fa8640c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 0x7fa8640c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39868/0 (socket says 172.21.15.92:39868) 2023-10-06T03:08:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 -- 172.21.15.92:0/1920691541 learned_addr learned my addr 172.21.15.92:0/1920691541 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa863fff700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa86416dbd0 0x7fa86416a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa862ffd700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8640cbf20 0x7fa86416d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 -- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa86416dbd0 msgr2=0x7fa86416a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa86416dbd0 0x7fa86416a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 -- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8640cbf20 msgr2=0x7fa86416d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8640cbf20 0x7fa86416d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.034+0000 7fa8637fe700 1 -- 172.21.15.92:0/1920691541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8640c75e0 con 0x7fa8640cbb50 2023-10-06T03:08:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.035+0000 7fa8637fe700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 0x7fa8640c8c80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa85400a450 tx=0x7fa854002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:14.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.035+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400ae80 con 0x7fa8640cbb50 2023-10-06T03:08:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.035+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854013600 con 0x7fa8640cbb50 2023-10-06T03:08:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.035+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8640c78c0 con 0x7fa8640cbb50 2023-10-06T03:08:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.035+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854013980 con 0x7fa8640cbb50 2023-10-06T03:08:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.035+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8640c7e80 con 0x7fa8640cbb50 2023-10-06T03:08:14.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.036+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa864088990 con 0x7fa8640cbb50 2023-10-06T03:08:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.036+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa85401c400 con 0x7fa8640cbb50 2023-10-06T03:08:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.036+0000 7fa84b7fe700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa850067320 0x7fa8500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.037+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa854085bf0 con 0x7fa8640cbb50 2023-10-06T03:08:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.037+0000 7fa862ffd700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa850067320 0x7fa8500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.037+0000 7fa862ffd700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa850067320 0x7fa8500697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa86415a830 tx=0x7fa858008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:14.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.041+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa854052a20 con 0x7fa8640cbb50 2023-10-06T03:08:14.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.231+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa8640cd900 con 0x7fa8640cbb50 2023-10-06T03:08:14.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.235+0000 7fa84b7fe700 1 -- 172.21.15.92:0/1920691541 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa854056040 con 0x7fa8640cbb50 2023-10-06T03:08:14.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.236+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa850067320 msgr2=0x7fa8500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.236+0000 7fa868c44700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa850067320 0x7fa8500697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa86415a830 tx=0x7fa858008040 comp rx=0 tx=0).stop 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.236+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 msgr2=0x7fa8640c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.236+0000 7fa868c44700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 0x7fa8640c8c80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa85400a450 tx=0x7fa854002210 comp rx=0 tx=0).stop 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 shutdown_connections 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa850067320 0x7fa8500697d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8640cbb50 0x7fa8640c8c80 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa86416dbd0 0x7fa86416a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 --2- 172.21.15.92:0/1920691541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8640cbf20 0x7fa86416d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 >> 172.21.15.92:0/1920691541 conn(0x7fa8640d10c0 msgr2=0x7fa864158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:14.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 shutdown_connections 2023-10-06T03:08:14.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.237+0000 7fa868c44700 1 -- 172.21.15.92:0/1920691541 wait complete. 2023-10-06T03:08:14.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-06T03:08:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.363+0000 7fd15d836700 1 Processor -- start 2023-10-06T03:08:14.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.363+0000 7fd15d836700 1 -- start start 2023-10-06T03:08:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d836700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1580ad870 0x7fd15806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d836700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd15806bf20 0x7fd15806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d836700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15806c350 con 0x7fd158142ea0 2023-10-06T03:08:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d836700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580b40f0 con 0x7fd15806bf20 2023-10-06T03:08:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d836700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158069640 con 0x7fd1580ad870 2023-10-06T03:08:14.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d035700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd15806bf20 0x7fd15806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d035700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd15806bf20 0x7fd15806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39874/0 (socket says 172.21.15.92:39874) 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd15d035700 1 -- 172.21.15.92:0/4069144313 learned_addr learned my addr 172.21.15.92:0/4069144313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.364+0000 7fd14ffff700 1 --2- 172.21.15.92:0/4069144313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1580ad870 0x7fd15806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d035700 1 -- 172.21.15.92:0/4069144313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1580ad870 msgr2=0x7fd15806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d035700 1 --2- 172.21.15.92:0/4069144313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1580ad870 0x7fd15806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d035700 1 -- 172.21.15.92:0/4069144313 >> v1:172.21.15.136:6789/0 conn(0x7fd158142ea0 legacy=0x7fd1580b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d035700 1 -- 172.21.15.92:0/4069144313 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158069920 con 0x7fd15806bf20 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d035700 1 --2- 172.21.15.92:0/4069144313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd15806bf20 0x7fd15806a120 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd1500099b0 tx=0x7fd150009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf866da5c6fb0958 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd14f7fe700 1 -- 172.21.15.92:0/4069144313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15000b860 con 0x7fd15806bf20 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd14f7fe700 1 -- 172.21.15.92:0/4069144313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15000b9c0 con 0x7fd15806bf20 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd14f7fe700 1 -- 172.21.15.92:0/4069144313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15000bd60 con 0x7fd15806bf20 2023-10-06T03:08:14.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 -- 172.21.15.92:0/4069144313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd15806bf20 msgr2=0x7fd15806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 --2- 172.21.15.92:0/4069144313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd15806bf20 0x7fd15806a120 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd1500099b0 tx=0x7fd150009250 comp rx=0 tx=0).stop 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 -- 172.21.15.92:0/4069144313 shutdown_connections 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 --2- 172.21.15.92:0/4069144313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd15806bf20 0x7fd15806a120 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 --2- 172.21.15.92:0/4069144313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1580ad870 0x7fd15806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 -- 172.21.15.92:0/4069144313 >> 172.21.15.92:0/4069144313 conn(0x7fd1580ce430 msgr2=0x7fd1580ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 -- 172.21.15.92:0/4069144313 shutdown_connections 2023-10-06T03:08:14.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.365+0000 7fd15d836700 1 -- 172.21.15.92:0/4069144313 wait complete. 2023-10-06T03:08:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.366+0000 7fd15d836700 1 Processor -- start 2023-10-06T03:08:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 -- start start 2023-10-06T03:08:14.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15806bf20 0x7fd15806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 0x7fd158158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd158158d70 0x7fd1580afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15816e490 con 0x7fd158142ea0 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158159210 con 0x7fd158158d70 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd15d836700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15816d880 con 0x7fd15806bf20 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd14ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 0x7fd158158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd14ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 0x7fd158158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39890/0 (socket says 172.21.15.92:39890) 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.367+0000 7fd14ffff700 1 -- 172.21.15.92:0/594725549 learned_addr learned my addr 172.21.15.92:0/594725549 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd15d035700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd158158d70 0x7fd1580afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd15c834700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15806bf20 0x7fd15806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14ffff700 1 -- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15806bf20 msgr2=0x7fd15806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14ffff700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15806bf20 0x7fd15806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14ffff700 1 -- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd158158d70 msgr2=0x7fd1580afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14ffff700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd158158d70 0x7fd1580afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14ffff700 1 -- 172.21.15.92:0/594725549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1500094f0 con 0x7fd158142ea0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd15c834700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15806bf20 0x7fd15806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14ffff700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 0x7fd158158830 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd14800d890 tx=0x7fd148008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1480125f0 con 0x7fd158142ea0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd148012750 con 0x7fd158142ea0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15816db60 con 0x7fd158142ea0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148012ad0 con 0x7fd158142ea0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.368+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15816e120 con 0x7fd158142ea0 2023-10-06T03:08:14.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.369+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15816e600 con 0x7fd158142ea0 2023-10-06T03:08:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.370+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd14801e460 con 0x7fd158142ea0 2023-10-06T03:08:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.370+0000 7fd14cff9700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd144067320 0x7fd1440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.370+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd1480850e0 con 0x7fd158142ea0 2023-10-06T03:08:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.370+0000 7fd15c834700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd144067320 0x7fd1440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:14.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.370+0000 7fd15c834700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd144067320 0x7fd1440697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd154007fd0 tx=0x7fd154009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:14.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.374+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd148051f10 con 0x7fd158142ea0 2023-10-06T03:08:14.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.636+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7fd15804c3c0 con 0x7fd158142ea0 2023-10-06T03:08:14.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.897+0000 7fd14cff9700 1 -- 172.21.15.92:0/594725549 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v322) v1 ==== 156+0+0 (secure 0 0 0) 0x7fd148055530 con 0x7fd158142ea0 2023-10-06T03:08:14.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd144067320 msgr2=0x7fd1440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd144067320 0x7fd1440697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd154007fd0 tx=0x7fd154009040 comp rx=0 tx=0).stop 2023-10-06T03:08:14.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 msgr2=0x7fd158158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:14.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 0x7fd158158830 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd14800d890 tx=0x7fd148008040 comp rx=0 tx=0).stop 2023-10-06T03:08:14.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 shutdown_connections 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd144067320 0x7fd1440697d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd158158d70 0x7fd1580afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15806bf20 0x7fd15806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 --2- 172.21.15.92:0/594725549 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd158142ea0 0x7fd158158830 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 >> 172.21.15.92:0/594725549 conn(0x7fd1580ce430 msgr2=0x7fd158141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 shutdown_connections 2023-10-06T03:08:14.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:14.899+0000 7fd15d836700 1 -- 172.21.15.92:0/594725549 wait complete. 2023-10-06T03:08:14.912 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command without subvolume group ... ok 2023-10-06T03:08:14.912 INFO:tasks.cephfs_test_runner:Starting test: test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) That the existing and the newly created volumes can be listed and 2023-10-06T03:08:14.913 INFO:tasks.cephfs_test_runner:test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:08:14.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-06T03:08:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.027+0000 7fa2e2beb700 1 Processor -- start 2023-10-06T03:08:15.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.027+0000 7fa2e2beb700 1 -- start start 2023-10-06T03:08:15.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e2beb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2dc06c0f0 0x7fa2dc159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:15.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e2beb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:15.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e2beb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc143420 con 0x7fa2dc143050 2023-10-06T03:08:15.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e2beb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc06a1e0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e2beb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0695d0 con 0x7fa2dc0b3810 2023-10-06T03:08:15.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e13e8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2dc06c0f0 0x7fa2dc159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:15.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e1be9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa2dc143050 0x7fa2dc06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49690/0 (socket says 172.21.15.92:49690) 2023-10-06T03:08:15.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e1be9700 1 -- 172.21.15.92:0/44821308 learned_addr learned my addr 172.21.15.92:0/44821308 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:15.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e23ea700 1 --2- 172.21.15.92:0/44821308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e0be7700 1 -- 172.21.15.92:0/44821308 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077496330 0 0) 0x7fa2dc143420 con 0x7fa2dc143050 2023-10-06T03:08:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e0be7700 1 -- 172.21.15.92:0/44821308 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c8003570 con 0x7fa2dc143050 2023-10-06T03:08:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e13e8700 1 -- 172.21.15.92:0/44821308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 msgr2=0x7fa2dc0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e13e8700 1 --2- 172.21.15.92:0/44821308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.028+0000 7fa2e13e8700 1 -- 172.21.15.92:0/44821308 >> v1:172.21.15.92:6789/0 conn(0x7fa2dc143050 legacy=0x7fa2dc06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:15.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e13e8700 1 -- 172.21.15.92:0/44821308 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0698b0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e13e8700 1 --2- 172.21.15.92:0/44821308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc159c90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0098e0 tx=0x7fa2cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b7b97955068ba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:15.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e0be7700 1 -- 172.21.15.92:0/44821308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00b640 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e0be7700 1 -- 172.21.15.92:0/44821308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00b7a0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e0be7700 1 -- 172.21.15.92:0/44821308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00bb40 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 -- 172.21.15.92:0/44821308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 msgr2=0x7fa2dc159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/44821308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc159c90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0098e0 tx=0x7fa2cc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 -- 172.21.15.92:0/44821308 shutdown_connections 2023-10-06T03:08:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/44821308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/44821308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc159c90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 -- 172.21.15.92:0/44821308 >> 172.21.15.92:0/44821308 conn(0x7fa2dc0ce430 msgr2=0x7fa2dc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:15.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.029+0000 7fa2e2beb700 1 -- 172.21.15.92:0/44821308 shutdown_connections 2023-10-06T03:08:15.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.030+0000 7fa2e2beb700 1 -- 172.21.15.92:0/44821308 wait complete. 2023-10-06T03:08:15.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 Processor -- start 2023-10-06T03:08:15.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 -- start start 2023-10-06T03:08:15.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc13ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:15.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:15.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2dc143050 0x7fa2dc158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc15b4b0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc159400 con 0x7fa2dc143050 2023-10-06T03:08:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e2beb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc13f360 con 0x7fa2dc0b3810 2023-10-06T03:08:15.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e1be9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc13ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e1be9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc13ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39908/0 (socket says 172.21.15.92:39908) 2023-10-06T03:08:15.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e1be9700 1 -- 172.21.15.92:0/202972020 learned_addr learned my addr 172.21.15.92:0/202972020 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e13e8700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.031+0000 7fa2e23ea700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2dc143050 0x7fa2dc158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:15.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e1be9700 1 -- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 msgr2=0x7fa2dc15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:15.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e1be9700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e1be9700 1 -- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2dc143050 msgr2=0x7fa2dc158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:15.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e1be9700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2dc143050 0x7fa2dc158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e1be9700 1 -- 172.21.15.92:0/202972020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc009420 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e13e8700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e1be9700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc13ec50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c900 tx=0x7fa2d8002740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:15.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800cea0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d8005d80 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc13f640 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d800a7e0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.032+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc13fc00 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.033+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc04c3c0 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.034+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa2d800ac10 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.034+0000 7fa2d1ffb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2c0067320 0x7fa2c00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.034+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa2d8084860 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.034+0000 7fa2e13e8700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2c0067320 0x7fa2c00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:15.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.034+0000 7fa2e13e8700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2c0067320 0x7fa2c00697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0093f0 tx=0x7fa2cc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:15.053 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.038+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2d8051710 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.297+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7fa2dc069b90 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.903+0000 7fa2d1ffb700 1 -- 172.21.15.92:0/202972020 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v323) v1 ==== 134+0+0 (secure 0 0 0) 0x7fa2d8054d30 con 0x7fa2dc06c0f0 2023-10-06T03:08:15.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2c0067320 msgr2=0x7fa2c00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2c0067320 0x7fa2c00697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0093f0 tx=0x7fa2cc017000 comp rx=0 tx=0).stop 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 msgr2=0x7fa2dc13ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc13ec50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa2d800c900 tx=0x7fa2d8002740 comp rx=0 tx=0).stop 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 shutdown_connections 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2c0067320 0x7fa2c00697d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2dc06c0f0 0x7fa2dc13ec50 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2dc143050 0x7fa2dc158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 --2- 172.21.15.92:0/202972020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2dc0b3810 0x7fa2dc15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 >> 172.21.15.92:0/202972020 conn(0x7fa2dc0ce430 msgr2=0x7fa2dc140420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 shutdown_connections 2023-10-06T03:08:15.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:15.906+0000 7fa2e2beb700 1 -- 172.21.15.92:0/202972020 wait complete. 2023-10-06T03:08:15.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:08:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.037+0000 7f131d8e8700 1 Processor -- start 2023-10-06T03:08:16.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.037+0000 7f131d8e8700 1 -- start start 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f131d8e8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1318159890 0x7f131815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f131d8e8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159c60 0x7f131815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f131d8e8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f131815a160 con 0x7f131815db10 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f131d8e8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180d13a0 con 0x7f1318159890 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f131d8e8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131815e140 con 0x7f1318159c60 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f1316ffd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1318159890 0x7f131815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f13177fe700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f131815db10 0x7f1318069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:54250/0 (socket says 172.21.15.92:54250) 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f130e7fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159c60 0x7f131815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.038+0000 7f13177fe700 1 -- 172.21.15.92:0/2147735170 learned_addr learned my addr 172.21.15.92:0/2147735170 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f130e7fc700 1 -- 172.21.15.92:0/2147735170 >> v2:172.21.15.136:3300/0 conn(0x7f1318159890 msgr2=0x7f131815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f130e7fc700 1 --2- 172.21.15.92:0/2147735170 >> v2:172.21.15.136:3300/0 conn(0x7f1318159890 0x7f131815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f130e7fc700 1 -- 172.21.15.92:0/2147735170 >> v1:172.21.15.136:6789/0 conn(0x7f131815db10 legacy=0x7f1318069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f130e7fc700 1 -- 172.21.15.92:0/2147735170 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131815a3d0 con 0x7f1318159c60 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f1316ffd700 1 --2- 172.21.15.92:0/2147735170 >> v2:172.21.15.136:3300/0 conn(0x7f1318159890 0x7f131815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f130e7fc700 1 --2- 172.21.15.92:0/2147735170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159c60 0x7f131815d5d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13000098e0 tx=0x7f13000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e137cdcda8755f0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f13167fc700 1 -- 172.21.15.92:0/2147735170 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130000b640 con 0x7f1318159c60 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f13167fc700 1 -- 172.21.15.92:0/2147735170 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130000b7a0 con 0x7f1318159c60 2023-10-06T03:08:16.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.039+0000 7f13167fc700 1 -- 172.21.15.92:0/2147735170 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130000bb40 con 0x7f1318159c60 2023-10-06T03:08:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 -- 172.21.15.92:0/2147735170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159c60 msgr2=0x7f131815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 --2- 172.21.15.92:0/2147735170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159c60 0x7f131815d5d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13000098e0 tx=0x7f13000091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 -- 172.21.15.92:0/2147735170 shutdown_connections 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 --2- 172.21.15.92:0/2147735170 >> v2:172.21.15.136:3300/0 conn(0x7f1318159890 0x7f131815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 --2- 172.21.15.92:0/2147735170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159c60 0x7f131815d5d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 -- 172.21.15.92:0/2147735170 >> 172.21.15.92:0/2147735170 conn(0x7f1318068e20 msgr2=0x7f13180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 -- 172.21.15.92:0/2147735170 shutdown_connections 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.040+0000 7f131d8e8700 1 -- 172.21.15.92:0/2147735170 wait complete. 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.041+0000 7f131d8e8700 1 Processor -- start 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.041+0000 7f131d8e8700 1 -- start start 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f131d8e8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159890 0x7f1318149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f131d8e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 0x7f131815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f131d8e8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f131815db10 0x7f1318151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f131d8e8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180cde70 con 0x7f1318159c60 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f131d8e8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131815d5f0 con 0x7f131815db10 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f131d8e8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13180cd260 con 0x7f1318159890 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 0x7f131815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 0x7f131815c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39924/0 (socket says 172.21.15.92:39924) 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 -- 172.21.15.92:0/1826732272 learned_addr learned my addr 172.21.15.92:0/1826732272 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f13177fe700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f131815db10 0x7f1318151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f1316ffd700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159890 0x7f1318149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 -- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159890 msgr2=0x7f1318149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159890 0x7f1318149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 -- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f131815db10 msgr2=0x7f1318151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f131815db10 0x7f1318151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 -- 172.21.15.92:0/1826732272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1308009420 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130e7fc700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 0x7f131815c6b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1300013000 tx=0x7f1300017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130001bc40 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.042+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130001bdc0 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.043+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1300009420 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.043+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300011670 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.043+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13180cd820 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.043+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131804c3c0 con 0x7f1318159c60 2023-10-06T03:08:16.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.044+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1300011a20 con 0x7f1318159c60 2023-10-06T03:08:16.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.044+0000 7f130f7fe700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12f8067320 0x7f12f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.044+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1300086020 con 0x7f1318159c60 2023-10-06T03:08:16.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.044+0000 7f1316ffd700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12f8067320 0x7f12f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.045+0000 7f1316ffd700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12f8067320 0x7f12f80697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1308009f40 tx=0x7f13080091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.048+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1300052e50 con 0x7f1318159c60 2023-10-06T03:08:16.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.230+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f13180cdfb0 con 0x7f1318159c60 2023-10-06T03:08:16.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.236+0000 7f130f7fe700 1 -- 172.21.15.92:0/1826732272 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1300056470 con 0x7f1318159c60 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12f8067320 msgr2=0x7f12f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12f8067320 0x7f12f80697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1308009f40 tx=0x7f13080091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 msgr2=0x7f131815c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 0x7f131815c6b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1300013000 tx=0x7f1300017000 comp rx=0 tx=0).stop 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 shutdown_connections 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12f8067320 0x7f12f80697d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f131815db10 0x7f1318151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1318159890 0x7f1318149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 --2- 172.21.15.92:0/1826732272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1318159c60 0x7f131815c6b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 >> 172.21.15.92:0/1826732272 conn(0x7f1318068e20 msgr2=0x7f13180d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:16.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 shutdown_connections 2023-10-06T03:08:16.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.238+0000 7f131d8e8700 1 -- 172.21.15.92:0/1826732272 wait complete. 2023-10-06T03:08:16.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:08:16.251 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:16.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:16.335 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:08:16.335 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:08:16.335 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:16.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.449+0000 7f3f0f44e700 1 Processor -- start 2023-10-06T03:08:16.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.449+0000 7f3f0f44e700 1 -- start start 2023-10-06T03:08:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0f44e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3f08159890 0x7f3f0815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0f44e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0815a160 con 0x7f3f08159c60 2023-10-06T03:08:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0f44e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f080d13a0 con 0x7f3f08069560 2023-10-06T03:08:16.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0f44e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0815e140 con 0x7f3f08159890 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0c9e9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3f08159c60 0x7f3f0815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49698/0 (socket says 172.21.15.92:49698) 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0c9e9700 1 -- 172.21.15.92:0/3670003751 learned_addr learned my addr 172.21.15.92:0/3670003751 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3f0d1ea700 1 --2- 172.21.15.92:0/3670003751 >> v2:172.21.15.136:3300/0 conn(0x7f3f08159890 0x7f3f0815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135175229 0 0) 0x7f3f0815a160 con 0x7f3f08159c60 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef4003570 con 0x7f3f08159c60 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419677860 0 0) 0x7f3f080d13a0 con 0x7f3f08069560 2023-10-06T03:08:16.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f0815a160 con 0x7f3f08069560 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574314394 0 0) 0x7f3ef4003570 con 0x7f3f08159c60 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 >> v2:172.21.15.136:3300/0 conn(0x7f3f08159890 msgr2=0x7f3f0815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 --2- 172.21.15.92:0/3670003751 >> v2:172.21.15.136:3300/0 conn(0x7f3f08159890 0x7f3f0815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 >> v1:172.21.15.136:6789/0 conn(0x7f3f08069560 legacy=0x7f3f080ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0815a3d0 con 0x7f3f08159c60 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f3ef8001470 con 0x7f3f08159c60 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 >> v1:172.21.15.92:6789/0 conn(0x7f3f08159c60 legacy=0x7f3f0815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef4003bf0 0x7f3ef4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.450+0000 7f3efffff700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef4004810 0x7f3ef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3ef400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0815e140 con 0x7f3ef40051c0 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400c0b0 con 0x7f3ef4003bf0 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef400c270 con 0x7f3ef4004810 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3ef400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d1ea700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef4003bf0 0x7f3ef4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0c9e9700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef4004810 0x7f3ef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 -- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef4004810 msgr2=0x7f3ef4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef4004810 0x7f3ef4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 -- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef4003bf0 msgr2=0x7f3ef4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef4003bf0 0x7f3ef4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 -- 172.21.15.92:0/3670003751 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4003570 con 0x7f3ef40051c0 2023-10-06T03:08:16.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d1ea700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef4003bf0 0x7f3ef4003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 -- 172.21.15.92:0/3670003751 reap_dead start 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3f0d9eb700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3ef400ba60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3f00002120 tx=0x7f3f000017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3c67f6c8aad695 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000c980 con 0x7f3ef40051c0 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0000cae0 con 0x7f3ef40051c0 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.451+0000 7f3efffff700 1 -- 172.21.15.92:0/3670003751 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0000ce80 con 0x7f3ef40051c0 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 -- 172.21.15.92:0/3670003751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 msgr2=0x7f3ef400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3ef400ba60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3f00002120 tx=0x7f3f000017a0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 -- 172.21.15.92:0/3670003751 shutdown_connections 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/3670003751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3ef400ba60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 -- 172.21.15.92:0/3670003751 >> 172.21.15.92:0/3670003751 conn(0x7f3f08068e20 msgr2=0x7f3f080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:16.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 -- 172.21.15.92:0/3670003751 shutdown_connections 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.452+0000 7f3f0f44e700 1 -- 172.21.15.92:0/3670003751 wait complete. 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.453+0000 7f3f0f44e700 1 Processor -- start 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.453+0000 7f3f0f44e700 1 -- start start 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0f44e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3f0813f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0f44e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f08151f80 0x7f3f081523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0f44e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f080ce880 0x7f3f080cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0f44e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08152930 con 0x7f3ef40051c0 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0f44e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080cf5f0 con 0x7f3f080ce880 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0f44e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f080cf8d0 con 0x7f3f08151f80 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3f0813f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3f0813f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39928/0 (socket says 172.21.15.92:39928) 2023-10-06T03:08:16.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 -- 172.21.15.92:0/901249409 learned_addr learned my addr 172.21.15.92:0/901249409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d9eb700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f080ce880 0x7f3f080cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0c9e9700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f08151f80 0x7f3f081523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 -- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f08151f80 msgr2=0x7f3f081523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f08151f80 0x7f3f081523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 -- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f080ce880 msgr2=0x7f3f080cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f080ce880 0x7f3f080cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 -- 172.21.15.92:0/901249409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f080cfbb0 con 0x7f3ef40051c0 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3f0d1ea700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3f0813f060 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3f04011d70 tx=0x7f3f040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.454+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0400fcd0 con 0x7f3ef40051c0 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.455+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0400fe30 con 0x7f3ef40051c0 2023-10-06T03:08:16.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.455+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f080cfe90 con 0x7f3ef40051c0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.455+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04014670 con 0x7f3ef40051c0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.455+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f080cf230 con 0x7f3ef40051c0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.456+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f080b3870 con 0x7f3ef40051c0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.456+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3f04014a20 con 0x7f3ef40051c0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.456+0000 7f3efd7fa700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3eec067350 0x7f3eec069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.456+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3f04018070 con 0x7f3ef40051c0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.456+0000 7f3f0c9e9700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3eec067350 0x7f3eec069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.457+0000 7f3f0c9e9700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3eec067350 0x7f3eec069800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8001c80 tx=0x7f3ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.460+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f080b3870 con 0x7f3ef40051c0 2023-10-06T03:08:16.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.686+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f080b3870 con 0x7f3ef40051c0 2023-10-06T03:08:16.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.686+0000 7f3efd7fa700 1 -- 172.21.15.92:0/901249409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f3f080b3870 con 0x7f3ef40051c0 2023-10-06T03:08:16.689 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:16.689 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/291658122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":291658122},{"type":"v1","addr":"172.21.15.136:6834","nonce":291658122}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63},{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2870841314","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2870841314},{"type":"v1","addr":"172.21.15.136:6835","nonce":2870841314}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":62},{"gid":6837,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/1979697353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1979697353},{"type":"v1","addr":"172.21.15.92:6836","nonce":1979697353}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":7158,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/279167919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":279167919},{"type":"v1","addr":"172.21.15.92:6837","nonce":279167919}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[]} 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3eec067350 msgr2=0x7f3eec069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3eec067350 0x7f3eec069800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8001c80 tx=0x7f3ef8009040 comp rx=0 tx=0).stop 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 msgr2=0x7f3f0813f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3f0813f060 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3f04011d70 tx=0x7f3f040091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 shutdown_connections 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3eec067350 0x7f3eec069800 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ef40051c0 0x7f3f0813f060 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f080ce880 0x7f3f080cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 --2- 172.21.15.92:0/901249409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f08151f80 0x7f3f081523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 >> 172.21.15.92:0/901249409 conn(0x7f3f08068e20 msgr2=0x7f3f0815d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:16.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 shutdown_connections 2023-10-06T03:08:16.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.689+0000 7f3f0f44e700 1 -- 172.21.15.92:0/901249409 wait complete. 2023-10-06T03:08:16.692 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 71 2023-10-06T03:08:16.703 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:08:16.703 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:08:16.703 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:08:16.703 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:16.703 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:08:16.704 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:08:16.704 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:08:16.704 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:16.704 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:08:16.704 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:08:16.704 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:08:16.704 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:16.705 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:08:16.705 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:08:16.705 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:08:16.705 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:16.712 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:08:16.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:08:16.713 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:08:16.713 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:08:16.715 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:08:16.715 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:08:16.715 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:08:16.716 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:08:16.755 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:08:16.755 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:16.755 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:08:16.756 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:16.758 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:08:16.758 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:16.758 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:08:16.759 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:16.759 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:08:16.828 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:08:16.829 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:08:16.837 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:08:16.837 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:08:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.878+0000 7f4e12d85700 1 Processor -- start 2023-10-06T03:08:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.878+0000 7f4e12d85700 1 -- start start 2023-10-06T03:08:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e12d85700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e0c159c60 0x7f4e0c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e12d85700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4e0c15db10 0x7f4e0c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e12d85700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c15a160 con 0x7f4e0c159890 2023-10-06T03:08:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e12d85700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c15df50 con 0x7f4e0c159c60 2023-10-06T03:08:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e12d85700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c0d13a0 con 0x7f4e0c15db10 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e0c159c60 0x7f4e0c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4e0c159c60 0x7f4e0c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39958/0 (socket says 172.21.15.92:39958) 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 -- 172.21.15.92:0/1058459789 learned_addr learned my addr 172.21.15.92:0/1058459789 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e11322700 1 --2- 172.21.15.92:0/1058459789 >> v2:172.21.15.136:3300/0 conn(0x7f4e0c15db10 0x7f4e0c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 -- 172.21.15.92:0/1058459789 >> v2:172.21.15.136:3300/0 conn(0x7f4e0c15db10 msgr2=0x7f4e0c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 --2- 172.21.15.92:0/1058459789 >> v2:172.21.15.136:3300/0 conn(0x7f4e0c15db10 0x7f4e0c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 -- 172.21.15.92:0/1058459789 >> v1:172.21.15.136:6789/0 conn(0x7f4e0c159890 legacy=0x7f4e0c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 -- 172.21.15.92:0/1058459789 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c15e140 con 0x7f4e0c159c60 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0bfff700 1 --2- 172.21.15.92:0/1058459789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c15d5d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c06a890 tx=0x7f4df8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b8b9422435093b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.879+0000 7f4e0b7fe700 1 -- 172.21.15.92:0/1058459789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800b820 con 0x7f4e0c159c60 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e0b7fe700 1 -- 172.21.15.92:0/1058459789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df800b980 con 0x7f4e0c159c60 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e0b7fe700 1 -- 172.21.15.92:0/1058459789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800bd20 con 0x7f4e0c159c60 2023-10-06T03:08:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 -- 172.21.15.92:0/1058459789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 msgr2=0x7f4e0c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 --2- 172.21.15.92:0/1058459789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c15d5d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c06a890 tx=0x7f4df8009250 comp rx=0 tx=0).stop 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 -- 172.21.15.92:0/1058459789 shutdown_connections 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 --2- 172.21.15.92:0/1058459789 >> v2:172.21.15.136:3300/0 conn(0x7f4e0c15db10 0x7f4e0c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 --2- 172.21.15.92:0/1058459789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c15d5d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 -- 172.21.15.92:0/1058459789 >> 172.21.15.92:0/1058459789 conn(0x7f4e0c068e20 msgr2=0x7f4e0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 -- 172.21.15.92:0/1058459789 shutdown_connections 2023-10-06T03:08:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.880+0000 7f4e12d85700 1 -- 172.21.15.92:0/1058459789 wait complete. 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.881+0000 7f4e12d85700 1 Processor -- start 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.881+0000 7f4e12d85700 1 -- start start 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e12d85700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0c159890 0x7f4e0c16b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e12d85700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e12d85700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0c15db10 0x7f4e0c16d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e12d85700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0b6480 con 0x7f4e0c159c60 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e12d85700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c16b5d0 con 0x7f4e0c15db10 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e12d85700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0b61e0 con 0x7f4e0c159890 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c0b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39970/0 (socket says 172.21.15.92:39970) 2023-10-06T03:08:16.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 -- 172.21.15.92:0/782113457 learned_addr learned my addr 172.21.15.92:0/782113457 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e10b21700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0c159890 0x7f4e0c16b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e11322700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0c15db10 0x7f4e0c16d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 -- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0c159890 msgr2=0x7f4e0c16b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0c159890 0x7f4e0c16b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 -- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0c15db10 msgr2=0x7f4e0c16d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0c15db10 0x7f4e0c16d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.882+0000 7f4e0bfff700 1 -- 172.21.15.92:0/782113457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80094f0 con 0x7f4e0c159c60 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.883+0000 7f4e0bfff700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c0b5ca0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c070e20 tx=0x7f4df8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.883+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8011c80 con 0x7f4e0c159c60 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.883+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df8020070 con 0x7f4e0c159c60 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.883+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c0b0670 con 0x7f4e0c159c60 2023-10-06T03:08:16.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.883+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df801b450 con 0x7f4e0c159c60 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.883+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c0b0c30 con 0x7f4e0c159c60 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.884+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c04c3c0 con 0x7f4e0c159c60 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.884+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4df8022030 con 0x7f4e0c159c60 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.884+0000 7f4e08ff9700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4df0067320 0x7f4df00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.885+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4df8086790 con 0x7f4e0c159c60 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.885+0000 7f4e10b21700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4df0067320 0x7f4df00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:16.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.885+0000 7f4e10b21700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4df0067320 0x7f4df00697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4e00006fd0 tx=0x7f4e000090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:16.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:16.889+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4df80536d0 con 0x7f4e0c159c60 2023-10-06T03:08:17.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.093+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4e0c04c3c0 con 0x7f4e0c159c60 2023-10-06T03:08:17.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.926+0000 7f4e08ff9700 1 -- 172.21.15.92:0/782113457 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4df8056cf0 con 0x7f4e0c159c60 2023-10-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4df0067320 msgr2=0x7f4df00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4df0067320 0x7f4df00697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4e00006fd0 tx=0x7f4e000090f0 comp rx=0 tx=0).stop 2023-10-06T03:08:17.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 msgr2=0x7f4e0c0b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c0b5ca0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c070e20 tx=0x7f4df8016000 comp rx=0 tx=0).stop 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 shutdown_connections 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4df0067320 0x7f4df00697d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e0c15db10 0x7f4e0c16d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e0c159890 0x7f4e0c16b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 --2- 172.21.15.92:0/782113457 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e0c159c60 0x7f4e0c0b5ca0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 >> 172.21.15.92:0/782113457 conn(0x7f4e0c068e20 msgr2=0x7f4e0c153290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 shutdown_connections 2023-10-06T03:08:17.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:17.929+0000 7f4e12d85700 1 -- 172.21.15.92:0/782113457 wait complete. 2023-10-06T03:08:17.933 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:08:17.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:08:18.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.062+0000 7f4d9275c700 1 Processor -- start 2023-10-06T03:08:18.064 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.062+0000 7f4d9275c700 1 -- start start 2023-10-06T03:08:18.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d9275c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d8c159890 0x7f4d8c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d9275c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c069560 0x7f4d8c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d9275c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8c069ee0 con 0x7f4d8c159c60 2023-10-06T03:08:18.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d9275c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8c15a160 con 0x7f4d8c159890 2023-10-06T03:08:18.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d9275c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c0d13a0 con 0x7f4d8c069560 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d8bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d8c159890 0x7f4d8c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d8bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d8c159890 0x7f4d8c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39978/0 (socket says 172.21.15.92:39978) 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.063+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4037747785 learned_addr learned my addr 172.21.15.92:0/4037747785 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d90cf9700 1 --2- 172.21.15.92:0/4037747785 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c069560 0x7f4d8c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4037747785 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c069560 msgr2=0x7f4d8c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8bfff700 1 --2- 172.21.15.92:0/4037747785 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c069560 0x7f4d8c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4037747785 >> v1:172.21.15.136:6789/0 conn(0x7f4d8c159c60 legacy=0x7f4d8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4037747785 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c15e140 con 0x7f4d8c159890 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8bfff700 1 --2- 172.21.15.92:0/4037747785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c159890 0x7f4d8c15abd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0099b0 tx=0x7f4d7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8067bee334ada5df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8affd700 1 -- 172.21.15.92:0/4037747785 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00b860 con 0x7f4d8c159890 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8affd700 1 -- 172.21.15.92:0/4037747785 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00b9c0 con 0x7f4d8c159890 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d8affd700 1 -- 172.21.15.92:0/4037747785 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00bd60 con 0x7f4d8c159890 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d9275c700 1 -- 172.21.15.92:0/4037747785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c159890 msgr2=0x7f4d8c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4037747785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c159890 0x7f4d8c15abd0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0099b0 tx=0x7f4d7c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d9275c700 1 -- 172.21.15.92:0/4037747785 shutdown_connections 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4037747785 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c159890 0x7f4d8c15abd0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.064+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4037747785 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c069560 0x7f4d8c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.065+0000 7f4d9275c700 1 -- 172.21.15.92:0/4037747785 >> 172.21.15.92:0/4037747785 conn(0x7f4d8c068e20 msgr2=0x7f4d8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:18.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.065+0000 7f4d9275c700 1 -- 172.21.15.92:0/4037747785 shutdown_connections 2023-10-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.065+0000 7f4d9275c700 1 -- 172.21.15.92:0/4037747785 wait complete. 2023-10-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 Processor -- start 2023-10-06T03:08:18.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 -- start start 2023-10-06T03:08:18.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 0x7f4d8c14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c159890 0x7f4d8c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c159c60 0x7f4d8c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c14d330 con 0x7f4d8c069560 2023-10-06T03:08:18.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c0ca380 con 0x7f4d8c159890 2023-10-06T03:08:18.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.066+0000 7f4d9275c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8c0c8b00 con 0x7f4d8c159c60 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 0x7f4d8c14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 0x7f4d8c14cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39980/0 (socket says 172.21.15.92:39980) 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4208945302 learned_addr learned my addr 172.21.15.92:0/4208945302 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d90cf9700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c159c60 0x7f4d8c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8b7fe700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c159890 0x7f4d8c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c159c60 msgr2=0x7f4d8c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c159c60 0x7f4d8c1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c159890 msgr2=0x7f4d8c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c159890 0x7f4d8c0c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 -- 172.21.15.92:0/4208945302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0094f0 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d8bfff700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 0x7f4d8c14cdf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0899c0 tx=0x7f4d7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c011e40 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c024440 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8c0c9770 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c024800 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.067+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8c0c9d30 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.068+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8c04c3c0 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.068+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4d7c02d440 con 0x7f4d8c069560 2023-10-06T03:08:18.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.069+0000 7f4d73fff700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4d78067320 0x7f4d780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.069+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4d7c0863b0 con 0x7f4d8c069560 2023-10-06T03:08:18.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.069+0000 7f4d8b7fe700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4d78067320 0x7f4d780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.069+0000 7f4d8b7fe700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4d78067320 0x7f4d780697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c151450 tx=0x7f4d80007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.073+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d7c053230 con 0x7f4d8c069560 2023-10-06T03:08:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.334+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4d8c0ca660 con 0x7f4d8c069560 2023-10-06T03:08:18.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.335+0000 7f4d73fff700 1 -- 172.21.15.92:0/4208945302 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4d7c056850 con 0x7f4d8c069560 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:18.337 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.338 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.339 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.340 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:08:18.341 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.342 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.343 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.344 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:08:18.345 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:18.346 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:08:18.347 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.348 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.349 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:08:18.350 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.351 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:18.352 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.353 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:18.353 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:08:18.353 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:08:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4d78067320 msgr2=0x7f4d780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4d78067320 0x7f4d780697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c151450 tx=0x7f4d80007ec0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 msgr2=0x7f4d8c14cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.354 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 0x7f4d8c14cdf0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0899c0 tx=0x7f4d7c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 shutdown_connections 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4d78067320 0x7f4d780697d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d8c069560 0x7f4d8c14cdf0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d8c159c60 0x7f4d8c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 --2- 172.21.15.92:0/4208945302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d8c159890 0x7f4d8c0c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 >> 172.21.15.92:0/4208945302 conn(0x7f4d8c068e20 msgr2=0x7f4d8c15c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 shutdown_connections 2023-10-06T03:08:18.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.337+0000 7f4d9275c700 1 -- 172.21.15.92:0/4208945302 wait complete. 2023-10-06T03:08:18.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 Processor -- start 2023-10-06T03:08:18.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 -- start start 2023-10-06T03:08:18.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdf2c159890 0x7fdf2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf2c069560 0x7fdf2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c069ee0 con 0x7fdf2c159c60 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c15a160 con 0x7fdf2c069560 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.468+0000 7fdf32706700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0d13a0 con 0x7fdf2c159890 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf2c069560 0x7fdf2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdf2c069560 0x7fdf2c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39992/0 (socket says 172.21.15.92:39992) 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 -- 172.21.15.92:0/3013113172 learned_addr learned my addr 172.21.15.92:0/3013113172 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2b7fe700 1 --1- 172.21.15.92:0/3013113172 >> v1:172.21.15.92:6789/0 conn(0x7fdf2c159c60 0x7fdf2c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49716/0 (socket says 172.21.15.92:49716) 2023-10-06T03:08:18.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2bfff700 1 --2- 172.21.15.92:0/3013113172 >> v2:172.21.15.136:3300/0 conn(0x7fdf2c159890 0x7fdf2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 -- 172.21.15.92:0/3013113172 >> v2:172.21.15.136:3300/0 conn(0x7fdf2c159890 msgr2=0x7fdf2c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 --2- 172.21.15.92:0/3013113172 >> v2:172.21.15.136:3300/0 conn(0x7fdf2c159890 0x7fdf2c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 -- 172.21.15.92:0/3013113172 >> v1:172.21.15.92:6789/0 conn(0x7fdf2c159c60 legacy=0x7fdf2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 -- 172.21.15.92:0/3013113172 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c15e140 con 0x7fdf2c069560 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2bfff700 1 --2- 172.21.15.92:0/3013113172 >> v2:172.21.15.136:3300/0 conn(0x7fdf2c159890 0x7fdf2c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2affd700 1 -- 172.21.15.92:0/3013113172 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385629339 0 0) 0x7fdf2c069ee0 con 0x7fdf2c159c60 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf30ca3700 1 --2- 172.21.15.92:0/3013113172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c069560 0x7fdf2c0699a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0d1090 tx=0x7fdf24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763656b47a152f2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2affd700 1 -- 172.21.15.92:0/3013113172 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400e070 con 0x7fdf2c069560 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2affd700 1 -- 172.21.15.92:0/3013113172 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400ba50 con 0x7fdf2c069560 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.469+0000 7fdf2affd700 1 -- 172.21.15.92:0/3013113172 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240092e0 con 0x7fdf2c069560 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 -- 172.21.15.92:0/3013113172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c069560 msgr2=0x7fdf2c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 --2- 172.21.15.92:0/3013113172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c069560 0x7fdf2c0699a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0d1090 tx=0x7fdf24009840 comp rx=0 tx=0).stop 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 -- 172.21.15.92:0/3013113172 shutdown_connections 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 --2- 172.21.15.92:0/3013113172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c069560 0x7fdf2c0699a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 --2- 172.21.15.92:0/3013113172 >> v2:172.21.15.136:3300/0 conn(0x7fdf2c159890 0x7fdf2c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 -- 172.21.15.92:0/3013113172 >> 172.21.15.92:0/3013113172 conn(0x7fdf2c068e20 msgr2=0x7fdf2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 -- 172.21.15.92:0/3013113172 shutdown_connections 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.470+0000 7fdf32706700 1 -- 172.21.15.92:0/3013113172 wait complete. 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.471+0000 7fdf32706700 1 Processor -- start 2023-10-06T03:08:18.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.471+0000 7fdf32706700 1 -- start start 2023-10-06T03:08:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf32706700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf2c069560 0x7fdf2c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf32706700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 0x7fdf2c0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf32706700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf2c159c60 0x7fdf2c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf32706700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0ce480 con 0x7fdf2c159890 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf32706700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c169930 con 0x7fdf2c159c60 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf32706700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c15b8e0 con 0x7fdf2c069560 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 0x7fdf2c0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 0x7fdf2c0b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:39996/0 (socket says 172.21.15.92:39996) 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 -- 172.21.15.92:0/1777042776 learned_addr learned my addr 172.21.15.92:0/1777042776 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf30ca3700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf2c159c60 0x7fdf2c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 -- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf2c069560 msgr2=0x7fdf2c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf2c069560 0x7fdf2c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 -- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf2c159c60 msgr2=0x7fdf2c1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf2c159c60 0x7fdf2c1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 -- 172.21.15.92:0/1777042776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24009ae0 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.472+0000 7fdf2b7fe700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 0x7fdf2c0b5380 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000c5e0 tx=0x7fdf20001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.473+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20005a70 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.473+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf20005bd0 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.473+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf200070d0 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.473+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0b5a70 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.473+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c15b570 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.474+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c04c3c0 con 0x7fdf2c159890 2023-10-06T03:08:18.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.474+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdf20007480 con 0x7fdf2c159890 2023-10-06T03:08:18.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.474+0000 7fdf13fff700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdf180673a0 0x7fdf18069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.474+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdf20084010 con 0x7fdf2c159890 2023-10-06T03:08:18.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.474+0000 7fdf2bfff700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdf180673a0 0x7fdf18069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.475+0000 7fdf2bfff700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdf180673a0 0x7fdf18069850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006fd0 tx=0x7fdf1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.479+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf20050e50 con 0x7fdf2c159890 2023-10-06T03:08:18.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.660+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2c15ba20 con 0x7fdf2c159890 2023-10-06T03:08:18.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.661+0000 7fdf13fff700 1 -- 172.21.15.92:0/1777042776 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fdf20017070 con 0x7fdf2c159890 2023-10-06T03:08:18.663 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:18.663 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":111,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:17.924601+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:18.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdf180673a0 msgr2=0x7fdf18069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdf180673a0 0x7fdf18069850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c006fd0 tx=0x7fdf1c008040 comp rx=0 tx=0).stop 2023-10-06T03:08:18.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 msgr2=0x7fdf2c0b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 0x7fdf2c0b5380 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000c5e0 tx=0x7fdf20001040 comp rx=0 tx=0).stop 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 shutdown_connections 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdf180673a0 0x7fdf18069850 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf2c159c60 0x7fdf2c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf2c069560 0x7fdf2c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 --2- 172.21.15.92:0/1777042776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf2c159890 0x7fdf2c0b5380 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 >> 172.21.15.92:0/1777042776 conn(0x7fdf2c068e20 msgr2=0x7fdf2c157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 shutdown_connections 2023-10-06T03:08:18.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.663+0000 7fdf32706700 1 -- 172.21.15.92:0/1777042776 wait complete. 2023-10-06T03:08:18.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:08:18.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:08:18.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.794+0000 7f75b1ca4700 1 Processor -- start 2023-10-06T03:08:18.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.794+0000 7f75b1ca4700 1 -- start start 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.794+0000 7f75b1ca4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75ac0aca80 0x7f75ac06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.794+0000 7f75b1ca4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac06bdb0 con 0x7f75ac069d60 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.794+0000 7f75b1ca4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac0cb370 con 0x7f75ac0c83c0 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.794+0000 7f75b1ca4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ac159f60 con 0x7f75ac0aca80 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75ab7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f75ac0aca80 0x7f75ac06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75aaffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f75ac069d60 0x7f75ac0c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49720/0 (socket says 172.21.15.92:49720) 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75aaffd700 1 -- 172.21.15.92:0/887523901 learned_addr learned my addr 172.21.15.92:0/887523901 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75ab7fe700 1 -- 172.21.15.92:0/887523901 >> v1:172.21.15.136:6789/0 conn(0x7f75ac0c83c0 legacy=0x7f75ac0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75ab7fe700 1 -- 172.21.15.92:0/887523901 >> v1:172.21.15.92:6789/0 conn(0x7f75ac069d60 legacy=0x7f75ac0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75ab7fe700 1 -- 172.21.15.92:0/887523901 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac06b510 con 0x7f75ac0aca80 2023-10-06T03:08:18.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75aa7fc700 1 -- 172.21.15.92:0/887523901 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262810997 0 0) 0x7f75ac06bdb0 con 0x7f75ac069d60 2023-10-06T03:08:18.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75ab7fe700 1 --2- 172.21.15.92:0/887523901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac06aba0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f759c009eb0 tx=0x7f759c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe69771d82ae52b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75aa7fc700 1 -- 172.21.15.92:0/887523901 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00e070 con 0x7f75ac0aca80 2023-10-06T03:08:18.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75aa7fc700 1 -- 172.21.15.92:0/887523901 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c00bc30 con 0x7f75ac0aca80 2023-10-06T03:08:18.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.795+0000 7f75aa7fc700 1 -- 172.21.15.92:0/887523901 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c012430 con 0x7f75ac0aca80 2023-10-06T03:08:18.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 -- 172.21.15.92:0/887523901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 msgr2=0x7f75ac06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/887523901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac06aba0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f759c009eb0 tx=0x7f759c009780 comp rx=0 tx=0).stop 2023-10-06T03:08:18.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 -- 172.21.15.92:0/887523901 shutdown_connections 2023-10-06T03:08:18.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/887523901 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac06aba0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 -- 172.21.15.92:0/887523901 >> 172.21.15.92:0/887523901 conn(0x7f75ac0cd670 msgr2=0x7f75ac0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:18.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 -- 172.21.15.92:0/887523901 shutdown_connections 2023-10-06T03:08:18.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.796+0000 7f75b1ca4700 1 -- 172.21.15.92:0/887523901 wait complete. 2023-10-06T03:08:18.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.797+0000 7f75b1ca4700 1 Processor -- start 2023-10-06T03:08:18.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.797+0000 7f75b1ca4700 1 -- start start 2023-10-06T03:08:18.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75ac069d60 0x7f75ac0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac0c83c0 0x7f75ac14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0ad500 con 0x7f75ac0aca80 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0acef0 con 0x7f75ac0c83c0 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0b14b0 con 0x7f75ac069d60 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac0af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40012/0 (socket says 172.21.15.92:40012) 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 -- 172.21.15.92:0/2033811942 learned_addr learned my addr 172.21.15.92:0/2033811942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75abfff700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac0c83c0 0x7f75ac14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75ab7fe700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75ac069d60 0x7f75ac0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 -- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75ac069d60 msgr2=0x7f75ac0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75ac069d60 0x7f75ac0b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 -- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac0c83c0 msgr2=0x7f75ac14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac0c83c0 0x7f75ac14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 -- 172.21.15.92:0/2033811942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c009a20 con 0x7f75ac0aca80 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75abfff700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac0c83c0 0x7f75ac14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75ab7fe700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75ac069d60 0x7f75ac0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75aaffd700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac0af870 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f759400c810 tx=0x7f7594007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594005c00 con 0x7f75ac0aca80 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7594005d60 con 0x7f75ac0aca80 2023-10-06T03:08:18.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac0ad190 con 0x7f75ac0aca80 2023-10-06T03:08:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594007320 con 0x7f75ac0aca80 2023-10-06T03:08:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.798+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac14c950 con 0x7f75ac0aca80 2023-10-06T03:08:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.799+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ac0ad670 con 0x7f75ac0aca80 2023-10-06T03:08:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.800+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f759401b400 con 0x7f75ac0aca80 2023-10-06T03:08:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.800+0000 7f75a37fe700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f758c067320 0x7f758c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:18.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.800+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7594084460 con 0x7f75ac0aca80 2023-10-06T03:08:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.800+0000 7f75ab7fe700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f758c067320 0x7f758c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:18.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.801+0000 7f75ab7fe700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f758c067320 0x7f758c0697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f759c009eb0 tx=0x7f759c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:18.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.805+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7594051460 con 0x7f75ac0aca80 2023-10-06T03:08:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.985+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75ac04c3c0 con 0x7f75ac0aca80 2023-10-06T03:08:18.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.986+0000 7f75a37fe700 1 -- 172.21.15.92:0/2033811942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7594054a80 con 0x7f75ac0aca80 2023-10-06T03:08:18.987 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:08:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.987+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f758c067320 msgr2=0x7f758c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.987+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f758c067320 0x7f758c0697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f759c009eb0 tx=0x7f759c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.987+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 msgr2=0x7f75ac0af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.987+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac0af870 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f759400c810 tx=0x7f7594007db0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 shutdown_connections 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f758c067320 0x7f758c0697d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75ac0c83c0 0x7f75ac14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75ac069d60 0x7f75ac0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 --2- 172.21.15.92:0/2033811942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75ac0aca80 0x7f75ac0af870 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 >> 172.21.15.92:0/2033811942 conn(0x7f75ac0cd670 msgr2=0x7f75ac06a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 shutdown_connections 2023-10-06T03:08:18.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:18.988+0000 7f75b1ca4700 1 -- 172.21.15.92:0/2033811942 wait complete. 2023-10-06T03:08:19.001 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:08:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.117+0000 7f0b4547e700 1 Processor -- start 2023-10-06T03:08:19.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.117+0000 7f0b4547e700 1 -- start start 2023-10-06T03:08:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b4547e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b4006b160 0x7f0b4006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b4547e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0b400637d0 0x7f0b40068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b4547e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400ac450 con 0x7f0b4006a2b0 2023-10-06T03:08:19.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b4547e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4006adc0 con 0x7f0b4006b160 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b4547e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b40069d00 con 0x7f0b400637d0 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b3effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b4006b160 0x7f0b4006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b3f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0b4006a2b0 0x7f0b4006a6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49730/0 (socket says 172.21.15.92:49730) 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b3f7fe700 1 -- 172.21.15.92:0/2307620569 learned_addr learned my addr 172.21.15.92:0/2307620569 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.118+0000 7f0b367fc700 1 --2- 172.21.15.92:0/2307620569 >> v2:172.21.15.136:3300/0 conn(0x7f0b400637d0 0x7f0b40068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3effd700 1 -- 172.21.15.92:0/2307620569 >> v2:172.21.15.136:3300/0 conn(0x7f0b400637d0 msgr2=0x7f0b40068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3effd700 1 --2- 172.21.15.92:0/2307620569 >> v2:172.21.15.136:3300/0 conn(0x7f0b400637d0 0x7f0b40068cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3effd700 1 -- 172.21.15.92:0/2307620569 >> v1:172.21.15.92:6789/0 conn(0x7f0b4006a2b0 legacy=0x7f0b4006a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3effd700 1 -- 172.21.15.92:0/2307620569 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b40068830 con 0x7f0b4006b160 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b367fc700 1 --2- 172.21.15.92:0/2307620569 >> v2:172.21.15.136:3300/0 conn(0x7f0b400637d0 0x7f0b40068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:19.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3effd700 1 --2- 172.21.15.92:0/2307620569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006b160 0x7f0b4006b530 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0b400af860 tx=0x7f0b300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e1eaf57a8d475a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3e7fc700 1 -- 172.21.15.92:0/2307620569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000b6b0 con 0x7f0b4006b160 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3e7fc700 1 -- 172.21.15.92:0/2307620569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3000b810 con 0x7f0b4006b160 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b3e7fc700 1 -- 172.21.15.92:0/2307620569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3000bbb0 con 0x7f0b4006b160 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 -- 172.21.15.92:0/2307620569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006b160 msgr2=0x7f0b4006b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2307620569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006b160 0x7f0b4006b530 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0b400af860 tx=0x7f0b300091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 -- 172.21.15.92:0/2307620569 shutdown_connections 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2307620569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006b160 0x7f0b4006b530 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2307620569 >> v2:172.21.15.136:3300/0 conn(0x7f0b400637d0 0x7f0b40068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 -- 172.21.15.92:0/2307620569 >> 172.21.15.92:0/2307620569 conn(0x7f0b4006ded0 msgr2=0x7f0b4006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 -- 172.21.15.92:0/2307620569 shutdown_connections 2023-10-06T03:08:19.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.119+0000 7f0b4547e700 1 -- 172.21.15.92:0/2307620569 wait complete. 2023-10-06T03:08:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 Processor -- start 2023-10-06T03:08:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 -- start start 2023-10-06T03:08:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b400637d0 0x7f0b401553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 0x7f0b400c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b4006b160 0x7f0b40156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400c1f60 con 0x7f0b4006a2b0 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b40155ab0 con 0x7f0b400637d0 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b4547e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b40155d90 con 0x7f0b4006b160 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b367fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 0x7f0b400c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.121+0000 7f0b367fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 0x7f0b400c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40034/0 (socket says 172.21.15.92:40034) 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 -- 172.21.15.92:0/2596417411 learned_addr learned my addr 172.21.15.92:0/2596417411 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b3f7fe700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b4006b160 0x7f0b40156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b3effd700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b400637d0 0x7f0b401553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 -- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b4006b160 msgr2=0x7f0b40156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b4006b160 0x7f0b40156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 -- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b400637d0 msgr2=0x7f0b401553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b400637d0 0x7f0b401553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 -- 172.21.15.92:0/2596417411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30009420 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b3f7fe700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b4006b160 0x7f0b40156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b367fc700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 0x7f0b400c1a20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800d860 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280125f0 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28012750 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28012ad0 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b40156070 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.122+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b40156630 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.123+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b401569a0 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.123+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0b28014430 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.124+0000 7f0b377fe700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0b20067320 0x7f0b200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.124+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0b28084f10 con 0x7f0b4006a2b0 2023-10-06T03:08:19.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.124+0000 7f0b3effd700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0b20067320 0x7f0b200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.124+0000 7f0b3effd700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0b20067320 0x7f0b200697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0b300093f0 tx=0x7f0b30017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.128+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b28055820 con 0x7f0b4006a2b0 2023-10-06T03:08:19.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.312+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b4004c3c0 con 0x7f0b4006a2b0 2023-10-06T03:08:19.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.313+0000 7f0b377fe700 1 -- 172.21.15.92:0/2596417411 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b280553b0 con 0x7f0b4006a2b0 2023-10-06T03:08:19.315 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:08:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0b20067320 msgr2=0x7f0b200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0b20067320 0x7f0b200697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0b300093f0 tx=0x7f0b30017000 comp rx=0 tx=0).stop 2023-10-06T03:08:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 msgr2=0x7f0b400c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 0x7f0b400c1a20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800d860 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2023-10-06T03:08:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 shutdown_connections 2023-10-06T03:08:19.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0b20067320 0x7f0b200697d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b4006b160 0x7f0b40156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b400637d0 0x7f0b401553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 --2- 172.21.15.92:0/2596417411 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b4006a2b0 0x7f0b400c1a20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 >> 172.21.15.92:0/2596417411 conn(0x7f0b4006ded0 msgr2=0x7f0b40153760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 shutdown_connections 2023-10-06T03:08:19.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.315+0000 7f0b4547e700 1 -- 172.21.15.92:0/2596417411 wait complete. 2023-10-06T03:08:19.329 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:08:19.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.441+0000 7fe5cd7e3700 1 Processor -- start 2023-10-06T03:08:19.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.441+0000 7fe5cd7e3700 1 -- start start 2023-10-06T03:08:19.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5cd7e3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5cd7e3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5cd7e3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c815a160 con 0x7fe5c815db10 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5cd7e3700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c80d13a0 con 0x7fe5c8159c60 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5cd7e3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c815e140 con 0x7fe5c8159890 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5c77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe5c815db10 0x7fe5c8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37508/0 (socket says 172.21.15.92:37508) 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5c77fe700 1 -- 172.21.15.92:0/238298223 learned_addr learned my addr 172.21.15.92:0/238298223 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5be7fc700 1 --2- 172.21.15.92:0/238298223 >> v2:172.21.15.136:3300/0 conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.442+0000 7fe5c6ffd700 1 --2- 172.21.15.92:0/238298223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c67fc700 1 -- 172.21.15.92:0/238298223 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825930601 0 0) 0x7fe5c815a160 con 0x7fe5c815db10 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c67fc700 1 -- 172.21.15.92:0/238298223 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b4003570 con 0x7fe5c815db10 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c6ffd700 1 -- 172.21.15.92:0/238298223 >> v2:172.21.15.136:3300/0 conn(0x7fe5c8159c60 msgr2=0x7fe5c815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c6ffd700 1 --2- 172.21.15.92:0/238298223 >> v2:172.21.15.136:3300/0 conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c6ffd700 1 -- 172.21.15.92:0/238298223 >> v1:172.21.15.92:6789/0 conn(0x7fe5c815db10 legacy=0x7fe5c8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c6ffd700 1 -- 172.21.15.92:0/238298223 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c815a3d0 con 0x7fe5c8159890 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5be7fc700 1 --2- 172.21.15.92:0/238298223 >> v2:172.21.15.136:3300/0 conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:19.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c6ffd700 1 --2- 172.21.15.92:0/238298223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159890 0x7fe5c815abd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80098e0 tx=0x7fe5b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984f86003891656c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c67fc700 1 -- 172.21.15.92:0/238298223 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8011630 con 0x7fe5c8159890 2023-10-06T03:08:19.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c67fc700 1 -- 172.21.15.92:0/238298223 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8011790 con 0x7fe5c8159890 2023-10-06T03:08:19.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.443+0000 7fe5c67fc700 1 -- 172.21.15.92:0/238298223 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8011b30 con 0x7fe5c8159890 2023-10-06T03:08:19.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/238298223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159890 msgr2=0x7fe5c815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/238298223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159890 0x7fe5c815abd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80098e0 tx=0x7fe5b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/238298223 shutdown_connections 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/238298223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/238298223 >> v2:172.21.15.136:3300/0 conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/238298223 >> 172.21.15.92:0/238298223 conn(0x7fe5c8068e20 msgr2=0x7fe5c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/238298223 shutdown_connections 2023-10-06T03:08:19.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.444+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/238298223 wait complete. 2023-10-06T03:08:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.445+0000 7fe5cd7e3700 1 Processor -- start 2023-10-06T03:08:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.445+0000 7fe5cd7e3700 1 -- start start 2023-10-06T03:08:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c8159890 0x7fe5c815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159c60 0x7fe5c8151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 0x7fe5c8151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c815b6d0 con 0x7fe5c815db10 2023-10-06T03:08:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c815c250 con 0x7fe5c8159890 2023-10-06T03:08:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c815b920 con 0x7fe5c8159c60 2023-10-06T03:08:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 0x7fe5c8151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 0x7fe5c8151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37884/0 (socket says 172.21.15.92:37884) 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 -- 172.21.15.92:0/2222188044 learned_addr learned my addr 172.21.15.92:0/2222188044 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5be7fc700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159c60 0x7fe5c8151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c6ffd700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c8159890 0x7fe5c815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 -- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159c60 msgr2=0x7fe5c8151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159c60 0x7fe5c8151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 -- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c8159890 msgr2=0x7fe5c815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c8159890 0x7fe5c815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 -- 172.21.15.92:0/2222188044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b0009420 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5c77fe700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 0x7fe5c8151d50 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0001460 tx=0x7fe5c0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000a870 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c000a9d0 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b8009420 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c000ad50 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.446+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c815bee0 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.447+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c815c400 con 0x7fe5c815db10 2023-10-06T03:08:19.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.448+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe5c0007300 con 0x7fe5c815db10 2023-10-06T03:08:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.448+0000 7fe5bf7fe700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a8067320 0x7fe5a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.448+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe5c0084d50 con 0x7fe5c815db10 2023-10-06T03:08:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.448+0000 7fe5c6ffd700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a8067320 0x7fe5a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.448+0000 7fe5c6ffd700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a8067320 0x7fe5a80697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80091e0 tx=0x7fe5b8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.452+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5c0051c80 con 0x7fe5c815db10 2023-10-06T03:08:19.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.637+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe5c804c3c0 con 0x7fe5c815db10 2023-10-06T03:08:19.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.637+0000 7fe5bf7fe700 1 -- 172.21.15.92:0/2222188044 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v111) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe5c00552a0 con 0x7fe5c815db10 2023-10-06T03:08:19.639 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:19.640 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:08:19.641 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:08:19.641 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:19.641 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:08:19.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a8067320 msgr2=0x7fe5a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a8067320 0x7fe5a80697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80091e0 tx=0x7fe5b8017000 comp rx=0 tx=0).stop 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 msgr2=0x7fe5c8151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 0x7fe5c8151d50 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0001460 tx=0x7fe5c0001a40 comp rx=0 tx=0).stop 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 shutdown_connections 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a8067320 0x7fe5a80697d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5c815db10 0x7fe5c8151d50 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5c8159890 0x7fe5c815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 --2- 172.21.15.92:0/2222188044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5c8159c60 0x7fe5c8151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 >> 172.21.15.92:0/2222188044 conn(0x7fe5c8068e20 msgr2=0x7fe5c814d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 shutdown_connections 2023-10-06T03:08:19.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.639+0000 7fe5cd7e3700 1 -- 172.21.15.92:0/2222188044 wait complete. 2023-10-06T03:08:19.653 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:08:19.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:08:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.771+0000 7fcee5a6e700 1 Processor -- start 2023-10-06T03:08:19.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.771+0000 7fcee5a6e700 1 -- start start 2023-10-06T03:08:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee5a6e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee00b03c0 0x7fcee00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee5a6e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcee01591e0 0x7fcee0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee5a6e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee00cfa00 con 0x7fcee00b0790 2023-10-06T03:08:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee5a6e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee00cb580 con 0x7fcee01591e0 2023-10-06T03:08:19.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee5a6e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee00cf160 con 0x7fcee00b03c0 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcee01591e0 0x7fcee0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcee01591e0 0x7fcee0159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37888/0 (socket says 172.21.15.92:37888) 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcedffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcee00b0790 0x7fcee00b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37518/0 (socket says 172.21.15.92:37518) 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 -- 172.21.15.92:0/1727189114 learned_addr learned my addr 172.21.15.92:0/1727189114 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee4a6c700 1 --2- 172.21.15.92:0/1727189114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee00b03c0 0x7fcee00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 -- 172.21.15.92:0/1727189114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee00b03c0 msgr2=0x7fcee00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 --2- 172.21.15.92:0/1727189114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee00b03c0 0x7fcee00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 -- 172.21.15.92:0/1727189114 >> v1:172.21.15.92:6789/0 conn(0x7fcee00b0790 legacy=0x7fcee00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 -- 172.21.15.92:0/1727189114 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee00cf440 con 0x7fcee01591e0 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcedf7fe700 1 -- 172.21.15.92:0/1727189114 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674215272 0 0) 0x7fcee00cfa00 con 0x7fcee00b0790 2023-10-06T03:08:19.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcee526d700 1 --2- 172.21.15.92:0/1727189114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee01591e0 0x7fcee0159620 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fced8009f20 tx=0x7fced8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f4982fce959888 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.772+0000 7fcedf7fe700 1 -- 172.21.15.92:0/1727189114 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800e070 con 0x7fcee01591e0 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcedf7fe700 1 -- 172.21.15.92:0/1727189114 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced800b9d0 con 0x7fcee01591e0 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcedf7fe700 1 -- 172.21.15.92:0/1727189114 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80092e0 con 0x7fcee01591e0 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 -- 172.21.15.92:0/1727189114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee01591e0 msgr2=0x7fcee0159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/1727189114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee01591e0 0x7fcee0159620 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fced8009f20 tx=0x7fced8009840 comp rx=0 tx=0).stop 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 -- 172.21.15.92:0/1727189114 shutdown_connections 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/1727189114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee00b03c0 0x7fcee00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/1727189114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee01591e0 0x7fcee0159620 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 -- 172.21.15.92:0/1727189114 >> 172.21.15.92:0/1727189114 conn(0x7fcee0063230 msgr2=0x7fcee0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 -- 172.21.15.92:0/1727189114 shutdown_connections 2023-10-06T03:08:19.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.773+0000 7fcee5a6e700 1 -- 172.21.15.92:0/1727189114 wait complete. 2023-10-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.774+0000 7fcee5a6e700 1 Processor -- start 2023-10-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.774+0000 7fcee5a6e700 1 -- start start 2023-10-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee5a6e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee00b03c0 0x7fcee00cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee5a6e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 0x7fcee015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee5a6e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee01591e0 0x7fcee015af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee5a6e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015c5e0 con 0x7fcee00b0790 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee5a6e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015b9d0 con 0x7fcee00b03c0 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee5a6e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee015bcb0 con 0x7fcee01591e0 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 0x7fcee015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 0x7fcee015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37902/0 (socket says 172.21.15.92:37902) 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 -- 172.21.15.92:0/3179301334 learned_addr learned my addr 172.21.15.92:0/3179301334 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee4a6c700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee00b03c0 0x7fcee00cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcee526d700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee01591e0 0x7fcee015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 -- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee01591e0 msgr2=0x7fcee015af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee01591e0 0x7fcee015af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 -- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee00b03c0 msgr2=0x7fcee00cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:19.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.775+0000 7fcedffff700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee00b03c0 0x7fcee00cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcedffff700 1 -- 172.21.15.92:0/3179301334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009ae0 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcee4a6c700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee00b03c0 0x7fcee00cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcee526d700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee01591e0 0x7fcee015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcedffff700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 0x7fcee015ca70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0011e0 tx=0x7fcecc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc007300 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc007460 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee015bf90 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004830 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.776+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee00ca2f0 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.777+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee015c750 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.777+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcecc004be0 con 0x7fcee00b0790 2023-10-06T03:08:19.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.778+0000 7fcedcff9700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcec4067320 0x7fcec40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.778+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcecc083fc0 con 0x7fcee00b0790 2023-10-06T03:08:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.778+0000 7fcee4a6c700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcec4067320 0x7fcec40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.778+0000 7fcee4a6c700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcec4067320 0x7fcec40697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fced4009ce0 tx=0x7fced4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:19.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.782+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcecc050ec0 con 0x7fcee00b0790 2023-10-06T03:08:19.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:19.965+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcee004c3c0 con 0x7fcee00b0790 2023-10-06T03:08:20.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.943+0000 7fcedcff9700 1 -- 172.21.15.92:0/3179301334 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v112) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcecc018020 con 0x7fcee00b0790 2023-10-06T03:08:20.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcec4067320 msgr2=0x7fcec40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcec4067320 0x7fcec40697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fced4009ce0 tx=0x7fced4008040 comp rx=0 tx=0).stop 2023-10-06T03:08:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 msgr2=0x7fcee015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 0x7fcee015ca70 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0011e0 tx=0x7fcecc007e10 comp rx=0 tx=0).stop 2023-10-06T03:08:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 shutdown_connections 2023-10-06T03:08:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcec4067320 0x7fcec40697d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:20.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcee01591e0 0x7fcee015af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcee00b03c0 0x7fcee00cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 --2- 172.21.15.92:0/3179301334 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcee00b0790 0x7fcee015ca70 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 >> 172.21.15.92:0/3179301334 conn(0x7fcee0063230 msgr2=0x7fcee0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 shutdown_connections 2023-10-06T03:08:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:20.947+0000 7fcee5a6e700 1 -- 172.21.15.92:0/3179301334 wait complete. 2023-10-06T03:08:20.951 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:08:20.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:08:21.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.087+0000 7f7e9de05700 1 Processor -- start 2023-10-06T03:08:21.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.087+0000 7f7e9de05700 1 -- start start 2023-10-06T03:08:21.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e9de05700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e98069c70 0x7f7e980cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e9de05700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e980ce380 con 0x7f7e980af320 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e9de05700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e980cec20 con 0x7f7e98159950 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e9de05700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980afa90 con 0x7f7e98069c70 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e96ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7e98069c70 0x7f7e980cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e97fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7e980af320 0x7f7e980cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37524/0 (socket says 172.21.15.92:37524) 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e97fff700 1 -- 172.21.15.92:0/3092971505 learned_addr learned my addr 172.21.15.92:0/3092971505 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:21.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e96ffd700 1 -- 172.21.15.92:0/3092971505 >> v1:172.21.15.136:6789/0 conn(0x7f7e98159950 legacy=0x7f7e98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e96ffd700 1 -- 172.21.15.92:0/3092971505 >> v1:172.21.15.92:6789/0 conn(0x7f7e980af320 legacy=0x7f7e980cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.088+0000 7f7e96ffd700 1 -- 172.21.15.92:0/3092971505 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980ce5d0 con 0x7f7e98069c70 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e96ffd700 1 --2- 172.21.15.92:0/3092971505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e98069c70 0x7f7e980cbb90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0098e0 tx=0x7f7e8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fed4f34ca21b8c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e967fc700 1 -- 172.21.15.92:0/3092971505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c009e80 con 0x7f7e98069c70 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e967fc700 1 -- 172.21.15.92:0/3092971505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c00b760 con 0x7f7e98069c70 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e967fc700 1 -- 172.21.15.92:0/3092971505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00bae0 con 0x7f7e98069c70 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 -- 172.21.15.92:0/3092971505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e98069c70 msgr2=0x7f7e980cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:21.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 --2- 172.21.15.92:0/3092971505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e98069c70 0x7f7e980cbb90 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0098e0 tx=0x7f7e8c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 -- 172.21.15.92:0/3092971505 shutdown_connections 2023-10-06T03:08:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 --2- 172.21.15.92:0/3092971505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e98069c70 0x7f7e980cbb90 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 -- 172.21.15.92:0/3092971505 >> 172.21.15.92:0/3092971505 conn(0x7f7e98068e00 msgr2=0x7f7e980d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 -- 172.21.15.92:0/3092971505 shutdown_connections 2023-10-06T03:08:21.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.089+0000 7f7e9de05700 1 -- 172.21.15.92:0/3092971505 wait complete. 2023-10-06T03:08:21.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.090+0000 7f7e9de05700 1 Processor -- start 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 -- start start 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98069c70 0x7f7e9816a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 0x7f7e980c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e98159950 0x7f7e9816ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980b62d0 con 0x7f7e980af320 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980c80b0 con 0x7f7e98069c70 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e9de05700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980c8390 con 0x7f7e98159950 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 0x7f7e980c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 0x7f7e980c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37924/0 (socket says 172.21.15.92:37924) 2023-10-06T03:08:21.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.091+0000 7f7e96ffd700 1 -- 172.21.15.92:0/1091604508 learned_addr learned my addr 172.21.15.92:0/1091604508 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e97fff700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e98159950 0x7f7e9816ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e977fe700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98069c70 0x7f7e9816a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e96ffd700 1 -- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e98159950 msgr2=0x7f7e9816ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e96ffd700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e98159950 0x7f7e9816ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e96ffd700 1 -- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98069c70 msgr2=0x7f7e9816a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e96ffd700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98069c70 0x7f7e9816a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e96ffd700 1 -- 172.21.15.92:0/1091604508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009420 con 0x7f7e980af320 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e96ffd700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 0x7f7e980c7860 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0093f0 tx=0x7f7e8c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00bee0 con 0x7f7e980af320 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c00bae0 con 0x7f7e980af320 2023-10-06T03:08:21.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e980c8670 con 0x7f7e980af320 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01c6b0 con 0x7f7e980af320 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.092+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e980c8c30 con 0x7f7e980af320 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.093+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e980c8e20 con 0x7f7e980af320 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.093+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7e8c01caa0 con 0x7f7e980af320 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.094+0000 7f7e7f7fe700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e84067320 0x7f7e840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.094+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f7e8c087060 con 0x7f7e980af320 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.094+0000 7f7e977fe700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e84067320 0x7f7e840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:21.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.094+0000 7f7e977fe700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e84067320 0x7f7e840697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e98159370 tx=0x7f7e88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:21.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.098+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e8c0b85a0 con 0x7f7e980af320 2023-10-06T03:08:21.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.282+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7e980c8e20 con 0x7f7e980af320 2023-10-06T03:08:21.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.948+0000 7f7e7f7fe700 1 -- 172.21.15.92:0/1091604508 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v113) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7e8c053d70 con 0x7f7e980af320 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.955+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e84067320 msgr2=0x7f7e840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.955+0000 7f7e9de05700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e84067320 0x7f7e840697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e98159370 tx=0x7f7e88009040 comp rx=0 tx=0).stop 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 msgr2=0x7f7e980c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 0x7f7e980c7860 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0093f0 tx=0x7f7e8c016000 comp rx=0 tx=0).stop 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 shutdown_connections 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e84067320 0x7f7e840697d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e98159950 0x7f7e9816ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e98069c70 0x7f7e9816a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 --2- 172.21.15.92:0/1091604508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e980af320 0x7f7e980c7860 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.956+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 >> 172.21.15.92:0/1091604508 conn(0x7f7e98068e00 msgr2=0x7f7e980b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.957+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 shutdown_connections 2023-10-06T03:08:21.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:21.958+0000 7f7e9de05700 1 -- 172.21.15.92:0/1091604508 wait complete. 2023-10-06T03:08:21.969 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:08:21.979 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:08:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.095+0000 7f6a5f4ec700 1 Processor -- start 2023-10-06T03:08:22.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.095+0000 7f6a5f4ec700 1 -- start start 2023-10-06T03:08:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5f4ec700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6a580b4e70 0x7f6a580ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:22.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5f4ec700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580cfb00 con 0x7f6a580b1b40 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5f4ec700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5815e060 con 0x7f6a580b16d0 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5f4ec700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a580b5c40 con 0x7f6a580b4e70 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5e4ea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6a580b4e70 0x7f6a580ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5eceb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6a580b1b40 0x7f6a5815d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37530/0 (socket says 172.21.15.92:37530) 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5eceb700 1 -- 172.21.15.92:0/2521465514 learned_addr learned my addr 172.21.15.92:0/2521465514 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5e4ea700 1 -- 172.21.15.92:0/2521465514 >> v1:172.21.15.136:6789/0 conn(0x7f6a580b16d0 legacy=0x7f6a5815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5e4ea700 1 -- 172.21.15.92:0/2521465514 >> v1:172.21.15.92:6789/0 conn(0x7f6a580b1b40 legacy=0x7f6a5815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5e4ea700 1 -- 172.21.15.92:0/2521465514 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a580b5f20 con 0x7f6a580b4e70 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.096+0000 7f6a5d4e8700 1 -- 172.21.15.92:0/2521465514 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461905474 0 0) 0x7f6a580cfb00 con 0x7f6a580b1b40 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5e4ea700 1 --2- 172.21.15.92:0/2521465514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b4e70 0x7f6a580ccff0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009e60 tx=0x7f6a54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1db24c86d56f67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:22.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5d4e8700 1 -- 172.21.15.92:0/2521465514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400e070 con 0x7f6a580b4e70 2023-10-06T03:08:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5d4e8700 1 -- 172.21.15.92:0/2521465514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5400bbf0 con 0x7f6a580b4e70 2023-10-06T03:08:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5d4e8700 1 -- 172.21.15.92:0/2521465514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54012430 con 0x7f6a580b4e70 2023-10-06T03:08:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/2521465514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b4e70 msgr2=0x7f6a580ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:22.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/2521465514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b4e70 0x7f6a580ccff0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009e60 tx=0x7f6a54009780 comp rx=0 tx=0).stop 2023-10-06T03:08:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/2521465514 shutdown_connections 2023-10-06T03:08:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/2521465514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b4e70 0x7f6a580ccff0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/2521465514 >> 172.21.15.92:0/2521465514 conn(0x7f6a580d10c0 msgr2=0x7f6a580affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/2521465514 shutdown_connections 2023-10-06T03:08:22.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.097+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/2521465514 wait complete. 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.098+0000 7f6a5f4ec700 1 Processor -- start 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.098+0000 7f6a5f4ec700 1 -- start start 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5f4ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a580b16d0 0x7f6a580cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5f4ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 0x7f6a58157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5f4ec700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a580b4e70 0x7f6a58157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5f4ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c7e60 con 0x7f6a580b1b40 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5f4ec700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c8fd0 con 0x7f6a580b16d0 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5f4ec700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a580c92b0 con 0x7f6a580b4e70 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5dce9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 0x7f6a58157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5dce9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 0x7f6a58157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37932/0 (socket says 172.21.15.92:37932) 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.099+0000 7f6a5dce9700 1 -- 172.21.15.92:0/564913080 learned_addr learned my addr 172.21.15.92:0/564913080 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:22.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5eceb700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a580b4e70 0x7f6a58157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5dce9700 1 -- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a580b4e70 msgr2=0x7f6a58157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5e4ea700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a580b16d0 0x7f6a580cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5dce9700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a580b4e70 0x7f6a58157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5dce9700 1 -- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a580b16d0 msgr2=0x7f6a580cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5dce9700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a580b16d0 0x7f6a580cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5dce9700 1 -- 172.21.15.92:0/564913080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a54009a20 con 0x7f6a580b1b40 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5dce9700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 0x7f6a58157770 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b600 tx=0x7f6a480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800f510 con 0x7f6a580b1b40 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4800f670 con 0x7f6a580b1b40 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a580c9590 con 0x7f6a580b1b40 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800fa30 con 0x7f6a580b1b40 2023-10-06T03:08:22.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.100+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a58146ad0 con 0x7f6a580b1b40 2023-10-06T03:08:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.101+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a580c8d20 con 0x7f6a580b1b40 2023-10-06T03:08:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.101+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6a4801a400 con 0x7f6a580b1b40 2023-10-06T03:08:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.102+0000 7f6a4e7fc700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a3c067320 0x7f6a3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:22.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.102+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f6a480848f0 con 0x7f6a580b1b40 2023-10-06T03:08:22.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.102+0000 7f6a5e4ea700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a3c067320 0x7f6a3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:22.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.102+0000 7f6a5e4ea700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a3c067320 0x7f6a3c0697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6a540099f0 tx=0x7f6a54018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:22.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.106+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a48051500 con 0x7f6a580b1b40 2023-10-06T03:08:22.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.331+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6a5804bb80 con 0x7f6a580b1b40 2023-10-06T03:08:22.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.948+0000 7f6a4e7fc700 1 -- 172.21.15.92:0/564913080 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6a48054b20 con 0x7f6a580b1b40 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.953+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a3c067320 msgr2=0x7f6a3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.953+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a3c067320 0x7f6a3c0697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6a540099f0 tx=0x7f6a54018000 comp rx=0 tx=0).stop 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.953+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 msgr2=0x7f6a58157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.953+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 0x7f6a58157770 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b600 tx=0x7f6a480091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 shutdown_connections 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a3c067320 0x7f6a3c0697d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a580b4e70 0x7f6a58157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a580b16d0 0x7f6a580cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 --2- 172.21.15.92:0/564913080 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a580b1b40 0x7f6a58157770 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:22.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 >> 172.21.15.92:0/564913080 conn(0x7f6a580d10c0 msgr2=0x7f6a580ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:22.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 shutdown_connections 2023-10-06T03:08:22.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:22.954+0000 7f6a5f4ec700 1 -- 172.21.15.92:0/564913080 wait complete. 2023-10-06T03:08:22.957 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T03:08:22.970 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:23.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.083+0000 7fb517fff700 1 Processor -- start 2023-10-06T03:08:23.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.083+0000 7fb517fff700 1 -- start start 2023-10-06T03:08:23.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb517fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb518159420 0x7fb5180b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb517fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb51815d230 0x7fb51815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb517fff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5180b2980 con 0x7fb5181597f0 2023-10-06T03:08:23.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb517fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51815db70 con 0x7fb518159420 2023-10-06T03:08:23.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb517fff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5180b11d0 con 0x7fb51815d230 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb518159420 0x7fb5180b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb518159420 0x7fb5180b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37934/0 (socket says 172.21.15.92:37934) 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 -- 172.21.15.92:0/3285710693 learned_addr learned my addr 172.21.15.92:0/3285710693 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb5177fe700 1 --2- 172.21.15.92:0/3285710693 >> v2:172.21.15.136:3300/0 conn(0x7fb51815d230 0x7fb51815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 -- 172.21.15.92:0/3285710693 >> v2:172.21.15.136:3300/0 conn(0x7fb51815d230 msgr2=0x7fb51815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 --2- 172.21.15.92:0/3285710693 >> v2:172.21.15.136:3300/0 conn(0x7fb51815d230 0x7fb51815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 -- 172.21.15.92:0/3285710693 >> v1:172.21.15.136:6789/0 conn(0x7fb5181597f0 legacy=0x7fb5180b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 -- 172.21.15.92:0/3285710693 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5180b14b0 con 0x7fb518159420 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.084+0000 7fb516ffd700 1 --2- 172.21.15.92:0/3285710693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb518159420 0x7fb5180b5f40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb51815d170 tx=0x7fb508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68014f9abc1ec516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb515ffb700 1 -- 172.21.15.92:0/3285710693 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800b860 con 0x7fb518159420 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb515ffb700 1 -- 172.21.15.92:0/3285710693 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800b9c0 con 0x7fb518159420 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb515ffb700 1 -- 172.21.15.92:0/3285710693 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800bd60 con 0x7fb518159420 2023-10-06T03:08:23.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 -- 172.21.15.92:0/3285710693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb518159420 msgr2=0x7fb5180b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 --2- 172.21.15.92:0/3285710693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb518159420 0x7fb5180b5f40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb51815d170 tx=0x7fb508009250 comp rx=0 tx=0).stop 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 -- 172.21.15.92:0/3285710693 shutdown_connections 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 --2- 172.21.15.92:0/3285710693 >> v2:172.21.15.136:3300/0 conn(0x7fb51815d230 0x7fb51815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 --2- 172.21.15.92:0/3285710693 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb518159420 0x7fb5180b5f40 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 -- 172.21.15.92:0/3285710693 >> 172.21.15.92:0/3285710693 conn(0x7fb5180d0660 msgr2=0x7fb5180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 -- 172.21.15.92:0/3285710693 shutdown_connections 2023-10-06T03:08:23.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.085+0000 7fb517fff700 1 -- 172.21.15.92:0/3285710693 wait complete. 2023-10-06T03:08:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.086+0000 7fb517fff700 1 Processor -- start 2023-10-06T03:08:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.086+0000 7fb517fff700 1 -- start start 2023-10-06T03:08:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb517fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb518159420 0x7fb518155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb517fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5181597f0 0x7fb518169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb517fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 0x7fb51816a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb517fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5181566b0 con 0x7fb51815d230 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb517fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51816a7b0 con 0x7fb518159420 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb517fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51816aa90 con 0x7fb5181597f0 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb5177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 0x7fb51816a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb5177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 0x7fb51816a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37938/0 (socket says 172.21.15.92:37938) 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb5177fe700 1 -- 172.21.15.92:0/3295729533 learned_addr learned my addr 172.21.15.92:0/3295729533 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb516ffd700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb518159420 0x7fb518155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.087+0000 7fb5167fc700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5181597f0 0x7fb518169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb5177fe700 1 -- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5181597f0 msgr2=0x7fb518169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb5177fe700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5181597f0 0x7fb518169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb5177fe700 1 -- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb518159420 msgr2=0x7fb518155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb5177fe700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb518159420 0x7fb518155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb5177fe700 1 -- 172.21.15.92:0/3295729533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5080094f0 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb5177fe700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 0x7fb51816a110 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb510007f20 tx=0x7fb510009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000eb90 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51000ed10 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb518159ca0 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510014670 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.088+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51815a260 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.089+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51804c3c0 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.089+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb5100149e0 con 0x7fb51815d230 2023-10-06T03:08:23.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.090+0000 7fb4feffd700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb504067320 0x7fb5040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.090+0000 7fb516ffd700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb504067320 0x7fb5040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.090+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb510085350 con 0x7fb51815d230 2023-10-06T03:08:23.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.090+0000 7fb516ffd700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb504067320 0x7fb5040697d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb51815d170 tx=0x7fb508017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:23.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.094+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb510051e80 con 0x7fb51815d230 2023-10-06T03:08:23.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.283+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb51815a450 con 0x7fb51815d230 2023-10-06T03:08:23.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.284+0000 7fb4feffd700 1 -- 172.21.15.92:0/3295729533 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fb5100554a0 con 0x7fb51815d230 2023-10-06T03:08:23.286 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:23.287 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":114,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:22.333732+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:08:19.983879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T03:08:21.301577+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb504067320 msgr2=0x7fb5040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb504067320 0x7fb5040697d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb51815d170 tx=0x7fb508017000 comp rx=0 tx=0).stop 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 msgr2=0x7fb51816a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 0x7fb51816a110 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb510007f20 tx=0x7fb510009860 comp rx=0 tx=0).stop 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 shutdown_connections 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb504067320 0x7fb5040697d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb51815d230 0x7fb51816a110 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb518159420 0x7fb518155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 --2- 172.21.15.92:0/3295729533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5181597f0 0x7fb518169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 >> 172.21.15.92:0/3295729533 conn(0x7fb5180d0660 msgr2=0x7fb518069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:23.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 shutdown_connections 2023-10-06T03:08:23.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.286+0000 7fb517fff700 1 -- 172.21.15.92:0/3295729533 wait complete. 2023-10-06T03:08:23.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:23.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.412+0000 7f5a816b7700 1 Processor -- start 2023-10-06T03:08:23.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.412+0000 7f5a816b7700 1 -- start start 2023-10-06T03:08:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a816b7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a7c1597c0 0x7f5a7c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a816b7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c0af210 0x7f5a7c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a816b7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7c0ae390 con 0x7f5a7c0aedc0 2023-10-06T03:08:23.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a816b7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7c15e390 con 0x7f5a7c1597c0 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a816b7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0ae930 con 0x7f5a7c0af210 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a7c1597c0 0x7f5a7c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5a7c1597c0 0x7f5a7c159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37954/0 (socket says 172.21.15.92:37954) 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 -- 172.21.15.92:0/2960596753 learned_addr learned my addr 172.21.15.92:0/2960596753 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a727fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5a7c0aedc0 0x7f5a7c15d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37532/0 (socket says 172.21.15.92:37532) 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7b7fe700 1 --2- 172.21.15.92:0/2960596753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c0af210 0x7f5a7c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 -- 172.21.15.92:0/2960596753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c0af210 msgr2=0x7f5a7c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 --2- 172.21.15.92:0/2960596753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c0af210 0x7f5a7c06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 -- 172.21.15.92:0/2960596753 >> v1:172.21.15.92:6789/0 conn(0x7f5a7c0aedc0 legacy=0x7f5a7c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7affd700 1 -- 172.21.15.92:0/2960596753 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c15db00 con 0x7f5a7c1597c0 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.413+0000 7f5a7a7fc700 1 -- 172.21.15.92:0/2960596753 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947050342 0 0) 0x7f5a7c0ae390 con 0x7f5a7c0aedc0 2023-10-06T03:08:23.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a7affd700 1 --2- 172.21.15.92:0/2960596753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c1597c0 0x7f5a7c159b90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0099b0 tx=0x7f5a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef44e6ff6c46af7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a7a7fc700 1 -- 172.21.15.92:0/2960596753 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c009ea0 con 0x7f5a7c1597c0 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a7a7fc700 1 -- 172.21.15.92:0/2960596753 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c00b860 con 0x7f5a7c1597c0 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a7a7fc700 1 -- 172.21.15.92:0/2960596753 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00bc00 con 0x7f5a7c1597c0 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 -- 172.21.15.92:0/2960596753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c1597c0 msgr2=0x7f5a7c159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 --2- 172.21.15.92:0/2960596753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c1597c0 0x7f5a7c159b90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0099b0 tx=0x7f5a6c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 -- 172.21.15.92:0/2960596753 shutdown_connections 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 --2- 172.21.15.92:0/2960596753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c0af210 0x7f5a7c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 --2- 172.21.15.92:0/2960596753 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c1597c0 0x7f5a7c159b90 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 -- 172.21.15.92:0/2960596753 >> 172.21.15.92:0/2960596753 conn(0x7f5a7c0c9f30 msgr2=0x7f5a7c06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 -- 172.21.15.92:0/2960596753 shutdown_connections 2023-10-06T03:08:23.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.414+0000 7f5a816b7700 1 -- 172.21.15.92:0/2960596753 wait complete. 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 Processor -- start 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 -- start start 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a7c0aedc0 0x7f5a7c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 0x7f5a7c0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c1597c0 0x7f5a7c16d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c6750 con 0x7f5a7c0af210 2023-10-06T03:08:23.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c0c6960 con 0x7f5a7c0aedc0 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a816b7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1548c0 con 0x7f5a7c1597c0 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 0x7f5a7c0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a727fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 0x7f5a7c0c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37966/0 (socket says 172.21.15.92:37966) 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a727fc700 1 -- 172.21.15.92:0/324048742 learned_addr learned my addr 172.21.15.92:0/324048742 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.416+0000 7f5a7b7fe700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c1597c0 0x7f5a7c16d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a7affd700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a7c0aedc0 0x7f5a7c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a727fc700 1 -- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c1597c0 msgr2=0x7f5a7c16d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a727fc700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c1597c0 0x7f5a7c16d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a727fc700 1 -- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a7c0aedc0 msgr2=0x7f5a7c0c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a727fc700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a7c0aedc0 0x7f5a7c0c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a727fc700 1 -- 172.21.15.92:0/324048742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0094f0 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a727fc700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 0x7f5a7c0c6210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5a640011e0 tx=0x7f5a64007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64005a70 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64005bd0 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c154ba0 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640070d0 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.417+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c16dcb0 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.418+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7c16dea0 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.418+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5a64007480 con 0x7f5a7c0af210 2023-10-06T03:08:23.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.419+0000 7f5a737fe700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a5c067320 0x7f5a5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.419+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5a64084330 con 0x7f5a7c0af210 2023-10-06T03:08:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.419+0000 7f5a7affd700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a5c067320 0x7f5a5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.419+0000 7f5a7affd700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a5c067320 0x7f5a5c0697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0099b0 tx=0x7f5a6c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:23.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.423+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a64050e60 con 0x7f5a7c0af210 2023-10-06T03:08:23.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.610+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a7c04c3c0 con 0x7f5a7c0af210 2023-10-06T03:08:24.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.611+0000 7f5a737fe700 1 -- 172.21.15.92:0/324048742 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f5a64017090 con 0x7f5a7c0af210 2023-10-06T03:08:24.723 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:24.723 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":115,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:23.524404+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:08:19.983879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T03:08:21.301577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:24.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a5c067320 msgr2=0x7f5a5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a5c067320 0x7f5a5c0697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0099b0 tx=0x7f5a6c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:24.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 msgr2=0x7f5a7c0c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 0x7f5a7c0c6210 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5a640011e0 tx=0x7f5a64007e10 comp rx=0 tx=0).stop 2023-10-06T03:08:24.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 shutdown_connections 2023-10-06T03:08:24.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5a5c067320 0x7f5a5c0697d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a7c1597c0 0x7f5a7c16d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a7c0aedc0 0x7f5a7c0c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 --2- 172.21.15.92:0/324048742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5a7c0af210 0x7f5a7c0c6210 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 >> 172.21.15.92:0/324048742 conn(0x7f5a7c0c9f30 msgr2=0x7f5a7c150810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:24.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 shutdown_connections 2023-10-06T03:08:24.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:23.613+0000 7f5a816b7700 1 -- 172.21.15.92:0/324048742 wait complete. 2023-10-06T03:08:24.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:08:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.856+0000 7f09dd17f700 1 Processor -- start 2023-10-06T03:08:24.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.856+0000 7f09dd17f700 1 -- start start 2023-10-06T03:08:24.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09dd17f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f09d8069560 0x7f09d80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09dd17f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d8069ee0 con 0x7f09d8159c60 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09dd17f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d815a160 con 0x7f09d8159890 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09dd17f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d80d13a0 con 0x7f09d8069560 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09ce59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09d8159c60 0x7f09d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37542/0 (socket says 172.21.15.92:37542) 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09ce59c700 1 -- 172.21.15.92:0/1454089988 learned_addr learned my addr 172.21.15.92:0/1454089988 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964436805 0 0) 0x7f09d8069ee0 con 0x7f09d8159c60 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c4003570 con 0x7f09d8159c60 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09d759e700 1 --2- 172.21.15.92:0/1454089988 >> v2:172.21.15.136:3300/0 conn(0x7f09d8069560 0x7f09d80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519929592 0 0) 0x7f09c4003570 con 0x7f09d8159c60 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.857+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 >> v2:172.21.15.136:3300/0 conn(0x7f09d8069560 msgr2=0x7f09d80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:24.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 --2- 172.21.15.92:0/1454089988 >> v2:172.21.15.136:3300/0 conn(0x7f09d8069560 0x7f09d80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 >> v1:172.21.15.136:6789/0 conn(0x7f09d8159890 legacy=0x7f09d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d815e140 con 0x7f09d8159c60 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f09c0001470 con 0x7f09d8159c60 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 >> v1:172.21.15.92:6789/0 conn(0x7f09d8159c60 legacy=0x7f09d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c4003a60 0x7f09c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4004810 0x7f09c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09c40051c0 0x7f09c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80d13a0 con 0x7f09c4003a60 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c400c0d0 con 0x7f09c4004810 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c400c290 con 0x7f09c40051c0 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c4003a60 0x7f09c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d759e700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09c40051c0 0x7f09c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09ce59c700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4004810 0x7f09c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 -- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09c40051c0 msgr2=0x7f09c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09c40051c0 0x7f09c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 -- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4004810 msgr2=0x7f09c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4004810 0x7f09c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d759e700 1 -- 172.21.15.92:0/1454089988 reap_dead start 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 -- 172.21.15.92:0/1454089988 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c4003570 con 0x7f09c4003a60 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09ce59c700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4004810 0x7f09c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c4003a60 0x7f09c4003e70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f09c80018b0 tx=0x7f09c8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8de9631f762fdd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8012420 con 0x7f09c4003a60 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c80125a0 con 0x7f09c4003a60 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.858+0000 7f09d659c700 1 -- 172.21.15.92:0/1454089988 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8012960 con 0x7f09c4003a60 2023-10-06T03:08:24.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 -- 172.21.15.92:0/1454089988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c4003a60 msgr2=0x7f09c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c4003a60 0x7f09c4003e70 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f09c80018b0 tx=0x7f09c8007d30 comp rx=0 tx=0).stop 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 -- 172.21.15.92:0/1454089988 shutdown_connections 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 --2- 172.21.15.92:0/1454089988 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09c4003a60 0x7f09c4003e70 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 -- 172.21.15.92:0/1454089988 >> 172.21.15.92:0/1454089988 conn(0x7f09d8068e20 msgr2=0x7f09d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 -- 172.21.15.92:0/1454089988 shutdown_connections 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.859+0000 7f09dd17f700 1 -- 172.21.15.92:0/1454089988 wait complete. 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.860+0000 7f09dd17f700 1 Processor -- start 2023-10-06T03:08:24.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.860+0000 7f09dd17f700 1 -- start start 2023-10-06T03:08:24.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09dd17f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4003a60 0x7f09d816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09dd17f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 0x7f09d816de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09dd17f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09d816e350 0x7f09d80b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09dd17f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d816afe0 con 0x7f09d816a550 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09dd17f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d81570e0 con 0x7f09c4003a60 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09dd17f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d816aa20 con 0x7f09d816e350 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 0x7f09d816de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 0x7f09d816de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37990/0 (socket says 172.21.15.92:37990) 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 -- 172.21.15.92:0/2802117175 learned_addr learned my addr 172.21.15.92:0/2802117175 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09d759e700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09d816e350 0x7f09d80b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4003a60 0x7f09d816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 -- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09d816e350 msgr2=0x7f09d80b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09d816e350 0x7f09d80b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 -- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4003a60 msgr2=0x7f09d816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4003a60 0x7f09d816a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09ce59c700 1 -- 172.21.15.92:0/2802117175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d816ad00 con 0x7f09d816a550 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.861+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4003a60 0x7f09d816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.862+0000 7f09ce59c700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 0x7f09d816de10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f09c000e810 tx=0x7f09c0001ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.862+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0012590 con 0x7f09d816a550 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.862+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c00126f0 con 0x7f09d816a550 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.862+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d80cd160 con 0x7f09d816a550 2023-10-06T03:08:24.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.862+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0012a90 con 0x7f09d816a550 2023-10-06T03:08:24.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.862+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d80b2110 con 0x7f09d816a550 2023-10-06T03:08:24.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.863+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d8069a10 con 0x7f09d816a550 2023-10-06T03:08:24.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.863+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f09c001d460 con 0x7f09d816a550 2023-10-06T03:08:24.870 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.863+0000 7f09cf7fe700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:24.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.864+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:24.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.864+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09c0084d50 con 0x7f09d816a550 2023-10-06T03:08:24.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.864+0000 7f09d6d9d700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09d80d17a0 tx=0x7f09c8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:24.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:24.868+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09c0051880 con 0x7f09d816a550 2023-10-06T03:08:25.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.095+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f09d80cbc80 con 0x7f09d816a550 2023-10-06T03:08:25.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.960+0000 7f09cf7fe700 1 -- 172.21.15.92:0/2802117175 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f09c0054ea0 con 0x7f09d816a550 2023-10-06T03:08:25.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 msgr2=0x7f09b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09d80d17a0 tx=0x7f09c8015000 comp rx=0 tx=0).stop 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 msgr2=0x7f09d816de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 0x7f09d816de10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f09c000e810 tx=0x7f09c0001ca0 comp rx=0 tx=0).stop 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 shutdown_connections 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09d816e350 0x7f09d80b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09c4003a60 0x7f09d816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 --2- 172.21.15.92:0/2802117175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09d816a550 0x7f09d816de10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 >> 172.21.15.92:0/2802117175 conn(0x7f09d8068e20 msgr2=0x7f09d815d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 shutdown_connections 2023-10-06T03:08:25.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:25.963+0000 7f09dd17f700 1 -- 172.21.15.92:0/2802117175 wait complete. 2023-10-06T03:08:25.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:08:26.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.094+0000 7f20b1b44700 1 Processor -- start 2023-10-06T03:08:26.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.094+0000 7f20b1b44700 1 -- start start 2023-10-06T03:08:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20b1b44700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f20ac0b4e70 0x7f20ac0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20b1b44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 0x7f20ac15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20b1b44700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac0cfb00 con 0x7f20ac0b16d0 2023-10-06T03:08:26.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20b1b44700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac0b5c40 con 0x7f20ac0b4e70 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20b1b44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0b5f20 con 0x7f20ac0b1b40 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20abfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f20ac0b16d0 0x7f20ac15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37552/0 (socket says 172.21.15.92:37552) 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20abfff700 1 -- 172.21.15.92:0/1997509149 learned_addr learned my addr 172.21.15.92:0/1997509149 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930396481 0 0) 0x7f20ac0cfb00 con 0x7f20ac0b16d0 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209c003570 con 0x7f20ac0b16d0 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.095+0000 7f20b1343700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 0x7f20ac15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20b0b42700 1 --2- 172.21.15.92:0/1997509149 >> v2:172.21.15.136:3300/0 conn(0x7f20ac0b4e70 0x7f20ac0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977020445 0 0) 0x7f209c003570 con 0x7f20ac0b16d0 2023-10-06T03:08:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 msgr2=0x7f20ac15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 0x7f20ac15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 >> v2:172.21.15.136:3300/0 conn(0x7f20ac0b4e70 msgr2=0x7f20ac0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 --2- 172.21.15.92:0/1997509149 >> v2:172.21.15.136:3300/0 conn(0x7f20ac0b4e70 0x7f20ac0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0b6200 con 0x7f20ac0b16d0 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2098001470 con 0x7f20ac0b16d0 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 >> v1:172.21.15.92:6789/0 conn(0x7f20ac0b16d0 legacy=0x7f20ac15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 0x7f209c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c003fe0 0x7f209c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f209c004d50 0x7f209c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0b5c40 con 0x7f209c003fe0 2023-10-06T03:08:26.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0b5f20 con 0x7f209c004d50 2023-10-06T03:08:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c00ba90 con 0x7f20ac0b1b40 2023-10-06T03:08:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c003fe0 0x7f209c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20b0b42700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 0x7f209c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20b1343700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f209c004d50 0x7f209c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 -- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 msgr2=0x7f209c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:26.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac0b1b40 0x7f209c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 -- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f209c004d50 msgr2=0x7f209c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f209c004d50 0x7f209c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 -- 172.21.15.92:0/1997509149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c003570 con 0x7f209c003fe0 2023-10-06T03:08:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20b1343700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f209c004d50 0x7f209c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20abfff700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c003fe0 0x7f209c004810 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f209800c3c0 tx=0x7f2098001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daa27f8a5a55fa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:26.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800cc30 con 0x7f209c003fe0 2023-10-06T03:08:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800cdb0 con 0x7f209c003fe0 2023-10-06T03:08:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.096+0000 7f20ab7fe700 1 -- 172.21.15.92:0/1997509149 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098005b70 con 0x7f209c003fe0 2023-10-06T03:08:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1b44700 1 -- 172.21.15.92:0/1997509149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c003fe0 msgr2=0x7f209c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1b44700 1 --2- 172.21.15.92:0/1997509149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c003fe0 0x7f209c004810 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f209800c3c0 tx=0x7f2098001e90 comp rx=0 tx=0).stop 2023-10-06T03:08:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1343700 1 -- 172.21.15.92:0/1997509149 reap_dead start 2023-10-06T03:08:26.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1b44700 1 -- 172.21.15.92:0/1997509149 shutdown_connections 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1b44700 1 -- 172.21.15.92:0/1997509149 >> 172.21.15.92:0/1997509149 conn(0x7f20ac0d10c0 msgr2=0x7f20ac0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1b44700 1 -- 172.21.15.92:0/1997509149 shutdown_connections 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.097+0000 7f20b1b44700 1 -- 172.21.15.92:0/1997509149 wait complete. 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.098+0000 7f20b1b44700 1 Processor -- start 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.098+0000 7f20b1b44700 1 -- start start 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1b44700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f209c003fe0 0x7f20ac16e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1b44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 0x7f20ac0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1b44700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac068390 0x7f20ac068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1b44700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac068d40 con 0x7f20ac0cacd0 2023-10-06T03:08:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1b44700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac169f50 con 0x7f20ac068390 2023-10-06T03:08:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1b44700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0cb140 con 0x7f209c003fe0 2023-10-06T03:08:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 0x7f20ac0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 0x7f20ac0cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38002/0 (socket says 172.21.15.92:38002) 2023-10-06T03:08:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 -- 172.21.15.92:0/620391798 learned_addr learned my addr 172.21.15.92:0/620391798 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b0b42700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f209c003fe0 0x7f20ac16e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b1343700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac068390 0x7f20ac068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 -- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f209c003fe0 msgr2=0x7f20ac16e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f209c003fe0 0x7f20ac16e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 -- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac068390 msgr2=0x7f20ac068800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac068390 0x7f20ac068800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20abfff700 1 -- 172.21.15.92:0/620391798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0cc050 con 0x7f20ac0cacd0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.099+0000 7f20b0b42700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f209c003fe0 0x7f20ac16e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.100+0000 7f20abfff700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 0x7f20ac0cbb10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f209c004730 tx=0x7f2098015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.100+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098001bb0 con 0x7f20ac0cacd0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.100+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2098000f50 con 0x7f20ac0cacd0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.100+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac16a190 con 0x7f20ac0cacd0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.100+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098007550 con 0x7f20ac0cacd0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.100+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac16a750 con 0x7f20ac0cacd0 2023-10-06T03:08:26.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.101+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac0b5510 con 0x7f20ac0cacd0 2023-10-06T03:08:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.101+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2098019030 con 0x7f20ac0cacd0 2023-10-06T03:08:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.101+0000 7f20a8ff9700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2090067350 0x7f2090069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.101+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2098085dd0 con 0x7f20ac0cacd0 2023-10-06T03:08:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.101+0000 7f20b0b42700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2090067350 0x7f2090069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.102+0000 7f20b0b42700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2090067350 0x7f2090069800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006f00 tx=0x7f20a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:26.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.106+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2098052900 con 0x7f20ac0cacd0 2023-10-06T03:08:26.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.329+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f20ac068d40 con 0x7f20ac0cacd0 2023-10-06T03:08:26.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.965+0000 7f20a8ff9700 1 -- 172.21.15.92:0/620391798 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20ac068d40 con 0x7f20ac0cacd0 2023-10-06T03:08:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2090067350 msgr2=0x7f2090069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2090067350 0x7f2090069800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f20a0006f00 tx=0x7f20a0008040 comp rx=0 tx=0).stop 2023-10-06T03:08:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 msgr2=0x7f20ac0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 0x7f20ac0cbb10 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f209c004730 tx=0x7f2098015000 comp rx=0 tx=0).stop 2023-10-06T03:08:26.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 shutdown_connections 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2090067350 0x7f2090069800 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac068390 0x7f20ac068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f209c003fe0 0x7f20ac16e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 --2- 172.21.15.92:0/620391798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f20ac0cacd0 0x7f20ac0cbb10 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 >> 172.21.15.92:0/620391798 conn(0x7f20ac0d10c0 msgr2=0x7f20ac0af470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 shutdown_connections 2023-10-06T03:08:26.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:26.967+0000 7f20b1b44700 1 -- 172.21.15.92:0/620391798 wait complete. 2023-10-06T03:08:26.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.096+0000 7f2c4495c700 1 Processor -- start 2023-10-06T03:08:27.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.096+0000 7f2c4495c700 1 -- start start 2023-10-06T03:08:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c4495c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c40159c60 0x7f2c4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c4495c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2c4015db10 0x7f2c40069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c4495c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4015a160 con 0x7f2c40159890 2023-10-06T03:08:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c4495c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4015df50 con 0x7f2c40159c60 2023-10-06T03:08:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c4495c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c400d13a0 con 0x7f2c4015db10 2023-10-06T03:08:27.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c35d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c40159c60 0x7f2c4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c35d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2c40159c60 0x7f2c4015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38006/0 (socket says 172.21.15.92:38006) 2023-10-06T03:08:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c35d9b700 1 -- 172.21.15.92:0/3144020467 learned_addr learned my addr 172.21.15.92:0/3144020467 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c3ed9d700 1 --2- 172.21.15.92:0/3144020467 >> v2:172.21.15.136:3300/0 conn(0x7f2c4015db10 0x7f2c40069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.097+0000 7f2c35d9b700 1 -- 172.21.15.92:0/3144020467 >> v2:172.21.15.136:3300/0 conn(0x7f2c4015db10 msgr2=0x7f2c40069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c35d9b700 1 --2- 172.21.15.92:0/3144020467 >> v2:172.21.15.136:3300/0 conn(0x7f2c4015db10 0x7f2c40069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c35d9b700 1 -- 172.21.15.92:0/3144020467 >> v1:172.21.15.136:6789/0 conn(0x7f2c40159890 legacy=0x7f2c4015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c35d9b700 1 -- 172.21.15.92:0/3144020467 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4015e140 con 0x7f2c40159c60 2023-10-06T03:08:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c35d9b700 1 --2- 172.21.15.92:0/3144020467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c4015d5d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006a890 tx=0x7f2c28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c443ffeb87a0d662 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c3dd9b700 1 -- 172.21.15.92:0/3144020467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800b820 con 0x7f2c40159c60 2023-10-06T03:08:27.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c3dd9b700 1 -- 172.21.15.92:0/3144020467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2800b980 con 0x7f2c40159c60 2023-10-06T03:08:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c3dd9b700 1 -- 172.21.15.92:0/3144020467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800bd20 con 0x7f2c40159c60 2023-10-06T03:08:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 -- 172.21.15.92:0/3144020467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 msgr2=0x7f2c4015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 --2- 172.21.15.92:0/3144020467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c4015d5d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006a890 tx=0x7f2c28009250 comp rx=0 tx=0).stop 2023-10-06T03:08:27.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 -- 172.21.15.92:0/3144020467 shutdown_connections 2023-10-06T03:08:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 --2- 172.21.15.92:0/3144020467 >> v2:172.21.15.136:3300/0 conn(0x7f2c4015db10 0x7f2c40069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 --2- 172.21.15.92:0/3144020467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c4015d5d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 -- 172.21.15.92:0/3144020467 >> 172.21.15.92:0/3144020467 conn(0x7f2c40068e20 msgr2=0x7f2c400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 -- 172.21.15.92:0/3144020467 shutdown_connections 2023-10-06T03:08:27.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.098+0000 7f2c4495c700 1 -- 172.21.15.92:0/3144020467 wait complete. 2023-10-06T03:08:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 Processor -- start 2023-10-06T03:08:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 -- start start 2023-10-06T03:08:27.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c40159890 0x7f2c4016b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c400b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c4015db10 0x7f2c4016d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c400b6480 con 0x7f2c40159c60 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4016b5d0 con 0x7f2c4015db10 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c4495c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c400b61e0 con 0x7f2c40159890 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c35d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c400b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.100+0000 7f2c35d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c400b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38022/0 (socket says 172.21.15.92:38022) 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 -- 172.21.15.92:0/1382239216 learned_addr learned my addr 172.21.15.92:0/1382239216 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c3e59c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c40159890 0x7f2c4016b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c3ed9d700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c4015db10 0x7f2c4016d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 -- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c40159890 msgr2=0x7f2c4016b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c40159890 0x7f2c4016b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 -- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c4015db10 msgr2=0x7f2c4016d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c4015db10 0x7f2c4016d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 -- 172.21.15.92:0/1382239216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c280094f0 con 0x7f2c40159c60 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c35d9b700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c400b5ca0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2c40070e20 tx=0x7f2c28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28011cc0 con 0x7f2c40159c60 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28020070 con 0x7f2c40159c60 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c400b0670 con 0x7f2c40159c60 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2801b6a0 con 0x7f2c40159c60 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.101+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c400b0c30 con 0x7f2c40159c60 2023-10-06T03:08:27.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.102+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4004c3c0 con 0x7f2c40159c60 2023-10-06T03:08:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.102+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2c28022030 con 0x7f2c40159c60 2023-10-06T03:08:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.103+0000 7f2c36ffd700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c20067320 0x7f2c200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.103+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c28086b20 con 0x7f2c40159c60 2023-10-06T03:08:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.103+0000 7f2c3e59c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c20067320 0x7f2c200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.103+0000 7f2c3e59c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c20067320 0x7f2c200697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c30003a30 tx=0x7f2c30007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.107+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c28053760 con 0x7f2c40159c60 2023-10-06T03:08:27.241 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:08:27.240+0000 7f6ea15b7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:08:27.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.334+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c4004c3c0 con 0x7f2c40159c60 2023-10-06T03:08:27.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.334+0000 7f2c36ffd700 1 -- 172.21.15.92:0/1382239216 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f2c28056d80 con 0x7f2c40159c60 2023-10-06T03:08:27.336 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:27.336 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3257155196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3257155196},{"type":"v1","addr":"172.21.15.136:6835","nonce":3257155196}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7218,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3995167741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3995167741},{"type":"v1","addr":"172.21.15.92:6837","nonce":3995167741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:08:22.334042+0000","modified":"2023-10-06T03:08:26.961080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7221},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7221":{"gid":7221,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/1746152484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1746152484},{"type":"v1","addr":"172.21.15.92:6836","nonce":1746152484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T03:08:27.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c20067320 msgr2=0x7f2c200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c20067320 0x7f2c200697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c30003a30 tx=0x7f2c30007ea0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 msgr2=0x7f2c400b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c400b5ca0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2c40070e20 tx=0x7f2c28016000 comp rx=0 tx=0).stop 2023-10-06T03:08:27.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 shutdown_connections 2023-10-06T03:08:27.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c20067320 0x7f2c200697d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c4015db10 0x7f2c4016d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c40159890 0x7f2c4016b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 --2- 172.21.15.92:0/1382239216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c40159c60 0x7f2c400b5ca0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 >> 172.21.15.92:0/1382239216 conn(0x7f2c40068e20 msgr2=0x7f2c40153290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:27.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 shutdown_connections 2023-10-06T03:08:27.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.336+0000 7f2c4495c700 1 -- 172.21.15.92:0/1382239216 wait complete. 2023-10-06T03:08:27.340 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 77 2023-10-06T03:08:27.350 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.463+0000 7f7abead8700 1 Processor -- start 2023-10-06T03:08:27.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.463+0000 7f7abead8700 1 -- start start 2023-10-06T03:08:27.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.464+0000 7f7abead8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7ab8069560 0x7f7ab80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.464+0000 7f7abead8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab8069ee0 con 0x7f7ab8159890 2023-10-06T03:08:27.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.464+0000 7f7abead8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab815a160 con 0x7f7ab8159c60 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.464+0000 7f7abead8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80d13a0 con 0x7f7ab8069560 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.464+0000 7f7abc874700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7ab8159890 0x7f7ab815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37558/0 (socket says 172.21.15.92:37558) 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.464+0000 7f7abc874700 1 -- 172.21.15.92:0/3756632405 learned_addr learned my addr 172.21.15.92:0/3756632405 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166839148 0 0) 0x7f7ab8069ee0 con 0x7f7ab8159890 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abd075700 1 --2- 172.21.15.92:0/3756632405 >> v2:172.21.15.136:3300/0 conn(0x7f7ab8069560 0x7f7ab80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa0003570 con 0x7f7ab8159890 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1446352003 0 0) 0x7f7aa0003570 con 0x7f7ab8159890 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 >> v2:172.21.15.136:3300/0 conn(0x7f7ab8069560 msgr2=0x7f7ab80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 --2- 172.21.15.92:0/3756632405 >> v2:172.21.15.136:3300/0 conn(0x7f7ab8069560 0x7f7ab80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 >> v1:172.21.15.136:6789/0 conn(0x7f7ab8159c60 legacy=0x7f7ab815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab815e140 con 0x7f7ab8159890 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7ab40017f0 con 0x7f7ab8159890 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 >> v1:172.21.15.92:6789/0 conn(0x7f7ab8159890 legacy=0x7f7ab815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa0003a60 0x7f7aa0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7aa0004810 0x7f7aa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa00051c0 0x7f7aa000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80d13a0 con 0x7f7aa0003a60 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa000c0d0 con 0x7f7aa00051c0 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa000c290 con 0x7f7aa0004810 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa0003a60 0x7f7aa0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaffff700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7aa0004810 0x7f7aa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abd075700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa00051c0 0x7f7aa000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 -- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7aa0004810 msgr2=0x7f7aa0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7aa0004810 0x7f7aa0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 -- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa00051c0 msgr2=0x7f7aa000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa00051c0 0x7f7aa000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 -- 172.21.15.92:0/3756632405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa0003570 con 0x7f7aa0003a60 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7aaffff700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7aa0004810 0x7f7aa0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abd075700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa00051c0 0x7f7aa000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abd075700 1 -- 172.21.15.92:0/3756632405 reap_dead start 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.465+0000 7f7abc874700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa0003a60 0x7f7aa0003e70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400a930 tx=0x7f7ab40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcf188cf76e962b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40070d0 con 0x7f7aa0003a60 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab4007230 con 0x7f7aa0003a60 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7aaf7fe700 1 -- 172.21.15.92:0/3756632405 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40075d0 con 0x7f7aa0003a60 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 -- 172.21.15.92:0/3756632405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa0003a60 msgr2=0x7f7aa0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa0003a60 0x7f7aa0003e70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400a930 tx=0x7f7ab40021d0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 -- 172.21.15.92:0/3756632405 shutdown_connections 2023-10-06T03:08:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 --2- 172.21.15.92:0/3756632405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7aa0003a60 0x7f7aa0003e70 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 -- 172.21.15.92:0/3756632405 >> 172.21.15.92:0/3756632405 conn(0x7f7ab8068e20 msgr2=0x7f7ab80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 -- 172.21.15.92:0/3756632405 shutdown_connections 2023-10-06T03:08:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.466+0000 7f7abead8700 1 -- 172.21.15.92:0/3756632405 wait complete. 2023-10-06T03:08:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.467+0000 7f7abead8700 1 Processor -- start 2023-10-06T03:08:27.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.467+0000 7f7abead8700 1 -- start start 2023-10-06T03:08:27.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abead8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0003a60 0x7f7ab8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abead8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ab80b36b0 0x7f7ab80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abead8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 0x7f7ab80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abead8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80d0fa0 con 0x7f7ab80d0b30 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abead8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80b5490 con 0x7f7aa0003a60 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abead8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80b5770 con 0x7f7ab80b36b0 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 0x7f7ab80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 0x7f7ab80c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38044/0 (socket says 172.21.15.92:38044) 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 -- 172.21.15.92:0/889590211 learned_addr learned my addr 172.21.15.92:0/889590211 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7aaffff700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ab80b36b0 0x7f7ab80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abc874700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0003a60 0x7f7ab8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 -- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ab80b36b0 msgr2=0x7f7ab80d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ab80b36b0 0x7f7ab80d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 -- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0003a60 msgr2=0x7f7ab8147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0003a60 0x7f7ab8147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.468+0000 7f7abd075700 1 -- 172.21.15.92:0/889590211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80b5a50 con 0x7f7ab80d0b30 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.469+0000 7f7abd075700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 0x7f7ab80c8870 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80d03b0 tx=0x7f7ab0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.469+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0015670 con 0x7f7ab80d0b30 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.469+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab00157f0 con 0x7f7ab80d0b30 2023-10-06T03:08:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.469+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab80b5d30 con 0x7f7ab80d0b30 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.469+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0015bb0 con 0x7f7ab80d0b30 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.469+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab8147750 con 0x7f7ab80d0b30 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.470+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab80b6250 con 0x7f7ab80d0b30 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.470+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7ab001f410 con 0x7f7ab80d0b30 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.470+0000 7f7aacff9700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7aa4067320 0x7f7aa40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.471+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7ab0084bb0 con 0x7f7ab80d0b30 2023-10-06T03:08:27.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.471+0000 7f7abc874700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7aa4067320 0x7f7aa40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.471+0000 7f7abc874700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7aa4067320 0x7f7aa40697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400a3f0 tx=0x7f7ab4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.475+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ab80b6250 con 0x7f7ab80d0b30 2023-10-06T03:08:27.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.661+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ab80b6250 con 0x7f7ab80d0b30 2023-10-06T03:08:27.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.662+0000 7f7aacff9700 1 -- 172.21.15.92:0/889590211 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f7ab80b6250 con 0x7f7ab80d0b30 2023-10-06T03:08:27.664 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:27.664 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":115,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:23.524404+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:08:19.983879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T03:08:21.301577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:27.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7aa4067320 msgr2=0x7f7aa40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7aa4067320 0x7f7aa40697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400a3f0 tx=0x7f7ab4016000 comp rx=0 tx=0).stop 2023-10-06T03:08:27.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 msgr2=0x7f7ab80c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 0x7f7ab80c8870 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80d03b0 tx=0x7f7ab0008040 comp rx=0 tx=0).stop 2023-10-06T03:08:27.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 shutdown_connections 2023-10-06T03:08:27.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7aa4067320 0x7f7aa40697d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7ab80d0b30 0x7f7ab80c8870 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7aa0003a60 0x7f7ab8147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 --2- 172.21.15.92:0/889590211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7ab80b36b0 0x7f7ab80d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 >> 172.21.15.92:0/889590211 conn(0x7f7ab8068e20 msgr2=0x7f7ab80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:27.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 shutdown_connections 2023-10-06T03:08:27.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.664+0000 7f7abead8700 1 -- 172.21.15.92:0/889590211 wait complete. 2023-10-06T03:08:27.678 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:08:27.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:08:27.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.791+0000 7ffaa8d3f700 1 Processor -- start 2023-10-06T03:08:27.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.791+0000 7ffaa8d3f700 1 -- start start 2023-10-06T03:08:27.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa8d3f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa4143050 0x7ffaa4159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa8d3f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffaa406b9e0 0x7ffaa406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa8d3f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffaa40b3810 0x7ffaa40b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa8d3f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa406c390 con 0x7ffaa406b9e0 2023-10-06T03:08:27.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa8d3f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa4143420 con 0x7ffaa40b3810 2023-10-06T03:08:27.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa8d3f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa406a1e0 con 0x7ffaa4143050 2023-10-06T03:08:27.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffaa406b9e0 0x7ffaa406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffaa406b9e0 0x7ffaa406be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38054/0 (socket says 172.21.15.92:38054) 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.792+0000 7ffaa2ffd700 1 -- 172.21.15.92:0/3826846439 learned_addr learned my addr 172.21.15.92:0/3826846439 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa37fe700 1 --2- 172.21.15.92:0/3826846439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa4143050 0x7ffaa4159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa3fff700 1 --2- 172.21.15.92:0/3826846439 >> v2:172.21.15.136:3300/0 conn(0x7ffaa40b3810 0x7ffaa40b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa2ffd700 1 -- 172.21.15.92:0/3826846439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa4143050 msgr2=0x7ffaa4159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa2ffd700 1 --2- 172.21.15.92:0/3826846439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa4143050 0x7ffaa4159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa2ffd700 1 -- 172.21.15.92:0/3826846439 >> v2:172.21.15.136:3300/0 conn(0x7ffaa40b3810 msgr2=0x7ffaa40b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa2ffd700 1 --2- 172.21.15.92:0/3826846439 >> v2:172.21.15.136:3300/0 conn(0x7ffaa40b3810 0x7ffaa40b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa2ffd700 1 -- 172.21.15.92:0/3826846439 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40695d0 con 0x7ffaa406b9e0 2023-10-06T03:08:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa2ffd700 1 --2- 172.21.15.92:0/3826846439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa406b9e0 0x7ffaa406be50 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ffaa406c760 tx=0x7ffa98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d171aaf9b71031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa27fc700 1 -- 172.21.15.92:0/3826846439 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800b860 con 0x7ffaa406b9e0 2023-10-06T03:08:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa27fc700 1 -- 172.21.15.92:0/3826846439 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9800b9c0 con 0x7ffaa406b9e0 2023-10-06T03:08:27.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa27fc700 1 -- 172.21.15.92:0/3826846439 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800bd80 con 0x7ffaa406b9e0 2023-10-06T03:08:27.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/3826846439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa406b9e0 msgr2=0x7ffaa406be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.793+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/3826846439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa406b9e0 0x7ffaa406be50 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ffaa406c760 tx=0x7ffa98009250 comp rx=0 tx=0).stop 2023-10-06T03:08:27.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/3826846439 shutdown_connections 2023-10-06T03:08:27.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/3826846439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa4143050 0x7ffaa4159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/3826846439 >> v2:172.21.15.136:3300/0 conn(0x7ffaa40b3810 0x7ffaa40b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/3826846439 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa406b9e0 0x7ffaa406be50 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/3826846439 >> 172.21.15.92:0/3826846439 conn(0x7ffaa40ce430 msgr2=0x7ffaa40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:27.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/3826846439 shutdown_connections 2023-10-06T03:08:27.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.794+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/3826846439 wait complete. 2023-10-06T03:08:27.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.795+0000 7ffaa8d3f700 1 Processor -- start 2023-10-06T03:08:27.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.795+0000 7ffaa8d3f700 1 -- start start 2023-10-06T03:08:27.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.795+0000 7ffaa8d3f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa406b9e0 0x7ffaa415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa40b3810 0x7ffaa40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 0x7ffaa415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa415d5f0 con 0x7ffaa4143050 2023-10-06T03:08:27.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa415e430 con 0x7ffaa40b3810 2023-10-06T03:08:27.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa413ecd0 con 0x7ffaa406b9e0 2023-10-06T03:08:27.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 0x7ffaa415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 0x7ffaa415d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38062/0 (socket says 172.21.15.92:38062) 2023-10-06T03:08:27.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 -- 172.21.15.92:0/2715910979 learned_addr learned my addr 172.21.15.92:0/2715910979 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:27.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa37fe700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa406b9e0 0x7ffaa415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa2ffd700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa40b3810 0x7ffaa40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 -- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa406b9e0 msgr2=0x7ffaa415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa406b9e0 0x7ffaa415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 -- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa40b3810 msgr2=0x7ffaa40c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:27.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa40b3810 0x7ffaa40c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:27.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 -- 172.21.15.92:0/2715910979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa980094f0 con 0x7ffaa4143050 2023-10-06T03:08:27.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa3fff700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 0x7ffaa415d0b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00d890 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0125f0 con 0x7ffaa4143050 2023-10-06T03:08:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c012770 con 0x7ffaa4143050 2023-10-06T03:08:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa415d820 con 0x7ffaa4143050 2023-10-06T03:08:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c012b30 con 0x7ffaa4143050 2023-10-06T03:08:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.796+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa415dde0 con 0x7ffaa4143050 2023-10-06T03:08:27.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.797+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa415e5a0 con 0x7ffaa4143050 2023-10-06T03:08:27.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.798+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffa9c01e460 con 0x7ffaa4143050 2023-10-06T03:08:27.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.798+0000 7ffa8b7fe700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa90067320 0x7ffa900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:27.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.798+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffa9c085290 con 0x7ffaa4143050 2023-10-06T03:08:27.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.798+0000 7ffaa37fe700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa90067320 0x7ffa900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:27.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.798+0000 7ffaa37fe700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa90067320 0x7ffa900697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffa940075e0 tx=0x7ffa94009830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:27.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.802+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa9c051dc0 con 0x7ffaa4143050 2023-10-06T03:08:27.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.983+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffaa404c3c0 con 0x7ffa90067320 2023-10-06T03:08:27.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.991+0000 7ffa8b7fe700 1 -- 172.21.15.92:0/2715910979 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312100 (secure 0 0 0) 0x7ffaa404c3c0 con 0x7ffa90067320 2023-10-06T03:08:27.999 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:28.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.994+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa90067320 msgr2=0x7ffa900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.994+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa90067320 0x7ffa900697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffa940075e0 tx=0x7ffa94009830 comp rx=0 tx=0).stop 2023-10-06T03:08:28.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.994+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 msgr2=0x7ffaa415d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.994+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 0x7ffaa415d0b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00d890 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 shutdown_connections 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffa90067320 0x7ffa900697d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffaa4143050 0x7ffaa415d0b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffaa406b9e0 0x7ffaa415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 --2- 172.21.15.92:0/2715910979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaa40b3810 0x7ffaa40c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 >> 172.21.15.92:0/2715910979 conn(0x7ffaa40ce430 msgr2=0x7ffaa415b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 shutdown_connections 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:27.995+0000 7ffaa8d3f700 1 -- 172.21.15.92:0/2715910979 wait complete. 2023-10-06T03:08:28.002 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:08:28.010 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-10-06T03:08:27.264789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2419372,"kb_used_data":12396,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752555348,"statfs":{"total":773094113280,"available":770616676352,"internally_reserved":0,"allocated":12693504,"data_stored":6421463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001068"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535246+0000","last_change":"2023-10-06T03:08:22.532629+0000","last_active":"2023-10-06T03:08:22.535246+0000","last_peered":"2023-10-06T03:08:22.535246+0000","last_clean":"2023-10-06T03:08:22.535246+0000","last_became_active":"2023-10-06T03:08:22.532391+0000","last_became_peered":"2023-10-06T03:08:22.532391+0000","last_unstale":"2023-10-06T03:08:22.535246+0000","last_undegraded":"2023-10-06T03:08:22.535246+0000","last_fullsized":"2023-10-06T03:08:22.535246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:15.303883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542314+0000","last_change":"2023-10-06T03:08:22.537320+0000","last_active":"2023-10-06T03:08:22.542314+0000","last_peered":"2023-10-06T03:08:22.542314+0000","last_clean":"2023-10-06T03:08:22.542314+0000","last_became_active":"2023-10-06T03:08:22.537024+0000","last_became_peered":"2023-10-06T03:08:22.537024+0000","last_unstale":"2023-10-06T03:08:22.542314+0000","last_undegraded":"2023-10-06T03:08:22.542314+0000","last_fullsized":"2023-10-06T03:08:22.542314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:26.910097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548523+0000","last_change":"2023-10-06T03:08:22.548523+0000","last_active":"2023-10-06T03:08:22.548523+0000","last_peered":"2023-10-06T03:08:22.548523+0000","last_clean":"2023-10-06T03:08:22.548523+0000","last_became_active":"2023-10-06T03:08:22.548228+0000","last_became_peered":"2023-10-06T03:08:22.548228+0000","last_unstale":"2023-10-06T03:08:22.548523+0000","last_undegraded":"2023-10-06T03:08:22.548523+0000","last_fullsized":"2023-10-06T03:08:22.548523+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:47.321425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.549224+0000","last_change":"2023-10-06T03:08:22.541832+0000","last_active":"2023-10-06T03:08:22.549224+0000","last_peered":"2023-10-06T03:08:22.549224+0000","last_clean":"2023-10-06T03:08:22.549224+0000","last_became_active":"2023-10-06T03:08:22.541518+0000","last_became_peered":"2023-10-06T03:08:22.541518+0000","last_unstale":"2023-10-06T03:08:22.549224+0000","last_undegraded":"2023-10-06T03:08:22.549224+0000","last_fullsized":"2023-10-06T03:08:22.549224+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:30.710396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.537092+0000","last_change":"2023-10-06T03:08:22.537092+0000","last_active":"2023-10-06T03:08:22.537092+0000","last_peered":"2023-10-06T03:08:22.537092+0000","last_clean":"2023-10-06T03:08:22.537092+0000","last_became_active":"2023-10-06T03:08:22.536841+0000","last_became_peered":"2023-10-06T03:08:22.536841+0000","last_unstale":"2023-10-06T03:08:22.537092+0000","last_undegraded":"2023-10-06T03:08:22.537092+0000","last_fullsized":"2023-10-06T03:08:22.537092+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:44.926723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548085+0000","last_change":"2023-10-06T03:08:22.548085+0000","last_active":"2023-10-06T03:08:22.548085+0000","last_peered":"2023-10-06T03:08:22.548085+0000","last_clean":"2023-10-06T03:08:22.548085+0000","last_became_active":"2023-10-06T03:08:22.547831+0000","last_became_peered":"2023-10-06T03:08:22.547831+0000","last_unstale":"2023-10-06T03:08:22.548085+0000","last_undegraded":"2023-10-06T03:08:22.548085+0000","last_fullsized":"2023-10-06T03:08:22.548085+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:20.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.550433+0000","last_change":"2023-10-06T03:08:22.550433+0000","last_active":"2023-10-06T03:08:22.550433+0000","last_peered":"2023-10-06T03:08:22.550433+0000","last_clean":"2023-10-06T03:08:22.550433+0000","last_became_active":"2023-10-06T03:08:22.550166+0000","last_became_peered":"2023-10-06T03:08:22.550166+0000","last_unstale":"2023-10-06T03:08:22.550433+0000","last_undegraded":"2023-10-06T03:08:22.550433+0000","last_fullsized":"2023-10-06T03:08:22.550433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:52.894864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547679+0000","last_change":"2023-10-06T03:08:22.541762+0000","last_active":"2023-10-06T03:08:22.547679+0000","last_peered":"2023-10-06T03:08:22.547679+0000","last_clean":"2023-10-06T03:08:22.547679+0000","last_became_active":"2023-10-06T03:08:22.541553+0000","last_became_peered":"2023-10-06T03:08:22.541553+0000","last_unstale":"2023-10-06T03:08:22.547679+0000","last_undegraded":"2023-10-06T03:08:22.547679+0000","last_fullsized":"2023-10-06T03:08:22.547679+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:03.294297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540961+0000","last_change":"2023-10-06T03:08:22.540961+0000","last_active":"2023-10-06T03:08:22.540961+0000","last_peered":"2023-10-06T03:08:22.540961+0000","last_clean":"2023-10-06T03:08:22.540961+0000","last_became_active":"2023-10-06T03:08:22.540610+0000","last_became_peered":"2023-10-06T03:08:22.540610+0000","last_unstale":"2023-10-06T03:08:22.540961+0000","last_undegraded":"2023-10-06T03:08:22.540961+0000","last_fullsized":"2023-10-06T03:08:22.540961+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:14.063036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535698+0000","last_change":"2023-10-06T03:08:22.535698+0000","last_active":"2023-10-06T03:08:22.535698+0000","last_peered":"2023-10-06T03:08:22.535698+0000","last_clean":"2023-10-06T03:08:22.535698+0000","last_became_active":"2023-10-06T03:08:22.535496+0000","last_became_peered":"2023-10-06T03:08:22.535496+0000","last_unstale":"2023-10-06T03:08:22.535698+0000","last_undegraded":"2023-10-06T03:08:22.535698+0000","last_fullsized":"2023-10-06T03:08:22.535698+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:04.787603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.554634+0000","last_change":"2023-10-06T03:08:22.549828+0000","last_active":"2023-10-06T03:08:22.554634+0000","last_peered":"2023-10-06T03:08:22.554634+0000","last_clean":"2023-10-06T03:08:22.554634+0000","last_became_active":"2023-10-06T03:08:22.549563+0000","last_became_peered":"2023-10-06T03:08:22.549563+0000","last_unstale":"2023-10-06T03:08:22.554634+0000","last_undegraded":"2023-10-06T03:08:22.554634+0000","last_fullsized":"2023-10-06T03:08:22.554634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:14.063036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543832+0000","last_change":"2023-10-06T03:08:22.537086+0000","last_active":"2023-10-06T03:08:22.543832+0000","last_peered":"2023-10-06T03:08:22.543832+0000","last_clean":"2023-10-06T03:08:22.543832+0000","last_became_active":"2023-10-06T03:08:22.536841+0000","last_became_peered":"2023-10-06T03:08:22.536841+0000","last_unstale":"2023-10-06T03:08:22.543832+0000","last_undegraded":"2023-10-06T03:08:22.543832+0000","last_fullsized":"2023-10-06T03:08:22.543832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:12:22.169025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541146+0000","last_change":"2023-10-06T03:08:22.541146+0000","last_active":"2023-10-06T03:08:22.541146+0000","last_peered":"2023-10-06T03:08:22.541146+0000","last_clean":"2023-10-06T03:08:22.541146+0000","last_became_active":"2023-10-06T03:08:22.540902+0000","last_became_peered":"2023-10-06T03:08:22.540902+0000","last_unstale":"2023-10-06T03:08:22.541146+0000","last_undegraded":"2023-10-06T03:08:22.541146+0000","last_fullsized":"2023-10-06T03:08:22.541146+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:12.931647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543693+0000","last_change":"2023-10-06T03:08:22.539488+0000","last_active":"2023-10-06T03:08:22.543693+0000","last_peered":"2023-10-06T03:08:22.543693+0000","last_clean":"2023-10-06T03:08:22.543693+0000","last_became_active":"2023-10-06T03:08:22.539161+0000","last_became_peered":"2023-10-06T03:08:22.539161+0000","last_unstale":"2023-10-06T03:08:22.543693+0000","last_undegraded":"2023-10-06T03:08:22.543693+0000","last_fullsized":"2023-10-06T03:08:22.543693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:58.391302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543808+0000","last_change":"2023-10-06T03:08:22.539481+0000","last_active":"2023-10-06T03:08:22.543808+0000","last_peered":"2023-10-06T03:08:22.543808+0000","last_clean":"2023-10-06T03:08:22.543808+0000","last_became_active":"2023-10-06T03:08:22.539203+0000","last_became_peered":"2023-10-06T03:08:22.539203+0000","last_unstale":"2023-10-06T03:08:22.543808+0000","last_undegraded":"2023-10-06T03:08:22.543808+0000","last_fullsized":"2023-10-06T03:08:22.543808+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:10.330303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540787+0000","last_change":"2023-10-06T03:08:22.540787+0000","last_active":"2023-10-06T03:08:22.540787+0000","last_peered":"2023-10-06T03:08:22.540787+0000","last_clean":"2023-10-06T03:08:22.540787+0000","last_became_active":"2023-10-06T03:08:22.540501+0000","last_became_peered":"2023-10-06T03:08:22.540501+0000","last_unstale":"2023-10-06T03:08:22.540787+0000","last_undegraded":"2023-10-06T03:08:22.540787+0000","last_fullsized":"2023-10-06T03:08:22.540787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:30.327476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.536703+0000","last_change":"2023-10-06T03:08:22.536703+0000","last_active":"2023-10-06T03:08:22.536703+0000","last_peered":"2023-10-06T03:08:22.536703+0000","last_clean":"2023-10-06T03:08:22.536703+0000","last_became_active":"2023-10-06T03:08:22.536350+0000","last_became_peered":"2023-10-06T03:08:22.536350+0000","last_unstale":"2023-10-06T03:08:22.536703+0000","last_undegraded":"2023-10-06T03:08:22.536703+0000","last_fullsized":"2023-10-06T03:08:22.536703+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:30.710396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.537015+0000","last_change":"2023-10-06T03:08:22.537015+0000","last_active":"2023-10-06T03:08:22.537015+0000","last_peered":"2023-10-06T03:08:22.537015+0000","last_clean":"2023-10-06T03:08:22.537015+0000","last_became_active":"2023-10-06T03:08:22.536775+0000","last_became_peered":"2023-10-06T03:08:22.536775+0000","last_unstale":"2023-10-06T03:08:22.537015+0000","last_undegraded":"2023-10-06T03:08:22.537015+0000","last_fullsized":"2023-10-06T03:08:22.537015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:42.103845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548506+0000","last_change":"2023-10-06T03:08:22.548506+0000","last_active":"2023-10-06T03:08:22.548506+0000","last_peered":"2023-10-06T03:08:22.548506+0000","last_clean":"2023-10-06T03:08:22.548506+0000","last_became_active":"2023-10-06T03:08:22.548235+0000","last_became_peered":"2023-10-06T03:08:22.548235+0000","last_unstale":"2023-10-06T03:08:22.548506+0000","last_undegraded":"2023-10-06T03:08:22.548506+0000","last_fullsized":"2023-10-06T03:08:22.548506+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:59.944171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.544866+0000","last_change":"2023-10-06T03:08:22.544866+0000","last_active":"2023-10-06T03:08:22.544866+0000","last_peered":"2023-10-06T03:08:22.544866+0000","last_clean":"2023-10-06T03:08:22.544866+0000","last_became_active":"2023-10-06T03:08:22.544621+0000","last_became_peered":"2023-10-06T03:08:22.544621+0000","last_unstale":"2023-10-06T03:08:22.544866+0000","last_undegraded":"2023-10-06T03:08:22.544866+0000","last_fullsized":"2023-10-06T03:08:22.544866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:21.390611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.552192+0000","last_change":"2023-10-06T03:08:22.546634+0000","last_active":"2023-10-06T03:08:22.552192+0000","last_peered":"2023-10-06T03:08:22.552192+0000","last_clean":"2023-10-06T03:08:22.552192+0000","last_became_active":"2023-10-06T03:08:22.546303+0000","last_became_peered":"2023-10-06T03:08:22.546303+0000","last_unstale":"2023-10-06T03:08:22.552192+0000","last_undegraded":"2023-10-06T03:08:22.552192+0000","last_fullsized":"2023-10-06T03:08:22.552192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.699820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543711+0000","last_change":"2023-10-06T03:08:22.539656+0000","last_active":"2023-10-06T03:08:22.543711+0000","last_peered":"2023-10-06T03:08:22.543711+0000","last_clean":"2023-10-06T03:08:22.543711+0000","last_became_active":"2023-10-06T03:08:22.539418+0000","last_became_peered":"2023-10-06T03:08:22.539418+0000","last_unstale":"2023-10-06T03:08:22.543711+0000","last_undegraded":"2023-10-06T03:08:22.543711+0000","last_fullsized":"2023-10-06T03:08:22.543711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:15.295825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.552183+0000","last_change":"2023-10-06T03:08:22.546420+0000","last_active":"2023-10-06T03:08:22.552183+0000","last_peered":"2023-10-06T03:08:22.552183+0000","last_clean":"2023-10-06T03:08:22.552183+0000","last_became_active":"2023-10-06T03:08:22.546126+0000","last_became_peered":"2023-10-06T03:08:22.546126+0000","last_unstale":"2023-10-06T03:08:22.552183+0000","last_undegraded":"2023-10-06T03:08:22.552183+0000","last_fullsized":"2023-10-06T03:08:22.552183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:07.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540952+0000","last_change":"2023-10-06T03:08:22.540952+0000","last_active":"2023-10-06T03:08:22.540952+0000","last_peered":"2023-10-06T03:08:22.540952+0000","last_clean":"2023-10-06T03:08:22.540952+0000","last_became_active":"2023-10-06T03:08:22.540599+0000","last_became_peered":"2023-10-06T03:08:22.540599+0000","last_unstale":"2023-10-06T03:08:22.540952+0000","last_undegraded":"2023-10-06T03:08:22.540952+0000","last_fullsized":"2023-10-06T03:08:22.540952+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:30.556468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.545041+0000","last_change":"2023-10-06T03:08:22.545041+0000","last_active":"2023-10-06T03:08:22.545041+0000","last_peered":"2023-10-06T03:08:22.545041+0000","last_clean":"2023-10-06T03:08:22.545041+0000","last_became_active":"2023-10-06T03:08:22.544714+0000","last_became_peered":"2023-10-06T03:08:22.544714+0000","last_unstale":"2023-10-06T03:08:22.545041+0000","last_undegraded":"2023-10-06T03:08:22.545041+0000","last_fullsized":"2023-10-06T03:08:22.545041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:26.910097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.545188+0000","last_change":"2023-10-06T03:08:22.545188+0000","last_active":"2023-10-06T03:08:22.545188+0000","last_peered":"2023-10-06T03:08:22.545188+0000","last_clean":"2023-10-06T03:08:22.545188+0000","last_became_active":"2023-10-06T03:08:22.544899+0000","last_became_peered":"2023-10-06T03:08:22.544899+0000","last_unstale":"2023-10-06T03:08:22.545188+0000","last_undegraded":"2023-10-06T03:08:22.545188+0000","last_fullsized":"2023-10-06T03:08:22.545188+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:01.225765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.551043+0000","last_change":"2023-10-06T03:08:22.551043+0000","last_active":"2023-10-06T03:08:22.551043+0000","last_peered":"2023-10-06T03:08:22.551043+0000","last_clean":"2023-10-06T03:08:22.551043+0000","last_became_active":"2023-10-06T03:08:22.550788+0000","last_became_peered":"2023-10-06T03:08:22.550788+0000","last_unstale":"2023-10-06T03:08:22.551043+0000","last_undegraded":"2023-10-06T03:08:22.551043+0000","last_fullsized":"2023-10-06T03:08:22.551043+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:28.345775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533655+0000","last_change":"2023-10-06T03:08:22.533655+0000","last_active":"2023-10-06T03:08:22.533655+0000","last_peered":"2023-10-06T03:08:22.533655+0000","last_clean":"2023-10-06T03:08:22.533655+0000","last_became_active":"2023-10-06T03:08:22.533423+0000","last_became_peered":"2023-10-06T03:08:22.533423+0000","last_unstale":"2023-10-06T03:08:22.533655+0000","last_undegraded":"2023-10-06T03:08:22.533655+0000","last_fullsized":"2023-10-06T03:08:22.533655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:13:48.467391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543614+0000","last_change":"2023-10-06T03:08:22.543614+0000","last_active":"2023-10-06T03:08:22.543614+0000","last_peered":"2023-10-06T03:08:22.543614+0000","last_clean":"2023-10-06T03:08:22.543614+0000","last_became_active":"2023-10-06T03:08:22.543342+0000","last_became_peered":"2023-10-06T03:08:22.543342+0000","last_unstale":"2023-10-06T03:08:22.543614+0000","last_undegraded":"2023-10-06T03:08:22.543614+0000","last_fullsized":"2023-10-06T03:08:22.543614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:30.556468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547334+0000","last_change":"2023-10-06T03:08:22.547334+0000","last_active":"2023-10-06T03:08:22.547334+0000","last_peered":"2023-10-06T03:08:22.547334+0000","last_clean":"2023-10-06T03:08:22.547334+0000","last_became_active":"2023-10-06T03:08:22.547010+0000","last_became_peered":"2023-10-06T03:08:22.547010+0000","last_unstale":"2023-10-06T03:08:22.547334+0000","last_undegraded":"2023-10-06T03:08:22.547334+0000","last_fullsized":"2023-10-06T03:08:22.547334+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.699820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547743+0000","last_change":"2023-10-06T03:08:22.543425+0000","last_active":"2023-10-06T03:08:22.547743+0000","last_peered":"2023-10-06T03:08:22.547743+0000","last_clean":"2023-10-06T03:08:22.547743+0000","last_became_active":"2023-10-06T03:08:22.543125+0000","last_became_peered":"2023-10-06T03:08:22.543125+0000","last_unstale":"2023-10-06T03:08:22.547743+0000","last_undegraded":"2023-10-06T03:08:22.547743+0000","last_fullsized":"2023-10-06T03:08:22.547743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:50.584290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543009+0000","last_change":"2023-10-06T03:08:22.543009+0000","last_active":"2023-10-06T03:08:22.543009+0000","last_peered":"2023-10-06T03:08:22.543009+0000","last_clean":"2023-10-06T03:08:22.543009+0000","last_became_active":"2023-10-06T03:08:22.542752+0000","last_became_peered":"2023-10-06T03:08:22.542752+0000","last_unstale":"2023-10-06T03:08:22.543009+0000","last_undegraded":"2023-10-06T03:08:22.543009+0000","last_fullsized":"2023-10-06T03:08:22.543009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:06.190251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548048+0000","last_change":"2023-10-06T03:08:22.545113+0000","last_active":"2023-10-06T03:08:22.548048+0000","last_peered":"2023-10-06T03:08:22.548048+0000","last_clean":"2023-10-06T03:08:22.548048+0000","last_became_active":"2023-10-06T03:08:22.544871+0000","last_became_peered":"2023-10-06T03:08:22.544871+0000","last_unstale":"2023-10-06T03:08:22.548048+0000","last_undegraded":"2023-10-06T03:08:22.548048+0000","last_fullsized":"2023-10-06T03:08:22.548048+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:05.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"114'4","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.957614+0000","last_change":"2023-10-06T03:08:21.966064+0000","last_active":"2023-10-06T03:08:22.957614+0000","last_peered":"2023-10-06T03:08:22.957614+0000","last_clean":"2023-10-06T03:08:22.957614+0000","last_became_active":"2023-10-06T03:08:21.965780+0000","last_became_peered":"2023-10-06T03:08:21.965780+0000","last_unstale":"2023-10-06T03:08:22.957614+0000","last_undegraded":"2023-10-06T03:08:22.957614+0000","last_fullsized":"2023-10-06T03:08:22.957614+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:52.927746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543790+0000","last_change":"2023-10-06T03:08:22.538002+0000","last_active":"2023-10-06T03:08:22.543790+0000","last_peered":"2023-10-06T03:08:22.543790+0000","last_clean":"2023-10-06T03:08:22.543790+0000","last_became_active":"2023-10-06T03:08:22.537736+0000","last_became_peered":"2023-10-06T03:08:22.537736+0000","last_unstale":"2023-10-06T03:08:22.543790+0000","last_undegraded":"2023-10-06T03:08:22.543790+0000","last_fullsized":"2023-10-06T03:08:22.543790+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:57.700996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"114'1","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530918+0000","last_change":"2023-10-06T03:08:21.966086+0000","last_active":"2023-10-06T03:08:22.530918+0000","last_peered":"2023-10-06T03:08:22.530918+0000","last_clean":"2023-10-06T03:08:22.530918+0000","last_became_active":"2023-10-06T03:08:21.965818+0000","last_became_peered":"2023-10-06T03:08:21.965818+0000","last_unstale":"2023-10-06T03:08:22.530918+0000","last_undegraded":"2023-10-06T03:08:22.530918+0000","last_fullsized":"2023-10-06T03:08:22.530918+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:00.218403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.554573+0000","last_change":"2023-10-06T03:08:22.549139+0000","last_active":"2023-10-06T03:08:22.554573+0000","last_peered":"2023-10-06T03:08:22.554573+0000","last_clean":"2023-10-06T03:08:22.554573+0000","last_became_active":"2023-10-06T03:08:22.548873+0000","last_became_peered":"2023-10-06T03:08:22.548873+0000","last_unstale":"2023-10-06T03:08:22.554573+0000","last_undegraded":"2023-10-06T03:08:22.554573+0000","last_fullsized":"2023-10-06T03:08:22.554573+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:28.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"114'2","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.957520+0000","last_change":"2023-10-06T03:08:21.967520+0000","last_active":"2023-10-06T03:08:22.957520+0000","last_peered":"2023-10-06T03:08:22.957520+0000","last_clean":"2023-10-06T03:08:22.957520+0000","last_became_active":"2023-10-06T03:08:21.967258+0000","last_became_peered":"2023-10-06T03:08:21.967258+0000","last_unstale":"2023-10-06T03:08:22.957520+0000","last_undegraded":"2023-10-06T03:08:22.957520+0000","last_fullsized":"2023-10-06T03:08:22.957520+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:59.697005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543743+0000","last_change":"2023-10-06T03:08:22.539474+0000","last_active":"2023-10-06T03:08:22.543743+0000","last_peered":"2023-10-06T03:08:22.543743+0000","last_clean":"2023-10-06T03:08:22.543743+0000","last_became_active":"2023-10-06T03:08:22.539195+0000","last_became_peered":"2023-10-06T03:08:22.539195+0000","last_unstale":"2023-10-06T03:08:22.543743+0000","last_undegraded":"2023-10-06T03:08:22.543743+0000","last_fullsized":"2023-10-06T03:08:22.543743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:21.633618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"114'2","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.958209+0000","last_change":"2023-10-06T03:08:21.965827+0000","last_active":"2023-10-06T03:08:22.958209+0000","last_peered":"2023-10-06T03:08:22.958209+0000","last_clean":"2023-10-06T03:08:22.958209+0000","last_became_active":"2023-10-06T03:08:21.965549+0000","last_became_peered":"2023-10-06T03:08:21.965549+0000","last_unstale":"2023-10-06T03:08:22.958209+0000","last_undegraded":"2023-10-06T03:08:22.958209+0000","last_fullsized":"2023-10-06T03:08:22.958209+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:43.919361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547825+0000","last_change":"2023-10-06T03:08:22.543353+0000","last_active":"2023-10-06T03:08:22.547825+0000","last_peered":"2023-10-06T03:08:22.547825+0000","last_clean":"2023-10-06T03:08:22.547825+0000","last_became_active":"2023-10-06T03:08:22.543068+0000","last_became_peered":"2023-10-06T03:08:22.543068+0000","last_unstale":"2023-10-06T03:08:22.547825+0000","last_undegraded":"2023-10-06T03:08:22.547825+0000","last_fullsized":"2023-10-06T03:08:22.547825+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:51.911342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.957899+0000","last_change":"2023-10-06T03:08:21.962167+0000","last_active":"2023-10-06T03:08:22.957899+0000","last_peered":"2023-10-06T03:08:22.957899+0000","last_clean":"2023-10-06T03:08:22.957899+0000","last_became_active":"2023-10-06T03:08:21.961916+0000","last_became_peered":"2023-10-06T03:08:21.961916+0000","last_unstale":"2023-10-06T03:08:22.957899+0000","last_undegraded":"2023-10-06T03:08:22.957899+0000","last_fullsized":"2023-10-06T03:08:22.957899+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:19.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535832+0000","last_change":"2023-10-06T03:08:22.533177+0000","last_active":"2023-10-06T03:08:22.535832+0000","last_peered":"2023-10-06T03:08:22.535832+0000","last_clean":"2023-10-06T03:08:22.535832+0000","last_became_active":"2023-10-06T03:08:22.532885+0000","last_became_peered":"2023-10-06T03:08:22.532885+0000","last_unstale":"2023-10-06T03:08:22.535832+0000","last_undegraded":"2023-10-06T03:08:22.535832+0000","last_fullsized":"2023-10-06T03:08:22.535832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:59.993003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"114'1","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.528264+0000","last_change":"2023-10-06T03:08:21.961877+0000","last_active":"2023-10-06T03:08:22.528264+0000","last_peered":"2023-10-06T03:08:22.528264+0000","last_clean":"2023-10-06T03:08:22.528264+0000","last_became_active":"2023-10-06T03:08:21.961619+0000","last_became_peered":"2023-10-06T03:08:21.961619+0000","last_unstale":"2023-10-06T03:08:22.528264+0000","last_undegraded":"2023-10-06T03:08:22.528264+0000","last_fullsized":"2023-10-06T03:08:22.528264+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:22.765332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548228+0000","last_change":"2023-10-06T03:08:22.548228+0000","last_active":"2023-10-06T03:08:22.548228+0000","last_peered":"2023-10-06T03:08:22.548228+0000","last_clean":"2023-10-06T03:08:22.548228+0000","last_became_active":"2023-10-06T03:08:22.548038+0000","last_became_peered":"2023-10-06T03:08:22.548038+0000","last_unstale":"2023-10-06T03:08:22.548228+0000","last_undegraded":"2023-10-06T03:08:22.548228+0000","last_fullsized":"2023-10-06T03:08:22.548228+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:05.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965377+0000","last_change":"2023-10-06T03:08:21.962654+0000","last_active":"2023-10-06T03:08:21.965377+0000","last_peered":"2023-10-06T03:08:21.965377+0000","last_clean":"2023-10-06T03:08:21.965377+0000","last_became_active":"2023-10-06T03:08:21.962399+0000","last_became_peered":"2023-10-06T03:08:21.962399+0000","last_unstale":"2023-10-06T03:08:21.965377+0000","last_undegraded":"2023-10-06T03:08:21.965377+0000","last_fullsized":"2023-10-06T03:08:21.965377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:17.643984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.551179+0000","last_change":"2023-10-06T03:08:22.551179+0000","last_active":"2023-10-06T03:08:22.551179+0000","last_peered":"2023-10-06T03:08:22.551179+0000","last_clean":"2023-10-06T03:08:22.551179+0000","last_became_active":"2023-10-06T03:08:22.550937+0000","last_became_peered":"2023-10-06T03:08:22.550937+0000","last_unstale":"2023-10-06T03:08:22.551179+0000","last_undegraded":"2023-10-06T03:08:22.551179+0000","last_fullsized":"2023-10-06T03:08:22.551179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:35.118953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.968404+0000","last_change":"2023-10-06T03:08:21.963880+0000","last_active":"2023-10-06T03:08:21.968404+0000","last_peered":"2023-10-06T03:08:21.968404+0000","last_clean":"2023-10-06T03:08:21.968404+0000","last_became_active":"2023-10-06T03:08:21.962883+0000","last_became_peered":"2023-10-06T03:08:21.962883+0000","last_unstale":"2023-10-06T03:08:21.968404+0000","last_undegraded":"2023-10-06T03:08:21.968404+0000","last_fullsized":"2023-10-06T03:08:21.968404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:03.153922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547257+0000","last_change":"2023-10-06T03:08:22.547257+0000","last_active":"2023-10-06T03:08:22.547257+0000","last_peered":"2023-10-06T03:08:22.547257+0000","last_clean":"2023-10-06T03:08:22.547257+0000","last_became_active":"2023-10-06T03:08:22.546975+0000","last_became_peered":"2023-10-06T03:08:22.546975+0000","last_unstale":"2023-10-06T03:08:22.547257+0000","last_undegraded":"2023-10-06T03:08:22.547257+0000","last_fullsized":"2023-10-06T03:08:22.547257+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:30.327476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.956741+0000","last_change":"2023-10-06T03:08:21.961399+0000","last_active":"2023-10-06T03:08:22.956741+0000","last_peered":"2023-10-06T03:08:22.956741+0000","last_clean":"2023-10-06T03:08:22.956741+0000","last_became_active":"2023-10-06T03:08:21.961141+0000","last_became_peered":"2023-10-06T03:08:21.961141+0000","last_unstale":"2023-10-06T03:08:22.956741+0000","last_undegraded":"2023-10-06T03:08:22.956741+0000","last_fullsized":"2023-10-06T03:08:22.956741+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:46.620477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547950+0000","last_change":"2023-10-06T03:08:22.547950+0000","last_active":"2023-10-06T03:08:22.547950+0000","last_peered":"2023-10-06T03:08:22.547950+0000","last_clean":"2023-10-06T03:08:22.547950+0000","last_became_active":"2023-10-06T03:08:22.547595+0000","last_became_peered":"2023-10-06T03:08:22.547595+0000","last_unstale":"2023-10-06T03:08:22.547950+0000","last_undegraded":"2023-10-06T03:08:22.547950+0000","last_fullsized":"2023-10-06T03:08:22.547950+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:43.735368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"114'2","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.956060+0000","last_change":"2023-10-06T03:08:21.965741+0000","last_active":"2023-10-06T03:08:22.956060+0000","last_peered":"2023-10-06T03:08:22.956060+0000","last_clean":"2023-10-06T03:08:22.956060+0000","last_became_active":"2023-10-06T03:08:21.965467+0000","last_became_peered":"2023-10-06T03:08:21.965467+0000","last_unstale":"2023-10-06T03:08:22.956060+0000","last_undegraded":"2023-10-06T03:08:22.956060+0000","last_fullsized":"2023-10-06T03:08:22.956060+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:01.099653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.551055+0000","last_change":"2023-10-06T03:08:22.551055+0000","last_active":"2023-10-06T03:08:22.551055+0000","last_peered":"2023-10-06T03:08:22.551055+0000","last_clean":"2023-10-06T03:08:22.551055+0000","last_became_active":"2023-10-06T03:08:22.550783+0000","last_became_peered":"2023-10-06T03:08:22.550783+0000","last_unstale":"2023-10-06T03:08:22.551055+0000","last_undegraded":"2023-10-06T03:08:22.551055+0000","last_fullsized":"2023-10-06T03:08:22.551055+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:26.731510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"114'1","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530864+0000","last_change":"2023-10-06T03:08:21.962321+0000","last_active":"2023-10-06T03:08:22.530864+0000","last_peered":"2023-10-06T03:08:22.530864+0000","last_clean":"2023-10-06T03:08:22.530864+0000","last_became_active":"2023-10-06T03:08:21.962038+0000","last_became_peered":"2023-10-06T03:08:21.962038+0000","last_unstale":"2023-10-06T03:08:22.530864+0000","last_undegraded":"2023-10-06T03:08:22.530864+0000","last_fullsized":"2023-10-06T03:08:22.530864+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:44.826974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541640+0000","last_change":"2023-10-06T03:08:22.541640+0000","last_active":"2023-10-06T03:08:22.541640+0000","last_peered":"2023-10-06T03:08:22.541640+0000","last_clean":"2023-10-06T03:08:22.541640+0000","last_became_active":"2023-10-06T03:08:22.541318+0000","last_became_peered":"2023-10-06T03:08:22.541318+0000","last_unstale":"2023-10-06T03:08:22.541640+0000","last_undegraded":"2023-10-06T03:08:22.541640+0000","last_fullsized":"2023-10-06T03:08:22.541640+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:09.772795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.966658+0000","last_change":"2023-10-06T03:08:21.964674+0000","last_active":"2023-10-06T03:08:21.966658+0000","last_peered":"2023-10-06T03:08:21.966658+0000","last_clean":"2023-10-06T03:08:21.966658+0000","last_became_active":"2023-10-06T03:08:21.963621+0000","last_became_peered":"2023-10-06T03:08:21.963621+0000","last_unstale":"2023-10-06T03:08:21.966658+0000","last_undegraded":"2023-10-06T03:08:21.966658+0000","last_fullsized":"2023-10-06T03:08:21.966658+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:29.549106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.536183+0000","last_change":"2023-10-06T03:08:22.536183+0000","last_active":"2023-10-06T03:08:22.536183+0000","last_peered":"2023-10-06T03:08:22.536183+0000","last_clean":"2023-10-06T03:08:22.536183+0000","last_became_active":"2023-10-06T03:08:22.535892+0000","last_became_peered":"2023-10-06T03:08:22.535892+0000","last_unstale":"2023-10-06T03:08:22.536183+0000","last_undegraded":"2023-10-06T03:08:22.536183+0000","last_fullsized":"2023-10-06T03:08:22.536183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:00.704367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960258+0000","last_change":"2023-10-06T03:08:21.966576+0000","last_active":"2023-10-06T03:08:22.960258+0000","last_peered":"2023-10-06T03:08:22.960258+0000","last_clean":"2023-10-06T03:08:22.960258+0000","last_became_active":"2023-10-06T03:08:21.966245+0000","last_became_peered":"2023-10-06T03:08:21.966245+0000","last_unstale":"2023-10-06T03:08:22.960258+0000","last_undegraded":"2023-10-06T03:08:22.960258+0000","last_fullsized":"2023-10-06T03:08:22.960258+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:20.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.546719+0000","last_change":"2023-10-06T03:08:22.544826+0000","last_active":"2023-10-06T03:08:22.546719+0000","last_peered":"2023-10-06T03:08:22.546719+0000","last_clean":"2023-10-06T03:08:22.546719+0000","last_became_active":"2023-10-06T03:08:22.544581+0000","last_became_peered":"2023-10-06T03:08:22.544581+0000","last_unstale":"2023-10-06T03:08:22.546719+0000","last_undegraded":"2023-10-06T03:08:22.546719+0000","last_fullsized":"2023-10-06T03:08:22.546719+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:59.393124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.526752+0000","last_change":"2023-10-06T03:08:21.962597+0000","last_active":"2023-10-06T03:08:22.526752+0000","last_peered":"2023-10-06T03:08:22.526752+0000","last_clean":"2023-10-06T03:08:22.526752+0000","last_became_active":"2023-10-06T03:08:21.960988+0000","last_became_peered":"2023-10-06T03:08:21.960988+0000","last_unstale":"2023-10-06T03:08:22.526752+0000","last_undegraded":"2023-10-06T03:08:22.526752+0000","last_fullsized":"2023-10-06T03:08:22.526752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:08.586968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540338+0000","last_change":"2023-10-06T03:08:22.540338+0000","last_active":"2023-10-06T03:08:22.540338+0000","last_peered":"2023-10-06T03:08:22.540338+0000","last_clean":"2023-10-06T03:08:22.540338+0000","last_became_active":"2023-10-06T03:08:22.540090+0000","last_became_peered":"2023-10-06T03:08:22.540090+0000","last_unstale":"2023-10-06T03:08:22.540338+0000","last_undegraded":"2023-10-06T03:08:22.540338+0000","last_fullsized":"2023-10-06T03:08:22.540338+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:50.584290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.973708+0000","last_change":"2023-10-06T03:08:21.967431+0000","last_active":"2023-10-06T03:08:21.973708+0000","last_peered":"2023-10-06T03:08:21.973708+0000","last_clean":"2023-10-06T03:08:21.973708+0000","last_became_active":"2023-10-06T03:08:21.966991+0000","last_became_peered":"2023-10-06T03:08:21.966991+0000","last_unstale":"2023-10-06T03:08:21.973708+0000","last_undegraded":"2023-10-06T03:08:21.973708+0000","last_fullsized":"2023-10-06T03:08:21.973708+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:11.924285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.554658+0000","last_change":"2023-10-06T03:08:22.549803+0000","last_active":"2023-10-06T03:08:22.554658+0000","last_peered":"2023-10-06T03:08:22.554658+0000","last_clean":"2023-10-06T03:08:22.554658+0000","last_became_active":"2023-10-06T03:08:22.549530+0000","last_became_peered":"2023-10-06T03:08:22.549530+0000","last_unstale":"2023-10-06T03:08:22.554658+0000","last_undegraded":"2023-10-06T03:08:22.554658+0000","last_fullsized":"2023-10-06T03:08:22.554658+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:20.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965457+0000","last_change":"2023-10-06T03:08:21.963695+0000","last_active":"2023-10-06T03:08:21.965457+0000","last_peered":"2023-10-06T03:08:21.965457+0000","last_clean":"2023-10-06T03:08:21.965457+0000","last_became_active":"2023-10-06T03:08:21.963441+0000","last_became_peered":"2023-10-06T03:08:21.963441+0000","last_unstale":"2023-10-06T03:08:21.965457+0000","last_undegraded":"2023-10-06T03:08:21.965457+0000","last_fullsized":"2023-10-06T03:08:21.965457+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:28.573484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548318+0000","last_change":"2023-10-06T03:08:22.548318+0000","last_active":"2023-10-06T03:08:22.548318+0000","last_peered":"2023-10-06T03:08:22.548318+0000","last_clean":"2023-10-06T03:08:22.548318+0000","last_became_active":"2023-10-06T03:08:22.548077+0000","last_became_peered":"2023-10-06T03:08:22.548077+0000","last_unstale":"2023-10-06T03:08:22.548318+0000","last_undegraded":"2023-10-06T03:08:22.548318+0000","last_fullsized":"2023-10-06T03:08:22.548318+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:08.821026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960404+0000","last_change":"2023-10-06T03:08:21.966590+0000","last_active":"2023-10-06T03:08:22.960404+0000","last_peered":"2023-10-06T03:08:22.960404+0000","last_clean":"2023-10-06T03:08:22.960404+0000","last_became_active":"2023-10-06T03:08:21.966290+0000","last_became_peered":"2023-10-06T03:08:21.966290+0000","last_unstale":"2023-10-06T03:08:22.960404+0000","last_undegraded":"2023-10-06T03:08:22.960404+0000","last_fullsized":"2023-10-06T03:08:22.960404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:34.069486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542066+0000","last_change":"2023-10-06T03:08:22.542066+0000","last_active":"2023-10-06T03:08:22.542066+0000","last_peered":"2023-10-06T03:08:22.542066+0000","last_clean":"2023-10-06T03:08:22.542066+0000","last_became_active":"2023-10-06T03:08:22.541802+0000","last_became_peered":"2023-10-06T03:08:22.541802+0000","last_unstale":"2023-10-06T03:08:22.542066+0000","last_undegraded":"2023-10-06T03:08:22.542066+0000","last_fullsized":"2023-10-06T03:08:22.542066+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:00.335970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530944+0000","last_change":"2023-10-06T03:08:21.966194+0000","last_active":"2023-10-06T03:08:22.530944+0000","last_peered":"2023-10-06T03:08:22.530944+0000","last_clean":"2023-10-06T03:08:22.530944+0000","last_became_active":"2023-10-06T03:08:21.965930+0000","last_became_peered":"2023-10-06T03:08:21.965930+0000","last_unstale":"2023-10-06T03:08:22.530944+0000","last_undegraded":"2023-10-06T03:08:22.530944+0000","last_fullsized":"2023-10-06T03:08:22.530944+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:13.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.550617+0000","last_change":"2023-10-06T03:08:22.550617+0000","last_active":"2023-10-06T03:08:22.550617+0000","last_peered":"2023-10-06T03:08:22.550617+0000","last_clean":"2023-10-06T03:08:22.550617+0000","last_became_active":"2023-10-06T03:08:22.550189+0000","last_became_peered":"2023-10-06T03:08:22.550189+0000","last_unstale":"2023-10-06T03:08:22.550617+0000","last_undegraded":"2023-10-06T03:08:22.550617+0000","last_fullsized":"2023-10-06T03:08:22.550617+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:51.062202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530395+0000","last_change":"2023-10-06T03:08:21.961487+0000","last_active":"2023-10-06T03:08:22.530395+0000","last_peered":"2023-10-06T03:08:22.530395+0000","last_clean":"2023-10-06T03:08:22.530395+0000","last_became_active":"2023-10-06T03:08:21.961210+0000","last_became_peered":"2023-10-06T03:08:21.961210+0000","last_unstale":"2023-10-06T03:08:22.530395+0000","last_undegraded":"2023-10-06T03:08:22.530395+0000","last_fullsized":"2023-10-06T03:08:22.530395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:11.924285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.534603+0000","last_change":"2023-10-06T03:08:22.534603+0000","last_active":"2023-10-06T03:08:22.534603+0000","last_peered":"2023-10-06T03:08:22.534603+0000","last_clean":"2023-10-06T03:08:22.534603+0000","last_became_active":"2023-10-06T03:08:22.534424+0000","last_became_peered":"2023-10-06T03:08:22.534424+0000","last_unstale":"2023-10-06T03:08:22.534603+0000","last_undegraded":"2023-10-06T03:08:22.534603+0000","last_fullsized":"2023-10-06T03:08:22.534603+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:57.475796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.968875+0000","last_change":"2023-10-06T03:08:21.967575+0000","last_active":"2023-10-06T03:08:21.968875+0000","last_peered":"2023-10-06T03:08:21.968875+0000","last_clean":"2023-10-06T03:08:21.968875+0000","last_became_active":"2023-10-06T03:08:21.967334+0000","last_became_peered":"2023-10-06T03:08:21.967334+0000","last_unstale":"2023-10-06T03:08:21.968875+0000","last_undegraded":"2023-10-06T03:08:21.968875+0000","last_fullsized":"2023-10-06T03:08:21.968875+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:02.695955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540776+0000","last_change":"2023-10-06T03:08:22.540776+0000","last_active":"2023-10-06T03:08:22.540776+0000","last_peered":"2023-10-06T03:08:22.540776+0000","last_clean":"2023-10-06T03:08:22.540776+0000","last_became_active":"2023-10-06T03:08:22.540513+0000","last_became_peered":"2023-10-06T03:08:22.540513+0000","last_unstale":"2023-10-06T03:08:22.540776+0000","last_undegraded":"2023-10-06T03:08:22.540776+0000","last_fullsized":"2023-10-06T03:08:22.540776+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:26.910097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533706+0000","last_change":"2023-10-06T03:08:21.964068+0000","last_active":"2023-10-06T03:08:22.533706+0000","last_peered":"2023-10-06T03:08:22.533706+0000","last_clean":"2023-10-06T03:08:22.533706+0000","last_became_active":"2023-10-06T03:08:21.963741+0000","last_became_peered":"2023-10-06T03:08:21.963741+0000","last_unstale":"2023-10-06T03:08:22.533706+0000","last_undegraded":"2023-10-06T03:08:22.533706+0000","last_fullsized":"2023-10-06T03:08:22.533706+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:17.582565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541418+0000","last_change":"2023-10-06T03:08:22.541418+0000","last_active":"2023-10-06T03:08:22.541418+0000","last_peered":"2023-10-06T03:08:22.541418+0000","last_clean":"2023-10-06T03:08:22.541418+0000","last_became_active":"2023-10-06T03:08:22.541182+0000","last_became_peered":"2023-10-06T03:08:22.541182+0000","last_unstale":"2023-10-06T03:08:22.541418+0000","last_undegraded":"2023-10-06T03:08:22.541418+0000","last_fullsized":"2023-10-06T03:08:22.541418+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.699820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.973756+0000","last_change":"2023-10-06T03:08:21.967949+0000","last_active":"2023-10-06T03:08:21.973756+0000","last_peered":"2023-10-06T03:08:21.973756+0000","last_clean":"2023-10-06T03:08:21.973756+0000","last_became_active":"2023-10-06T03:08:21.967689+0000","last_became_peered":"2023-10-06T03:08:21.967689+0000","last_unstale":"2023-10-06T03:08:21.973756+0000","last_undegraded":"2023-10-06T03:08:21.973756+0000","last_fullsized":"2023-10-06T03:08:21.973756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:41.096483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.544945+0000","last_change":"2023-10-06T03:08:22.544945+0000","last_active":"2023-10-06T03:08:22.544945+0000","last_peered":"2023-10-06T03:08:22.544945+0000","last_clean":"2023-10-06T03:08:22.544945+0000","last_became_active":"2023-10-06T03:08:22.544704+0000","last_became_peered":"2023-10-06T03:08:22.544704+0000","last_unstale":"2023-10-06T03:08:22.544945+0000","last_undegraded":"2023-10-06T03:08:22.544945+0000","last_fullsized":"2023-10-06T03:08:22.544945+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:28.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965396+0000","last_change":"2023-10-06T03:08:21.963183+0000","last_active":"2023-10-06T03:08:21.965396+0000","last_peered":"2023-10-06T03:08:21.965396+0000","last_clean":"2023-10-06T03:08:21.965396+0000","last_became_active":"2023-10-06T03:08:21.962930+0000","last_became_peered":"2023-10-06T03:08:21.962930+0000","last_unstale":"2023-10-06T03:08:21.965396+0000","last_undegraded":"2023-10-06T03:08:21.965396+0000","last_fullsized":"2023-10-06T03:08:21.965396+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:51.453402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.550511+0000","last_change":"2023-10-06T03:08:22.550511+0000","last_active":"2023-10-06T03:08:22.550511+0000","last_peered":"2023-10-06T03:08:22.550511+0000","last_clean":"2023-10-06T03:08:22.550511+0000","last_became_active":"2023-10-06T03:08:22.550222+0000","last_became_peered":"2023-10-06T03:08:22.550222+0000","last_unstale":"2023-10-06T03:08:22.550511+0000","last_undegraded":"2023-10-06T03:08:22.550511+0000","last_fullsized":"2023-10-06T03:08:22.550511+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:49.949858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965257+0000","last_change":"2023-10-06T03:08:21.963258+0000","last_active":"2023-10-06T03:08:21.965257+0000","last_peered":"2023-10-06T03:08:21.965257+0000","last_clean":"2023-10-06T03:08:21.965257+0000","last_became_active":"2023-10-06T03:08:21.962955+0000","last_became_peered":"2023-10-06T03:08:21.962955+0000","last_unstale":"2023-10-06T03:08:21.965257+0000","last_undegraded":"2023-10-06T03:08:21.965257+0000","last_fullsized":"2023-10-06T03:08:21.965257+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:59.328608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541630+0000","last_change":"2023-10-06T03:08:22.541630+0000","last_active":"2023-10-06T03:08:22.541630+0000","last_peered":"2023-10-06T03:08:22.541630+0000","last_clean":"2023-10-06T03:08:22.541630+0000","last_became_active":"2023-10-06T03:08:22.541374+0000","last_became_peered":"2023-10-06T03:08:22.541374+0000","last_unstale":"2023-10-06T03:08:22.541630+0000","last_undegraded":"2023-10-06T03:08:22.541630+0000","last_fullsized":"2023-10-06T03:08:22.541630+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:07.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.526506+0000","last_change":"2023-10-06T03:08:21.960552+0000","last_active":"2023-10-06T03:08:22.526506+0000","last_peered":"2023-10-06T03:08:22.526506+0000","last_clean":"2023-10-06T03:08:22.526506+0000","last_became_active":"2023-10-06T03:08:21.960290+0000","last_became_peered":"2023-10-06T03:08:21.960290+0000","last_unstale":"2023-10-06T03:08:22.526506+0000","last_undegraded":"2023-10-06T03:08:22.526506+0000","last_fullsized":"2023-10-06T03:08:22.526506+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:10.706495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.962131+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:08:21.962131+0000","last_peered":"2023-10-06T03:08:21.962131+0000","last_clean":"2023-10-06T03:08:21.962131+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:08:21.962131+0000","last_undegraded":"2023-10-06T03:08:21.962131+0000","last_fullsized":"2023-10-06T03:08:21.962131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548640+0000","last_change":"2023-10-06T03:08:22.547218+0000","last_active":"2023-10-06T03:08:22.548640+0000","last_peered":"2023-10-06T03:08:22.548640+0000","last_clean":"2023-10-06T03:08:22.548640+0000","last_became_active":"2023-10-06T03:08:22.546992+0000","last_became_peered":"2023-10-06T03:08:22.546992+0000","last_unstale":"2023-10-06T03:08:22.548640+0000","last_undegraded":"2023-10-06T03:08:22.548640+0000","last_fullsized":"2023-10-06T03:08:22.548640+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:25.036920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533668+0000","last_change":"2023-10-06T03:08:21.964143+0000","last_active":"2023-10-06T03:08:22.533668+0000","last_peered":"2023-10-06T03:08:22.533668+0000","last_clean":"2023-10-06T03:08:22.533668+0000","last_became_active":"2023-10-06T03:08:21.963892+0000","last_became_peered":"2023-10-06T03:08:21.963892+0000","last_unstale":"2023-10-06T03:08:22.533668+0000","last_undegraded":"2023-10-06T03:08:22.533668+0000","last_fullsized":"2023-10-06T03:08:22.533668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:50.462283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.957037+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:08:21.957037+0000","last_peered":"2023-10-06T03:08:21.957037+0000","last_clean":"2023-10-06T03:08:21.957037+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:08:21.957037+0000","last_undegraded":"2023-10-06T03:08:21.957037+0000","last_fullsized":"2023-10-06T03:08:21.957037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542693+0000","last_change":"2023-10-06T03:08:22.542693+0000","last_active":"2023-10-06T03:08:22.542693+0000","last_peered":"2023-10-06T03:08:22.542693+0000","last_clean":"2023-10-06T03:08:22.542693+0000","last_became_active":"2023-10-06T03:08:22.542433+0000","last_became_peered":"2023-10-06T03:08:22.542433+0000","last_unstale":"2023-10-06T03:08:22.542693+0000","last_undegraded":"2023-10-06T03:08:22.542693+0000","last_fullsized":"2023-10-06T03:08:22.542693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:23.772694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"114'6","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960043+0000","last_change":"2023-10-06T03:08:21.968418+0000","last_active":"2023-10-06T03:08:22.960043+0000","last_peered":"2023-10-06T03:08:22.960043+0000","last_clean":"2023-10-06T03:08:22.960043+0000","last_became_active":"2023-10-06T03:08:21.967240+0000","last_became_peered":"2023-10-06T03:08:21.967240+0000","last_unstale":"2023-10-06T03:08:22.960043+0000","last_undegraded":"2023-10-06T03:08:22.960043+0000","last_fullsized":"2023-10-06T03:08:22.960043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:13.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.960966+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:08:21.960966+0000","last_peered":"2023-10-06T03:08:21.960966+0000","last_clean":"2023-10-06T03:08:21.960966+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:08:21.960966+0000","last_undegraded":"2023-10-06T03:08:21.960966+0000","last_fullsized":"2023-10-06T03:08:21.960966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547426+0000","last_change":"2023-10-06T03:08:22.547426+0000","last_active":"2023-10-06T03:08:22.547426+0000","last_peered":"2023-10-06T03:08:22.547426+0000","last_clean":"2023-10-06T03:08:22.547426+0000","last_became_active":"2023-10-06T03:08:22.547153+0000","last_became_peered":"2023-10-06T03:08:22.547153+0000","last_unstale":"2023-10-06T03:08:22.547426+0000","last_undegraded":"2023-10-06T03:08:22.547426+0000","last_fullsized":"2023-10-06T03:08:22.547426+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:32.180596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.969255+0000","last_change":"2023-10-06T03:08:21.966497+0000","last_active":"2023-10-06T03:08:21.969255+0000","last_peered":"2023-10-06T03:08:21.969255+0000","last_clean":"2023-10-06T03:08:21.969255+0000","last_became_active":"2023-10-06T03:08:21.966231+0000","last_became_peered":"2023-10-06T03:08:21.966231+0000","last_unstale":"2023-10-06T03:08:21.969255+0000","last_undegraded":"2023-10-06T03:08:21.969255+0000","last_fullsized":"2023-10-06T03:08:21.969255+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:28.384668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.959984+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:08:21.959984+0000","last_peered":"2023-10-06T03:08:21.959984+0000","last_clean":"2023-10-06T03:08:21.959984+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:08:21.959984+0000","last_undegraded":"2023-10-06T03:08:21.959984+0000","last_fullsized":"2023-10-06T03:08:21.959984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547105+0000","last_change":"2023-10-06T03:08:22.547105+0000","last_active":"2023-10-06T03:08:22.547105+0000","last_peered":"2023-10-06T03:08:22.547105+0000","last_clean":"2023-10-06T03:08:22.547105+0000","last_became_active":"2023-10-06T03:08:22.546840+0000","last_became_peered":"2023-10-06T03:08:22.546840+0000","last_unstale":"2023-10-06T03:08:22.547105+0000","last_undegraded":"2023-10-06T03:08:22.547105+0000","last_fullsized":"2023-10-06T03:08:22.547105+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:11.945719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.970831+0000","last_change":"2023-10-06T03:08:21.966435+0000","last_active":"2023-10-06T03:08:21.970831+0000","last_peered":"2023-10-06T03:08:21.970831+0000","last_clean":"2023-10-06T03:08:21.970831+0000","last_became_active":"2023-10-06T03:08:21.966186+0000","last_became_peered":"2023-10-06T03:08:21.966186+0000","last_unstale":"2023-10-06T03:08:21.970831+0000","last_undegraded":"2023-10-06T03:08:21.970831+0000","last_fullsized":"2023-10-06T03:08:21.970831+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:20.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":242,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.960431+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:08:21.960431+0000","last_peered":"2023-10-06T03:08:21.960431+0000","last_clean":"2023-10-06T03:08:21.960431+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:08:21.960431+0000","last_undegraded":"2023-10-06T03:08:21.960431+0000","last_fullsized":"2023-10-06T03:08:21.960431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":100,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533756+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:08:22.533756+0000","last_peered":"2023-10-06T03:08:22.533756+0000","last_clean":"2023-10-06T03:08:22.533756+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:08:22.533756+0000","last_undegraded":"2023-10-06T03:08:22.533756+0000","last_fullsized":"2023-10-06T03:08:22.533756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542373+0000","last_change":"2023-10-06T03:08:22.542373+0000","last_active":"2023-10-06T03:08:22.542373+0000","last_peered":"2023-10-06T03:08:22.542373+0000","last_clean":"2023-10-06T03:08:22.542373+0000","last_became_active":"2023-10-06T03:08:22.541998+0000","last_became_peered":"2023-10-06T03:08:22.541998+0000","last_unstale":"2023-10-06T03:08:22.542373+0000","last_undegraded":"2023-10-06T03:08:22.542373+0000","last_fullsized":"2023-10-06T03:08:22.542373+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:08.345180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"114'8","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.961308+0000","last_change":"2023-10-06T03:08:21.967181+0000","last_active":"2023-10-06T03:08:22.961308+0000","last_peered":"2023-10-06T03:08:22.961308+0000","last_clean":"2023-10-06T03:08:22.961308+0000","last_became_active":"2023-10-06T03:08:21.966898+0000","last_became_peered":"2023-10-06T03:08:21.966898+0000","last_unstale":"2023-10-06T03:08:22.961308+0000","last_undegraded":"2023-10-06T03:08:22.961308+0000","last_fullsized":"2023-10-06T03:08:22.961308+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:00.219437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":91,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.526421+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:08:22.526421+0000","last_peered":"2023-10-06T03:08:22.526421+0000","last_clean":"2023-10-06T03:08:22.526421+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:08:22.526421+0000","last_undegraded":"2023-10-06T03:08:22.526421+0000","last_fullsized":"2023-10-06T03:08:22.526421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535888+0000","last_change":"2023-10-06T03:08:22.535888+0000","last_active":"2023-10-06T03:08:22.535888+0000","last_peered":"2023-10-06T03:08:22.535888+0000","last_clean":"2023-10-06T03:08:22.535888+0000","last_became_active":"2023-10-06T03:08:22.535635+0000","last_became_peered":"2023-10-06T03:08:22.535635+0000","last_unstale":"2023-10-06T03:08:22.535888+0000","last_undegraded":"2023-10-06T03:08:22.535888+0000","last_fullsized":"2023-10-06T03:08:22.535888+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:17.649792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.956749+0000","last_change":"2023-10-06T03:08:21.960604+0000","last_active":"2023-10-06T03:08:22.956749+0000","last_peered":"2023-10-06T03:08:22.956749+0000","last_clean":"2023-10-06T03:08:22.956749+0000","last_became_active":"2023-10-06T03:08:21.960295+0000","last_became_peered":"2023-10-06T03:08:21.960295+0000","last_unstale":"2023-10-06T03:08:22.956749+0000","last_undegraded":"2023-10-06T03:08:22.956749+0000","last_fullsized":"2023-10-06T03:08:22.956749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:21.730880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":100,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533507+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:08:22.533507+0000","last_peered":"2023-10-06T03:08:22.533507+0000","last_clean":"2023-10-06T03:08:22.533507+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:08:22.533507+0000","last_undegraded":"2023-10-06T03:08:22.533507+0000","last_fullsized":"2023-10-06T03:08:22.533507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.539940+0000","last_change":"2023-10-06T03:08:22.539940+0000","last_active":"2023-10-06T03:08:22.539940+0000","last_peered":"2023-10-06T03:08:22.539940+0000","last_clean":"2023-10-06T03:08:22.539940+0000","last_became_active":"2023-10-06T03:08:22.539692+0000","last_became_peered":"2023-10-06T03:08:22.539692+0000","last_unstale":"2023-10-06T03:08:22.539940+0000","last_undegraded":"2023-10-06T03:08:22.539940+0000","last_fullsized":"2023-10-06T03:08:22.539940+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:27.230608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"114'3","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960441+0000","last_change":"2023-10-06T03:08:21.968958+0000","last_active":"2023-10-06T03:08:22.960441+0000","last_peered":"2023-10-06T03:08:22.960441+0000","last_clean":"2023-10-06T03:08:22.960441+0000","last_became_active":"2023-10-06T03:08:21.968652+0000","last_became_peered":"2023-10-06T03:08:21.968652+0000","last_unstale":"2023-10-06T03:08:22.960441+0000","last_undegraded":"2023-10-06T03:08:22.960441+0000","last_fullsized":"2023-10-06T03:08:22.960441+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:19.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.957059+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:08:21.957059+0000","last_peered":"2023-10-06T03:08:21.957059+0000","last_clean":"2023-10-06T03:08:21.957059+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:08:21.957059+0000","last_undegraded":"2023-10-06T03:08:21.957059+0000","last_fullsized":"2023-10-06T03:08:21.957059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303372,"kb_used_data":1988,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068468,"statfs":{"total":96636764160,"available":96326111232,"internally_reserved":0,"allocated":2035712,"data_stored":1244018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":1.264},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":2.286},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":1.1890000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":4.0289999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":2.504}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":1.141},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":1.639}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":2.3359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302784,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069056,"statfs":{"total":96636764160,"available":96326713344,"internally_reserved":0,"allocated":1433600,"data_stored":652668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.628},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298684,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073156,"statfs":{"total":96636764160,"available":96330911744,"internally_reserved":0,"allocated":1429504,"data_stored":652094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.29699999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.441},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.46899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303364,"kb_used_data":1980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068476,"statfs":{"total":96636764160,"available":96326119424,"internally_reserved":0,"allocated":2027520,"data_stored":1242448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.217},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.27100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.251}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.377},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302796,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069044,"statfs":{"total":96636764160,"available":96326701056,"internally_reserved":0,"allocated":1445888,"data_stored":659253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.374}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302792,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069048,"statfs":{"total":96636764160,"available":96326705152,"internally_reserved":0,"allocated":1441792,"data_stored":653630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.26700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.52700000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.495},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.505},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302788,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069052,"statfs":{"total":96636764160,"available":96326709248,"internally_reserved":0,"allocated":1437696,"data_stored":658665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.437}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302792,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069048,"statfs":{"total":96636764160,"available":96326705152,"internally_reserved":0,"allocated":1441792,"data_stored":658687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.498}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:08:28.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:08:28.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.163+0000 7f72e0c8d700 1 Processor -- start 2023-10-06T03:08:28.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.163+0000 7f72e0c8d700 1 -- start start 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72e0c8d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72dc159890 0x7f72dc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72e0c8d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f72dc159c60 0x7f72dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72e0c8d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc15a160 con 0x7f72dc15db10 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72e0c8d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc15df50 con 0x7f72dc159890 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72e0c8d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0d13a0 con 0x7f72dc159c60 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72da59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f72dc159890 0x7f72dc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72dad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f72dc15db10 0x7f72dc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37562/0 (socket says 172.21.15.92:37562) 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72dad9d700 1 -- 172.21.15.92:0/3945704912 learned_addr learned my addr 172.21.15.92:0/3945704912 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:28.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72da59c700 1 -- 172.21.15.92:0/3945704912 >> v2:172.21.15.136:3300/0 conn(0x7f72dc159c60 msgr2=0x7f72dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72da59c700 1 --2- 172.21.15.92:0/3945704912 >> v2:172.21.15.136:3300/0 conn(0x7f72dc159c60 0x7f72dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72da59c700 1 -- 172.21.15.92:0/3945704912 >> v1:172.21.15.92:6789/0 conn(0x7f72dc15db10 legacy=0x7f72dc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72da59c700 1 -- 172.21.15.92:0/3945704912 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc15e140 con 0x7f72dc159890 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72da59c700 1 --2- 172.21.15.92:0/3945704912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc159890 0x7f72dc15abd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b67f0 tx=0x7f72d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fc89a5ad723e19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72d959a700 1 -- 172.21.15.92:0/3945704912 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d400e070 con 0x7f72dc159890 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72d959a700 1 -- 172.21.15.92:0/3945704912 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d400bc30 con 0x7f72dc159890 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.164+0000 7f72d959a700 1 -- 172.21.15.92:0/3945704912 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4012430 con 0x7f72dc159890 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 -- 172.21.15.92:0/3945704912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc159890 msgr2=0x7f72dc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/3945704912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc159890 0x7f72dc15abd0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b67f0 tx=0x7f72d4009780 comp rx=0 tx=0).stop 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 -- 172.21.15.92:0/3945704912 shutdown_connections 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/3945704912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc159890 0x7f72dc15abd0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/3945704912 >> v2:172.21.15.136:3300/0 conn(0x7f72dc159c60 0x7f72dc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 -- 172.21.15.92:0/3945704912 >> 172.21.15.92:0/3945704912 conn(0x7f72dc068e20 msgr2=0x7f72dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:28.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 -- 172.21.15.92:0/3945704912 shutdown_connections 2023-10-06T03:08:28.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.165+0000 7f72e0c8d700 1 -- 172.21.15.92:0/3945704912 wait complete. 2023-10-06T03:08:28.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.166+0000 7f72e0c8d700 1 Processor -- start 2023-10-06T03:08:28.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.166+0000 7f72e0c8d700 1 -- start start 2023-10-06T03:08:28.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72dc159890 0x7f72dc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72dc159c60 0x7f72dc152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 0x7f72dc152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0b3f60 con 0x7f72dc15db10 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0ce020 con 0x7f72dc159c60 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0b3630 con 0x7f72dc159890 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 0x7f72dc152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 0x7f72dc152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38092/0 (socket says 172.21.15.92:38092) 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 -- 172.21.15.92:0/4261048650 learned_addr learned my addr 172.21.15.92:0/4261048650 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72d9d9b700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72dc159c60 0x7f72dc152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72da59c700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72dc159890 0x7f72dc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 -- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72dc159890 msgr2=0x7f72dc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72dc159890 0x7f72dc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 -- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72dc159c60 msgr2=0x7f72dc152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72dc159c60 0x7f72dc152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 -- 172.21.15.92:0/4261048650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d4009a20 con 0x7f72dc15db10 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72d9d9b700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72dc159c60 0x7f72dc152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72da59c700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72dc159890 0x7f72dc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72dad9d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 0x7f72dc152b50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c560 tx=0x7f72cc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc005b30 con 0x7f72dc15db10 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc005c90 con 0x7f72dc15db10 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.167+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc0b3910 con 0x7f72dc15db10 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.168+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0072e0 con 0x7f72dc15db10 2023-10-06T03:08:28.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.168+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc0b43f0 con 0x7f72dc15db10 2023-10-06T03:08:28.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.169+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc04c3c0 con 0x7f72dc15db10 2023-10-06T03:08:28.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.169+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f72cc007690 con 0x7f72dc15db10 2023-10-06T03:08:28.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.169+0000 7f72ca7fc700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72bc067320 0x7f72bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.169+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72cc084130 con 0x7f72dc15db10 2023-10-06T03:08:28.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.169+0000 7f72da59c700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72bc067320 0x7f72bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.170+0000 7f72da59c700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72bc067320 0x7f72bc0697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b67f0 tx=0x7f72d4019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.173+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72cc04b5a0 con 0x7f72dc15db10 2023-10-06T03:08:28.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.356+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72dc06a5b0 con 0x7f72bc067320 2023-10-06T03:08:28.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.364+0000 7f72ca7fc700 1 -- 172.21.15.92:0/4261048650 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312100 (secure 0 0 0) 0x7f72dc06a5b0 con 0x7f72bc067320 2023-10-06T03:08:28.369 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72bc067320 msgr2=0x7f72bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72bc067320 0x7f72bc0697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0b67f0 tx=0x7f72d4019000 comp rx=0 tx=0).stop 2023-10-06T03:08:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 msgr2=0x7f72dc152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 0x7f72dc152b50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c560 tx=0x7f72cc007f60 comp rx=0 tx=0).stop 2023-10-06T03:08:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 shutdown_connections 2023-10-06T03:08:28.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72bc067320 0x7f72bc0697d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72dc15db10 0x7f72dc152b50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72dc159890 0x7f72dc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 --2- 172.21.15.92:0/4261048650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72dc159c60 0x7f72dc152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.367+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 >> 172.21.15.92:0/4261048650 conn(0x7f72dc068e20 msgr2=0x7f72dc158840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.368+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 shutdown_connections 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.368+0000 7f72e0c8d700 1 -- 172.21.15.92:0/4261048650 wait complete. 2023-10-06T03:08:28.373 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:08:28.381 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-10-06T03:08:27.264789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2419372,"kb_used_data":12396,"kb_used_omap":0,"kb_used_meta":2406912,"kb_avail":752555348,"statfs":{"total":773094113280,"available":770616676352,"internally_reserved":0,"allocated":12693504,"data_stored":6421463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2464677888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001068"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535246+0000","last_change":"2023-10-06T03:08:22.532629+0000","last_active":"2023-10-06T03:08:22.535246+0000","last_peered":"2023-10-06T03:08:22.535246+0000","last_clean":"2023-10-06T03:08:22.535246+0000","last_became_active":"2023-10-06T03:08:22.532391+0000","last_became_peered":"2023-10-06T03:08:22.532391+0000","last_unstale":"2023-10-06T03:08:22.535246+0000","last_undegraded":"2023-10-06T03:08:22.535246+0000","last_fullsized":"2023-10-06T03:08:22.535246+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:15.303883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542314+0000","last_change":"2023-10-06T03:08:22.537320+0000","last_active":"2023-10-06T03:08:22.542314+0000","last_peered":"2023-10-06T03:08:22.542314+0000","last_clean":"2023-10-06T03:08:22.542314+0000","last_became_active":"2023-10-06T03:08:22.537024+0000","last_became_peered":"2023-10-06T03:08:22.537024+0000","last_unstale":"2023-10-06T03:08:22.542314+0000","last_undegraded":"2023-10-06T03:08:22.542314+0000","last_fullsized":"2023-10-06T03:08:22.542314+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:26.910097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548523+0000","last_change":"2023-10-06T03:08:22.548523+0000","last_active":"2023-10-06T03:08:22.548523+0000","last_peered":"2023-10-06T03:08:22.548523+0000","last_clean":"2023-10-06T03:08:22.548523+0000","last_became_active":"2023-10-06T03:08:22.548228+0000","last_became_peered":"2023-10-06T03:08:22.548228+0000","last_unstale":"2023-10-06T03:08:22.548523+0000","last_undegraded":"2023-10-06T03:08:22.548523+0000","last_fullsized":"2023-10-06T03:08:22.548523+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:47.321425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.549224+0000","last_change":"2023-10-06T03:08:22.541832+0000","last_active":"2023-10-06T03:08:22.549224+0000","last_peered":"2023-10-06T03:08:22.549224+0000","last_clean":"2023-10-06T03:08:22.549224+0000","last_became_active":"2023-10-06T03:08:22.541518+0000","last_became_peered":"2023-10-06T03:08:22.541518+0000","last_unstale":"2023-10-06T03:08:22.549224+0000","last_undegraded":"2023-10-06T03:08:22.549224+0000","last_fullsized":"2023-10-06T03:08:22.549224+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:30.710396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.537092+0000","last_change":"2023-10-06T03:08:22.537092+0000","last_active":"2023-10-06T03:08:22.537092+0000","last_peered":"2023-10-06T03:08:22.537092+0000","last_clean":"2023-10-06T03:08:22.537092+0000","last_became_active":"2023-10-06T03:08:22.536841+0000","last_became_peered":"2023-10-06T03:08:22.536841+0000","last_unstale":"2023-10-06T03:08:22.537092+0000","last_undegraded":"2023-10-06T03:08:22.537092+0000","last_fullsized":"2023-10-06T03:08:22.537092+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:44.926723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548085+0000","last_change":"2023-10-06T03:08:22.548085+0000","last_active":"2023-10-06T03:08:22.548085+0000","last_peered":"2023-10-06T03:08:22.548085+0000","last_clean":"2023-10-06T03:08:22.548085+0000","last_became_active":"2023-10-06T03:08:22.547831+0000","last_became_peered":"2023-10-06T03:08:22.547831+0000","last_unstale":"2023-10-06T03:08:22.548085+0000","last_undegraded":"2023-10-06T03:08:22.548085+0000","last_fullsized":"2023-10-06T03:08:22.548085+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:20.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.550433+0000","last_change":"2023-10-06T03:08:22.550433+0000","last_active":"2023-10-06T03:08:22.550433+0000","last_peered":"2023-10-06T03:08:22.550433+0000","last_clean":"2023-10-06T03:08:22.550433+0000","last_became_active":"2023-10-06T03:08:22.550166+0000","last_became_peered":"2023-10-06T03:08:22.550166+0000","last_unstale":"2023-10-06T03:08:22.550433+0000","last_undegraded":"2023-10-06T03:08:22.550433+0000","last_fullsized":"2023-10-06T03:08:22.550433+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:52.894864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547679+0000","last_change":"2023-10-06T03:08:22.541762+0000","last_active":"2023-10-06T03:08:22.547679+0000","last_peered":"2023-10-06T03:08:22.547679+0000","last_clean":"2023-10-06T03:08:22.547679+0000","last_became_active":"2023-10-06T03:08:22.541553+0000","last_became_peered":"2023-10-06T03:08:22.541553+0000","last_unstale":"2023-10-06T03:08:22.547679+0000","last_undegraded":"2023-10-06T03:08:22.547679+0000","last_fullsized":"2023-10-06T03:08:22.547679+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:03.294297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540961+0000","last_change":"2023-10-06T03:08:22.540961+0000","last_active":"2023-10-06T03:08:22.540961+0000","last_peered":"2023-10-06T03:08:22.540961+0000","last_clean":"2023-10-06T03:08:22.540961+0000","last_became_active":"2023-10-06T03:08:22.540610+0000","last_became_peered":"2023-10-06T03:08:22.540610+0000","last_unstale":"2023-10-06T03:08:22.540961+0000","last_undegraded":"2023-10-06T03:08:22.540961+0000","last_fullsized":"2023-10-06T03:08:22.540961+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:14.063036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535698+0000","last_change":"2023-10-06T03:08:22.535698+0000","last_active":"2023-10-06T03:08:22.535698+0000","last_peered":"2023-10-06T03:08:22.535698+0000","last_clean":"2023-10-06T03:08:22.535698+0000","last_became_active":"2023-10-06T03:08:22.535496+0000","last_became_peered":"2023-10-06T03:08:22.535496+0000","last_unstale":"2023-10-06T03:08:22.535698+0000","last_undegraded":"2023-10-06T03:08:22.535698+0000","last_fullsized":"2023-10-06T03:08:22.535698+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:04.787603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.554634+0000","last_change":"2023-10-06T03:08:22.549828+0000","last_active":"2023-10-06T03:08:22.554634+0000","last_peered":"2023-10-06T03:08:22.554634+0000","last_clean":"2023-10-06T03:08:22.554634+0000","last_became_active":"2023-10-06T03:08:22.549563+0000","last_became_peered":"2023-10-06T03:08:22.549563+0000","last_unstale":"2023-10-06T03:08:22.554634+0000","last_undegraded":"2023-10-06T03:08:22.554634+0000","last_fullsized":"2023-10-06T03:08:22.554634+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:14.063036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543832+0000","last_change":"2023-10-06T03:08:22.537086+0000","last_active":"2023-10-06T03:08:22.543832+0000","last_peered":"2023-10-06T03:08:22.543832+0000","last_clean":"2023-10-06T03:08:22.543832+0000","last_became_active":"2023-10-06T03:08:22.536841+0000","last_became_peered":"2023-10-06T03:08:22.536841+0000","last_unstale":"2023-10-06T03:08:22.543832+0000","last_undegraded":"2023-10-06T03:08:22.543832+0000","last_fullsized":"2023-10-06T03:08:22.543832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:12:22.169025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541146+0000","last_change":"2023-10-06T03:08:22.541146+0000","last_active":"2023-10-06T03:08:22.541146+0000","last_peered":"2023-10-06T03:08:22.541146+0000","last_clean":"2023-10-06T03:08:22.541146+0000","last_became_active":"2023-10-06T03:08:22.540902+0000","last_became_peered":"2023-10-06T03:08:22.540902+0000","last_unstale":"2023-10-06T03:08:22.541146+0000","last_undegraded":"2023-10-06T03:08:22.541146+0000","last_fullsized":"2023-10-06T03:08:22.541146+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:12.931647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543693+0000","last_change":"2023-10-06T03:08:22.539488+0000","last_active":"2023-10-06T03:08:22.543693+0000","last_peered":"2023-10-06T03:08:22.543693+0000","last_clean":"2023-10-06T03:08:22.543693+0000","last_became_active":"2023-10-06T03:08:22.539161+0000","last_became_peered":"2023-10-06T03:08:22.539161+0000","last_unstale":"2023-10-06T03:08:22.543693+0000","last_undegraded":"2023-10-06T03:08:22.543693+0000","last_fullsized":"2023-10-06T03:08:22.543693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:58.391302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543808+0000","last_change":"2023-10-06T03:08:22.539481+0000","last_active":"2023-10-06T03:08:22.543808+0000","last_peered":"2023-10-06T03:08:22.543808+0000","last_clean":"2023-10-06T03:08:22.543808+0000","last_became_active":"2023-10-06T03:08:22.539203+0000","last_became_peered":"2023-10-06T03:08:22.539203+0000","last_unstale":"2023-10-06T03:08:22.543808+0000","last_undegraded":"2023-10-06T03:08:22.543808+0000","last_fullsized":"2023-10-06T03:08:22.543808+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:10.330303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540787+0000","last_change":"2023-10-06T03:08:22.540787+0000","last_active":"2023-10-06T03:08:22.540787+0000","last_peered":"2023-10-06T03:08:22.540787+0000","last_clean":"2023-10-06T03:08:22.540787+0000","last_became_active":"2023-10-06T03:08:22.540501+0000","last_became_peered":"2023-10-06T03:08:22.540501+0000","last_unstale":"2023-10-06T03:08:22.540787+0000","last_undegraded":"2023-10-06T03:08:22.540787+0000","last_fullsized":"2023-10-06T03:08:22.540787+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:30.327476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.536703+0000","last_change":"2023-10-06T03:08:22.536703+0000","last_active":"2023-10-06T03:08:22.536703+0000","last_peered":"2023-10-06T03:08:22.536703+0000","last_clean":"2023-10-06T03:08:22.536703+0000","last_became_active":"2023-10-06T03:08:22.536350+0000","last_became_peered":"2023-10-06T03:08:22.536350+0000","last_unstale":"2023-10-06T03:08:22.536703+0000","last_undegraded":"2023-10-06T03:08:22.536703+0000","last_fullsized":"2023-10-06T03:08:22.536703+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:30.710396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.537015+0000","last_change":"2023-10-06T03:08:22.537015+0000","last_active":"2023-10-06T03:08:22.537015+0000","last_peered":"2023-10-06T03:08:22.537015+0000","last_clean":"2023-10-06T03:08:22.537015+0000","last_became_active":"2023-10-06T03:08:22.536775+0000","last_became_peered":"2023-10-06T03:08:22.536775+0000","last_unstale":"2023-10-06T03:08:22.537015+0000","last_undegraded":"2023-10-06T03:08:22.537015+0000","last_fullsized":"2023-10-06T03:08:22.537015+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:42.103845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548506+0000","last_change":"2023-10-06T03:08:22.548506+0000","last_active":"2023-10-06T03:08:22.548506+0000","last_peered":"2023-10-06T03:08:22.548506+0000","last_clean":"2023-10-06T03:08:22.548506+0000","last_became_active":"2023-10-06T03:08:22.548235+0000","last_became_peered":"2023-10-06T03:08:22.548235+0000","last_unstale":"2023-10-06T03:08:22.548506+0000","last_undegraded":"2023-10-06T03:08:22.548506+0000","last_fullsized":"2023-10-06T03:08:22.548506+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:54:59.944171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.544866+0000","last_change":"2023-10-06T03:08:22.544866+0000","last_active":"2023-10-06T03:08:22.544866+0000","last_peered":"2023-10-06T03:08:22.544866+0000","last_clean":"2023-10-06T03:08:22.544866+0000","last_became_active":"2023-10-06T03:08:22.544621+0000","last_became_peered":"2023-10-06T03:08:22.544621+0000","last_unstale":"2023-10-06T03:08:22.544866+0000","last_undegraded":"2023-10-06T03:08:22.544866+0000","last_fullsized":"2023-10-06T03:08:22.544866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:21.390611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.552192+0000","last_change":"2023-10-06T03:08:22.546634+0000","last_active":"2023-10-06T03:08:22.552192+0000","last_peered":"2023-10-06T03:08:22.552192+0000","last_clean":"2023-10-06T03:08:22.552192+0000","last_became_active":"2023-10-06T03:08:22.546303+0000","last_became_peered":"2023-10-06T03:08:22.546303+0000","last_unstale":"2023-10-06T03:08:22.552192+0000","last_undegraded":"2023-10-06T03:08:22.552192+0000","last_fullsized":"2023-10-06T03:08:22.552192+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.699820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543711+0000","last_change":"2023-10-06T03:08:22.539656+0000","last_active":"2023-10-06T03:08:22.543711+0000","last_peered":"2023-10-06T03:08:22.543711+0000","last_clean":"2023-10-06T03:08:22.543711+0000","last_became_active":"2023-10-06T03:08:22.539418+0000","last_became_peered":"2023-10-06T03:08:22.539418+0000","last_unstale":"2023-10-06T03:08:22.543711+0000","last_undegraded":"2023-10-06T03:08:22.543711+0000","last_fullsized":"2023-10-06T03:08:22.543711+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:15.295825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.552183+0000","last_change":"2023-10-06T03:08:22.546420+0000","last_active":"2023-10-06T03:08:22.552183+0000","last_peered":"2023-10-06T03:08:22.552183+0000","last_clean":"2023-10-06T03:08:22.552183+0000","last_became_active":"2023-10-06T03:08:22.546126+0000","last_became_peered":"2023-10-06T03:08:22.546126+0000","last_unstale":"2023-10-06T03:08:22.552183+0000","last_undegraded":"2023-10-06T03:08:22.552183+0000","last_fullsized":"2023-10-06T03:08:22.552183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:07.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540952+0000","last_change":"2023-10-06T03:08:22.540952+0000","last_active":"2023-10-06T03:08:22.540952+0000","last_peered":"2023-10-06T03:08:22.540952+0000","last_clean":"2023-10-06T03:08:22.540952+0000","last_became_active":"2023-10-06T03:08:22.540599+0000","last_became_peered":"2023-10-06T03:08:22.540599+0000","last_unstale":"2023-10-06T03:08:22.540952+0000","last_undegraded":"2023-10-06T03:08:22.540952+0000","last_fullsized":"2023-10-06T03:08:22.540952+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:30.556468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.545041+0000","last_change":"2023-10-06T03:08:22.545041+0000","last_active":"2023-10-06T03:08:22.545041+0000","last_peered":"2023-10-06T03:08:22.545041+0000","last_clean":"2023-10-06T03:08:22.545041+0000","last_became_active":"2023-10-06T03:08:22.544714+0000","last_became_peered":"2023-10-06T03:08:22.544714+0000","last_unstale":"2023-10-06T03:08:22.545041+0000","last_undegraded":"2023-10-06T03:08:22.545041+0000","last_fullsized":"2023-10-06T03:08:22.545041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:26.910097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.545188+0000","last_change":"2023-10-06T03:08:22.545188+0000","last_active":"2023-10-06T03:08:22.545188+0000","last_peered":"2023-10-06T03:08:22.545188+0000","last_clean":"2023-10-06T03:08:22.545188+0000","last_became_active":"2023-10-06T03:08:22.544899+0000","last_became_peered":"2023-10-06T03:08:22.544899+0000","last_unstale":"2023-10-06T03:08:22.545188+0000","last_undegraded":"2023-10-06T03:08:22.545188+0000","last_fullsized":"2023-10-06T03:08:22.545188+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:01.225765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.551043+0000","last_change":"2023-10-06T03:08:22.551043+0000","last_active":"2023-10-06T03:08:22.551043+0000","last_peered":"2023-10-06T03:08:22.551043+0000","last_clean":"2023-10-06T03:08:22.551043+0000","last_became_active":"2023-10-06T03:08:22.550788+0000","last_became_peered":"2023-10-06T03:08:22.550788+0000","last_unstale":"2023-10-06T03:08:22.551043+0000","last_undegraded":"2023-10-06T03:08:22.551043+0000","last_fullsized":"2023-10-06T03:08:22.551043+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:28.345775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533655+0000","last_change":"2023-10-06T03:08:22.533655+0000","last_active":"2023-10-06T03:08:22.533655+0000","last_peered":"2023-10-06T03:08:22.533655+0000","last_clean":"2023-10-06T03:08:22.533655+0000","last_became_active":"2023-10-06T03:08:22.533423+0000","last_became_peered":"2023-10-06T03:08:22.533423+0000","last_unstale":"2023-10-06T03:08:22.533655+0000","last_undegraded":"2023-10-06T03:08:22.533655+0000","last_fullsized":"2023-10-06T03:08:22.533655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:13:48.467391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543614+0000","last_change":"2023-10-06T03:08:22.543614+0000","last_active":"2023-10-06T03:08:22.543614+0000","last_peered":"2023-10-06T03:08:22.543614+0000","last_clean":"2023-10-06T03:08:22.543614+0000","last_became_active":"2023-10-06T03:08:22.543342+0000","last_became_peered":"2023-10-06T03:08:22.543342+0000","last_unstale":"2023-10-06T03:08:22.543614+0000","last_undegraded":"2023-10-06T03:08:22.543614+0000","last_fullsized":"2023-10-06T03:08:22.543614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:30.556468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547334+0000","last_change":"2023-10-06T03:08:22.547334+0000","last_active":"2023-10-06T03:08:22.547334+0000","last_peered":"2023-10-06T03:08:22.547334+0000","last_clean":"2023-10-06T03:08:22.547334+0000","last_became_active":"2023-10-06T03:08:22.547010+0000","last_became_peered":"2023-10-06T03:08:22.547010+0000","last_unstale":"2023-10-06T03:08:22.547334+0000","last_undegraded":"2023-10-06T03:08:22.547334+0000","last_fullsized":"2023-10-06T03:08:22.547334+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.699820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547743+0000","last_change":"2023-10-06T03:08:22.543425+0000","last_active":"2023-10-06T03:08:22.547743+0000","last_peered":"2023-10-06T03:08:22.547743+0000","last_clean":"2023-10-06T03:08:22.547743+0000","last_became_active":"2023-10-06T03:08:22.543125+0000","last_became_peered":"2023-10-06T03:08:22.543125+0000","last_unstale":"2023-10-06T03:08:22.547743+0000","last_undegraded":"2023-10-06T03:08:22.547743+0000","last_fullsized":"2023-10-06T03:08:22.547743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:50.584290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543009+0000","last_change":"2023-10-06T03:08:22.543009+0000","last_active":"2023-10-06T03:08:22.543009+0000","last_peered":"2023-10-06T03:08:22.543009+0000","last_clean":"2023-10-06T03:08:22.543009+0000","last_became_active":"2023-10-06T03:08:22.542752+0000","last_became_peered":"2023-10-06T03:08:22.542752+0000","last_unstale":"2023-10-06T03:08:22.543009+0000","last_undegraded":"2023-10-06T03:08:22.543009+0000","last_fullsized":"2023-10-06T03:08:22.543009+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:06.190251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548048+0000","last_change":"2023-10-06T03:08:22.545113+0000","last_active":"2023-10-06T03:08:22.548048+0000","last_peered":"2023-10-06T03:08:22.548048+0000","last_clean":"2023-10-06T03:08:22.548048+0000","last_became_active":"2023-10-06T03:08:22.544871+0000","last_became_peered":"2023-10-06T03:08:22.544871+0000","last_unstale":"2023-10-06T03:08:22.548048+0000","last_undegraded":"2023-10-06T03:08:22.548048+0000","last_fullsized":"2023-10-06T03:08:22.548048+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:05.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"114'4","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.957614+0000","last_change":"2023-10-06T03:08:21.966064+0000","last_active":"2023-10-06T03:08:22.957614+0000","last_peered":"2023-10-06T03:08:22.957614+0000","last_clean":"2023-10-06T03:08:22.957614+0000","last_became_active":"2023-10-06T03:08:21.965780+0000","last_became_peered":"2023-10-06T03:08:21.965780+0000","last_unstale":"2023-10-06T03:08:22.957614+0000","last_undegraded":"2023-10-06T03:08:22.957614+0000","last_fullsized":"2023-10-06T03:08:22.957614+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:52.927746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543790+0000","last_change":"2023-10-06T03:08:22.538002+0000","last_active":"2023-10-06T03:08:22.543790+0000","last_peered":"2023-10-06T03:08:22.543790+0000","last_clean":"2023-10-06T03:08:22.543790+0000","last_became_active":"2023-10-06T03:08:22.537736+0000","last_became_peered":"2023-10-06T03:08:22.537736+0000","last_unstale":"2023-10-06T03:08:22.543790+0000","last_undegraded":"2023-10-06T03:08:22.543790+0000","last_fullsized":"2023-10-06T03:08:22.543790+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:45:57.700996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"114'1","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530918+0000","last_change":"2023-10-06T03:08:21.966086+0000","last_active":"2023-10-06T03:08:22.530918+0000","last_peered":"2023-10-06T03:08:22.530918+0000","last_clean":"2023-10-06T03:08:22.530918+0000","last_became_active":"2023-10-06T03:08:21.965818+0000","last_became_peered":"2023-10-06T03:08:21.965818+0000","last_unstale":"2023-10-06T03:08:22.530918+0000","last_undegraded":"2023-10-06T03:08:22.530918+0000","last_fullsized":"2023-10-06T03:08:22.530918+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:00.218403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.554573+0000","last_change":"2023-10-06T03:08:22.549139+0000","last_active":"2023-10-06T03:08:22.554573+0000","last_peered":"2023-10-06T03:08:22.554573+0000","last_clean":"2023-10-06T03:08:22.554573+0000","last_became_active":"2023-10-06T03:08:22.548873+0000","last_became_peered":"2023-10-06T03:08:22.548873+0000","last_unstale":"2023-10-06T03:08:22.554573+0000","last_undegraded":"2023-10-06T03:08:22.554573+0000","last_fullsized":"2023-10-06T03:08:22.554573+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:28.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"114'2","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.957520+0000","last_change":"2023-10-06T03:08:21.967520+0000","last_active":"2023-10-06T03:08:22.957520+0000","last_peered":"2023-10-06T03:08:22.957520+0000","last_clean":"2023-10-06T03:08:22.957520+0000","last_became_active":"2023-10-06T03:08:21.967258+0000","last_became_peered":"2023-10-06T03:08:21.967258+0000","last_unstale":"2023-10-06T03:08:22.957520+0000","last_undegraded":"2023-10-06T03:08:22.957520+0000","last_fullsized":"2023-10-06T03:08:22.957520+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:59.697005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.543743+0000","last_change":"2023-10-06T03:08:22.539474+0000","last_active":"2023-10-06T03:08:22.543743+0000","last_peered":"2023-10-06T03:08:22.543743+0000","last_clean":"2023-10-06T03:08:22.543743+0000","last_became_active":"2023-10-06T03:08:22.539195+0000","last_became_peered":"2023-10-06T03:08:22.539195+0000","last_unstale":"2023-10-06T03:08:22.543743+0000","last_undegraded":"2023-10-06T03:08:22.543743+0000","last_fullsized":"2023-10-06T03:08:22.543743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:21.633618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"114'2","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.958209+0000","last_change":"2023-10-06T03:08:21.965827+0000","last_active":"2023-10-06T03:08:22.958209+0000","last_peered":"2023-10-06T03:08:22.958209+0000","last_clean":"2023-10-06T03:08:22.958209+0000","last_became_active":"2023-10-06T03:08:21.965549+0000","last_became_peered":"2023-10-06T03:08:21.965549+0000","last_unstale":"2023-10-06T03:08:22.958209+0000","last_undegraded":"2023-10-06T03:08:22.958209+0000","last_fullsized":"2023-10-06T03:08:22.958209+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:43.919361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547825+0000","last_change":"2023-10-06T03:08:22.543353+0000","last_active":"2023-10-06T03:08:22.547825+0000","last_peered":"2023-10-06T03:08:22.547825+0000","last_clean":"2023-10-06T03:08:22.547825+0000","last_became_active":"2023-10-06T03:08:22.543068+0000","last_became_peered":"2023-10-06T03:08:22.543068+0000","last_unstale":"2023-10-06T03:08:22.547825+0000","last_undegraded":"2023-10-06T03:08:22.547825+0000","last_fullsized":"2023-10-06T03:08:22.547825+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:51.911342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.957899+0000","last_change":"2023-10-06T03:08:21.962167+0000","last_active":"2023-10-06T03:08:22.957899+0000","last_peered":"2023-10-06T03:08:22.957899+0000","last_clean":"2023-10-06T03:08:22.957899+0000","last_became_active":"2023-10-06T03:08:21.961916+0000","last_became_peered":"2023-10-06T03:08:21.961916+0000","last_unstale":"2023-10-06T03:08:22.957899+0000","last_undegraded":"2023-10-06T03:08:22.957899+0000","last_fullsized":"2023-10-06T03:08:22.957899+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:19.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535832+0000","last_change":"2023-10-06T03:08:22.533177+0000","last_active":"2023-10-06T03:08:22.535832+0000","last_peered":"2023-10-06T03:08:22.535832+0000","last_clean":"2023-10-06T03:08:22.535832+0000","last_became_active":"2023-10-06T03:08:22.532885+0000","last_became_peered":"2023-10-06T03:08:22.532885+0000","last_unstale":"2023-10-06T03:08:22.535832+0000","last_undegraded":"2023-10-06T03:08:22.535832+0000","last_fullsized":"2023-10-06T03:08:22.535832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:59.993003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"114'1","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.528264+0000","last_change":"2023-10-06T03:08:21.961877+0000","last_active":"2023-10-06T03:08:22.528264+0000","last_peered":"2023-10-06T03:08:22.528264+0000","last_clean":"2023-10-06T03:08:22.528264+0000","last_became_active":"2023-10-06T03:08:21.961619+0000","last_became_peered":"2023-10-06T03:08:21.961619+0000","last_unstale":"2023-10-06T03:08:22.528264+0000","last_undegraded":"2023-10-06T03:08:22.528264+0000","last_fullsized":"2023-10-06T03:08:22.528264+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:22.765332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548228+0000","last_change":"2023-10-06T03:08:22.548228+0000","last_active":"2023-10-06T03:08:22.548228+0000","last_peered":"2023-10-06T03:08:22.548228+0000","last_clean":"2023-10-06T03:08:22.548228+0000","last_became_active":"2023-10-06T03:08:22.548038+0000","last_became_peered":"2023-10-06T03:08:22.548038+0000","last_unstale":"2023-10-06T03:08:22.548228+0000","last_undegraded":"2023-10-06T03:08:22.548228+0000","last_fullsized":"2023-10-06T03:08:22.548228+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:05.056493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965377+0000","last_change":"2023-10-06T03:08:21.962654+0000","last_active":"2023-10-06T03:08:21.965377+0000","last_peered":"2023-10-06T03:08:21.965377+0000","last_clean":"2023-10-06T03:08:21.965377+0000","last_became_active":"2023-10-06T03:08:21.962399+0000","last_became_peered":"2023-10-06T03:08:21.962399+0000","last_unstale":"2023-10-06T03:08:21.965377+0000","last_undegraded":"2023-10-06T03:08:21.965377+0000","last_fullsized":"2023-10-06T03:08:21.965377+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:17.643984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.551179+0000","last_change":"2023-10-06T03:08:22.551179+0000","last_active":"2023-10-06T03:08:22.551179+0000","last_peered":"2023-10-06T03:08:22.551179+0000","last_clean":"2023-10-06T03:08:22.551179+0000","last_became_active":"2023-10-06T03:08:22.550937+0000","last_became_peered":"2023-10-06T03:08:22.550937+0000","last_unstale":"2023-10-06T03:08:22.551179+0000","last_undegraded":"2023-10-06T03:08:22.551179+0000","last_fullsized":"2023-10-06T03:08:22.551179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:35.118953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.968404+0000","last_change":"2023-10-06T03:08:21.963880+0000","last_active":"2023-10-06T03:08:21.968404+0000","last_peered":"2023-10-06T03:08:21.968404+0000","last_clean":"2023-10-06T03:08:21.968404+0000","last_became_active":"2023-10-06T03:08:21.962883+0000","last_became_peered":"2023-10-06T03:08:21.962883+0000","last_unstale":"2023-10-06T03:08:21.968404+0000","last_undegraded":"2023-10-06T03:08:21.968404+0000","last_fullsized":"2023-10-06T03:08:21.968404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:41:03.153922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547257+0000","last_change":"2023-10-06T03:08:22.547257+0000","last_active":"2023-10-06T03:08:22.547257+0000","last_peered":"2023-10-06T03:08:22.547257+0000","last_clean":"2023-10-06T03:08:22.547257+0000","last_became_active":"2023-10-06T03:08:22.546975+0000","last_became_peered":"2023-10-06T03:08:22.546975+0000","last_unstale":"2023-10-06T03:08:22.547257+0000","last_undegraded":"2023-10-06T03:08:22.547257+0000","last_fullsized":"2023-10-06T03:08:22.547257+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:30.327476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"114'1","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.956741+0000","last_change":"2023-10-06T03:08:21.961399+0000","last_active":"2023-10-06T03:08:22.956741+0000","last_peered":"2023-10-06T03:08:22.956741+0000","last_clean":"2023-10-06T03:08:22.956741+0000","last_became_active":"2023-10-06T03:08:21.961141+0000","last_became_peered":"2023-10-06T03:08:21.961141+0000","last_unstale":"2023-10-06T03:08:22.956741+0000","last_undegraded":"2023-10-06T03:08:22.956741+0000","last_fullsized":"2023-10-06T03:08:22.956741+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:46.620477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547950+0000","last_change":"2023-10-06T03:08:22.547950+0000","last_active":"2023-10-06T03:08:22.547950+0000","last_peered":"2023-10-06T03:08:22.547950+0000","last_clean":"2023-10-06T03:08:22.547950+0000","last_became_active":"2023-10-06T03:08:22.547595+0000","last_became_peered":"2023-10-06T03:08:22.547595+0000","last_unstale":"2023-10-06T03:08:22.547950+0000","last_undegraded":"2023-10-06T03:08:22.547950+0000","last_fullsized":"2023-10-06T03:08:22.547950+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:43.735368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"114'2","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.956060+0000","last_change":"2023-10-06T03:08:21.965741+0000","last_active":"2023-10-06T03:08:22.956060+0000","last_peered":"2023-10-06T03:08:22.956060+0000","last_clean":"2023-10-06T03:08:22.956060+0000","last_became_active":"2023-10-06T03:08:21.965467+0000","last_became_peered":"2023-10-06T03:08:21.965467+0000","last_unstale":"2023-10-06T03:08:22.956060+0000","last_undegraded":"2023-10-06T03:08:22.956060+0000","last_fullsized":"2023-10-06T03:08:22.956060+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:01.099653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.551055+0000","last_change":"2023-10-06T03:08:22.551055+0000","last_active":"2023-10-06T03:08:22.551055+0000","last_peered":"2023-10-06T03:08:22.551055+0000","last_clean":"2023-10-06T03:08:22.551055+0000","last_became_active":"2023-10-06T03:08:22.550783+0000","last_became_peered":"2023-10-06T03:08:22.550783+0000","last_unstale":"2023-10-06T03:08:22.551055+0000","last_undegraded":"2023-10-06T03:08:22.551055+0000","last_fullsized":"2023-10-06T03:08:22.551055+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:22:26.731510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"114'1","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530864+0000","last_change":"2023-10-06T03:08:21.962321+0000","last_active":"2023-10-06T03:08:22.530864+0000","last_peered":"2023-10-06T03:08:22.530864+0000","last_clean":"2023-10-06T03:08:22.530864+0000","last_became_active":"2023-10-06T03:08:21.962038+0000","last_became_peered":"2023-10-06T03:08:21.962038+0000","last_unstale":"2023-10-06T03:08:22.530864+0000","last_undegraded":"2023-10-06T03:08:22.530864+0000","last_fullsized":"2023-10-06T03:08:22.530864+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:44.826974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541640+0000","last_change":"2023-10-06T03:08:22.541640+0000","last_active":"2023-10-06T03:08:22.541640+0000","last_peered":"2023-10-06T03:08:22.541640+0000","last_clean":"2023-10-06T03:08:22.541640+0000","last_became_active":"2023-10-06T03:08:22.541318+0000","last_became_peered":"2023-10-06T03:08:22.541318+0000","last_unstale":"2023-10-06T03:08:22.541640+0000","last_undegraded":"2023-10-06T03:08:22.541640+0000","last_fullsized":"2023-10-06T03:08:22.541640+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:09.772795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.966658+0000","last_change":"2023-10-06T03:08:21.964674+0000","last_active":"2023-10-06T03:08:21.966658+0000","last_peered":"2023-10-06T03:08:21.966658+0000","last_clean":"2023-10-06T03:08:21.966658+0000","last_became_active":"2023-10-06T03:08:21.963621+0000","last_became_peered":"2023-10-06T03:08:21.963621+0000","last_unstale":"2023-10-06T03:08:21.966658+0000","last_undegraded":"2023-10-06T03:08:21.966658+0000","last_fullsized":"2023-10-06T03:08:21.966658+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:29.549106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.536183+0000","last_change":"2023-10-06T03:08:22.536183+0000","last_active":"2023-10-06T03:08:22.536183+0000","last_peered":"2023-10-06T03:08:22.536183+0000","last_clean":"2023-10-06T03:08:22.536183+0000","last_became_active":"2023-10-06T03:08:22.535892+0000","last_became_peered":"2023-10-06T03:08:22.535892+0000","last_unstale":"2023-10-06T03:08:22.536183+0000","last_undegraded":"2023-10-06T03:08:22.536183+0000","last_fullsized":"2023-10-06T03:08:22.536183+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:00.704367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960258+0000","last_change":"2023-10-06T03:08:21.966576+0000","last_active":"2023-10-06T03:08:22.960258+0000","last_peered":"2023-10-06T03:08:22.960258+0000","last_clean":"2023-10-06T03:08:22.960258+0000","last_became_active":"2023-10-06T03:08:21.966245+0000","last_became_peered":"2023-10-06T03:08:21.966245+0000","last_unstale":"2023-10-06T03:08:22.960258+0000","last_undegraded":"2023-10-06T03:08:22.960258+0000","last_fullsized":"2023-10-06T03:08:22.960258+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:20.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.546719+0000","last_change":"2023-10-06T03:08:22.544826+0000","last_active":"2023-10-06T03:08:22.546719+0000","last_peered":"2023-10-06T03:08:22.546719+0000","last_clean":"2023-10-06T03:08:22.546719+0000","last_became_active":"2023-10-06T03:08:22.544581+0000","last_became_peered":"2023-10-06T03:08:22.544581+0000","last_unstale":"2023-10-06T03:08:22.546719+0000","last_undegraded":"2023-10-06T03:08:22.546719+0000","last_fullsized":"2023-10-06T03:08:22.546719+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:59.393124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.526752+0000","last_change":"2023-10-06T03:08:21.962597+0000","last_active":"2023-10-06T03:08:22.526752+0000","last_peered":"2023-10-06T03:08:22.526752+0000","last_clean":"2023-10-06T03:08:22.526752+0000","last_became_active":"2023-10-06T03:08:21.960988+0000","last_became_peered":"2023-10-06T03:08:21.960988+0000","last_unstale":"2023-10-06T03:08:22.526752+0000","last_undegraded":"2023-10-06T03:08:22.526752+0000","last_fullsized":"2023-10-06T03:08:22.526752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:08.586968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540338+0000","last_change":"2023-10-06T03:08:22.540338+0000","last_active":"2023-10-06T03:08:22.540338+0000","last_peered":"2023-10-06T03:08:22.540338+0000","last_clean":"2023-10-06T03:08:22.540338+0000","last_became_active":"2023-10-06T03:08:22.540090+0000","last_became_peered":"2023-10-06T03:08:22.540090+0000","last_unstale":"2023-10-06T03:08:22.540338+0000","last_undegraded":"2023-10-06T03:08:22.540338+0000","last_fullsized":"2023-10-06T03:08:22.540338+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:50.584290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.973708+0000","last_change":"2023-10-06T03:08:21.967431+0000","last_active":"2023-10-06T03:08:21.973708+0000","last_peered":"2023-10-06T03:08:21.973708+0000","last_clean":"2023-10-06T03:08:21.973708+0000","last_became_active":"2023-10-06T03:08:21.966991+0000","last_became_peered":"2023-10-06T03:08:21.966991+0000","last_unstale":"2023-10-06T03:08:21.973708+0000","last_undegraded":"2023-10-06T03:08:21.973708+0000","last_fullsized":"2023-10-06T03:08:21.973708+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:11.924285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.554658+0000","last_change":"2023-10-06T03:08:22.549803+0000","last_active":"2023-10-06T03:08:22.554658+0000","last_peered":"2023-10-06T03:08:22.554658+0000","last_clean":"2023-10-06T03:08:22.554658+0000","last_became_active":"2023-10-06T03:08:22.549530+0000","last_became_peered":"2023-10-06T03:08:22.549530+0000","last_unstale":"2023-10-06T03:08:22.554658+0000","last_undegraded":"2023-10-06T03:08:22.554658+0000","last_fullsized":"2023-10-06T03:08:22.554658+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:20.537385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965457+0000","last_change":"2023-10-06T03:08:21.963695+0000","last_active":"2023-10-06T03:08:21.965457+0000","last_peered":"2023-10-06T03:08:21.965457+0000","last_clean":"2023-10-06T03:08:21.965457+0000","last_became_active":"2023-10-06T03:08:21.963441+0000","last_became_peered":"2023-10-06T03:08:21.963441+0000","last_unstale":"2023-10-06T03:08:21.965457+0000","last_undegraded":"2023-10-06T03:08:21.965457+0000","last_fullsized":"2023-10-06T03:08:21.965457+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:28.573484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548318+0000","last_change":"2023-10-06T03:08:22.548318+0000","last_active":"2023-10-06T03:08:22.548318+0000","last_peered":"2023-10-06T03:08:22.548318+0000","last_clean":"2023-10-06T03:08:22.548318+0000","last_became_active":"2023-10-06T03:08:22.548077+0000","last_became_peered":"2023-10-06T03:08:22.548077+0000","last_unstale":"2023-10-06T03:08:22.548318+0000","last_undegraded":"2023-10-06T03:08:22.548318+0000","last_fullsized":"2023-10-06T03:08:22.548318+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:45:08.821026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960404+0000","last_change":"2023-10-06T03:08:21.966590+0000","last_active":"2023-10-06T03:08:22.960404+0000","last_peered":"2023-10-06T03:08:22.960404+0000","last_clean":"2023-10-06T03:08:22.960404+0000","last_became_active":"2023-10-06T03:08:21.966290+0000","last_became_peered":"2023-10-06T03:08:21.966290+0000","last_unstale":"2023-10-06T03:08:22.960404+0000","last_undegraded":"2023-10-06T03:08:22.960404+0000","last_fullsized":"2023-10-06T03:08:22.960404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:34.069486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542066+0000","last_change":"2023-10-06T03:08:22.542066+0000","last_active":"2023-10-06T03:08:22.542066+0000","last_peered":"2023-10-06T03:08:22.542066+0000","last_clean":"2023-10-06T03:08:22.542066+0000","last_became_active":"2023-10-06T03:08:22.541802+0000","last_became_peered":"2023-10-06T03:08:22.541802+0000","last_unstale":"2023-10-06T03:08:22.542066+0000","last_undegraded":"2023-10-06T03:08:22.542066+0000","last_fullsized":"2023-10-06T03:08:22.542066+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:00.335970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530944+0000","last_change":"2023-10-06T03:08:21.966194+0000","last_active":"2023-10-06T03:08:22.530944+0000","last_peered":"2023-10-06T03:08:22.530944+0000","last_clean":"2023-10-06T03:08:22.530944+0000","last_became_active":"2023-10-06T03:08:21.965930+0000","last_became_peered":"2023-10-06T03:08:21.965930+0000","last_unstale":"2023-10-06T03:08:22.530944+0000","last_undegraded":"2023-10-06T03:08:22.530944+0000","last_fullsized":"2023-10-06T03:08:22.530944+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:13.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.550617+0000","last_change":"2023-10-06T03:08:22.550617+0000","last_active":"2023-10-06T03:08:22.550617+0000","last_peered":"2023-10-06T03:08:22.550617+0000","last_clean":"2023-10-06T03:08:22.550617+0000","last_became_active":"2023-10-06T03:08:22.550189+0000","last_became_peered":"2023-10-06T03:08:22.550189+0000","last_unstale":"2023-10-06T03:08:22.550617+0000","last_undegraded":"2023-10-06T03:08:22.550617+0000","last_fullsized":"2023-10-06T03:08:22.550617+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:51.062202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.530395+0000","last_change":"2023-10-06T03:08:21.961487+0000","last_active":"2023-10-06T03:08:22.530395+0000","last_peered":"2023-10-06T03:08:22.530395+0000","last_clean":"2023-10-06T03:08:22.530395+0000","last_became_active":"2023-10-06T03:08:21.961210+0000","last_became_peered":"2023-10-06T03:08:21.961210+0000","last_unstale":"2023-10-06T03:08:22.530395+0000","last_undegraded":"2023-10-06T03:08:22.530395+0000","last_fullsized":"2023-10-06T03:08:22.530395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:11.924285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.534603+0000","last_change":"2023-10-06T03:08:22.534603+0000","last_active":"2023-10-06T03:08:22.534603+0000","last_peered":"2023-10-06T03:08:22.534603+0000","last_clean":"2023-10-06T03:08:22.534603+0000","last_became_active":"2023-10-06T03:08:22.534424+0000","last_became_peered":"2023-10-06T03:08:22.534424+0000","last_unstale":"2023-10-06T03:08:22.534603+0000","last_undegraded":"2023-10-06T03:08:22.534603+0000","last_fullsized":"2023-10-06T03:08:22.534603+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:57.475796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.968875+0000","last_change":"2023-10-06T03:08:21.967575+0000","last_active":"2023-10-06T03:08:21.968875+0000","last_peered":"2023-10-06T03:08:21.968875+0000","last_clean":"2023-10-06T03:08:21.968875+0000","last_became_active":"2023-10-06T03:08:21.967334+0000","last_became_peered":"2023-10-06T03:08:21.967334+0000","last_unstale":"2023-10-06T03:08:21.968875+0000","last_undegraded":"2023-10-06T03:08:21.968875+0000","last_fullsized":"2023-10-06T03:08:21.968875+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:02.695955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.540776+0000","last_change":"2023-10-06T03:08:22.540776+0000","last_active":"2023-10-06T03:08:22.540776+0000","last_peered":"2023-10-06T03:08:22.540776+0000","last_clean":"2023-10-06T03:08:22.540776+0000","last_became_active":"2023-10-06T03:08:22.540513+0000","last_became_peered":"2023-10-06T03:08:22.540513+0000","last_unstale":"2023-10-06T03:08:22.540776+0000","last_undegraded":"2023-10-06T03:08:22.540776+0000","last_fullsized":"2023-10-06T03:08:22.540776+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:02:26.910097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533706+0000","last_change":"2023-10-06T03:08:21.964068+0000","last_active":"2023-10-06T03:08:22.533706+0000","last_peered":"2023-10-06T03:08:22.533706+0000","last_clean":"2023-10-06T03:08:22.533706+0000","last_became_active":"2023-10-06T03:08:21.963741+0000","last_became_peered":"2023-10-06T03:08:21.963741+0000","last_unstale":"2023-10-06T03:08:22.533706+0000","last_undegraded":"2023-10-06T03:08:22.533706+0000","last_fullsized":"2023-10-06T03:08:22.533706+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:17.582565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541418+0000","last_change":"2023-10-06T03:08:22.541418+0000","last_active":"2023-10-06T03:08:22.541418+0000","last_peered":"2023-10-06T03:08:22.541418+0000","last_clean":"2023-10-06T03:08:22.541418+0000","last_became_active":"2023-10-06T03:08:22.541182+0000","last_became_peered":"2023-10-06T03:08:22.541182+0000","last_unstale":"2023-10-06T03:08:22.541418+0000","last_undegraded":"2023-10-06T03:08:22.541418+0000","last_fullsized":"2023-10-06T03:08:22.541418+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:53.699820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.973756+0000","last_change":"2023-10-06T03:08:21.967949+0000","last_active":"2023-10-06T03:08:21.973756+0000","last_peered":"2023-10-06T03:08:21.973756+0000","last_clean":"2023-10-06T03:08:21.973756+0000","last_became_active":"2023-10-06T03:08:21.967689+0000","last_became_peered":"2023-10-06T03:08:21.967689+0000","last_unstale":"2023-10-06T03:08:21.973756+0000","last_undegraded":"2023-10-06T03:08:21.973756+0000","last_fullsized":"2023-10-06T03:08:21.973756+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:41.096483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.544945+0000","last_change":"2023-10-06T03:08:22.544945+0000","last_active":"2023-10-06T03:08:22.544945+0000","last_peered":"2023-10-06T03:08:22.544945+0000","last_clean":"2023-10-06T03:08:22.544945+0000","last_became_active":"2023-10-06T03:08:22.544704+0000","last_became_peered":"2023-10-06T03:08:22.544704+0000","last_unstale":"2023-10-06T03:08:22.544945+0000","last_undegraded":"2023-10-06T03:08:22.544945+0000","last_fullsized":"2023-10-06T03:08:22.544945+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:28.024701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965396+0000","last_change":"2023-10-06T03:08:21.963183+0000","last_active":"2023-10-06T03:08:21.965396+0000","last_peered":"2023-10-06T03:08:21.965396+0000","last_clean":"2023-10-06T03:08:21.965396+0000","last_became_active":"2023-10-06T03:08:21.962930+0000","last_became_peered":"2023-10-06T03:08:21.962930+0000","last_unstale":"2023-10-06T03:08:21.965396+0000","last_undegraded":"2023-10-06T03:08:21.965396+0000","last_fullsized":"2023-10-06T03:08:21.965396+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:51.453402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.550511+0000","last_change":"2023-10-06T03:08:22.550511+0000","last_active":"2023-10-06T03:08:22.550511+0000","last_peered":"2023-10-06T03:08:22.550511+0000","last_clean":"2023-10-06T03:08:22.550511+0000","last_became_active":"2023-10-06T03:08:22.550222+0000","last_became_peered":"2023-10-06T03:08:22.550222+0000","last_unstale":"2023-10-06T03:08:22.550511+0000","last_undegraded":"2023-10-06T03:08:22.550511+0000","last_fullsized":"2023-10-06T03:08:22.550511+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:49.949858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.965257+0000","last_change":"2023-10-06T03:08:21.963258+0000","last_active":"2023-10-06T03:08:21.965257+0000","last_peered":"2023-10-06T03:08:21.965257+0000","last_clean":"2023-10-06T03:08:21.965257+0000","last_became_active":"2023-10-06T03:08:21.962955+0000","last_became_peered":"2023-10-06T03:08:21.962955+0000","last_unstale":"2023-10-06T03:08:21.965257+0000","last_undegraded":"2023-10-06T03:08:21.965257+0000","last_fullsized":"2023-10-06T03:08:21.965257+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:59.328608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.541630+0000","last_change":"2023-10-06T03:08:22.541630+0000","last_active":"2023-10-06T03:08:22.541630+0000","last_peered":"2023-10-06T03:08:22.541630+0000","last_clean":"2023-10-06T03:08:22.541630+0000","last_became_active":"2023-10-06T03:08:22.541374+0000","last_became_peered":"2023-10-06T03:08:22.541374+0000","last_unstale":"2023-10-06T03:08:22.541630+0000","last_undegraded":"2023-10-06T03:08:22.541630+0000","last_fullsized":"2023-10-06T03:08:22.541630+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:07.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.526506+0000","last_change":"2023-10-06T03:08:21.960552+0000","last_active":"2023-10-06T03:08:22.526506+0000","last_peered":"2023-10-06T03:08:22.526506+0000","last_clean":"2023-10-06T03:08:22.526506+0000","last_became_active":"2023-10-06T03:08:21.960290+0000","last_became_peered":"2023-10-06T03:08:21.960290+0000","last_unstale":"2023-10-06T03:08:22.526506+0000","last_undegraded":"2023-10-06T03:08:22.526506+0000","last_fullsized":"2023-10-06T03:08:22.526506+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:10.706495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.962131+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:08:21.962131+0000","last_peered":"2023-10-06T03:08:21.962131+0000","last_clean":"2023-10-06T03:08:21.962131+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:08:21.962131+0000","last_undegraded":"2023-10-06T03:08:21.962131+0000","last_fullsized":"2023-10-06T03:08:21.962131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.548640+0000","last_change":"2023-10-06T03:08:22.547218+0000","last_active":"2023-10-06T03:08:22.548640+0000","last_peered":"2023-10-06T03:08:22.548640+0000","last_clean":"2023-10-06T03:08:22.548640+0000","last_became_active":"2023-10-06T03:08:22.546992+0000","last_became_peered":"2023-10-06T03:08:22.546992+0000","last_unstale":"2023-10-06T03:08:22.548640+0000","last_undegraded":"2023-10-06T03:08:22.548640+0000","last_fullsized":"2023-10-06T03:08:22.548640+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:56:25.036920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533668+0000","last_change":"2023-10-06T03:08:21.964143+0000","last_active":"2023-10-06T03:08:22.533668+0000","last_peered":"2023-10-06T03:08:22.533668+0000","last_clean":"2023-10-06T03:08:22.533668+0000","last_became_active":"2023-10-06T03:08:21.963892+0000","last_became_peered":"2023-10-06T03:08:21.963892+0000","last_unstale":"2023-10-06T03:08:22.533668+0000","last_undegraded":"2023-10-06T03:08:22.533668+0000","last_fullsized":"2023-10-06T03:08:22.533668+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:50.462283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.957037+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:08:21.957037+0000","last_peered":"2023-10-06T03:08:21.957037+0000","last_clean":"2023-10-06T03:08:21.957037+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:08:21.957037+0000","last_undegraded":"2023-10-06T03:08:21.957037+0000","last_fullsized":"2023-10-06T03:08:21.957037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542693+0000","last_change":"2023-10-06T03:08:22.542693+0000","last_active":"2023-10-06T03:08:22.542693+0000","last_peered":"2023-10-06T03:08:22.542693+0000","last_clean":"2023-10-06T03:08:22.542693+0000","last_became_active":"2023-10-06T03:08:22.542433+0000","last_became_peered":"2023-10-06T03:08:22.542433+0000","last_unstale":"2023-10-06T03:08:22.542693+0000","last_undegraded":"2023-10-06T03:08:22.542693+0000","last_fullsized":"2023-10-06T03:08:22.542693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:23.772694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"114'6","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960043+0000","last_change":"2023-10-06T03:08:21.968418+0000","last_active":"2023-10-06T03:08:22.960043+0000","last_peered":"2023-10-06T03:08:22.960043+0000","last_clean":"2023-10-06T03:08:22.960043+0000","last_became_active":"2023-10-06T03:08:21.967240+0000","last_became_peered":"2023-10-06T03:08:21.967240+0000","last_unstale":"2023-10-06T03:08:22.960043+0000","last_undegraded":"2023-10-06T03:08:22.960043+0000","last_fullsized":"2023-10-06T03:08:22.960043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:13.055674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.960966+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:08:21.960966+0000","last_peered":"2023-10-06T03:08:21.960966+0000","last_clean":"2023-10-06T03:08:21.960966+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:08:21.960966+0000","last_undegraded":"2023-10-06T03:08:21.960966+0000","last_fullsized":"2023-10-06T03:08:21.960966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547426+0000","last_change":"2023-10-06T03:08:22.547426+0000","last_active":"2023-10-06T03:08:22.547426+0000","last_peered":"2023-10-06T03:08:22.547426+0000","last_clean":"2023-10-06T03:08:22.547426+0000","last_became_active":"2023-10-06T03:08:22.547153+0000","last_became_peered":"2023-10-06T03:08:22.547153+0000","last_unstale":"2023-10-06T03:08:22.547426+0000","last_undegraded":"2023-10-06T03:08:22.547426+0000","last_fullsized":"2023-10-06T03:08:22.547426+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:32.180596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.969255+0000","last_change":"2023-10-06T03:08:21.966497+0000","last_active":"2023-10-06T03:08:21.969255+0000","last_peered":"2023-10-06T03:08:21.969255+0000","last_clean":"2023-10-06T03:08:21.969255+0000","last_became_active":"2023-10-06T03:08:21.966231+0000","last_became_peered":"2023-10-06T03:08:21.966231+0000","last_unstale":"2023-10-06T03:08:21.969255+0000","last_undegraded":"2023-10-06T03:08:21.969255+0000","last_fullsized":"2023-10-06T03:08:21.969255+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:28.384668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":89,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.959984+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:08:21.959984+0000","last_peered":"2023-10-06T03:08:21.959984+0000","last_clean":"2023-10-06T03:08:21.959984+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:08:21.959984+0000","last_undegraded":"2023-10-06T03:08:21.959984+0000","last_fullsized":"2023-10-06T03:08:21.959984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.547105+0000","last_change":"2023-10-06T03:08:22.547105+0000","last_active":"2023-10-06T03:08:22.547105+0000","last_peered":"2023-10-06T03:08:22.547105+0000","last_clean":"2023-10-06T03:08:22.547105+0000","last_became_active":"2023-10-06T03:08:22.546840+0000","last_became_peered":"2023-10-06T03:08:22.546840+0000","last_unstale":"2023-10-06T03:08:22.547105+0000","last_undegraded":"2023-10-06T03:08:22.547105+0000","last_fullsized":"2023-10-06T03:08:22.547105+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:11.945719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.970831+0000","last_change":"2023-10-06T03:08:21.966435+0000","last_active":"2023-10-06T03:08:21.970831+0000","last_peered":"2023-10-06T03:08:21.970831+0000","last_clean":"2023-10-06T03:08:21.970831+0000","last_became_active":"2023-10-06T03:08:21.966186+0000","last_became_peered":"2023-10-06T03:08:21.966186+0000","last_unstale":"2023-10-06T03:08:21.970831+0000","last_undegraded":"2023-10-06T03:08:21.970831+0000","last_fullsized":"2023-10-06T03:08:21.970831+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:20.507671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":242,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.960431+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:08:21.960431+0000","last_peered":"2023-10-06T03:08:21.960431+0000","last_clean":"2023-10-06T03:08:21.960431+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:08:21.960431+0000","last_undegraded":"2023-10-06T03:08:21.960431+0000","last_fullsized":"2023-10-06T03:08:21.960431+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":100,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533756+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:08:22.533756+0000","last_peered":"2023-10-06T03:08:22.533756+0000","last_clean":"2023-10-06T03:08:22.533756+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:08:22.533756+0000","last_undegraded":"2023-10-06T03:08:22.533756+0000","last_fullsized":"2023-10-06T03:08:22.533756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.542373+0000","last_change":"2023-10-06T03:08:22.542373+0000","last_active":"2023-10-06T03:08:22.542373+0000","last_peered":"2023-10-06T03:08:22.542373+0000","last_clean":"2023-10-06T03:08:22.542373+0000","last_became_active":"2023-10-06T03:08:22.541998+0000","last_became_peered":"2023-10-06T03:08:22.541998+0000","last_unstale":"2023-10-06T03:08:22.542373+0000","last_undegraded":"2023-10-06T03:08:22.542373+0000","last_fullsized":"2023-10-06T03:08:22.542373+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:08.345180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"114'8","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.961308+0000","last_change":"2023-10-06T03:08:21.967181+0000","last_active":"2023-10-06T03:08:22.961308+0000","last_peered":"2023-10-06T03:08:22.961308+0000","last_clean":"2023-10-06T03:08:22.961308+0000","last_became_active":"2023-10-06T03:08:21.966898+0000","last_became_peered":"2023-10-06T03:08:21.966898+0000","last_unstale":"2023-10-06T03:08:22.961308+0000","last_undegraded":"2023-10-06T03:08:22.961308+0000","last_fullsized":"2023-10-06T03:08:22.961308+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:18:00.219437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":91,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.526421+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:08:22.526421+0000","last_peered":"2023-10-06T03:08:22.526421+0000","last_clean":"2023-10-06T03:08:22.526421+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:08:22.526421+0000","last_undegraded":"2023-10-06T03:08:22.526421+0000","last_fullsized":"2023-10-06T03:08:22.526421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.535888+0000","last_change":"2023-10-06T03:08:22.535888+0000","last_active":"2023-10-06T03:08:22.535888+0000","last_peered":"2023-10-06T03:08:22.535888+0000","last_clean":"2023-10-06T03:08:22.535888+0000","last_became_active":"2023-10-06T03:08:22.535635+0000","last_became_peered":"2023-10-06T03:08:22.535635+0000","last_unstale":"2023-10-06T03:08:22.535888+0000","last_undegraded":"2023-10-06T03:08:22.535888+0000","last_fullsized":"2023-10-06T03:08:22.535888+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:17.649792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"114'2","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.956749+0000","last_change":"2023-10-06T03:08:21.960604+0000","last_active":"2023-10-06T03:08:22.956749+0000","last_peered":"2023-10-06T03:08:22.956749+0000","last_clean":"2023-10-06T03:08:22.956749+0000","last_became_active":"2023-10-06T03:08:21.960295+0000","last_became_peered":"2023-10-06T03:08:21.960295+0000","last_unstale":"2023-10-06T03:08:22.956749+0000","last_undegraded":"2023-10-06T03:08:22.956749+0000","last_fullsized":"2023-10-06T03:08:22.956749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:21.730880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":100,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.533507+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:08:22.533507+0000","last_peered":"2023-10-06T03:08:22.533507+0000","last_clean":"2023-10-06T03:08:22.533507+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:08:22.533507+0000","last_undegraded":"2023-10-06T03:08:22.533507+0000","last_fullsized":"2023-10-06T03:08:22.533507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":8,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.539940+0000","last_change":"2023-10-06T03:08:22.539940+0000","last_active":"2023-10-06T03:08:22.539940+0000","last_peered":"2023-10-06T03:08:22.539940+0000","last_clean":"2023-10-06T03:08:22.539940+0000","last_became_active":"2023-10-06T03:08:22.539692+0000","last_became_peered":"2023-10-06T03:08:22.539692+0000","last_unstale":"2023-10-06T03:08:22.539940+0000","last_undegraded":"2023-10-06T03:08:22.539940+0000","last_fullsized":"2023-10-06T03:08:22.539940+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:21.940033+0000","last_clean_scrub_stamp":"2023-10-06T03:08:21.940033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:27.230608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"114'3","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2023-10-06T03:08:22.960441+0000","last_change":"2023-10-06T03:08:21.968958+0000","last_active":"2023-10-06T03:08:22.960441+0000","last_peered":"2023-10-06T03:08:22.960441+0000","last_clean":"2023-10-06T03:08:22.960441+0000","last_became_active":"2023-10-06T03:08:21.968652+0000","last_became_peered":"2023-10-06T03:08:21.968652+0000","last_unstale":"2023-10-06T03:08:22.960441+0000","last_undegraded":"2023-10-06T03:08:22.960441+0000","last_fullsized":"2023-10-06T03:08:22.960441+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:08:20.932671+0000","last_clean_scrub_stamp":"2023-10-06T03:08:20.932671+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:41:19.530023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2023-10-06T03:08:21.957059+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:08:21.957059+0000","last_peered":"2023-10-06T03:08:21.957059+0000","last_clean":"2023-10-06T03:08:21.957059+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:08:21.957059+0000","last_undegraded":"2023-10-06T03:08:21.957059+0000","last_fullsized":"2023-10-06T03:08:21.957059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303372,"kb_used_data":1988,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068468,"statfs":{"total":96636764160,"available":96326111232,"internally_reserved":0,"allocated":2035712,"data_stored":1244018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.70699999999999996},"min":{"1min":0.34399999999999997,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.72999999999999998,"5min":0.76200000000000001,"15min":6.0110000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51100000000000001,"15min":0.51700000000000002},"min":{"1min":0.41199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.69799999999999995,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":0.749,"15min":0.749},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52400000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.875},"last":1.5049999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.81299999999999994,"15min":0.999},"last":1.264},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.90200000000000002},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79000000000000004,"5min":0.83499999999999996,"15min":0.91900000000000004},"last":2.286},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.94599999999999995},"last":1.1890000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.50800000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":4.0289999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.026},"last":2.504}]},{"osd":5,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":1.141},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56999999999999995,"15min":0.55900000000000005},"min":{"1min":0.36699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":1.639}]},{"osd":6,"last update":"Fri Oct 6 03:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.91300000000000003},"last":1.5649999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.60699999999999998,"15min":0.58999999999999997},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":2.3359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302784,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069056,"statfs":{"total":96636764160,"available":96326713344,"internally_reserved":0,"allocated":1433600,"data_stored":652668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49099999999999999,"15min":0.50700000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59499999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.40300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66300000000000003,"5min":0.755,"15min":0.755},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.747,"5min":0.747,"15min":0.79000000000000004},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72599999999999998,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.504,"15min":0.54300000000000004},"min":{"1min":0.32300000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80400000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.70999999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55900000000000005,"15min":0.59899999999999998},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.628},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58799999999999997},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.86799999999999999},"last":0.76300000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298684,"kb_used_data":1396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073156,"statfs":{"total":96636764160,"available":96330911744,"internally_reserved":0,"allocated":1429504,"data_stored":652094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.34899999999999998,"15min":0.33300000000000002},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"min":{"1min":0.29199999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":3.6520000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33300000000000002},"min":{"1min":0.245,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.22900000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.29699999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.33500000000000002,"15min":0.33900000000000002},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.372,"5min":0.36099999999999999,"15min":0.36799999999999999},"min":{"1min":0.29399999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.51600000000000001,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.57999999999999996},"min":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.441},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.40999999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.52900000000000003,"15min":0.56299999999999994},"min":{"1min":0.39800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.85799999999999998,"15min":0.96299999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.629,"15min":0.60299999999999998},"min":{"1min":0.44800000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.363,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.47699999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.496}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40500000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.57699999999999996,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.46899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303364,"kb_used_data":1980,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068476,"statfs":{"total":96636764160,"available":96326119424,"internally_reserved":0,"allocated":2027520,"data_stored":1242448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.307,"15min":0.31},"min":{"1min":0.21299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.217},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.317,"15min":0.32600000000000001},"min":{"1min":0.223,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.44500000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.33600000000000002,"15min":0.33200000000000002},"min":{"1min":0.27100000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.64100000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34300000000000003,"15min":0.34999999999999998},"min":{"1min":0.27700000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.27100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33000000000000002,"15min":0.34100000000000003},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.43099999999999999,"5min":0.52400000000000002,"15min":0.52400000000000002},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.497,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.251}]},{"osd":4,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.871},"last":0.377},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.76100000000000001,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.41899999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58099999999999996,"15min":0.58999999999999997},"min":{"1min":0.41499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82899999999999996,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.5},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.59199999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.72999999999999998,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.39200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.746,"5min":0.83899999999999997,"15min":0.93700000000000006},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79500000000000004,"5min":0.878,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:07:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.91200000000000003,"15min":0.96099999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.61799999999999999},"min":{"1min":0.435,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.69699999999999995,"5min":0.85399999999999998,"15min":0.97999999999999998},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302796,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069044,"statfs":{"total":96636764160,"available":96326701056,"internally_reserved":0,"allocated":1445888,"data_stored":659253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.29699999999999999,"5min":0.30399999999999999,"15min":0.32000000000000001},"min":{"1min":0.214,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.317},"min":{"1min":0.219,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.41399999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.33300000000000002,"15min":0.32800000000000001},"min":{"1min":0.24299999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.375,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34100000000000003,"15min":0.34899999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.41299999999999998,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32800000000000001,"15min":0.32200000000000001},"min":{"1min":0.27700000000000002,"5min":0.222,"15min":0.222},"max":{"1min":0.39900000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.36099999999999999,"15min":0.379},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.39400000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.374}]},{"osd":4,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.748,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.76100000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56699999999999995,"15min":0.51900000000000002},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57299999999999995,"15min":0.54300000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72299999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.55200000000000005},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74299999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72199999999999998,"5min":0.87,"15min":1.8740000000000001},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:07:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.433,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78800000000000003,"5min":0.84999999999999998,"15min":1.903},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.75900000000000001,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.56899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302792,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069048,"statfs":{"total":96636764160,"available":96326705152,"internally_reserved":0,"allocated":1441792,"data_stored":653630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32000000000000001,"5min":0.313,"15min":0.30599999999999999},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.44,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.33400000000000002,"15min":0.33000000000000002},"min":{"1min":0.26600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.26700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.30299999999999999,"15min":0.309},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.41399999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34899999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.40200000000000002,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.35399999999999998,"15min":0.35499999999999998},"min":{"1min":0.248,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.50900000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.315}]},{"osd":4,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56999999999999995,"15min":0.54700000000000004},"min":{"1min":0.38500000000000001,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71999999999999997,"5min":0.997,"15min":0.997},"last":0.52700000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57099999999999995,"15min":0.55100000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71399999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.495},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.56599999999999995,"15min":0.55600000000000005},"min":{"1min":0.40300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.75700000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:07:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67700000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.505},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.747,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302788,"kb_used_data":1404,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069052,"statfs":{"total":96636764160,"available":96326709248,"internally_reserved":0,"allocated":1437696,"data_stored":658665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53300000000000003,"15min":0.501},"min":{"1min":0.40300000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.61799999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":0.78600000000000003,"15min":1.4410000000000001},"last":0.437}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64600000000000002,"5min":0.72399999999999998,"15min":1.0669999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.51800000000000002},"min":{"1min":0.40500000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52600000000000002,"15min":0.58599999999999997},"min":{"1min":0.34599999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.65900000000000003,"5min":0.751,"15min":3},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53200000000000003,"15min":0.79200000000000004},"min":{"1min":0.39200000000000002,"5min":0.34999999999999998,"15min":0.34399999999999997},"max":{"1min":0.70299999999999996,"5min":0.72099999999999997,"15min":10.084},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.68700000000000006},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.51400000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.83999999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.58399999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":1.54},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53200000000000003,"15min":0.54700000000000004},"min":{"1min":0.36599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.88200000000000001,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54400000000000004,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.58899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58799999999999997,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302792,"kb_used_data":1408,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069048,"statfs":{"total":96636764160,"available":96326705152,"internally_reserved":0,"allocated":1441792,"data_stored":658687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.143,"5min":1.143,"15min":2.3599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":2.371},"last":0.54700000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.496,"15min":0.54300000000000004},"min":{"1min":0.34899999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":4.1219999999999999,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.58499999999999996,"15min":0.60699999999999998},"min":{"1min":0.38200000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":6.532,"5min":6.532,"15min":6.532},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.58999999999999997,"15min":0.73099999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39100000000000001},"max":{"1min":5.524,"5min":5.524,"15min":5.524},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49399999999999999,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.58699999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.498}]},{"osd":6,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.56599999999999995},"min":{"1min":0.32400000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78200000000000003,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.63200000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.052},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.63200000000000001},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.86799999999999999,"15min":1.175},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.61499999999999999},"min":{"1min":0.38800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75600000000000001,"5min":0.77500000000000002,"15min":1.145},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:08:28.388 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:08:28.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:08:28.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.541+0000 7f818a975700 1 Processor -- start 2023-10-06T03:08:28.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.541+0000 7f818a975700 1 -- start start 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f818a975700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f81840cbb90 0x7f81840af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f818a975700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81840af860 con 0x7f8184069c70 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f818a975700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81840ce380 con 0x7f8184159950 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f818a975700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840cec20 con 0x7f81840cbb90 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f81837fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8184069c70 0x7f81840cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37568/0 (socket says 172.21.15.92:37568) 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f81837fe700 1 -- 172.21.15.92:0/3912456212 learned_addr learned my addr 172.21.15.92:0/3912456212 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.542+0000 7f8188f12700 1 --2- 172.21.15.92:0/3912456212 >> v2:172.21.15.136:3300/0 conn(0x7f81840cbb90 0x7f81840af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122750638 0 0) 0x7f81840af860 con 0x7f8184069c70 2023-10-06T03:08:28.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f816c003570 con 0x7f8184069c70 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 276082757 0 0) 0x7f816c003570 con 0x7f8184069c70 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 >> v2:172.21.15.136:3300/0 conn(0x7f81840cbb90 msgr2=0x7f81840af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 --2- 172.21.15.92:0/3912456212 >> v2:172.21.15.136:3300/0 conn(0x7f81840cbb90 0x7f81840af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 >> v1:172.21.15.136:6789/0 conn(0x7f8184159950 legacy=0x7f8184069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840cbfb0 con 0x7f8184069c70 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8178001470 con 0x7f8184069c70 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 >> v1:172.21.15.92:6789/0 conn(0x7f8184069c70 legacy=0x7f81840cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c003a60 0x7f816c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c004810 0x7f816c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f816c0051c0 0x7f816c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840cec20 con 0x7f816c0051c0 2023-10-06T03:08:28.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c00c160 con 0x7f816c004810 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c00c320 con 0x7f816c003a60 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f816c0051c0 0x7f816c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f81837fe700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c004810 0x7f816c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8183fff700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c003a60 0x7f816c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 -- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c003a60 msgr2=0x7f816c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c003a60 0x7f816c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 -- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c004810 msgr2=0x7f816c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c004810 0x7f816c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 -- 172.21.15.92:0/3912456212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c003570 con 0x7f816c0051c0 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 -- 172.21.15.92:0/3912456212 reap_dead start 2023-10-06T03:08:28.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.543+0000 7f8188f12700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f816c0051c0 0x7f816c00ba60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e210 tx=0x7f817c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aa12731e9cf7d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00eac0 con 0x7f816c0051c0 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00f040 con 0x7f816c0051c0 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f8182ffd700 1 -- 172.21.15.92:0/3912456212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00ee70 con 0x7f816c0051c0 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 -- 172.21.15.92:0/3912456212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f816c0051c0 msgr2=0x7f816c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f816c0051c0 0x7f816c00ba60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e210 tx=0x7f817c00b040 comp rx=0 tx=0).stop 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 -- 172.21.15.92:0/3912456212 shutdown_connections 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 --2- 172.21.15.92:0/3912456212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f816c0051c0 0x7f816c00ba60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 -- 172.21.15.92:0/3912456212 >> 172.21.15.92:0/3912456212 conn(0x7f8184068e00 msgr2=0x7f81840d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 -- 172.21.15.92:0/3912456212 shutdown_connections 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.544+0000 7f818a975700 1 -- 172.21.15.92:0/3912456212 wait complete. 2023-10-06T03:08:28.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.545+0000 7f818a975700 1 Processor -- start 2023-10-06T03:08:28.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.545+0000 7f818a975700 1 -- start start 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f818a975700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c0051c0 0x7f8184169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f818a975700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 0x7f8184158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f818a975700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f81841560f0 0x7f8184156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f818a975700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c9b80 con 0x7f8184158120 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f818a975700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184158ad0 con 0x7f816c0051c0 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f818a975700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8184156ab0 con 0x7f81841560f0 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 0x7f8184158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 0x7f8184158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38120/0 (socket says 172.21.15.92:38120) 2023-10-06T03:08:28.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 -- 172.21.15.92:0/661724320 learned_addr learned my addr 172.21.15.92:0/661724320 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f8188f12700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f81841560f0 0x7f8184156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f8183fff700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c0051c0 0x7f8184169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 -- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f81841560f0 msgr2=0x7f8184156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f81841560f0 0x7f8184156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 -- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c0051c0 msgr2=0x7f8184169c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c0051c0 0x7f8184169c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.546+0000 7f81837fe700 1 -- 172.21.15.92:0/661724320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8184156d20 con 0x7f8184158120 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.547+0000 7f81837fe700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 0x7f8184158590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8178001050 tx=0x7f81840ced60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.547+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178007710 con 0x7f8184158120 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.547+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817800fa50 con 0x7f8184158120 2023-10-06T03:08:28.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.547+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8184157000 con 0x7f8184158120 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.547+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817800fe10 con 0x7f8184158120 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.547+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81841575c0 con 0x7f8184158120 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.548+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81841591e0 con 0x7f8184158120 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.548+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8178013600 con 0x7f8184158120 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.548+0000 7f816bfff700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8170067320 0x7f81700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.549+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8178084de0 con 0x7f8184158120 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.549+0000 7f8183fff700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8170067320 0x7f81700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.549+0000 7f8183fff700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8170067320 0x7f81700697d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8174007e40 tx=0x7f8174007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.553+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8178051ae0 con 0x7f8184158120 2023-10-06T03:08:28.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.808+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f81840718b0 con 0x7f8184158120 2023-10-06T03:08:28.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.812+0000 7f816bfff700 1 -- 172.21.15.92:0/661724320 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8178055100 con 0x7f8184158120 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8170067320 msgr2=0x7f81700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8170067320 0x7f81700697d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8174007e40 tx=0x7f8174007d30 comp rx=0 tx=0).stop 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 msgr2=0x7f8184158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 0x7f8184158590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8178001050 tx=0x7f81840ced60 comp rx=0 tx=0).stop 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 shutdown_connections 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8170067320 0x7f81700697d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f81841560f0 0x7f8184156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c0051c0 0x7f8184169c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 --2- 172.21.15.92:0/661724320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8184158120 0x7f8184158590 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 >> 172.21.15.92:0/661724320 conn(0x7f8184068e00 msgr2=0x7f81840b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:28.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 shutdown_connections 2023-10-06T03:08:28.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.814+0000 7f818a975700 1 -- 172.21.15.92:0/661724320 wait complete. 2023-10-06T03:08:28.817 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:08:28.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:28.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.943+0000 7f56204a0700 1 Processor -- start 2023-10-06T03:08:28.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.943+0000 7f56204a0700 1 -- start start 2023-10-06T03:08:28.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.944+0000 7f56204a0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f561806aa40 0x7f56180cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.944+0000 7f56204a0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f561806ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.944+0000 7f56204a0700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56180cc210 con 0x7f561806b350 2023-10-06T03:08:28.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.944+0000 7f56204a0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56180cc4f0 con 0x7f561806aa40 2023-10-06T03:08:28.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.944+0000 7f56204a0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56180cc7d0 con 0x7f561806bac0 2023-10-06T03:08:28.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f561806aa40 0x7f56180cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f561806aa40 0x7f56180cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38128/0 (socket says 172.21.15.92:38128) 2023-10-06T03:08:28.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 -- 172.21.15.92:0/2692915252 learned_addr learned my addr 172.21.15.92:0/2692915252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:28.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561da3b700 1 --2- 172.21.15.92:0/2692915252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f561806ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 -- 172.21.15.92:0/2692915252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 msgr2=0x7f561806ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:28.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 --2- 172.21.15.92:0/2692915252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f561806ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 -- 172.21.15.92:0/2692915252 >> v1:172.21.15.136:6789/0 conn(0x7f561806b350 legacy=0x7f56180c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 -- 172.21.15.92:0/2692915252 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56180ccab0 con 0x7f561806aa40 2023-10-06T03:08:28.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561e23c700 1 --2- 172.21.15.92:0/2692915252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806aa40 0x7f56180cbc00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5614009f70 tx=0x7f5614009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7306ac01ab5518 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561d23a700 1 -- 172.21.15.92:0/2692915252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400e070 con 0x7f561806aa40 2023-10-06T03:08:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561d23a700 1 -- 172.21.15.92:0/2692915252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561400ba20 con 0x7f561806aa40 2023-10-06T03:08:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.945+0000 7f561d23a700 1 -- 172.21.15.92:0/2692915252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140092e0 con 0x7f561806aa40 2023-10-06T03:08:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 -- 172.21.15.92:0/2692915252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806aa40 msgr2=0x7f56180cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 --2- 172.21.15.92:0/2692915252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806aa40 0x7f56180cbc00 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f5614009f70 tx=0x7f5614009840 comp rx=0 tx=0).stop 2023-10-06T03:08:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 -- 172.21.15.92:0/2692915252 shutdown_connections 2023-10-06T03:08:28.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 --2- 172.21.15.92:0/2692915252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806aa40 0x7f56180cbc00 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 --2- 172.21.15.92:0/2692915252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f561806ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 -- 172.21.15.92:0/2692915252 >> 172.21.15.92:0/2692915252 conn(0x7f56180cd670 msgr2=0x7f56180cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:28.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 -- 172.21.15.92:0/2692915252 shutdown_connections 2023-10-06T03:08:28.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.946+0000 7f56204a0700 1 -- 172.21.15.92:0/2692915252 wait complete. 2023-10-06T03:08:28.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.947+0000 7f56204a0700 1 Processor -- start 2023-10-06T03:08:28.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.947+0000 7f56204a0700 1 -- start start 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f561806aa40 0x7f5618151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 0x7f56180af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f56180c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180af000 con 0x7f561806b350 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561816a320 con 0x7f561806aa40 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561816a600 con 0x7f561806bac0 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 0x7f56180af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 0x7f56180af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:38130/0 (socket says 172.21.15.92:38130) 2023-10-06T03:08:28.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 -- 172.21.15.92:0/2261095394 learned_addr learned my addr 172.21.15.92:0/2261095394 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:28.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561e23c700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f561806aa40 0x7f5618151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561ea3d700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f56180c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 -- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 msgr2=0x7f56180c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f56180c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 -- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f561806aa40 msgr2=0x7f5618151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:28.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f561806aa40 0x7f5618151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:28.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 -- 172.21.15.92:0/2261095394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614009ae0 con 0x7f561806b350 2023-10-06T03:08:28.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561ea3d700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f56180c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:28.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f561da3b700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 0x7f56180af490 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f560800e900 tx=0x7f56080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608013680 con 0x7f561806b350 2023-10-06T03:08:28.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56080137e0 con 0x7f561806b350 2023-10-06T03:08:28.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56180aec90 con 0x7f561806b350 2023-10-06T03:08:28.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608013ba0 con 0x7f561806b350 2023-10-06T03:08:28.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.948+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5618151d60 con 0x7f561806b350 2023-10-06T03:08:28.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.949+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56180af1b0 con 0x7f561806b350 2023-10-06T03:08:28.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.950+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f560800c460 con 0x7f561806b350 2023-10-06T03:08:28.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.950+0000 7f560e7fc700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55fc067320 0x7f55fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:28.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.950+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5608084c50 con 0x7f561806b350 2023-10-06T03:08:28.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.950+0000 7f561e23c700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55fc067320 0x7f55fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:28.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.951+0000 7f561e23c700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55fc067320 0x7f55fc0697d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5614009880 tx=0x7f5614018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:28.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:28.955+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56180af1b0 con 0x7f561806b350 2023-10-06T03:08:29.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.179+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f561804c3c0 con 0x7f561806b350 2023-10-06T03:08:29.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.180+0000 7f560e7fc700 1 -- 172.21.15.92:0/2261095394 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f5608017090 con 0x7f561806b350 2023-10-06T03:08:29.182 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:29.182 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6625,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3257155196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3257155196},{"type":"v1","addr":"172.21.15.136:6835","nonce":3257155196}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7218,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3995167741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3995167741},{"type":"v1","addr":"172.21.15.92:6837","nonce":3995167741}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:08:22.334042+0000","modified":"2023-10-06T03:08:26.961080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7221},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7221":{"gid":7221,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/1746152484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1746152484},{"type":"v1","addr":"172.21.15.92:6836","nonce":1746152484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T03:08:29.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55fc067320 msgr2=0x7f55fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55fc067320 0x7f55fc0697d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5614009880 tx=0x7f5614018000 comp rx=0 tx=0).stop 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 msgr2=0x7f56180af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 0x7f56180af490 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f560800e900 tx=0x7f56080091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 shutdown_connections 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f55fc067320 0x7f55fc0697d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f561806bac0 0x7f56180c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f561806aa40 0x7f5618151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 --2- 172.21.15.92:0/2261095394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f561806b350 0x7f56180af490 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 >> 172.21.15.92:0/2261095394 conn(0x7f56180cd670 msgr2=0x7f56180c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 shutdown_connections 2023-10-06T03:08:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.182+0000 7f56204a0700 1 -- 172.21.15.92:0/2261095394 wait complete. 2023-10-06T03:08:29.186 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 77 2023-10-06T03:08:29.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:08:22.334042+0000', 'modified': '2023-10-06T03:08:26.961080+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7221}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7221': {'gid': 7221, 'name': 'a', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6836/1746152484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 1746152484}, {'type': 'v1', 'addr': '172.21.15.92:6836', 'nonce': 1746152484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:08:29.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:08:29.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:08:29.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.716+0000 7f9caaaaa700 1 Processor -- start 2023-10-06T03:08:29.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.716+0000 7f9caaaaa700 1 -- start start 2023-10-06T03:08:29.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.717+0000 7f9caaaaa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9ca40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.717+0000 7f9caaaaa700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca4069ee0 con 0x7f9ca4159c60 2023-10-06T03:08:29.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.717+0000 7f9caaaaa700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca415a160 con 0x7f9ca4159890 2023-10-06T03:08:29.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.717+0000 7f9caaaaa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40d13a0 con 0x7f9ca4069560 2023-10-06T03:08:29.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.717+0000 7f9c9bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ca4159c60 0x7f9ca415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51510/0 (socket says 172.21.15.92:51510) 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.717+0000 7f9c9bfff700 1 -- 172.21.15.92:0/252037162 learned_addr learned my addr 172.21.15.92:0/252037162 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9ca9047700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9ca40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73034925 0 0) 0x7f9ca4069ee0 con 0x7f9ca4159c60 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c8c003570 con 0x7f9ca4159c60 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2445719309 0 0) 0x7f9c8c003570 con 0x7f9ca4159c60 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 msgr2=0x7f9ca40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9ca40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 >> v1:172.21.15.136:6789/0 conn(0x7f9ca4159890 legacy=0x7f9ca415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca415e140 con 0x7f9ca4159c60 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f9c94001470 con 0x7f9ca4159c60 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 >> v1:172.21.15.92:6789/0 conn(0x7f9ca4159c60 legacy=0x7f9ca415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8c003a90 0x7f9c8c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9c8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9c8c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40d13a0 con 0x7f9c8c004810 2023-10-06T03:08:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c00bb90 con 0x7f9c8c003a90 2023-10-06T03:08:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c00bd50 con 0x7f9ca4069560 2023-10-06T03:08:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9c8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9ca8846700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8c003a90 0x7f9c8c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9ca9047700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9c8c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9bfff700 1 -- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 msgr2=0x7f9c8c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9c8c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9bfff700 1 -- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8c003a90 msgr2=0x7f9c8c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c8c003a90 0x7f9c8c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9c9bfff700 1 -- 172.21.15.92:0/252037162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c003570 con 0x7f9c8c004810 2023-10-06T03:08:29.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.718+0000 7f9ca9047700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca4069560 0x7f9c8c0051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9c8c004c80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c3c0 tx=0x7f9c94001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad55cfdf04545c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9400cc90 con 0x7f9c8c004810 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9400cdf0 con 0x7f9c8c004810 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9c9b7fe700 1 -- 172.21.15.92:0/252037162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94005b50 con 0x7f9c8c004810 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9caaaaa700 1 -- 172.21.15.92:0/252037162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 msgr2=0x7f9c8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/252037162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9c8c004c80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c3c0 tx=0x7f9c94001e50 comp rx=0 tx=0).stop 2023-10-06T03:08:29.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9ca9047700 1 -- 172.21.15.92:0/252037162 reap_dead start 2023-10-06T03:08:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9caaaaa700 1 -- 172.21.15.92:0/252037162 shutdown_connections 2023-10-06T03:08:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9caaaaa700 1 -- 172.21.15.92:0/252037162 >> 172.21.15.92:0/252037162 conn(0x7f9ca4068e20 msgr2=0x7f9ca40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9caaaaa700 1 -- 172.21.15.92:0/252037162 shutdown_connections 2023-10-06T03:08:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.719+0000 7f9caaaaa700 1 -- 172.21.15.92:0/252037162 wait complete. 2023-10-06T03:08:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.720+0000 7f9caaaaa700 1 Processor -- start 2023-10-06T03:08:29.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 -- start start 2023-10-06T03:08:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9ca40c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca40c94e0 0x7f9ca40c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca40c7de0 0x7f9ca4143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40c8250 con 0x7f9c8c004810 2023-10-06T03:08:29.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca40c9050 con 0x7f9ca40c94e0 2023-10-06T03:08:29.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9caaaaa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4152b00 con 0x7f9ca40c7de0 2023-10-06T03:08:29.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9ca8846700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9ca40c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9ca8846700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9ca40c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54924/0 (socket says 172.21.15.92:54924) 2023-10-06T03:08:29.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9ca8846700 1 -- 172.21.15.92:0/3096840138 learned_addr learned my addr 172.21.15.92:0/3096840138 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca40c94e0 0x7f9ca40c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.721+0000 7f9ca9047700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca40c7de0 0x7f9ca4143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca8846700 1 -- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca40c7de0 msgr2=0x7f9ca4143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca40c7de0 0x7f9ca4143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca8846700 1 -- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca40c94e0 msgr2=0x7f9ca40c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca40c94e0 0x7f9ca40c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca8846700 1 -- 172.21.15.92:0/3096840138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40c8440 con 0x7f9c8c004810 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca9047700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca40c7de0 0x7f9ca4143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:29.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9ca40c9f30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000fbe0 tx=0x7f9ca0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9c98ff9700 1 -- 172.21.15.92:0/3096840138 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0001b60 con 0x7f9c8c004810 2023-10-06T03:08:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9c98ff9700 1 -- 172.21.15.92:0/3096840138 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0001cc0 con 0x7f9c8c004810 2023-10-06T03:08:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca40c8720 con 0x7f9c8c004810 2023-10-06T03:08:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9c98ff9700 1 -- 172.21.15.92:0/3096840138 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000d940 con 0x7f9c8c004810 2023-10-06T03:08:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.722+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca40c8ce0 con 0x7f9c8c004810 2023-10-06T03:08:29.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.723+0000 7f9c98ff9700 1 -- 172.21.15.92:0/3096840138 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9ca000dcb0 con 0x7f9c8c004810 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.724+0000 7f9c98ff9700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c90067320 0x7f9c900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.724+0000 7f9c98ff9700 1 -- 172.21.15.92:0/3096840138 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ca0085350 con 0x7f9c8c004810 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.724+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c90067320 0x7f9c900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.724+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c90067320 0x7f9c900697d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9c94007cd0 tx=0x7f9c94015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.725+0000 7f9cb3da2b80 1 Processor -- start 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.725+0000 7f9cb3da2b80 1 -- start start 2023-10-06T03:08:29.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a543a076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a08050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54390ea70 con 0x55a5439b8cc0 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54390c270 con 0x55a543a07c00 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a543908060 con 0x55a5439b7840 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9c9bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a543a076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9c9bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a543a076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54938/0 (socket says 172.21.15.92:54938) 2023-10-06T03:08:29.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9c9bfff700 1 -- 172.21.15.92:0/1808608045 learned_addr learned my addr 172.21.15.92:0/1808608045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9ca8846700 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.726+0000 7f9ca9047700 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a08050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c9bfff700 1 -- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 msgr2=0x55a5439b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c9bfff700 1 -- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 msgr2=0x55a543a08050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a08050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c9bfff700 1 -- 172.21.15.92:0/1808608045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5437a2270 con 0x55a5439b8cc0 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a543a076c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00bb60 tx=0x7f9c94021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66dbe46a41a8051 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c89ffb700 1 -- 172.21.15.92:0/1808608045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9402f070 con 0x55a5439b8cc0 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c89ffb700 1 -- 172.21.15.92:0/1808608045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940017c0 con 0x55a5439b8cc0 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9c89ffb700 1 -- 172.21.15.92:0/1808608045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94004c90 con 0x55a5439b8cc0 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/1808608045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 msgr2=0x55a543a076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a543a076c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00bb60 tx=0x7f9c94021040 comp rx=0 tx=0).stop 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/1808608045 shutdown_connections 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a08050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/1808608045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a543a076c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/1808608045 >> 172.21.15.92:0/1808608045 conn(0x55a5439b9160 msgr2=0x55a5439b9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.727+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/1808608045 shutdown_connections 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.728+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/1808608045 wait complete. 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.728+0000 7f9cb3da2b80 1 -- start start 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.728+0000 7f9cb3da2b80 1 Processor -- start 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a5439fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a5439fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439fed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5438b3890 con 0x55a5439b7840 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5438ceea0 con 0x55a543a07c00 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a54378ae30 con 0x55a5439b8cc0 2023-10-06T03:08:29.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9cb3da2b80 10 client.0 _ll_register_callbacks cb 0x55a54397ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a5439fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a5439fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54946/0 (socket says 172.21.15.92:54946) 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 -- 172.21.15.92:0/3501557483 learned_addr learned my addr 172.21.15.92:0/3501557483 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a5439fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca9047700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 -- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 msgr2=0x55a5439fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a5439fe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 -- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 msgr2=0x55a5439fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.729+0000 7f9ca8846700 1 -- 172.21.15.92:0/3501557483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c003570 con 0x55a5439b7840 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a5439fe2c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0055a90 tx=0x7f9ca0085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55a3a3977e2f7ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3501557483 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00558d0 con 0x55a5439b7840 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3501557483 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ca0056200 con 0x55a5439b7840 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5439ac940 con 0x55a5439b7840 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3501557483 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca0055b00 con 0x55a5439b7840 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 10 client.7386 fetch_fsmap learned FSMap version 77 2023-10-06T03:08:29.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55a543744fe0 con 0x55a5439b7840 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3501557483 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a5439ac940 con 0x55a5439b7840 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3501557483 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 77) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9ca005c610 con 0x55a5439b7840 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 10 client.7386 fetch_fsmap finished waiting for FSMap version 77 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 10 client.7386 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 20 client.7386 populate_metadata read hostname 'smithi092' 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a5439e3180 0x55a5439e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 4 client.7386 mds_command: new command op to 7221 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:08:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 --> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a543989870 con 0x55a5439e3180 2023-10-06T03:08:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a5439e3180 0x55a5439e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c82ffd700 10 client.7386 ms_handle_connect on v2:172.21.15.92:6834/1746152484 2023-10-06T03:08:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.730+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a5439e3180 0x55a5439e7090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20817342a366fd5d server_cookie=fedb6c9a74044bb7 in_seq=0 out_seq=0 2023-10-06T03:08:29.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3501557483 <== mds.0 v2:172.21.15.92:6834/1746152484 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a543989870 con 0x55a5439e3180 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9c82ffd700 10 client.7386 handle_command_reply: tid=0 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 client.7386 shutdown 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 msgr2=0x55a5439fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a5439fe2c0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0055a90 tx=0x7f9ca0085040 comp rx=0 tx=0).stop 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 shutdown_connections 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a5439e3180 0x55a5439e7090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a5439fe2c0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9c82ffd700 0 client.7386 ms_handle_reset on v2:172.21.15.92:6834/1746152484 2023-10-06T03:08:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3501557483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a5439fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 >> 172.21.15.92:0/3501557483 conn(0x55a5439b9160 msgr2=0x55a5439fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 shutdown_connections 2023-10-06T03:08:29.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3501557483 wait complete. 2023-10-06T03:08:29.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.731+0000 7f9cb3da2b80 20 client.7386 trim_cache size 0 max 16384 2023-10-06T03:08:29.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.748+0000 7f9cb3da2b80 1 Processor -- start 2023-10-06T03:08:29.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.748+0000 7f9cb3da2b80 1 -- start start 2023-10-06T03:08:29.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a5439e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a543a10ab0 con 0x55a5439b8cc0 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5439e5960 con 0x55a543a07c00 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5439ac8a0 con 0x55a5439b7840 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a5439e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a5439e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54962/0 (socket says 172.21.15.92:54962) 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 -- 172.21.15.92:0/79369779 learned_addr learned my addr 172.21.15.92:0/79369779 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9ca8846700 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9ca9047700 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 -- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 msgr2=0x55a5439e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 -- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 msgr2=0x55a543a0fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a0fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 -- 172.21.15.92:0/79369779 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5438d6780 con 0x55a5439b8cc0 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9ca8846700 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a5439e2bc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9c94001ca0 tx=0x7f9c9403b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfc968dac10ff1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c8a7fc700 1 -- 172.21.15.92:0/79369779 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94041070 con 0x55a5439b8cc0 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c8a7fc700 1 -- 172.21.15.92:0/79369779 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940072e0 con 0x55a5439b8cc0 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.749+0000 7f9c8a7fc700 1 -- 172.21.15.92:0/79369779 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9403b070 con 0x55a5439b8cc0 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/79369779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 msgr2=0x55a5439e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a5439e2bc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9c94001ca0 tx=0x7f9c9403b250 comp rx=0 tx=0).stop 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/79369779 shutdown_connections 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a543a0fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b7840 0x55a5439e2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/79369779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b8cc0 0x55a5439e2bc0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/79369779 >> 172.21.15.92:0/79369779 conn(0x55a5439b9160 msgr2=0x55a5439fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/79369779 shutdown_connections 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/79369779 wait complete. 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.750+0000 7f9cb3da2b80 1 -- start start 2023-10-06T03:08:29.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 Processor -- start 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a543a0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a543a10410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439ff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a543a00570 con 0x55a5439b7840 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5439fbd60 con 0x55a543a07c00 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5439e29a0 con 0x55a5439b8cc0 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9cb3da2b80 10 client.0 _ll_register_callbacks cb 0x55a54397ae70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9ca8846700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a543a0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9ca8846700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a543a0fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54970/0 (socket says 172.21.15.92:54970) 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.751+0000 7f9ca8846700 1 -- 172.21.15.92:0/3984374748 learned_addr learned my addr 172.21.15.92:0/3984374748 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca9047700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a543a10410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca8846700 1 -- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 msgr2=0x55a543a10410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a543a10410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca8846700 1 -- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 msgr2=0x55a5439ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca8846700 1 -- 172.21.15.92:0/3984374748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5437a2270 con 0x55a5439b7840 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9ca8846700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a543a0fed0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000a040 tx=0x7f9ca006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd5f1f4baabd445 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3984374748 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00558d0 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3984374748 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ca0056200 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c940017c0 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3984374748 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca0023300 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 10 client.7392 fetch_fsmap learned FSMap version 77 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55a5438bd470 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3984374748 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940017c0 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3984374748 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 77) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9ca005c610 con 0x55a5439b7840 2023-10-06T03:08:29.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 10 client.7392 fetch_fsmap finished waiting for FSMap version 77 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 10 client.7392 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 20 client.7392 populate_metadata read hostname 'smithi092' 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a543a63460 0x55a5439ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 4 client.7392 mds_command: new command op to 7221 tid=0[{"prefix": "status"}] 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.752+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 --> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a5438aae00 con 0x55a543a63460 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a543a63460 0x55a5439ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9c82ffd700 10 client.7392 ms_handle_connect on v2:172.21.15.92:6834/1746152484 2023-10-06T03:08:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9c9bfff700 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a543a63460 0x55a5439ffa80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f231ca55fcf2496d server_cookie=5becdaf2c6830822 in_seq=0 out_seq=0 2023-10-06T03:08:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9c82ffd700 1 -- 172.21.15.92:0/3984374748 <== mds.0 v2:172.21.15.92:6834/1746152484 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a5438aae00 con 0x55a543a63460 2023-10-06T03:08:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9c82ffd700 10 client.7392 handle_command_reply: tid=0 2023-10-06T03:08:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 client.7392 shutdown 2023-10-06T03:08:29.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 msgr2=0x55a543a0fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a543a0fed0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000a040 tx=0x7f9ca006d000 comp rx=0 tx=0).stop 2023-10-06T03:08:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 shutdown_connections 2023-10-06T03:08:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:6834/1746152484,v1:172.21.15.92:6836/1746152484] conn(0x55a543a63460 0x55a5439ffa80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a5439b7840 0x55a543a0fed0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9c82ffd700 0 client.7392 ms_handle_reset on v2:172.21.15.92:6834/1746152484 2023-10-06T03:08:29.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a543a07c00 0x55a5439ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 --2- 172.21.15.92:0/3984374748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55a5439b8cc0 0x55a543a10410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 >> 172.21.15.92:0/3984374748 conn(0x55a5439b9160 msgr2=0x55a5439e4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 shutdown_connections 2023-10-06T03:08:29.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 1 -- 172.21.15.92:0/3984374748 wait complete. 2023-10-06T03:08:29.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.753+0000 7f9cb3da2b80 20 client.7392 trim_cache size 0 max 16384 2023-10-06T03:08:29.781 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:08:29.781 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:08:29.781 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:08:29.781 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7221, 2023-10-06T03:08:29.781 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:08:29.781 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:08:29.782 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:08:29.782 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 6.8063879800000002, 2023-10-06T03:08:29.782 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 77, 2023-10-06T03:08:29.782 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 115, 2023-10-06T03:08:29.782 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 115, 2023-10-06T03:08:29.783 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 12.92496309 2023-10-06T03:08:29.783 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c90067320 msgr2=0x7f9c900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c90067320 0x7f9c900697d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9c94007cd0 tx=0x7f9c94015000 comp rx=0 tx=0).stop 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 msgr2=0x7f9ca40c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9ca40c9f30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000fbe0 tx=0x7f9ca0001640 comp rx=0 tx=0).stop 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 shutdown_connections 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c90067320 0x7f9c900697d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c8c004810 0x7f9ca40c9f30 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ca40c7de0 0x7f9ca4143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 --2- 172.21.15.92:0/3096840138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ca40c94e0 0x7f9ca40c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 >> 172.21.15.92:0/3096840138 conn(0x7f9ca4068e20 msgr2=0x7f9ca40d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 shutdown_connections 2023-10-06T03:08:29.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.754+0000 7f9caaaaa700 1 -- 172.21.15.92:0/3096840138 wait complete. 2023-10-06T03:08:29.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:08:29.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.900+0000 7fcd8e63b700 1 Processor -- start 2023-10-06T03:08:29.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.900+0000 7fcd8e63b700 1 -- start start 2023-10-06T03:08:29.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd8e63b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcd88159890 0x7fcd8815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd8e63b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd88159c60 0x7fcd8815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd8e63b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8815a160 con 0x7fcd8815db10 2023-10-06T03:08:29.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd8e63b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8815df50 con 0x7fcd88159c60 2023-10-06T03:08:29.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd8e63b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd880d13a0 con 0x7fcd88159890 2023-10-06T03:08:29.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd88159c60 0x7fcd8815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd877fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcd88159c60 0x7fcd8815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54986/0 (socket says 172.21.15.92:54986) 2023-10-06T03:08:29.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd8cbd8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcd8815db10 0x7fcd88069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51520/0 (socket says 172.21.15.92:51520) 2023-10-06T03:08:29.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.901+0000 7fcd877fe700 1 -- 172.21.15.92:0/4106013874 learned_addr learned my addr 172.21.15.92:0/4106013874 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd87fff700 1 --2- 172.21.15.92:0/4106013874 >> v2:172.21.15.136:3300/0 conn(0x7fcd88159890 0x7fcd8815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd877fe700 1 -- 172.21.15.92:0/4106013874 >> v2:172.21.15.136:3300/0 conn(0x7fcd88159890 msgr2=0x7fcd8815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd877fe700 1 --2- 172.21.15.92:0/4106013874 >> v2:172.21.15.136:3300/0 conn(0x7fcd88159890 0x7fcd8815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd877fe700 1 -- 172.21.15.92:0/4106013874 >> v1:172.21.15.92:6789/0 conn(0x7fcd8815db10 legacy=0x7fcd88069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd86ffd700 1 -- 172.21.15.92:0/4106013874 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799208917 0 0) 0x7fcd8815a160 con 0x7fcd8815db10 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd877fe700 1 -- 172.21.15.92:0/4106013874 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8815e140 con 0x7fcd88159c60 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd877fe700 1 --2- 172.21.15.92:0/4106013874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd88159c60 0x7fcd8815d5d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcd880d1090 tx=0x7fcd7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa6d21163bb54c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd86ffd700 1 -- 172.21.15.92:0/4106013874 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00b860 con 0x7fcd88159c60 2023-10-06T03:08:29.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd86ffd700 1 -- 172.21.15.92:0/4106013874 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7c00b9c0 con 0x7fcd88159c60 2023-10-06T03:08:29.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd86ffd700 1 -- 172.21.15.92:0/4106013874 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00bd60 con 0x7fcd88159c60 2023-10-06T03:08:29.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 -- 172.21.15.92:0/4106013874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd88159c60 msgr2=0x7fcd8815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/4106013874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd88159c60 0x7fcd8815d5d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcd880d1090 tx=0x7fcd7c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:29.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 -- 172.21.15.92:0/4106013874 shutdown_connections 2023-10-06T03:08:29.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/4106013874 >> v2:172.21.15.136:3300/0 conn(0x7fcd88159890 0x7fcd8815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/4106013874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd88159c60 0x7fcd8815d5d0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 -- 172.21.15.92:0/4106013874 >> 172.21.15.92:0/4106013874 conn(0x7fcd88068e20 msgr2=0x7fcd880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:29.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.902+0000 7fcd8e63b700 1 -- 172.21.15.92:0/4106013874 shutdown_connections 2023-10-06T03:08:29.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.903+0000 7fcd8e63b700 1 -- 172.21.15.92:0/4106013874 wait complete. 2023-10-06T03:08:29.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 Processor -- start 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 -- start start 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd88159890 0x7fcd880b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd88159c60 0x7fcd881523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 0x7fcd88069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd880ce800 con 0x7fcd8815db10 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8815bd60 con 0x7fcd88159c60 2023-10-06T03:08:29.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.904+0000 7fcd8e63b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8815c040 con 0x7fcd88159890 2023-10-06T03:08:29.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 0x7fcd88069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 0x7fcd88069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:54996/0 (socket says 172.21.15.92:54996) 2023-10-06T03:08:29.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 -- 172.21.15.92:0/1861796881 learned_addr learned my addr 172.21.15.92:0/1861796881 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:29.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd877fe700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd88159c60 0x7fcd881523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd87fff700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd88159890 0x7fcd880b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 -- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd88159890 msgr2=0x7fcd880b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd88159890 0x7fcd880b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 -- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd88159c60 msgr2=0x7fcd881523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd88159c60 0x7fcd881523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 -- 172.21.15.92:0/1861796881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0094f0 con 0x7fcd8815db10 2023-10-06T03:08:29.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8cbd8700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 0x7fcd88069530 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000ccc0 tx=0x7fcd80001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80005df0 con 0x7fcd8815db10 2023-10-06T03:08:29.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8000a600 con 0x7fcd8815db10 2023-10-06T03:08:29.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8000a9c0 con 0x7fcd8815db10 2023-10-06T03:08:29.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8815c320 con 0x7fcd8815db10 2023-10-06T03:08:29.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.905+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8815c8e0 con 0x7fcd8815db10 2023-10-06T03:08:29.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.906+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8804c3c0 con 0x7fcd8815db10 2023-10-06T03:08:29.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.907+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcd800070f0 con 0x7fcd8815db10 2023-10-06T03:08:29.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.907+0000 7fcd6ffff700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd74067320 0x7fcd740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:29.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.907+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcd80084e00 con 0x7fcd8815db10 2023-10-06T03:08:29.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.907+0000 7fcd87fff700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd74067320 0x7fcd740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:29.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.907+0000 7fcd87fff700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd74067320 0x7fcd740697d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800acb0 tx=0x7fcd78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:29.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:29.911+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd80051a60 con 0x7fcd8815db10 2023-10-06T03:08:30.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.140+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd8815cad0 con 0x7fcd8815db10 2023-10-06T03:08:30.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.140+0000 7fcd6ffff700 1 -- 172.21.15.92:0/1861796881 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcd80055080 con 0x7fcd8815db10 2023-10-06T03:08:30.142 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd74067320 msgr2=0x7fcd740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd74067320 0x7fcd740697d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800acb0 tx=0x7fcd78008040 comp rx=0 tx=0).stop 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 msgr2=0x7fcd88069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 0x7fcd88069530 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000ccc0 tx=0x7fcd80001a70 comp rx=0 tx=0).stop 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 shutdown_connections 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd74067320 0x7fcd740697d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd8815db10 0x7fcd88069530 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcd88159890 0x7fcd880b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 --2- 172.21.15.92:0/1861796881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd88159c60 0x7fcd881523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 >> 172.21.15.92:0/1861796881 conn(0x7fcd88068e20 msgr2=0x7fcd881573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 shutdown_connections 2023-10-06T03:08:30.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.142+0000 7fcd8e63b700 1 -- 172.21.15.92:0/1861796881 wait complete. 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:08:30.156 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:08:30.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.157 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:08:30.171 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:30.172 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42096sec preferred_lft 42096sec 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:30.173 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:08:30.174 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:30.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.175 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:08:30.236 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:08:30.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.237 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:08:30.293 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:08:30.293 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:08:30.293 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:08:30.294 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:08:30.294 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:08:30.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.294 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:08:30.294 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:08:30.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:08:30.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:08:30.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:08:30.295 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:08:30.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.393 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:08:30.407 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:08:30.407 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:08:30.408 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:08:30.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:08:30.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:30.472 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:30.527 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:08:30.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:30.528 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:30.582 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:08:30.582 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:08:30.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:30.662 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:30.662 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:30.680 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:30.681 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:30.682 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:30.682 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:30.702 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:08:30.703 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:08:30.704 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:08:30.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:30.705 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:30.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:08:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.922+0000 7fea8561f700 1 Processor -- start 2023-10-06T03:08:30.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.922+0000 7fea8561f700 1 -- start start 2023-10-06T03:08:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea8561f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fea80159420 0x7fea800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea8561f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea800b2980 con 0x7fea801597f0 2023-10-06T03:08:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea8561f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea800b11d0 con 0x7fea8015d230 2023-10-06T03:08:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea8561f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea800b14b0 con 0x7fea80159420 2023-10-06T03:08:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea7f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fea801597f0 0x7fea800b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51534/0 (socket says 172.21.15.92:51534) 2023-10-06T03:08:30.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea7f7fe700 1 -- 172.21.15.92:0/2173074454 learned_addr learned my addr 172.21.15.92:0/2173074454 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea7ffff700 1 --2- 172.21.15.92:0/2173074454 >> v2:172.21.15.136:3300/0 conn(0x7fea80159420 0x7fea800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346203646 0 0) 0x7fea800b2980 con 0x7fea801597f0 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.923+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea68003570 con 0x7fea801597f0 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3634124888 0 0) 0x7fea68003570 con 0x7fea801597f0 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 >> v2:172.21.15.136:3300/0 conn(0x7fea80159420 msgr2=0x7fea800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 --2- 172.21.15.92:0/2173074454 >> v2:172.21.15.136:3300/0 conn(0x7fea80159420 0x7fea800b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 >> v1:172.21.15.136:6789/0 conn(0x7fea8015d230 legacy=0x7fea8015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800b1790 con 0x7fea801597f0 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fea74001470 con 0x7fea801597f0 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 >> v1:172.21.15.92:6789/0 conn(0x7fea801597f0 legacy=0x7fea800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea68003a60 0x7fea68003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68004810 0x7fea68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea680051c0 0x7fea6800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800b14b0 con 0x7fea680051c0 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6800c0d0 con 0x7fea68004810 2023-10-06T03:08:30.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6800c290 con 0x7fea68003a60 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea680051c0 0x7fea6800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7ffff700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea68003a60 0x7fea68003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7f7fe700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68004810 0x7fea68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 -- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea68003a60 msgr2=0x7fea68003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea68003a60 0x7fea68003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 -- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68004810 msgr2=0x7fea68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea68004810 0x7fea68004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 -- 172.21.15.92:0/2173074454 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68003570 con 0x7fea680051c0 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 -- 172.21.15.92:0/2173074454 reap_dead start 2023-10-06T03:08:30.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea84e1e700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea680051c0 0x7fea6800ba60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fea78001f90 tx=0x7fea78001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e889f003cdda5ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78001950 con 0x7fea680051c0 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.924+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800c8b0 con 0x7fea680051c0 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea7effd700 1 -- 172.21.15.92:0/2173074454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800cc50 con 0x7fea680051c0 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 -- 172.21.15.92:0/2173074454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea680051c0 msgr2=0x7fea6800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea680051c0 0x7fea6800ba60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fea78001f90 tx=0x7fea78001640 comp rx=0 tx=0).stop 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 -- 172.21.15.92:0/2173074454 shutdown_connections 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 --2- 172.21.15.92:0/2173074454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea680051c0 0x7fea6800ba60 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 -- 172.21.15.92:0/2173074454 >> 172.21.15.92:0/2173074454 conn(0x7fea800d0660 msgr2=0x7fea800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:30.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 -- 172.21.15.92:0/2173074454 shutdown_connections 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.925+0000 7fea8561f700 1 -- 172.21.15.92:0/2173074454 wait complete. 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.926+0000 7fea8561f700 1 Processor -- start 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.926+0000 7fea8561f700 1 -- start start 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea8561f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea680051c0 0x7fea800cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea8561f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 0x7fea800b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea8561f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea800b5e70 0x7fea801516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea8561f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800b62f0 con 0x7fea800cc050 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea8561f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80169db0 con 0x7fea680051c0 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea8561f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea801523d0 con 0x7fea800b5e70 2023-10-06T03:08:30.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 0x7fea800b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 0x7fea800b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55016/0 (socket says 172.21.15.92:55016) 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 -- 172.21.15.92:0/3081282516 learned_addr learned my addr 172.21.15.92:0/3081282516 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7ffff700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea680051c0 0x7fea800cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea84e1e700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea800b5e70 0x7fea801516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 -- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea800b5e70 msgr2=0x7fea801516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea800b5e70 0x7fea801516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 -- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea680051c0 msgr2=0x7fea800cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea680051c0 0x7fea800cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:30.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.927+0000 7fea7f7fe700 1 -- 172.21.15.92:0/3081282516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea801526b0 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.928+0000 7fea7f7fe700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 0x7fea800b5930 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fea740073f0 tx=0x7fea74001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.928+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74018030 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.928+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7400f6c0 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.928+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea80152990 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.928+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400fa80 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.928+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea80151be0 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.929+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea80169f40 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.929+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fea7401b070 con 0x7fea800cc050 2023-10-06T03:08:30.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.929+0000 7fea67fff700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea6c067320 0x7fea6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.930+0000 7fea7ffff700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea6c067320 0x7fea6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.930+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fea74019070 con 0x7fea800cc050 2023-10-06T03:08:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.930+0000 7fea7ffff700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea6c067320 0x7fea6c0697d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea70006f00 tx=0x7fea700091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:30.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:30.934+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea740519d0 con 0x7fea800cc050 2023-10-06T03:08:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.159+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fea8004c3c0 con 0x7fea800cc050 2023-10-06T03:08:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.160+0000 7fea67fff700 1 -- 172.21.15.92:0/3081282516 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7fea74054ff0 con 0x7fea800cc050 2023-10-06T03:08:31.162 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea6c067320 msgr2=0x7fea6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea6c067320 0x7fea6c0697d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea70006f00 tx=0x7fea700091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 msgr2=0x7fea800b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 0x7fea800b5930 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fea740073f0 tx=0x7fea74001e50 comp rx=0 tx=0).stop 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 shutdown_connections 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea6c067320 0x7fea6c0697d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea800b5e70 0x7fea801516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea680051c0 0x7fea800cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 --2- 172.21.15.92:0/3081282516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea800cc050 0x7fea800b5930 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 >> 172.21.15.92:0/3081282516 conn(0x7fea800d0660 msgr2=0x7fea801577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:31.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 shutdown_connections 2023-10-06T03:08:31.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.162+0000 7fea8561f700 1 -- 172.21.15.92:0/3081282516 wait complete. 2023-10-06T03:08:31.175 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:08:31.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:08:31.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.176 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.195 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:31.196 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42096sec preferred_lft 42096sec 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:31.197 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:08:31.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.198 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:08:31.263 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:08:31.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.264 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:08:31.322 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:08:31.322 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:08:31.322 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:08:31.322 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:08:31.323 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:08:31.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.323 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:08:31.323 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:08:31.324 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:08:31.324 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:08:31.324 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:08:31.324 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:08:31.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.429 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:08:31.444 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:08:31.445 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:08:31.445 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:08:31.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.445 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:08:31.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:31.511 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:31.565 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:08:31.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:08:31.566 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:31.619 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:08:31.620 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:08:31.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:31.699 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:31.700 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:31.718 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:31.740 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:08:31.740 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:08:31.740 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:08:31.740 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:08:31.741 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:08:31.742 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:08:31.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:31.742 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:31.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:08:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.926+0000 7fcc0a1d5700 1 Processor -- start 2023-10-06T03:08:31.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.926+0000 7fcc0a1d5700 1 -- start start 2023-10-06T03:08:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc0a1d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc040cd910 0x7fcc040ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc0a1d5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc040b55a0 0x7fcc040ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc0a1d5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040b5b00 con 0x7fcc04159ec0 2023-10-06T03:08:31.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc0a1d5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040b5da0 con 0x7fcc040b55a0 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc0a1d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040b6080 con 0x7fcc040cd910 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc03fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc040b55a0 0x7fcc040ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc037fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcc04159ec0 0x7fcc040b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51550/0 (socket says 172.21.15.92:51550) 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcc037fe700 1 -- 172.21.15.92:0/1301126663 learned_addr learned my addr 172.21.15.92:0/1301126663 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.927+0000 7fcbfbfff700 1 --2- 172.21.15.92:0/1301126663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc040cd910 0x7fcc040ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc03fff700 1 -- 172.21.15.92:0/1301126663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc040cd910 msgr2=0x7fcc040ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc03fff700 1 --2- 172.21.15.92:0/1301126663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc040cd910 0x7fcc040ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc03fff700 1 -- 172.21.15.92:0/1301126663 >> v1:172.21.15.92:6789/0 conn(0x7fcc04159ec0 legacy=0x7fcc040b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc03fff700 1 -- 172.21.15.92:0/1301126663 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040cc100 con 0x7fcc040b55a0 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc02ffd700 1 -- 172.21.15.92:0/1301126663 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949552171 0 0) 0x7fcc040b5b00 con 0x7fcc04159ec0 2023-10-06T03:08:31.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc03fff700 1 --2- 172.21.15.92:0/1301126663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc040b55a0 0x7fcc040ce880 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009e60 tx=0x7fcbfc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6364deb9d922fc51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc02ffd700 1 -- 172.21.15.92:0/1301126663 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc00e070 con 0x7fcc040b55a0 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc02ffd700 1 -- 172.21.15.92:0/1301126663 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc00bc30 con 0x7fcc040b55a0 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc02ffd700 1 -- 172.21.15.92:0/1301126663 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc012430 con 0x7fcc040b55a0 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/1301126663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc040b55a0 msgr2=0x7fcc040ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/1301126663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc040b55a0 0x7fcc040ce880 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009e60 tx=0x7fcbfc009780 comp rx=0 tx=0).stop 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/1301126663 shutdown_connections 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/1301126663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc040b55a0 0x7fcc040ce880 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/1301126663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc040cd910 0x7fcc040ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.928+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/1301126663 >> 172.21.15.92:0/1301126663 conn(0x7fcc040999f0 msgr2=0x7fcc040b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.929+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/1301126663 shutdown_connections 2023-10-06T03:08:31.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.929+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/1301126663 wait complete. 2023-10-06T03:08:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 Processor -- start 2023-10-06T03:08:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 -- start start 2023-10-06T03:08:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040b55a0 0x7fcc0409be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:31.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc04159ec0 0x7fcc0416d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 0x7fcc04156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0416ded0 con 0x7fcc041699e0 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0409c4d0 con 0x7fcc040b55a0 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc0a1d5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0409c7b0 con 0x7fcc04159ec0 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 0x7fcc04156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc03fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 0x7fcc04156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55024/0 (socket says 172.21.15.92:55024) 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.930+0000 7fcc03fff700 1 -- 172.21.15.92:0/3510355823 learned_addr learned my addr 172.21.15.92:0/3510355823 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcbfbfff700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc04159ec0 0x7fcc0416d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc03fff700 1 -- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc04159ec0 msgr2=0x7fcc0416d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc03fff700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc04159ec0 0x7fcc0416d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc037fe700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040b55a0 0x7fcc0409be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:31.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc03fff700 1 -- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040b55a0 msgr2=0x7fcc0409be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc03fff700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040b55a0 0x7fcc0409be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc03fff700 1 -- 172.21.15.92:0/3510355823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009a20 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcbfbfff700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc04159ec0 0x7fcc0416d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc03fff700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 0x7fcc04156d10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc013000 tx=0x7fcbfc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc012e50 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc01d630 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0409ca90 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc01d9f0 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.931+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc04069360 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.932+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc04063c90 con 0x7fcc041699e0 2023-10-06T03:08:31.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.932+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcbfc024070 con 0x7fcc041699e0 2023-10-06T03:08:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.933+0000 7fcbfb7fe700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcbe4067320 0x7fcbe40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.933+0000 7fcc037fe700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcbe4067320 0x7fcbe40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.933+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcbfc0876c0 con 0x7fcc041699e0 2023-10-06T03:08:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.933+0000 7fcc037fe700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcbe4067320 0x7fcbe40697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcc0409bba0 tx=0x7fcbf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:31.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:31.937+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbfc0541f0 con 0x7fcc041699e0 2023-10-06T03:08:32.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.164+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcc04069550 con 0x7fcc041699e0 2023-10-06T03:08:32.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.987+0000 7fcbfb7fe700 1 -- 172.21.15.92:0/3510355823 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 136+0+0 (secure 0 0 0) 0x7fcbfc057810 con 0x7fcc041699e0 2023-10-06T03:08:32.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcbe4067320 msgr2=0x7fcbe40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:32.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcbe4067320 0x7fcbe40697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcc0409bba0 tx=0x7fcbf4009040 comp rx=0 tx=0).stop 2023-10-06T03:08:32.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 msgr2=0x7fcc04156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 0x7fcc04156d10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc013000 tx=0x7fcbfc019000 comp rx=0 tx=0).stop 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 shutdown_connections 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcbe4067320 0x7fcbe40697d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc041699e0 0x7fcc04156d10 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc040b55a0 0x7fcc0409be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 --2- 172.21.15.92:0/3510355823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc04159ec0 0x7fcc0416d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 >> 172.21.15.92:0/3510355823 conn(0x7fcc040999f0 msgr2=0x7fcc040cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.989+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 shutdown_connections 2023-10-06T03:08:32.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:32.990+0000 7fcc0a1d5700 1 -- 172.21.15.92:0/3510355823 wait complete. 2023-10-06T03:08:33.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:08:33.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.126+0000 7f0c2031a700 1 Processor -- start 2023-10-06T03:08:33.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.126+0000 7f0c2031a700 1 -- start start 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c2031a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0c1806bcd0 0x7f0c180caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c2031a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c1815df50 0x7f0c1815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c2031a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1815d7e0 con 0x7f0c1806c0a0 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c2031a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1815dac0 con 0x7f0c1815df50 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c2031a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1815aa30 con 0x7f0c1806bcd0 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c1815df50 0x7f0c1815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0c1815df50 0x7f0c1815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55032/0 (socket says 172.21.15.92:55032) 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/4129726108 learned_addr learned my addr 172.21.15.92:0/4129726108 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e0b6700 1 --2- 172.21.15.92:0/4129726108 >> v2:172.21.15.136:3300/0 conn(0x7f0c1806bcd0 0x7f0c180caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/4129726108 >> v2:172.21.15.136:3300/0 conn(0x7f0c1806bcd0 msgr2=0x7f0c180caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:33.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 --2- 172.21.15.92:0/4129726108 >> v2:172.21.15.136:3300/0 conn(0x7f0c1806bcd0 0x7f0c180caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/4129726108 >> v1:172.21.15.136:6789/0 conn(0x7f0c1806c0a0 legacy=0x7f0c1815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/4129726108 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1815ad10 con 0x7f0c1815df50 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1e8b7700 1 --2- 172.21.15.92:0/4129726108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c1815e3b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0c180691a0 tx=0x7f0c10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d8846c7a5c3530 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1d0b4700 1 -- 172.21.15.92:0/4129726108 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1000e070 con 0x7f0c1815df50 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1d0b4700 1 -- 172.21.15.92:0/4129726108 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1000ba50 con 0x7f0c1815df50 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.127+0000 7f0c1d0b4700 1 -- 172.21.15.92:0/4129726108 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100092e0 con 0x7f0c1815df50 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 -- 172.21.15.92:0/4129726108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 msgr2=0x7f0c1815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 --2- 172.21.15.92:0/4129726108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c1815e3b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0c180691a0 tx=0x7f0c10009840 comp rx=0 tx=0).stop 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 -- 172.21.15.92:0/4129726108 shutdown_connections 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 --2- 172.21.15.92:0/4129726108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c1815e3b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 --2- 172.21.15.92:0/4129726108 >> v2:172.21.15.136:3300/0 conn(0x7f0c1806bcd0 0x7f0c180caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 -- 172.21.15.92:0/4129726108 >> 172.21.15.92:0/4129726108 conn(0x7f0c180ce600 msgr2=0x7f0c180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 -- 172.21.15.92:0/4129726108 shutdown_connections 2023-10-06T03:08:33.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.128+0000 7f0c2031a700 1 -- 172.21.15.92:0/4129726108 wait complete. 2023-10-06T03:08:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.129+0000 7f0c2031a700 1 Processor -- start 2023-10-06T03:08:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.129+0000 7f0c2031a700 1 -- start start 2023-10-06T03:08:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c1806bcd0 0x7f0c180c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1806c0a0 0x7f0c1815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c18154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180b4120 con 0x7f0c1815df50 2023-10-06T03:08:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180c88c0 con 0x7f0c1806bcd0 2023-10-06T03:08:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180b1830 con 0x7f0c1806c0a0 2023-10-06T03:08:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c18154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c18154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55034/0 (socket says 172.21.15.92:55034) 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/2364212085 learned_addr learned my addr 172.21.15.92:0/2364212085 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e0b6700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c1806bcd0 0x7f0c180c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1d8b5700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1806c0a0 0x7f0c1815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1806c0a0 msgr2=0x7f0c1815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1806c0a0 0x7f0c1815b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c1806bcd0 msgr2=0x7f0c180c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c1806bcd0 0x7f0c180c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 -- 172.21.15.92:0/2364212085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10009ae0 con 0x7f0c1815df50 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1d8b5700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1806c0a0 0x7f0c1815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c1e8b7700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c18154fa0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009fd0 tx=0x7f0c10018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007db0 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10007000 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c180b1b10 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.130+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1001c660 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.131+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c180b20d0 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.131+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c180b22f0 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.132+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0c10003050 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.132+0000 7f0c0e7fc700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bfc067320 0x7f0bfc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.132+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c10086870 con 0x7f0c1815df50 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.132+0000 7f0c1e0b6700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bfc067320 0x7f0bfc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.133+0000 7f0c1e0b6700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bfc067320 0x7f0bfc0697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006fd0 tx=0x7f0c14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.137+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c100532b0 con 0x7f0c1815df50 2023-10-06T03:08:33.346 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.345+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c1804c3c0 con 0x7f0bfc067320 2023-10-06T03:08:33.347 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.346+0000 7f0c0e7fc700 1 -- 172.21.15.92:0/2364212085 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0c1804c3c0 con 0x7f0bfc067320 2023-10-06T03:08:33.347 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:08:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:08:33.348 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:33.348 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bfc067320 msgr2=0x7f0bfc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bfc067320 0x7f0bfc0697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c14006fd0 tx=0x7f0c14008040 comp rx=0 tx=0).stop 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 msgr2=0x7f0c18154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c18154fa0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009fd0 tx=0x7f0c10018000 comp rx=0 tx=0).stop 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 shutdown_connections 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bfc067320 0x7f0bfc0697d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0c1815df50 0x7f0c18154fa0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c1806bcd0 0x7f0c180c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 --2- 172.21.15.92:0/2364212085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1806c0a0 0x7f0c1815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 >> 172.21.15.92:0/2364212085 conn(0x7f0c180ce600 msgr2=0x7f0c18159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:33.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 shutdown_connections 2023-10-06T03:08:33.351 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.348+0000 7f0c2031a700 1 -- 172.21.15.92:0/2364212085 wait complete. 2023-10-06T03:08:33.361 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:08:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.478+0000 7f5ea5c76700 1 Processor -- start 2023-10-06T03:08:33.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.478+0000 7f5ea5c76700 1 -- start start 2023-10-06T03:08:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5ea5c76700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5ea0159c60 0x7f5ea015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5ea5c76700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5ea5c76700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea015a160 con 0x7f5ea0159890 2023-10-06T03:08:33.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5ea5c76700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea015df50 con 0x7f5ea0159c60 2023-10-06T03:08:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5ea5c76700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00d13a0 con 0x7f5ea015db10 2023-10-06T03:08:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5e9f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5ea0159890 0x7f5ea015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51554/0 (socket says 172.21.15.92:51554) 2023-10-06T03:08:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5e9f7fe700 1 -- 172.21.15.92:0/2712085693 learned_addr learned my addr 172.21.15.92:0/2712085693 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5e9effd700 1 --2- 172.21.15.92:0/2712085693 >> v2:172.21.15.136:3300/0 conn(0x7f5ea0159c60 0x7f5ea015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.479+0000 7f5e9ffff700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9e7fc700 1 -- 172.21.15.92:0/2712085693 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725865361 0 0) 0x7f5ea015a160 con 0x7f5ea0159890 2023-10-06T03:08:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9e7fc700 1 -- 172.21.15.92:0/2712085693 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e88003570 con 0x7f5ea0159890 2023-10-06T03:08:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9effd700 1 -- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 msgr2=0x7f5ea0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9effd700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9effd700 1 -- 172.21.15.92:0/2712085693 >> v1:172.21.15.92:6789/0 conn(0x7f5ea0159890 legacy=0x7f5ea015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9effd700 1 -- 172.21.15.92:0/2712085693 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea015e140 con 0x7f5ea0159c60 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9ffff700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9effd700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159c60 0x7f5ea015d5d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e940098e0 tx=0x7f5e940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71bf6d265631fd8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9e7fc700 1 -- 172.21.15.92:0/2712085693 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94011630 con 0x7f5ea0159c60 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9e7fc700 1 -- 172.21.15.92:0/2712085693 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94011790 con 0x7f5ea0159c60 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.480+0000 7f5e9e7fc700 1 -- 172.21.15.92:0/2712085693 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94011b30 con 0x7f5ea0159c60 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 -- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159c60 msgr2=0x7f5ea015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159c60 0x7f5ea015d5d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5e940098e0 tx=0x7f5e940091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 -- 172.21.15.92:0/2712085693 shutdown_connections 2023-10-06T03:08:33.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/2712085693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159c60 0x7f5ea015d5d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 -- 172.21.15.92:0/2712085693 >> 172.21.15.92:0/2712085693 conn(0x7f5ea0068e20 msgr2=0x7f5ea00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 -- 172.21.15.92:0/2712085693 shutdown_connections 2023-10-06T03:08:33.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.481+0000 7f5ea5c76700 1 -- 172.21.15.92:0/2712085693 wait complete. 2023-10-06T03:08:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.482+0000 7f5ea5c76700 1 Processor -- start 2023-10-06T03:08:33.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.482+0000 7f5ea5c76700 1 -- start start 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159890 0x7f5ea016a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 0x7f5ea016c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00c8a80 con 0x7f5ea0159c60 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea016aec0 con 0x7f5ea0159890 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00c8190 con 0x7f5ea015db10 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 0x7f5ea016c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 0x7f5ea016c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55050/0 (socket says 172.21.15.92:55050) 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 -- 172.21.15.92:0/3415718170 learned_addr learned my addr 172.21.15.92:0/3415718170 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9f7fe700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159890 0x7f5ea016a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9ffff700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 -- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 msgr2=0x7f5ea00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 -- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159890 msgr2=0x7f5ea016a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159890 0x7f5ea016a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 -- 172.21.15.92:0/3415718170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009420 con 0x7f5ea0159c60 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9f7fe700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159890 0x7f5ea016a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e9effd700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 0x7f5ea016c570 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009ed0 tx=0x7f5e94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94011b10 con 0x7f5ea0159c60 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e940233f0 con 0x7f5ea0159c60 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e94023770 con 0x7f5ea0159c60 2023-10-06T03:08:33.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e94009420 con 0x7f5ea0159c60 2023-10-06T03:08:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.483+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea00c8710 con 0x7f5ea0159c60 2023-10-06T03:08:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.484+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea00c8bf0 con 0x7f5ea0159c60 2023-10-06T03:08:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.485+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e9402c3b0 con 0x7f5ea0159c60 2023-10-06T03:08:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.485+0000 7f5e877fe700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e8c067320 0x7f5e8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.485+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e94086290 con 0x7f5ea0159c60 2023-10-06T03:08:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.485+0000 7f5e9f7fe700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e8c067320 0x7f5e8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.485+0000 7f5e9f7fe700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e8c067320 0x7f5e8c0697d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e90005c50 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.489+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e94052e40 con 0x7f5ea0159c60 2023-10-06T03:08:33.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.670+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5ea004c3c0 con 0x7f5ea0159c60 2023-10-06T03:08:33.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.670+0000 7f5e877fe700 1 -- 172.21.15.92:0/3415718170 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e94056460 con 0x7f5ea0159c60 2023-10-06T03:08:33.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.672+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e8c067320 msgr2=0x7f5e8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.672+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e8c067320 0x7f5e8c0697d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e90005c50 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.672+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 msgr2=0x7f5ea016c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.672+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 0x7f5ea016c570 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009ed0 tx=0x7f5e94017000 comp rx=0 tx=0).stop 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 shutdown_connections 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e8c067320 0x7f5e8c0697d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ea015db10 0x7f5ea00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ea0159890 0x7f5ea016a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 --2- 172.21.15.92:0/3415718170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ea0159c60 0x7f5ea016c570 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 >> 172.21.15.92:0/3415718170 conn(0x7f5ea0068e20 msgr2=0x7f5ea00c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:33.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 shutdown_connections 2023-10-06T03:08:33.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.673+0000 7f5ea5c76700 1 -- 172.21.15.92:0/3415718170 wait complete. 2023-10-06T03:08:33.686 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:08:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.801+0000 7f3e45fa2700 1 Processor -- start 2023-10-06T03:08:33.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.801+0000 7f3e45fa2700 1 -- start start 2023-10-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e45fa2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3e4006aa40 0x7f3e400cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e45fa2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e400cc210 con 0x7f3e400c91e0 2023-10-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e45fa2700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e400cc4f0 con 0x7f3e4006bac0 2023-10-06T03:08:33.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e45fa2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e400cc7d0 con 0x7f3e4006aa40 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e3ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3e400c91e0 0x7f3e40160390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51558/0 (socket says 172.21.15.92:51558) 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e3ffff700 1 -- 172.21.15.92:0/194148924 learned_addr learned my addr 172.21.15.92:0/194148924 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e3f7fe700 1 --2- 172.21.15.92:0/194148924 >> v2:172.21.15.136:3300/0 conn(0x7f3e4006aa40 0x7f3e400cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e3effd700 1 -- 172.21.15.92:0/194148924 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4120824979 0 0) 0x7f3e400cc210 con 0x7f3e400c91e0 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.802+0000 7f3e3effd700 1 -- 172.21.15.92:0/194148924 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e2c003570 con 0x7f3e400c91e0 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3effd700 1 -- 172.21.15.92:0/194148924 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631827644 0 0) 0x7f3e2c003570 con 0x7f3e400c91e0 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/194148924 >> v1:172.21.15.136:6789/0 conn(0x7f3e4006bac0 legacy=0x7f3e4006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/194148924 >> v1:172.21.15.92:6789/0 conn(0x7f3e400c91e0 legacy=0x7f3e40160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/194148924 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400ccab0 con 0x7f3e4006aa40 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3f7fe700 1 --2- 172.21.15.92:0/194148924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e4006aa40 0x7f3e400cbc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e300098d0 tx=0x7f3e30009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee3cb0c46c036ccf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3effd700 1 -- 172.21.15.92:0/194148924 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e380017f0 con 0x7f3e4006aa40 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3effd700 1 -- 172.21.15.92:0/194148924 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30011420 con 0x7f3e4006aa40 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.803+0000 7f3e3effd700 1 -- 172.21.15.92:0/194148924 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300117c0 con 0x7f3e4006aa40 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 -- 172.21.15.92:0/194148924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e4006aa40 msgr2=0x7f3e400cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/194148924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e4006aa40 0x7f3e400cbc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3e300098d0 tx=0x7f3e30009120 comp rx=0 tx=0).stop 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 -- 172.21.15.92:0/194148924 shutdown_connections 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/194148924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e4006aa40 0x7f3e400cbc00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 -- 172.21.15.92:0/194148924 >> 172.21.15.92:0/194148924 conn(0x7f3e400cd670 msgr2=0x7f3e400cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 -- 172.21.15.92:0/194148924 shutdown_connections 2023-10-06T03:08:33.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.804+0000 7f3e45fa2700 1 -- 172.21.15.92:0/194148924 wait complete. 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.805+0000 7f3e45fa2700 1 Processor -- start 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.805+0000 7f3e45fa2700 1 -- start start 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 0x7f3e40150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e4006bac0 0x7f3e400b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e400c91e0 0x7f3e400b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400b0410 con 0x7f3e4006aa40 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e401678a0 con 0x7f3e400c91e0 2023-10-06T03:08:33.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e401669d0 con 0x7f3e4006bac0 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 0x7f3e40150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 0x7f3e40150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55056/0 (socket says 172.21.15.92:55056) 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/3638620082 learned_addr learned my addr 172.21.15.92:0/3638620082 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e4006bac0 msgr2=0x7f3e400b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e4006bac0 0x7f3e400b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e400c91e0 msgr2=0x7f3e400b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3ffff700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e400c91e0 0x7f3e400b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e400c91e0 0x7f3e400b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 -- 172.21.15.92:0/3638620082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e300093c0 con 0x7f3e4006aa40 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e3f7fe700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 0x7f3e40150d70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3e4006c6a0 tx=0x7f3e30017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30020070 con 0x7f3e4006aa40 2023-10-06T03:08:33.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3001b2e0 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e40166c90 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3001bdc0 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.806+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e40167250 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.807+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4004c3c0 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.808+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3e300196d0 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.808+0000 7f3e37fff700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e20067320 0x7f3e200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.808+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3e30087940 con 0x7f3e4006aa40 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.808+0000 7f3e36ffd700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e20067320 0x7f3e200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:33.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.808+0000 7f3e36ffd700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e20067320 0x7f3e200697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3e40151710 tx=0x7f3e28007ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:33.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:33.812+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e30054470 con 0x7f3e4006aa40 2023-10-06T03:08:34.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.023+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e401675f0 con 0x7f3e20067320 2023-10-06T03:08:34.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.024+0000 7f3e37fff700 1 -- 172.21.15.92:0/3638620082 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3e401675f0 con 0x7f3e20067320 2023-10-06T03:08:34.026 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:08:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:08:34.026 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:34.026 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:08:34.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e20067320 msgr2=0x7f3e200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e20067320 0x7f3e200697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3e40151710 tx=0x7f3e28007ac0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 msgr2=0x7f3e40150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 0x7f3e40150d70 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3e4006c6a0 tx=0x7f3e30017000 comp rx=0 tx=0).stop 2023-10-06T03:08:34.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 shutdown_connections 2023-10-06T03:08:34.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e20067320 0x7f3e200697d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3e4006aa40 0x7f3e40150d70 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e400c91e0 0x7f3e400b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 --2- 172.21.15.92:0/3638620082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e4006bac0 0x7f3e400b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 >> 172.21.15.92:0/3638620082 conn(0x7f3e400cd670 msgr2=0x7f3e40152220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:34.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 shutdown_connections 2023-10-06T03:08:34.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.026+0000 7f3e45fa2700 1 -- 172.21.15.92:0/3638620082 wait complete. 2023-10-06T03:08:34.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000001007350 2023-10-06T03:08:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.154+0000 7f9ec455c700 1 Processor -- start 2023-10-06T03:08:34.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.154+0000 7f9ec455c700 1 -- start start 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec455c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9ebc06c0a0 0x7f9ebc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec455c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc15d7e0 con 0x7f9ebc15df50 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec455c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc15dac0 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec455c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc15aa30 con 0x7f9ebc06c0a0 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec2af9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ebc15df50 0x7f9ebc0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51564/0 (socket says 172.21.15.92:51564) 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec2af9700 1 -- 172.21.15.92:0/3765740636 learned_addr learned my addr 172.21.15.92:0/3765740636 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.155+0000 7f9ec1af7700 1 --2- 172.21.15.92:0/3765740636 >> v2:172.21.15.136:3300/0 conn(0x7f9ebc06c0a0 0x7f9ebc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:34.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec12f6700 1 -- 172.21.15.92:0/3765740636 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668646629 0 0) 0x7f9ebc15d7e0 con 0x7f9ebc15df50 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec12f6700 1 -- 172.21.15.92:0/3765740636 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea8003570 con 0x7f9ebc15df50 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec1af7700 1 -- 172.21.15.92:0/3765740636 >> v1:172.21.15.136:6789/0 conn(0x7f9ebc06bcd0 legacy=0x7f9ebc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec1af7700 1 -- 172.21.15.92:0/3765740636 >> v1:172.21.15.92:6789/0 conn(0x7f9ebc15df50 legacy=0x7f9ebc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec1af7700 1 -- 172.21.15.92:0/3765740636 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc15ad10 con 0x7f9ebc06c0a0 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec1af7700 1 --2- 172.21.15.92:0/3765740636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc06c0a0 0x7f9ebc0caa90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0098e0 tx=0x7f9eac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6e53c922c51502 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec12f6700 1 -- 172.21.15.92:0/3765740636 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00b640 con 0x7f9ebc06c0a0 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec12f6700 1 -- 172.21.15.92:0/3765740636 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac00b7a0 con 0x7f9ebc06c0a0 2023-10-06T03:08:34.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.156+0000 7f9ec12f6700 1 -- 172.21.15.92:0/3765740636 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac00bb40 con 0x7f9ebc06c0a0 2023-10-06T03:08:34.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 -- 172.21.15.92:0/3765740636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc06c0a0 msgr2=0x7f9ebc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 --2- 172.21.15.92:0/3765740636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc06c0a0 0x7f9ebc0caa90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0098e0 tx=0x7f9eac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 -- 172.21.15.92:0/3765740636 shutdown_connections 2023-10-06T03:08:34.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 --2- 172.21.15.92:0/3765740636 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc06c0a0 0x7f9ebc0caa90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 -- 172.21.15.92:0/3765740636 >> 172.21.15.92:0/3765740636 conn(0x7f9ebc0ce600 msgr2=0x7f9ebc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:34.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 -- 172.21.15.92:0/3765740636 shutdown_connections 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.157+0000 7f9ec455c700 1 -- 172.21.15.92:0/3765740636 wait complete. 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.158+0000 7f9ec455c700 1 Processor -- start 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.158+0000 7f9ec455c700 1 -- start start 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec455c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 0x7f9ebc0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec455c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ebc06c0a0 0x7f9ebc0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec455c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc15df50 0x7f9ebc14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec455c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b2750 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec455c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b18e0 con 0x7f9ebc15df50 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec455c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0b1b40 con 0x7f9ebc06c0a0 2023-10-06T03:08:34.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 0x7f9ebc0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:34.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 0x7f9ebc0c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55064/0 (socket says 172.21.15.92:55064) 2023-10-06T03:08:34.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 -- 172.21.15.92:0/549864909 learned_addr learned my addr 172.21.15.92:0/549864909 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:34.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec1af7700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ebc06c0a0 0x7f9ebc0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:34.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec2af9700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc15df50 0x7f9ebc14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 -- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ebc06c0a0 msgr2=0x7f9ebc0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ebc06c0a0 0x7f9ebc0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 -- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc15df50 msgr2=0x7f9ebc14c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc15df50 0x7f9ebc14c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 -- 172.21.15.92:0/549864909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac009420 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec2af9700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc15df50 0x7f9ebc14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.159+0000 7f9ec22f8700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 0x7f9ebc0c8260 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c4e0 tx=0x7f9eb800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.160+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8011590 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.160+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8011710 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.160+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8011ad0 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.160+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc0b1e20 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.160+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc0b23e0 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.161+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc04c3c0 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.161+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9eb801b3b0 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.161+0000 7f9eb27fc700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ea0067320 0x7f9ea00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.161+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9eb8084240 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.161+0000 7f9ec1af7700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ea0067320 0x7f9ea00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:34.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.162+0000 7f9ec1af7700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ea0067320 0x7f9ea00697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0093f0 tx=0x7f9eac017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:34.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.165+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9eb8050d70 con 0x7f9ebc06bcd0 2023-10-06T03:08:34.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:34.377+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000001007350", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ebc0b2890 con 0x7f9ea0067320 2023-10-06T03:08:37.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.015+0000 7f9eb27fc700 1 -- 172.21.15.92:0/549864909 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f9eac00b910 con 0x7f9ea0067320 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.019+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ea0067320 msgr2=0x7f9ea00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.019+0000 7f9ec455c700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ea0067320 0x7f9ea00697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0093f0 tx=0x7f9eac017000 comp rx=0 tx=0).stop 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.019+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 msgr2=0x7f9ebc0c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.019+0000 7f9ec455c700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 0x7f9ebc0c8260 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c4e0 tx=0x7f9eb800c7f0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 shutdown_connections 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ea0067320 0x7f9ea00697d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ebc06bcd0 0x7f9ebc0c8260 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ebc15df50 0x7f9ebc14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 --2- 172.21.15.92:0/549864909 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ebc06c0a0 0x7f9ebc0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 >> 172.21.15.92:0/549864909 conn(0x7f9ebc0ce600 msgr2=0x7f9ebc15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 shutdown_connections 2023-10-06T03:08:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.020+0000 7f9ec455c700 1 -- 172.21.15.92:0/549864909 wait complete. 2023-10-06T03:08:37.026 INFO:teuthology.orchestra.run.smithi092.stderr:Volume created successfully (no MDS daemons created) 2023-10-06T03:08:37.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000001007351 2023-10-06T03:08:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 Processor -- start 2023-10-06T03:08:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 -- start start 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e900b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7e900b33b0 0x7f7e900b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e900b2080 con 0x7f7e900b4e70 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e900cfb00 con 0x7f7e900b33b0 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900b5c40 con 0x7f7e900b16d0 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7e900b4e70 0x7f7e900ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:51568/0 (socket says 172.21.15.92:51568) 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8effd700 1 -- 172.21.15.92:0/2638565093 learned_addr learned my addr 172.21.15.92:0/2638565093 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8e7fc700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e900b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:37.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8f7fe700 1 --2- 172.21.15.92:0/2638565093 >> v2:172.21.15.136:3300/0 conn(0x7f7e900b33b0 0x7f7e900b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8dffb700 1 -- 172.21.15.92:0/2638565093 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133583352 0 0) 0x7f7e900b2080 con 0x7f7e900b4e70 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8dffb700 1 -- 172.21.15.92:0/2638565093 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7c003570 con 0x7f7e900b4e70 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8dffb700 1 -- 172.21.15.92:0/2638565093 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854923854 0 0) 0x7f7e7c003570 con 0x7f7e900b4e70 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 msgr2=0x7f7e900b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8f7fe700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e900b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/2638565093 >> v1:172.21.15.92:6789/0 conn(0x7f7e900b4e70 legacy=0x7f7e900ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/2638565093 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900b5f20 con 0x7f7e900b33b0 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8e7fc700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e900b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.163+0000 7f7e8f7fe700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900b3830 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7e800098e0 tx=0x7f7e800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86992c434a17298 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8dffb700 1 -- 172.21.15.92:0/2638565093 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880017f0 con 0x7f7e900b33b0 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8dffb700 1 -- 172.21.15.92:0/2638565093 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80011420 con 0x7f7e900b33b0 2023-10-06T03:08:37.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8dffb700 1 -- 172.21.15.92:0/2638565093 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800117c0 con 0x7f7e900b33b0 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 -- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 msgr2=0x7f7e900b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900b3830 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7e800098e0 tx=0x7f7e800091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 -- 172.21.15.92:0/2638565093 shutdown_connections 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900b3830 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/2638565093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e900b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 -- 172.21.15.92:0/2638565093 >> 172.21.15.92:0/2638565093 conn(0x7f7e900d10c0 msgr2=0x7f7e900affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 -- 172.21.15.92:0/2638565093 shutdown_connections 2023-10-06T03:08:37.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.164+0000 7f7e8ffff700 1 -- 172.21.15.92:0/2638565093 wait complete. 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.165+0000 7f7e8ffff700 1 Processor -- start 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.165+0000 7f7e8ffff700 1 -- start start 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e90145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 0x7f7e900cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e90145cd0 con 0x7f7e900b4e70 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900c8260 con 0x7f7e900b33b0 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e900c8540 con 0x7f7e900b16d0 2023-10-06T03:08:37.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 0x7f7e900cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 0x7f7e900cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55068/0 (socket says 172.21.15.92:55068) 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/3436881574 learned_addr learned my addr 172.21.15.92:0/3436881574 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.166+0000 7f7e8e7fc700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8effd700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e90145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 msgr2=0x7f7e90145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8f7fe700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e90145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 msgr2=0x7f7e900cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8f7fe700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8f7fe700 1 -- 172.21.15.92:0/3436881574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009420 con 0x7f7e900b4e70 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8e7fc700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8effd700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e90145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:37.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8f7fe700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 0x7f7e900cb0f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7e800093f0 tx=0x7f7e80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001ccd0 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8001ce30 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e80009420 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8001a6a0 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.167+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e900c8b00 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.168+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9006a1f0 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.168+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7e8001aa50 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.169+0000 7f7e86ffd700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e70067320 0x7f7e700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.169+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6956+0+0 (secure 0 0 0) 0x7f7e80086ce0 con 0x7f7e900b4e70 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.169+0000 7f7e8effd700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e70067320 0x7f7e700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:37.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.169+0000 7f7e8effd700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e70067320 0x7f7e700697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7e880019a0 tx=0x7f7e88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:37.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.173+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e80053490 con 0x7f7e900b4e70 2023-10-06T03:08:37.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:37.385+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000001007351", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e90069840 con 0x7f7e70067320 2023-10-06T03:08:40.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.012+0000 7f7e86ffd700 1 -- 172.21.15.92:0/3436881574 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f7e90069840 con 0x7f7e70067320 2023-10-06T03:08:40.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.016+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e70067320 msgr2=0x7f7e700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.016+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e70067320 0x7f7e700697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7e880019a0 tx=0x7f7e88009040 comp rx=0 tx=0).stop 2023-10-06T03:08:40.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.016+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 msgr2=0x7f7e900cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.016+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 0x7f7e900cb0f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7e800093f0 tx=0x7f7e80016000 comp rx=0 tx=0).stop 2023-10-06T03:08:40.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 shutdown_connections 2023-10-06T03:08:40.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7e70067320 0x7f7e700697d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7e900b4e70 0x7f7e900cb0f0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e900b16d0 0x7f7e90145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 --2- 172.21.15.92:0/3436881574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e900b33b0 0x7f7e900cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 >> 172.21.15.92:0/3436881574 conn(0x7f7e900d10c0 msgr2=0x7f7e90069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:40.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.017+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 shutdown_connections 2023-10-06T03:08:40.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.018+0000 7f7e8ffff700 1 -- 172.21.15.92:0/3436881574 wait complete. 2023-10-06T03:08:40.020 INFO:teuthology.orchestra.run.smithi092.stderr:Volume created successfully (no MDS daemons created) 2023-10-06T03:08:40.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:08:40.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.165+0000 7f37de0d0700 1 Processor -- start 2023-10-06T03:08:40.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.165+0000 7f37de0d0700 1 -- start start 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37de0d0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159c60 0x7f37d815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37de0d0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37d815db10 0x7f37d8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37de0d0700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d815a160 con 0x7f37d8159890 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37de0d0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d80d13a0 con 0x7f37d815db10 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37de0d0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d815e140 con 0x7f37d8159c60 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37d815db10 0x7f37d8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f37d815db10 0x7f37d8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40200/0 (socket says 172.21.15.92:40200) 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 -- 172.21.15.92:0/1636287682 learned_addr learned my addr 172.21.15.92:0/1636287682 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d6ffd700 1 --2- 172.21.15.92:0/1636287682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159c60 0x7f37d815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 -- 172.21.15.92:0/1636287682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159c60 msgr2=0x7f37d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 --2- 172.21.15.92:0/1636287682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159c60 0x7f37d815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 -- 172.21.15.92:0/1636287682 >> v1:172.21.15.136:6789/0 conn(0x7f37d8159890 legacy=0x7f37d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 -- 172.21.15.92:0/1636287682 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d815a3d0 con 0x7f37d815db10 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d7fff700 1 --2- 172.21.15.92:0/1636287682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d8069560 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009f20 tx=0x7f37d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bb4ca0885d1399 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d67fc700 1 -- 172.21.15.92:0/1636287682 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000e070 con 0x7f37d815db10 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d67fc700 1 -- 172.21.15.92:0/1636287682 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d000ba80 con 0x7f37d815db10 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.166+0000 7f37d67fc700 1 -- 172.21.15.92:0/1636287682 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00092e0 con 0x7f37d815db10 2023-10-06T03:08:40.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 -- 172.21.15.92:0/1636287682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 msgr2=0x7f37d8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1636287682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d8069560 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009f20 tx=0x7f37d0009840 comp rx=0 tx=0).stop 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 -- 172.21.15.92:0/1636287682 shutdown_connections 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1636287682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d8069560 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1636287682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159c60 0x7f37d815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 -- 172.21.15.92:0/1636287682 >> 172.21.15.92:0/1636287682 conn(0x7f37d8068e20 msgr2=0x7f37d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 -- 172.21.15.92:0/1636287682 shutdown_connections 2023-10-06T03:08:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.167+0000 7f37de0d0700 1 -- 172.21.15.92:0/1636287682 wait complete. 2023-10-06T03:08:40.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.168+0000 7f37de0d0700 1 Processor -- start 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.168+0000 7f37de0d0700 1 -- start start 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37de0d0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159890 0x7f37d80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37de0d0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37d8159c60 0x7f37d816d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37de0d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d80cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37de0d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d816e450 con 0x7f37d815db10 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37de0d0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80b0c50 con 0x7f37d8159c60 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37de0d0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80b0f30 con 0x7f37d8159890 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d80cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d80cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40202/0 (socket says 172.21.15.92:40202) 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 -- 172.21.15.92:0/1396215336 learned_addr learned my addr 172.21.15.92:0/1396215336 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:40.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d6ffd700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37d8159c60 0x7f37d816d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d77fe700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159890 0x7f37d80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 -- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159890 msgr2=0x7f37d80b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159890 0x7f37d80b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 -- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37d8159c60 msgr2=0x7f37d816d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37d8159c60 0x7f37d816d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d7fff700 1 -- 172.21.15.92:0/1396215336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0009ae0 con 0x7f37d815db10 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d6ffd700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37d8159c60 0x7f37d816d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.169+0000 7f37d77fe700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159890 0x7f37d80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.170+0000 7f37d7fff700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d80cdce0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b6c0 tx=0x7f37d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.170+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00032f0 con 0x7f37d815db10 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.170+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0008e80 con 0x7f37d815db10 2023-10-06T03:08:40.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.170+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d80b1210 con 0x7f37d815db10 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.170+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d001c620 con 0x7f37d815db10 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.170+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d80b17d0 con 0x7f37d815db10 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.171+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d804c3c0 con 0x7f37d815db10 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.171+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37d0003040 con 0x7f37d815db10 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.172+0000 7f37bf7fe700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37c4067320 0x7f37c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.172+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7854+0+0 (secure 0 0 0) 0x7f37d0087340 con 0x7f37d815db10 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.172+0000 7f37d77fe700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37c4067320 0x7f37c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.172+0000 7f37d77fe700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37c4067320 0x7f37c40697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f37c8007fd0 tx=0x7f37c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:40.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.176+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37d00536e0 con 0x7f37d815db10 2023-10-06T03:08:40.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.386+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f37d806a5b0 con 0x7f37c4067320 2023-10-06T03:08:40.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.388+0000 7f37bf7fe700 1 -- 172.21.15.92:0/1396215336 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+151 (secure 0 0 0) 0x7f37d806a5b0 con 0x7f37c4067320 2023-10-06T03:08:40.388 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:08:40.388 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volume_0000000001007350" 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volume_0000000001007351" 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:40.389 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37c4067320 msgr2=0x7f37c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37c4067320 0x7f37c40697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f37c8007fd0 tx=0x7f37c8009040 comp rx=0 tx=0).stop 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 msgr2=0x7f37d80cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d80cdce0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b6c0 tx=0x7f37d0018000 comp rx=0 tx=0).stop 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 shutdown_connections 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37c4067320 0x7f37c40697d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37d815db10 0x7f37d80cdce0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37d8159890 0x7f37d80b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 --2- 172.21.15.92:0/1396215336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37d8159c60 0x7f37d816d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 >> 172.21.15.92:0/1396215336 conn(0x7f37d8068e20 msgr2=0x7f37d8153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:40.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 shutdown_connections 2023-10-06T03:08:40.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.390+0000 7f37de0d0700 1 -- 172.21.15.92:0/1396215336 wait complete. 2023-10-06T03:08:40.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000001007350 --yes-i-really-mean-it 2023-10-06T03:08:40.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.518+0000 7fe5bb743700 1 Processor -- start 2023-10-06T03:08:40.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.518+0000 7fe5bb743700 1 -- start start 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.519+0000 7fe5bb743700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.519+0000 7fe5bb743700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 0x7fe5b4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.519+0000 7fe5bb743700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b415c2a0 con 0x7fe5b40c8e00 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.519+0000 7fe5bb743700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b415b690 con 0x7fe5b406c0a0 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.519+0000 7fe5bb743700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b415b970 con 0x7fe5b406bcd0 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b8cde700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 0x7fe5b4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b4140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:46566/0 (socket says 172.21.15.92:46566) 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b8cde700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 0x7fe5b4141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:54802/0 (socket says 172.21.15.92:54802) 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 -- 172.21.15.92:0/4127978811 learned_addr learned my addr 172.21.15.92:0/4127978811 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b9ce0700 1 --1- 172.21.15.92:0/4127978811 >> v1:172.21.15.136:6789/0 conn(0x7fe5b40c8e00 0x7fe5b40c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:44998/0 (socket says 172.21.15.92:44998) 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 -- 172.21.15.92:0/4127978811 >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 msgr2=0x7fe5b4141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 --2- 172.21.15.92:0/4127978811 >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 0x7fe5b4141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 -- 172.21.15.92:0/4127978811 >> v1:172.21.15.136:6789/0 conn(0x7fe5b40c8e00 legacy=0x7fe5b40c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 -- 172.21.15.92:0/4127978811 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b415bc50 con 0x7fe5b406bcd0 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b8cde700 1 --2- 172.21.15.92:0/4127978811 >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 0x7fe5b4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5b94df700 1 --2- 172.21.15.92:0/4127978811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b4140df0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00099b0 tx=0x7fe5b0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1b61f4a70aba30c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:40.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5abfff700 1 -- 172.21.15.92:0/4127978811 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0009ea0 con 0x7fe5b406bcd0 2023-10-06T03:08:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5abfff700 1 -- 172.21.15.92:0/4127978811 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b000b860 con 0x7fe5b406bcd0 2023-10-06T03:08:40.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.520+0000 7fe5abfff700 1 -- 172.21.15.92:0/4127978811 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b000bc00 con 0x7fe5b406bcd0 2023-10-06T03:08:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 -- 172.21.15.92:0/4127978811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 msgr2=0x7fe5b4140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 --2- 172.21.15.92:0/4127978811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b4140df0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00099b0 tx=0x7fe5b0009250 comp rx=0 tx=0).stop 2023-10-06T03:08:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 -- 172.21.15.92:0/4127978811 shutdown_connections 2023-10-06T03:08:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 --2- 172.21.15.92:0/4127978811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b4140df0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 --2- 172.21.15.92:0/4127978811 >> v2:172.21.15.136:3300/0 conn(0x7fe5b406c0a0 0x7fe5b4141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 -- 172.21.15.92:0/4127978811 >> 172.21.15.92:0/4127978811 conn(0x7fe5b40ce600 msgr2=0x7fe5b40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 -- 172.21.15.92:0/4127978811 shutdown_connections 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.521+0000 7fe5bb743700 1 -- 172.21.15.92:0/4127978811 wait complete. 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.522+0000 7fe5bb743700 1 Processor -- start 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.522+0000 7fe5bb743700 1 -- start start 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5bb743700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5bb743700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 0x7fe5b413d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5bb743700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b40c8e00 0x7fe5b40cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5bb743700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b413d680 con 0x7fe5b406c0a0 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5bb743700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b413d890 con 0x7fe5b40c8e00 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5bb743700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b413db70 con 0x7fe5b406bcd0 2023-10-06T03:08:40.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 0x7fe5b413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 0x7fe5b413d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40208/0 (socket says 172.21.15.92:40208) 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 -- 172.21.15.92:0/1025431784 learned_addr learned my addr 172.21.15.92:0/1025431784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b94df700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b9ce0700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b40c8e00 0x7fe5b40cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 -- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 msgr2=0x7fe5b41566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b41566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 -- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b40c8e00 msgr2=0x7fe5b40cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.523+0000 7fe5b8cde700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b40c8e00 0x7fe5b40cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5b8cde700 1 -- 172.21.15.92:0/1025431784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a4009420 con 0x7fe5b406c0a0 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5b8cde700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 0x7fe5b413d140 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400ec80 tx=0x7fe5a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400c920 con 0x7fe5b406c0a0 2023-10-06T03:08:40.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a400ca80 con 0x7fe5b406c0a0 2023-10-06T03:08:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00094f0 con 0x7fe5b406c0a0 2023-10-06T03:08:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400ce00 con 0x7fe5b406c0a0 2023-10-06T03:08:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.524+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b413e130 con 0x7fe5b406c0a0 2023-10-06T03:08:40.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.525+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b40c65e0 con 0x7fe5b406c0a0 2023-10-06T03:08:40.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.525+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe5a4013680 con 0x7fe5b406c0a0 2023-10-06T03:08:40.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.525+0000 7fe5a97fa700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe598067320 0x7fe5980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:40.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.526+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 7854+0+0 (secure 0 0 0) 0x7fe5a40858b0 con 0x7fe5b406c0a0 2023-10-06T03:08:40.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.526+0000 7fe5b94df700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe598067320 0x7fe5980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:40.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.526+0000 7fe5b94df700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe598067320 0x7fe5980697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00099b0 tx=0x7fe5b0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:40.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.530+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5b40c65e0 con 0x7fe5b406c0a0 2023-10-06T03:08:40.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:40.741+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000001007350", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5b40c65e0 con 0x7fe598067320 2023-10-06T03:08:40.745 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:08:40.744+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:08:40.768 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:08:42.728 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:08:42.731 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:08:43.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.770+0000 7fe5a97fa700 1 -- 172.21.15.92:0/1025431784 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7fe5b40c65e0 con 0x7fe598067320 2023-10-06T03:08:43.771 INFO:teuthology.orchestra.run.smithi092.stdout:metadata pool: cephfs.volume_0000000001007350.meta data pool: ['cephfs.volume_0000000001007350.data'] removed 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe598067320 msgr2=0x7fe5980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe598067320 0x7fe5980697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00099b0 tx=0x7fe5b0017000 comp rx=0 tx=0).stop 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 msgr2=0x7fe5b413d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 0x7fe5b413d140 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400ec80 tx=0x7fe5a40091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 shutdown_connections 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe598067320 0x7fe5980697d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b40c8e00 0x7fe5b40cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5b406bcd0 0x7fe5b41566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 --2- 172.21.15.92:0/1025431784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5b406c0a0 0x7fe5b413d140 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 >> 172.21.15.92:0/1025431784 conn(0x7fe5b40ce600 msgr2=0x7fe5b413c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 shutdown_connections 2023-10-06T03:08:43.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.773+0000 7fe5bb743700 1 -- 172.21.15.92:0/1025431784 wait complete. 2023-10-06T03:08:43.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000001007351 --yes-i-really-mean-it 2023-10-06T03:08:43.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.918+0000 7f07df145700 1 Processor -- start 2023-10-06T03:08:43.918 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.918+0000 7f07df145700 1 -- start start 2023-10-06T03:08:43.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.918+0000 7f07df145700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f07d8159890 0x7f07d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.918+0000 7f07df145700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d815a160 con 0x7f07d8069560 2023-10-06T03:08:43.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.918+0000 7f07df145700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d80d13a0 con 0x7f07d8159c60 2023-10-06T03:08:43.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.918+0000 7f07df145700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d815e140 con 0x7f07d8159890 2023-10-06T03:08:43.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dd6e2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f07d8069560 0x7f07d80ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56676/0 (socket says 172.21.15.92:56676) 2023-10-06T03:08:43.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dd6e2700 1 -- 172.21.15.92:0/900760203 learned_addr learned my addr 172.21.15.92:0/900760203 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 --2- 172.21.15.92:0/900760203 >> v2:172.21.15.136:3300/0 conn(0x7f07d8159890 0x7f07d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798010566 0 0) 0x7f07d815a160 con 0x7f07d8069560 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c0003570 con 0x7f07d8069560 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3568895802 0 0) 0x7f07c0003570 con 0x7f07d8069560 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 >> v2:172.21.15.136:3300/0 conn(0x7f07d8159890 msgr2=0x7f07d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 --2- 172.21.15.92:0/900760203 >> v2:172.21.15.136:3300/0 conn(0x7f07d8159890 0x7f07d815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 >> v1:172.21.15.136:6789/0 conn(0x7f07d8159c60 legacy=0x7f07d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d815a3d0 con 0x7f07d8069560 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f07d0001470 con 0x7f07d8069560 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 >> v1:172.21.15.92:6789/0 conn(0x7f07d8069560 legacy=0x7f07d80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07c0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07c0004810 0x7f07c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07c00051c0 0x7f07c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d815e140 con 0x7f07c0003a60 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c000c0d0 con 0x7f07c00051c0 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c000c290 con 0x7f07c0004810 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07c0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dd6e2700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07c00051c0 0x7f07c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07cffff700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07c0004810 0x7f07c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 -- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07c0004810 msgr2=0x7f07c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07c0004810 0x7f07c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 -- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07c00051c0 msgr2=0x7f07c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07c00051c0 0x7f07c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dd6e2700 1 -- 172.21.15.92:0/900760203 reap_dead start 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.919+0000 7f07dcee1700 1 -- 172.21.15.92:0/900760203 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0003570 con 0x7f07c0003a60 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07dcee1700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07c0003e70 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f07d400e210 tx=0x7f07d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcfcdaa287c42 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d400eac0 con 0x7f07c0003a60 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d400f040 con 0x7f07c0003a60 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07cf7fe700 1 -- 172.21.15.92:0/900760203 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d400ee70 con 0x7f07c0003a60 2023-10-06T03:08:43.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 -- 172.21.15.92:0/900760203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 msgr2=0x7f07c0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07c0003e70 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f07d400e210 tx=0x7f07d400b040 comp rx=0 tx=0).stop 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 -- 172.21.15.92:0/900760203 shutdown_connections 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 --2- 172.21.15.92:0/900760203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07c0003e70 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 -- 172.21.15.92:0/900760203 >> 172.21.15.92:0/900760203 conn(0x7f07d8068e20 msgr2=0x7f07d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 -- 172.21.15.92:0/900760203 shutdown_connections 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.920+0000 7f07df145700 1 -- 172.21.15.92:0/900760203 wait complete. 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.921+0000 7f07df145700 1 Processor -- start 2023-10-06T03:08:43.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 -- start start 2023-10-06T03:08:43.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07d81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07d8151b00 0x7f07d81554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07d814c2e0 0x7f07d80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80cfc70 con 0x7f07c0003a60 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8151670 con 0x7f07d814c2e0 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07df145700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8150a60 con 0x7f07d8151b00 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07dcee1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07d81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07dcee1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07d81503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40256/0 (socket says 172.21.15.92:40256) 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07dcee1700 1 -- 172.21.15.92:0/3989166199 learned_addr learned my addr 172.21.15.92:0/3989166199 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07cffff700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07d8151b00 0x7f07d81554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.922+0000 7f07dd6e2700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07d814c2e0 0x7f07d80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07dcee1700 1 -- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07d8151b00 msgr2=0x7f07d81554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07dcee1700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07d8151b00 0x7f07d81554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07dcee1700 1 -- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07d814c2e0 msgr2=0x7f07d80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07dcee1700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07d814c2e0 0x7f07d80ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07dcee1700 1 -- 172.21.15.92:0/3989166199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8150d40 con 0x7f07c0003a60 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07dcee1700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07d81503d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f07d400afd0 tx=0x7f07d401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4014e80 con 0x7f07c0003a60 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d4029450 con 0x7f07c0003a60 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d8151020 con 0x7f07c0003a60 2023-10-06T03:08:43.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40297f0 con 0x7f07c0003a60 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.923+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d81559e0 con 0x7f07c0003a60 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.924+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d804c3c0 con 0x7f07c0003a60 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.925+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f07d4003040 con 0x7f07c0003a60 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.925+0000 7f07ccff9700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f07c4067320 0x7f07c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.925+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7042+0+0 (secure 0 0 0) 0x7f07d40875f0 con 0x7f07c0003a60 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.925+0000 7f07cffff700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f07c4067320 0x7f07c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:43.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.925+0000 7f07cffff700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f07c4067320 0x7f07c40697d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f07c8007fd0 tx=0x7f07c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:43.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:43.929+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07d4053c30 con 0x7f07c0003a60 2023-10-06T03:08:44.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:44.143+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000001007351", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f07d81517b0 con 0x7f07c4067320 2023-10-06T03:08:44.147 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:08:44.146+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:08:44.177 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:08:47.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.160+0000 7f07ccff9700 1 -- 172.21.15.92:0/3989166199 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f07c8006820 con 0x7f07c4067320 2023-10-06T03:08:47.161 INFO:teuthology.orchestra.run.smithi092.stdout:metadata pool: cephfs.volume_0000000001007351.meta data pool: ['cephfs.volume_0000000001007351.data'] removed 2023-10-06T03:08:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f07c4067320 msgr2=0x7f07c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f07c4067320 0x7f07c40697d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f07c8007fd0 tx=0x7f07c8009040 comp rx=0 tx=0).stop 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 msgr2=0x7f07d81503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07d81503d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f07d400afd0 tx=0x7f07d401a000 comp rx=0 tx=0).stop 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 shutdown_connections 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f07c4067320 0x7f07c40697d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07c0003a60 0x7f07d81503d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07d814c2e0 0x7f07d80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 --2- 172.21.15.92:0/3989166199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07d8151b00 0x7f07d81554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.163+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 >> 172.21.15.92:0/3989166199 conn(0x7f07d8068e20 msgr2=0x7f07d815c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.164+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 shutdown_connections 2023-10-06T03:08:47.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.164+0000 7f07df145700 1 -- 172.21.15.92:0/3989166199 wait complete. 2023-10-06T03:08:47.179 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:08:47.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:47.222 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:47.260 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:08:47.261 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:08:47.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:08:47.261 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:08:47.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:08:47.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:47.303 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:08:47.359 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:08:47.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:08:47.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:08:47.407 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:08:47.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:08:47.408 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:08:47.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:47.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.538+0000 7f404ef24700 1 Processor -- start 2023-10-06T03:08:47.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.538+0000 7f404ef24700 1 -- start start 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.539+0000 7f404ef24700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 0x7f40480699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.539+0000 7f404ef24700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4048069ee0 con 0x7f4048159c60 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.539+0000 7f404ef24700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404815a160 con 0x7f4048159890 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.539+0000 7f404ef24700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40480d13a0 con 0x7f4048069560 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.539+0000 7f403ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4048159c60 0x7f404815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56692/0 (socket says 172.21.15.92:56692) 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.539+0000 7f403ffff700 1 -- 172.21.15.92:0/1921876626 learned_addr learned my addr 172.21.15.92:0/1921876626 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:47.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100712863 0 0) 0x7f4048069ee0 con 0x7f4048159c60 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4030003570 con 0x7f4048159c60 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404d4c1700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 0x7f40480699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444650503 0 0) 0x7f4030003570 con 0x7f4048159c60 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 msgr2=0x7f40480699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 0x7f40480699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 >> v1:172.21.15.136:6789/0 conn(0x7f4048159890 legacy=0x7f404815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404815e140 con 0x7f4048159c60 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f4038001470 con 0x7f4048159c60 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 >> v1:172.21.15.92:6789/0 conn(0x7f4048159c60 legacy=0x7f404815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f4030003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4030004810 0x7f4030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 0x7f40300051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40480d13a0 con 0x7f4030003a90 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403000bb90 con 0x7f4030004810 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403000bd50 con 0x7f4048069560 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404ccc0700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f4030003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404d4c1700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 0x7f40300051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f403ffff700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4030004810 0x7f4030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404ccc0700 1 -- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 msgr2=0x7f40300051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404ccc0700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4048069560 0x7f40300051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404ccc0700 1 -- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4030004810 msgr2=0x7f4030004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404ccc0700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4030004810 0x7f4030004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.540+0000 7f404ccc0700 1 -- 172.21.15.92:0/1921876626 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030003570 con 0x7f4030003a90 2023-10-06T03:08:47.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ccc0700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f4030003ea0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4044001f70 tx=0x7f4044001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ba67336ab2f064 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044004070 con 0x7f4030003a90 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404400c8b0 con 0x7f4030003a90 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f403f7fe700 1 -- 172.21.15.92:0/1921876626 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044001930 con 0x7f4030003a90 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ef24700 1 -- 172.21.15.92:0/1921876626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 msgr2=0x7f4030003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ef24700 1 --2- 172.21.15.92:0/1921876626 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f4030003ea0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4044001f70 tx=0x7f4044001640 comp rx=0 tx=0).stop 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404d4c1700 1 -- 172.21.15.92:0/1921876626 reap_dead start 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ef24700 1 -- 172.21.15.92:0/1921876626 shutdown_connections 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ef24700 1 -- 172.21.15.92:0/1921876626 >> 172.21.15.92:0/1921876626 conn(0x7f4048068e20 msgr2=0x7f40480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ef24700 1 -- 172.21.15.92:0/1921876626 shutdown_connections 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.541+0000 7f404ef24700 1 -- 172.21.15.92:0/1921876626 wait complete. 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.542+0000 7f404ef24700 1 Processor -- start 2023-10-06T03:08:47.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 -- start start 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f40480b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40480ca800 0x7f40480ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4048146e70 0x7f4048147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40480c9260 con 0x7f4030003a90 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048147850 con 0x7f4048146e70 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ef24700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4048169cc0 con 0x7f40480ca800 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ccc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f40480b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ccc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f40480b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40276/0 (socket says 172.21.15.92:40276) 2023-10-06T03:08:47.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404ccc0700 1 -- 172.21.15.92:0/4168303154 learned_addr learned my addr 172.21.15.92:0/4168303154 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f404d4c1700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4048146e70 0x7f4048147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.543+0000 7f403ffff700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40480ca800 0x7f40480ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ccc0700 1 -- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40480ca800 msgr2=0x7f40480ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ccc0700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40480ca800 0x7f40480ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ccc0700 1 -- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4048146e70 msgr2=0x7f4048147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ccc0700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4048146e70 0x7f4048147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ccc0700 1 -- 172.21.15.92:0/4168303154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480b3e90 con 0x7f4030003a90 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404d4c1700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4048146e70 0x7f4048147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ccc0700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f40480b3590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f404400c6a0 tx=0x7f4044015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044004070 con 0x7f4030003a90 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404400ce80 con 0x7f4030003a90 2023-10-06T03:08:47.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40480b4170 con 0x7f4030003a90 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044006ba0 con 0x7f4030003a90 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.544+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40480b4ae0 con 0x7f4030003a90 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.545+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40480b5130 con 0x7f4030003a90 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.545+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f404401f410 con 0x7f4030003a90 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.546+0000 7f403cff9700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4034067320 0x7f40340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.546+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f4044085da0 con 0x7f4030003a90 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.546+0000 7f403ffff700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4034067320 0x7f40340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.546+0000 7f403ffff700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4034067320 0x7f40340697d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f40480b0240 tx=0x7f4038001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:47.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.550+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4044052820 con 0x7f4030003a90 2023-10-06T03:08:47.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.776+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f404804c3c0 con 0x7f4030003a90 2023-10-06T03:08:47.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.776+0000 7f403cff9700 1 -- 172.21.15.92:0/4168303154 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f4044017090 con 0x7f4030003a90 2023-10-06T03:08:47.777 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:47.777 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1888979047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1888979047},{"type":"v1","addr":"172.21.15.136:6835","nonce":1888979047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7464,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/628951753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":628951753},{"type":"v1","addr":"172.21.15.92:6837","nonce":628951753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:08:22.334042+0000","modified":"2023-10-06T03:08:26.961080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7221},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7221":{"gid":7221,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/1746152484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1746152484},{"type":"v1","addr":"172.21.15.92:6836","nonce":1746152484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T03:08:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4034067320 msgr2=0x7f40340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4034067320 0x7f40340697d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f40480b0240 tx=0x7f4038001e50 comp rx=0 tx=0).stop 2023-10-06T03:08:47.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 msgr2=0x7f40480b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f40480b3590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f404400c6a0 tx=0x7f4044015000 comp rx=0 tx=0).stop 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 shutdown_connections 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4034067320 0x7f40340697d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4030003a90 0x7f40480b3590 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4048146e70 0x7f4048147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 --2- 172.21.15.92:0/4168303154 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40480ca800 0x7f40480ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.778+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 >> 172.21.15.92:0/4168303154 conn(0x7f4048068e20 msgr2=0x7f4048154600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.779+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 shutdown_connections 2023-10-06T03:08:47.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.779+0000 7f404ef24700 1 -- 172.21.15.92:0/4168303154 wait complete. 2023-10-06T03:08:47.781 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 86 2023-10-06T03:08:47.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:47.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.905+0000 7fb46759e700 1 Processor -- start 2023-10-06T03:08:47.906 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.905+0000 7fb46759e700 1 -- start start 2023-10-06T03:08:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb46759e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb4681597f0 0x7fb4680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb46759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 0x7fb46815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb46759e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46815dbb0 con 0x7fb468159420 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb46759e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4680b11d0 con 0x7fb4681597f0 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb46759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4680b14b0 con 0x7fb46815d230 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb466d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 0x7fb46815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb465d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb4681597f0 0x7fb4680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb466d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 0x7fb46815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:46662/0 (socket says 172.21.15.92:46662) 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb466d9d700 1 -- 172.21.15.92:0/2279128794 learned_addr learned my addr 172.21.15.92:0/2279128794 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb465d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb4681597f0 0x7fb4680b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:54880/0 (socket says 172.21.15.92:54880) 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb46659c700 1 --1- 172.21.15.92:0/2279128794 >> v1:172.21.15.136:6789/0 conn(0x7fb468159420 0x7fb4680b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:45024/0 (socket says 172.21.15.92:45024) 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb466d9d700 1 -- 172.21.15.92:0/2279128794 >> v2:172.21.15.136:3300/0 conn(0x7fb4681597f0 msgr2=0x7fb4680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:47.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb466d9d700 1 --2- 172.21.15.92:0/2279128794 >> v2:172.21.15.136:3300/0 conn(0x7fb4681597f0 0x7fb4680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.906+0000 7fb466d9d700 1 -- 172.21.15.92:0/2279128794 >> v1:172.21.15.136:6789/0 conn(0x7fb468159420 legacy=0x7fb4680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb466d9d700 1 -- 172.21.15.92:0/2279128794 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680b1790 con 0x7fb46815d230 2023-10-06T03:08:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb466d9d700 1 --2- 172.21.15.92:0/2279128794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 0x7fb46815d670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb458009f20 tx=0x7fb458009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d882c873828a2cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46559a700 1 -- 172.21.15.92:0/2279128794 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800e070 con 0x7fb46815d230 2023-10-06T03:08:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46559a700 1 -- 172.21.15.92:0/2279128794 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800b9d0 con 0x7fb46815d230 2023-10-06T03:08:47.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46559a700 1 -- 172.21.15.92:0/2279128794 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580092e0 con 0x7fb46815d230 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 -- 172.21.15.92:0/2279128794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 msgr2=0x7fb46815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 --2- 172.21.15.92:0/2279128794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 0x7fb46815d670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb458009f20 tx=0x7fb458009840 comp rx=0 tx=0).stop 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 -- 172.21.15.92:0/2279128794 shutdown_connections 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 --2- 172.21.15.92:0/2279128794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb46815d230 0x7fb46815d670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 --2- 172.21.15.92:0/2279128794 >> v2:172.21.15.136:3300/0 conn(0x7fb4681597f0 0x7fb4680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 -- 172.21.15.92:0/2279128794 >> 172.21.15.92:0/2279128794 conn(0x7fb4680d0660 msgr2=0x7fb4680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 -- 172.21.15.92:0/2279128794 shutdown_connections 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.907+0000 7fb46759e700 1 -- 172.21.15.92:0/2279128794 wait complete. 2023-10-06T03:08:47.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.908+0000 7fb46759e700 1 Processor -- start 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 -- start start 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb468159420 0x7fb46814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4681597f0 0x7fb468154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 0x7fb468153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb468157390 con 0x7fb46815d230 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb468156dd0 con 0x7fb4681597f0 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4681570b0 con 0x7fb468159420 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb466d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 0x7fb468153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb46659c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb468159420 0x7fb46814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb465d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4681597f0 0x7fb468154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb466d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 0x7fb468153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40288/0 (socket says 172.21.15.92:40288) 2023-10-06T03:08:47.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.909+0000 7fb466d9d700 1 -- 172.21.15.92:0/2964102272 learned_addr learned my addr 172.21.15.92:0/2964102272 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb466d9d700 1 -- 172.21.15.92:0/2964102272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb468159420 msgr2=0x7fb46814ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb466d9d700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb468159420 0x7fb46814ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb466d9d700 1 -- 172.21.15.92:0/2964102272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4681597f0 msgr2=0x7fb468154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb466d9d700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4681597f0 0x7fb468154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb466d9d700 1 -- 172.21.15.92:0/2964102272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009ae0 con 0x7fb46815d230 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb466d9d700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 0x7fb468153f80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b610 tx=0x7fb458018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458008ad0 con 0x7fb46815d230 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458008c30 con 0x7fb46815d230 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46815a6b0 con 0x7fb46815d230 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45801c410 con 0x7fb46815d230 2023-10-06T03:08:47.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.910+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4681544c0 con 0x7fb46815d230 2023-10-06T03:08:47.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.911+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46815abd0 con 0x7fb46815d230 2023-10-06T03:08:47.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.911+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb458003040 con 0x7fb46815d230 2023-10-06T03:08:47.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.912+0000 7fb45e7fc700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb448067320 0x7fb4480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:47.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.912+0000 7fb46659c700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb448067320 0x7fb4480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:47.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.912+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7fb458086a80 con 0x7fb46815d230 2023-10-06T03:08:47.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.912+0000 7fb46659c700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb448067320 0x7fb4480697d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb460004fd0 tx=0x7fb460007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:47.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:47.916+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb46815abd0 con 0x7fb46815d230 2023-10-06T03:08:48.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.099+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb46815abd0 con 0x7fb46815d230 2023-10-06T03:08:48.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.100+0000 7fb45e7fc700 1 -- 172.21.15.92:0/2964102272 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17141 (secure 0 0 0) 0x7fb46815abd0 con 0x7fb46815d230 2023-10-06T03:08:48.101 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:48.101 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:47.155134+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:08:19.983879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T03:08:21.301577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6835/3257155196":"2023-10-07T03:08:44.146778+0000","172.21.15.136:6833/3257155196":"2023-10-07T03:08:44.146778+0000","172.21.15.92:6837/3995167741":"2023-10-07T03:08:40.744848+0000","172.21.15.92:6835/3995167741":"2023-10-07T03:08:40.744848+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.102+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb448067320 msgr2=0x7fb4480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.102+0000 7fb46759e700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb448067320 0x7fb4480697d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb460004fd0 tx=0x7fb460007eb0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.102+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 msgr2=0x7fb468153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.102+0000 7fb46759e700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 0x7fb468153f80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b610 tx=0x7fb458018000 comp rx=0 tx=0).stop 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 shutdown_connections 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb448067320 0x7fb4480697d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb46815d230 0x7fb468153f80 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb468159420 0x7fb46814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 --2- 172.21.15.92:0/2964102272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4681597f0 0x7fb468154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 >> 172.21.15.92:0/2964102272 conn(0x7fb4680d0660 msgr2=0x7fb468068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 shutdown_connections 2023-10-06T03:08:48.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.103+0000 7fb46759e700 1 -- 172.21.15.92:0/2964102272 wait complete. 2023-10-06T03:08:48.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:08:19.983879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:08:21.301577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:08:48.116 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:08:48.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.233+0000 7f5eaffff700 1 Processor -- start 2023-10-06T03:08:48.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.233+0000 7f5eaffff700 1 -- start start 2023-10-06T03:08:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5eb00b4e70 0x7f5eb00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 0x7f5eb00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00b2080 con 0x7f5eb015d240 2023-10-06T03:08:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00cfb00 con 0x7f5eb00b4e70 2023-10-06T03:08:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00b5c40 con 0x7f5eb00b16d0 2023-10-06T03:08:48.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5eb015d240 0x7f5eb015d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56698/0 (socket says 172.21.15.92:56698) 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaf7fe700 1 -- 172.21.15.92:0/1996279212 learned_addr learned my addr 172.21.15.92:0/1996279212 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eae7fc700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 0x7f5eb00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1996279212 >> v2:172.21.15.136:3300/0 conn(0x7f5eb00b4e70 0x7f5eb00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261191282 0 0) 0x7f5eb00b2080 con 0x7f5eb015d240 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e9c003570 con 0x7f5eb015d240 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250177099 0 0) 0x7f5e9c003570 con 0x7f5eb015d240 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 msgr2=0x7f5eb00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 0x7f5eb00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 >> v2:172.21.15.136:3300/0 conn(0x7f5eb00b4e70 msgr2=0x7f5eb00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 --2- 172.21.15.92:0/1996279212 >> v2:172.21.15.136:3300/0 conn(0x7f5eb00b4e70 0x7f5eb00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00b5f20 con 0x7f5eb015d240 2023-10-06T03:08:48.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.234+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5ea8001470 con 0x7f5eb015d240 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 >> v1:172.21.15.92:6789/0 conn(0x7f5eb015d240 legacy=0x7f5eb015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9c003a60 0x7f5e9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9c004810 0x7f5e9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 0x7f5e9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cfb00 con 0x7f5e9c003a60 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00b5c40 con 0x7f5e9c004810 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9c00ba60 con 0x7f5eb00b16d0 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9c003a60 0x7f5e9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eae7fc700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9c004810 0x7f5e9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaf7fe700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 0x7f5e9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 -- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 msgr2=0x7f5e9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb00b16d0 0x7f5e9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 -- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9c004810 msgr2=0x7f5e9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9c004810 0x7f5e9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 -- 172.21.15.92:0/1996279212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c003570 con 0x7f5e9c003a60 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9c003a60 0x7f5e9c003e70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000e240 tx=0x7f5ea000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40d01a2f39c4c1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000eaf0 con 0x7f5e9c003a60 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea000f040 con 0x7f5e9c003a60 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.235+0000 7f5eadffb700 1 -- 172.21.15.92:0/1996279212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000eea0 con 0x7f5e9c003a60 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaffff700 1 -- 172.21.15.92:0/1996279212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9c003a60 msgr2=0x7f5e9c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1996279212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9c003a60 0x7f5e9c003e70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000e240 tx=0x7f5ea000b040 comp rx=0 tx=0).stop 2023-10-06T03:08:48.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaf7fe700 1 -- 172.21.15.92:0/1996279212 reap_dead start 2023-10-06T03:08:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaffff700 1 -- 172.21.15.92:0/1996279212 shutdown_connections 2023-10-06T03:08:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaffff700 1 -- 172.21.15.92:0/1996279212 >> 172.21.15.92:0/1996279212 conn(0x7f5eb00d10c0 msgr2=0x7f5eb00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaffff700 1 -- 172.21.15.92:0/1996279212 shutdown_connections 2023-10-06T03:08:48.241 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.236+0000 7f5eaffff700 1 -- 172.21.15.92:0/1996279212 wait complete. 2023-10-06T03:08:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.237+0000 7f5eaffff700 1 Processor -- start 2023-10-06T03:08:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.237+0000 7f5eaffff700 1 -- start start 2023-10-06T03:08:48.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e9c003a60 0x7f5eb00c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb0169ed0 0x7f5eb00ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 0x7f5eb00ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb016a340 con 0x7f5eb0156d30 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00c7d50 con 0x7f5eb0169ed0 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb016a520 con 0x7f5e9c003a60 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 0x7f5eb00ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 0x7f5eb00ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40302/0 (socket says 172.21.15.92:40302) 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 -- 172.21.15.92:0/1901102403 learned_addr learned my addr 172.21.15.92:0/1901102403 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e9c003a60 0x7f5eb00c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eae7fc700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb0169ed0 0x7f5eb00ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 -- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e9c003a60 msgr2=0x7f5eb00c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e9c003a60 0x7f5eb00c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 -- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb0169ed0 msgr2=0x7f5eb00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb0169ed0 0x7f5eb00ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 -- 172.21.15.92:0/1901102403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb016a800 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaf7fe700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 0x7f5eb00ca440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0071380 tx=0x7f5ea8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8000f80 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800f8a0 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.238+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb016aae0 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.239+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800fc60 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.239+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb00c84c0 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.239+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb00c8310 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.240+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5ea80135c0 con 0x7f5eb0156d30 2023-10-06T03:08:48.245 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.240+0000 7f5ea6ffd700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e90067320 0x7f5e900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.240+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f5ea80845d0 con 0x7f5eb0156d30 2023-10-06T03:08:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.240+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e90067320 0x7f5e900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.241+0000 7f5eaeffd700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e90067320 0x7f5e900697d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00b3ff0 tx=0x7f5ea001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.246 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.245+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ea8051160 con 0x7f5eb0156d30 2023-10-06T03:08:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.427+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5eb00c86e0 con 0x7f5eb0156d30 2023-10-06T03:08:48.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.427+0000 7f5ea6ffd700 1 -- 172.21.15.92:0/1901102403 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ea801a0a0 con 0x7f5eb0156d30 2023-10-06T03:08:48.428 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:08:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e90067320 msgr2=0x7f5e900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e90067320 0x7f5e900697d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00b3ff0 tx=0x7f5ea001a000 comp rx=0 tx=0).stop 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 msgr2=0x7f5eb00ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 0x7f5eb00ca440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0071380 tx=0x7f5ea8001e90 comp rx=0 tx=0).stop 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 shutdown_connections 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e90067320 0x7f5e900697d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5eb0156d30 0x7f5eb00ca440 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e9c003a60 0x7f5eb00c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 --2- 172.21.15.92:0/1901102403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb0169ed0 0x7f5eb00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 >> 172.21.15.92:0/1901102403 conn(0x7f5eb00d10c0 msgr2=0x7f5eb015d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 shutdown_connections 2023-10-06T03:08:48.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.430+0000 7f5eaffff700 1 -- 172.21.15.92:0/1901102403 wait complete. 2023-10-06T03:08:48.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:08:48.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.560+0000 7f5bdbbb4700 1 Processor -- start 2023-10-06T03:08:48.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.560+0000 7f5bdbbb4700 1 -- start start 2023-10-06T03:08:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bdbbb4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5bd4159c60 0x7f5bd415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bdbbb4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bdbbb4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd415a160 con 0x7f5bd4159890 2023-10-06T03:08:48.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bdbbb4700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd415df50 con 0x7f5bd4159c60 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bdbbb4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40d13a0 con 0x7f5bd415db10 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bd9950700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5bd4159890 0x7f5bd415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56712/0 (socket says 172.21.15.92:56712) 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bd9950700 1 -- 172.21.15.92:0/3610596355 learned_addr learned my addr 172.21.15.92:0/3610596355 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bda151700 1 --2- 172.21.15.92:0/3610596355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bd914f700 1 --2- 172.21.15.92:0/3610596355 >> v2:172.21.15.136:3300/0 conn(0x7f5bd4159c60 0x7f5bd415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bd894e700 1 -- 172.21.15.92:0/3610596355 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391355351 0 0) 0x7f5bd415a160 con 0x7f5bd4159890 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bd894e700 1 -- 172.21.15.92:0/3610596355 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc0003570 con 0x7f5bd4159890 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bd894e700 1 -- 172.21.15.92:0/3610596355 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 254560013 0 0) 0x7f5bc0003570 con 0x7f5bd4159890 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bda151700 1 -- 172.21.15.92:0/3610596355 >> v2:172.21.15.136:3300/0 conn(0x7f5bd4159c60 msgr2=0x7f5bd415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bda151700 1 --2- 172.21.15.92:0/3610596355 >> v2:172.21.15.136:3300/0 conn(0x7f5bd4159c60 0x7f5bd415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.561+0000 7f5bda151700 1 -- 172.21.15.92:0/3610596355 >> v1:172.21.15.92:6789/0 conn(0x7f5bd4159890 legacy=0x7f5bd415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bda151700 1 -- 172.21.15.92:0/3610596355 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd415e140 con 0x7f5bd415db10 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bd914f700 1 --2- 172.21.15.92:0/3610596355 >> v2:172.21.15.136:3300/0 conn(0x7f5bd4159c60 0x7f5bd415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bda151700 1 --2- 172.21.15.92:0/3610596355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd4069560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0098e0 tx=0x7f5bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c767f577da1629 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bd894e700 1 -- 172.21.15.92:0/3610596355 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00017f0 con 0x7f5bd415db10 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bd894e700 1 -- 172.21.15.92:0/3610596355 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc011420 con 0x7f5bd415db10 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bd894e700 1 -- 172.21.15.92:0/3610596355 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0117c0 con 0x7f5bd415db10 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3610596355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 msgr2=0x7f5bd4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3610596355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd4069560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0098e0 tx=0x7f5bcc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3610596355 shutdown_connections 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3610596355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd4069560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3610596355 >> v2:172.21.15.136:3300/0 conn(0x7f5bd4159c60 0x7f5bd415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.562+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3610596355 >> 172.21.15.92:0/3610596355 conn(0x7f5bd4068e20 msgr2=0x7f5bd40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:48.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.563+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3610596355 shutdown_connections 2023-10-06T03:08:48.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.563+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3610596355 wait complete. 2023-10-06T03:08:48.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 Processor -- start 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 -- start start 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 0x7f5bd4149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bd4159c60 0x7f5bd40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40cdaf0 con 0x7f5bd4159890 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b05e0 con 0x7f5bd4159c60 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bdbbb4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40b08c0 con 0x7f5bd415db10 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bd9950700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 0x7f5bd4149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bd9950700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 0x7f5bd4149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40314/0 (socket says 172.21.15.92:40314) 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bd9950700 1 -- 172.21.15.92:0/3441032532 learned_addr learned my addr 172.21.15.92:0/3441032532 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:48.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bda151700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.564+0000 7f5bd914f700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bd4159c60 0x7f5bd40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bd9950700 1 -- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 msgr2=0x7f5bd415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bd9950700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd415c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bd9950700 1 -- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bd4159c60 msgr2=0x7f5bd40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bd9950700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bd4159c60 0x7f5bd40cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bd9950700 1 -- 172.21.15.92:0/3441032532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009420 con 0x7f5bd4159890 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bd9950700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 0x7f5bd4149000 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000cc10 tx=0x7f5bd0007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0004dc0 con 0x7f5bd4159890 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000a6a0 con 0x7f5bd4159890 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc009420 con 0x7f5bd4159890 2023-10-06T03:08:48.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000aa40 con 0x7f5bd4159890 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.565+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd40b0e80 con 0x7f5bd4159890 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.566+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd40b10a0 con 0x7f5bd4159890 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.566+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5bd0007150 con 0x7f5bd4159890 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.567+0000 7f5bc9ffb700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5bb8067320 0x7f5bb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.567+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f5bd0084880 con 0x7f5bd4159890 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.567+0000 7f5bd914f700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5bb8067320 0x7f5bb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.567+0000 7f5bd914f700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5bb8067320 0x7f5bb80697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4154cc0 tx=0x7f5bc40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.571+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bd0051480 con 0x7f5bd4159890 2023-10-06T03:08:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.751+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5bd404c3c0 con 0x7f5bd4159890 2023-10-06T03:08:48.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.751+0000 7f5bc9ffb700 1 -- 172.21.15.92:0/3441032532 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v128) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5bd0018030 con 0x7f5bd4159890 2023-10-06T03:08:48.752 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:08:48.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5bb8067320 msgr2=0x7f5bb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5bb8067320 0x7f5bb80697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4154cc0 tx=0x7f5bc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 msgr2=0x7f5bd4149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 0x7f5bd4149000 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000cc10 tx=0x7f5bd0007d20 comp rx=0 tx=0).stop 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 shutdown_connections 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5bb8067320 0x7f5bb80697d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bd4159890 0x7f5bd4149000 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5bd415db10 0x7f5bd415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 --2- 172.21.15.92:0/3441032532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5bd4159c60 0x7f5bd40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 >> 172.21.15.92:0/3441032532 conn(0x7f5bd4068e20 msgr2=0x7f5bd4157730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 shutdown_connections 2023-10-06T03:08:48.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.753+0000 7f5bdbbb4700 1 -- 172.21.15.92:0/3441032532 wait complete. 2023-10-06T03:08:48.766 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:08:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.883+0000 7f71259b0700 1 Processor -- start 2023-10-06T03:08:48.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.883+0000 7f71259b0700 1 -- start start 2023-10-06T03:08:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f71259b0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7120155eb0 0x7f712006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f71259b0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71200c70b0 0x7f7120069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f71259b0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712006eb90 con 0x7f7120069cb0 2023-10-06T03:08:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f71259b0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712006a130 con 0x7f71200c70b0 2023-10-06T03:08:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f71259b0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712006ee70 con 0x7f7120155eb0 2023-10-06T03:08:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f71167fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71200c70b0 0x7f7120069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f711f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7120069cb0 0x7f7120159a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56722/0 (socket says 172.21.15.92:56722) 2023-10-06T03:08:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f711f7fe700 1 -- 172.21.15.92:0/852318521 learned_addr learned my addr 172.21.15.92:0/852318521 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:48.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.884+0000 7f711effd700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7120155eb0 0x7f712006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71167fc700 1 -- 172.21.15.92:0/852318521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7120155eb0 msgr2=0x7f712006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71167fc700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7120155eb0 0x7f712006e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71167fc700 1 -- 172.21.15.92:0/852318521 >> v1:172.21.15.92:6789/0 conn(0x7f7120069cb0 legacy=0x7f7120159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71167fc700 1 -- 172.21.15.92:0/852318521 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712006f150 con 0x7f71200c70b0 2023-10-06T03:08:48.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f711e7fc700 1 -- 172.21.15.92:0/852318521 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274335063 0 0) 0x7f712006eb90 con 0x7f7120069cb0 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f711effd700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7120155eb0 0x7f712006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71167fc700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71200c70b0 0x7f7120069770 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f71080098e0 tx=0x7f71080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9f8fcc00b76338 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f711e7fc700 1 -- 172.21.15.92:0/852318521 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108009e80 con 0x7f71200c70b0 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f711e7fc700 1 -- 172.21.15.92:0/852318521 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800b860 con 0x7f71200c70b0 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f711e7fc700 1 -- 172.21.15.92:0/852318521 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800bbe0 con 0x7f71200c70b0 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 -- 172.21.15.92:0/852318521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71200c70b0 msgr2=0x7f7120069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71200c70b0 0x7f7120069770 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f71080098e0 tx=0x7f71080091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 -- 172.21.15.92:0/852318521 shutdown_connections 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7120155eb0 0x7f712006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 --2- 172.21.15.92:0/852318521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71200c70b0 0x7f7120069770 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 -- 172.21.15.92:0/852318521 >> 172.21.15.92:0/852318521 conn(0x7f7120068e10 msgr2=0x7f71200cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 -- 172.21.15.92:0/852318521 shutdown_connections 2023-10-06T03:08:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.885+0000 7f71259b0700 1 -- 172.21.15.92:0/852318521 wait complete. 2023-10-06T03:08:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 Processor -- start 2023-10-06T03:08:48.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 -- start start 2023-10-06T03:08:48.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120069cb0 0x7f71201626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71200c70b0 0x7f712013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 0x7f71200b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71200b1050 con 0x7f7120155eb0 2023-10-06T03:08:48.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71200ae0f0 con 0x7f7120069cb0 2023-10-06T03:08:48.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.887+0000 7f71259b0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71200ae3d0 con 0x7f71200c70b0 2023-10-06T03:08:48.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 0x7f71200b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 0x7f71200b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40328/0 (socket says 172.21.15.92:40328) 2023-10-06T03:08:48.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 -- 172.21.15.92:0/212655162 learned_addr learned my addr 172.21.15.92:0/212655162 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:48.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f71167fc700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71200c70b0 0x7f712013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711effd700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120069cb0 0x7f71201626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 -- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71200c70b0 msgr2=0x7f712013ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71200c70b0 0x7f712013ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 -- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120069cb0 msgr2=0x7f71201626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:48.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120069cb0 0x7f71201626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:48.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 -- 172.21.15.92:0/212655162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108009420 con 0x7f7120155eb0 2023-10-06T03:08:48.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f711f7fe700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 0x7f71200b14e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f711800cc90 tx=0x7f7118001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800a8b0 con 0x7f7120155eb0 2023-10-06T03:08:48.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800aa10 con 0x7f7120155eb0 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71200b0a00 con 0x7f7120155eb0 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800add0 con 0x7f7120155eb0 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.888+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7120140440 con 0x7f7120155eb0 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.889+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712004c3c0 con 0x7f7120155eb0 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.889+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7118007300 con 0x7f7120155eb0 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.890+0000 7f71177fe700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7100067320 0x7f71000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:48.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.890+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f71180851a0 con 0x7f7120155eb0 2023-10-06T03:08:48.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.890+0000 7f711effd700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7100067320 0x7f71000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:48.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.890+0000 7f711effd700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7100067320 0x7f71000697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7110006fd0 tx=0x7f7110008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:48.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:48.894+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7118051ca0 con 0x7f7120155eb0 2023-10-06T03:08:49.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.077+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71200b1190 con 0x7f7120155eb0 2023-10-06T03:08:49.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.078+0000 7f71177fe700 1 -- 172.21.15.92:0/212655162 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7f71180552c0 con 0x7f7120155eb0 2023-10-06T03:08:49.079 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:08:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7100067320 msgr2=0x7f71000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7100067320 0x7f71000697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7110006fd0 tx=0x7f7110008040 comp rx=0 tx=0).stop 2023-10-06T03:08:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 msgr2=0x7f71200b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 0x7f71200b14e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f711800cc90 tx=0x7f7118001a80 comp rx=0 tx=0).stop 2023-10-06T03:08:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 shutdown_connections 2023-10-06T03:08:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7100067320 0x7f71000697d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7120155eb0 0x7f71200b14e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7120069cb0 0x7f71201626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 --2- 172.21.15.92:0/212655162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71200c70b0 0x7f712013ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 >> 172.21.15.92:0/212655162 conn(0x7f7120068e10 msgr2=0x7f71200c1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:49.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 shutdown_connections 2023-10-06T03:08:49.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.080+0000 7f71259b0700 1 -- 172.21.15.92:0/212655162 wait complete. 2023-10-06T03:08:49.093 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:08:49.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.210+0000 7f7946dd5700 1 Processor -- start 2023-10-06T03:08:49.211 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.210+0000 7f7946dd5700 1 -- start start 2023-10-06T03:08:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7946dd5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7940156fb0 0x7f794006a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7946dd5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f794006ab20 0x7f79400cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7946dd5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f79400cbae0 0x7f7940159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7946dd5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794006af70 con 0x7f7940156fb0 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7946dd5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400cbf40 con 0x7f79400cbae0 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7946dd5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7940069d30 con 0x7f794006ab20 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7944b71700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7940156fb0 0x7f794006a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7944b71700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7940156fb0 0x7f794006a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40344/0 (socket says 172.21.15.92:40344) 2023-10-06T03:08:49.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.211+0000 7f7944b71700 1 -- 172.21.15.92:0/1480110007 learned_addr learned my addr 172.21.15.92:0/1480110007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7944b71700 1 -- 172.21.15.92:0/1480110007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f794006ab20 msgr2=0x7f79400cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7945372700 1 --2- 172.21.15.92:0/1480110007 >> v2:172.21.15.136:3300/0 conn(0x7f79400cbae0 0x7f7940159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f793ffff700 1 --2- 172.21.15.92:0/1480110007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f794006ab20 0x7f79400cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7944b71700 1 --2- 172.21.15.92:0/1480110007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f794006ab20 0x7f79400cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7944b71700 1 -- 172.21.15.92:0/1480110007 >> v2:172.21.15.136:3300/0 conn(0x7f79400cbae0 msgr2=0x7f7940159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7944b71700 1 --2- 172.21.15.92:0/1480110007 >> v2:172.21.15.136:3300/0 conn(0x7f79400cbae0 0x7f7940159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7944b71700 1 -- 172.21.15.92:0/1480110007 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79400c8110 con 0x7f7940156fb0 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7944b71700 1 --2- 172.21.15.92:0/1480110007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f794006a5e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f79401540e0 tx=0x7f793800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196a4dd03e8c8b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f793f7fe700 1 -- 172.21.15.92:0/1480110007 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938010070 con 0x7f7940156fb0 2023-10-06T03:08:49.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f793f7fe700 1 -- 172.21.15.92:0/1480110007 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793800dc80 con 0x7f7940156fb0 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f793f7fe700 1 -- 172.21.15.92:0/1480110007 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938016400 con 0x7f7940156fb0 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.212+0000 7f7946dd5700 1 -- 172.21.15.92:0/1480110007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 msgr2=0x7f794006a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 --2- 172.21.15.92:0/1480110007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f794006a5e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f79401540e0 tx=0x7f793800ba50 comp rx=0 tx=0).stop 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 -- 172.21.15.92:0/1480110007 shutdown_connections 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 --2- 172.21.15.92:0/1480110007 >> v2:172.21.15.136:3300/0 conn(0x7f79400cbae0 0x7f7940159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 --2- 172.21.15.92:0/1480110007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f794006a5e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 --2- 172.21.15.92:0/1480110007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f794006ab20 0x7f79400cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 -- 172.21.15.92:0/1480110007 >> 172.21.15.92:0/1480110007 conn(0x7f79400c9870 msgr2=0x7f79400c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 -- 172.21.15.92:0/1480110007 shutdown_connections 2023-10-06T03:08:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.213+0000 7f7946dd5700 1 -- 172.21.15.92:0/1480110007 wait complete. 2023-10-06T03:08:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.214+0000 7f7946dd5700 1 Processor -- start 2023-10-06T03:08:49.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.214+0000 7f7946dd5700 1 -- start start 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f794006ab20 0x7f794014faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79400cbae0 0x7f79400c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f79400c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400c6b10 con 0x7f7940156fb0 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b10c0 con 0x7f794006ab20 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79400b0820 con 0x7f79400cbae0 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f79400c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f79400c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40356/0 (socket says 172.21.15.92:40356) 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 -- 172.21.15.92:0/2506703266 learned_addr learned my addr 172.21.15.92:0/2506703266 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7944b71700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f794006ab20 0x7f794014faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f793ffff700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79400cbae0 0x7f79400c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 -- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79400cbae0 msgr2=0x7f79400c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79400cbae0 0x7f79400c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 -- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f794006ab20 msgr2=0x7f794014faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f794006ab20 0x7f794014faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 -- 172.21.15.92:0/2506703266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793800bcf0 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7945372700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f79400c4d00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f793000c880 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930011660 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79300117c0 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930011b60 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79400b0b00 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.215+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7940157520 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.216+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940157380 con 0x7f7940156fb0 2023-10-06T03:08:49.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.217+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f79300193f0 con 0x7f7940156fb0 2023-10-06T03:08:49.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.217+0000 7f793cff9700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f792c067320 0x7f792c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.217+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f7930085220 con 0x7f7940156fb0 2023-10-06T03:08:49.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.217+0000 7f7944b71700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f792c067320 0x7f792c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.217+0000 7f7944b71700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f792c067320 0x7f792c0697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f79401540e0 tx=0x7f793801c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.221+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7940157380 con 0x7f7940156fb0 2023-10-06T03:08:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.403+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7940157380 con 0x7f7940156fb0 2023-10-06T03:08:49.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.404+0000 7f793cff9700 1 -- 172.21.15.92:0/2506703266 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7940157380 con 0x7f7940156fb0 2023-10-06T03:08:49.405 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:08:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f792c067320 msgr2=0x7f792c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f792c067320 0x7f792c0697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f79401540e0 tx=0x7f793801c000 comp rx=0 tx=0).stop 2023-10-06T03:08:49.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 msgr2=0x7f79400c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f79400c4d00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f793000c880 tx=0x7f7930008040 comp rx=0 tx=0).stop 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 shutdown_connections 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f792c067320 0x7f792c0697d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7940156fb0 0x7f79400c4d00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f794006ab20 0x7f794014faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 --2- 172.21.15.92:0/2506703266 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79400cbae0 0x7f79400c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 >> 172.21.15.92:0/2506703266 conn(0x7f79400c9870 msgr2=0x7f79401537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 shutdown_connections 2023-10-06T03:08:49.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.406+0000 7f7946dd5700 1 -- 172.21.15.92:0/2506703266 wait complete. 2023-10-06T03:08:49.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:49.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.532+0000 7f803f1c5700 1 Processor -- start 2023-10-06T03:08:49.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.532+0000 7f803f1c5700 1 -- start start 2023-10-06T03:08:49.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803f1c5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f80380b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803f1c5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f803815d230 0x7f803815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803f1c5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803815dbd0 con 0x7f80381597f0 2023-10-06T03:08:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803f1c5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80380b2980 con 0x7f803815d230 2023-10-06T03:08:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803f1c5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80380b11d0 con 0x7f8038159420 2023-10-06T03:08:49.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803d9c2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f80381597f0 0x7f80380b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52948/0 (socket says 172.21.15.92:52948) 2023-10-06T03:08:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803d9c2700 1 -- 172.21.15.92:0/896584830 learned_addr learned my addr 172.21.15.92:0/896584830 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597737072 0 0) 0x7f802c001110 con 0x7f80381597f0 2023-10-06T03:08:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803e1c3700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f80380b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8020003570 con 0x7f80381597f0 2023-10-06T03:08:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.533+0000 7f803e9c4700 1 --2- 172.21.15.92:0/896584830 >> v2:172.21.15.136:3300/0 conn(0x7f803815d230 0x7f803815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3362893728 0 0) 0x7f8020003570 con 0x7f80381597f0 2023-10-06T03:08:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 msgr2=0x7f80380b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f80380b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 >> v2:172.21.15.136:3300/0 conn(0x7f803815d230 msgr2=0x7f803815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 --2- 172.21.15.92:0/896584830 >> v2:172.21.15.136:3300/0 conn(0x7f803815d230 0x7f803815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380b14b0 con 0x7f80381597f0 2023-10-06T03:08:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f802c001600 con 0x7f80381597f0 2023-10-06T03:08:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 >> v1:172.21.15.92:6789/0 conn(0x7f80381597f0 legacy=0x7f80380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8020003a60 0x7f8020003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f8020004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8020004d50 0x7f80200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380b2980 con 0x7f8020003a60 2023-10-06T03:08:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80380b11d0 con 0x7f8020004d50 2023-10-06T03:08:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802000ba60 con 0x7f8038159420 2023-10-06T03:08:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8020003a60 0x7f8020003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d9c2700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f8020004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e9c4700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8020004d50 0x7f80200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 -- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 msgr2=0x7f8020004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f8020004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 -- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8020004d50 msgr2=0x7f80200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8020004d50 0x7f80200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 -- 172.21.15.92:0/896584830 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8020003570 con 0x7f8020003a60 2023-10-06T03:08:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d9c2700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8038159420 0x7f8020004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:49.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e9c4700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8020004d50 0x7f80200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803e1c3700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8020003a60 0x7f8020003e70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f8030009c00 tx=0x7f80300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d9de3a61b59a4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000e6b0 con 0x7f8020003a60 2023-10-06T03:08:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803000e810 con 0x7f8020003a60 2023-10-06T03:08:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.534+0000 7f803d1c1700 1 -- 172.21.15.92:0/896584830 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803000ebd0 con 0x7f8020003a60 2023-10-06T03:08:49.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803f1c5700 1 -- 172.21.15.92:0/896584830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8020003a60 msgr2=0x7f8020003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803f1c5700 1 --2- 172.21.15.92:0/896584830 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8020003a60 0x7f8020003e70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f8030009c00 tx=0x7f80300091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803e9c4700 1 -- 172.21.15.92:0/896584830 reap_dead start 2023-10-06T03:08:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803f1c5700 1 -- 172.21.15.92:0/896584830 shutdown_connections 2023-10-06T03:08:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803f1c5700 1 -- 172.21.15.92:0/896584830 >> 172.21.15.92:0/896584830 conn(0x7f80380d0660 msgr2=0x7f80380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803f1c5700 1 -- 172.21.15.92:0/896584830 shutdown_connections 2023-10-06T03:08:49.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.535+0000 7f803f1c5700 1 -- 172.21.15.92:0/896584830 wait complete. 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.536+0000 7f803f1c5700 1 Processor -- start 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.536+0000 7f803f1c5700 1 -- start start 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803f1c5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8020003a60 0x7f803815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803f1c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 0x7f803814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803f1c5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8038156c50 0x7f80381570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803f1c5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803814ddf0 con 0x7f803816a620 2023-10-06T03:08:49.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803f1c5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803815b600 con 0x7f8038156c50 2023-10-06T03:08:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803f1c5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80381575e0 con 0x7f8020003a60 2023-10-06T03:08:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 0x7f803814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 0x7f803814bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56208/0 (socket says 172.21.15.92:56208) 2023-10-06T03:08:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 -- 172.21.15.92:0/2320384271 learned_addr learned my addr 172.21.15.92:0/2320384271 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803e1c3700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8020003a60 0x7f803815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803e9c4700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8038156c50 0x7f80381570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 -- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8020003a60 msgr2=0x7f803815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8020003a60 0x7f803815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 -- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8038156c50 msgr2=0x7f80381570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8038156c50 0x7f80381570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 -- 172.21.15.92:0/2320384271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803815cee0 con 0x7f803816a620 2023-10-06T03:08:49.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803e1c3700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8020003a60 0x7f803815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803e9c4700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8038156c50 0x7f80381570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.537+0000 7f803d9c2700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 0x7f803814bea0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f802c000fd0 tx=0x7f802c000da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.538+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00e890 con 0x7f803816a620 2023-10-06T03:08:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.538+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c00e9f0 con 0x7f803816a620 2023-10-06T03:08:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.538+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803815acd0 con 0x7f803816a620 2023-10-06T03:08:49.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.538+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00ed70 con 0x7f803816a620 2023-10-06T03:08:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.538+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803815b290 con 0x7f803816a620 2023-10-06T03:08:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.539+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803804c3c0 con 0x7f803816a620 2023-10-06T03:08:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.539+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f802c013630 con 0x7f803816a620 2023-10-06T03:08:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.539+0000 7f80367fc700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80240672f0 0x7f80240697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.540+0000 7f803e1c3700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80240672f0 0x7f80240697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.540+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f802c084fe0 con 0x7f803816a620 2023-10-06T03:08:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.540+0000 7f803e1c3700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80240672f0 0x7f80240697a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80300091e0 tx=0x7f803001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.544+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f802c051990 con 0x7f803816a620 2023-10-06T03:08:49.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.772+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f803815dbd0 con 0x7f803816a620 2023-10-06T03:08:49.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.772+0000 7f80367fc700 1 -- 172.21.15.92:0/2320384271 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f803815dbd0 con 0x7f803816a620 2023-10-06T03:08:49.773 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:49.774 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1888979047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1888979047},{"type":"v1","addr":"172.21.15.136:6835","nonce":1888979047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7464,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/628951753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":628951753},{"type":"v1","addr":"172.21.15.92:6837","nonce":628951753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:08:22.334042+0000","modified":"2023-10-06T03:08:26.961080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7221},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7221":{"gid":7221,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/1746152484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1746152484},{"type":"v1","addr":"172.21.15.92:6836","nonce":1746152484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80240672f0 msgr2=0x7f80240697a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80240672f0 0x7f80240697a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f80300091e0 tx=0x7f803001a000 comp rx=0 tx=0).stop 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 msgr2=0x7f803814bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 0x7f803814bea0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f802c000fd0 tx=0x7f802c000da0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 shutdown_connections 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80240672f0 0x7f80240697a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8038156c50 0x7f80381570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8020003a60 0x7f803815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 --2- 172.21.15.92:0/2320384271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f803816a620 0x7f803814bea0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 >> 172.21.15.92:0/2320384271 conn(0x7f80380d0660 msgr2=0x7f803814d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:49.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 shutdown_connections 2023-10-06T03:08:49.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.774+0000 7f803f1c5700 1 -- 172.21.15.92:0/2320384271 wait complete. 2023-10-06T03:08:49.777 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 86 2023-10-06T03:08:49.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:49.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.900+0000 7f6690f19700 1 Processor -- start 2023-10-06T03:08:49.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.900+0000 7f6690f19700 1 -- start start 2023-10-06T03:08:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f6690f19700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f668c1597f0 0x7f668c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f6690f19700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0b11d0 con 0x7f668c159420 2023-10-06T03:08:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f6690f19700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0b14b0 con 0x7f668c15d230 2023-10-06T03:08:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f6690f19700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c0b1790 con 0x7f668c1597f0 2023-10-06T03:08:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f668c1597f0 0x7f668c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f668c159420 0x7f668c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52958/0 (socket says 172.21.15.92:52958) 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668b7fe700 1 -- 172.21.15.92:0/4110915171 learned_addr learned my addr 172.21.15.92:0/4110915171 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668affd700 1 -- 172.21.15.92:0/4110915171 >> v1:172.21.15.136:6789/0 conn(0x7f668c15d230 legacy=0x7f668c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668affd700 1 -- 172.21.15.92:0/4110915171 >> v1:172.21.15.92:6789/0 conn(0x7f668c159420 legacy=0x7f668c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668a7fc700 1 -- 172.21.15.92:0/4110915171 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372060722 0 0) 0x7f668c0b11d0 con 0x7f668c159420 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.901+0000 7f668affd700 1 -- 172.21.15.92:0/4110915171 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0b1a70 con 0x7f668c1597f0 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f668affd700 1 --2- 172.21.15.92:0/4110915171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c1597f0 0x7f668c0b2270 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f66800098e0 tx=0x7f66800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41536ab091b9445 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f668a7fc700 1 -- 172.21.15.92:0/4110915171 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680009e80 con 0x7f668c1597f0 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f668a7fc700 1 -- 172.21.15.92:0/4110915171 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000b820 con 0x7f668c1597f0 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f668a7fc700 1 -- 172.21.15.92:0/4110915171 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000bba0 con 0x7f668c1597f0 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 -- 172.21.15.92:0/4110915171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c1597f0 msgr2=0x7f668c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 --2- 172.21.15.92:0/4110915171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c1597f0 0x7f668c0b2270 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f66800098e0 tx=0x7f66800091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 -- 172.21.15.92:0/4110915171 shutdown_connections 2023-10-06T03:08:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 --2- 172.21.15.92:0/4110915171 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c1597f0 0x7f668c0b2270 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 -- 172.21.15.92:0/4110915171 >> 172.21.15.92:0/4110915171 conn(0x7f668c0d0660 msgr2=0x7f668c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 -- 172.21.15.92:0/4110915171 shutdown_connections 2023-10-06T03:08:49.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.902+0000 7f6690f19700 1 -- 172.21.15.92:0/4110915171 wait complete. 2023-10-06T03:08:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 Processor -- start 2023-10-06T03:08:49.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 -- start start 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f668c159420 0x7f668c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f668c1597f0 0x7f668c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 0x7f668c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0ca2e0 con 0x7f668c15d230 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c15bbc0 con 0x7f668c159420 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f6690f19700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0c5400 con 0x7f668c1597f0 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f668bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 0x7f668c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f668bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 0x7f668c155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56218/0 (socket says 172.21.15.92:56218) 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.904+0000 7f668bfff700 1 -- 172.21.15.92:0/2695226288 learned_addr learned my addr 172.21.15.92:0/2695226288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668affd700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f668c1597f0 0x7f668c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668b7fe700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f668c159420 0x7f668c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668bfff700 1 -- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f668c1597f0 msgr2=0x7f668c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668bfff700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f668c1597f0 0x7f668c154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668bfff700 1 -- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f668c159420 msgr2=0x7f668c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668bfff700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f668c159420 0x7f668c15b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668bfff700 1 -- 172.21.15.92:0/2695226288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009420 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f668bfff700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 0x7f668c155bc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f668400b5e0 tx=0x7f6684009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400f510 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668400f670 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c0c96d0 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400fa30 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.905+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c0c9c90 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.906+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c0ca450 con 0x7f668c15d230 2023-10-06T03:08:49.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.906+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f668401a400 con 0x7f668c15d230 2023-10-06T03:08:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.907+0000 7f66737fe700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6678067320 0x7f66780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.907+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f6684084870 con 0x7f668c15d230 2023-10-06T03:08:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.907+0000 7f668b7fe700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6678067320 0x7f66780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:49.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.907+0000 7f668b7fe700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6678067320 0x7f66780697d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f667c002220 tx=0x7f667c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:49.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:49.911+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6684051470 con 0x7f668c15d230 2023-10-06T03:08:50.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.096+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f668c04c3c0 con 0x7f668c15d230 2023-10-06T03:08:50.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.097+0000 7f66737fe700 1 -- 172.21.15.92:0/2695226288 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17141 (secure 0 0 0) 0x7f6684054a90 con 0x7f668c15d230 2023-10-06T03:08:50.097 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:50.098 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:47.155134+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:08:19.983879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T03:08:21.301577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6835/3257155196":"2023-10-07T03:08:44.146778+0000","172.21.15.136:6833/3257155196":"2023-10-07T03:08:44.146778+0000","172.21.15.92:6837/3995167741":"2023-10-07T03:08:40.744848+0000","172.21.15.92:6835/3995167741":"2023-10-07T03:08:40.744848+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6678067320 msgr2=0x7f66780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6678067320 0x7f66780697d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f667c002220 tx=0x7f667c009040 comp rx=0 tx=0).stop 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 msgr2=0x7f668c155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 0x7f668c155bc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f668400b5e0 tx=0x7f6684009060 comp rx=0 tx=0).stop 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 shutdown_connections 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6678067320 0x7f66780697d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f668c15d230 0x7f668c155bc0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f668c159420 0x7f668c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 --2- 172.21.15.92:0/2695226288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f668c1597f0 0x7f668c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 >> 172.21.15.92:0/2695226288 conn(0x7f668c0d0660 msgr2=0x7f668c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:50.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 shutdown_connections 2023-10-06T03:08:50.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.099+0000 7f6690f19700 1 -- 172.21.15.92:0/2695226288 wait complete. 2023-10-06T03:08:50.112 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:08:50.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:50.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.227+0000 7f0580c94700 1 Processor -- start 2023-10-06T03:08:50.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.227+0000 7f0580c94700 1 -- start start 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f0580c94700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f057c159890 0x7f057c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f0580c94700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057c15a160 con 0x7f057c069560 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f0580c94700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057c0d13a0 con 0x7f057c159c60 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f0580c94700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f057c15e140 con 0x7f057c159890 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f057c069560 0x7f057c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52964/0 (socket says 172.21.15.92:52964) 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057ad9d700 1 -- 172.21.15.92:0/2059020401 learned_addr learned my addr 172.21.15.92:0/2059020401 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134977677 0 0) 0x7f057c15a160 con 0x7f057c069560 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057a59c700 1 --2- 172.21.15.92:0/2059020401 >> v2:172.21.15.136:3300/0 conn(0x7f057c159890 0x7f057c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0564003570 con 0x7f057c069560 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255720773 0 0) 0x7f0564003570 con 0x7f057c069560 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 >> v2:172.21.15.136:3300/0 conn(0x7f057c159890 msgr2=0x7f057c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 --2- 172.21.15.92:0/2059020401 >> v2:172.21.15.136:3300/0 conn(0x7f057c159890 0x7f057c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 >> v1:172.21.15.136:6789/0 conn(0x7f057c159c60 legacy=0x7f057c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c15a3d0 con 0x7f057c069560 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f05740017f0 con 0x7f057c069560 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 >> v1:172.21.15.92:6789/0 conn(0x7f057c069560 legacy=0x7f057c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0564003a60 0x7f0564003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0564004810 0x7f0564004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05640051c0 0x7f056400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c15e140 con 0x7f05640051c0 2023-10-06T03:08:50.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056400c0d0 con 0x7f0564003a60 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.228+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056400c290 con 0x7f0564004810 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05640051c0 0x7f056400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057a59c700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0564003a60 0x7f0564003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f0579d9b700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0564004810 0x7f0564004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 -- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0564004810 msgr2=0x7f0564004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0564004810 0x7f0564004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 -- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0564003a60 msgr2=0x7f0564003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0564003a60 0x7f0564003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 -- 172.21.15.92:0/2059020401 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564003570 con 0x7f05640051c0 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 -- 172.21.15.92:0/2059020401 reap_dead start 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057ad9d700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05640051c0 0x7f056400ba60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f057400a930 tx=0x7f05740021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d178a2674820f4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05740070d0 con 0x7f05640051c0 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574007230 con 0x7f05640051c0 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.229+0000 7f057959a700 1 -- 172.21.15.92:0/2059020401 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05740075d0 con 0x7f05640051c0 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 -- 172.21.15.92:0/2059020401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05640051c0 msgr2=0x7f056400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05640051c0 0x7f056400ba60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f057400a930 tx=0x7f05740021d0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 -- 172.21.15.92:0/2059020401 shutdown_connections 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 --2- 172.21.15.92:0/2059020401 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05640051c0 0x7f056400ba60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 -- 172.21.15.92:0/2059020401 >> 172.21.15.92:0/2059020401 conn(0x7f057c068e20 msgr2=0x7f057c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 -- 172.21.15.92:0/2059020401 shutdown_connections 2023-10-06T03:08:50.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.230+0000 7f0580c94700 1 -- 172.21.15.92:0/2059020401 wait complete. 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 Processor -- start 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 -- start start 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05640051c0 0x7f057c1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 0x7f057c0b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0b4440 0x7f057c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0b24e0 con 0x7f057c0b1b30 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0cde00 con 0x7f057c0b4440 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.231+0000 7f0580c94700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c0cd1f0 con 0x7f05640051c0 2023-10-06T03:08:50.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 0x7f057c0b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 0x7f057c0b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56240/0 (socket says 172.21.15.92:56240) 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 -- 172.21.15.92:0/2606035857 learned_addr learned my addr 172.21.15.92:0/2606035857 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f057a59c700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05640051c0 0x7f057c1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f057ad9d700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0b4440 0x7f057c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 -- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05640051c0 msgr2=0x7f057c1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05640051c0 0x7f057c1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 -- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0b4440 msgr2=0x7f057c0b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0b4440 0x7f057c0b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 -- 172.21.15.92:0/2606035857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c0cd4d0 con 0x7f057c0b1b30 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0579d9b700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 0x7f057c0b1fa0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0570002950 tx=0x7f0570007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570003390 con 0x7f057c0b1b30 2023-10-06T03:08:50.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0570006820 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570006bc0 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c0cd7b0 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.232+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c0b4df0 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.233+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c0cdf70 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.234+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0570007030 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.234+0000 7f056a7fc700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f055c067320 0x7f055c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.234+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f0570084e70 con 0x7f057c0b1b30 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.234+0000 7f057a59c700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f055c067320 0x7f055c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.234+0000 7f057a59c700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f055c067320 0x7f055c0697d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f057c0b4380 tx=0x7f056c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.238+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05700518f0 con 0x7f057c0b1b30 2023-10-06T03:08:50.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.465+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f057c04c3c0 con 0x7f057c0b1b30 2023-10-06T03:08:50.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.466+0000 7f056a7fc700 1 -- 172.21.15.92:0/2606035857 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f0570054f10 con 0x7f057c0b1b30 2023-10-06T03:08:50.467 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:50.467 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1888979047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1888979047},{"type":"v1","addr":"172.21.15.136:6835","nonce":1888979047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7464,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/628951753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":628951753},{"type":"v1","addr":"172.21.15.92:6837","nonce":628951753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:08:22.334042+0000","modified":"2023-10-06T03:08:26.961080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7221},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7221":{"gid":7221,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/1746152484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1746152484},{"type":"v1","addr":"172.21.15.92:6836","nonce":1746152484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f055c067320 msgr2=0x7f055c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f055c067320 0x7f055c0697d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f057c0b4380 tx=0x7f056c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 msgr2=0x7f057c0b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 0x7f057c0b1fa0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0570002950 tx=0x7f0570007d30 comp rx=0 tx=0).stop 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 shutdown_connections 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f055c067320 0x7f055c0697d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f057c0b4440 0x7f057c0b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05640051c0 0x7f057c1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 --2- 172.21.15.92:0/2606035857 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f057c0b1b30 0x7f057c0b1fa0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 >> 172.21.15.92:0/2606035857 conn(0x7f057c068e20 msgr2=0x7f057c15d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 shutdown_connections 2023-10-06T03:08:50.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.468+0000 7f0580c94700 1 -- 172.21.15.92:0/2606035857 wait complete. 2023-10-06T03:08:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 86 2023-10-06T03:08:50.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:50.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.594+0000 7f65a3455700 1 Processor -- start 2023-10-06T03:08:50.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.594+0000 7f65a3455700 1 -- start start 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a3455700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c159890 0x7f659c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a3455700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f659c069560 0x7f659c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a3455700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659c15a160 con 0x7f659c159c60 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a3455700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c069f20 con 0x7f659c069560 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a3455700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0d13a0 con 0x7f659c159890 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f659c069560 0x7f659c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f659c069560 0x7f659c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56250/0 (socket says 172.21.15.92:56250) 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 -- 172.21.15.92:0/313873662 learned_addr learned my addr 172.21.15.92:0/313873662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a11f1700 1 --2- 172.21.15.92:0/313873662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c159890 0x7f659c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 -- 172.21.15.92:0/313873662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c159890 msgr2=0x7f659c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 --2- 172.21.15.92:0/313873662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c159890 0x7f659c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 -- 172.21.15.92:0/313873662 >> v1:172.21.15.136:6789/0 conn(0x7f659c159c60 legacy=0x7f659c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 -- 172.21.15.92:0/313873662 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c15e140 con 0x7f659c069560 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f65a19f2700 1 --2- 172.21.15.92:0/313873662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c069560 0x7f659c0699e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f658c0099b0 tx=0x7f658c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394011c6559e74e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.595+0000 7f659bfff700 1 -- 172.21.15.92:0/313873662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00b860 con 0x7f659c069560 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f659bfff700 1 -- 172.21.15.92:0/313873662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c00b9c0 con 0x7f659c069560 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f659bfff700 1 -- 172.21.15.92:0/313873662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00bd60 con 0x7f659c069560 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 -- 172.21.15.92:0/313873662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c069560 msgr2=0x7f659c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 --2- 172.21.15.92:0/313873662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c069560 0x7f659c0699e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f658c0099b0 tx=0x7f658c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:50.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 -- 172.21.15.92:0/313873662 shutdown_connections 2023-10-06T03:08:50.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 --2- 172.21.15.92:0/313873662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c069560 0x7f659c0699e0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 --2- 172.21.15.92:0/313873662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c159890 0x7f659c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 -- 172.21.15.92:0/313873662 >> 172.21.15.92:0/313873662 conn(0x7f659c068e20 msgr2=0x7f659c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:50.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 -- 172.21.15.92:0/313873662 shutdown_connections 2023-10-06T03:08:50.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.596+0000 7f65a3455700 1 -- 172.21.15.92:0/313873662 wait complete. 2023-10-06T03:08:50.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.597+0000 7f65a3455700 1 Processor -- start 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.597+0000 7f65a3455700 1 -- start start 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a3455700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c069560 0x7f659c146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a3455700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f659c159890 0x7f659c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a3455700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 0x7f659c0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a3455700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0b4610 con 0x7f659c159c60 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a3455700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0b4aa0 con 0x7f659c159890 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a3455700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0b3a00 con 0x7f659c069560 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a19f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 0x7f659c0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a19f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 0x7f659c0d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56266/0 (socket says 172.21.15.92:56266) 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a19f2700 1 -- 172.21.15.92:0/4024828609 learned_addr learned my addr 172.21.15.92:0/4024828609 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a19f2700 1 -- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c069560 msgr2=0x7f659c146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a11f1700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c069560 0x7f659c146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a19f2700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c069560 0x7f659c146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.598+0000 7f65a09f0700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f659c159890 0x7f659c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65a19f2700 1 -- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f659c159890 msgr2=0x7f659c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65a19f2700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f659c159890 0x7f659c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65a19f2700 1 -- 172.21.15.92:0/4024828609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c0094f0 con 0x7f659c159c60 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65a19f2700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 0x7f659c0d05f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f659c0899c0 tx=0x7f658c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c01a630 con 0x7f659c159c60 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c01a7b0 con 0x7f659c159c60 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c0b3ce0 con 0x7f659c159c60 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c01ab70 con 0x7f659c159c60 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.599+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c0b42a0 con 0x7f659c159c60 2023-10-06T03:08:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.600+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f659c04c3c0 con 0x7f659c159c60 2023-10-06T03:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.600+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f658c02d440 con 0x7f659c159c60 2023-10-06T03:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.600+0000 7f65997fa700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6588067320 0x7f65880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.601+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f658c086b00 con 0x7f659c159c60 2023-10-06T03:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.601+0000 7f65a11f1700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6588067320 0x7f65880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.601+0000 7f65a11f1700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6588067320 0x7f65880697d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f659c16d0e0 tx=0x7f659400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.605+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f658c088e60 con 0x7f659c159c60 2023-10-06T03:08:50.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.829+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f659c0b1f90 con 0x7f659c159c60 2023-10-06T03:08:50.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.829+0000 7f65997fa700 1 -- 172.21.15.92:0/4024828609 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f658c053580 con 0x7f659c159c60 2023-10-06T03:08:50.830 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:50.830 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":86,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1888979047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1888979047},{"type":"v1","addr":"172.21.15.136:6835","nonce":1888979047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7464,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/628951753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":628951753},{"type":"v1","addr":"172.21.15.92:6837","nonce":628951753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:08:22.334042+0000","modified":"2023-10-06T03:08:26.961080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7221},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7221":{"gid":7221,"name":"a","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6836/1746152484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":1746152484},{"type":"v1","addr":"172.21.15.92:6836","nonce":1746152484}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T03:08:50.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6588067320 msgr2=0x7f65880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6588067320 0x7f65880697d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f659c16d0e0 tx=0x7f659400b040 comp rx=0 tx=0).stop 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 msgr2=0x7f659c0d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 0x7f659c0d05f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f659c0899c0 tx=0x7f658c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 shutdown_connections 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6588067320 0x7f65880697d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f659c159c60 0x7f659c0d05f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f659c069560 0x7f659c146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 --2- 172.21.15.92:0/4024828609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f659c159890 0x7f659c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 >> 172.21.15.92:0/4024828609 conn(0x7f659c068e20 msgr2=0x7f659c157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:50.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 shutdown_connections 2023-10-06T03:08:50.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.831+0000 7f65a3455700 1 -- 172.21.15.92:0/4024828609 wait complete. 2023-10-06T03:08:50.834 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 86 2023-10-06T03:08:50.844 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:50.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.957+0000 7eff47dd9700 1 Processor -- start 2023-10-06T03:08:50.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.957+0000 7eff47dd9700 1 -- start start 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff47dd9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff400b0a10 0x7eff40155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff47dd9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff400ad930 con 0x7eff40155c10 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff47dd9700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff400acd20 con 0x7eff401578b0 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff47dd9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400ad000 con 0x7eff400b0a10 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff400b0a10 0x7eff40155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff400b0a10 0x7eff40155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56282/0 (socket says 172.21.15.92:56282) 2023-10-06T03:08:50.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff45b75700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7eff40155c10 0x7eff400ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52978/0 (socket says 172.21.15.92:52978) 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 -- 172.21.15.92:0/2463391295 learned_addr learned my addr 172.21.15.92:0/2463391295 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 -- 172.21.15.92:0/2463391295 >> v1:172.21.15.136:6789/0 conn(0x7eff401578b0 legacy=0x7eff400af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 -- 172.21.15.92:0/2463391295 >> v1:172.21.15.92:6789/0 conn(0x7eff40155c10 legacy=0x7eff400ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff44b73700 1 -- 172.21.15.92:0/2463391295 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795831931 0 0) 0x7eff400ad930 con 0x7eff40155c10 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 -- 172.21.15.92:0/2463391295 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400ad2e0 con 0x7eff400b0a10 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.958+0000 7eff46376700 1 --2- 172.21.15.92:0/2463391295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff400b0a10 0x7eff40155fe0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7eff3800d120 tx=0x7eff3800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fc502c59f7b08e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff44b73700 1 -- 172.21.15.92:0/2463391295 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38010070 con 0x7eff400b0a10 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff44b73700 1 -- 172.21.15.92:0/2463391295 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3800ddc0 con 0x7eff400b0a10 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff44b73700 1 -- 172.21.15.92:0/2463391295 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3800b320 con 0x7eff400b0a10 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 -- 172.21.15.92:0/2463391295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff400b0a10 msgr2=0x7eff40155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2463391295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff400b0a10 0x7eff40155fe0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7eff3800d120 tx=0x7eff3800ba50 comp rx=0 tx=0).stop 2023-10-06T03:08:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 -- 172.21.15.92:0/2463391295 shutdown_connections 2023-10-06T03:08:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2463391295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff400b0a10 0x7eff40155fe0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 -- 172.21.15.92:0/2463391295 >> 172.21.15.92:0/2463391295 conn(0x7eff4006a320 msgr2=0x7eff400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 -- 172.21.15.92:0/2463391295 shutdown_connections 2023-10-06T03:08:50.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.959+0000 7eff47dd9700 1 -- 172.21.15.92:0/2463391295 wait complete. 2023-10-06T03:08:50.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.960+0000 7eff47dd9700 1 Processor -- start 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.960+0000 7eff47dd9700 1 -- start start 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff47dd9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff400b0a10 0x7eff400cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff47dd9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff40155c10 0x7eff400c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff47dd9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 0x7eff4014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff47dd9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400ccf20 con 0x7eff401578b0 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff47dd9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cc310 con 0x7eff40155c10 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff47dd9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400cc5f0 con 0x7eff400b0a10 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 0x7eff4014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 0x7eff4014df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56298/0 (socket says 172.21.15.92:56298) 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 -- 172.21.15.92:0/2518923853 learned_addr learned my addr 172.21.15.92:0/2518923853 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff45b75700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff400b0a10 0x7eff400cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff45374700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff40155c10 0x7eff400c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 -- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff400b0a10 msgr2=0x7eff400cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff400b0a10 0x7eff400cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 -- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff40155c10 msgr2=0x7eff400c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff40155c10 0x7eff400c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff46376700 1 -- 172.21.15.92:0/2518923853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3800bcf0 con 0x7eff401578b0 2023-10-06T03:08:50.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.961+0000 7eff45b75700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff400b0a10 0x7eff400cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.962+0000 7eff46376700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 0x7eff4014df20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7eff3800ba90 tx=0x7eff38019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.962+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38009dd0 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.962+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff38014430 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.962+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff400cc8d0 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.962+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380147f0 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.962+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff400cbec0 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.963+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4004c3c0 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.963+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7eff38014ba0 con 0x7eff401578b0 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.964+0000 7eff35ffb700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff24067320 0x7eff240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:50.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.964+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7eff38088a20 con 0x7eff401578b0 2023-10-06T03:08:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.964+0000 7eff45b75700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff24067320 0x7eff240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.964+0000 7eff45b75700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff24067320 0x7eff240697d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff3c000c00 tx=0x7eff3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:50.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:50.968+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff38055390 con 0x7eff401578b0 2023-10-06T03:08:51.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.150+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff400ad5c0 con 0x7eff401578b0 2023-10-06T03:08:51.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.151+0000 7eff35ffb700 1 -- 172.21.15.92:0/2518923853 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17141 (secure 0 0 0) 0x7eff380589b0 con 0x7eff401578b0 2023-10-06T03:08:51.152 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:51.152 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:47.155134+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:08:19.983879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T03:08:21.301577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6835/3257155196":"2023-10-07T03:08:44.146778+0000","172.21.15.136:6833/3257155196":"2023-10-07T03:08:44.146778+0000","172.21.15.92:6837/3995167741":"2023-10-07T03:08:40.744848+0000","172.21.15.92:6835/3995167741":"2023-10-07T03:08:40.744848+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff24067320 msgr2=0x7eff240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff24067320 0x7eff240697d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7eff3c000c00 tx=0x7eff3c009040 comp rx=0 tx=0).stop 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 msgr2=0x7eff4014df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 0x7eff4014df20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7eff3800ba90 tx=0x7eff38019000 comp rx=0 tx=0).stop 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 shutdown_connections 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff24067320 0x7eff240697d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff401578b0 0x7eff4014df20 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff400b0a10 0x7eff400cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 --2- 172.21.15.92:0/2518923853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff40155c10 0x7eff400c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 >> 172.21.15.92:0/2518923853 conn(0x7eff4006a320 msgr2=0x7eff4006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 shutdown_connections 2023-10-06T03:08:51.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.153+0000 7eff47dd9700 1 -- 172.21.15.92:0/2518923853 wait complete. 2023-10-06T03:08:51.166 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:08:51.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.280+0000 7f8ff669e700 1 Processor -- start 2023-10-06T03:08:51.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.281+0000 7f8ff669e700 1 -- start start 2023-10-06T03:08:51.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.281+0000 7f8ff669e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ff0159c60 0x7f8ff015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.281+0000 7f8ff669e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff015db10 0x7f8ff0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.281+0000 7f8ff669e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff015a160 con 0x7f8ff0159890 2023-10-06T03:08:51.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.281+0000 7f8ff669e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff015df50 con 0x7f8ff0159c60 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.281+0000 7f8ff669e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00d13a0 con 0x7f8ff015db10 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ff0159c60 0x7f8ff015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ff0159c60 0x7f8ff015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56306/0 (socket says 172.21.15.92:56306) 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 -- 172.21.15.92:0/857688692 learned_addr learned my addr 172.21.15.92:0/857688692 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8ff4c3b700 1 --2- 172.21.15.92:0/857688692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff015db10 0x7f8ff0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 -- 172.21.15.92:0/857688692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff015db10 msgr2=0x7f8ff0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 --2- 172.21.15.92:0/857688692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff015db10 0x7f8ff0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 -- 172.21.15.92:0/857688692 >> v1:172.21.15.136:6789/0 conn(0x7f8ff0159890 legacy=0x7f8ff015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 -- 172.21.15.92:0/857688692 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff015e140 con 0x7f8ff0159c60 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8fef7fe700 1 --2- 172.21.15.92:0/857688692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff015d5d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40099b0 tx=0x7f8fe4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e92364f48101b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:51.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8feeffd700 1 -- 172.21.15.92:0/857688692 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400b820 con 0x7f8ff0159c60 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8feeffd700 1 -- 172.21.15.92:0/857688692 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400b980 con 0x7f8ff0159c60 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.282+0000 7f8feeffd700 1 -- 172.21.15.92:0/857688692 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400bd20 con 0x7f8ff0159c60 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 -- 172.21.15.92:0/857688692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 msgr2=0x7f8ff015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 --2- 172.21.15.92:0/857688692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff015d5d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40099b0 tx=0x7f8fe4009250 comp rx=0 tx=0).stop 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 -- 172.21.15.92:0/857688692 shutdown_connections 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 --2- 172.21.15.92:0/857688692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff015db10 0x7f8ff0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 --2- 172.21.15.92:0/857688692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff015d5d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 -- 172.21.15.92:0/857688692 >> 172.21.15.92:0/857688692 conn(0x7f8ff0068e20 msgr2=0x7f8ff00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 -- 172.21.15.92:0/857688692 shutdown_connections 2023-10-06T03:08:51.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.283+0000 7f8ff669e700 1 -- 172.21.15.92:0/857688692 wait complete. 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.284+0000 7f8ff669e700 1 Processor -- start 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.284+0000 7f8ff669e700 1 -- start start 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff669e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff0159890 0x7f8ff00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff669e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff00cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff669e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff015db10 0x7f8ff0153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff669e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00ce140 con 0x7f8ff0159c60 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff669e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00cd770 con 0x7f8ff015db10 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff669e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00cd160 con 0x7f8ff0159890 2023-10-06T03:08:51.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff00cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff00cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56314/0 (socket says 172.21.15.92:56314) 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 -- 172.21.15.92:0/274054963 learned_addr learned my addr 172.21.15.92:0/274054963 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8feffff700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff0159890 0x7f8ff00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff4c3b700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff015db10 0x7f8ff0153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 -- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff0159890 msgr2=0x7f8ff00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff0159890 0x7f8ff00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 -- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff015db10 msgr2=0x7f8ff0153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff015db10 0x7f8ff0153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 -- 172.21.15.92:0/274054963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe40094f0 con 0x7f8ff0159c60 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8ff4c3b700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff015db10 0x7f8ff0153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fef7fe700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff00cdc00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009290 tx=0x7f8fe4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:51.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.285+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4011d90 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.286+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4020070 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.286+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff00cfbc0 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.286+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401b6a0 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.286+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff00cfea0 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.287+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff00cd8e0 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.287+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8fe4022030 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.287+0000 7f8fd7fff700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8fdc067320 0x7f8fdc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.287+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6230+0+0 (secure 0 0 0) 0x7f8fe4086cf0 con 0x7f8ff0159c60 2023-10-06T03:08:51.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.287+0000 7f8feffff700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8fdc067320 0x7f8fdc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.288+0000 7f8feffff700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8fdc067320 0x7f8fdc0697d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0003870 tx=0x7f8fe0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:51.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.292+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fe4053880 con 0x7f8ff0159c60 2023-10-06T03:08:51.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.516+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8ff00cd8e0 con 0x7f8ff0159c60 2023-10-06T03:08:51.517 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:08:51.517+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:08:51.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.520+0000 7f8fd7fff700 1 -- 172.21.15.92:0/274054963 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8fe4056ea0 con 0x7f8ff0159c60 2023-10-06T03:08:51.522 INFO:tasks.ceph.mds.a.smithi092.stderr: -1420> 2023-10-06T03:08:27.240+0000 7f6ea15b7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:08:51.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8fdc067320 msgr2=0x7f8fdc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8fdc067320 0x7f8fdc0697d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0003870 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 msgr2=0x7f8ff00cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff00cdc00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009290 tx=0x7f8fe4016000 comp rx=0 tx=0).stop 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 shutdown_connections 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8fdc067320 0x7f8fdc0697d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ff015db10 0x7f8ff0153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ff0159890 0x7f8ff00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.522+0000 7f8ff669e700 1 --2- 172.21.15.92:0/274054963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff0159c60 0x7f8ff00cdc00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.523+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 >> 172.21.15.92:0/274054963 conn(0x7f8ff0068e20 msgr2=0x7f8ff0158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:51.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.523+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 shutdown_connections 2023-10-06T03:08:51.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.523+0000 7f8ff669e700 1 -- 172.21.15.92:0/274054963 wait complete. 2023-10-06T03:08:51.525 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:08:51.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:08:51.545 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:08:51.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.652+0000 7f694cbb6700 1 Processor -- start 2023-10-06T03:08:51.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.652+0000 7f694cbb6700 1 -- start start 2023-10-06T03:08:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694cbb6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f694806bcd0 0x7f69480caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694cbb6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f694815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694cbb6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694815d7e0 con 0x7f694806c0a0 2023-10-06T03:08:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694cbb6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694815dac0 con 0x7f694806bcd0 2023-10-06T03:08:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694cbb6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694815aa30 con 0x7f694815df50 2023-10-06T03:08:51.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f694806bcd0 0x7f69480caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f694806bcd0 0x7f69480caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56336/0 (socket says 172.21.15.92:56336) 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694659c700 1 -- 172.21.15.92:0/1385001447 learned_addr learned my addr 172.21.15.92:0/1385001447 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f6946d9d700 1 --2- 172.21.15.92:0/1385001447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f694815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694659c700 1 -- 172.21.15.92:0/1385001447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 msgr2=0x7f694815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.653+0000 7f694659c700 1 --2- 172.21.15.92:0/1385001447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f694815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694659c700 1 -- 172.21.15.92:0/1385001447 >> v1:172.21.15.136:6789/0 conn(0x7f694806c0a0 legacy=0x7f694815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694659c700 1 -- 172.21.15.92:0/1385001447 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694815ad10 con 0x7f694806bcd0 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694659c700 1 --2- 172.21.15.92:0/1385001447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806bcd0 0x7f69480caa90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f69380099b0 tx=0x7f6938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e457d2eaf2651d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f6945d9b700 1 -- 172.21.15.92:0/1385001447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800b860 con 0x7f694806bcd0 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f6945d9b700 1 -- 172.21.15.92:0/1385001447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693800b9c0 con 0x7f694806bcd0 2023-10-06T03:08:51.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f6945d9b700 1 -- 172.21.15.92:0/1385001447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800bd80 con 0x7f694806bcd0 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 -- 172.21.15.92:0/1385001447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806bcd0 msgr2=0x7f69480caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 --2- 172.21.15.92:0/1385001447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806bcd0 0x7f69480caa90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f69380099b0 tx=0x7f6938009250 comp rx=0 tx=0).stop 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 -- 172.21.15.92:0/1385001447 shutdown_connections 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 --2- 172.21.15.92:0/1385001447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806bcd0 0x7f69480caa90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 --2- 172.21.15.92:0/1385001447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f694815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 -- 172.21.15.92:0/1385001447 >> 172.21.15.92:0/1385001447 conn(0x7f69480ce600 msgr2=0x7f69480cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 -- 172.21.15.92:0/1385001447 shutdown_connections 2023-10-06T03:08:51.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.654+0000 7f694cbb6700 1 -- 172.21.15.92:0/1385001447 wait complete. 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 Processor -- start 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 -- start start 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f694806bcd0 0x7f69480b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 0x7f69480c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f6948156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480c56b0 con 0x7f694806c0a0 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480c7c90 con 0x7f694806bcd0 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.656+0000 7f694cbb6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480c7f70 con 0x7f694815df50 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 0x7f69480c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 0x7f69480c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56350/0 (socket says 172.21.15.92:56350) 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 -- 172.21.15.92:0/48420884 learned_addr learned my addr 172.21.15.92:0/48420884 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f6946d9d700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f6948156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 -- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 msgr2=0x7f6948156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f6948156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 -- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f694806bcd0 msgr2=0x7f69480b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f694806bcd0 0x7f69480b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 -- 172.21.15.92:0/48420884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69380094f0 con 0x7f694806c0a0 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f6946d9d700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f6948156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693dd9b700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 0x7f69480c7340 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f693000c4e0 tx=0x7f693000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930011590 con 0x7f694806c0a0 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6930011710 con 0x7f694806c0a0 2023-10-06T03:08:51.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480c8250 con 0x7f694806c0a0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930011ad0 con 0x7f694806c0a0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.657+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69480b2ad0 con 0x7f694806c0a0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.658+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69480c5b40 con 0x7f694806c0a0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.658+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f693001b3b0 con 0x7f694806c0a0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.659+0000 7f693effd700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6928067320 0x7f69280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.659+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f6930084420 con 0x7f694806c0a0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.659+0000 7f694659c700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6928067320 0x7f69280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:51.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.659+0000 7f694659c700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6928067320 0x7f69280697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f69380099b0 tx=0x7f6938017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:51.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.663+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69480c5b40 con 0x7f694806c0a0 2023-10-06T03:08:51.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:51.889+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f69480c5b40 con 0x7f694806c0a0 2023-10-06T03:08:52.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.520+0000 7f693effd700 1 -- 172.21.15.92:0/48420884 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 104+0+0 (secure 0 0 0) 0x7f69480c5b40 con 0x7f694806c0a0 2023-10-06T03:08:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6928067320 msgr2=0x7f69280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6928067320 0x7f69280697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f69380099b0 tx=0x7f6938017000 comp rx=0 tx=0).stop 2023-10-06T03:08:52.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 msgr2=0x7f69480c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 0x7f69480c7340 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f693000c4e0 tx=0x7f693000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 shutdown_connections 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6928067320 0x7f69280697d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694815df50 0x7f6948156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f694806bcd0 0x7f69480b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 --2- 172.21.15.92:0/48420884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f694806c0a0 0x7f69480c7340 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 >> 172.21.15.92:0/48420884 conn(0x7f69480ce600 msgr2=0x7f69480c39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 shutdown_connections 2023-10-06T03:08:52.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.522+0000 7f694cbb6700 1 -- 172.21.15.92:0/48420884 wait complete. 2023-10-06T03:08:52.538 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:08:52.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.697+0000 7fc44359e700 1 Processor -- start 2023-10-06T03:08:52.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.697+0000 7fc44359e700 1 -- start start 2023-10-06T03:08:52.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc44359e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc4440ac0e0 0x7fc4440afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:52.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc44359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 0x7fc44415d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:52.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc44359e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44406e510 con 0x7fc4441597d0 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc44359e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc444069180 con 0x7fc4440ac0e0 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc44359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44406dec0 con 0x7fc44406f0b0 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 0x7fc44415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc439d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc4440ac0e0 0x7fc4440afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 0x7fc44415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:43956/0 (socket says 172.21.15.92:43956) 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc442d9d700 1 -- 172.21.15.92:0/2396308683 learned_addr learned my addr 172.21.15.92:0/2396308683 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc439d9b700 1 -- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 msgr2=0x7fc44415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc439d9b700 1 --2- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 0x7fc44415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc439d9b700 1 -- 172.21.15.92:0/2396308683 >> v1:172.21.15.136:6789/0 conn(0x7fc4441597d0 legacy=0x7fc44406e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc439d9b700 1 -- 172.21.15.92:0/2396308683 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44406e1a0 con 0x7fc4440ac0e0 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.698+0000 7fc442d9d700 1 --2- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 0x7fc44415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:52.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc439d9b700 1 --2- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4440ac0e0 0x7fc4440afa40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0098e0 tx=0x7fc42c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd97f5a890a6541d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc441d9b700 1 -- 172.21.15.92:0/2396308683 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00b640 con 0x7fc4440ac0e0 2023-10-06T03:08:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc441d9b700 1 -- 172.21.15.92:0/2396308683 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c00b7a0 con 0x7fc4440ac0e0 2023-10-06T03:08:52.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc441d9b700 1 -- 172.21.15.92:0/2396308683 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00bb40 con 0x7fc4440ac0e0 2023-10-06T03:08:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 -- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4440ac0e0 msgr2=0x7fc4440afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 --2- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4440ac0e0 0x7fc4440afa40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0098e0 tx=0x7fc42c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 -- 172.21.15.92:0/2396308683 shutdown_connections 2023-10-06T03:08:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 --2- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc44406f0b0 0x7fc44415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 --2- 172.21.15.92:0/2396308683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4440ac0e0 0x7fc4440afa40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 -- 172.21.15.92:0/2396308683 >> 172.21.15.92:0/2396308683 conn(0x7fc44406adf0 msgr2=0x7fc4440ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 -- 172.21.15.92:0/2396308683 shutdown_connections 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.699+0000 7fc44359e700 1 -- 172.21.15.92:0/2396308683 wait complete. 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 Processor -- start 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 -- start start 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 0x7fc44415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4441597d0 0x7fc44414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc444158720 0x7fc44416a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44414ffc0 con 0x7fc44406f0b0 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc444169de0 con 0x7fc4441597d0 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4441503d0 con 0x7fc444158720 2023-10-06T03:08:52.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.701+0000 7fc44259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 0x7fc44415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 0x7fc44415db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56366/0 (socket says 172.21.15.92:56366) 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 -- 172.21.15.92:0/491959431 learned_addr learned my addr 172.21.15.92:0/491959431 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc442d9d700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc444158720 0x7fc44416a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc439d9b700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4441597d0 0x7fc44414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 -- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc444158720 msgr2=0x7fc44416a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc444158720 0x7fc44416a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 -- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4441597d0 msgr2=0x7fc44414fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4441597d0 0x7fc44414fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 -- 172.21.15.92:0/491959431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009ae0 con 0x7fc44406f0b0 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44259c700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 0x7fc44415db80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c510 tx=0x7fc43400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:52.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434011590 con 0x7fc44406f0b0 2023-10-06T03:08:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434011710 con 0x7fc44406f0b0 2023-10-06T03:08:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c009420 con 0x7fc44406f0b0 2023-10-06T03:08:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434011ad0 con 0x7fc44406f0b0 2023-10-06T03:08:52.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.702+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44415e560 con 0x7fc44406f0b0 2023-10-06T03:08:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.703+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44416a0c0 con 0x7fc44406f0b0 2023-10-06T03:08:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.704+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc43401b3b0 con 0x7fc44406f0b0 2023-10-06T03:08:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.704+0000 7fc43affd700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc424067320 0x7fc4240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.704+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fc434084290 con 0x7fc44406f0b0 2023-10-06T03:08:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.704+0000 7fc439d9b700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc424067320 0x7fc4240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:52.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.704+0000 7fc439d9b700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc424067320 0x7fc4240697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0093f0 tx=0x7fc42c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:52.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.708+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc434050cc0 con 0x7fc44406f0b0 2023-10-06T03:08:52.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:52.890+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc44404c3c0 con 0x7fc44406f0b0 2023-10-06T03:08:53.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.532+0000 7fc43affd700 1 -- 172.21.15.92:0/491959431 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc4340170d0 con 0x7fc44406f0b0 2023-10-06T03:08:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc424067320 msgr2=0x7fc4240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc424067320 0x7fc4240697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0093f0 tx=0x7fc42c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 msgr2=0x7fc44415db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:53.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 0x7fc44415db80 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fc43400c510 tx=0x7fc43400c820 comp rx=0 tx=0).stop 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 shutdown_connections 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc424067320 0x7fc4240697d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc44406f0b0 0x7fc44415db80 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc444158720 0x7fc44416a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 --2- 172.21.15.92:0/491959431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4441597d0 0x7fc44414fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 >> 172.21.15.92:0/491959431 conn(0x7fc44406adf0 msgr2=0x7fc444151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 shutdown_connections 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.534+0000 7fc44359e700 1 -- 172.21.15.92:0/491959431 wait complete. 2023-10-06T03:08:53.537 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:08:53.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:08:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 Processor -- start 2023-10-06T03:08:53.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 -- start start 2023-10-06T03:08:53.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd4cc0cd910 0x7fd4cc0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc0b55a0 0x7fd4cc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0b5b00 con 0x7fd4cc159ec0 2023-10-06T03:08:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0b5da0 con 0x7fd4cc0cd910 2023-10-06T03:08:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.667+0000 7fd4d095e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0b6080 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4ca59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd4cc159ec0 0x7fd4cc0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52994/0 (socket says 172.21.15.92:52994) 2023-10-06T03:08:53.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4ca59c700 1 -- 172.21.15.92:0/2537072555 learned_addr learned my addr 172.21.15.92:0/2537072555 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c1d9b700 1 --2- 172.21.15.92:0/2537072555 >> v2:172.21.15.92:3300/0 conn(0x7fd4cc0cd910 0x7fd4cc0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4cad9d700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc0b55a0 0x7fd4cc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c1d9b700 1 -- 172.21.15.92:0/2537072555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc0b55a0 msgr2=0x7fd4cc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c1d9b700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc0b55a0 0x7fd4cc0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c1d9b700 1 -- 172.21.15.92:0/2537072555 >> v1:172.21.15.92:6789/0 conn(0x7fd4cc159ec0 legacy=0x7fd4cc0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c1d9b700 1 -- 172.21.15.92:0/2537072555 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0cc100 con 0x7fd4cc0cd910 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4cad9d700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc0b55a0 0x7fd4cc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:53.672 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c1d9b700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0cd910 0x7fd4cc0ce340 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009910 tx=0x7fd4b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42b0e4a1cc5cbfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c9d9b700 1 -- 172.21.15.92:0/2537072555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400b820 con 0x7fd4cc0cd910 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c9d9b700 1 -- 172.21.15.92:0/2537072555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b400b980 con 0x7fd4cc0cd910 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.668+0000 7fd4c9d9b700 1 -- 172.21.15.92:0/2537072555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b400bd20 con 0x7fd4cc0cd910 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 -- 172.21.15.92:0/2537072555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0cd910 msgr2=0x7fd4cc0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0cd910 0x7fd4cc0ce340 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009910 tx=0x7fd4b40091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 -- 172.21.15.92:0/2537072555 shutdown_connections 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc0b55a0 0x7fd4cc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 --2- 172.21.15.92:0/2537072555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0cd910 0x7fd4cc0ce340 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 -- 172.21.15.92:0/2537072555 >> 172.21.15.92:0/2537072555 conn(0x7fd4cc0999f0 msgr2=0x7fd4cc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 -- 172.21.15.92:0/2537072555 shutdown_connections 2023-10-06T03:08:53.673 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.669+0000 7fd4d095e700 1 -- 172.21.15.92:0/2537072555 wait complete. 2023-10-06T03:08:53.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.670+0000 7fd4d095e700 1 Processor -- start 2023-10-06T03:08:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.670+0000 7fd4d095e700 1 -- start start 2023-10-06T03:08:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 0x7fd4cc16a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4cc159ec0 0x7fd4cc1450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc1561e0 0x7fd4cc158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:53.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc152c90 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc16aba0 con 0x7fd4cc159ec0 2023-10-06T03:08:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1566e0 con 0x7fd4cc1561e0 2023-10-06T03:08:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 0x7fd4cc16a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 0x7fd4cc16a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56374/0 (socket says 172.21.15.92:56374) 2023-10-06T03:08:53.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 -- 172.21.15.92:0/1962609743 learned_addr learned my addr 172.21.15.92:0/1962609743 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4cad9d700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc1561e0 0x7fd4cc158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4c1d9b700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4cc159ec0 0x7fd4cc1450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 -- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc1561e0 msgr2=0x7fd4cc158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc1561e0 0x7fd4cc158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 -- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4cc159ec0 msgr2=0x7fd4cc1450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4cc159ec0 0x7fd4cc1450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 -- 172.21.15.92:0/1962609743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b4009420 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4ca59c700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 0x7fd4cc16a660 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c790 tx=0x7fd4bc007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0073b0 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc007530 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc1580c0 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc005a40 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.671+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc158700 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.672+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc063c90 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.673+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd4bc01b3b0 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.673+0000 7fd4c2ffd700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4ac067320 0x7fd4ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.673+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5878+0+0 (secure 0 0 0) 0x7fd4bc084220 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.673+0000 7fd4c1d9b700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4ac067320 0x7fd4ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.674+0000 7fd4c1d9b700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4ac067320 0x7fd4ac0697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0c75d0 tx=0x7fd4b4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.677+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4bc050e00 con 0x7fd4cc0b55a0 2023-10-06T03:08:53.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:53.859+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4cc0718b0 con 0x7fd4cc0b55a0 2023-10-06T03:08:54.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.540+0000 7fd4c2ffd700 1 -- 172.21.15.92:0/1962609743 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd4bc054420 con 0x7fd4cc0b55a0 2023-10-06T03:08:54.545 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4ac067320 msgr2=0x7fd4ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4ac067320 0x7fd4ac0697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0c75d0 tx=0x7fd4b4017000 comp rx=0 tx=0).stop 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 msgr2=0x7fd4cc16a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 0x7fd4cc16a660 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c790 tx=0x7fd4bc007da0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 shutdown_connections 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd4ac067320 0x7fd4ac0697d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4cc0b55a0 0x7fd4cc16a660 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4cc1561e0 0x7fd4cc158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 --2- 172.21.15.92:0/1962609743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4cc159ec0 0x7fd4cc1450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 >> 172.21.15.92:0/1962609743 conn(0x7fd4cc0999f0 msgr2=0x7fd4cc0d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 shutdown_connections 2023-10-06T03:08:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.544+0000 7fd4d095e700 1 -- 172.21.15.92:0/1962609743 wait complete. 2023-10-06T03:08:54.547 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:08:54.559 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:08:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.674+0000 7fd2f7112700 1 Processor -- start 2023-10-06T03:08:54.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.674+0000 7fd2f7112700 1 -- start start 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f7112700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 0x7fd2f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f7112700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd2f015db10 0x7fd2f0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f7112700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f015a160 con 0x7fd2f0159890 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f7112700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f00d13a0 con 0x7fd2f015db10 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f7112700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015e140 con 0x7fd2f0159c60 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f4eae700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd2f0159890 0x7fd2f015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52996/0 (socket says 172.21.15.92:52996) 2023-10-06T03:08:54.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.675+0000 7fd2f4eae700 1 -- 172.21.15.92:0/2139837241 learned_addr learned my addr 172.21.15.92:0/2139837241 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f56af700 1 --2- 172.21.15.92:0/2139837241 >> v2:172.21.15.136:3300/0 conn(0x7fd2f015db10 0x7fd2f0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900166914 0 0) 0x7fd2f015a160 con 0x7fd2f0159890 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d8003570 con 0x7fd2f0159890 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e7fff700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 0x7fd2f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565605466 0 0) 0x7fd2d8003570 con 0x7fd2f0159890 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 msgr2=0x7fd2f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 0x7fd2f015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 >> v2:172.21.15.136:3300/0 conn(0x7fd2f015db10 msgr2=0x7fd2f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 --2- 172.21.15.92:0/2139837241 >> v2:172.21.15.136:3300/0 conn(0x7fd2f015db10 0x7fd2f0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f015a3d0 con 0x7fd2f0159890 2023-10-06T03:08:54.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd2ec0017f0 con 0x7fd2f0159890 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 >> v1:172.21.15.92:6789/0 conn(0x7fd2f0159890 legacy=0x7fd2f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2d8003a60 0x7fd2d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 0x7fd2d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8004d50 0x7fd2d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00d13a0 con 0x7fd2d8003a60 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015e140 con 0x7fd2d8004d50 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d800ba60 con 0x7fd2f0159c60 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2d8003a60 0x7fd2d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2e7fff700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 0x7fd2d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f56af700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8004d50 0x7fd2d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f4eae700 1 -- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 msgr2=0x7fd2d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:54.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f0159c60 0x7fd2d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f4eae700 1 -- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8004d50 msgr2=0x7fd2d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8004d50 0x7fd2d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f4eae700 1 -- 172.21.15.92:0/2139837241 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8003570 con 0x7fd2d8003a60 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.676+0000 7fd2f56af700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8004d50 0x7fd2d80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2d8003a60 0x7fd2d8003e70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00a970 tx=0x7fd2ec002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc11c4a4b4c6b8ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec012440 con 0x7fd2d8003a60 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec0125c0 con 0x7fd2d8003a60 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2e77fe700 1 -- 172.21.15.92:0/2139837241 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec012980 con 0x7fd2d8003a60 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f7112700 1 -- 172.21.15.92:0/2139837241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2d8003a60 msgr2=0x7fd2d8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f7112700 1 --2- 172.21.15.92:0/2139837241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2d8003a60 0x7fd2d8003e70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00a970 tx=0x7fd2ec002210 comp rx=0 tx=0).stop 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f56af700 1 -- 172.21.15.92:0/2139837241 reap_dead start 2023-10-06T03:08:54.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f7112700 1 -- 172.21.15.92:0/2139837241 shutdown_connections 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f7112700 1 -- 172.21.15.92:0/2139837241 >> 172.21.15.92:0/2139837241 conn(0x7fd2f0068e20 msgr2=0x7fd2f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f7112700 1 -- 172.21.15.92:0/2139837241 shutdown_connections 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.677+0000 7fd2f7112700 1 -- 172.21.15.92:0/2139837241 wait complete. 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.678+0000 7fd2f7112700 1 Processor -- start 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.678+0000 7fd2f7112700 1 -- start start 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f7112700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8003a60 0x7fd2f01472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f7112700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f00b5a60 0x7fd2f00b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f7112700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 0x7fd2f00c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f7112700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0147830 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f7112700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00b6410 con 0x7fd2d8003a60 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f7112700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00b3be0 con 0x7fd2f00b5a60 2023-10-06T03:08:54.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f56af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 0x7fd2f00c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f56af700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 0x7fd2f00c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56402/0 (socket says 172.21.15.92:56402) 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f56af700 1 -- 172.21.15.92:0/1425757349 learned_addr learned my addr 172.21.15.92:0/1425757349 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2e7fff700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f00b5a60 0x7fd2f00b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.679+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8003a60 0x7fd2f01472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f56af700 1 -- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f00b5a60 msgr2=0x7fd2f00b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f56af700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f00b5a60 0x7fd2f00b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f56af700 1 -- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8003a60 msgr2=0x7fd2f01472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f56af700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8003a60 0x7fd2f01472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f56af700 1 -- 172.21.15.92:0/1425757349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00c8820 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f56af700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 0x7fd2f00c7f40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00d03b0 tx=0x7fd2e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8015670 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e80157d0 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f00c8b00 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8015b90 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.680+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f00b3870 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.681+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f00b3d90 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.688 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.681+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd2e8020460 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.682+0000 7fd2e4ff9700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2dc067320 0x7fd2dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:54.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.682+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2dc067320 0x7fd2dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:54.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.682+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd2e8084af0 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.682+0000 7fd2f4eae700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2dc067320 0x7fd2dc0697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00afd0 tx=0x7fd2ec015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:54.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.686+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2f00b3d90 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.871+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2f00b3d50 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.874+0000 7fd2e4ff9700 1 -- 172.21.15.92:0/1425757349 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2e8051870 con 0x7fd2f00c7ad0 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2dc067320 msgr2=0x7fd2dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2dc067320 0x7fd2dc0697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00afd0 tx=0x7fd2ec015000 comp rx=0 tx=0).stop 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 msgr2=0x7fd2f00c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 0x7fd2f00c7f40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00d03b0 tx=0x7fd2e8008040 comp rx=0 tx=0).stop 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 shutdown_connections 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd2dc067320 0x7fd2dc0697d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd2f00c7ad0 0x7fd2f00c7f40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2d8003a60 0x7fd2f01472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 --2- 172.21.15.92:0/1425757349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2f00b5a60 0x7fd2f00b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:54.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 >> 172.21.15.92:0/1425757349 conn(0x7fd2f0068e20 msgr2=0x7fd2f00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:54.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 shutdown_connections 2023-10-06T03:08:54.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:54.877+0000 7fd2f7112700 1 -- 172.21.15.92:0/1425757349 wait complete. 2023-10-06T03:08:54.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-06T03:08:55.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.003+0000 7fdc22705700 1 Processor -- start 2023-10-06T03:08:55.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.003+0000 7fdc22705700 1 -- start start 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.004+0000 7fdc22705700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdc1c159420 0x7fdc1c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.004+0000 7fdc22705700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 0x7fdc1c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.004+0000 7fdc22705700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0b11d0 con 0x7fdc1c15d230 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.004+0000 7fdc22705700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b14b0 con 0x7fdc1c159420 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.004+0000 7fdc22705700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b1790 con 0x7fdc1c1597f0 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.004+0000 7fdc20f02700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 0x7fdc1c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc21703700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdc1c159420 0x7fdc1c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 0x7fdc1c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:44014/0 (socket says 172.21.15.92:44014) 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 -- 172.21.15.92:0/220389362 learned_addr learned my addr 172.21.15.92:0/220389362 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 -- 172.21.15.92:0/220389362 >> v2:172.21.15.136:3300/0 conn(0x7fdc1c159420 msgr2=0x7fdc1c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:55.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 --2- 172.21.15.92:0/220389362 >> v2:172.21.15.136:3300/0 conn(0x7fdc1c159420 0x7fdc1c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 -- 172.21.15.92:0/220389362 >> v1:172.21.15.136:6789/0 conn(0x7fdc1c15d230 legacy=0x7fdc1c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 -- 172.21.15.92:0/220389362 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0b1a70 con 0x7fdc1c1597f0 2023-10-06T03:08:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc20f02700 1 --2- 172.21.15.92:0/220389362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 0x7fdc1c0b2270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0099b0 tx=0x7fdc0c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdfeead5dddf3a88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc13fff700 1 -- 172.21.15.92:0/220389362 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c009ea0 con 0x7fdc1c1597f0 2023-10-06T03:08:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc13fff700 1 -- 172.21.15.92:0/220389362 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c00b860 con 0x7fdc1c1597f0 2023-10-06T03:08:55.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.005+0000 7fdc13fff700 1 -- 172.21.15.92:0/220389362 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00bc00 con 0x7fdc1c1597f0 2023-10-06T03:08:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 -- 172.21.15.92:0/220389362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 msgr2=0x7fdc1c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 --2- 172.21.15.92:0/220389362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 0x7fdc1c0b2270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0099b0 tx=0x7fdc0c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 -- 172.21.15.92:0/220389362 shutdown_connections 2023-10-06T03:08:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 --2- 172.21.15.92:0/220389362 >> v2:172.21.15.136:3300/0 conn(0x7fdc1c159420 0x7fdc1c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 --2- 172.21.15.92:0/220389362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c1597f0 0x7fdc1c0b2270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 -- 172.21.15.92:0/220389362 >> 172.21.15.92:0/220389362 conn(0x7fdc1c0d0660 msgr2=0x7fdc1c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 -- 172.21.15.92:0/220389362 shutdown_connections 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.006+0000 7fdc22705700 1 -- 172.21.15.92:0/220389362 wait complete. 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.007+0000 7fdc22705700 1 Processor -- start 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.007+0000 7fdc22705700 1 -- start start 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc22705700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 0x7fdc1c14d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc22705700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1c1597f0 0x7fdc1c16cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc22705700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c15d230 0x7fdc1c16d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc22705700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c14dac0 con 0x7fdc1c159420 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc22705700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c14dda0 con 0x7fdc1c1597f0 2023-10-06T03:08:55.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc22705700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c14e080 con 0x7fdc1c15d230 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 0x7fdc1c14d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 0x7fdc1c14d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56418/0 (socket says 172.21.15.92:56418) 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 -- 172.21.15.92:0/2419704024 learned_addr learned my addr 172.21.15.92:0/2419704024 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21f04700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c15d230 0x7fdc1c16d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc20f02700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1c1597f0 0x7fdc1c16cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 -- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c15d230 msgr2=0x7fdc1c16d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c15d230 0x7fdc1c16d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 -- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1c1597f0 msgr2=0x7fdc1c16cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1c1597f0 0x7fdc1c16cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 -- 172.21.15.92:0/2419704024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0094f0 con 0x7fdc1c159420 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21f04700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c15d230 0x7fdc1c16d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc21703700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 0x7fdc1c14d020 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ec80 tx=0x7fdc180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:55.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.008+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800c850 con 0x7fdc1c159420 2023-10-06T03:08:55.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.009+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1800c9b0 con 0x7fdc1c159420 2023-10-06T03:08:55.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.009+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c14e360 con 0x7fdc1c159420 2023-10-06T03:08:55.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.009+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800cd50 con 0x7fdc1c159420 2023-10-06T03:08:55.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.009+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c14d560 con 0x7fdc1c159420 2023-10-06T03:08:55.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.010+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c14e9b0 con 0x7fdc1c159420 2023-10-06T03:08:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.010+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdc18013680 con 0x7fdc1c159420 2023-10-06T03:08:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.010+0000 7fdc117fa700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc00067320 0x7fdc000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.011+0000 7fdc20f02700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc00067320 0x7fdc000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.011+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fdc18084d70 con 0x7fdc1c159420 2023-10-06T03:08:55.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.011+0000 7fdc20f02700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc00067320 0x7fdc000697d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0099b0 tx=0x7fdc0c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:55.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.015+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc18051af0 con 0x7fdc1c159420 2023-10-06T03:08:55.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.271+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7fdc1c04c3c0 con 0x7fdc1c159420 2023-10-06T03:08:55.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.538+0000 7fdc117fa700 1 -- 172.21.15.92:0/2419704024 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v361) v1 ==== 131+0+0 (secure 0 0 0) 0x7fdc18055110 con 0x7fdc1c159420 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc00067320 msgr2=0x7fdc000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc00067320 0x7fdc000697d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0099b0 tx=0x7fdc0c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 msgr2=0x7fdc1c14d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 0x7fdc1c14d020 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ec80 tx=0x7fdc180091a0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 shutdown_connections 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc00067320 0x7fdc000697d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc1c159420 0x7fdc1c14d020 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc1c15d230 0x7fdc1c16d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 --2- 172.21.15.92:0/2419704024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc1c1597f0 0x7fdc1c16cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 >> 172.21.15.92:0/2419704024 conn(0x7fdc1c0d0660 msgr2=0x7fdc1c0b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:55.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 shutdown_connections 2023-10-06T03:08:55.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.541+0000 7fdc22705700 1 -- 172.21.15.92:0/2419704024 wait complete. 2023-10-06T03:08:55.555 INFO:tasks.cephfs_test_runner:That the existing and the newly created volumes can be listed and ... ok 2023-10-06T03:08:55.555 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) That volume, its file system and pools, can be renamed. 2023-10-06T03:08:55.555 INFO:tasks.cephfs_test_runner:test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:08:55.555 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-06T03:08:55.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.669+0000 7faba0b2b700 1 Processor -- start 2023-10-06T03:08:55.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.669+0000 7faba0b2b700 1 -- start start 2023-10-06T03:08:55.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.669+0000 7faba0b2b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fab9c140350 0x7fab9c069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.669+0000 7faba0b2b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0c7de0 con 0x7fab9c068aa0 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.669+0000 7faba0b2b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c140c10 con 0x7fab9c06a1c0 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.669+0000 7faba0b2b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c141490 con 0x7fab9c140350 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab9ad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fab9c068aa0 0x7fab9c0c8270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53010/0 (socket says 172.21.15.92:53010) 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab9ad9d700 1 -- 172.21.15.92:0/2257347831 learned_addr learned my addr 172.21.15.92:0/2257347831 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322213060 0 0) 0x7fab9c0c7de0 con 0x7fab9c068aa0 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab9a59c700 1 --2- 172.21.15.92:0/2257347831 >> v2:172.21.15.136:3300/0 conn(0x7fab9c140350 0x7fab9c069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab88003570 con 0x7fab9c068aa0 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252595711 0 0) 0x7fab88003570 con 0x7fab9c068aa0 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 >> v2:172.21.15.136:3300/0 conn(0x7fab9c140350 msgr2=0x7fab9c069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:55.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 --2- 172.21.15.92:0/2257347831 >> v2:172.21.15.136:3300/0 conn(0x7fab9c140350 0x7fab9c069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 >> v1:172.21.15.136:6789/0 conn(0x7fab9c06a1c0 legacy=0x7fab9c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c140ed0 con 0x7fab9c068aa0 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fab940017f0 con 0x7fab9c068aa0 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 >> v1:172.21.15.92:6789/0 conn(0x7fab9c068aa0 legacy=0x7fab9c0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003a60 0x7fab88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88004810 0x7fab88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab880051c0 0x7fab8800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c141490 con 0x7fab88004810 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800c0d0 con 0x7fab88003a60 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800c290 con 0x7fab880051c0 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab91d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88004810 0x7fab88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.670+0000 7fab9a59c700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003a60 0x7fab88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab9ad9d700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab880051c0 0x7fab8800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab91d9b700 1 -- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab880051c0 msgr2=0x7fab8800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab91d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab880051c0 0x7fab8800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab91d9b700 1 -- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003a60 msgr2=0x7fab88003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab91d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003a60 0x7fab88003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab91d9b700 1 -- 172.21.15.92:0/2257347831 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88003570 con 0x7fab88004810 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab9ad9d700 1 -- 172.21.15.92:0/2257347831 reap_dead start 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab91d9b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88004810 0x7fab88004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fab840018b0 tx=0x7fab84007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c20d61eaf2c76a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84012420 con 0x7fab88004810 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab840125a0 con 0x7fab88004810 2023-10-06T03:08:55.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7fab99d9b700 1 -- 172.21.15.92:0/2257347831 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84012960 con 0x7fab88004810 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7faba0b2b700 1 -- 172.21.15.92:0/2257347831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88004810 msgr2=0x7fab88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7faba0b2b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88004810 0x7fab88004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fab840018b0 tx=0x7fab84007d30 comp rx=0 tx=0).stop 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7faba0b2b700 1 -- 172.21.15.92:0/2257347831 shutdown_connections 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7faba0b2b700 1 --2- 172.21.15.92:0/2257347831 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88004810 0x7fab88004c80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7faba0b2b700 1 -- 172.21.15.92:0/2257347831 >> 172.21.15.92:0/2257347831 conn(0x7fab9c0cd670 msgr2=0x7fab9c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.671+0000 7faba0b2b700 1 -- 172.21.15.92:0/2257347831 shutdown_connections 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.672+0000 7faba0b2b700 1 -- 172.21.15.92:0/2257347831 wait complete. 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 Processor -- start 2023-10-06T03:08:55.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 -- start start 2023-10-06T03:08:55.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004810 0x7fab9c0ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c13f0a0 0x7fab9c13f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 0x7fab9c16a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0cad90 con 0x7fab9c169c50 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c13fa50 con 0x7fab88004810 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.673+0000 7faba0b2b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c16d0e0 con 0x7fab9c13f0a0 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 0x7fab9c16a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 0x7fab9c16a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56446/0 (socket says 172.21.15.92:56446) 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 -- 172.21.15.92:0/3372708772 learned_addr learned my addr 172.21.15.92:0/3372708772 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9a59c700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004810 0x7fab9c0ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab91d9b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c13f0a0 0x7fab9c13f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 -- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c13f0a0 msgr2=0x7fab9c13f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c13f0a0 0x7fab9c13f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 -- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004810 msgr2=0x7fab9c0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004810 0x7fab9c0ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 -- 172.21.15.92:0/3372708772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c16d3a0 con 0x7fab9c169c50 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab9ad9d700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 0x7fab9c16a0c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fab9c06d8d0 tx=0x7fab94001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94017070 con 0x7fab9c169c50 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9400eb80 con 0x7fab9c169c50 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c16d680 con 0x7fab9c169c50 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94013420 con 0x7fab9c169c50 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.674+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c16dc40 con 0x7fab9c169c50 2023-10-06T03:08:55.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.675+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c04c3c0 con 0x7fab9c169c50 2023-10-06T03:08:55.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.675+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fab940137d0 con 0x7fab9c169c50 2023-10-06T03:08:55.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.676+0000 7fab92ffd700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab7c067320 0x7fab7c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:55.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.676+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fab940849a0 con 0x7fab9c169c50 2023-10-06T03:08:55.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.676+0000 7fab9a59c700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab7c067320 0x7fab7c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:55.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.676+0000 7fab9a59c700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab7c067320 0x7fab7c0697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab9c061160 tx=0x7fab8c009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:55.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.680+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab94051720 con 0x7fab9c169c50 2023-10-06T03:08:55.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:55.937+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7fab9c1411b0 con 0x7fab9c169c50 2023-10-06T03:08:56.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.539+0000 7fab92ffd700 1 -- 172.21.15.92:0/3372708772 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v362) v1 ==== 138+0+0 (secure 0 0 0) 0x7fab94054d40 con 0x7fab9c169c50 2023-10-06T03:08:56.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab7c067320 msgr2=0x7fab7c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab7c067320 0x7fab7c0697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab9c061160 tx=0x7fab8c009120 comp rx=0 tx=0).stop 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 msgr2=0x7fab9c16a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 0x7fab9c16a0c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fab9c06d8d0 tx=0x7fab94001300 comp rx=0 tx=0).stop 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 shutdown_connections 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab7c067320 0x7fab7c0697d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c169c50 0x7fab9c16a0c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004810 0x7fab9c0ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 --2- 172.21.15.92:0/3372708772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c13f0a0 0x7fab9c13f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 >> 172.21.15.92:0/3372708772 conn(0x7fab9c0cd670 msgr2=0x7fab9c13be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 shutdown_connections 2023-10-06T03:08:56.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.542+0000 7faba0b2b700 1 -- 172.21.15.92:0/3372708772 wait complete. 2023-10-06T03:08:56.556 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:08:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.672+0000 7f1ec21a5700 1 Processor -- start 2023-10-06T03:08:56.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.672+0000 7f1ec21a5700 1 -- start start 2023-10-06T03:08:56.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ec21a5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ec21a5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ebc159c60 0x7f1ebc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ec21a5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ebc15db10 0x7f1ebc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ec21a5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc15a160 con 0x7f1ebc159c60 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ec21a5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0d13a0 con 0x7f1ebc15db10 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ec21a5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc15e140 con 0x7f1ebc159890 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ebc159c60 0x7f1ebc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ebc159c60 0x7f1ebc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56448/0 (socket says 172.21.15.92:56448) 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 -- 172.21.15.92:0/953625542 learned_addr learned my addr 172.21.15.92:0/953625542 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 -- 172.21.15.92:0/953625542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 msgr2=0x7f1ebc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 --2- 172.21.15.92:0/953625542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 -- 172.21.15.92:0/953625542 >> v2:172.21.15.136:3300/0 conn(0x7f1ebc15db10 msgr2=0x7f1ebc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 --2- 172.21.15.92:0/953625542 >> v2:172.21.15.136:3300/0 conn(0x7f1ebc15db10 0x7f1ebc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.673+0000 7f1ebaffd700 1 -- 172.21.15.92:0/953625542 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc15a3d0 con 0x7f1ebc159c60 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ebaffd700 1 --2- 172.21.15.92:0/953625542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc159c60 0x7f1ebc15d5d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc089b50 tx=0x7f1eb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4486a9ffb7ec2f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1eba7fc700 1 -- 172.21.15.92:0/953625542 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000b860 con 0x7f1ebc159c60 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1eba7fc700 1 -- 172.21.15.92:0/953625542 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb000b9c0 con 0x7f1ebc159c60 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1eba7fc700 1 -- 172.21.15.92:0/953625542 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb000bd60 con 0x7f1ebc159c60 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 -- 172.21.15.92:0/953625542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc159c60 msgr2=0x7f1ebc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/953625542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc159c60 0x7f1ebc15d5d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc089b50 tx=0x7f1eb0009250 comp rx=0 tx=0).stop 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 -- 172.21.15.92:0/953625542 shutdown_connections 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/953625542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/953625542 >> v2:172.21.15.136:3300/0 conn(0x7f1ebc15db10 0x7f1ebc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/953625542 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc159c60 0x7f1ebc15d5d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 -- 172.21.15.92:0/953625542 >> 172.21.15.92:0/953625542 conn(0x7f1ebc068e20 msgr2=0x7f1ebc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:56.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 -- 172.21.15.92:0/953625542 shutdown_connections 2023-10-06T03:08:56.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.674+0000 7f1ec21a5700 1 -- 172.21.15.92:0/953625542 wait complete. 2023-10-06T03:08:56.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 Processor -- start 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 -- start start 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ebc159c60 0x7f1ebc16a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 0x7f1ebc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0b6140 con 0x7f1ebc15db10 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cf2a0 con 0x7f1ebc159c60 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ec21a5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cfcb0 con 0x7f1ebc159890 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.676+0000 7f1ebbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 0x7f1ebc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 0x7f1ebc0cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56460/0 (socket says 172.21.15.92:56460) 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 -- 172.21.15.92:0/869972578 learned_addr learned my addr 172.21.15.92:0/869972578 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:56.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebb7fe700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebaffd700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ebc159c60 0x7f1ebc16a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 -- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 msgr2=0x7f1ebc0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 -- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ebc159c60 msgr2=0x7f1ebc16a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ebc159c60 0x7f1ebc16a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 -- 172.21.15.92:0/869972578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00094f0 con 0x7f1ebc15db10 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ebbfff700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 0x7f1ebc0cb180 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b510 tx=0x7f1eb40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400f510 con 0x7f1ebc15db10 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb400f690 con 0x7f1ebc15db10 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400fa50 con 0x7f1ebc15db10 2023-10-06T03:08:56.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc16ab50 con 0x7f1ebc15db10 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.677+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc0b5810 con 0x7f1ebc15db10 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.678+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc0b62f0 con 0x7f1ebc15db10 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.679+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1eb401a400 con 0x7f1ebc15db10 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.679+0000 7f1ea37fe700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ea8067320 0x7f1ea80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.679+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1eb4084430 con 0x7f1ebc15db10 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.679+0000 7f1ebb7fe700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ea8067320 0x7f1ea80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.679+0000 7f1ebb7fe700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ea8067320 0x7f1ea80697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1eac007580 tx=0x7f1eac009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:56.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.683+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1eb4051230 con 0x7f1ebc15db10 2023-10-06T03:08:56.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.867+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ebc04c3c0 con 0x7f1ebc15db10 2023-10-06T03:08:56.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.872+0000 7f1ea37fe700 1 -- 172.21.15.92:0/869972578 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1eb4082e60 con 0x7f1ebc15db10 2023-10-06T03:08:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ea8067320 msgr2=0x7f1ea80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ea8067320 0x7f1ea80697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1eac007580 tx=0x7f1eac009780 comp rx=0 tx=0).stop 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 msgr2=0x7f1ebc0cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 0x7f1ebc0cb180 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b510 tx=0x7f1eb40090f0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 shutdown_connections 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ea8067320 0x7f1ea80697d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.874+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc15db10 0x7f1ebc0cb180 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.875+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ebc159890 0x7f1ebc0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.875+0000 7f1ec21a5700 1 --2- 172.21.15.92:0/869972578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ebc159c60 0x7f1ebc16a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.875+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 >> 172.21.15.92:0/869972578 conn(0x7f1ebc068e20 msgr2=0x7f1ebc0d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.875+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 shutdown_connections 2023-10-06T03:08:56.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:56.875+0000 7f1ec21a5700 1 -- 172.21.15.92:0/869972578 wait complete. 2023-10-06T03:08:56.888 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:08:56.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:56.928 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:08:56.969 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:08:56.969 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:08:56.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:08:57.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.082+0000 7f86f576b700 1 Processor -- start 2023-10-06T03:08:57.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.082+0000 7f86f576b700 1 -- start start 2023-10-06T03:08:57.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86f576b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f86f015d230 0x7f86f015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86f576b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f00b2980 con 0x7f86f0159420 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86f576b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f015db70 con 0x7f86f01597f0 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86f576b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f00b11d0 con 0x7f86f015d230 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86effff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f86f0159420 0x7f86f00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53020/0 (socket says 172.21.15.92:53020) 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86effff700 1 -- 172.21.15.92:0/2615122637 learned_addr learned my addr 172.21.15.92:0/2615122637 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86f4f6a700 1 --2- 172.21.15.92:0/2615122637 >> v2:172.21.15.136:3300/0 conn(0x7f86f015d230 0x7f86f015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431771981 0 0) 0x7f86f00b2980 con 0x7f86f0159420 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d8003570 con 0x7f86f0159420 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273822738 0 0) 0x7f86d8003570 con 0x7f86f0159420 2023-10-06T03:08:57.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 >> v2:172.21.15.136:3300/0 conn(0x7f86f015d230 msgr2=0x7f86f015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 --2- 172.21.15.92:0/2615122637 >> v2:172.21.15.136:3300/0 conn(0x7f86f015d230 0x7f86f015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 >> v1:172.21.15.136:6789/0 conn(0x7f86f01597f0 legacy=0x7f86f00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.083+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00b14b0 con 0x7f86f0159420 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f86e0001470 con 0x7f86f0159420 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 >> v1:172.21.15.92:6789/0 conn(0x7f86f0159420 legacy=0x7f86f00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d8003a60 0x7f86d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86d8004810 0x7f86d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00b11d0 con 0x7f86d80051c0 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d800c0d0 con 0x7f86d8003a60 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d800c290 con 0x7f86d8004810 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86effff700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d8003a60 0x7f86d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86ef7fe700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86d8004810 0x7f86d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 -- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86d8004810 msgr2=0x7f86d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86d8004810 0x7f86d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 -- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d8003a60 msgr2=0x7f86d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d8003a60 0x7f86d8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 -- 172.21.15.92:0/2615122637 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d8003570 con 0x7f86d80051c0 2023-10-06T03:08:57.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 -- 172.21.15.92:0/2615122637 reap_dead start 2023-10-06T03:08:57.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86f4f6a700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86d800ba60 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e210 tx=0x7f86e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3309dd034cf3652a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800eac0 con 0x7f86d80051c0 2023-10-06T03:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e800f040 con 0x7f86d80051c0 2023-10-06T03:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.084+0000 7f86eeffd700 1 -- 172.21.15.92:0/2615122637 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800ee70 con 0x7f86d80051c0 2023-10-06T03:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 -- 172.21.15.92:0/2615122637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 msgr2=0x7f86d800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86d800ba60 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e210 tx=0x7f86e800b040 comp rx=0 tx=0).stop 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 -- 172.21.15.92:0/2615122637 shutdown_connections 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 --2- 172.21.15.92:0/2615122637 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86d800ba60 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 -- 172.21.15.92:0/2615122637 >> 172.21.15.92:0/2615122637 conn(0x7f86f00d0660 msgr2=0x7f86f00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 -- 172.21.15.92:0/2615122637 shutdown_connections 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.085+0000 7f86f576b700 1 -- 172.21.15.92:0/2615122637 wait complete. 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.086+0000 7f86f576b700 1 Processor -- start 2023-10-06T03:08:57.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.086+0000 7f86f576b700 1 -- start start 2023-10-06T03:08:57.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86f00c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86f0156030 0x7f86f01564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f0169f90 0x7f86f016a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00c7400 con 0x7f86d80051c0 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00c6780 con 0x7f86f0169f90 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00c6a60 con 0x7f86f0156030 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86f00c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86f00c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56486/0 (socket says 172.21.15.92:56486) 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 -- 172.21.15.92:0/2954572619 learned_addr learned my addr 172.21.15.92:0/2954572619 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:57.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 -- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86f0156030 msgr2=0x7f86f01564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86ef7fe700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86f0156030 0x7f86f01564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86f0156030 0x7f86f01564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 -- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f0169f90 msgr2=0x7f86f016a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f0169f90 0x7f86f016a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 -- 172.21.15.92:0/2954572619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f00c6d40 con 0x7f86d80051c0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86effff700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86f00c5e10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f86e0007390 tx=0x7f86f015dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0018030 con 0x7f86d80051c0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e000f8d0 con 0x7f86d80051c0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f00c7020 con 0x7f86d80051c0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e000fc70 con 0x7f86d80051c0 2023-10-06T03:08:57.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.087+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f00c6510 con 0x7f86d80051c0 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.088+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f004c3c0 con 0x7f86d80051c0 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.089+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f86e001b070 con 0x7f86d80051c0 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.089+0000 7f86d7fff700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f86dc067320 0x7f86dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.089+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f86e0019070 con 0x7f86d80051c0 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.089+0000 7f86ef7fe700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f86dc067320 0x7f86dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.090+0000 7f86ef7fe700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f86dc067320 0x7f86dc0697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f86f01569e0 tx=0x7f86e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:57.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.093+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86e0051900 con 0x7f86d80051c0 2023-10-06T03:08:57.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.319+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86f00b1790 con 0x7f86d80051c0 2023-10-06T03:08:57.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.320+0000 7f86d7fff700 1 -- 172.21.15.92:0/2954572619 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f86e0054f20 con 0x7f86d80051c0 2023-10-06T03:08:57.322 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:57.322 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":88,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5954,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/2278434588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":2278434588},{"type":"v1","addr":"172.21.15.136:6834","nonce":2278434588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6817,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1888979047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":1888979047},{"type":"v1","addr":"172.21.15.136:6835","nonce":1888979047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7464,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/628951753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":628951753},{"type":"v1","addr":"172.21.15.92:6837","nonce":628951753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84},{"gid":7560,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2176219452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2176219452},{"type":"v1","addr":"172.21.15.92:6836","nonce":2176219452}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[]} 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f86dc067320 msgr2=0x7f86dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f86dc067320 0x7f86dc0697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f86f01569e0 tx=0x7f86e4009040 comp rx=0 tx=0).stop 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 msgr2=0x7f86f00c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86f00c5e10 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f86e0007390 tx=0x7f86f015dd00 comp rx=0 tx=0).stop 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 shutdown_connections 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f86dc067320 0x7f86dc0697d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f86d80051c0 0x7f86f00c5e10 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86f0169f90 0x7f86f016a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 --2- 172.21.15.92:0/2954572619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86f0156030 0x7f86f01564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 >> 172.21.15.92:0/2954572619 conn(0x7f86f00d0660 msgr2=0x7f86f01570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:57.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 shutdown_connections 2023-10-06T03:08:57.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.322+0000 7f86f576b700 1 -- 172.21.15.92:0/2954572619 wait complete. 2023-10-06T03:08:57.325 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 88 2023-10-06T03:08:57.336 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:08:57.336 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:08:57.336 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:08:57.337 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:57.337 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:08:57.337 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:08:57.337 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:08:57.337 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:57.337 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:08:57.338 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:08:57.338 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:08:57.338 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:57.338 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:08:57.338 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:08:57.339 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:08:57.339 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:08:57.345 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:08:57.345 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:08:57.346 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:08:57.346 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:08:57.348 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:08:57.349 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:08:57.349 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:08:57.350 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:08:57.388 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:08:57.388 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:57.389 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:08:57.389 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:57.391 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:08:57.391 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:57.392 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:08:57.392 DEBUG:teuthology.parallel:result is None 2023-10-06T03:08:57.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:08:57.460 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:08:57.461 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:08:57.473 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:08:57.474 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:08:57.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.510+0000 7fefd2341700 1 Processor -- start 2023-10-06T03:08:57.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.510+0000 7fefd2341700 1 -- start start 2023-10-06T03:08:57.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.511+0000 7fefd2341700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fefcc069560 0x7fefcc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.511+0000 7fefd2341700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc069ee0 con 0x7fefcc159c60 2023-10-06T03:08:57.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.511+0000 7fefd2341700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc15a160 con 0x7fefcc159890 2023-10-06T03:08:57.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.511+0000 7fefd2341700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc0d13a0 con 0x7fefcc069560 2023-10-06T03:08:57.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.511+0000 7fefcb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fefcc159c60 0x7fefcc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53042/0 (socket says 172.21.15.92:53042) 2023-10-06T03:08:57.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.511+0000 7fefcb7fe700 1 -- 172.21.15.92:0/1818096152 learned_addr learned my addr 172.21.15.92:0/1818096152 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefd08de700 1 --2- 172.21.15.92:0/1818096152 >> v2:172.21.15.136:3300/0 conn(0x7fefcc069560 0x7fefcc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71505174 0 0) 0x7fefcc069ee0 con 0x7fefcc159c60 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefb4003570 con 0x7fefcc159c60 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1825493945 0 0) 0x7fefb4003570 con 0x7fefcc159c60 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 >> v2:172.21.15.136:3300/0 conn(0x7fefcc069560 msgr2=0x7fefcc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 --2- 172.21.15.92:0/1818096152 >> v2:172.21.15.136:3300/0 conn(0x7fefcc069560 0x7fefcc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 >> v1:172.21.15.136:6789/0 conn(0x7fefcc159890 legacy=0x7fefcc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc15e140 con 0x7fefcc159c60 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fefc0001470 con 0x7fefcc159c60 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 >> v1:172.21.15.92:6789/0 conn(0x7fefcc159c60 legacy=0x7fefcc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb4003a60 0x7fefb4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefb4004810 0x7fefb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb40051c0 0x7fefb400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0d13a0 con 0x7fefb4003a60 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb400c0d0 con 0x7fefb4004810 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb400c290 con 0x7fefb40051c0 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb4003a60 0x7fefb4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcb7fe700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefb4004810 0x7fefb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefd08de700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb40051c0 0x7fefb400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 -- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb40051c0 msgr2=0x7fefb400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb40051c0 0x7fefb400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 -- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefb4004810 msgr2=0x7fefb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefb4004810 0x7fefb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefd08de700 1 -- 172.21.15.92:0/1818096152 reap_dead start 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 -- 172.21.15.92:0/1818096152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4003570 con 0x7fefb4003a60 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.512+0000 7fefcbfff700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb4003a60 0x7fefb4003e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0018b0 tx=0x7fefbc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964568b4f4a74901 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc012420 con 0x7fefb4003a60 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefbc0125a0 con 0x7fefb4003a60 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefcaffd700 1 -- 172.21.15.92:0/1818096152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc012960 con 0x7fefb4003a60 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 -- 172.21.15.92:0/1818096152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb4003a60 msgr2=0x7fefb4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb4003a60 0x7fefb4003e70 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0018b0 tx=0x7fefbc007d30 comp rx=0 tx=0).stop 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 -- 172.21.15.92:0/1818096152 shutdown_connections 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 --2- 172.21.15.92:0/1818096152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefb4003a60 0x7fefb4003e70 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 -- 172.21.15.92:0/1818096152 >> 172.21.15.92:0/1818096152 conn(0x7fefcc068e20 msgr2=0x7fefcc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:57.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 -- 172.21.15.92:0/1818096152 shutdown_connections 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.513+0000 7fefd2341700 1 -- 172.21.15.92:0/1818096152 wait complete. 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.514+0000 7fefd2341700 1 Processor -- start 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.514+0000 7fefd2341700 1 -- start start 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd2341700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb4003a60 0x7fefcc14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd2341700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 0x7fefcc0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd2341700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefcc0b5e90 0x7fefcc151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd2341700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc152c10 con 0x7fefcc155cc0 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd2341700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0ce2a0 con 0x7fefcc0b5e90 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd2341700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc14cab0 con 0x7fefb4003a60 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 0x7fefcc0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 0x7fefcc0b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56546/0 (socket says 172.21.15.92:56546) 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 -- 172.21.15.92:0/911119019 learned_addr learned my addr 172.21.15.92:0/911119019 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:57.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcbfff700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb4003a60 0x7fefcc14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefd08de700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefcc0b5e90 0x7fefcc151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 -- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb4003a60 msgr2=0x7fefcc14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb4003a60 0x7fefcc14c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 -- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefcc0b5e90 msgr2=0x7fefcc151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.515+0000 7fefcb7fe700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefcc0b5e90 0x7fefcc151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefcb7fe700 1 -- 172.21.15.92:0/911119019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc152930 con 0x7fefcc155cc0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefd08de700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefcc0b5e90 0x7fefcc151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefcb7fe700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 0x7fefcc0b5950 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fefc00073c0 tx=0x7fefc0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000f8d0 con 0x7fefcc155cc0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc000fa50 con 0x7fefcc155cc0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc0cdc50 con 0x7fefcc155cc0 2023-10-06T03:08:57.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000fe10 con 0x7fefcc155cc0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.516+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc0b6330 con 0x7fefcc155cc0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.517+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc04c3c0 con 0x7fefcc155cc0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.517+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fefc00135c0 con 0x7fefcc155cc0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.518+0000 7fefb3fff700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefb8067320 0x7fefb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.518+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fefc0084360 con 0x7fefcc155cc0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.518+0000 7fefcbfff700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefb8067320 0x7fefb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.518+0000 7fefcbfff700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefb8067320 0x7fefb80697d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefbc001690 tx=0x7fefbc015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:57.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.522+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefc00511f0 con 0x7fefcc155cc0 2023-10-06T03:08:57.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:57.723+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fefcc0ce3e0 con 0x7fefcc155cc0 2023-10-06T03:08:58.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.560+0000 7fefb3fff700 1 -- 172.21.15.92:0/911119019 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7fefc0054810 con 0x7fefcc155cc0 2023-10-06T03:08:58.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefb8067320 msgr2=0x7fefb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefb8067320 0x7fefb80697d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fefbc001690 tx=0x7fefbc015000 comp rx=0 tx=0).stop 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 msgr2=0x7fefcc0b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 0x7fefcc0b5950 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fefc00073c0 tx=0x7fefc0001e50 comp rx=0 tx=0).stop 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 shutdown_connections 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fefb8067320 0x7fefb80697d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fefcc0b5e90 0x7fefcc151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fefb4003a60 0x7fefcc14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 --2- 172.21.15.92:0/911119019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefcc155cc0 0x7fefcc0b5950 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 >> 172.21.15.92:0/911119019 conn(0x7fefcc068e20 msgr2=0x7fefcc15ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:58.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 shutdown_connections 2023-10-06T03:08:58.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.563+0000 7fefd2341700 1 -- 172.21.15.92:0/911119019 wait complete. 2023-10-06T03:08:58.566 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:08:58.579 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:08:58.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.697+0000 7fe2fea13700 1 Processor -- start 2023-10-06T03:08:58.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.697+0000 7fe2fea13700 1 -- start start 2023-10-06T03:08:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.698+0000 7fe2fea13700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe2f80cd1e0 0x7fe2f80c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.698+0000 7fe2fea13700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f815c2a0 con 0x7fe2f806bcd0 2023-10-06T03:08:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.698+0000 7fe2fea13700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f815b690 con 0x7fe2f8141500 2023-10-06T03:08:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.698+0000 7fe2fea13700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f815b970 con 0x7fe2f80cd1e0 2023-10-06T03:08:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.698+0000 7fe2f7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe2f806bcd0 0x7fe2f8140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:53044/0 (socket says 172.21.15.92:53044) 2023-10-06T03:08:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.698+0000 7fe2f7fff700 1 -- 172.21.15.92:0/227761929 learned_addr learned my addr 172.21.15.92:0/227761929 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/227761929 >> v2:172.21.15.136:3300/0 conn(0x7fe2f80cd1e0 0x7fe2f80c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877327046 0 0) 0x7fe2f815c2a0 con 0x7fe2f806bcd0 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e0003570 con 0x7fe2f806bcd0 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1098634337 0 0) 0x7fe2e0003570 con 0x7fe2f806bcd0 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 >> v2:172.21.15.136:3300/0 conn(0x7fe2f80cd1e0 msgr2=0x7fe2f80c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 --2- 172.21.15.92:0/227761929 >> v2:172.21.15.136:3300/0 conn(0x7fe2f80cd1e0 0x7fe2f80c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 >> v1:172.21.15.136:6789/0 conn(0x7fe2f8141500 legacy=0x7fe2f80c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f815bc50 con 0x7fe2f806bcd0 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fe2e8001470 con 0x7fe2f806bcd0 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/227761929 >> v2:172.21.15.136:3300/0 conn(0x7fe2f80cd1e0 0x7fe2f80c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 >> v1:172.21.15.92:6789/0 conn(0x7fe2f806bcd0 legacy=0x7fe2f8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e0003a60 0x7fe2e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e0004810 0x7fe2e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e00051c0 0x7fe2e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f815b970 con 0x7fe2e00051c0 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e000c0d0 con 0x7fe2e0004810 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e000c290 con 0x7fe2e0003a60 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e00051c0 0x7fe2e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f7fff700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e0003a60 0x7fe2e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2f77fe700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e0004810 0x7fe2e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e0003a60 msgr2=0x7fe2e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e0003a60 0x7fe2e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e0004810 msgr2=0x7fe2e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e0004810 0x7fe2e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.699+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/227761929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0003570 con 0x7fe2e00051c0 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/227761929 reap_dead start 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e00051c0 0x7fe2e000ba60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000e3c0 tx=0x7fe2f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b3cfca93511629 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0009c50 con 0x7fe2e00051c0 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f000f040 con 0x7fe2e00051c0 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2f6ffd700 1 -- 172.21.15.92:0/227761929 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000eec0 con 0x7fe2e00051c0 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 -- 172.21.15.92:0/227761929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e00051c0 msgr2=0x7fe2e000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e00051c0 0x7fe2e000ba60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000e3c0 tx=0x7fe2f000b040 comp rx=0 tx=0).stop 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 -- 172.21.15.92:0/227761929 shutdown_connections 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 --2- 172.21.15.92:0/227761929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2e00051c0 0x7fe2e000ba60 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 -- 172.21.15.92:0/227761929 >> 172.21.15.92:0/227761929 conn(0x7fe2f80ce600 msgr2=0x7fe2f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 -- 172.21.15.92:0/227761929 shutdown_connections 2023-10-06T03:08:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.700+0000 7fe2fea13700 1 -- 172.21.15.92:0/227761929 wait complete. 2023-10-06T03:08:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.701+0000 7fe2fea13700 1 Processor -- start 2023-10-06T03:08:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 -- start start 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e00051c0 0x7fe2f8157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2f816a9f0 0x7fe2f80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 0x7fe2f815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f8157d90 con 0x7fe2f80b3890 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f816a560 con 0x7fe2e00051c0 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fea13700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80b4180 con 0x7fe2f816a9f0 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fcfb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 0x7fe2f815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fcfb0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 0x7fe2f815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56560/0 (socket says 172.21.15.92:56560) 2023-10-06T03:08:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/2030143824 learned_addr learned my addr 172.21.15.92:0/2030143824 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2f7fff700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e00051c0 0x7fe2f8157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.702+0000 7fe2f77fe700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2f816a9f0 0x7fe2f80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2f816a9f0 msgr2=0x7fe2f80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2f816a9f0 0x7fe2f80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e00051c0 msgr2=0x7fe2f8157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e00051c0 0x7fe2f8157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fcfb0700 1 -- 172.21.15.92:0/2030143824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f80b3ea0 con 0x7fe2f80b3890 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fcfb0700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 0x7fe2f815a300 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80ce5d0 tx=0x7fe2f001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f001d660 con 0x7fe2f80b3890 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f001d7e0 con 0x7fe2f80b3890 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f8169f10 con 0x7fe2f80b3890 2023-10-06T03:08:58.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f001dba0 con 0x7fe2f80b3890 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.703+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f816ae60 con 0x7fe2f80b3890 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.704+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f816a6f0 con 0x7fe2f80b3890 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.704+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe2f0003020 con 0x7fe2f80b3890 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.705+0000 7fe2dffff700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2e4067320 0x7fe2e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.705+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe2f0085d30 con 0x7fe2f80b3890 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.705+0000 7fe2f7fff700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2e4067320 0x7fe2e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.705+0000 7fe2f7fff700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2e4067320 0x7fe2e40697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8001ed0 tx=0x7fe2f815c210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:58.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.709+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2f0052bd0 con 0x7fe2f80b3890 2023-10-06T03:08:58.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.965+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2f804c3c0 con 0x7fe2f80b3890 2023-10-06T03:08:58.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.965+0000 7fe2dffff700 1 -- 172.21.15.92:0/2030143824 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe2f0018090 con 0x7fe2f80b3890 2023-10-06T03:08:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:58.967 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:08:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:08:58.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:08:58.968 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:08:58.970 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.973 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:08:58.974 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.975 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:08:58.977 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:08:58.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:08:58.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:08:58.981 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:08:58.982 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:08:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:08:58.983 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:08:58.983 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:08:58.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2e4067320 msgr2=0x7fe2e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2e4067320 0x7fe2e40697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8001ed0 tx=0x7fe2f815c210 comp rx=0 tx=0).stop 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 msgr2=0x7fe2f815a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 0x7fe2f815a300 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80ce5d0 tx=0x7fe2f001a000 comp rx=0 tx=0).stop 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 shutdown_connections 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe2e4067320 0x7fe2e40697d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe2f80b3890 0x7fe2f815a300 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e00051c0 0x7fe2f8157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 --2- 172.21.15.92:0/2030143824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2f816a9f0 0x7fe2f80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 >> 172.21.15.92:0/2030143824 conn(0x7fe2f80ce600 msgr2=0x7fe2f80c3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:58.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 shutdown_connections 2023-10-06T03:08:58.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:58.967+0000 7fe2fea13700 1 -- 172.21.15.92:0/2030143824 wait complete. 2023-10-06T03:08:58.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:08:59.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.099+0000 7f324098a700 1 Processor -- start 2023-10-06T03:08:59.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.099+0000 7f324098a700 1 -- start start 2023-10-06T03:08:59.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f324098a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f323c159890 0x7f323c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f324098a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f323c159c60 0x7f323c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f324098a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c15db10 0x7f323c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f324098a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c15df50 con 0x7f323c159890 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f324098a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c15a160 con 0x7f323c159c60 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f324098a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0d13a0 con 0x7f323c15db10 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f323c159890 0x7f323c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f323c159890 0x7f323c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56570/0 (socket says 172.21.15.92:56570) 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 -- 172.21.15.92:0/1158470339 learned_addr learned my addr 172.21.15.92:0/1158470339 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:59.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323ad9d700 1 --2- 172.21.15.92:0/1158470339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c15db10 0x7f323c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f3231d9b700 1 --2- 172.21.15.92:0/1158470339 >> v2:172.21.15.136:3300/0 conn(0x7f323c159c60 0x7f323c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 -- 172.21.15.92:0/1158470339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c15db10 msgr2=0x7f323c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 --2- 172.21.15.92:0/1158470339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c15db10 0x7f323c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 -- 172.21.15.92:0/1158470339 >> v2:172.21.15.136:3300/0 conn(0x7f323c159c60 msgr2=0x7f323c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 --2- 172.21.15.92:0/1158470339 >> v2:172.21.15.136:3300/0 conn(0x7f323c159c60 0x7f323c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 -- 172.21.15.92:0/1158470339 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c15e140 con 0x7f323c159890 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f3231d9b700 1 --2- 172.21.15.92:0/1158470339 >> v2:172.21.15.136:3300/0 conn(0x7f323c159c60 0x7f323c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f323a59c700 1 --2- 172.21.15.92:0/1158470339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c159890 0x7f323c15abd0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b67f0 tx=0x7f322c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5672367deee799b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.100+0000 7f3239d9b700 1 -- 172.21.15.92:0/1158470339 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c009e70 con 0x7f323c159890 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f3239d9b700 1 -- 172.21.15.92:0/1158470339 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c00b860 con 0x7f323c159890 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f3239d9b700 1 -- 172.21.15.92:0/1158470339 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c00bc00 con 0x7f323c159890 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 -- 172.21.15.92:0/1158470339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c159890 msgr2=0x7f323c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 --2- 172.21.15.92:0/1158470339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c159890 0x7f323c15abd0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b67f0 tx=0x7f322c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 -- 172.21.15.92:0/1158470339 shutdown_connections 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 --2- 172.21.15.92:0/1158470339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c15db10 0x7f323c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 --2- 172.21.15.92:0/1158470339 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c159890 0x7f323c15abd0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 --2- 172.21.15.92:0/1158470339 >> v2:172.21.15.136:3300/0 conn(0x7f323c159c60 0x7f323c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 -- 172.21.15.92:0/1158470339 >> 172.21.15.92:0/1158470339 conn(0x7f323c068e20 msgr2=0x7f323c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 -- 172.21.15.92:0/1158470339 shutdown_connections 2023-10-06T03:08:59.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.101+0000 7f324098a700 1 -- 172.21.15.92:0/1158470339 wait complete. 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.102+0000 7f324098a700 1 Processor -- start 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.102+0000 7f324098a700 1 -- start start 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f324098a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f323c159890 0x7f323c148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f324098a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c159c60 0x7f323c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f324098a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 0x7f323c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f324098a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c155890 con 0x7f323c15db10 2023-10-06T03:08:59.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f324098a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0ced80 con 0x7f323c159890 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f324098a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0cf060 con 0x7f323c159c60 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f323ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 0x7f323c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f323ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 0x7f323c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:56576/0 (socket says 172.21.15.92:56576) 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f323ad9d700 1 -- 172.21.15.92:0/423980168 learned_addr learned my addr 172.21.15.92:0/423980168 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f3231d9b700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c159c60 0x7f323c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.103+0000 7f323a59c700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f323c159890 0x7f323c148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f323ad9d700 1 -- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c159c60 msgr2=0x7f323c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f323ad9d700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c159c60 0x7f323c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f323ad9d700 1 -- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f323c159890 msgr2=0x7f323c148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f323ad9d700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f323c159890 0x7f323c148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f323ad9d700 1 -- 172.21.15.92:0/423980168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0094f0 con 0x7f323c15db10 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f323ad9d700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 0x7f323c0b3f30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e90 tx=0x7f323400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400cb70 con 0x7f323c15db10 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323400f040 con 0x7f323c15db10 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c0cf340 con 0x7f323c15db10 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234014480 con 0x7f323c15db10 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.104+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c148740 con 0x7f323c15db10 2023-10-06T03:08:59.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.105+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c04c3c0 con 0x7f323c15db10 2023-10-06T03:08:59.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.105+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3234014830 con 0x7f323c15db10 2023-10-06T03:08:59.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.106+0000 7f3232ffd700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f321c067320 0x7f321c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.106+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3234085410 con 0x7f323c15db10 2023-10-06T03:08:59.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.106+0000 7f323a59c700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f321c067320 0x7f321c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.106+0000 7f323a59c700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f321c067320 0x7f321c0697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b67f0 tx=0x7f322c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.110+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3234052280 con 0x7f323c15db10 2023-10-06T03:08:59.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.297+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f323c0d0e70 con 0x7f323c15db10 2023-10-06T03:08:59.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.298+0000 7f3232ffd700 1 -- 172.21.15.92:0/423980168 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f32340558a0 con 0x7f323c15db10 2023-10-06T03:08:59.300 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:08:59.300 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":136,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:08:58.558311+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:08:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f321c067320 msgr2=0x7f321c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f321c067320 0x7f321c0697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f323c0b67f0 tx=0x7f322c017000 comp rx=0 tx=0).stop 2023-10-06T03:08:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 msgr2=0x7f323c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 0x7f323c0b3f30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f3234009e90 tx=0x7f323400b040 comp rx=0 tx=0).stop 2023-10-06T03:08:59.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 shutdown_connections 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f321c067320 0x7f321c0697d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f323c15db10 0x7f323c0b3f30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f323c159890 0x7f323c148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 --2- 172.21.15.92:0/423980168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f323c159c60 0x7f323c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 >> 172.21.15.92:0/423980168 conn(0x7f323c068e20 msgr2=0x7f323c1584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 shutdown_connections 2023-10-06T03:08:59.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.300+0000 7f324098a700 1 -- 172.21.15.92:0/423980168 wait complete. 2023-10-06T03:08:59.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:08:59.314 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:08:59.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.430+0000 7f16cffff700 1 Processor -- start 2023-10-06T03:08:59.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.430+0000 7f16cffff700 1 -- start start 2023-10-06T03:08:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f16d00cc600 0x7f16d00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d0150b50 con 0x7f16d015d990 2023-10-06T03:08:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cffff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d00cede0 con 0x7f16d00cc9d0 2023-10-06T03:08:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d00b2260 con 0x7f16d00cc600 2023-10-06T03:08:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cf7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f16d015d990 0x7f16d00cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40420/0 (socket says 172.21.15.92:40420) 2023-10-06T03:08:59.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cf7fe700 1 -- 172.21.15.92:0/3458036322 learned_addr learned my addr 172.21.15.92:0/3458036322 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989286908 0 0) 0x7f16d0150b50 con 0x7f16d015d990 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16b8003570 con 0x7f16d015d990 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16ceffd700 1 --2- 172.21.15.92:0/3458036322 >> v2:172.21.15.136:3300/0 conn(0x7f16d00cc600 0x7f16d00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 504802039 0 0) 0x7f16b8003570 con 0x7f16d015d990 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 >> v2:172.21.15.136:3300/0 conn(0x7f16d00cc600 msgr2=0x7f16d00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 --2- 172.21.15.92:0/3458036322 >> v2:172.21.15.136:3300/0 conn(0x7f16d00cc600 0x7f16d00b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 >> v1:172.21.15.136:6789/0 conn(0x7f16d00cc9d0 legacy=0x7f16d015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00b2540 con 0x7f16d015d990 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f16c80017f0 con 0x7f16d015d990 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 >> v1:172.21.15.92:6789/0 conn(0x7f16d015d990 legacy=0x7f16d00cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b8003a60 0x7f16b8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16b8004810 0x7f16b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b80051c0 0x7f16b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00b2260 con 0x7f16b80051c0 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.431+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b800c0d0 con 0x7f16b8003a60 2023-10-06T03:08:59.440 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b800c290 con 0x7f16b8004810 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b80051c0 0x7f16b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16ceffd700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b8003a60 0x7f16b8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16ce7fc700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16b8004810 0x7f16b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 -- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16b8004810 msgr2=0x7f16b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16b8004810 0x7f16b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 -- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b8003a60 msgr2=0x7f16b8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b8003a60 0x7f16b8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 -- 172.21.15.92:0/3458036322 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8003570 con 0x7f16b80051c0 2023-10-06T03:08:59.441 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16ce7fc700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16b8004810 0x7f16b8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 -- 172.21.15.92:0/3458036322 reap_dead start 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cf7fe700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b80051c0 0x7f16b800ba60 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f16c800a930 tx=0x7f16c80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55e0dab8e317d4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80070d0 con 0x7f16b80051c0 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8007230 con 0x7f16b80051c0 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.432+0000 7f16cdffb700 1 -- 172.21.15.92:0/3458036322 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80075d0 con 0x7f16b80051c0 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 -- 172.21.15.92:0/3458036322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b80051c0 msgr2=0x7f16b800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b80051c0 0x7f16b800ba60 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f16c800a930 tx=0x7f16c80021d0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 -- 172.21.15.92:0/3458036322 shutdown_connections 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 --2- 172.21.15.92:0/3458036322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16b80051c0 0x7f16b800ba60 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 -- 172.21.15.92:0/3458036322 >> 172.21.15.92:0/3458036322 conn(0x7f16d00d0630 msgr2=0x7f16d00d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 -- 172.21.15.92:0/3458036322 shutdown_connections 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.433+0000 7f16cffff700 1 -- 172.21.15.92:0/3458036322 wait complete. 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.434+0000 7f16cffff700 1 Processor -- start 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.434+0000 7f16cffff700 1 -- start start 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b80051c0 0x7f16d013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 0x7f16d0069b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16d016d410 0x7f16d016d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00b3de0 con 0x7f16d00696d0 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d006a080 con 0x7f16b80051c0 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d00b4920 con 0x7f16d016d410 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 0x7f16d0069b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 0x7f16d0069b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52640/0 (socket says 172.21.15.92:52640) 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 -- 172.21.15.92:0/2581479550 learned_addr learned my addr 172.21.15.92:0/2581479550 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cf7fe700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16d016d410 0x7f16d016d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ceffd700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b80051c0 0x7f16d013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 -- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16d016d410 msgr2=0x7f16d016d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16d016d410 0x7f16d016d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 -- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b80051c0 msgr2=0x7f16d013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b80051c0 0x7f16d013fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 -- 172.21.15.92:0/2581479550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d00b3ff0 con 0x7f16d00696d0 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cf7fe700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16d016d410 0x7f16d016d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ceffd700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b80051c0 0x7f16d013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16ce7fc700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 0x7f16d0069b40 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f16c400e5b0 tx=0x7f16c4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4010490 con 0x7f16d00696d0 2023-10-06T03:08:59.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c4010040 con 0x7f16d00696d0 2023-10-06T03:08:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d00b42d0 con 0x7f16d00696d0 2023-10-06T03:08:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4010820 con 0x7f16d00696d0 2023-10-06T03:08:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.435+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d016ddc0 con 0x7f16d00696d0 2023-10-06T03:08:59.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.436+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d00b4770 con 0x7f16d00696d0 2023-10-06T03:08:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.437+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f16c4010bd0 con 0x7f16d00696d0 2023-10-06T03:08:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.437+0000 7f16b6ffd700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16bc067320 0x7f16bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.437+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f16c4083e20 con 0x7f16d00696d0 2023-10-06T03:08:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.437+0000 7f16ceffd700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16bc067320 0x7f16bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.438+0000 7f16ceffd700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16bc067320 0x7f16bc0697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f16c0006f00 tx=0x7f16c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.441+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16c404d5a0 con 0x7f16d00696d0 2023-10-06T03:08:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.624+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16d00b2820 con 0x7f16d00696d0 2023-10-06T03:08:59.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.625+0000 7f16b6ffd700 1 -- 172.21.15.92:0/2581479550 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16c40190a0 con 0x7f16d00696d0 2023-10-06T03:08:59.627 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16bc067320 msgr2=0x7f16bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16bc067320 0x7f16bc0697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f16c0006f00 tx=0x7f16c0008040 comp rx=0 tx=0).stop 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 msgr2=0x7f16d0069b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 0x7f16d0069b40 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f16c400e5b0 tx=0x7f16c4009040 comp rx=0 tx=0).stop 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 shutdown_connections 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f16bc067320 0x7f16bc0697d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16d016d410 0x7f16d016d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16b80051c0 0x7f16d013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 --2- 172.21.15.92:0/2581479550 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16d00696d0 0x7f16d0069b40 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 >> 172.21.15.92:0/2581479550 conn(0x7f16d00d0630 msgr2=0x7f16d01512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:59.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 shutdown_connections 2023-10-06T03:08:59.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.627+0000 7f16cffff700 1 -- 172.21.15.92:0/2581479550 wait complete. 2023-10-06T03:08:59.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:08:59.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.756+0000 7f7bb9d19700 1 Processor -- start 2023-10-06T03:08:59.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.756+0000 7f7bb9d19700 1 -- start start 2023-10-06T03:08:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9d19700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bb40b4e70 0x7f7bb40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9d19700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb40b16d0 0x7f7bb40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9d19700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bb415d240 0x7f7bb415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9d19700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40b2080 con 0x7f7bb415d240 2023-10-06T03:08:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9d19700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb40cfb00 con 0x7f7bb40b4e70 2023-10-06T03:08:59.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9d19700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40b5c40 con 0x7f7bb40b16d0 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bb415d240 0x7f7bb415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7bb415d240 0x7f7bb415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52646/0 (socket says 172.21.15.92:52646) 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 -- 172.21.15.92:0/2256283316 learned_addr learned my addr 172.21.15.92:0/2256283316 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb3fff700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb40b16d0 0x7f7bb40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb8d17700 1 --2- 172.21.15.92:0/2256283316 >> v2:172.21.15.136:3300/0 conn(0x7f7bb40b4e70 0x7f7bb40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 -- 172.21.15.92:0/2256283316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb40b16d0 msgr2=0x7f7bb40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb40b16d0 0x7f7bb40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 -- 172.21.15.92:0/2256283316 >> v2:172.21.15.136:3300/0 conn(0x7f7bb40b4e70 msgr2=0x7f7bb40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:08:59.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 --2- 172.21.15.92:0/2256283316 >> v2:172.21.15.136:3300/0 conn(0x7f7bb40b4e70 0x7f7bb40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 -- 172.21.15.92:0/2256283316 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40b5f20 con 0x7f7bb415d240 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb3fff700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb40b16d0 0x7f7bb40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb8d17700 1 --2- 172.21.15.92:0/2256283316 >> v2:172.21.15.136:3300/0 conn(0x7f7bb40b4e70 0x7f7bb40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.757+0000 7f7bb9518700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb415d240 0x7f7bb415d6c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4064910 tx=0x7f7b9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5877a31a4b0d1681 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb37fe700 1 -- 172.21.15.92:0/2256283316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00b860 con 0x7f7bb415d240 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb37fe700 1 -- 172.21.15.92:0/2256283316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00b9c0 con 0x7f7bb415d240 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb37fe700 1 -- 172.21.15.92:0/2256283316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00bd60 con 0x7f7bb415d240 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 -- 172.21.15.92:0/2256283316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb415d240 msgr2=0x7f7bb415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb415d240 0x7f7bb415d6c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4064910 tx=0x7f7b9c009250 comp rx=0 tx=0).stop 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 -- 172.21.15.92:0/2256283316 shutdown_connections 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb415d240 0x7f7bb415d6c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/2256283316 >> v2:172.21.15.136:3300/0 conn(0x7f7bb40b4e70 0x7f7bb40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/2256283316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb40b16d0 0x7f7bb40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 -- 172.21.15.92:0/2256283316 >> 172.21.15.92:0/2256283316 conn(0x7f7bb40d10c0 msgr2=0x7f7bb40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 -- 172.21.15.92:0/2256283316 shutdown_connections 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.758+0000 7f7bb9d19700 1 -- 172.21.15.92:0/2256283316 wait complete. 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.759+0000 7f7bb9d19700 1 Processor -- start 2023-10-06T03:08:59.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.759+0000 7f7bb9d19700 1 -- start start 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb9d19700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb40b16d0 0x7f7bb4158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb9d19700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 0x7f7bb415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb9d19700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb415d240 0x7f7bb415fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb9d19700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb416cf90 con 0x7f7bb40b4e70 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb9d19700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40c60a0 con 0x7f7bb40b16d0 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb9d19700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb416d1e0 con 0x7f7bb415d240 2023-10-06T03:08:59.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 0x7f7bb415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 0x7f7bb415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52658/0 (socket says 172.21.15.92:52658) 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.760+0000 7f7bb3fff700 1 -- 172.21.15.92:0/941128977 learned_addr learned my addr 172.21.15.92:0/941128977 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb9518700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb415d240 0x7f7bb415fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb3fff700 1 -- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb415d240 msgr2=0x7f7bb415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb3fff700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb415d240 0x7f7bb415fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb8d17700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb40b16d0 0x7f7bb4158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb3fff700 1 -- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb40b16d0 msgr2=0x7f7bb4158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb3fff700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb40b16d0 0x7f7bb4158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb3fff700 1 -- 172.21.15.92:0/941128977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0094f0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb3fff700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 0x7f7bb415f790 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400e8d0 tx=0x7f7ba4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400ee70 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400c5e0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400c980 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb416d4c0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.761+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4159330 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.762+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb416d9e0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.762+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7ba4013410 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.763+0000 7f7bb0ff9700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b94067320 0x7f7b940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.763+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7ba40849f0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.763+0000 7f7bb8d17700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b94067320 0x7f7b940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:08:59.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.763+0000 7f7bb8d17700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b94067320 0x7f7b940697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0094d0 tx=0x7f7bac0097a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:08:59.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.767+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bb416d9e0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.948+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7bb416d9e0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.948+0000 7f7bb0ff9700 1 -- 172.21.15.92:0/941128977 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7bb416d9e0 con 0x7f7bb40b4e70 2023-10-06T03:08:59.950 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:08:59.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b94067320 msgr2=0x7f7b940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b94067320 0x7f7b940697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0094d0 tx=0x7f7bac0097a0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 msgr2=0x7f7bb415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:08:59.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 0x7f7bb415f790 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400e8d0 tx=0x7f7ba4008040 comp rx=0 tx=0).stop 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 shutdown_connections 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b94067320 0x7f7b940697d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bb415d240 0x7f7bb415fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb40b16d0 0x7f7bb4158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 --2- 172.21.15.92:0/941128977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7bb40b4e70 0x7f7bb415f790 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 >> 172.21.15.92:0/941128977 conn(0x7f7bb40d10c0 msgr2=0x7f7bb40b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 shutdown_connections 2023-10-06T03:08:59.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:08:59.950+0000 7f7bb9d19700 1 -- 172.21.15.92:0/941128977 wait complete. 2023-10-06T03:08:59.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:09:00.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.077+0000 7f0bd3fff700 1 Processor -- start 2023-10-06T03:09:00.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.077+0000 7f0bd3fff700 1 -- start start 2023-10-06T03:09:00.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd4159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd3fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40cfe20 con 0x7f0bd40b1e50 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd3fff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40cf210 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd3fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40cf4f0 con 0x7f0bd41593b0 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0bd40b1e50 0x7f0bd40b3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40424/0 (socket says 172.21.15.92:40424) 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd37fe700 1 -- 172.21.15.92:0/513271455 learned_addr learned my addr 172.21.15.92:0/513271455 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd2ffd700 1 --2- 172.21.15.92:0/513271455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd4159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd27fc700 1 -- 172.21.15.92:0/513271455 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401434505 0 0) 0x7f0bd40cfe20 con 0x7f0bd40b1e50 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd27fc700 1 -- 172.21.15.92:0/513271455 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc0003570 con 0x7f0bd40b1e50 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd2ffd700 1 -- 172.21.15.92:0/513271455 >> v1:172.21.15.136:6789/0 conn(0x7f0bd40ceaa0 legacy=0x7f0bd40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.078+0000 7f0bd2ffd700 1 -- 172.21.15.92:0/513271455 >> v1:172.21.15.92:6789/0 conn(0x7f0bd40b1e50 legacy=0x7f0bd40b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd2ffd700 1 -- 172.21.15.92:0/513271455 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40cf7d0 con 0x7f0bd41593b0 2023-10-06T03:09:00.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd2ffd700 1 --2- 172.21.15.92:0/513271455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd4159780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009910 tx=0x7f0bc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a711fa607fd0c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd27fc700 1 -- 172.21.15.92:0/513271455 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400b680 con 0x7f0bd41593b0 2023-10-06T03:09:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd27fc700 1 -- 172.21.15.92:0/513271455 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc400b7e0 con 0x7f0bd41593b0 2023-10-06T03:09:00.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd27fc700 1 -- 172.21.15.92:0/513271455 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400bba0 con 0x7f0bd41593b0 2023-10-06T03:09:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 -- 172.21.15.92:0/513271455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 msgr2=0x7f0bd4159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/513271455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd4159780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009910 tx=0x7f0bc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 -- 172.21.15.92:0/513271455 shutdown_connections 2023-10-06T03:09:00.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/513271455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd4159780 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 -- 172.21.15.92:0/513271455 >> 172.21.15.92:0/513271455 conn(0x7f0bd40d1130 msgr2=0x7f0bd40b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 -- 172.21.15.92:0/513271455 shutdown_connections 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.079+0000 7f0bd3fff700 1 -- 172.21.15.92:0/513271455 wait complete. 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 Processor -- start 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 -- start start 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd40b1e50 0x7f0bd4150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 0x7f0bd4152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4159780 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd415a020 con 0x7f0bd40b1e50 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.081+0000 7f0bd3fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd414f8c0 con 0x7f0bd41593b0 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 0x7f0bd4152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 0x7f0bd4152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52668/0 (socket says 172.21.15.92:52668) 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 -- 172.21.15.92:0/1326238353 learned_addr learned my addr 172.21.15.92:0/1326238353 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bd2ffd700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd40b1e50 0x7f0bd4150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bd37fe700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 -- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 msgr2=0x7f0bd414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd414e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 -- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd40b1e50 msgr2=0x7f0bd4150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd40b1e50 0x7f0bd4150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 -- 172.21.15.92:0/1326238353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc4009420 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bca7fc700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 0x7f0bd4152a30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c590 tx=0x7f0bbc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:00.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc011590 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc011710 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd41510b0 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc011ad0 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.082+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd4159cb0 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.083+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd415a190 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.083+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0bbc01b3b0 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.084+0000 7f0bcb7fe700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4067320 0x7f0bb40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.084+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0bbc083e90 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.084+0000 7f0bd2ffd700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4067320 0x7f0bb40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.084+0000 7f0bd2ffd700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4067320 0x7f0bb40697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40093f0 tx=0x7f0bc4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:00.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.088+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bbc050d10 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.271+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bd404c3c0 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.272+0000 7f0bcb7fe700 1 -- 172.21.15.92:0/1326238353 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0bbc017070 con 0x7f0bd40ceaa0 2023-10-06T03:09:00.637 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:00.638 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:09:00.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:00.638 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:09:00.638 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:09:00.638 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:00.638 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:00.639 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:09:00.639 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:09:00.639 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:00.639 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:09:00.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4067320 msgr2=0x7f0bb40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4067320 0x7f0bb40697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40093f0 tx=0x7f0bc4017000 comp rx=0 tx=0).stop 2023-10-06T03:09:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 msgr2=0x7f0bd4152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 0x7f0bd4152a30 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c590 tx=0x7f0bbc00c8a0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 shutdown_connections 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0bb4067320 0x7f0bb40697d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bd41593b0 0x7f0bd414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bd40b1e50 0x7f0bd4150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 --2- 172.21.15.92:0/1326238353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0bd40ceaa0 0x7f0bd4152a30 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 >> 172.21.15.92:0/1326238353 conn(0x7f0bd40d1130 msgr2=0x7f0bd40b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 shutdown_connections 2023-10-06T03:09:00.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.274+0000 7f0bd3fff700 1 -- 172.21.15.92:0/1326238353 wait complete. 2023-10-06T03:09:00.644 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:09:00.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:09:00.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.764+0000 7f463a749700 1 Processor -- start 2023-10-06T03:09:00.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 -- start start 2023-10-06T03:09:00.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634153f50 0x7f46340b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4634154320 0x7f463406d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f463415a190 0x7f463406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340b4570 con 0x7f463415a190 2023-10-06T03:09:00.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463415a730 con 0x7f4634154320 2023-10-06T03:09:00.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f463a749700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463415aa10 con 0x7f4634153f50 2023-10-06T03:09:00.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.765+0000 7f4638ce6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f463415a190 0x7f463406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f463415a190 0x7f463406d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52680/0 (socket says 172.21.15.92:52680) 2023-10-06T03:09:00.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 -- 172.21.15.92:0/269946126 learned_addr learned my addr 172.21.15.92:0/269946126 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:00.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4633fff700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634153f50 0x7f46340b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f46337fe700 1 --2- 172.21.15.92:0/269946126 >> v2:172.21.15.136:3300/0 conn(0x7f4634154320 0x7f463406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 -- 172.21.15.92:0/269946126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634153f50 msgr2=0x7f46340b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634153f50 0x7f46340b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 -- 172.21.15.92:0/269946126 >> v2:172.21.15.136:3300/0 conn(0x7f4634154320 msgr2=0x7f463406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 --2- 172.21.15.92:0/269946126 >> v2:172.21.15.136:3300/0 conn(0x7f4634154320 0x7f463406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 -- 172.21.15.92:0/269946126 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463415acf0 con 0x7f463415a190 2023-10-06T03:09:00.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4633fff700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634153f50 0x7f46340b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:00.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f46337fe700 1 --2- 172.21.15.92:0/269946126 >> v2:172.21.15.136:3300/0 conn(0x7f4634154320 0x7f463406d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:00.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4638ce6700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f463406d670 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f463406f4f0 tx=0x7f462c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc70baa4281aadf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:00.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4632ffd700 1 -- 172.21.15.92:0/269946126 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00e070 con 0x7f463415a190 2023-10-06T03:09:00.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4632ffd700 1 -- 172.21.15.92:0/269946126 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00ba50 con 0x7f463415a190 2023-10-06T03:09:00.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.766+0000 7f4632ffd700 1 -- 172.21.15.92:0/269946126 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0092e0 con 0x7f463415a190 2023-10-06T03:09:00.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 -- 172.21.15.92:0/269946126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 msgr2=0x7f463406d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f463406d670 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f463406f4f0 tx=0x7f462c009840 comp rx=0 tx=0).stop 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 -- 172.21.15.92:0/269946126 shutdown_connections 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634153f50 0x7f46340b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 --2- 172.21.15.92:0/269946126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f463406d670 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 --2- 172.21.15.92:0/269946126 >> v2:172.21.15.136:3300/0 conn(0x7f4634154320 0x7f463406d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 -- 172.21.15.92:0/269946126 >> 172.21.15.92:0/269946126 conn(0x7f463406baf0 msgr2=0x7f463406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:00.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 -- 172.21.15.92:0/269946126 shutdown_connections 2023-10-06T03:09:00.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.767+0000 7f463a749700 1 -- 172.21.15.92:0/269946126 wait complete. 2023-10-06T03:09:00.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.768+0000 7f463a749700 1 Processor -- start 2023-10-06T03:09:00.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.768+0000 7f463a749700 1 -- start start 2023-10-06T03:09:00.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4634153f50 0x7f4634150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634154320 0x7f46340b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f46340b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340af150 con 0x7f463415a190 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b0240 con 0x7f4634153f50 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340af670 con 0x7f4634154320 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f46340b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f46340b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52686/0 (socket says 172.21.15.92:52686) 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 -- 172.21.15.92:0/2003723645 learned_addr learned my addr 172.21.15.92:0/2003723645 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f46337fe700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634154320 0x7f46340b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4633fff700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4634153f50 0x7f4634150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 -- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634154320 msgr2=0x7f46340b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634154320 0x7f46340b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 -- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4634153f50 msgr2=0x7f4634150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4634153f50 0x7f4634150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 -- 172.21.15.92:0/2003723645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c009ae0 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4633fff700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4634153f50 0x7f4634150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f4638ce6700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f46340b0c10 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f462c009880 tx=0x7f462c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0032f0 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c008dd0 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46340af910 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c01c640 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.769+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46340afed0 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.770+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46340718b0 con 0x7f463415a190 2023-10-06T03:09:00.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.771+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f462c003040 con 0x7f463415a190 2023-10-06T03:09:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.771+0000 7f461bfff700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4620067320 0x7f46200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.771+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f462c086350 con 0x7f463415a190 2023-10-06T03:09:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.771+0000 7f4633fff700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4620067320 0x7f46200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.771+0000 7f4633fff700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4620067320 0x7f46200697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4624006f00 tx=0x7f4624008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:00.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.775+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f462c0530c0 con 0x7f463415a190 2023-10-06T03:09:00.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:00.956+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4634069650 con 0x7f463415a190 2023-10-06T03:09:01.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.571+0000 7f461bfff700 1 -- 172.21.15.92:0/2003723645 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) v1 ==== 138+0+0 (secure 0 0 0) 0x7f462c0566e0 con 0x7f463415a190 2023-10-06T03:09:01.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4620067320 msgr2=0x7f46200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:01.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4620067320 0x7f46200697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4624006f00 tx=0x7f4624008040 comp rx=0 tx=0).stop 2023-10-06T03:09:01.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 msgr2=0x7f46340b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:01.575 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f46340b0c10 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f462c009880 tx=0x7f462c018000 comp rx=0 tx=0).stop 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 shutdown_connections 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4620067320 0x7f46200697d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f463415a190 0x7f46340b0c10 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4634153f50 0x7f4634150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 --2- 172.21.15.92:0/2003723645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4634154320 0x7f46340b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 >> 172.21.15.92:0/2003723645 conn(0x7f463406baf0 msgr2=0x7f46340b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:01.576 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.574+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 shutdown_connections 2023-10-06T03:09:01.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.577+0000 7f463a749700 1 -- 172.21.15.92:0/2003723645 wait complete. 2023-10-06T03:09:01.579 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:09:01.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:09:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.711+0000 7fb655516700 1 Processor -- start 2023-10-06T03:09:01.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb655516700 1 -- start start 2023-10-06T03:09:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb655516700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb6500cc2c0 0x7fb6500cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:01.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb655516700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb65006c860 0x7fb6500af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb655516700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6500af9a0 con 0x7fb65015af50 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb655516700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6500afc80 con 0x7fb65006c860 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb655516700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65015a470 con 0x7fb6500cc2c0 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb64e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb65006c860 0x7fb6500af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb64e7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb65006c860 0x7fb6500af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52692/0 (socket says 172.21.15.92:52692) 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.712+0000 7fb64e7fc700 1 -- 172.21.15.92:0/3884909442 learned_addr learned my addr 172.21.15.92:0/3884909442 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64effd700 1 --2- 172.21.15.92:0/3884909442 >> v2:172.21.15.136:3300/0 conn(0x7fb6500cc2c0 0x7fb6500cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64e7fc700 1 -- 172.21.15.92:0/3884909442 >> v2:172.21.15.136:3300/0 conn(0x7fb6500cc2c0 msgr2=0x7fb6500cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/3884909442 >> v2:172.21.15.136:3300/0 conn(0x7fb6500cc2c0 0x7fb6500cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64e7fc700 1 -- 172.21.15.92:0/3884909442 >> v1:172.21.15.136:6789/0 conn(0x7fb65015af50 legacy=0x7fb650153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:01.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64e7fc700 1 -- 172.21.15.92:0/3884909442 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65015a750 con 0x7fb65006c860 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/3884909442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb6500af0a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099b0 tx=0x7fb638009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7174ac7d7fffd166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64dffb700 1 -- 172.21.15.92:0/3884909442 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800b7e0 con 0x7fb65006c860 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64dffb700 1 -- 172.21.15.92:0/3884909442 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63800b940 con 0x7fb65006c860 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.713+0000 7fb64dffb700 1 -- 172.21.15.92:0/3884909442 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800bce0 con 0x7fb65006c860 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 -- 172.21.15.92:0/3884909442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 msgr2=0x7fb6500af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 --2- 172.21.15.92:0/3884909442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb6500af0a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099b0 tx=0x7fb638009250 comp rx=0 tx=0).stop 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 -- 172.21.15.92:0/3884909442 shutdown_connections 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 --2- 172.21.15.92:0/3884909442 >> v2:172.21.15.136:3300/0 conn(0x7fb6500cc2c0 0x7fb6500cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 --2- 172.21.15.92:0/3884909442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb6500af0a0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 -- 172.21.15.92:0/3884909442 >> 172.21.15.92:0/3884909442 conn(0x7fb6500cdd80 msgr2=0x7fb6500c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:01.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 -- 172.21.15.92:0/3884909442 shutdown_connections 2023-10-06T03:09:01.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.714+0000 7fb655516700 1 -- 172.21.15.92:0/3884909442 wait complete. 2023-10-06T03:09:01.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.715+0000 7fb655516700 1 Processor -- start 2023-10-06T03:09:01.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.715+0000 7fb655516700 1 -- start start 2023-10-06T03:09:01.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb65006e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb6500cc2c0 0x7fb6500c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65015af50 0x7fb650157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6501597a0 con 0x7fb65006c860 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b0500 con 0x7fb65015af50 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b1af0 con 0x7fb6500cc2c0 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb65006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb65006e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52700/0 (socket says 172.21.15.92:52700) 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 -- 172.21.15.92:0/2763504187 learned_addr learned my addr 172.21.15.92:0/2763504187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb6500cc2c0 0x7fb6500c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64f7fe700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65015af50 0x7fb650157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:01.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 -- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb6500cc2c0 msgr2=0x7fb6500c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb6500cc2c0 0x7fb6500c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 -- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65015af50 msgr2=0x7fb650157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65015af50 0x7fb650157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 -- 172.21.15.92:0/2763504187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380094f0 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb64effd700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb65006e750 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb64000e190 tx=0x7fb640009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64000ecd0 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64000ee30 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6500b1dd0 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640014670 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.716+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6500b20b0 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.717+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65004c3c0 con 0x7fb65006c860 2023-10-06T03:09:01.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.718+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb640014a20 con 0x7fb65006c860 2023-10-06T03:09:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.718+0000 7fb646ffd700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb630067350 0x7fb630069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.718+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fb640085280 con 0x7fb65006c860 2023-10-06T03:09:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.718+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb630067350 0x7fb630069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:01.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.719+0000 7fb64e7fc700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb630067350 0x7fb630069800 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099b0 tx=0x7fb638016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:01.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.723+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb640051f90 con 0x7fb65006c860 2023-10-06T03:09:01.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:01.906+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb65015aa30 con 0x7fb65006c860 2023-10-06T03:09:02.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.577+0000 7fb646ffd700 1 -- 172.21.15.92:0/2763504187 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb640009a60 con 0x7fb65006c860 2023-10-06T03:09:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.580+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb630067350 msgr2=0x7fb630069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.580+0000 7fb655516700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb630067350 0x7fb630069800 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099b0 tx=0x7fb638016000 comp rx=0 tx=0).stop 2023-10-06T03:09:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.580+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 msgr2=0x7fb65006e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.580+0000 7fb655516700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb65006e750 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb64000e190 tx=0x7fb640009920 comp rx=0 tx=0).stop 2023-10-06T03:09:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.581+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 shutdown_connections 2023-10-06T03:09:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.581+0000 7fb655516700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb630067350 0x7fb630069800 secure :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb6380099b0 tx=0x7fb638016000 comp rx=0 tx=0).stop 2023-10-06T03:09:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.581+0000 7fb655516700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb65006c860 0x7fb65006e750 secure :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb64000e190 tx=0x7fb640009920 comp rx=0 tx=0).stop 2023-10-06T03:09:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.581+0000 7fb655516700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65015af50 0x7fb650157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.581+0000 7fb655516700 1 --2- 172.21.15.92:0/2763504187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb6500cc2c0 0x7fb6500c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.581+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 >> 172.21.15.92:0/2763504187 conn(0x7fb6500cdd80 msgr2=0x7fb6500aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.588+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 shutdown_connections 2023-10-06T03:09:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.589+0000 7fb655516700 1 -- 172.21.15.92:0/2763504187 wait complete. 2023-10-06T03:09:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:09:02.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:09:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.724+0000 7f47cc528700 1 Processor -- start 2023-10-06T03:09:02.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.724+0000 7f47cc528700 1 -- start start 2023-10-06T03:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.725+0000 7f47cc528700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f47c4069560 0x7f47c40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.725+0000 7f47cc528700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c4069ee0 con 0x7f47c4159890 2023-10-06T03:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.725+0000 7f47cc528700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c415a160 con 0x7f47c4159c60 2023-10-06T03:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.725+0000 7f47cc528700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c40d13a0 con 0x7f47c4069560 2023-10-06T03:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.725+0000 7f47ca2c4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f47c4159890 0x7f47c415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40428/0 (socket says 172.21.15.92:40428) 2023-10-06T03:09:02.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.725+0000 7f47ca2c4700 1 -- 172.21.15.92:0/1971748862 learned_addr learned my addr 172.21.15.92:0/1971748862 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:02.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47caac5700 1 --2- 172.21.15.92:0/1971748862 >> v2:172.21.15.136:3300/0 conn(0x7f47c4069560 0x7f47c40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808840229 0 0) 0x7f47c4069ee0 con 0x7f47c4159890 2023-10-06T03:09:02.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b0003570 con 0x7f47c4159890 2023-10-06T03:09:02.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4116750223 0 0) 0x7f47b0003570 con 0x7f47c4159890 2023-10-06T03:09:02.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 >> v2:172.21.15.136:3300/0 conn(0x7f47c4069560 msgr2=0x7f47c40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:02.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 --2- 172.21.15.92:0/1971748862 >> v2:172.21.15.136:3300/0 conn(0x7f47c4069560 0x7f47c40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 >> v1:172.21.15.136:6789/0 conn(0x7f47c4159c60 legacy=0x7f47c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c415e140 con 0x7f47c4159890 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f47c00017f0 con 0x7f47c4159890 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 >> v1:172.21.15.92:6789/0 conn(0x7f47c4159890 legacy=0x7f47c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b0003a60 0x7f47b0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47b00051c0 0x7f47b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40d13a0 con 0x7f47b0004810 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b000c0d0 con 0x7f47b0003a60 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b000c290 con 0x7f47b00051c0 2023-10-06T03:09:02.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47ca2c4700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b0003a60 0x7f47b0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47caac5700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47b00051c0 0x7f47b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c9ac3700 1 -- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47b00051c0 msgr2=0x7f47b000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47b00051c0 0x7f47b000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c9ac3700 1 -- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b0003a60 msgr2=0x7f47b0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47b0003a60 0x7f47b0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47caac5700 1 -- 172.21.15.92:0/1971748862 reap_dead start 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.726+0000 7f47c9ac3700 1 -- 172.21.15.92:0/1971748862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0003570 con 0x7f47b0004810 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47b0004c80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f47b40018b0 tx=0x7f47b4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5162c1330f6a4bd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4012420 con 0x7f47b0004810 2023-10-06T03:09:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4012580 con 0x7f47b0004810 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47c92c2700 1 -- 172.21.15.92:0/1971748862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4012920 con 0x7f47b0004810 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 -- 172.21.15.92:0/1971748862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 msgr2=0x7f47b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47b0004c80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f47b40018b0 tx=0x7f47b4007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 -- 172.21.15.92:0/1971748862 shutdown_connections 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 --2- 172.21.15.92:0/1971748862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47b0004c80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 -- 172.21.15.92:0/1971748862 >> 172.21.15.92:0/1971748862 conn(0x7f47c4068e20 msgr2=0x7f47c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 -- 172.21.15.92:0/1971748862 shutdown_connections 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.727+0000 7f47cc528700 1 -- 172.21.15.92:0/1971748862 wait complete. 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.728+0000 7f47cc528700 1 Processor -- start 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.728+0000 7f47cc528700 1 -- start start 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47cc528700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47c40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47cc528700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47c41496d0 0x7f47c415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47cc528700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4154930 0x7f47c4154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47cc528700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c415c2d0 con 0x7f47b0004810 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47cc528700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41552e0 con 0x7f47c4154930 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47cc528700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c415b650 con 0x7f47c41496d0 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47ca2c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47c40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47ca2c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47c40c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52718/0 (socket says 172.21.15.92:52718) 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47ca2c4700 1 -- 172.21.15.92:0/2333928029 learned_addr learned my addr 172.21.15.92:0/2333928029 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:02.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.729+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47c41496d0 0x7f47c415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47caac5700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4154930 0x7f47c4154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ca2c4700 1 -- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47c41496d0 msgr2=0x7f47c415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ca2c4700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47c41496d0 0x7f47c415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ca2c4700 1 -- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4154930 msgr2=0x7f47c4154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ca2c4700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4154930 0x7f47c4154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ca2c4700 1 -- 172.21.15.92:0/2333928029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c415b930 con 0x7f47b0004810 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ca2c4700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47c40c8e60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f47c0007a70 tx=0x7f47c00021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000ae40 con 0x7f47b0004810 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c00133f0 con 0x7f47b0004810 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c415bc10 con 0x7f47b0004810 2023-10-06T03:09:02.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00137b0 con 0x7f47b0004810 2023-10-06T03:09:02.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.730+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c4149b40 con 0x7f47b0004810 2023-10-06T03:09:02.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.731+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c404c3c0 con 0x7f47b0004810 2023-10-06T03:09:02.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.731+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f47c0013b20 con 0x7f47b0004810 2023-10-06T03:09:02.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.732+0000 7f47ba7fc700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47a8067320 0x7f47a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:02.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.732+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47a8067320 0x7f47a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:02.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.732+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f47c0085b60 con 0x7f47b0004810 2023-10-06T03:09:02.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.732+0000 7f47c9ac3700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47a8067320 0x7f47a80697d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f47c4157640 tx=0x7f47b4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:02.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.736+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47c00526f0 con 0x7f47b0004810 2023-10-06T03:09:02.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:02.962+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f47c406a5b0 con 0x7f47b0004810 2023-10-06T03:09:03.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.600+0000 7f47ba7fc700 1 -- 172.21.15.92:0/2333928029 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7f47c00170c0 con 0x7f47b0004810 2023-10-06T03:09:03.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47a8067320 msgr2=0x7f47a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47a8067320 0x7f47a80697d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f47c4157640 tx=0x7f47b4015000 comp rx=0 tx=0).stop 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 msgr2=0x7f47c40c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47c40c8e60 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f47c0007a70 tx=0x7f47c00021d0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 shutdown_connections 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f47a8067320 0x7f47a80697d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f47b0004810 0x7f47c40c8e60 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4154930 0x7f47c4154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 --2- 172.21.15.92:0/2333928029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47c41496d0 0x7f47c415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 >> 172.21.15.92:0/2333928029 conn(0x7f47c4068e20 msgr2=0x7f47c4158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 shutdown_connections 2023-10-06T03:09:03.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.613+0000 7f47cc528700 1 -- 172.21.15.92:0/2333928029 wait complete. 2023-10-06T03:09:03.617 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 25 and data pool 26 2023-10-06T03:09:03.627 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.740+0000 7fabadd0a700 1 Processor -- start 2023-10-06T03:09:03.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 -- start start 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faba81598d0 0x7faba815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7faba80c7bd0 0x7faba806ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba806f350 0x7faba806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba815af50 con 0x7faba81598d0 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faba806e980 con 0x7faba80c7bd0 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabadd0a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba80ce180 con 0x7faba806f350 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabacd08700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faba81598d0 0x7faba815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabacd08700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faba81598d0 0x7faba815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52734/0 (socket says 172.21.15.92:52734) 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.741+0000 7fabacd08700 1 -- 172.21.15.92:0/1371147902 learned_addr learned my addr 172.21.15.92:0/1371147902 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:03.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabacd08700 1 -- 172.21.15.92:0/1371147902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba806f350 msgr2=0x7faba806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabacd08700 1 --2- 172.21.15.92:0/1371147902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba806f350 0x7faba806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabacd08700 1 -- 172.21.15.92:0/1371147902 >> v2:172.21.15.136:3300/0 conn(0x7faba80c7bd0 msgr2=0x7faba806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabacd08700 1 --2- 172.21.15.92:0/1371147902 >> v2:172.21.15.136:3300/0 conn(0x7faba80c7bd0 0x7faba806ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabacd08700 1 -- 172.21.15.92:0/1371147902 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80c80f0 con 0x7faba81598d0 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabacd08700 1 --2- 172.21.15.92:0/1371147902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba81598d0 0x7faba815aa10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7faba80659d0 tx=0x7fab98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9c9c073322301 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7faba77fe700 1 -- 172.21.15.92:0/1371147902 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9800e070 con 0x7faba81598d0 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7faba77fe700 1 -- 172.21.15.92:0/1371147902 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9800ba50 con 0x7faba81598d0 2023-10-06T03:09:03.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7faba77fe700 1 -- 172.21.15.92:0/1371147902 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980092e0 con 0x7faba81598d0 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabadd0a700 1 -- 172.21.15.92:0/1371147902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba81598d0 msgr2=0x7faba815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.742+0000 7fabadd0a700 1 --2- 172.21.15.92:0/1371147902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba81598d0 0x7faba815aa10 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7faba80659d0 tx=0x7fab98009840 comp rx=0 tx=0).stop 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 -- 172.21.15.92:0/1371147902 shutdown_connections 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 --2- 172.21.15.92:0/1371147902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba806f350 0x7faba806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 --2- 172.21.15.92:0/1371147902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba81598d0 0x7faba815aa10 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 --2- 172.21.15.92:0/1371147902 >> v2:172.21.15.136:3300/0 conn(0x7faba80c7bd0 0x7faba806ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 -- 172.21.15.92:0/1371147902 >> 172.21.15.92:0/1371147902 conn(0x7faba806b490 msgr2=0x7faba80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 -- 172.21.15.92:0/1371147902 shutdown_connections 2023-10-06T03:09:03.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.743+0000 7fabadd0a700 1 -- 172.21.15.92:0/1371147902 wait complete. 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.744+0000 7fabadd0a700 1 Processor -- start 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.744+0000 7fabadd0a700 1 -- start start 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 0x7faba8150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faba80c7bd0 0x7faba815dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba81598d0 0x7faba80c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba80b1780 con 0x7faba806f350 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba81510b0 con 0x7faba80c7bd0 2023-10-06T03:09:03.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba815d110 con 0x7faba81598d0 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 0x7faba8150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 0x7faba8150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52738/0 (socket says 172.21.15.92:52738) 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 -- 172.21.15.92:0/3675849102 learned_addr learned my addr 172.21.15.92:0/3675849102 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7faba7fff700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faba80c7bd0 0x7faba815dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabad509700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba81598d0 0x7faba80c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 -- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba81598d0 msgr2=0x7faba80c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba81598d0 0x7faba80c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 -- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faba80c7bd0 msgr2=0x7faba815dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faba80c7bd0 0x7faba815dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 -- 172.21.15.92:0/3675849102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009ae0 con 0x7faba806f350 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabad509700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba81598d0 0x7faba80c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabacd08700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 0x7faba8150b70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7faba806de70 tx=0x7fab98018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab98008bb0 con 0x7faba806f350 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98008d10 con 0x7faba806f350 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980266b0 con 0x7faba806f350 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba815d3f0 con 0x7faba806f350 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.745+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba815e260 con 0x7faba806f350 2023-10-06T03:09:03.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.746+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba815d910 con 0x7faba806f350 2023-10-06T03:09:03.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.747+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fab98003040 con 0x7faba806f350 2023-10-06T03:09:03.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.747+0000 7faba4ff9700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab94067320 0x7fab940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:03.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.747+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab98086cc0 con 0x7faba806f350 2023-10-06T03:09:03.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.747+0000 7faba7fff700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab94067320 0x7fab940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:03.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.747+0000 7faba7fff700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab94067320 0x7fab940697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faba815d040 tx=0x7fab9c0090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:03.754 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.751+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faba815d910 con 0x7faba806f350 2023-10-06T03:09:03.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.936+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faba80b1550 con 0x7faba806f350 2023-10-06T03:09:03.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.937+0000 7faba4ff9700 1 -- 172.21.15.92:0/3675849102 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16893 (secure 0 0 0) 0x7faba80b1550 con 0x7faba806f350 2023-10-06T03:09:03.938 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:03.939 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":139,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:02.963938+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:01.925339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab94067320 msgr2=0x7fab940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab94067320 0x7fab940697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faba815d040 tx=0x7fab9c0090f0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 msgr2=0x7faba8150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 0x7faba8150b70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7faba806de70 tx=0x7fab98018000 comp rx=0 tx=0).stop 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 shutdown_connections 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab94067320 0x7fab940697d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faba806f350 0x7faba8150b70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faba81598d0 0x7faba80c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 --2- 172.21.15.92:0/3675849102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faba80c7bd0 0x7faba815dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 >> 172.21.15.92:0/3675849102 conn(0x7faba806b490 msgr2=0x7faba8157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:03.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 shutdown_connections 2023-10-06T03:09:03.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:03.939+0000 7fabadd0a700 1 -- 172.21.15.92:0/3675849102 wait complete. 2023-10-06T03:09:03.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:04.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.065+0000 7fbc6485c700 1 Processor -- start 2023-10-06T03:09:04.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.065+0000 7fbc6485c700 1 -- start start 2023-10-06T03:09:04.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc6485c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbc60159890 0x7fbc6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc6485c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc60069560 0x7fbc600699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc6485c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc60069ee0 con 0x7fbc60159c60 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc6485c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6015a160 con 0x7fbc60069560 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc6485c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc600d13a0 con 0x7fbc60159890 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc5ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc60069560 0x7fbc600699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc5ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc60069560 0x7fbc600699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52744/0 (socket says 172.21.15.92:52744) 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc5ed9d700 1 -- 172.21.15.92:0/711291511 learned_addr learned my addr 172.21.15.92:0/711291511 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.066+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/711291511 >> v2:172.21.15.136:3300/0 conn(0x7fbc60159890 0x7fbc6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5ed9d700 1 -- 172.21.15.92:0/711291511 >> v2:172.21.15.136:3300/0 conn(0x7fbc60159890 msgr2=0x7fbc6015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5ed9d700 1 --2- 172.21.15.92:0/711291511 >> v2:172.21.15.136:3300/0 conn(0x7fbc60159890 0x7fbc6015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5ed9d700 1 -- 172.21.15.92:0/711291511 >> v1:172.21.15.136:6789/0 conn(0x7fbc60159c60 legacy=0x7fbc6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5ed9d700 1 -- 172.21.15.92:0/711291511 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6015e140 con 0x7fbc60069560 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5ed9d700 1 --2- 172.21.15.92:0/711291511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600699a0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fbc600d1090 tx=0x7fbc58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c82770c20e08f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5dd9b700 1 -- 172.21.15.92:0/711291511 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800e070 con 0x7fbc60069560 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5dd9b700 1 -- 172.21.15.92:0/711291511 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5800ba20 con 0x7fbc60069560 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc5dd9b700 1 -- 172.21.15.92:0/711291511 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580092e0 con 0x7fbc60069560 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 -- 172.21.15.92:0/711291511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 msgr2=0x7fbc600699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 --2- 172.21.15.92:0/711291511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600699a0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fbc600d1090 tx=0x7fbc58009840 comp rx=0 tx=0).stop 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 -- 172.21.15.92:0/711291511 shutdown_connections 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 --2- 172.21.15.92:0/711291511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600699a0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 --2- 172.21.15.92:0/711291511 >> v2:172.21.15.136:3300/0 conn(0x7fbc60159890 0x7fbc6015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 -- 172.21.15.92:0/711291511 >> 172.21.15.92:0/711291511 conn(0x7fbc60068e20 msgr2=0x7fbc600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 -- 172.21.15.92:0/711291511 shutdown_connections 2023-10-06T03:09:04.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.067+0000 7fbc6485c700 1 -- 172.21.15.92:0/711291511 wait complete. 2023-10-06T03:09:04.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 Processor -- start 2023-10-06T03:09:04.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 -- start start 2023-10-06T03:09:04.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc60159890 0x7fbc600b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc60159c60 0x7fbc600c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b6270 con 0x7fbc60069560 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6016d270 con 0x7fbc60159c60 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.069+0000 7fbc6485c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6016d550 con 0x7fbc60159890 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600b5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52746/0 (socket says 172.21.15.92:52746) 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 -- 172.21.15.92:0/3314209639 learned_addr learned my addr 172.21.15.92:0/3314209639 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5ed9d700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc60159c60 0x7fbc600c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc55d9b700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc60159890 0x7fbc600b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 -- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc60159890 msgr2=0x7fbc600b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc60159890 0x7fbc600b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 -- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc60159c60 msgr2=0x7fbc600c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc60159c60 0x7fbc600c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 -- 172.21.15.92:0/3314209639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009ae0 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc5e59c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600b5bf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000ec80 tx=0x7fbc500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5000c850 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5000c9b0 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6016d830 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5000cd50 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.070+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc600b46c0 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.071+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6016dcd0 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.072+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbc50013680 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.072+0000 7fbc56ffd700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc40067320 0x7fbc400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.072+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbc50085070 con 0x7fbc60069560 2023-10-06T03:09:04.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.072+0000 7fbc55d9b700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc40067320 0x7fbc400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.072+0000 7fbc55d9b700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc40067320 0x7fbc400697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc60153660 tx=0x7fbc48007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:04.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.076+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc500b65a0 con 0x7fbc60069560 2023-10-06T03:09:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.260+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6016de80 con 0x7fbc60069560 2023-10-06T03:09:04.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.261+0000 7fbc56ffd700 1 -- 172.21.15.92:0/3314209639 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fbc50051ba0 con 0x7fbc60069560 2023-10-06T03:09:04.263 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:04.263 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":139,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:02.963938+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:01.925339+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc40067320 msgr2=0x7fbc400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc40067320 0x7fbc400697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc60153660 tx=0x7fbc48007ec0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 msgr2=0x7fbc600b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600b5bf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000ec80 tx=0x7fbc500091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 shutdown_connections 2023-10-06T03:09:04.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbc40067320 0x7fbc400697d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc60069560 0x7fbc600b5bf0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc60159c60 0x7fbc600c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 --2- 172.21.15.92:0/3314209639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc60159890 0x7fbc600b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 >> 172.21.15.92:0/3314209639 conn(0x7fbc60068e20 msgr2=0x7fbc6014d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 shutdown_connections 2023-10-06T03:09:04.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.263+0000 7fbc6485c700 1 -- 172.21.15.92:0/3314209639 wait complete. 2023-10-06T03:09:04.277 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:09:04.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.392+0000 7f213c995700 1 Processor -- start 2023-10-06T03:09:04.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.392+0000 7f213c995700 1 -- start start 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f213c995700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 0x7f21380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f213c995700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2138069ee0 con 0x7f2138159c60 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f213c995700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213815a160 con 0x7f2138159890 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f213c995700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380d13a0 con 0x7f2138069560 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f212dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2138159c60 0x7f213815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40440/0 (socket says 172.21.15.92:40440) 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f212dd9b700 1 -- 172.21.15.92:0/2300042517 learned_addr learned my addr 172.21.15.92:0/2300042517 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534851351 0 0) 0x7f2138069ee0 con 0x7f2138159c60 2023-10-06T03:09:04.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f2136d9d700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 0x7f21380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.393+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2124003570 con 0x7f2138159c60 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3031097502 0 0) 0x7f2124003570 con 0x7f2138159c60 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 msgr2=0x7f21380699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 0x7f21380699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 >> v1:172.21.15.136:6789/0 conn(0x7f2138159890 legacy=0x7f213815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213815e140 con 0x7f2138159c60 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2120001470 con 0x7f2138159c60 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 >> v1:172.21.15.92:6789/0 conn(0x7f2138159c60 legacy=0x7f213815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2124003a90 0x7f2124003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 0x7f2124004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2124004d50 0x7f21240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380d13a0 con 0x7f2124004d50 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212400bba0 con 0x7f2124003a90 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212400bd60 con 0x7f2138069560 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2124004d50 0x7f21240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f213659c700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2124003a90 0x7f2124003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f212dd9b700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 0x7f2124004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 -- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 msgr2=0x7f2124004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2138069560 0x7f2124004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 -- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2124003a90 msgr2=0x7f2124003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:04.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2124003a90 0x7f2124003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 -- 172.21.15.92:0/2300042517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124003570 con 0x7f2124004d50 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2136d9d700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2124004d50 0x7f21240051c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f213000e210 tx=0x7f213000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7951560fb49b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000eac0 con 0x7f2124004d50 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000f040 con 0x7f2124004d50 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.394+0000 7f2135d9b700 1 -- 172.21.15.92:0/2300042517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000ee70 con 0x7f2124004d50 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f213c995700 1 -- 172.21.15.92:0/2300042517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2124004d50 msgr2=0x7f21240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f213c995700 1 --2- 172.21.15.92:0/2300042517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2124004d50 0x7f21240051c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f213000e210 tx=0x7f213000b040 comp rx=0 tx=0).stop 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f2136d9d700 1 -- 172.21.15.92:0/2300042517 reap_dead start 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f213c995700 1 -- 172.21.15.92:0/2300042517 shutdown_connections 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f213c995700 1 -- 172.21.15.92:0/2300042517 >> 172.21.15.92:0/2300042517 conn(0x7f2138068e20 msgr2=0x7f21380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:04.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f213c995700 1 -- 172.21.15.92:0/2300042517 shutdown_connections 2023-10-06T03:09:04.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.395+0000 7f213c995700 1 -- 172.21.15.92:0/2300042517 wait complete. 2023-10-06T03:09:04.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.396+0000 7f213c995700 1 Processor -- start 2023-10-06T03:09:04.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.396+0000 7f213c995700 1 -- start start 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213c995700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2124004d50 0x7f21380b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213c995700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2138158bf0 0x7f213814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213c995700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 0x7f21380cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213c995700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380cfbf0 con 0x7f21380cf210 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213c995700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380b6400 con 0x7f2138158bf0 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213c995700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21380b5290 con 0x7f2124004d50 2023-10-06T03:09:04.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 0x7f21380cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 0x7f21380cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52774/0 (socket says 172.21.15.92:52774) 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 -- 172.21.15.92:0/1211195752 learned_addr learned my addr 172.21.15.92:0/1211195752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213659c700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2124004d50 0x7f21380b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f212dd9b700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2138158bf0 0x7f213814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 -- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2124004d50 msgr2=0x7f21380b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2124004d50 0x7f21380b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 -- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2138158bf0 msgr2=0x7f213814d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2138158bf0 0x7f213814d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 -- 172.21.15.92:0/1211195752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380b36c0 con 0x7f21380cf210 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f213659c700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2124004d50 0x7f21380b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:04.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f212dd9b700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2138158bf0 0x7f213814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.397+0000 7f2136d9d700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 0x7f21380cf6b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2130016000 tx=0x7f213001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.398+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213001e640 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.398+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2130022070 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.398+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21380b39a0 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.398+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213001ea10 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.398+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21380b3f60 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.399+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21380b4150 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.399+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2130003040 con 0x7f21380cf210 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.399+0000 7f212effd700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2118067320 0x7f21180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.400+0000 7f213659c700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2118067320 0x7f21180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:04.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.400+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2130024070 con 0x7f21380cf210 2023-10-06T03:09:04.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.400+0000 7f213659c700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2118067320 0x7f21180697d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f213806a070 tx=0x7f2128009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:04.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.404+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2130053670 con 0x7f21380cf210 2023-10-06T03:09:04.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:04.630+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f213804c3c0 con 0x7f21380cf210 2023-10-06T03:09:05.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.608+0000 7f212effd700 1 -- 172.21.15.92:0/1211195752 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2130018020 con 0x7f21380cf210 2023-10-06T03:09:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2118067320 msgr2=0x7f21180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2118067320 0x7f21180697d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f213806a070 tx=0x7f2128009040 comp rx=0 tx=0).stop 2023-10-06T03:09:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 msgr2=0x7f21380cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 0x7f21380cf6b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2130016000 tx=0x7f213001a000 comp rx=0 tx=0).stop 2023-10-06T03:09:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 shutdown_connections 2023-10-06T03:09:05.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2118067320 0x7f21180697d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21380cf210 0x7f21380cf6b0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2124004d50 0x7f21380b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 --2- 172.21.15.92:0/1211195752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2138158bf0 0x7f213814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 >> 172.21.15.92:0/1211195752 conn(0x7f2138068e20 msgr2=0x7f21380b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 shutdown_connections 2023-10-06T03:09:05.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.611+0000 7f213c995700 1 -- 172.21.15.92:0/1211195752 wait complete. 2023-10-06T03:09:05.628 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:09:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.750+0000 7f832d925700 1 Processor -- start 2023-10-06T03:09:05.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.750+0000 7f832d925700 1 -- start start 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.750+0000 7f832d925700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8328159890 0x7f832815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.750+0000 7f832d925700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8328159c60 0x7f832815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f832d925700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832815a160 con 0x7f832815db10 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f832d925700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83280d13a0 con 0x7f8328159c60 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f832d925700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832815e140 con 0x7f8328159890 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8328159c60 0x7f832815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:05.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8328159c60 0x7f832815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52784/0 (socket says 172.21.15.92:52784) 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 -- 172.21.15.92:0/2237793054 learned_addr learned my addr 172.21.15.92:0/2237793054 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f8326ffd700 1 --2- 172.21.15.92:0/2237793054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8328159890 0x7f832815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 -- 172.21.15.92:0/2237793054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8328159890 msgr2=0x7f832815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 --2- 172.21.15.92:0/2237793054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8328159890 0x7f832815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 -- 172.21.15.92:0/2237793054 >> v1:172.21.15.136:6789/0 conn(0x7f832815db10 legacy=0x7f8328069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 -- 172.21.15.92:0/2237793054 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832815a3d0 con 0x7f8328159c60 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f83267fc700 1 --2- 172.21.15.92:0/2237793054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159c60 0x7f832815d5d0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8310009f20 tx=0x7f8310009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2104ec1a903e7454 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f8325ffb700 1 -- 172.21.15.92:0/2237793054 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831000e070 con 0x7f8328159c60 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f8325ffb700 1 -- 172.21.15.92:0/2237793054 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831000ba80 con 0x7f8328159c60 2023-10-06T03:09:05.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.751+0000 7f8325ffb700 1 -- 172.21.15.92:0/2237793054 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83100092e0 con 0x7f8328159c60 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 -- 172.21.15.92:0/2237793054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159c60 msgr2=0x7f832815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 --2- 172.21.15.92:0/2237793054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159c60 0x7f832815d5d0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8310009f20 tx=0x7f8310009840 comp rx=0 tx=0).stop 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 -- 172.21.15.92:0/2237793054 shutdown_connections 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 --2- 172.21.15.92:0/2237793054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8328159890 0x7f832815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 --2- 172.21.15.92:0/2237793054 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159c60 0x7f832815d5d0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 -- 172.21.15.92:0/2237793054 >> 172.21.15.92:0/2237793054 conn(0x7f8328068e20 msgr2=0x7f83280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 -- 172.21.15.92:0/2237793054 shutdown_connections 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.752+0000 7f832d925700 1 -- 172.21.15.92:0/2237793054 wait complete. 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.753+0000 7f832d925700 1 Processor -- start 2023-10-06T03:09:05.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.753+0000 7f832d925700 1 -- start start 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f832d925700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 0x7f8328158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f832d925700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8328159c60 0x7f832816a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f832d925700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832815db10 0x7f83280c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f832d925700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328153e40 con 0x7f8328159890 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f832d925700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83280c8800 con 0x7f8328159c60 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f832d925700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832816aea0 con 0x7f832815db10 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 0x7f8328158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 0x7f8328158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52794/0 (socket says 172.21.15.92:52794) 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 -- 172.21.15.92:0/562251697 learned_addr learned my addr 172.21.15.92:0/562251697 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f83277fe700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832815db10 0x7f83280c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:05.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f83267fc700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8328159c60 0x7f832816a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 -- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832815db10 msgr2=0x7f83280c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832815db10 0x7f83280c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 -- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8328159c60 msgr2=0x7f832816a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8328159c60 0x7f832816a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 -- 172.21.15.92:0/562251697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8310009ae0 con 0x7f8328159890 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.754+0000 7f8326ffd700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 0x7f8328158bd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f832000d8c0 tx=0x7f8320008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.755+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000de60 con 0x7f8328159890 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.755+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832000b780 con 0x7f8328159890 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.755+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000bb20 con 0x7f8328159890 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.755+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83280c7bf0 con 0x7f8328159890 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.755+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83280c81b0 con 0x7f8328159890 2023-10-06T03:09:05.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.756+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832804c3c0 con 0x7f8328159890 2023-10-06T03:09:05.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.756+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f832001e460 con 0x7f8328159890 2023-10-06T03:09:05.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.756+0000 7f831effd700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308067320 0x7f83080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:05.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.756+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8320085460 con 0x7f8328159890 2023-10-06T03:09:05.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.756+0000 7f83267fc700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308067320 0x7f83080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:05.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.757+0000 7f83267fc700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308067320 0x7f83080697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f832815a090 tx=0x7f8310018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:05.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.760+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8320051f90 con 0x7f8328159890 2023-10-06T03:09:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:05.989+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f83280c6de0 con 0x7f8328159890 2023-10-06T03:09:06.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.613+0000 7f831effd700 1 -- 172.21.15.92:0/562251697 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83200555b0 con 0x7f8328159890 2023-10-06T03:09:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308067320 msgr2=0x7f83080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308067320 0x7f83080697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f832815a090 tx=0x7f8310018000 comp rx=0 tx=0).stop 2023-10-06T03:09:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 msgr2=0x7f8328158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 0x7f8328158bd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f832000d8c0 tx=0x7f8320008040 comp rx=0 tx=0).stop 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 shutdown_connections 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8308067320 0x7f83080697d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8328159890 0x7f8328158bd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f832815db10 0x7f83280c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 --2- 172.21.15.92:0/562251697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8328159c60 0x7f832816a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 >> 172.21.15.92:0/562251697 conn(0x7f8328068e20 msgr2=0x7f8328069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 shutdown_connections 2023-10-06T03:09:06.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.615+0000 7f832d925700 1 -- 172.21.15.92:0/562251697 wait complete. 2023-10-06T03:09:06.631 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.747+0000 7f76dfe71700 1 Processor -- start 2023-10-06T03:09:06.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.747+0000 7f76dfe71700 1 -- start start 2023-10-06T03:09:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dfe71700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76d8069560 0x7f76d80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dfe71700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d8069ee0 con 0x7f76d8159c60 2023-10-06T03:09:06.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dfe71700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d815a160 con 0x7f76d8159890 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dfe71700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80d13a0 con 0x7f76d8069560 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76de40e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76d8069560 0x7f76d80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dd40c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f76d8159c60 0x7f76d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40452/0 (socket says 172.21.15.92:40452) 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76de40e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f76d8069560 0x7f76d80699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52806/0 (socket says 172.21.15.92:52806) 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dd40c700 1 -- 172.21.15.92:0/2310093832 learned_addr learned my addr 172.21.15.92:0/2310093832 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76de40e700 1 -- 172.21.15.92:0/2310093832 >> v1:172.21.15.136:6789/0 conn(0x7f76d8159890 legacy=0x7f76d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76de40e700 1 -- 172.21.15.92:0/2310093832 >> v1:172.21.15.92:6789/0 conn(0x7f76d8159c60 legacy=0x7f76d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76de40e700 1 -- 172.21.15.92:0/2310093832 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d815e140 con 0x7f76d8069560 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.748+0000 7f76dcc0b700 1 -- 172.21.15.92:0/2310093832 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642279395 0 0) 0x7f76d8069ee0 con 0x7f76d8159c60 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76de40e700 1 --2- 172.21.15.92:0/2310093832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8069560 0x7f76d80699a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098e0 tx=0x7f76d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31cbcc5daeb5caf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dcc0b700 1 -- 172.21.15.92:0/2310093832 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0009e80 con 0x7f76d8069560 2023-10-06T03:09:06.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dcc0b700 1 -- 172.21.15.92:0/2310093832 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d000b820 con 0x7f76d8069560 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dcc0b700 1 -- 172.21.15.92:0/2310093832 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d000bba0 con 0x7f76d8069560 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 -- 172.21.15.92:0/2310093832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8069560 msgr2=0x7f76d80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2310093832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8069560 0x7f76d80699a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f76d00098e0 tx=0x7f76d00091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 -- 172.21.15.92:0/2310093832 shutdown_connections 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2310093832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8069560 0x7f76d80699a0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 -- 172.21.15.92:0/2310093832 >> 172.21.15.92:0/2310093832 conn(0x7f76d8068e20 msgr2=0x7f76d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 -- 172.21.15.92:0/2310093832 shutdown_connections 2023-10-06T03:09:06.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.749+0000 7f76dfe71700 1 -- 172.21.15.92:0/2310093832 wait complete. 2023-10-06T03:09:06.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.750+0000 7f76dfe71700 1 Processor -- start 2023-10-06T03:09:06.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.750+0000 7f76dfe71700 1 -- start start 2023-10-06T03:09:06.756 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dfe71700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8069560 0x7f76d816dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dfe71700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159890 0x7f76d80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dfe71700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 0x7f76d8144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dfe71700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c78a0 con 0x7f76d8159c60 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dfe71700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d8144780 con 0x7f76d8069560 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dfe71700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c7af0 con 0x7f76d8159890 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 0x7f76d8144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 0x7f76d8144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52820/0 (socket says 172.21.15.92:52820) 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 -- 172.21.15.92:0/2018380599 learned_addr learned my addr 172.21.15.92:0/2018380599 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76dd40c700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159890 0x7f76d80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76ddc0d700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8069560 0x7f76d816dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 -- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159890 msgr2=0x7f76d80ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159890 0x7f76d80ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 -- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8069560 msgr2=0x7f76d816dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.751+0000 7f76de40e700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8069560 0x7f76d816dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76de40e700 1 -- 172.21.15.92:0/2018380599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0009420 con 0x7f76d8159c60 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76de40e700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 0x7f76d8144240 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f76d00093f0 tx=0x7f76d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d001b450 con 0x7f76d8159c60 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0020070 con 0x7f76d8159c60 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80c7dd0 con 0x7f76d8159c60 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d001b800 con 0x7f76d8159c60 2023-10-06T03:09:06.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.752+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d80cea20 con 0x7f76d8159c60 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.753+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d804c3c0 con 0x7f76d8159c60 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.753+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f76d0022030 con 0x7f76d8159c60 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.754+0000 7f76cdffb700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76bc067320 0x7f76bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.754+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f76d0086f20 con 0x7f76d8159c60 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.754+0000 7f76ddc0d700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76bc067320 0x7f76bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.754+0000 7f76ddc0d700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76bc067320 0x7f76bc0697d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009fd0 tx=0x7f76d4009620 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:06.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.758+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76d0053b60 con 0x7f76d8159c60 2023-10-06T03:09:06.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.980+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76d804c3c0 con 0x7f76d8159c60 2023-10-06T03:09:06.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.981+0000 7f76cdffb700 1 -- 172.21.15.92:0/2018380599 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f76d0057180 con 0x7f76d8159c60 2023-10-06T03:09:06.983 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:06.983 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:06.608148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:06.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76bc067320 msgr2=0x7f76bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76bc067320 0x7f76bc0697d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009fd0 tx=0x7f76d4009620 comp rx=0 tx=0).stop 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 msgr2=0x7f76d8144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 0x7f76d8144240 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f76d00093f0 tx=0x7f76d0016000 comp rx=0 tx=0).stop 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 shutdown_connections 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76bc067320 0x7f76bc0697d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76d8159c60 0x7f76d8144240 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8069560 0x7f76d816dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 --2- 172.21.15.92:0/2018380599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159890 0x7f76d80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:06.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 >> 172.21.15.92:0/2018380599 conn(0x7f76d8068e20 msgr2=0x7f76d8152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:06.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 shutdown_connections 2023-10-06T03:09:06.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:06.983+0000 7f76dfe71700 1 -- 172.21.15.92:0/2018380599 wait complete. 2023-10-06T03:09:06.987 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 94 2023-10-06T03:09:06.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.109+0000 7ff60db97700 1 Processor -- start 2023-10-06T03:09:07.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.109+0000 7ff60db97700 1 -- start start 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff60db97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff60806c0a0 0x7ff6080caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff60db97700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60815d7e0 con 0x7ff60815df50 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff60db97700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60815dac0 con 0x7ff60806bcd0 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff60db97700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60815aa30 con 0x7ff60806c0a0 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff606ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff60806c0a0 0x7ff6080caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff607fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff60815df50 0x7ff6080c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40456/0 (socket says 172.21.15.92:40456) 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.110+0000 7ff607fff700 1 -- 172.21.15.92:0/296126227 learned_addr learned my addr 172.21.15.92:0/296126227 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff606ffd700 1 -- 172.21.15.92:0/296126227 >> v1:172.21.15.136:6789/0 conn(0x7ff60806bcd0 legacy=0x7ff60815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:07.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff606ffd700 1 -- 172.21.15.92:0/296126227 >> v1:172.21.15.92:6789/0 conn(0x7ff60815df50 legacy=0x7ff6080c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff606ffd700 1 -- 172.21.15.92:0/296126227 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60815ad10 con 0x7ff60806c0a0 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff606ffd700 1 --2- 172.21.15.92:0/296126227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60806c0a0 0x7ff6080caa90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0098e0 tx=0x7ff5fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c120078738ca50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff6067fc700 1 -- 172.21.15.92:0/296126227 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc009e80 con 0x7ff60806c0a0 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff6067fc700 1 -- 172.21.15.92:0/296126227 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc00b820 con 0x7ff60806c0a0 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff6067fc700 1 -- 172.21.15.92:0/296126227 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00bba0 con 0x7ff60806c0a0 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 -- 172.21.15.92:0/296126227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60806c0a0 msgr2=0x7ff6080caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 --2- 172.21.15.92:0/296126227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60806c0a0 0x7ff6080caa90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0098e0 tx=0x7ff5fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 -- 172.21.15.92:0/296126227 shutdown_connections 2023-10-06T03:09:07.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 --2- 172.21.15.92:0/296126227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60806c0a0 0x7ff6080caa90 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 -- 172.21.15.92:0/296126227 >> 172.21.15.92:0/296126227 conn(0x7ff6080ce600 msgr2=0x7ff6080cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 -- 172.21.15.92:0/296126227 shutdown_connections 2023-10-06T03:09:07.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.111+0000 7ff60db97700 1 -- 172.21.15.92:0/296126227 wait complete. 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 Processor -- start 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 -- start start 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60806bcd0 0x7ff60816ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff60806c0a0 0x7ff6080b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 0x7ff60815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608153bc0 con 0x7ff60815df50 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608153020 con 0x7ff60806c0a0 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.113+0000 7ff60db97700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60816d3a0 con 0x7ff60806bcd0 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 0x7ff60815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 0x7ff60815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52834/0 (socket says 172.21.15.92:52834) 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 -- 172.21.15.92:0/1302838231 learned_addr learned my addr 172.21.15.92:0/1302838231 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff606ffd700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff60806c0a0 0x7ff6080b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff6077fe700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60806bcd0 0x7ff60816ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 -- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60806bcd0 msgr2=0x7ff60816ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60806bcd0 0x7ff60816ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 -- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff60806c0a0 msgr2=0x7ff6080b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff60806c0a0 0x7ff6080b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 -- 172.21.15.92:0/1302838231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc009420 con 0x7ff60815df50 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff607fff700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 0x7ff60815b8b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c560 tx=0x7ff600007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600005b30 con 0x7ff60815df50 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600005c90 con 0x7ff60815df50 2023-10-06T03:09:07.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000072e0 con 0x7ff60815df50 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff608153290 con 0x7ff60815df50 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.114+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff608153850 con 0x7ff60815df50 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.115+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff608153d30 con 0x7ff60815df50 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.116+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff600007690 con 0x7ff60815df50 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.116+0000 7ff5ef7fe700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4067320 0x7ff5f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.116+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff600083f70 con 0x7ff60815df50 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.116+0000 7ff6077fe700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4067320 0x7ff5f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.116+0000 7ff6077fe700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4067320 0x7ff5f40697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8007560 tx=0x7ff5f8009760 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.120+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff60004b5a0 con 0x7ff60815df50 2023-10-06T03:09:07.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.304+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff60804c3c0 con 0x7ff60815df50 2023-10-06T03:09:07.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.304+0000 7ff5ef7fe700 1 -- 172.21.15.92:0/1302838231 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16880 (secure 0 0 0) 0x7ff600017020 con 0x7ff60815df50 2023-10-06T03:09:07.306 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:07.306 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":140,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:04.586383+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:01.925339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4067320 msgr2=0x7ff5f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4067320 0x7ff5f40697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8007560 tx=0x7ff5f8009760 comp rx=0 tx=0).stop 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 msgr2=0x7ff60815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 0x7ff60815b8b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c560 tx=0x7ff600007f60 comp rx=0 tx=0).stop 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 shutdown_connections 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff5f4067320 0x7ff5f40697d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff60815df50 0x7ff60815b8b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60806bcd0 0x7ff60816ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 --2- 172.21.15.92:0/1302838231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff60806c0a0 0x7ff6080b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 >> 172.21.15.92:0/1302838231 conn(0x7ff6080ce600 msgr2=0x7ff6080c3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 shutdown_connections 2023-10-06T03:09:07.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.307+0000 7ff60db97700 1 -- 172.21.15.92:0/1302838231 wait complete. 2023-10-06T03:09:07.321 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:09:07.321 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:09:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.433+0000 7efc043d6700 1 Processor -- start 2023-10-06T03:09:07.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.433+0000 7efc043d6700 1 -- start start 2023-10-06T03:09:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.434+0000 7efc043d6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efbfc06bcd0 0x7efbfc140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.434+0000 7efc043d6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc15c2a0 con 0x7efbfc0c9510 2023-10-06T03:09:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.434+0000 7efc043d6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc15b690 con 0x7efbfc06c0a0 2023-10-06T03:09:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.434+0000 7efc043d6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc0c9900 con 0x7efbfc06bcd0 2023-10-06T03:09:07.442 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.434+0000 7efc02973700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efbfc0c9510 0x7efbfc0cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40458/0 (socket says 172.21.15.92:40458) 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.434+0000 7efc02973700 1 -- 172.21.15.92:0/3906689563 learned_addr learned my addr 172.21.15.92:0/3906689563 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 --2- 172.21.15.92:0/3906689563 >> v2:172.21.15.136:3300/0 conn(0x7efbfc06bcd0 0x7efbfc140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919987333 0 0) 0x7efbfc15c2a0 con 0x7efbfc0c9510 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbe8003570 con 0x7efbfc0c9510 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1242103353 0 0) 0x7efbe8003570 con 0x7efbfc0c9510 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 >> v2:172.21.15.136:3300/0 conn(0x7efbfc06bcd0 msgr2=0x7efbfc140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:07.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 --2- 172.21.15.92:0/3906689563 >> v2:172.21.15.136:3300/0 conn(0x7efbfc06bcd0 0x7efbfc140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 >> v1:172.21.15.136:6789/0 conn(0x7efbfc06c0a0 legacy=0x7efbfc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc15b970 con 0x7efbfc0c9510 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7efbf4001470 con 0x7efbfc0c9510 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 >> v1:172.21.15.92:6789/0 conn(0x7efbfc0c9510 legacy=0x7efbfc0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbe8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbe8004810 0x7efbe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbe80051c0 0x7efbe800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0c9900 con 0x7efbe8003a60 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbe800c0d0 con 0x7efbe8004810 2023-10-06T03:09:07.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbe800c290 con 0x7efbe80051c0 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbe8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc01971700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbe8004810 0x7efbe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02973700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbe80051c0 0x7efbe800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 -- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbe80051c0 msgr2=0x7efbe800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbe80051c0 0x7efbe800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 -- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbe8004810 msgr2=0x7efbe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbe8004810 0x7efbe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02172700 1 -- 172.21.15.92:0/3906689563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe8003570 con 0x7efbe8003a60 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.435+0000 7efc02973700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbe80051c0 0x7efbe800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:07.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc02973700 1 -- 172.21.15.92:0/3906689563 reap_dead start 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc02172700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbe8003e70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7efbf800e210 tx=0x7efbf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba9189406784896 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800eac0 con 0x7efbe8003a60 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf800f040 con 0x7efbe8003a60 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc01170700 1 -- 172.21.15.92:0/3906689563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf800ee70 con 0x7efbe8003a60 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 -- 172.21.15.92:0/3906689563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 msgr2=0x7efbe8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbe8003e70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7efbf800e210 tx=0x7efbf800b040 comp rx=0 tx=0).stop 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 -- 172.21.15.92:0/3906689563 shutdown_connections 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 --2- 172.21.15.92:0/3906689563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbe8003e70 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 -- 172.21.15.92:0/3906689563 >> 172.21.15.92:0/3906689563 conn(0x7efbfc0ce600 msgr2=0x7efbfc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 -- 172.21.15.92:0/3906689563 shutdown_connections 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.436+0000 7efc043d6700 1 -- 172.21.15.92:0/3906689563 wait complete. 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.437+0000 7efc043d6700 1 Processor -- start 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 -- start start 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbfc13f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbfc06add0 0x7efbfc154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc06b240 0x7efbfc154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0b2290 con 0x7efbe8003a60 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc13f860 con 0x7efbfc06b240 2023-10-06T03:09:07.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc043d6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc06a7c0 con 0x7efbfc06add0 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc02172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbfc13f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc02172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbfc13f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52858/0 (socket says 172.21.15.92:52858) 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc02172700 1 -- 172.21.15.92:0/4126918077 learned_addr learned my addr 172.21.15.92:0/4126918077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc01971700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbfc06add0 0x7efbfc154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.438+0000 7efc02973700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc06b240 0x7efbfc154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc02172700 1 -- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbfc06add0 msgr2=0x7efbfc154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc02172700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbfc06add0 0x7efbfc154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc02172700 1 -- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc06b240 msgr2=0x7efbfc154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc02172700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc06b240 0x7efbfc154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc02172700 1 -- 172.21.15.92:0/4126918077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0b13a0 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc02172700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbfc13f320 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7efbe800c0a0 tx=0x7efbf801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf801d420 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf801d580 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc0b1680 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf801d940 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.439+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc0b1c40 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.440+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc0b2570 con 0x7efbe8003a60 2023-10-06T03:09:07.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.440+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efbf8003040 con 0x7efbe8003a60 2023-10-06T03:09:07.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.441+0000 7efbf27fc700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efbe0067320 0x7efbe00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.441+0000 7efc01971700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efbe0067320 0x7efbe00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.441+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efbf80871f0 con 0x7efbe8003a60 2023-10-06T03:09:07.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.441+0000 7efc01971700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efbe0067320 0x7efbe00697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efbfc06d2c0 tx=0x7efbec007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.445+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efbf8053d80 con 0x7efbe8003a60 2023-10-06T03:09:07.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.626+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efbfc04c3c0 con 0x7efbe0067320 2023-10-06T03:09:07.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.634+0000 7efbf27fc700 1 -- 172.21.15.92:0/4126918077 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311869 (secure 0 0 0) 0x7efbfc04c3c0 con 0x7efbe0067320 2023-10-06T03:09:07.640 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.637+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efbe0067320 msgr2=0x7efbe00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.637+0000 7efc043d6700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efbe0067320 0x7efbe00697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efbfc06d2c0 tx=0x7efbec007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.637+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 msgr2=0x7efbfc13f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.637+0000 7efc043d6700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbfc13f320 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7efbe800c0a0 tx=0x7efbf801a000 comp rx=0 tx=0).stop 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 shutdown_connections 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efbe0067320 0x7efbe00697d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efbe8003a60 0x7efbfc13f320 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbfc06b240 0x7efbfc154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 --2- 172.21.15.92:0/4126918077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbfc06add0 0x7efbfc154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 >> 172.21.15.92:0/4126918077 conn(0x7efbfc0ce600 msgr2=0x7efbfc155d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 shutdown_connections 2023-10-06T03:09:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.638+0000 7efc043d6700 1 -- 172.21.15.92:0/4126918077 wait complete. 2023-10-06T03:09:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:09:07.651 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2023-10-06T03:09:07.273824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425952,"kb_used_data":14880,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548768,"statfs":{"total":773094113280,"available":770609938432,"internally_reserved":0,"allocated":15237120,"data_stored":7786406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703405"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604866+0000","last_change":"2023-10-06T03:09:03.604866+0000","last_active":"2023-10-06T03:09:03.604866+0000","last_peered":"2023-10-06T03:09:03.604866+0000","last_clean":"2023-10-06T03:09:03.604866+0000","last_became_active":"2023-10-06T03:09:03.604298+0000","last_became_peered":"2023-10-06T03:09:03.604298+0000","last_unstale":"2023-10-06T03:09:03.604866+0000","last_undegraded":"2023-10-06T03:09:03.604866+0000","last_fullsized":"2023-10-06T03:09:03.604866+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:39.571557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604542+0000","last_change":"2023-10-06T03:09:03.604542+0000","last_active":"2023-10-06T03:09:03.604542+0000","last_peered":"2023-10-06T03:09:03.604542+0000","last_clean":"2023-10-06T03:09:03.604542+0000","last_became_active":"2023-10-06T03:09:03.603295+0000","last_became_peered":"2023-10-06T03:09:03.603295+0000","last_unstale":"2023-10-06T03:09:03.604542+0000","last_undegraded":"2023-10-06T03:09:03.604542+0000","last_fullsized":"2023-10-06T03:09:03.604542+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:11.961093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611900+0000","last_change":"2023-10-06T03:09:03.611900+0000","last_active":"2023-10-06T03:09:03.611900+0000","last_peered":"2023-10-06T03:09:03.611900+0000","last_clean":"2023-10-06T03:09:03.611900+0000","last_became_active":"2023-10-06T03:09:03.611635+0000","last_became_peered":"2023-10-06T03:09:03.611635+0000","last_unstale":"2023-10-06T03:09:03.611900+0000","last_undegraded":"2023-10-06T03:09:03.611900+0000","last_fullsized":"2023-10-06T03:09:03.611900+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:29.398296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601263+0000","last_change":"2023-10-06T03:09:03.601263+0000","last_active":"2023-10-06T03:09:03.601263+0000","last_peered":"2023-10-06T03:09:03.601263+0000","last_clean":"2023-10-06T03:09:03.601263+0000","last_became_active":"2023-10-06T03:09:03.601003+0000","last_became_peered":"2023-10-06T03:09:03.601003+0000","last_unstale":"2023-10-06T03:09:03.601263+0000","last_undegraded":"2023-10-06T03:09:03.601263+0000","last_fullsized":"2023-10-06T03:09:03.601263+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:52.782664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602930+0000","last_change":"2023-10-06T03:09:03.602930+0000","last_active":"2023-10-06T03:09:03.602930+0000","last_peered":"2023-10-06T03:09:03.602930+0000","last_clean":"2023-10-06T03:09:03.602930+0000","last_became_active":"2023-10-06T03:09:03.602680+0000","last_became_peered":"2023-10-06T03:09:03.602680+0000","last_unstale":"2023-10-06T03:09:03.602930+0000","last_undegraded":"2023-10-06T03:09:03.602930+0000","last_fullsized":"2023-10-06T03:09:03.602930+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:37.782342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601014+0000","last_change":"2023-10-06T03:09:03.601014+0000","last_active":"2023-10-06T03:09:03.601014+0000","last_peered":"2023-10-06T03:09:03.601014+0000","last_clean":"2023-10-06T03:09:03.601014+0000","last_became_active":"2023-10-06T03:09:03.600745+0000","last_became_peered":"2023-10-06T03:09:03.600745+0000","last_unstale":"2023-10-06T03:09:03.601014+0000","last_undegraded":"2023-10-06T03:09:03.601014+0000","last_fullsized":"2023-10-06T03:09:03.601014+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:41.123215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604567+0000","last_change":"2023-10-06T03:09:03.604567+0000","last_active":"2023-10-06T03:09:03.604567+0000","last_peered":"2023-10-06T03:09:03.604567+0000","last_clean":"2023-10-06T03:09:03.604567+0000","last_became_active":"2023-10-06T03:09:03.604296+0000","last_became_peered":"2023-10-06T03:09:03.604296+0000","last_unstale":"2023-10-06T03:09:03.604567+0000","last_undegraded":"2023-10-06T03:09:03.604567+0000","last_fullsized":"2023-10-06T03:09:03.604567+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:36.275549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.608958+0000","last_change":"2023-10-06T03:09:03.608958+0000","last_active":"2023-10-06T03:09:03.608958+0000","last_peered":"2023-10-06T03:09:03.608958+0000","last_clean":"2023-10-06T03:09:03.608958+0000","last_became_active":"2023-10-06T03:09:03.608453+0000","last_became_peered":"2023-10-06T03:09:03.608453+0000","last_unstale":"2023-10-06T03:09:03.608958+0000","last_undegraded":"2023-10-06T03:09:03.608958+0000","last_fullsized":"2023-10-06T03:09:03.608958+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:55.433899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.608393+0000","last_change":"2023-10-06T03:09:03.608393+0000","last_active":"2023-10-06T03:09:03.608393+0000","last_peered":"2023-10-06T03:09:03.608393+0000","last_clean":"2023-10-06T03:09:03.608393+0000","last_became_active":"2023-10-06T03:09:03.608078+0000","last_became_peered":"2023-10-06T03:09:03.608078+0000","last_unstale":"2023-10-06T03:09:03.608393+0000","last_undegraded":"2023-10-06T03:09:03.608393+0000","last_fullsized":"2023-10-06T03:09:03.608393+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:42.997705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603336+0000","last_change":"2023-10-06T03:09:03.603336+0000","last_active":"2023-10-06T03:09:03.603336+0000","last_peered":"2023-10-06T03:09:03.603336+0000","last_clean":"2023-10-06T03:09:03.603336+0000","last_became_active":"2023-10-06T03:09:03.603089+0000","last_became_peered":"2023-10-06T03:09:03.603089+0000","last_unstale":"2023-10-06T03:09:03.603336+0000","last_undegraded":"2023-10-06T03:09:03.603336+0000","last_fullsized":"2023-10-06T03:09:03.603336+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:09.080161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602872+0000","last_change":"2023-10-06T03:09:03.602872+0000","last_active":"2023-10-06T03:09:03.602872+0000","last_peered":"2023-10-06T03:09:03.602872+0000","last_clean":"2023-10-06T03:09:03.602872+0000","last_became_active":"2023-10-06T03:09:03.602617+0000","last_became_peered":"2023-10-06T03:09:03.602617+0000","last_unstale":"2023-10-06T03:09:03.602872+0000","last_undegraded":"2023-10-06T03:09:03.602872+0000","last_fullsized":"2023-10-06T03:09:03.602872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:46.371480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603080+0000","last_change":"2023-10-06T03:09:03.603080+0000","last_active":"2023-10-06T03:09:03.603080+0000","last_peered":"2023-10-06T03:09:03.603080+0000","last_clean":"2023-10-06T03:09:03.603080+0000","last_became_active":"2023-10-06T03:09:03.602820+0000","last_became_peered":"2023-10-06T03:09:03.602820+0000","last_unstale":"2023-10-06T03:09:03.603080+0000","last_undegraded":"2023-10-06T03:09:03.603080+0000","last_fullsized":"2023-10-06T03:09:03.603080+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:32.238086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603751+0000","last_change":"2023-10-06T03:09:03.603751+0000","last_active":"2023-10-06T03:09:03.603751+0000","last_peered":"2023-10-06T03:09:03.603751+0000","last_clean":"2023-10-06T03:09:03.603751+0000","last_became_active":"2023-10-06T03:09:03.603448+0000","last_became_peered":"2023-10-06T03:09:03.603448+0000","last_unstale":"2023-10-06T03:09:03.603751+0000","last_undegraded":"2023-10-06T03:09:03.603751+0000","last_fullsized":"2023-10-06T03:09:03.603751+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:31.713110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604871+0000","last_change":"2023-10-06T03:09:03.604871+0000","last_active":"2023-10-06T03:09:03.604871+0000","last_peered":"2023-10-06T03:09:03.604871+0000","last_clean":"2023-10-06T03:09:03.604871+0000","last_became_active":"2023-10-06T03:09:03.603994+0000","last_became_peered":"2023-10-06T03:09:03.603994+0000","last_unstale":"2023-10-06T03:09:03.604871+0000","last_undegraded":"2023-10-06T03:09:03.604871+0000","last_fullsized":"2023-10-06T03:09:03.604871+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:16.765052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600174+0000","last_change":"2023-10-06T03:09:03.600174+0000","last_active":"2023-10-06T03:09:03.600174+0000","last_peered":"2023-10-06T03:09:03.600174+0000","last_clean":"2023-10-06T03:09:03.600174+0000","last_became_active":"2023-10-06T03:09:03.599861+0000","last_became_peered":"2023-10-06T03:09:03.599861+0000","last_unstale":"2023-10-06T03:09:03.600174+0000","last_undegraded":"2023-10-06T03:09:03.600174+0000","last_fullsized":"2023-10-06T03:09:03.600174+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:16.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.594453+0000","last_change":"2023-10-06T03:09:03.594453+0000","last_active":"2023-10-06T03:09:03.594453+0000","last_peered":"2023-10-06T03:09:03.594453+0000","last_clean":"2023-10-06T03:09:03.594453+0000","last_became_active":"2023-10-06T03:09:03.593990+0000","last_became_peered":"2023-10-06T03:09:03.593990+0000","last_unstale":"2023-10-06T03:09:03.594453+0000","last_undegraded":"2023-10-06T03:09:03.594453+0000","last_fullsized":"2023-10-06T03:09:03.594453+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:32.238086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603499+0000","last_change":"2023-10-06T03:09:03.603499+0000","last_active":"2023-10-06T03:09:03.603499+0000","last_peered":"2023-10-06T03:09:03.603499+0000","last_clean":"2023-10-06T03:09:03.603499+0000","last_became_active":"2023-10-06T03:09:03.603246+0000","last_became_peered":"2023-10-06T03:09:03.603246+0000","last_unstale":"2023-10-06T03:09:03.603499+0000","last_undegraded":"2023-10-06T03:09:03.603499+0000","last_fullsized":"2023-10-06T03:09:03.603499+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:07.456293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605109+0000","last_change":"2023-10-06T03:09:03.605109+0000","last_active":"2023-10-06T03:09:03.605109+0000","last_peered":"2023-10-06T03:09:03.605109+0000","last_clean":"2023-10-06T03:09:03.605109+0000","last_became_active":"2023-10-06T03:09:03.604587+0000","last_became_peered":"2023-10-06T03:09:03.604587+0000","last_unstale":"2023-10-06T03:09:03.605109+0000","last_undegraded":"2023-10-06T03:09:03.605109+0000","last_fullsized":"2023-10-06T03:09:03.605109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:26.740899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.606935+0000","last_change":"2023-10-06T03:09:03.606935+0000","last_active":"2023-10-06T03:09:03.606935+0000","last_peered":"2023-10-06T03:09:03.606935+0000","last_clean":"2023-10-06T03:09:03.606935+0000","last_became_active":"2023-10-06T03:09:03.606564+0000","last_became_peered":"2023-10-06T03:09:03.606564+0000","last_unstale":"2023-10-06T03:09:03.606935+0000","last_undegraded":"2023-10-06T03:09:03.606935+0000","last_fullsized":"2023-10-06T03:09:03.606935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:39.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603202+0000","last_change":"2023-10-06T03:09:03.603202+0000","last_active":"2023-10-06T03:09:03.603202+0000","last_peered":"2023-10-06T03:09:03.603202+0000","last_clean":"2023-10-06T03:09:03.603202+0000","last_became_active":"2023-10-06T03:09:03.602943+0000","last_became_peered":"2023-10-06T03:09:03.602943+0000","last_unstale":"2023-10-06T03:09:03.603202+0000","last_undegraded":"2023-10-06T03:09:03.603202+0000","last_fullsized":"2023-10-06T03:09:03.603202+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:53.646433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604741+0000","last_change":"2023-10-06T03:09:03.604741+0000","last_active":"2023-10-06T03:09:03.604741+0000","last_peered":"2023-10-06T03:09:03.604741+0000","last_clean":"2023-10-06T03:09:03.604741+0000","last_became_active":"2023-10-06T03:09:03.604498+0000","last_became_peered":"2023-10-06T03:09:03.604498+0000","last_unstale":"2023-10-06T03:09:03.604741+0000","last_undegraded":"2023-10-06T03:09:03.604741+0000","last_fullsized":"2023-10-06T03:09:03.604741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:09.933703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.607098+0000","last_change":"2023-10-06T03:09:03.607098+0000","last_active":"2023-10-06T03:09:03.607098+0000","last_peered":"2023-10-06T03:09:03.607098+0000","last_clean":"2023-10-06T03:09:03.607098+0000","last_became_active":"2023-10-06T03:09:03.606832+0000","last_became_peered":"2023-10-06T03:09:03.606832+0000","last_unstale":"2023-10-06T03:09:03.607098+0000","last_undegraded":"2023-10-06T03:09:03.607098+0000","last_fullsized":"2023-10-06T03:09:03.607098+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:26.740899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.609571+0000","last_change":"2023-10-06T03:09:03.609571+0000","last_active":"2023-10-06T03:09:03.609571+0000","last_peered":"2023-10-06T03:09:03.609571+0000","last_clean":"2023-10-06T03:09:03.609571+0000","last_became_active":"2023-10-06T03:09:03.609294+0000","last_became_peered":"2023-10-06T03:09:03.609294+0000","last_unstale":"2023-10-06T03:09:03.609571+0000","last_undegraded":"2023-10-06T03:09:03.609571+0000","last_fullsized":"2023-10-06T03:09:03.609571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:11.713416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603971+0000","last_change":"2023-10-06T03:09:03.603971+0000","last_active":"2023-10-06T03:09:03.603971+0000","last_peered":"2023-10-06T03:09:03.603971+0000","last_clean":"2023-10-06T03:09:03.603971+0000","last_became_active":"2023-10-06T03:09:03.603701+0000","last_became_peered":"2023-10-06T03:09:03.603701+0000","last_unstale":"2023-10-06T03:09:03.603971+0000","last_undegraded":"2023-10-06T03:09:03.603971+0000","last_fullsized":"2023-10-06T03:09:03.603971+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:31.713110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603858+0000","last_change":"2023-10-06T03:09:03.603858+0000","last_active":"2023-10-06T03:09:03.603858+0000","last_peered":"2023-10-06T03:09:03.603858+0000","last_clean":"2023-10-06T03:09:03.603858+0000","last_became_active":"2023-10-06T03:09:03.603607+0000","last_became_peered":"2023-10-06T03:09:03.603607+0000","last_unstale":"2023-10-06T03:09:03.603858+0000","last_undegraded":"2023-10-06T03:09:03.603858+0000","last_fullsized":"2023-10-06T03:09:03.603858+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:13.567731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600263+0000","last_change":"2023-10-06T03:09:03.600263+0000","last_active":"2023-10-06T03:09:03.600263+0000","last_peered":"2023-10-06T03:09:03.600263+0000","last_clean":"2023-10-06T03:09:03.600263+0000","last_became_active":"2023-10-06T03:09:03.599870+0000","last_became_peered":"2023-10-06T03:09:03.599870+0000","last_unstale":"2023-10-06T03:09:03.600263+0000","last_undegraded":"2023-10-06T03:09:03.600263+0000","last_fullsized":"2023-10-06T03:09:03.600263+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:54.428380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.612259+0000","last_change":"2023-10-06T03:09:03.612259+0000","last_active":"2023-10-06T03:09:03.612259+0000","last_peered":"2023-10-06T03:09:03.612259+0000","last_clean":"2023-10-06T03:09:03.612259+0000","last_became_active":"2023-10-06T03:09:03.612007+0000","last_became_peered":"2023-10-06T03:09:03.612007+0000","last_unstale":"2023-10-06T03:09:03.612259+0000","last_undegraded":"2023-10-06T03:09:03.612259+0000","last_fullsized":"2023-10-06T03:09:03.612259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:48.578908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605395+0000","last_change":"2023-10-06T03:09:03.605395+0000","last_active":"2023-10-06T03:09:03.605395+0000","last_peered":"2023-10-06T03:09:03.605395+0000","last_clean":"2023-10-06T03:09:03.605395+0000","last_became_active":"2023-10-06T03:09:03.605151+0000","last_became_peered":"2023-10-06T03:09:03.605151+0000","last_unstale":"2023-10-06T03:09:03.605395+0000","last_undegraded":"2023-10-06T03:09:03.605395+0000","last_fullsized":"2023-10-06T03:09:03.605395+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:06.153319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597827+0000","last_change":"2023-10-06T03:09:03.597827+0000","last_active":"2023-10-06T03:09:03.597827+0000","last_peered":"2023-10-06T03:09:03.597827+0000","last_clean":"2023-10-06T03:09:03.597827+0000","last_became_active":"2023-10-06T03:09:03.597568+0000","last_became_peered":"2023-10-06T03:09:03.597568+0000","last_unstale":"2023-10-06T03:09:03.597827+0000","last_undegraded":"2023-10-06T03:09:03.597827+0000","last_fullsized":"2023-10-06T03:09:03.597827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:41.161716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601195+0000","last_change":"2023-10-06T03:09:03.601195+0000","last_active":"2023-10-06T03:09:03.601195+0000","last_peered":"2023-10-06T03:09:03.601195+0000","last_clean":"2023-10-06T03:09:03.601195+0000","last_became_active":"2023-10-06T03:09:03.600604+0000","last_became_peered":"2023-10-06T03:09:03.600604+0000","last_unstale":"2023-10-06T03:09:03.601195+0000","last_undegraded":"2023-10-06T03:09:03.601195+0000","last_fullsized":"2023-10-06T03:09:03.601195+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:27.600148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605976+0000","last_change":"2023-10-06T03:09:03.605976+0000","last_active":"2023-10-06T03:09:03.605976+0000","last_peered":"2023-10-06T03:09:03.605976+0000","last_clean":"2023-10-06T03:09:03.605976+0000","last_became_active":"2023-10-06T03:09:03.602004+0000","last_became_peered":"2023-10-06T03:09:03.602004+0000","last_unstale":"2023-10-06T03:09:03.605976+0000","last_undegraded":"2023-10-06T03:09:03.605976+0000","last_fullsized":"2023-10-06T03:09:03.605976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:27.409021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599232+0000","last_change":"2023-10-06T03:09:03.599232+0000","last_active":"2023-10-06T03:09:03.599232+0000","last_peered":"2023-10-06T03:09:03.599232+0000","last_clean":"2023-10-06T03:09:03.599232+0000","last_became_active":"2023-10-06T03:09:03.598963+0000","last_became_peered":"2023-10-06T03:09:03.598963+0000","last_unstale":"2023-10-06T03:09:03.599232+0000","last_undegraded":"2023-10-06T03:09:03.599232+0000","last_fullsized":"2023-10-06T03:09:03.599232+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.523660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.608202+0000","last_change":"2023-10-06T03:09:03.608202+0000","last_active":"2023-10-06T03:09:03.608202+0000","last_peered":"2023-10-06T03:09:03.608202+0000","last_clean":"2023-10-06T03:09:03.608202+0000","last_became_active":"2023-10-06T03:09:03.607925+0000","last_became_peered":"2023-10-06T03:09:03.607925+0000","last_unstale":"2023-10-06T03:09:03.608202+0000","last_undegraded":"2023-10-06T03:09:03.608202+0000","last_fullsized":"2023-10-06T03:09:03.608202+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.596702+0000","last_change":"2023-10-06T03:09:02.594352+0000","last_active":"2023-10-06T03:09:02.596702+0000","last_peered":"2023-10-06T03:09:02.596702+0000","last_clean":"2023-10-06T03:09:02.596702+0000","last_became_active":"2023-10-06T03:09:02.594116+0000","last_became_peered":"2023-10-06T03:09:02.594116+0000","last_unstale":"2023-10-06T03:09:02.596702+0000","last_undegraded":"2023-10-06T03:09:02.596702+0000","last_fullsized":"2023-10-06T03:09:02.596702+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:26.956521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597024+0000","last_change":"2023-10-06T03:09:03.597024+0000","last_active":"2023-10-06T03:09:03.597024+0000","last_peered":"2023-10-06T03:09:03.597024+0000","last_clean":"2023-10-06T03:09:03.597024+0000","last_became_active":"2023-10-06T03:09:03.596730+0000","last_became_peered":"2023-10-06T03:09:03.596730+0000","last_unstale":"2023-10-06T03:09:03.597024+0000","last_undegraded":"2023-10-06T03:09:03.597024+0000","last_fullsized":"2023-10-06T03:09:03.597024+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:52.782664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.598730+0000","last_change":"2023-10-06T03:09:02.594280+0000","last_active":"2023-10-06T03:09:02.598730+0000","last_peered":"2023-10-06T03:09:02.598730+0000","last_clean":"2023-10-06T03:09:02.598730+0000","last_became_active":"2023-10-06T03:09:02.594043+0000","last_became_peered":"2023-10-06T03:09:02.594043+0000","last_unstale":"2023-10-06T03:09:02.598730+0000","last_undegraded":"2023-10-06T03:09:02.598730+0000","last_fullsized":"2023-10-06T03:09:02.598730+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:57.130837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601112+0000","last_change":"2023-10-06T03:09:03.601112+0000","last_active":"2023-10-06T03:09:03.601112+0000","last_peered":"2023-10-06T03:09:03.601112+0000","last_clean":"2023-10-06T03:09:03.601112+0000","last_became_active":"2023-10-06T03:09:03.599380+0000","last_became_peered":"2023-10-06T03:09:03.599380+0000","last_unstale":"2023-10-06T03:09:03.601112+0000","last_undegraded":"2023-10-06T03:09:03.601112+0000","last_fullsized":"2023-10-06T03:09:03.601112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"139'3","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611193+0000","last_change":"2023-10-06T03:09:02.596986+0000","last_active":"2023-10-06T03:09:03.611193+0000","last_peered":"2023-10-06T03:09:03.611193+0000","last_clean":"2023-10-06T03:09:03.611193+0000","last_became_active":"2023-10-06T03:09:02.596722+0000","last_became_peered":"2023-10-06T03:09:02.596722+0000","last_unstale":"2023-10-06T03:09:03.611193+0000","last_undegraded":"2023-10-06T03:09:03.611193+0000","last_fullsized":"2023-10-06T03:09:03.611193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:15.392027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605044+0000","last_change":"2023-10-06T03:09:03.605044+0000","last_active":"2023-10-06T03:09:03.605044+0000","last_peered":"2023-10-06T03:09:03.605044+0000","last_clean":"2023-10-06T03:09:03.605044+0000","last_became_active":"2023-10-06T03:09:03.604621+0000","last_became_peered":"2023-10-06T03:09:03.604621+0000","last_unstale":"2023-10-06T03:09:03.605044+0000","last_undegraded":"2023-10-06T03:09:03.605044+0000","last_fullsized":"2023-10-06T03:09:03.605044+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"139'6","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.616253+0000","last_change":"2023-10-06T03:09:02.583136+0000","last_active":"2023-10-06T03:09:03.616253+0000","last_peered":"2023-10-06T03:09:03.616253+0000","last_clean":"2023-10-06T03:09:03.616253+0000","last_became_active":"2023-10-06T03:09:02.582851+0000","last_became_peered":"2023-10-06T03:09:02.582851+0000","last_unstale":"2023-10-06T03:09:03.616253+0000","last_undegraded":"2023-10-06T03:09:03.616253+0000","last_fullsized":"2023-10-06T03:09:03.616253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:57.308794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604572+0000","last_change":"2023-10-06T03:09:03.604572+0000","last_active":"2023-10-06T03:09:03.604572+0000","last_peered":"2023-10-06T03:09:03.604572+0000","last_clean":"2023-10-06T03:09:03.604572+0000","last_became_active":"2023-10-06T03:09:03.604320+0000","last_became_peered":"2023-10-06T03:09:03.604320+0000","last_unstale":"2023-10-06T03:09:03.604572+0000","last_undegraded":"2023-10-06T03:09:03.604572+0000","last_fullsized":"2023-10-06T03:09:03.604572+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:06.646632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613143+0000","last_change":"2023-10-06T03:09:02.593978+0000","last_active":"2023-10-06T03:09:03.613143+0000","last_peered":"2023-10-06T03:09:03.613143+0000","last_clean":"2023-10-06T03:09:03.613143+0000","last_became_active":"2023-10-06T03:09:02.593670+0000","last_became_peered":"2023-10-06T03:09:02.593670+0000","last_unstale":"2023-10-06T03:09:03.613143+0000","last_undegraded":"2023-10-06T03:09:03.613143+0000","last_fullsized":"2023-10-06T03:09:03.613143+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:24.165492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605983+0000","last_change":"2023-10-06T03:09:03.605983+0000","last_active":"2023-10-06T03:09:03.605983+0000","last_peered":"2023-10-06T03:09:03.605983+0000","last_clean":"2023-10-06T03:09:03.605983+0000","last_became_active":"2023-10-06T03:09:03.605715+0000","last_became_peered":"2023-10-06T03:09:03.605715+0000","last_unstale":"2023-10-06T03:09:03.605983+0000","last_undegraded":"2023-10-06T03:09:03.605983+0000","last_fullsized":"2023-10-06T03:09:03.605983+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.598493+0000","last_change":"2023-10-06T03:09:02.588724+0000","last_active":"2023-10-06T03:09:02.598493+0000","last_peered":"2023-10-06T03:09:02.598493+0000","last_clean":"2023-10-06T03:09:02.598493+0000","last_became_active":"2023-10-06T03:09:02.588455+0000","last_became_peered":"2023-10-06T03:09:02.588455+0000","last_unstale":"2023-10-06T03:09:02.598493+0000","last_undegraded":"2023-10-06T03:09:02.598493+0000","last_fullsized":"2023-10-06T03:09:02.598493+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:28.520118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599135+0000","last_change":"2023-10-06T03:09:03.599135+0000","last_active":"2023-10-06T03:09:03.599135+0000","last_peered":"2023-10-06T03:09:03.599135+0000","last_clean":"2023-10-06T03:09:03.599135+0000","last_became_active":"2023-10-06T03:09:03.598858+0000","last_became_peered":"2023-10-06T03:09:03.598858+0000","last_unstale":"2023-10-06T03:09:03.599135+0000","last_undegraded":"2023-10-06T03:09:03.599135+0000","last_fullsized":"2023-10-06T03:09:03.599135+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.373485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.590302+0000","last_change":"2023-10-06T03:09:02.589625+0000","last_active":"2023-10-06T03:09:02.590302+0000","last_peered":"2023-10-06T03:09:02.590302+0000","last_clean":"2023-10-06T03:09:02.590302+0000","last_became_active":"2023-10-06T03:09:02.589381+0000","last_became_peered":"2023-10-06T03:09:02.589381+0000","last_unstale":"2023-10-06T03:09:02.590302+0000","last_undegraded":"2023-10-06T03:09:02.590302+0000","last_fullsized":"2023-10-06T03:09:02.590302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:47.574762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600682+0000","last_change":"2023-10-06T03:09:03.600682+0000","last_active":"2023-10-06T03:09:03.600682+0000","last_peered":"2023-10-06T03:09:03.600682+0000","last_clean":"2023-10-06T03:09:03.600682+0000","last_became_active":"2023-10-06T03:09:03.600177+0000","last_became_peered":"2023-10-06T03:09:03.600177+0000","last_unstale":"2023-10-06T03:09:03.600682+0000","last_undegraded":"2023-10-06T03:09:03.600682+0000","last_fullsized":"2023-10-06T03:09:03.600682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:19.560203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"139'8","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.615651+0000","last_change":"2023-10-06T03:09:02.587225+0000","last_active":"2023-10-06T03:09:03.615651+0000","last_peered":"2023-10-06T03:09:03.615651+0000","last_clean":"2023-10-06T03:09:03.615651+0000","last_became_active":"2023-10-06T03:09:02.586972+0000","last_became_peered":"2023-10-06T03:09:02.586972+0000","last_unstale":"2023-10-06T03:09:03.615651+0000","last_undegraded":"2023-10-06T03:09:03.615651+0000","last_fullsized":"2023-10-06T03:09:03.615651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:06.753575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604149+0000","last_change":"2023-10-06T03:09:03.604149+0000","last_active":"2023-10-06T03:09:03.604149+0000","last_peered":"2023-10-06T03:09:03.604149+0000","last_clean":"2023-10-06T03:09:03.604149+0000","last_became_active":"2023-10-06T03:09:03.601859+0000","last_became_peered":"2023-10-06T03:09:03.601859+0000","last_unstale":"2023-10-06T03:09:03.604149+0000","last_undegraded":"2023-10-06T03:09:03.604149+0000","last_fullsized":"2023-10-06T03:09:03.604149+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:31.358739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.590445+0000","last_change":"2023-10-06T03:09:02.590110+0000","last_active":"2023-10-06T03:09:03.590445+0000","last_peered":"2023-10-06T03:09:03.590445+0000","last_clean":"2023-10-06T03:09:03.590445+0000","last_became_active":"2023-10-06T03:09:02.589851+0000","last_became_peered":"2023-10-06T03:09:02.589851+0000","last_unstale":"2023-10-06T03:09:03.590445+0000","last_undegraded":"2023-10-06T03:09:03.590445+0000","last_fullsized":"2023-10-06T03:09:03.590445+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:22.528211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599172+0000","last_change":"2023-10-06T03:09:03.599172+0000","last_active":"2023-10-06T03:09:03.599172+0000","last_peered":"2023-10-06T03:09:03.599172+0000","last_clean":"2023-10-06T03:09:03.599172+0000","last_became_active":"2023-10-06T03:09:03.598930+0000","last_became_peered":"2023-10-06T03:09:03.598930+0000","last_unstale":"2023-10-06T03:09:03.599172+0000","last_undegraded":"2023-10-06T03:09:03.599172+0000","last_fullsized":"2023-10-06T03:09:03.599172+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:11.713416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613202+0000","last_change":"2023-10-06T03:09:02.588048+0000","last_active":"2023-10-06T03:09:03.613202+0000","last_peered":"2023-10-06T03:09:03.613202+0000","last_clean":"2023-10-06T03:09:03.613202+0000","last_became_active":"2023-10-06T03:09:02.587787+0000","last_became_peered":"2023-10-06T03:09:02.587787+0000","last_unstale":"2023-10-06T03:09:03.613202+0000","last_undegraded":"2023-10-06T03:09:03.613202+0000","last_fullsized":"2023-10-06T03:09:03.613202+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:38.567412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601862+0000","last_change":"2023-10-06T03:09:03.601862+0000","last_active":"2023-10-06T03:09:03.601862+0000","last_peered":"2023-10-06T03:09:03.601862+0000","last_clean":"2023-10-06T03:09:03.601862+0000","last_became_active":"2023-10-06T03:09:03.600210+0000","last_became_peered":"2023-10-06T03:09:03.600210+0000","last_unstale":"2023-10-06T03:09:03.601862+0000","last_undegraded":"2023-10-06T03:09:03.601862+0000","last_fullsized":"2023-10-06T03:09:03.601862+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:09.933703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.596885+0000","last_change":"2023-10-06T03:09:02.594641+0000","last_active":"2023-10-06T03:09:03.596885+0000","last_peered":"2023-10-06T03:09:03.596885+0000","last_clean":"2023-10-06T03:09:03.596885+0000","last_became_active":"2023-10-06T03:09:02.594351+0000","last_became_peered":"2023-10-06T03:09:02.594351+0000","last_unstale":"2023-10-06T03:09:03.596885+0000","last_undegraded":"2023-10-06T03:09:03.596885+0000","last_fullsized":"2023-10-06T03:09:03.596885+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:35.271403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600917+0000","last_change":"2023-10-06T03:09:03.600917+0000","last_active":"2023-10-06T03:09:03.600917+0000","last_peered":"2023-10-06T03:09:03.600917+0000","last_clean":"2023-10-06T03:09:03.600917+0000","last_became_active":"2023-10-06T03:09:03.600669+0000","last_became_peered":"2023-10-06T03:09:03.600669+0000","last_unstale":"2023-10-06T03:09:03.600917+0000","last_undegraded":"2023-10-06T03:09:03.600917+0000","last_fullsized":"2023-10-06T03:09:03.600917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:55.433899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.593814+0000","last_change":"2023-10-06T03:09:02.590839+0000","last_active":"2023-10-06T03:09:02.593814+0000","last_peered":"2023-10-06T03:09:02.593814+0000","last_clean":"2023-10-06T03:09:02.593814+0000","last_became_active":"2023-10-06T03:09:02.590530+0000","last_became_peered":"2023-10-06T03:09:02.590530+0000","last_unstale":"2023-10-06T03:09:02.593814+0000","last_undegraded":"2023-10-06T03:09:02.593814+0000","last_fullsized":"2023-10-06T03:09:02.593814+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:21.899272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597759+0000","last_change":"2023-10-06T03:09:03.597759+0000","last_active":"2023-10-06T03:09:03.597759+0000","last_peered":"2023-10-06T03:09:03.597759+0000","last_clean":"2023-10-06T03:09:03.597759+0000","last_became_active":"2023-10-06T03:09:03.597424+0000","last_became_peered":"2023-10-06T03:09:03.597424+0000","last_unstale":"2023-10-06T03:09:03.597759+0000","last_undegraded":"2023-10-06T03:09:03.597759+0000","last_fullsized":"2023-10-06T03:09:03.597759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:02.247707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613278+0000","last_change":"2023-10-06T03:09:02.595147+0000","last_active":"2023-10-06T03:09:03.613278+0000","last_peered":"2023-10-06T03:09:03.613278+0000","last_clean":"2023-10-06T03:09:03.613278+0000","last_became_active":"2023-10-06T03:09:02.594828+0000","last_became_peered":"2023-10-06T03:09:02.594828+0000","last_unstale":"2023-10-06T03:09:03.613278+0000","last_undegraded":"2023-10-06T03:09:03.613278+0000","last_fullsized":"2023-10-06T03:09:03.613278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.293211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600556+0000","last_change":"2023-10-06T03:09:03.600556+0000","last_active":"2023-10-06T03:09:03.600556+0000","last_peered":"2023-10-06T03:09:03.600556+0000","last_clean":"2023-10-06T03:09:03.600556+0000","last_became_active":"2023-10-06T03:09:03.600292+0000","last_became_peered":"2023-10-06T03:09:03.600292+0000","last_unstale":"2023-10-06T03:09:03.600556+0000","last_undegraded":"2023-10-06T03:09:03.600556+0000","last_fullsized":"2023-10-06T03:09:03.600556+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:06.153319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.596108+0000","last_change":"2023-10-06T03:09:02.592540+0000","last_active":"2023-10-06T03:09:02.596108+0000","last_peered":"2023-10-06T03:09:02.596108+0000","last_clean":"2023-10-06T03:09:02.596108+0000","last_became_active":"2023-10-06T03:09:02.592298+0000","last_became_peered":"2023-10-06T03:09:02.592298+0000","last_unstale":"2023-10-06T03:09:02.596108+0000","last_undegraded":"2023-10-06T03:09:02.596108+0000","last_fullsized":"2023-10-06T03:09:02.596108+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:48.348722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604620+0000","last_change":"2023-10-06T03:09:03.604620+0000","last_active":"2023-10-06T03:09:03.604620+0000","last_peered":"2023-10-06T03:09:03.604620+0000","last_clean":"2023-10-06T03:09:03.604620+0000","last_became_active":"2023-10-06T03:09:03.604362+0000","last_became_peered":"2023-10-06T03:09:03.604362+0000","last_unstale":"2023-10-06T03:09:03.604620+0000","last_undegraded":"2023-10-06T03:09:03.604620+0000","last_fullsized":"2023-10-06T03:09:03.604620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:53.646433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.600287+0000","last_change":"2023-10-06T03:09:02.599435+0000","last_active":"2023-10-06T03:09:02.600287+0000","last_peered":"2023-10-06T03:09:02.600287+0000","last_clean":"2023-10-06T03:09:02.600287+0000","last_became_active":"2023-10-06T03:09:02.598893+0000","last_became_peered":"2023-10-06T03:09:02.598893+0000","last_unstale":"2023-10-06T03:09:02.600287+0000","last_undegraded":"2023-10-06T03:09:02.600287+0000","last_fullsized":"2023-10-06T03:09:02.600287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:56.030638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.606194+0000","last_change":"2023-10-06T03:09:03.606194+0000","last_active":"2023-10-06T03:09:03.606194+0000","last_peered":"2023-10-06T03:09:03.606194+0000","last_clean":"2023-10-06T03:09:03.606194+0000","last_became_active":"2023-10-06T03:09:03.605882+0000","last_became_peered":"2023-10-06T03:09:03.605882+0000","last_unstale":"2023-10-06T03:09:03.606194+0000","last_undegraded":"2023-10-06T03:09:03.606194+0000","last_fullsized":"2023-10-06T03:09:03.606194+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:08.981267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.594228+0000","last_change":"2023-10-06T03:09:02.589875+0000","last_active":"2023-10-06T03:09:02.594228+0000","last_peered":"2023-10-06T03:09:02.594228+0000","last_clean":"2023-10-06T03:09:02.594228+0000","last_became_active":"2023-10-06T03:09:02.589625+0000","last_became_peered":"2023-10-06T03:09:02.589625+0000","last_unstale":"2023-10-06T03:09:02.594228+0000","last_undegraded":"2023-10-06T03:09:02.594228+0000","last_fullsized":"2023-10-06T03:09:02.594228+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:30.354593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.596714+0000","last_change":"2023-10-06T03:09:03.596714+0000","last_active":"2023-10-06T03:09:03.596714+0000","last_peered":"2023-10-06T03:09:03.596714+0000","last_clean":"2023-10-06T03:09:03.596714+0000","last_became_active":"2023-10-06T03:09:03.596448+0000","last_became_peered":"2023-10-06T03:09:03.596448+0000","last_unstale":"2023-10-06T03:09:03.596714+0000","last_undegraded":"2023-10-06T03:09:03.596714+0000","last_fullsized":"2023-10-06T03:09:03.596714+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:21.131263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613023+0000","last_change":"2023-10-06T03:09:02.591080+0000","last_active":"2023-10-06T03:09:03.613023+0000","last_peered":"2023-10-06T03:09:03.613023+0000","last_clean":"2023-10-06T03:09:03.613023+0000","last_became_active":"2023-10-06T03:09:02.590372+0000","last_became_peered":"2023-10-06T03:09:02.590372+0000","last_unstale":"2023-10-06T03:09:03.613023+0000","last_undegraded":"2023-10-06T03:09:03.613023+0000","last_fullsized":"2023-10-06T03:09:03.613023+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:43.755069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611222+0000","last_change":"2023-10-06T03:09:03.611222+0000","last_active":"2023-10-06T03:09:03.611222+0000","last_peered":"2023-10-06T03:09:03.611222+0000","last_clean":"2023-10-06T03:09:03.611222+0000","last_became_active":"2023-10-06T03:09:03.610922+0000","last_became_peered":"2023-10-06T03:09:03.610922+0000","last_unstale":"2023-10-06T03:09:03.611222+0000","last_undegraded":"2023-10-06T03:09:03.611222+0000","last_fullsized":"2023-10-06T03:09:03.611222+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.373485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.600281+0000","last_change":"2023-10-06T03:09:02.599168+0000","last_active":"2023-10-06T03:09:02.600281+0000","last_peered":"2023-10-06T03:09:02.600281+0000","last_clean":"2023-10-06T03:09:02.600281+0000","last_became_active":"2023-10-06T03:09:02.598919+0000","last_became_peered":"2023-10-06T03:09:02.598919+0000","last_unstale":"2023-10-06T03:09:02.600281+0000","last_undegraded":"2023-10-06T03:09:02.600281+0000","last_fullsized":"2023-10-06T03:09:02.600281+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:53.973425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.607000+0000","last_change":"2023-10-06T03:09:03.607000+0000","last_active":"2023-10-06T03:09:03.607000+0000","last_peered":"2023-10-06T03:09:03.607000+0000","last_clean":"2023-10-06T03:09:03.607000+0000","last_became_active":"2023-10-06T03:09:03.606724+0000","last_became_peered":"2023-10-06T03:09:03.606724+0000","last_unstale":"2023-10-06T03:09:03.607000+0000","last_undegraded":"2023-10-06T03:09:03.607000+0000","last_fullsized":"2023-10-06T03:09:03.607000+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:16.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.596789+0000","last_change":"2023-10-06T03:09:02.595716+0000","last_active":"2023-10-06T03:09:02.596789+0000","last_peered":"2023-10-06T03:09:02.596789+0000","last_clean":"2023-10-06T03:09:02.596789+0000","last_became_active":"2023-10-06T03:09:02.595036+0000","last_became_peered":"2023-10-06T03:09:02.595036+0000","last_unstale":"2023-10-06T03:09:02.596789+0000","last_undegraded":"2023-10-06T03:09:02.596789+0000","last_fullsized":"2023-10-06T03:09:02.596789+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:12.659974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605602+0000","last_change":"2023-10-06T03:09:03.605602+0000","last_active":"2023-10-06T03:09:03.605602+0000","last_peered":"2023-10-06T03:09:03.605602+0000","last_clean":"2023-10-06T03:09:03.605602+0000","last_became_active":"2023-10-06T03:09:03.605375+0000","last_became_peered":"2023-10-06T03:09:03.605375+0000","last_unstale":"2023-10-06T03:09:03.605602+0000","last_undegraded":"2023-10-06T03:09:03.605602+0000","last_fullsized":"2023-10-06T03:09:03.605602+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:24.336606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.600063+0000","last_change":"2023-10-06T03:09:02.599232+0000","last_active":"2023-10-06T03:09:02.600063+0000","last_peered":"2023-10-06T03:09:02.600063+0000","last_clean":"2023-10-06T03:09:02.600063+0000","last_became_active":"2023-10-06T03:09:02.598896+0000","last_became_peered":"2023-10-06T03:09:02.598896+0000","last_unstale":"2023-10-06T03:09:02.600063+0000","last_undegraded":"2023-10-06T03:09:02.600063+0000","last_fullsized":"2023-10-06T03:09:02.600063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:53.424235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597922+0000","last_change":"2023-10-06T03:09:03.597922+0000","last_active":"2023-10-06T03:09:03.597922+0000","last_peered":"2023-10-06T03:09:03.597922+0000","last_clean":"2023-10-06T03:09:03.597922+0000","last_became_active":"2023-10-06T03:09:03.597664+0000","last_became_peered":"2023-10-06T03:09:03.597664+0000","last_unstale":"2023-10-06T03:09:03.597922+0000","last_undegraded":"2023-10-06T03:09:03.597922+0000","last_fullsized":"2023-10-06T03:09:03.597922+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:44.846505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.593833+0000","last_change":"2023-10-06T03:09:02.590939+0000","last_active":"2023-10-06T03:09:02.593833+0000","last_peered":"2023-10-06T03:09:02.593833+0000","last_clean":"2023-10-06T03:09:02.593833+0000","last_became_active":"2023-10-06T03:09:02.590679+0000","last_became_peered":"2023-10-06T03:09:02.590679+0000","last_unstale":"2023-10-06T03:09:02.593833+0000","last_undegraded":"2023-10-06T03:09:02.593833+0000","last_fullsized":"2023-10-06T03:09:02.593833+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:38.118380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.607868+0000","last_change":"2023-10-06T03:09:03.607868+0000","last_active":"2023-10-06T03:09:03.607868+0000","last_peered":"2023-10-06T03:09:03.607868+0000","last_clean":"2023-10-06T03:09:03.607868+0000","last_became_active":"2023-10-06T03:09:03.607612+0000","last_became_peered":"2023-10-06T03:09:03.607612+0000","last_unstale":"2023-10-06T03:09:03.607868+0000","last_undegraded":"2023-10-06T03:09:03.607868+0000","last_fullsized":"2023-10-06T03:09:03.607868+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:02.247707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.597512+0000","last_change":"2023-10-06T03:09:02.597043+0000","last_active":"2023-10-06T03:09:02.597512+0000","last_peered":"2023-10-06T03:09:02.597512+0000","last_clean":"2023-10-06T03:09:02.597512+0000","last_became_active":"2023-10-06T03:09:02.596801+0000","last_became_peered":"2023-10-06T03:09:02.596801+0000","last_unstale":"2023-10-06T03:09:02.597512+0000","last_undegraded":"2023-10-06T03:09:02.597512+0000","last_fullsized":"2023-10-06T03:09:02.597512+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:26.956521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600884+0000","last_change":"2023-10-06T03:09:03.600884+0000","last_active":"2023-10-06T03:09:03.600884+0000","last_peered":"2023-10-06T03:09:03.600884+0000","last_clean":"2023-10-06T03:09:03.600884+0000","last_became_active":"2023-10-06T03:09:03.600620+0000","last_became_peered":"2023-10-06T03:09:03.600620+0000","last_unstale":"2023-10-06T03:09:03.600884+0000","last_undegraded":"2023-10-06T03:09:03.600884+0000","last_fullsized":"2023-10-06T03:09:03.600884+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:19.560203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.591478+0000","last_change":"2023-10-06T03:09:02.587671+0000","last_active":"2023-10-06T03:09:02.591478+0000","last_peered":"2023-10-06T03:09:02.591478+0000","last_clean":"2023-10-06T03:09:02.591478+0000","last_became_active":"2023-10-06T03:09:02.587399+0000","last_became_peered":"2023-10-06T03:09:02.587399+0000","last_unstale":"2023-10-06T03:09:02.591478+0000","last_undegraded":"2023-10-06T03:09:02.591478+0000","last_fullsized":"2023-10-06T03:09:02.591478+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:34.570119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604687+0000","last_change":"2023-10-06T03:09:03.604687+0000","last_active":"2023-10-06T03:09:03.604687+0000","last_peered":"2023-10-06T03:09:03.604687+0000","last_clean":"2023-10-06T03:09:03.604687+0000","last_became_active":"2023-10-06T03:09:03.604154+0000","last_became_peered":"2023-10-06T03:09:03.604154+0000","last_unstale":"2023-10-06T03:09:03.604687+0000","last_undegraded":"2023-10-06T03:09:03.604687+0000","last_fullsized":"2023-10-06T03:09:03.604687+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:11.961093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.594462+0000","last_change":"2023-10-06T03:09:02.593421+0000","last_active":"2023-10-06T03:09:02.594462+0000","last_peered":"2023-10-06T03:09:02.594462+0000","last_clean":"2023-10-06T03:09:02.594462+0000","last_became_active":"2023-10-06T03:09:02.593135+0000","last_became_peered":"2023-10-06T03:09:02.593135+0000","last_unstale":"2023-10-06T03:09:02.594462+0000","last_undegraded":"2023-10-06T03:09:02.594462+0000","last_fullsized":"2023-10-06T03:09:02.594462+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.563585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603781+0000","last_change":"2023-10-06T03:09:03.603781+0000","last_active":"2023-10-06T03:09:03.603781+0000","last_peered":"2023-10-06T03:09:03.603781+0000","last_clean":"2023-10-06T03:09:03.603781+0000","last_became_active":"2023-10-06T03:09:03.602754+0000","last_became_peered":"2023-10-06T03:09:03.602754+0000","last_unstale":"2023-10-06T03:09:03.603781+0000","last_undegraded":"2023-10-06T03:09:03.603781+0000","last_fullsized":"2023-10-06T03:09:03.603781+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:29.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613694+0000","last_change":"2023-10-06T03:09:02.587566+0000","last_active":"2023-10-06T03:09:03.613694+0000","last_peered":"2023-10-06T03:09:03.613694+0000","last_clean":"2023-10-06T03:09:03.613694+0000","last_became_active":"2023-10-06T03:09:02.587231+0000","last_became_peered":"2023-10-06T03:09:02.587231+0000","last_unstale":"2023-10-06T03:09:03.613694+0000","last_undegraded":"2023-10-06T03:09:03.613694+0000","last_fullsized":"2023-10-06T03:09:03.613694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:46.132169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.585358+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:09:02.585358+0000","last_peered":"2023-10-06T03:09:02.585358+0000","last_clean":"2023-10-06T03:09:02.585358+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:09:02.585358+0000","last_undegraded":"2023-10-06T03:09:02.585358+0000","last_fullsized":"2023-10-06T03:09:02.585358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601262+0000","last_change":"2023-10-06T03:09:03.601262+0000","last_active":"2023-10-06T03:09:03.601262+0000","last_peered":"2023-10-06T03:09:03.601262+0000","last_clean":"2023-10-06T03:09:03.601262+0000","last_became_active":"2023-10-06T03:09:03.600839+0000","last_became_peered":"2023-10-06T03:09:03.600839+0000","last_unstale":"2023-10-06T03:09:03.601262+0000","last_undegraded":"2023-10-06T03:09:03.601262+0000","last_fullsized":"2023-10-06T03:09:03.601262+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:08.114339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613866+0000","last_change":"2023-10-06T03:09:02.584840+0000","last_active":"2023-10-06T03:09:03.613866+0000","last_peered":"2023-10-06T03:09:03.613866+0000","last_clean":"2023-10-06T03:09:03.613866+0000","last_became_active":"2023-10-06T03:09:02.584571+0000","last_became_peered":"2023-10-06T03:09:02.584571+0000","last_unstale":"2023-10-06T03:09:03.613866+0000","last_undegraded":"2023-10-06T03:09:03.613866+0000","last_fullsized":"2023-10-06T03:09:03.613866+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:52.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577586+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:09:02.577586+0000","last_peered":"2023-10-06T03:09:02.577586+0000","last_clean":"2023-10-06T03:09:02.577586+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:09:02.577586+0000","last_undegraded":"2023-10-06T03:09:02.577586+0000","last_fullsized":"2023-10-06T03:09:02.577586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604987+0000","last_change":"2023-10-06T03:09:03.604987+0000","last_active":"2023-10-06T03:09:03.604987+0000","last_peered":"2023-10-06T03:09:03.604987+0000","last_clean":"2023-10-06T03:09:03.604987+0000","last_became_active":"2023-10-06T03:09:03.601903+0000","last_became_peered":"2023-10-06T03:09:03.601903+0000","last_unstale":"2023-10-06T03:09:03.604987+0000","last_undegraded":"2023-10-06T03:09:03.604987+0000","last_fullsized":"2023-10-06T03:09:03.604987+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:29.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.588544+0000","last_change":"2023-10-06T03:09:02.595428+0000","last_active":"2023-10-06T03:09:03.588544+0000","last_peered":"2023-10-06T03:09:03.588544+0000","last_clean":"2023-10-06T03:09:03.588544+0000","last_became_active":"2023-10-06T03:09:02.595144+0000","last_became_peered":"2023-10-06T03:09:02.595144+0000","last_unstale":"2023-10-06T03:09:03.588544+0000","last_undegraded":"2023-10-06T03:09:03.588544+0000","last_fullsized":"2023-10-06T03:09:03.588544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:36.112186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.585773+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:09:02.585773+0000","last_peered":"2023-10-06T03:09:02.585773+0000","last_clean":"2023-10-06T03:09:02.585773+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:09:02.585773+0000","last_undegraded":"2023-10-06T03:09:02.585773+0000","last_fullsized":"2023-10-06T03:09:02.585773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599682+0000","last_change":"2023-10-06T03:09:03.599682+0000","last_active":"2023-10-06T03:09:03.599682+0000","last_peered":"2023-10-06T03:09:03.599682+0000","last_clean":"2023-10-06T03:09:03.599682+0000","last_became_active":"2023-10-06T03:09:03.599210+0000","last_became_peered":"2023-10-06T03:09:03.599210+0000","last_unstale":"2023-10-06T03:09:03.599682+0000","last_undegraded":"2023-10-06T03:09:03.599682+0000","last_fullsized":"2023-10-06T03:09:03.599682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:48.578908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.588670+0000","last_change":"2023-10-06T03:09:02.595589+0000","last_active":"2023-10-06T03:09:03.588670+0000","last_peered":"2023-10-06T03:09:03.588670+0000","last_clean":"2023-10-06T03:09:03.588670+0000","last_became_active":"2023-10-06T03:09:02.594526+0000","last_became_peered":"2023-10-06T03:09:02.594526+0000","last_unstale":"2023-10-06T03:09:03.588670+0000","last_undegraded":"2023-10-06T03:09:03.588670+0000","last_fullsized":"2023-10-06T03:09:03.588670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:08.929557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":111,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.583811+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:09:02.583811+0000","last_peered":"2023-10-06T03:09:02.583811+0000","last_clean":"2023-10-06T03:09:02.583811+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:09:02.583811+0000","last_undegraded":"2023-10-06T03:09:02.583811+0000","last_fullsized":"2023-10-06T03:09:02.583811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603575+0000","last_change":"2023-10-06T03:09:03.603575+0000","last_active":"2023-10-06T03:09:03.603575+0000","last_peered":"2023-10-06T03:09:03.603575+0000","last_clean":"2023-10-06T03:09:03.603575+0000","last_became_active":"2023-10-06T03:09:03.603174+0000","last_became_peered":"2023-10-06T03:09:03.603174+0000","last_unstale":"2023-10-06T03:09:03.603575+0000","last_undegraded":"2023-10-06T03:09:03.603575+0000","last_fullsized":"2023-10-06T03:09:03.603575+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:26.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.597068+0000","last_change":"2023-10-06T03:09:02.594260+0000","last_active":"2023-10-06T03:09:02.597068+0000","last_peered":"2023-10-06T03:09:02.597068+0000","last_clean":"2023-10-06T03:09:02.597068+0000","last_became_active":"2023-10-06T03:09:02.594011+0000","last_became_peered":"2023-10-06T03:09:02.594011+0000","last_unstale":"2023-10-06T03:09:02.597068+0000","last_undegraded":"2023-10-06T03:09:02.597068+0000","last_fullsized":"2023-10-06T03:09:02.597068+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:10.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.585607+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:09:02.585607+0000","last_peered":"2023-10-06T03:09:02.585607+0000","last_clean":"2023-10-06T03:09:02.585607+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:09:02.585607+0000","last_undegraded":"2023-10-06T03:09:02.585607+0000","last_fullsized":"2023-10-06T03:09:02.585607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":123,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577955+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:09:02.577955+0000","last_peered":"2023-10-06T03:09:02.577955+0000","last_clean":"2023-10-06T03:09:02.577955+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:09:02.577955+0000","last_undegraded":"2023-10-06T03:09:02.577955+0000","last_fullsized":"2023-10-06T03:09:02.577955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611479+0000","last_change":"2023-10-06T03:09:03.611479+0000","last_active":"2023-10-06T03:09:03.611479+0000","last_peered":"2023-10-06T03:09:03.611479+0000","last_clean":"2023-10-06T03:09:03.611479+0000","last_became_active":"2023-10-06T03:09:03.611101+0000","last_became_peered":"2023-10-06T03:09:03.611101+0000","last_unstale":"2023-10-06T03:09:03.611479+0000","last_undegraded":"2023-10-06T03:09:03.611479+0000","last_fullsized":"2023-10-06T03:09:03.611479+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:48.578908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.592175+0000","last_change":"2023-10-06T03:09:02.597026+0000","last_active":"2023-10-06T03:09:03.592175+0000","last_peered":"2023-10-06T03:09:03.592175+0000","last_clean":"2023-10-06T03:09:03.592175+0000","last_became_active":"2023-10-06T03:09:02.596775+0000","last_became_peered":"2023-10-06T03:09:02.596775+0000","last_unstale":"2023-10-06T03:09:03.592175+0000","last_undegraded":"2023-10-06T03:09:03.592175+0000","last_fullsized":"2023-10-06T03:09:03.592175+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:10.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.584315+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:09:02.584315+0000","last_peered":"2023-10-06T03:09:02.584315+0000","last_clean":"2023-10-06T03:09:02.584315+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:09:02.584315+0000","last_undegraded":"2023-10-06T03:09:02.584315+0000","last_fullsized":"2023-10-06T03:09:02.584315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602307+0000","last_change":"2023-10-06T03:09:03.602307+0000","last_active":"2023-10-06T03:09:03.602307+0000","last_peered":"2023-10-06T03:09:03.602307+0000","last_clean":"2023-10-06T03:09:03.602307+0000","last_became_active":"2023-10-06T03:09:03.602032+0000","last_became_peered":"2023-10-06T03:09:03.602032+0000","last_unstale":"2023-10-06T03:09:03.602307+0000","last_undegraded":"2023-10-06T03:09:03.602307+0000","last_fullsized":"2023-10-06T03:09:03.602307+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:44.759215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.590045+0000","last_change":"2023-10-06T03:09:02.588029+0000","last_active":"2023-10-06T03:09:02.590045+0000","last_peered":"2023-10-06T03:09:02.590045+0000","last_clean":"2023-10-06T03:09:02.590045+0000","last_became_active":"2023-10-06T03:09:02.587749+0000","last_became_peered":"2023-10-06T03:09:02.587749+0000","last_unstale":"2023-10-06T03:09:02.590045+0000","last_undegraded":"2023-10-06T03:09:02.590045+0000","last_fullsized":"2023-10-06T03:09:02.590045+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:08.128505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":123,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577848+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:09:02.577848+0000","last_peered":"2023-10-06T03:09:02.577848+0000","last_clean":"2023-10-06T03:09:02.577848+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:09:02.577848+0000","last_undegraded":"2023-10-06T03:09:02.577848+0000","last_fullsized":"2023-10-06T03:09:02.577848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602951+0000","last_change":"2023-10-06T03:09:03.602951+0000","last_active":"2023-10-06T03:09:03.602951+0000","last_peered":"2023-10-06T03:09:03.602951+0000","last_clean":"2023-10-06T03:09:03.602951+0000","last_became_active":"2023-10-06T03:09:03.602652+0000","last_became_peered":"2023-10-06T03:09:03.602652+0000","last_unstale":"2023-10-06T03:09:03.602951+0000","last_undegraded":"2023-10-06T03:09:03.602951+0000","last_fullsized":"2023-10-06T03:09:03.602951+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:44.759215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"139'4","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613517+0000","last_change":"2023-10-06T03:09:02.588094+0000","last_active":"2023-10-06T03:09:03.613517+0000","last_peered":"2023-10-06T03:09:03.613517+0000","last_clean":"2023-10-06T03:09:03.613517+0000","last_became_active":"2023-10-06T03:09:02.587840+0000","last_became_peered":"2023-10-06T03:09:02.587840+0000","last_unstale":"2023-10-06T03:09:03.613517+0000","last_undegraded":"2023-10-06T03:09:03.613517+0000","last_fullsized":"2023-10-06T03:09:03.613517+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:02.326914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577534+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:09:02.577534+0000","last_peered":"2023-10-06T03:09:02.577534+0000","last_clean":"2023-10-06T03:09:02.577534+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:09:02.577534+0000","last_undegraded":"2023-10-06T03:09:02.577534+0000","last_fullsized":"2023-10-06T03:09:02.577534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1412157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.68700000000000006},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.5},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.436}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.376},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.56299999999999994,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.59999999999999998},"min":{"1min":0.51700000000000002,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":828975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.33400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71799999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.434},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.505},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.627,"5min":0.755,"15min":0.755},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66500000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.79000000000000004},"last":0.504}]},{"osd":3,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.65100000000000002,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.57199999999999995},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.28499999999999998},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.58799999999999997},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78900000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":821789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34399999999999997,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.40699999999999997,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.252},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39300000000000002,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.98899999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32500000000000001,"15min":0.33200000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.40500000000000003,"5min":1.282,"15min":1.282},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33500000000000002},"min":{"1min":0.247,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42799999999999999,"5min":1.25,"15min":1.25},"last":0.372},{"interface":"front","average":{"1min":0.374,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.42899999999999999,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.29199999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66200000000000003,"5min":1.302,"15min":1.302},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.127,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52900000000000003,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.96299999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63,"15min":0.60299999999999998},"min":{"1min":0.47599999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71099999999999997,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.74199999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56499999999999995,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.054,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.379,"15min":1.379},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303696,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068144,"statfs":{"total":96636764160,"available":96325779456,"internally_reserved":0,"allocated":2367488,"data_stored":1419973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.307,"15min":0.309},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.32600000000000001},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38800000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.435}]},{"osd":2,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33300000000000002},"min":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.64100000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.311}]},{"osd":3,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.443,"5min":0.52600000000000002,"15min":0.61699999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.81999999999999995,"15min":0.871},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.754,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.496},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78400000000000003,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.96099999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.97999999999999998},"last":0.53200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":829459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.29999999999999999,"15min":0.318},"min":{"1min":0.23999999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.443,"5min":0.498,"15min":0.53100000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46700000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.32900000000000001},"min":{"1min":0.253,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.34799999999999998},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44600000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.318}]},{"osd":3,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32300000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.42999999999999999,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.378},"min":{"1min":0.31,"5min":0.249,"15min":0.249},"max":{"1min":0.503,"5min":0.55300000000000005,"15min":0.57699999999999996},"last":0.29999999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.54200000000000004},"min":{"1min":0.504,"5min":0.374,"15min":0.374},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.40300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72899999999999998,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.55700000000000005},"min":{"1min":0.48099999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.87,"15min":1.8740000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.61299999999999999},"min":{"1min":0.49399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.903},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77000000000000002,"5min":0.78800000000000003,"15min":0.84599999999999997},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":821789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.308},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47399999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.47099999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.30399999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.307,"15min":0.309},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.40600000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.436,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35199999999999998,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.44500000000000001,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.35499999999999998},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.26400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.873,"15min":5.5220000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75800000000000001,"5min":0.997,"15min":0.997},"last":0.379}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.72899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.49199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.441},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71999999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303116,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068724,"statfs":{"total":96636764160,"available":96326373376,"internally_reserved":0,"allocated":1773568,"data_stored":830475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.66300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.57799999999999996},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":0.746,"15min":3},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.76400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":10.084},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33000000000000002},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.752,"15min":0.83999999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.57999999999999996},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":1.54},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.72599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.438}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":821789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53600000000000003,"15min":0.57799999999999996},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72699999999999998,"5min":1.143,"15min":2.3599999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54700000000000004,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.86299999999999999,"15min":2.371},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.55900000000000005,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69599999999999995,"5min":6.532,"15min":6.532},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.57899999999999996,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.75,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76500000000000001,"5min":5.524,"15min":5.524},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.58099999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72199999999999998,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60099999999999998,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.753,"5min":0.84199999999999997,"15min":1.052},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.624},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.86799999999999999,"15min":1.175},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.60899999999999999},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.77500000000000002,"15min":1.145},"last":0.65800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:09:07.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:09:07.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.804+0000 7f0aa7bf1700 1 Processor -- start 2023-10-06T03:09:07.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.804+0000 7f0aa7bf1700 1 -- start start 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.805+0000 7f0aa7bf1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0aa0159890 0x7f0aa015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.805+0000 7f0aa7bf1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 0x7f0aa015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.805+0000 7f0aa7bf1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa015a160 con 0x7f0aa015db10 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.805+0000 7f0aa7bf1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa00d13a0 con 0x7f0aa0159890 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.805+0000 7f0aa7bf1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa015e140 con 0x7f0aa0159c60 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa598d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0aa0159890 0x7f0aa015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 0x7f0aa015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 0x7f0aa015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:40686/0 (socket says 172.21.15.92:40686) 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 -- 172.21.15.92:0/2832546590 learned_addr learned my addr 172.21.15.92:0/2832546590 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 -- 172.21.15.92:0/2832546590 >> v2:172.21.15.136:3300/0 conn(0x7f0aa0159890 msgr2=0x7f0aa015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 --2- 172.21.15.92:0/2832546590 >> v2:172.21.15.136:3300/0 conn(0x7f0aa0159890 0x7f0aa015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 -- 172.21.15.92:0/2832546590 >> v1:172.21.15.136:6789/0 conn(0x7f0aa015db10 legacy=0x7f0aa0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 -- 172.21.15.92:0/2832546590 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa015a3d0 con 0x7f0aa0159c60 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa598d700 1 --2- 172.21.15.92:0/2832546590 >> v2:172.21.15.136:3300/0 conn(0x7f0aa0159890 0x7f0aa015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa518c700 1 --2- 172.21.15.92:0/2832546590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 0x7f0aa015d5d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a900099b0 tx=0x7f0a90009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d07cb80e16642e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa498b700 1 -- 172.21.15.92:0/2832546590 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90009ea0 con 0x7f0aa0159c60 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.806+0000 7f0aa498b700 1 -- 172.21.15.92:0/2832546590 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9000b860 con 0x7f0aa0159c60 2023-10-06T03:09:07.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa498b700 1 -- 172.21.15.92:0/2832546590 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000bc00 con 0x7f0aa0159c60 2023-10-06T03:09:07.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2832546590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 msgr2=0x7f0aa015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2832546590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 0x7f0aa015d5d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0a900099b0 tx=0x7f0a90009250 comp rx=0 tx=0).stop 2023-10-06T03:09:07.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2832546590 shutdown_connections 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2832546590 >> v2:172.21.15.136:3300/0 conn(0x7f0aa0159890 0x7f0aa015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2832546590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa0159c60 0x7f0aa015d5d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2832546590 >> 172.21.15.92:0/2832546590 conn(0x7f0aa0068e20 msgr2=0x7f0aa00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2832546590 shutdown_connections 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.807+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2832546590 wait complete. 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.808+0000 7f0aa7bf1700 1 Processor -- start 2023-10-06T03:09:07.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.808+0000 7f0aa7bf1700 1 -- start start 2023-10-06T03:09:07.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa7bf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 0x7f0aa00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa7bf1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aa0159c60 0x7f0aa00cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa7bf1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa015db10 0x7f0aa0169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa7bf1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa016a450 con 0x7f0aa0159890 2023-10-06T03:09:07.814 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa7bf1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa016a6f0 con 0x7f0aa0159c60 2023-10-06T03:09:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa7bf1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa016a9d0 con 0x7f0aa015db10 2023-10-06T03:09:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 0x7f0aa00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 0x7f0aa00ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52864/0 (socket says 172.21.15.92:52864) 2023-10-06T03:09:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 -- 172.21.15.92:0/2605626662 learned_addr learned my addr 172.21.15.92:0/2605626662 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa518c700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aa0159c60 0x7f0aa00cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa618e700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa015db10 0x7f0aa0169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 -- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa015db10 msgr2=0x7f0aa0169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa015db10 0x7f0aa0169f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 -- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aa0159c60 msgr2=0x7f0aa00cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aa0159c60 0x7f0aa00cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.809+0000 7f0aa598d700 1 -- 172.21.15.92:0/2605626662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009420 con 0x7f0aa0159890 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.810+0000 7f0aa598d700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 0x7f0aa00ce9b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00ec80 tx=0x7f0a9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.810+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00c9d0 con 0x7f0aa0159890 2023-10-06T03:09:07.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.810+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00cb30 con 0x7f0aa0159890 2023-10-06T03:09:07.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.810+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00ceb0 con 0x7f0aa0159890 2023-10-06T03:09:07.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.810+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a900094f0 con 0x7f0aa0159890 2023-10-06T03:09:07.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.810+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa00cd260 con 0x7f0aa0159890 2023-10-06T03:09:07.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.811+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa00cd5d0 con 0x7f0aa0159890 2023-10-06T03:09:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.811+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0a9c013680 con 0x7f0aa0159890 2023-10-06T03:09:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.812+0000 7f0a95ffb700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a840673a0 0x7f0a84069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.812+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0a9c085260 con 0x7f0aa0159890 2023-10-06T03:09:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.812+0000 7f0aa518c700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a840673a0 0x7f0a84069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.812+0000 7f0aa518c700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a840673a0 0x7f0a84069850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0a900099b0 tx=0x7f0a90017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:07.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.816+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a9c051d50 con 0x7f0aa0159890 2023-10-06T03:09:07.883 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:09:07.882+0000 7f8484791700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:09:07.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:07.996+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0aa016aef0 con 0x7f0a840673a0 2023-10-06T03:09:08.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.004+0000 7f0a95ffb700 1 -- 172.21.15.92:0/2605626662 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311869 (secure 0 0 0) 0x7f0aa016aef0 con 0x7f0a840673a0 2023-10-06T03:09:08.009 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a840673a0 msgr2=0x7f0a84069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a840673a0 0x7f0a84069850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0a900099b0 tx=0x7f0a90017000 comp rx=0 tx=0).stop 2023-10-06T03:09:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 msgr2=0x7f0aa00ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 0x7f0aa00ce9b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00ec80 tx=0x7f0a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 shutdown_connections 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a840673a0 0x7f0a84069850 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aa0159890 0x7f0aa00ce9b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aa015db10 0x7f0aa0169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 --2- 172.21.15.92:0/2605626662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aa0159c60 0x7f0aa00cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 >> 172.21.15.92:0/2605626662 conn(0x7f0aa0068e20 msgr2=0x7f0aa00d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 shutdown_connections 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.007+0000 7f0aa7bf1700 1 -- 172.21.15.92:0/2605626662 wait complete. 2023-10-06T03:09:08.013 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:09:08.020 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2023-10-06T03:09:07.273824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425952,"kb_used_data":14880,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548768,"statfs":{"total":773094113280,"available":770609938432,"internally_reserved":0,"allocated":15237120,"data_stored":7786406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.703405"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604866+0000","last_change":"2023-10-06T03:09:03.604866+0000","last_active":"2023-10-06T03:09:03.604866+0000","last_peered":"2023-10-06T03:09:03.604866+0000","last_clean":"2023-10-06T03:09:03.604866+0000","last_became_active":"2023-10-06T03:09:03.604298+0000","last_became_peered":"2023-10-06T03:09:03.604298+0000","last_unstale":"2023-10-06T03:09:03.604866+0000","last_undegraded":"2023-10-06T03:09:03.604866+0000","last_fullsized":"2023-10-06T03:09:03.604866+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:39.571557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604542+0000","last_change":"2023-10-06T03:09:03.604542+0000","last_active":"2023-10-06T03:09:03.604542+0000","last_peered":"2023-10-06T03:09:03.604542+0000","last_clean":"2023-10-06T03:09:03.604542+0000","last_became_active":"2023-10-06T03:09:03.603295+0000","last_became_peered":"2023-10-06T03:09:03.603295+0000","last_unstale":"2023-10-06T03:09:03.604542+0000","last_undegraded":"2023-10-06T03:09:03.604542+0000","last_fullsized":"2023-10-06T03:09:03.604542+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:11.961093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611900+0000","last_change":"2023-10-06T03:09:03.611900+0000","last_active":"2023-10-06T03:09:03.611900+0000","last_peered":"2023-10-06T03:09:03.611900+0000","last_clean":"2023-10-06T03:09:03.611900+0000","last_became_active":"2023-10-06T03:09:03.611635+0000","last_became_peered":"2023-10-06T03:09:03.611635+0000","last_unstale":"2023-10-06T03:09:03.611900+0000","last_undegraded":"2023-10-06T03:09:03.611900+0000","last_fullsized":"2023-10-06T03:09:03.611900+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:29.398296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601263+0000","last_change":"2023-10-06T03:09:03.601263+0000","last_active":"2023-10-06T03:09:03.601263+0000","last_peered":"2023-10-06T03:09:03.601263+0000","last_clean":"2023-10-06T03:09:03.601263+0000","last_became_active":"2023-10-06T03:09:03.601003+0000","last_became_peered":"2023-10-06T03:09:03.601003+0000","last_unstale":"2023-10-06T03:09:03.601263+0000","last_undegraded":"2023-10-06T03:09:03.601263+0000","last_fullsized":"2023-10-06T03:09:03.601263+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:52.782664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602930+0000","last_change":"2023-10-06T03:09:03.602930+0000","last_active":"2023-10-06T03:09:03.602930+0000","last_peered":"2023-10-06T03:09:03.602930+0000","last_clean":"2023-10-06T03:09:03.602930+0000","last_became_active":"2023-10-06T03:09:03.602680+0000","last_became_peered":"2023-10-06T03:09:03.602680+0000","last_unstale":"2023-10-06T03:09:03.602930+0000","last_undegraded":"2023-10-06T03:09:03.602930+0000","last_fullsized":"2023-10-06T03:09:03.602930+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:08:37.782342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601014+0000","last_change":"2023-10-06T03:09:03.601014+0000","last_active":"2023-10-06T03:09:03.601014+0000","last_peered":"2023-10-06T03:09:03.601014+0000","last_clean":"2023-10-06T03:09:03.601014+0000","last_became_active":"2023-10-06T03:09:03.600745+0000","last_became_peered":"2023-10-06T03:09:03.600745+0000","last_unstale":"2023-10-06T03:09:03.601014+0000","last_undegraded":"2023-10-06T03:09:03.601014+0000","last_fullsized":"2023-10-06T03:09:03.601014+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:41.123215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604567+0000","last_change":"2023-10-06T03:09:03.604567+0000","last_active":"2023-10-06T03:09:03.604567+0000","last_peered":"2023-10-06T03:09:03.604567+0000","last_clean":"2023-10-06T03:09:03.604567+0000","last_became_active":"2023-10-06T03:09:03.604296+0000","last_became_peered":"2023-10-06T03:09:03.604296+0000","last_unstale":"2023-10-06T03:09:03.604567+0000","last_undegraded":"2023-10-06T03:09:03.604567+0000","last_fullsized":"2023-10-06T03:09:03.604567+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:36.275549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.608958+0000","last_change":"2023-10-06T03:09:03.608958+0000","last_active":"2023-10-06T03:09:03.608958+0000","last_peered":"2023-10-06T03:09:03.608958+0000","last_clean":"2023-10-06T03:09:03.608958+0000","last_became_active":"2023-10-06T03:09:03.608453+0000","last_became_peered":"2023-10-06T03:09:03.608453+0000","last_unstale":"2023-10-06T03:09:03.608958+0000","last_undegraded":"2023-10-06T03:09:03.608958+0000","last_fullsized":"2023-10-06T03:09:03.608958+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:55.433899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.608393+0000","last_change":"2023-10-06T03:09:03.608393+0000","last_active":"2023-10-06T03:09:03.608393+0000","last_peered":"2023-10-06T03:09:03.608393+0000","last_clean":"2023-10-06T03:09:03.608393+0000","last_became_active":"2023-10-06T03:09:03.608078+0000","last_became_peered":"2023-10-06T03:09:03.608078+0000","last_unstale":"2023-10-06T03:09:03.608393+0000","last_undegraded":"2023-10-06T03:09:03.608393+0000","last_fullsized":"2023-10-06T03:09:03.608393+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:42.997705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603336+0000","last_change":"2023-10-06T03:09:03.603336+0000","last_active":"2023-10-06T03:09:03.603336+0000","last_peered":"2023-10-06T03:09:03.603336+0000","last_clean":"2023-10-06T03:09:03.603336+0000","last_became_active":"2023-10-06T03:09:03.603089+0000","last_became_peered":"2023-10-06T03:09:03.603089+0000","last_unstale":"2023-10-06T03:09:03.603336+0000","last_undegraded":"2023-10-06T03:09:03.603336+0000","last_fullsized":"2023-10-06T03:09:03.603336+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:09.080161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602872+0000","last_change":"2023-10-06T03:09:03.602872+0000","last_active":"2023-10-06T03:09:03.602872+0000","last_peered":"2023-10-06T03:09:03.602872+0000","last_clean":"2023-10-06T03:09:03.602872+0000","last_became_active":"2023-10-06T03:09:03.602617+0000","last_became_peered":"2023-10-06T03:09:03.602617+0000","last_unstale":"2023-10-06T03:09:03.602872+0000","last_undegraded":"2023-10-06T03:09:03.602872+0000","last_fullsized":"2023-10-06T03:09:03.602872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:46.371480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603080+0000","last_change":"2023-10-06T03:09:03.603080+0000","last_active":"2023-10-06T03:09:03.603080+0000","last_peered":"2023-10-06T03:09:03.603080+0000","last_clean":"2023-10-06T03:09:03.603080+0000","last_became_active":"2023-10-06T03:09:03.602820+0000","last_became_peered":"2023-10-06T03:09:03.602820+0000","last_unstale":"2023-10-06T03:09:03.603080+0000","last_undegraded":"2023-10-06T03:09:03.603080+0000","last_fullsized":"2023-10-06T03:09:03.603080+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:32.238086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603751+0000","last_change":"2023-10-06T03:09:03.603751+0000","last_active":"2023-10-06T03:09:03.603751+0000","last_peered":"2023-10-06T03:09:03.603751+0000","last_clean":"2023-10-06T03:09:03.603751+0000","last_became_active":"2023-10-06T03:09:03.603448+0000","last_became_peered":"2023-10-06T03:09:03.603448+0000","last_unstale":"2023-10-06T03:09:03.603751+0000","last_undegraded":"2023-10-06T03:09:03.603751+0000","last_fullsized":"2023-10-06T03:09:03.603751+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:31.713110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604871+0000","last_change":"2023-10-06T03:09:03.604871+0000","last_active":"2023-10-06T03:09:03.604871+0000","last_peered":"2023-10-06T03:09:03.604871+0000","last_clean":"2023-10-06T03:09:03.604871+0000","last_became_active":"2023-10-06T03:09:03.603994+0000","last_became_peered":"2023-10-06T03:09:03.603994+0000","last_unstale":"2023-10-06T03:09:03.604871+0000","last_undegraded":"2023-10-06T03:09:03.604871+0000","last_fullsized":"2023-10-06T03:09:03.604871+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:16.765052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600174+0000","last_change":"2023-10-06T03:09:03.600174+0000","last_active":"2023-10-06T03:09:03.600174+0000","last_peered":"2023-10-06T03:09:03.600174+0000","last_clean":"2023-10-06T03:09:03.600174+0000","last_became_active":"2023-10-06T03:09:03.599861+0000","last_became_peered":"2023-10-06T03:09:03.599861+0000","last_unstale":"2023-10-06T03:09:03.600174+0000","last_undegraded":"2023-10-06T03:09:03.600174+0000","last_fullsized":"2023-10-06T03:09:03.600174+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:16.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.594453+0000","last_change":"2023-10-06T03:09:03.594453+0000","last_active":"2023-10-06T03:09:03.594453+0000","last_peered":"2023-10-06T03:09:03.594453+0000","last_clean":"2023-10-06T03:09:03.594453+0000","last_became_active":"2023-10-06T03:09:03.593990+0000","last_became_peered":"2023-10-06T03:09:03.593990+0000","last_unstale":"2023-10-06T03:09:03.594453+0000","last_undegraded":"2023-10-06T03:09:03.594453+0000","last_fullsized":"2023-10-06T03:09:03.594453+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:36:32.238086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603499+0000","last_change":"2023-10-06T03:09:03.603499+0000","last_active":"2023-10-06T03:09:03.603499+0000","last_peered":"2023-10-06T03:09:03.603499+0000","last_clean":"2023-10-06T03:09:03.603499+0000","last_became_active":"2023-10-06T03:09:03.603246+0000","last_became_peered":"2023-10-06T03:09:03.603246+0000","last_unstale":"2023-10-06T03:09:03.603499+0000","last_undegraded":"2023-10-06T03:09:03.603499+0000","last_fullsized":"2023-10-06T03:09:03.603499+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:07.456293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605109+0000","last_change":"2023-10-06T03:09:03.605109+0000","last_active":"2023-10-06T03:09:03.605109+0000","last_peered":"2023-10-06T03:09:03.605109+0000","last_clean":"2023-10-06T03:09:03.605109+0000","last_became_active":"2023-10-06T03:09:03.604587+0000","last_became_peered":"2023-10-06T03:09:03.604587+0000","last_unstale":"2023-10-06T03:09:03.605109+0000","last_undegraded":"2023-10-06T03:09:03.605109+0000","last_fullsized":"2023-10-06T03:09:03.605109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:26.740899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.606935+0000","last_change":"2023-10-06T03:09:03.606935+0000","last_active":"2023-10-06T03:09:03.606935+0000","last_peered":"2023-10-06T03:09:03.606935+0000","last_clean":"2023-10-06T03:09:03.606935+0000","last_became_active":"2023-10-06T03:09:03.606564+0000","last_became_peered":"2023-10-06T03:09:03.606564+0000","last_unstale":"2023-10-06T03:09:03.606935+0000","last_undegraded":"2023-10-06T03:09:03.606935+0000","last_fullsized":"2023-10-06T03:09:03.606935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:39.122526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603202+0000","last_change":"2023-10-06T03:09:03.603202+0000","last_active":"2023-10-06T03:09:03.603202+0000","last_peered":"2023-10-06T03:09:03.603202+0000","last_clean":"2023-10-06T03:09:03.603202+0000","last_became_active":"2023-10-06T03:09:03.602943+0000","last_became_peered":"2023-10-06T03:09:03.602943+0000","last_unstale":"2023-10-06T03:09:03.603202+0000","last_undegraded":"2023-10-06T03:09:03.603202+0000","last_fullsized":"2023-10-06T03:09:03.603202+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:53.646433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604741+0000","last_change":"2023-10-06T03:09:03.604741+0000","last_active":"2023-10-06T03:09:03.604741+0000","last_peered":"2023-10-06T03:09:03.604741+0000","last_clean":"2023-10-06T03:09:03.604741+0000","last_became_active":"2023-10-06T03:09:03.604498+0000","last_became_peered":"2023-10-06T03:09:03.604498+0000","last_unstale":"2023-10-06T03:09:03.604741+0000","last_undegraded":"2023-10-06T03:09:03.604741+0000","last_fullsized":"2023-10-06T03:09:03.604741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:09.933703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.607098+0000","last_change":"2023-10-06T03:09:03.607098+0000","last_active":"2023-10-06T03:09:03.607098+0000","last_peered":"2023-10-06T03:09:03.607098+0000","last_clean":"2023-10-06T03:09:03.607098+0000","last_became_active":"2023-10-06T03:09:03.606832+0000","last_became_peered":"2023-10-06T03:09:03.606832+0000","last_unstale":"2023-10-06T03:09:03.607098+0000","last_undegraded":"2023-10-06T03:09:03.607098+0000","last_fullsized":"2023-10-06T03:09:03.607098+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:26.740899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.609571+0000","last_change":"2023-10-06T03:09:03.609571+0000","last_active":"2023-10-06T03:09:03.609571+0000","last_peered":"2023-10-06T03:09:03.609571+0000","last_clean":"2023-10-06T03:09:03.609571+0000","last_became_active":"2023-10-06T03:09:03.609294+0000","last_became_peered":"2023-10-06T03:09:03.609294+0000","last_unstale":"2023-10-06T03:09:03.609571+0000","last_undegraded":"2023-10-06T03:09:03.609571+0000","last_fullsized":"2023-10-06T03:09:03.609571+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:11.713416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603971+0000","last_change":"2023-10-06T03:09:03.603971+0000","last_active":"2023-10-06T03:09:03.603971+0000","last_peered":"2023-10-06T03:09:03.603971+0000","last_clean":"2023-10-06T03:09:03.603971+0000","last_became_active":"2023-10-06T03:09:03.603701+0000","last_became_peered":"2023-10-06T03:09:03.603701+0000","last_unstale":"2023-10-06T03:09:03.603971+0000","last_undegraded":"2023-10-06T03:09:03.603971+0000","last_fullsized":"2023-10-06T03:09:03.603971+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:31.713110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603858+0000","last_change":"2023-10-06T03:09:03.603858+0000","last_active":"2023-10-06T03:09:03.603858+0000","last_peered":"2023-10-06T03:09:03.603858+0000","last_clean":"2023-10-06T03:09:03.603858+0000","last_became_active":"2023-10-06T03:09:03.603607+0000","last_became_peered":"2023-10-06T03:09:03.603607+0000","last_unstale":"2023-10-06T03:09:03.603858+0000","last_undegraded":"2023-10-06T03:09:03.603858+0000","last_fullsized":"2023-10-06T03:09:03.603858+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:13.567731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600263+0000","last_change":"2023-10-06T03:09:03.600263+0000","last_active":"2023-10-06T03:09:03.600263+0000","last_peered":"2023-10-06T03:09:03.600263+0000","last_clean":"2023-10-06T03:09:03.600263+0000","last_became_active":"2023-10-06T03:09:03.599870+0000","last_became_peered":"2023-10-06T03:09:03.599870+0000","last_unstale":"2023-10-06T03:09:03.600263+0000","last_undegraded":"2023-10-06T03:09:03.600263+0000","last_fullsized":"2023-10-06T03:09:03.600263+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:54.428380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.612259+0000","last_change":"2023-10-06T03:09:03.612259+0000","last_active":"2023-10-06T03:09:03.612259+0000","last_peered":"2023-10-06T03:09:03.612259+0000","last_clean":"2023-10-06T03:09:03.612259+0000","last_became_active":"2023-10-06T03:09:03.612007+0000","last_became_peered":"2023-10-06T03:09:03.612007+0000","last_unstale":"2023-10-06T03:09:03.612259+0000","last_undegraded":"2023-10-06T03:09:03.612259+0000","last_fullsized":"2023-10-06T03:09:03.612259+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:48.578908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605395+0000","last_change":"2023-10-06T03:09:03.605395+0000","last_active":"2023-10-06T03:09:03.605395+0000","last_peered":"2023-10-06T03:09:03.605395+0000","last_clean":"2023-10-06T03:09:03.605395+0000","last_became_active":"2023-10-06T03:09:03.605151+0000","last_became_peered":"2023-10-06T03:09:03.605151+0000","last_unstale":"2023-10-06T03:09:03.605395+0000","last_undegraded":"2023-10-06T03:09:03.605395+0000","last_fullsized":"2023-10-06T03:09:03.605395+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:06.153319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597827+0000","last_change":"2023-10-06T03:09:03.597827+0000","last_active":"2023-10-06T03:09:03.597827+0000","last_peered":"2023-10-06T03:09:03.597827+0000","last_clean":"2023-10-06T03:09:03.597827+0000","last_became_active":"2023-10-06T03:09:03.597568+0000","last_became_peered":"2023-10-06T03:09:03.597568+0000","last_unstale":"2023-10-06T03:09:03.597827+0000","last_undegraded":"2023-10-06T03:09:03.597827+0000","last_fullsized":"2023-10-06T03:09:03.597827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:41.161716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601195+0000","last_change":"2023-10-06T03:09:03.601195+0000","last_active":"2023-10-06T03:09:03.601195+0000","last_peered":"2023-10-06T03:09:03.601195+0000","last_clean":"2023-10-06T03:09:03.601195+0000","last_became_active":"2023-10-06T03:09:03.600604+0000","last_became_peered":"2023-10-06T03:09:03.600604+0000","last_unstale":"2023-10-06T03:09:03.601195+0000","last_undegraded":"2023-10-06T03:09:03.601195+0000","last_fullsized":"2023-10-06T03:09:03.601195+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:27.600148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605976+0000","last_change":"2023-10-06T03:09:03.605976+0000","last_active":"2023-10-06T03:09:03.605976+0000","last_peered":"2023-10-06T03:09:03.605976+0000","last_clean":"2023-10-06T03:09:03.605976+0000","last_became_active":"2023-10-06T03:09:03.602004+0000","last_became_peered":"2023-10-06T03:09:03.602004+0000","last_unstale":"2023-10-06T03:09:03.605976+0000","last_undegraded":"2023-10-06T03:09:03.605976+0000","last_fullsized":"2023-10-06T03:09:03.605976+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:27.409021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599232+0000","last_change":"2023-10-06T03:09:03.599232+0000","last_active":"2023-10-06T03:09:03.599232+0000","last_peered":"2023-10-06T03:09:03.599232+0000","last_clean":"2023-10-06T03:09:03.599232+0000","last_became_active":"2023-10-06T03:09:03.598963+0000","last_became_peered":"2023-10-06T03:09:03.598963+0000","last_unstale":"2023-10-06T03:09:03.599232+0000","last_undegraded":"2023-10-06T03:09:03.599232+0000","last_fullsized":"2023-10-06T03:09:03.599232+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.523660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.608202+0000","last_change":"2023-10-06T03:09:03.608202+0000","last_active":"2023-10-06T03:09:03.608202+0000","last_peered":"2023-10-06T03:09:03.608202+0000","last_clean":"2023-10-06T03:09:03.608202+0000","last_became_active":"2023-10-06T03:09:03.607925+0000","last_became_peered":"2023-10-06T03:09:03.607925+0000","last_unstale":"2023-10-06T03:09:03.608202+0000","last_undegraded":"2023-10-06T03:09:03.608202+0000","last_fullsized":"2023-10-06T03:09:03.608202+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.596702+0000","last_change":"2023-10-06T03:09:02.594352+0000","last_active":"2023-10-06T03:09:02.596702+0000","last_peered":"2023-10-06T03:09:02.596702+0000","last_clean":"2023-10-06T03:09:02.596702+0000","last_became_active":"2023-10-06T03:09:02.594116+0000","last_became_peered":"2023-10-06T03:09:02.594116+0000","last_unstale":"2023-10-06T03:09:02.596702+0000","last_undegraded":"2023-10-06T03:09:02.596702+0000","last_fullsized":"2023-10-06T03:09:02.596702+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:26.956521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597024+0000","last_change":"2023-10-06T03:09:03.597024+0000","last_active":"2023-10-06T03:09:03.597024+0000","last_peered":"2023-10-06T03:09:03.597024+0000","last_clean":"2023-10-06T03:09:03.597024+0000","last_became_active":"2023-10-06T03:09:03.596730+0000","last_became_peered":"2023-10-06T03:09:03.596730+0000","last_unstale":"2023-10-06T03:09:03.597024+0000","last_undegraded":"2023-10-06T03:09:03.597024+0000","last_fullsized":"2023-10-06T03:09:03.597024+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:52.782664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.598730+0000","last_change":"2023-10-06T03:09:02.594280+0000","last_active":"2023-10-06T03:09:02.598730+0000","last_peered":"2023-10-06T03:09:02.598730+0000","last_clean":"2023-10-06T03:09:02.598730+0000","last_became_active":"2023-10-06T03:09:02.594043+0000","last_became_peered":"2023-10-06T03:09:02.594043+0000","last_unstale":"2023-10-06T03:09:02.598730+0000","last_undegraded":"2023-10-06T03:09:02.598730+0000","last_fullsized":"2023-10-06T03:09:02.598730+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:57.130837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601112+0000","last_change":"2023-10-06T03:09:03.601112+0000","last_active":"2023-10-06T03:09:03.601112+0000","last_peered":"2023-10-06T03:09:03.601112+0000","last_clean":"2023-10-06T03:09:03.601112+0000","last_became_active":"2023-10-06T03:09:03.599380+0000","last_became_peered":"2023-10-06T03:09:03.599380+0000","last_unstale":"2023-10-06T03:09:03.601112+0000","last_undegraded":"2023-10-06T03:09:03.601112+0000","last_fullsized":"2023-10-06T03:09:03.601112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:13.664120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"139'3","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611193+0000","last_change":"2023-10-06T03:09:02.596986+0000","last_active":"2023-10-06T03:09:03.611193+0000","last_peered":"2023-10-06T03:09:03.611193+0000","last_clean":"2023-10-06T03:09:03.611193+0000","last_became_active":"2023-10-06T03:09:02.596722+0000","last_became_peered":"2023-10-06T03:09:02.596722+0000","last_unstale":"2023-10-06T03:09:03.611193+0000","last_undegraded":"2023-10-06T03:09:03.611193+0000","last_fullsized":"2023-10-06T03:09:03.611193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:15.392027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605044+0000","last_change":"2023-10-06T03:09:03.605044+0000","last_active":"2023-10-06T03:09:03.605044+0000","last_peered":"2023-10-06T03:09:03.605044+0000","last_clean":"2023-10-06T03:09:03.605044+0000","last_became_active":"2023-10-06T03:09:03.604621+0000","last_became_peered":"2023-10-06T03:09:03.604621+0000","last_unstale":"2023-10-06T03:09:03.605044+0000","last_undegraded":"2023-10-06T03:09:03.605044+0000","last_fullsized":"2023-10-06T03:09:03.605044+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"139'6","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.616253+0000","last_change":"2023-10-06T03:09:02.583136+0000","last_active":"2023-10-06T03:09:03.616253+0000","last_peered":"2023-10-06T03:09:03.616253+0000","last_clean":"2023-10-06T03:09:03.616253+0000","last_became_active":"2023-10-06T03:09:02.582851+0000","last_became_peered":"2023-10-06T03:09:02.582851+0000","last_unstale":"2023-10-06T03:09:03.616253+0000","last_undegraded":"2023-10-06T03:09:03.616253+0000","last_fullsized":"2023-10-06T03:09:03.616253+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:57.308794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604572+0000","last_change":"2023-10-06T03:09:03.604572+0000","last_active":"2023-10-06T03:09:03.604572+0000","last_peered":"2023-10-06T03:09:03.604572+0000","last_clean":"2023-10-06T03:09:03.604572+0000","last_became_active":"2023-10-06T03:09:03.604320+0000","last_became_peered":"2023-10-06T03:09:03.604320+0000","last_unstale":"2023-10-06T03:09:03.604572+0000","last_undegraded":"2023-10-06T03:09:03.604572+0000","last_fullsized":"2023-10-06T03:09:03.604572+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:06.646632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613143+0000","last_change":"2023-10-06T03:09:02.593978+0000","last_active":"2023-10-06T03:09:03.613143+0000","last_peered":"2023-10-06T03:09:03.613143+0000","last_clean":"2023-10-06T03:09:03.613143+0000","last_became_active":"2023-10-06T03:09:02.593670+0000","last_became_peered":"2023-10-06T03:09:02.593670+0000","last_unstale":"2023-10-06T03:09:03.613143+0000","last_undegraded":"2023-10-06T03:09:03.613143+0000","last_fullsized":"2023-10-06T03:09:03.613143+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:24.165492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605983+0000","last_change":"2023-10-06T03:09:03.605983+0000","last_active":"2023-10-06T03:09:03.605983+0000","last_peered":"2023-10-06T03:09:03.605983+0000","last_clean":"2023-10-06T03:09:03.605983+0000","last_became_active":"2023-10-06T03:09:03.605715+0000","last_became_peered":"2023-10-06T03:09:03.605715+0000","last_unstale":"2023-10-06T03:09:03.605983+0000","last_undegraded":"2023-10-06T03:09:03.605983+0000","last_fullsized":"2023-10-06T03:09:03.605983+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:49.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.598493+0000","last_change":"2023-10-06T03:09:02.588724+0000","last_active":"2023-10-06T03:09:02.598493+0000","last_peered":"2023-10-06T03:09:02.598493+0000","last_clean":"2023-10-06T03:09:02.598493+0000","last_became_active":"2023-10-06T03:09:02.588455+0000","last_became_peered":"2023-10-06T03:09:02.588455+0000","last_unstale":"2023-10-06T03:09:02.598493+0000","last_undegraded":"2023-10-06T03:09:02.598493+0000","last_fullsized":"2023-10-06T03:09:02.598493+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:28.520118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599135+0000","last_change":"2023-10-06T03:09:03.599135+0000","last_active":"2023-10-06T03:09:03.599135+0000","last_peered":"2023-10-06T03:09:03.599135+0000","last_clean":"2023-10-06T03:09:03.599135+0000","last_became_active":"2023-10-06T03:09:03.598858+0000","last_became_peered":"2023-10-06T03:09:03.598858+0000","last_unstale":"2023-10-06T03:09:03.599135+0000","last_undegraded":"2023-10-06T03:09:03.599135+0000","last_fullsized":"2023-10-06T03:09:03.599135+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.373485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.590302+0000","last_change":"2023-10-06T03:09:02.589625+0000","last_active":"2023-10-06T03:09:02.590302+0000","last_peered":"2023-10-06T03:09:02.590302+0000","last_clean":"2023-10-06T03:09:02.590302+0000","last_became_active":"2023-10-06T03:09:02.589381+0000","last_became_peered":"2023-10-06T03:09:02.589381+0000","last_unstale":"2023-10-06T03:09:02.590302+0000","last_undegraded":"2023-10-06T03:09:02.590302+0000","last_fullsized":"2023-10-06T03:09:02.590302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:47.574762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600682+0000","last_change":"2023-10-06T03:09:03.600682+0000","last_active":"2023-10-06T03:09:03.600682+0000","last_peered":"2023-10-06T03:09:03.600682+0000","last_clean":"2023-10-06T03:09:03.600682+0000","last_became_active":"2023-10-06T03:09:03.600177+0000","last_became_peered":"2023-10-06T03:09:03.600177+0000","last_unstale":"2023-10-06T03:09:03.600682+0000","last_undegraded":"2023-10-06T03:09:03.600682+0000","last_fullsized":"2023-10-06T03:09:03.600682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:19.560203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"139'8","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.615651+0000","last_change":"2023-10-06T03:09:02.587225+0000","last_active":"2023-10-06T03:09:03.615651+0000","last_peered":"2023-10-06T03:09:03.615651+0000","last_clean":"2023-10-06T03:09:03.615651+0000","last_became_active":"2023-10-06T03:09:02.586972+0000","last_became_peered":"2023-10-06T03:09:02.586972+0000","last_unstale":"2023-10-06T03:09:03.615651+0000","last_undegraded":"2023-10-06T03:09:03.615651+0000","last_fullsized":"2023-10-06T03:09:03.615651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:06.753575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604149+0000","last_change":"2023-10-06T03:09:03.604149+0000","last_active":"2023-10-06T03:09:03.604149+0000","last_peered":"2023-10-06T03:09:03.604149+0000","last_clean":"2023-10-06T03:09:03.604149+0000","last_became_active":"2023-10-06T03:09:03.601859+0000","last_became_peered":"2023-10-06T03:09:03.601859+0000","last_unstale":"2023-10-06T03:09:03.604149+0000","last_undegraded":"2023-10-06T03:09:03.604149+0000","last_fullsized":"2023-10-06T03:09:03.604149+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:31.358739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.590445+0000","last_change":"2023-10-06T03:09:02.590110+0000","last_active":"2023-10-06T03:09:03.590445+0000","last_peered":"2023-10-06T03:09:03.590445+0000","last_clean":"2023-10-06T03:09:03.590445+0000","last_became_active":"2023-10-06T03:09:02.589851+0000","last_became_peered":"2023-10-06T03:09:02.589851+0000","last_unstale":"2023-10-06T03:09:03.590445+0000","last_undegraded":"2023-10-06T03:09:03.590445+0000","last_fullsized":"2023-10-06T03:09:03.590445+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:22.528211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599172+0000","last_change":"2023-10-06T03:09:03.599172+0000","last_active":"2023-10-06T03:09:03.599172+0000","last_peered":"2023-10-06T03:09:03.599172+0000","last_clean":"2023-10-06T03:09:03.599172+0000","last_became_active":"2023-10-06T03:09:03.598930+0000","last_became_peered":"2023-10-06T03:09:03.598930+0000","last_unstale":"2023-10-06T03:09:03.599172+0000","last_undegraded":"2023-10-06T03:09:03.599172+0000","last_fullsized":"2023-10-06T03:09:03.599172+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:11.713416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613202+0000","last_change":"2023-10-06T03:09:02.588048+0000","last_active":"2023-10-06T03:09:03.613202+0000","last_peered":"2023-10-06T03:09:03.613202+0000","last_clean":"2023-10-06T03:09:03.613202+0000","last_became_active":"2023-10-06T03:09:02.587787+0000","last_became_peered":"2023-10-06T03:09:02.587787+0000","last_unstale":"2023-10-06T03:09:03.613202+0000","last_undegraded":"2023-10-06T03:09:03.613202+0000","last_fullsized":"2023-10-06T03:09:03.613202+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:38.567412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601862+0000","last_change":"2023-10-06T03:09:03.601862+0000","last_active":"2023-10-06T03:09:03.601862+0000","last_peered":"2023-10-06T03:09:03.601862+0000","last_clean":"2023-10-06T03:09:03.601862+0000","last_became_active":"2023-10-06T03:09:03.600210+0000","last_became_peered":"2023-10-06T03:09:03.600210+0000","last_unstale":"2023-10-06T03:09:03.601862+0000","last_undegraded":"2023-10-06T03:09:03.601862+0000","last_fullsized":"2023-10-06T03:09:03.601862+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:09.933703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.596885+0000","last_change":"2023-10-06T03:09:02.594641+0000","last_active":"2023-10-06T03:09:03.596885+0000","last_peered":"2023-10-06T03:09:03.596885+0000","last_clean":"2023-10-06T03:09:03.596885+0000","last_became_active":"2023-10-06T03:09:02.594351+0000","last_became_peered":"2023-10-06T03:09:02.594351+0000","last_unstale":"2023-10-06T03:09:03.596885+0000","last_undegraded":"2023-10-06T03:09:03.596885+0000","last_fullsized":"2023-10-06T03:09:03.596885+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:33:35.271403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600917+0000","last_change":"2023-10-06T03:09:03.600917+0000","last_active":"2023-10-06T03:09:03.600917+0000","last_peered":"2023-10-06T03:09:03.600917+0000","last_clean":"2023-10-06T03:09:03.600917+0000","last_became_active":"2023-10-06T03:09:03.600669+0000","last_became_peered":"2023-10-06T03:09:03.600669+0000","last_unstale":"2023-10-06T03:09:03.600917+0000","last_undegraded":"2023-10-06T03:09:03.600917+0000","last_fullsized":"2023-10-06T03:09:03.600917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:55.433899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.593814+0000","last_change":"2023-10-06T03:09:02.590839+0000","last_active":"2023-10-06T03:09:02.593814+0000","last_peered":"2023-10-06T03:09:02.593814+0000","last_clean":"2023-10-06T03:09:02.593814+0000","last_became_active":"2023-10-06T03:09:02.590530+0000","last_became_peered":"2023-10-06T03:09:02.590530+0000","last_unstale":"2023-10-06T03:09:02.593814+0000","last_undegraded":"2023-10-06T03:09:02.593814+0000","last_fullsized":"2023-10-06T03:09:02.593814+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:21.899272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597759+0000","last_change":"2023-10-06T03:09:03.597759+0000","last_active":"2023-10-06T03:09:03.597759+0000","last_peered":"2023-10-06T03:09:03.597759+0000","last_clean":"2023-10-06T03:09:03.597759+0000","last_became_active":"2023-10-06T03:09:03.597424+0000","last_became_peered":"2023-10-06T03:09:03.597424+0000","last_unstale":"2023-10-06T03:09:03.597759+0000","last_undegraded":"2023-10-06T03:09:03.597759+0000","last_fullsized":"2023-10-06T03:09:03.597759+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:02.247707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613278+0000","last_change":"2023-10-06T03:09:02.595147+0000","last_active":"2023-10-06T03:09:03.613278+0000","last_peered":"2023-10-06T03:09:03.613278+0000","last_clean":"2023-10-06T03:09:03.613278+0000","last_became_active":"2023-10-06T03:09:02.594828+0000","last_became_peered":"2023-10-06T03:09:02.594828+0000","last_unstale":"2023-10-06T03:09:03.613278+0000","last_undegraded":"2023-10-06T03:09:03.613278+0000","last_fullsized":"2023-10-06T03:09:03.613278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.293211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600556+0000","last_change":"2023-10-06T03:09:03.600556+0000","last_active":"2023-10-06T03:09:03.600556+0000","last_peered":"2023-10-06T03:09:03.600556+0000","last_clean":"2023-10-06T03:09:03.600556+0000","last_became_active":"2023-10-06T03:09:03.600292+0000","last_became_peered":"2023-10-06T03:09:03.600292+0000","last_unstale":"2023-10-06T03:09:03.600556+0000","last_undegraded":"2023-10-06T03:09:03.600556+0000","last_fullsized":"2023-10-06T03:09:03.600556+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:06.153319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.596108+0000","last_change":"2023-10-06T03:09:02.592540+0000","last_active":"2023-10-06T03:09:02.596108+0000","last_peered":"2023-10-06T03:09:02.596108+0000","last_clean":"2023-10-06T03:09:02.596108+0000","last_became_active":"2023-10-06T03:09:02.592298+0000","last_became_peered":"2023-10-06T03:09:02.592298+0000","last_unstale":"2023-10-06T03:09:02.596108+0000","last_undegraded":"2023-10-06T03:09:02.596108+0000","last_fullsized":"2023-10-06T03:09:02.596108+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:52:48.348722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604620+0000","last_change":"2023-10-06T03:09:03.604620+0000","last_active":"2023-10-06T03:09:03.604620+0000","last_peered":"2023-10-06T03:09:03.604620+0000","last_clean":"2023-10-06T03:09:03.604620+0000","last_became_active":"2023-10-06T03:09:03.604362+0000","last_became_peered":"2023-10-06T03:09:03.604362+0000","last_unstale":"2023-10-06T03:09:03.604620+0000","last_undegraded":"2023-10-06T03:09:03.604620+0000","last_fullsized":"2023-10-06T03:09:03.604620+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:50:53.646433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.600287+0000","last_change":"2023-10-06T03:09:02.599435+0000","last_active":"2023-10-06T03:09:02.600287+0000","last_peered":"2023-10-06T03:09:02.600287+0000","last_clean":"2023-10-06T03:09:02.600287+0000","last_became_active":"2023-10-06T03:09:02.598893+0000","last_became_peered":"2023-10-06T03:09:02.598893+0000","last_unstale":"2023-10-06T03:09:02.600287+0000","last_undegraded":"2023-10-06T03:09:02.600287+0000","last_fullsized":"2023-10-06T03:09:02.600287+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:56.030638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.606194+0000","last_change":"2023-10-06T03:09:03.606194+0000","last_active":"2023-10-06T03:09:03.606194+0000","last_peered":"2023-10-06T03:09:03.606194+0000","last_clean":"2023-10-06T03:09:03.606194+0000","last_became_active":"2023-10-06T03:09:03.605882+0000","last_became_peered":"2023-10-06T03:09:03.605882+0000","last_unstale":"2023-10-06T03:09:03.606194+0000","last_undegraded":"2023-10-06T03:09:03.606194+0000","last_fullsized":"2023-10-06T03:09:03.606194+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:08.981267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.594228+0000","last_change":"2023-10-06T03:09:02.589875+0000","last_active":"2023-10-06T03:09:02.594228+0000","last_peered":"2023-10-06T03:09:02.594228+0000","last_clean":"2023-10-06T03:09:02.594228+0000","last_became_active":"2023-10-06T03:09:02.589625+0000","last_became_peered":"2023-10-06T03:09:02.589625+0000","last_unstale":"2023-10-06T03:09:02.594228+0000","last_undegraded":"2023-10-06T03:09:02.594228+0000","last_fullsized":"2023-10-06T03:09:02.594228+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:30.354593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.596714+0000","last_change":"2023-10-06T03:09:03.596714+0000","last_active":"2023-10-06T03:09:03.596714+0000","last_peered":"2023-10-06T03:09:03.596714+0000","last_clean":"2023-10-06T03:09:03.596714+0000","last_became_active":"2023-10-06T03:09:03.596448+0000","last_became_peered":"2023-10-06T03:09:03.596448+0000","last_unstale":"2023-10-06T03:09:03.596714+0000","last_undegraded":"2023-10-06T03:09:03.596714+0000","last_fullsized":"2023-10-06T03:09:03.596714+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:21.131263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613023+0000","last_change":"2023-10-06T03:09:02.591080+0000","last_active":"2023-10-06T03:09:03.613023+0000","last_peered":"2023-10-06T03:09:03.613023+0000","last_clean":"2023-10-06T03:09:03.613023+0000","last_became_active":"2023-10-06T03:09:02.590372+0000","last_became_peered":"2023-10-06T03:09:02.590372+0000","last_unstale":"2023-10-06T03:09:03.613023+0000","last_undegraded":"2023-10-06T03:09:03.613023+0000","last_fullsized":"2023-10-06T03:09:03.613023+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:43.755069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611222+0000","last_change":"2023-10-06T03:09:03.611222+0000","last_active":"2023-10-06T03:09:03.611222+0000","last_peered":"2023-10-06T03:09:03.611222+0000","last_clean":"2023-10-06T03:09:03.611222+0000","last_became_active":"2023-10-06T03:09:03.610922+0000","last_became_peered":"2023-10-06T03:09:03.610922+0000","last_unstale":"2023-10-06T03:09:03.611222+0000","last_undegraded":"2023-10-06T03:09:03.611222+0000","last_fullsized":"2023-10-06T03:09:03.611222+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:24.373485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.600281+0000","last_change":"2023-10-06T03:09:02.599168+0000","last_active":"2023-10-06T03:09:02.600281+0000","last_peered":"2023-10-06T03:09:02.600281+0000","last_clean":"2023-10-06T03:09:02.600281+0000","last_became_active":"2023-10-06T03:09:02.598919+0000","last_became_peered":"2023-10-06T03:09:02.598919+0000","last_unstale":"2023-10-06T03:09:02.600281+0000","last_undegraded":"2023-10-06T03:09:02.600281+0000","last_fullsized":"2023-10-06T03:09:02.600281+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:53.973425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.607000+0000","last_change":"2023-10-06T03:09:03.607000+0000","last_active":"2023-10-06T03:09:03.607000+0000","last_peered":"2023-10-06T03:09:03.607000+0000","last_clean":"2023-10-06T03:09:03.607000+0000","last_became_active":"2023-10-06T03:09:03.606724+0000","last_became_peered":"2023-10-06T03:09:03.606724+0000","last_unstale":"2023-10-06T03:09:03.607000+0000","last_undegraded":"2023-10-06T03:09:03.607000+0000","last_fullsized":"2023-10-06T03:09:03.607000+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:28:16.396173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.596789+0000","last_change":"2023-10-06T03:09:02.595716+0000","last_active":"2023-10-06T03:09:02.596789+0000","last_peered":"2023-10-06T03:09:02.596789+0000","last_clean":"2023-10-06T03:09:02.596789+0000","last_became_active":"2023-10-06T03:09:02.595036+0000","last_became_peered":"2023-10-06T03:09:02.595036+0000","last_unstale":"2023-10-06T03:09:02.596789+0000","last_undegraded":"2023-10-06T03:09:02.596789+0000","last_fullsized":"2023-10-06T03:09:02.596789+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:12.659974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.605602+0000","last_change":"2023-10-06T03:09:03.605602+0000","last_active":"2023-10-06T03:09:03.605602+0000","last_peered":"2023-10-06T03:09:03.605602+0000","last_clean":"2023-10-06T03:09:03.605602+0000","last_became_active":"2023-10-06T03:09:03.605375+0000","last_became_peered":"2023-10-06T03:09:03.605375+0000","last_unstale":"2023-10-06T03:09:03.605602+0000","last_undegraded":"2023-10-06T03:09:03.605602+0000","last_fullsized":"2023-10-06T03:09:03.605602+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:24.336606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.600063+0000","last_change":"2023-10-06T03:09:02.599232+0000","last_active":"2023-10-06T03:09:02.600063+0000","last_peered":"2023-10-06T03:09:02.600063+0000","last_clean":"2023-10-06T03:09:02.600063+0000","last_became_active":"2023-10-06T03:09:02.598896+0000","last_became_peered":"2023-10-06T03:09:02.598896+0000","last_unstale":"2023-10-06T03:09:02.600063+0000","last_undegraded":"2023-10-06T03:09:02.600063+0000","last_fullsized":"2023-10-06T03:09:02.600063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:53.424235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.597922+0000","last_change":"2023-10-06T03:09:03.597922+0000","last_active":"2023-10-06T03:09:03.597922+0000","last_peered":"2023-10-06T03:09:03.597922+0000","last_clean":"2023-10-06T03:09:03.597922+0000","last_became_active":"2023-10-06T03:09:03.597664+0000","last_became_peered":"2023-10-06T03:09:03.597664+0000","last_unstale":"2023-10-06T03:09:03.597922+0000","last_undegraded":"2023-10-06T03:09:03.597922+0000","last_fullsized":"2023-10-06T03:09:03.597922+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:44.846505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.593833+0000","last_change":"2023-10-06T03:09:02.590939+0000","last_active":"2023-10-06T03:09:02.593833+0000","last_peered":"2023-10-06T03:09:02.593833+0000","last_clean":"2023-10-06T03:09:02.593833+0000","last_became_active":"2023-10-06T03:09:02.590679+0000","last_became_peered":"2023-10-06T03:09:02.590679+0000","last_unstale":"2023-10-06T03:09:02.593833+0000","last_undegraded":"2023-10-06T03:09:02.593833+0000","last_fullsized":"2023-10-06T03:09:02.593833+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:40:38.118380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.607868+0000","last_change":"2023-10-06T03:09:03.607868+0000","last_active":"2023-10-06T03:09:03.607868+0000","last_peered":"2023-10-06T03:09:03.607868+0000","last_clean":"2023-10-06T03:09:03.607868+0000","last_became_active":"2023-10-06T03:09:03.607612+0000","last_became_peered":"2023-10-06T03:09:03.607612+0000","last_unstale":"2023-10-06T03:09:03.607868+0000","last_undegraded":"2023-10-06T03:09:03.607868+0000","last_fullsized":"2023-10-06T03:09:03.607868+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:02.247707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.597512+0000","last_change":"2023-10-06T03:09:02.597043+0000","last_active":"2023-10-06T03:09:02.597512+0000","last_peered":"2023-10-06T03:09:02.597512+0000","last_clean":"2023-10-06T03:09:02.597512+0000","last_became_active":"2023-10-06T03:09:02.596801+0000","last_became_peered":"2023-10-06T03:09:02.596801+0000","last_unstale":"2023-10-06T03:09:02.597512+0000","last_undegraded":"2023-10-06T03:09:02.597512+0000","last_fullsized":"2023-10-06T03:09:02.597512+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:26.956521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.600884+0000","last_change":"2023-10-06T03:09:03.600884+0000","last_active":"2023-10-06T03:09:03.600884+0000","last_peered":"2023-10-06T03:09:03.600884+0000","last_clean":"2023-10-06T03:09:03.600884+0000","last_became_active":"2023-10-06T03:09:03.600620+0000","last_became_peered":"2023-10-06T03:09:03.600620+0000","last_unstale":"2023-10-06T03:09:03.600884+0000","last_undegraded":"2023-10-06T03:09:03.600884+0000","last_fullsized":"2023-10-06T03:09:03.600884+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:19.560203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.591478+0000","last_change":"2023-10-06T03:09:02.587671+0000","last_active":"2023-10-06T03:09:02.591478+0000","last_peered":"2023-10-06T03:09:02.591478+0000","last_clean":"2023-10-06T03:09:02.591478+0000","last_became_active":"2023-10-06T03:09:02.587399+0000","last_became_peered":"2023-10-06T03:09:02.587399+0000","last_unstale":"2023-10-06T03:09:02.591478+0000","last_undegraded":"2023-10-06T03:09:02.591478+0000","last_fullsized":"2023-10-06T03:09:02.591478+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:34.570119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604687+0000","last_change":"2023-10-06T03:09:03.604687+0000","last_active":"2023-10-06T03:09:03.604687+0000","last_peered":"2023-10-06T03:09:03.604687+0000","last_clean":"2023-10-06T03:09:03.604687+0000","last_became_active":"2023-10-06T03:09:03.604154+0000","last_became_peered":"2023-10-06T03:09:03.604154+0000","last_unstale":"2023-10-06T03:09:03.604687+0000","last_undegraded":"2023-10-06T03:09:03.604687+0000","last_fullsized":"2023-10-06T03:09:03.604687+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:11.961093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.594462+0000","last_change":"2023-10-06T03:09:02.593421+0000","last_active":"2023-10-06T03:09:02.594462+0000","last_peered":"2023-10-06T03:09:02.594462+0000","last_clean":"2023-10-06T03:09:02.594462+0000","last_became_active":"2023-10-06T03:09:02.593135+0000","last_became_peered":"2023-10-06T03:09:02.593135+0000","last_unstale":"2023-10-06T03:09:02.594462+0000","last_undegraded":"2023-10-06T03:09:02.594462+0000","last_fullsized":"2023-10-06T03:09:02.594462+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:33:12.563585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603781+0000","last_change":"2023-10-06T03:09:03.603781+0000","last_active":"2023-10-06T03:09:03.603781+0000","last_peered":"2023-10-06T03:09:03.603781+0000","last_clean":"2023-10-06T03:09:03.603781+0000","last_became_active":"2023-10-06T03:09:03.602754+0000","last_became_peered":"2023-10-06T03:09:03.602754+0000","last_unstale":"2023-10-06T03:09:03.603781+0000","last_undegraded":"2023-10-06T03:09:03.603781+0000","last_fullsized":"2023-10-06T03:09:03.603781+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:29.522686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613694+0000","last_change":"2023-10-06T03:09:02.587566+0000","last_active":"2023-10-06T03:09:03.613694+0000","last_peered":"2023-10-06T03:09:03.613694+0000","last_clean":"2023-10-06T03:09:03.613694+0000","last_became_active":"2023-10-06T03:09:02.587231+0000","last_became_peered":"2023-10-06T03:09:02.587231+0000","last_unstale":"2023-10-06T03:09:03.613694+0000","last_undegraded":"2023-10-06T03:09:03.613694+0000","last_fullsized":"2023-10-06T03:09:03.613694+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:46.132169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.585358+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:09:02.585358+0000","last_peered":"2023-10-06T03:09:02.585358+0000","last_clean":"2023-10-06T03:09:02.585358+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:09:02.585358+0000","last_undegraded":"2023-10-06T03:09:02.585358+0000","last_fullsized":"2023-10-06T03:09:02.585358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.601262+0000","last_change":"2023-10-06T03:09:03.601262+0000","last_active":"2023-10-06T03:09:03.601262+0000","last_peered":"2023-10-06T03:09:03.601262+0000","last_clean":"2023-10-06T03:09:03.601262+0000","last_became_active":"2023-10-06T03:09:03.600839+0000","last_became_peered":"2023-10-06T03:09:03.600839+0000","last_unstale":"2023-10-06T03:09:03.601262+0000","last_undegraded":"2023-10-06T03:09:03.601262+0000","last_fullsized":"2023-10-06T03:09:03.601262+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:08.114339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"139'2","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613866+0000","last_change":"2023-10-06T03:09:02.584840+0000","last_active":"2023-10-06T03:09:03.613866+0000","last_peered":"2023-10-06T03:09:03.613866+0000","last_clean":"2023-10-06T03:09:03.613866+0000","last_became_active":"2023-10-06T03:09:02.584571+0000","last_became_peered":"2023-10-06T03:09:02.584571+0000","last_unstale":"2023-10-06T03:09:03.613866+0000","last_undegraded":"2023-10-06T03:09:03.613866+0000","last_fullsized":"2023-10-06T03:09:03.613866+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:52.558587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577586+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:09:02.577586+0000","last_peered":"2023-10-06T03:09:02.577586+0000","last_clean":"2023-10-06T03:09:02.577586+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:09:02.577586+0000","last_undegraded":"2023-10-06T03:09:02.577586+0000","last_fullsized":"2023-10-06T03:09:02.577586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.604987+0000","last_change":"2023-10-06T03:09:03.604987+0000","last_active":"2023-10-06T03:09:03.604987+0000","last_peered":"2023-10-06T03:09:03.604987+0000","last_clean":"2023-10-06T03:09:03.604987+0000","last_became_active":"2023-10-06T03:09:03.601903+0000","last_became_peered":"2023-10-06T03:09:03.601903+0000","last_unstale":"2023-10-06T03:09:03.604987+0000","last_undegraded":"2023-10-06T03:09:03.604987+0000","last_fullsized":"2023-10-06T03:09:03.604987+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:19:29.357290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.588544+0000","last_change":"2023-10-06T03:09:02.595428+0000","last_active":"2023-10-06T03:09:03.588544+0000","last_peered":"2023-10-06T03:09:03.588544+0000","last_clean":"2023-10-06T03:09:03.588544+0000","last_became_active":"2023-10-06T03:09:02.595144+0000","last_became_peered":"2023-10-06T03:09:02.595144+0000","last_unstale":"2023-10-06T03:09:03.588544+0000","last_undegraded":"2023-10-06T03:09:03.588544+0000","last_fullsized":"2023-10-06T03:09:03.588544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:36.112186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.585773+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:09:02.585773+0000","last_peered":"2023-10-06T03:09:02.585773+0000","last_clean":"2023-10-06T03:09:02.585773+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:09:02.585773+0000","last_undegraded":"2023-10-06T03:09:02.585773+0000","last_fullsized":"2023-10-06T03:09:02.585773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.599682+0000","last_change":"2023-10-06T03:09:03.599682+0000","last_active":"2023-10-06T03:09:03.599682+0000","last_peered":"2023-10-06T03:09:03.599682+0000","last_clean":"2023-10-06T03:09:03.599682+0000","last_became_active":"2023-10-06T03:09:03.599210+0000","last_became_peered":"2023-10-06T03:09:03.599210+0000","last_unstale":"2023-10-06T03:09:03.599682+0000","last_undegraded":"2023-10-06T03:09:03.599682+0000","last_fullsized":"2023-10-06T03:09:03.599682+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:48.578908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.588670+0000","last_change":"2023-10-06T03:09:02.595589+0000","last_active":"2023-10-06T03:09:03.588670+0000","last_peered":"2023-10-06T03:09:03.588670+0000","last_clean":"2023-10-06T03:09:03.588670+0000","last_became_active":"2023-10-06T03:09:02.594526+0000","last_became_peered":"2023-10-06T03:09:02.594526+0000","last_unstale":"2023-10-06T03:09:03.588670+0000","last_undegraded":"2023-10-06T03:09:03.588670+0000","last_fullsized":"2023-10-06T03:09:03.588670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:08.929557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":111,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.583811+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:09:02.583811+0000","last_peered":"2023-10-06T03:09:02.583811+0000","last_clean":"2023-10-06T03:09:02.583811+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:09:02.583811+0000","last_undegraded":"2023-10-06T03:09:02.583811+0000","last_fullsized":"2023-10-06T03:09:02.583811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.603575+0000","last_change":"2023-10-06T03:09:03.603575+0000","last_active":"2023-10-06T03:09:03.603575+0000","last_peered":"2023-10-06T03:09:03.603575+0000","last_clean":"2023-10-06T03:09:03.603575+0000","last_became_active":"2023-10-06T03:09:03.603174+0000","last_became_peered":"2023-10-06T03:09:03.603174+0000","last_unstale":"2023-10-06T03:09:03.603575+0000","last_undegraded":"2023-10-06T03:09:03.603575+0000","last_fullsized":"2023-10-06T03:09:03.603575+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:26.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.597068+0000","last_change":"2023-10-06T03:09:02.594260+0000","last_active":"2023-10-06T03:09:02.597068+0000","last_peered":"2023-10-06T03:09:02.597068+0000","last_clean":"2023-10-06T03:09:02.597068+0000","last_became_active":"2023-10-06T03:09:02.594011+0000","last_became_peered":"2023-10-06T03:09:02.594011+0000","last_unstale":"2023-10-06T03:09:02.597068+0000","last_undegraded":"2023-10-06T03:09:02.597068+0000","last_fullsized":"2023-10-06T03:09:02.597068+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:10.709270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.585607+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:09:02.585607+0000","last_peered":"2023-10-06T03:09:02.585607+0000","last_clean":"2023-10-06T03:09:02.585607+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:09:02.585607+0000","last_undegraded":"2023-10-06T03:09:02.585607+0000","last_fullsized":"2023-10-06T03:09:02.585607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":123,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577955+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:09:02.577955+0000","last_peered":"2023-10-06T03:09:02.577955+0000","last_clean":"2023-10-06T03:09:02.577955+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:09:02.577955+0000","last_undegraded":"2023-10-06T03:09:02.577955+0000","last_fullsized":"2023-10-06T03:09:02.577955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.611479+0000","last_change":"2023-10-06T03:09:03.611479+0000","last_active":"2023-10-06T03:09:03.611479+0000","last_peered":"2023-10-06T03:09:03.611479+0000","last_clean":"2023-10-06T03:09:03.611479+0000","last_became_active":"2023-10-06T03:09:03.611101+0000","last_became_peered":"2023-10-06T03:09:03.611101+0000","last_unstale":"2023-10-06T03:09:03.611479+0000","last_undegraded":"2023-10-06T03:09:03.611479+0000","last_fullsized":"2023-10-06T03:09:03.611479+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:48.578908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.592175+0000","last_change":"2023-10-06T03:09:02.597026+0000","last_active":"2023-10-06T03:09:03.592175+0000","last_peered":"2023-10-06T03:09:03.592175+0000","last_clean":"2023-10-06T03:09:03.592175+0000","last_became_active":"2023-10-06T03:09:02.596775+0000","last_became_peered":"2023-10-06T03:09:02.596775+0000","last_unstale":"2023-10-06T03:09:03.592175+0000","last_undegraded":"2023-10-06T03:09:03.592175+0000","last_fullsized":"2023-10-06T03:09:03.592175+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:10.956947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.584315+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:09:02.584315+0000","last_peered":"2023-10-06T03:09:02.584315+0000","last_clean":"2023-10-06T03:09:02.584315+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:09:02.584315+0000","last_undegraded":"2023-10-06T03:09:02.584315+0000","last_fullsized":"2023-10-06T03:09:02.584315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602307+0000","last_change":"2023-10-06T03:09:03.602307+0000","last_active":"2023-10-06T03:09:03.602307+0000","last_peered":"2023-10-06T03:09:03.602307+0000","last_clean":"2023-10-06T03:09:03.602307+0000","last_became_active":"2023-10-06T03:09:03.602032+0000","last_became_peered":"2023-10-06T03:09:03.602032+0000","last_unstale":"2023-10-06T03:09:03.602307+0000","last_undegraded":"2023-10-06T03:09:03.602307+0000","last_fullsized":"2023-10-06T03:09:03.602307+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:44.759215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.590045+0000","last_change":"2023-10-06T03:09:02.588029+0000","last_active":"2023-10-06T03:09:02.590045+0000","last_peered":"2023-10-06T03:09:02.590045+0000","last_clean":"2023-10-06T03:09:02.590045+0000","last_became_active":"2023-10-06T03:09:02.587749+0000","last_became_peered":"2023-10-06T03:09:02.587749+0000","last_unstale":"2023-10-06T03:09:02.590045+0000","last_undegraded":"2023-10-06T03:09:02.590045+0000","last_fullsized":"2023-10-06T03:09:02.590045+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:08.128505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":123,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577848+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:09:02.577848+0000","last_peered":"2023-10-06T03:09:02.577848+0000","last_clean":"2023-10-06T03:09:02.577848+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:09:02.577848+0000","last_undegraded":"2023-10-06T03:09:02.577848+0000","last_fullsized":"2023-10-06T03:09:02.577848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.602951+0000","last_change":"2023-10-06T03:09:03.602951+0000","last_active":"2023-10-06T03:09:03.602951+0000","last_peered":"2023-10-06T03:09:03.602951+0000","last_clean":"2023-10-06T03:09:03.602951+0000","last_became_active":"2023-10-06T03:09:03.602652+0000","last_became_peered":"2023-10-06T03:09:03.602652+0000","last_unstale":"2023-10-06T03:09:03.602951+0000","last_undegraded":"2023-10-06T03:09:03.602951+0000","last_fullsized":"2023-10-06T03:09:03.602951+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:02.570063+0000","last_clean_scrub_stamp":"2023-10-06T03:09:02.570063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:14:44.759215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"139'4","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-06T03:09:03.613517+0000","last_change":"2023-10-06T03:09:02.588094+0000","last_active":"2023-10-06T03:09:03.613517+0000","last_peered":"2023-10-06T03:09:03.613517+0000","last_clean":"2023-10-06T03:09:03.613517+0000","last_became_active":"2023-10-06T03:09:02.587840+0000","last_became_peered":"2023-10-06T03:09:02.587840+0000","last_unstale":"2023-10-06T03:09:03.613517+0000","last_undegraded":"2023-10-06T03:09:03.613517+0000","last_fullsized":"2023-10-06T03:09:03.613517+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:01.565918+0000","last_clean_scrub_stamp":"2023-10-06T03:09:01.565918+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:50:02.326914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":113,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-06T03:09:02.577534+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:09:02.577534+0000","last_peered":"2023-10-06T03:09:02.577534+0000","last_clean":"2023-10-06T03:09:02.577534+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:09:02.577534+0000","last_undegraded":"2023-10-06T03:09:02.577534+0000","last_fullsized":"2023-10-06T03:09:02.577534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1412157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.68700000000000006},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.40300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.5},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.436}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.376},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.56299999999999994,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.59999999999999998},"min":{"1min":0.51700000000000002,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":828975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.33400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71799999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.434},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.505},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.627,"5min":0.755,"15min":0.755},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66500000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.79000000000000004},"last":0.504}]},{"osd":3,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.65100000000000002,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.57199999999999995},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.28499999999999998},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.58799999999999997},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78900000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":821789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34399999999999997,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.40699999999999997,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.252},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39300000000000002,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.98899999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32500000000000001,"15min":0.33200000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.40500000000000003,"5min":1.282,"15min":1.282},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33500000000000002},"min":{"1min":0.247,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42799999999999999,"5min":1.25,"15min":1.25},"last":0.372},{"interface":"front","average":{"1min":0.374,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.42899999999999999,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.29199999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66200000000000003,"5min":1.302,"15min":1.302},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.127,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52900000000000003,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.96299999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63,"15min":0.60299999999999998},"min":{"1min":0.47599999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71099999999999997,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.74199999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56499999999999995,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.054,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.379,"15min":1.379},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303696,"kb_used_data":2312,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068144,"statfs":{"total":96636764160,"available":96325779456,"internally_reserved":0,"allocated":2367488,"data_stored":1419973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.307,"15min":0.309},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.32600000000000001},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38800000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.435}]},{"osd":2,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33300000000000002},"min":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.64100000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.311}]},{"osd":3,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.443,"5min":0.52600000000000002,"15min":0.61699999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.81999999999999995,"15min":0.871},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.754,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.496},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78400000000000003,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.47199999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.96099999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.97999999999999998},"last":0.53200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303104,"kb_used_data":1720,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068736,"statfs":{"total":96636764160,"available":96326385664,"internally_reserved":0,"allocated":1761280,"data_stored":829459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.29999999999999999,"15min":0.318},"min":{"1min":0.23999999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.443,"5min":0.498,"15min":0.53100000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46700000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.32900000000000001},"min":{"1min":0.253,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.34799999999999998},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44600000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.318}]},{"osd":3,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32300000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.42999999999999999,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.378},"min":{"1min":0.31,"5min":0.249,"15min":0.249},"max":{"1min":0.503,"5min":0.55300000000000005,"15min":0.57699999999999996},"last":0.29999999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.54200000000000004},"min":{"1min":0.504,"5min":0.374,"15min":0.374},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.40300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72899999999999998,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.57099999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.55700000000000005},"min":{"1min":0.48099999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.87,"15min":1.8740000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.61299999999999999},"min":{"1min":0.49399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.903},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77000000000000002,"5min":0.78800000000000003,"15min":0.84599999999999997},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":821789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.308},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47399999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.47099999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.30399999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.307,"15min":0.309},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.40600000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.436,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.28399999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35199999999999998,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.44500000000000001,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.35499999999999998},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.26400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.873,"15min":5.5220000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75800000000000001,"5min":0.997,"15min":0.997},"last":0.379}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.72899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.49199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.441},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71999999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303116,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068724,"statfs":{"total":96636764160,"available":96326373376,"internally_reserved":0,"allocated":1773568,"data_stored":830475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.66300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.57799999999999996},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":0.746,"15min":3},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.76400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":10.084},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33000000000000002},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.752,"15min":0.83999999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.57999999999999996},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":1.54},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.72599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.438}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":821789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53600000000000003,"15min":0.57799999999999996},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72699999999999998,"5min":1.143,"15min":2.3599999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54700000000000004,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.86299999999999999,"15min":2.371},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.55900000000000005,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69599999999999995,"5min":6.532,"15min":6.532},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.57899999999999996,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.75,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76500000000000001,"5min":5.524,"15min":5.524},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.499},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.58099999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72199999999999998,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60099999999999998,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.753,"5min":0.84199999999999997,"15min":1.052},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.624},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.86799999999999999,"15min":1.175},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.60899999999999999},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.77500000000000002,"15min":1.145},"last":0.65800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:09:08.028 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:09:08.028 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:09:08.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.182+0000 7f90671f7700 1 Processor -- start 2023-10-06T03:09:08.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.182+0000 7f90671f7700 1 -- start start 2023-10-06T03:09:08.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f90671f7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90600cbb90 0x7f90600af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f90671f7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600af860 con 0x7f9060069c70 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f90671f7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600ce380 con 0x7f9060159950 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f90671f7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90600cec20 con 0x7f90600cbb90 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f9057fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9060069c70 0x7f90600cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40472/0 (socket says 172.21.15.92:40472) 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f9057fff700 1 -- 172.21.15.92:0/2834820079 learned_addr learned my addr 172.21.15.92:0/2834820079 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.183+0000 7f9065794700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90600cbb90 0x7f90600af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f9065794700 1 -- 172.21.15.92:0/2834820079 >> v1:172.21.15.136:6789/0 conn(0x7f9060159950 legacy=0x7f9060069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f9065794700 1 -- 172.21.15.92:0/2834820079 >> v1:172.21.15.92:6789/0 conn(0x7f9060069c70 legacy=0x7f90600cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f9065794700 1 -- 172.21.15.92:0/2834820079 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600cbfb0 con 0x7f90600cbb90 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f9065794700 1 --2- 172.21.15.92:0/2834820079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90600cbb90 0x7f90600af320 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f90580098e0 tx=0x7f90580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77518d10b8c43c8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:08.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90577fe700 1 -- 172.21.15.92:0/2834820079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058009e80 con 0x7f90600cbb90 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90577fe700 1 -- 172.21.15.92:0/2834820079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905800b820 con 0x7f90600cbb90 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90577fe700 1 -- 172.21.15.92:0/2834820079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905800bba0 con 0x7f90600cbb90 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 -- 172.21.15.92:0/2834820079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90600cbb90 msgr2=0x7f90600af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 --2- 172.21.15.92:0/2834820079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90600cbb90 0x7f90600af320 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f90580098e0 tx=0x7f90580091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 -- 172.21.15.92:0/2834820079 shutdown_connections 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 --2- 172.21.15.92:0/2834820079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90600cbb90 0x7f90600af320 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 -- 172.21.15.92:0/2834820079 >> 172.21.15.92:0/2834820079 conn(0x7f9060068e00 msgr2=0x7f90600d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 -- 172.21.15.92:0/2834820079 shutdown_connections 2023-10-06T03:09:08.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.184+0000 7f90671f7700 1 -- 172.21.15.92:0/2834820079 wait complete. 2023-10-06T03:09:08.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.185+0000 7f90671f7700 1 Processor -- start 2023-10-06T03:09:08.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 -- start start 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9060069c70 0x7f9060154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90600cbb90 0x7f906015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 0x7f90600b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9060154b80 con 0x7f9060159950 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906016acb0 con 0x7f90600cbb90 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f90671f7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9060169e30 con 0x7f9060069c70 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f9065794700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 0x7f90600b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f9065794700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 0x7f90600b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52880/0 (socket says 172.21.15.92:52880) 2023-10-06T03:09:08.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.186+0000 7f9065794700 1 -- 172.21.15.92:0/3248155767 learned_addr learned my addr 172.21.15.92:0/3248155767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9057fff700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90600cbb90 0x7f906015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9064f93700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9060069c70 0x7f9060154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9065794700 1 -- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9060069c70 msgr2=0x7f9060154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9065794700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9060069c70 0x7f9060154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9065794700 1 -- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90600cbb90 msgr2=0x7f906015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9065794700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90600cbb90 0x7f906015bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9065794700 1 -- 172.21.15.92:0/3248155767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009420 con 0x7f9060159950 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9057fff700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90600cbb90 0x7f906015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:08.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9064f93700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9060069c70 0x7f9060154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9065794700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 0x7f90600b5820 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9060070e20 tx=0x7f9058016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905801c450 con 0x7f9060159950 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058020070 con 0x7f9060159950 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906016a0a0 con 0x7f9060159950 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905801c820 con 0x7f9060159950 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.187+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906016a660 con 0x7f9060159950 2023-10-06T03:09:08.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.188+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906016ae40 con 0x7f9060159950 2023-10-06T03:09:08.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.188+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9058022030 con 0x7f9060159950 2023-10-06T03:09:08.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.189+0000 7f9054ff9700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f904c067320 0x7f904c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.189+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9058086df0 con 0x7f9060159950 2023-10-06T03:09:08.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.189+0000 7f9064f93700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f904c067320 0x7f904c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.189+0000 7f9064f93700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f904c067320 0x7f904c0697d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f90600b64e0 tx=0x7f905c0074f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:08.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.193+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9058053a30 con 0x7f9060159950 2023-10-06T03:09:08.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.451+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f906016adf0 con 0x7f9060159950 2023-10-06T03:09:08.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.455+0000 7f9054ff9700 1 -- 172.21.15.92:0/3248155767 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9058057050 con 0x7f9060159950 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f904c067320 msgr2=0x7f904c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f904c067320 0x7f904c0697d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f90600b64e0 tx=0x7f905c0074f0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 msgr2=0x7f90600b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 0x7f90600b5820 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9060070e20 tx=0x7f9058016000 comp rx=0 tx=0).stop 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 shutdown_connections 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f904c067320 0x7f904c0697d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9060159950 0x7f90600b5820 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9060069c70 0x7f9060154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 --2- 172.21.15.92:0/3248155767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90600cbb90 0x7f906015bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 >> 172.21.15.92:0/3248155767 conn(0x7f9060068e00 msgr2=0x7f9060155490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 shutdown_connections 2023-10-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.457+0000 7f90671f7700 1 -- 172.21.15.92:0/3248155767 wait complete. 2023-10-06T03:09:08.460 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:09:08.471 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:08.584 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.582+0000 7effa78b5700 1 Processor -- start 2023-10-06T03:09:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.582+0000 7effa78b5700 1 -- start start 2023-10-06T03:09:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa78b5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7effa006bcd0 0x7effa00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa78b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effa006c0a0 0x7effa015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa78b5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa78b5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa015d7e0 con 0x7effa006c0a0 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa78b5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa015dac0 con 0x7effa006bcd0 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa78b5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa015aa30 con 0x7effa015df50 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.583+0000 7effa4e50700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effa006c0a0 0x7effa015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7effa006c0a0 0x7effa015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52888/0 (socket says 172.21.15.92:52888) 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 -- 172.21.15.92:0/2796601581 learned_addr learned my addr 172.21.15.92:0/2796601581 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa5e52700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa5651700 1 --2- 172.21.15.92:0/2796601581 >> v2:172.21.15.136:3300/0 conn(0x7effa006bcd0 0x7effa00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 -- 172.21.15.92:0/2796601581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 msgr2=0x7effa015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 -- 172.21.15.92:0/2796601581 >> v2:172.21.15.136:3300/0 conn(0x7effa006bcd0 msgr2=0x7effa00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:08.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 --2- 172.21.15.92:0/2796601581 >> v2:172.21.15.136:3300/0 conn(0x7effa006bcd0 0x7effa00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 -- 172.21.15.92:0/2796601581 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa015ad10 con 0x7effa006c0a0 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa5651700 1 --2- 172.21.15.92:0/2796601581 >> v2:172.21.15.136:3300/0 conn(0x7effa006bcd0 0x7effa00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa5e52700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7effa4e50700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015a1e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7effa00691a0 tx=0x7eff90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323917f93bc65170 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7eff97fff700 1 -- 172.21.15.92:0/2796601581 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000b860 con 0x7effa006c0a0 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7eff97fff700 1 -- 172.21.15.92:0/2796601581 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9000b9c0 con 0x7effa006c0a0 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.584+0000 7eff97fff700 1 -- 172.21.15.92:0/2796601581 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000bd80 con 0x7effa006c0a0 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 -- 172.21.15.92:0/2796601581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 msgr2=0x7effa015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015a1e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7effa00691a0 tx=0x7eff90009250 comp rx=0 tx=0).stop 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 -- 172.21.15.92:0/2796601581 shutdown_connections 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 --2- 172.21.15.92:0/2796601581 >> v2:172.21.15.136:3300/0 conn(0x7effa006bcd0 0x7effa00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 --2- 172.21.15.92:0/2796601581 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015a1e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 -- 172.21.15.92:0/2796601581 >> 172.21.15.92:0/2796601581 conn(0x7effa00ce600 msgr2=0x7effa00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 -- 172.21.15.92:0/2796601581 shutdown_connections 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.585+0000 7effa78b5700 1 -- 172.21.15.92:0/2796601581 wait complete. 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.586+0000 7effa78b5700 1 Processor -- start 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.586+0000 7effa78b5700 1 -- start start 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa78b5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa006bcd0 0x7effa014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa78b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa78b5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa78b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c59f0 con 0x7effa006c0a0 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa78b5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa014a8a0 con 0x7effa006bcd0 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa78b5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa00b3070 con 0x7effa015df50 2023-10-06T03:09:08.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52896/0 (socket says 172.21.15.92:52896) 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 -- 172.21.15.92:0/2853070498 learned_addr learned my addr 172.21.15.92:0/2853070498 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa5651700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa006bcd0 0x7effa014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa5e52700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 -- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 msgr2=0x7effa015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 -- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa006bcd0 msgr2=0x7effa014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa006bcd0 0x7effa014a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 -- 172.21.15.92:0/2853070498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff900094f0 con 0x7effa006c0a0 2023-10-06T03:09:08.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7effa4e50700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015c330 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7effa0069220 tx=0x7eff90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.587+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90011d70 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.588+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90026450 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.588+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa00b3350 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.588+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90026810 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.588+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa00b3910 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.589+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa00c7c60 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.589+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7eff90026bc0 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.589+0000 7eff957fa700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff84067320 0x7eff840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.589+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7eff900866e0 con 0x7effa006c0a0 2023-10-06T03:09:08.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.589+0000 7effa5651700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff84067320 0x7eff840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:08.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.590+0000 7effa5651700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff84067320 0x7eff840697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0076a0 tx=0x7eff9c009930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:08.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.594+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff90053290 con 0x7effa006c0a0 2023-10-06T03:09:08.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.820+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa004c3c0 con 0x7effa006c0a0 2023-10-06T03:09:08.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.820+0000 7eff957fa700 1 -- 172.21.15.92:0/2853070498 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4601 (secure 0 0 0) 0x7eff900568b0 con 0x7effa006c0a0 2023-10-06T03:09:08.822 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:08.822 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":94,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:06.608148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:08.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.822+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff84067320 msgr2=0x7eff840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.822+0000 7effa78b5700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff84067320 0x7eff840697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0076a0 tx=0x7eff9c009930 comp rx=0 tx=0).stop 2023-10-06T03:09:08.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.822+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 msgr2=0x7effa015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:08.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.822+0000 7effa78b5700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015c330 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7effa0069220 tx=0x7eff90017000 comp rx=0 tx=0).stop 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.822+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 shutdown_connections 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.822+0000 7effa78b5700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff84067320 0x7eff840697d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.823+0000 7effa78b5700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa015df50 0x7effa015c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.823+0000 7effa78b5700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa006bcd0 0x7effa014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.823+0000 7effa78b5700 1 --2- 172.21.15.92:0/2853070498 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7effa006c0a0 0x7effa015c330 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.823+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 >> 172.21.15.92:0/2853070498 conn(0x7effa00ce600 msgr2=0x7effa00c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.823+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 shutdown_connections 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:08.823+0000 7effa78b5700 1 -- 172.21.15.92:0/2853070498 wait complete. 2023-10-06T03:09:08.825 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 94 2023-10-06T03:09:09.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:09:02.964243+0000', 'modified': '2023-10-06T03:09:06.608148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7620': {'gid': 7620, 'name': 'a', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/499097126', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 499097126}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 499097126}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:09:09.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:09:09.239 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:09:09.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.357+0000 7fc125bcc700 1 Processor -- start 2023-10-06T03:09:09.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.357+0000 7fc125bcc700 1 -- start start 2023-10-06T03:09:09.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc125bcc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc120159890 0x7fc12015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc125bcc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc120069560 0x7fc1200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc125bcc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc120069ee0 con 0x7fc120159c60 2023-10-06T03:09:09.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc125bcc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12015a160 con 0x7fc120159890 2023-10-06T03:09:09.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc125bcc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1200d13a0 con 0x7fc120069560 2023-10-06T03:09:09.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc120159890 0x7fc12015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc120159890 0x7fc12015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44794/0 (socket says 172.21.15.92:44794) 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 -- 172.21.15.92:0/73539133 learned_addr learned my addr 172.21.15.92:0/73539133 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11ffff700 1 --2- 172.21.15.92:0/73539133 >> v2:172.21.15.136:3300/0 conn(0x7fc120069560 0x7fc1200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 -- 172.21.15.92:0/73539133 >> v2:172.21.15.136:3300/0 conn(0x7fc120069560 msgr2=0x7fc1200699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/73539133 >> v2:172.21.15.136:3300/0 conn(0x7fc120069560 0x7fc1200699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 -- 172.21.15.92:0/73539133 >> v1:172.21.15.136:6789/0 conn(0x7fc120159c60 legacy=0x7fc12015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 -- 172.21.15.92:0/73539133 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12015e140 con 0x7fc120159890 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/73539133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc12015abd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc12006a890 tx=0x7fc110009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650c02b35adc10a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.358+0000 7fc11e7fc700 1 -- 172.21.15.92:0/73539133 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110009e70 con 0x7fc120159890 2023-10-06T03:09:09.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc11e7fc700 1 -- 172.21.15.92:0/73539133 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000b860 con 0x7fc120159890 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc11e7fc700 1 -- 172.21.15.92:0/73539133 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bc00 con 0x7fc120159890 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 -- 172.21.15.92:0/73539133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 msgr2=0x7fc12015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 --2- 172.21.15.92:0/73539133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc12015abd0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc12006a890 tx=0x7fc110009250 comp rx=0 tx=0).stop 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 -- 172.21.15.92:0/73539133 shutdown_connections 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 --2- 172.21.15.92:0/73539133 >> v2:172.21.15.136:3300/0 conn(0x7fc120069560 0x7fc1200699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 --2- 172.21.15.92:0/73539133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc12015abd0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 -- 172.21.15.92:0/73539133 >> 172.21.15.92:0/73539133 conn(0x7fc120068e20 msgr2=0x7fc1200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 -- 172.21.15.92:0/73539133 shutdown_connections 2023-10-06T03:09:09.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.359+0000 7fc125bcc700 1 -- 172.21.15.92:0/73539133 wait complete. 2023-10-06T03:09:09.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.360+0000 7fc125bcc700 1 Processor -- start 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.360+0000 7fc125bcc700 1 -- start start 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc125bcc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc120069560 0x7fc1200c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc125bcc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc1200b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc125bcc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc120159c60 0x7fc12016d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc125bcc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200b64c0 con 0x7fc120159890 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc125bcc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200b4430 con 0x7fc120159c60 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc125bcc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1200b4710 con 0x7fc120069560 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc1200b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc1200b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44802/0 (socket says 172.21.15.92:44802) 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 -- 172.21.15.92:0/1612280200 learned_addr learned my addr 172.21.15.92:0/1612280200 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11ffff700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc120159c60 0x7fc12016d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc120069560 0x7fc1200c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 -- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc120069560 msgr2=0x7fc1200c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc120069560 0x7fc1200c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 -- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc120159c60 msgr2=0x7fc12016d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc120159c60 0x7fc12016d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.361+0000 7fc11effd700 1 -- 172.21.15.92:0/1612280200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100094f0 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.362+0000 7fc11effd700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc1200b5c70 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c4e0 tx=0x7fc11400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.362+0000 7fc1077fe700 1 -- 172.21.15.92:0/1612280200 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114011590 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.362+0000 7fc1077fe700 1 -- 172.21.15.92:0/1612280200 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114011710 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.362+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1200b49f0 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.362+0000 7fc1077fe700 1 -- 172.21.15.92:0/1612280200 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114011ad0 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.362+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1200b61b0 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.363+0000 7fc1077fe700 1 -- 172.21.15.92:0/1612280200 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc11401b3b0 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.364+0000 7fc1077fe700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc10c067320 0x7fc10c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.364+0000 7fc1077fe700 1 -- 172.21.15.92:0/1612280200 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc114084200 con 0x7fc120159890 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.364+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc10c067320 0x7fc10c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.364+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc10c067320 0x7fc10c0697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc12006a890 tx=0x7fc110017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.365+0000 7fc12eec4b80 1 Processor -- start 2023-10-06T03:09:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 -- start start 2023-10-06T03:09:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3842c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3843cc0 0x5574b38926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 0x5574b3893050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b3799a70 con 0x5574b3892c00 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b3797270 con 0x5574b3842840 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b3793060 con 0x5574b3843cc0 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc11ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 0x5574b3893050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc11ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 0x5574b3893050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44806/0 (socket says 172.21.15.92:44806) 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.366+0000 7fc11ffff700 1 -- 172.21.15.92:0/254977986 learned_addr learned my addr 172.21.15.92:0/254977986 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3842c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11effd700 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3843cc0 0x5574b38926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11ffff700 1 -- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3843cc0 msgr2=0x5574b38926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11ffff700 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3843cc0 0x5574b38926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.376 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11ffff700 1 -- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 msgr2=0x5574b3842c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11ffff700 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3842c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11ffff700 1 -- 172.21.15.92:0/254977986 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b362d270 con 0x5574b3892c00 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc11ffff700 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 0x5574b3893050 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc118011550 tx=0x7fc11800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251ae452fc9d225a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc104ff9700 1 -- 172.21.15.92:0/254977986 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800e040 con 0x5574b3892c00 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc104ff9700 1 -- 172.21.15.92:0/254977986 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11800f040 con 0x5574b3892c00 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.367+0000 7fc104ff9700 1 -- 172.21.15.92:0/254977986 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118015870 con 0x5574b3892c00 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 -- 172.21.15.92:0/254977986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 msgr2=0x5574b3893050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 0x5574b3893050 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc118011550 tx=0x7fc11800b040 comp rx=0 tx=0).stop 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 -- 172.21.15.92:0/254977986 shutdown_connections 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3892c00 0x5574b3893050 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3842c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/254977986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3843cc0 0x5574b38926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 -- 172.21.15.92:0/254977986 >> 172.21.15.92:0/254977986 conn(0x5574b3844160 msgr2=0x5574b3844530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 -- 172.21.15.92:0/254977986 shutdown_connections 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 -- 172.21.15.92:0/254977986 wait complete. 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 -- start start 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.368+0000 7fc12eec4b80 1 Processor -- start 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 0x5574b3881e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3843cc0 0x5574b388b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b388bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b373e890 con 0x5574b3842840 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b3759ea0 con 0x5574b3843cc0 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b3615e30 con 0x5574b3892c00 2023-10-06T03:09:09.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc12eec4b80 10 client.0 _ll_register_callbacks cb 0x5574b3805e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 0x5574b3881e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 0x5574b3881e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44808/0 (socket says 172.21.15.92:44808) 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 -- 172.21.15.92:0/947687721 learned_addr learned my addr 172.21.15.92:0/947687721 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11ffff700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b388bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11effd700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3843cc0 0x5574b388b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 -- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 msgr2=0x5574b388bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b388bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 -- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3843cc0 msgr2=0x5574b388b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.369+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3843cc0 0x5574b388b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc11f7fe700 1 -- 172.21.15.92:0/947687721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc118011250 con 0x5574b3842840 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 0x5574b3881e40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc11000a010 tx=0x7fc110020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c11b6a74d97f85e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc0fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc0fdffb700 1 -- 172.21.15.92:0/947687721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11002e070 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc0fdffb700 1 -- 172.21.15.92:0/947687721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc110033c20 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574b3837940 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc0fdffb700 1 -- 172.21.15.92:0/947687721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc110020070 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 10 client.7764 fetch_fsmap learned FSMap version 94 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5574b35cffe0 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc0fdffb700 1 -- 172.21.15.92:0/947687721 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574b3837940 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc0fdffb700 1 -- 172.21.15.92:0/947687721 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 94) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc11001a420 con 0x5574b3842840 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 10 client.7764 fetch_fsmap finished waiting for FSMap version 94 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 10 client.7764 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 20 client.7764 populate_metadata read hostname 'smithi092' 2023-10-06T03:09:09.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b3889ae0 0x5574b3889eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 4 client.7764 mds_command: new command op to 7620 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 --> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574b3814870 con 0x5574b3889ae0 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.370+0000 7fc11effd700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b3889ae0 0x5574b3889eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc0fdffb700 10 client.7764 ms_handle_connect on v2:172.21.15.92:6835/499097126 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc11effd700 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b3889ae0 0x5574b3889eb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580900fe8e1de5f7 server_cookie=1c412096fe810f48 in_seq=0 out_seq=0 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc0fdffb700 1 -- 172.21.15.92:0/947687721 <== mds.0 v2:172.21.15.92:6835/499097126 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5574b3814870 con 0x5574b3889ae0 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc0fdffb700 10 client.7764 handle_command_reply: tid=0 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 client.7764 shutdown 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 msgr2=0x5574b3881e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 0x5574b3881e40 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc11000a010 tx=0x7fc110020000 comp rx=0 tx=0).stop 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 shutdown_connections 2023-10-06T03:09:09.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b3889ae0 0x5574b3889eb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3842840 0x5574b3881e40 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b388bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/947687721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3843cc0 0x5574b388b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc0fdffb700 0 client.7764 ms_handle_reset on v2:172.21.15.92:6835/499097126 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 >> 172.21.15.92:0/947687721 conn(0x5574b3844160 msgr2=0x5574b3886360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 shutdown_connections 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.371+0000 7fc12eec4b80 1 -- 172.21.15.92:0/947687721 wait complete. 2023-10-06T03:09:09.382 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.372+0000 7fc12eec4b80 20 client.7764 trim_cache size 0 max 16384 2023-10-06T03:09:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.388+0000 7fc12eec4b80 1 Processor -- start 2023-10-06T03:09:09.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.388+0000 7fc12eec4b80 1 -- start start 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3888c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b38838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b3886360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b383e1a0 con 0x5574b3843cc0 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b3844dc0 con 0x5574b3842840 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b3898df0 con 0x5574b3892c00 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b38838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b38838c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44820/0 (socket says 172.21.15.92:44820) 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 -- 172.21.15.92:0/1073024122 learned_addr learned my addr 172.21.15.92:0/1073024122 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11ffff700 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b3886360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3888c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 -- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 msgr2=0x5574b3886360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b3886360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 -- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 msgr2=0x5574b3888c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3888c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11effd700 1 -- 172.21.15.92:0/1073024122 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574b3761780 con 0x5574b3843cc0 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.389+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3888c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc11effd700 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b38838c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc1140546f0 tx=0x7fc114003040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992ce47c3fe82a27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc1057fa700 1 -- 172.21.15.92:0/1073024122 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11405b990 con 0x5574b3843cc0 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc1057fa700 1 -- 172.21.15.92:0/1073024122 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114011e80 con 0x5574b3843cc0 2023-10-06T03:09:09.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc1057fa700 1 -- 172.21.15.92:0/1073024122 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114052d90 con 0x5574b3843cc0 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 -- 172.21.15.92:0/1073024122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 msgr2=0x5574b38838c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b38838c0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc1140546f0 tx=0x7fc114003040 comp rx=0 tx=0).stop 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 -- 172.21.15.92:0/1073024122 shutdown_connections 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3892c00 0x5574b3886360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3842840 0x5574b3888c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/1073024122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b38838c0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 -- 172.21.15.92:0/1073024122 >> 172.21.15.92:0/1073024122 conn(0x5574b3844160 msgr2=0x5574b3889eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 -- 172.21.15.92:0/1073024122 shutdown_connections 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.390+0000 7fc12eec4b80 1 -- 172.21.15.92:0/1073024122 wait complete. 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.391+0000 7fc12eec4b80 1 -- start start 2023-10-06T03:09:09.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.391+0000 7fc12eec4b80 1 Processor -- start 2023-10-06T03:09:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3842840 0x5574b38d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b387f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3892c00 0x5574b387f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b38eea20 con 0x5574b3843cc0 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574b388e630 con 0x5574b3892c00 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574b38ef960 con 0x5574b3842840 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 10 client.0 _ll_register_callbacks cb 0x5574b3805e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b387f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b387f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44826/0 (socket says 172.21.15.92:44826) 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 -- 172.21.15.92:0/772349567 learned_addr learned my addr 172.21.15.92:0/772349567 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3842840 0x5574b38d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11ffff700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3892c00 0x5574b387f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 -- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3842840 msgr2=0x5574b38d7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3842840 0x5574b38d7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 -- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3892c00 msgr2=0x5574b387f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3892c00 0x5574b387f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 -- 172.21.15.92:0/772349567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1200b61b0 con 0x5574b3843cc0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3842840 0x5574b38d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11ffff700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3892c00 0x5574b387f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc11effd700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b387f350 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc1140546f0 tx=0x7fc114014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0f76c7d0431a76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc0fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc0fdffb700 1 -- 172.21.15.92:0/772349567 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114081fd0 con 0x5574b3843cc0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc0fdffb700 1 -- 172.21.15.92:0/772349567 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc114011c30 con 0x5574b3843cc0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc11800f040 con 0x5574b3843cc0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc0fdffb700 1 -- 172.21.15.92:0/772349567 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc11405bbe0 con 0x5574b3843cc0 2023-10-06T03:09:09.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 10 client.7770 fetch_fsmap learned FSMap version 94 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.392+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x5574b3748470 con 0x5574b3843cc0 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc0fdffb700 1 -- 172.21.15.92:0/772349567 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114065020 con 0x5574b3843cc0 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc0fdffb700 1 -- 172.21.15.92:0/772349567 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 94) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc114053ee0 con 0x5574b3843cc0 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 10 client.7770 fetch_fsmap finished waiting for FSMap version 94 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 10 client.7770 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 20 client.7770 populate_metadata read hostname 'smithi092' 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b383eb10 0x5574b383eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 4 client.7770 mds_command: new command op to 7620 tid=0[{"prefix": "status"}] 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 --> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574b3735e00 con 0x5574b383eb10 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b383eb10 0x5574b383eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc0fdffb700 10 client.7770 ms_handle_connect on v2:172.21.15.92:6835/499097126 2023-10-06T03:09:09.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc11f7fe700 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b383eb10 0x5574b383eee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769093ebc17e4815 server_cookie=66c04e1e1b91a050 in_seq=0 out_seq=0 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc0fdffb700 1 -- 172.21.15.92:0/772349567 <== mds.0 v2:172.21.15.92:6835/499097126 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5574b3735e00 con 0x5574b383eb10 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc0fdffb700 10 client.7770 handle_command_reply: tid=0 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 client.7770 shutdown 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 msgr2=0x5574b387f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b387f350 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fc1140546f0 tx=0x7fc114014000 comp rx=0 tx=0).stop 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 shutdown_connections 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:6835/499097126,v1:172.21.15.92:6837/499097126] conn(0x5574b383eb10 0x5574b383eee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5574b3892c00 0x5574b387f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5574b3842840 0x5574b38d7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 --2- 172.21.15.92:0/772349567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5574b3843cc0 0x5574b387f350 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc0fdffb700 0 client.7770 ms_handle_reset on v2:172.21.15.92:6835/499097126 2023-10-06T03:09:09.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 >> 172.21.15.92:0/772349567 conn(0x5574b3844160 msgr2=0x5574b38727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 shutdown_connections 2023-10-06T03:09:09.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.393+0000 7fc12eec4b80 1 -- 172.21.15.92:0/772349567 wait complete. 2023-10-06T03:09:09.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc12eec4b80 20 client.7770 trim_cache size 0 max 16384 2023-10-06T03:09:09.404 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:09:09.404 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7620, 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.7983590869999997, 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 94, 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 140, 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 140, 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.933008272 2023-10-06T03:09:09.405 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:09:09.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc10c067320 msgr2=0x7fc10c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc10c067320 0x7fc10c0697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc12006a890 tx=0x7fc110017000 comp rx=0 tx=0).stop 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 msgr2=0x7fc1200b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc1200b5c70 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c4e0 tx=0x7fc11400c7f0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 shutdown_connections 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc10c067320 0x7fc10c0697d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc120159c60 0x7fc12016d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc120069560 0x7fc1200c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 --2- 172.21.15.92:0/1612280200 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc120159890 0x7fc1200b5c70 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 >> 172.21.15.92:0/1612280200 conn(0x7fc120068e20 msgr2=0x7fc1201534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 shutdown_connections 2023-10-06T03:09:09.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.394+0000 7fc125bcc700 1 -- 172.21.15.92:0/1612280200 wait complete. 2023-10-06T03:09:09.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:09:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.522+0000 7f3b95282700 1 Processor -- start 2023-10-06T03:09:09.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.522+0000 7f3b95282700 1 -- start start 2023-10-06T03:09:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b95282700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b900b1c20 0x7f3b901572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b95282700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b90157c60 con 0x7f3b900b1850 2023-10-06T03:09:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b95282700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b900cb300 con 0x7f3b90157820 2023-10-06T03:09:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b95282700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9006fc30 con 0x7f3b900b1c20 2023-10-06T03:09:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b867fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b900b1c20 0x7f3b901572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b8effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3b900b1850 0x7f3b900ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40800/0 (socket says 172.21.15.92:40800) 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b867fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3b900b1c20 0x7f3b901572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44836/0 (socket says 172.21.15.92:44836) 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b8effd700 1 -- 172.21.15.92:0/833181082 learned_addr learned my addr 172.21.15.92:0/833181082 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b867fc700 1 -- 172.21.15.92:0/833181082 >> v1:172.21.15.136:6789/0 conn(0x7f3b90157820 legacy=0x7f3b900abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b867fc700 1 -- 172.21.15.92:0/833181082 >> v1:172.21.15.92:6789/0 conn(0x7f3b900b1850 legacy=0x7f3b900ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b867fc700 1 -- 172.21.15.92:0/833181082 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b900cb5e0 con 0x7f3b900b1c20 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.523+0000 7f3b8e7fc700 1 -- 172.21.15.92:0/833181082 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168616252 0 0) 0x7f3b90157c60 con 0x7f3b900b1850 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b867fc700 1 --2- 172.21.15.92:0/833181082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b900b1c20 0x7f3b901572e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098e0 tx=0x7f3b780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2152f196939a7e7c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b8e7fc700 1 -- 172.21.15.92:0/833181082 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78009e80 con 0x7f3b900b1c20 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b8e7fc700 1 -- 172.21.15.92:0/833181082 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7800b860 con 0x7f3b900b1c20 2023-10-06T03:09:09.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b8e7fc700 1 -- 172.21.15.92:0/833181082 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800bbe0 con 0x7f3b900b1c20 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 -- 172.21.15.92:0/833181082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b900b1c20 msgr2=0x7f3b901572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 --2- 172.21.15.92:0/833181082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b900b1c20 0x7f3b901572e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098e0 tx=0x7f3b780091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 -- 172.21.15.92:0/833181082 shutdown_connections 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 --2- 172.21.15.92:0/833181082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b900b1c20 0x7f3b901572e0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 -- 172.21.15.92:0/833181082 >> 172.21.15.92:0/833181082 conn(0x7f3b9006a320 msgr2=0x7f3b900cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 -- 172.21.15.92:0/833181082 shutdown_connections 2023-10-06T03:09:09.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.524+0000 7f3b95282700 1 -- 172.21.15.92:0/833181082 wait complete. 2023-10-06T03:09:09.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.525+0000 7f3b95282700 1 Processor -- start 2023-10-06T03:09:09.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 -- start start 2023-10-06T03:09:09.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b900b1850 0x7f3b900a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b900b1c20 0x7f3b90144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 0x7f3b9006db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900cc260 con 0x7f3b90157820 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90151490 con 0x7f3b900b1850 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b95282700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9016b9c0 con 0x7f3b900b1c20 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b8f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 0x7f3b9006db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b8f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 0x7f3b9006db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44846/0 (socket says 172.21.15.92:44846) 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b8f7fe700 1 -- 172.21.15.92:0/859435316 learned_addr learned my addr 172.21.15.92:0/859435316 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b8effd700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b900b1850 0x7f3b900a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.526+0000 7f3b867fc700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b900b1c20 0x7f3b90144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8f7fe700 1 -- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b900b1c20 msgr2=0x7f3b90144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8f7fe700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b900b1c20 0x7f3b90144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8f7fe700 1 -- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b900b1850 msgr2=0x7f3b900a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8f7fe700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b900b1850 0x7f3b900a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8f7fe700 1 -- 172.21.15.92:0/859435316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78009420 con 0x7f3b90157820 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8effd700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b900b1850 0x7f3b900a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b8f7fe700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 0x7f3b9006db50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c870 tx=0x7f3b88001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88012630 con 0x7f3b90157820 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b880127b0 con 0x7f3b90157820 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b900a5a00 con 0x7f3b90157820 2023-10-06T03:09:09.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88012b70 con 0x7f3b90157820 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.527+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9016b370 con 0x7f3b90157820 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.528+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9004c3c0 con 0x7f3b90157820 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.528+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3b88003030 con 0x7f3b90157820 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.529+0000 7f3b877fe700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b70067320 0x7f3b700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.529+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3b88084cf0 con 0x7f3b90157820 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.529+0000 7f3b8effd700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b70067320 0x7f3b700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:09.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.529+0000 7f3b8effd700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b70067320 0x7f3b700697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b800011e0 tx=0x7f3b80009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:09.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.533+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b88051820 con 0x7f3b90157820 2023-10-06T03:09:09.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.763+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3b900678d0 con 0x7f3b90157820 2023-10-06T03:09:09.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.764+0000 7f3b877fe700 1 -- 172.21.15.92:0/859435316 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3b88054e40 con 0x7f3b90157820 2023-10-06T03:09:09.766 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:09:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b70067320 msgr2=0x7f3b700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b70067320 0x7f3b700697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b800011e0 tx=0x7f3b80009040 comp rx=0 tx=0).stop 2023-10-06T03:09:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 msgr2=0x7f3b9006db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 0x7f3b9006db50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800c870 tx=0x7f3b88001670 comp rx=0 tx=0).stop 2023-10-06T03:09:09.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 shutdown_connections 2023-10-06T03:09:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3b70067320 0x7f3b700697d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b90157820 0x7f3b9006db50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3b900b1850 0x7f3b900a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 --2- 172.21.15.92:0/859435316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3b900b1c20 0x7f3b90144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:09.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 >> 172.21.15.92:0/859435316 conn(0x7f3b9006a320 msgr2=0x7f3b900caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 shutdown_connections 2023-10-06T03:09:09.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:09.766+0000 7f3b95282700 1 -- 172.21.15.92:0/859435316 wait complete. 2023-10-06T03:09:09.779 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:09:09.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:09:09.780 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:09:09.780 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:09:09.780 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:09.780 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:09:09.780 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:09:09.780 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:09:09.781 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:09:09.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:09.781 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:09:09.796 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:09.797 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42056sec preferred_lft 42056sec 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.798 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:09.799 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:09:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:09:09.800 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:09.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:09.800 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:09:09.859 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:09:09.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:09.861 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:09:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:09:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:09:09.917 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:09:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:09:09.918 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:09:09.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:09.918 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:09:09.918 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:09:09.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:09:09.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:09:09.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:09:09.918 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:09:10.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.017 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:09:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:09:10.032 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:09:10.032 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:09:10.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.032 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:09:10.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:10.097 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:10.151 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:09:10.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:10.152 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:10.205 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:09:10.205 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:09:10.206 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:10.285 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:10.285 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:10.303 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:10.303 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:10.303 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:10.304 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:10.304 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:10.304 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:10.304 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:10.304 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:10.325 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:10.325 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:10.325 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:09:10.326 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:09:10.327 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:09:10.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:10.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:09:10.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.545+0000 7ff62fa17700 1 Processor -- start 2023-10-06T03:09:10.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.545+0000 7ff62fa17700 1 -- start start 2023-10-06T03:09:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62fa17700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff6280b5ea0 0x7ff628068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62fa17700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62815c480 con 0x7ff628068bb0 2023-10-06T03:09:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62fa17700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62815b870 con 0x7ff628159c10 2023-10-06T03:09:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62fa17700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6280b2a40 con 0x7ff6280b5ea0 2023-10-06T03:09:10.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62dfb4700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff628068bb0 0x7ff62815c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40810/0 (socket says 172.21.15.92:40810) 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62dfb4700 1 -- 172.21.15.92:0/3767160805 learned_addr learned my addr 172.21.15.92:0/3767160805 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424382469 0 0) 0x7ff62815c480 con 0x7ff628068bb0 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff614003570 con 0x7ff628068bb0 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62cfb2700 1 --2- 172.21.15.92:0/3767160805 >> v2:172.21.15.136:3300/0 conn(0x7ff6280b5ea0 0x7ff628068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 757974668 0 0) 0x7ff614003570 con 0x7ff628068bb0 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 >> v2:172.21.15.136:3300/0 conn(0x7ff6280b5ea0 msgr2=0x7ff628068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 --2- 172.21.15.92:0/3767160805 >> v2:172.21.15.136:3300/0 conn(0x7ff6280b5ea0 0x7ff628068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 >> v1:172.21.15.136:6789/0 conn(0x7ff628159c10 legacy=0x7ff62815b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62815bb50 con 0x7ff628068bb0 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7ff6200017f0 con 0x7ff628068bb0 2023-10-06T03:09:10.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 >> v1:172.21.15.92:6789/0 conn(0x7ff628068bb0 legacy=0x7ff62815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff614003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614004810 0x7ff614004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6140053c0 0x7ff61400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c2c0 con 0x7ff614003b30 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c430 con 0x7ff6140053c0 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400c5f0 con 0x7ff614004810 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.546+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff614003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62dfb4700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6140053c0 0x7ff61400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62cfb2700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614004810 0x7ff614004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62d7b3700 1 -- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614004810 msgr2=0x7ff614004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff614004810 0x7ff614004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62d7b3700 1 -- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6140053c0 msgr2=0x7ff61400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6140053c0 0x7ff61400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62d7b3700 1 -- 172.21.15.92:0/3767160805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614003570 con 0x7ff614003b30 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62dfb4700 1 -- 172.21.15.92:0/3767160805 reap_dead start 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff614003f40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff6240018b0 tx=0x7ff624007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f8ad30b883e77f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624012420 con 0x7ff614003b30 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240125a0 con 0x7ff614003b30 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff61ffff700 1 -- 172.21.15.92:0/3767160805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624012960 con 0x7ff614003b30 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62fa17700 1 -- 172.21.15.92:0/3767160805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 msgr2=0x7ff614003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62fa17700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff614003f40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7ff6240018b0 tx=0x7ff624007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62fa17700 1 -- 172.21.15.92:0/3767160805 shutdown_connections 2023-10-06T03:09:10.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62fa17700 1 --2- 172.21.15.92:0/3767160805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff614003f40 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.547+0000 7ff62fa17700 1 -- 172.21.15.92:0/3767160805 >> 172.21.15.92:0/3767160805 conn(0x7ff628099a20 msgr2=0x7ff6280af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.548+0000 7ff62fa17700 1 -- 172.21.15.92:0/3767160805 shutdown_connections 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.548+0000 7ff62fa17700 1 -- 172.21.15.92:0/3767160805 wait complete. 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.549+0000 7ff62fa17700 1 Processor -- start 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.549+0000 7ff62fa17700 1 -- start start 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.549+0000 7ff62fa17700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff6280c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.549+0000 7ff62fa17700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6280c59d0 0x7ff62806a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.549+0000 7ff62fa17700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6280c5e40 0x7ff62806a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62fa17700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6281692e0 con 0x7ff614003b30 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62fa17700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280c6400 con 0x7ff6280c5e40 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62fa17700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280c66e0 con 0x7ff6280c59d0 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff6280c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff6280c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44860/0 (socket says 172.21.15.92:44860) 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 -- 172.21.15.92:0/1097078952 learned_addr learned my addr 172.21.15.92:0/1097078952 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62cfb2700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6280c59d0 0x7ff62806a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62dfb4700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6280c5e40 0x7ff62806a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 -- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6280c59d0 msgr2=0x7ff62806a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6280c59d0 0x7ff62806a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 -- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6280c5e40 msgr2=0x7ff62806a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6280c5e40 0x7ff62806a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 -- 172.21.15.92:0/1097078952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280c69c0 con 0x7ff614003b30 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62dfb4700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6280c5e40 0x7ff62806a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62d7b3700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff6280c7f10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c3f0 tx=0x7ff624015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:10.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624001200 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624012cd0 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624012e40 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6280c6ca0 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.550+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6280b1090 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.551+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6280c6820 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.552+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff62401f620 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.552+0000 7ff61d7fa700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff60c067320 0x7ff60c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.552+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff6240867e0 con 0x7ff614003b30 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.552+0000 7ff62cfb2700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff60c067320 0x7ff60c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:10.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.553+0000 7ff62cfb2700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff60c067320 0x7ff60c0697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff6280b1930 tx=0x7ff618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:10.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.556+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6280c6820 con 0x7ff614003b30 2023-10-06T03:09:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.784+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff6280c6820 con 0x7ff614003b30 2023-10-06T03:09:10.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.784+0000 7ff61d7fa700 1 -- 172.21.15.92:0/1097078952 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff6280c6820 con 0x7ff614003b30 2023-10-06T03:09:10.786 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:09:10.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff60c067320 msgr2=0x7ff60c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff60c067320 0x7ff60c0697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff6280b1930 tx=0x7ff618008040 comp rx=0 tx=0).stop 2023-10-06T03:09:10.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 msgr2=0x7ff6280c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:10.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff6280c7f10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c3f0 tx=0x7ff624015000 comp rx=0 tx=0).stop 2023-10-06T03:09:10.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 shutdown_connections 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff60c067320 0x7ff60c0697d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff614003b30 0x7ff6280c7f10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6280c5e40 0x7ff62806a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 --2- 172.21.15.92:0/1097078952 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6280c59d0 0x7ff62806a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 >> 172.21.15.92:0/1097078952 conn(0x7ff628099a20 msgr2=0x7ff62815b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 shutdown_connections 2023-10-06T03:09:10.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:10.786+0000 7ff62fa17700 1 -- 172.21.15.92:0/1097078952 wait complete. 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:09:10.800 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:09:10.801 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:09:10.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.801 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:09:10.819 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:09:10.819 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:09:10.819 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:10.820 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42056sec preferred_lft 42056sec 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:09:10.821 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:10.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.823 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:09:10.887 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:09:10.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.889 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:09:10.947 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:09:10.948 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:09:10.948 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:09:10.948 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:09:10.948 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:09:10.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:10.949 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:09:10.949 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:09:10.949 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:09:10.949 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:09:10.949 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:09:10.949 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:09:11.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:11.053 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:09:11.068 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:09:11.068 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:09:11.069 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:09:11.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:11.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:09:11.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:11.134 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:11.190 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:09:11.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:11.190 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:11.245 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:09:11.246 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:09:11.246 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:11.325 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:11.325 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:11.343 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:11.343 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:11.344 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:11.344 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:11.344 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:11.344 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:11.344 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:11.344 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:11.366 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:11.366 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:11.366 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:09:11.367 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:09:11.368 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:09:11.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:11.368 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:11.433 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:09:11.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.551+0000 7f5837b6d700 1 Processor -- start 2023-10-06T03:09:11.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5837b6d700 1 -- start start 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5837b6d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5830159ec0 0x7f58300b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5837b6d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f58300b53d0 0x7f58300cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5837b6d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58300b5b00 con 0x7f58300cce40 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5837b6d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58300b5da0 con 0x7f58300b53d0 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5837b6d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58300b6080 con 0x7f5830159ec0 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f5835108700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f58300b53d0 0x7f58300cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f583610a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f58300cce40 0x7f58300cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40814/0 (socket says 172.21.15.92:40814) 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.552+0000 7f583610a700 1 -- 172.21.15.92:0/62618379 learned_addr learned my addr 172.21.15.92:0/62618379 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:11.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5835909700 1 --2- 172.21.15.92:0/62618379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5830159ec0 0x7f58300b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5835108700 1 -- 172.21.15.92:0/62618379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5830159ec0 msgr2=0x7f58300b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5835108700 1 --2- 172.21.15.92:0/62618379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5830159ec0 0x7f58300b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5835108700 1 -- 172.21.15.92:0/62618379 >> v1:172.21.15.92:6789/0 conn(0x7f58300cce40 legacy=0x7f58300cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5834907700 1 -- 172.21.15.92:0/62618379 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614468873 0 0) 0x7f58300b5b00 con 0x7f58300cce40 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5835108700 1 -- 172.21.15.92:0/62618379 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58300cc100 con 0x7f58300b53d0 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5835108700 1 --2- 172.21.15.92:0/62618379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300b53d0 0x7f58300cc900 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f58200098e0 tx=0x7f58200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f596e89336a5fff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5834907700 1 -- 172.21.15.92:0/62618379 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000b640 con 0x7f58300b53d0 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5834907700 1 -- 172.21.15.92:0/62618379 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582000b7a0 con 0x7f58300b53d0 2023-10-06T03:09:11.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.553+0000 7f5834907700 1 -- 172.21.15.92:0/62618379 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000bb40 con 0x7f58300b53d0 2023-10-06T03:09:11.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 -- 172.21.15.92:0/62618379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300b53d0 msgr2=0x7f58300cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:11.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 --2- 172.21.15.92:0/62618379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300b53d0 0x7f58300cc900 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f58200098e0 tx=0x7f58200091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:11.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 -- 172.21.15.92:0/62618379 shutdown_connections 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 --2- 172.21.15.92:0/62618379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5830159ec0 0x7f58300b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 --2- 172.21.15.92:0/62618379 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300b53d0 0x7f58300cc900 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 -- 172.21.15.92:0/62618379 >> 172.21.15.92:0/62618379 conn(0x7f58300999f0 msgr2=0x7f58300b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 -- 172.21.15.92:0/62618379 shutdown_connections 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.554+0000 7f5837b6d700 1 -- 172.21.15.92:0/62618379 wait complete. 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.555+0000 7f5837b6d700 1 Processor -- start 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.555+0000 7f5837b6d700 1 -- start start 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58300b53d0 0x7f5830148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 0x7f58301458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830159ec0 0x7f5830145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:11.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58301463d0 con 0x7f58300cce40 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58301466b0 con 0x7f5830159ec0 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5830146990 con 0x7f58300b53d0 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 0x7f58301458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 0x7f58301458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44882/0 (socket says 172.21.15.92:44882) 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 -- 172.21.15.92:0/1276581905 learned_addr learned my addr 172.21.15.92:0/1276581905 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835909700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58300b53d0 0x7f5830148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f583610a700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830159ec0 0x7f5830145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 -- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58300b53d0 msgr2=0x7f5830148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58300b53d0 0x7f5830148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 -- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830159ec0 msgr2=0x7f5830145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830159ec0 0x7f5830145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:11.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 -- 172.21.15.92:0/1276581905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820009420 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5835108700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 0x7f58301458b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5830089390 tx=0x7f5820017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000bb20 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820011420 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5830146c70 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200117a0 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.556+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583006a5a0 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.557+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5830063c90 con 0x7f58300cce40 2023-10-06T03:09:11.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.558+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5820011b50 con 0x7f58300cce40 2023-10-06T03:09:11.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.558+0000 7f5825ffb700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5814067320 0x7f58140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:11.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.558+0000 7f5835909700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5814067320 0x7f58140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:11.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.558+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5820086520 con 0x7f58300cce40 2023-10-06T03:09:11.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.559+0000 7f5835909700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5814067320 0x7f58140697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f582c009fd0 tx=0x7f582c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:11.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.563+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5820053050 con 0x7f58300cce40 2023-10-06T03:09:11.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:11.793+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5830147170 con 0x7f58300cce40 2023-10-06T03:09:12.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.648+0000 7f5825ffb700 1 -- 172.21.15.92:0/1276581905 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v95) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5820056670 con 0x7f58300cce40 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5814067320 msgr2=0x7f58140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5814067320 0x7f58140697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f582c009fd0 tx=0x7f582c00b040 comp rx=0 tx=0).stop 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 msgr2=0x7f58301458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 0x7f58301458b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5830089390 tx=0x7f5820017000 comp rx=0 tx=0).stop 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 shutdown_connections 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5814067320 0x7f58140697d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5830159ec0 0x7f5830145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58300b53d0 0x7f5830148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 --2- 172.21.15.92:0/1276581905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58300cce40 0x7f58301458b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 >> 172.21.15.92:0/1276581905 conn(0x7f58300999f0 msgr2=0x7f58300c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:12.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 shutdown_connections 2023-10-06T03:09:12.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.651+0000 7f5837b6d700 1 -- 172.21.15.92:0/1276581905 wait complete. 2023-10-06T03:09:12.665 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:09:12.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.780+0000 7fad9809a700 1 Processor -- start 2023-10-06T03:09:12.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.780+0000 7fad9809a700 1 -- start start 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fad90159890 0x7fad9015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad90069560 0x7fad900699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad90069ee0 con 0x7fad90159c60 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9015a160 con 0x7fad90069560 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad900d13a0 con 0x7fad90159890 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad90069560 0x7fad900699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad90069560 0x7fad900699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44890/0 (socket says 172.21.15.92:44890) 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 -- 172.21.15.92:0/437783423 learned_addr learned my addr 172.21.15.92:0/437783423 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:12.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad95e36700 1 --2- 172.21.15.92:0/437783423 >> v2:172.21.15.136:3300/0 conn(0x7fad90159890 0x7fad9015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 -- 172.21.15.92:0/437783423 >> v2:172.21.15.136:3300/0 conn(0x7fad90159890 msgr2=0x7fad9015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 --2- 172.21.15.92:0/437783423 >> v2:172.21.15.136:3300/0 conn(0x7fad90159890 0x7fad9015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 -- 172.21.15.92:0/437783423 >> v1:172.21.15.136:6789/0 conn(0x7fad90159c60 legacy=0x7fad9015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 -- 172.21.15.92:0/437783423 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9015e140 con 0x7fad90069560 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad96637700 1 --2- 172.21.15.92:0/437783423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90069560 0x7fad900699a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fad900d1090 tx=0x7fad88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f407250632194a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad94e34700 1 -- 172.21.15.92:0/437783423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800e070 con 0x7fad90069560 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad94e34700 1 -- 172.21.15.92:0/437783423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8800ba20 con 0x7fad90069560 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad94e34700 1 -- 172.21.15.92:0/437783423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad880092e0 con 0x7fad90069560 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- 172.21.15.92:0/437783423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90069560 msgr2=0x7fad900699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 --2- 172.21.15.92:0/437783423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90069560 0x7fad900699a0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fad900d1090 tx=0x7fad88009840 comp rx=0 tx=0).stop 2023-10-06T03:09:12.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- 172.21.15.92:0/437783423 shutdown_connections 2023-10-06T03:09:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 --2- 172.21.15.92:0/437783423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90069560 0x7fad900699a0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 --2- 172.21.15.92:0/437783423 >> v2:172.21.15.136:3300/0 conn(0x7fad90159890 0x7fad9015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- 172.21.15.92:0/437783423 >> 172.21.15.92:0/437783423 conn(0x7fad90068e20 msgr2=0x7fad900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- 172.21.15.92:0/437783423 shutdown_connections 2023-10-06T03:09:12.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.782+0000 7fad9809a700 1 -- 172.21.15.92:0/437783423 wait complete. 2023-10-06T03:09:12.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.783+0000 7fad9809a700 1 Processor -- start 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.783+0000 7fad9809a700 1 -- start start 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad9809a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad90069560 0x7fad900b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad9809a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 0x7fad900b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad9809a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad90159c60 0x7fad900c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad9809a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad900b6270 con 0x7fad90159890 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad9809a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9016d270 con 0x7fad90069560 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad9809a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9016d550 con 0x7fad90159c60 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 0x7fad900b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 0x7fad900b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44894/0 (socket says 172.21.15.92:44894) 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 -- 172.21.15.92:0/1357111319 learned_addr learned my addr 172.21.15.92:0/1357111319 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:12.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad96637700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad90159c60 0x7fad900c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95e36700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad90069560 0x7fad900b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 -- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad90159c60 msgr2=0x7fad900c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad90159c60 0x7fad900c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 -- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad90069560 msgr2=0x7fad900b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad90069560 0x7fad900b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 -- 172.21.15.92:0/1357111319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88009ae0 con 0x7fad90159890 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad96637700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad90159c60 0x7fad900c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95e36700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad90069560 0x7fad900b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad95635700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 0x7fad900b4180 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fad8000c4e0 tx=0x7fad8000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.784+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80011590 con 0x7fad90159890 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.785+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80011710 con 0x7fad90159890 2023-10-06T03:09:12.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.785+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9016d830 con 0x7fad90159890 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.785+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80011ad0 con 0x7fad90159890 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.785+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad900b46c0 con 0x7fad90159890 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.786+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9016dd50 con 0x7fad90159890 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.786+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad8001b3b0 con 0x7fad90159890 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.786+0000 7fad85ffb700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad74067320 0x7fad740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.786+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad80084300 con 0x7fad90159890 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.787+0000 7fad95e36700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad74067320 0x7fad740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:12.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.787+0000 7fad95e36700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad74067320 0x7fad740697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009d80 tx=0x7fad8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:12.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:12.792+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad9016dd50 con 0x7fad90159890 2023-10-06T03:09:12.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:09:12.810 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:09:13.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.003+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fad9016dd50 con 0x7fad74067320 2023-10-06T03:09:13.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.004+0000 7fad85ffb700 1 -- 172.21.15.92:0/1357111319 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fad9016dd50 con 0x7fad74067320 2023-10-06T03:09:13.006 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:09:13.006 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:13.006 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:09:13.006 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:13.006 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:09:13.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad74067320 msgr2=0x7fad740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad74067320 0x7fad740697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009d80 tx=0x7fad8c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:13.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 msgr2=0x7fad900b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 0x7fad900b4180 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fad8000c4e0 tx=0x7fad8000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 shutdown_connections 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad74067320 0x7fad740697d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad90159c60 0x7fad900c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad90069560 0x7fad900b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 --2- 172.21.15.92:0/1357111319 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad90159890 0x7fad900b4180 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 >> 172.21.15.92:0/1357111319 conn(0x7fad90068e20 msgr2=0x7fad9014d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 shutdown_connections 2023-10-06T03:09:13.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.006+0000 7fad9809a700 1 -- 172.21.15.92:0/1357111319 wait complete. 2023-10-06T03:09:13.020 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:09:13.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862cad7700 1 Processor -- start 2023-10-06T03:09:13.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862cad7700 1 -- start start 2023-10-06T03:09:13.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862cad7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628069560 0x7f86280699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862cad7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8628069ee0 con 0x7f8628159890 2023-10-06T03:09:13.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862cad7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862815a160 con 0x7f8628159c60 2023-10-06T03:09:13.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862cad7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86280d13a0 con 0x7f8628069560 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8628159890 0x7f862815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40830/0 (socket says 172.21.15.92:40830) 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f862659c700 1 -- 172.21.15.92:0/1592768790 learned_addr learned my addr 172.21.15.92:0/1592768790 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8626d9d700 1 --2- 172.21.15.92:0/1592768790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628069560 0x7f86280699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171301546 0 0) 0x7f8628069ee0 con 0x7f8628159890 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8614003570 con 0x7f8628159890 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035366815 0 0) 0x7f862815a160 con 0x7f8628159c60 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8628069ee0 con 0x7f8628159c60 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657500430 0 0) 0x7f8614003570 con 0x7f8628159890 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8626d9d700 1 -- 172.21.15.92:0/1592768790 >> v1:172.21.15.136:6789/0 conn(0x7f8628159c60 legacy=0x7f862815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8626d9d700 1 -- 172.21.15.92:0/1592768790 >> v1:172.21.15.92:6789/0 conn(0x7f8628159890 legacy=0x7f862815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8626d9d700 1 -- 172.21.15.92:0/1592768790 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862815e140 con 0x7f8628069560 2023-10-06T03:09:13.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.137+0000 7f8626d9d700 1 --2- 172.21.15.92:0/1592768790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628069560 0x7f86280699a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8620009e90 tx=0x7f8620009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b092d47fad316daa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862000e070 con 0x7f8628069560 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862000b870 con 0x7f8628069560 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f8625d9b700 1 -- 172.21.15.92:0/1592768790 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618001470 con 0x7f8628069560 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 -- 172.21.15.92:0/1592768790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628069560 msgr2=0x7f86280699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 --2- 172.21.15.92:0/1592768790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628069560 0x7f86280699a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8620009e90 tx=0x7f8620009780 comp rx=0 tx=0).stop 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 -- 172.21.15.92:0/1592768790 shutdown_connections 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 --2- 172.21.15.92:0/1592768790 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628069560 0x7f86280699a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 -- 172.21.15.92:0/1592768790 >> 172.21.15.92:0/1592768790 conn(0x7f8628068e20 msgr2=0x7f86280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 -- 172.21.15.92:0/1592768790 shutdown_connections 2023-10-06T03:09:13.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.138+0000 7f862cad7700 1 -- 172.21.15.92:0/1592768790 wait complete. 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.139+0000 7f862cad7700 1 Processor -- start 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.139+0000 7f862cad7700 1 -- start start 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862cad7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 0x7f86281540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862cad7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8628159890 0x7f86280b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862cad7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628159c60 0x7f862815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862cad7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862815c510 con 0x7f8628069560 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862cad7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862815bf50 con 0x7f8628159890 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862cad7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862815c230 con 0x7f8628159c60 2023-10-06T03:09:13.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 0x7f86281540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 0x7f86281540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44908/0 (socket says 172.21.15.92:44908) 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 -- 172.21.15.92:0/4283237520 learned_addr learned my addr 172.21.15.92:0/4283237520 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f861ffff700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8628159890 0x7f86280b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f8626d9d700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628159c60 0x7f862815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 -- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628159c60 msgr2=0x7f862815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628159c60 0x7f862815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 -- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8628159890 msgr2=0x7f86280b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8628159890 0x7f86280b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.140+0000 7f862659c700 1 -- 172.21.15.92:0/4283237520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620009a20 con 0x7f8628069560 2023-10-06T03:09:13.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.141+0000 7f862659c700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 0x7f86281540c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f861800cb70 tx=0x7f8618001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.141+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861800ce80 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.141+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618007360 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.141+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618007700 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.141+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86280ce7e0 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.141+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862815d460 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.142+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86280ced00 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.142+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8618004a40 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.143+0000 7f861e7fc700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8608067320 0x7f86080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.143+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8618084200 con 0x7f8628069560 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.143+0000 7f861ffff700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8608067320 0x7f86080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.143+0000 7f861ffff700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8608067320 0x7f86080697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f86280b2630 tx=0x7f8610007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:13.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.147+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86280ced00 con 0x7f8628069560 2023-10-06T03:09:13.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.332+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f86280ced00 con 0x7f8628069560 2023-10-06T03:09:13.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.333+0000 7f861e7fc700 1 -- 172.21.15.92:0/4283237520 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8618050d70 con 0x7f8628069560 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8608067320 msgr2=0x7f86080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8608067320 0x7f86080697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f86280b2630 tx=0x7f8610007d50 comp rx=0 tx=0).stop 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 msgr2=0x7f86281540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 0x7f86281540c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f861800cb70 tx=0x7f8618001b50 comp rx=0 tx=0).stop 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 shutdown_connections 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8608067320 0x7f86080697d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8628069560 0x7f86281540c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8628159c60 0x7f862815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 --2- 172.21.15.92:0/4283237520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8628159890 0x7f86280b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 >> 172.21.15.92:0/4283237520 conn(0x7f8628068e20 msgr2=0x7f86280c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 shutdown_connections 2023-10-06T03:09:13.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.335+0000 7f862cad7700 1 -- 172.21.15.92:0/4283237520 wait complete. 2023-10-06T03:09:13.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:09:13.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:09:13.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:13.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:09:13.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:13.385 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:09:13.439 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:09:13.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:09:13.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:13.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:09:13.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:13.478 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:09:13.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:09:13.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:13.534 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T03:09:13.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:09:13.549 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T03:09:13.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:09:13.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:13.549 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T03:09:13.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:09:13.589 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T03:09:13.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000645996 --yes-i-really-mean-it 2023-10-06T03:09:13.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.709+0000 7fc532cc6700 1 Processor -- start 2023-10-06T03:09:13.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.709+0000 7fc532cc6700 1 -- start start 2023-10-06T03:09:13.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc532cc6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc52c0b55a0 0x7fc52c164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc532cc6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0cfa60 con 0x7fc52c0cd910 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc532cc6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52c0b5b00 con 0x7fc52c159ec0 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc532cc6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc52c0b5da0 con 0x7fc52c0b55a0 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc52c0cd910 0x7fc52c0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40840/0 (socket says 172.21.15.92:40840) 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52bfff700 1 -- 172.21.15.92:0/1340962121 learned_addr learned my addr 172.21.15.92:0/1340962121 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc531263700 1 --2- 172.21.15.92:0/1340962121 >> v2:172.21.15.136:3300/0 conn(0x7fc52c0b55a0 0x7fc52c164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497202913 0 0) 0x7fc52c0cfa60 con 0x7fc52c0cd910 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc51c003570 con 0x7fc52c0cd910 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3457342780 0 0) 0x7fc51c003570 con 0x7fc52c0cd910 2023-10-06T03:09:13.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 >> v2:172.21.15.136:3300/0 conn(0x7fc52c0b55a0 msgr2=0x7fc52c164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 --2- 172.21.15.92:0/1340962121 >> v2:172.21.15.136:3300/0 conn(0x7fc52c0b55a0 0x7fc52c164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 >> v1:172.21.15.136:6789/0 conn(0x7fc52c159ec0 legacy=0x7fc52c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0b6080 con 0x7fc52c0cd910 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc518001470 con 0x7fc52c0cd910 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.710+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 >> v1:172.21.15.92:6789/0 conn(0x7fc52c0cd910 legacy=0x7fc52c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc51c003a60 0x7fc51c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc51c004810 0x7fc51c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc51c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c0b5da0 con 0x7fc51c005210 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c00c130 con 0x7fc51c004810 2023-10-06T03:09:13.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c00c2c0 con 0x7fc51c003a60 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc51c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52bfff700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc51c004810 0x7fc51c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc530a62700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc51c003a60 0x7fc51c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 -- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc51c003a60 msgr2=0x7fc51c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc51c003a60 0x7fc51c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 -- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc51c004810 msgr2=0x7fc51c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc51c004810 0x7fc51c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 -- 172.21.15.92:0/1340962121 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c003570 con 0x7fc51c005210 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 -- 172.21.15.92:0/1340962121 reap_dead start 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc531263700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc51c00ba60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc52400e240 tx=0x7fc52400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9886031aacf4b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:13.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524009ac0 con 0x7fc51c005210 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52400f040 con 0x7fc51c005210 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.711+0000 7fc52b7fe700 1 -- 172.21.15.92:0/1340962121 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52400ed40 con 0x7fc51c005210 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 -- 172.21.15.92:0/1340962121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 msgr2=0x7fc51c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc51c00ba60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc52400e240 tx=0x7fc52400b040 comp rx=0 tx=0).stop 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 -- 172.21.15.92:0/1340962121 shutdown_connections 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 --2- 172.21.15.92:0/1340962121 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc51c00ba60 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 -- 172.21.15.92:0/1340962121 >> 172.21.15.92:0/1340962121 conn(0x7fc52c0999f0 msgr2=0x7fc52c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 -- 172.21.15.92:0/1340962121 shutdown_connections 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.712+0000 7fc532cc6700 1 -- 172.21.15.92:0/1340962121 wait complete. 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.713+0000 7fc532cc6700 1 Processor -- start 2023-10-06T03:09:13.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.713+0000 7fc532cc6700 1 -- start start 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc52c169d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c09a460 0x7fc52c09b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc52c14b130 0x7fc52c09cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c068390 con 0x7fc51c005210 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c068c30 con 0x7fc52c09a460 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c09bda0 con 0x7fc52c14b130 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc52c169d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc52c169d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44918/0 (socket says 172.21.15.92:44918) 2023-10-06T03:09:13.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 -- 172.21.15.92:0/174442841 learned_addr learned my addr 172.21.15.92:0/174442841 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc531263700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc52c14b130 0x7fc52c09cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc52bfff700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c09a460 0x7fc52c09b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 -- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc52c14b130 msgr2=0x7fc52c09cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc52c14b130 0x7fc52c09cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 -- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c09a460 msgr2=0x7fc52c09b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c09a460 0x7fc52c09b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 -- 172.21.15.92:0/174442841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0685e0 con 0x7fc51c005210 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc530a62700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc52c169d90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc520002970 tx=0x7fc520007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520013600 con 0x7fc51c005210 2023-10-06T03:09:13.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc520013760 con 0x7fc51c005210 2023-10-06T03:09:13.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520013ae0 con 0x7fc51c005210 2023-10-06T03:09:13.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.714+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c0688c0 con 0x7fc51c005210 2023-10-06T03:09:13.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.715+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c14b5a0 con 0x7fc51c005210 2023-10-06T03:09:13.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.715+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c14b7c0 con 0x7fc51c005210 2023-10-06T03:09:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.716+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc52001c400 con 0x7fc51c005210 2023-10-06T03:09:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.716+0000 7fc528ff9700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc510067320 0x7fc5100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:13.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.716+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc520085180 con 0x7fc51c005210 2023-10-06T03:09:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.716+0000 7fc52bfff700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc510067320 0x7fc5100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.717+0000 7fc52bfff700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc510067320 0x7fc5100697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc52c09c560 tx=0x7fc518001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:13.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.720+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc52c14b7c0 con 0x7fc51c005210 2023-10-06T03:09:13.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:13.931+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000645996", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc52c14b7c0 con 0x7fc510067320 2023-10-06T03:09:16.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.665+0000 7fc528ff9700 1 -- 172.21.15.92:0/174442841 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7fc52c14b7c0 con 0x7fc510067320 2023-10-06T03:09:16.666 INFO:teuthology.orchestra.run.smithi092.stdout:FS volume 'cephfs' renamed to 'volume_0000000000645996' 2023-10-06T03:09:16.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc510067320 msgr2=0x7fc5100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:16.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc510067320 0x7fc5100697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc52c09c560 tx=0x7fc518001e50 comp rx=0 tx=0).stop 2023-10-06T03:09:16.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 msgr2=0x7fc52c169d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc52c169d90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fc520002970 tx=0x7fc520007d80 comp rx=0 tx=0).stop 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 shutdown_connections 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc510067320 0x7fc5100697d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc51c005210 0x7fc52c169d90 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc52c14b130 0x7fc52c09cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 --2- 172.21.15.92:0/174442841 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc52c09a460 0x7fc52c09b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.667+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 >> 172.21.15.92:0/174442841 conn(0x7fc52c0999f0 msgr2=0x7fc52c0c5610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.668+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 shutdown_connections 2023-10-06T03:09:16.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.668+0000 7fc532cc6700 1 -- 172.21.15.92:0/174442841 wait complete. 2023-10-06T03:09:16.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:09:16.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.807+0000 7fad7bfff700 1 Processor -- start 2023-10-06T03:09:16.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.807+0000 7fad7bfff700 1 -- start start 2023-10-06T03:09:16.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad7c0cc3c0 0x7fad7c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:16.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7bfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c0b16a0 con 0x7fad7c1574d0 2023-10-06T03:09:16.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7bfff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c0b1c60 con 0x7fad7c06bf20 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7bfff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7c0cd980 con 0x7fad7c0cc3c0 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad727fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad7c0cc3c0 0x7fad7c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad7c1574d0 0x7fad7c0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40846/0 (socket says 172.21.15.92:40846) 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad727fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad7c0cc3c0 0x7fad7c06b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44934/0 (socket says 172.21.15.92:44934) 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7affd700 1 -- 172.21.15.92:0/3145632774 learned_addr learned my addr 172.21.15.92:0/3145632774 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad727fc700 1 -- 172.21.15.92:0/3145632774 >> v1:172.21.15.136:6789/0 conn(0x7fad7c06bf20 legacy=0x7fad7c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad727fc700 1 -- 172.21.15.92:0/3145632774 >> v1:172.21.15.92:6789/0 conn(0x7fad7c1574d0 legacy=0x7fad7c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad727fc700 1 -- 172.21.15.92:0/3145632774 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c06c360 con 0x7fad7c0cc3c0 2023-10-06T03:09:16.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.808+0000 7fad7a7fc700 1 -- 172.21.15.92:0/3145632774 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736980128 0 0) 0x7fad7c0b16a0 con 0x7fad7c1574d0 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad727fc700 1 --2- 172.21.15.92:0/3145632774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c0cc3c0 0x7fad7c06b9e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fad640098e0 tx=0x7fad640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0886fed18ba07e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7a7fc700 1 -- 172.21.15.92:0/3145632774 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64009e80 con 0x7fad7c0cc3c0 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7a7fc700 1 -- 172.21.15.92:0/3145632774 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6400b820 con 0x7fad7c0cc3c0 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7a7fc700 1 -- 172.21.15.92:0/3145632774 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400bba0 con 0x7fad7c0cc3c0 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 -- 172.21.15.92:0/3145632774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c0cc3c0 msgr2=0x7fad7c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 --2- 172.21.15.92:0/3145632774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c0cc3c0 0x7fad7c06b9e0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fad640098e0 tx=0x7fad640091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 -- 172.21.15.92:0/3145632774 shutdown_connections 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 --2- 172.21.15.92:0/3145632774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c0cc3c0 0x7fad7c06b9e0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 -- 172.21.15.92:0/3145632774 >> 172.21.15.92:0/3145632774 conn(0x7fad7c0ce430 msgr2=0x7fad7c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:16.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 -- 172.21.15.92:0/3145632774 shutdown_connections 2023-10-06T03:09:16.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.809+0000 7fad7bfff700 1 -- 172.21.15.92:0/3145632774 wait complete. 2023-10-06T03:09:16.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.810+0000 7fad7bfff700 1 Processor -- start 2023-10-06T03:09:16.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.810+0000 7fad7bfff700 1 -- start start 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 0x7fad7c148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0cc3c0 0x7fad7c06a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1574d0 0x7fad7c154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7bfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c15ae40 con 0x7fad7c06bf20 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7bfff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c15bc60 con 0x7fad7c1574d0 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7bfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c06b380 con 0x7fad7c0cc3c0 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 0x7fad7c148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 0x7fad7c148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44946/0 (socket says 172.21.15.92:44946) 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad7affd700 1 -- 172.21.15.92:0/2334419079 learned_addr learned my addr 172.21.15.92:0/2334419079 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:16.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.811+0000 7fad727fc700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0cc3c0 0x7fad7c06a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7b7fe700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1574d0 0x7fad7c154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7affd700 1 -- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0cc3c0 msgr2=0x7fad7c06a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7affd700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0cc3c0 0x7fad7c06a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7affd700 1 -- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1574d0 msgr2=0x7fad7c154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7affd700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1574d0 0x7fad7c154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7affd700 1 -- 172.21.15.92:0/2334419079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64009420 con 0x7fad7c06bf20 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad727fc700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0cc3c0 0x7fad7c06a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7affd700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 0x7fad7c148dd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c5e0 tx=0x7fad6c007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0072e0 con 0x7fad7c06bf20 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c007440 con 0x7fad7c06bf20 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c169a80 con 0x7fad7c06bf20 2023-10-06T03:09:16.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0057f0 con 0x7fad7c06bf20 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.812+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c15b610 con 0x7fad7c06bf20 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.813+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c04c3c0 con 0x7fad7c06bf20 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.813+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad6c005c00 con 0x7fad7c06bf20 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.814+0000 7fad737fe700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.814+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fad6c0843c0 con 0x7fad7c06bf20 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.814+0000 7fad727fc700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:16.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.814+0000 7fad727fc700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0cc800 tx=0x7fad64016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:16.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:16.818+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad6c050ea0 con 0x7fad7c06bf20 2023-10-06T03:09:17.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.028+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fad7c0adad0 con 0x7fad5c067320 2023-10-06T03:09:17.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.029+0000 7fad737fe700 1 -- 172.21.15.92:0/2334419079 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7fad7c0adad0 con 0x7fad5c067320 2023-10-06T03:09:17.030 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:09:17.030 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:17.030 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volume_0000000000645996" 2023-10-06T03:09:17.030 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:17.030 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:09:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 msgr2=0x7fad5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0cc800 tx=0x7fad64016000 comp rx=0 tx=0).stop 2023-10-06T03:09:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 msgr2=0x7fad7c148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 0x7fad7c148dd0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c5e0 tx=0x7fad6c007e80 comp rx=0 tx=0).stop 2023-10-06T03:09:17.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 shutdown_connections 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad7c06bf20 0x7fad7c148dd0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1574d0 0x7fad7c154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 --2- 172.21.15.92:0/2334419079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0cc3c0 0x7fad7c06a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 >> 172.21.15.92:0/2334419079 conn(0x7fad7c0ce430 msgr2=0x7fad7c155b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 shutdown_connections 2023-10-06T03:09:17.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.031+0000 7fad7bfff700 1 -- 172.21.15.92:0/2334419079 wait complete. 2023-10-06T03:09:17.044 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.157+0000 7fc57da0e700 1 Processor -- start 2023-10-06T03:09:17.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.157+0000 7fc57da0e700 1 -- start start 2023-10-06T03:09:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57da0e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc578159420 0x7fc5780b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57da0e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 0x7fc5780b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57da0e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5780b11d0 con 0x7fc57815d230 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57da0e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5780b14b0 con 0x7fc578159420 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57da0e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780b1790 con 0x7fc5781597f0 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57d20d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc57815d230 0x7fc5780b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40862/0 (socket says 172.21.15.92:40862) 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57d20d700 1 -- 172.21.15.92:0/3463719261 learned_addr learned my addr 172.21.15.92:0/3463719261 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc577fff700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 0x7fc5780b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/3463719261 >> v2:172.21.15.136:3300/0 conn(0x7fc578159420 0x7fc5780b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.158+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563199735 0 0) 0x7fc5780b11d0 con 0x7fc57815d230 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc568003570 con 0x7fc57815d230 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2084083615 0 0) 0x7fc568003570 con 0x7fc57815d230 2023-10-06T03:09:17.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 msgr2=0x7fc5780b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 0x7fc5780b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 >> v2:172.21.15.136:3300/0 conn(0x7fc578159420 msgr2=0x7fc5780b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 --2- 172.21.15.92:0/3463719261 >> v2:172.21.15.136:3300/0 conn(0x7fc578159420 0x7fc5780b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5780b1a70 con 0x7fc57815d230 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc5700017f0 con 0x7fc57815d230 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 >> v1:172.21.15.92:6789/0 conn(0x7fc57815d230 legacy=0x7fc5780b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc568003a60 0x7fc568003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc568004810 0x7fc568004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 0x7fc5680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780b14b0 con 0x7fc568003a60 2023-10-06T03:09:17.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780b1790 con 0x7fc568004810 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56800ba60 con 0x7fc5781597f0 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc568003a60 0x7fc568003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc577fff700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc568004810 0x7fc568004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57d20d700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 0x7fc5680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 -- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 msgr2=0x7fc5680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5781597f0 0x7fc5680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 -- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc568004810 msgr2=0x7fc568004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc568004810 0x7fc568004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 -- 172.21.15.92:0/3463719261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc568003570 con 0x7fc568003a60 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc577fff700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc568004810 0x7fc568004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc568003a60 0x7fc568003e70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009c30 tx=0x7fc56c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca36772052dfb3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00e710 con 0x7fc568003a60 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c00e870 con 0x7fc568003a60 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.159+0000 7fc5777fe700 1 -- 172.21.15.92:0/3463719261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00ec30 con 0x7fc568003a60 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57da0e700 1 -- 172.21.15.92:0/3463719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc568003a60 msgr2=0x7fc568003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57da0e700 1 --2- 172.21.15.92:0/3463719261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc568003a60 0x7fc568003e70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009c30 tx=0x7fc56c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57d20d700 1 -- 172.21.15.92:0/3463719261 reap_dead start 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57da0e700 1 -- 172.21.15.92:0/3463719261 shutdown_connections 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57da0e700 1 -- 172.21.15.92:0/3463719261 >> 172.21.15.92:0/3463719261 conn(0x7fc5780d0660 msgr2=0x7fc5780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57da0e700 1 -- 172.21.15.92:0/3463719261 shutdown_connections 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.160+0000 7fc57da0e700 1 -- 172.21.15.92:0/3463719261 wait complete. 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.161+0000 7fc57da0e700 1 Processor -- start 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.161+0000 7fc57da0e700 1 -- start start 2023-10-06T03:09:17.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57da0e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc568003a60 0x7fc57814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57da0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 0x7fc5780b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57da0e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc57815f4d0 0x7fc5780b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57da0e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57815b460 con 0x7fc57815f060 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57da0e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57815b740 con 0x7fc57815f4d0 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57da0e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57815ba20 con 0x7fc568003a60 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 0x7fc5780b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 0x7fc5780b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44958/0 (socket says 172.21.15.92:44958) 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 -- 172.21.15.92:0/1909948008 learned_addr learned my addr 172.21.15.92:0/1909948008 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc568003a60 0x7fc57814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57d20d700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc57815f4d0 0x7fc5780b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 -- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc568003a60 msgr2=0x7fc57814f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc568003a60 0x7fc57814f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 -- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc57815f4d0 msgr2=0x7fc5780b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc57815f4d0 0x7fc5780b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc577fff700 1 -- 172.21.15.92:0/1909948008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc578151450 con 0x7fc57815f060 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc568003a60 0x7fc57814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.162+0000 7fc57d20d700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc57815f4d0 0x7fc5780b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.163+0000 7fc577fff700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 0x7fc5780b5090 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc5640118e0 tx=0x7fc564008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.163+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc564011e80 con 0x7fc57815f060 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.163+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56400f8e0 con 0x7fc57815f060 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.163+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57814f6b0 con 0x7fc57815f060 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.163+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400fc80 con 0x7fc57815f060 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.163+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57815b0a0 con 0x7fc57815f060 2023-10-06T03:09:17.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.164+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57815b880 con 0x7fc57815f060 2023-10-06T03:09:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.164+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc564015460 con 0x7fc57815f060 2023-10-06T03:09:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.164+0000 7fc574ff9700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc55c067320 0x7fc55c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.165+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fc564085810 con 0x7fc57815f060 2023-10-06T03:09:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.165+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc55c067320 0x7fc55c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.165+0000 7fc57ca0c700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc55c067320 0x7fc55c0697d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0091e0 tx=0x7fc56c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.169+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc57815b880 con 0x7fc57815f060 2023-10-06T03:09:17.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.395+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc57815b880 con 0x7fc57815f060 2023-10-06T03:09:17.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.395+0000 7fc574ff9700 1 -- 172.21.15.92:0/1909948008 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4618 (secure 0 0 0) 0x7fc57815b880 con 0x7fc57815f060 2023-10-06T03:09:17.396 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:17.396 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:14.655190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000645996","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc55c067320 msgr2=0x7fc55c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc55c067320 0x7fc55c0697d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc56c0091e0 tx=0x7fc56c01a000 comp rx=0 tx=0).stop 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 msgr2=0x7fc5780b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 0x7fc5780b5090 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fc5640118e0 tx=0x7fc564008040 comp rx=0 tx=0).stop 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 shutdown_connections 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc55c067320 0x7fc55c0697d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc57815f4d0 0x7fc5780b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc568003a60 0x7fc57814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 --2- 172.21.15.92:0/1909948008 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57815f060 0x7fc5780b5090 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 >> 172.21.15.92:0/1909948008 conn(0x7fc5780d0660 msgr2=0x7fc57815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 shutdown_connections 2023-10-06T03:09:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.397+0000 7fc57da0e700 1 -- 172.21.15.92:0/1909948008 wait complete. 2023-10-06T03:09:17.399 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 96 2023-10-06T03:09:17.410 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.524+0000 7fb0a6aa8700 1 Processor -- start 2023-10-06T03:09:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.524+0000 7fb0a6aa8700 1 -- start start 2023-10-06T03:09:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a6aa8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb0a015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a6aa8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb0a015db10 0x7fb0a0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a6aa8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a015a160 con 0x7fb0a0159890 2023-10-06T03:09:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a6aa8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a00d13a0 con 0x7fb0a015db10 2023-10-06T03:09:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a6aa8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015e140 con 0x7fb0a0159c60 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a4844700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb0a0159890 0x7fb0a015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40874/0 (socket says 172.21.15.92:40874) 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a4844700 1 -- 172.21.15.92:0/1278611190 learned_addr learned my addr 172.21.15.92:0/1278611190 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714744737 0 0) 0x7fb0a015a160 con 0x7fb0a0159890 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb090003570 con 0x7fb0a0159890 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb0a5045700 1 --2- 172.21.15.92:0/1278611190 >> v2:172.21.15.136:3300/0 conn(0x7fb0a015db10 0x7fb0a0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09ffff700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb0a015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4162840259 0 0) 0x7fb090003570 con 0x7fb0a0159890 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 msgr2=0x7fb0a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb0a015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 >> v2:172.21.15.136:3300/0 conn(0x7fb0a015db10 msgr2=0x7fb0a0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 --2- 172.21.15.92:0/1278611190 >> v2:172.21.15.136:3300/0 conn(0x7fb0a015db10 0x7fb0a0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a015a3d0 con 0x7fb0a0159890 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb094001470 con 0x7fb0a0159890 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 >> v1:172.21.15.92:6789/0 conn(0x7fb0a0159890 legacy=0x7fb0a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.525+0000 7fb09f7fe700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb090003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb090004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090004d50 0x7fb0900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00d13a0 con 0x7fb090003fe0 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015e140 con 0x7fb090004d50 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09000ba90 con 0x7fb0a0159c60 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb090004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb0a4844700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb090003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb0a5045700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090004d50 0x7fb0900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 -- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 msgr2=0x7fb090003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb090003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 -- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090004d50 msgr2=0x7fb0900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090004d50 0x7fb0900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 -- 172.21.15.92:0/1278611190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090003570 con 0x7fb090003fe0 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb0a4844700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0159c60 0x7fb090003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09ffff700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb090004810 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb0a015a090 tx=0x7fb08c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781f1f0225df01af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00e6b0 con 0x7fb090003fe0 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c00e810 con 0x7fb090003fe0 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.526+0000 7fb09f7fe700 1 -- 172.21.15.92:0/1278611190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00ebd0 con 0x7fb090003fe0 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/1278611190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 msgr2=0x7fb090004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/1278611190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb090004810 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb0a015a090 tx=0x7fb08c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a5045700 1 -- 172.21.15.92:0/1278611190 reap_dead start 2023-10-06T03:09:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/1278611190 shutdown_connections 2023-10-06T03:09:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/1278611190 >> 172.21.15.92:0/1278611190 conn(0x7fb0a0068e20 msgr2=0x7fb0a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/1278611190 shutdown_connections 2023-10-06T03:09:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.527+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/1278611190 wait complete. 2023-10-06T03:09:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.528+0000 7fb0a6aa8700 1 Processor -- start 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.528+0000 7fb0a6aa8700 1 -- start start 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb0a016df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0a014a750 0x7fb0a015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0149e50 0x7fb0a0152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00cd5b0 con 0x7fb090003fe0 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00cd2d0 con 0x7fb0a014a750 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0152f70 con 0x7fb0a0149e50 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb0a016df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb0a016df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44968/0 (socket says 172.21.15.92:44968) 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 -- 172.21.15.92:0/199530944 learned_addr learned my addr 172.21.15.92:0/199530944 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:17.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb09ffff700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0a014a750 0x7fb0a015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a5045700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0149e50 0x7fb0a0152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 -- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0149e50 msgr2=0x7fb0a0152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0149e50 0x7fb0a0152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 -- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0a014a750 msgr2=0x7fb0a015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0a014a750 0x7fb0a015b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 -- 172.21.15.92:0/199530944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a015bcf0 con 0x7fb090003fe0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a5045700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0149e50 0x7fb0a0152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a4844700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb0a016df40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb0940015e0 tx=0x7fb094001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400c7b0 con 0x7fb090003fe0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09400c910 con 0x7fb090003fe0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a01517d0 con 0x7fb090003fe0 2023-10-06T03:09:17.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400ccd0 con 0x7fb090003fe0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.529+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a0151d70 con 0x7fb090003fe0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.530+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a00cd890 con 0x7fb090003fe0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.531+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb094013630 con 0x7fb090003fe0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.531+0000 7fb09cff9700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb084067320 0x7fb0840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.531+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fb094084e90 con 0x7fb090003fe0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.531+0000 7fb09ffff700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb084067320 0x7fb0840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.532+0000 7fb09ffff700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb084067320 0x7fb0840697d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0091e0 tx=0x7fb08c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.535+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb094051970 con 0x7fb090003fe0 2023-10-06T03:09:17.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.724+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0a0069c80 con 0x7fb090003fe0 2023-10-06T03:09:17.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.724+0000 7fb09cff9700 1 -- 172.21.15.92:0/199530944 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fb094054f90 con 0x7fb090003fe0 2023-10-06T03:09:17.725 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:17.725 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":143,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:16.658258+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs.volume_0000000000645996.meta","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000645996"}}},{"pool":26,"pool_name":"cephfs.volume_0000000000645996.data","create_time":"2023-10-06T03:09:01.925339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000645996"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:17.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb084067320 msgr2=0x7fb0840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb084067320 0x7fb0840697d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0091e0 tx=0x7fb08c01a000 comp rx=0 tx=0).stop 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 msgr2=0x7fb0a016df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb0a016df40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb0940015e0 tx=0x7fb094001570 comp rx=0 tx=0).stop 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 shutdown_connections 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb084067320 0x7fb0840697d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb090003fe0 0x7fb0a016df40 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0a0149e50 0x7fb0a0152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 --2- 172.21.15.92:0/199530944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0a014a750 0x7fb0a015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.726+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 >> 172.21.15.92:0/199530944 conn(0x7fb0a0068e20 msgr2=0x7fb0a0154050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.727+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 shutdown_connections 2023-10-06T03:09:17.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.727+0000 7fb0a6aa8700 1 -- 172.21.15.92:0/199530944 wait complete. 2023-10-06T03:09:17.740 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:09:17.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:17.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:17.823 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:09:17.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:17.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.937+0000 7f7520593700 1 Processor -- start 2023-10-06T03:09:17.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.937+0000 7f7520593700 1 -- start start 2023-10-06T03:09:17.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f7520593700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7518159c60 0x7f751815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f7520593700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751815a160 con 0x7f7518159890 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f7520593700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75180d13a0 con 0x7f751815db10 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f7520593700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751815df50 con 0x7f7518159c60 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f751db2e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7518159c60 0x7f751815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f751e32f700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7518159890 0x7f751815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40880/0 (socket says 172.21.15.92:40880) 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.938+0000 7f751e32f700 1 -- 172.21.15.92:0/344230454 learned_addr learned my addr 172.21.15.92:0/344230454 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751db2e700 1 -- 172.21.15.92:0/344230454 >> v1:172.21.15.136:6789/0 conn(0x7f751815db10 legacy=0x7f7518069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751db2e700 1 -- 172.21.15.92:0/344230454 >> v1:172.21.15.92:6789/0 conn(0x7f7518159890 legacy=0x7f751815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751d32d700 1 -- 172.21.15.92:0/344230454 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568449348 0 0) 0x7f751815a160 con 0x7f7518159890 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751db2e700 1 -- 172.21.15.92:0/344230454 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751815e140 con 0x7f7518159c60 2023-10-06T03:09:17.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751db2e700 1 --2- 172.21.15.92:0/344230454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f751815d5d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f75080098e0 tx=0x7f75080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318cc8395d2ad57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751d32d700 1 -- 172.21.15.92:0/344230454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508009e80 con 0x7f7518159c60 2023-10-06T03:09:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751d32d700 1 -- 172.21.15.92:0/344230454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750800b820 con 0x7f7518159c60 2023-10-06T03:09:17.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f751d32d700 1 -- 172.21.15.92:0/344230454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750800bba0 con 0x7f7518159c60 2023-10-06T03:09:17.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 -- 172.21.15.92:0/344230454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 msgr2=0x7f751815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 --2- 172.21.15.92:0/344230454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f751815d5d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f75080098e0 tx=0x7f75080091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 -- 172.21.15.92:0/344230454 shutdown_connections 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 --2- 172.21.15.92:0/344230454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f751815d5d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 -- 172.21.15.92:0/344230454 >> 172.21.15.92:0/344230454 conn(0x7f7518068e20 msgr2=0x7f75180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 -- 172.21.15.92:0/344230454 shutdown_connections 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.939+0000 7f7520593700 1 -- 172.21.15.92:0/344230454 wait complete. 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.940+0000 7f7520593700 1 Processor -- start 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 -- start start 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7518159890 0x7f75181501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f75180cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f751815db10 0x7f75180c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751816a6a0 con 0x7f7518159c60 2023-10-06T03:09:17.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7518159060 con 0x7f751815db10 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f7520593700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75180cdba0 con 0x7f7518159890 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f751db2e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f75180cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f751db2e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f75180cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44984/0 (socket says 172.21.15.92:44984) 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f751db2e700 1 -- 172.21.15.92:0/4177247236 learned_addr learned my addr 172.21.15.92:0/4177247236 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f751eb30700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f751815db10 0x7f75180c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.941+0000 7f751e32f700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7518159890 0x7f75181501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f751db2e700 1 -- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7518159890 msgr2=0x7f75181501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f751db2e700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7518159890 0x7f75181501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f751db2e700 1 -- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f751815db10 msgr2=0x7f75180c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f751db2e700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f751815db10 0x7f75180c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f751db2e700 1 -- 172.21.15.92:0/4177247236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508009420 con 0x7f7518159c60 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f751db2e700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f75180cd100 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f75080030c0 tx=0x7f7508016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750800ba10 con 0x7f7518159c60 2023-10-06T03:09:17.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7508020070 con 0x7f7518159c60 2023-10-06T03:09:17.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7518169d70 con 0x7f7518159c60 2023-10-06T03:09:17.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750801b6a0 con 0x7f7518159c60 2023-10-06T03:09:17.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.942+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751816a330 con 0x7f7518159c60 2023-10-06T03:09:17.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.943+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751816a810 con 0x7f7518159c60 2023-10-06T03:09:17.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.943+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7508022030 con 0x7f7518159c60 2023-10-06T03:09:17.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.944+0000 7f750e7fc700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f74fc067320 0x7f74fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:17.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.944+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f7508086d30 con 0x7f7518159c60 2023-10-06T03:09:17.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.944+0000 7f751e32f700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f74fc067320 0x7f74fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:17.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.944+0000 7f751e32f700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f74fc067320 0x7f74fc0697d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7518157090 tx=0x7f7514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:17.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:17.948+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7508053920 con 0x7f7518159c60 2023-10-06T03:09:18.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.174+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f751816a7e0 con 0x7f7518159c60 2023-10-06T03:09:18.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.174+0000 7f750e7fc700 1 -- 172.21.15.92:0/4177247236 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f7508056f40 con 0x7f7518159c60 2023-10-06T03:09:18.175 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:18.175 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:14.655190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000645996","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f74fc067320 msgr2=0x7f74fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f74fc067320 0x7f74fc0697d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7518157090 tx=0x7f7514009040 comp rx=0 tx=0).stop 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 msgr2=0x7f75180cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f75180cd100 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f75080030c0 tx=0x7f7508016000 comp rx=0 tx=0).stop 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 shutdown_connections 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f74fc067320 0x7f74fc0697d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f751815db10 0x7f75180c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7518159890 0x7f75181501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 --2- 172.21.15.92:0/4177247236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7518159c60 0x7f75180cd100 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 >> 172.21.15.92:0/4177247236 conn(0x7f7518068e20 msgr2=0x7f75181570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 shutdown_connections 2023-10-06T03:09:18.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.177+0000 7f7520593700 1 -- 172.21.15.92:0/4177247236 wait complete. 2023-10-06T03:09:18.179 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 96 2023-10-06T03:09:18.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:18.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.303+0000 7f0ab0f1a700 1 Processor -- start 2023-10-06T03:09:18.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.303+0000 7f0ab0f1a700 1 -- start start 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0ab0f1a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0aac069560 0x7f0aac0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0ab0f1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac069ee0 con 0x7f0aac159c60 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0ab0f1a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac15a160 con 0x7f0aac159890 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0ab0f1a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac0d13a0 con 0x7f0aac069560 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aaad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0aac069560 0x7f0aac0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aa9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0aac159c60 0x7f0aac15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40896/0 (socket says 172.21.15.92:40896) 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aaad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0aac069560 0x7f0aac0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44992/0 (socket says 172.21.15.92:44992) 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aa9d9b700 1 -- 172.21.15.92:0/2172962858 learned_addr learned my addr 172.21.15.92:0/2172962858 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:18.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2172962858 >> v1:172.21.15.136:6789/0 conn(0x7f0aac159890 legacy=0x7f0aac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:18.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2172962858 >> v1:172.21.15.92:6789/0 conn(0x7f0aac159c60 legacy=0x7f0aac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2172962858 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac15e140 con 0x7f0aac069560 2023-10-06T03:09:18.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aa959a700 1 -- 172.21.15.92:0/2172962858 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985502976 0 0) 0x7f0aac069ee0 con 0x7f0aac159c60 2023-10-06T03:09:18.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aaad9d700 1 --2- 172.21.15.92:0/2172962858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac069560 0x7f0aac0699a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e60 tx=0x7f0aa4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912076298fdbc4e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.304+0000 7f0aa959a700 1 -- 172.21.15.92:0/2172962858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400e070 con 0x7f0aac069560 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0aa959a700 1 -- 172.21.15.92:0/2172962858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa400bbf0 con 0x7f0aac069560 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0aa959a700 1 -- 172.21.15.92:0/2172962858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4012430 con 0x7f0aac069560 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2172962858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac069560 msgr2=0x7f0aac0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2172962858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac069560 0x7f0aac0699a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e60 tx=0x7f0aa4009780 comp rx=0 tx=0).stop 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2172962858 shutdown_connections 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2172962858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac069560 0x7f0aac0699a0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2172962858 >> 172.21.15.92:0/2172962858 conn(0x7f0aac068e20 msgr2=0x7f0aac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:18.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2172962858 shutdown_connections 2023-10-06T03:09:18.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.305+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2172962858 wait complete. 2023-10-06T03:09:18.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.306+0000 7f0ab0f1a700 1 Processor -- start 2023-10-06T03:09:18.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.306+0000 7f0ab0f1a700 1 -- start start 2023-10-06T03:09:18.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0ab0f1a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aac069560 0x7f0aac14d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0ab0f1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aac159890 0x7f0aac153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0ab0f1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 0x7f0aac153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0ab0f1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac15c4e0 con 0x7f0aac159c60 2023-10-06T03:09:18.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0ab0f1a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0ce170 con 0x7f0aac069560 2023-10-06T03:09:18.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0ab0f1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0ce450 con 0x7f0aac159890 2023-10-06T03:09:18.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 0x7f0aac153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 0x7f0aac153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44998/0 (socket says 172.21.15.92:44998) 2023-10-06T03:09:18.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2416502557 learned_addr learned my addr 172.21.15.92:0/2416502557 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:18.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aac159890 msgr2=0x7f0aac153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aa9d9b700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aac159890 0x7f0aac153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaa59c700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aac069560 0x7f0aac14d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aac159890 0x7f0aac153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aac069560 msgr2=0x7f0aac14d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aac069560 0x7f0aac14d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.307+0000 7f0aaad9d700 1 -- 172.21.15.92:0/2416502557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4009a20 con 0x7f0aac159c60 2023-10-06T03:09:18.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.308+0000 7f0aaad9d700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 0x7f0aac153d70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b6c0 tx=0x7f0aa4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.308+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa401d640 con 0x7f0aac159c60 2023-10-06T03:09:18.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.308+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa401d7c0 con 0x7f0aac159c60 2023-10-06T03:09:18.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.308+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac0ce730 con 0x7f0aac159c60 2023-10-06T03:09:18.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.308+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa401db80 con 0x7f0aac159c60 2023-10-06T03:09:18.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.308+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac0cd5c0 con 0x7f0aac159c60 2023-10-06T03:09:18.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.309+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac15c650 con 0x7f0aac159c60 2023-10-06T03:09:18.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.309+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0aa4024070 con 0x7f0aac159c60 2023-10-06T03:09:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.309+0000 7f0a9a7fc700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a8c067320 0x7f0a8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.309+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f0aa40877f0 con 0x7f0aac159c60 2023-10-06T03:09:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.309+0000 7f0aaa59c700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a8c067320 0x7f0a8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.310+0000 7f0aaa59c700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a8c067320 0x7f0a8c0697d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0027e0 tx=0x7f0a9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.314+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aa4054480 con 0x7f0aac159c60 2023-10-06T03:09:18.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.496+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0aac04c3c0 con 0x7f0aac159c60 2023-10-06T03:09:18.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.497+0000 7f0a9a7fc700 1 -- 172.21.15.92:0/2416502557 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f0aa4057aa0 con 0x7f0aac159c60 2023-10-06T03:09:18.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:18.499 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":143,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:16.658258+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs.volume_0000000000645996.meta","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000645996"}}},{"pool":26,"pool_name":"cephfs.volume_0000000000645996.data","create_time":"2023-10-06T03:09:01.925339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000645996"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:18.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a8c067320 msgr2=0x7f0a8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a8c067320 0x7f0a8c0697d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0027e0 tx=0x7f0a9c009040 comp rx=0 tx=0).stop 2023-10-06T03:09:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 msgr2=0x7f0aac153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 0x7f0aac153d70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400b6c0 tx=0x7f0aa4018000 comp rx=0 tx=0).stop 2023-10-06T03:09:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 shutdown_connections 2023-10-06T03:09:18.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a8c067320 0x7f0a8c0697d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0aac159c60 0x7f0aac153d70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0aac069560 0x7f0aac14d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 --2- 172.21.15.92:0/2416502557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0aac159890 0x7f0aac153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 >> 172.21.15.92:0/2416502557 conn(0x7f0aac068e20 msgr2=0x7f0aac157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 shutdown_connections 2023-10-06T03:09:18.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.499+0000 7f0ab0f1a700 1 -- 172.21.15.92:0/2416502557 wait complete. 2023-10-06T03:09:18.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs.volume_0000000000645996.meta', 'create_time': '2023-10-06T03:09:00.975583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000645996'}}}, {'pool': 26, 'pool_name': 'cephfs.volume_0000000000645996.data', 'create_time': '2023-10-06T03:09:01.925339+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000645996'}}}] 2023-10-06T03:09:18.514 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:09:18.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.632+0000 7ff0e642b700 1 Processor -- start 2023-10-06T03:09:18.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 -- start start 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff0e0159c60 0x7ff0e015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff0e015db10 0x7ff0e0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e015a160 con 0x7ff0e0159c60 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e00d13a0 con 0x7ff0e015db10 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0e642b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e015e140 con 0x7ff0e0159890 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.633+0000 7ff0df7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff0e0159c60 0x7ff0e015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff0e0159c60 0x7ff0e015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45010/0 (socket says 172.21.15.92:45010) 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 -- 172.21.15.92:0/764010623 learned_addr learned my addr 172.21.15.92:0/764010623 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:18.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0e49c8700 1 --2- 172.21.15.92:0/764010623 >> v2:172.21.15.136:3300/0 conn(0x7ff0e015db10 0x7ff0e0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0dffff700 1 --2- 172.21.15.92:0/764010623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 -- 172.21.15.92:0/764010623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 msgr2=0x7ff0e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 --2- 172.21.15.92:0/764010623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 -- 172.21.15.92:0/764010623 >> v2:172.21.15.136:3300/0 conn(0x7ff0e015db10 msgr2=0x7ff0e0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 --2- 172.21.15.92:0/764010623 >> v2:172.21.15.136:3300/0 conn(0x7ff0e015db10 0x7ff0e0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 -- 172.21.15.92:0/764010623 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e015a3d0 con 0x7ff0e0159c60 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0df7fe700 1 --2- 172.21.15.92:0/764010623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e015d5d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0089b50 tx=0x7ff0d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c391ab4412c2238 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0deffd700 1 -- 172.21.15.92:0/764010623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400b640 con 0x7ff0e0159c60 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0deffd700 1 -- 172.21.15.92:0/764010623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400b7a0 con 0x7ff0e0159c60 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.634+0000 7ff0deffd700 1 -- 172.21.15.92:0/764010623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400bb60 con 0x7ff0e0159c60 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 -- 172.21.15.92:0/764010623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 msgr2=0x7ff0e015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 --2- 172.21.15.92:0/764010623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e015d5d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0089b50 tx=0x7ff0d4009250 comp rx=0 tx=0).stop 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 -- 172.21.15.92:0/764010623 shutdown_connections 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 --2- 172.21.15.92:0/764010623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 --2- 172.21.15.92:0/764010623 >> v2:172.21.15.136:3300/0 conn(0x7ff0e015db10 0x7ff0e0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 --2- 172.21.15.92:0/764010623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e015d5d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 -- 172.21.15.92:0/764010623 >> 172.21.15.92:0/764010623 conn(0x7ff0e0068e20 msgr2=0x7ff0e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 -- 172.21.15.92:0/764010623 shutdown_connections 2023-10-06T03:09:18.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.635+0000 7ff0e642b700 1 -- 172.21.15.92:0/764010623 wait complete. 2023-10-06T03:09:18.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.636+0000 7ff0e642b700 1 Processor -- start 2023-10-06T03:09:18.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.636+0000 7ff0e642b700 1 -- start start 2023-10-06T03:09:18.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e642b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e00b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e642b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e00ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e642b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0e015db10 0x7ff0e00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e642b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00cac70 con 0x7ff0e0159c60 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e642b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00b5a00 con 0x7ff0e015db10 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e642b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00b63d0 con 0x7ff0e0159890 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e00ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e00ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45020/0 (socket says 172.21.15.92:45020) 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 -- 172.21.15.92:0/2950360261 learned_addr learned my addr 172.21.15.92:0/2950360261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0dffff700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e00b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 -- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 msgr2=0x7ff0e00b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e00b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 -- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0e015db10 msgr2=0x7ff0e00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0e49c8700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0e015db10 0x7ff0e00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0e015db10 0x7ff0e00b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 -- 172.21.15.92:0/2950360261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40094f0 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.637+0000 7ff0df7fe700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e00ca730 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0070e20 tx=0x7ff0d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.638+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4000d10 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.638+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400bb20 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.638+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00b4df0 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.638+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4011670 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.638+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e00b53b0 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.639+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e00b5b90 con 0x7ff0e0159c60 2023-10-06T03:09:18.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.639+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff0d40119e0 con 0x7ff0e0159c60 2023-10-06T03:09:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.639+0000 7ff0c7fff700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff0cc067320 0x7ff0cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.639+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7ff0d40866d0 con 0x7ff0e0159c60 2023-10-06T03:09:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.639+0000 7ff0dffff700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff0cc067320 0x7ff0cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.640+0000 7ff0dffff700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff0cc067320 0x7ff0cc0697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00093f0 tx=0x7ff0d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.644+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0d40531b0 con 0x7ff0e0159c60 2023-10-06T03:09:18.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.828+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0e004c3c0 con 0x7ff0e0159c60 2023-10-06T03:09:18.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.828+0000 7ff0c7fff700 1 -- 172.21.15.92:0/2950360261 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0d40567d0 con 0x7ff0e0159c60 2023-10-06T03:09:18.829 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff0cc067320 msgr2=0x7ff0cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff0cc067320 0x7ff0cc0697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00093f0 tx=0x7ff0d0008040 comp rx=0 tx=0).stop 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 msgr2=0x7ff0e00ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e00ca730 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0070e20 tx=0x7ff0d4017000 comp rx=0 tx=0).stop 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 shutdown_connections 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff0cc067320 0x7ff0cc0697d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0e015db10 0x7ff0e00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0e0159890 0x7ff0e00b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 --2- 172.21.15.92:0/2950360261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff0e0159c60 0x7ff0e00ca730 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 >> 172.21.15.92:0/2950360261 conn(0x7ff0e0068e20 msgr2=0x7ff0e0152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 shutdown_connections 2023-10-06T03:09:18.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.830+0000 7ff0e642b700 1 -- 172.21.15.92:0/2950360261 wait complete. 2023-10-06T03:09:18.843 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:09:18.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.959+0000 7f6998241700 1 Processor -- start 2023-10-06T03:09:18.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.959+0000 7f6998241700 1 -- start start 2023-10-06T03:09:18.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f6998241700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6990159c60 0x7f699015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f6998241700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699015a160 con 0x7f699015db10 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f6998241700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69900d13a0 con 0x7f6990159890 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f6998241700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699015e140 con 0x7f6990159c60 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f69967de700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f699015db10 0x7f6990069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40906/0 (socket says 172.21.15.92:40906) 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f69967de700 1 -- 172.21.15.92:0/3601310967 learned_addr learned my addr 172.21.15.92:0/3601310967 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f69957dc700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6990159c60 0x7f699015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905983371 0 0) 0x7f699015a160 con 0x7f699015db10 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.960+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f697c003570 con 0x7f699015db10 2023-10-06T03:09:18.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128844471 0 0) 0x7f69900d13a0 con 0x7f6990159890 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699015a160 con 0x7f6990159890 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352037906 0 0) 0x7f697c003570 con 0x7f699015db10 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6990159c60 msgr2=0x7f699015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6990159c60 0x7f699015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 >> v1:172.21.15.136:6789/0 conn(0x7f6990159890 legacy=0x7f699015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699015a3d0 con 0x7f699015db10 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f6988001470 con 0x7f699015db10 2023-10-06T03:09:18.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 >> v1:172.21.15.92:6789/0 conn(0x7f699015db10 legacy=0x7f6990069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c003c20 0x7f697c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f697c004810 0x7f697c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f697c0051c0 0x7f697c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699015e140 con 0x7f697c004810 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c00c020 con 0x7f697c003c20 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c00c1e0 con 0x7f697c0051c0 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6990159c60 0x7f699015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f697c004810 0x7f697c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f6995fdd700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c003c20 0x7f697c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69967de700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f697c0051c0 0x7f697c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 -- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f697c0051c0 msgr2=0x7f697c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f697c0051c0 0x7f697c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 -- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c003c20 msgr2=0x7f697c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c003c20 0x7f697c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 -- 172.21.15.92:0/3601310967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c003570 con 0x7f697c004810 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.961+0000 7f69957dc700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f697c004810 0x7f697c004c80 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f69900892f0 tx=0x7f69800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31870b10ebb39c04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000e6e0 con 0x7f697c004810 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000e840 con 0x7f697c004810 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6994fdb700 1 -- 172.21.15.92:0/3601310967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000ebe0 con 0x7f697c004810 2023-10-06T03:09:18.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6998241700 1 -- 172.21.15.92:0/3601310967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f697c004810 msgr2=0x7f697c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6998241700 1 --2- 172.21.15.92:0/3601310967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f697c004810 0x7f697c004c80 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f69900892f0 tx=0x7f69800091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f69967de700 1 -- 172.21.15.92:0/3601310967 reap_dead start 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6998241700 1 -- 172.21.15.92:0/3601310967 shutdown_connections 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6998241700 1 -- 172.21.15.92:0/3601310967 >> 172.21.15.92:0/3601310967 conn(0x7f6990068e20 msgr2=0x7f69900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6998241700 1 -- 172.21.15.92:0/3601310967 shutdown_connections 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.962+0000 7f6998241700 1 -- 172.21.15.92:0/3601310967 wait complete. 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.963+0000 7f6998241700 1 Processor -- start 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.963+0000 7f6998241700 1 -- start start 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6998241700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c004810 0x7f699014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6998241700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69900b5ca0 0x7f69900b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6998241700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 0x7f69900b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6998241700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699016abf0 con 0x7f69900b6110 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6998241700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69900b4510 con 0x7f697c004810 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6998241700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69900b3900 con 0x7f69900b5ca0 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 0x7f69900b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 0x7f69900b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45032/0 (socket says 172.21.15.92:45032) 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 -- 172.21.15.92:0/1121244999 learned_addr learned my addr 172.21.15.92:0/1121244999 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69957dc700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69900b5ca0 0x7f69900b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6995fdd700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c004810 0x7f699014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 -- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69900b5ca0 msgr2=0x7f69900b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69900b5ca0 0x7f69900b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 -- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c004810 msgr2=0x7f699014c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c004810 0x7f699014c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69967de700 1 -- 172.21.15.92:0/1121244999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69900b3be0 con 0x7f69900b6110 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f6995fdd700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c004810 0x7f699014c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.964+0000 7f69957dc700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69900b5ca0 0x7f69900b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.965+0000 7f69967de700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 0x7f69900b1520 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f699015dee0 tx=0x7f6988001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.965+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988018030 con 0x7f69900b6110 2023-10-06T03:09:18.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.965+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988011590 con 0x7f69900b6110 2023-10-06T03:09:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.965+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988011950 con 0x7f69900b6110 2023-10-06T03:09:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.965+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69900b3ec0 con 0x7f69900b6110 2023-10-06T03:09:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.965+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69900b3590 con 0x7f69900b6110 2023-10-06T03:09:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.966+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69900b46a0 con 0x7f69900b6110 2023-10-06T03:09:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.966+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f698801b070 con 0x7f69900b6110 2023-10-06T03:09:18.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.967+0000 7f6985ffb700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6974067320 0x7f69740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.967+0000 7f6995fdd700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6974067320 0x7f69740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.967+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f6988019070 con 0x7f69900b6110 2023-10-06T03:09:18.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.967+0000 7f6995fdd700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6974067320 0x7f69740697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f699014cbd0 tx=0x7f698c0017a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:18.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:18.971+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69880514f0 con 0x7f69900b6110 2023-10-06T03:09:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.153+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f699004c3c0 con 0x7f69900b6110 2023-10-06T03:09:19.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.153+0000 7f6985ffb700 1 -- 172.21.15.92:0/1121244999 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v143) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6988054b10 con 0x7f69900b6110 2023-10-06T03:09:19.154 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:09:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6974067320 msgr2=0x7f69740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6974067320 0x7f69740697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f699014cbd0 tx=0x7f698c0017a0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 msgr2=0x7f69900b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 0x7f69900b1520 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f699015dee0 tx=0x7f6988001df0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 shutdown_connections 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6974067320 0x7f69740697d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69900b6110 0x7f69900b1520 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f697c004810 0x7f699014c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 --2- 172.21.15.92:0/1121244999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69900b5ca0 0x7f69900b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 >> 172.21.15.92:0/1121244999 conn(0x7f6990068e20 msgr2=0x7f699014d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 shutdown_connections 2023-10-06T03:09:19.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.155+0000 7f6998241700 1 -- 172.21.15.92:0/1121244999 wait complete. 2023-10-06T03:09:19.168 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000645996.meta pg_num 2023-10-06T03:09:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.285+0000 7f6768789700 1 Processor -- start 2023-10-06T03:09:19.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.285+0000 7f6768789700 1 -- start start 2023-10-06T03:09:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.286+0000 7f6768789700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 0x7f6760069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.286+0000 7f6768789700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6760068aa0 0x7f67600c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.286+0000 7f6768789700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67600c7de0 con 0x7f676006a1c0 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.286+0000 7f6768789700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6760140c10 con 0x7f6760068aa0 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.286+0000 7f6768789700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6760141490 con 0x7f6760140350 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.286+0000 7f6765d24700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f676006a1c0 0x7f6760068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:40922/0 (socket says 172.21.15.92:40922) 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 -- 172.21.15.92:0/463406876 learned_addr learned my addr 172.21.15.92:0/463406876 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6766d26700 1 --2- 172.21.15.92:0/463406876 >> v2:172.21.15.136:3300/0 conn(0x7f6760068aa0 0x7f67600c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6766525700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 0x7f6760069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175713988 0 0) 0x7f67600c7de0 con 0x7f676006a1c0 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f674c003570 con 0x7f676006a1c0 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3432896548 0 0) 0x7f674c003570 con 0x7f676006a1c0 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 msgr2=0x7f6760069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 0x7f6760069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 >> v2:172.21.15.136:3300/0 conn(0x7f6760068aa0 msgr2=0x7f67600c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 --2- 172.21.15.92:0/463406876 >> v2:172.21.15.136:3300/0 conn(0x7f6760068aa0 0x7f67600c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760140ed0 con 0x7f676006a1c0 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f6750001470 con 0x7f676006a1c0 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 >> v1:172.21.15.92:6789/0 conn(0x7f676006a1c0 legacy=0x7f6760068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 0x7f674c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f674c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f674c004d50 0x7f674c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6760140c10 con 0x7f674c003fe0 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6760141490 con 0x7f674c004d50 2023-10-06T03:09:19.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c00ba90 con 0x7f6760140350 2023-10-06T03:09:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f674c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6766525700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 0x7f674c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6766d26700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f674c004d50 0x7f674c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 -- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 msgr2=0x7f674c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6760140350 0x7f674c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 -- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f674c004d50 msgr2=0x7f674c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f674c004d50 0x7f674c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.287+0000 7f6765d24700 1 -- 172.21.15.92:0/463406876 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c003570 con 0x7f674c003fe0 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6765d24700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f674c004810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f675000c3c0 tx=0x7f67600c7d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbb5b0203be1884 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000cc30 con 0x7f674c003fe0 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675000cd90 con 0x7f674c003fe0 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6765523700 1 -- 172.21.15.92:0/463406876 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750005b70 con 0x7f674c003fe0 2023-10-06T03:09:19.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6768789700 1 -- 172.21.15.92:0/463406876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 msgr2=0x7f674c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6768789700 1 --2- 172.21.15.92:0/463406876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f674c004810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f675000c3c0 tx=0x7f67600c7d50 comp rx=0 tx=0).stop 2023-10-06T03:09:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6766d26700 1 -- 172.21.15.92:0/463406876 reap_dead start 2023-10-06T03:09:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6768789700 1 -- 172.21.15.92:0/463406876 shutdown_connections 2023-10-06T03:09:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6768789700 1 -- 172.21.15.92:0/463406876 >> 172.21.15.92:0/463406876 conn(0x7f67600cd670 msgr2=0x7f67600cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6768789700 1 -- 172.21.15.92:0/463406876 shutdown_connections 2023-10-06T03:09:19.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.288+0000 7f6768789700 1 -- 172.21.15.92:0/463406876 wait complete. 2023-10-06T03:09:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.289+0000 7f6768789700 1 Processor -- start 2023-10-06T03:09:19.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.289+0000 7f6768789700 1 -- start start 2023-10-06T03:09:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6768789700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f67600c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6768789700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67600cc5b0 0x7f676015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6768789700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67600c51a0 0x7f67600c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6768789700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600c5b60 con 0x7f674c003fe0 2023-10-06T03:09:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6768789700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600b2ba0 con 0x7f67600cc5b0 2023-10-06T03:09:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6768789700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600c4f10 con 0x7f67600c51a0 2023-10-06T03:09:19.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6766525700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f67600c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6766525700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f67600c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45062/0 (socket says 172.21.15.92:45062) 2023-10-06T03:09:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6766525700 1 -- 172.21.15.92:0/2821239212 learned_addr learned my addr 172.21.15.92:0/2821239212 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6766d26700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67600c51a0 0x7f67600c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.290+0000 7f6765d24700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67600cc5b0 0x7f676015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6766525700 1 -- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67600c51a0 msgr2=0x7f67600c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6766525700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67600c51a0 0x7f67600c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6766525700 1 -- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67600cc5b0 msgr2=0x7f676015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6766525700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67600cc5b0 0x7f676015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6766525700 1 -- 172.21.15.92:0/2821239212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600b1f90 con 0x7f674c003fe0 2023-10-06T03:09:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6765d24700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67600cc5b0 0x7f676015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6766525700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f67600c49d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f675c011ae0 tx=0x7f675c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c005c10 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c005d70 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67600b2270 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c00f980 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.291+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600b2830 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.292+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676006e3b0 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.292+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f675c00fd70 con 0x7f674c003fe0 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.292+0000 7f67567fc700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067350 0x7f6744069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.293+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f675c085460 con 0x7f674c003fe0 2023-10-06T03:09:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.293+0000 7f6765d24700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067350 0x7f6744069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.293+0000 7f6765d24700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067350 0x7f6744069800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f67600b3100 tx=0x7f6750015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.297+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f675c051f40 con 0x7f674c003fe0 2023-10-06T03:09:19.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.477+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000645996.meta", "var": "pg_num"} v 0) v1 -- 0x7f676004c3c0 con 0x7f674c003fe0 2023-10-06T03:09:19.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.477+0000 7f67567fc700 1 -- 172.21.15.92:0/2821239212 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000645996.meta", "var": "pg_num"}]=0 v143) v1 ==== 124+0+11 (secure 0 0 0) 0x7f675c055560 con 0x7f674c003fe0 2023-10-06T03:09:19.478 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:09:19.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067350 msgr2=0x7f6744069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067350 0x7f6744069800 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f67600b3100 tx=0x7f6750015000 comp rx=0 tx=0).stop 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 msgr2=0x7f67600c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f67600c49d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f675c011ae0 tx=0x7f675c009120 comp rx=0 tx=0).stop 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 shutdown_connections 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067350 0x7f6744069800 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f674c003fe0 0x7f67600c49d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67600c51a0 0x7f67600c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 --2- 172.21.15.92:0/2821239212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67600cc5b0 0x7f676015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.479+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 >> 172.21.15.92:0/2821239212 conn(0x7f67600cd670 msgr2=0x7f67600ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.480+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 shutdown_connections 2023-10-06T03:09:19.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.480+0000 7f6768789700 1 -- 172.21.15.92:0/2821239212 wait complete. 2023-10-06T03:09:19.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000645996.data pg_num 2023-10-06T03:09:19.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.610+0000 7f997f33b700 1 Processor -- start 2023-10-06T03:09:19.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.610+0000 7f997f33b700 1 -- start start 2023-10-06T03:09:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997f33b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f99780b16d0 0x7f99780b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997f33b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b33b0 0x7f99780b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997f33b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99780b2080 con 0x7f99780b4e70 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997f33b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99780cfb00 con 0x7f99780b16d0 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997f33b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780b5c40 con 0x7f99780b33b0 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997eb3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b33b0 0x7f99780b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997db38700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f99780b16d0 0x7f99780b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997db38700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f99780b16d0 0x7f99780b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:55022/0 (socket says 172.21.15.92:55022) 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997db38700 1 -- 172.21.15.92:0/1686845961 learned_addr learned my addr 172.21.15.92:0/1686845961 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997e339700 1 --1- 172.21.15.92:0/1686845961 >> v1:172.21.15.136:6789/0 conn(0x7f99780b4e70 0x7f99780ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:35788/0 (socket says 172.21.15.92:35788) 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997eb3a700 1 -- 172.21.15.92:0/1686845961 >> v2:172.21.15.136:3300/0 conn(0x7f99780b16d0 msgr2=0x7f99780b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997eb3a700 1 --2- 172.21.15.92:0/1686845961 >> v2:172.21.15.136:3300/0 conn(0x7f99780b16d0 0x7f99780b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997eb3a700 1 -- 172.21.15.92:0/1686845961 >> v1:172.21.15.136:6789/0 conn(0x7f99780b4e70 legacy=0x7f99780ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.611+0000 7f997eb3a700 1 -- 172.21.15.92:0/1686845961 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780b5f20 con 0x7f99780b33b0 2023-10-06T03:09:19.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997eb3a700 1 --2- 172.21.15.92:0/1686845961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b33b0 0x7f99780b3810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9970009e60 tx=0x7f9970009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ff8a6661b0daae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997d337700 1 -- 172.21.15.92:0/1686845961 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000e070 con 0x7f99780b33b0 2023-10-06T03:09:19.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997d337700 1 -- 172.21.15.92:0/1686845961 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000bb60 con 0x7f99780b33b0 2023-10-06T03:09:19.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997d337700 1 -- 172.21.15.92:0/1686845961 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970012430 con 0x7f99780b33b0 2023-10-06T03:09:19.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 -- 172.21.15.92:0/1686845961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b33b0 msgr2=0x7f99780b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 --2- 172.21.15.92:0/1686845961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b33b0 0x7f99780b3810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9970009e60 tx=0x7f9970009780 comp rx=0 tx=0).stop 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 -- 172.21.15.92:0/1686845961 shutdown_connections 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 --2- 172.21.15.92:0/1686845961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b33b0 0x7f99780b3810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 --2- 172.21.15.92:0/1686845961 >> v2:172.21.15.136:3300/0 conn(0x7f99780b16d0 0x7f99780b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 -- 172.21.15.92:0/1686845961 >> 172.21.15.92:0/1686845961 conn(0x7f99780d10c0 msgr2=0x7f99780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 -- 172.21.15.92:0/1686845961 shutdown_connections 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.612+0000 7f997f33b700 1 -- 172.21.15.92:0/1686845961 wait complete. 2023-10-06T03:09:19.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 Processor -- start 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 -- start start 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 0x7f99780c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99780b33b0 0x7f99780b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b4e70 0x7f9978152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c8130 con 0x7f99780b16d0 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c75b0 con 0x7f99780b33b0 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997f33b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780c7890 con 0x7f99780b4e70 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997e339700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 0x7f99780c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997e339700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 0x7f99780c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50328/0 (socket says 172.21.15.92:50328) 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997e339700 1 -- 172.21.15.92:0/737842769 learned_addr learned my addr 172.21.15.92:0/737842769 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997eb3a700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b4e70 0x7f9978152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.614+0000 7f997db38700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99780b33b0 0x7f99780b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997e339700 1 -- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b4e70 msgr2=0x7f9978152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997e339700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b4e70 0x7f9978152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997e339700 1 -- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99780b33b0 msgr2=0x7f99780b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997e339700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99780b33b0 0x7f99780b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997e339700 1 -- 172.21.15.92:0/737842769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970009a20 con 0x7f99780b16d0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997e339700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 0x7f99780c6cf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f997400c7c0 tx=0x7f9974007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974005a70 con 0x7f99780b16d0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974005bd0 con 0x7f99780b16d0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99780c7b70 con 0x7f99780b16d0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740070d0 con 0x7f99780b16d0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.615+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99780c7230 con 0x7f99780b16d0 2023-10-06T03:09:19.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.616+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9978088990 con 0x7f99780b16d0 2023-10-06T03:09:19.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.616+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9974007480 con 0x7f99780b16d0 2023-10-06T03:09:19.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.617+0000 7f996e7fc700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f995c067320 0x7f995c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.617+0000 7f997db38700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f995c067320 0x7f995c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.617+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f9974084230 con 0x7f99780b16d0 2023-10-06T03:09:19.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.617+0000 7f997db38700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f995c067320 0x7f995c0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f99781597f0 tx=0x7f9968009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.621+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9974050d10 con 0x7f99780b16d0 2023-10-06T03:09:19.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.806+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000645996.data", "var": "pg_num"} v 0) v1 -- 0x7f997804bb80 con 0x7f99780b16d0 2023-10-06T03:09:19.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.806+0000 7f996e7fc700 1 -- 172.21.15.92:0/737842769 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000645996.data", "var": "pg_num"}]=0 v143) v1 ==== 124+0+11 (secure 0 0 0) 0x7f9974017020 con 0x7f99780b16d0 2023-10-06T03:09:19.807 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:09:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f995c067320 msgr2=0x7f995c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f995c067320 0x7f995c0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f99781597f0 tx=0x7f9968009250 comp rx=0 tx=0).stop 2023-10-06T03:09:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 msgr2=0x7f99780c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 0x7f99780c6cf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f997400c7c0 tx=0x7f9974007f70 comp rx=0 tx=0).stop 2023-10-06T03:09:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 shutdown_connections 2023-10-06T03:09:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f995c067320 0x7f995c0697d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99780b16d0 0x7f99780c6cf0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99780b4e70 0x7f9978152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 --2- 172.21.15.92:0/737842769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99780b33b0 0x7f99780b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.808+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 >> 172.21.15.92:0/737842769 conn(0x7f99780d10c0 msgr2=0x7f99780b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:19.812 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.809+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 shutdown_connections 2023-10-06T03:09:19.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.809+0000 7f997f33b700 1 -- 172.21.15.92:0/737842769 wait complete. 2023-10-06T03:09:19.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:19.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.937+0000 7f77455a2700 1 Processor -- start 2023-10-06T03:09:19.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.937+0000 7f77455a2700 1 -- start start 2023-10-06T03:09:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f77455a2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7740159890 0x7f774015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f77455a2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159c60 0x7f774015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f77455a2700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774015a160 con 0x7f774015db10 2023-10-06T03:09:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f77455a2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77400d13a0 con 0x7f7740159890 2023-10-06T03:09:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f77455a2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774015e140 con 0x7f7740159c60 2023-10-06T03:09:19.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7740159890 0x7f774015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7740159890 0x7f774015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50344/0 (socket says 172.21.15.92:50344) 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 -- 172.21.15.92:0/3311182984 learned_addr learned my addr 172.21.15.92:0/3311182984 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f7737fff700 1 --2- 172.21.15.92:0/3311182984 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159c60 0x7f774015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 -- 172.21.15.92:0/3311182984 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159c60 msgr2=0x7f774015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 --2- 172.21.15.92:0/3311182984 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159c60 0x7f774015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 -- 172.21.15.92:0/3311182984 >> v1:172.21.15.136:6789/0 conn(0x7f774015db10 legacy=0x7f7740069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 -- 172.21.15.92:0/3311182984 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774015a3d0 con 0x7f7740159890 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.938+0000 7f773effd700 1 --2- 172.21.15.92:0/3311182984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159890 0x7f774015abd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f77300099b0 tx=0x7f7730009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2935171a1678f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f773e7fc700 1 -- 172.21.15.92:0/3311182984 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730009ea0 con 0x7f7740159890 2023-10-06T03:09:19.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f773e7fc700 1 -- 172.21.15.92:0/3311182984 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000b860 con 0x7f7740159890 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f773e7fc700 1 -- 172.21.15.92:0/3311182984 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000bc00 con 0x7f7740159890 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 -- 172.21.15.92:0/3311182984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159890 msgr2=0x7f774015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 --2- 172.21.15.92:0/3311182984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159890 0x7f774015abd0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f77300099b0 tx=0x7f7730009250 comp rx=0 tx=0).stop 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 -- 172.21.15.92:0/3311182984 shutdown_connections 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 --2- 172.21.15.92:0/3311182984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159890 0x7f774015abd0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 --2- 172.21.15.92:0/3311182984 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159c60 0x7f774015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 -- 172.21.15.92:0/3311182984 >> 172.21.15.92:0/3311182984 conn(0x7f7740068e20 msgr2=0x7f77400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 -- 172.21.15.92:0/3311182984 shutdown_connections 2023-10-06T03:09:19.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.939+0000 7f77455a2700 1 -- 172.21.15.92:0/3311182984 wait complete. 2023-10-06T03:09:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.940+0000 7f77455a2700 1 Processor -- start 2023-10-06T03:09:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.940+0000 7f77455a2700 1 -- start start 2023-10-06T03:09:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f77455a2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159890 0x7f774016e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f77455a2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 0x7f77400b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f77455a2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f774015db10 0x7f77400b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f77455a2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400b3590 con 0x7f7740159c60 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f77455a2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400c79b0 con 0x7f774015db10 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f77455a2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77400b43b0 con 0x7f7740159890 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 0x7f77400b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 0x7f77400b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50356/0 (socket says 172.21.15.92:50356) 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 -- 172.21.15.92:0/3016715646 learned_addr learned my addr 172.21.15.92:0/3016715646 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f773effd700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159890 0x7f774016e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f773f7fe700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f774015db10 0x7f77400b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 -- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159890 msgr2=0x7f774016e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159890 0x7f774016e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 -- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f774015db10 msgr2=0x7f77400b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f774015db10 0x7f77400b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 -- 172.21.15.92:0/3016715646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300094f0 con 0x7f7740159c60 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.941+0000 7f7737fff700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 0x7f77400b4840 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f772800ec50 tx=0x7f77280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.942+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800c8e0 con 0x7f7740159c60 2023-10-06T03:09:19.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.942+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772800ca40 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.942+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774016e820 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.942+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800ce00 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.942+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77400b3a80 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.943+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77401590e0 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.943+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7728013680 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.943+0000 7f7736ffd700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7720067320 0x7f77200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.943+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f77280850b0 con 0x7f7740159c60 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.943+0000 7f773effd700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7720067320 0x7f77200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:19.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.944+0000 7f773effd700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7720067320 0x7f77200697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f77300099b0 tx=0x7f7730017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:19.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:19.947+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7728051b90 con 0x7f7740159c60 2023-10-06T03:09:20.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.170+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f774004c3c0 con 0x7f7740159c60 2023-10-06T03:09:20.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.171+0000 7f7736ffd700 1 -- 172.21.15.92:0/3016715646 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f77280551b0 con 0x7f7740159c60 2023-10-06T03:09:20.171 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:20.172 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:14.655190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000645996","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.172+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7720067320 msgr2=0x7f77200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.172+0000 7f77455a2700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7720067320 0x7f77200697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f77300099b0 tx=0x7f7730017000 comp rx=0 tx=0).stop 2023-10-06T03:09:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.172+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 msgr2=0x7f77400b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.172+0000 7f77455a2700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 0x7f77400b4840 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f772800ec50 tx=0x7f77280091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 shutdown_connections 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7720067320 0x7f77200697d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f774015db10 0x7f77400b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7740159890 0x7f774016e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 --2- 172.21.15.92:0/3016715646 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7740159c60 0x7f77400b4840 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 >> 172.21.15.92:0/3016715646 conn(0x7f7740068e20 msgr2=0x7f774015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 shutdown_connections 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.173+0000 7f77455a2700 1 -- 172.21.15.92:0/3016715646 wait complete. 2023-10-06T03:09:20.174 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 96 2023-10-06T03:09:20.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:20.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.300+0000 7f960e379700 1 Processor -- start 2023-10-06T03:09:20.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.300+0000 7f960e379700 1 -- start start 2023-10-06T03:09:20.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f960e379700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9608159c60 0x7f960815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f960e379700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960815a160 con 0x7f960815db10 2023-10-06T03:09:20.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f960e379700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080d13a0 con 0x7f9608159890 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f960e379700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960815df50 con 0x7f9608159c60 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f96077fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9608159c60 0x7f960815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f960c916700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f960815db10 0x7f9608069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52636/0 (socket says 172.21.15.92:52636) 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.301+0000 7f960c916700 1 -- 172.21.15.92:0/3559277910 learned_addr learned my addr 172.21.15.92:0/3559277910 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f96077fe700 1 -- 172.21.15.92:0/3559277910 >> v1:172.21.15.136:6789/0 conn(0x7f9608159890 legacy=0x7f960815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f96077fe700 1 -- 172.21.15.92:0/3559277910 >> v1:172.21.15.92:6789/0 conn(0x7f960815db10 legacy=0x7f9608069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f96077fe700 1 -- 172.21.15.92:0/3559277910 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960815e140 con 0x7f9608159c60 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f9606ffd700 1 -- 172.21.15.92:0/3559277910 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098023415 0 0) 0x7f960815a160 con 0x7f960815db10 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f96077fe700 1 --2- 172.21.15.92:0/3559277910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f960815d5d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0098e0 tx=0x7f95fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b67abe95903f8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f9606ffd700 1 -- 172.21.15.92:0/3559277910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc009e80 con 0x7f9608159c60 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f9606ffd700 1 -- 172.21.15.92:0/3559277910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc00b820 con 0x7f9608159c60 2023-10-06T03:09:20.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f9606ffd700 1 -- 172.21.15.92:0/3559277910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00bba0 con 0x7f9608159c60 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 -- 172.21.15.92:0/3559277910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 msgr2=0x7f960815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 --2- 172.21.15.92:0/3559277910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f960815d5d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0098e0 tx=0x7f95fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 -- 172.21.15.92:0/3559277910 shutdown_connections 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 --2- 172.21.15.92:0/3559277910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f960815d5d0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 -- 172.21.15.92:0/3559277910 >> 172.21.15.92:0/3559277910 conn(0x7f9608068e20 msgr2=0x7f96080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 -- 172.21.15.92:0/3559277910 shutdown_connections 2023-10-06T03:09:20.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.302+0000 7f960e379700 1 -- 172.21.15.92:0/3559277910 wait complete. 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.303+0000 7f960e379700 1 Processor -- start 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 -- start start 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9608159890 0x7f96081474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f96080cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f960815db10 0x7f96080b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960816db40 con 0x7f9608159c60 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960816cf70 con 0x7f9608159890 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.304+0000 7f960e379700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960816d210 con 0x7f960815db10 2023-10-06T03:09:20.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f96080cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f96080cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50374/0 (socket says 172.21.15.92:50374) 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 -- 172.21.15.92:0/3639035093 learned_addr learned my addr 172.21.15.92:0/3639035093 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 -- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f960815db10 msgr2=0x7f96080b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f960c916700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f960815db10 0x7f96080b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f9607fff700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9608159890 0x7f96081474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f960815db10 0x7f96080b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 -- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9608159890 msgr2=0x7f96081474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9608159890 0x7f96081474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 -- 172.21.15.92:0/3639035093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc009420 con 0x7f9608159c60 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f960c916700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f960815db10 0x7f96080b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:20.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f9607fff700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9608159890 0x7f96081474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f96077fe700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f96080cebf0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0030c0 tx=0x7f95fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00bae0 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc020070 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960816d4f0 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc01b6a0 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.305+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96080cf130 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.306+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960804c3c0 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.306+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f95fc022030 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.307+0000 7f95effff700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95f4067320 0x7f95f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.307+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f95fc086e00 con 0x7f9608159c60 2023-10-06T03:09:20.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.307+0000 7f9607fff700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95f4067320 0x7f95f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.307+0000 7f9607fff700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95f4067320 0x7f95f40697d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f95f8006fd0 tx=0x7f95f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:20.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.311+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95fc0539f0 con 0x7f9608159c60 2023-10-06T03:09:20.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.496+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f960804c3c0 con 0x7f9608159c60 2023-10-06T03:09:20.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.497+0000 7f95effff700 1 -- 172.21.15.92:0/3639035093 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f95fc057010 con 0x7f9608159c60 2023-10-06T03:09:20.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:20.498 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":143,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:16.658258+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs.volume_0000000000645996.meta","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000645996"}}},{"pool":26,"pool_name":"cephfs.volume_0000000000645996.data","create_time":"2023-10-06T03:09:01.925339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000645996"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95f4067320 msgr2=0x7f95f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95f4067320 0x7f95f40697d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f95f8006fd0 tx=0x7f95f8008040 comp rx=0 tx=0).stop 2023-10-06T03:09:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 msgr2=0x7f96080cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f96080cebf0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0030c0 tx=0x7f95fc016000 comp rx=0 tx=0).stop 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 shutdown_connections 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95f4067320 0x7f95f40697d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f960815db10 0x7f96080b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9608159890 0x7f96081474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 --2- 172.21.15.92:0/3639035093 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9608159c60 0x7f96080cebf0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 >> 172.21.15.92:0/3639035093 conn(0x7f9608068e20 msgr2=0x7f96081574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 shutdown_connections 2023-10-06T03:09:20.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.499+0000 7f960e379700 1 -- 172.21.15.92:0/3639035093 wait complete. 2023-10-06T03:09:20.512 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000645996 and related pools 2023-10-06T03:09:20.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.626+0000 7fcdad3eb700 1 Processor -- start 2023-10-06T03:09:20.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.626+0000 7fcdad3eb700 1 -- start start 2023-10-06T03:09:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcdad3eb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcdad3eb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcdad3eb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda815a160 con 0x7fcda815db10 2023-10-06T03:09:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcdad3eb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda80d13a0 con 0x7fcda8159c60 2023-10-06T03:09:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcdad3eb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda815e140 con 0x7fcda8159890 2023-10-06T03:09:20.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcda77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcda815db10 0x7fcda8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52638/0 (socket says 172.21.15.92:52638) 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.627+0000 7fcda77fe700 1 -- 172.21.15.92:0/3534470800 learned_addr learned my addr 172.21.15.92:0/3534470800 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/3534470800 >> v2:172.21.15.136:3300/0 conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda6ffd700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134343380 0 0) 0x7fcda815a160 con 0x7fcda815db10 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd94003570 con 0x7fcda815db10 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508727299 0 0) 0x7fcd94003570 con 0x7fcda815db10 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 msgr2=0x7fcda815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 >> v2:172.21.15.136:3300/0 conn(0x7fcda8159c60 msgr2=0x7fcda815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:20.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 --2- 172.21.15.92:0/3534470800 >> v2:172.21.15.136:3300/0 conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda815a3d0 con 0x7fcda815db10 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fcda00017f0 con 0x7fcda815db10 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 >> v1:172.21.15.92:6789/0 conn(0x7fcda815db10 legacy=0x7fcda8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd94003a60 0x7fcd94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcd94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 0x7fcd940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80d13a0 con 0x7fcd94004810 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda815e140 con 0x7fcd94003a60 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9400ba60 con 0x7fcda8159890 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcd94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda6ffd700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd94003a60 0x7fcd94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcda77fe700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 0x7fcd940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 -- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 msgr2=0x7fcd940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda8159890 0x7fcd940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 -- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd94003a60 msgr2=0x7fcd94003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcd94003a60 0x7fcd94003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.628+0000 7fcd9e7fc700 1 -- 172.21.15.92:0/3534470800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94003570 con 0x7fcd94004810 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcd94004c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009c00 tx=0x7fcd900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c493138e15a2d35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000e710 con 0x7fcd94004810 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9000e870 con 0x7fcd94004810 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcda67fc700 1 -- 172.21.15.92:0/3534470800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000ec30 con 0x7fcd94004810 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcdad3eb700 1 -- 172.21.15.92:0/3534470800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 msgr2=0x7fcd94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/3534470800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcd94004c80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009c00 tx=0x7fcd900091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcda77fe700 1 -- 172.21.15.92:0/3534470800 reap_dead start 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcdad3eb700 1 -- 172.21.15.92:0/3534470800 shutdown_connections 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcdad3eb700 1 -- 172.21.15.92:0/3534470800 >> 172.21.15.92:0/3534470800 conn(0x7fcda8068e20 msgr2=0x7fcda80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcdad3eb700 1 -- 172.21.15.92:0/3534470800 shutdown_connections 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.629+0000 7fcdad3eb700 1 -- 172.21.15.92:0/3534470800 wait complete. 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 Processor -- start 2023-10-06T03:09:20.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 -- start start 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcda8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcda80b6150 0x7fcda80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda80b3590 0x7fcda80cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80cd410 con 0x7fcd94004810 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80b4e80 con 0x7fcda80b6150 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcdad3eb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80b5cc0 con 0x7fcda80b3590 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcda6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcda8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcda6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcda8150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50402/0 (socket says 172.21.15.92:50402) 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.631+0000 7fcda6ffd700 1 -- 172.21.15.92:0/2767799368 learned_addr learned my addr 172.21.15.92:0/2767799368 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:20.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda77fe700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda80b3590 0x7fcda80cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcda80b6150 0x7fcda80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda6ffd700 1 -- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda80b3590 msgr2=0x7fcda80cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda6ffd700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda80b3590 0x7fcda80cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda6ffd700 1 -- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcda80b6150 msgr2=0x7fcda80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda6ffd700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcda80b6150 0x7fcda80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda6ffd700 1 -- 172.21.15.92:0/2767799368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda80b50b0 con 0x7fcd94004810 2023-10-06T03:09:20.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcda6ffd700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcda8150ec0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fcd98011da0 tx=0x7fcd980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:20.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9800fa10 con 0x7fcd94004810 2023-10-06T03:09:20.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9800fb70 con 0x7fcd94004810 2023-10-06T03:09:20.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda80b5390 con 0x7fcd94004810 2023-10-06T03:09:20.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98014440 con 0x7fcd94004810 2023-10-06T03:09:20.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.632+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda80b5950 con 0x7fcd94004810 2023-10-06T03:09:20.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.633+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda804c3c0 con 0x7fcd94004810 2023-10-06T03:09:20.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.633+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcd98014870 con 0x7fcd94004810 2023-10-06T03:09:20.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.634+0000 7fcd9f7fe700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd88067320 0x7fcd880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:20.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.634+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fcd98018030 con 0x7fcd94004810 2023-10-06T03:09:20.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.634+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd88067320 0x7fcd880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:20.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.634+0000 7fcd9e7fc700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd88067320 0x7fcd880697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcd900091e0 tx=0x7fcd9001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:20.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.638+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd98052250 con 0x7fcd94004810 2023-10-06T03:09:20.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.865+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcda806a5b0 con 0x7fcd94004810 2023-10-06T03:09:20.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.865+0000 7fcd9f7fe700 1 -- 172.21.15.92:0/2767799368 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4618 (secure 0 0 0) 0x7fcd98055870 con 0x7fcd94004810 2023-10-06T03:09:20.866 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:20.866 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:14.655190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000645996","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:20.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd88067320 msgr2=0x7fcd880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd88067320 0x7fcd880697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcd900091e0 tx=0x7fcd9001a000 comp rx=0 tx=0).stop 2023-10-06T03:09:20.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 msgr2=0x7fcda8150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:20.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcda8150ec0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fcd98011da0 tx=0x7fcd980091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 shutdown_connections 2023-10-06T03:09:20.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcd88067320 0x7fcd880697d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcd94004810 0x7fcda8150ec0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcda80b3590 0x7fcda80cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 --2- 172.21.15.92:0/2767799368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcda80b6150 0x7fcda80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 >> 172.21.15.92:0/2767799368 conn(0x7fcda8068e20 msgr2=0x7fcda815cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 shutdown_connections 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.867+0000 7fcdad3eb700 1 -- 172.21.15.92:0/2767799368 wait complete. 2023-10-06T03:09:20.869 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 96 2023-10-06T03:09:20.880 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:20.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.994+0000 7f65879cb700 1 Processor -- start 2023-10-06T03:09:20.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.994+0000 7f65879cb700 1 -- start start 2023-10-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f65879cb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6580159890 0x7f658015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f65879cb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6580159c60 0x7f658015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f65879cb700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658015a160 con 0x7f658015db10 2023-10-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f65879cb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658015df50 con 0x7f6580159890 2023-10-06T03:09:21.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f65879cb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65800d13a0 con 0x7f6580159c60 2023-10-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6580159890 0x7f658015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6580159890 0x7f658015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50412/0 (socket says 172.21.15.92:50412) 2023-10-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 -- 172.21.15.92:0/279946369 learned_addr learned my addr 172.21.15.92:0/279946369 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:21.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6584f66700 1 --2- 172.21.15.92:0/279946369 >> v2:172.21.15.136:3300/0 conn(0x7f6580159c60 0x7f658015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 -- 172.21.15.92:0/279946369 >> v2:172.21.15.136:3300/0 conn(0x7f6580159c60 msgr2=0x7f658015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:21.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 --2- 172.21.15.92:0/279946369 >> v2:172.21.15.136:3300/0 conn(0x7f6580159c60 0x7f658015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 -- 172.21.15.92:0/279946369 >> v1:172.21.15.136:6789/0 conn(0x7f658015db10 legacy=0x7f6580069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:21.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 -- 172.21.15.92:0/279946369 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658015e140 con 0x7f6580159890 2023-10-06T03:09:21.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6585767700 1 --2- 172.21.15.92:0/279946369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f658015abd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f65800b67f0 tx=0x7f657c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6627e40e713d08b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.995+0000 7f6577fff700 1 -- 172.21.15.92:0/279946369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c00b820 con 0x7f6580159890 2023-10-06T03:09:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f6577fff700 1 -- 172.21.15.92:0/279946369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c00b980 con 0x7f6580159890 2023-10-06T03:09:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f6577fff700 1 -- 172.21.15.92:0/279946369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c00bd20 con 0x7f6580159890 2023-10-06T03:09:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 -- 172.21.15.92:0/279946369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 msgr2=0x7f658015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 --2- 172.21.15.92:0/279946369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f658015abd0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f65800b67f0 tx=0x7f657c009250 comp rx=0 tx=0).stop 2023-10-06T03:09:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 -- 172.21.15.92:0/279946369 shutdown_connections 2023-10-06T03:09:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 --2- 172.21.15.92:0/279946369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f658015abd0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 --2- 172.21.15.92:0/279946369 >> v2:172.21.15.136:3300/0 conn(0x7f6580159c60 0x7f658015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 -- 172.21.15.92:0/279946369 >> 172.21.15.92:0/279946369 conn(0x7f6580068e20 msgr2=0x7f65800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:21.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 -- 172.21.15.92:0/279946369 shutdown_connections 2023-10-06T03:09:21.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.996+0000 7f65879cb700 1 -- 172.21.15.92:0/279946369 wait complete. 2023-10-06T03:09:21.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.997+0000 7f65879cb700 1 Processor -- start 2023-10-06T03:09:21.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.997+0000 7f65879cb700 1 -- start start 2023-10-06T03:09:21.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f65879cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f65800b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f65879cb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6580159c60 0x7f6580158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f65879cb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658015db10 0x7f658016acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f65879cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800c9320 con 0x7f6580159890 2023-10-06T03:09:21.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f65879cb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800b4680 con 0x7f658015db10 2023-10-06T03:09:21.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f65879cb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65800c89f0 con 0x7f6580159c60 2023-10-06T03:09:21.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f65800b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f65800b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50420/0 (socket says 172.21.15.92:50420) 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 -- 172.21.15.92:0/1647335037 learned_addr learned my addr 172.21.15.92:0/1647335037 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6584f66700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6580159c60 0x7f6580158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585f68700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658015db10 0x7f658016acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 -- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6580159c60 msgr2=0x7f6580158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6580159c60 0x7f6580158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 -- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658015db10 msgr2=0x7f658016acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658015db10 0x7f658016acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585767700 1 -- 172.21.15.92:0/1647335037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0094f0 con 0x7f6580159890 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.998+0000 7f6585f68700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658015db10 0x7f658016acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.999+0000 7f6585767700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f65800b3f80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f657c009290 tx=0x7f657c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.999+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c011e20 con 0x7f6580159890 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.999+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c020070 con 0x7f6580159890 2023-10-06T03:09:21.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.999+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65800c8cd0 con 0x7f6580159890 2023-10-06T03:09:21.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.999+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c01b6a0 con 0x7f6580159890 2023-10-06T03:09:21.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:20.999+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658016b1f0 con 0x7f6580159890 2023-10-06T03:09:21.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.000+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658004c3c0 con 0x7f6580159890 2023-10-06T03:09:21.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.000+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f657c022030 con 0x7f6580159890 2023-10-06T03:09:21.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.001+0000 7f65757fa700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6564067320 0x7f65640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.001+0000 7f6584f66700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6564067320 0x7f65640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.001+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f657c086ce0 con 0x7f6580159890 2023-10-06T03:09:21.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.001+0000 7f6584f66700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6564067320 0x7f65640697d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f65800b2880 tx=0x7f6570008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.005+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f657c0538d0 con 0x7f6580159890 2023-10-06T03:09:21.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.231+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f658004c3c0 con 0x7f6580159890 2023-10-06T03:09:21.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.231+0000 7f65757fa700 1 -- 172.21.15.92:0/1647335037 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f657c056ef0 con 0x7f6580159890 2023-10-06T03:09:21.232 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:21.232 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:02.964243+0000","modified":"2023-10-06T03:09:14.655190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/499097126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":499097126},{"type":"v1","addr":"172.21.15.92:6837","nonce":499097126}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"volume_0000000000645996","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T03:09:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6564067320 msgr2=0x7f65640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6564067320 0x7f65640697d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f65800b2880 tx=0x7f6570008040 comp rx=0 tx=0).stop 2023-10-06T03:09:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 msgr2=0x7f65800b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f65800b3f80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f657c009290 tx=0x7f657c016000 comp rx=0 tx=0).stop 2023-10-06T03:09:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 shutdown_connections 2023-10-06T03:09:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6564067320 0x7f65640697d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6580159890 0x7f65800b3f80 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658015db10 0x7f658016acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 --2- 172.21.15.92:0/1647335037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6580159c60 0x7f6580158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 >> 172.21.15.92:0/1647335037 conn(0x7f6580068e20 msgr2=0x7f6580153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 shutdown_connections 2023-10-06T03:09:21.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.233+0000 7f65879cb700 1 -- 172.21.15.92:0/1647335037 wait complete. 2023-10-06T03:09:21.237 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 96 2023-10-06T03:09:21.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:21.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.361+0000 7f57d47b1700 1 Processor -- start 2023-10-06T03:09:21.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.361+0000 7f57d47b1700 1 -- start start 2023-10-06T03:09:21.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.362+0000 7f57d47b1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 0x7f57cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.362+0000 7f57d47b1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f57cc159c60 0x7f57cc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.362+0000 7f57d47b1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57cc15a160 con 0x7f57cc15db10 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.362+0000 7f57d47b1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57cc0d13a0 con 0x7f57cc159c60 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.362+0000 7f57d47b1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc15e140 con 0x7f57cc159890 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.362+0000 7f57d1d4c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f57cc159c60 0x7f57cc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 0x7f57cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d2d4e700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f57cc15db10 0x7f57cc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:35830/0 (socket says 172.21.15.92:35830) 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 0x7f57cc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:48904/0 (socket says 172.21.15.92:48904) 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d2d4e700 1 -- 172.21.15.92:0/2587760750 learned_addr learned my addr 172.21.15.92:0/2587760750 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:21.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 -- 172.21.15.92:0/2587760750 >> v2:172.21.15.136:3300/0 conn(0x7f57cc159c60 msgr2=0x7f57cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 --2- 172.21.15.92:0/2587760750 >> v2:172.21.15.136:3300/0 conn(0x7f57cc159c60 0x7f57cc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 -- 172.21.15.92:0/2587760750 >> v1:172.21.15.136:6789/0 conn(0x7f57cc15db10 legacy=0x7f57cc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 -- 172.21.15.92:0/2587760750 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc15a3d0 con 0x7f57cc159890 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d254d700 1 --2- 172.21.15.92:0/2587760750 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 0x7f57cc15abd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009e60 tx=0x7f57c8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c413c38ccd2420 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d154b700 1 -- 172.21.15.92:0/2587760750 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800e070 con 0x7f57cc159890 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d154b700 1 -- 172.21.15.92:0/2587760750 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c800bb60 con 0x7f57cc159890 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.363+0000 7f57d154b700 1 -- 172.21.15.92:0/2587760750 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8012430 con 0x7f57cc159890 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 -- 172.21.15.92:0/2587760750 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 msgr2=0x7f57cc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2587760750 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 0x7f57cc15abd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009e60 tx=0x7f57c8009780 comp rx=0 tx=0).stop 2023-10-06T03:09:21.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 -- 172.21.15.92:0/2587760750 shutdown_connections 2023-10-06T03:09:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2587760750 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc159890 0x7f57cc15abd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2587760750 >> v2:172.21.15.136:3300/0 conn(0x7f57cc159c60 0x7f57cc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 -- 172.21.15.92:0/2587760750 >> 172.21.15.92:0/2587760750 conn(0x7f57cc068e20 msgr2=0x7f57cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 -- 172.21.15.92:0/2587760750 shutdown_connections 2023-10-06T03:09:21.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.364+0000 7f57d47b1700 1 -- 172.21.15.92:0/2587760750 wait complete. 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.365+0000 7f57d47b1700 1 Processor -- start 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.365+0000 7f57d47b1700 1 -- start start 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d47b1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57cc159890 0x7f57cc0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d47b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 0x7f57cc0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d47b1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc15db10 0x7f57cc15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d47b1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0b3e30 con 0x7f57cc159c60 2023-10-06T03:09:21.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d47b1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0c8210 con 0x7f57cc159890 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d47b1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc0c84f0 con 0x7f57cc15db10 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 0x7f57cc0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 0x7f57cc0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50426/0 (socket says 172.21.15.92:50426) 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 -- 172.21.15.92:0/2004313069 learned_addr learned my addr 172.21.15.92:0/2004313069 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d2d4e700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc15db10 0x7f57cc15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d254d700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57cc159890 0x7f57cc0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 -- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc15db10 msgr2=0x7f57cc15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc15db10 0x7f57cc15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 -- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57cc159890 msgr2=0x7f57cc0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57cc159890 0x7f57cc0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 -- 172.21.15.92:0/2004313069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c8009a20 con 0x7f57cc159c60 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d2d4e700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc15db10 0x7f57cc15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d254d700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57cc159890 0x7f57cc0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.366+0000 7f57d1d4c700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 0x7f57cc0c9620 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00ec50 tx=0x7f57bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.367+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00c9d0 con 0x7f57cc159c60 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.367+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00cb30 con 0x7f57cc159c60 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.367+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc0c87d0 con 0x7f57cc159c60 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.367+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc013470 con 0x7f57cc159c60 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.367+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc0c7de0 con 0x7f57cc159c60 2023-10-06T03:09:21.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.367+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc04c3c0 con 0x7f57cc159c60 2023-10-06T03:09:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.368+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f57bc013860 con 0x7f57cc159c60 2023-10-06T03:09:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.368+0000 7f57c27fc700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f57b0067320 0x7f57b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.369+0000 7f57d254d700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f57b0067320 0x7f57b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.369+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f57bc085170 con 0x7f57cc159c60 2023-10-06T03:09:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.369+0000 7f57d254d700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f57b0067320 0x7f57b00697d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f57c80097c0 tx=0x7f57c8019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.373+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57bc051c50 con 0x7f57cc159c60 2023-10-06T03:09:21.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.557+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57cc06a5b0 con 0x7f57cc159c60 2023-10-06T03:09:21.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.558+0000 7f57c27fc700 1 -- 172.21.15.92:0/2004313069 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f57bc055270 con 0x7f57cc159c60 2023-10-06T03:09:21.558 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:21.559 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":143,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:16.658258+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs.volume_0000000000645996.meta","create_time":"2023-10-06T03:09:00.975583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000645996"}}},{"pool":26,"pool_name":"cephfs.volume_0000000000645996.data","create_time":"2023-10-06T03:09:01.925339+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000645996"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.559+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f57b0067320 msgr2=0x7f57b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f57b0067320 0x7f57b00697d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f57c80097c0 tx=0x7f57c8019000 comp rx=0 tx=0).stop 2023-10-06T03:09:21.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 msgr2=0x7f57cc0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 0x7f57cc0c9620 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00ec50 tx=0x7f57bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 shutdown_connections 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f57b0067320 0x7f57b00697d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57cc15db10 0x7f57cc15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57cc159890 0x7f57cc0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 --2- 172.21.15.92:0/2004313069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f57cc159c60 0x7f57cc0c9620 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 >> 172.21.15.92:0/2004313069 conn(0x7f57cc068e20 msgr2=0x7f57cc0c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:21.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 shutdown_connections 2023-10-06T03:09:21.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.560+0000 7f57d47b1700 1 -- 172.21.15.92:0/2004313069 wait complete. 2023-10-06T03:09:21.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000645996 2023-10-06T03:09:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.688+0000 7fed6ca5b700 1 Processor -- start 2023-10-06T03:09:21.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.688+0000 7fed6ca5b700 1 -- start start 2023-10-06T03:09:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed6ca5b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed68159420 0x7fed680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed6ca5b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed680b2980 con 0x7fed6815d230 2023-10-06T03:09:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed6ca5b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed680b11d0 con 0x7fed681597f0 2023-10-06T03:09:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed6ca5b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed680b14b0 con 0x7fed68159420 2023-10-06T03:09:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed677fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fed68159420 0x7fed680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed67fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fed6815d230 0x7fed6815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52652/0 (socket says 172.21.15.92:52652) 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed67fff700 1 -- 172.21.15.92:0/501442308 learned_addr learned my addr 172.21.15.92:0/501442308 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed677fe700 1 -- 172.21.15.92:0/501442308 >> v1:172.21.15.136:6789/0 conn(0x7fed681597f0 legacy=0x7fed680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed677fe700 1 -- 172.21.15.92:0/501442308 >> v1:172.21.15.92:6789/0 conn(0x7fed6815d230 legacy=0x7fed6815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed677fe700 1 -- 172.21.15.92:0/501442308 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680b1790 con 0x7fed68159420 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed667fc700 1 -- 172.21.15.92:0/501442308 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334993841 0 0) 0x7fed680b2980 con 0x7fed6815d230 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.689+0000 7fed677fe700 1 --2- 172.21.15.92:0/501442308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680b5f40 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fed58009e60 tx=0x7fed58009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d921323cc0512bd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed667fc700 1 -- 172.21.15.92:0/501442308 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5800e070 con 0x7fed68159420 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed667fc700 1 -- 172.21.15.92:0/501442308 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5800bbf0 con 0x7fed68159420 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed667fc700 1 -- 172.21.15.92:0/501442308 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58012430 con 0x7fed68159420 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 -- 172.21.15.92:0/501442308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 msgr2=0x7fed680b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/501442308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680b5f40 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fed58009e60 tx=0x7fed58009780 comp rx=0 tx=0).stop 2023-10-06T03:09:21.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 -- 172.21.15.92:0/501442308 shutdown_connections 2023-10-06T03:09:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/501442308 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680b5f40 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 -- 172.21.15.92:0/501442308 >> 172.21.15.92:0/501442308 conn(0x7fed680d0660 msgr2=0x7fed680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 -- 172.21.15.92:0/501442308 shutdown_connections 2023-10-06T03:09:21.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.690+0000 7fed6ca5b700 1 -- 172.21.15.92:0/501442308 wait complete. 2023-10-06T03:09:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.691+0000 7fed6ca5b700 1 Processor -- start 2023-10-06T03:09:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.691+0000 7fed6ca5b700 1 -- start start 2023-10-06T03:09:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed6ca5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed6ca5b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed681597f0 0x7fed68155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed6ca5b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed6815d230 0x7fed681581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed6ca5b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed680c5bd0 con 0x7fed68159420 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed6ca5b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6816a520 con 0x7fed681597f0 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed6ca5b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed680ca430 con 0x7fed6815d230 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed677fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50432/0 (socket says 172.21.15.92:50432) 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed677fe700 1 -- 172.21.15.92:0/257883969 learned_addr learned my addr 172.21.15.92:0/257883969 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed67fff700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed6815d230 0x7fed681581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed66ffd700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed681597f0 0x7fed68155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed677fe700 1 -- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed6815d230 msgr2=0x7fed681581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed677fe700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed6815d230 0x7fed681581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.692+0000 7fed677fe700 1 -- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed681597f0 msgr2=0x7fed68155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed677fe700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed681597f0 0x7fed68155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed677fe700 1 -- 172.21.15.92:0/257883969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed58009a20 con 0x7fed68159420 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed66ffd700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed681597f0 0x7fed68155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed67fff700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed6815d230 0x7fed681581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed677fe700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680c60c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b6c0 tx=0x7fed58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58012e80 con 0x7fed68159420 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5801e640 con 0x7fed68159420 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed680ca710 con 0x7fed68159420 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.693+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5801e9e0 con 0x7fed68159420 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.694+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed680c6600 con 0x7fed68159420 2023-10-06T03:09:21.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.694+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed680ca290 con 0x7fed68159420 2023-10-06T03:09:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.695+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fed58024070 con 0x7fed68159420 2023-10-06T03:09:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.695+0000 7fed4f7fe700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed54067320 0x7fed540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.695+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fed58087820 con 0x7fed68159420 2023-10-06T03:09:21.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.695+0000 7fed66ffd700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed54067320 0x7fed540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:21.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.696+0000 7fed66ffd700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed54067320 0x7fed540697d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fed68159c70 tx=0x7fed5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:21.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.701+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed680ca290 con 0x7fed68159420 2023-10-06T03:09:21.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.926+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000645996"} v 0) v1 -- 0x7fed680ca290 con 0x7fed68159420 2023-10-06T03:09:21.927 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:09:21.927+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:09:21.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.932+0000 7fed4f7fe700 1 -- 172.21.15.92:0/257883969 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000645996"}]=0 volume_0000000000645996 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 195+0+0 (secure 0 0 0) 0x7fed680ca290 con 0x7fed68159420 2023-10-06T03:09:21.932 INFO:tasks.ceph.mds.a.smithi092.stderr: -777> 2023-10-06T03:09:07.882+0000 7f8484791700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:09:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed54067320 msgr2=0x7fed540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed54067320 0x7fed540697d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fed68159c70 tx=0x7fed5c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 msgr2=0x7fed680c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680c60c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fed5800b6c0 tx=0x7fed58018000 comp rx=0 tx=0).stop 2023-10-06T03:09:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 shutdown_connections 2023-10-06T03:09:21.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed54067320 0x7fed540697d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed68159420 0x7fed680c60c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed6815d230 0x7fed681581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 --2- 172.21.15.92:0/257883969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed681597f0 0x7fed68155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 >> 172.21.15.92:0/257883969 conn(0x7fed680d0660 msgr2=0x7fed68156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 shutdown_connections 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:21.934+0000 7fed6ca5b700 1 -- 172.21.15.92:0/257883969 wait complete. 2023-10-06T03:09:21.936 INFO:teuthology.orchestra.run.smithi092.stderr:volume_0000000000645996 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:09:21.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000645996 --yes-i-really-mean-it 2023-10-06T03:09:21.952 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:09:22.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.064+0000 7f633a1b2700 1 Processor -- start 2023-10-06T03:09:22.065 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f633a1b2700 1 -- start start 2023-10-06T03:09:22.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f633a1b2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f63340c70b0 0x7f6334069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f633a1b2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6334069cb0 0x7f63340ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f633a1b2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633406e950 con 0x7f6334155eb0 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f633a1b2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633406a130 con 0x7f6334069cb0 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f633a1b2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633406eb90 con 0x7f63340c70b0 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.065+0000 7f6333fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6334069cb0 0x7f63340ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f63337fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6334155eb0 0x7f633406e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52678/0 (socket says 172.21.15.92:52678) 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f63337fe700 1 -- 172.21.15.92:0/3670733898 learned_addr learned my addr 172.21.15.92:0/3670733898 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f6332ffd700 1 --2- 172.21.15.92:0/3670733898 >> v2:172.21.15.136:3300/0 conn(0x7f63340c70b0 0x7f6334069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f6333fff700 1 -- 172.21.15.92:0/3670733898 >> v2:172.21.15.136:3300/0 conn(0x7f63340c70b0 msgr2=0x7f6334069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f6333fff700 1 --2- 172.21.15.92:0/3670733898 >> v2:172.21.15.136:3300/0 conn(0x7f63340c70b0 0x7f6334069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f6333fff700 1 -- 172.21.15.92:0/3670733898 >> v1:172.21.15.92:6789/0 conn(0x7f6334155eb0 legacy=0x7f633406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f6333fff700 1 -- 172.21.15.92:0/3670733898 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633406ee70 con 0x7f6334069cb0 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f63327fc700 1 -- 172.21.15.92:0/3670733898 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141720190 0 0) 0x7f6324001490 con 0x7f6334155eb0 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f6333fff700 1 --2- 172.21.15.92:0/3670733898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334069cb0 0x7f63340ca4c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6334065130 tx=0x7f632c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4a5919c317d847 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f63327fc700 1 -- 172.21.15.92:0/3670733898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00b6b0 con 0x7f6334069cb0 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f63327fc700 1 -- 172.21.15.92:0/3670733898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00b810 con 0x7f6334069cb0 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.066+0000 7f63327fc700 1 -- 172.21.15.92:0/3670733898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00bbb0 con 0x7f6334069cb0 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 -- 172.21.15.92:0/3670733898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334069cb0 msgr2=0x7f63340ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 --2- 172.21.15.92:0/3670733898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334069cb0 0x7f63340ca4c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6334065130 tx=0x7f632c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 -- 172.21.15.92:0/3670733898 shutdown_connections 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 --2- 172.21.15.92:0/3670733898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334069cb0 0x7f63340ca4c0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 --2- 172.21.15.92:0/3670733898 >> v2:172.21.15.136:3300/0 conn(0x7f63340c70b0 0x7f6334069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 -- 172.21.15.92:0/3670733898 >> 172.21.15.92:0/3670733898 conn(0x7f6334068e10 msgr2=0x7f63340cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 -- 172.21.15.92:0/3670733898 shutdown_connections 2023-10-06T03:09:22.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.067+0000 7f633a1b2700 1 -- 172.21.15.92:0/3670733898 wait complete. 2023-10-06T03:09:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.068+0000 7f633a1b2700 1 Processor -- start 2023-10-06T03:09:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.068+0000 7f633a1b2700 1 -- start start 2023-10-06T03:09:22.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6334069cb0 0x7f63340c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63340c70b0 0x7f63341426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 0x7f63340ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340c5850 con 0x7f6334155eb0 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340c6220 con 0x7f63340c70b0 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63340c4ca0 con 0x7f6334069cb0 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 0x7f63340ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 0x7f63340ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50476/0 (socket says 172.21.15.92:50476) 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 -- 172.21.15.92:0/240616369 learned_addr learned my addr 172.21.15.92:0/240616369 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6332ffd700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63340c70b0 0x7f63341426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f63337fe700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6334069cb0 0x7f63340c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:22.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 -- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6334069cb0 msgr2=0x7f63340c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6334069cb0 0x7f63340c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 -- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63340c70b0 msgr2=0x7f63341426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63340c70b0 0x7f63341426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 -- 172.21.15.92:0/240616369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009420 con 0x7f6334155eb0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f6333fff700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 0x7f63340ccb50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0e0 tx=0x7f632c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c00bb50 con 0x7f6334155eb0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c011630 con 0x7f6334155eb0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c0119d0 con 0x7f6334155eb0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633414c660 con 0x7f6334155eb0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.069+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340c5200 con 0x7f6334155eb0 2023-10-06T03:09:22.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.070+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633404c3c0 con 0x7f6334155eb0 2023-10-06T03:09:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.071+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f632c011d40 con 0x7f6334155eb0 2023-10-06T03:09:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.071+0000 7f631b7fe700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6320067320 0x7f63200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.071+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f632c086a20 con 0x7f6334155eb0 2023-10-06T03:09:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.071+0000 7f63337fe700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6320067320 0x7f63200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:22.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.072+0000 7f63337fe700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6320067320 0x7f63200697d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6324002230 tx=0x7f6324009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:22.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.075+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f632c0534b0 con 0x7f6334155eb0 2023-10-06T03:09:22.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.301+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000645996", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f63340678d0 con 0x7f6334155eb0 2023-10-06T03:09:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.931+0000 7f631b7fe700 1 -- 172.21.15.92:0/240616369 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000645996", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 121+0+0 (secure 0 0 0) 0x7f632c0150c0 con 0x7f6334155eb0 2023-10-06T03:09:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6320067320 msgr2=0x7f63200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6320067320 0x7f63200697d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6324002230 tx=0x7f6324009040 comp rx=0 tx=0).stop 2023-10-06T03:09:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 msgr2=0x7f63340ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 0x7f63340ccb50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0e0 tx=0x7f632c017000 comp rx=0 tx=0).stop 2023-10-06T03:09:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 shutdown_connections 2023-10-06T03:09:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6320067320 0x7f63200697d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6334155eb0 0x7f63340ccb50 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6334069cb0 0x7f63340c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 --2- 172.21.15.92:0/240616369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63340c70b0 0x7f63341426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 >> 172.21.15.92:0/240616369 conn(0x7f6334068e10 msgr2=0x7f633414ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 shutdown_connections 2023-10-06T03:09:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:22.934+0000 7f633a1b2700 1 -- 172.21.15.92:0/240616369 wait complete. 2023-10-06T03:09:22.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000645996.meta cephfs.volume_0000000000645996.meta --yes-i-really-really-mean-it 2023-10-06T03:09:23.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.113+0000 7fde5ca18700 1 Processor -- start 2023-10-06T03:09:23.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.113+0000 7fde5ca18700 1 -- start start 2023-10-06T03:09:23.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde5ca18700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde580b03c0 0x7fde580cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde5ca18700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde581591e0 0x7fde58159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde5ca18700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde580cfa00 con 0x7fde580b0790 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde5ca18700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde580cb580 con 0x7fde580b03c0 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde5ca18700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde580cf160 con 0x7fde581591e0 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde580b03c0 0x7fde580cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fde580b03c0 0x7fde580cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50492/0 (socket says 172.21.15.92:50492) 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 -- 172.21.15.92:0/2998307765 learned_addr learned my addr 172.21.15.92:0/2998307765 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 -- 172.21.15.92:0/2998307765 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde581591e0 msgr2=0x7fde58159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:23.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 --2- 172.21.15.92:0/2998307765 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde581591e0 0x7fde58159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 -- 172.21.15.92:0/2998307765 >> v1:172.21.15.136:6789/0 conn(0x7fde580b0790 legacy=0x7fde580b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 -- 172.21.15.92:0/2998307765 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580cf440 con 0x7fde580b03c0 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde577fe700 1 --2- 172.21.15.92:0/2998307765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde580b03c0 0x7fde580cea70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fde48009f50 tx=0x7fde48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e3ac693d0b9845 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde567fc700 1 -- 172.21.15.92:0/2998307765 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800e070 con 0x7fde580b03c0 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde567fc700 1 -- 172.21.15.92:0/2998307765 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800ba30 con 0x7fde580b03c0 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.114+0000 7fde567fc700 1 -- 172.21.15.92:0/2998307765 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde480092e0 con 0x7fde580b03c0 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 -- 172.21.15.92:0/2998307765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde580b03c0 msgr2=0x7fde580cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 --2- 172.21.15.92:0/2998307765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde580b03c0 0x7fde580cea70 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fde48009f50 tx=0x7fde48009840 comp rx=0 tx=0).stop 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 -- 172.21.15.92:0/2998307765 shutdown_connections 2023-10-06T03:09:23.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 --2- 172.21.15.92:0/2998307765 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde580b03c0 0x7fde580cea70 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 --2- 172.21.15.92:0/2998307765 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde581591e0 0x7fde58159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 -- 172.21.15.92:0/2998307765 >> 172.21.15.92:0/2998307765 conn(0x7fde58063230 msgr2=0x7fde58063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 -- 172.21.15.92:0/2998307765 shutdown_connections 2023-10-06T03:09:23.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.115+0000 7fde5ca18700 1 -- 172.21.15.92:0/2998307765 wait complete. 2023-10-06T03:09:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.116+0000 7fde5ca18700 1 Processor -- start 2023-10-06T03:09:23.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.116+0000 7fde5ca18700 1 -- start start 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde5ca18700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde580b03c0 0x7fde5815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde5ca18700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde580b0790 0x7fde58156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde5ca18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 0x7fde580c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde5ca18700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c62a0 con 0x7fde581591e0 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde5ca18700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c8d20 con 0x7fde580b0790 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde5ca18700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5815bb80 con 0x7fde580b03c0 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 0x7fde580c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 0x7fde580c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50500/0 (socket says 172.21.15.92:50500) 2023-10-06T03:09:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 -- 172.21.15.92:0/1124116883 learned_addr learned my addr 172.21.15.92:0/1124116883 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde577fe700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde580b03c0 0x7fde5815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde56ffd700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde580b0790 0x7fde58156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 -- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde580b03c0 msgr2=0x7fde5815b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde580b03c0 0x7fde5815b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 -- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde580b0790 msgr2=0x7fde58156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde580b0790 0x7fde58156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 -- 172.21.15.92:0/1124116883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48009ae0 con 0x7fde581591e0 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde577fe700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde580b03c0 0x7fde5815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.117+0000 7fde57fff700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 0x7fde580c5400 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fde5000b840 tx=0x7fde500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.118+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5000f510 con 0x7fde581591e0 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.118+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5000f670 con 0x7fde581591e0 2023-10-06T03:09:23.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.118+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde580c6d70 con 0x7fde581591e0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.118+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5000fa30 con 0x7fde581591e0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.118+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde58156ea0 con 0x7fde581591e0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.119+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde580c8ed0 con 0x7fde581591e0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.119+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fde5001a400 con 0x7fde581591e0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.119+0000 7fde3f7fe700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde440673a0 0x7fde44069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.120+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6222+0+0 (secure 0 0 0) 0x7fde50084a30 con 0x7fde581591e0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.120+0000 7fde577fe700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde440673a0 0x7fde44069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:23.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.120+0000 7fde577fe700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde440673a0 0x7fde44069850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde48009880 tx=0x7fde48017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.124+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde580c8ed0 con 0x7fde581591e0 2023-10-06T03:09:23.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.312+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000645996.meta", "pool2": "cephfs.volume_0000000000645996.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde580c8ed0 con 0x7fde581591e0 2023-10-06T03:09:23.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.934+0000 7fde3f7fe700 1 -- 172.21.15.92:0/1124116883 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000645996.meta", "pool2": "cephfs.volume_0000000000645996.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000645996.meta' removed v145) v1 ==== 241+0+0 (secure 0 0 0) 0x7fde580c8ed0 con 0x7fde581591e0 2023-10-06T03:09:23.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.939+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde440673a0 msgr2=0x7fde44069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:23.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.939+0000 7fde5ca18700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde440673a0 0x7fde44069850 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde48009880 tx=0x7fde48017000 comp rx=0 tx=0).stop 2023-10-06T03:09:23.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.939+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 msgr2=0x7fde580c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:23.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.939+0000 7fde5ca18700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 0x7fde580c5400 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fde5000b840 tx=0x7fde500091d0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 shutdown_connections 2023-10-06T03:09:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde440673a0 0x7fde44069850 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fde581591e0 0x7fde580c5400 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde580b03c0 0x7fde5815b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 --2- 172.21.15.92:0/1124116883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde580b0790 0x7fde58156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:23.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 >> 172.21.15.92:0/1124116883 conn(0x7fde58063230 msgr2=0x7fde580b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:23.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 shutdown_connections 2023-10-06T03:09:23.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:23.940+0000 7fde5ca18700 1 -- 172.21.15.92:0/1124116883 wait complete. 2023-10-06T03:09:23.943 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs.volume_0000000000645996.meta' removed 2023-10-06T03:09:23.956 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000645996.data cephfs.volume_0000000000645996.data --yes-i-really-really-mean-it 2023-10-06T03:09:24.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.077+0000 7fa819205700 1 Processor -- start 2023-10-06T03:09:24.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.077+0000 7fa819205700 1 -- start start 2023-10-06T03:09:24.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa819205700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 0x7fa8140b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa819205700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8140b53d0 0x7fa8140cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa819205700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8140b5b00 con 0x7fa8140cce40 2023-10-06T03:09:24.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa819205700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8140b5da0 con 0x7fa8140b53d0 2023-10-06T03:09:24.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa819205700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b6080 con 0x7fa814159ec0 2023-10-06T03:09:24.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa81359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa8140cce40 0x7fa8140cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52680/0 (socket says 172.21.15.92:52680) 2023-10-06T03:09:24.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa81359e700 1 -- 172.21.15.92:0/263996729 learned_addr learned my addr 172.21.15.92:0/263996729 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:24.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005448846 0 0) 0x7fa8140b5b00 con 0x7fa8140cce40 2023-10-06T03:09:24.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa800003570 con 0x7fa8140cce40 2023-10-06T03:09:24.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa812d9d700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 0x7fa8140b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa81259c700 1 --2- 172.21.15.92:0/263996729 >> v2:172.21.15.136:3300/0 conn(0x7fa8140b53d0 0x7fa8140cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4141595467 0 0) 0x7fa800003570 con 0x7fa8140cce40 2023-10-06T03:09:24.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 msgr2=0x7fa8140b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:24.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 0x7fa8140b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 >> v2:172.21.15.136:3300/0 conn(0x7fa8140b53d0 msgr2=0x7fa8140cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:24.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 --2- 172.21.15.92:0/263996729 >> v2:172.21.15.136:3300/0 conn(0x7fa8140b53d0 0x7fa8140cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140cc100 con 0x7fa8140cce40 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa804001470 con 0x7fa8140cce40 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 >> v1:172.21.15.92:6789/0 conn(0x7fa8140cce40 legacy=0x7fa8140cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800003a60 0x7fa800003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa800004810 0x7fa800004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 0x7fa8000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b5da0 con 0x7fa800004810 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b6080 con 0x7fa800003a60 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80000ba60 con 0x7fa814159ec0 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.078+0000 7fa81259c700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa800004810 0x7fa800004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa812d9d700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800003a60 0x7fa800003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81359e700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 0x7fa8000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81259c700 1 -- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 msgr2=0x7fa8000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81259c700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa814159ec0 0x7fa8000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81259c700 1 -- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800003a60 msgr2=0x7fa800003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81259c700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800003a60 0x7fa800003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81259c700 1 -- 172.21.15.92:0/263996729 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa800003570 con 0x7fa800004810 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa812d9d700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800003a60 0x7fa800003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81259c700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa800004810 0x7fa800004c80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009c30 tx=0x7fa7fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7a4309d7a83af3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00e710 con 0x7fa800004810 2023-10-06T03:09:24.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc00e870 con 0x7fa800004810 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa811d9b700 1 -- 172.21.15.92:0/263996729 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00ec30 con 0x7fa800004810 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa819205700 1 -- 172.21.15.92:0/263996729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa800004810 msgr2=0x7fa800004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa819205700 1 --2- 172.21.15.92:0/263996729 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa800004810 0x7fa800004c80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009c30 tx=0x7fa7fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.079+0000 7fa81359e700 1 -- 172.21.15.92:0/263996729 reap_dead start 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.080+0000 7fa819205700 1 -- 172.21.15.92:0/263996729 shutdown_connections 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.080+0000 7fa819205700 1 -- 172.21.15.92:0/263996729 >> 172.21.15.92:0/263996729 conn(0x7fa8140999f0 msgr2=0x7fa8140b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.080+0000 7fa819205700 1 -- 172.21.15.92:0/263996729 shutdown_connections 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.080+0000 7fa819205700 1 -- 172.21.15.92:0/263996729 wait complete. 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 Processor -- start 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 -- start start 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800004810 0x7fa81409b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 0x7fa8140b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa81416ab20 0x7fa81409d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814069d00 con 0x7fa81409ab30 2023-10-06T03:09:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81415f440 con 0x7fa800004810 2023-10-06T03:09:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.081+0000 7fa819205700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81409afc0 con 0x7fa81416ab20 2023-10-06T03:09:24.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 0x7fa8140b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 0x7fa8140b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50524/0 (socket says 172.21.15.92:50524) 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 -- 172.21.15.92:0/4071640511 learned_addr learned my addr 172.21.15.92:0/4071640511 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81359e700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa81416ab20 0x7fa81409d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa812d9d700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800004810 0x7fa81409b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 -- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa81416ab20 msgr2=0x7fa81409d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa81416ab20 0x7fa81409d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 -- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800004810 msgr2=0x7fa81409b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800004810 0x7fa81409b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 -- 172.21.15.92:0/4071640511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140696b0 con 0x7fa81409ab30 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa81259c700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 0x7fa8140b2a60 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa8000046d0 tx=0x7fa7fc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc022070 con 0x7fa81409ab30 2023-10-06T03:09:24.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc00ebf0 con 0x7fa81409ab30 2023-10-06T03:09:24.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa814069990 con 0x7fa81409ab30 2023-10-06T03:09:24.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc01c680 con 0x7fa81409ab30 2023-10-06T03:09:24.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.082+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81409da00 con 0x7fa81409ab30 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.083+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814069eb0 con 0x7fa81409ab30 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.083+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa7fc01cab0 con 0x7fa81409ab30 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.084+0000 7fa80affd700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa7f4067320 0x7fa7f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.084+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5834+0+0 (secure 0 0 0) 0x7fa7fc086680 con 0x7fa81409ab30 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.084+0000 7fa812d9d700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa7f4067320 0x7fa7f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.084+0000 7fa812d9d700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa7f4067320 0x7fa7f40697d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa80c011780 tx=0x7fa80c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:24.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.088+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa814069eb0 con 0x7fa81409ab30 2023-10-06T03:09:24.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.270+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000645996.data", "pool2": "cephfs.volume_0000000000645996.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa814069510 con 0x7fa81409ab30 2023-10-06T03:09:24.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.939+0000 7fa80affd700 1 -- 172.21.15.92:0/4071640511 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000645996.data", "pool2": "cephfs.volume_0000000000645996.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000645996.data' removed v146) v1 ==== 241+0+0 (secure 0 0 0) 0x7fa814069510 con 0x7fa81409ab30 2023-10-06T03:09:24.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.943+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa7f4067320 msgr2=0x7fa7f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:24.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.943+0000 7fa819205700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa7f4067320 0x7fa7f40697d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa80c011780 tx=0x7fa80c00b040 comp rx=0 tx=0).stop 2023-10-06T03:09:24.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.943+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 msgr2=0x7fa8140b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:24.944 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 0x7fa8140b2a60 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fa8000046d0 tx=0x7fa7fc01a000 comp rx=0 tx=0).stop 2023-10-06T03:09:24.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 shutdown_connections 2023-10-06T03:09:24.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa7f4067320 0x7fa7f40697d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa81416ab20 0x7fa81409d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800004810 0x7fa81409b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 --2- 172.21.15.92:0/4071640511 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa81409ab30 0x7fa8140b2a60 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:24.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 >> 172.21.15.92:0/4071640511 conn(0x7fa8140999f0 msgr2=0x7fa8140ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:24.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 shutdown_connections 2023-10-06T03:09:24.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:24.944+0000 7fa819205700 1 -- 172.21.15.92:0/4071640511 wait complete. 2023-10-06T03:09:24.946 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs.volume_0000000000645996.data' removed 2023-10-06T03:09:24.958 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:09:25.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.073+0000 7fa90a11e700 1 Processor -- start 2023-10-06T03:09:25.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.073+0000 7fa90a11e700 1 -- start start 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa90a11e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159890 0x7fa90415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa90a11e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa904159c60 0x7fa90415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa90a11e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90415a160 con 0x7fa90415db10 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa90a11e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040d13a0 con 0x7fa904159c60 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa90a11e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90415e140 con 0x7fa904159890 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa902ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa904159c60 0x7fa90415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa902ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa904159c60 0x7fa90415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50532/0 (socket says 172.21.15.92:50532) 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa902ffd700 1 -- 172.21.15.92:0/1286795529 learned_addr learned my addr 172.21.15.92:0/1286795529 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.074+0000 7fa9037fe700 1 --2- 172.21.15.92:0/1286795529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159890 0x7fa90415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa902ffd700 1 -- 172.21.15.92:0/1286795529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159890 msgr2=0x7fa90415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa902ffd700 1 --2- 172.21.15.92:0/1286795529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159890 0x7fa90415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa902ffd700 1 -- 172.21.15.92:0/1286795529 >> v1:172.21.15.136:6789/0 conn(0x7fa90415db10 legacy=0x7fa904069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa902ffd700 1 -- 172.21.15.92:0/1286795529 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90415a3d0 con 0x7fa904159c60 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa902ffd700 1 --2- 172.21.15.92:0/1286795529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa904159c60 0x7fa90415d5d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80099b0 tx=0x7fa8f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b8247601e866 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa9027fc700 1 -- 172.21.15.92:0/1286795529 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800b860 con 0x7fa904159c60 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa9027fc700 1 -- 172.21.15.92:0/1286795529 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f800b9c0 con 0x7fa904159c60 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa9027fc700 1 -- 172.21.15.92:0/1286795529 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800bd60 con 0x7fa904159c60 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 -- 172.21.15.92:0/1286795529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa904159c60 msgr2=0x7fa90415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 --2- 172.21.15.92:0/1286795529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa904159c60 0x7fa90415d5d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80099b0 tx=0x7fa8f8009250 comp rx=0 tx=0).stop 2023-10-06T03:09:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 -- 172.21.15.92:0/1286795529 shutdown_connections 2023-10-06T03:09:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 --2- 172.21.15.92:0/1286795529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159890 0x7fa90415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 --2- 172.21.15.92:0/1286795529 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa904159c60 0x7fa90415d5d0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 -- 172.21.15.92:0/1286795529 >> 172.21.15.92:0/1286795529 conn(0x7fa904068e20 msgr2=0x7fa9040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.075+0000 7fa90a11e700 1 -- 172.21.15.92:0/1286795529 shutdown_connections 2023-10-06T03:09:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.076+0000 7fa90a11e700 1 -- 172.21.15.92:0/1286795529 wait complete. 2023-10-06T03:09:25.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 Processor -- start 2023-10-06T03:09:25.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 -- start start 2023-10-06T03:09:25.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa904159890 0x7fa904148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159c60 0x7fa9040b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 0x7fa9040b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa904153770 con 0x7fa90415db10 2023-10-06T03:09:25.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90416a310 con 0x7fa904159890 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.077+0000 7fa90a11e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040b62d0 con 0x7fa904159c60 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 0x7fa9040b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 0x7fa9040b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50548/0 (socket says 172.21.15.92:50548) 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 -- 172.21.15.92:0/3769996213 learned_addr learned my addr 172.21.15.92:0/3769996213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa9037fe700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa904159890 0x7fa904148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa902ffd700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159c60 0x7fa9040b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 -- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159c60 msgr2=0x7fa9040b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159c60 0x7fa9040b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 -- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa904159890 msgr2=0x7fa904148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa904159890 0x7fa904148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 -- 172.21.15.92:0/3769996213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f80094f0 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa903fff700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 0x7fa9040b5c30 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c4c0 tx=0x7fa8fc007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011590 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc011710 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa904169cc0 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011ad0 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.078+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9041699b0 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.079+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90416a480 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.079+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa8fc01b3b0 con 0x7fa90415db10 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.080+0000 7fa8eb7fe700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8f0067320 0x7fa8f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.080+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa8fc083fd0 con 0x7fa90415db10 2023-10-06T03:09:25.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.080+0000 7fa9037fe700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8f0067320 0x7fa8f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.080+0000 7fa9037fe700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8f0067320 0x7fa8f00697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40075a0 tx=0x7fa8f40097a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:25.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.084+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8fc050e00 con 0x7fa90415db10 2023-10-06T03:09:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.266+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa90404c3c0 con 0x7fa90415db10 2023-10-06T03:09:25.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.270+0000 7fa8eb7fe700 1 -- 172.21.15.92:0/3769996213 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8fc017020 con 0x7fa90415db10 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8f0067320 msgr2=0x7fa8f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8f0067320 0x7fa8f00697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40075a0 tx=0x7fa8f40097a0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 msgr2=0x7fa9040b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 0x7fa9040b5c30 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c4c0 tx=0x7fa8fc007f50 comp rx=0 tx=0).stop 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 shutdown_connections 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8f0067320 0x7fa8f00697d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90415db10 0x7fa9040b5c30 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa904159890 0x7fa904148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 --2- 172.21.15.92:0/3769996213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa904159c60 0x7fa9040b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 >> 172.21.15.92:0/3769996213 conn(0x7fa904068e20 msgr2=0x7fa904154310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 shutdown_connections 2023-10-06T03:09:25.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.272+0000 7fa90a11e700 1 -- 172.21.15.92:0/3769996213 wait complete. 2023-10-06T03:09:25.285 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-06T03:09:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.398+0000 7fa3b42f7700 1 Processor -- start 2023-10-06T03:09:25.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.399+0000 7fa3b42f7700 1 -- start start 2023-10-06T03:09:25.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.399+0000 7fa3b42f7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3ac068920 0x7fa3ac06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b42f7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa3ac06f640 0x7fa3ac159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b42f7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac06ec70 con 0x7fa3ac068550 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b42f7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac068d90 con 0x7fa3ac068920 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b42f7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac0b3430 con 0x7fa3ac06f640 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3ac068920 0x7fa3ac06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa3ac068920 0x7fa3ac06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50558/0 (socket says 172.21.15.92:50558) 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 -- 172.21.15.92:0/480468271 learned_addr learned my addr 172.21.15.92:0/480468271 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b2894700 1 --2- 172.21.15.92:0/480468271 >> v2:172.21.15.136:3300/0 conn(0x7fa3ac06f640 0x7fa3ac159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 -- 172.21.15.92:0/480468271 >> v2:172.21.15.136:3300/0 conn(0x7fa3ac06f640 msgr2=0x7fa3ac159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 --2- 172.21.15.92:0/480468271 >> v2:172.21.15.136:3300/0 conn(0x7fa3ac06f640 0x7fa3ac159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 -- 172.21.15.92:0/480468271 >> v1:172.21.15.136:6789/0 conn(0x7fa3ac068550 legacy=0x7fa3ac15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 -- 172.21.15.92:0/480468271 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0b2e70 con 0x7fa3ac068920 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1892700 1 --2- 172.21.15.92:0/480468271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068920 0x7fa3ac06f100 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0aceb0 tx=0x7fa39c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f2b40d764b4a84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1091700 1 -- 172.21.15.92:0/480468271 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00b7e0 con 0x7fa3ac068920 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1091700 1 -- 172.21.15.92:0/480468271 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c00b940 con 0x7fa3ac068920 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.400+0000 7fa3b1091700 1 -- 172.21.15.92:0/480468271 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00bce0 con 0x7fa3ac068920 2023-10-06T03:09:25.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 -- 172.21.15.92:0/480468271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068920 msgr2=0x7fa3ac06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/480468271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068920 0x7fa3ac06f100 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0aceb0 tx=0x7fa39c009250 comp rx=0 tx=0).stop 2023-10-06T03:09:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 -- 172.21.15.92:0/480468271 shutdown_connections 2023-10-06T03:09:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/480468271 >> v2:172.21.15.136:3300/0 conn(0x7fa3ac06f640 0x7fa3ac159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/480468271 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068920 0x7fa3ac06f100 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 -- 172.21.15.92:0/480468271 >> 172.21.15.92:0/480468271 conn(0x7fa3ac0cd480 msgr2=0x7fa3ac0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:25.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 -- 172.21.15.92:0/480468271 shutdown_connections 2023-10-06T03:09:25.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.401+0000 7fa3b42f7700 1 -- 172.21.15.92:0/480468271 wait complete. 2023-10-06T03:09:25.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.402+0000 7fa3b42f7700 1 Processor -- start 2023-10-06T03:09:25.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.402+0000 7fa3b42f7700 1 -- start start 2023-10-06T03:09:25.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 0x7fa3ac159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3ac068920 0x7fa3ac0ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3ac06f640 0x7fa3ac0ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15b200 con 0x7fa3ac068550 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0acb20 con 0x7fa3ac06f640 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0af0d0 con 0x7fa3ac068920 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 0x7fa3ac159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 0x7fa3ac159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50566/0 (socket says 172.21.15.92:50566) 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 -- 172.21.15.92:0/1912498153 learned_addr learned my addr 172.21.15.92:0/1912498153 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2894700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3ac06f640 0x7fa3ac0ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b1892700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3ac068920 0x7fa3ac0ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 -- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3ac068920 msgr2=0x7fa3ac0ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3ac068920 0x7fa3ac0ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 -- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3ac06f640 msgr2=0x7fa3ac0ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3ac06f640 0x7fa3ac0ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 -- 172.21.15.92:0/1912498153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0094f0 con 0x7fa3ac068550 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b2093700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 0x7fa3ac159610 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800b600 tx=0x7fa3a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800f510 con 0x7fa3ac068550 2023-10-06T03:09:25.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a800f670 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac0af3b0 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a800fa30 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.403+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac0aedb0 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.404+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac04c3c0 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.405+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa3a801a400 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.405+0000 7fa3a27fc700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa390067320 0x7fa3900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.405+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa3a80844a0 con 0x7fa3ac068550 2023-10-06T03:09:25.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.405+0000 7fa3b1892700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa390067320 0x7fa3900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:25.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.406+0000 7fa3b1892700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa390067320 0x7fa3900697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0aceb0 tx=0x7fa39c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:25.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.409+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3a8051450 con 0x7fa3ac068550 2023-10-06T03:09:25.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.671+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7fa3ac0b3150 con 0x7fa3ac068550 2023-10-06T03:09:25.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.938+0000 7fa3a27fc700 1 -- 172.21.15.92:0/1912498153 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v391) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa3a8054a70 con 0x7fa3ac068550 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa390067320 msgr2=0x7fa3900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa390067320 0x7fa3900697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0aceb0 tx=0x7fa39c016000 comp rx=0 tx=0).stop 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 msgr2=0x7fa3ac159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 0x7fa3ac159610 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800b600 tx=0x7fa3a80091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 shutdown_connections 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa390067320 0x7fa3900697d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3ac068550 0x7fa3ac159610 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3ac06f640 0x7fa3ac0ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 --2- 172.21.15.92:0/1912498153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3ac068920 0x7fa3ac0ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:25.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 >> 172.21.15.92:0/1912498153 conn(0x7fa3ac0cd480 msgr2=0x7fa3ac156d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:25.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 shutdown_connections 2023-10-06T03:09:25.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:25.940+0000 7fa3b42f7700 1 -- 172.21.15.92:0/1912498153 wait complete. 2023-10-06T03:09:25.953 INFO:tasks.cephfs_test_runner:That volume, its file system and pools, can be renamed. ... ok 2023-10-06T03:09:25.953 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) That renaming volume fails without --yes-i-really-mean-it flag. 2023-10-06T03:09:25.953 INFO:tasks.cephfs_test_runner:test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:09:25.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-06T03:09:26.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.069+0000 7f772098b700 1 Processor -- start 2023-10-06T03:09:26.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.069+0000 7f772098b700 1 -- start start 2023-10-06T03:09:26.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f772098b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f771c069ac0 0x7f771c06a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:26.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f772098b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c0c96a0 con 0x7f771c1599e0 2023-10-06T03:09:26.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f772098b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c0cb9e0 con 0x7f771c159db0 2023-10-06T03:09:26.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f772098b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c0cab60 con 0x7f771c069ac0 2023-10-06T03:09:26.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f771a59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f771c1599e0 0x7f771c069080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52690/0 (socket says 172.21.15.92:52690) 2023-10-06T03:09:26.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f771a59c700 1 -- 172.21.15.92:0/801819584 learned_addr learned my addr 172.21.15.92:0/801819584 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f771ad9d700 1 --2- 172.21.15.92:0/801819584 >> v2:172.21.15.92:3300/0 conn(0x7f771c069ac0 0x7f771c06a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f7719d9b700 1 -- 172.21.15.92:0/801819584 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309646205 0 0) 0x7f771c0c96a0 con 0x7f771c1599e0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f7719d9b700 1 -- 172.21.15.92:0/801819584 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7708003570 con 0x7f771c1599e0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f771ad9d700 1 -- 172.21.15.92:0/801819584 >> v1:172.21.15.136:6789/0 conn(0x7f771c159db0 legacy=0x7f771c06a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f771ad9d700 1 -- 172.21.15.92:0/801819584 >> v1:172.21.15.92:6789/0 conn(0x7f771c1599e0 legacy=0x7f771c069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.070+0000 7f771ad9d700 1 -- 172.21.15.92:0/801819584 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c0cb420 con 0x7f771c069ac0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f771ad9d700 1 --2- 172.21.15.92:0/801819584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c069ac0 0x7f771c06a9c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f7714009e60 tx=0x7f7714009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4aa78cff536092 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f7719d9b700 1 -- 172.21.15.92:0/801819584 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771400e070 con 0x7f771c069ac0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f7719d9b700 1 -- 172.21.15.92:0/801819584 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771400bb20 con 0x7f771c069ac0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f7719d9b700 1 -- 172.21.15.92:0/801819584 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714012430 con 0x7f771c069ac0 2023-10-06T03:09:26.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 -- 172.21.15.92:0/801819584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c069ac0 msgr2=0x7f771c06a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 --2- 172.21.15.92:0/801819584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c069ac0 0x7f771c06a9c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f7714009e60 tx=0x7f7714009780 comp rx=0 tx=0).stop 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 -- 172.21.15.92:0/801819584 shutdown_connections 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 --2- 172.21.15.92:0/801819584 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c069ac0 0x7f771c06a9c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 -- 172.21.15.92:0/801819584 >> 172.21.15.92:0/801819584 conn(0x7f771c0c9860 msgr2=0x7f771c0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 -- 172.21.15.92:0/801819584 shutdown_connections 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.071+0000 7f772098b700 1 -- 172.21.15.92:0/801819584 wait complete. 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 Processor -- start 2023-10-06T03:09:26.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 -- start start 2023-10-06T03:09:26.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f771c069ac0 0x7f771c0c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:26.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 0x7f771c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:26.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f771c159db0 0x7f771c15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:26.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c15b070 con 0x7f771c1599e0 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0c50e0 con 0x7f771c159db0 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f772098b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0c4a30 con 0x7f771c069ac0 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f7711d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 0x7f771c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f7711d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 0x7f771c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50584/0 (socket says 172.21.15.92:50584) 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.073+0000 7f7711d9b700 1 -- 172.21.15.92:0/1190896270 learned_addr learned my addr 172.21.15.92:0/1190896270 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7711d9b700 1 -- 172.21.15.92:0/1190896270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f771c069ac0 msgr2=0x7f771c0c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7711d9b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f771c069ac0 0x7f771c0c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7711d9b700 1 -- 172.21.15.92:0/1190896270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f771c159db0 msgr2=0x7f771c15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7711d9b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f771c159db0 0x7f771c15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7711d9b700 1 -- 172.21.15.92:0/1190896270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714009a20 con 0x7f771c1599e0 2023-10-06T03:09:26.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7711d9b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 0x7f771c154f60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f770400b5e0 tx=0x7f77040090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770400f510 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770400f670 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770400fa30 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c1554a0 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.074+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c0c55d0 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.075+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c04c3c0 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.075+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f770401a400 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.076+0000 7f7712ffd700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76fc067320 0x7f76fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.076+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7704084560 con 0x7f771c1599e0 2023-10-06T03:09:26.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.076+0000 7f771a59c700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76fc067320 0x7f76fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:26.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.076+0000 7f771a59c700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76fc067320 0x7f76fc0697d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f770c0058d0 tx=0x7f770c0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:26.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.080+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7704051490 con 0x7f771c1599e0 2023-10-06T03:09:26.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.339+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f771c063490 con 0x7f771c1599e0 2023-10-06T03:09:26.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.939+0000 7f7712ffd700 1 -- 172.21.15.92:0/1190896270 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v392) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7704054ab0 con 0x7f771c1599e0 2023-10-06T03:09:26.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.941+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76fc067320 msgr2=0x7f76fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:26.942 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.941+0000 7f772098b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76fc067320 0x7f76fc0697d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f770c0058d0 tx=0x7f770c0079d0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.941+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 msgr2=0x7f771c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.941+0000 7f772098b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 0x7f771c154f60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f770400b5e0 tx=0x7f77040090f0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 shutdown_connections 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f76fc067320 0x7f76fc0697d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f771c159db0 0x7f771c15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f771c069ac0 0x7f771c0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 --2- 172.21.15.92:0/1190896270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f771c1599e0 0x7f771c154f60 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 >> 172.21.15.92:0/1190896270 conn(0x7f771c0c9860 msgr2=0x7f771c0704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 shutdown_connections 2023-10-06T03:09:26.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:26.942+0000 7f772098b700 1 -- 172.21.15.92:0/1190896270 wait complete. 2023-10-06T03:09:26.954 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:09:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.071+0000 7f4fbec1d700 1 Processor -- start 2023-10-06T03:09:27.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.071+0000 7f4fbec1d700 1 -- start start 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbec1d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fb80b4e70 0x7f4fb80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbec1d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbec1d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb80cfb00 con 0x7f4fb80b16d0 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbec1d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb80b5c40 con 0x7f4fb80b4e70 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbec1d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb80b5f20 con 0x7f4fb80b1b40 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbdc1b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4fb80b4e70 0x7f4fb80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbd41a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4fb80b16d0 0x7f4fb815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52692/0 (socket says 172.21.15.92:52692) 2023-10-06T03:09:27.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbd41a700 1 -- 172.21.15.92:0/3569131373 learned_addr learned my addr 172.21.15.92:0/3569131373 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbe41c700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbdc1b700 1 -- 172.21.15.92:0/3569131373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 msgr2=0x7f4fb815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbdc1b700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbcc19700 1 -- 172.21.15.92:0/3569131373 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452810558 0 0) 0x7f4fb80cfb00 con 0x7f4fb80b16d0 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbdc1b700 1 -- 172.21.15.92:0/3569131373 >> v1:172.21.15.92:6789/0 conn(0x7f4fb80b16d0 legacy=0x7f4fb815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbdc1b700 1 -- 172.21.15.92:0/3569131373 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb80b6200 con 0x7f4fb80b4e70 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbe41c700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbdc1b700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb80b4e70 0x7f4fb80ccff0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098e0 tx=0x7f4fb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aedd21d1b45494f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbcc19700 1 -- 172.21.15.92:0/3569131373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4009e80 con 0x7f4fb80b4e70 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbcc19700 1 -- 172.21.15.92:0/3569131373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb400b860 con 0x7f4fb80b4e70 2023-10-06T03:09:27.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.072+0000 7f4fbcc19700 1 -- 172.21.15.92:0/3569131373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb400bbe0 con 0x7f4fb80b4e70 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3569131373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb80b4e70 msgr2=0x7f4fb80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb80b4e70 0x7f4fb80ccff0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098e0 tx=0x7f4fb40091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3569131373 shutdown_connections 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3569131373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb80b4e70 0x7f4fb80ccff0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3569131373 >> 172.21.15.92:0/3569131373 conn(0x7f4fb80d10c0 msgr2=0x7f4fb80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3569131373 shutdown_connections 2023-10-06T03:09:27.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.073+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3569131373 wait complete. 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.074+0000 7f4fbec1d700 1 Processor -- start 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.074+0000 7f4fbec1d700 1 -- start start 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbec1d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fb80b16d0 0x7f4fb80c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbec1d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb80c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbec1d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 0x7f4fb816a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbec1d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb816ae60 con 0x7f4fb8169c90 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbec1d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb816bc50 con 0x7f4fb80b16d0 2023-10-06T03:09:27.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbec1d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8147580 con 0x7f4fb80b1b40 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 0x7f4fb816a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 0x7f4fb816a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50610/0 (socket says 172.21.15.92:50610) 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 -- 172.21.15.92:0/3308962364 learned_addr learned my addr 172.21.15.92:0/3308962364 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbdc1b700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fb80b16d0 0x7f4fb80c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbd41a700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb80c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 -- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 msgr2=0x7f4fb80c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb80c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 -- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fb80b16d0 msgr2=0x7f4fb80c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fb80b16d0 0x7f4fb80c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 -- 172.21.15.92:0/3308962364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4009420 con 0x7f4fb8169c90 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.075+0000 7f4fbe41c700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 0x7f4fb816a130 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000cef0 tx=0x7f4fb0007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.080 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.076+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000ae00 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.076+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0016420 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.076+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb816b040 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.076+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00167e0 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.076+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb816b600 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.077+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb816bdc0 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.077+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4fb0016bd0 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.077+0000 7f4fadffb700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f9c067320 0x7f4f9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.077+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4fb0012070 con 0x7f4fb8169c90 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.077+0000 7f4fbdc1b700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f9c067320 0x7f4f9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.078+0000 7f4fbdc1b700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f9c067320 0x7f4f9c0697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8071410 tx=0x7f4fb4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.081+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fb004e5a0 con 0x7f4fb8169c90 2023-10-06T03:09:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.265+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4fb80cc4a0 con 0x7f4fb8169c90 2023-10-06T03:09:27.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.268+0000 7f4fadffb700 1 -- 172.21.15.92:0/3308962364 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fb00522a0 con 0x7f4fb8169c90 2023-10-06T03:09:27.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f9c067320 msgr2=0x7f4f9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f9c067320 0x7f4f9c0697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8071410 tx=0x7f4fb4017000 comp rx=0 tx=0).stop 2023-10-06T03:09:27.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 msgr2=0x7f4fb816a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 0x7f4fb816a130 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000cef0 tx=0x7f4fb0007860 comp rx=0 tx=0).stop 2023-10-06T03:09:27.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 shutdown_connections 2023-10-06T03:09:27.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4f9c067320 0x7f4f9c0697d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4fb8169c90 0x7f4fb816a130 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fb80b16d0 0x7f4fb80c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 --2- 172.21.15.92:0/3308962364 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fb80b1b40 0x7f4fb80c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 >> 172.21.15.92:0/3308962364 conn(0x7f4fb80d10c0 msgr2=0x7f4fb8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:27.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 shutdown_connections 2023-10-06T03:09:27.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.271+0000 7f4fbec1d700 1 -- 172.21.15.92:0/3308962364 wait complete. 2023-10-06T03:09:27.283 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:09:27.284 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:27.319 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:27.363 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:09:27.363 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:09:27.363 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.478+0000 7f5011101700 1 Processor -- start 2023-10-06T03:09:27.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.478+0000 7f5011101700 1 -- start start 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f5011101700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c159890 0x7f500c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f5011101700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f500c159c60 0x7f500c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f5011101700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f500c15db10 0x7f500c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f5011101700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c15a160 con 0x7f500c159c60 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f5011101700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c0d13a0 con 0x7f500c15db10 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f5011101700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c15e140 con 0x7f500c159890 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f500c159c60 0x7f500c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f500c159c60 0x7f500c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50612/0 (socket says 172.21.15.92:50612) 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 -- 172.21.15.92:0/927302616 learned_addr learned my addr 172.21.15.92:0/927302616 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500ad9d700 1 --2- 172.21.15.92:0/927302616 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c159890 0x7f500c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500b59e700 1 --2- 172.21.15.92:0/927302616 >> v2:172.21.15.136:3300/0 conn(0x7f500c15db10 0x7f500c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 -- 172.21.15.92:0/927302616 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c159890 msgr2=0x7f500c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 --2- 172.21.15.92:0/927302616 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c159890 0x7f500c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 -- 172.21.15.92:0/927302616 >> v2:172.21.15.136:3300/0 conn(0x7f500c15db10 msgr2=0x7f500c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 --2- 172.21.15.92:0/927302616 >> v2:172.21.15.136:3300/0 conn(0x7f500c15db10 0x7f500c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 -- 172.21.15.92:0/927302616 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c15a3d0 con 0x7f500c159c60 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.479+0000 7f500259c700 1 --2- 172.21.15.92:0/927302616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159c60 0x7f500c15d5d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f500c089b50 tx=0x7f4ff4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff27ef29311dd53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f500a59c700 1 -- 172.21.15.92:0/927302616 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400b860 con 0x7f500c159c60 2023-10-06T03:09:27.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f500a59c700 1 -- 172.21.15.92:0/927302616 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff400b9c0 con 0x7f500c159c60 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f500a59c700 1 -- 172.21.15.92:0/927302616 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400bd60 con 0x7f500c159c60 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 -- 172.21.15.92:0/927302616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159c60 msgr2=0x7f500c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 --2- 172.21.15.92:0/927302616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159c60 0x7f500c15d5d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f500c089b50 tx=0x7f4ff4009250 comp rx=0 tx=0).stop 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 -- 172.21.15.92:0/927302616 shutdown_connections 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 --2- 172.21.15.92:0/927302616 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c159890 0x7f500c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 --2- 172.21.15.92:0/927302616 >> v2:172.21.15.136:3300/0 conn(0x7f500c15db10 0x7f500c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 --2- 172.21.15.92:0/927302616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159c60 0x7f500c15d5d0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 -- 172.21.15.92:0/927302616 >> 172.21.15.92:0/927302616 conn(0x7f500c068e20 msgr2=0x7f500c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 -- 172.21.15.92:0/927302616 shutdown_connections 2023-10-06T03:09:27.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.480+0000 7f5011101700 1 -- 172.21.15.92:0/927302616 wait complete. 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.481+0000 7f5011101700 1 Processor -- start 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.481+0000 7f5011101700 1 -- start start 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f5011101700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 0x7f500c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f5011101700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f500c159c60 0x7f500c151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f5011101700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c15db10 0x7f500c0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f5011101700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0ce9d0 con 0x7f500c159890 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f5011101700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0c9150 con 0x7f500c159c60 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f5011101700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0c7bf0 con 0x7f500c15db10 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 0x7f500c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 0x7f500c0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50622/0 (socket says 172.21.15.92:50622) 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 -- 172.21.15.92:0/385198124 learned_addr learned my addr 172.21.15.92:0/385198124 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500259c700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f500c159c60 0x7f500c151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500b59e700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c15db10 0x7f500c0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 -- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c15db10 msgr2=0x7f500c0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c15db10 0x7f500c0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 -- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f500c159c60 msgr2=0x7f500c151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f500c159c60 0x7f500c151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500ad9d700 1 -- 172.21.15.92:0/385198124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40094f0 con 0x7f500c159890 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.482+0000 7f500b59e700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c15db10 0x7f500c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.483+0000 7f500ad9d700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 0x7f500c0b3590 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00d8c0 tx=0x7f4ffc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.483+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0125f0 con 0x7f500c159890 2023-10-06T03:09:27.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.483+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc012750 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.483+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c0c96e0 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.483+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc012af0 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.483+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c0c9370 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.484+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c04c3c0 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.484+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4ffc01e460 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.484+0000 7f50037fe700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fec067320 0x7f4fec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.484+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4ffc085070 con 0x7f500c159890 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.485+0000 7f500259c700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fec067320 0x7f4fec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.485+0000 7f500259c700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fec067320 0x7f4fec0697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f500c089b50 tx=0x7f4ff4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.489+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ffc051ea0 con 0x7f500c159890 2023-10-06T03:09:27.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.717+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f500c06a5b0 con 0x7f500c159890 2023-10-06T03:09:27.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.718+0000 7f50037fe700 1 -- 172.21.15.92:0/385198124 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f4ffc0554c0 con 0x7f500c159890 2023-10-06T03:09:27.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:27.719 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/512035529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":512035529},{"type":"v1","addr":"172.21.15.136:6834","nonce":512035529}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":6940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2076888959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":2076888959},{"type":"v1","addr":"172.21.15.136:6835","nonce":2076888959}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2660704139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2660704139},{"type":"v1","addr":"172.21.15.92:6836","nonce":2660704139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":7923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2055212738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":2055212738},{"type":"v1","addr":"172.21.15.92:6837","nonce":2055212738}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98}],"filesystems":[]} 2023-10-06T03:09:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fec067320 msgr2=0x7f4fec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fec067320 0x7f4fec0697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f500c089b50 tx=0x7f4ff4017000 comp rx=0 tx=0).stop 2023-10-06T03:09:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 msgr2=0x7f500c0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 0x7f500c0b3590 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00d8c0 tx=0x7f4ffc008040 comp rx=0 tx=0).stop 2023-10-06T03:09:27.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 shutdown_connections 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4fec067320 0x7f4fec0697d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f500c159890 0x7f500c0b3590 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f500c15db10 0x7f500c0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 --2- 172.21.15.92:0/385198124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f500c159c60 0x7f500c151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 >> 172.21.15.92:0/385198124 conn(0x7f500c068e20 msgr2=0x7f500c154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 shutdown_connections 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.720+0000 7f5011101700 1 -- 172.21.15.92:0/385198124 wait complete. 2023-10-06T03:09:27.722 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 98 2023-10-06T03:09:27.733 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:09:27.733 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:09:27.733 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:09:27.733 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:27.734 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:09:27.734 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:09:27.734 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:09:27.734 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:27.734 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:09:27.734 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:09:27.734 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:09:27.735 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:27.735 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:09:27.735 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:09:27.735 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:09:27.735 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:27.743 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:09:27.743 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:09:27.744 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:09:27.744 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:09:27.745 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:09:27.745 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:09:27.746 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:09:27.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:09:27.786 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:09:27.786 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:27.787 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:09:27.787 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:27.788 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:09:27.788 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:27.788 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:09:27.789 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:27.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:09:27.859 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:09:27.859 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:09:27.866 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:09:27.867 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:09:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.907+0000 7f7c6eb28700 1 Processor -- start 2023-10-06T03:09:27.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.907+0000 7f7c6eb28700 1 -- start start 2023-10-06T03:09:27.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6eb28700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 0x7f7c680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6eb28700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680b11d0 con 0x7f7c68159420 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6eb28700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680b14b0 con 0x7f7c6815d230 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6eb28700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b1790 con 0x7f7c681597f0 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6db26700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7c68159420 0x7f7c680b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:52710/0 (socket says 172.21.15.92:52710) 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6db26700 1 -- 172.21.15.92:0/4173627926 learned_addr learned my addr 172.21.15.92:0/4173627926 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6d325700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 0x7f7c680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452125154 0 0) 0x7f7c680b11d0 con 0x7f7c68159420 2023-10-06T03:09:27.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.908+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c54003570 con 0x7f7c68159420 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797246811 0 0) 0x7f7c680b14b0 con 0x7f7c6815d230 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c680b11d0 con 0x7f7c6815d230 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046258193 0 0) 0x7f7c54003570 con 0x7f7c68159420 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 msgr2=0x7f7c680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 0x7f7c680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 >> v1:172.21.15.136:6789/0 conn(0x7f7c6815d230 legacy=0x7f7c680b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680b1a70 con 0x7f7c68159420 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7c64001470 con 0x7f7c68159420 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 >> v1:172.21.15.92:6789/0 conn(0x7f7c68159420 legacy=0x7f7c680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c54003c20 0x7f7c54004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 0x7f7c54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c54004d50 0x7f7c540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b1790 con 0x7f7c54004d50 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5400bbf0 con 0x7f7c54003c20 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5400bd60 con 0x7f7c681597f0 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6db26700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c54003c20 0x7f7c54004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c54004d50 0x7f7c540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6d325700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 0x7f7c54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 -- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 msgr2=0x7f7c54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c681597f0 0x7f7c54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 -- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c54003c20 msgr2=0x7f7c54004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c54003c20 0x7f7c54004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 -- 172.21.15.92:0/4173627926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54003570 con 0x7f7c54004d50 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6e327700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c54004d50 0x7f7c540051c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7c600020f0 tx=0x7f7c600017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71327ff59046966 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c60001ab0 con 0x7f7c54004d50 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6000c8b0 con 0x7f7c54004d50 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.909+0000 7f7c6cb24700 1 -- 172.21.15.92:0/4173627926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6000cc70 con 0x7f7c54004d50 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4173627926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c54004d50 msgr2=0x7f7c540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4173627926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c54004d50 0x7f7c540051c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7c600020f0 tx=0x7f7c600017a0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6e327700 1 -- 172.21.15.92:0/4173627926 reap_dead start 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4173627926 shutdown_connections 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4173627926 >> 172.21.15.92:0/4173627926 conn(0x7f7c680d0660 msgr2=0x7f7c680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:27.916 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4173627926 shutdown_connections 2023-10-06T03:09:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.910+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4173627926 wait complete. 2023-10-06T03:09:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.911+0000 7f7c6eb28700 1 Processor -- start 2023-10-06T03:09:27.917 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.911+0000 7f7c6eb28700 1 -- start start 2023-10-06T03:09:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 0x7f7c680c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.919 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c680b4470 0x7f7c680ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c68155080 0x7f7c681554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680cbc00 con 0x7f7c680b40a0 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c68155a00 con 0x7f7c68155080 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680cad50 con 0x7f7c680b4470 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 0x7f7c680c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 0x7f7c680c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50664/0 (socket says 172.21.15.92:50664) 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 -- 172.21.15.92:0/4273991711 learned_addr learned my addr 172.21.15.92:0/4273991711 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6e327700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c68155080 0x7f7c681554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6d325700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c680b4470 0x7f7c680ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 -- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c680b4470 msgr2=0x7f7c680ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.920 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c680b4470 0x7f7c680ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 -- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c68155080 msgr2=0x7f7c681554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c68155080 0x7f7c681554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 -- 172.21.15.92:0/4273991711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c680caff0 con 0x7f7c680b40a0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6e327700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c68155080 0x7f7c681554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6d325700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c680b4470 0x7f7c680ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6db26700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 0x7f7c680c8e30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7c64001e30 tx=0x7f7c64001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64018030 con 0x7f7c680b40a0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6400f610 con 0x7f7c680b40a0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c680cb2d0 con 0x7f7c680b40a0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.912+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6400f9d0 con 0x7f7c680b40a0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.913+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680cb890 con 0x7f7c680b40a0 2023-10-06T03:09:27.921 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.914+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6804c3c0 con 0x7f7c680b40a0 2023-10-06T03:09:27.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.914+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7c6401b070 con 0x7f7c680b40a0 2023-10-06T03:09:27.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.914+0000 7f7c59ffb700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c4c067320 0x7f7c4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:27.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.914+0000 7f7c6d325700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c4c067320 0x7f7c4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:27.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.914+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7c64019070 con 0x7f7c680b40a0 2023-10-06T03:09:27.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.915+0000 7f7c6d325700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c4c067320 0x7f7c4c0697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0093f0 tx=0x7f7c5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:27.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:27.919+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c64051720 con 0x7f7c680b40a0 2023-10-06T03:09:28.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.120+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7c680b1d50 con 0x7f7c680b40a0 2023-10-06T03:09:28.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.961+0000 7f7c59ffb700 1 -- 172.21.15.92:0/4273991711 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v151) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7c64054d40 con 0x7f7c680b40a0 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c4c067320 msgr2=0x7f7c4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c4c067320 0x7f7c4c0697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0093f0 tx=0x7f7c5c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 msgr2=0x7f7c680c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 0x7f7c680c8e30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7c64001e30 tx=0x7f7c64001d40 comp rx=0 tx=0).stop 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 shutdown_connections 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c4c067320 0x7f7c4c0697d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c680b40a0 0x7f7c680c8e30 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c68155080 0x7f7c681554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:28.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 --2- 172.21.15.92:0/4273991711 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c680b4470 0x7f7c680ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 >> 172.21.15.92:0/4273991711 conn(0x7f7c680d0660 msgr2=0x7f7c680699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 shutdown_connections 2023-10-06T03:09:28.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:28.963+0000 7f7c6eb28700 1 -- 172.21.15.92:0/4273991711 wait complete. 2023-10-06T03:09:28.966 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:09:28.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:09:29.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.091+0000 7fe1cffff700 1 Processor -- start 2023-10-06T03:09:29.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.091+0000 7fe1cffff700 1 -- start start 2023-10-06T03:09:29.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1d00cf6c0 0x7fe1d00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d015a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cffff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d00b5b40 con 0x7fe1d0159cc0 2023-10-06T03:09:29.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d00ce260 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00b5e20 con 0x7fe1d015a090 2023-10-06T03:09:29.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1c67fc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1d00cf6c0 0x7fe1d00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cf7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1c67fc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1d00cf6c0 0x7fe1d00b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:55308/0 (socket says 172.21.15.92:55308) 2023-10-06T03:09:29.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1cf7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d015a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:49148/0 (socket says 172.21.15.92:49148) 2023-10-06T03:09:29.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1ceffd700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fe1d0159cc0 0x7fe1d00b2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:35910/0 (socket says 172.21.15.92:35910) 2023-10-06T03:09:29.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.092+0000 7fe1c67fc700 1 -- 172.21.15.92:0/3983472345 learned_addr learned my addr 172.21.15.92:0/3983472345 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:29.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1c67fc700 1 -- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 msgr2=0x7fe1d015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:29.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1c67fc700 1 --2- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d015a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1c67fc700 1 -- 172.21.15.92:0/3983472345 >> v1:172.21.15.136:6789/0 conn(0x7fe1d0159cc0 legacy=0x7fe1d00b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1c67fc700 1 -- 172.21.15.92:0/3983472345 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00b6100 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1ce7fc700 1 -- 172.21.15.92:0/3983472345 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886593377 0 0) 0x7fe1d00b5b40 con 0x7fe1d0159cc0 2023-10-06T03:09:29.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1cf7fe700 1 --2- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:29.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1c67fc700 1 --2- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b5530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80099b0 tx=0x7fe1b8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6299283ee01b4cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1ce7fc700 1 -- 172.21.15.92:0/3983472345 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8009ea0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1ce7fc700 1 -- 172.21.15.92:0/3983472345 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b800b860 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1ce7fc700 1 -- 172.21.15.92:0/3983472345 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800bc00 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1cffff700 1 -- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d00cf6c0 msgr2=0x7fe1d00b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.093+0000 7fe1cffff700 1 --2- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b5530 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80099b0 tx=0x7fe1b8009250 comp rx=0 tx=0).stop 2023-10-06T03:09:29.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.094+0000 7fe1cffff700 1 -- 172.21.15.92:0/3983472345 shutdown_connections 2023-10-06T03:09:29.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.094+0000 7fe1cffff700 1 --2- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.094+0000 7fe1cffff700 1 --2- 172.21.15.92:0/3983472345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b5530 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.094+0000 7fe1cffff700 1 -- 172.21.15.92:0/3983472345 >> 172.21.15.92:0/3983472345 conn(0x7fe1d00d0560 msgr2=0x7fe1d00d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:29.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.094+0000 7fe1cffff700 1 -- 172.21.15.92:0/3983472345 shutdown_connections 2023-10-06T03:09:29.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.094+0000 7fe1cffff700 1 -- 172.21.15.92:0/3983472345 wait complete. 2023-10-06T03:09:29.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.095+0000 7fe1cffff700 1 Processor -- start 2023-10-06T03:09:29.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.095+0000 7fe1cffff700 1 -- start start 2023-10-06T03:09:29.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.095+0000 7fe1cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d0159cc0 0x7fe1d00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d0155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d006a1c0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0158890 con 0x7fe1d0159cc0 2023-10-06T03:09:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d0159410 con 0x7fe1d015a090 2023-10-06T03:09:29.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:50668/0 (socket says 172.21.15.92:50668) 2023-10-06T03:09:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 -- 172.21.15.92:0/1307912895 learned_addr learned my addr 172.21.15.92:0/1307912895 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1c67fc700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d0159cc0 0x7fe1d00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cf7fe700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d0155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 -- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 msgr2=0x7fe1d0155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d0155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 -- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d0159cc0 msgr2=0x7fe1d00b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d0159cc0 0x7fe1d00b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 -- 172.21.15.92:0/1307912895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8009a20 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1ceffd700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b1700 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c590 tx=0x7fe1c0001030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0005ac0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c0005c20 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80094f0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00070d0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.096+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d0158dc0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.097+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d004c3c0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.098+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe1c0007480 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.098+0000 7fe1c77fe700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe1b0067320 0x7fe1b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.098+0000 7fe1c67fc700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe1b0067320 0x7fe1b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.098+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe1c0083ec0 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.099+0000 7fe1c67fc700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe1b0067320 0x7fe1b00697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00b5a70 tx=0x7fe1b8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.102+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1c0050e00 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.364+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe1d015aa70 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.364+0000 7fe1c77fe700 1 -- 172.21.15.92:0/1307912895 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe1c0017070 con 0x7fe1d00cf6c0 2023-10-06T03:09:29.366 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:29.366 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.367 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.368 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.369 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:09:29.370 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.371 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.372 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:09:29.373 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.374 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.375 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.376 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:09:29.377 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.378 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.379 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:09:29.380 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:09:29.381 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.382 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:09:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:09:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:09:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:09:29.383 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:09:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:09:29.384 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:09:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:09:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:29.385 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:09:29.386 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:09:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe1b0067320 msgr2=0x7fe1b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe1b0067320 0x7fe1b00697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00b5a70 tx=0x7fe1b8017000 comp rx=0 tx=0).stop 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 msgr2=0x7fe1d00b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b1700 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c590 tx=0x7fe1c0001030 comp rx=0 tx=0).stop 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 shutdown_connections 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe1b0067320 0x7fe1b00697d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1d00cf6c0 0x7fe1d00b1700 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1d015a090 0x7fe1d0155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 --2- 172.21.15.92:0/1307912895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1d0159cc0 0x7fe1d00b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 >> 172.21.15.92:0/1307912895 conn(0x7fe1d00d0560 msgr2=0x7fe1d00cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:29.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 shutdown_connections 2023-10-06T03:09:29.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.367+0000 7fe1cffff700 1 -- 172.21.15.92:0/1307912895 wait complete. 2023-10-06T03:09:29.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:29.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.503+0000 7fc71fe0e700 1 Processor -- start 2023-10-06T03:09:29.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.503+0000 7fc71fe0e700 1 -- start start 2023-10-06T03:09:29.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71fe0e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc718159b10 0x7fc7180cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71fe0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc71806b420 0x7fc71815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71fe0e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7180c74b0 con 0x7fc71815d730 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71fe0e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180c8700 con 0x7fc71806b420 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71fe0e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7180c9070 con 0x7fc718159b10 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71e3ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc71806b420 0x7fc71815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71e3ab700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc71806b420 0x7fc71815dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34418/0 (socket says 172.21.15.92:34418) 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.504+0000 7fc71e3ab700 1 -- 172.21.15.92:0/122664104 learned_addr learned my addr 172.21.15.92:0/122664104 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71e3ab700 1 -- 172.21.15.92:0/122664104 >> v2:172.21.15.136:3300/0 conn(0x7fc718159b10 msgr2=0x7fc7180cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71dbaa700 1 --2- 172.21.15.92:0/122664104 >> v2:172.21.15.136:3300/0 conn(0x7fc718159b10 0x7fc7180cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71cba8700 1 -- 172.21.15.92:0/122664104 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200368808 0 0) 0x7fc7180c74b0 con 0x7fc71815d730 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71e3ab700 1 --2- 172.21.15.92:0/122664104 >> v2:172.21.15.136:3300/0 conn(0x7fc718159b10 0x7fc7180cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71e3ab700 1 -- 172.21.15.92:0/122664104 >> v1:172.21.15.92:6789/0 conn(0x7fc71815d730 legacy=0x7fc718159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71e3ab700 1 -- 172.21.15.92:0/122664104 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71815e0e0 con 0x7fc71806b420 2023-10-06T03:09:29.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71dbaa700 1 --2- 172.21.15.92:0/122664104 >> v2:172.21.15.136:3300/0 conn(0x7fc718159b10 0x7fc7180cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71e3ab700 1 --2- 172.21.15.92:0/122664104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71815dba0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc718063e90 tx=0x7fc710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5af5b20205574fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71cba8700 1 -- 172.21.15.92:0/122664104 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000b8f0 con 0x7fc71806b420 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71cba8700 1 -- 172.21.15.92:0/122664104 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71000ba50 con 0x7fc71806b420 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71cba8700 1 -- 172.21.15.92:0/122664104 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71000bdf0 con 0x7fc71806b420 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71fe0e700 1 -- 172.21.15.92:0/122664104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 msgr2=0x7fc71815dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.505+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/122664104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71815dba0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc718063e90 tx=0x7fc710009250 comp rx=0 tx=0).stop 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.506+0000 7fc71fe0e700 1 -- 172.21.15.92:0/122664104 shutdown_connections 2023-10-06T03:09:29.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.506+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/122664104 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71815dba0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.506+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/122664104 >> v2:172.21.15.136:3300/0 conn(0x7fc718159b10 0x7fc7180cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.506+0000 7fc71fe0e700 1 -- 172.21.15.92:0/122664104 >> 172.21.15.92:0/122664104 conn(0x7fc7180c9870 msgr2=0x7fc7180c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.506+0000 7fc71fe0e700 1 -- 172.21.15.92:0/122664104 shutdown_connections 2023-10-06T03:09:29.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.506+0000 7fc71fe0e700 1 -- 172.21.15.92:0/122664104 wait complete. 2023-10-06T03:09:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 Processor -- start 2023-10-06T03:09:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 -- start start 2023-10-06T03:09:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71814aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc718159b10 0x7fc71815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc71815d730 0x7fc718169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71815aa40 con 0x7fc71806b420 2023-10-06T03:09:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71815a710 con 0x7fc71815d730 2023-10-06T03:09:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71fe0e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71816a780 con 0x7fc718159b10 2023-10-06T03:09:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.507+0000 7fc71dbaa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71814aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71814aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34422/0 (socket says 172.21.15.92:34422) 2023-10-06T03:09:29.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 -- 172.21.15.92:0/2532751546 learned_addr learned my addr 172.21.15.92:0/2532751546 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71e3ab700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc71815d730 0x7fc718169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71d3a9700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc718159b10 0x7fc71815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 -- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc718159b10 msgr2=0x7fc71815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc718159b10 0x7fc71815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 -- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc71815d730 msgr2=0x7fc718169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc71815d730 0x7fc718169db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 -- 172.21.15.92:0/2532751546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100094f0 con 0x7fc71806b420 2023-10-06T03:09:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71dbaa700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71814aa10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc71400a400 tx=0x7fc714007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400d650 con 0x7fc71806b420 2023-10-06T03:09:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71400d7b0 con 0x7fc71806b420 2023-10-06T03:09:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71816aa60 con 0x7fc71806b420 2023-10-06T03:09:29.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400db50 con 0x7fc71806b420 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.508+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71814af50 con 0x7fc71806b420 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.509+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71815d570 con 0x7fc71806b420 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.509+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc714015440 con 0x7fc71806b420 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.510+0000 7fc70dffb700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc6fc067320 0x7fc6fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.510+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc714085330 con 0x7fc71806b420 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.510+0000 7fc71d3a9700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc6fc067320 0x7fc6fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.510+0000 7fc71d3a9700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc6fc067320 0x7fc6fc0697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc7180640a0 tx=0x7fc708009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.514+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc71404e5a0 con 0x7fc71806b420 2023-10-06T03:09:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.699+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc71815ad20 con 0x7fc71806b420 2023-10-06T03:09:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.699+0000 7fc70dffb700 1 -- 172.21.15.92:0/2532751546 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+13660 (secure 0 0 0) 0x7fc714083e60 con 0x7fc71806b420 2023-10-06T03:09:29.701 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:29.701 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":151,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:28.959708+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc6fc067320 msgr2=0x7fc6fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc6fc067320 0x7fc6fc0697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc7180640a0 tx=0x7fc708009040 comp rx=0 tx=0).stop 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 msgr2=0x7fc71814aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71814aa10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc71400a400 tx=0x7fc714007910 comp rx=0 tx=0).stop 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 shutdown_connections 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc6fc067320 0x7fc6fc0697d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.701+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc71806b420 0x7fc71814aa10 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.702+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc71815d730 0x7fc718169db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.702+0000 7fc71fe0e700 1 --2- 172.21.15.92:0/2532751546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc718159b10 0x7fc71815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.702+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 >> 172.21.15.92:0/2532751546 conn(0x7fc7180c9870 msgr2=0x7fc718156870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:29.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.702+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 shutdown_connections 2023-10-06T03:09:29.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.702+0000 7fc71fe0e700 1 -- 172.21.15.92:0/2532751546 wait complete. 2023-10-06T03:09:29.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:09:29.716 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:09:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.833+0000 7efcc2097700 1 Processor -- start 2023-10-06T03:09:29.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.833+0000 7efcc2097700 1 -- start start 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcc2097700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcbc157960 0x7efcbc0af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcc2097700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcc2097700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc06b030 con 0x7efcbc06b4d0 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcc2097700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0ac760 con 0x7efcbc157960 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcc2097700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0af830 con 0x7efcbc06b970 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcbb7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcbc157960 0x7efcbc0af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcbbfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcb2ffd700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7efcbc06b4d0 0x7efcbc068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:37498/0 (socket says 172.21.15.92:37498) 2023-10-06T03:09:29.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcbb7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcbc157960 0x7efcbc0af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:57858/0 (socket says 172.21.15.92:57858) 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcbbfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42502/0 (socket says 172.21.15.92:42502) 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.834+0000 7efcb2ffd700 1 -- 172.21.15.92:0/3927721058 learned_addr learned my addr 172.21.15.92:0/3927721058 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbb7fe700 1 -- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 msgr2=0x7efcbc157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbb7fe700 1 --2- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbb7fe700 1 -- 172.21.15.92:0/3927721058 >> v1:172.21.15.136:6789/0 conn(0x7efcbc06b4d0 legacy=0x7efcbc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbb7fe700 1 -- 172.21.15.92:0/3927721058 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0c7f60 con 0x7efcbc157960 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbaffd700 1 -- 172.21.15.92:0/3927721058 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115370042 0 0) 0x7efcbc06b030 con 0x7efcbc06b4d0 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbb7fe700 1 --2- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc0af250 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098e0 tx=0x7efcac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11c5fd81743fc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbaffd700 1 -- 172.21.15.92:0/3927721058 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00b640 con 0x7efcbc157960 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbaffd700 1 -- 172.21.15.92:0/3927721058 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac00b7a0 con 0x7efcbc157960 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcbaffd700 1 -- 172.21.15.92:0/3927721058 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00bb40 con 0x7efcbc157960 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcc2097700 1 -- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 msgr2=0x7efcbc0af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.835+0000 7efcc2097700 1 --2- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc0af250 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098e0 tx=0x7efcac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.836+0000 7efcc2097700 1 -- 172.21.15.92:0/3927721058 shutdown_connections 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.836+0000 7efcc2097700 1 --2- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc0af250 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.836+0000 7efcc2097700 1 --2- 172.21.15.92:0/3927721058 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.836+0000 7efcc2097700 1 -- 172.21.15.92:0/3927721058 >> 172.21.15.92:0/3927721058 conn(0x7efcbc0c9870 msgr2=0x7efcbc0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.836+0000 7efcc2097700 1 -- 172.21.15.92:0/3927721058 shutdown_connections 2023-10-06T03:09:29.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.836+0000 7efcc2097700 1 -- 172.21.15.92:0/3927721058 wait complete. 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.837+0000 7efcc2097700 1 Processor -- start 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.837+0000 7efcc2097700 1 -- start start 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.837+0000 7efcc2097700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 0x7efcbc0ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.837+0000 7efcc2097700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcc2097700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcc2097700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc153fa0 con 0x7efcbc06b4d0 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcc2097700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0adcd0 con 0x7efcbc157960 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcc2097700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0c4af0 con 0x7efcbc06b970 2023-10-06T03:09:29.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 0x7efcbc0ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 0x7efcbc0ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34438/0 (socket says 172.21.15.92:34438) 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 -- 172.21.15.92:0/2696146018 learned_addr learned my addr 172.21.15.92:0/2696146018 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbbfff700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcb2ffd700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 -- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 msgr2=0x7efcbc0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 -- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 msgr2=0x7efcbc1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 -- 172.21.15.92:0/2696146018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009420 con 0x7efcbc06b4d0 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcbb7fe700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 0x7efcbc0ad790 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7efcac009f80 tx=0x7efcac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00b910 con 0x7efcbc06b4d0 2023-10-06T03:09:29.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac00ba70 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc0c3f50 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac024670 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.838+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc0c44a0 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.839+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc04c3c0 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.840+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efcac011420 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.840+0000 7efcb3fff700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc9c067320 0x7efc9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.840+0000 7efcb2ffd700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc9c067320 0x7efc9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.840+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efcac086420 con 0x7efcbc06b4d0 2023-10-06T03:09:29.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.840+0000 7efcb2ffd700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc9c067320 0x7efc9c0697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0ae260 tx=0x7efca4007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:29.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:29.844+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcac0533a0 con 0x7efcbc06b4d0 2023-10-06T03:09:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.025+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcbc063300 con 0x7efcbc06b4d0 2023-10-06T03:09:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.025+0000 7efcb3fff700 1 -- 172.21.15.92:0/2696146018 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcac0569c0 con 0x7efcbc06b4d0 2023-10-06T03:09:30.027 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:09:30.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc9c067320 msgr2=0x7efc9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc9c067320 0x7efc9c0697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0ae260 tx=0x7efca4007f30 comp rx=0 tx=0).stop 2023-10-06T03:09:30.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 msgr2=0x7efcbc0ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 0x7efcbc0ad790 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7efcac009f80 tx=0x7efcac017000 comp rx=0 tx=0).stop 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 shutdown_connections 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc9c067320 0x7efc9c0697d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcbc06b4d0 0x7efcbc0ad790 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcbc157960 0x7efcbc1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 --2- 172.21.15.92:0/2696146018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcbc06b970 0x7efcbc0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 >> 172.21.15.92:0/2696146018 conn(0x7efcbc0c9870 msgr2=0x7efcbc1557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 shutdown_connections 2023-10-06T03:09:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.027+0000 7efcc2097700 1 -- 172.21.15.92:0/2696146018 wait complete. 2023-10-06T03:09:30.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:09:30.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.157+0000 7fee1d11c700 1 Processor -- start 2023-10-06T03:09:30.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.157+0000 7fee1d11c700 1 -- start start 2023-10-06T03:09:30.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1d11c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee1806bcd0 0x7fee180caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1d11c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1815d7e0 con 0x7fee1815df50 2023-10-06T03:09:30.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1d11c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1815dac0 con 0x7fee1806c0a0 2023-10-06T03:09:30.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1d11c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1815aa30 con 0x7fee1806bcd0 2023-10-06T03:09:30.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee16d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fee1806bcd0 0x7fee180caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1759e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fee1815df50 0x7fee180c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44752/0 (socket says 172.21.15.92:44752) 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1759e700 1 -- 172.21.15.92:0/1174843137 learned_addr learned my addr 172.21.15.92:0/1174843137 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee16d9d700 1 --2- 172.21.15.92:0/1174843137 >> v2:172.21.15.92:3300/0 conn(0x7fee1806bcd0 0x7fee180caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34440/0 (socket says 172.21.15.92:34440) 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee16d9d700 1 -- 172.21.15.92:0/1174843137 >> v1:172.21.15.136:6789/0 conn(0x7fee1806c0a0 legacy=0x7fee1815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee16d9d700 1 -- 172.21.15.92:0/1174843137 >> v1:172.21.15.92:6789/0 conn(0x7fee1815df50 legacy=0x7fee180c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee16d9d700 1 -- 172.21.15.92:0/1174843137 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1815ad10 con 0x7fee1806bcd0 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.158+0000 7fee1659c700 1 -- 172.21.15.92:0/1174843137 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345009039 0 0) 0x7fee1815d7e0 con 0x7fee1815df50 2023-10-06T03:09:30.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee16d9d700 1 --2- 172.21.15.92:0/1174843137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1806bcd0 0x7fee180caa90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fee080098e0 tx=0x7fee080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdf2b7debc66877 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1659c700 1 -- 172.21.15.92:0/1174843137 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08009e80 con 0x7fee1806bcd0 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1659c700 1 -- 172.21.15.92:0/1174843137 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0800b820 con 0x7fee1806bcd0 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1659c700 1 -- 172.21.15.92:0/1174843137 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0800bba0 con 0x7fee1806bcd0 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 -- 172.21.15.92:0/1174843137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1806bcd0 msgr2=0x7fee180caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 --2- 172.21.15.92:0/1174843137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1806bcd0 0x7fee180caa90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fee080098e0 tx=0x7fee080091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 -- 172.21.15.92:0/1174843137 shutdown_connections 2023-10-06T03:09:30.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 --2- 172.21.15.92:0/1174843137 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1806bcd0 0x7fee180caa90 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 -- 172.21.15.92:0/1174843137 >> 172.21.15.92:0/1174843137 conn(0x7fee180ce600 msgr2=0x7fee180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:30.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 -- 172.21.15.92:0/1174843137 shutdown_connections 2023-10-06T03:09:30.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.159+0000 7fee1d11c700 1 -- 172.21.15.92:0/1174843137 wait complete. 2023-10-06T03:09:30.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 Processor -- start 2023-10-06T03:09:30.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 -- start start 2023-10-06T03:09:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee1806bcd0 0x7fee180b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1806c0a0 0x7fee180c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 0x7fee18169f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18154f40 con 0x7fee1815df50 2023-10-06T03:09:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c85e0 con 0x7fee1806c0a0 2023-10-06T03:09:30.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1d11c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee180c6490 con 0x7fee1806bcd0 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 0x7fee18169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 0x7fee18169f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34454/0 (socket says 172.21.15.92:34454) 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee1759e700 1 -- 172.21.15.92:0/3757774310 learned_addr learned my addr 172.21.15.92:0/3757774310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee16d9d700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee1806bcd0 0x7fee180b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.161+0000 7fee0e59c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1806c0a0 0x7fee180c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1759e700 1 -- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee1806bcd0 msgr2=0x7fee180b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1759e700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee1806bcd0 0x7fee180b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1759e700 1 -- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1806c0a0 msgr2=0x7fee180c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1759e700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1806c0a0 0x7fee180c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1759e700 1 -- 172.21.15.92:0/3757774310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee08009420 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1759e700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 0x7fee18169f80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fee1000cbf0 tx=0x7fee100019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000a8b0 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1000aa10 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee180c9100 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000add0 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.162+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee180c8ab0 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.163+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee180c9290 con 0x7fee1815df50 2023-10-06T03:09:30.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.163+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fee10007300 con 0x7fee1815df50 2023-10-06T03:09:30.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.164+0000 7fee0f7fe700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedf8067320 0x7fedf80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.164+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fee10084d00 con 0x7fee1815df50 2023-10-06T03:09:30.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.164+0000 7fee16d9d700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedf8067320 0x7fedf80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.164+0000 7fee16d9d700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedf8067320 0x7fedf80697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fee080030c0 tx=0x7fee08016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.168+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee10051c00 con 0x7fee1815df50 2023-10-06T03:09:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.351+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fee1804c3c0 con 0x7fee1815df50 2023-10-06T03:09:30.353 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.352+0000 7fee0f7fe700 1 -- 172.21.15.92:0/3757774310 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v151) v1 ==== 92+0+10 (secure 0 0 0) 0x7fee10055220 con 0x7fee1815df50 2023-10-06T03:09:30.353 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedf8067320 msgr2=0x7fedf80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedf8067320 0x7fedf80697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fee080030c0 tx=0x7fee08016000 comp rx=0 tx=0).stop 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 msgr2=0x7fee18169f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 0x7fee18169f80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fee1000cbf0 tx=0x7fee100019b0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 shutdown_connections 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fedf8067320 0x7fedf80697d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee1815df50 0x7fee18169f80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee1806bcd0 0x7fee180b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 --2- 172.21.15.92:0/3757774310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1806c0a0 0x7fee180c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 >> 172.21.15.92:0/3757774310 conn(0x7fee180ce600 msgr2=0x7fee1806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 shutdown_connections 2023-10-06T03:09:30.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.354+0000 7fee1d11c700 1 -- 172.21.15.92:0/3757774310 wait complete. 2023-10-06T03:09:30.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:09:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.481+0000 7efc598dc700 1 Processor -- start 2023-10-06T03:09:30.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.481+0000 7efc598dc700 1 -- start start 2023-10-06T03:09:30.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc598dc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efc541597f0 0x7efc540b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc598dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc5415d230 0x7efc5415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc598dc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5415db70 con 0x7efc54159420 2023-10-06T03:09:30.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc598dc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc540b11d0 con 0x7efc5415d230 2023-10-06T03:09:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc598dc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc540b14b0 con 0x7efc541597f0 2023-10-06T03:09:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc5415d230 0x7efc5415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efc5415d230 0x7efc5415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34460/0 (socket says 172.21.15.92:34460) 2023-10-06T03:09:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 -- 172.21.15.92:0/344807942 learned_addr learned my addr 172.21.15.92:0/344807942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc4bfff700 1 --2- 172.21.15.92:0/344807942 >> v2:172.21.15.136:3300/0 conn(0x7efc541597f0 0x7efc540b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 -- 172.21.15.92:0/344807942 >> v2:172.21.15.136:3300/0 conn(0x7efc541597f0 msgr2=0x7efc540b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 --2- 172.21.15.92:0/344807942 >> v2:172.21.15.136:3300/0 conn(0x7efc541597f0 0x7efc540b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 -- 172.21.15.92:0/344807942 >> v1:172.21.15.136:6789/0 conn(0x7efc54159420 legacy=0x7efc540b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.482+0000 7efc590db700 1 -- 172.21.15.92:0/344807942 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc540b1790 con 0x7efc5415d230 2023-10-06T03:09:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc590db700 1 --2- 172.21.15.92:0/344807942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc5415d230 0x7efc5415d630 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efc54089750 tx=0x7efc4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b7354c7b3de45e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc4b7fe700 1 -- 172.21.15.92:0/344807942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00e070 con 0x7efc5415d230 2023-10-06T03:09:30.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc4b7fe700 1 -- 172.21.15.92:0/344807942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c00ba20 con 0x7efc5415d230 2023-10-06T03:09:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc4b7fe700 1 -- 172.21.15.92:0/344807942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c0092e0 con 0x7efc5415d230 2023-10-06T03:09:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 -- 172.21.15.92:0/344807942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc5415d230 msgr2=0x7efc5415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 --2- 172.21.15.92:0/344807942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc5415d230 0x7efc5415d630 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efc54089750 tx=0x7efc4c009840 comp rx=0 tx=0).stop 2023-10-06T03:09:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 -- 172.21.15.92:0/344807942 shutdown_connections 2023-10-06T03:09:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 --2- 172.21.15.92:0/344807942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc5415d230 0x7efc5415d630 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.492 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 --2- 172.21.15.92:0/344807942 >> v2:172.21.15.136:3300/0 conn(0x7efc541597f0 0x7efc540b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 -- 172.21.15.92:0/344807942 >> 172.21.15.92:0/344807942 conn(0x7efc540d0660 msgr2=0x7efc540d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 -- 172.21.15.92:0/344807942 shutdown_connections 2023-10-06T03:09:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.483+0000 7efc598dc700 1 -- 172.21.15.92:0/344807942 wait complete. 2023-10-06T03:09:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.484+0000 7efc598dc700 1 Processor -- start 2023-10-06T03:09:30.493 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 -- start start 2023-10-06T03:09:30.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 0x7efc5415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541597f0 0x7efc5415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc5415d230 0x7efc54153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415b650 con 0x7efc54159420 2023-10-06T03:09:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415b090 con 0x7efc541597f0 2023-10-06T03:09:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc598dc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5415b370 con 0x7efc5415d230 2023-10-06T03:09:30.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.485+0000 7efc588da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 0x7efc5415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 0x7efc5415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34472/0 (socket says 172.21.15.92:34472) 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 -- 172.21.15.92:0/761736099 learned_addr learned my addr 172.21.15.92:0/761736099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc4bfff700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541597f0 0x7efc5415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc590db700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc5415d230 0x7efc54153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 -- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc5415d230 msgr2=0x7efc54153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc5415d230 0x7efc54153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 -- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541597f0 msgr2=0x7efc5415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541597f0 0x7efc5415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 -- 172.21.15.92:0/761736099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009ae0 con 0x7efc54159420 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc590db700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc5415d230 0x7efc54153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc588da700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 0x7efc5415c940 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c590 tx=0x7efc5000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011590 con 0x7efc54159420 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50011710 con 0x7efc54159420 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc540c55a0 con 0x7efc54159420 2023-10-06T03:09:30.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50011ad0 con 0x7efc54159420 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.486+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc540c7430 con 0x7efc54159420 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.487+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc540c5400 con 0x7efc54159420 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.487+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efc5001b3b0 con 0x7efc54159420 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.488+0000 7efc48ff9700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc40067320 0x7efc400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.488+0000 7efc4bfff700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc40067320 0x7efc400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.488+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efc50083f30 con 0x7efc54159420 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.488+0000 7efc4bfff700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc40067320 0x7efc400697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efc540af880 tx=0x7efc440091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.492+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc540c5400 con 0x7efc54159420 2023-10-06T03:09:30.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.677+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efc540c5400 con 0x7efc54159420 2023-10-06T03:09:30.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.677+0000 7efc48ff9700 1 -- 172.21.15.92:0/761736099 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) v1 ==== 84+0+126 (secure 0 0 0) 0x7efc540c5400 con 0x7efc54159420 2023-10-06T03:09:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:30.679 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:09:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:09:30.679 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:09:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:09:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:09:30.680 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:30.680 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:09:30.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc40067320 msgr2=0x7efc400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc40067320 0x7efc400697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efc540af880 tx=0x7efc440091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 msgr2=0x7efc5415c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 0x7efc5415c940 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c590 tx=0x7efc5000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 shutdown_connections 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc40067320 0x7efc400697d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efc54159420 0x7efc5415c940 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc5415d230 0x7efc54153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 --2- 172.21.15.92:0/761736099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc541597f0 0x7efc5415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 >> 172.21.15.92:0/761736099 conn(0x7efc540d0660 msgr2=0x7efc540b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 shutdown_connections 2023-10-06T03:09:30.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.679+0000 7efc598dc700 1 -- 172.21.15.92:0/761736099 wait complete. 2023-10-06T03:09:30.694 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:09:30.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:09:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.811+0000 7fd095f5c700 1 Processor -- start 2023-10-06T03:09:30.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.811+0000 7fd095f5c700 1 -- start start 2023-10-06T03:09:30.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd095f5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0900cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd095f5c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900cfa00 con 0x7fd0901591e0 2023-10-06T03:09:30.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd095f5c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0900cb580 con 0x7fd0900b03c0 2023-10-06T03:09:30.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd095f5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900cf160 con 0x7fd0900b0790 2023-10-06T03:09:30.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd09575b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd0901591e0 0x7fd090159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44764/0 (socket says 172.21.15.92:44764) 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd09575b700 1 -- 172.21.15.92:0/2852658043 learned_addr learned my addr 172.21.15.92:0/2852658043 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd08ffff700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0900cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989119958 0 0) 0x7fd0900cfa00 con 0x7fd0901591e0 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.812+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd080003570 con 0x7fd0901591e0 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671656948 0 0) 0x7fd080003570 con 0x7fd0901591e0 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 msgr2=0x7fd0900cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0900cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 >> v1:172.21.15.136:6789/0 conn(0x7fd0900b03c0 legacy=0x7fd0900b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0900cf440 con 0x7fd0901591e0 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd0880017f0 con 0x7fd0901591e0 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 >> v1:172.21.15.92:6789/0 conn(0x7fd0901591e0 legacy=0x7fd090159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd080003a90 0x7fd080003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd080004810 0x7fd080004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0800051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900cf160 con 0x7fd080003a90 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08000bb90 con 0x7fd080004810 2023-10-06T03:09:30.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08000bd50 con 0x7fd0900b0790 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd080003a90 0x7fd080003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd09575b700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0800051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08ffff700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd080004810 0x7fd080004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 -- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 msgr2=0x7fd0800051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0800051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 -- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd080004810 msgr2=0x7fd080004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd080004810 0x7fd080004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 -- 172.21.15.92:0/2852658043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080003570 con 0x7fd080003a90 2023-10-06T03:09:30.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd09575b700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900b0790 0x7fd0800051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd094f5a700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd080003a90 0x7fd080003ea0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd0840018b0 tx=0x7fd084007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b780415cac88454d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084012420 con 0x7fd080003a90 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084012580 con 0x7fd080003a90 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.813+0000 7fd08f7fe700 1 -- 172.21.15.92:0/2852658043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084012920 con 0x7fd080003a90 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd095f5c700 1 -- 172.21.15.92:0/2852658043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd080003a90 msgr2=0x7fd080003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd095f5c700 1 --2- 172.21.15.92:0/2852658043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd080003a90 0x7fd080003ea0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fd0840018b0 tx=0x7fd084007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd09575b700 1 -- 172.21.15.92:0/2852658043 reap_dead start 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd095f5c700 1 -- 172.21.15.92:0/2852658043 shutdown_connections 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd095f5c700 1 -- 172.21.15.92:0/2852658043 >> 172.21.15.92:0/2852658043 conn(0x7fd090063230 msgr2=0x7fd090063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd095f5c700 1 -- 172.21.15.92:0/2852658043 shutdown_connections 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.814+0000 7fd095f5c700 1 -- 172.21.15.92:0/2852658043 wait complete. 2023-10-06T03:09:30.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.815+0000 7fd095f5c700 1 Processor -- start 2023-10-06T03:09:30.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.815+0000 7fd095f5c700 1 -- start start 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd095f5c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd090151e70 0x7fd0901565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd095f5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 0x7fd090158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd095f5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900c5160 0x7fd090158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd095f5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900c66e0 con 0x7fd090152240 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd095f5c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0900c6250 con 0x7fd090151e70 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd095f5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd090157b10 con 0x7fd0900c5160 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 0x7fd090158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 0x7fd090158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34498/0 (socket says 172.21.15.92:34498) 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 -- 172.21.15.92:0/3063282312 learned_addr learned my addr 172.21.15.92:0/3063282312 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd094f5a700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd090151e70 0x7fd0901565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd09575b700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900c5160 0x7fd090158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 -- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900c5160 msgr2=0x7fd090158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900c5160 0x7fd090158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 -- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd090151e70 msgr2=0x7fd0901565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd090151e70 0x7fd0901565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd08ffff700 1 -- 172.21.15.92:0/3063282312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0900c5640 con 0x7fd090152240 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.816+0000 7fd09575b700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900c5160 0x7fd090158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.817+0000 7fd08ffff700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 0x7fd090158530 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd07c011d70 tx=0x7fd07c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.817+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00fac0 con 0x7fd090152240 2023-10-06T03:09:30.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.817+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c00fc20 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.817+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c014440 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.817+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0900c5920 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.817+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0900c5ee0 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.818+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0900c71b0 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.818+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd07c014870 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.818+0000 7fd08cff9700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd074067320 0x7fd0740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.819+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd07c018030 con 0x7fd090152240 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.819+0000 7fd094f5a700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd074067320 0x7fd0740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:30.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.819+0000 7fd094f5a700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd074067320 0x7fd0740697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd084001690 tx=0x7fd084015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:30.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:30.823+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd07c0522d0 con 0x7fd090152240 2023-10-06T03:09:31.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.005+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd09004c3c0 con 0x7fd090152240 2023-10-06T03:09:31.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.973+0000 7fd08cff9700 1 -- 172.21.15.92:0/3063282312 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v152) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd07c0558f0 con 0x7fd090152240 2023-10-06T03:09:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.977+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd074067320 msgr2=0x7fd0740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.977+0000 7fd095f5c700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd074067320 0x7fd0740697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd084001690 tx=0x7fd084015000 comp rx=0 tx=0).stop 2023-10-06T03:09:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.977+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 msgr2=0x7fd090158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.977+0000 7fd095f5c700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 0x7fd090158530 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd07c011d70 tx=0x7fd07c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:31.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.978+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 shutdown_connections 2023-10-06T03:09:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.978+0000 7fd095f5c700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd074067320 0x7fd0740697d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.978+0000 7fd095f5c700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0900c5160 0x7fd090158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.978+0000 7fd095f5c700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd090151e70 0x7fd0901565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.978+0000 7fd095f5c700 1 --2- 172.21.15.92:0/3063282312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd090152240 0x7fd090158530 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:31.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.978+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 >> 172.21.15.92:0/3063282312 conn(0x7fd090063230 msgr2=0x7fd090069940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:31.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.979+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 shutdown_connections 2023-10-06T03:09:31.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:31.979+0000 7fd095f5c700 1 -- 172.21.15.92:0/3063282312 wait complete. 2023-10-06T03:09:31.982 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:09:31.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:09:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.122+0000 7fd98c268700 1 Processor -- start 2023-10-06T03:09:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.122+0000 7fd98c268700 1 -- start start 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.123+0000 7fd98c268700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 0x7fd9840cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.123+0000 7fd98c268700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9840ce380 con 0x7fd984159950 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.123+0000 7fd98c268700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9840cec20 con 0x7fd9840af320 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.123+0000 7fd98c268700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840afa90 con 0x7fd984069c70 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.123+0000 7fd98a004700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd984159950 0x7fd984069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44776/0 (socket says 172.21.15.92:44776) 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.123+0000 7fd98a004700 1 -- 172.21.15.92:0/1412282938 learned_addr learned my addr 172.21.15.92:0/1412282938 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989803700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 0x7fd9840cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524957487 0 0) 0x7fd9840ce380 con 0x7fd984159950 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd970003570 con 0x7fd984159950 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404885646 0 0) 0x7fd9840cec20 con 0x7fd9840af320 2023-10-06T03:09:32.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9840ce380 con 0x7fd9840af320 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2606749325 0 0) 0x7fd970003570 con 0x7fd984159950 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 msgr2=0x7fd9840cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 0x7fd9840cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 >> v1:172.21.15.136:6789/0 conn(0x7fd9840af320 legacy=0x7fd9840cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840ce5d0 con 0x7fd984159950 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd980001470 con 0x7fd984159950 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 >> v1:172.21.15.92:6789/0 conn(0x7fd984159950 legacy=0x7fd984069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970003c20 0x7fd970004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 0x7fd970004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840afa90 con 0x7fd970004d50 2023-10-06T03:09:32.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97000bb40 con 0x7fd970003c20 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97000bd00 con 0x7fd984069c70 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd98a805700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd98a004700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970003c20 0x7fd970004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.124+0000 7fd989803700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 0x7fd970004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd98a805700 1 -- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 msgr2=0x7fd970004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd98a805700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd984069c70 0x7fd970004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd98a805700 1 -- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970003c20 msgr2=0x7fd970004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd98a805700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970003c20 0x7fd970004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd98a805700 1 -- 172.21.15.92:0/1412282938 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd970003570 con 0x7fd970004d50 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd98a805700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd97c002120 tx=0x7fd97c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f25214000c1cb49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c001ab0 con 0x7fd970004d50 2023-10-06T03:09:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c00c980 con 0x7fd970004d50 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.125+0000 7fd989002700 1 -- 172.21.15.92:0/1412282938 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00cd20 con 0x7fd970004d50 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98c268700 1 -- 172.21.15.92:0/1412282938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd970004d50 msgr2=0x7fd9700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98c268700 1 --2- 172.21.15.92:0/1412282938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd970004d50 0x7fd9700051c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd97c002120 tx=0x7fd97c0017a0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98a805700 1 -- 172.21.15.92:0/1412282938 reap_dead start 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98c268700 1 -- 172.21.15.92:0/1412282938 shutdown_connections 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98c268700 1 -- 172.21.15.92:0/1412282938 >> 172.21.15.92:0/1412282938 conn(0x7fd984068e00 msgr2=0x7fd9840d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98c268700 1 -- 172.21.15.92:0/1412282938 shutdown_connections 2023-10-06T03:09:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.126+0000 7fd98c268700 1 -- 172.21.15.92:0/1412282938 wait complete. 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.127+0000 7fd98c268700 1 Processor -- start 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.127+0000 7fd98c268700 1 -- start start 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970004d50 0x7fd98414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9840c97c0 0x7fd9840c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 0x7fd9840c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840c8f70 con 0x7fd9840c9c30 2023-10-06T03:09:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840b3c40 con 0x7fd970004d50 2023-10-06T03:09:32.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9840c8440 con 0x7fd9840c97c0 2023-10-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 0x7fd9840c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 0x7fd9840c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34516/0 (socket says 172.21.15.92:34516) 2023-10-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 -- 172.21.15.92:0/137349409 learned_addr learned my addr 172.21.15.92:0/137349409 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a004700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970004d50 0x7fd98414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd989803700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9840c97c0 0x7fd9840c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.137 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 -- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9840c97c0 msgr2=0x7fd9840c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9840c97c0 0x7fd9840c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 -- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970004d50 msgr2=0x7fd98414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970004d50 0x7fd98414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 -- 172.21.15.92:0/137349409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840b35f0 con 0x7fd9840c9c30 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd989803700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9840c97c0 0x7fd9840c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98a805700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 0x7fd9840c7da0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd97c003000 tx=0x7fd97c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c00c6d0 con 0x7fd9840c9c30 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c006db0 con 0x7fd9840c9c30 2023-10-06T03:09:32.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9840b38d0 con 0x7fd9840c9c30 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c012da0 con 0x7fd9840c9c30 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.128+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98414d310 con 0x7fd9840c9c30 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.129+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9840b3db0 con 0x7fd9840c9c30 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.130+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd97c003030 con 0x7fd9840c9c30 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.130+0000 7fd97a7fc700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd968067320 0x7fd9680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.130+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fd97c086eb0 con 0x7fd9840c9c30 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.130+0000 7fd98a004700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd968067320 0x7fd9680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:32.139 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.131+0000 7fd98a004700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd968067320 0x7fd9680697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd980001200 tx=0x7fd980001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:32.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.135+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd97c053cd0 con 0x7fd9840c9c30 2023-10-06T03:09:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.320+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd9840718b0 con 0x7fd9840c9c30 2023-10-06T03:09:32.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.986+0000 7fd97a7fc700 1 -- 172.21.15.92:0/137349409 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v153) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd97c0572f0 con 0x7fd9840c9c30 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd968067320 msgr2=0x7fd9680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd968067320 0x7fd9680697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd980001200 tx=0x7fd980001df0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 msgr2=0x7fd9840c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 0x7fd9840c7da0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd97c003000 tx=0x7fd97c016000 comp rx=0 tx=0).stop 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 shutdown_connections 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd968067320 0x7fd9680697d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9840c9c30 0x7fd9840c7da0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd970004d50 0x7fd98414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 --2- 172.21.15.92:0/137349409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9840c97c0 0x7fd9840c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:32.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.993+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 >> 172.21.15.92:0/137349409 conn(0x7fd984068e00 msgr2=0x7fd9840b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.997+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 shutdown_connections 2023-10-06T03:09:32.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:32.998+0000 7fd98c268700 1 -- 172.21.15.92:0/137349409 wait complete. 2023-10-06T03:09:33.004 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:09:33.019 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:09:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.138+0000 7f2e2a984700 1 Processor -- start 2023-10-06T03:09:33.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.138+0000 7f2e2a984700 1 -- start start 2023-10-06T03:09:33.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e2a984700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 0x7f2e2415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e2a984700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2e240b5ea0 0x7f2e24068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e2a984700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2415c480 con 0x7f2e24068bb0 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e2a984700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240b2a40 con 0x7f2e240b5ea0 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e2a984700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2415b5d0 con 0x7f2e24159c10 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 0x7f2e2415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e237fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2e240b5ea0 0x7f2e24068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e237fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2e240b5ea0 0x7f2e24068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:57954/0 (socket says 172.21.15.92:57954) 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 0x7f2e2415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:42594/0 (socket says 172.21.15.92:42594) 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e237fe700 1 -- 172.21.15.92:0/24747007 learned_addr learned my addr 172.21.15.92:0/24747007 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.139+0000 7f2e28f21700 1 --1- 172.21.15.92:0/24747007 >> v1:172.21.15.136:6789/0 conn(0x7f2e24068bb0 0x7f2e2415c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:37532/0 (socket says 172.21.15.92:37532) 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e237fe700 1 -- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 msgr2=0x7f2e2415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:33.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e237fe700 1 --2- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 0x7f2e2415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e237fe700 1 -- 172.21.15.92:0/24747007 >> v1:172.21.15.136:6789/0 conn(0x7f2e24068bb0 legacy=0x7f2e2415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e237fe700 1 -- 172.21.15.92:0/24747007 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2415b870 con 0x7f2e240b5ea0 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e23fff700 1 --2- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 0x7f2e2415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e22ffd700 1 -- 172.21.15.92:0/24747007 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338036365 0 0) 0x7f2e2415c480 con 0x7f2e24068bb0 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e237fe700 1 --2- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e24068670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2e180099b0 tx=0x7f2e18009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b648d25774b25bf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e22ffd700 1 -- 172.21.15.92:0/24747007 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18009ea0 con 0x7f2e240b5ea0 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e22ffd700 1 -- 172.21.15.92:0/24747007 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1800b860 con 0x7f2e240b5ea0 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.140+0000 7f2e22ffd700 1 -- 172.21.15.92:0/24747007 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800bc00 con 0x7f2e240b5ea0 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 -- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 msgr2=0x7f2e24068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 --2- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e24068670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2e180099b0 tx=0x7f2e18009250 comp rx=0 tx=0).stop 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 -- 172.21.15.92:0/24747007 shutdown_connections 2023-10-06T03:09:33.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 --2- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24159c10 0x7f2e2415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 --2- 172.21.15.92:0/24747007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e24068670 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 -- 172.21.15.92:0/24747007 >> 172.21.15.92:0/24747007 conn(0x7f2e24099a20 msgr2=0x7f2e240af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 -- 172.21.15.92:0/24747007 shutdown_connections 2023-10-06T03:09:33.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.141+0000 7f2e2a984700 1 -- 172.21.15.92:0/24747007 wait complete. 2023-10-06T03:09:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.142+0000 7f2e2a984700 1 Processor -- start 2023-10-06T03:09:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.142+0000 7f2e2a984700 1 -- start start 2023-10-06T03:09:33.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24068bb0 0x7f2e24169610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e240c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 0x7f2e240c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240c5930 con 0x7f2e24159c10 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24169e60 con 0x7f2e240b5ea0 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2416a140 con 0x7f2e24068bb0 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 0x7f2e240c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 0x7f2e240c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34518/0 (socket says 172.21.15.92:34518) 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 -- 172.21.15.92:0/3048505803 learned_addr learned my addr 172.21.15.92:0/3048505803 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e237fe700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e240c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e23fff700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24068bb0 0x7f2e24169610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:33.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 -- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24068bb0 msgr2=0x7f2e24169610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24068bb0 0x7f2e24169610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 -- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 msgr2=0x7f2e240c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e240c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 -- 172.21.15.92:0/3048505803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14009420 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e237fe700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e240c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e28f21700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 0x7f2e240c5dc0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00cd50 tx=0x7f2e1c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c005df0 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e180094f0 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.143+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c00a960 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.144+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c00ad20 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.144+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240c55c0 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.145+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e240c5aa0 con 0x7f2e24159c10 2023-10-06T03:09:33.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.145+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2e1c007300 con 0x7f2e24159c10 2023-10-06T03:09:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.145+0000 7f2e0bfff700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2e10067320 0x7f2e100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.145+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f2e1c084fe0 con 0x7f2e24159c10 2023-10-06T03:09:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.145+0000 7f2e23fff700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2e10067320 0x7f2e100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:33.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.146+0000 7f2e23fff700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2e10067320 0x7f2e100697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2e140093f0 tx=0x7f2e14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:33.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.150+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e1c051b70 con 0x7f2e24159c10 2023-10-06T03:09:33.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:33.375+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2e2404c3c0 con 0x7f2e24159c10 2023-10-06T03:09:34.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.001+0000 7f2e0bfff700 1 -- 172.21.15.92:0/3048505803 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v100) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2e1c055190 con 0x7f2e24159c10 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2e10067320 msgr2=0x7f2e100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2e10067320 0x7f2e100697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2e140093f0 tx=0x7f2e14008040 comp rx=0 tx=0).stop 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 msgr2=0x7f2e240c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 0x7f2e240c5dc0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00cd50 tx=0x7f2e1c001b10 comp rx=0 tx=0).stop 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 shutdown_connections 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2e10067320 0x7f2e100697d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2e24159c10 0x7f2e240c5dc0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e24068bb0 0x7f2e24169610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 --2- 172.21.15.92:0/3048505803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e240b5ea0 0x7f2e240c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 >> 172.21.15.92:0/3048505803 conn(0x7f2e24099a20 msgr2=0x7f2e240b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:34.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 shutdown_connections 2023-10-06T03:09:34.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.010+0000 7f2e2a984700 1 -- 172.21.15.92:0/3048505803 wait complete. 2023-10-06T03:09:34.014 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 27 and data pool 28 2023-10-06T03:09:34.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.139+0000 7fcfe80f8700 1 Processor -- start 2023-10-06T03:09:34.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.139+0000 7fcfe80f8700 1 -- start start 2023-10-06T03:09:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe80f8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcfe0159c60 0x7fcfe015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe80f8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe015a160 con 0x7fcfe0159890 2023-10-06T03:09:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe80f8700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe00d13a0 con 0x7fcfe015db10 2023-10-06T03:09:34.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe80f8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe015df50 con 0x7fcfe0159c60 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe5693700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcfe0159c60 0x7fcfe015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe5693700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcfe0159c60 0x7fcfe015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34530/0 (socket says 172.21.15.92:34530) 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe5e94700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcfe0159890 0x7fcfe015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44778/0 (socket says 172.21.15.92:44778) 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.140+0000 7fcfe5693700 1 -- 172.21.15.92:0/2353169385 learned_addr learned my addr 172.21.15.92:0/2353169385 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe5693700 1 -- 172.21.15.92:0/2353169385 >> v1:172.21.15.136:6789/0 conn(0x7fcfe015db10 legacy=0x7fcfe0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe5693700 1 -- 172.21.15.92:0/2353169385 >> v1:172.21.15.92:6789/0 conn(0x7fcfe0159890 legacy=0x7fcfe015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe5693700 1 -- 172.21.15.92:0/2353169385 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe015e140 con 0x7fcfe0159c60 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe4e92700 1 -- 172.21.15.92:0/2353169385 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044170173 0 0) 0x7fcfe015a160 con 0x7fcfe0159890 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe5693700 1 --2- 172.21.15.92:0/2353169385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159c60 0x7fcfe015d5d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00099b0 tx=0x7fcfd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295714008268b759 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe4e92700 1 -- 172.21.15.92:0/2353169385 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000b820 con 0x7fcfe0159c60 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe4e92700 1 -- 172.21.15.92:0/2353169385 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd000b980 con 0x7fcfe0159c60 2023-10-06T03:09:34.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe4e92700 1 -- 172.21.15.92:0/2353169385 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000bd20 con 0x7fcfe0159c60 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 -- 172.21.15.92:0/2353169385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159c60 msgr2=0x7fcfe015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/2353169385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159c60 0x7fcfe015d5d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00099b0 tx=0x7fcfd0009250 comp rx=0 tx=0).stop 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 -- 172.21.15.92:0/2353169385 shutdown_connections 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/2353169385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159c60 0x7fcfe015d5d0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 -- 172.21.15.92:0/2353169385 >> 172.21.15.92:0/2353169385 conn(0x7fcfe0068e20 msgr2=0x7fcfe00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 -- 172.21.15.92:0/2353169385 shutdown_connections 2023-10-06T03:09:34.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.141+0000 7fcfe80f8700 1 -- 172.21.15.92:0/2353169385 wait complete. 2023-10-06T03:09:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.142+0000 7fcfe80f8700 1 Processor -- start 2023-10-06T03:09:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 -- start start 2023-10-06T03:09:34.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 0x7fcfe01501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe0159c60 0x7fcfe00cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfe015db10 0x7fcfe00c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe016a6a0 con 0x7fcfe0159890 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0159060 con 0x7fcfe0159c60 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe80f8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00cdba0 con 0x7fcfe015db10 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe5e94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 0x7fcfe01501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe5e94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 0x7fcfe01501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34536/0 (socket says 172.21.15.92:34536) 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.143+0000 7fcfe5e94700 1 -- 172.21.15.92:0/4291388219 learned_addr learned my addr 172.21.15.92:0/4291388219 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe5e94700 1 -- 172.21.15.92:0/4291388219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfe015db10 msgr2=0x7fcfe00c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe5e94700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfe015db10 0x7fcfe00c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe5e94700 1 -- 172.21.15.92:0/4291388219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe0159c60 msgr2=0x7fcfe00cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe5e94700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe0159c60 0x7fcfe00cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe5e94700 1 -- 172.21.15.92:0/4291388219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00094f0 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe5e94700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 0x7fcfe01501e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00cd30 tx=0x7fcfdc001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc005df0 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc00a960 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe0169d70 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00ad20 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.144+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe016a330 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.145+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe004c3c0 con 0x7fcfe0159890 2023-10-06T03:09:34.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.146+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcfdc007300 con 0x7fcfe0159890 2023-10-06T03:09:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.146+0000 7fcfd5ffb700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcfc4067320 0x7fcfc40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.146+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcfdc084fa0 con 0x7fcfe0159890 2023-10-06T03:09:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.146+0000 7fcfe5693700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcfc4067320 0x7fcfc40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.146+0000 7fcfe5693700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcfc4067320 0x7fcfc40697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00099b0 tx=0x7fcfd0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.150+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfdc051b50 con 0x7fcfe0159890 2023-10-06T03:09:34.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.336+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfe006a5b0 con 0x7fcfe0159890 2023-10-06T03:09:34.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.337+0000 7fcfd5ffb700 1 -- 172.21.15.92:0/4291388219 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fcfdc055170 con 0x7fcfe0159890 2023-10-06T03:09:34.339 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:34.340 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":154,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:33.377194+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:31.023983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:32.339528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:34.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcfc4067320 msgr2=0x7fcfc40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcfc4067320 0x7fcfc40697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00099b0 tx=0x7fcfd0016000 comp rx=0 tx=0).stop 2023-10-06T03:09:34.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 msgr2=0x7fcfe01501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 0x7fcfe01501e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00cd30 tx=0x7fcfdc001af0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 shutdown_connections 2023-10-06T03:09:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcfc4067320 0x7fcfc40697d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcfe0159890 0x7fcfe01501e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfe015db10 0x7fcfe00c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 --2- 172.21.15.92:0/4291388219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe0159c60 0x7fcfe00cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 >> 172.21.15.92:0/4291388219 conn(0x7fcfe0068e20 msgr2=0x7fcfe01570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:34.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 shutdown_connections 2023-10-06T03:09:34.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.339+0000 7fcfe80f8700 1 -- 172.21.15.92:0/4291388219 wait complete. 2023-10-06T03:09:34.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:34.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.467+0000 7f6d07fff700 1 Processor -- start 2023-10-06T03:09:34.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.467+0000 7f6d07fff700 1 -- start start 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d0815a7a0 0x7f6d080b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d07fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080b5b40 con 0x7f6d08159cc0 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d07fff700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d080b5e20 con 0x7f6d080cf6c0 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d07fff700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d080ce260 con 0x7f6d0815a7a0 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d077fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d0815a7a0 0x7f6d080b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d06ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6d08159cc0 0x7f6d080b2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44790/0 (socket says 172.21.15.92:44790) 2023-10-06T03:09:34.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d06ffd700 1 -- 172.21.15.92:0/813996023 learned_addr learned my addr 172.21.15.92:0/813996023 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d077fe700 1 -- 172.21.15.92:0/813996023 >> v1:172.21.15.136:6789/0 conn(0x7f6d080cf6c0 legacy=0x7f6d0815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d077fe700 1 -- 172.21.15.92:0/813996023 >> v1:172.21.15.92:6789/0 conn(0x7f6d08159cc0 legacy=0x7f6d080b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d05ffb700 1 -- 172.21.15.92:0/813996023 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229539747 0 0) 0x7f6d080b5b40 con 0x7f6d08159cc0 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.468+0000 7f6d077fe700 1 -- 172.21.15.92:0/813996023 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080b6100 con 0x7f6d0815a7a0 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d077fe700 1 --2- 172.21.15.92:0/813996023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d0815a7a0 0x7f6d080b5530 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009e60 tx=0x7f6d00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7507e534de94209c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d05ffb700 1 -- 172.21.15.92:0/813996023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000e070 con 0x7f6d0815a7a0 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d05ffb700 1 -- 172.21.15.92:0/813996023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0000bbf0 con 0x7f6d0815a7a0 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d05ffb700 1 -- 172.21.15.92:0/813996023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00012430 con 0x7f6d0815a7a0 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 -- 172.21.15.92:0/813996023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d0815a7a0 msgr2=0x7f6d080b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 --2- 172.21.15.92:0/813996023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d0815a7a0 0x7f6d080b5530 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009e60 tx=0x7f6d00009780 comp rx=0 tx=0).stop 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 -- 172.21.15.92:0/813996023 shutdown_connections 2023-10-06T03:09:34.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 --2- 172.21.15.92:0/813996023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d0815a7a0 0x7f6d080b5530 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 -- 172.21.15.92:0/813996023 >> 172.21.15.92:0/813996023 conn(0x7f6d080d0560 msgr2=0x7f6d080d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:34.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 -- 172.21.15.92:0/813996023 shutdown_connections 2023-10-06T03:09:34.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.469+0000 7f6d07fff700 1 -- 172.21.15.92:0/813996023 wait complete. 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.470+0000 7f6d07fff700 1 Processor -- start 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.470+0000 7f6d07fff700 1 -- start start 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d07fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d080cf6c0 0x7f6d08156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d07fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 0x7f6d08158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d07fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d0815a7a0 0x7f6d08168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d07fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0815d650 con 0x7f6d08159cc0 2023-10-06T03:09:34.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d07fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0815d930 con 0x7f6d0815a7a0 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d07fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0815dc10 con 0x7f6d080cf6c0 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 0x7f6d08158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 0x7f6d08158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34562/0 (socket says 172.21.15.92:34562) 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 -- 172.21.15.92:0/1483393112 learned_addr learned my addr 172.21.15.92:0/1483393112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 -- 172.21.15.92:0/1483393112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d080cf6c0 msgr2=0x7f6d08156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d080cf6c0 0x7f6d08156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 -- 172.21.15.92:0/1483393112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d0815a7a0 msgr2=0x7f6d08168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d0815a7a0 0x7f6d08168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.471+0000 7f6d067fc700 1 -- 172.21.15.92:0/1483393112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00009a20 con 0x7f6d08159cc0 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.472+0000 7f6d067fc700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 0x7f6d08158a90 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00b5e0 tx=0x7f6cfc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.472+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00f510 con 0x7f6d08159cc0 2023-10-06T03:09:34.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.472+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00f690 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.472+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0815def0 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.472+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00fa50 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.472+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0815d1e0 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.473+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d08159210 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.473+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6cfc01a400 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.474+0000 7f6ceeffd700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6cf4067320 0x7f6cf40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.474+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6cfc084970 con 0x7f6d08159cc0 2023-10-06T03:09:34.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.474+0000 7f6d06ffd700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6cf4067320 0x7f6cf40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.474+0000 7f6d06ffd700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6cf4067320 0x7f6cf40697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8001640 tx=0x7f6cf8007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.478+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cfc051520 con 0x7f6d08159cc0 2023-10-06T03:09:34.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.659+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d0804c3c0 con 0x7f6d08159cc0 2023-10-06T03:09:34.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.660+0000 7f6ceeffd700 1 -- 172.21.15.92:0/1483393112 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f6cfc05a030 con 0x7f6d08159cc0 2023-10-06T03:09:34.662 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:34.662 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":154,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:33.377194+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:31.023983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:32.339528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6cf4067320 msgr2=0x7f6cf40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6cf4067320 0x7f6cf40697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8001640 tx=0x7f6cf8007a40 comp rx=0 tx=0).stop 2023-10-06T03:09:34.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 msgr2=0x7f6d08158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 0x7f6d08158a90 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00b5e0 tx=0x7f6cfc0090f0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 shutdown_connections 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6cf4067320 0x7f6cf40697d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d0815a7a0 0x7f6d08168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d080cf6c0 0x7f6d08156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 --2- 172.21.15.92:0/1483393112 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d08159cc0 0x7f6d08158a90 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 >> 172.21.15.92:0/1483393112 conn(0x7f6d080d0560 msgr2=0x7f6d08153640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 shutdown_connections 2023-10-06T03:09:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.662+0000 7f6d07fff700 1 -- 172.21.15.92:0/1483393112 wait complete. 2023-10-06T03:09:34.676 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:09:34.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.792+0000 7f8efee29700 1 Processor -- start 2023-10-06T03:09:34.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.792+0000 7f8efee29700 1 -- start start 2023-10-06T03:09:34.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efee29700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8ef8159890 0x7f8ef815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efee29700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 0x7f8ef815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efee29700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef815a160 con 0x7f8ef815db10 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efee29700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef80d13a0 con 0x7f8ef8159890 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efee29700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef815e140 con 0x7f8ef8159c60 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efd3c6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8ef815db10 0x7f8ef8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44792/0 (socket says 172.21.15.92:44792) 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efd3c6700 1 -- 172.21.15.92:0/2627951667 learned_addr learned my addr 172.21.15.92:0/2627951667 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8efcbc5700 1 --2- 172.21.15.92:0/2627951667 >> v2:172.21.15.136:3300/0 conn(0x7f8ef8159890 0x7f8ef815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 0x7f8ef815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265811193 0 0) 0x7f8ef815a160 con 0x7f8ef815db10 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee4003570 con 0x7f8ef815db10 2023-10-06T03:09:34.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069586014 0 0) 0x7f8ee4003570 con 0x7f8ef815db10 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 msgr2=0x7f8ef815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 0x7f8ef815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 >> v2:172.21.15.136:3300/0 conn(0x7f8ef8159890 msgr2=0x7f8ef815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 --2- 172.21.15.92:0/2627951667 >> v2:172.21.15.136:3300/0 conn(0x7f8ef8159890 0x7f8ef815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef815a3d0 con 0x7f8ef815db10 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8ee0001470 con 0x7f8ef815db10 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 >> v1:172.21.15.92:6789/0 conn(0x7f8ef815db10 legacy=0x7f8ef8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 0x7f8ee4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4003fe0 0x7f8ee4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004d50 0x7f8ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80d13a0 con 0x7f8ee4003fe0 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef815e140 con 0x7f8ee4004d50 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.793+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee400ba90 con 0x7f8ef8159c60 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4003fe0 0x7f8ee4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8efcbc5700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 0x7f8ee4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8efd3c6700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004d50 0x7f8ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 -- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 msgr2=0x7f8ee4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ef8159c60 0x7f8ee4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 -- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004d50 msgr2=0x7f8ee40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004d50 0x7f8ee40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 -- 172.21.15.92:0/2627951667 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee4003570 con 0x7f8ee4003fe0 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8efd3c6700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004d50 0x7f8ee40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4003fe0 0x7f8ee4004810 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8ef815a090 tx=0x7f8ee800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4aaa93e6df3a442 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800ea30 con 0x7f8ee4003fe0 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee800f040 con 0x7f8ee4003fe0 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.794+0000 7f8ef77fe700 1 -- 172.21.15.92:0/2627951667 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800ede0 con 0x7f8ee4003fe0 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efee29700 1 -- 172.21.15.92:0/2627951667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4003fe0 msgr2=0x7f8ee4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efee29700 1 --2- 172.21.15.92:0/2627951667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4003fe0 0x7f8ee4004810 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8ef815a090 tx=0x7f8ee800b040 comp rx=0 tx=0).stop 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efd3c6700 1 -- 172.21.15.92:0/2627951667 reap_dead start 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efee29700 1 -- 172.21.15.92:0/2627951667 shutdown_connections 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efee29700 1 -- 172.21.15.92:0/2627951667 >> 172.21.15.92:0/2627951667 conn(0x7f8ef8068e20 msgr2=0x7f8ef80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efee29700 1 -- 172.21.15.92:0/2627951667 shutdown_connections 2023-10-06T03:09:34.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.795+0000 7f8efee29700 1 -- 172.21.15.92:0/2627951667 wait complete. 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.796+0000 7f8efee29700 1 Processor -- start 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.796+0000 7f8efee29700 1 -- start start 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003fe0 0x7f8ef80b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 0x7f8ef80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef80b1e90 0x7f8ef814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80cd890 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80cdb70 con 0x7f8ef80b1e90 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80cde50 con 0x7f8ee4003fe0 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 0x7f8ef80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 0x7f8ef80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34582/0 (socket says 172.21.15.92:34582) 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 -- 172.21.15.92:0/1074722045 learned_addr learned my addr 172.21.15.92:0/1074722045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efcbc5700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003fe0 0x7f8ef80b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efd3c6700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef80b1e90 0x7f8ef814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 -- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003fe0 msgr2=0x7f8ef80b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003fe0 0x7f8ef80b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 -- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef80b1e90 msgr2=0x7f8ef814bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef80b1e90 0x7f8ef814bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:34.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 -- 172.21.15.92:0/1074722045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80ce130 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efd3c6700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef80b1e90 0x7f8ef814bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef7fff700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 0x7f8ef80ce930 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800afd0 tx=0x7f8ee801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee801d420 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee801d580 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef80ce410 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee801d980 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.797+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef80cee70 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.798+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef80ce5d0 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.799+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8ee8003040 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.799+0000 7f8ef4ff9700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ed8067320 0x7f8ed80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:34.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.799+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ee8087210 con 0x7f8ef80cd0d0 2023-10-06T03:09:34.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.799+0000 7f8efcbc5700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ed8067320 0x7f8ed80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:34.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.799+0000 7f8efcbc5700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ed8067320 0x7f8ed80697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006f00 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:34.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:34.803+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ee8053c30 con 0x7f8ef80cd0d0 2023-10-06T03:09:35.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:35.032+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8ef804c3c0 con 0x7f8ef80cd0d0 2023-10-06T03:09:36.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.016+0000 7f8ef4ff9700 1 -- 172.21.15.92:0/1074722045 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v103) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8ee8057250 con 0x7f8ef80cd0d0 2023-10-06T03:09:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ed8067320 msgr2=0x7f8ed80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ed8067320 0x7f8ed80697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0006f00 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2023-10-06T03:09:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 msgr2=0x7f8ef80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 0x7f8ef80ce930 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800afd0 tx=0x7f8ee801a000 comp rx=0 tx=0).stop 2023-10-06T03:09:36.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 shutdown_connections 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ed8067320 0x7f8ed80697d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ef80b1e90 0x7f8ef814bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003fe0 0x7f8ef80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 --2- 172.21.15.92:0/1074722045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ef80cd0d0 0x7f8ef80ce930 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 >> 172.21.15.92:0/1074722045 conn(0x7f8ef8068e20 msgr2=0x7f8ef815cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 shutdown_connections 2023-10-06T03:09:36.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.019+0000 7f8efee29700 1 -- 172.21.15.92:0/1074722045 wait complete. 2023-10-06T03:09:36.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:09:36.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.157+0000 7fe496df5700 1 Processor -- start 2023-10-06T03:09:36.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.157+0000 7fe496df5700 1 -- start start 2023-10-06T03:09:36.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe496df5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe49006bcd0 0x7fe490140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe496df5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe49006c0a0 0x7fe490141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe496df5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe4900c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe496df5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49015c2a0 con 0x7fe49006bcd0 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe496df5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49015b690 con 0x7fe49006c0a0 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe496df5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49015b970 con 0x7fe4900c8e00 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe494b91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe49006bcd0 0x7fe490140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe494b91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe49006bcd0 0x7fe490140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34584/0 (socket says 172.21.15.92:34584) 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.158+0000 7fe494b91700 1 -- 172.21.15.92:0/2528824667 learned_addr learned my addr 172.21.15.92:0/2528824667 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe487fff700 1 --2- 172.21.15.92:0/2528824667 >> v2:172.21.15.136:3300/0 conn(0x7fe49006c0a0 0x7fe490141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe495392700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe4900c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe494b91700 1 -- 172.21.15.92:0/2528824667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 msgr2=0x7fe4900c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:36.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe494b91700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe4900c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe494b91700 1 -- 172.21.15.92:0/2528824667 >> v2:172.21.15.136:3300/0 conn(0x7fe49006c0a0 msgr2=0x7fe490141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe494b91700 1 --2- 172.21.15.92:0/2528824667 >> v2:172.21.15.136:3300/0 conn(0x7fe49006c0a0 0x7fe490141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe494b91700 1 -- 172.21.15.92:0/2528824667 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49015bc50 con 0x7fe49006bcd0 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe487fff700 1 --2- 172.21.15.92:0/2528824667 >> v2:172.21.15.136:3300/0 conn(0x7fe49006c0a0 0x7fe490141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe495392700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe4900c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe494b91700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006bcd0 0x7fe490140df0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe49006c6e0 tx=0x7fe48c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d109c9d3ef1f3a31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe4877fe700 1 -- 172.21.15.92:0/2528824667 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00b860 con 0x7fe49006bcd0 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe4877fe700 1 -- 172.21.15.92:0/2528824667 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c00b9c0 con 0x7fe49006bcd0 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe4877fe700 1 -- 172.21.15.92:0/2528824667 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c00bd60 con 0x7fe49006bcd0 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 -- 172.21.15.92:0/2528824667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006bcd0 msgr2=0x7fe490140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006bcd0 0x7fe490140df0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe49006c6e0 tx=0x7fe48c009250 comp rx=0 tx=0).stop 2023-10-06T03:09:36.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 -- 172.21.15.92:0/2528824667 shutdown_connections 2023-10-06T03:09:36.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe4900c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 --2- 172.21.15.92:0/2528824667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006bcd0 0x7fe490140df0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 --2- 172.21.15.92:0/2528824667 >> v2:172.21.15.136:3300/0 conn(0x7fe49006c0a0 0x7fe490141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 -- 172.21.15.92:0/2528824667 >> 172.21.15.92:0/2528824667 conn(0x7fe4900ce600 msgr2=0x7fe4900cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:36.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.159+0000 7fe496df5700 1 -- 172.21.15.92:0/2528824667 shutdown_connections 2023-10-06T03:09:36.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.160+0000 7fe496df5700 1 -- 172.21.15.92:0/2528824667 wait complete. 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 Processor -- start 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 -- start start 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe49006bcd0 0x7fe4900b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 0x7fe4900b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe49013c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900b36f0 con 0x7fe49006c0a0 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49013c7c0 con 0x7fe49006bcd0 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.161+0000 7fe496df5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900c6360 con 0x7fe4900c8e00 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 0x7fe4900b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 0x7fe4900b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34586/0 (socket says 172.21.15.92:34586) 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 -- 172.21.15.92:0/4282179794 learned_addr learned my addr 172.21.15.92:0/4282179794 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe494b91700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe49006bcd0 0x7fe4900b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe495392700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe49013c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 -- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 msgr2=0x7fe49013c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe49013c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 -- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe49006bcd0 msgr2=0x7fe4900b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe49006bcd0 0x7fe4900b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 -- 172.21.15.92:0/4282179794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c0094f0 con 0x7fe49006c0a0 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe487fff700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 0x7fe4900b31b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c8b0 tx=0x7fe480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480011600 con 0x7fe49006c0a0 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480011760 con 0x7fe49006c0a0 2023-10-06T03:09:36.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480011b00 con 0x7fe49006c0a0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4900c5750 con 0x7fe49006c0a0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.162+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4900c5d10 con 0x7fe49006c0a0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.163+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49004c3c0 con 0x7fe49006c0a0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.164+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe480018350 con 0x7fe49006c0a0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.164+0000 7fe484ff9700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe47c067320 0x7fe47c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.164+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe480084d00 con 0x7fe49006c0a0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.164+0000 7fe494b91700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe47c067320 0x7fe47c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.164+0000 7fe494b91700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe47c067320 0x7fe47c0697d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe49006c6e0 tx=0x7fe48c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:36.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.168+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4800551b0 con 0x7fe49006c0a0 2023-10-06T03:09:36.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:36.392+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe49006c9d0 con 0x7fe49006c0a0 2023-10-06T03:09:37.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.018+0000 7fe484ff9700 1 -- 172.21.15.92:0/4282179794 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v104) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe480054d40 con 0x7fe49006c0a0 2023-10-06T03:09:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe47c067320 msgr2=0x7fe47c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe47c067320 0x7fe47c0697d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe49006c6e0 tx=0x7fe48c017000 comp rx=0 tx=0).stop 2023-10-06T03:09:37.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 msgr2=0x7fe4900b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 0x7fe4900b31b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c8b0 tx=0x7fe480008040 comp rx=0 tx=0).stop 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 shutdown_connections 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe47c067320 0x7fe47c0697d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4900c8e00 0x7fe49013c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe49006bcd0 0x7fe4900b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 --2- 172.21.15.92:0/4282179794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe49006c0a0 0x7fe4900b31b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.020+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 >> 172.21.15.92:0/4282179794 conn(0x7fe4900ce600 msgr2=0x7fe4900cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.021+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 shutdown_connections 2023-10-06T03:09:37.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.021+0000 7fe496df5700 1 -- 172.21.15.92:0/4282179794 wait complete. 2023-10-06T03:09:37.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:37.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.152+0000 7f5cea5e3700 1 Processor -- start 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.152+0000 7f5cea5e3700 1 -- start start 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5cea5e3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ce4159890 0x7f5ce415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5cea5e3700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce415a160 con 0x7f5ce4159c60 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5cea5e3700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce40d13a0 con 0x7f5ce4069560 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5cea5e3700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce415e140 con 0x7f5ce4159890 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5ce3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ce4159890 0x7f5ce415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5ce37fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5ce4159c60 0x7f5ce415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44804/0 (socket says 172.21.15.92:44804) 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5ce37fe700 1 -- 172.21.15.92:0/738147944 learned_addr learned my addr 172.21.15.92:0/738147944 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.153+0000 7f5ce3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5ce4159890 0x7f5ce415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34596/0 (socket says 172.21.15.92:34596) 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce3fff700 1 -- 172.21.15.92:0/738147944 >> v1:172.21.15.136:6789/0 conn(0x7f5ce4069560 legacy=0x7f5ce40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:37.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce3fff700 1 -- 172.21.15.92:0/738147944 >> v1:172.21.15.92:6789/0 conn(0x7f5ce4159c60 legacy=0x7f5ce415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce3fff700 1 -- 172.21.15.92:0/738147944 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce415a3d0 con 0x7f5ce4159890 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce2ffd700 1 -- 172.21.15.92:0/738147944 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620918193 0 0) 0x7f5ce415a160 con 0x7f5ce4159c60 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce3fff700 1 --2- 172.21.15.92:0/738147944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4159890 0x7f5ce415abd0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40098e0 tx=0x7f5cd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26e6fb5f719206a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce2ffd700 1 -- 172.21.15.92:0/738147944 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4009e80 con 0x7f5ce4159890 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce2ffd700 1 -- 172.21.15.92:0/738147944 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd400b820 con 0x7f5ce4159890 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5ce2ffd700 1 -- 172.21.15.92:0/738147944 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400bba0 con 0x7f5ce4159890 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5cea5e3700 1 -- 172.21.15.92:0/738147944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4159890 msgr2=0x7f5ce415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/738147944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4159890 0x7f5ce415abd0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40098e0 tx=0x7f5cd40091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5cea5e3700 1 -- 172.21.15.92:0/738147944 shutdown_connections 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/738147944 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4159890 0x7f5ce415abd0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5cea5e3700 1 -- 172.21.15.92:0/738147944 >> 172.21.15.92:0/738147944 conn(0x7f5ce4068e20 msgr2=0x7f5ce40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:37.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.154+0000 7f5cea5e3700 1 -- 172.21.15.92:0/738147944 shutdown_connections 2023-10-06T03:09:37.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.155+0000 7f5cea5e3700 1 -- 172.21.15.92:0/738147944 wait complete. 2023-10-06T03:09:37.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 Processor -- start 2023-10-06T03:09:37.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 -- start start 2023-10-06T03:09:37.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 0x7f5ce414d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ce4159890 0x7f5ce415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ce4159c60 0x7f5ce40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b4850 con 0x7f5ce4069560 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b4ac0 con 0x7f5ce4159890 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.156+0000 7f5cea5e3700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40b4da0 con 0x7f5ce4159c60 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 0x7f5ce414d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 0x7f5ce414d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34604/0 (socket says 172.21.15.92:34604) 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 -- 172.21.15.92:0/1004187562 learned_addr learned my addr 172.21.15.92:0/1004187562 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce37fe700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ce4159890 0x7f5ce415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce8b80700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ce4159c60 0x7f5ce40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 -- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ce4159c60 msgr2=0x7f5ce40ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ce4159c60 0x7f5ce40ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 -- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ce4159890 msgr2=0x7f5ce415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ce4159890 0x7f5ce415d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 -- 172.21.15.92:0/1004187562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd4009420 con 0x7f5ce4069560 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce8b80700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ce4159c60 0x7f5ce40ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ce3fff700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 0x7f5ce414d3f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40030c0 tx=0x7f5cd4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd400ba60 con 0x7f5ce4069560 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd4020070 con 0x7f5ce4069560 2023-10-06T03:09:37.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce40b5080 con 0x7f5ce4069560 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd401b6a0 con 0x7f5ce4069560 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.157+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce414d930 con 0x7f5ce4069560 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.158+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce40b55a0 con 0x7f5ce4069560 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.159+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5cd4022030 con 0x7f5ce4069560 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.159+0000 7f5ccbfff700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cd0067320 0x7f5cd00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.159+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5cd4086c90 con 0x7f5ce4069560 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.159+0000 7f5ce37fe700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cd0067320 0x7f5cd00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.159+0000 7f5ce37fe700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cd0067320 0x7f5cd00697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40caa20 tx=0x7f5cd8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.163+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ce40b55a0 con 0x7f5ce4069560 2023-10-06T03:09:37.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.386+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ce40cc820 con 0x7f5ce4069560 2023-10-06T03:09:37.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.387+0000 7f5ccbfff700 1 -- 172.21.15.92:0/1004187562 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f5cd40538d0 con 0x7f5ce4069560 2023-10-06T03:09:37.388 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:37.388 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:33.377515+0000","modified":"2023-10-06T03:09:37.016548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2005327996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2005327996},{"type":"v1","addr":"172.21.15.92:6837","nonce":2005327996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cd0067320 msgr2=0x7f5cd00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cd0067320 0x7f5cd00697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40caa20 tx=0x7f5cd8007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 msgr2=0x7f5ce414d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 0x7f5ce414d3f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40030c0 tx=0x7f5cd4016000 comp rx=0 tx=0).stop 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 shutdown_connections 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5cd0067320 0x7f5cd00697d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ce4069560 0x7f5ce414d3f0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ce4159c60 0x7f5ce40ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 --2- 172.21.15.92:0/1004187562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ce4159890 0x7f5ce415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 >> 172.21.15.92:0/1004187562 conn(0x7f5ce4068e20 msgr2=0x7f5ce40cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 shutdown_connections 2023-10-06T03:09:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.389+0000 7f5cea5e3700 1 -- 172.21.15.92:0/1004187562 wait complete. 2023-10-06T03:09:37.392 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 104 2023-10-06T03:09:37.403 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:37.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.515+0000 7fa91191c700 1 Processor -- start 2023-10-06T03:09:37.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.515+0000 7fa91191c700 1 -- start start 2023-10-06T03:09:37.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa91191c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa90c159890 0x7fa90c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa91191c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c15a160 con 0x7fa90c159c60 2023-10-06T03:09:37.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa91191c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90c0d13a0 con 0x7fa90c069560 2023-10-06T03:09:37.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa91191c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90c15e140 con 0x7fa90c159890 2023-10-06T03:09:37.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa90affd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa90c159890 0x7fa90c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa9027fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa90c159c60 0x7fa90c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44808/0 (socket says 172.21.15.92:44808) 2023-10-06T03:09:37.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.516+0000 7fa9027fc700 1 -- 172.21.15.92:0/2086686233 learned_addr learned my addr 172.21.15.92:0/2086686233 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:37.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90affd700 1 -- 172.21.15.92:0/2086686233 >> v1:172.21.15.136:6789/0 conn(0x7fa90c069560 legacy=0x7fa90c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:37.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90affd700 1 -- 172.21.15.92:0/2086686233 >> v1:172.21.15.92:6789/0 conn(0x7fa90c159c60 legacy=0x7fa90c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90a7fc700 1 -- 172.21.15.92:0/2086686233 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029021225 0 0) 0x7fa90c15a160 con 0x7fa90c159c60 2023-10-06T03:09:37.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90affd700 1 -- 172.21.15.92:0/2086686233 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c15a3d0 con 0x7fa90c159890 2023-10-06T03:09:37.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90affd700 1 --2- 172.21.15.92:0/2086686233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15abd0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098e0 tx=0x7fa8fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2f7c183e3d86d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90a7fc700 1 -- 172.21.15.92:0/2086686233 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc009e80 con 0x7fa90c159890 2023-10-06T03:09:37.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90a7fc700 1 -- 172.21.15.92:0/2086686233 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00b820 con 0x7fa90c159890 2023-10-06T03:09:37.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa90a7fc700 1 -- 172.21.15.92:0/2086686233 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00bba0 con 0x7fa90c159890 2023-10-06T03:09:37.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa91191c700 1 -- 172.21.15.92:0/2086686233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 msgr2=0x7fa90c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa91191c700 1 --2- 172.21.15.92:0/2086686233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15abd0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0098e0 tx=0x7fa8fc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa91191c700 1 -- 172.21.15.92:0/2086686233 shutdown_connections 2023-10-06T03:09:37.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa91191c700 1 --2- 172.21.15.92:0/2086686233 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15abd0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa91191c700 1 -- 172.21.15.92:0/2086686233 >> 172.21.15.92:0/2086686233 conn(0x7fa90c068e20 msgr2=0x7fa90c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.517+0000 7fa91191c700 1 -- 172.21.15.92:0/2086686233 shutdown_connections 2023-10-06T03:09:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.518+0000 7fa91191c700 1 -- 172.21.15.92:0/2086686233 wait complete. 2023-10-06T03:09:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.519+0000 7fa91191c700 1 Processor -- start 2023-10-06T03:09:37.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.519+0000 7fa91191c700 1 -- start start 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.519+0000 7fa91191c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa90c069560 0x7fa90c14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.519+0000 7fa91191c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.519+0000 7fa91191c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa90c159c60 0x7fa90c151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.519+0000 7fa91191c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c14db50 con 0x7fa90c159890 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa91191c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0b5a40 con 0x7fa90c159c60 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa91191c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c15ca70 con 0x7fa90c069560 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34614/0 (socket says 172.21.15.92:34614) 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 -- 172.21.15.92:0/4020676187 learned_addr learned my addr 172.21.15.92:0/4020676187 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:37.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa90affd700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa90c069560 0x7fa90c14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa90b7fe700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa90c159c60 0x7fa90c151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 -- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa90c069560 msgr2=0x7fa90c14d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa90c069560 0x7fa90c14d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 -- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa90c159c60 msgr2=0x7fa90c151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa90c159c60 0x7fa90c151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 -- 172.21.15.92:0/4020676187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009420 con 0x7fa90c159890 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa90b7fe700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa90c159c60 0x7fa90c151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9027fc700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15c530 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c7e0 tx=0x7fa8f4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4005b30 con 0x7fa90c159890 2023-10-06T03:09:37.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f4005c90 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c15d600 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4007300 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.520+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c0b5110 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.521+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c0b5bf0 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.522+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa8f401b400 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.522+0000 7fa9037fe700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8ec067320 0x7fa8ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.522+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa8f4084950 con 0x7fa90c159890 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.522+0000 7fa90affd700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8ec067320 0x7fa8ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.523+0000 7fa90affd700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8ec067320 0x7fa8ec0697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0030c0 tx=0x7fa8fc016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.526+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa90c0b5bf0 con 0x7fa90c159890 2023-10-06T03:09:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.712+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa90c0b5bf0 con 0x7fa90c159890 2023-10-06T03:09:37.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.713+0000 7fa9037fe700 1 -- 172.21.15.92:0/4020676187 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fa90c0b5bf0 con 0x7fa90c159890 2023-10-06T03:09:37.715 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:37.716 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":155,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:34.995208+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:31.023983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:32.339528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8ec067320 msgr2=0x7fa8ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8ec067320 0x7fa8ec0697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0030c0 tx=0x7fa8fc016000 comp rx=0 tx=0).stop 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 msgr2=0x7fa90c15c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15c530 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400c7e0 tx=0x7fa8f4007df0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 shutdown_connections 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa8ec067320 0x7fa8ec0697d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa90c159c60 0x7fa90c151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa90c069560 0x7fa90c14d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 --2- 172.21.15.92:0/4020676187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa90c159890 0x7fa90c15c530 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 >> 172.21.15.92:0/4020676187 conn(0x7fa90c068e20 msgr2=0x7fa90c0b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 shutdown_connections 2023-10-06T03:09:37.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.716+0000 7fa91191c700 1 -- 172.21.15.92:0/4020676187 wait complete. 2023-10-06T03:09:37.730 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:09:37.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:09:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 Processor -- start 2023-10-06T03:09:37.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 -- start start 2023-10-06T03:09:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fad7006c0a0 0x7fad700caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 0x7fad7015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7015d7e0 con 0x7fad7006bcd0 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7015dac0 con 0x7fad7006c0a0 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.843+0000 7fad76883700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015aa30 con 0x7fad7015df50 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad7006bcd0 0x7fad7015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44812/0 (socket says 172.21.15.92:44812) 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6ffff700 1 -- 172.21.15.92:0/2801092980 learned_addr learned my addr 172.21.15.92:0/2801092980 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731801334 0 0) 0x7fad7015d7e0 con 0x7fad7006bcd0 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6f7fe700 1 --2- 172.21.15.92:0/2801092980 >> v2:172.21.15.136:3300/0 conn(0x7fad7006c0a0 0x7fad700caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad74e20700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 0x7fad7015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad58003570 con 0x7fad7006bcd0 2023-10-06T03:09:37.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351717827 0 0) 0x7fad58003570 con 0x7fad7006bcd0 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 msgr2=0x7fad7015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 0x7fad7015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 >> v2:172.21.15.136:3300/0 conn(0x7fad7006c0a0 msgr2=0x7fad700caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 --2- 172.21.15.92:0/2801092980 >> v2:172.21.15.136:3300/0 conn(0x7fad7006c0a0 0x7fad700caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7015ad10 con 0x7fad7006bcd0 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fad60001470 con 0x7fad7006bcd0 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 >> v1:172.21.15.92:6789/0 conn(0x7fad7006bcd0 legacy=0x7fad7015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 0x7fad58003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad58003fe0 0x7fad58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015dac0 con 0x7fad58004d50 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015aa30 con 0x7fad58003fe0 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5800bb40 con 0x7fad7015df50 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad74e20700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6ffff700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 0x7fad58003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.844+0000 7fad6f7fe700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad58003fe0 0x7fad58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 -- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 msgr2=0x7fad58003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:37.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7015df50 0x7fad58003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 -- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad58003fe0 msgr2=0x7fad58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad58003fe0 0x7fad58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 -- 172.21.15.92:0/2801092980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58003570 con 0x7fad58004d50 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad580051c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fad680094d0 tx=0x7fad6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae75c85ab39adfc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800ea90 con 0x7fad58004d50 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6800f040 con 0x7fad58004d50 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad6effd700 1 -- 172.21.15.92:0/2801092980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800ee20 con 0x7fad58004d50 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad76883700 1 -- 172.21.15.92:0/2801092980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 msgr2=0x7fad580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad76883700 1 --2- 172.21.15.92:0/2801092980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad580051c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fad680094d0 tx=0x7fad6800b040 comp rx=0 tx=0).stop 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad74e20700 1 -- 172.21.15.92:0/2801092980 reap_dead start 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad76883700 1 -- 172.21.15.92:0/2801092980 shutdown_connections 2023-10-06T03:09:37.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad76883700 1 -- 172.21.15.92:0/2801092980 >> 172.21.15.92:0/2801092980 conn(0x7fad700ce600 msgr2=0x7fad700cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:37.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad76883700 1 -- 172.21.15.92:0/2801092980 shutdown_connections 2023-10-06T03:09:37.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.845+0000 7fad76883700 1 -- 172.21.15.92:0/2801092980 wait complete. 2023-10-06T03:09:37.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.846+0000 7fad76883700 1 Processor -- start 2023-10-06T03:09:37.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 -- start start 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad7016b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7016aef0 0x7fad700b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7016d1a0 0x7fad70158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700b3470 con 0x7fad58004d50 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7016b360 con 0x7fad7016aef0 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad76883700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700b3f20 con 0x7fad7016d1a0 2023-10-06T03:09:37.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad7016b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad6ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad7016b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34642/0 (socket says 172.21.15.92:34642) 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.847+0000 7fad6ffff700 1 -- 172.21.15.92:0/2972825617 learned_addr learned my addr 172.21.15.92:0/2972825617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6f7fe700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7016aef0 0x7fad700b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad74e20700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7016d1a0 0x7fad70158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6ffff700 1 -- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7016d1a0 msgr2=0x7fad70158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6ffff700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7016d1a0 0x7fad70158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6ffff700 1 -- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7016aef0 msgr2=0x7fad700b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6ffff700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7016aef0 0x7fad700b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6ffff700 1 -- 172.21.15.92:0/2972825617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700b14d0 con 0x7fad58004d50 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad6ffff700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad7016b560 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fad600079e0 tx=0x7fad60001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60018030 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6000f8a0 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6000fc40 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad700b3680 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.848+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad700b3c40 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.849+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad700648d0 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.849+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad6001b070 con 0x7fad58004d50 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.850+0000 7fad57fff700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:37.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.850+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad60019070 con 0x7fad58004d50 2023-10-06T03:09:37.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.850+0000 7fad6f7fe700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:37.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.850+0000 7fad6f7fe700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad64006ed0 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:37.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:37.854+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad60051910 con 0x7fad58004d50 2023-10-06T03:09:38.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.034+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad7004c3c0 con 0x7fad5c067320 2023-10-06T03:09:38.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.043+0000 7fad57fff700 1 -- 172.21.15.92:0/2972825617 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311885 (secure 0 0 0) 0x7fad7004c3c0 con 0x7fad5c067320 2023-10-06T03:09:38.048 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:38.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 msgr2=0x7fad5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad64006ed0 tx=0x7fad64008040 comp rx=0 tx=0).stop 2023-10-06T03:09:38.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 msgr2=0x7fad7016b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad7016b560 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fad600079e0 tx=0x7fad60001e90 comp rx=0 tx=0).stop 2023-10-06T03:09:38.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 shutdown_connections 2023-10-06T03:09:38.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad5c067320 0x7fad5c0697d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad58004d50 0x7fad7016b560 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7016d1a0 0x7fad70158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 --2- 172.21.15.92:0/2972825617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7016aef0 0x7fad700b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 >> 172.21.15.92:0/2972825617 conn(0x7fad700ce600 msgr2=0x7fad70157180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 shutdown_connections 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.046+0000 7fad76883700 1 -- 172.21.15.92:0/2972825617 wait complete. 2023-10-06T03:09:38.052 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:09:38.060 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2023-10-06T03:09:37.279329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427424,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547296,"statfs":{"total":773094113280,"available":770608431104,"internally_reserved":0,"allocated":16744448,"data_stored":8521220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.307253"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000228+0000","last_change":"2023-10-06T03:09:34.000228+0000","last_active":"2023-10-06T03:09:34.000228+0000","last_peered":"2023-10-06T03:09:34.000228+0000","last_clean":"2023-10-06T03:09:34.000228+0000","last_became_active":"2023-10-06T03:09:33.999962+0000","last_became_peered":"2023-10-06T03:09:33.999962+0000","last_unstale":"2023-10-06T03:09:34.000228+0000","last_undegraded":"2023-10-06T03:09:34.000228+0000","last_fullsized":"2023-10-06T03:09:34.000228+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:59.329077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003591+0000","last_change":"2023-10-06T03:09:34.003591+0000","last_active":"2023-10-06T03:09:34.003591+0000","last_peered":"2023-10-06T03:09:34.003591+0000","last_clean":"2023-10-06T03:09:34.003591+0000","last_became_active":"2023-10-06T03:09:34.003002+0000","last_became_peered":"2023-10-06T03:09:34.003002+0000","last_unstale":"2023-10-06T03:09:34.003591+0000","last_undegraded":"2023-10-06T03:09:34.003591+0000","last_fullsized":"2023-10-06T03:09:34.003591+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:48.232570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002441+0000","last_change":"2023-10-06T03:09:34.002441+0000","last_active":"2023-10-06T03:09:34.002441+0000","last_peered":"2023-10-06T03:09:34.002441+0000","last_clean":"2023-10-06T03:09:34.002441+0000","last_became_active":"2023-10-06T03:09:34.002150+0000","last_became_peered":"2023-10-06T03:09:34.002150+0000","last_unstale":"2023-10-06T03:09:34.002441+0000","last_undegraded":"2023-10-06T03:09:34.002441+0000","last_fullsized":"2023-10-06T03:09:34.002441+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:52.259726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000566+0000","last_change":"2023-10-06T03:09:34.000566+0000","last_active":"2023-10-06T03:09:34.000566+0000","last_peered":"2023-10-06T03:09:34.000566+0000","last_clean":"2023-10-06T03:09:34.000566+0000","last_became_active":"2023-10-06T03:09:34.000281+0000","last_became_peered":"2023-10-06T03:09:34.000281+0000","last_unstale":"2023-10-06T03:09:34.000566+0000","last_undegraded":"2023-10-06T03:09:34.000566+0000","last_fullsized":"2023-10-06T03:09:34.000566+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:18.520193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005665+0000","last_change":"2023-10-06T03:09:34.005665+0000","last_active":"2023-10-06T03:09:34.005665+0000","last_peered":"2023-10-06T03:09:34.005665+0000","last_clean":"2023-10-06T03:09:34.005665+0000","last_became_active":"2023-10-06T03:09:34.005397+0000","last_became_peered":"2023-10-06T03:09:34.005397+0000","last_unstale":"2023-10-06T03:09:34.005665+0000","last_undegraded":"2023-10-06T03:09:34.005665+0000","last_fullsized":"2023-10-06T03:09:34.005665+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:48.925475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.004897+0000","last_change":"2023-10-06T03:09:34.004897+0000","last_active":"2023-10-06T03:09:34.004897+0000","last_peered":"2023-10-06T03:09:34.004897+0000","last_clean":"2023-10-06T03:09:34.004897+0000","last_became_active":"2023-10-06T03:09:34.004599+0000","last_became_peered":"2023-10-06T03:09:34.004599+0000","last_unstale":"2023-10-06T03:09:34.004897+0000","last_undegraded":"2023-10-06T03:09:34.004897+0000","last_fullsized":"2023-10-06T03:09:34.004897+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:33.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999896+0000","last_change":"2023-10-06T03:09:33.999896+0000","last_active":"2023-10-06T03:09:33.999896+0000","last_peered":"2023-10-06T03:09:33.999896+0000","last_clean":"2023-10-06T03:09:33.999896+0000","last_became_active":"2023-10-06T03:09:33.999683+0000","last_became_peered":"2023-10-06T03:09:33.999683+0000","last_unstale":"2023-10-06T03:09:33.999896+0000","last_undegraded":"2023-10-06T03:09:33.999896+0000","last_fullsized":"2023-10-06T03:09:33.999896+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:19.754683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007086+0000","last_change":"2023-10-06T03:09:34.007086+0000","last_active":"2023-10-06T03:09:34.007086+0000","last_peered":"2023-10-06T03:09:34.007086+0000","last_clean":"2023-10-06T03:09:34.007086+0000","last_became_active":"2023-10-06T03:09:34.006827+0000","last_became_peered":"2023-10-06T03:09:34.006827+0000","last_unstale":"2023-10-06T03:09:34.007086+0000","last_undegraded":"2023-10-06T03:09:34.007086+0000","last_fullsized":"2023-10-06T03:09:34.007086+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:55.616344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.994330+0000","last_change":"2023-10-06T03:09:33.994330+0000","last_active":"2023-10-06T03:09:33.994330+0000","last_peered":"2023-10-06T03:09:33.994330+0000","last_clean":"2023-10-06T03:09:33.994330+0000","last_became_active":"2023-10-06T03:09:33.994049+0000","last_became_peered":"2023-10-06T03:09:33.994049+0000","last_unstale":"2023-10-06T03:09:33.994330+0000","last_undegraded":"2023-10-06T03:09:33.994330+0000","last_fullsized":"2023-10-06T03:09:33.994330+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:42.460007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003280+0000","last_change":"2023-10-06T03:09:34.003280+0000","last_active":"2023-10-06T03:09:34.003280+0000","last_peered":"2023-10-06T03:09:34.003280+0000","last_clean":"2023-10-06T03:09:34.003280+0000","last_became_active":"2023-10-06T03:09:34.002891+0000","last_became_peered":"2023-10-06T03:09:34.002891+0000","last_unstale":"2023-10-06T03:09:34.003280+0000","last_undegraded":"2023-10-06T03:09:34.003280+0000","last_fullsized":"2023-10-06T03:09:34.003280+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:08.184156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007161+0000","last_change":"2023-10-06T03:09:34.007161+0000","last_active":"2023-10-06T03:09:34.007161+0000","last_peered":"2023-10-06T03:09:34.007161+0000","last_clean":"2023-10-06T03:09:34.007161+0000","last_became_active":"2023-10-06T03:09:34.006921+0000","last_became_peered":"2023-10-06T03:09:34.006921+0000","last_unstale":"2023-10-06T03:09:34.007161+0000","last_undegraded":"2023-10-06T03:09:34.007161+0000","last_fullsized":"2023-10-06T03:09:34.007161+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:38.584278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999351+0000","last_change":"2023-10-06T03:09:33.999351+0000","last_active":"2023-10-06T03:09:33.999351+0000","last_peered":"2023-10-06T03:09:33.999351+0000","last_clean":"2023-10-06T03:09:33.999351+0000","last_became_active":"2023-10-06T03:09:33.999093+0000","last_became_peered":"2023-10-06T03:09:33.999093+0000","last_unstale":"2023-10-06T03:09:33.999351+0000","last_undegraded":"2023-10-06T03:09:33.999351+0000","last_fullsized":"2023-10-06T03:09:33.999351+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:37.858107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.012910+0000","last_change":"2023-10-06T03:09:34.012910+0000","last_active":"2023-10-06T03:09:34.012910+0000","last_peered":"2023-10-06T03:09:34.012910+0000","last_clean":"2023-10-06T03:09:34.012910+0000","last_became_active":"2023-10-06T03:09:34.012607+0000","last_became_peered":"2023-10-06T03:09:34.012607+0000","last_unstale":"2023-10-06T03:09:34.012910+0000","last_undegraded":"2023-10-06T03:09:34.012910+0000","last_fullsized":"2023-10-06T03:09:34.012910+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:48.232570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999868+0000","last_change":"2023-10-06T03:09:33.999868+0000","last_active":"2023-10-06T03:09:33.999868+0000","last_peered":"2023-10-06T03:09:33.999868+0000","last_clean":"2023-10-06T03:09:33.999868+0000","last_became_active":"2023-10-06T03:09:33.999608+0000","last_became_peered":"2023-10-06T03:09:33.999608+0000","last_unstale":"2023-10-06T03:09:33.999868+0000","last_undegraded":"2023-10-06T03:09:33.999868+0000","last_fullsized":"2023-10-06T03:09:33.999868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:39.383082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999544+0000","last_change":"2023-10-06T03:09:33.999544+0000","last_active":"2023-10-06T03:09:33.999544+0000","last_peered":"2023-10-06T03:09:33.999544+0000","last_clean":"2023-10-06T03:09:33.999544+0000","last_became_active":"2023-10-06T03:09:33.999253+0000","last_became_peered":"2023-10-06T03:09:33.999253+0000","last_unstale":"2023-10-06T03:09:33.999544+0000","last_undegraded":"2023-10-06T03:09:33.999544+0000","last_fullsized":"2023-10-06T03:09:33.999544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:28.754607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003746+0000","last_change":"2023-10-06T03:09:34.003746+0000","last_active":"2023-10-06T03:09:34.003746+0000","last_peered":"2023-10-06T03:09:34.003746+0000","last_clean":"2023-10-06T03:09:34.003746+0000","last_became_active":"2023-10-06T03:09:34.003476+0000","last_became_peered":"2023-10-06T03:09:34.003476+0000","last_unstale":"2023-10-06T03:09:34.003746+0000","last_undegraded":"2023-10-06T03:09:34.003746+0000","last_fullsized":"2023-10-06T03:09:34.003746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:01.566725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003002+0000","last_change":"2023-10-06T03:09:34.003002+0000","last_active":"2023-10-06T03:09:34.003002+0000","last_peered":"2023-10-06T03:09:34.003002+0000","last_clean":"2023-10-06T03:09:34.003002+0000","last_became_active":"2023-10-06T03:09:34.002756+0000","last_became_peered":"2023-10-06T03:09:34.002756+0000","last_unstale":"2023-10-06T03:09:34.003002+0000","last_undegraded":"2023-10-06T03:09:34.003002+0000","last_fullsized":"2023-10-06T03:09:34.003002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:59.754210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000960+0000","last_change":"2023-10-06T03:09:34.000960+0000","last_active":"2023-10-06T03:09:34.000960+0000","last_peered":"2023-10-06T03:09:34.000960+0000","last_clean":"2023-10-06T03:09:34.000960+0000","last_became_active":"2023-10-06T03:09:34.000744+0000","last_became_peered":"2023-10-06T03:09:34.000744+0000","last_unstale":"2023-10-06T03:09:34.000960+0000","last_undegraded":"2023-10-06T03:09:34.000960+0000","last_fullsized":"2023-10-06T03:09:34.000960+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:52.813869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.998565+0000","last_change":"2023-10-06T03:09:33.998565+0000","last_active":"2023-10-06T03:09:33.998565+0000","last_peered":"2023-10-06T03:09:33.998565+0000","last_clean":"2023-10-06T03:09:33.998565+0000","last_became_active":"2023-10-06T03:09:33.998231+0000","last_became_peered":"2023-10-06T03:09:33.998231+0000","last_unstale":"2023-10-06T03:09:33.998565+0000","last_undegraded":"2023-10-06T03:09:33.998565+0000","last_fullsized":"2023-10-06T03:09:33.998565+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:19.282888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007848+0000","last_change":"2023-10-06T03:09:34.007848+0000","last_active":"2023-10-06T03:09:34.007848+0000","last_peered":"2023-10-06T03:09:34.007848+0000","last_clean":"2023-10-06T03:09:34.007848+0000","last_became_active":"2023-10-06T03:09:34.007554+0000","last_became_peered":"2023-10-06T03:09:34.007554+0000","last_unstale":"2023-10-06T03:09:34.007848+0000","last_undegraded":"2023-10-06T03:09:34.007848+0000","last_fullsized":"2023-10-06T03:09:34.007848+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:08.650534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.006024+0000","last_change":"2023-10-06T03:09:34.006024+0000","last_active":"2023-10-06T03:09:34.006024+0000","last_peered":"2023-10-06T03:09:34.006024+0000","last_clean":"2023-10-06T03:09:34.006024+0000","last_became_active":"2023-10-06T03:09:34.002682+0000","last_became_peered":"2023-10-06T03:09:34.002682+0000","last_unstale":"2023-10-06T03:09:34.006024+0000","last_undegraded":"2023-10-06T03:09:34.006024+0000","last_fullsized":"2023-10-06T03:09:34.006024+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:28.413431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999698+0000","last_change":"2023-10-06T03:09:33.999698+0000","last_active":"2023-10-06T03:09:33.999698+0000","last_peered":"2023-10-06T03:09:33.999698+0000","last_clean":"2023-10-06T03:09:33.999698+0000","last_became_active":"2023-10-06T03:09:33.999436+0000","last_became_peered":"2023-10-06T03:09:33.999436+0000","last_unstale":"2023-10-06T03:09:33.999698+0000","last_undegraded":"2023-10-06T03:09:33.999698+0000","last_fullsized":"2023-10-06T03:09:33.999698+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:11.525029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003349+0000","last_change":"2023-10-06T03:09:34.003349+0000","last_active":"2023-10-06T03:09:34.003349+0000","last_peered":"2023-10-06T03:09:34.003349+0000","last_clean":"2023-10-06T03:09:34.003349+0000","last_became_active":"2023-10-06T03:09:34.003102+0000","last_became_peered":"2023-10-06T03:09:34.003102+0000","last_unstale":"2023-10-06T03:09:34.003349+0000","last_undegraded":"2023-10-06T03:09:34.003349+0000","last_fullsized":"2023-10-06T03:09:34.003349+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:33.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005638+0000","last_change":"2023-10-06T03:09:34.005638+0000","last_active":"2023-10-06T03:09:34.005638+0000","last_peered":"2023-10-06T03:09:34.005638+0000","last_clean":"2023-10-06T03:09:34.005638+0000","last_became_active":"2023-10-06T03:09:34.005358+0000","last_became_peered":"2023-10-06T03:09:34.005358+0000","last_unstale":"2023-10-06T03:09:34.005638+0000","last_undegraded":"2023-10-06T03:09:34.005638+0000","last_fullsized":"2023-10-06T03:09:34.005638+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:39.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001089+0000","last_change":"2023-10-06T03:09:34.001089+0000","last_active":"2023-10-06T03:09:34.001089+0000","last_peered":"2023-10-06T03:09:34.001089+0000","last_clean":"2023-10-06T03:09:34.001089+0000","last_became_active":"2023-10-06T03:09:34.000822+0000","last_became_peered":"2023-10-06T03:09:34.000822+0000","last_unstale":"2023-10-06T03:09:34.001089+0000","last_undegraded":"2023-10-06T03:09:34.001089+0000","last_fullsized":"2023-10-06T03:09:34.001089+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:22.013014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003249+0000","last_change":"2023-10-06T03:09:34.003249+0000","last_active":"2023-10-06T03:09:34.003249+0000","last_peered":"2023-10-06T03:09:34.003249+0000","last_clean":"2023-10-06T03:09:34.003249+0000","last_became_active":"2023-10-06T03:09:34.000577+0000","last_became_peered":"2023-10-06T03:09:34.000577+0000","last_unstale":"2023-10-06T03:09:34.003249+0000","last_undegraded":"2023-10-06T03:09:34.003249+0000","last_fullsized":"2023-10-06T03:09:34.003249+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:28.754607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002408+0000","last_change":"2023-10-06T03:09:34.002408+0000","last_active":"2023-10-06T03:09:34.002408+0000","last_peered":"2023-10-06T03:09:34.002408+0000","last_clean":"2023-10-06T03:09:34.002408+0000","last_became_active":"2023-10-06T03:09:34.002164+0000","last_became_peered":"2023-10-06T03:09:34.002164+0000","last_unstale":"2023-10-06T03:09:34.002408+0000","last_undegraded":"2023-10-06T03:09:34.002408+0000","last_fullsized":"2023-10-06T03:09:34.002408+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:44.559848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005601+0000","last_change":"2023-10-06T03:09:34.005601+0000","last_active":"2023-10-06T03:09:34.005601+0000","last_peered":"2023-10-06T03:09:34.005601+0000","last_clean":"2023-10-06T03:09:34.005601+0000","last_became_active":"2023-10-06T03:09:34.005333+0000","last_became_peered":"2023-10-06T03:09:34.005333+0000","last_unstale":"2023-10-06T03:09:34.005601+0000","last_undegraded":"2023-10-06T03:09:34.005601+0000","last_fullsized":"2023-10-06T03:09:34.005601+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:42.115231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996970+0000","last_change":"2023-10-06T03:09:33.996970+0000","last_active":"2023-10-06T03:09:33.996970+0000","last_peered":"2023-10-06T03:09:33.996970+0000","last_clean":"2023-10-06T03:09:33.996970+0000","last_became_active":"2023-10-06T03:09:33.996700+0000","last_became_peered":"2023-10-06T03:09:33.996700+0000","last_unstale":"2023-10-06T03:09:33.996970+0000","last_undegraded":"2023-10-06T03:09:33.996970+0000","last_fullsized":"2023-10-06T03:09:33.996970+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:29.386954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999962+0000","last_change":"2023-10-06T03:09:33.999962+0000","last_active":"2023-10-06T03:09:33.999962+0000","last_peered":"2023-10-06T03:09:33.999962+0000","last_clean":"2023-10-06T03:09:33.999962+0000","last_became_active":"2023-10-06T03:09:33.999716+0000","last_became_peered":"2023-10-06T03:09:33.999716+0000","last_unstale":"2023-10-06T03:09:33.999962+0000","last_undegraded":"2023-10-06T03:09:33.999962+0000","last_fullsized":"2023-10-06T03:09:33.999962+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:03.259521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005386+0000","last_change":"2023-10-06T03:09:34.005386+0000","last_active":"2023-10-06T03:09:34.005386+0000","last_peered":"2023-10-06T03:09:34.005386+0000","last_clean":"2023-10-06T03:09:34.005386+0000","last_became_active":"2023-10-06T03:09:34.005075+0000","last_became_peered":"2023-10-06T03:09:34.005075+0000","last_unstale":"2023-10-06T03:09:34.005386+0000","last_undegraded":"2023-10-06T03:09:34.005386+0000","last_fullsized":"2023-10-06T03:09:34.005386+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:36.191349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.004377+0000","last_change":"2023-10-06T03:09:34.004377+0000","last_active":"2023-10-06T03:09:34.004377+0000","last_peered":"2023-10-06T03:09:34.004377+0000","last_clean":"2023-10-06T03:09:34.004377+0000","last_became_active":"2023-10-06T03:09:34.004128+0000","last_became_peered":"2023-10-06T03:09:34.004128+0000","last_unstale":"2023-10-06T03:09:34.004377+0000","last_undegraded":"2023-10-06T03:09:34.004377+0000","last_fullsized":"2023-10-06T03:09:34.004377+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:45.968998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996475+0000","last_change":"2023-10-06T03:09:33.996475+0000","last_active":"2023-10-06T03:09:33.996475+0000","last_peered":"2023-10-06T03:09:33.996475+0000","last_clean":"2023-10-06T03:09:33.996475+0000","last_became_active":"2023-10-06T03:09:33.996213+0000","last_became_peered":"2023-10-06T03:09:33.996213+0000","last_unstale":"2023-10-06T03:09:33.996475+0000","last_undegraded":"2023-10-06T03:09:33.996475+0000","last_fullsized":"2023-10-06T03:09:33.996475+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:13.018067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.012248+0000","last_change":"2023-10-06T03:09:33.007956+0000","last_active":"2023-10-06T03:09:33.012248+0000","last_peered":"2023-10-06T03:09:33.012248+0000","last_clean":"2023-10-06T03:09:33.012248+0000","last_became_active":"2023-10-06T03:09:33.007664+0000","last_became_peered":"2023-10-06T03:09:33.007664+0000","last_unstale":"2023-10-06T03:09:33.012248+0000","last_undegraded":"2023-10-06T03:09:33.012248+0000","last_fullsized":"2023-10-06T03:09:33.012248+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:56.137718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007491+0000","last_change":"2023-10-06T03:09:34.007491+0000","last_active":"2023-10-06T03:09:34.007491+0000","last_peered":"2023-10-06T03:09:34.007491+0000","last_clean":"2023-10-06T03:09:34.007491+0000","last_became_active":"2023-10-06T03:09:34.006259+0000","last_became_peered":"2023-10-06T03:09:34.006259+0000","last_unstale":"2023-10-06T03:09:34.007491+0000","last_undegraded":"2023-10-06T03:09:34.007491+0000","last_fullsized":"2023-10-06T03:09:34.007491+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:32.691475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.010393+0000","last_change":"2023-10-06T03:09:33.006063+0000","last_active":"2023-10-06T03:09:33.010393+0000","last_peered":"2023-10-06T03:09:33.010393+0000","last_clean":"2023-10-06T03:09:33.010393+0000","last_became_active":"2023-10-06T03:09:33.005802+0000","last_became_peered":"2023-10-06T03:09:33.005802+0000","last_unstale":"2023-10-06T03:09:33.010393+0000","last_undegraded":"2023-10-06T03:09:33.010393+0000","last_fullsized":"2023-10-06T03:09:33.010393+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:06.438493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.994596+0000","last_change":"2023-10-06T03:09:33.994596+0000","last_active":"2023-10-06T03:09:33.994596+0000","last_peered":"2023-10-06T03:09:33.994596+0000","last_clean":"2023-10-06T03:09:33.994596+0000","last_became_active":"2023-10-06T03:09:33.994336+0000","last_became_peered":"2023-10-06T03:09:33.994336+0000","last_unstale":"2023-10-06T03:09:33.994596+0000","last_undegraded":"2023-10-06T03:09:33.994596+0000","last_fullsized":"2023-10-06T03:09:33.994596+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:27.989998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"154'3","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.015483+0000","last_change":"2023-10-06T03:09:33.003350+0000","last_active":"2023-10-06T03:09:34.015483+0000","last_peered":"2023-10-06T03:09:34.015483+0000","last_clean":"2023-10-06T03:09:34.015483+0000","last_became_active":"2023-10-06T03:09:33.003081+0000","last_became_peered":"2023-10-06T03:09:33.003081+0000","last_unstale":"2023-10-06T03:09:34.015483+0000","last_undegraded":"2023-10-06T03:09:34.015483+0000","last_fullsized":"2023-10-06T03:09:34.015483+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:35.186353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001027+0000","last_change":"2023-10-06T03:09:34.001027+0000","last_active":"2023-10-06T03:09:34.001027+0000","last_peered":"2023-10-06T03:09:34.001027+0000","last_clean":"2023-10-06T03:09:34.001027+0000","last_became_active":"2023-10-06T03:09:34.000769+0000","last_became_peered":"2023-10-06T03:09:34.000769+0000","last_unstale":"2023-10-06T03:09:34.001027+0000","last_undegraded":"2023-10-06T03:09:34.001027+0000","last_fullsized":"2023-10-06T03:09:34.001027+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:24.048248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"154'6","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.015661+0000","last_change":"2023-10-06T03:09:33.005953+0000","last_active":"2023-10-06T03:09:34.015661+0000","last_peered":"2023-10-06T03:09:34.015661+0000","last_clean":"2023-10-06T03:09:34.015661+0000","last_became_active":"2023-10-06T03:09:33.005699+0000","last_became_peered":"2023-10-06T03:09:33.005699+0000","last_unstale":"2023-10-06T03:09:34.015661+0000","last_undegraded":"2023-10-06T03:09:34.015661+0000","last_fullsized":"2023-10-06T03:09:34.015661+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:18.756288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007825+0000","last_change":"2023-10-06T03:09:34.007825+0000","last_active":"2023-10-06T03:09:34.007825+0000","last_peered":"2023-10-06T03:09:34.007825+0000","last_clean":"2023-10-06T03:09:34.007825+0000","last_became_active":"2023-10-06T03:09:34.007112+0000","last_became_peered":"2023-10-06T03:09:34.007112+0000","last_unstale":"2023-10-06T03:09:34.007825+0000","last_undegraded":"2023-10-06T03:09:34.007825+0000","last_fullsized":"2023-10-06T03:09:34.007825+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:29.996895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018571+0000","last_change":"2023-10-06T03:09:32.999392+0000","last_active":"2023-10-06T03:09:34.018571+0000","last_peered":"2023-10-06T03:09:34.018571+0000","last_clean":"2023-10-06T03:09:34.018571+0000","last_became_active":"2023-10-06T03:09:32.999127+0000","last_became_peered":"2023-10-06T03:09:32.999127+0000","last_unstale":"2023-10-06T03:09:34.018571+0000","last_undegraded":"2023-10-06T03:09:34.018571+0000","last_fullsized":"2023-10-06T03:09:34.018571+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:24.592942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000972+0000","last_change":"2023-10-06T03:09:34.000972+0000","last_active":"2023-10-06T03:09:34.000972+0000","last_peered":"2023-10-06T03:09:34.000972+0000","last_clean":"2023-10-06T03:09:34.000972+0000","last_became_active":"2023-10-06T03:09:34.000707+0000","last_became_peered":"2023-10-06T03:09:34.000707+0000","last_unstale":"2023-10-06T03:09:34.000972+0000","last_undegraded":"2023-10-06T03:09:34.000972+0000","last_fullsized":"2023-10-06T03:09:34.000972+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:25.597937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.012154+0000","last_change":"2023-10-06T03:09:33.006979+0000","last_active":"2023-10-06T03:09:33.012154+0000","last_peered":"2023-10-06T03:09:33.012154+0000","last_clean":"2023-10-06T03:09:33.012154+0000","last_became_active":"2023-10-06T03:09:33.006716+0000","last_became_peered":"2023-10-06T03:09:33.006716+0000","last_unstale":"2023-10-06T03:09:33.012154+0000","last_undegraded":"2023-10-06T03:09:33.012154+0000","last_fullsized":"2023-10-06T03:09:33.012154+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:53.510372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007093+0000","last_change":"2023-10-06T03:09:34.007093+0000","last_active":"2023-10-06T03:09:34.007093+0000","last_peered":"2023-10-06T03:09:34.007093+0000","last_clean":"2023-10-06T03:09:34.007093+0000","last_became_active":"2023-10-06T03:09:34.006847+0000","last_became_peered":"2023-10-06T03:09:34.006847+0000","last_unstale":"2023-10-06T03:09:34.007093+0000","last_undegraded":"2023-10-06T03:09:34.007093+0000","last_fullsized":"2023-10-06T03:09:34.007093+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:40.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.009187+0000","last_change":"2023-10-06T03:09:33.006102+0000","last_active":"2023-10-06T03:09:33.009187+0000","last_peered":"2023-10-06T03:09:33.009187+0000","last_clean":"2023-10-06T03:09:33.009187+0000","last_became_active":"2023-10-06T03:09:33.005796+0000","last_became_peered":"2023-10-06T03:09:33.005796+0000","last_unstale":"2023-10-06T03:09:33.009187+0000","last_undegraded":"2023-10-06T03:09:33.009187+0000","last_fullsized":"2023-10-06T03:09:33.009187+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:32.523040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001908+0000","last_change":"2023-10-06T03:09:34.001908+0000","last_active":"2023-10-06T03:09:34.001908+0000","last_peered":"2023-10-06T03:09:34.001908+0000","last_clean":"2023-10-06T03:09:34.001908+0000","last_became_active":"2023-10-06T03:09:34.001643+0000","last_became_peered":"2023-10-06T03:09:34.001643+0000","last_unstale":"2023-10-06T03:09:34.001908+0000","last_undegraded":"2023-10-06T03:09:34.001908+0000","last_fullsized":"2023-10-06T03:09:34.001908+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:29.996895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"154'8","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018071+0000","last_change":"2023-10-06T03:09:33.005584+0000","last_active":"2023-10-06T03:09:34.018071+0000","last_peered":"2023-10-06T03:09:34.018071+0000","last_clean":"2023-10-06T03:09:34.018071+0000","last_became_active":"2023-10-06T03:09:33.005382+0000","last_became_peered":"2023-10-06T03:09:33.005382+0000","last_unstale":"2023-10-06T03:09:34.018071+0000","last_undegraded":"2023-10-06T03:09:34.018071+0000","last_fullsized":"2023-10-06T03:09:34.018071+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.644526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005834+0000","last_change":"2023-10-06T03:09:34.005834+0000","last_active":"2023-10-06T03:09:34.005834+0000","last_peered":"2023-10-06T03:09:34.005834+0000","last_clean":"2023-10-06T03:09:34.005834+0000","last_became_active":"2023-10-06T03:09:34.003027+0000","last_became_peered":"2023-10-06T03:09:34.003027+0000","last_unstale":"2023-10-06T03:09:34.005834+0000","last_undegraded":"2023-10-06T03:09:34.005834+0000","last_fullsized":"2023-10-06T03:09:34.005834+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:39.507488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.991504+0000","last_change":"2023-10-06T03:09:33.001222+0000","last_active":"2023-10-06T03:09:33.991504+0000","last_peered":"2023-10-06T03:09:33.991504+0000","last_clean":"2023-10-06T03:09:33.991504+0000","last_became_active":"2023-10-06T03:09:33.000957+0000","last_became_peered":"2023-10-06T03:09:33.000957+0000","last_unstale":"2023-10-06T03:09:33.991504+0000","last_undegraded":"2023-10-06T03:09:33.991504+0000","last_fullsized":"2023-10-06T03:09:33.991504+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:59.787797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002994+0000","last_change":"2023-10-06T03:09:34.002994+0000","last_active":"2023-10-06T03:09:34.002994+0000","last_peered":"2023-10-06T03:09:34.002994+0000","last_clean":"2023-10-06T03:09:34.002994+0000","last_became_active":"2023-10-06T03:09:34.000730+0000","last_became_peered":"2023-10-06T03:09:34.000730+0000","last_unstale":"2023-10-06T03:09:34.002994+0000","last_undegraded":"2023-10-06T03:09:34.002994+0000","last_fullsized":"2023-10-06T03:09:34.002994+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:17.482211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.014858+0000","last_change":"2023-10-06T03:09:33.003072+0000","last_active":"2023-10-06T03:09:34.014858+0000","last_peered":"2023-10-06T03:09:34.014858+0000","last_clean":"2023-10-06T03:09:34.014858+0000","last_became_active":"2023-10-06T03:09:33.002651+0000","last_became_peered":"2023-10-06T03:09:33.002651+0000","last_unstale":"2023-10-06T03:09:34.014858+0000","last_undegraded":"2023-10-06T03:09:34.014858+0000","last_fullsized":"2023-10-06T03:09:34.014858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:54.424138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005487+0000","last_change":"2023-10-06T03:09:34.005487+0000","last_active":"2023-10-06T03:09:34.005487+0000","last_peered":"2023-10-06T03:09:34.005487+0000","last_clean":"2023-10-06T03:09:34.005487+0000","last_became_active":"2023-10-06T03:09:34.005157+0000","last_became_peered":"2023-10-06T03:09:34.005157+0000","last_unstale":"2023-10-06T03:09:34.005487+0000","last_undegraded":"2023-10-06T03:09:34.005487+0000","last_fullsized":"2023-10-06T03:09:34.005487+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:56.939127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.998439+0000","last_change":"2023-10-06T03:09:32.994895+0000","last_active":"2023-10-06T03:09:33.998439+0000","last_peered":"2023-10-06T03:09:33.998439+0000","last_clean":"2023-10-06T03:09:33.998439+0000","last_became_active":"2023-10-06T03:09:32.994620+0000","last_became_peered":"2023-10-06T03:09:32.994620+0000","last_unstale":"2023-10-06T03:09:33.998439+0000","last_undegraded":"2023-10-06T03:09:33.998439+0000","last_fullsized":"2023-10-06T03:09:33.998439+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:45.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002556+0000","last_change":"2023-10-06T03:09:34.002556+0000","last_active":"2023-10-06T03:09:34.002556+0000","last_peered":"2023-10-06T03:09:34.002556+0000","last_clean":"2023-10-06T03:09:34.002556+0000","last_became_active":"2023-10-06T03:09:34.002134+0000","last_became_peered":"2023-10-06T03:09:34.002134+0000","last_unstale":"2023-10-06T03:09:34.002556+0000","last_undegraded":"2023-10-06T03:09:34.002556+0000","last_fullsized":"2023-10-06T03:09:34.002556+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:23.184479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.002144+0000","last_change":"2023-10-06T03:09:32.999264+0000","last_active":"2023-10-06T03:09:33.002144+0000","last_peered":"2023-10-06T03:09:33.002144+0000","last_clean":"2023-10-06T03:09:33.002144+0000","last_became_active":"2023-10-06T03:09:32.999048+0000","last_became_peered":"2023-10-06T03:09:32.999048+0000","last_unstale":"2023-10-06T03:09:33.002144+0000","last_undegraded":"2023-10-06T03:09:33.002144+0000","last_fullsized":"2023-10-06T03:09:33.002144+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:01.109929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003357+0000","last_change":"2023-10-06T03:09:34.003357+0000","last_active":"2023-10-06T03:09:34.003357+0000","last_peered":"2023-10-06T03:09:34.003357+0000","last_clean":"2023-10-06T03:09:34.003357+0000","last_became_active":"2023-10-06T03:09:34.003065+0000","last_became_peered":"2023-10-06T03:09:34.003065+0000","last_unstale":"2023-10-06T03:09:34.003357+0000","last_undegraded":"2023-10-06T03:09:34.003357+0000","last_fullsized":"2023-10-06T03:09:34.003357+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.560766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.012877+0000","last_change":"2023-10-06T03:09:33.003384+0000","last_active":"2023-10-06T03:09:34.012877+0000","last_peered":"2023-10-06T03:09:34.012877+0000","last_clean":"2023-10-06T03:09:34.012877+0000","last_became_active":"2023-10-06T03:09:33.003126+0000","last_became_peered":"2023-10-06T03:09:33.003126+0000","last_unstale":"2023-10-06T03:09:34.012877+0000","last_undegraded":"2023-10-06T03:09:34.012877+0000","last_fullsized":"2023-10-06T03:09:34.012877+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:53.770304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.004308+0000","last_change":"2023-10-06T03:09:34.004308+0000","last_active":"2023-10-06T03:09:34.004308+0000","last_peered":"2023-10-06T03:09:34.004308+0000","last_clean":"2023-10-06T03:09:34.004308+0000","last_became_active":"2023-10-06T03:09:34.003952+0000","last_became_peered":"2023-10-06T03:09:34.003952+0000","last_unstale":"2023-10-06T03:09:34.004308+0000","last_undegraded":"2023-10-06T03:09:34.004308+0000","last_fullsized":"2023-10-06T03:09:34.004308+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.560766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.006374+0000","last_change":"2023-10-06T03:09:33.001622+0000","last_active":"2023-10-06T03:09:33.006374+0000","last_peered":"2023-10-06T03:09:33.006374+0000","last_clean":"2023-10-06T03:09:33.006374+0000","last_became_active":"2023-10-06T03:09:33.001286+0000","last_became_peered":"2023-10-06T03:09:33.001286+0000","last_unstale":"2023-10-06T03:09:33.006374+0000","last_undegraded":"2023-10-06T03:09:33.006374+0000","last_fullsized":"2023-10-06T03:09:33.006374+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:51.254730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001544+0000","last_change":"2023-10-06T03:09:34.001544+0000","last_active":"2023-10-06T03:09:34.001544+0000","last_peered":"2023-10-06T03:09:34.001544+0000","last_clean":"2023-10-06T03:09:34.001544+0000","last_became_active":"2023-10-06T03:09:34.001311+0000","last_became_peered":"2023-10-06T03:09:34.001311+0000","last_unstale":"2023-10-06T03:09:34.001544+0000","last_undegraded":"2023-10-06T03:09:34.001544+0000","last_fullsized":"2023-10-06T03:09:34.001544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:59.395151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.010281+0000","last_change":"2023-10-06T03:09:33.005953+0000","last_active":"2023-10-06T03:09:33.010281+0000","last_peered":"2023-10-06T03:09:33.010281+0000","last_clean":"2023-10-06T03:09:33.010281+0000","last_became_active":"2023-10-06T03:09:33.005751+0000","last_became_peered":"2023-10-06T03:09:33.005751+0000","last_unstale":"2023-10-06T03:09:33.010281+0000","last_undegraded":"2023-10-06T03:09:33.010281+0000","last_fullsized":"2023-10-06T03:09:33.010281+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:06.297875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.011245+0000","last_change":"2023-10-06T03:09:34.011245+0000","last_active":"2023-10-06T03:09:34.011245+0000","last_peered":"2023-10-06T03:09:34.011245+0000","last_clean":"2023-10-06T03:09:34.011245+0000","last_became_active":"2023-10-06T03:09:34.010238+0000","last_became_peered":"2023-10-06T03:09:34.010238+0000","last_unstale":"2023-10-06T03:09:34.011245+0000","last_undegraded":"2023-10-06T03:09:34.011245+0000","last_fullsized":"2023-10-06T03:09:34.011245+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:55.938721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.005274+0000","last_change":"2023-10-06T03:09:32.998653+0000","last_active":"2023-10-06T03:09:33.005274+0000","last_peered":"2023-10-06T03:09:33.005274+0000","last_clean":"2023-10-06T03:09:33.005274+0000","last_became_active":"2023-10-06T03:09:32.998362+0000","last_became_peered":"2023-10-06T03:09:32.998362+0000","last_unstale":"2023-10-06T03:09:33.005274+0000","last_undegraded":"2023-10-06T03:09:33.005274+0000","last_fullsized":"2023-10-06T03:09:33.005274+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:18.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005842+0000","last_change":"2023-10-06T03:09:34.005842+0000","last_active":"2023-10-06T03:09:34.005842+0000","last_peered":"2023-10-06T03:09:34.005842+0000","last_clean":"2023-10-06T03:09:34.005842+0000","last_became_active":"2023-10-06T03:09:34.003426+0000","last_became_peered":"2023-10-06T03:09:34.003426+0000","last_unstale":"2023-10-06T03:09:34.005842+0000","last_undegraded":"2023-10-06T03:09:34.005842+0000","last_fullsized":"2023-10-06T03:09:34.005842+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:30.548098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018615+0000","last_change":"2023-10-06T03:09:33.008304+0000","last_active":"2023-10-06T03:09:34.018615+0000","last_peered":"2023-10-06T03:09:34.018615+0000","last_clean":"2023-10-06T03:09:34.018615+0000","last_became_active":"2023-10-06T03:09:33.008021+0000","last_became_peered":"2023-10-06T03:09:33.008021+0000","last_unstale":"2023-10-06T03:09:34.018615+0000","last_undegraded":"2023-10-06T03:09:34.018615+0000","last_fullsized":"2023-10-06T03:09:34.018615+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:28.991899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001786+0000","last_change":"2023-10-06T03:09:34.001786+0000","last_active":"2023-10-06T03:09:34.001786+0000","last_peered":"2023-10-06T03:09:34.001786+0000","last_clean":"2023-10-06T03:09:34.001786+0000","last_became_active":"2023-10-06T03:09:34.001494+0000","last_became_peered":"2023-10-06T03:09:34.001494+0000","last_unstale":"2023-10-06T03:09:34.001786+0000","last_undegraded":"2023-10-06T03:09:34.001786+0000","last_fullsized":"2023-10-06T03:09:34.001786+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:46.005378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.007829+0000","last_change":"2023-10-06T03:09:33.003086+0000","last_active":"2023-10-06T03:09:33.007829+0000","last_peered":"2023-10-06T03:09:33.007829+0000","last_clean":"2023-10-06T03:09:33.007829+0000","last_became_active":"2023-10-06T03:09:33.002746+0000","last_became_peered":"2023-10-06T03:09:33.002746+0000","last_unstale":"2023-10-06T03:09:33.007829+0000","last_undegraded":"2023-10-06T03:09:33.007829+0000","last_fullsized":"2023-10-06T03:09:33.007829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:48.957022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007628+0000","last_change":"2023-10-06T03:09:34.007628+0000","last_active":"2023-10-06T03:09:34.007628+0000","last_peered":"2023-10-06T03:09:34.007628+0000","last_clean":"2023-10-06T03:09:34.007628+0000","last_became_active":"2023-10-06T03:09:34.007393+0000","last_became_peered":"2023-10-06T03:09:34.007393+0000","last_unstale":"2023-10-06T03:09:34.007628+0000","last_undegraded":"2023-10-06T03:09:34.007628+0000","last_fullsized":"2023-10-06T03:09:34.007628+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:58.426269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.007796+0000","last_change":"2023-10-06T03:09:33.001914+0000","last_active":"2023-10-06T03:09:33.007796+0000","last_peered":"2023-10-06T03:09:33.007796+0000","last_clean":"2023-10-06T03:09:33.007796+0000","last_became_active":"2023-10-06T03:09:33.001596+0000","last_became_peered":"2023-10-06T03:09:33.001596+0000","last_unstale":"2023-10-06T03:09:33.007796+0000","last_undegraded":"2023-10-06T03:09:33.007796+0000","last_fullsized":"2023-10-06T03:09:33.007796+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:43.636061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003328+0000","last_change":"2023-10-06T03:09:34.003328+0000","last_active":"2023-10-06T03:09:34.003328+0000","last_peered":"2023-10-06T03:09:34.003328+0000","last_clean":"2023-10-06T03:09:34.003328+0000","last_became_active":"2023-10-06T03:09:34.003039+0000","last_became_peered":"2023-10-06T03:09:34.003039+0000","last_unstale":"2023-10-06T03:09:34.003328+0000","last_undegraded":"2023-10-06T03:09:34.003328+0000","last_fullsized":"2023-10-06T03:09:34.003328+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:22.219017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.008062+0000","last_change":"2023-10-06T03:09:33.001476+0000","last_active":"2023-10-06T03:09:33.008062+0000","last_peered":"2023-10-06T03:09:33.008062+0000","last_clean":"2023-10-06T03:09:33.008062+0000","last_became_active":"2023-10-06T03:09:33.001179+0000","last_became_peered":"2023-10-06T03:09:33.001179+0000","last_unstale":"2023-10-06T03:09:33.008062+0000","last_undegraded":"2023-10-06T03:09:33.008062+0000","last_fullsized":"2023-10-06T03:09:33.008062+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:27.682444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005976+0000","last_change":"2023-10-06T03:09:34.005976+0000","last_active":"2023-10-06T03:09:34.005976+0000","last_peered":"2023-10-06T03:09:34.005976+0000","last_clean":"2023-10-06T03:09:34.005976+0000","last_became_active":"2023-10-06T03:09:34.003295+0000","last_became_peered":"2023-10-06T03:09:34.003295+0000","last_unstale":"2023-10-06T03:09:34.005976+0000","last_undegraded":"2023-10-06T03:09:34.005976+0000","last_fullsized":"2023-10-06T03:09:34.005976+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:42.730425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.007853+0000","last_change":"2023-10-06T03:09:33.004595+0000","last_active":"2023-10-06T03:09:33.007853+0000","last_peered":"2023-10-06T03:09:33.007853+0000","last_clean":"2023-10-06T03:09:33.007853+0000","last_became_active":"2023-10-06T03:09:33.004343+0000","last_became_peered":"2023-10-06T03:09:33.004343+0000","last_unstale":"2023-10-06T03:09:33.007853+0000","last_undegraded":"2023-10-06T03:09:33.007853+0000","last_fullsized":"2023-10-06T03:09:33.007853+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:17.975727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.012921+0000","last_change":"2023-10-06T03:09:34.012921+0000","last_active":"2023-10-06T03:09:34.012921+0000","last_peered":"2023-10-06T03:09:34.012921+0000","last_clean":"2023-10-06T03:09:34.012921+0000","last_became_active":"2023-10-06T03:09:34.012680+0000","last_became_peered":"2023-10-06T03:09:34.012680+0000","last_unstale":"2023-10-06T03:09:34.012921+0000","last_undegraded":"2023-10-06T03:09:34.012921+0000","last_fullsized":"2023-10-06T03:09:34.012921+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:35.806118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.002562+0000","last_change":"2023-10-06T03:09:33.001437+0000","last_active":"2023-10-06T03:09:33.002562+0000","last_peered":"2023-10-06T03:09:33.002562+0000","last_clean":"2023-10-06T03:09:33.002562+0000","last_became_active":"2023-10-06T03:09:33.001155+0000","last_became_peered":"2023-10-06T03:09:33.001155+0000","last_unstale":"2023-10-06T03:09:33.002562+0000","last_undegraded":"2023-10-06T03:09:33.002562+0000","last_fullsized":"2023-10-06T03:09:33.002562+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:08.555770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.013011+0000","last_change":"2023-10-06T03:09:34.013011+0000","last_active":"2023-10-06T03:09:34.013011+0000","last_peered":"2023-10-06T03:09:34.013011+0000","last_clean":"2023-10-06T03:09:34.013011+0000","last_became_active":"2023-10-06T03:09:34.012549+0000","last_became_peered":"2023-10-06T03:09:34.012549+0000","last_unstale":"2023-10-06T03:09:34.013011+0000","last_undegraded":"2023-10-06T03:09:34.013011+0000","last_fullsized":"2023-10-06T03:09:34.013011+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:53.157206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.012691+0000","last_change":"2023-10-06T03:09:33.006032+0000","last_active":"2023-10-06T03:09:33.012691+0000","last_peered":"2023-10-06T03:09:33.012691+0000","last_clean":"2023-10-06T03:09:33.012691+0000","last_became_active":"2023-10-06T03:09:33.005765+0000","last_became_peered":"2023-10-06T03:09:33.005765+0000","last_unstale":"2023-10-06T03:09:33.012691+0000","last_undegraded":"2023-10-06T03:09:33.012691+0000","last_fullsized":"2023-10-06T03:09:33.012691+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:45.280785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003861+0000","last_change":"2023-10-06T03:09:34.003861+0000","last_active":"2023-10-06T03:09:34.003861+0000","last_peered":"2023-10-06T03:09:34.003861+0000","last_clean":"2023-10-06T03:09:34.003861+0000","last_became_active":"2023-10-06T03:09:34.003602+0000","last_became_peered":"2023-10-06T03:09:34.003602+0000","last_unstale":"2023-10-06T03:09:34.003861+0000","last_undegraded":"2023-10-06T03:09:34.003861+0000","last_fullsized":"2023-10-06T03:09:34.003861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:53.418852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.006611+0000","last_change":"2023-10-06T03:09:33.002773+0000","last_active":"2023-10-06T03:09:33.006611+0000","last_peered":"2023-10-06T03:09:33.006611+0000","last_clean":"2023-10-06T03:09:33.006611+0000","last_became_active":"2023-10-06T03:09:33.002496+0000","last_became_peered":"2023-10-06T03:09:33.002496+0000","last_unstale":"2023-10-06T03:09:33.006611+0000","last_undegraded":"2023-10-06T03:09:33.006611+0000","last_fullsized":"2023-10-06T03:09:33.006611+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:08.519345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996974+0000","last_change":"2023-10-06T03:09:33.996974+0000","last_active":"2023-10-06T03:09:33.996974+0000","last_peered":"2023-10-06T03:09:33.996974+0000","last_clean":"2023-10-06T03:09:33.996974+0000","last_became_active":"2023-10-06T03:09:33.996669+0000","last_became_peered":"2023-10-06T03:09:33.996669+0000","last_unstale":"2023-10-06T03:09:33.996974+0000","last_undegraded":"2023-10-06T03:09:33.996974+0000","last_fullsized":"2023-10-06T03:09:33.996974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:13.018067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.991887+0000","last_change":"2023-10-06T03:09:33.002894+0000","last_active":"2023-10-06T03:09:33.991887+0000","last_peered":"2023-10-06T03:09:33.991887+0000","last_clean":"2023-10-06T03:09:33.991887+0000","last_became_active":"2023-10-06T03:09:33.002298+0000","last_became_peered":"2023-10-06T03:09:33.002298+0000","last_unstale":"2023-10-06T03:09:33.991887+0000","last_undegraded":"2023-10-06T03:09:33.991887+0000","last_fullsized":"2023-10-06T03:09:33.991887+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:13.004992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.002454+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:09:33.002454+0000","last_peered":"2023-10-06T03:09:33.002454+0000","last_clean":"2023-10-06T03:09:33.002454+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:09:33.002454+0000","last_undegraded":"2023-10-06T03:09:33.002454+0000","last_fullsized":"2023-10-06T03:09:33.002454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000997+0000","last_change":"2023-10-06T03:09:34.000997+0000","last_active":"2023-10-06T03:09:34.000997+0000","last_peered":"2023-10-06T03:09:34.000997+0000","last_clean":"2023-10-06T03:09:34.000997+0000","last_became_active":"2023-10-06T03:09:34.000761+0000","last_became_peered":"2023-10-06T03:09:34.000761+0000","last_unstale":"2023-10-06T03:09:34.000997+0000","last_undegraded":"2023-10-06T03:09:34.000997+0000","last_fullsized":"2023-10-06T03:09:34.000997+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:36.191349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"154'4","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.015117+0000","last_change":"2023-10-06T03:09:33.007058+0000","last_active":"2023-10-06T03:09:34.015117+0000","last_peered":"2023-10-06T03:09:34.015117+0000","last_clean":"2023-10-06T03:09:34.015117+0000","last_became_active":"2023-10-06T03:09:33.006803+0000","last_became_peered":"2023-10-06T03:09:33.006803+0000","last_unstale":"2023-10-06T03:09:34.015117+0000","last_undegraded":"2023-10-06T03:09:34.015117+0000","last_fullsized":"2023-10-06T03:09:34.015117+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:59.310367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991007+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:09:32.991007+0000","last_peered":"2023-10-06T03:09:32.991007+0000","last_clean":"2023-10-06T03:09:32.991007+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:09:32.991007+0000","last_undegraded":"2023-10-06T03:09:32.991007+0000","last_fullsized":"2023-10-06T03:09:32.991007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003601+0000","last_change":"2023-10-06T03:09:34.003601+0000","last_active":"2023-10-06T03:09:34.003601+0000","last_peered":"2023-10-06T03:09:34.003601+0000","last_clean":"2023-10-06T03:09:34.003601+0000","last_became_active":"2023-10-06T03:09:34.003346+0000","last_became_peered":"2023-10-06T03:09:34.003346+0000","last_unstale":"2023-10-06T03:09:34.003601+0000","last_undegraded":"2023-10-06T03:09:34.003601+0000","last_fullsized":"2023-10-06T03:09:34.003601+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:42.362908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.014327+0000","last_change":"2023-10-06T03:09:33.001383+0000","last_active":"2023-10-06T03:09:34.014327+0000","last_peered":"2023-10-06T03:09:34.014327+0000","last_clean":"2023-10-06T03:09:34.014327+0000","last_became_active":"2023-10-06T03:09:33.001101+0000","last_became_peered":"2023-10-06T03:09:33.001101+0000","last_unstale":"2023-10-06T03:09:34.014327+0000","last_undegraded":"2023-10-06T03:09:34.014327+0000","last_fullsized":"2023-10-06T03:09:34.014327+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:47.227575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.992694+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:09:32.992694+0000","last_peered":"2023-10-06T03:09:32.992694+0000","last_clean":"2023-10-06T03:09:32.992694+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:09:32.992694+0000","last_undegraded":"2023-10-06T03:09:32.992694+0000","last_fullsized":"2023-10-06T03:09:32.992694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.006072+0000","last_change":"2023-10-06T03:09:34.006072+0000","last_active":"2023-10-06T03:09:34.006072+0000","last_peered":"2023-10-06T03:09:34.006072+0000","last_clean":"2023-10-06T03:09:34.006072+0000","last_became_active":"2023-10-06T03:09:34.005793+0000","last_became_peered":"2023-10-06T03:09:34.005793+0000","last_unstale":"2023-10-06T03:09:34.006072+0000","last_undegraded":"2023-10-06T03:09:34.006072+0000","last_fullsized":"2023-10-06T03:09:34.006072+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:54.515367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.003082+0000","last_change":"2023-10-06T03:09:32.999363+0000","last_active":"2023-10-06T03:09:33.003082+0000","last_peered":"2023-10-06T03:09:33.003082+0000","last_clean":"2023-10-06T03:09:33.003082+0000","last_became_active":"2023-10-06T03:09:32.999146+0000","last_became_peered":"2023-10-06T03:09:32.999146+0000","last_unstale":"2023-10-06T03:09:33.003082+0000","last_undegraded":"2023-10-06T03:09:33.003082+0000","last_fullsized":"2023-10-06T03:09:33.003082+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:00.561729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991771+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:09:32.991771+0000","last_peered":"2023-10-06T03:09:32.991771+0000","last_clean":"2023-10-06T03:09:32.991771+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:09:32.991771+0000","last_undegraded":"2023-10-06T03:09:32.991771+0000","last_fullsized":"2023-10-06T03:09:32.991771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999665+0000","last_change":"2023-10-06T03:09:33.999665+0000","last_active":"2023-10-06T03:09:33.999665+0000","last_peered":"2023-10-06T03:09:33.999665+0000","last_clean":"2023-10-06T03:09:33.999665+0000","last_became_active":"2023-10-06T03:09:33.999417+0000","last_became_peered":"2023-10-06T03:09:33.999417+0000","last_unstale":"2023-10-06T03:09:33.999665+0000","last_undegraded":"2023-10-06T03:09:33.999665+0000","last_fullsized":"2023-10-06T03:09:33.999665+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:54.515367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.987832+0000","last_change":"2023-10-06T03:09:33.001979+0000","last_active":"2023-10-06T03:09:33.987832+0000","last_peered":"2023-10-06T03:09:33.987832+0000","last_clean":"2023-10-06T03:09:33.987832+0000","last_became_active":"2023-10-06T03:09:33.001694+0000","last_became_peered":"2023-10-06T03:09:33.001694+0000","last_unstale":"2023-10-06T03:09:33.987832+0000","last_undegraded":"2023-10-06T03:09:33.987832+0000","last_fullsized":"2023-10-06T03:09:33.987832+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:52.152210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":276,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991869+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:09:32.991869+0000","last_peered":"2023-10-06T03:09:32.991869+0000","last_clean":"2023-10-06T03:09:32.991869+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:09:32.991869+0000","last_undegraded":"2023-10-06T03:09:32.991869+0000","last_fullsized":"2023-10-06T03:09:32.991869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.996010+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:09:32.996010+0000","last_peered":"2023-10-06T03:09:32.996010+0000","last_clean":"2023-10-06T03:09:32.996010+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:09:32.996010+0000","last_undegraded":"2023-10-06T03:09:32.996010+0000","last_fullsized":"2023-10-06T03:09:32.996010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.997377+0000","last_change":"2023-10-06T03:09:33.997377+0000","last_active":"2023-10-06T03:09:33.997377+0000","last_peered":"2023-10-06T03:09:33.997377+0000","last_clean":"2023-10-06T03:09:33.997377+0000","last_became_active":"2023-10-06T03:09:33.997119+0000","last_became_peered":"2023-10-06T03:09:33.997119+0000","last_unstale":"2023-10-06T03:09:33.997377+0000","last_undegraded":"2023-10-06T03:09:33.997377+0000","last_fullsized":"2023-10-06T03:09:33.997377+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:27.709595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.003398+0000","last_change":"2023-10-06T03:09:33.001401+0000","last_active":"2023-10-06T03:09:33.003398+0000","last_peered":"2023-10-06T03:09:33.003398+0000","last_clean":"2023-10-06T03:09:33.003398+0000","last_became_active":"2023-10-06T03:09:33.001148+0000","last_became_peered":"2023-10-06T03:09:33.001148+0000","last_unstale":"2023-10-06T03:09:33.003398+0000","last_undegraded":"2023-10-06T03:09:33.003398+0000","last_fullsized":"2023-10-06T03:09:33.003398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:21.214022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.993388+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:09:32.993388+0000","last_peered":"2023-10-06T03:09:32.993388+0000","last_clean":"2023-10-06T03:09:32.993388+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:09:32.993388+0000","last_undegraded":"2023-10-06T03:09:32.993388+0000","last_fullsized":"2023-10-06T03:09:32.993388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002813+0000","last_change":"2023-10-06T03:09:34.002813+0000","last_active":"2023-10-06T03:09:34.002813+0000","last_peered":"2023-10-06T03:09:34.002813+0000","last_clean":"2023-10-06T03:09:34.002813+0000","last_became_active":"2023-10-06T03:09:34.002435+0000","last_became_peered":"2023-10-06T03:09:34.002435+0000","last_unstale":"2023-10-06T03:09:34.002813+0000","last_undegraded":"2023-10-06T03:09:34.002813+0000","last_fullsized":"2023-10-06T03:09:34.002813+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:22.219017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996375+0000","last_change":"2023-10-06T03:09:32.998526+0000","last_active":"2023-10-06T03:09:33.996375+0000","last_peered":"2023-10-06T03:09:33.996375+0000","last_clean":"2023-10-06T03:09:33.996375+0000","last_became_active":"2023-10-06T03:09:32.998258+0000","last_became_peered":"2023-10-06T03:09:32.998258+0000","last_unstale":"2023-10-06T03:09:33.996375+0000","last_undegraded":"2023-10-06T03:09:33.996375+0000","last_fullsized":"2023-10-06T03:09:33.996375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:03.277759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.995779+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:09:32.995779+0000","last_peered":"2023-10-06T03:09:32.995779+0000","last_clean":"2023-10-06T03:09:32.995779+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:09:32.995779+0000","last_undegraded":"2023-10-06T03:09:32.995779+0000","last_fullsized":"2023-10-06T03:09:32.995779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.010140+0000","last_change":"2023-10-06T03:09:34.010140+0000","last_active":"2023-10-06T03:09:34.010140+0000","last_peered":"2023-10-06T03:09:34.010140+0000","last_clean":"2023-10-06T03:09:34.010140+0000","last_became_active":"2023-10-06T03:09:34.009829+0000","last_became_peered":"2023-10-06T03:09:34.009829+0000","last_unstale":"2023-10-06T03:09:34.010140+0000","last_undegraded":"2023-10-06T03:09:34.010140+0000","last_fullsized":"2023-10-06T03:09:34.010140+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:24.048248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018621+0000","last_change":"2023-10-06T03:09:32.998894+0000","last_active":"2023-10-06T03:09:34.018621+0000","last_peered":"2023-10-06T03:09:34.018621+0000","last_clean":"2023-10-06T03:09:34.018621+0000","last_became_active":"2023-10-06T03:09:32.998574+0000","last_became_peered":"2023-10-06T03:09:32.998574+0000","last_unstale":"2023-10-06T03:09:34.018621+0000","last_undegraded":"2023-10-06T03:09:34.018621+0000","last_fullsized":"2023-10-06T03:09:34.018621+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:37.579282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991003+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:09:32.991003+0000","last_peered":"2023-10-06T03:09:32.991003+0000","last_clean":"2023-10-06T03:09:32.991003+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:09:32.991003+0000","last_undegraded":"2023-10-06T03:09:32.991003+0000","last_fullsized":"2023-10-06T03:09:32.991003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1513770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.68700000000000006},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.495},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.56299999999999994,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.59999999999999998},"min":{"1min":0.51700000000000002,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.55800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.33400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71799999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.505},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.627,"5min":0.755,"15min":0.755},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66500000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.79000000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.65100000000000002,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.57199999999999995},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.28499999999999998},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35399999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.58799999999999997},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78900000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34399999999999997,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.40699999999999997,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39300000000000002,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.98899999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32500000000000001,"15min":0.33200000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.40500000000000003,"5min":1.282,"15min":1.282},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33500000000000002},"min":{"1min":0.247,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42799999999999999,"5min":1.25,"15min":1.25},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.42899999999999999,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66200000000000003,"5min":1.302,"15min":1.302},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.127,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52900000000000003,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.96299999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63,"15min":0.60299999999999998},"min":{"1min":0.47599999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71099999999999997,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.74199999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56499999999999995,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.054,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.379,"15min":1.379},"last":0.58799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2543616,"data_stored":1512164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.307,"15min":0.309},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.32600000000000001},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38800000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33300000000000002},"min":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.64100000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.28199999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.443,"5min":0.52600000000000002,"15min":0.61699999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.81999999999999995,"15min":0.871},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.754,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78400000000000003,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.437}]},{"osd":6,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.96099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.97999999999999998},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":921886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.29999999999999999,"15min":0.318},"min":{"1min":0.23999999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.443,"5min":0.498,"15min":0.53100000000000003},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46700000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.378}]},{"osd":1,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.32900000000000001},"min":{"1min":0.253,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.34799999999999998},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44600000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32300000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.42999999999999999,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.378},"min":{"1min":0.31,"5min":0.249,"15min":0.249},"max":{"1min":0.503,"5min":0.55300000000000005,"15min":0.57699999999999996},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.54200000000000004},"min":{"1min":0.504,"5min":0.374,"15min":0.374},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.40300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72899999999999998,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.55700000000000005},"min":{"1min":0.48099999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.436},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.87,"15min":1.8740000000000001},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.61299999999999999},"min":{"1min":0.49399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.903},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77000000000000002,"5min":0.78800000000000003,"15min":0.84599999999999997},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.308},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47399999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.47099999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.31}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.307,"15min":0.309},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.40600000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.436,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35199999999999998,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.44500000000000001,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.35499999999999998},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.873,"15min":5.5220000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75800000000000001,"5min":0.997,"15min":0.997},"last":0.441}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.72899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.49199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71999999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.438},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.57799999999999996},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":0.746,"15min":3},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.76400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":10.084},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33000000000000002},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.496}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.752,"15min":0.83999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.57999999999999996},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":1.54},"last":0.316}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.72599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.371}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.41199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53600000000000003,"15min":0.57799999999999996},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72699999999999998,"5min":1.143,"15min":2.3599999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54700000000000004,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.86299999999999999,"15min":2.371},"last":0.68100000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.55900000000000005,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69599999999999995,"5min":6.532,"15min":6.532},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.57899999999999996,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.75,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76500000000000001,"5min":5.524,"15min":5.524},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.58099999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72199999999999998,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60099999999999998,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.753,"5min":0.84199999999999997,"15min":1.052},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.624},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.86799999999999999,"15min":1.175},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.60899999999999999},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.77500000000000002,"15min":1.145},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:09:38.067 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:09:38.214 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.213+0000 7fe84770e700 1 Processor -- start 2023-10-06T03:09:38.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.213+0000 7fe84770e700 1 -- start start 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe84770e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe840159890 0x7fe84015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe84770e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe840159c60 0x7fe84015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe84770e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84015a160 con 0x7fe84015db10 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe84770e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400d13a0 con 0x7fe840159c60 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe84770e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84015e140 con 0x7fe840159890 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe844ca9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe840159c60 0x7fe84015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe845cab700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe84015db10 0x7fe840069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44826/0 (socket says 172.21.15.92:44826) 2023-10-06T03:09:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe845cab700 1 -- 172.21.15.92:0/1830512496 learned_addr learned my addr 172.21.15.92:0/1830512496 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe844ca9700 1 -- 172.21.15.92:0/1830512496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe840159890 msgr2=0x7fe84015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe8454aa700 1 --2- 172.21.15.92:0/1830512496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe840159890 0x7fe84015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe837fff700 1 -- 172.21.15.92:0/1830512496 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983876222 0 0) 0x7fe84015a160 con 0x7fe84015db10 2023-10-06T03:09:38.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe844ca9700 1 --2- 172.21.15.92:0/1830512496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe840159890 0x7fe84015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe844ca9700 1 -- 172.21.15.92:0/1830512496 >> v1:172.21.15.92:6789/0 conn(0x7fe84015db10 legacy=0x7fe840069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe844ca9700 1 -- 172.21.15.92:0/1830512496 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84015a3d0 con 0x7fe840159c60 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe844ca9700 1 --2- 172.21.15.92:0/1830512496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159c60 0x7fe84015d5d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe8300098e0 tx=0x7fe8300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8243b1dc4bd4157 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.214+0000 7fe837fff700 1 -- 172.21.15.92:0/1830512496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830009e80 con 0x7fe840159c60 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe837fff700 1 -- 172.21.15.92:0/1830512496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83000b860 con 0x7fe840159c60 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe837fff700 1 -- 172.21.15.92:0/1830512496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000bbe0 con 0x7fe840159c60 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 -- 172.21.15.92:0/1830512496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159c60 msgr2=0x7fe84015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 --2- 172.21.15.92:0/1830512496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159c60 0x7fe84015d5d0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe8300098e0 tx=0x7fe8300091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 -- 172.21.15.92:0/1830512496 shutdown_connections 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 --2- 172.21.15.92:0/1830512496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe840159890 0x7fe84015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 --2- 172.21.15.92:0/1830512496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159c60 0x7fe84015d5d0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 -- 172.21.15.92:0/1830512496 >> 172.21.15.92:0/1830512496 conn(0x7fe840068e20 msgr2=0x7fe8400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 -- 172.21.15.92:0/1830512496 shutdown_connections 2023-10-06T03:09:38.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.215+0000 7fe84770e700 1 -- 172.21.15.92:0/1830512496 wait complete. 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.216+0000 7fe84770e700 1 Processor -- start 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.216+0000 7fe84770e700 1 -- start start 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe84770e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 0x7fe8400c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe84770e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe840159c60 0x7fe8401471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe84770e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe84015db10 0x7fe8400c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe84770e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400ce7a0 con 0x7fe840159890 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe84770e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400cea40 con 0x7fe840159c60 2023-10-06T03:09:38.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe84770e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400ced20 con 0x7fe84015db10 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe8454aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 0x7fe8400c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe8454aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 0x7fe8400c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34660/0 (socket says 172.21.15.92:34660) 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe8454aa700 1 -- 172.21.15.92:0/1693253359 learned_addr learned my addr 172.21.15.92:0/1693253359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe844ca9700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe840159c60 0x7fe8401471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe845cab700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe84015db10 0x7fe8400c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.217+0000 7fe8454aa700 1 -- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe84015db10 msgr2=0x7fe8400c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8454aa700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe84015db10 0x7fe8400c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8454aa700 1 -- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe840159c60 msgr2=0x7fe8401471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8454aa700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe840159c60 0x7fe8401471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8454aa700 1 -- 172.21.15.92:0/1693253359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830009420 con 0x7fe840159890 2023-10-06T03:09:38.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe844ca9700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe840159c60 0x7fe8401471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe845cab700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe84015db10 0x7fe8400c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8454aa700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 0x7fe8400c9df0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c7d0 tx=0x7fe83c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c011600 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c011760 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8400cf000 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c011b20 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.218+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe840147730 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.219+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8400cf1f0 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.219+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe83c0193f0 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.220+0000 7fe8357fa700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe824067320 0x7fe8240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.220+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe83c0852e0 con 0x7fe840159890 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.220+0000 7fe844ca9700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe824067320 0x7fe8240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.220+0000 7fe844ca9700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe824067320 0x7fe8240697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe84015a090 tx=0x7fe830017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.224+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe83c051d00 con 0x7fe840159890 2023-10-06T03:09:38.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.402+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe84004c3c0 con 0x7fe824067320 2023-10-06T03:09:38.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.410+0000 7fe8357fa700 1 -- 172.21.15.92:0/1693253359 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311885 (secure 0 0 0) 0x7fe84004c3c0 con 0x7fe824067320 2023-10-06T03:09:38.415 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe824067320 msgr2=0x7fe8240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe824067320 0x7fe8240697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe84015a090 tx=0x7fe830017000 comp rx=0 tx=0).stop 2023-10-06T03:09:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 msgr2=0x7fe8400c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 0x7fe8400c9df0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c7d0 tx=0x7fe83c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:38.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 shutdown_connections 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe824067320 0x7fe8240697d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe840159890 0x7fe8400c9df0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe84015db10 0x7fe8400c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 --2- 172.21.15.92:0/1693253359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe840159c60 0x7fe8401471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.413+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 >> 172.21.15.92:0/1693253359 conn(0x7fe840068e20 msgr2=0x7fe840152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.414+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 shutdown_connections 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.414+0000 7fe84770e700 1 -- 172.21.15.92:0/1693253359 wait complete. 2023-10-06T03:09:38.419 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:09:38.427 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2023-10-06T03:09:37.279329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427424,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547296,"statfs":{"total":773094113280,"available":770608431104,"internally_reserved":0,"allocated":16744448,"data_stored":8521220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.307253"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000228+0000","last_change":"2023-10-06T03:09:34.000228+0000","last_active":"2023-10-06T03:09:34.000228+0000","last_peered":"2023-10-06T03:09:34.000228+0000","last_clean":"2023-10-06T03:09:34.000228+0000","last_became_active":"2023-10-06T03:09:33.999962+0000","last_became_peered":"2023-10-06T03:09:33.999962+0000","last_unstale":"2023-10-06T03:09:34.000228+0000","last_undegraded":"2023-10-06T03:09:34.000228+0000","last_fullsized":"2023-10-06T03:09:34.000228+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:59.329077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003591+0000","last_change":"2023-10-06T03:09:34.003591+0000","last_active":"2023-10-06T03:09:34.003591+0000","last_peered":"2023-10-06T03:09:34.003591+0000","last_clean":"2023-10-06T03:09:34.003591+0000","last_became_active":"2023-10-06T03:09:34.003002+0000","last_became_peered":"2023-10-06T03:09:34.003002+0000","last_unstale":"2023-10-06T03:09:34.003591+0000","last_undegraded":"2023-10-06T03:09:34.003591+0000","last_fullsized":"2023-10-06T03:09:34.003591+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:48.232570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002441+0000","last_change":"2023-10-06T03:09:34.002441+0000","last_active":"2023-10-06T03:09:34.002441+0000","last_peered":"2023-10-06T03:09:34.002441+0000","last_clean":"2023-10-06T03:09:34.002441+0000","last_became_active":"2023-10-06T03:09:34.002150+0000","last_became_peered":"2023-10-06T03:09:34.002150+0000","last_unstale":"2023-10-06T03:09:34.002441+0000","last_undegraded":"2023-10-06T03:09:34.002441+0000","last_fullsized":"2023-10-06T03:09:34.002441+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:52.259726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000566+0000","last_change":"2023-10-06T03:09:34.000566+0000","last_active":"2023-10-06T03:09:34.000566+0000","last_peered":"2023-10-06T03:09:34.000566+0000","last_clean":"2023-10-06T03:09:34.000566+0000","last_became_active":"2023-10-06T03:09:34.000281+0000","last_became_peered":"2023-10-06T03:09:34.000281+0000","last_unstale":"2023-10-06T03:09:34.000566+0000","last_undegraded":"2023-10-06T03:09:34.000566+0000","last_fullsized":"2023-10-06T03:09:34.000566+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:18.520193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005665+0000","last_change":"2023-10-06T03:09:34.005665+0000","last_active":"2023-10-06T03:09:34.005665+0000","last_peered":"2023-10-06T03:09:34.005665+0000","last_clean":"2023-10-06T03:09:34.005665+0000","last_became_active":"2023-10-06T03:09:34.005397+0000","last_became_peered":"2023-10-06T03:09:34.005397+0000","last_unstale":"2023-10-06T03:09:34.005665+0000","last_undegraded":"2023-10-06T03:09:34.005665+0000","last_fullsized":"2023-10-06T03:09:34.005665+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:48.925475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.004897+0000","last_change":"2023-10-06T03:09:34.004897+0000","last_active":"2023-10-06T03:09:34.004897+0000","last_peered":"2023-10-06T03:09:34.004897+0000","last_clean":"2023-10-06T03:09:34.004897+0000","last_became_active":"2023-10-06T03:09:34.004599+0000","last_became_peered":"2023-10-06T03:09:34.004599+0000","last_unstale":"2023-10-06T03:09:34.004897+0000","last_undegraded":"2023-10-06T03:09:34.004897+0000","last_fullsized":"2023-10-06T03:09:34.004897+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:33.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999896+0000","last_change":"2023-10-06T03:09:33.999896+0000","last_active":"2023-10-06T03:09:33.999896+0000","last_peered":"2023-10-06T03:09:33.999896+0000","last_clean":"2023-10-06T03:09:33.999896+0000","last_became_active":"2023-10-06T03:09:33.999683+0000","last_became_peered":"2023-10-06T03:09:33.999683+0000","last_unstale":"2023-10-06T03:09:33.999896+0000","last_undegraded":"2023-10-06T03:09:33.999896+0000","last_fullsized":"2023-10-06T03:09:33.999896+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:19.754683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007086+0000","last_change":"2023-10-06T03:09:34.007086+0000","last_active":"2023-10-06T03:09:34.007086+0000","last_peered":"2023-10-06T03:09:34.007086+0000","last_clean":"2023-10-06T03:09:34.007086+0000","last_became_active":"2023-10-06T03:09:34.006827+0000","last_became_peered":"2023-10-06T03:09:34.006827+0000","last_unstale":"2023-10-06T03:09:34.007086+0000","last_undegraded":"2023-10-06T03:09:34.007086+0000","last_fullsized":"2023-10-06T03:09:34.007086+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:23:55.616344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.994330+0000","last_change":"2023-10-06T03:09:33.994330+0000","last_active":"2023-10-06T03:09:33.994330+0000","last_peered":"2023-10-06T03:09:33.994330+0000","last_clean":"2023-10-06T03:09:33.994330+0000","last_became_active":"2023-10-06T03:09:33.994049+0000","last_became_peered":"2023-10-06T03:09:33.994049+0000","last_unstale":"2023-10-06T03:09:33.994330+0000","last_undegraded":"2023-10-06T03:09:33.994330+0000","last_fullsized":"2023-10-06T03:09:33.994330+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:42.460007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003280+0000","last_change":"2023-10-06T03:09:34.003280+0000","last_active":"2023-10-06T03:09:34.003280+0000","last_peered":"2023-10-06T03:09:34.003280+0000","last_clean":"2023-10-06T03:09:34.003280+0000","last_became_active":"2023-10-06T03:09:34.002891+0000","last_became_peered":"2023-10-06T03:09:34.002891+0000","last_unstale":"2023-10-06T03:09:34.003280+0000","last_undegraded":"2023-10-06T03:09:34.003280+0000","last_fullsized":"2023-10-06T03:09:34.003280+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:08.184156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007161+0000","last_change":"2023-10-06T03:09:34.007161+0000","last_active":"2023-10-06T03:09:34.007161+0000","last_peered":"2023-10-06T03:09:34.007161+0000","last_clean":"2023-10-06T03:09:34.007161+0000","last_became_active":"2023-10-06T03:09:34.006921+0000","last_became_peered":"2023-10-06T03:09:34.006921+0000","last_unstale":"2023-10-06T03:09:34.007161+0000","last_undegraded":"2023-10-06T03:09:34.007161+0000","last_fullsized":"2023-10-06T03:09:34.007161+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:38.584278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999351+0000","last_change":"2023-10-06T03:09:33.999351+0000","last_active":"2023-10-06T03:09:33.999351+0000","last_peered":"2023-10-06T03:09:33.999351+0000","last_clean":"2023-10-06T03:09:33.999351+0000","last_became_active":"2023-10-06T03:09:33.999093+0000","last_became_peered":"2023-10-06T03:09:33.999093+0000","last_unstale":"2023-10-06T03:09:33.999351+0000","last_undegraded":"2023-10-06T03:09:33.999351+0000","last_fullsized":"2023-10-06T03:09:33.999351+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:37.858107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.012910+0000","last_change":"2023-10-06T03:09:34.012910+0000","last_active":"2023-10-06T03:09:34.012910+0000","last_peered":"2023-10-06T03:09:34.012910+0000","last_clean":"2023-10-06T03:09:34.012910+0000","last_became_active":"2023-10-06T03:09:34.012607+0000","last_became_peered":"2023-10-06T03:09:34.012607+0000","last_unstale":"2023-10-06T03:09:34.012910+0000","last_undegraded":"2023-10-06T03:09:34.012910+0000","last_fullsized":"2023-10-06T03:09:34.012910+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:48.232570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999868+0000","last_change":"2023-10-06T03:09:33.999868+0000","last_active":"2023-10-06T03:09:33.999868+0000","last_peered":"2023-10-06T03:09:33.999868+0000","last_clean":"2023-10-06T03:09:33.999868+0000","last_became_active":"2023-10-06T03:09:33.999608+0000","last_became_peered":"2023-10-06T03:09:33.999608+0000","last_unstale":"2023-10-06T03:09:33.999868+0000","last_undegraded":"2023-10-06T03:09:33.999868+0000","last_fullsized":"2023-10-06T03:09:33.999868+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:39.383082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999544+0000","last_change":"2023-10-06T03:09:33.999544+0000","last_active":"2023-10-06T03:09:33.999544+0000","last_peered":"2023-10-06T03:09:33.999544+0000","last_clean":"2023-10-06T03:09:33.999544+0000","last_became_active":"2023-10-06T03:09:33.999253+0000","last_became_peered":"2023-10-06T03:09:33.999253+0000","last_unstale":"2023-10-06T03:09:33.999544+0000","last_undegraded":"2023-10-06T03:09:33.999544+0000","last_fullsized":"2023-10-06T03:09:33.999544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:28.754607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003746+0000","last_change":"2023-10-06T03:09:34.003746+0000","last_active":"2023-10-06T03:09:34.003746+0000","last_peered":"2023-10-06T03:09:34.003746+0000","last_clean":"2023-10-06T03:09:34.003746+0000","last_became_active":"2023-10-06T03:09:34.003476+0000","last_became_peered":"2023-10-06T03:09:34.003476+0000","last_unstale":"2023-10-06T03:09:34.003746+0000","last_undegraded":"2023-10-06T03:09:34.003746+0000","last_fullsized":"2023-10-06T03:09:34.003746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:01.566725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003002+0000","last_change":"2023-10-06T03:09:34.003002+0000","last_active":"2023-10-06T03:09:34.003002+0000","last_peered":"2023-10-06T03:09:34.003002+0000","last_clean":"2023-10-06T03:09:34.003002+0000","last_became_active":"2023-10-06T03:09:34.002756+0000","last_became_peered":"2023-10-06T03:09:34.002756+0000","last_unstale":"2023-10-06T03:09:34.003002+0000","last_undegraded":"2023-10-06T03:09:34.003002+0000","last_fullsized":"2023-10-06T03:09:34.003002+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:14:59.754210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000960+0000","last_change":"2023-10-06T03:09:34.000960+0000","last_active":"2023-10-06T03:09:34.000960+0000","last_peered":"2023-10-06T03:09:34.000960+0000","last_clean":"2023-10-06T03:09:34.000960+0000","last_became_active":"2023-10-06T03:09:34.000744+0000","last_became_peered":"2023-10-06T03:09:34.000744+0000","last_unstale":"2023-10-06T03:09:34.000960+0000","last_undegraded":"2023-10-06T03:09:34.000960+0000","last_fullsized":"2023-10-06T03:09:34.000960+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:52.813869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.998565+0000","last_change":"2023-10-06T03:09:33.998565+0000","last_active":"2023-10-06T03:09:33.998565+0000","last_peered":"2023-10-06T03:09:33.998565+0000","last_clean":"2023-10-06T03:09:33.998565+0000","last_became_active":"2023-10-06T03:09:33.998231+0000","last_became_peered":"2023-10-06T03:09:33.998231+0000","last_unstale":"2023-10-06T03:09:33.998565+0000","last_undegraded":"2023-10-06T03:09:33.998565+0000","last_fullsized":"2023-10-06T03:09:33.998565+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:19.282888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007848+0000","last_change":"2023-10-06T03:09:34.007848+0000","last_active":"2023-10-06T03:09:34.007848+0000","last_peered":"2023-10-06T03:09:34.007848+0000","last_clean":"2023-10-06T03:09:34.007848+0000","last_became_active":"2023-10-06T03:09:34.007554+0000","last_became_peered":"2023-10-06T03:09:34.007554+0000","last_unstale":"2023-10-06T03:09:34.007848+0000","last_undegraded":"2023-10-06T03:09:34.007848+0000","last_fullsized":"2023-10-06T03:09:34.007848+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:08.650534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.006024+0000","last_change":"2023-10-06T03:09:34.006024+0000","last_active":"2023-10-06T03:09:34.006024+0000","last_peered":"2023-10-06T03:09:34.006024+0000","last_clean":"2023-10-06T03:09:34.006024+0000","last_became_active":"2023-10-06T03:09:34.002682+0000","last_became_peered":"2023-10-06T03:09:34.002682+0000","last_unstale":"2023-10-06T03:09:34.006024+0000","last_undegraded":"2023-10-06T03:09:34.006024+0000","last_fullsized":"2023-10-06T03:09:34.006024+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:28.413431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999698+0000","last_change":"2023-10-06T03:09:33.999698+0000","last_active":"2023-10-06T03:09:33.999698+0000","last_peered":"2023-10-06T03:09:33.999698+0000","last_clean":"2023-10-06T03:09:33.999698+0000","last_became_active":"2023-10-06T03:09:33.999436+0000","last_became_peered":"2023-10-06T03:09:33.999436+0000","last_unstale":"2023-10-06T03:09:33.999698+0000","last_undegraded":"2023-10-06T03:09:33.999698+0000","last_fullsized":"2023-10-06T03:09:33.999698+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:11.525029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003349+0000","last_change":"2023-10-06T03:09:34.003349+0000","last_active":"2023-10-06T03:09:34.003349+0000","last_peered":"2023-10-06T03:09:34.003349+0000","last_clean":"2023-10-06T03:09:34.003349+0000","last_became_active":"2023-10-06T03:09:34.003102+0000","last_became_peered":"2023-10-06T03:09:34.003102+0000","last_unstale":"2023-10-06T03:09:34.003349+0000","last_undegraded":"2023-10-06T03:09:34.003349+0000","last_fullsized":"2023-10-06T03:09:34.003349+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:33.528035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005638+0000","last_change":"2023-10-06T03:09:34.005638+0000","last_active":"2023-10-06T03:09:34.005638+0000","last_peered":"2023-10-06T03:09:34.005638+0000","last_clean":"2023-10-06T03:09:34.005638+0000","last_became_active":"2023-10-06T03:09:34.005358+0000","last_became_peered":"2023-10-06T03:09:34.005358+0000","last_unstale":"2023-10-06T03:09:34.005638+0000","last_undegraded":"2023-10-06T03:09:34.005638+0000","last_fullsized":"2023-10-06T03:09:34.005638+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:39.481976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001089+0000","last_change":"2023-10-06T03:09:34.001089+0000","last_active":"2023-10-06T03:09:34.001089+0000","last_peered":"2023-10-06T03:09:34.001089+0000","last_clean":"2023-10-06T03:09:34.001089+0000","last_became_active":"2023-10-06T03:09:34.000822+0000","last_became_peered":"2023-10-06T03:09:34.000822+0000","last_unstale":"2023-10-06T03:09:34.001089+0000","last_undegraded":"2023-10-06T03:09:34.001089+0000","last_fullsized":"2023-10-06T03:09:34.001089+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:22.013014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003249+0000","last_change":"2023-10-06T03:09:34.003249+0000","last_active":"2023-10-06T03:09:34.003249+0000","last_peered":"2023-10-06T03:09:34.003249+0000","last_clean":"2023-10-06T03:09:34.003249+0000","last_became_active":"2023-10-06T03:09:34.000577+0000","last_became_peered":"2023-10-06T03:09:34.000577+0000","last_unstale":"2023-10-06T03:09:34.003249+0000","last_undegraded":"2023-10-06T03:09:34.003249+0000","last_fullsized":"2023-10-06T03:09:34.003249+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:28.754607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002408+0000","last_change":"2023-10-06T03:09:34.002408+0000","last_active":"2023-10-06T03:09:34.002408+0000","last_peered":"2023-10-06T03:09:34.002408+0000","last_clean":"2023-10-06T03:09:34.002408+0000","last_became_active":"2023-10-06T03:09:34.002164+0000","last_became_peered":"2023-10-06T03:09:34.002164+0000","last_unstale":"2023-10-06T03:09:34.002408+0000","last_undegraded":"2023-10-06T03:09:34.002408+0000","last_fullsized":"2023-10-06T03:09:34.002408+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:44.559848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005601+0000","last_change":"2023-10-06T03:09:34.005601+0000","last_active":"2023-10-06T03:09:34.005601+0000","last_peered":"2023-10-06T03:09:34.005601+0000","last_clean":"2023-10-06T03:09:34.005601+0000","last_became_active":"2023-10-06T03:09:34.005333+0000","last_became_peered":"2023-10-06T03:09:34.005333+0000","last_unstale":"2023-10-06T03:09:34.005601+0000","last_undegraded":"2023-10-06T03:09:34.005601+0000","last_fullsized":"2023-10-06T03:09:34.005601+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:42.115231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996970+0000","last_change":"2023-10-06T03:09:33.996970+0000","last_active":"2023-10-06T03:09:33.996970+0000","last_peered":"2023-10-06T03:09:33.996970+0000","last_clean":"2023-10-06T03:09:33.996970+0000","last_became_active":"2023-10-06T03:09:33.996700+0000","last_became_peered":"2023-10-06T03:09:33.996700+0000","last_unstale":"2023-10-06T03:09:33.996970+0000","last_undegraded":"2023-10-06T03:09:33.996970+0000","last_fullsized":"2023-10-06T03:09:33.996970+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:29.386954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999962+0000","last_change":"2023-10-06T03:09:33.999962+0000","last_active":"2023-10-06T03:09:33.999962+0000","last_peered":"2023-10-06T03:09:33.999962+0000","last_clean":"2023-10-06T03:09:33.999962+0000","last_became_active":"2023-10-06T03:09:33.999716+0000","last_became_peered":"2023-10-06T03:09:33.999716+0000","last_unstale":"2023-10-06T03:09:33.999962+0000","last_undegraded":"2023-10-06T03:09:33.999962+0000","last_fullsized":"2023-10-06T03:09:33.999962+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:03.259521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005386+0000","last_change":"2023-10-06T03:09:34.005386+0000","last_active":"2023-10-06T03:09:34.005386+0000","last_peered":"2023-10-06T03:09:34.005386+0000","last_clean":"2023-10-06T03:09:34.005386+0000","last_became_active":"2023-10-06T03:09:34.005075+0000","last_became_peered":"2023-10-06T03:09:34.005075+0000","last_unstale":"2023-10-06T03:09:34.005386+0000","last_undegraded":"2023-10-06T03:09:34.005386+0000","last_fullsized":"2023-10-06T03:09:34.005386+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:36.191349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.004377+0000","last_change":"2023-10-06T03:09:34.004377+0000","last_active":"2023-10-06T03:09:34.004377+0000","last_peered":"2023-10-06T03:09:34.004377+0000","last_clean":"2023-10-06T03:09:34.004377+0000","last_became_active":"2023-10-06T03:09:34.004128+0000","last_became_peered":"2023-10-06T03:09:34.004128+0000","last_unstale":"2023-10-06T03:09:34.004377+0000","last_undegraded":"2023-10-06T03:09:34.004377+0000","last_fullsized":"2023-10-06T03:09:34.004377+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:32:45.968998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996475+0000","last_change":"2023-10-06T03:09:33.996475+0000","last_active":"2023-10-06T03:09:33.996475+0000","last_peered":"2023-10-06T03:09:33.996475+0000","last_clean":"2023-10-06T03:09:33.996475+0000","last_became_active":"2023-10-06T03:09:33.996213+0000","last_became_peered":"2023-10-06T03:09:33.996213+0000","last_unstale":"2023-10-06T03:09:33.996475+0000","last_undegraded":"2023-10-06T03:09:33.996475+0000","last_fullsized":"2023-10-06T03:09:33.996475+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:13.018067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.012248+0000","last_change":"2023-10-06T03:09:33.007956+0000","last_active":"2023-10-06T03:09:33.012248+0000","last_peered":"2023-10-06T03:09:33.012248+0000","last_clean":"2023-10-06T03:09:33.012248+0000","last_became_active":"2023-10-06T03:09:33.007664+0000","last_became_peered":"2023-10-06T03:09:33.007664+0000","last_unstale":"2023-10-06T03:09:33.012248+0000","last_undegraded":"2023-10-06T03:09:33.012248+0000","last_fullsized":"2023-10-06T03:09:33.012248+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:56.137718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007491+0000","last_change":"2023-10-06T03:09:34.007491+0000","last_active":"2023-10-06T03:09:34.007491+0000","last_peered":"2023-10-06T03:09:34.007491+0000","last_clean":"2023-10-06T03:09:34.007491+0000","last_became_active":"2023-10-06T03:09:34.006259+0000","last_became_peered":"2023-10-06T03:09:34.006259+0000","last_unstale":"2023-10-06T03:09:34.007491+0000","last_undegraded":"2023-10-06T03:09:34.007491+0000","last_fullsized":"2023-10-06T03:09:34.007491+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:32.691475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.010393+0000","last_change":"2023-10-06T03:09:33.006063+0000","last_active":"2023-10-06T03:09:33.010393+0000","last_peered":"2023-10-06T03:09:33.010393+0000","last_clean":"2023-10-06T03:09:33.010393+0000","last_became_active":"2023-10-06T03:09:33.005802+0000","last_became_peered":"2023-10-06T03:09:33.005802+0000","last_unstale":"2023-10-06T03:09:33.010393+0000","last_undegraded":"2023-10-06T03:09:33.010393+0000","last_fullsized":"2023-10-06T03:09:33.010393+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:06.438493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.994596+0000","last_change":"2023-10-06T03:09:33.994596+0000","last_active":"2023-10-06T03:09:33.994596+0000","last_peered":"2023-10-06T03:09:33.994596+0000","last_clean":"2023-10-06T03:09:33.994596+0000","last_became_active":"2023-10-06T03:09:33.994336+0000","last_became_peered":"2023-10-06T03:09:33.994336+0000","last_unstale":"2023-10-06T03:09:33.994596+0000","last_undegraded":"2023-10-06T03:09:33.994596+0000","last_fullsized":"2023-10-06T03:09:33.994596+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:27.989998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"154'3","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.015483+0000","last_change":"2023-10-06T03:09:33.003350+0000","last_active":"2023-10-06T03:09:34.015483+0000","last_peered":"2023-10-06T03:09:34.015483+0000","last_clean":"2023-10-06T03:09:34.015483+0000","last_became_active":"2023-10-06T03:09:33.003081+0000","last_became_peered":"2023-10-06T03:09:33.003081+0000","last_unstale":"2023-10-06T03:09:34.015483+0000","last_undegraded":"2023-10-06T03:09:34.015483+0000","last_fullsized":"2023-10-06T03:09:34.015483+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:35.186353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001027+0000","last_change":"2023-10-06T03:09:34.001027+0000","last_active":"2023-10-06T03:09:34.001027+0000","last_peered":"2023-10-06T03:09:34.001027+0000","last_clean":"2023-10-06T03:09:34.001027+0000","last_became_active":"2023-10-06T03:09:34.000769+0000","last_became_peered":"2023-10-06T03:09:34.000769+0000","last_unstale":"2023-10-06T03:09:34.001027+0000","last_undegraded":"2023-10-06T03:09:34.001027+0000","last_fullsized":"2023-10-06T03:09:34.001027+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:24.048248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"154'6","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.015661+0000","last_change":"2023-10-06T03:09:33.005953+0000","last_active":"2023-10-06T03:09:34.015661+0000","last_peered":"2023-10-06T03:09:34.015661+0000","last_clean":"2023-10-06T03:09:34.015661+0000","last_became_active":"2023-10-06T03:09:33.005699+0000","last_became_peered":"2023-10-06T03:09:33.005699+0000","last_unstale":"2023-10-06T03:09:34.015661+0000","last_undegraded":"2023-10-06T03:09:34.015661+0000","last_fullsized":"2023-10-06T03:09:34.015661+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:18.756288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007825+0000","last_change":"2023-10-06T03:09:34.007825+0000","last_active":"2023-10-06T03:09:34.007825+0000","last_peered":"2023-10-06T03:09:34.007825+0000","last_clean":"2023-10-06T03:09:34.007825+0000","last_became_active":"2023-10-06T03:09:34.007112+0000","last_became_peered":"2023-10-06T03:09:34.007112+0000","last_unstale":"2023-10-06T03:09:34.007825+0000","last_undegraded":"2023-10-06T03:09:34.007825+0000","last_fullsized":"2023-10-06T03:09:34.007825+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:29.996895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018571+0000","last_change":"2023-10-06T03:09:32.999392+0000","last_active":"2023-10-06T03:09:34.018571+0000","last_peered":"2023-10-06T03:09:34.018571+0000","last_clean":"2023-10-06T03:09:34.018571+0000","last_became_active":"2023-10-06T03:09:32.999127+0000","last_became_peered":"2023-10-06T03:09:32.999127+0000","last_unstale":"2023-10-06T03:09:34.018571+0000","last_undegraded":"2023-10-06T03:09:34.018571+0000","last_fullsized":"2023-10-06T03:09:34.018571+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:24.592942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000972+0000","last_change":"2023-10-06T03:09:34.000972+0000","last_active":"2023-10-06T03:09:34.000972+0000","last_peered":"2023-10-06T03:09:34.000972+0000","last_clean":"2023-10-06T03:09:34.000972+0000","last_became_active":"2023-10-06T03:09:34.000707+0000","last_became_peered":"2023-10-06T03:09:34.000707+0000","last_unstale":"2023-10-06T03:09:34.000972+0000","last_undegraded":"2023-10-06T03:09:34.000972+0000","last_fullsized":"2023-10-06T03:09:34.000972+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:25.597937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.012154+0000","last_change":"2023-10-06T03:09:33.006979+0000","last_active":"2023-10-06T03:09:33.012154+0000","last_peered":"2023-10-06T03:09:33.012154+0000","last_clean":"2023-10-06T03:09:33.012154+0000","last_became_active":"2023-10-06T03:09:33.006716+0000","last_became_peered":"2023-10-06T03:09:33.006716+0000","last_unstale":"2023-10-06T03:09:33.012154+0000","last_undegraded":"2023-10-06T03:09:33.012154+0000","last_fullsized":"2023-10-06T03:09:33.012154+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:53.510372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007093+0000","last_change":"2023-10-06T03:09:34.007093+0000","last_active":"2023-10-06T03:09:34.007093+0000","last_peered":"2023-10-06T03:09:34.007093+0000","last_clean":"2023-10-06T03:09:34.007093+0000","last_became_active":"2023-10-06T03:09:34.006847+0000","last_became_peered":"2023-10-06T03:09:34.006847+0000","last_unstale":"2023-10-06T03:09:34.007093+0000","last_undegraded":"2023-10-06T03:09:34.007093+0000","last_fullsized":"2023-10-06T03:09:34.007093+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:40.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.009187+0000","last_change":"2023-10-06T03:09:33.006102+0000","last_active":"2023-10-06T03:09:33.009187+0000","last_peered":"2023-10-06T03:09:33.009187+0000","last_clean":"2023-10-06T03:09:33.009187+0000","last_became_active":"2023-10-06T03:09:33.005796+0000","last_became_peered":"2023-10-06T03:09:33.005796+0000","last_unstale":"2023-10-06T03:09:33.009187+0000","last_undegraded":"2023-10-06T03:09:33.009187+0000","last_fullsized":"2023-10-06T03:09:33.009187+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:32.523040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001908+0000","last_change":"2023-10-06T03:09:34.001908+0000","last_active":"2023-10-06T03:09:34.001908+0000","last_peered":"2023-10-06T03:09:34.001908+0000","last_clean":"2023-10-06T03:09:34.001908+0000","last_became_active":"2023-10-06T03:09:34.001643+0000","last_became_peered":"2023-10-06T03:09:34.001643+0000","last_unstale":"2023-10-06T03:09:34.001908+0000","last_undegraded":"2023-10-06T03:09:34.001908+0000","last_fullsized":"2023-10-06T03:09:34.001908+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:29.996895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"154'8","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018071+0000","last_change":"2023-10-06T03:09:33.005584+0000","last_active":"2023-10-06T03:09:34.018071+0000","last_peered":"2023-10-06T03:09:34.018071+0000","last_clean":"2023-10-06T03:09:34.018071+0000","last_became_active":"2023-10-06T03:09:33.005382+0000","last_became_peered":"2023-10-06T03:09:33.005382+0000","last_unstale":"2023-10-06T03:09:34.018071+0000","last_undegraded":"2023-10-06T03:09:34.018071+0000","last_fullsized":"2023-10-06T03:09:34.018071+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:31.644526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005834+0000","last_change":"2023-10-06T03:09:34.005834+0000","last_active":"2023-10-06T03:09:34.005834+0000","last_peered":"2023-10-06T03:09:34.005834+0000","last_clean":"2023-10-06T03:09:34.005834+0000","last_became_active":"2023-10-06T03:09:34.003027+0000","last_became_peered":"2023-10-06T03:09:34.003027+0000","last_unstale":"2023-10-06T03:09:34.005834+0000","last_undegraded":"2023-10-06T03:09:34.005834+0000","last_fullsized":"2023-10-06T03:09:34.005834+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:39.507488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.991504+0000","last_change":"2023-10-06T03:09:33.001222+0000","last_active":"2023-10-06T03:09:33.991504+0000","last_peered":"2023-10-06T03:09:33.991504+0000","last_clean":"2023-10-06T03:09:33.991504+0000","last_became_active":"2023-10-06T03:09:33.000957+0000","last_became_peered":"2023-10-06T03:09:33.000957+0000","last_unstale":"2023-10-06T03:09:33.991504+0000","last_undegraded":"2023-10-06T03:09:33.991504+0000","last_fullsized":"2023-10-06T03:09:33.991504+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:59.787797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002994+0000","last_change":"2023-10-06T03:09:34.002994+0000","last_active":"2023-10-06T03:09:34.002994+0000","last_peered":"2023-10-06T03:09:34.002994+0000","last_clean":"2023-10-06T03:09:34.002994+0000","last_became_active":"2023-10-06T03:09:34.000730+0000","last_became_peered":"2023-10-06T03:09:34.000730+0000","last_unstale":"2023-10-06T03:09:34.002994+0000","last_undegraded":"2023-10-06T03:09:34.002994+0000","last_fullsized":"2023-10-06T03:09:34.002994+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:17.482211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.014858+0000","last_change":"2023-10-06T03:09:33.003072+0000","last_active":"2023-10-06T03:09:34.014858+0000","last_peered":"2023-10-06T03:09:34.014858+0000","last_clean":"2023-10-06T03:09:34.014858+0000","last_became_active":"2023-10-06T03:09:33.002651+0000","last_became_peered":"2023-10-06T03:09:33.002651+0000","last_unstale":"2023-10-06T03:09:34.014858+0000","last_undegraded":"2023-10-06T03:09:34.014858+0000","last_fullsized":"2023-10-06T03:09:34.014858+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:54.424138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005487+0000","last_change":"2023-10-06T03:09:34.005487+0000","last_active":"2023-10-06T03:09:34.005487+0000","last_peered":"2023-10-06T03:09:34.005487+0000","last_clean":"2023-10-06T03:09:34.005487+0000","last_became_active":"2023-10-06T03:09:34.005157+0000","last_became_peered":"2023-10-06T03:09:34.005157+0000","last_unstale":"2023-10-06T03:09:34.005487+0000","last_undegraded":"2023-10-06T03:09:34.005487+0000","last_fullsized":"2023-10-06T03:09:34.005487+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:56.939127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.998439+0000","last_change":"2023-10-06T03:09:32.994895+0000","last_active":"2023-10-06T03:09:33.998439+0000","last_peered":"2023-10-06T03:09:33.998439+0000","last_clean":"2023-10-06T03:09:33.998439+0000","last_became_active":"2023-10-06T03:09:32.994620+0000","last_became_peered":"2023-10-06T03:09:32.994620+0000","last_unstale":"2023-10-06T03:09:33.998439+0000","last_undegraded":"2023-10-06T03:09:33.998439+0000","last_fullsized":"2023-10-06T03:09:33.998439+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:45.000382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002556+0000","last_change":"2023-10-06T03:09:34.002556+0000","last_active":"2023-10-06T03:09:34.002556+0000","last_peered":"2023-10-06T03:09:34.002556+0000","last_clean":"2023-10-06T03:09:34.002556+0000","last_became_active":"2023-10-06T03:09:34.002134+0000","last_became_peered":"2023-10-06T03:09:34.002134+0000","last_unstale":"2023-10-06T03:09:34.002556+0000","last_undegraded":"2023-10-06T03:09:34.002556+0000","last_fullsized":"2023-10-06T03:09:34.002556+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:23.184479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.002144+0000","last_change":"2023-10-06T03:09:32.999264+0000","last_active":"2023-10-06T03:09:33.002144+0000","last_peered":"2023-10-06T03:09:33.002144+0000","last_clean":"2023-10-06T03:09:33.002144+0000","last_became_active":"2023-10-06T03:09:32.999048+0000","last_became_peered":"2023-10-06T03:09:32.999048+0000","last_unstale":"2023-10-06T03:09:33.002144+0000","last_undegraded":"2023-10-06T03:09:33.002144+0000","last_fullsized":"2023-10-06T03:09:33.002144+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:01.109929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003357+0000","last_change":"2023-10-06T03:09:34.003357+0000","last_active":"2023-10-06T03:09:34.003357+0000","last_peered":"2023-10-06T03:09:34.003357+0000","last_clean":"2023-10-06T03:09:34.003357+0000","last_became_active":"2023-10-06T03:09:34.003065+0000","last_became_peered":"2023-10-06T03:09:34.003065+0000","last_unstale":"2023-10-06T03:09:34.003357+0000","last_undegraded":"2023-10-06T03:09:34.003357+0000","last_fullsized":"2023-10-06T03:09:34.003357+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.560766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.012877+0000","last_change":"2023-10-06T03:09:33.003384+0000","last_active":"2023-10-06T03:09:34.012877+0000","last_peered":"2023-10-06T03:09:34.012877+0000","last_clean":"2023-10-06T03:09:34.012877+0000","last_became_active":"2023-10-06T03:09:33.003126+0000","last_became_peered":"2023-10-06T03:09:33.003126+0000","last_unstale":"2023-10-06T03:09:34.012877+0000","last_undegraded":"2023-10-06T03:09:34.012877+0000","last_fullsized":"2023-10-06T03:09:34.012877+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:53.770304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.004308+0000","last_change":"2023-10-06T03:09:34.004308+0000","last_active":"2023-10-06T03:09:34.004308+0000","last_peered":"2023-10-06T03:09:34.004308+0000","last_clean":"2023-10-06T03:09:34.004308+0000","last_became_active":"2023-10-06T03:09:34.003952+0000","last_became_peered":"2023-10-06T03:09:34.003952+0000","last_unstale":"2023-10-06T03:09:34.004308+0000","last_undegraded":"2023-10-06T03:09:34.004308+0000","last_fullsized":"2023-10-06T03:09:34.004308+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:09.560766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.006374+0000","last_change":"2023-10-06T03:09:33.001622+0000","last_active":"2023-10-06T03:09:33.006374+0000","last_peered":"2023-10-06T03:09:33.006374+0000","last_clean":"2023-10-06T03:09:33.006374+0000","last_became_active":"2023-10-06T03:09:33.001286+0000","last_became_peered":"2023-10-06T03:09:33.001286+0000","last_unstale":"2023-10-06T03:09:33.006374+0000","last_undegraded":"2023-10-06T03:09:33.006374+0000","last_fullsized":"2023-10-06T03:09:33.006374+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:51.254730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001544+0000","last_change":"2023-10-06T03:09:34.001544+0000","last_active":"2023-10-06T03:09:34.001544+0000","last_peered":"2023-10-06T03:09:34.001544+0000","last_clean":"2023-10-06T03:09:34.001544+0000","last_became_active":"2023-10-06T03:09:34.001311+0000","last_became_peered":"2023-10-06T03:09:34.001311+0000","last_unstale":"2023-10-06T03:09:34.001544+0000","last_undegraded":"2023-10-06T03:09:34.001544+0000","last_fullsized":"2023-10-06T03:09:34.001544+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:48:59.395151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.010281+0000","last_change":"2023-10-06T03:09:33.005953+0000","last_active":"2023-10-06T03:09:33.010281+0000","last_peered":"2023-10-06T03:09:33.010281+0000","last_clean":"2023-10-06T03:09:33.010281+0000","last_became_active":"2023-10-06T03:09:33.005751+0000","last_became_peered":"2023-10-06T03:09:33.005751+0000","last_unstale":"2023-10-06T03:09:33.010281+0000","last_undegraded":"2023-10-06T03:09:33.010281+0000","last_fullsized":"2023-10-06T03:09:33.010281+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:22:06.297875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.011245+0000","last_change":"2023-10-06T03:09:34.011245+0000","last_active":"2023-10-06T03:09:34.011245+0000","last_peered":"2023-10-06T03:09:34.011245+0000","last_clean":"2023-10-06T03:09:34.011245+0000","last_became_active":"2023-10-06T03:09:34.010238+0000","last_became_peered":"2023-10-06T03:09:34.010238+0000","last_unstale":"2023-10-06T03:09:34.011245+0000","last_undegraded":"2023-10-06T03:09:34.011245+0000","last_fullsized":"2023-10-06T03:09:34.011245+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:38:55.938721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.005274+0000","last_change":"2023-10-06T03:09:32.998653+0000","last_active":"2023-10-06T03:09:33.005274+0000","last_peered":"2023-10-06T03:09:33.005274+0000","last_clean":"2023-10-06T03:09:33.005274+0000","last_became_active":"2023-10-06T03:09:32.998362+0000","last_became_peered":"2023-10-06T03:09:32.998362+0000","last_unstale":"2023-10-06T03:09:33.005274+0000","last_undegraded":"2023-10-06T03:09:33.005274+0000","last_fullsized":"2023-10-06T03:09:33.005274+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:18.291435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005842+0000","last_change":"2023-10-06T03:09:34.005842+0000","last_active":"2023-10-06T03:09:34.005842+0000","last_peered":"2023-10-06T03:09:34.005842+0000","last_clean":"2023-10-06T03:09:34.005842+0000","last_became_active":"2023-10-06T03:09:34.003426+0000","last_became_peered":"2023-10-06T03:09:34.003426+0000","last_unstale":"2023-10-06T03:09:34.005842+0000","last_undegraded":"2023-10-06T03:09:34.005842+0000","last_fullsized":"2023-10-06T03:09:34.005842+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:30.548098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018615+0000","last_change":"2023-10-06T03:09:33.008304+0000","last_active":"2023-10-06T03:09:34.018615+0000","last_peered":"2023-10-06T03:09:34.018615+0000","last_clean":"2023-10-06T03:09:34.018615+0000","last_became_active":"2023-10-06T03:09:33.008021+0000","last_became_peered":"2023-10-06T03:09:33.008021+0000","last_unstale":"2023-10-06T03:09:34.018615+0000","last_undegraded":"2023-10-06T03:09:34.018615+0000","last_fullsized":"2023-10-06T03:09:34.018615+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:28.991899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.001786+0000","last_change":"2023-10-06T03:09:34.001786+0000","last_active":"2023-10-06T03:09:34.001786+0000","last_peered":"2023-10-06T03:09:34.001786+0000","last_clean":"2023-10-06T03:09:34.001786+0000","last_became_active":"2023-10-06T03:09:34.001494+0000","last_became_peered":"2023-10-06T03:09:34.001494+0000","last_unstale":"2023-10-06T03:09:34.001786+0000","last_undegraded":"2023-10-06T03:09:34.001786+0000","last_fullsized":"2023-10-06T03:09:34.001786+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:46.005378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.007829+0000","last_change":"2023-10-06T03:09:33.003086+0000","last_active":"2023-10-06T03:09:33.007829+0000","last_peered":"2023-10-06T03:09:33.007829+0000","last_clean":"2023-10-06T03:09:33.007829+0000","last_became_active":"2023-10-06T03:09:33.002746+0000","last_became_peered":"2023-10-06T03:09:33.002746+0000","last_unstale":"2023-10-06T03:09:33.007829+0000","last_undegraded":"2023-10-06T03:09:33.007829+0000","last_fullsized":"2023-10-06T03:09:33.007829+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:48.957022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.007628+0000","last_change":"2023-10-06T03:09:34.007628+0000","last_active":"2023-10-06T03:09:34.007628+0000","last_peered":"2023-10-06T03:09:34.007628+0000","last_clean":"2023-10-06T03:09:34.007628+0000","last_became_active":"2023-10-06T03:09:34.007393+0000","last_became_peered":"2023-10-06T03:09:34.007393+0000","last_unstale":"2023-10-06T03:09:34.007628+0000","last_undegraded":"2023-10-06T03:09:34.007628+0000","last_fullsized":"2023-10-06T03:09:34.007628+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:58.426269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.007796+0000","last_change":"2023-10-06T03:09:33.001914+0000","last_active":"2023-10-06T03:09:33.007796+0000","last_peered":"2023-10-06T03:09:33.007796+0000","last_clean":"2023-10-06T03:09:33.007796+0000","last_became_active":"2023-10-06T03:09:33.001596+0000","last_became_peered":"2023-10-06T03:09:33.001596+0000","last_unstale":"2023-10-06T03:09:33.007796+0000","last_undegraded":"2023-10-06T03:09:33.007796+0000","last_fullsized":"2023-10-06T03:09:33.007796+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:43.636061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003328+0000","last_change":"2023-10-06T03:09:34.003328+0000","last_active":"2023-10-06T03:09:34.003328+0000","last_peered":"2023-10-06T03:09:34.003328+0000","last_clean":"2023-10-06T03:09:34.003328+0000","last_became_active":"2023-10-06T03:09:34.003039+0000","last_became_peered":"2023-10-06T03:09:34.003039+0000","last_unstale":"2023-10-06T03:09:34.003328+0000","last_undegraded":"2023-10-06T03:09:34.003328+0000","last_fullsized":"2023-10-06T03:09:34.003328+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:22.219017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.008062+0000","last_change":"2023-10-06T03:09:33.001476+0000","last_active":"2023-10-06T03:09:33.008062+0000","last_peered":"2023-10-06T03:09:33.008062+0000","last_clean":"2023-10-06T03:09:33.008062+0000","last_became_active":"2023-10-06T03:09:33.001179+0000","last_became_peered":"2023-10-06T03:09:33.001179+0000","last_unstale":"2023-10-06T03:09:33.008062+0000","last_undegraded":"2023-10-06T03:09:33.008062+0000","last_fullsized":"2023-10-06T03:09:33.008062+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:27.682444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.005976+0000","last_change":"2023-10-06T03:09:34.005976+0000","last_active":"2023-10-06T03:09:34.005976+0000","last_peered":"2023-10-06T03:09:34.005976+0000","last_clean":"2023-10-06T03:09:34.005976+0000","last_became_active":"2023-10-06T03:09:34.003295+0000","last_became_peered":"2023-10-06T03:09:34.003295+0000","last_unstale":"2023-10-06T03:09:34.005976+0000","last_undegraded":"2023-10-06T03:09:34.005976+0000","last_fullsized":"2023-10-06T03:09:34.005976+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:27:42.730425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.007853+0000","last_change":"2023-10-06T03:09:33.004595+0000","last_active":"2023-10-06T03:09:33.007853+0000","last_peered":"2023-10-06T03:09:33.007853+0000","last_clean":"2023-10-06T03:09:33.007853+0000","last_became_active":"2023-10-06T03:09:33.004343+0000","last_became_peered":"2023-10-06T03:09:33.004343+0000","last_unstale":"2023-10-06T03:09:33.007853+0000","last_undegraded":"2023-10-06T03:09:33.007853+0000","last_fullsized":"2023-10-06T03:09:33.007853+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:17.975727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.012921+0000","last_change":"2023-10-06T03:09:34.012921+0000","last_active":"2023-10-06T03:09:34.012921+0000","last_peered":"2023-10-06T03:09:34.012921+0000","last_clean":"2023-10-06T03:09:34.012921+0000","last_became_active":"2023-10-06T03:09:34.012680+0000","last_became_peered":"2023-10-06T03:09:34.012680+0000","last_unstale":"2023-10-06T03:09:34.012921+0000","last_undegraded":"2023-10-06T03:09:34.012921+0000","last_fullsized":"2023-10-06T03:09:34.012921+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:35.806118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.002562+0000","last_change":"2023-10-06T03:09:33.001437+0000","last_active":"2023-10-06T03:09:33.002562+0000","last_peered":"2023-10-06T03:09:33.002562+0000","last_clean":"2023-10-06T03:09:33.002562+0000","last_became_active":"2023-10-06T03:09:33.001155+0000","last_became_peered":"2023-10-06T03:09:33.001155+0000","last_unstale":"2023-10-06T03:09:33.002562+0000","last_undegraded":"2023-10-06T03:09:33.002562+0000","last_fullsized":"2023-10-06T03:09:33.002562+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:08.555770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.013011+0000","last_change":"2023-10-06T03:09:34.013011+0000","last_active":"2023-10-06T03:09:34.013011+0000","last_peered":"2023-10-06T03:09:34.013011+0000","last_clean":"2023-10-06T03:09:34.013011+0000","last_became_active":"2023-10-06T03:09:34.012549+0000","last_became_peered":"2023-10-06T03:09:34.012549+0000","last_unstale":"2023-10-06T03:09:34.013011+0000","last_undegraded":"2023-10-06T03:09:34.013011+0000","last_fullsized":"2023-10-06T03:09:34.013011+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:53.157206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.012691+0000","last_change":"2023-10-06T03:09:33.006032+0000","last_active":"2023-10-06T03:09:33.012691+0000","last_peered":"2023-10-06T03:09:33.012691+0000","last_clean":"2023-10-06T03:09:33.012691+0000","last_became_active":"2023-10-06T03:09:33.005765+0000","last_became_peered":"2023-10-06T03:09:33.005765+0000","last_unstale":"2023-10-06T03:09:33.012691+0000","last_undegraded":"2023-10-06T03:09:33.012691+0000","last_fullsized":"2023-10-06T03:09:33.012691+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:45.280785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003861+0000","last_change":"2023-10-06T03:09:34.003861+0000","last_active":"2023-10-06T03:09:34.003861+0000","last_peered":"2023-10-06T03:09:34.003861+0000","last_clean":"2023-10-06T03:09:34.003861+0000","last_became_active":"2023-10-06T03:09:34.003602+0000","last_became_peered":"2023-10-06T03:09:34.003602+0000","last_unstale":"2023-10-06T03:09:34.003861+0000","last_undegraded":"2023-10-06T03:09:34.003861+0000","last_fullsized":"2023-10-06T03:09:34.003861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:53.418852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.006611+0000","last_change":"2023-10-06T03:09:33.002773+0000","last_active":"2023-10-06T03:09:33.006611+0000","last_peered":"2023-10-06T03:09:33.006611+0000","last_clean":"2023-10-06T03:09:33.006611+0000","last_became_active":"2023-10-06T03:09:33.002496+0000","last_became_peered":"2023-10-06T03:09:33.002496+0000","last_unstale":"2023-10-06T03:09:33.006611+0000","last_undegraded":"2023-10-06T03:09:33.006611+0000","last_fullsized":"2023-10-06T03:09:33.006611+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:08.519345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996974+0000","last_change":"2023-10-06T03:09:33.996974+0000","last_active":"2023-10-06T03:09:33.996974+0000","last_peered":"2023-10-06T03:09:33.996974+0000","last_clean":"2023-10-06T03:09:33.996974+0000","last_became_active":"2023-10-06T03:09:33.996669+0000","last_became_peered":"2023-10-06T03:09:33.996669+0000","last_unstale":"2023-10-06T03:09:33.996974+0000","last_undegraded":"2023-10-06T03:09:33.996974+0000","last_fullsized":"2023-10-06T03:09:33.996974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:13.018067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.991887+0000","last_change":"2023-10-06T03:09:33.002894+0000","last_active":"2023-10-06T03:09:33.991887+0000","last_peered":"2023-10-06T03:09:33.991887+0000","last_clean":"2023-10-06T03:09:33.991887+0000","last_became_active":"2023-10-06T03:09:33.002298+0000","last_became_peered":"2023-10-06T03:09:33.002298+0000","last_unstale":"2023-10-06T03:09:33.991887+0000","last_undegraded":"2023-10-06T03:09:33.991887+0000","last_fullsized":"2023-10-06T03:09:33.991887+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:13.004992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.002454+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:09:33.002454+0000","last_peered":"2023-10-06T03:09:33.002454+0000","last_clean":"2023-10-06T03:09:33.002454+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:09:33.002454+0000","last_undegraded":"2023-10-06T03:09:33.002454+0000","last_fullsized":"2023-10-06T03:09:33.002454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.000997+0000","last_change":"2023-10-06T03:09:34.000997+0000","last_active":"2023-10-06T03:09:34.000997+0000","last_peered":"2023-10-06T03:09:34.000997+0000","last_clean":"2023-10-06T03:09:34.000997+0000","last_became_active":"2023-10-06T03:09:34.000761+0000","last_became_peered":"2023-10-06T03:09:34.000761+0000","last_unstale":"2023-10-06T03:09:34.000997+0000","last_undegraded":"2023-10-06T03:09:34.000997+0000","last_fullsized":"2023-10-06T03:09:34.000997+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:42:36.191349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"154'4","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.015117+0000","last_change":"2023-10-06T03:09:33.007058+0000","last_active":"2023-10-06T03:09:34.015117+0000","last_peered":"2023-10-06T03:09:34.015117+0000","last_clean":"2023-10-06T03:09:34.015117+0000","last_became_active":"2023-10-06T03:09:33.006803+0000","last_became_peered":"2023-10-06T03:09:33.006803+0000","last_unstale":"2023-10-06T03:09:34.015117+0000","last_undegraded":"2023-10-06T03:09:34.015117+0000","last_fullsized":"2023-10-06T03:09:34.015117+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:44:59.310367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991007+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:09:32.991007+0000","last_peered":"2023-10-06T03:09:32.991007+0000","last_clean":"2023-10-06T03:09:32.991007+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:09:32.991007+0000","last_undegraded":"2023-10-06T03:09:32.991007+0000","last_fullsized":"2023-10-06T03:09:32.991007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.003601+0000","last_change":"2023-10-06T03:09:34.003601+0000","last_active":"2023-10-06T03:09:34.003601+0000","last_peered":"2023-10-06T03:09:34.003601+0000","last_clean":"2023-10-06T03:09:34.003601+0000","last_became_active":"2023-10-06T03:09:34.003346+0000","last_became_peered":"2023-10-06T03:09:34.003346+0000","last_unstale":"2023-10-06T03:09:34.003601+0000","last_undegraded":"2023-10-06T03:09:34.003601+0000","last_fullsized":"2023-10-06T03:09:34.003601+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:42.362908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.014327+0000","last_change":"2023-10-06T03:09:33.001383+0000","last_active":"2023-10-06T03:09:34.014327+0000","last_peered":"2023-10-06T03:09:34.014327+0000","last_clean":"2023-10-06T03:09:34.014327+0000","last_became_active":"2023-10-06T03:09:33.001101+0000","last_became_peered":"2023-10-06T03:09:33.001101+0000","last_unstale":"2023-10-06T03:09:34.014327+0000","last_undegraded":"2023-10-06T03:09:34.014327+0000","last_fullsized":"2023-10-06T03:09:34.014327+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:05:47.227575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":123,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.992694+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:09:32.992694+0000","last_peered":"2023-10-06T03:09:32.992694+0000","last_clean":"2023-10-06T03:09:32.992694+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:09:32.992694+0000","last_undegraded":"2023-10-06T03:09:32.992694+0000","last_fullsized":"2023-10-06T03:09:32.992694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.006072+0000","last_change":"2023-10-06T03:09:34.006072+0000","last_active":"2023-10-06T03:09:34.006072+0000","last_peered":"2023-10-06T03:09:34.006072+0000","last_clean":"2023-10-06T03:09:34.006072+0000","last_became_active":"2023-10-06T03:09:34.005793+0000","last_became_peered":"2023-10-06T03:09:34.005793+0000","last_unstale":"2023-10-06T03:09:34.006072+0000","last_undegraded":"2023-10-06T03:09:34.006072+0000","last_fullsized":"2023-10-06T03:09:34.006072+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:54.515367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.003082+0000","last_change":"2023-10-06T03:09:32.999363+0000","last_active":"2023-10-06T03:09:33.003082+0000","last_peered":"2023-10-06T03:09:33.003082+0000","last_clean":"2023-10-06T03:09:33.003082+0000","last_became_active":"2023-10-06T03:09:32.999146+0000","last_became_peered":"2023-10-06T03:09:32.999146+0000","last_unstale":"2023-10-06T03:09:33.003082+0000","last_undegraded":"2023-10-06T03:09:33.003082+0000","last_fullsized":"2023-10-06T03:09:33.003082+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:00.561729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":123,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991771+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:09:32.991771+0000","last_peered":"2023-10-06T03:09:32.991771+0000","last_clean":"2023-10-06T03:09:32.991771+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:09:32.991771+0000","last_undegraded":"2023-10-06T03:09:32.991771+0000","last_fullsized":"2023-10-06T03:09:32.991771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.999665+0000","last_change":"2023-10-06T03:09:33.999665+0000","last_active":"2023-10-06T03:09:33.999665+0000","last_peered":"2023-10-06T03:09:33.999665+0000","last_clean":"2023-10-06T03:09:33.999665+0000","last_became_active":"2023-10-06T03:09:33.999417+0000","last_became_peered":"2023-10-06T03:09:33.999417+0000","last_unstale":"2023-10-06T03:09:33.999665+0000","last_undegraded":"2023-10-06T03:09:33.999665+0000","last_fullsized":"2023-10-06T03:09:33.999665+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:54.515367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.987832+0000","last_change":"2023-10-06T03:09:33.001979+0000","last_active":"2023-10-06T03:09:33.987832+0000","last_peered":"2023-10-06T03:09:33.987832+0000","last_clean":"2023-10-06T03:09:33.987832+0000","last_became_active":"2023-10-06T03:09:33.001694+0000","last_became_peered":"2023-10-06T03:09:33.001694+0000","last_unstale":"2023-10-06T03:09:33.987832+0000","last_undegraded":"2023-10-06T03:09:33.987832+0000","last_fullsized":"2023-10-06T03:09:33.987832+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:52.152210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":276,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991869+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:09:32.991869+0000","last_peered":"2023-10-06T03:09:32.991869+0000","last_clean":"2023-10-06T03:09:32.991869+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:09:32.991869+0000","last_undegraded":"2023-10-06T03:09:32.991869+0000","last_fullsized":"2023-10-06T03:09:32.991869+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.996010+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:09:32.996010+0000","last_peered":"2023-10-06T03:09:32.996010+0000","last_clean":"2023-10-06T03:09:32.996010+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:09:32.996010+0000","last_undegraded":"2023-10-06T03:09:32.996010+0000","last_fullsized":"2023-10-06T03:09:32.996010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.997377+0000","last_change":"2023-10-06T03:09:33.997377+0000","last_active":"2023-10-06T03:09:33.997377+0000","last_peered":"2023-10-06T03:09:33.997377+0000","last_clean":"2023-10-06T03:09:33.997377+0000","last_became_active":"2023-10-06T03:09:33.997119+0000","last_became_peered":"2023-10-06T03:09:33.997119+0000","last_unstale":"2023-10-06T03:09:33.997377+0000","last_undegraded":"2023-10-06T03:09:33.997377+0000","last_fullsized":"2023-10-06T03:09:33.997377+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:27.709595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.003398+0000","last_change":"2023-10-06T03:09:33.001401+0000","last_active":"2023-10-06T03:09:33.003398+0000","last_peered":"2023-10-06T03:09:33.003398+0000","last_clean":"2023-10-06T03:09:33.003398+0000","last_became_active":"2023-10-06T03:09:33.001148+0000","last_became_peered":"2023-10-06T03:09:33.001148+0000","last_unstale":"2023-10-06T03:09:33.003398+0000","last_undegraded":"2023-10-06T03:09:33.003398+0000","last_fullsized":"2023-10-06T03:09:33.003398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:21.214022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.993388+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:09:32.993388+0000","last_peered":"2023-10-06T03:09:32.993388+0000","last_clean":"2023-10-06T03:09:32.993388+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:09:32.993388+0000","last_undegraded":"2023-10-06T03:09:32.993388+0000","last_fullsized":"2023-10-06T03:09:32.993388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.002813+0000","last_change":"2023-10-06T03:09:34.002813+0000","last_active":"2023-10-06T03:09:34.002813+0000","last_peered":"2023-10-06T03:09:34.002813+0000","last_clean":"2023-10-06T03:09:34.002813+0000","last_became_active":"2023-10-06T03:09:34.002435+0000","last_became_peered":"2023-10-06T03:09:34.002435+0000","last_unstale":"2023-10-06T03:09:34.002813+0000","last_undegraded":"2023-10-06T03:09:34.002813+0000","last_fullsized":"2023-10-06T03:09:34.002813+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:22.219017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"154'1","reported_seq":10,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:33.996375+0000","last_change":"2023-10-06T03:09:32.998526+0000","last_active":"2023-10-06T03:09:33.996375+0000","last_peered":"2023-10-06T03:09:33.996375+0000","last_clean":"2023-10-06T03:09:33.996375+0000","last_became_active":"2023-10-06T03:09:32.998258+0000","last_became_peered":"2023-10-06T03:09:32.998258+0000","last_unstale":"2023-10-06T03:09:33.996375+0000","last_undegraded":"2023-10-06T03:09:33.996375+0000","last_fullsized":"2023-10-06T03:09:33.996375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:41:03.277759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":137,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.995779+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:09:32.995779+0000","last_peered":"2023-10-06T03:09:32.995779+0000","last_clean":"2023-10-06T03:09:32.995779+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:09:32.995779+0000","last_undegraded":"2023-10-06T03:09:32.995779+0000","last_fullsized":"2023-10-06T03:09:32.995779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":8,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.010140+0000","last_change":"2023-10-06T03:09:34.010140+0000","last_active":"2023-10-06T03:09:34.010140+0000","last_peered":"2023-10-06T03:09:34.010140+0000","last_clean":"2023-10-06T03:09:34.010140+0000","last_became_active":"2023-10-06T03:09:34.009829+0000","last_became_peered":"2023-10-06T03:09:34.009829+0000","last_unstale":"2023-10-06T03:09:34.010140+0000","last_undegraded":"2023-10-06T03:09:34.010140+0000","last_fullsized":"2023-10-06T03:09:34.010140+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:32.971878+0000","last_clean_scrub_stamp":"2023-10-06T03:09:32.971878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:24.048248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"154'2","reported_seq":11,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-06T03:09:34.018621+0000","last_change":"2023-10-06T03:09:32.998894+0000","last_active":"2023-10-06T03:09:34.018621+0000","last_peered":"2023-10-06T03:09:34.018621+0000","last_clean":"2023-10-06T03:09:34.018621+0000","last_became_active":"2023-10-06T03:09:32.998574+0000","last_became_peered":"2023-10-06T03:09:32.998574+0000","last_unstale":"2023-10-06T03:09:34.018621+0000","last_undegraded":"2023-10-06T03:09:34.018621+0000","last_fullsized":"2023-10-06T03:09:34.018621+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:31.966882+0000","last_clean_scrub_stamp":"2023-10-06T03:09:31.966882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:37.579282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":125,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-06T03:09:32.991003+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:09:32.991003+0000","last_peered":"2023-10-06T03:09:32.991003+0000","last_clean":"2023-10-06T03:09:32.991003+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:09:32.991003+0000","last_undegraded":"2023-10-06T03:09:32.991003+0000","last_fullsized":"2023-10-06T03:09:32.991003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1513770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.68700000000000006},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.34899999999999998,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.495},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.55200000000000005,"15min":0.53100000000000003},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54900000000000004,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.39200000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.56299999999999994,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":4.0289999999999999,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.45700000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.504,"5min":2.504,"15min":2.504},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.52700000000000002},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.58999999999999997,"15min":0.56699999999999995},"min":{"1min":0.46999999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.51100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.63100000000000001,"15min":0.59999999999999998},"min":{"1min":0.51700000000000002,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.55800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55300000000000005,"15min":0.55000000000000004},"min":{"1min":0.33400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71799999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56499999999999995,"15min":0.56999999999999995},"min":{"1min":0.43099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76300000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49199999999999999,"15min":0.505},"min":{"1min":0.372,"5min":0.318,"15min":0.318},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.38200000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.627,"5min":0.755,"15min":0.755},"last":0.38600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66500000000000004,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.42499999999999999,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.747,"15min":0.79000000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.54700000000000004,"15min":0.55800000000000005},"min":{"1min":0.39900000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.65100000000000002,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.497},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55800000000000005,"15min":0.57199999999999995},"min":{"1min":0.45900000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.44}]},{"osd":4,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.28499999999999998},"max":{"1min":0.81200000000000006,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.35399999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76900000000000002,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56899999999999995,"15min":0.58799999999999997},"min":{"1min":0.47599999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.59699999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78900000000000003,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34399999999999997,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.40699999999999997,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39300000000000002,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.98899999999999999,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.32500000000000001,"15min":0.33200000000000002},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.40500000000000003,"5min":1.282,"15min":1.282},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.33000000000000002,"15min":0.33500000000000002},"min":{"1min":0.247,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.46200000000000002,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.33200000000000002,"15min":0.33600000000000002},"min":{"1min":0.218,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.42799999999999999,"5min":1.25,"15min":1.25},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.374,"5min":0.36299999999999999,"15min":0.36799999999999999},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.42899999999999999,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66200000000000003,"5min":1.302,"15min":1.302},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.127,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52900000000000003,"15min":0.56200000000000006},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.96299999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63,"15min":0.60299999999999998},"min":{"1min":0.47599999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71099999999999997,"5min":5.0270000000000001,"15min":5.0270000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.74199999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56499999999999995,"15min":0.59599999999999997},"min":{"1min":0.42599999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.054,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55000000000000004,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71299999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57799999999999996,"15min":0.59899999999999998},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.087,"5min":1.379,"15min":1.379},"last":0.58799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303868,"kb_used_data":2484,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067972,"statfs":{"total":96636764160,"available":96325603328,"internally_reserved":0,"allocated":2543616,"data_stored":1512164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.307,"15min":0.309},"min":{"1min":0.217,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46300000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.32600000000000001},"min":{"1min":0.22900000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.38800000000000001,"5min":0.54000000000000004,"15min":0.57599999999999996},"last":0.32000000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.33300000000000002},"min":{"1min":0.24399999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.64100000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.56100000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.28199999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.223,"15min":0.20899999999999999},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.443,"5min":0.52600000000000002,"15min":0.61699999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78900000000000003,"5min":0.81999999999999995,"15min":0.871},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56599999999999995,"15min":0.59699999999999998},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.754,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76300000000000001,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.59199999999999997},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.78400000000000003,"5min":0.81299999999999994,"15min":0.94799999999999995},"last":0.437}]},{"osd":6,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58199999999999996,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58199999999999996,"15min":0.60199999999999998},"min":{"1min":0.42799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 03:08:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58499999999999996,"15min":0.60099999999999998},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77500000000000002,"5min":0.89600000000000002,"15min":0.96099999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.61499999999999999},"min":{"1min":0.47699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":0.83299999999999996,"15min":0.97999999999999998},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303292,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068548,"statfs":{"total":96636764160,"available":96326193152,"internally_reserved":0,"allocated":1953792,"data_stored":921886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.29999999999999999,"15min":0.318},"min":{"1min":0.23999999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.443,"5min":0.498,"15min":0.53100000000000003},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46700000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.378}]},{"osd":1,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.32900000000000001},"min":{"1min":0.253,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.503,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.34799999999999998},"min":{"1min":0.26100000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44600000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.33900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.32600000000000001,"15min":0.32300000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.42999999999999999,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.378},"min":{"1min":0.31,"5min":0.249,"15min":0.249},"max":{"1min":0.503,"5min":0.55300000000000005,"15min":0.57699999999999996},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.436,"5min":0.38,"15min":0.38},"max":{"1min":0.79000000000000004,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.54200000000000004},"min":{"1min":0.504,"5min":0.374,"15min":0.374},"max":{"1min":0.74299999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56899999999999995,"15min":0.52500000000000002},"min":{"1min":0.40300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72899999999999998,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.48899999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.55700000000000005},"min":{"1min":0.48099999999999998,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.436},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.745,"5min":0.87,"15min":1.8740000000000001},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.61299999999999999},"min":{"1min":0.49399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":1.903},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.77000000000000002,"5min":0.78800000000000003,"15min":0.84599999999999997},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.317,"15min":0.308},"min":{"1min":0.20100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.47399999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.26700000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.47099999999999997,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.31}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.307,"15min":0.309},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.40600000000000003,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.314,"5min":0.33000000000000002,"15min":0.32800000000000001},"min":{"1min":0.24399999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.436,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.35199999999999998,"15min":0.34899999999999998},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.44500000000000001,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35199999999999998,"15min":0.35499999999999998},"min":{"1min":0.24399999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.51900000000000002,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.34300000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.372,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79700000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.54800000000000004},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.77300000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.58099999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.873,"15min":5.5220000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75800000000000001,"5min":0.997,"15min":0.997},"last":0.441}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78600000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57799999999999996,"15min":0.55900000000000005},"min":{"1min":0.45400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.72899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56599999999999995,"15min":0.55200000000000005},"min":{"1min":0.49199999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71999999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52000000000000002,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.69299999999999995,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55000000000000004,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50800000000000001,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.438},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.57799999999999996},"min":{"1min":0.371,"5min":0.32000000000000001,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":0.746,"15min":3},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51900000000000002,"15min":0.76400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":10.084},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.33000000000000002},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54400000000000004,"15min":0.67100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70499999999999996,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.496}]},{"osd":5,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.51500000000000001},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.752,"15min":0.83999999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.57999999999999996},"min":{"1min":0.33300000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":1.54},"last":0.316}]},{"osd":6,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.30099999999999999},"max":{"1min":0.72599999999999998,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.57099999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.371}]},{"osd":7,"last update":"Fri Oct 6 03:08:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.58899999999999997},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58299999999999996,"15min":0.61199999999999999},"min":{"1min":0.376,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.41199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":914680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53600000000000003,"15min":0.57799999999999996},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72699999999999998,"5min":1.143,"15min":2.3599999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54700000000000004,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.78600000000000003,"5min":0.86299999999999999,"15min":2.371},"last":0.68100000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.499,"15min":0.53900000000000003},"min":{"1min":0.34100000000000003,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.71899999999999997,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.55900000000000005,"15min":0.59099999999999997},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.35599999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69599999999999995,"5min":6.532,"15min":6.532},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.57899999999999996,"15min":0.71099999999999997},"min":{"1min":0.32600000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.75,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.69499999999999995},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76500000000000001,"5min":5.524,"15min":5.524},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50800000000000001,"15min":0.52600000000000002},"min":{"1min":0.34799999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68200000000000005,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54100000000000004,"15min":0.58099999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72199999999999998,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52500000000000002,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.31900000000000001},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60099999999999998,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.753,"5min":0.84199999999999997,"15min":1.052},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Oct 6 03:08:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56899999999999995,"15min":0.624},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76800000000000002,"5min":0.86799999999999999,"15min":1.175},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56699999999999995,"15min":0.60899999999999999},"min":{"1min":0.441,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.77500000000000002,"15min":1.145},"last":0.504}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:09:38.464 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:09:38.465 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:09:38.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.587+0000 7fdec8c6f700 1 Processor -- start 2023-10-06T03:09:38.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.587+0000 7fdec8c6f700 1 -- start start 2023-10-06T03:09:38.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec8c6f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec41595d0 0x7fdec40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec8c6f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdec40cf480 0x7fdec415d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec8c6f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdec415d760 0x7fdec41599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec8c6f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec415dbc0 con 0x7fdec415d760 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec8c6f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec40b10f0 con 0x7fdec40cf480 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec8c6f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40cd530 con 0x7fdec41595d0 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdec415d760 0x7fdec41599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdec415d760 0x7fdec41599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34672/0 (socket says 172.21.15.92:34672) 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 -- 172.21.15.92:0/1383227910 learned_addr learned my addr 172.21.15.92:0/1383227910 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec2ffd700 1 --2- 172.21.15.92:0/1383227910 >> v2:172.21.15.136:3300/0 conn(0x7fdec40cf480 0x7fdec415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec37fe700 1 --2- 172.21.15.92:0/1383227910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec41595d0 0x7fdec40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 -- 172.21.15.92:0/1383227910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec41595d0 msgr2=0x7fdec40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 --2- 172.21.15.92:0/1383227910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec41595d0 0x7fdec40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 -- 172.21.15.92:0/1383227910 >> v2:172.21.15.136:3300/0 conn(0x7fdec40cf480 msgr2=0x7fdec415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 --2- 172.21.15.92:0/1383227910 >> v2:172.21.15.136:3300/0 conn(0x7fdec40cf480 0x7fdec415d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 -- 172.21.15.92:0/1383227910 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec415e000 con 0x7fdec415d760 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.588+0000 7fdec3fff700 1 --2- 172.21.15.92:0/1383227910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec415d760 0x7fdec41599a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdec40712a0 tx=0x7fdebc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ed769c42571d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec27fc700 1 -- 172.21.15.92:0/1383227910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00e070 con 0x7fdec415d760 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec27fc700 1 -- 172.21.15.92:0/1383227910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc00ba50 con 0x7fdec415d760 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec27fc700 1 -- 172.21.15.92:0/1383227910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0092e0 con 0x7fdec415d760 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 -- 172.21.15.92:0/1383227910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec415d760 msgr2=0x7fdec41599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/1383227910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec415d760 0x7fdec41599a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fdec40712a0 tx=0x7fdebc009840 comp rx=0 tx=0).stop 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 -- 172.21.15.92:0/1383227910 shutdown_connections 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/1383227910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec41595d0 0x7fdec40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/1383227910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec415d760 0x7fdec41599a0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/1383227910 >> v2:172.21.15.136:3300/0 conn(0x7fdec40cf480 0x7fdec415d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 -- 172.21.15.92:0/1383227910 >> 172.21.15.92:0/1383227910 conn(0x7fdec40d05f0 msgr2=0x7fdec40d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:38.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 -- 172.21.15.92:0/1383227910 shutdown_connections 2023-10-06T03:09:38.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.589+0000 7fdec8c6f700 1 -- 172.21.15.92:0/1383227910 wait complete. 2023-10-06T03:09:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.590+0000 7fdec8c6f700 1 Processor -- start 2023-10-06T03:09:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.590+0000 7fdec8c6f700 1 -- start start 2023-10-06T03:09:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec8c6f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec40cf480 0x7fdec415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec8c6f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 0x7fdec415f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec8c6f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec415d760 0x7fdec415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec8c6f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec414e6a0 con 0x7fdec41595d0 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec8c6f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4150ae0 con 0x7fdec415d760 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec8c6f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec414fed0 con 0x7fdec40cf480 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 0x7fdec415f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 0x7fdec415f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34678/0 (socket says 172.21.15.92:34678) 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 -- 172.21.15.92:0/3970927268 learned_addr learned my addr 172.21.15.92:0/3970927268 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 -- 172.21.15.92:0/3970927268 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec40cf480 msgr2=0x7fdec415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec40cf480 0x7fdec415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 -- 172.21.15.92:0/3970927268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec415d760 msgr2=0x7fdec415fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec415d760 0x7fdec415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.591+0000 7fdec2ffd700 1 -- 172.21.15.92:0/3970927268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc009ae0 con 0x7fdec41595d0 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.592+0000 7fdec2ffd700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 0x7fdec415f700 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800d860 tx=0x7fdeb8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.592+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8009610 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.592+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb8009770 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.592+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec41501b0 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.592+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8009b30 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.592+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec4150770 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.593+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec404c3c0 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.593+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdeb801e460 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.593+0000 7fdeab7fe700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdeb0067320 0x7fdeb00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.594+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdeb8085400 con 0x7fdec41595d0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.594+0000 7fdec37fe700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdeb0067320 0x7fdeb00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.594+0000 7fdec37fe700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdeb0067320 0x7fdeb00697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdec4063e20 tx=0x7fdeb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.598+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeb8051f30 con 0x7fdec41595d0 2023-10-06T03:09:38.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.856+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdec4150c70 con 0x7fdec41595d0 2023-10-06T03:09:38.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.860+0000 7fdeab7fe700 1 -- 172.21.15.92:0/3970927268 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdeb8055550 con 0x7fdec41595d0 2023-10-06T03:09:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdeb0067320 msgr2=0x7fdeb00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdeb0067320 0x7fdeb00697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdec4063e20 tx=0x7fdeb4008040 comp rx=0 tx=0).stop 2023-10-06T03:09:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 msgr2=0x7fdec415f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 0x7fdec415f700 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800d860 tx=0x7fdeb8008040 comp rx=0 tx=0).stop 2023-10-06T03:09:38.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 shutdown_connections 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdeb0067320 0x7fdeb00697d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec415d760 0x7fdec415fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdec40cf480 0x7fdec415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 --2- 172.21.15.92:0/3970927268 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdec41595d0 0x7fdec415f700 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 >> 172.21.15.92:0/3970927268 conn(0x7fdec40d05f0 msgr2=0x7fdec41599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 shutdown_connections 2023-10-06T03:09:38.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.862+0000 7fdec8c6f700 1 -- 172.21.15.92:0/3970927268 wait complete. 2023-10-06T03:09:38.866 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:09:38.876 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:38.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.988+0000 7f4ace3b5700 1 Processor -- start 2023-10-06T03:09:38.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.988+0000 7f4ace3b5700 1 -- start start 2023-10-06T03:09:38.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ace3b5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4ac8159890 0x7f4ac815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ace3b5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159c60 0x7f4ac815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ace3b5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ac815db10 0x7f4ac8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ace3b5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac815a160 con 0x7f4ac815db10 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ace3b5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80d13a0 con 0x7f4ac8159890 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ace3b5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac815e140 con 0x7f4ac8159c60 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4acc952700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ac815db10 0x7f4ac8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4acc952700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ac815db10 0x7f4ac8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34686/0 (socket says 172.21.15.92:34686) 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4acc952700 1 -- 172.21.15.92:0/3484912722 learned_addr learned my addr 172.21.15.92:0/3484912722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ac77fe700 1 --2- 172.21.15.92:0/3484912722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159c60 0x7f4ac815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.989+0000 7f4ac7fff700 1 --2- 172.21.15.92:0/3484912722 >> v2:172.21.15.136:3300/0 conn(0x7f4ac8159890 0x7f4ac815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4acc952700 1 -- 172.21.15.92:0/3484912722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159c60 msgr2=0x7f4ac815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4acc952700 1 --2- 172.21.15.92:0/3484912722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159c60 0x7f4ac815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4acc952700 1 -- 172.21.15.92:0/3484912722 >> v2:172.21.15.136:3300/0 conn(0x7f4ac8159890 msgr2=0x7f4ac815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4acc952700 1 --2- 172.21.15.92:0/3484912722 >> v2:172.21.15.136:3300/0 conn(0x7f4ac8159890 0x7f4ac815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4acc952700 1 -- 172.21.15.92:0/3484912722 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac815a3d0 con 0x7f4ac815db10 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4acc952700 1 --2- 172.21.15.92:0/3484912722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac8069560 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80d1090 tx=0x7f4ac0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5a2f410440aaf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ac6ffd700 1 -- 172.21.15.92:0/3484912722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000e070 con 0x7f4ac815db10 2023-10-06T03:09:38.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ac6ffd700 1 -- 172.21.15.92:0/3484912722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac000ba50 con 0x7f4ac815db10 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ac6ffd700 1 -- 172.21.15.92:0/3484912722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00092e0 con 0x7f4ac815db10 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 -- 172.21.15.92:0/3484912722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 msgr2=0x7f4ac8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/3484912722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac8069560 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80d1090 tx=0x7f4ac0009840 comp rx=0 tx=0).stop 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 -- 172.21.15.92:0/3484912722 shutdown_connections 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/3484912722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac8069560 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/3484912722 >> v2:172.21.15.136:3300/0 conn(0x7f4ac8159890 0x7f4ac815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/3484912722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159c60 0x7f4ac815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 -- 172.21.15.92:0/3484912722 >> 172.21.15.92:0/3484912722 conn(0x7f4ac8068e20 msgr2=0x7f4ac80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:38.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 -- 172.21.15.92:0/3484912722 shutdown_connections 2023-10-06T03:09:38.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.990+0000 7f4ace3b5700 1 -- 172.21.15.92:0/3484912722 wait complete. 2023-10-06T03:09:38.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 Processor -- start 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 -- start start 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159890 0x7f4ac80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ac8159c60 0x7f4ac80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8153e70 con 0x7f4ac815db10 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8153860 con 0x7f4ac8159c60 2023-10-06T03:09:38.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.992+0000 7f4ace3b5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80cfbc0 con 0x7f4ac8159890 2023-10-06T03:09:39.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4ac7fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159890 0x7f4ac80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4ac77fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ac8159c60 0x7f4ac80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac80cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:34698/0 (socket says 172.21.15.92:34698) 2023-10-06T03:09:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 -- 172.21.15.92:0/1180913604 learned_addr learned my addr 172.21.15.92:0/1180913604 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 -- 172.21.15.92:0/1180913604 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159890 msgr2=0x7f4ac80b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159890 0x7f4ac80b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 -- 172.21.15.92:0/1180913604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ac8159c60 msgr2=0x7f4ac80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ac8159c60 0x7f4ac80cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 -- 172.21.15.92:0/1180913604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0009ae0 con 0x7f4ac815db10 2023-10-06T03:09:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4acc952700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac80cade0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8070e20 tx=0x7f4ac0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00032f0 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0008d20 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac8153b00 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00256b0 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.993+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac8151010 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.994+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac804c3c0 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.995+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4ac0003040 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.995+0000 7f4aaffff700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ab4067320 0x7f4ab40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.995+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ac0086b10 con 0x7f4ac815db10 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.995+0000 7f4ac7fff700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ab4067320 0x7f4ab40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.995+0000 7f4ac7fff700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ab4067320 0x7f4ab40697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006fd0 tx=0x7f4ab8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:38.999+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ac0053530 con 0x7f4ac815db10 2023-10-06T03:09:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.224+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ac8153fb0 con 0x7f4ac815db10 2023-10-06T03:09:39.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.225+0000 7f4aaffff700 1 -- 172.21.15.92:0/1180913604 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f4ac0009940 con 0x7f4ac815db10 2023-10-06T03:09:39.227 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:39.227 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":104,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:33.377515+0000","modified":"2023-10-06T03:09:37.016548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2005327996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2005327996},{"type":"v1","addr":"172.21.15.92:6837","nonce":2005327996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ab4067320 msgr2=0x7f4ab40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ab4067320 0x7f4ab40697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8006fd0 tx=0x7f4ab8008040 comp rx=0 tx=0).stop 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 msgr2=0x7f4ac80cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac80cade0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8070e20 tx=0x7f4ac0018000 comp rx=0 tx=0).stop 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 shutdown_connections 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ab4067320 0x7f4ab40697d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac815db10 0x7f4ac80cade0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ac8159890 0x7f4ac80b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 --2- 172.21.15.92:0/1180913604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ac8159c60 0x7f4ac80cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 >> 172.21.15.92:0/1180913604 conn(0x7f4ac8068e20 msgr2=0x7f4ac8152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 shutdown_connections 2023-10-06T03:09:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.227+0000 7f4ace3b5700 1 -- 172.21.15.92:0/1180913604 wait complete. 2023-10-06T03:09:39.230 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 104 2023-10-06T03:09:39.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:09:33.377515+0000', 'modified': '2023-10-06T03:09:37.016548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7992}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7992': {'gid': 7992, 'name': 'a', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/2005327996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 2005327996}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2005327996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:09:39.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:09:39.644 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:09:39.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.759+0000 7f098574d700 1 Processor -- start 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.759+0000 7f098574d700 1 -- start start 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f098574d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0980159c60 0x7f098015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f098574d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f098015db10 0x7f0980069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f098574d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098015a160 con 0x7f0980159890 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f098574d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098015df50 con 0x7f098015db10 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f098574d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800d13a0 con 0x7f0980159c60 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f097f7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f098015db10 0x7f0980069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f097effd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0980159890 0x7f098015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47374/0 (socket says 172.21.15.92:47374) 2023-10-06T03:09:39.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.760+0000 7f097effd700 1 -- 172.21.15.92:0/166806228 learned_addr learned my addr 172.21.15.92:0/166806228 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097f7fe700 1 -- 172.21.15.92:0/166806228 >> v2:172.21.15.136:3300/0 conn(0x7f0980159c60 msgr2=0x7f098015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097f7fe700 1 --2- 172.21.15.92:0/166806228 >> v2:172.21.15.136:3300/0 conn(0x7f0980159c60 0x7f098015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097f7fe700 1 -- 172.21.15.92:0/166806228 >> v1:172.21.15.92:6789/0 conn(0x7f0980159890 legacy=0x7f098015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097e7fc700 1 -- 172.21.15.92:0/166806228 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565586460 0 0) 0x7f098015a160 con 0x7f0980159890 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097f7fe700 1 -- 172.21.15.92:0/166806228 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098015e140 con 0x7f098015db10 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097f7fe700 1 --2- 172.21.15.92:0/166806228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f0980069560 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f09800b67f0 tx=0x7f0978009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aeaa48c7a2f61b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097e7fc700 1 -- 172.21.15.92:0/166806228 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800e070 con 0x7f098015db10 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097e7fc700 1 -- 172.21.15.92:0/166806228 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097800bc30 con 0x7f098015db10 2023-10-06T03:09:39.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f097e7fc700 1 -- 172.21.15.92:0/166806228 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978012430 con 0x7f098015db10 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 -- 172.21.15.92:0/166806228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 msgr2=0x7f0980069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 --2- 172.21.15.92:0/166806228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f0980069560 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f09800b67f0 tx=0x7f0978009780 comp rx=0 tx=0).stop 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 -- 172.21.15.92:0/166806228 shutdown_connections 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 --2- 172.21.15.92:0/166806228 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f0980069560 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 --2- 172.21.15.92:0/166806228 >> v2:172.21.15.136:3300/0 conn(0x7f0980159c60 0x7f098015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 -- 172.21.15.92:0/166806228 >> 172.21.15.92:0/166806228 conn(0x7f0980068e20 msgr2=0x7f09800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.761+0000 7f098574d700 1 -- 172.21.15.92:0/166806228 shutdown_connections 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.762+0000 7f098574d700 1 -- 172.21.15.92:0/166806228 wait complete. 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 Processor -- start 2023-10-06T03:09:39.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 -- start start 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0980159890 0x7f0980149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0980159c60 0x7f0980151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f098016d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cd0c0 con 0x7f098015db10 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cd3a0 con 0x7f0980159c60 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.763+0000 7f098574d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0980152520 con 0x7f0980159890 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f098016d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f098016d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37290/0 (socket says 172.21.15.92:37290) 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 -- 172.21.15.92:0/2834294972 learned_addr learned my addr 172.21.15.92:0/2834294972 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097effd700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0980159890 0x7f0980149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f09767fc700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0980159c60 0x7f0980151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 -- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0980159890 msgr2=0x7f0980149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0980159890 0x7f0980149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 -- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0980159c60 msgr2=0x7f0980151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0980159c60 0x7f0980151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 -- 172.21.15.92:0/2834294972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978009a20 con 0x7f098015db10 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f09767fc700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0980159c60 0x7f0980151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097effd700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0980159890 0x7f0980149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f097f7fe700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f098016d0c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0980070e20 tx=0x7f0978019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f09777fe700 1 -- 172.21.15.92:0/2834294972 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097801d630 con 0x7f098015db10 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f09777fe700 1 -- 172.21.15.92:0/2834294972 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097801d790 con 0x7f098015db10 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f09777fe700 1 -- 172.21.15.92:0/2834294972 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097801db30 con 0x7f098015db10 2023-10-06T03:09:39.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0980152800 con 0x7f098015db10 2023-10-06T03:09:39.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.764+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098014a390 con 0x7f098015db10 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.766+0000 7f09777fe700 1 -- 172.21.15.92:0/2834294972 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0978024070 con 0x7f098015db10 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.766+0000 7f09777fe700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0960067320 0x7f09600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.766+0000 7f09777fe700 1 -- 172.21.15.92:0/2834294972 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0978087670 con 0x7f098015db10 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.766+0000 7f097effd700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0960067320 0x7f09600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.767+0000 7f097effd700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0960067320 0x7f09600697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f09700011e0 tx=0x7f0970009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.768+0000 7f098ea45b80 1 Processor -- start 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.768+0000 7f098ea45b80 1 -- start start 2023-10-06T03:09:39.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 0x55bc236f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236f5c00 0x55bc236f6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f098ea45b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc235fca70 con 0x55bc236a6cc0 2023-10-06T03:09:39.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc235fa270 con 0x55bc236a5840 2023-10-06T03:09:39.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc235f6060 con 0x55bc236f5c00 2023-10-06T03:09:39.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 0x55bc236f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 0x55bc236f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37296/0 (socket says 172.21.15.92:37296) 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 -- 172.21.15.92:0/3036821257 learned_addr learned my addr 172.21.15.92:0/3036821257 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f097f7fe700 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236f5c00 0x55bc236f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f097effd700 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 -- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236f5c00 msgr2=0x55bc236f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236f5c00 0x55bc236f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 -- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 msgr2=0x55bc236a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.769+0000 7f09767fc700 1 -- 172.21.15.92:0/3036821257 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc23490270 con 0x55bc236a6cc0 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f09767fc700 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 0x55bc236f56c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0968007920 tx=0x7f0968008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaaf8e34ac87bb8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f095ffff700 1 -- 172.21.15.92:0/3036821257 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096800d5a0 con 0x55bc236a6cc0 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f095ffff700 1 -- 172.21.15.92:0/3036821257 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096800d700 con 0x55bc236a6cc0 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f095ffff700 1 -- 172.21.15.92:0/3036821257 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096800dac0 con 0x55bc236a6cc0 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 -- 172.21.15.92:0/3036821257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 msgr2=0x55bc236f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 0x55bc236f56c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f0968007920 tx=0x7f0968008040 comp rx=0 tx=0).stop 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 -- 172.21.15.92:0/3036821257 shutdown_connections 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236f5c00 0x55bc236f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 --2- 172.21.15.92:0/3036821257 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236a6cc0 0x55bc236f56c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.770+0000 7f098ea45b80 1 -- 172.21.15.92:0/3036821257 >> 172.21.15.92:0/3036821257 conn(0x55bc236a7160 msgr2=0x55bc236a7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.771+0000 7f098ea45b80 1 -- 172.21.15.92:0/3036821257 shutdown_connections 2023-10-06T03:09:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.771+0000 7f098ea45b80 1 -- 172.21.15.92:0/3036821257 wait complete. 2023-10-06T03:09:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.771+0000 7f098ea45b80 1 -- start start 2023-10-06T03:09:39.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.771+0000 7f098ea45b80 1 Processor -- start 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc235a1890 con 0x55bc236f5c00 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc235bcea0 con 0x55bc236a5840 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc23478e30 con 0x55bc236a6cc0 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f098ea45b80 10 client.0 _ll_register_callbacks cb 0x55bc23668e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236d4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37300/0 (socket says 172.21.15.92:37300) 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f097f7fe700 1 -- 172.21.15.92:0/2169239752 learned_addr learned my addr 172.21.15.92:0/2169239752 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f097effd700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.772+0000 7f09767fc700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097f7fe700 1 -- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 msgr2=0x55bc236d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097f7fe700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097f7fe700 1 -- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 msgr2=0x55bc236ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097f7fe700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097f7fe700 1 -- 172.21.15.92:0/2169239752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096800e040 con 0x55bc236f5c00 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097f7fe700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236d4940 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0978012580 tx=0x7f0978014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2626e78d6fbd01d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f095d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f095d7fa700 1 -- 172.21.15.92:0/2169239752 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097806e070 con 0x55bc236f5c00 2023-10-06T03:09:39.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f095d7fa700 1 -- 172.21.15.92:0/2169239752 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0978057e60 con 0x55bc236f5c00 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc2369a940 con 0x55bc236f5c00 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f095d7fa700 1 -- 172.21.15.92:0/2169239752 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f097800e040 con 0x55bc236f5c00 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 10 client.8148 fetch_fsmap learned FSMap version 104 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55bc23432fe0 con 0x55bc236f5c00 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f095d7fa700 1 -- 172.21.15.92:0/2169239752 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978083070 con 0x55bc236f5c00 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f095d7fa700 1 -- 172.21.15.92:0/2169239752 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 104) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0978057350 con 0x55bc236f5c00 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 10 client.8148 fetch_fsmap finished waiting for FSMap version 104 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 10 client.8148 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 20 client.8148 populate_metadata read hostname 'smithi092' 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d0410 0x55bc236d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 4 client.8148 mds_command: new command op to 7992 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:09:39.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 --> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc23677870 con 0x55bc236d0410 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.773+0000 7f097effd700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d0410 0x55bc236d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f095d7fa700 10 client.8148 ms_handle_connect on v2:172.21.15.92:6836/2005327996 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f097effd700 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d0410 0x55bc236d07e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cd6c6541dbba6e server_cookie=cb416b3dbaccd941 in_seq=0 out_seq=0 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f095d7fa700 1 -- 172.21.15.92:0/2169239752 <== mds.0 v2:172.21.15.92:6836/2005327996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55bc23677870 con 0x55bc236d0410 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f095d7fa700 10 client.8148 handle_command_reply: tid=0 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 client.8148 shutdown 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 msgr2=0x55bc236d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236d4940 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f0978012580 tx=0x7f0978014040 comp rx=0 tx=0).stop 2023-10-06T03:09:39.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 shutdown_connections 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d0410 0x55bc236d07e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236d4940 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f095d7fa700 0 client.8148 ms_handle_reset on v2:172.21.15.92:6836/2005327996 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 --2- 172.21.15.92:0/2169239752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.774+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 >> 172.21.15.92:0/2169239752 conn(0x55bc236a7160 msgr2=0x55bc236a61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.775+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 shutdown_connections 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.775+0000 7f098ea45b80 1 -- 172.21.15.92:0/2169239752 wait complete. 2023-10-06T03:09:39.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.775+0000 7f098ea45b80 20 client.8148 trim_cache size 0 max 16384 2023-10-06T03:09:39.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.791+0000 7f098ea45b80 1 Processor -- start 2023-10-06T03:09:39.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 -- start start 2023-10-06T03:09:39.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a5840 0x55bc236a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a6cc0 0x55bc236e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc236a7b80 con 0x55bc236f5c00 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc236a6b20 con 0x55bc236a6cc0 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc236d3080 con 0x55bc236a5840 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37306/0 (socket says 172.21.15.92:37306) 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.792+0000 7f097f7fe700 1 -- 172.21.15.92:0/369755358 learned_addr learned my addr 172.21.15.92:0/369755358 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f097f7fe700 1 -- 172.21.15.92:0/369755358 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a5840 msgr2=0x55bc236a7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f097f7fe700 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a5840 0x55bc236a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f097f7fe700 1 -- 172.21.15.92:0/369755358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a6cc0 msgr2=0x55bc236e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f097f7fe700 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a6cc0 0x55bc236e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f097f7fe700 1 -- 172.21.15.92:0/369755358 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc235c4780 con 0x55bc236f5c00 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f097f7fe700 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236cd7c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f0978057b60 tx=0x7f0978029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941caa06fb4d1e7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f0974ff9700 1 -- 172.21.15.92:0/369755358 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800e040 con 0x55bc236f5c00 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f0974ff9700 1 -- 172.21.15.92:0/369755358 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978062070 con 0x55bc236f5c00 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f0974ff9700 1 -- 172.21.15.92:0/369755358 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097806e040 con 0x55bc236f5c00 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 -- 172.21.15.92:0/369755358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 msgr2=0x55bc236cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236cd7c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f0978057b60 tx=0x7f0978029000 comp rx=0 tx=0).stop 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 -- 172.21.15.92:0/369755358 shutdown_connections 2023-10-06T03:09:39.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc236cd7c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a5840 0x55bc236a7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 --2- 172.21.15.92:0/369755358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a6cc0 0x55bc236e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 -- 172.21.15.92:0/369755358 >> 172.21.15.92:0/369755358 conn(0x55bc236a7160 msgr2=0x55bc2369e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.793+0000 7f098ea45b80 1 -- 172.21.15.92:0/369755358 shutdown_connections 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.794+0000 7f098ea45b80 1 -- 172.21.15.92:0/369755358 wait complete. 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.794+0000 7f098ea45b80 1 -- start start 2023-10-06T03:09:39.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.794+0000 7f098ea45b80 1 Processor -- start 2023-10-06T03:09:39.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc23754c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc237527b0 con 0x55bc236f5c00 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc237547e0 con 0x55bc236a5840 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc23751620 con 0x55bc236a6cc0 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f098ea45b80 10 client.0 _ll_register_callbacks cb 0x55bc236efd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc23754c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc23754c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37320/0 (socket says 172.21.15.92:37320) 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 -- 172.21.15.92:0/1844423539 learned_addr learned my addr 172.21.15.92:0/1844423539 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f09767fc700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 -- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 msgr2=0x55bc236d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097effd700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 -- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 msgr2=0x55bc236d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 -- 172.21.15.92:0/1844423539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc23432fe0 con 0x55bc236f5c00 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f097f7fe700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc23754c80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f097805a010 tx=0x7f0978014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61defce3e5df30e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.795+0000 7f095d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f095d7fa700 1 -- 172.21.15.92:0/1844423539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097800e040 con 0x55bc236f5c00 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f095d7fa700 1 -- 172.21.15.92:0/1844423539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f097805ee00 con 0x55bc236f5c00 2023-10-06T03:09:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f096800d700 con 0x55bc236f5c00 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f095d7fa700 1 -- 172.21.15.92:0/1844423539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f097805b0b0 con 0x55bc236f5c00 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 10 client.8154 fetch_fsmap learned FSMap version 104 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55bc235ab470 con 0x55bc236f5c00 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f095d7fa700 1 -- 172.21.15.92:0/1844423539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978078020 con 0x55bc236f5c00 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f095d7fa700 1 -- 172.21.15.92:0/1844423539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 104) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0978057350 con 0x55bc236f5c00 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 10 client.8154 fetch_fsmap finished waiting for FSMap version 104 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 10 client.8154 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 20 client.8154 populate_metadata read hostname 'smithi092' 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d4470 0x55bc237551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 4 client.8154 mds_command: new command op to 7992 tid=0[{"prefix": "status"}] 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 --> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc23598e00 con 0x55bc236d4470 2023-10-06T03:09:39.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f097effd700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d4470 0x55bc237551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f095d7fa700 10 client.8154 ms_handle_connect on v2:172.21.15.92:6836/2005327996 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.796+0000 7f097effd700 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d4470 0x55bc237551c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b640cc9fdabae1 server_cookie=74a2d3ce0975636e in_seq=0 out_seq=0 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f095d7fa700 1 -- 172.21.15.92:0/1844423539 <== mds.0 v2:172.21.15.92:6836/2005327996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55bc23598e00 con 0x55bc236d4470 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f095d7fa700 10 client.8154 handle_command_reply: tid=0 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 client.8154 shutdown 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 msgr2=0x55bc23754c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc23754c80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f097805a010 tx=0x7f0978014070 comp rx=0 tx=0).stop 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 shutdown_connections 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:6836/2005327996,v1:172.21.15.92:6837/2005327996] conn(0x55bc236d4470 0x55bc237551c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bc236f5c00 0x55bc23754c80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f095d7fa700 0 client.8154 ms_handle_reset on v2:172.21.15.92:6836/2005327996 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bc236a5840 0x55bc236d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 --2- 172.21.15.92:0/1844423539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55bc236a6cc0 0x55bc236d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 >> 172.21.15.92:0/1844423539 conn(0x55bc236a7160 msgr2=0x55bc2369e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 shutdown_connections 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 1 -- 172.21.15.92:0/1844423539 wait complete. 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098ea45b80 20 client.8154 trim_cache size 0 max 16384 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:09:39.806 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 7992, 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.8021957640000004, 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 104, 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 155, 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 155, 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.936817312000001 2023-10-06T03:09:39.807 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:09:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0960067320 msgr2=0x7f09600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0960067320 0x7f09600697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f09700011e0 tx=0x7f0970009040 comp rx=0 tx=0).stop 2023-10-06T03:09:39.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 msgr2=0x7f098016d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f098016d0c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f0980070e20 tx=0x7f0978019000 comp rx=0 tx=0).stop 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 shutdown_connections 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0960067320 0x7f09600697d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f098015db10 0x7f098016d0c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0980159890 0x7f0980149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 --2- 172.21.15.92:0/2834294972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0980159c60 0x7f0980151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.797+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 >> 172.21.15.92:0/2834294972 conn(0x7f0980068e20 msgr2=0x7f0980155010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.798+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 shutdown_connections 2023-10-06T03:09:39.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.798+0000 7f098574d700 1 -- 172.21.15.92:0/2834294972 wait complete. 2023-10-06T03:09:39.811 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:09:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.924+0000 7f38106bf700 1 Processor -- start 2023-10-06T03:09:39.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.924+0000 7f38106bf700 1 -- start start 2023-10-06T03:09:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f38106bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f380806bcd0 0x7f38080caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f38106bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806c0a0 0x7f380815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f38106bf700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f380815df50 0x7f380815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f38106bf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380815d7e0 con 0x7f380806bcd0 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f38106bf700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380815dac0 con 0x7f380815df50 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f38106bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380815aa30 con 0x7f380806c0a0 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f380e45b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f380806bcd0 0x7f38080caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f380e45b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f380806bcd0 0x7f38080caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37332/0 (socket says 172.21.15.92:37332) 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.925+0000 7f380e45b700 1 -- 172.21.15.92:0/3381007177 learned_addr learned my addr 172.21.15.92:0/3381007177 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380e45b700 1 -- 172.21.15.92:0/3381007177 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806c0a0 msgr2=0x7f380815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380ec5c700 1 --2- 172.21.15.92:0/3381007177 >> v2:172.21.15.136:3300/0 conn(0x7f380815df50 0x7f380815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380dc5a700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806c0a0 0x7f380815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380e45b700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806c0a0 0x7f380815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380e45b700 1 -- 172.21.15.92:0/3381007177 >> v2:172.21.15.136:3300/0 conn(0x7f380815df50 msgr2=0x7f380815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380e45b700 1 --2- 172.21.15.92:0/3381007177 >> v2:172.21.15.136:3300/0 conn(0x7f380815df50 0x7f380815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380e45b700 1 -- 172.21.15.92:0/3381007177 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380815ad10 con 0x7f380806bcd0 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380ec5c700 1 --2- 172.21.15.92:0/3381007177 >> v2:172.21.15.136:3300/0 conn(0x7f380815df50 0x7f380815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380dc5a700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806c0a0 0x7f380815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380e45b700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806bcd0 0x7f38080caa90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f38080b4600 tx=0x7f3804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee1078473e757e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380d459700 1 -- 172.21.15.92:0/3381007177 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380400b860 con 0x7f380806bcd0 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380d459700 1 -- 172.21.15.92:0/3381007177 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380400b9c0 con 0x7f380806bcd0 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f380d459700 1 -- 172.21.15.92:0/3381007177 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380400bd80 con 0x7f380806bcd0 2023-10-06T03:09:39.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 -- 172.21.15.92:0/3381007177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806bcd0 msgr2=0x7f38080caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806bcd0 0x7f38080caa90 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f38080b4600 tx=0x7f3804009250 comp rx=0 tx=0).stop 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 -- 172.21.15.92:0/3381007177 shutdown_connections 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 --2- 172.21.15.92:0/3381007177 >> v2:172.21.15.136:3300/0 conn(0x7f380815df50 0x7f380815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806bcd0 0x7f38080caa90 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 --2- 172.21.15.92:0/3381007177 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806c0a0 0x7f380815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.926+0000 7f38106bf700 1 -- 172.21.15.92:0/3381007177 >> 172.21.15.92:0/3381007177 conn(0x7f38080ce600 msgr2=0x7f38080cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.927+0000 7f38106bf700 1 -- 172.21.15.92:0/3381007177 shutdown_connections 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.927+0000 7f38106bf700 1 -- 172.21.15.92:0/3381007177 wait complete. 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 Processor -- start 2023-10-06T03:09:39.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 -- start start 2023-10-06T03:09:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806bcd0 0x7f3808153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 0x7f38080b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380815df50 0x7f38080c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080c6c90 con 0x7f380806c0a0 2023-10-06T03:09:39.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b1e30 con 0x7f380815df50 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f38106bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b2110 con 0x7f380806bcd0 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f380dc5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 0x7f38080b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f380dc5a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 0x7f38080b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37338/0 (socket says 172.21.15.92:37338) 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f380dc5a700 1 -- 172.21.15.92:0/4266459250 learned_addr learned my addr 172.21.15.92:0/4266459250 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.928+0000 7f380e45b700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806bcd0 0x7f3808153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380ec5c700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380815df50 0x7f38080c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380dc5a700 1 -- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806bcd0 msgr2=0x7f3808153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380dc5a700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806bcd0 0x7f3808153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380dc5a700 1 -- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380815df50 msgr2=0x7f38080c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380dc5a700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380815df50 0x7f38080c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:39.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380dc5a700 1 -- 172.21.15.92:0/4266459250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040094f0 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f380dc5a700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 0x7f38080b14c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c880 tx=0x7f37f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8011600 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8011760 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38080b23f0 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8011b20 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.929+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3808153a60 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.930+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380804c3c0 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.930+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f37f80193b0 con 0x7f380806c0a0 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.931+0000 7f37fe7fc700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37ec067320 0x7f37ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:39.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.931+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f37f8084d00 con 0x7f380806c0a0 2023-10-06T03:09:39.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.931+0000 7f380e45b700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37ec067320 0x7f37ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:39.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.931+0000 7f380e45b700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37ec067320 0x7f37ec0697d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f38080b4600 tx=0x7f3804017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:39.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:39.935+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37f80551b0 con 0x7f380806c0a0 2023-10-06T03:09:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.166+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f380815aff0 con 0x7f380806c0a0 2023-10-06T03:09:40.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.167+0000 7f37fe7fc700 1 -- 172.21.15.92:0/4266459250 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7f37f801f070 con 0x7f380806c0a0 2023-10-06T03:09:40.168 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:09:40.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37ec067320 msgr2=0x7f37ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37ec067320 0x7f37ec0697d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f38080b4600 tx=0x7f3804017000 comp rx=0 tx=0).stop 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 msgr2=0x7f38080b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 0x7f38080b14c0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f37f800c880 tx=0x7f37f8008040 comp rx=0 tx=0).stop 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 shutdown_connections 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f37ec067320 0x7f37ec0697d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f380815df50 0x7f38080c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f380806bcd0 0x7f3808153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 --2- 172.21.15.92:0/4266459250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f380806c0a0 0x7f38080b14c0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 >> 172.21.15.92:0/4266459250 conn(0x7f38080ce600 msgr2=0x7f38081565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:40.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 shutdown_connections 2023-10-06T03:09:40.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.169+0000 7f38106bf700 1 -- 172.21.15.92:0/4266459250 wait complete. 2023-10-06T03:09:40.182 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:09:40.183 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:09:40.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.183 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:09:40.198 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:40.199 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 42026sec preferred_lft 42026sec 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:40.200 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:09:40.201 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:40.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.202 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:09:40.261 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:09:40.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.263 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:09:40.319 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:09:40.319 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:09:40.319 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:09:40.319 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:09:40.319 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:09:40.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.320 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:09:40.320 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:09:40.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:09:40.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:09:40.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:09:40.320 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:09:40.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.419 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:09:40.434 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:09:40.434 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:09:40.434 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:09:40.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.434 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:09:40.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:40.499 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:40.554 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:09:40.554 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:40.555 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:40.608 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:09:40.608 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:09:40.608 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:40.688 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:40.688 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:40.707 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:40.729 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:40.729 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:40.729 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:40.729 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:40.729 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:09:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:09:40.730 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:09:40.731 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:09:40.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:40.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:40.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:09:40.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.949+0000 7f09b8852700 1 Processor -- start 2023-10-06T03:09:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.949+0000 7f09b8852700 1 -- start start 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b8852700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09b4159c60 0x7f09b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b8852700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b415a160 con 0x7f09b4159890 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b8852700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b40d13a0 con 0x7f09b415db10 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b8852700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b415df50 con 0x7f09b4159c60 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b1d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09b4159c60 0x7f09b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09b4159890 0x7f09b415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47382/0 (socket says 172.21.15.92:47382) 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b259c700 1 -- 172.21.15.92:0/1955968661 learned_addr learned my addr 172.21.15.92:0/1955968661 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b1d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09b4159c60 0x7f09b415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37354/0 (socket says 172.21.15.92:37354) 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b1d9b700 1 -- 172.21.15.92:0/1955968661 >> v1:172.21.15.136:6789/0 conn(0x7f09b415db10 legacy=0x7f09b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b1d9b700 1 -- 172.21.15.92:0/1955968661 >> v1:172.21.15.92:6789/0 conn(0x7f09b4159890 legacy=0x7f09b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:40.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b1d9b700 1 -- 172.21.15.92:0/1955968661 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b415e140 con 0x7f09b4159c60 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b159a700 1 -- 172.21.15.92:0/1955968661 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549888999 0 0) 0x7f09b415a160 con 0x7f09b4159890 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b1d9b700 1 --2- 172.21.15.92:0/1955968661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159c60 0x7f09b415d5d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f09a80098e0 tx=0x7f09a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82115c387b71541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b159a700 1 -- 172.21.15.92:0/1955968661 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8009e80 con 0x7f09b4159c60 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.950+0000 7f09b159a700 1 -- 172.21.15.92:0/1955968661 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800b820 con 0x7f09b4159c60 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b159a700 1 -- 172.21.15.92:0/1955968661 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800bba0 con 0x7f09b4159c60 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 -- 172.21.15.92:0/1955968661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159c60 msgr2=0x7f09b415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 --2- 172.21.15.92:0/1955968661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159c60 0x7f09b415d5d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f09a80098e0 tx=0x7f09a80091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 -- 172.21.15.92:0/1955968661 shutdown_connections 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 --2- 172.21.15.92:0/1955968661 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159c60 0x7f09b415d5d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 -- 172.21.15.92:0/1955968661 >> 172.21.15.92:0/1955968661 conn(0x7f09b4068e20 msgr2=0x7f09b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:40.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 -- 172.21.15.92:0/1955968661 shutdown_connections 2023-10-06T03:09:40.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.951+0000 7f09b8852700 1 -- 172.21.15.92:0/1955968661 wait complete. 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.952+0000 7f09b8852700 1 Processor -- start 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.952+0000 7f09b8852700 1 -- start start 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b8852700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 0x7f09b40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b8852700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b4159c60 0x7f09b40b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b8852700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b415db10 0x7f09b40b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b8852700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40cd410 con 0x7f09b4159890 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b8852700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40b6230 con 0x7f09b4159c60 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b8852700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b40cd6d0 con 0x7f09b415db10 2023-10-06T03:09:40.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 0x7f09b40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 0x7f09b40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37356/0 (socket says 172.21.15.92:37356) 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 -- 172.21.15.92:0/3842162460 learned_addr learned my addr 172.21.15.92:0/3842162460 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 -- 172.21.15.92:0/3842162460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b415db10 msgr2=0x7f09b40b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b415db10 0x7f09b40b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 -- 172.21.15.92:0/3842162460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b4159c60 msgr2=0x7f09b40b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b4159c60 0x7f09b40b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 -- 172.21.15.92:0/3842162460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009420 con 0x7f09b4159890 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.953+0000 7f09b259c700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 0x7f09b40b3590 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c5b0 tx=0x7f09a4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.954+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4005b30 con 0x7f09b4159890 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.954+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a4005c90 con 0x7f09b4159890 2023-10-06T03:09:40.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.954+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b40cd9b0 con 0x7f09b4159890 2023-10-06T03:09:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.954+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007300 con 0x7f09b4159890 2023-10-06T03:09:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.954+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b40b5ec0 con 0x7f09b4159890 2023-10-06T03:09:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.954+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b404c3c0 con 0x7f09b4159890 2023-10-06T03:09:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.955+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f09a401b3b0 con 0x7f09b4159890 2023-10-06T03:09:40.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.955+0000 7f09a27fc700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0994067320 0x7f09940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.955+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09a4084320 con 0x7f09b4159890 2023-10-06T03:09:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.955+0000 7f09b1d9b700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0994067320 0x7f09940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:40.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.956+0000 7f09b1d9b700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0994067320 0x7f09940697d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f09b40cf7d0 tx=0x7f09a8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:40.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:40.959+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09a4050e50 con 0x7f09b4159890 2023-10-06T03:09:41.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.190+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f09b40b0120 con 0x7f09b4159890 2023-10-06T03:09:41.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.191+0000 7f09a27fc700 1 -- 172.21.15.92:0/3842162460 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7f09a4017060 con 0x7f09b4159890 2023-10-06T03:09:41.192 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0994067320 msgr2=0x7f09940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0994067320 0x7f09940697d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f09b40cf7d0 tx=0x7f09a8016000 comp rx=0 tx=0).stop 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 msgr2=0x7f09b40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 0x7f09b40b3590 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c5b0 tx=0x7f09a4001070 comp rx=0 tx=0).stop 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 shutdown_connections 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0994067320 0x7f09940697d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4159890 0x7f09b40b3590 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b415db10 0x7f09b40b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 --2- 172.21.15.92:0/3842162460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b4159c60 0x7f09b40b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 >> 172.21.15.92:0/3842162460 conn(0x7f09b4068e20 msgr2=0x7f09b41532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:41.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 shutdown_connections 2023-10-06T03:09:41.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.193+0000 7f09b8852700 1 -- 172.21.15.92:0/3842162460 wait complete. 2023-10-06T03:09:41.206 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:09:41.207 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:09:41.207 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:09:41.207 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:09:41.207 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:41.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:09:41.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:09:41.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:09:41.208 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:09:41.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.209 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:09:41.228 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:09:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:09:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:09:41.229 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:09:41.230 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.230 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:09:41.231 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:41.231 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:09:41.232 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:09:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:41.232 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:09:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:09:41.232 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 42026sec preferred_lft 42026sec 2023-10-06T03:09:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:09:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.233 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:09:41.233 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:09:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:09:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:09:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.234 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:41.234 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:09:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:09:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.235 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:09:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:09:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:09:41.235 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:09:41.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.237 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:09:41.295 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:09:41.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.297 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:09:41.357 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:09:41.358 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:09:41.358 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:09:41.358 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:09:41.358 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:09:41.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.358 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:09:41.358 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:09:41.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:09:41.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:09:41.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:09:41.359 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:09:41.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.463 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:09:41.477 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:09:41.478 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:09:41.478 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:09:41.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.478 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:09:41.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:41.543 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:41.598 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:09:41.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:09:41.599 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:41.653 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:09:41.654 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:09:41.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:41.740 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:41.740 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:41.758 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:41.758 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:41.758 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:41.759 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:41.759 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:41.760 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:41.760 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:41.760 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:41.780 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:09:41.780 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:09:41.781 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:09:41.782 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:09:41.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:41.782 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:41.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.970+0000 7f90f8d1a700 1 Processor -- start 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.970+0000 7f90f8d1a700 1 -- start start 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f8d1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 0x7f90f40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f8d1a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40b5b00 con 0x7f90f40b53d0 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f8d1a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40b5da0 con 0x7f90f40cf350 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f8d1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40b6080 con 0x7f90f4159ec0 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90e9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f90f40b53d0 0x7f90f40cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47388/0 (socket says 172.21.15.92:47388) 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90e9d9b700 1 -- 172.21.15.92:0/763095042 learned_addr learned my addr 172.21.15.92:0/763095042 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:41.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f259c700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 0x7f90f40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146623905 0 0) 0x7f90f40b5b00 con 0x7f90f40b53d0 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e0003570 con 0x7f90f40b53d0 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627594378 0 0) 0x7f90e0003570 con 0x7f90f40b53d0 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 msgr2=0x7f90f40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 0x7f90f40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 >> v1:172.21.15.136:6789/0 conn(0x7f90f40cf350 legacy=0x7f90f40cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40cc100 con 0x7f90f40b53d0 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.971+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f90dc001470 con 0x7f90f40b53d0 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 >> v1:172.21.15.92:6789/0 conn(0x7f90f40b53d0 legacy=0x7f90f40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e0003a90 0x7f90e0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 0x7f90e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e0004d50 0x7f90e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40b6080 con 0x7f90e0004d50 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e000bb70 con 0x7f90e0003a90 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e000bd30 con 0x7f90f4159ec0 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e0004d50 0x7f90e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f259c700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e0003a90 0x7f90e0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90e9d9b700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 0x7f90e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 -- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 msgr2=0x7f90e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4159ec0 0x7f90e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 -- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e0003a90 msgr2=0x7f90e0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e0003a90 0x7f90e0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 -- 172.21.15.92:0/763095042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0003570 con 0x7f90e0004d50 2023-10-06T03:09:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f2d9d700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e0004d50 0x7f90e00051c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f90ec001fc0 tx=0x7f90ec001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d40195b41c09c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec001950 con 0x7f90e0004d50 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00c8b0 con 0x7f90e0004d50 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.972+0000 7f90f1d9b700 1 -- 172.21.15.92:0/763095042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00cc50 con 0x7f90e0004d50 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f8d1a700 1 -- 172.21.15.92:0/763095042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e0004d50 msgr2=0x7f90e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/763095042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e0004d50 0x7f90e00051c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f90ec001fc0 tx=0x7f90ec001640 comp rx=0 tx=0).stop 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f2d9d700 1 -- 172.21.15.92:0/763095042 reap_dead start 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f8d1a700 1 -- 172.21.15.92:0/763095042 shutdown_connections 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f8d1a700 1 -- 172.21.15.92:0/763095042 >> 172.21.15.92:0/763095042 conn(0x7f90f40999f0 msgr2=0x7f90f40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f8d1a700 1 -- 172.21.15.92:0/763095042 shutdown_connections 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.973+0000 7f90f8d1a700 1 -- 172.21.15.92:0/763095042 wait complete. 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.974+0000 7f90f8d1a700 1 Processor -- start 2023-10-06T03:09:41.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.974+0000 7f90f8d1a700 1 -- start start 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 0x7f90f41697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4157390 0x7f90f409c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90f406b0b0 0x7f90f409a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f409d090 con 0x7f90f4156fc0 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4068390 con 0x7f90f406b0b0 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4068f10 con 0x7f90f4157390 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 0x7f90f41697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 0x7f90f41697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37384/0 (socket says 172.21.15.92:37384) 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 -- 172.21.15.92:0/1068089445 learned_addr learned my addr 172.21.15.92:0/1068089445 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 -- 172.21.15.92:0/1068089445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4157390 msgr2=0x7f90f409c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4157390 0x7f90f409c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 -- 172.21.15.92:0/1068089445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90f406b0b0 msgr2=0x7f90f409a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90f406b0b0 0x7f90f409a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 -- 172.21.15.92:0/1068089445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40685e0 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f259c700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 0x7f90f41697f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f90e4011b40 tx=0x7f90e4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4005c10 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4005d70 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40688c0 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e400fa90 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.975+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f4169d30 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.976+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f4063c90 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.977+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f90e4006050 con 0x7f90f4156fc0 2023-10-06T03:09:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.977+0000 7f90eaffd700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d4067320 0x7f90d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.977+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f90e40857e0 con 0x7f90f4156fc0 2023-10-06T03:09:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.977+0000 7f90e9d9b700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d4067320 0x7f90d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.977+0000 7f90e9d9b700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d4067320 0x7f90d40697d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f90f4157760 tx=0x7f90dc001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:41.981+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90e4052310 con 0x7f90f4156fc0 2023-10-06T03:09:42.216 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:42.214+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f90f40718b0 con 0x7f90f4156fc0 2023-10-06T03:09:42.864 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:09:42.866 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:09:43.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.042+0000 7f90eaffd700 1 -- 172.21.15.92:0/1068089445 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v105) v1 ==== 136+0+0 (secure 0 0 0) 0x7f90e4055930 con 0x7f90f4156fc0 2023-10-06T03:09:43.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d4067320 msgr2=0x7f90d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d4067320 0x7f90d40697d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f90f4157760 tx=0x7f90dc001e50 comp rx=0 tx=0).stop 2023-10-06T03:09:43.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 msgr2=0x7f90f41697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 0x7f90f41697f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f90e4011b40 tx=0x7f90e4009120 comp rx=0 tx=0).stop 2023-10-06T03:09:43.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 shutdown_connections 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d4067320 0x7f90d40697d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f4156fc0 0x7f90f41697f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90f406b0b0 0x7f90f409a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 --2- 172.21.15.92:0/1068089445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90f4157390 0x7f90f409c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 >> 172.21.15.92:0/1068089445 conn(0x7f90f40999f0 msgr2=0x7f90f4158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 shutdown_connections 2023-10-06T03:09:43.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.044+0000 7f90f8d1a700 1 -- 172.21.15.92:0/1068089445 wait complete. 2023-10-06T03:09:43.060 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:09:43.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.176+0000 7f020731d700 1 Processor -- start 2023-10-06T03:09:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.176+0000 7f020731d700 1 -- start start 2023-10-06T03:09:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f020731d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f020731d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 0x7f020006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f020731d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020015af50 con 0x7f02001598d0 2023-10-06T03:09:43.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f020731d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020006e980 con 0x7f020006bbe0 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f020731d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000ce180 con 0x7f02000c7bd0 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f0205b1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f0206b1c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 0x7f020006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f0205b1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:60852/0 (socket says 172.21.15.92:60852) 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f0206b1c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 0x7f020006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:44838/0 (socket says 172.21.15.92:44838) 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f020631b700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f02001598d0 0x7f020006ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:58240/0 (socket says 172.21.15.92:58240) 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.177+0000 7f0205b1a700 1 -- 172.21.15.92:0/2603712885 learned_addr learned my addr 172.21.15.92:0/2603712885 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:43.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205b1a700 1 -- 172.21.15.92:0/2603712885 >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 msgr2=0x7f020006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205b1a700 1 --2- 172.21.15.92:0/2603712885 >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 0x7f020006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205b1a700 1 -- 172.21.15.92:0/2603712885 >> v1:172.21.15.136:6789/0 conn(0x7f02001598d0 legacy=0x7f020006ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205b1a700 1 -- 172.21.15.92:0/2603712885 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000c80f0 con 0x7f02000c7bd0 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205319700 1 -- 172.21.15.92:0/2603712885 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381372661 0 0) 0x7f01fc001490 con 0x7f02001598d0 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0206b1c700 1 --2- 172.21.15.92:0/2603712885 >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 0x7f020006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205b1a700 1 --2- 172.21.15.92:0/2603712885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015aa10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01f00099b0 tx=0x7f01f0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f29256c900d00b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205319700 1 -- 172.21.15.92:0/2603712885 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0009ea0 con 0x7f02000c7bd0 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205319700 1 -- 172.21.15.92:0/2603712885 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000b860 con 0x7f02000c7bd0 2023-10-06T03:09:43.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f0205319700 1 -- 172.21.15.92:0/2603712885 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000bc00 con 0x7f02000c7bd0 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f020731d700 1 -- 172.21.15.92:0/2603712885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 msgr2=0x7f020015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.178+0000 7f020731d700 1 --2- 172.21.15.92:0/2603712885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015aa10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f01f00099b0 tx=0x7f01f0009250 comp rx=0 tx=0).stop 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.179+0000 7f020731d700 1 -- 172.21.15.92:0/2603712885 shutdown_connections 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.179+0000 7f020731d700 1 --2- 172.21.15.92:0/2603712885 >> v2:172.21.15.136:3300/0 conn(0x7f020006bbe0 0x7f020006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.179+0000 7f020731d700 1 --2- 172.21.15.92:0/2603712885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015aa10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.179+0000 7f020731d700 1 -- 172.21.15.92:0/2603712885 >> 172.21.15.92:0/2603712885 conn(0x7f020006b490 msgr2=0x7f02000cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.179+0000 7f020731d700 1 -- 172.21.15.92:0/2603712885 shutdown_connections 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.179+0000 7f020731d700 1 -- 172.21.15.92:0/2603712885 wait complete. 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 Processor -- start 2023-10-06T03:09:43.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 -- start start 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 0x7f020016d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02001598d0 0x7f020015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020015d860 con 0x7f020006bbe0 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020016a230 con 0x7f02001598d0 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020731d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0200156110 con 0x7f02000c7bd0 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.180+0000 7f020631b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 0x7f020016d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 0x7f020016d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37396/0 (socket says 172.21.15.92:37396) 2023-10-06T03:09:43.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 -- 172.21.15.92:0/1804213900 learned_addr learned my addr 172.21.15.92:0/1804213900 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f0206b1c700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02001598d0 0x7f020015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f0205b1a700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 -- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 msgr2=0x7f020015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 -- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02001598d0 msgr2=0x7f020015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02001598d0 0x7f020015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 -- 172.21.15.92:0/1804213900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f8009420 con 0x7f020006bbe0 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f0206b1c700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02001598d0 0x7f020015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020631b700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 0x7f020016d0c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00cdd0 tx=0x7f01fc001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc005df0 con 0x7f020006bbe0 2023-10-06T03:09:43.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00a960 con 0x7f020006bbe0 2023-10-06T03:09:43.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f00094f0 con 0x7f020006bbe0 2023-10-06T03:09:43.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00ad20 con 0x7f020006bbe0 2023-10-06T03:09:43.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.181+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020015c350 con 0x7f020006bbe0 2023-10-06T03:09:43.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.182+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02000c79d0 con 0x7f020006bbe0 2023-10-06T03:09:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.183+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f01fc007300 con 0x7f020006bbe0 2023-10-06T03:09:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.183+0000 7f01f67fc700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f01e4067320 0x7f01e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.183+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01fc084db0 con 0x7f020006bbe0 2023-10-06T03:09:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.183+0000 7f0205b1a700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f01e4067320 0x7f01e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.183+0000 7f0205b1a700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f01e4067320 0x7f01e40697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02000c8020 tx=0x7f01f0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.187+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01fc089070 con 0x7f020006bbe0 2023-10-06T03:09:43.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.395+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f020016d600 con 0x7f01e4067320 2023-10-06T03:09:43.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.396+0000 7f01f67fc700 1 -- 172.21.15.92:0/1804213900 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f01f0000d00 con 0x7f01e4067320 2023-10-06T03:09:43.397 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:09:43.398 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:43.398 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:09:43.398 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:43.398 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f01e4067320 msgr2=0x7f01e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f01e4067320 0x7f01e40697d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f02000c8020 tx=0x7f01f0017000 comp rx=0 tx=0).stop 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 msgr2=0x7f020016d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 0x7f020016d0c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00cdd0 tx=0x7f01fc001b90 comp rx=0 tx=0).stop 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 shutdown_connections 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f01e4067320 0x7f01e40697d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f020006bbe0 0x7f020016d0c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02001598d0 0x7f020015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 --2- 172.21.15.92:0/1804213900 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02000c7bd0 0x7f020015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 >> 172.21.15.92:0/1804213900 conn(0x7f020006b490 msgr2=0x7f02000b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:43.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 shutdown_connections 2023-10-06T03:09:43.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.398+0000 7f020731d700 1 -- 172.21.15.92:0/1804213900 wait complete. 2023-10-06T03:09:43.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.527+0000 7f8c8af93700 1 Processor -- start 2023-10-06T03:09:43.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.527+0000 7f8c8af93700 1 -- start start 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.528+0000 7f8c8af93700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8c84069560 0x7f8c840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.528+0000 7f8c8af93700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c84069ee0 con 0x7f8c84159c60 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.528+0000 7f8c8af93700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8415a160 con 0x7f8c84159890 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.528+0000 7f8c8af93700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c840d13a0 con 0x7f8c84069560 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.528+0000 7f8c7bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8c84159c60 0x7f8c8415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47398/0 (socket says 172.21.15.92:47398) 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.528+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3021460193 learned_addr learned my addr 172.21.15.92:0/3021460193 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336175645 0 0) 0x7f8c84069ee0 con 0x7f8c84159c60 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6c003570 con 0x7f8c84159c60 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c89530700 1 --2- 172.21.15.92:0/3021460193 >> v2:172.21.15.136:3300/0 conn(0x7f8c84069560 0x7f8c840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395801982 0 0) 0x7f8c6c003570 con 0x7f8c84159c60 2023-10-06T03:09:43.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 >> v2:172.21.15.136:3300/0 conn(0x7f8c84069560 msgr2=0x7f8c840699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3021460193 >> v2:172.21.15.136:3300/0 conn(0x7f8c84069560 0x7f8c840699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 >> v1:172.21.15.136:6789/0 conn(0x7f8c84159890 legacy=0x7f8c8415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8415e140 con 0x7f8c84159c60 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8c74001470 con 0x7f8c84159c60 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 >> v1:172.21.15.92:6789/0 conn(0x7f8c84159c60 legacy=0x7f8c8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c003a60 0x7f8c6c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c6c004810 0x7f8c6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c0051c0 0x7f8c6c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840d13a0 con 0x7f8c6c004810 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c00c0d0 con 0x7f8c6c0051c0 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c00c290 con 0x7f8c6c003a60 2023-10-06T03:09:43.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c6c004810 0x7f8c6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c88d2f700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c003a60 0x7f8c6c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c89530700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c0051c0 0x7f8c6c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c003a60 msgr2=0x7f8c6c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c003a60 0x7f8c6c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c0051c0 msgr2=0x7f8c6c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c6c0051c0 0x7f8c6c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c7bfff700 1 -- 172.21.15.92:0/3021460193 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c003570 con 0x7f8c6c004810 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.529+0000 7f8c89530700 1 -- 172.21.15.92:0/3021460193 reap_dead start 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c6c004810 0x7f8c6c004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c3c0 tx=0x7f8c74001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb83c7c3236fcbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400cc10 con 0x7f8c6c004810 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7400cd70 con 0x7f8c6c004810 2023-10-06T03:09:43.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c7b7fe700 1 -- 172.21.15.92:0/3021460193 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74005b50 con 0x7f8c6c004810 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 -- 172.21.15.92:0/3021460193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c6c004810 msgr2=0x7f8c6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c6c004810 0x7f8c6c004c80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c3c0 tx=0x7f8c74001e50 comp rx=0 tx=0).stop 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 -- 172.21.15.92:0/3021460193 shutdown_connections 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 --2- 172.21.15.92:0/3021460193 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c6c004810 0x7f8c6c004c80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 -- 172.21.15.92:0/3021460193 >> 172.21.15.92:0/3021460193 conn(0x7f8c84068e20 msgr2=0x7f8c840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 -- 172.21.15.92:0/3021460193 shutdown_connections 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.530+0000 7f8c8af93700 1 -- 172.21.15.92:0/3021460193 wait complete. 2023-10-06T03:09:43.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.531+0000 7f8c8af93700 1 Processor -- start 2023-10-06T03:09:43.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 -- start start 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c004810 0x7f8c840cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c8416aac0 0x7f8c84154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 0x7f8c840b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8416bc20 con 0x7f8c840b5e10 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840cd600 con 0x7f8c8416aac0 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c8af93700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8416b010 con 0x7f8c6c004810 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c89530700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 0x7f8c840b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c89530700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 0x7f8c840b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37414/0 (socket says 172.21.15.92:37414) 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c89530700 1 -- 172.21.15.92:0/1109622656 learned_addr learned my addr 172.21.15.92:0/1109622656 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c88d2f700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c004810 0x7f8c840cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.539 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.532+0000 7f8c7bfff700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c8416aac0 0x7f8c84154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c89530700 1 -- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c004810 msgr2=0x7f8c840cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c89530700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c004810 0x7f8c840cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c89530700 1 -- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c8416aac0 msgr2=0x7f8c84154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c89530700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c8416aac0 0x7f8c84154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c89530700 1 -- 172.21.15.92:0/1109622656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8416b2f0 con 0x7f8c840b5e10 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c89530700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 0x7f8c840b3fa0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8c8406a300 tx=0x7f8c7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00f7d0 con 0x7f8c840b5e10 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c00f930 con 0x7f8c840b5e10 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00fcd0 con 0x7f8c840b5e10 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8416b5d0 con 0x7f8c840b5e10 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.533+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c841553d0 con 0x7f8c840b5e10 2023-10-06T03:09:43.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.534+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8416bd90 con 0x7f8c840b5e10 2023-10-06T03:09:43.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.534+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8c7c014650 con 0x7f8c840b5e10 2023-10-06T03:09:43.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.535+0000 7f8c78ff9700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c70067320 0x7f8c700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.535+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8c7c085460 con 0x7f8c840b5e10 2023-10-06T03:09:43.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.535+0000 7f8c88d2f700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c70067320 0x7f8c700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.535+0000 7f8c88d2f700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c70067320 0x7f8c700697d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c841590d0 tx=0x7f8c80001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.539+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c7c052090 con 0x7f8c840b5e10 2023-10-06T03:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.723+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8c8404c3c0 con 0x7f8c840b5e10 2023-10-06T03:09:43.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.723+0000 7f8c78ff9700 1 -- 172.21.15.92:0/1109622656 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8c7c0556b0 con 0x7f8c840b5e10 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c70067320 msgr2=0x7f8c700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c70067320 0x7f8c700697d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c841590d0 tx=0x7f8c80001640 comp rx=0 tx=0).stop 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 msgr2=0x7f8c840b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 0x7f8c840b3fa0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8c8406a300 tx=0x7f8c7c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 shutdown_connections 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8c70067320 0x7f8c700697d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8c840b5e10 0x7f8c840b3fa0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8c6c004810 0x7f8c840cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 --2- 172.21.15.92:0/1109622656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c8416aac0 0x7f8c84154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 >> 172.21.15.92:0/1109622656 conn(0x7f8c84068e20 msgr2=0x7f8c841583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:43.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 shutdown_connections 2023-10-06T03:09:43.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.725+0000 7f8c8af93700 1 -- 172.21.15.92:0/1109622656 wait complete. 2023-10-06T03:09:43.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000422517 2023-10-06T03:09:43.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.854+0000 7f901759e700 1 Processor -- start 2023-10-06T03:09:43.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.854+0000 7f901759e700 1 -- start start 2023-10-06T03:09:43.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f901759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 0x7f90180b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f901759e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180b2080 con 0x7f90180b4e70 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f901759e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180cfb00 con 0x7f90180b33b0 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f901759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180b5c40 con 0x7f90180b16d0 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f901659c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f90180b4e70 0x7f90180ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47408/0 (socket says 172.21.15.92:47408) 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f901659c700 1 -- 172.21.15.92:0/1655307050 learned_addr learned my addr 172.21.15.92:0/1655307050 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f900dd9b700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 0x7f90180b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097475028 0 0) 0x7f90180b2080 con 0x7f90180b4e70 2023-10-06T03:09:43.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9004003570 con 0x7f90180b4e70 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2580753716 0 0) 0x7f9004003570 con 0x7f90180b4e70 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 msgr2=0x7f90180b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 0x7f90180b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 >> v1:172.21.15.136:6789/0 conn(0x7f90180b33b0 legacy=0x7f90180b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180b5f20 con 0x7f90180b4e70 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.855+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f9008001470 con 0x7f90180b4e70 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 >> v1:172.21.15.92:6789/0 conn(0x7f90180b4e70 legacy=0x7f90180ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f9004003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9004004810 0x7f9004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 0x7f9004005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180b5c40 con 0x7f9004003a90 2023-10-06T03:09:43.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900400bb90 con 0x7f9004004810 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900400bd50 con 0x7f90180b16d0 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f9004003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f900dd9b700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9004004810 0x7f9004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9016d9d700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 0x7f9004005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 -- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 msgr2=0x7f9004005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180b16d0 0x7f9004005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 -- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9004004810 msgr2=0x7f9004004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9004004810 0x7f9004004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 -- 172.21.15.92:0/1655307050 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004003570 con 0x7f9004003a90 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f901659c700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f9004003e60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f900800c2d0 tx=0x7f9008001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635a18b152af6451 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900800cbc0 con 0x7f9004003a90 2023-10-06T03:09:43.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900800cd20 con 0x7f9004003a90 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.856+0000 7f9015d9b700 1 -- 172.21.15.92:0/1655307050 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008005b70 con 0x7f9004003a90 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f901759e700 1 -- 172.21.15.92:0/1655307050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 msgr2=0x7f9004003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f901759e700 1 --2- 172.21.15.92:0/1655307050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f9004003e60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f900800c2d0 tx=0x7f9008001df0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f9016d9d700 1 -- 172.21.15.92:0/1655307050 reap_dead start 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f901759e700 1 -- 172.21.15.92:0/1655307050 shutdown_connections 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f901759e700 1 -- 172.21.15.92:0/1655307050 >> 172.21.15.92:0/1655307050 conn(0x7f90180d10c0 msgr2=0x7f90180affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f901759e700 1 -- 172.21.15.92:0/1655307050 shutdown_connections 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.857+0000 7f901759e700 1 -- 172.21.15.92:0/1655307050 wait complete. 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.858+0000 7f901759e700 1 Processor -- start 2023-10-06T03:09:43.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.858+0000 7f901759e700 1 -- start start 2023-10-06T03:09:43.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f901814c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180cade0 0x7f90180cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90180c8660 0x7f90180c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018159680 con 0x7f9004003a90 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180cb790 con 0x7f90180c8660 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180c9010 con 0x7f90180cade0 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f901814c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f901814c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37430/0 (socket says 172.21.15.92:37430) 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 -- 172.21.15.92:0/576568152 learned_addr learned my addr 172.21.15.92:0/576568152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f900dd9b700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180cade0 0x7f90180cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.866 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 -- 172.21.15.92:0/576568152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180cade0 msgr2=0x7f90180cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90180cade0 0x7f90180cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 -- 172.21.15.92:0/576568152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90180c8660 msgr2=0x7f90180c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90180c8660 0x7f90180c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 -- 172.21.15.92:0/576568152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180c9270 con 0x7f9004003a90 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901659c700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9004003a90 0x7f901814c040 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f9008011000 tx=0x7f9008015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080014c0 con 0x7f9004003a90 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080072e0 con 0x7f9004003a90 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 -- 172.21.15.92:0/576568152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90180c9550 con 0x7f9004003a90 2023-10-06T03:09:43.867 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080076e0 con 0x7f9004003a90 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.859+0000 7f901759e700 1 -- 172.21.15.92:0/576568152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90180c9b10 con 0x7f9004003a90 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.860+0000 7f901759e700 1 -- 172.21.15.92:0/576568152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9018088990 con 0x7f9004003a90 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.861+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9008019030 con 0x7f9004003a90 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.861+0000 7f900effd700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ff8067350 0x7f8ff8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.861+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9008086610 con 0x7f9004003a90 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.861+0000 7f900dd9b700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ff8067350 0x7f8ff8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.862+0000 7f900dd9b700 1 --2- 172.21.15.92:0/576568152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ff8067350 0x7f8ff8069800 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9000006f00 tx=0x7f9000008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:43.868 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:43.865+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90080532b0 con 0x7f9004003a90 2023-10-06T03:09:44.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.075+0000 7f901759e700 1 -- 172.21.15.92:0/576568152 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000422517", "target": ["mon-mgr", ""]}) v1 -- 0x7f901804bb80 con 0x7f8ff8067350 2023-10-06T03:09:44.078 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:09:44.076+0000 7fdd0eb80700 -1 mgr.server reply reply (1) Operation not permitted WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-06T03:09:44.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.076+0000 7f900effd700 1 -- 172.21.15.92:0/576568152 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: -1 WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue.) v1 ==== 283+0+0 (secure 0 0 0) 0x7f901804bb80 con 0x7f8ff8067350 2023-10-06T03:09:44.080 INFO:teuthology.orchestra.run.smithi092.stderr:Error EPERM: WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-06T03:09:44.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:09:44.083 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:09:44.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:44.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:44.170 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:09:44.170 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:09:44.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:09:44.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:09:44.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:09:44.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:44.209 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:09:44.264 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:09:44.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:09:44.265 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:09:44.307 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:09:44.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:09:44.308 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:09:44.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.478+0000 7f9fa6455700 1 Processor -- start 2023-10-06T03:09:44.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.478+0000 7f9fa6455700 1 -- start start 2023-10-06T03:09:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa6455700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa6455700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9fa015db10 0x7f9fa0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa6455700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa015a160 con 0x7f9fa0159890 2023-10-06T03:09:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa6455700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00d13a0 con 0x7f9fa015db10 2023-10-06T03:09:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa6455700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa015e140 con 0x7f9fa0159c60 2023-10-06T03:09:44.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9fa015db10 0x7f9fa0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9fa015db10 0x7f9fa0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37446/0 (socket says 172.21.15.92:37446) 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 -- 172.21.15.92:0/2880788858 learned_addr learned my addr 172.21.15.92:0/2880788858 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9f9f7fe700 1 --2- 172.21.15.92:0/2880788858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 -- 172.21.15.92:0/2880788858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 msgr2=0x7f9fa015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 --2- 172.21.15.92:0/2880788858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 -- 172.21.15.92:0/2880788858 >> v1:172.21.15.136:6789/0 conn(0x7f9fa0159890 legacy=0x7f9fa015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.479+0000 7f9fa49f2700 1 -- 172.21.15.92:0/2880788858 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa015a3d0 con 0x7f9fa015db10 2023-10-06T03:09:44.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa49f2700 1 --2- 172.21.15.92:0/2880788858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa015db10 0x7f9fa0069560 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009f20 tx=0x7f9f98009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f70d2bbd2acafe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9f9effd700 1 -- 172.21.15.92:0/2880788858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9800e070 con 0x7f9fa015db10 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9f9effd700 1 -- 172.21.15.92:0/2880788858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9800ba80 con 0x7f9fa015db10 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9f9effd700 1 -- 172.21.15.92:0/2880788858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980092e0 con 0x7f9fa015db10 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 -- 172.21.15.92:0/2880788858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa015db10 msgr2=0x7f9fa0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 --2- 172.21.15.92:0/2880788858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa015db10 0x7f9fa0069560 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f9f98009f20 tx=0x7f9f98009840 comp rx=0 tx=0).stop 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 -- 172.21.15.92:0/2880788858 shutdown_connections 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 --2- 172.21.15.92:0/2880788858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa015db10 0x7f9fa0069560 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 --2- 172.21.15.92:0/2880788858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 -- 172.21.15.92:0/2880788858 >> 172.21.15.92:0/2880788858 conn(0x7f9fa0068e20 msgr2=0x7f9fa00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:44.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 -- 172.21.15.92:0/2880788858 shutdown_connections 2023-10-06T03:09:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.480+0000 7f9fa6455700 1 -- 172.21.15.92:0/2880788858 wait complete. 2023-10-06T03:09:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.481+0000 7f9fa6455700 1 Processor -- start 2023-10-06T03:09:44.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 -- start start 2023-10-06T03:09:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 0x7f9fa00cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa00cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fa015db10 0x7f9fa016d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa016abd0 con 0x7f9fa0159890 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa0169f50 con 0x7f9fa015db10 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa6455700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa016a230 con 0x7f9fa0159c60 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9f9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 0x7f9fa00cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9f9ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 0x7f9fa00cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37462/0 (socket says 172.21.15.92:37462) 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9f9ffff700 1 -- 172.21.15.92:0/4205064735 learned_addr learned my addr 172.21.15.92:0/4205064735 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9f9f7fe700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa00cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.482+0000 7f9fa49f2700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fa015db10 0x7f9fa016d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f9ffff700 1 -- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 msgr2=0x7f9fa00cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f9ffff700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa00cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f9ffff700 1 -- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fa015db10 msgr2=0x7f9fa016d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f9ffff700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fa015db10 0x7f9fa016d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f9ffff700 1 -- 172.21.15.92:0/4205064735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98009ae0 con 0x7f9fa0159890 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f9ffff700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 0x7f9fa00cf210 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c590 tx=0x7f9f9000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011590 con 0x7f9fa0159890 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90011710 con 0x7f9fa0159890 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa016a510 con 0x7f9fa0159890 2023-10-06T03:09:44.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90011ad0 con 0x7f9fa0159890 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.483+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa00c74f0 con 0x7f9fa0159890 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.484+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00c7710 con 0x7f9fa0159890 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.484+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9f9001b3b0 con 0x7f9fa0159890 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.485+0000 7f9f87fff700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f8c067320 0x7f9f8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.485+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f900841a0 con 0x7f9fa0159890 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.485+0000 7f9f9f7fe700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f8c067320 0x7f9f8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.485+0000 7f9f9f7fe700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f8c067320 0x7f9f8c0697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00cff50 tx=0x7f9f94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:44.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.489+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f90050cd0 con 0x7f9fa0159890 2023-10-06T03:09:44.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.714+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fa004c3c0 con 0x7f9fa0159890 2023-10-06T03:09:44.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.714+0000 7f9f87fff700 1 -- 172.21.15.92:0/4205064735 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f9f90017090 con 0x7f9fa0159890 2023-10-06T03:09:44.716 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:44.716 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:33.377515+0000","modified":"2023-10-06T03:09:37.016548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2005327996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2005327996},{"type":"v1","addr":"172.21.15.92:6837","nonce":2005327996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T03:09:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f8c067320 msgr2=0x7f9f8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f8c067320 0x7f9f8c0697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00cff50 tx=0x7f9f94008040 comp rx=0 tx=0).stop 2023-10-06T03:09:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 msgr2=0x7f9fa00cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 0x7f9fa00cf210 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c590 tx=0x7f9f9000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 shutdown_connections 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9f8c067320 0x7f9f8c0697d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9fa0159890 0x7f9fa00cf210 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fa015db10 0x7f9fa016d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 --2- 172.21.15.92:0/4205064735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fa0159c60 0x7f9fa00cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.716+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 >> 172.21.15.92:0/4205064735 conn(0x7f9fa0068e20 msgr2=0x7f9fa0152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.717+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 shutdown_connections 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.717+0000 7f9fa6455700 1 -- 172.21.15.92:0/4205064735 wait complete. 2023-10-06T03:09:44.719 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 105 2023-10-06T03:09:44.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:44.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.843+0000 7f28b02c8700 1 Processor -- start 2023-10-06T03:09:44.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.843+0000 7f28b02c8700 1 -- start start 2023-10-06T03:09:44.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28b02c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8159890 0x7f28a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28b02c8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f28a8069560 0x7f28a80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28b02c8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a815a160 con 0x7f28a8159c60 2023-10-06T03:09:44.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28b02c8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a8069f20 con 0x7f28a8069560 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28b02c8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80d13a0 con 0x7f28a8159890 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ad863700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f28a8159c60 0x7f28a815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47410/0 (socket says 172.21.15.92:47410) 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ad863700 1 -- 172.21.15.92:0/2142135350 learned_addr learned my addr 172.21.15.92:0/2142135350 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae865700 1 --2- 172.21.15.92:0/2142135350 >> v2:172.21.15.136:3300/0 conn(0x7f28a8069560 0x7f28a80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae064700 1 --2- 172.21.15.92:0/2142135350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8159890 0x7f28a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ad062700 1 -- 172.21.15.92:0/2142135350 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033026074 0 0) 0x7f28a815a160 con 0x7f28a8159c60 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ad062700 1 -- 172.21.15.92:0/2142135350 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2890003570 con 0x7f28a8159c60 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae064700 1 -- 172.21.15.92:0/2142135350 >> v2:172.21.15.136:3300/0 conn(0x7f28a8069560 msgr2=0x7f28a80699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae064700 1 --2- 172.21.15.92:0/2142135350 >> v2:172.21.15.136:3300/0 conn(0x7f28a8069560 0x7f28a80699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae064700 1 -- 172.21.15.92:0/2142135350 >> v1:172.21.15.92:6789/0 conn(0x7f28a8159c60 legacy=0x7f28a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae064700 1 -- 172.21.15.92:0/2142135350 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a815e140 con 0x7f28a8159890 2023-10-06T03:09:44.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.844+0000 7f28ae865700 1 --2- 172.21.15.92:0/2142135350 >> v2:172.21.15.136:3300/0 conn(0x7f28a8069560 0x7f28a80699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28ae064700 1 --2- 172.21.15.92:0/2142135350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8159890 0x7f28a815abd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009e60 tx=0x7f28a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee9ca63b225c04f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28ad062700 1 -- 172.21.15.92:0/2142135350 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000e070 con 0x7f28a8159890 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28ad062700 1 -- 172.21.15.92:0/2142135350 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28a000bab0 con 0x7f28a8159890 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28ad062700 1 -- 172.21.15.92:0/2142135350 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000be70 con 0x7f28a8159890 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 -- 172.21.15.92:0/2142135350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8159890 msgr2=0x7f28a815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 --2- 172.21.15.92:0/2142135350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8159890 0x7f28a815abd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009e60 tx=0x7f28a0009780 comp rx=0 tx=0).stop 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 -- 172.21.15.92:0/2142135350 shutdown_connections 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 --2- 172.21.15.92:0/2142135350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8159890 0x7f28a815abd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 --2- 172.21.15.92:0/2142135350 >> v2:172.21.15.136:3300/0 conn(0x7f28a8069560 0x7f28a80699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 -- 172.21.15.92:0/2142135350 >> 172.21.15.92:0/2142135350 conn(0x7f28a8068e20 msgr2=0x7f28a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 -- 172.21.15.92:0/2142135350 shutdown_connections 2023-10-06T03:09:44.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.845+0000 7f28b02c8700 1 -- 172.21.15.92:0/2142135350 wait complete. 2023-10-06T03:09:44.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.846+0000 7f28b02c8700 1 Processor -- start 2023-10-06T03:09:44.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.846+0000 7f28b02c8700 1 -- start start 2023-10-06T03:09:44.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28b02c8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8069560 0x7f28a80ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28b02c8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8159890 0x7f28a80c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28b02c8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 0x7f28a80b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28b02c8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80ce280 con 0x7f28a8159c60 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28b02c8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80cd670 con 0x7f28a8159890 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28b02c8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80cd950 con 0x7f28a8069560 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28ae865700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 0x7f28a80b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28ae865700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 0x7f28a80b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37464/0 (socket says 172.21.15.92:37464) 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28ae865700 1 -- 172.21.15.92:0/3744450767 learned_addr learned my addr 172.21.15.92:0/3744450767 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28ae064700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8069560 0x7f28a80ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28ae865700 1 -- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8069560 msgr2=0x7f28a80ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.847+0000 7f28ad863700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8159890 0x7f28a80c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ae865700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8069560 0x7f28a80ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ae865700 1 -- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8159890 msgr2=0x7f28a80c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ae865700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8159890 0x7f28a80c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ae865700 1 -- 172.21.15.92:0/3744450767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2898009420 con 0x7f28a8159c60 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ae064700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8069560 0x7f28a80ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ad863700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8159890 0x7f28a80c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28ae865700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 0x7f28a80b5870 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f289800ea20 tx=0x7f28980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800c850 con 0x7f28a8159c60 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289800c9b0 con 0x7f28a8159c60 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a0009a20 con 0x7f28a8159c60 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800cd70 con 0x7f28a8159c60 2023-10-06T03:09:44.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.848+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a80cdf10 con 0x7f28a8159c60 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.849+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a80ce3f0 con 0x7f28a8159c60 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.849+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2898013680 con 0x7f28a8159c60 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.849+0000 7f28a67fc700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2894067320 0x7f28940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.850+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2898084ef0 con 0x7f28a8159c60 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.850+0000 7f28ae064700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2894067320 0x7f28940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.850+0000 7f28ae064700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2894067320 0x7f28940697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f28a00097c0 tx=0x7f28a0019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:44.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:44.854+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28980519e0 con 0x7f28a8159c60 2023-10-06T03:09:45.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.038+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28a804c3c0 con 0x7f28a8159c60 2023-10-06T03:09:45.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.039+0000 7f28a67fc700 1 -- 172.21.15.92:0/3744450767 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f2898017090 con 0x7f28a8159c60 2023-10-06T03:09:45.042 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:45.042 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":155,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:34.995208+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:31.023983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:32.339528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2894067320 msgr2=0x7f28940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2894067320 0x7f28940697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f28a00097c0 tx=0x7f28a0019000 comp rx=0 tx=0).stop 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 msgr2=0x7f28a80b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 0x7f28a80b5870 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f289800ea20 tx=0x7f28980091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 shutdown_connections 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2894067320 0x7f28940697d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28a8159c60 0x7f28a80b5870 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8069560 0x7f28a80ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 --2- 172.21.15.92:0/3744450767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8159890 0x7f28a80c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 >> 172.21.15.92:0/3744450767 conn(0x7f28a8068e20 msgr2=0x7f28a80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:45.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 shutdown_connections 2023-10-06T03:09:45.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.041+0000 7f28b02c8700 1 -- 172.21.15.92:0/3744450767 wait complete. 2023-10-06T03:09:45.056 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:09:31.023983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:09:32.339528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:09:45.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:09:45.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.171+0000 7fb06fc94700 1 Processor -- start 2023-10-06T03:09:45.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.171+0000 7fb06fc94700 1 -- start start 2023-10-06T03:09:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06fc94700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb068159890 0x7fb06815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06fc94700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb068159c60 0x7fb06815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06fc94700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06815a160 con 0x7fb06815db10 2023-10-06T03:09:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06fc94700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06815df50 con 0x7fb068159c60 2023-10-06T03:09:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06fc94700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680d13a0 con 0x7fb068159890 2023-10-06T03:09:45.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb068159c60 0x7fb06815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06e231700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb06815db10 0x7fb068069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47422/0 (socket says 172.21.15.92:47422) 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06e231700 1 -- 172.21.15.92:0/1400517052 learned_addr learned my addr 172.21.15.92:0/1400517052 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb068159c60 0x7fb06815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37468/0 (socket says 172.21.15.92:37468) 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06da30700 1 --2- 172.21.15.92:0/1400517052 >> v2:172.21.15.136:3300/0 conn(0x7fb068159890 0x7fb06815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 -- 172.21.15.92:0/1400517052 >> v2:172.21.15.136:3300/0 conn(0x7fb068159890 msgr2=0x7fb06815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 --2- 172.21.15.92:0/1400517052 >> v2:172.21.15.136:3300/0 conn(0x7fb068159890 0x7fb06815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 -- 172.21.15.92:0/1400517052 >> v1:172.21.15.92:6789/0 conn(0x7fb06815db10 legacy=0x7fb068069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 -- 172.21.15.92:0/1400517052 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06815e140 con 0x7fb068159c60 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06ca2e700 1 -- 172.21.15.92:0/1400517052 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58668690 0 0) 0x7fb06815a160 con 0x7fb06815db10 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06da30700 1 --2- 172.21.15.92:0/1400517052 >> v2:172.21.15.136:3300/0 conn(0x7fb068159890 0x7fb06815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06d22f700 1 --2- 172.21.15.92:0/1400517052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb068159c60 0x7fb06815d5d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb0680d1090 tx=0x7fb05c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc62d904c9a3bcff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.172+0000 7fb06ca2e700 1 -- 172.21.15.92:0/1400517052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00b6b0 con 0x7fb068159c60 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06ca2e700 1 -- 172.21.15.92:0/1400517052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c00b810 con 0x7fb068159c60 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06ca2e700 1 -- 172.21.15.92:0/1400517052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00bbb0 con 0x7fb068159c60 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 -- 172.21.15.92:0/1400517052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb068159c60 msgr2=0x7fb06815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 --2- 172.21.15.92:0/1400517052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb068159c60 0x7fb06815d5d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb0680d1090 tx=0x7fb05c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 -- 172.21.15.92:0/1400517052 shutdown_connections 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 --2- 172.21.15.92:0/1400517052 >> v2:172.21.15.136:3300/0 conn(0x7fb068159890 0x7fb06815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 --2- 172.21.15.92:0/1400517052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb068159c60 0x7fb06815d5d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 -- 172.21.15.92:0/1400517052 >> 172.21.15.92:0/1400517052 conn(0x7fb068068e20 msgr2=0x7fb0680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 -- 172.21.15.92:0/1400517052 shutdown_connections 2023-10-06T03:09:45.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.173+0000 7fb06fc94700 1 -- 172.21.15.92:0/1400517052 wait complete. 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.174+0000 7fb06fc94700 1 Processor -- start 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.174+0000 7fb06fc94700 1 -- start start 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06fc94700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb068159890 0x7fb068143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06fc94700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068159c60 0x7fb0680c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06fc94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 0x7fb0680c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06fc94700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068154860 con 0x7fb06815db10 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06fc94700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0681537c0 con 0x7fb068159c60 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06fc94700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb068153aa0 con 0x7fb068159890 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 0x7fb0680c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 0x7fb0680c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37484/0 (socket says 172.21.15.92:37484) 2023-10-06T03:09:45.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 -- 172.21.15.92:0/3827816961 learned_addr learned my addr 172.21.15.92:0/3827816961 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 -- 172.21.15.92:0/3827816961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb068159890 msgr2=0x7fb068143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb068159890 0x7fb068143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 -- 172.21.15.92:0/3827816961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068159c60 msgr2=0x7fb0680c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068159c60 0x7fb0680c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 -- 172.21.15.92:0/3827816961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c009420 con 0x7fb06815db10 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb06e231700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 0x7fb0680c97d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb05800cbf0 tx=0x7fb0580019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.175+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800a9c0 con 0x7fb06815db10 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.176+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05800ab20 con 0x7fb06815db10 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.176+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580070f0 con 0x7fb06815db10 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.176+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068153d80 con 0x7fb06815db10 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.176+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb068154340 con 0x7fb06815db10 2023-10-06T03:09:45.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.177+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068154560 con 0x7fb06815db10 2023-10-06T03:09:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.177+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb058007520 con 0x7fb06815db10 2023-10-06T03:09:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.177+0000 7fb065ffb700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb054067320 0x7fb0540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.177+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb0580851b0 con 0x7fb06815db10 2023-10-06T03:09:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.177+0000 7fb06da30700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb054067320 0x7fb0540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.178+0000 7fb06da30700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb054067320 0x7fb0540697d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb060007aa0 tx=0x7fb060008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.181+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb058051d60 con 0x7fb06815db10 2023-10-06T03:09:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.364+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb06804c3c0 con 0x7fb06815db10 2023-10-06T03:09:45.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.364+0000 7fb065ffb700 1 -- 172.21.15.92:0/3827816961 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb058055380 con 0x7fb06815db10 2023-10-06T03:09:45.366 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:09:45.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb054067320 msgr2=0x7fb0540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb054067320 0x7fb0540697d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb060007aa0 tx=0x7fb060008040 comp rx=0 tx=0).stop 2023-10-06T03:09:45.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 msgr2=0x7fb0680c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 0x7fb0680c97d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fb05800cbf0 tx=0x7fb0580019b0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 shutdown_connections 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb054067320 0x7fb0540697d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06815db10 0x7fb0680c97d0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb068159890 0x7fb068143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 --2- 172.21.15.92:0/3827816961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb068159c60 0x7fb0680c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 >> 172.21.15.92:0/3827816961 conn(0x7fb068068e20 msgr2=0x7fb068157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 shutdown_connections 2023-10-06T03:09:45.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.366+0000 7fb06fc94700 1 -- 172.21.15.92:0/3827816961 wait complete. 2023-10-06T03:09:45.380 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:09:45.497 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.496+0000 7f60f0977700 1 Processor -- start 2023-10-06T03:09:45.498 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.496+0000 7f60f0977700 1 -- start start 2023-10-06T03:09:45.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60f0977700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 0x7f60ec15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.499 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60f0977700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec069560 0x7f60ec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60f0977700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec069ee0 con 0x7f60ec159c60 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60f0977700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec15a160 con 0x7f60ec159890 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60f0977700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec0d13a0 con 0x7f60ec069560 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ea59c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 0x7f60ec15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ead9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec069560 0x7f60ec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ea59c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 0x7f60ec15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:44924/0 (socket says 172.21.15.92:44924) 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ea59c700 1 -- 172.21.15.92:0/3512722164 learned_addr learned my addr 172.21.15.92:0/3512722164 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60e9d9b700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f60ec159c60 0x7f60ec15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:58264/0 (socket says 172.21.15.92:58264) 2023-10-06T03:09:45.500 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ead9d700 1 -- 172.21.15.92:0/3512722164 >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 msgr2=0x7f60ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ead9d700 1 --2- 172.21.15.92:0/3512722164 >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 0x7f60ec15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ead9d700 1 -- 172.21.15.92:0/3512722164 >> v1:172.21.15.136:6789/0 conn(0x7f60ec159c60 legacy=0x7f60ec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ead9d700 1 -- 172.21.15.92:0/3512722164 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec15e140 con 0x7f60ec069560 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60e959a700 1 -- 172.21.15.92:0/3512722164 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066796716 0 0) 0x7f60ec069ee0 con 0x7f60ec159c60 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.497+0000 7f60ea59c700 1 --2- 172.21.15.92:0/3512722164 >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 0x7f60ec15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60ead9d700 1 --2- 172.21.15.92:0/3512722164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec069560 0x7f60ec0699a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098e0 tx=0x7f60dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1b1317389bae93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60e959a700 1 -- 172.21.15.92:0/3512722164 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00b640 con 0x7f60ec069560 2023-10-06T03:09:45.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60e959a700 1 -- 172.21.15.92:0/3512722164 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc00b7a0 con 0x7f60ec069560 2023-10-06T03:09:45.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60e959a700 1 -- 172.21.15.92:0/3512722164 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00bb40 con 0x7f60ec069560 2023-10-06T03:09:45.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 -- 172.21.15.92:0/3512722164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec069560 msgr2=0x7f60ec0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 --2- 172.21.15.92:0/3512722164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec069560 0x7f60ec0699a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0098e0 tx=0x7f60dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 -- 172.21.15.92:0/3512722164 shutdown_connections 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 --2- 172.21.15.92:0/3512722164 >> v2:172.21.15.136:3300/0 conn(0x7f60ec159890 0x7f60ec15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 --2- 172.21.15.92:0/3512722164 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec069560 0x7f60ec0699a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 -- 172.21.15.92:0/3512722164 >> 172.21.15.92:0/3512722164 conn(0x7f60ec068e20 msgr2=0x7f60ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 -- 172.21.15.92:0/3512722164 shutdown_connections 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.498+0000 7f60f0977700 1 -- 172.21.15.92:0/3512722164 wait complete. 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.499+0000 7f60f0977700 1 Processor -- start 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.499+0000 7f60f0977700 1 -- start start 2023-10-06T03:09:45.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60f0977700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60ec069560 0x7f60ec14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60f0977700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec159890 0x7f60ec153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60f0977700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 0x7f60ec154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60f0977700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec152700 con 0x7f60ec159c60 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60f0977700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec14d890 con 0x7f60ec069560 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60f0977700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec151af0 con 0x7f60ec159890 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60ead9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 0x7f60ec154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60ead9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 0x7f60ec154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37488/0 (socket says 172.21.15.92:37488) 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.500+0000 7f60ead9d700 1 -- 172.21.15.92:0/427769745 learned_addr learned my addr 172.21.15.92:0/427769745 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60e9d9b700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec159890 0x7f60ec153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ea59c700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60ec069560 0x7f60ec14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ead9d700 1 -- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec159890 msgr2=0x7f60ec153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ead9d700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec159890 0x7f60ec153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ead9d700 1 -- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60ec069560 msgr2=0x7f60ec14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ead9d700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60ec069560 0x7f60ec14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ead9d700 1 -- 172.21.15.92:0/427769745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4009ae0 con 0x7f60ec159c60 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60ead9d700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 0x7f60ec154000 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f60ec070e20 tx=0x7f60dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc011420 con 0x7f60ec159c60 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc011580 con 0x7f60ec159c60 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc009420 con 0x7f60ec159c60 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc011920 con 0x7f60ec159c60 2023-10-06T03:09:45.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.501+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec1520b0 con 0x7f60ec159c60 2023-10-06T03:09:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.502+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec04c3c0 con 0x7f60ec159c60 2023-10-06T03:09:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.502+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f60dc011c90 con 0x7f60ec159c60 2023-10-06T03:09:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.503+0000 7f60e27fc700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f60cc067320 0x7f60cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.503+0000 7f60ea59c700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f60cc067320 0x7f60cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.503+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f60dc086770 con 0x7f60ec159c60 2023-10-06T03:09:45.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.503+0000 7f60ea59c700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f60cc067320 0x7f60cc0697d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009ab0 tx=0x7f60e400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.507+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60dc0533a0 con 0x7f60ec159c60 2023-10-06T03:09:45.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.687+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60ec152840 con 0x7f60ec159c60 2023-10-06T03:09:45.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.687+0000 7f60e27fc700 1 -- 172.21.15.92:0/427769745 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v155) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60ec152840 con 0x7f60ec159c60 2023-10-06T03:09:45.689 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:09:45.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f60cc067320 msgr2=0x7f60cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f60cc067320 0x7f60cc0697d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009ab0 tx=0x7f60e400b040 comp rx=0 tx=0).stop 2023-10-06T03:09:45.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 msgr2=0x7f60ec154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 0x7f60ec154000 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f60ec070e20 tx=0x7f60dc017000 comp rx=0 tx=0).stop 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 shutdown_connections 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f60cc067320 0x7f60cc0697d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f60ec159c60 0x7f60ec154000 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60ec069560 0x7f60ec14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 --2- 172.21.15.92:0/427769745 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60ec159890 0x7f60ec153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 >> 172.21.15.92:0/427769745 conn(0x7f60ec068e20 msgr2=0x7f60ec15d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 shutdown_connections 2023-10-06T03:09:45.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.689+0000 7f60f0977700 1 -- 172.21.15.92:0/427769745 wait complete. 2023-10-06T03:09:45.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:09:45.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.819+0000 7f09d20b8700 1 Processor -- start 2023-10-06T03:09:45.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.819+0000 7f09d20b8700 1 -- start start 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09d20b8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09cc159890 0x7f09cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09d20b8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f09cc159c60 0x7f09cc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09d20b8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc15a160 con 0x7f09cc15db10 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09d20b8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc15df50 con 0x7f09cc159890 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09d20b8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0d13a0 con 0x7f09cc159c60 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09cc159890 0x7f09cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f09cc15db10 0x7f09cc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47436/0 (socket says 172.21.15.92:47436) 2023-10-06T03:09:45.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cbfff700 1 -- 172.21.15.92:0/1664526180 learned_addr learned my addr 172.21.15.92:0/1664526180 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09cc159890 0x7f09cc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37492/0 (socket says 172.21.15.92:37492) 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 -- 172.21.15.92:0/1664526180 >> v2:172.21.15.136:3300/0 conn(0x7f09cc159c60 msgr2=0x7f09cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 --2- 172.21.15.92:0/1664526180 >> v2:172.21.15.136:3300/0 conn(0x7f09cc159c60 0x7f09cc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 -- 172.21.15.92:0/1664526180 >> v1:172.21.15.92:6789/0 conn(0x7f09cc15db10 legacy=0x7f09cc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09ca7fc700 1 -- 172.21.15.92:0/1664526180 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609471654 0 0) 0x7f09cc15a160 con 0x7f09cc15db10 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 -- 172.21.15.92:0/1664526180 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc15e140 con 0x7f09cc159890 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09cb7fe700 1 --2- 172.21.15.92:0/1664526180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159890 0x7f09cc15abd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0b67f0 tx=0x7f09bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae1f5c146162d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09ca7fc700 1 -- 172.21.15.92:0/1664526180 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00b6b0 con 0x7f09cc159890 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09ca7fc700 1 -- 172.21.15.92:0/1664526180 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc00b810 con 0x7f09cc159890 2023-10-06T03:09:45.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.820+0000 7f09ca7fc700 1 -- 172.21.15.92:0/1664526180 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00bbb0 con 0x7f09cc159890 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 -- 172.21.15.92:0/1664526180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159890 msgr2=0x7f09cc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 --2- 172.21.15.92:0/1664526180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159890 0x7f09cc15abd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0b67f0 tx=0x7f09bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 -- 172.21.15.92:0/1664526180 shutdown_connections 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 --2- 172.21.15.92:0/1664526180 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159890 0x7f09cc15abd0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 --2- 172.21.15.92:0/1664526180 >> v2:172.21.15.136:3300/0 conn(0x7f09cc159c60 0x7f09cc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 -- 172.21.15.92:0/1664526180 >> 172.21.15.92:0/1664526180 conn(0x7f09cc068e20 msgr2=0x7f09cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 -- 172.21.15.92:0/1664526180 shutdown_connections 2023-10-06T03:09:45.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.821+0000 7f09d20b8700 1 -- 172.21.15.92:0/1664526180 wait complete. 2023-10-06T03:09:45.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.822+0000 7f09d20b8700 1 Processor -- start 2023-10-06T03:09:45.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.822+0000 7f09d20b8700 1 -- start start 2023-10-06T03:09:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09d20b8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09cc159890 0x7f09cc0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09d20b8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 0x7f09cc151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09d20b8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09cc15db10 0x7f09cc152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09d20b8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0b46e0 con 0x7f09cc159c60 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09d20b8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0b61b0 con 0x7f09cc159890 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09d20b8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0b3ef0 con 0x7f09cc15db10 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 0x7f09cc151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 0x7f09cc151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37508/0 (socket says 172.21.15.92:37508) 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 -- 172.21.15.92:0/3737738194 learned_addr learned my addr 172.21.15.92:0/3737738194 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09cb7fe700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09cc159890 0x7f09cc0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 -- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09cc15db10 msgr2=0x7f09cc152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09cc15db10 0x7f09cc152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 -- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09cc159890 msgr2=0x7f09cc0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09cc159890 0x7f09cc0b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:45.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 -- 172.21.15.92:0/3737738194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc009420 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.823+0000 7f09caffd700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 0x7f09cc151d00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b810 tx=0x7f09c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.824+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000f5f0 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.824+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c000f770 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.824+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc0b5ba0 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.824+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000fb30 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.824+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc0b4400 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.825+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc0b6360 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.825+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f09c0010480 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.825+0000 7f09b37fe700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.825+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09c0084d30 con 0x7f09cc159c60 2023-10-06T03:09:45.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.825+0000 7f09cb7fe700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:45.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.826+0000 7f09cb7fe700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f09cc153660 tx=0x7f09bc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:45.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:45.829+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09cc0b6360 con 0x7f09cc159c60 2023-10-06T03:09:46.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.009+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09cc0b6360 con 0x7f09cc159c60 2023-10-06T03:09:46.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.010+0000 7f09b37fe700 1 -- 172.21.15.92:0/3737738194 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v155) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09cc0b6360 con 0x7f09cc159c60 2023-10-06T03:09:46.012 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:09:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 msgr2=0x7f09b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f09cc153660 tx=0x7f09bc017000 comp rx=0 tx=0).stop 2023-10-06T03:09:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 msgr2=0x7f09cc151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 0x7f09cc151d00 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f09c000b810 tx=0x7f09c0009250 comp rx=0 tx=0).stop 2023-10-06T03:09:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 shutdown_connections 2023-10-06T03:09:46.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09b8067320 0x7f09b80697d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09cc15db10 0x7f09cc152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09cc159890 0x7f09cc0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 --2- 172.21.15.92:0/3737738194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09cc159c60 0x7f09cc151d00 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 >> 172.21.15.92:0/3737738194 conn(0x7f09cc068e20 msgr2=0x7f09cc14d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:46.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 shutdown_connections 2023-10-06T03:09:46.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.012+0000 7f09d20b8700 1 -- 172.21.15.92:0/3737738194 wait complete. 2023-10-06T03:09:46.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:09:46.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.142+0000 7f00c8f01700 1 Processor -- start 2023-10-06T03:09:46.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.142+0000 7f00c8f01700 1 -- start start 2023-10-06T03:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c8f01700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c8f01700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00c4157a50 0x7f00c40cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c8f01700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f00c40b1450 0x7f00c40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c8f01700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c40cc700 con 0x7f00c4157a50 2023-10-06T03:09:46.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c8f01700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c40cb900 con 0x7f00c40b1450 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c8f01700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40cd240 con 0x7f00c4157680 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00c4157a50 0x7f00c40cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00c4157a50 0x7f00c40cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37518/0 (socket says 172.21.15.92:37518) 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 -- 172.21.15.92:0/3881598453 learned_addr learned my addr 172.21.15.92:0/3881598453 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c37fe700 1 --2- 172.21.15.92:0/3881598453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c3fff700 1 --2- 172.21.15.92:0/3881598453 >> v2:172.21.15.136:3300/0 conn(0x7f00c40b1450 0x7f00c40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 -- 172.21.15.92:0/3881598453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 msgr2=0x7f00c40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 --2- 172.21.15.92:0/3881598453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c40cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 -- 172.21.15.92:0/3881598453 >> v2:172.21.15.136:3300/0 conn(0x7f00c40b1450 msgr2=0x7f00c40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 --2- 172.21.15.92:0/3881598453 >> v2:172.21.15.136:3300/0 conn(0x7f00c40b1450 0x7f00c40c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 -- 172.21.15.92:0/3881598453 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40c9a20 con 0x7f00c4157a50 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.143+0000 7f00c2ffd700 1 --2- 172.21.15.92:0/3881598453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40cb3c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f00c40cd1b0 tx=0x7f00b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef054b6ccd586a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c27fc700 1 -- 172.21.15.92:0/3881598453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800b860 con 0x7f00c4157a50 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c27fc700 1 -- 172.21.15.92:0/3881598453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b800b9c0 con 0x7f00c4157a50 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c27fc700 1 -- 172.21.15.92:0/3881598453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b800bd60 con 0x7f00c4157a50 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 -- 172.21.15.92:0/3881598453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 msgr2=0x7f00c40cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 --2- 172.21.15.92:0/3881598453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40cb3c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f00c40cd1b0 tx=0x7f00b8009250 comp rx=0 tx=0).stop 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 -- 172.21.15.92:0/3881598453 shutdown_connections 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 --2- 172.21.15.92:0/3881598453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c40cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 --2- 172.21.15.92:0/3881598453 >> v2:172.21.15.136:3300/0 conn(0x7f00c40b1450 0x7f00c40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 --2- 172.21.15.92:0/3881598453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40cb3c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 -- 172.21.15.92:0/3881598453 >> 172.21.15.92:0/3881598453 conn(0x7f00c406baa0 msgr2=0x7f00c406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 -- 172.21.15.92:0/3881598453 shutdown_connections 2023-10-06T03:09:46.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.144+0000 7f00c8f01700 1 -- 172.21.15.92:0/3881598453 wait complete. 2023-10-06T03:09:46.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.145+0000 7f00c8f01700 1 Processor -- start 2023-10-06T03:09:46.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 -- start start 2023-10-06T03:09:46.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1450 0x7f00c4151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c4167a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c5740 con 0x7f00c4157a50 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c62c0 con 0x7f00c40b1450 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c8f01700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c5990 con 0x7f00c4157680 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37522/0 (socket says 172.21.15.92:37522) 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.146+0000 7f00c3fff700 1 -- 172.21.15.92:0/2956500249 learned_addr learned my addr 172.21.15.92:0/2956500249 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c37fe700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1450 0x7f00c4151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c2ffd700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c4167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c3fff700 1 -- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 msgr2=0x7f00c4167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c3fff700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c4167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c3fff700 1 -- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1450 msgr2=0x7f00c4151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c3fff700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1450 0x7f00c4151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c3fff700 1 -- 172.21.15.92:0/2956500249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80094f0 con 0x7f00c4157a50 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c2ffd700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c4167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c3fff700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40c6750 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d890 tx=0x7f00bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0125f0 con 0x7f00c4157a50 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc012770 con 0x7f00c4157a50 2023-10-06T03:09:46.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40c5c70 con 0x7f00c4157a50 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc012b30 con 0x7f00c4157a50 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.147+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c4151b50 con 0x7f00c4157a50 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.148+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c40c6430 con 0x7f00c4157a50 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.148+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f00bc01e460 con 0x7f00c4157a50 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.149+0000 7f00ab7fe700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00b0067320 0x7f00b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.149+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f00bc085370 con 0x7f00c4157a50 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.149+0000 7f00c37fe700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00b0067320 0x7f00b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.149+0000 7f00c37fe700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00b0067320 0x7f00b00697d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009fd0 tx=0x7f00b4009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.153+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00bc055930 con 0x7f00c4157a50 2023-10-06T03:09:46.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.336+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00c40cc910 con 0x7f00c4157a50 2023-10-06T03:09:46.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.336+0000 7f00ab7fe700 1 -- 172.21.15.92:0/2956500249 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v155) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00bc0554c0 con 0x7f00c4157a50 2023-10-06T03:09:46.338 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:09:46.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00b0067320 msgr2=0x7f00b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00b0067320 0x7f00b00697d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009fd0 tx=0x7f00b4009880 comp rx=0 tx=0).stop 2023-10-06T03:09:46.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 msgr2=0x7f00c40c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40c6750 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d890 tx=0x7f00bc008040 comp rx=0 tx=0).stop 2023-10-06T03:09:46.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 shutdown_connections 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f00b0067320 0x7f00b00697d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00c4157a50 0x7f00c40c6750 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00c40b1450 0x7f00c4151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 --2- 172.21.15.92:0/2956500249 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00c4157680 0x7f00c4167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 >> 172.21.15.92:0/2956500249 conn(0x7f00c406baa0 msgr2=0x7f00c41569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 shutdown_connections 2023-10-06T03:09:46.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.338+0000 7f00c8f01700 1 -- 172.21.15.92:0/2956500249 wait complete. 2023-10-06T03:09:46.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:46.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.465+0000 7f7b8f0c9700 1 Processor -- start 2023-10-06T03:09:46.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.465+0000 7f7b8f0c9700 1 -- start start 2023-10-06T03:09:46.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8f0c9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7b88068550 0x7f7b88068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8f0c9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b8815e010 0x7f7b8806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8f0c9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b8806f640 0x7f7b8806a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8f0c9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b880b3430 con 0x7f7b8806f640 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8f0c9700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8806ec70 con 0x7f7b88068550 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8f0c9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880b2e70 con 0x7f7b8815e010 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8d666700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b8806f640 0x7f7b8806a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8d666700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7b8806f640 0x7f7b8806a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37524/0 (socket says 172.21.15.92:37524) 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b8d666700 1 -- 172.21.15.92:0/304977583 learned_addr learned my addr 172.21.15.92:0/304977583 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.466+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/304977583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b8815e010 0x7f7b8806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8d666700 1 -- 172.21.15.92:0/304977583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b8815e010 msgr2=0x7f7b8806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:46.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8ce65700 1 --2- 172.21.15.92:0/304977583 >> v2:172.21.15.136:3300/0 conn(0x7f7b88068550 0x7f7b88068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8d666700 1 --2- 172.21.15.92:0/304977583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b8815e010 0x7f7b8806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8d666700 1 -- 172.21.15.92:0/304977583 >> v2:172.21.15.136:3300/0 conn(0x7f7b88068550 msgr2=0x7f7b88068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8d666700 1 --2- 172.21.15.92:0/304977583 >> v2:172.21.15.136:3300/0 conn(0x7f7b88068550 0x7f7b88068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8d666700 1 -- 172.21.15.92:0/304977583 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880b3150 con 0x7f7b8806f640 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8d666700 1 --2- 172.21.15.92:0/304977583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b8806a7e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806e170 tx=0x7f7b80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37b8032144f69bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b7f7fe700 1 -- 172.21.15.92:0/304977583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000e070 con 0x7f7b8806f640 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b7f7fe700 1 -- 172.21.15.92:0/304977583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8000ba50 con 0x7f7b8806f640 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b7f7fe700 1 -- 172.21.15.92:0/304977583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800092e0 con 0x7f7b8806f640 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/304977583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 msgr2=0x7f7b8806a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/304977583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b8806a7e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806e170 tx=0x7f7b80009840 comp rx=0 tx=0).stop 2023-10-06T03:09:46.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/304977583 shutdown_connections 2023-10-06T03:09:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/304977583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b8806a7e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/304977583 >> v2:172.21.15.136:3300/0 conn(0x7f7b88068550 0x7f7b88068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/304977583 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b8815e010 0x7f7b8806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/304977583 >> 172.21.15.92:0/304977583 conn(0x7f7b880cd480 msgr2=0x7f7b880cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/304977583 shutdown_connections 2023-10-06T03:09:46.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.467+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/304977583 wait complete. 2023-10-06T03:09:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.468+0000 7f7b8f0c9700 1 Processor -- start 2023-10-06T03:09:46.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 -- start start 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b88068550 0x7f7b88145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b880abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b8815e010 0x7f7b880ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b88159ad0 con 0x7f7b8806f640 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b881464e0 con 0x7f7b8815e010 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.469+0000 7f7b8f0c9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880ab2f0 con 0x7f7b88068550 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b880abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b880abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37528/0 (socket says 172.21.15.92:37528) 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 -- 172.21.15.92:0/663409825 learned_addr learned my addr 172.21.15.92:0/663409825 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b8ce65700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b88068550 0x7f7b88145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b8d666700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b8815e010 0x7f7b880ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 -- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b88068550 msgr2=0x7f7b88145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b88068550 0x7f7b88145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 -- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b8815e010 msgr2=0x7f7b880ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b8815e010 0x7f7b880ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 -- 172.21.15.92:0/663409825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80009ae0 con 0x7f7b8806f640 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7ffff700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b880abb90 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800c880 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78011600 con 0x7f7b8806f640 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b78011760 con 0x7f7b8806f640 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b88159480 con 0x7f7b8806f640 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78011b20 con 0x7f7b8806f640 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.470+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b880aa530 con 0x7f7b8806f640 2023-10-06T03:09:46.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.471+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b88159c80 con 0x7f7b8806f640 2023-10-06T03:09:46.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.471+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7b780193b0 con 0x7f7b8806f640 2023-10-06T03:09:46.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.472+0000 7f7b7cff9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b74067320 0x7f7b740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.472+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7b78085150 con 0x7f7b8806f640 2023-10-06T03:09:46.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.472+0000 7f7b8ce65700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b74067320 0x7f7b740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.472+0000 7f7b8ce65700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b74067320 0x7f7b740697d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400acb0 tx=0x7f7b84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.476+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b88159c80 con 0x7f7b8806f640 2023-10-06T03:09:46.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.700+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b881592c0 con 0x7f7b8806f640 2023-10-06T03:09:46.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.701+0000 7f7b7cff9700 1 -- 172.21.15.92:0/663409825 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f7b78051b70 con 0x7f7b8806f640 2023-10-06T03:09:46.703 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:46.703 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:33.377515+0000","modified":"2023-10-06T03:09:37.016548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2005327996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2005327996},{"type":"v1","addr":"172.21.15.92:6837","nonce":2005327996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b74067320 msgr2=0x7f7b740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b74067320 0x7f7b740697d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400acb0 tx=0x7f7b84008040 comp rx=0 tx=0).stop 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 msgr2=0x7f7b880abb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b880abb90 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800c880 tx=0x7f7b78008040 comp rx=0 tx=0).stop 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 shutdown_connections 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7b74067320 0x7f7b740697d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b8815e010 0x7f7b880ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b88068550 0x7f7b88145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 --2- 172.21.15.92:0/663409825 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7b8806f640 0x7f7b880abb90 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 >> 172.21.15.92:0/663409825 conn(0x7f7b880cd480 msgr2=0x7f7b88152a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 shutdown_connections 2023-10-06T03:09:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.703+0000 7f7b8f0c9700 1 -- 172.21.15.92:0/663409825 wait complete. 2023-10-06T03:09:46.706 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 105 2023-10-06T03:09:46.717 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.829+0000 7f7c9ad31700 1 Processor -- start 2023-10-06T03:09:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.829+0000 7f7c9ad31700 1 -- start start 2023-10-06T03:09:46.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c9ad31700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7c94159890 0x7f7c9415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c9ad31700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7c94069560 0x7f7c940699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c9ad31700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c94069ee0 con 0x7f7c94159c60 2023-10-06T03:09:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c9ad31700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9415a160 con 0x7f7c94159890 2023-10-06T03:09:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c9ad31700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c940d13a0 con 0x7f7c94069560 2023-10-06T03:09:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c98acd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7c94159890 0x7f7c9415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c8bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7c94159c60 0x7f7c9415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47446/0 (socket says 172.21.15.92:47446) 2023-10-06T03:09:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c98acd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7c94159890 0x7f7c9415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37544/0 (socket says 172.21.15.92:37544) 2023-10-06T03:09:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c8bfff700 1 -- 172.21.15.92:0/4163006440 learned_addr learned my addr 172.21.15.92:0/4163006440 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c992ce700 1 --2- 172.21.15.92:0/4163006440 >> v2:172.21.15.136:3300/0 conn(0x7f7c94069560 0x7f7c940699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c98acd700 1 -- 172.21.15.92:0/4163006440 >> v2:172.21.15.136:3300/0 conn(0x7f7c94069560 msgr2=0x7f7c940699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c98acd700 1 --2- 172.21.15.92:0/4163006440 >> v2:172.21.15.136:3300/0 conn(0x7f7c94069560 0x7f7c940699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.830+0000 7f7c98acd700 1 -- 172.21.15.92:0/4163006440 >> v1:172.21.15.92:6789/0 conn(0x7f7c94159c60 legacy=0x7f7c9415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c98acd700 1 -- 172.21.15.92:0/4163006440 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9415e140 con 0x7f7c94159890 2023-10-06T03:09:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c8b7fe700 1 -- 172.21.15.92:0/4163006440 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395115336 0 0) 0x7f7c94069ee0 con 0x7f7c94159c60 2023-10-06T03:09:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c98acd700 1 --2- 172.21.15.92:0/4163006440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c9415abd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7c9406a890 tx=0x7f7c900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21cb8da0eef205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c8b7fe700 1 -- 172.21.15.92:0/4163006440 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000b6b0 con 0x7f7c94159890 2023-10-06T03:09:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c8b7fe700 1 -- 172.21.15.92:0/4163006440 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9000b810 con 0x7f7c94159890 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c8b7fe700 1 -- 172.21.15.92:0/4163006440 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000bbb0 con 0x7f7c94159890 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 -- 172.21.15.92:0/4163006440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 msgr2=0x7f7c9415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/4163006440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c9415abd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7c9406a890 tx=0x7f7c900091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 -- 172.21.15.92:0/4163006440 shutdown_connections 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/4163006440 >> v2:172.21.15.136:3300/0 conn(0x7f7c94069560 0x7f7c940699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/4163006440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c9415abd0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 -- 172.21.15.92:0/4163006440 >> 172.21.15.92:0/4163006440 conn(0x7f7c94068e20 msgr2=0x7f7c940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 -- 172.21.15.92:0/4163006440 shutdown_connections 2023-10-06T03:09:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.831+0000 7f7c9ad31700 1 -- 172.21.15.92:0/4163006440 wait complete. 2023-10-06T03:09:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.832+0000 7f7c9ad31700 1 Processor -- start 2023-10-06T03:09:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 -- start start 2023-10-06T03:09:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c94069560 0x7f7c94156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c940b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94159c60 0x7f7c94153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940b3fd0 con 0x7f7c94159890 2023-10-06T03:09:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940b36a0 con 0x7f7c94159c60 2023-10-06T03:09:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.833+0000 7f7c9ad31700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c940b3980 con 0x7f7c94069560 2023-10-06T03:09:46.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c940b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c940b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37560/0 (socket says 172.21.15.92:37560) 2023-10-06T03:09:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 -- 172.21.15.92:0/706381246 learned_addr learned my addr 172.21.15.92:0/706381246 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c992ce700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94159c60 0x7f7c94153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c98acd700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c94069560 0x7f7c94156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 -- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c94069560 msgr2=0x7f7c94156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c94069560 0x7f7c94156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 -- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94159c60 msgr2=0x7f7c94153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94159c60 0x7f7c94153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 -- 172.21.15.92:0/706381246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90009420 con 0x7f7c94159890 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c98acd700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c94069560 0x7f7c94156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c8bfff700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c940b5bd0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c5b0 tx=0x7f7c84001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84005b30 con 0x7f7c94159890 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c84005c90 con 0x7f7c94159890 2023-10-06T03:09:46.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c940b3c60 con 0x7f7c94159890 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84007300 con 0x7f7c94159890 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.834+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c94156970 con 0x7f7c94159890 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.835+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c940b4140 con 0x7f7c94159890 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.835+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7c8401b3b0 con 0x7f7c94159890 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.836+0000 7f7c88ff9700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c80067320 0x7f7c800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.836+0000 7f7c98acd700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c80067320 0x7f7c800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.836+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7c840842b0 con 0x7f7c94159890 2023-10-06T03:09:46.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.836+0000 7f7c98acd700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c80067320 0x7f7c800697d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7c900093f0 tx=0x7f7c90017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:46.840+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c84050de0 con 0x7f7c94159890 2023-10-06T03:09:47.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.023+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c9404c3c0 con 0x7f7c94159890 2023-10-06T03:09:47.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.024+0000 7f7c88ff9700 1 -- 172.21.15.92:0/706381246 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f7c84054400 con 0x7f7c94159890 2023-10-06T03:09:47.026 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:47.026 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":155,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:34.995208+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:31.023983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:32.339528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c80067320 msgr2=0x7f7c800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c80067320 0x7f7c800697d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7c900093f0 tx=0x7f7c90017000 comp rx=0 tx=0).stop 2023-10-06T03:09:47.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 msgr2=0x7f7c940b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c940b5bd0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c5b0 tx=0x7f7c84001070 comp rx=0 tx=0).stop 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 shutdown_connections 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7c80067320 0x7f7c800697d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c94159c60 0x7f7c94153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c94069560 0x7f7c94156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 --2- 172.21.15.92:0/706381246 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7c94159890 0x7f7c940b5bd0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 >> 172.21.15.92:0/706381246 conn(0x7f7c94068e20 msgr2=0x7f7c94155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 shutdown_connections 2023-10-06T03:09:47.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.026+0000 7f7c9ad31700 1 -- 172.21.15.92:0/706381246 wait complete. 2023-10-06T03:09:47.041 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:09:47.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:47.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.158+0000 7fca8d2b6700 1 Processor -- start 2023-10-06T03:09:47.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.158+0000 7fca8d2b6700 1 -- start start 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8d2b6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca88159420 0x7fca880b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8d2b6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fca881597f0 0x7fca880b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8d2b6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8815d230 0x7fca8815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8d2b6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8815dbb0 con 0x7fca88159420 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8d2b6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca880b11d0 con 0x7fca881597f0 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8d2b6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880b14b0 con 0x7fca8815d230 2023-10-06T03:09:47.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca88159420 0x7fca880b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fca88159420 0x7fca880b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37576/0 (socket says 172.21.15.92:37576) 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 -- 172.21.15.92:0/3474652687 learned_addr learned my addr 172.21.15.92:0/3474652687 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8cab5700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8815d230 0x7fca8815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca877fe700 1 --2- 172.21.15.92:0/3474652687 >> v2:172.21.15.136:3300/0 conn(0x7fca881597f0 0x7fca880b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 -- 172.21.15.92:0/3474652687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8815d230 msgr2=0x7fca8815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8815d230 0x7fca8815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 -- 172.21.15.92:0/3474652687 >> v2:172.21.15.136:3300/0 conn(0x7fca881597f0 msgr2=0x7fca880b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 --2- 172.21.15.92:0/3474652687 >> v2:172.21.15.136:3300/0 conn(0x7fca881597f0 0x7fca880b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 -- 172.21.15.92:0/3474652687 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880b1790 con 0x7fca88159420 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca8cab5700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8815d230 0x7fca8815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca877fe700 1 --2- 172.21.15.92:0/3474652687 >> v2:172.21.15.136:3300/0 conn(0x7fca881597f0 0x7fca880b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:47.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.159+0000 7fca87fff700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca880b5f40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fca88089750 tx=0x7fca78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d0060a6d39d6fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca86ffd700 1 -- 172.21.15.92:0/3474652687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800b860 con 0x7fca88159420 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca86ffd700 1 -- 172.21.15.92:0/3474652687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7800b9c0 con 0x7fca88159420 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca86ffd700 1 -- 172.21.15.92:0/3474652687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800bd60 con 0x7fca88159420 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 -- 172.21.15.92:0/3474652687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 msgr2=0x7fca880b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca880b5f40 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fca88089750 tx=0x7fca78009250 comp rx=0 tx=0).stop 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 -- 172.21.15.92:0/3474652687 shutdown_connections 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8815d230 0x7fca8815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/3474652687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca880b5f40 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/3474652687 >> v2:172.21.15.136:3300/0 conn(0x7fca881597f0 0x7fca880b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 -- 172.21.15.92:0/3474652687 >> 172.21.15.92:0/3474652687 conn(0x7fca880d0660 msgr2=0x7fca880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:47.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 -- 172.21.15.92:0/3474652687 shutdown_connections 2023-10-06T03:09:47.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.160+0000 7fca8d2b6700 1 -- 172.21.15.92:0/3474652687 wait complete. 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.161+0000 7fca8d2b6700 1 Processor -- start 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.161+0000 7fca8d2b6700 1 -- start start 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca8d2b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca88169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca8d2b6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca881597f0 0x7fca88157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca8d2b6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8815d230 0x7fca880c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca8d2b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8816a450 con 0x7fca88159420 2023-10-06T03:09:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca8d2b6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca881579d0 con 0x7fca8815d230 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca8d2b6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880c7100 con 0x7fca881597f0 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca88169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca88169f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37578/0 (socket says 172.21.15.92:37578) 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 -- 172.21.15.92:0/1594277069 learned_addr learned my addr 172.21.15.92:0/1594277069 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 -- 172.21.15.92:0/1594277069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca881597f0 msgr2=0x7fca88157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca881597f0 0x7fca88157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 -- 172.21.15.92:0/1594277069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8815d230 msgr2=0x7fca880c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8815d230 0x7fca880c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.162+0000 7fca87fff700 1 -- 172.21.15.92:0/1594277069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780094f0 con 0x7fca88159420 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.163+0000 7fca87fff700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca88169f10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fca88089340 tx=0x7fca78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.163+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78024440 con 0x7fca88159420 2023-10-06T03:09:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.163+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780245a0 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.163+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca880c73e0 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.163+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78024960 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.163+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880c79a0 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.164+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8804c3c0 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.164+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fca7801a420 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.165+0000 7fca6ffff700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca740673a0 0x7fca74069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.165+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca78086c30 con 0x7fca88159420 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.165+0000 7fca877fe700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca740673a0 0x7fca74069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.165+0000 7fca877fe700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca740673a0 0x7fca74069850 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fca8815d200 tx=0x7fca7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.169+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca78053720 con 0x7fca88159420 2023-10-06T03:09:47.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.392+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca880b1a70 con 0x7fca88159420 2023-10-06T03:09:47.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.393+0000 7fca6ffff700 1 -- 172.21.15.92:0/1594277069 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fca78056d40 con 0x7fca88159420 2023-10-06T03:09:47.395 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:47.395 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:33.377515+0000","modified":"2023-10-06T03:09:37.016548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2005327996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2005327996},{"type":"v1","addr":"172.21.15.92:6837","nonce":2005327996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca740673a0 msgr2=0x7fca74069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca740673a0 0x7fca74069850 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fca8815d200 tx=0x7fca7c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 msgr2=0x7fca88169f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca88169f10 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fca88089340 tx=0x7fca78017000 comp rx=0 tx=0).stop 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 shutdown_connections 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fca740673a0 0x7fca74069850 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca88159420 0x7fca88169f10 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8815d230 0x7fca880c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 --2- 172.21.15.92:0/1594277069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca881597f0 0x7fca88157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 >> 172.21.15.92:0/1594277069 conn(0x7fca880d0660 msgr2=0x7fca880ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 shutdown_connections 2023-10-06T03:09:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.395+0000 7fca8d2b6700 1 -- 172.21.15.92:0/1594277069 wait complete. 2023-10-06T03:09:47.398 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 105 2023-10-06T03:09:47.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:47.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.521+0000 7fad57c4f700 1 Processor -- start 2023-10-06T03:09:47.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.521+0000 7fad57c4f700 1 -- start start 2023-10-06T03:09:47.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad57c4f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fad50159890 0x7fad5015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad57c4f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad50159c60 0x7fad5015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad57c4f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5015a160 con 0x7fad5015db10 2023-10-06T03:09:47.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad57c4f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5015df50 con 0x7fad50159c60 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad57c4f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad500d13a0 con 0x7fad50159890 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad551ea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fad50159c60 0x7fad5015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad561ec700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fad5015db10 0x7fad50069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47456/0 (socket says 172.21.15.92:47456) 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.522+0000 7fad561ec700 1 -- 172.21.15.92:0/3993414583 learned_addr learned my addr 172.21.15.92:0/3993414583 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad559eb700 1 --2- 172.21.15.92:0/3993414583 >> v2:172.21.15.136:3300/0 conn(0x7fad50159890 0x7fad5015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad551ea700 1 -- 172.21.15.92:0/3993414583 >> v2:172.21.15.136:3300/0 conn(0x7fad50159890 msgr2=0x7fad5015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad551ea700 1 --2- 172.21.15.92:0/3993414583 >> v2:172.21.15.136:3300/0 conn(0x7fad50159890 0x7fad5015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad551ea700 1 -- 172.21.15.92:0/3993414583 >> v1:172.21.15.92:6789/0 conn(0x7fad5015db10 legacy=0x7fad50069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad551ea700 1 -- 172.21.15.92:0/3993414583 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5015e140 con 0x7fad50159c60 2023-10-06T03:09:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad549e9700 1 -- 172.21.15.92:0/3993414583 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188950278 0 0) 0x7fad5015a160 con 0x7fad5015db10 2023-10-06T03:09:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad551ea700 1 --2- 172.21.15.92:0/3993414583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159c60 0x7fad5015d5d0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fad500d1090 tx=0x7fad400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bf69d97a803697 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad549e9700 1 -- 172.21.15.92:0/3993414583 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000b6b0 con 0x7fad50159c60 2023-10-06T03:09:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad549e9700 1 -- 172.21.15.92:0/3993414583 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4000b810 con 0x7fad50159c60 2023-10-06T03:09:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad549e9700 1 -- 172.21.15.92:0/3993414583 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000bbb0 con 0x7fad50159c60 2023-10-06T03:09:47.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad57c4f700 1 -- 172.21.15.92:0/3993414583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159c60 msgr2=0x7fad5015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad57c4f700 1 --2- 172.21.15.92:0/3993414583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159c60 0x7fad5015d5d0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fad500d1090 tx=0x7fad400091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad57c4f700 1 -- 172.21.15.92:0/3993414583 shutdown_connections 2023-10-06T03:09:47.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad57c4f700 1 --2- 172.21.15.92:0/3993414583 >> v2:172.21.15.136:3300/0 conn(0x7fad50159890 0x7fad5015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad57c4f700 1 --2- 172.21.15.92:0/3993414583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159c60 0x7fad5015d5d0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.523+0000 7fad57c4f700 1 -- 172.21.15.92:0/3993414583 >> 172.21.15.92:0/3993414583 conn(0x7fad50068e20 msgr2=0x7fad500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.524+0000 7fad57c4f700 1 -- 172.21.15.92:0/3993414583 shutdown_connections 2023-10-06T03:09:47.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.524+0000 7fad57c4f700 1 -- 172.21.15.92:0/3993414583 wait complete. 2023-10-06T03:09:47.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 Processor -- start 2023-10-06T03:09:47.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 -- start start 2023-10-06T03:09:47.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 0x7fad50152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad50159c60 0x7fad5015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5015db10 0x7fad50157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5015c100 con 0x7fad50159890 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500cde00 con 0x7fad5015db10 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad57c4f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5015c310 con 0x7fad50159c60 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad559eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 0x7fad50152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad559eb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 0x7fad50152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37596/0 (socket says 172.21.15.92:37596) 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.525+0000 7fad559eb700 1 -- 172.21.15.92:0/2913753077 learned_addr learned my addr 172.21.15.92:0/2913753077 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad551ea700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad50159c60 0x7fad5015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad561ec700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5015db10 0x7fad50157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad559eb700 1 -- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad50159c60 msgr2=0x7fad5015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad559eb700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad50159c60 0x7fad5015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad559eb700 1 -- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5015db10 msgr2=0x7fad50157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad559eb700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5015db10 0x7fad50157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad559eb700 1 -- 172.21.15.92:0/2913753077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009420 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad561ec700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5015db10 0x7fad50157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad559eb700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 0x7fad50152890 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e190 tx=0x7fad4c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00ee30 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c014420 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0147e0 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5015c5f0 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.526+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5015cbb0 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.527+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad500cdbd0 con 0x7fad50159890 2023-10-06T03:09:47.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.528+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fad4c014b90 con 0x7fad50159890 2023-10-06T03:09:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.528+0000 7fad45ffb700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad34067320 0x7fad340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.528+0000 7fad551ea700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad34067320 0x7fad340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.528+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad4c085630 con 0x7fad50159890 2023-10-06T03:09:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.528+0000 7fad551ea700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad34067320 0x7fad340697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fad400093f0 tx=0x7fad40017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.532+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad4c052160 con 0x7fad50159890 2023-10-06T03:09:47.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.760+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad5004c3c0 con 0x7fad50159890 2023-10-06T03:09:47.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.760+0000 7fad45ffb700 1 -- 172.21.15.92:0/2913753077 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fad4c009a60 con 0x7fad50159890 2023-10-06T03:09:47.762 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:47.762 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":105,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:33.377515+0000","modified":"2023-10-06T03:09:37.016548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7992},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7992":{"gid":7992,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/2005327996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2005327996},{"type":"v1","addr":"172.21.15.92:6837","nonce":2005327996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-06T03:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad34067320 msgr2=0x7fad340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad34067320 0x7fad340697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fad400093f0 tx=0x7fad40017000 comp rx=0 tx=0).stop 2023-10-06T03:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 msgr2=0x7fad50152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 0x7fad50152890 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00e190 tx=0x7fad4c009920 comp rx=0 tx=0).stop 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 shutdown_connections 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fad34067320 0x7fad340697d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fad50159890 0x7fad50152890 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad5015db10 0x7fad50157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 --2- 172.21.15.92:0/2913753077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad50159c60 0x7fad5015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.762+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 >> 172.21.15.92:0/2913753077 conn(0x7fad50068e20 msgr2=0x7fad50069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.763+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 shutdown_connections 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.763+0000 7fad57c4f700 1 -- 172.21.15.92:0/2913753077 wait complete. 2023-10-06T03:09:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 105 2023-10-06T03:09:47.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.889+0000 7f52488a7700 1 Processor -- start 2023-10-06T03:09:47.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.889+0000 7f52488a7700 1 -- start start 2023-10-06T03:09:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f52488a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f52488a7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f524415db10 0x7f5244069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f52488a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524415a160 con 0x7f5244159890 2023-10-06T03:09:47.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f52488a7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52440d13a0 con 0x7f524415db10 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f52488a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524415e140 con 0x7f5244159c60 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f5242d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f524415db10 0x7f5244069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f524259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5244159890 0x7f524415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47468/0 (socket says 172.21.15.92:47468) 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f524259c700 1 -- 172.21.15.92:0/4028170763 learned_addr learned my addr 172.21.15.92:0/4028170763 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.890+0000 7f5239d9b700 1 --2- 172.21.15.92:0/4028170763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5242d9d700 1 -- 172.21.15.92:0/4028170763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 msgr2=0x7f524415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5242d9d700 1 --2- 172.21.15.92:0/4028170763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5242d9d700 1 -- 172.21.15.92:0/4028170763 >> v1:172.21.15.92:6789/0 conn(0x7f5244159890 legacy=0x7f524415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5242d9d700 1 -- 172.21.15.92:0/4028170763 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524415a3d0 con 0x7f524415db10 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5241d9b700 1 -- 172.21.15.92:0/4028170763 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036871774 0 0) 0x7f524415a160 con 0x7f5244159890 2023-10-06T03:09:47.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5242d9d700 1 --2- 172.21.15.92:0/4028170763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f524415db10 0x7f5244069560 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f523c009e60 tx=0x7f523c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b12977836e6982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5241d9b700 1 -- 172.21.15.92:0/4028170763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c00e070 con 0x7f524415db10 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5241d9b700 1 -- 172.21.15.92:0/4028170763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c00bc30 con 0x7f524415db10 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f5241d9b700 1 -- 172.21.15.92:0/4028170763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c012430 con 0x7f524415db10 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 -- 172.21.15.92:0/4028170763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f524415db10 msgr2=0x7f5244069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 --2- 172.21.15.92:0/4028170763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f524415db10 0x7f5244069560 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f523c009e60 tx=0x7f523c009780 comp rx=0 tx=0).stop 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 -- 172.21.15.92:0/4028170763 shutdown_connections 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 --2- 172.21.15.92:0/4028170763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f524415db10 0x7f5244069560 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 --2- 172.21.15.92:0/4028170763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 -- 172.21.15.92:0/4028170763 >> 172.21.15.92:0/4028170763 conn(0x7f5244068e20 msgr2=0x7f52440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 -- 172.21.15.92:0/4028170763 shutdown_connections 2023-10-06T03:09:47.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.891+0000 7f52488a7700 1 -- 172.21.15.92:0/4028170763 wait complete. 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 Processor -- start 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 -- start start 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 0x7f52440b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524414a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524415db10 0x7f5244151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524414ad30 con 0x7f5244159890 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52440b2340 con 0x7f524415db10 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f52488a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52441523e0 con 0x7f5244159c60 2023-10-06T03:09:47.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f524259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 0x7f52440b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f524259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 0x7f52440b13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37624/0 (socket says 172.21.15.92:37624) 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.893+0000 7f524259c700 1 -- 172.21.15.92:0/4222837327 learned_addr learned my addr 172.21.15.92:0/4222837327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f524259c700 1 -- 172.21.15.92:0/4222837327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 msgr2=0x7f524414a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f524259c700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524414a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f524259c700 1 -- 172.21.15.92:0/4222837327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524415db10 msgr2=0x7f5244151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f524259c700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524415db10 0x7f5244151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f524259c700 1 -- 172.21.15.92:0/4222837327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c009a20 con 0x7f5244159890 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f524259c700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 0x7f52440b13e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f523400c810 tx=0x7f5234007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234005b30 con 0x7f5244159890 2023-10-06T03:09:47.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234005c90 con 0x7f5244159890 2023-10-06T03:09:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524414b060 con 0x7f5244159890 2023-10-06T03:09:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234007300 con 0x7f5244159890 2023-10-06T03:09:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.894+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52440b1b20 con 0x7f5244159890 2023-10-06T03:09:47.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.895+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524414b4c0 con 0x7f5244159890 2023-10-06T03:09:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.895+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f523401b400 con 0x7f5244159890 2023-10-06T03:09:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.896+0000 7f523affd700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5224067320 0x7f52240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.896+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f52340847a0 con 0x7f5244159890 2023-10-06T03:09:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.896+0000 7f5239d9b700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5224067320 0x7f52240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.896+0000 7f5239d9b700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5224067320 0x7f52240697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f52440b6630 tx=0x7f522c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:47.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:47.900+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5234051450 con 0x7f5244159890 2023-10-06T03:09:48.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.085+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f524404c3c0 con 0x7f5244159890 2023-10-06T03:09:48.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.086+0000 7f523affd700 1 -- 172.21.15.92:0/4222837327 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f5234054a70 con 0x7f5244159890 2023-10-06T03:09:48.087 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:48.087 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":155,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:34.995208+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:31.023983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:32.339528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5224067320 msgr2=0x7f52240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5224067320 0x7f52240697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f52440b6630 tx=0x7f522c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 msgr2=0x7f52440b13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 0x7f52440b13e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f523400c810 tx=0x7f5234007df0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 shutdown_connections 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5224067320 0x7f52240697d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5244159890 0x7f52440b13e0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f524415db10 0x7f5244151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 --2- 172.21.15.92:0/4222837327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5244159c60 0x7f524414a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 >> 172.21.15.92:0/4222837327 conn(0x7f5244068e20 msgr2=0x7f5244156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 shutdown_connections 2023-10-06T03:09:48.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.088+0000 7f52488a7700 1 -- 172.21.15.92:0/4222837327 wait complete. 2023-10-06T03:09:48.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:09:48.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.218+0000 7f4137330700 1 Processor -- start 2023-10-06T03:09:48.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.218+0000 7f4137330700 1 -- start start 2023-10-06T03:09:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4137330700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f41301597f0 0x7f41300b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4137330700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 0x7f413015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4137330700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413015dbb0 con 0x7f4130159420 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4137330700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300b11d0 con 0x7f41301597f0 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4137330700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b14b0 con 0x7f413015d230 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413632e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4130159420 0x7f41300b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47472/0 (socket says 172.21.15.92:47472) 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413632e700 1 -- 172.21.15.92:0/1277158441 learned_addr learned my addr 172.21.15.92:0/1277158441 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4136b2f700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 0x7f413015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 --2- 172.21.15.92:0/1277158441 >> v2:172.21.15.136:3300/0 conn(0x7f41301597f0 0x7f41300b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817664520 0 0) 0x7f413015dbb0 con 0x7f4130159420 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c003570 con 0x7f4130159420 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143801371 0 0) 0x7f411c003570 con 0x7f4130159420 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 msgr2=0x7f413015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 0x7f413015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 >> v2:172.21.15.136:3300/0 conn(0x7f41301597f0 msgr2=0x7f41300b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 --2- 172.21.15.92:0/1277158441 >> v2:172.21.15.136:3300/0 conn(0x7f41301597f0 0x7f41300b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300b1790 con 0x7f4130159420 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f412c001470 con 0x7f4130159420 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 >> v1:172.21.15.92:6789/0 conn(0x7f4130159420 legacy=0x7f41300b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 0x7f411c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f411c003fe0 0x7f411c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c004d50 0x7f411c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b11d0 con 0x7f411c003fe0 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b14b0 con 0x7f411c004d50 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c00ba90 con 0x7f413015d230 2023-10-06T03:09:48.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f411c003fe0 0x7f411c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413632e700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 0x7f411c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4136b2f700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c004d50 0x7f411c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 -- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 msgr2=0x7f411c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015d230 0x7f411c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 -- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c004d50 msgr2=0x7f411c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c004d50 0x7f411c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 -- 172.21.15.92:0/1277158441 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c003570 con 0x7f411c003fe0 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4136b2f700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c004d50 0x7f411c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f4135b2d700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f411c003fe0 0x7f411c004810 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4130159c70 tx=0x7f41200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd6f7218c9af8fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000e6b0 con 0x7f411c003fe0 2023-10-06T03:09:48.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412000e810 con 0x7f411c003fe0 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.219+0000 7f413532c700 1 -- 172.21.15.92:0/1277158441 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000ebd0 con 0x7f411c003fe0 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4137330700 1 -- 172.21.15.92:0/1277158441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f411c003fe0 msgr2=0x7f411c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4137330700 1 --2- 172.21.15.92:0/1277158441 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f411c003fe0 0x7f411c004810 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4130159c70 tx=0x7f41200091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4136b2f700 1 -- 172.21.15.92:0/1277158441 reap_dead start 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4137330700 1 -- 172.21.15.92:0/1277158441 shutdown_connections 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4137330700 1 -- 172.21.15.92:0/1277158441 >> 172.21.15.92:0/1277158441 conn(0x7f41300d0660 msgr2=0x7f41300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4137330700 1 -- 172.21.15.92:0/1277158441 shutdown_connections 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.220+0000 7f4137330700 1 -- 172.21.15.92:0/1277158441 wait complete. 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.221+0000 7f4137330700 1 Processor -- start 2023-10-06T03:09:48.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.221+0000 7f4137330700 1 -- start start 2023-10-06T03:09:48.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c003fe0 0x7f41301543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015c1e0 0x7f41300c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 0x7f41300698c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130154910 con 0x7f4130069440 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4130069e00 con 0x7f411c003fe0 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300c5d00 con 0x7f413015c1e0 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 0x7f41300698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 0x7f41300698c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37642/0 (socket says 172.21.15.92:37642) 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 -- 172.21.15.92:0/3232958114 learned_addr learned my addr 172.21.15.92:0/3232958114 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4135b2d700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015c1e0 0x7f41300c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f413632e700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c003fe0 0x7f41301543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 -- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015c1e0 msgr2=0x7f41300c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015c1e0 0x7f41300c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 -- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c003fe0 msgr2=0x7f41301543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c003fe0 0x7f41301543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 -- 172.21.15.92:0/3232958114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300c5fe0 con 0x7f4130069440 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4136b2f700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 0x7f41300698c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f413015e340 tx=0x7f412800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412801b070 con 0x7f4130069440 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412800f040 con 0x7f4130069440 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41300c62c0 con 0x7f4130069440 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800e050 con 0x7f4130069440 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.222+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41300c9190 con 0x7f4130069440 2023-10-06T03:09:48.230 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.223+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41300c9720 con 0x7f4130069440 2023-10-06T03:09:48.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.224+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4128003020 con 0x7f4130069440 2023-10-06T03:09:48.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.224+0000 7f41267fc700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4114067320 0x7f41140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.224+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f41280851f0 con 0x7f4130069440 2023-10-06T03:09:48.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.224+0000 7f413632e700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4114067320 0x7f41140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.224+0000 7f413632e700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4114067320 0x7f41140697d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f412c001620 tx=0x7f412c001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:48.231 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.228+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41300c9720 con 0x7f4130069440 2023-10-06T03:09:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.454+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f41300c9720 con 0x7f4130069440 2023-10-06T03:09:48.456 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:09:48.454+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:09:48.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.462+0000 7f41267fc700 1 -- 172.21.15.92:0/3232958114 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f41300c9720 con 0x7f4130069440 2023-10-06T03:09:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4114067320 msgr2=0x7f41140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4114067320 0x7f41140697d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f412c001620 tx=0x7f412c001e90 comp rx=0 tx=0).stop 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 msgr2=0x7f41300698c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 0x7f41300698c0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f413015e340 tx=0x7f412800b040 comp rx=0 tx=0).stop 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 shutdown_connections 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4114067320 0x7f41140697d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4130069440 0x7f41300698c0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f411c003fe0 0x7f41301543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 --2- 172.21.15.92:0/3232958114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f413015c1e0 0x7f41300c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 >> 172.21.15.92:0/3232958114 conn(0x7f41300d0660 msgr2=0x7f4130155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 shutdown_connections 2023-10-06T03:09:48.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.465+0000 7f4137330700 1 -- 172.21.15.92:0/3232958114 wait complete. 2023-10-06T03:09:48.468 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:09:48.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:09:48.483 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:09:48.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.595+0000 7fc0ac7df700 1 Processor -- start 2023-10-06T03:09:48.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.595+0000 7fc0ac7df700 1 -- start start 2023-10-06T03:09:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0ac7df700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc0a4069560 0x7fc0a40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0ac7df700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a4069ee0 con 0x7fc0a4159890 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0ac7df700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a415a160 con 0x7fc0a4159c60 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0ac7df700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a40d13a0 con 0x7fc0a4069560 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0aa57b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc0a4159890 0x7fc0a415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47490/0 (socket says 172.21.15.92:47490) 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0aa57b700 1 -- 172.21.15.92:0/2742022599 learned_addr learned my addr 172.21.15.92:0/2742022599 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446276625 0 0) 0x7fc0a4069ee0 con 0x7fc0a4159890 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc090003570 con 0x7fc0a4159890 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0aad7c700 1 --2- 172.21.15.92:0/2742022599 >> v2:172.21.15.136:3300/0 conn(0x7fc0a4069560 0x7fc0a40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1719536417 0 0) 0x7fc090003570 con 0x7fc0a4159890 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 >> v2:172.21.15.136:3300/0 conn(0x7fc0a4069560 msgr2=0x7fc0a40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 --2- 172.21.15.92:0/2742022599 >> v2:172.21.15.136:3300/0 conn(0x7fc0a4069560 0x7fc0a40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 >> v1:172.21.15.136:6789/0 conn(0x7fc0a4159c60 legacy=0x7fc0a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a415e140 con 0x7fc0a4159890 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc0a00017f0 con 0x7fc0a4159890 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 >> v1:172.21.15.92:6789/0 conn(0x7fc0a4159890 legacy=0x7fc0a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc090003a60 0x7fc090003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc090004810 0x7fc090004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc0900051c0 0x7fc09000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40d13a0 con 0x7fc090004810 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09000c0d0 con 0x7fc090003a60 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09000c290 con 0x7fc0900051c0 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc090004810 0x7fc090004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0aa57b700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc090003a60 0x7fc090003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0aad7c700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc0900051c0 0x7fc09000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc0900051c0 msgr2=0x7fc09000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc0900051c0 0x7fc09000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc090003a60 msgr2=0x7fc090003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc090003a60 0x7fc090003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2742022599 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc090003570 con 0x7fc090004810 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0aad7c700 1 -- 172.21.15.92:0/2742022599 reap_dead start 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.597+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc090004810 0x7fc090004c80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc0940018b0 tx=0x7fc094007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80a9d7ae1aae4a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094012420 con 0x7fc090004810 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094012580 con 0x7fc090004810 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0a9579700 1 -- 172.21.15.92:0/2742022599 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094012920 con 0x7fc090004810 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2742022599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc090004810 msgr2=0x7fc090004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc090004810 0x7fc090004c80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc0940018b0 tx=0x7fc094007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2742022599 shutdown_connections 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2742022599 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc090004810 0x7fc090004c80 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2742022599 >> 172.21.15.92:0/2742022599 conn(0x7fc0a4068e20 msgr2=0x7fc0a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2742022599 shutdown_connections 2023-10-06T03:09:48.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.598+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2742022599 wait complete. 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.599+0000 7fc0ac7df700 1 Processor -- start 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.599+0000 7fc0ac7df700 1 -- start start 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0ac7df700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc090004810 0x7fc0a40b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0ac7df700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 0x7fc0a40ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0ac7df700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a414d250 0x7fc0a4151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0ac7df700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40b1650 con 0x7fc0a40b0540 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0ac7df700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40b0a40 con 0x7fc0a414d250 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0ac7df700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40b0d20 con 0x7fc090004810 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 0x7fc0a40ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 0x7fc0a40ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37698/0 (socket says 172.21.15.92:37698) 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2396019545 learned_addr learned my addr 172.21.15.92:0/2396019545 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:48.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0aa57b700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc090004810 0x7fc0a40b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0aad7c700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a414d250 0x7fc0a4151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc090004810 msgr2=0x7fc0a40b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc090004810 0x7fc0a40b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a414d250 msgr2=0x7fc0a4151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a414d250 0x7fc0a4151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 -- 172.21.15.92:0/2396019545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40b1000 con 0x7fc0a40b0540 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.600+0000 7fc0a9d7a700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 0x7fc0a40ce4a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c3f0 tx=0x7fc094015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.601+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094001200 con 0x7fc0a40b0540 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.601+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094012c90 con 0x7fc0a40b0540 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.601+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a40b12e0 con 0x7fc0a40b0540 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.601+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094012920 con 0x7fc0a40b0540 2023-10-06T03:09:48.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.601+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a414d6a0 con 0x7fc0a40b0540 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.601+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a40b1800 con 0x7fc0a40b0540 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.602+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc09401f620 con 0x7fc0a40b0540 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.602+0000 7fc09a7fc700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc088067320 0x7fc0880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.602+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fc094086770 con 0x7fc0a40b0540 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.602+0000 7fc0aa57b700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc088067320 0x7fc0880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.603+0000 7fc0aa57b700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc088067320 0x7fc0880697d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0001920 tx=0x7fc0a0006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:48.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.607+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0a40b1800 con 0x7fc0a40b0540 2023-10-06T03:09:48.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:48.834+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0a404c3c0 con 0x7fc0a40b0540 2023-10-06T03:09:49.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.464+0000 7fc09a7fc700 1 -- 172.21.15.92:0/2396019545 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc094053250 con 0x7fc0a40b0540 2023-10-06T03:09:49.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc088067320 msgr2=0x7fc0880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:49.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc088067320 0x7fc0880697d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0001920 tx=0x7fc0a0006040 comp rx=0 tx=0).stop 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 msgr2=0x7fc0a40ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 0x7fc0a40ce4a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c3f0 tx=0x7fc094015000 comp rx=0 tx=0).stop 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 shutdown_connections 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc088067320 0x7fc0880697d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0a414d250 0x7fc0a4151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc090004810 0x7fc0a40b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 --2- 172.21.15.92:0/2396019545 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc0a40b0540 0x7fc0a40ce4a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 >> 172.21.15.92:0/2396019545 conn(0x7fc0a4068e20 msgr2=0x7fc0a40d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 shutdown_connections 2023-10-06T03:09:49.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.467+0000 7fc0ac7df700 1 -- 172.21.15.92:0/2396019545 wait complete. 2023-10-06T03:09:49.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:09:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.637+0000 7f95472bc700 1 Processor -- start 2023-10-06T03:09:49.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.637+0000 7f95472bc700 1 -- start start 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f95472bc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95400b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f95472bc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95400b53d0 0x7f95400cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f95472bc700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95400b5b00 con 0x7f95400cce40 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f95472bc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95400b5da0 con 0x7f95400b53d0 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f95472bc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400b6080 con 0x7f9540159ec0 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f9544857700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95400b53d0 0x7f95400cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f9545859700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f95400cce40 0x7f95400cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60276/0 (socket says 172.21.15.92:60276) 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f9544857700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95400b53d0 0x7f95400cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57776/0 (socket says 172.21.15.92:57776) 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.638+0000 7f9545859700 1 -- 172.21.15.92:0/3448969387 learned_addr learned my addr 172.21.15.92:0/3448969387 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:49.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9544857700 1 -- 172.21.15.92:0/3448969387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 msgr2=0x7f95400b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9545058700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95400b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9537fff700 1 -- 172.21.15.92:0/3448969387 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321286754 0 0) 0x7f95400b5b00 con 0x7f95400cce40 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9544857700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95400b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9544857700 1 -- 172.21.15.92:0/3448969387 >> v1:172.21.15.92:6789/0 conn(0x7f95400cce40 legacy=0x7f95400cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9544857700 1 -- 172.21.15.92:0/3448969387 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400cc100 con 0x7f95400b53d0 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9545058700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95400b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9544857700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400b53d0 0x7f95400cc900 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f95300098e0 tx=0x7f95300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07284171ab5bc6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9537fff700 1 -- 172.21.15.92:0/3448969387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530009e80 con 0x7f95400b53d0 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9537fff700 1 -- 172.21.15.92:0/3448969387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953000b860 con 0x7f95400b53d0 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f9537fff700 1 -- 172.21.15.92:0/3448969387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000bbe0 con 0x7f95400b53d0 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f95472bc700 1 -- 172.21.15.92:0/3448969387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400b53d0 msgr2=0x7f95400cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:49.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.639+0000 7f95472bc700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400b53d0 0x7f95400cc900 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f95300098e0 tx=0x7f95300091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.640+0000 7f95472bc700 1 -- 172.21.15.92:0/3448969387 shutdown_connections 2023-10-06T03:09:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.640+0000 7f95472bc700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95400b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.640+0000 7f95472bc700 1 --2- 172.21.15.92:0/3448969387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400b53d0 0x7f95400cc900 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.640+0000 7f95472bc700 1 -- 172.21.15.92:0/3448969387 >> 172.21.15.92:0/3448969387 conn(0x7f95400999f0 msgr2=0x7f95400b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.640+0000 7f95472bc700 1 -- 172.21.15.92:0/3448969387 shutdown_connections 2023-10-06T03:09:49.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.640+0000 7f95472bc700 1 -- 172.21.15.92:0/3448969387 wait complete. 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.641+0000 7f95472bc700 1 Processor -- start 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.641+0000 7f95472bc700 1 -- start start 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95400b53d0 0x7f954006a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 0x7f9540152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95401516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540153170 con 0x7f95400cce40 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540153450 con 0x7f95400b53d0 2023-10-06T03:09:49.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9540069a60 con 0x7f9540159ec0 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 0x7f9540152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 0x7f9540152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57784/0 (socket says 172.21.15.92:57784) 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 -- 172.21.15.92:0/1895197182 learned_addr learned my addr 172.21.15.92:0/1895197182 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9545859700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95401516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9545058700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95400b53d0 0x7f954006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 -- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 msgr2=0x7f95401516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95401516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 -- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95400b53d0 msgr2=0x7f954006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:49.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95400b53d0 0x7f954006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 -- 172.21.15.92:0/1895197182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9530009420 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f9544857700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 0x7f9540152940 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9540089390 tx=0x7f9530017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000b9d0 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953000bb50 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9540069d40 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530025670 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.642+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9540152e80 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.643+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954006a260 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.644+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9530023070 con 0x7f95400cce40 2023-10-06T03:09:49.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.644+0000 7f95357fa700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9524067320 0x7f95240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.644+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9530086520 con 0x7f95400cce40 2023-10-06T03:09:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.644+0000 7f9545058700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9524067320 0x7f95240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.644+0000 7f9545058700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9524067320 0x7f95240697d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f953c00ab80 tx=0x7f953c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:49.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.648+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f954006a260 con 0x7f95400cce40 2023-10-06T03:09:49.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:49.830+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f954006a260 con 0x7f95400cce40 2023-10-06T03:09:50.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.466+0000 7f95357fa700 1 -- 172.21.15.92:0/1895197182 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v157) v1 ==== 181+0+0 (secure 0 0 0) 0x7f954006a260 con 0x7f95400cce40 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9524067320 msgr2=0x7f95240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9524067320 0x7f95240697d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f953c00ab80 tx=0x7f953c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 msgr2=0x7f9540152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 0x7f9540152940 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9540089390 tx=0x7f9530017000 comp rx=0 tx=0).stop 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 shutdown_connections 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9524067320 0x7f95240697d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9540159ec0 0x7f95401516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95400b53d0 0x7f954006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 --2- 172.21.15.92:0/1895197182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95400cce40 0x7f9540152940 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 >> 172.21.15.92:0/1895197182 conn(0x7f95400999f0 msgr2=0x7f95400c5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 shutdown_connections 2023-10-06T03:09:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.469+0000 7f95472bc700 1 -- 172.21.15.92:0/1895197182 wait complete. 2023-10-06T03:09:50.471 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:09:50.484 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:09:50.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.603+0000 7f8b5686c700 1 Processor -- start 2023-10-06T03:09:50.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.603+0000 7f8b5686c700 1 -- start start 2023-10-06T03:09:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.603+0000 7f8b5686c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8b50159ec0 0x7f8b500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.603+0000 7f8b5686c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8b500cf350 0x7f8b500cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b5686c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500b5b00 con 0x7f8b500b53d0 2023-10-06T03:09:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b5686c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500b5850 con 0x7f8b50159ec0 2023-10-06T03:09:50.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b5686c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b500b5da0 con 0x7f8b500cf350 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8b50159ec0 0x7f8b500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8b50159ec0 0x7f8b500b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57800/0 (socket says 172.21.15.92:57800) 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 -- 172.21.15.92:0/2159763810 learned_addr learned my addr 172.21.15.92:0/2159763810 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b54e09700 1 --2- 172.21.15.92:0/2159763810 >> v2:172.21.15.136:3300/0 conn(0x7f8b500cf350 0x7f8b500cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 -- 172.21.15.92:0/2159763810 >> v2:172.21.15.136:3300/0 conn(0x7f8b500cf350 msgr2=0x7f8b500cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 --2- 172.21.15.92:0/2159763810 >> v2:172.21.15.136:3300/0 conn(0x7f8b500cf350 0x7f8b500cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 -- 172.21.15.92:0/2159763810 >> v1:172.21.15.136:6789/0 conn(0x7f8b500b53d0 legacy=0x7f8b500cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 -- 172.21.15.92:0/2159763810 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500b6080 con 0x7f8b50159ec0 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4ffff700 1 --2- 172.21.15.92:0/2159763810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b500b4e90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f8b500cb4a0 tx=0x7f8b40009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947ef87865dbe032 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4effd700 1 -- 172.21.15.92:0/2159763810 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40005c10 con 0x7f8b50159ec0 2023-10-06T03:09:50.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4effd700 1 -- 172.21.15.92:0/2159763810 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40005d70 con 0x7f8b50159ec0 2023-10-06T03:09:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.604+0000 7f8b4effd700 1 -- 172.21.15.92:0/2159763810 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000b840 con 0x7f8b50159ec0 2023-10-06T03:09:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 -- 172.21.15.92:0/2159763810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 msgr2=0x7f8b500b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 --2- 172.21.15.92:0/2159763810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b500b4e90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f8b500cb4a0 tx=0x7f8b40009220 comp rx=0 tx=0).stop 2023-10-06T03:09:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 -- 172.21.15.92:0/2159763810 shutdown_connections 2023-10-06T03:09:50.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 --2- 172.21.15.92:0/2159763810 >> v2:172.21.15.136:3300/0 conn(0x7f8b500cf350 0x7f8b500cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 --2- 172.21.15.92:0/2159763810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b500b4e90 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 -- 172.21.15.92:0/2159763810 >> 172.21.15.92:0/2159763810 conn(0x7f8b500999f0 msgr2=0x7f8b500b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:50.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 -- 172.21.15.92:0/2159763810 shutdown_connections 2023-10-06T03:09:50.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.605+0000 7f8b5686c700 1 -- 172.21.15.92:0/2159763810 wait complete. 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.606+0000 7f8b5686c700 1 Processor -- start 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.606+0000 7f8b5686c700 1 -- start start 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b5686c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b500b53d0 0x7f8b5016aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b5686c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b500cf350 0x7f8b50156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b5686c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b50068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b5686c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5016af60 con 0x7f8b50159ec0 2023-10-06T03:09:50.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b5686c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50156890 con 0x7f8b500cf350 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b5686c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5009b2d0 con 0x7f8b500b53d0 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b50068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b50068c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57816/0 (socket says 172.21.15.92:57816) 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 -- 172.21.15.92:0/544186456 learned_addr learned my addr 172.21.15.92:0/544186456 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b4ffff700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b500b53d0 0x7f8b5016aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b4f7fe700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b500cf350 0x7f8b50156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 -- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b500b53d0 msgr2=0x7f8b5016aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b500b53d0 0x7f8b5016aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 -- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b500cf350 msgr2=0x7f8b50156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b500cf350 0x7f8b50156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 -- 172.21.15.92:0/544186456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40009480 con 0x7f8b50159ec0 2023-10-06T03:09:50.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.607+0000 7f8b54e09700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b50068c30 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009fa0 tx=0x7f8b48009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.608+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800ed20 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.608+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4800ee80 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.608+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5009b5b0 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.608+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48014670 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.608+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b50068430 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.608+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b500718b0 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.609+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b480149e0 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.609+0000 7f8b37fff700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b3c067320 0x7f8b3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.609+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f8b48085460 con 0x7f8b50159ec0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.609+0000 7f8b4ffff700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b3c067320 0x7f8b3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:50.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.610+0000 7f8b4ffff700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b3c067320 0x7f8b3c0697d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8b400030c0 tx=0x7f8b40015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:50.615 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.613+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b480520f0 con 0x7f8b50159ec0 2023-10-06T03:09:50.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:50.797+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b50063c40 con 0x7f8b50159ec0 2023-10-06T03:09:51.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.485+0000 7f8b37fff700 1 -- 172.21.15.92:0/544186456 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v158) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b48055710 con 0x7f8b50159ec0 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b3c067320 msgr2=0x7f8b3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b3c067320 0x7f8b3c0697d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8b400030c0 tx=0x7f8b40015000 comp rx=0 tx=0).stop 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 msgr2=0x7f8b50068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b50068c30 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009fa0 tx=0x7f8b48009900 comp rx=0 tx=0).stop 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 shutdown_connections 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b3c067320 0x7f8b3c0697d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b50159ec0 0x7f8b50068c30 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b500b53d0 0x7f8b5016aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 --2- 172.21.15.92:0/544186456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b500cf350 0x7f8b50156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 >> 172.21.15.92:0/544186456 conn(0x7f8b500999f0 msgr2=0x7f8b501591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 shutdown_connections 2023-10-06T03:09:51.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.488+0000 7f8b5686c700 1 -- 172.21.15.92:0/544186456 wait complete. 2023-10-06T03:09:51.491 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:09:51.503 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:09:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.619+0000 7fcc6e12d700 1 Processor -- start 2023-10-06T03:09:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.619+0000 7fcc6e12d700 1 -- start start 2023-10-06T03:09:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6e12d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 0x7fcc680b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6e12d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcc680b33b0 0x7fcc680b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6e12d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680b2080 con 0x7fcc680b4e70 2023-10-06T03:09:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6e12d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680cfb00 con 0x7fcc680b33b0 2023-10-06T03:09:51.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6e12d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b5c40 con 0x7fcc680b16d0 2023-10-06T03:09:51.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6d92c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcc680b33b0 0x7fcc680b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6c92a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 0x7fcc680b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6c92a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 0x7fcc680b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:47294/0 (socket says 172.21.15.92:47294) 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6d92c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcc680b33b0 0x7fcc680b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:46156/0 (socket says 172.21.15.92:46156) 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6d12b700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fcc680b4e70 0x7fcc680ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:36674/0 (socket says 172.21.15.92:36674) 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.620+0000 7fcc6c92a700 1 -- 172.21.15.92:0/2528173830 learned_addr learned my addr 172.21.15.92:0/2528173830 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc6d92c700 1 -- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 msgr2=0x7fcc680b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc6d92c700 1 --2- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 0x7fcc680b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc6d92c700 1 -- 172.21.15.92:0/2528173830 >> v1:172.21.15.136:6789/0 conn(0x7fcc680b4e70 legacy=0x7fcc680ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc5ffff700 1 -- 172.21.15.92:0/2528173830 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190163372 0 0) 0x7fcc680b2080 con 0x7fcc680b4e70 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc6d92c700 1 -- 172.21.15.92:0/2528173830 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680b5f20 con 0x7fcc680b33b0 2023-10-06T03:09:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc6c92a700 1 --2- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 0x7fcc680b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc6d92c700 1 --2- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc680b3830 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc600098e0 tx=0x7fcc600091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e315011f63a8eb06 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc5ffff700 1 -- 172.21.15.92:0/2528173830 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000b640 con 0x7fcc680b33b0 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc5ffff700 1 -- 172.21.15.92:0/2528173830 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6000b7a0 con 0x7fcc680b33b0 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.621+0000 7fcc5ffff700 1 -- 172.21.15.92:0/2528173830 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000bb40 con 0x7fcc680b33b0 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 -- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 msgr2=0x7fcc680b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc680b3830 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc600098e0 tx=0x7fcc600091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 -- 172.21.15.92:0/2528173830 shutdown_connections 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc680b3830 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/2528173830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b16d0 0x7fcc680b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 -- 172.21.15.92:0/2528173830 >> 172.21.15.92:0/2528173830 conn(0x7fcc680d10c0 msgr2=0x7fcc680affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 -- 172.21.15.92:0/2528173830 shutdown_connections 2023-10-06T03:09:51.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.622+0000 7fcc6e12d700 1 -- 172.21.15.92:0/2528173830 wait complete. 2023-10-06T03:09:51.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.623+0000 7fcc6e12d700 1 Processor -- start 2023-10-06T03:09:51.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.623+0000 7fcc6e12d700 1 -- start start 2023-10-06T03:09:51.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 0x7fcc680c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc68169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b4e70 0x7fcc681574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680c5d30 con 0x7fcc680b16d0 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680c6010 con 0x7fcc680b33b0 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6816a410 con 0x7fcc680b4e70 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 0x7fcc680c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 0x7fcc680c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57830/0 (socket says 172.21.15.92:57830) 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 -- 172.21.15.92:0/4017921554 learned_addr learned my addr 172.21.15.92:0/4017921554 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6c92a700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc68169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d92c700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b4e70 0x7fcc681574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 -- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b4e70 msgr2=0x7fcc681574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b4e70 0x7fcc681574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 -- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 msgr2=0x7fcc68169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc68169bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 -- 172.21.15.92:0/4017921554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580094f0 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6c92a700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc68169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6d12b700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 0x7fcc680c7cc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400cd60 tx=0x7fcc64001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64005df0 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6400a920 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400ace0 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc60009420 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.624+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6816a9d0 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.625+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc68088990 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.626+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcc64007300 con 0x7fcc680b16d0 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.626+0000 7fcc5d7fa700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc4c067320 0x7fcc4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.626+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fcc64084c20 con 0x7fcc680b16d0 2023-10-06T03:09:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.626+0000 7fcc6c92a700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc4c067320 0x7fcc4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.626+0000 7fcc6c92a700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc4c067320 0x7fcc4c0697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006d40 tx=0x7fcc58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:51.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.630+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc64051ad0 con 0x7fcc680b16d0 2023-10-06T03:09:51.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.815+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcc680b14b0 con 0x7fcc680b16d0 2023-10-06T03:09:51.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.818+0000 7fcc5d7fa700 1 -- 172.21.15.92:0/4017921554 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcc640550f0 con 0x7fcc680b16d0 2023-10-06T03:09:51.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc4c067320 msgr2=0x7fcc4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc4c067320 0x7fcc4c0697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc58006d40 tx=0x7fcc58008040 comp rx=0 tx=0).stop 2023-10-06T03:09:51.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 msgr2=0x7fcc680c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 0x7fcc680c7cc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400cd60 tx=0x7fcc64001af0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 shutdown_connections 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc4c067320 0x7fcc4c0697d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc680b16d0 0x7fcc680c7cc0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc680b4e70 0x7fcc681574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 --2- 172.21.15.92:0/4017921554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc680b33b0 0x7fcc68169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 >> 172.21.15.92:0/4017921554 conn(0x7fcc680d10c0 msgr2=0x7fcc68159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 shutdown_connections 2023-10-06T03:09:51.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.820+0000 7fcc6e12d700 1 -- 172.21.15.92:0/4017921554 wait complete. 2023-10-06T03:09:51.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-06T03:09:51.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.947+0000 7f8108e1a700 1 Processor -- start 2023-10-06T03:09:51.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.947+0000 7f8108e1a700 1 -- start start 2023-10-06T03:09:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f8108e1a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8104153df0 0x7f8104063230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f8108e1a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f810406e8c0 0x7f81040c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f8108e1a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810415aa30 con 0x7f81040c8660 2023-10-06T03:09:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f8108e1a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81040c8a60 con 0x7f8104153df0 2023-10-06T03:09:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f8108e1a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810415a100 con 0x7f810406e8c0 2023-10-06T03:09:51.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f81037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8104153df0 0x7f8104063230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f81037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8104153df0 0x7f8104063230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57842/0 (socket says 172.21.15.92:57842) 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.948+0000 7f81037fe700 1 -- 172.21.15.92:0/3955193058 learned_addr learned my addr 172.21.15.92:0/3955193058 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81037fe700 1 -- 172.21.15.92:0/3955193058 >> v2:172.21.15.136:3300/0 conn(0x7f810406e8c0 msgr2=0x7f81040c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81037fe700 1 --2- 172.21.15.92:0/3955193058 >> v2:172.21.15.136:3300/0 conn(0x7f810406e8c0 0x7f81040c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81037fe700 1 -- 172.21.15.92:0/3955193058 >> v1:172.21.15.136:6789/0 conn(0x7f81040c8660 legacy=0x7f810415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81037fe700 1 -- 172.21.15.92:0/3955193058 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810415a3e0 con 0x7f8104153df0 2023-10-06T03:09:51.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81037fe700 1 --2- 172.21.15.92:0/3955193058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8104153df0 0x7f8104063230 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f810406d4e0 tx=0x7f80f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea985d07c7bb70e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81027fc700 1 -- 172.21.15.92:0/3955193058 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f400e070 con 0x7f8104153df0 2023-10-06T03:09:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81027fc700 1 -- 172.21.15.92:0/3955193058 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f400ba10 con 0x7f8104153df0 2023-10-06T03:09:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f81027fc700 1 -- 172.21.15.92:0/3955193058 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f40092e0 con 0x7f8104153df0 2023-10-06T03:09:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f8108e1a700 1 -- 172.21.15.92:0/3955193058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8104153df0 msgr2=0x7f8104063230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3955193058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8104153df0 0x7f8104063230 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f810406d4e0 tx=0x7f80f4009840 comp rx=0 tx=0).stop 2023-10-06T03:09:51.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f8108e1a700 1 -- 172.21.15.92:0/3955193058 shutdown_connections 2023-10-06T03:09:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3955193058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8104153df0 0x7f8104063230 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3955193058 >> v2:172.21.15.136:3300/0 conn(0x7f810406e8c0 0x7f81040c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.949+0000 7f8108e1a700 1 -- 172.21.15.92:0/3955193058 >> 172.21.15.92:0/3955193058 conn(0x7f810406ddb0 msgr2=0x7f810406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.950+0000 7f8108e1a700 1 -- 172.21.15.92:0/3955193058 shutdown_connections 2023-10-06T03:09:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.950+0000 7f8108e1a700 1 -- 172.21.15.92:0/3955193058 wait complete. 2023-10-06T03:09:51.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 Processor -- start 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 -- start start 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 0x7f8104145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81040c8660 0x7f81041562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8104153df0 0x7f8104156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8104145c60 con 0x7f810406e8c0 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810414c9d0 con 0x7f81040c8660 2023-10-06T03:09:51.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.951+0000 7f8108e1a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810414d830 con 0x7f8104153df0 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 0x7f8104145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 0x7f8104145720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57856/0 (socket says 172.21.15.92:57856) 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 -- 172.21.15.92:0/3959881653 learned_addr learned my addr 172.21.15.92:0/3959881653 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f8102ffd700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81040c8660 0x7f81041562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f8103fff700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8104153df0 0x7f8104156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 -- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8104153df0 msgr2=0x7f8104156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8104153df0 0x7f8104156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.961 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 -- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81040c8660 msgr2=0x7f81041562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81040c8660 0x7f81041562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 -- 172.21.15.92:0/3959881653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f4009ae0 con 0x7f810406e8c0 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f8103fff700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8104153df0 0x7f8104156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f81037fe700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 0x7f8104145720 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009fd0 tx=0x7f80f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f40128d0 con 0x7f810406e8c0 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f4012a50 con 0x7f810406e8c0 2023-10-06T03:09:51.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810414cc20 con 0x7f810406e8c0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f4012e10 con 0x7f810406e8c0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.952+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810414d1e0 con 0x7f810406e8c0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.953+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810404c3c0 con 0x7f810406e8c0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.953+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f80f401e640 con 0x7f810406e8c0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.954+0000 7f80eb7fe700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80f0067320 0x7f80f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.954+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f80f4086520 con 0x7f810406e8c0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.954+0000 7f8102ffd700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80f0067320 0x7f80f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:51.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.954+0000 7f8102ffd700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80f0067320 0x7f80f00697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f810414e2d0 tx=0x7f80f80091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:51.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:51.958+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80f4053260 con 0x7f810406e8c0 2023-10-06T03:09:52.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.223+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f8104146dc0 con 0x7f810406e8c0 2023-10-06T03:09:52.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.486+0000 7f80eb7fe700 1 -- 172.21.15.92:0/3959881653 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v417) v1 ==== 167+0+0 (secure 0 0 0) 0x7f80f4056880 con 0x7f810406e8c0 2023-10-06T03:09:52.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80f0067320 msgr2=0x7f80f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:52.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80f0067320 0x7f80f00697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f810414e2d0 tx=0x7f80f80091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 msgr2=0x7f8104145720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:52.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 0x7f8104145720 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009fd0 tx=0x7f80f4017000 comp rx=0 tx=0).stop 2023-10-06T03:09:52.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 shutdown_connections 2023-10-06T03:09:52.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80f0067320 0x7f80f00697d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f810406e8c0 0x7f8104145720 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8104153df0 0x7f8104156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 --2- 172.21.15.92:0/3959881653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81040c8660 0x7f81041562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 >> 172.21.15.92:0/3959881653 conn(0x7f810406ddb0 msgr2=0x7f81040cbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:52.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 shutdown_connections 2023-10-06T03:09:52.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.488+0000 7f8108e1a700 1 -- 172.21.15.92:0/3959881653 wait complete. 2023-10-06T03:09:52.501 INFO:tasks.cephfs_test_runner:That renaming volume fails without --yes-i-really-mean-it flag. ... ok 2023-10-06T03:09:52.502 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) That renaming a volume with more than one data pool does not change 2023-10-06T03:09:52.502 INFO:tasks.cephfs_test_runner:test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:09:52.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-06T03:09:52.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.617+0000 7fdadc365700 1 Processor -- start 2023-10-06T03:09:52.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.617+0000 7fdadc365700 1 -- start start 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.617+0000 7fdadc365700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.617+0000 7fdadc365700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdad4159c60 0x7fdad415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdadc365700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad415a160 con 0x7fdad415db10 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdadc365700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40d13a0 con 0x7fdad4159c60 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdadc365700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad415e140 con 0x7fdad4159890 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad9900700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdad4159c60 0x7fdad415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:52.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdada902700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdad415db10 0x7fdad4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60288/0 (socket says 172.21.15.92:60288) 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdada902700 1 -- 172.21.15.92:0/3171838049 learned_addr learned my addr 172.21.15.92:0/3171838049 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdada101700 1 --2- 172.21.15.92:0/3171838049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad9900700 1 -- 172.21.15.92:0/3171838049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 msgr2=0x7fdad415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad9900700 1 --2- 172.21.15.92:0/3171838049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad9900700 1 -- 172.21.15.92:0/3171838049 >> v1:172.21.15.92:6789/0 conn(0x7fdad415db10 legacy=0x7fdad4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad90ff700 1 -- 172.21.15.92:0/3171838049 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482378590 0 0) 0x7fdad415a160 con 0x7fdad415db10 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad9900700 1 -- 172.21.15.92:0/3171838049 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad415a3d0 con 0x7fdad4159c60 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad9900700 1 --2- 172.21.15.92:0/3171838049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad415d5d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fdac40098e0 tx=0x7fdac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b5c02e73e317e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad90ff700 1 -- 172.21.15.92:0/3171838049 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4009e80 con 0x7fdad4159c60 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad90ff700 1 -- 172.21.15.92:0/3171838049 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400b860 con 0x7fdad4159c60 2023-10-06T03:09:52.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.618+0000 7fdad90ff700 1 -- 172.21.15.92:0/3171838049 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400bbe0 con 0x7fdad4159c60 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 -- 172.21.15.92:0/3171838049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 msgr2=0x7fdad415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 --2- 172.21.15.92:0/3171838049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad415d5d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fdac40098e0 tx=0x7fdac40091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 -- 172.21.15.92:0/3171838049 shutdown_connections 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 --2- 172.21.15.92:0/3171838049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 --2- 172.21.15.92:0/3171838049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad415d5d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 -- 172.21.15.92:0/3171838049 >> 172.21.15.92:0/3171838049 conn(0x7fdad4068e20 msgr2=0x7fdad40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 -- 172.21.15.92:0/3171838049 shutdown_connections 2023-10-06T03:09:52.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.619+0000 7fdadc365700 1 -- 172.21.15.92:0/3171838049 wait complete. 2023-10-06T03:09:52.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.620+0000 7fdadc365700 1 Processor -- start 2023-10-06T03:09:52.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.620+0000 7fdadc365700 1 -- start start 2023-10-06T03:09:52.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdadc365700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad4153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:52.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdadc365700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:52.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdadc365700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad415db10 0x7fdad4152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:52.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdadc365700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4169ed0 con 0x7fdad4159c60 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdadc365700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40b4370 con 0x7fdad415db10 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdadc365700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40b4090 con 0x7fdad4159890 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57860/0 (socket says 172.21.15.92:57860) 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 -- 172.21.15.92:0/3598910589 learned_addr learned my addr 172.21.15.92:0/3598910589 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdada101700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad4153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdada902700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad415db10 0x7fdad4152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 -- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 msgr2=0x7fdad4153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad4153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 -- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad415db10 msgr2=0x7fdad4152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:52.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad415db10 0x7fdad4152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 -- 172.21.15.92:0/3598910589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac4009420 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdada101700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad4153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdada902700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad415db10 0x7fdad4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.621+0000 7fdad9900700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad40b3590 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fdac4013000 tx=0x7fdac4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.622+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400bbe0 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.622+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac4011420 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.622+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad416a100 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.622+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40117c0 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.622+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad416a490 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.623+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad416a8e0 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.623+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdac4011bb0 con 0x7fdad4159c60 2023-10-06T03:09:52.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.623+0000 7fdaca7fc700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.623+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fdac4086470 con 0x7fdad4159c60 2023-10-06T03:09:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.624+0000 7fdada101700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.624+0000 7fdada101700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b6630 tx=0x7fdad000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:52.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.628+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdac40532a0 con 0x7fdad4159c60 2023-10-06T03:09:52.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:52.883+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7fdad404c3c0 con 0x7fdad4159c60 2023-10-06T03:09:53.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.487+0000 7fdaca7fc700 1 -- 172.21.15.92:0/3598910589 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v418) v1 ==== 166+0+0 (secure 0 0 0) 0x7fdac4011e60 con 0x7fdad4159c60 2023-10-06T03:09:53.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 msgr2=0x7fdab80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b6630 tx=0x7fdad000b040 comp rx=0 tx=0).stop 2023-10-06T03:09:53.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 msgr2=0x7fdad40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad40b3590 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fdac4013000 tx=0x7fdac4017000 comp rx=0 tx=0).stop 2023-10-06T03:09:53.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 shutdown_connections 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdad415db10 0x7fdad4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad4159890 0x7fdad4153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 --2- 172.21.15.92:0/3598910589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad4159c60 0x7fdad40b3590 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 >> 172.21.15.92:0/3598910589 conn(0x7fdad4068e20 msgr2=0x7fdad40c6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 shutdown_connections 2023-10-06T03:09:53.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.489+0000 7fdadc365700 1 -- 172.21.15.92:0/3598910589 wait complete. 2023-10-06T03:09:53.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:09:53.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.620+0000 7f05aeb94700 1 Processor -- start 2023-10-06T03:09:53.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.620+0000 7f05aeb94700 1 -- start start 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f05aeb94700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f05a8069560 0x7f05a80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f05aeb94700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a8069ee0 con 0x7f05a8159c60 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f05aeb94700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a815a160 con 0x7f05a8159890 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f05aeb94700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80d13a0 con 0x7f05a8069560 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f059ffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f05a8159c60 0x7f05a815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60304/0 (socket says 172.21.15.92:60304) 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f059ffff700 1 -- 172.21.15.92:0/2525642517 learned_addr learned my addr 172.21.15.92:0/2525642517 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f05ad131700 1 --2- 172.21.15.92:0/2525642517 >> v2:172.21.15.136:3300/0 conn(0x7f05a8069560 0x7f05a80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900178349 0 0) 0x7f05a8069ee0 con 0x7f05a8159c60 2023-10-06T03:09:53.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.621+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0590003570 con 0x7f05a8159c60 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728756480 0 0) 0x7f0590003570 con 0x7f05a8159c60 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 >> v2:172.21.15.136:3300/0 conn(0x7f05a8069560 msgr2=0x7f05a80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 --2- 172.21.15.92:0/2525642517 >> v2:172.21.15.136:3300/0 conn(0x7f05a8069560 0x7f05a80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 >> v1:172.21.15.136:6789/0 conn(0x7f05a8159890 legacy=0x7f05a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a815e140 con 0x7f05a8159c60 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f0598001470 con 0x7f05a8159c60 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 >> v1:172.21.15.92:6789/0 conn(0x7f05a8159c60 legacy=0x7f05a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0590003a60 0x7f0590003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0590004810 0x7f0590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05900051c0 0x7f059000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80d13a0 con 0x7f0590003a60 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059000c0d0 con 0x7f0590004810 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059000c290 con 0x7f05900051c0 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0590003a60 0x7f0590003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059ffff700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0590004810 0x7f0590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ad131700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05900051c0 0x7f059000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 -- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05900051c0 msgr2=0x7f059000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05900051c0 0x7f059000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 -- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0590004810 msgr2=0x7f0590004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0590004810 0x7f0590004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 -- 172.21.15.92:0/2525642517 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590003570 con 0x7f0590003a60 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ad131700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05900051c0 0x7f059000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059ffff700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0590004810 0x7f0590004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ad131700 1 -- 172.21.15.92:0/2525642517 reap_dead start 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f05ac930700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0590003a60 0x7f0590003e70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f05a40018b0 tx=0x7f05a4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9580bc2217bbfa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4012420 con 0x7f0590003a60 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a40125a0 con 0x7f0590003a60 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.622+0000 7f059f7fe700 1 -- 172.21.15.92:0/2525642517 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4012960 con 0x7f0590003a60 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 -- 172.21.15.92:0/2525642517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0590003a60 msgr2=0x7f0590003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0590003a60 0x7f0590003e70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f05a40018b0 tx=0x7f05a4007d30 comp rx=0 tx=0).stop 2023-10-06T03:09:53.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 -- 172.21.15.92:0/2525642517 shutdown_connections 2023-10-06T03:09:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2525642517 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0590003a60 0x7f0590003e70 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 -- 172.21.15.92:0/2525642517 >> 172.21.15.92:0/2525642517 conn(0x7f05a8068e20 msgr2=0x7f05a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 -- 172.21.15.92:0/2525642517 shutdown_connections 2023-10-06T03:09:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.623+0000 7f05aeb94700 1 -- 172.21.15.92:0/2525642517 wait complete. 2023-10-06T03:09:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.624+0000 7f05aeb94700 1 Processor -- start 2023-10-06T03:09:53.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.624+0000 7f05aeb94700 1 -- start start 2023-10-06T03:09:53.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05aeb94700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0590003a60 0x7f05a8149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05aeb94700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 0x7f05a81537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05aeb94700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a81485d0 0x7f05a8153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05aeb94700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a814a6f0 con 0x7f05a8148160 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05aeb94700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8149ae0 con 0x7f05a81485d0 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05aeb94700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8149dc0 con 0x7f0590003a60 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 0x7f05a81537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 0x7f05a81537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57886/0 (socket says 172.21.15.92:57886) 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 -- 172.21.15.92:0/2686945420 learned_addr learned my addr 172.21.15.92:0/2686945420 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05ad131700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a81485d0 0x7f05a8153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05ac930700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0590003a60 0x7f05a8149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 -- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0590003a60 msgr2=0x7f05a8149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0590003a60 0x7f05a8149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 -- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a81485d0 msgr2=0x7f05a8153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a81485d0 0x7f05a8153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f059ffff700 1 -- 172.21.15.92:0/2686945420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a814a0a0 con 0x7f05a8148160 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05ad131700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a81485d0 0x7f05a8153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.625+0000 7f05ac930700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0590003a60 0x7f05a8149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.626+0000 7f059ffff700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 0x7f05a81537c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f059800eb90 tx=0x7f05980014c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.626+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800c7e0 con 0x7f05a8148160 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.626+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800c940 con 0x7f05a8148160 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.626+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a814a380 con 0x7f05a8148160 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.626+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800ccc0 con 0x7f05a8148160 2023-10-06T03:09:53.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.626+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8149870 con 0x7f05a8148160 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.627+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a814aa80 con 0x7f05a8148160 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.627+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0598013630 con 0x7f05a8148160 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.627+0000 7f059cff9700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0594067320 0x7f05940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.628+0000 7f05ac930700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0594067320 0x7f05940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.628+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0598084a80 con 0x7f05a8148160 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.628+0000 7f05ac930700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0594067320 0x7f05940697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f05a4001010 tx=0x7f05a4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:53.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.632+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05a814aa80 con 0x7f05a8148160 2023-10-06T03:09:53.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.813+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f05a815bbe0 con 0x7f05a8148160 2023-10-06T03:09:53.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.816+0000 7f059cff9700 1 -- 172.21.15.92:0/2686945420 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f05980518b0 con 0x7f05a8148160 2023-10-06T03:09:53.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0594067320 msgr2=0x7f05940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0594067320 0x7f05940697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f05a4001010 tx=0x7f05a4015000 comp rx=0 tx=0).stop 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 msgr2=0x7f05a81537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 0x7f05a81537c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f059800eb90 tx=0x7f05980014c0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 shutdown_connections 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0594067320 0x7f05940697d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a81485d0 0x7f05a8153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0590003a60 0x7f05a8149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 --2- 172.21.15.92:0/2686945420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05a8148160 0x7f05a81537c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 >> 172.21.15.92:0/2686945420 conn(0x7f05a8068e20 msgr2=0x7f05a815b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:53.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 shutdown_connections 2023-10-06T03:09:53.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:53.819+0000 7f05aeb94700 1 -- 172.21.15.92:0/2686945420 wait complete. 2023-10-06T03:09:53.832 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:09:53.832 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:53.872 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:09:53.924 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:09:53.924 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:09:53.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:09:54.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.038+0000 7fac29d3d700 1 Processor -- start 2023-10-06T03:09:54.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.038+0000 7fac29d3d700 1 -- start start 2023-10-06T03:09:54.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac29d3d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac24159890 0x7fac2415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac29d3d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac29d3d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2415a160 con 0x7fac2415db10 2023-10-06T03:09:54.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac29d3d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2415df50 con 0x7fac24159890 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac29d3d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240d13a0 con 0x7fac24159c60 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac237fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fac24159890 0x7fac2415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac23fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fac2415db10 0x7fac24069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60308/0 (socket says 172.21.15.92:60308) 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.039+0000 7fac23fff700 1 -- 172.21.15.92:0/4240804659 learned_addr learned my addr 172.21.15.92:0/4240804659 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac22ffd700 1 --2- 172.21.15.92:0/4240804659 >> v2:172.21.15.136:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac237fe700 1 -- 172.21.15.92:0/4240804659 >> v2:172.21.15.136:3300/0 conn(0x7fac24159c60 msgr2=0x7fac2415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac237fe700 1 --2- 172.21.15.92:0/4240804659 >> v2:172.21.15.136:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac237fe700 1 -- 172.21.15.92:0/4240804659 >> v1:172.21.15.92:6789/0 conn(0x7fac2415db10 legacy=0x7fac24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac237fe700 1 -- 172.21.15.92:0/4240804659 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2415e140 con 0x7fac24159890 2023-10-06T03:09:54.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac237fe700 1 --2- 172.21.15.92:0/4240804659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159890 0x7fac2415abd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fac240b67f0 tx=0x7fac14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928e4fabafea2bf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac227fc700 1 -- 172.21.15.92:0/4240804659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400e070 con 0x7fac24159890 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac227fc700 1 -- 172.21.15.92:0/4240804659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1400bc30 con 0x7fac24159890 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac227fc700 1 -- 172.21.15.92:0/4240804659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14012430 con 0x7fac24159890 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 -- 172.21.15.92:0/4240804659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159890 msgr2=0x7fac2415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 --2- 172.21.15.92:0/4240804659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159890 0x7fac2415abd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fac240b67f0 tx=0x7fac14009780 comp rx=0 tx=0).stop 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 -- 172.21.15.92:0/4240804659 shutdown_connections 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 --2- 172.21.15.92:0/4240804659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159890 0x7fac2415abd0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 --2- 172.21.15.92:0/4240804659 >> v2:172.21.15.136:3300/0 conn(0x7fac24159c60 0x7fac2415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 -- 172.21.15.92:0/4240804659 >> 172.21.15.92:0/4240804659 conn(0x7fac24068e20 msgr2=0x7fac240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:54.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 -- 172.21.15.92:0/4240804659 shutdown_connections 2023-10-06T03:09:54.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.040+0000 7fac29d3d700 1 -- 172.21.15.92:0/4240804659 wait complete. 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 Processor -- start 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 -- start start 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac240c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac24158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac240c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240b41c0 con 0x7fac24159c60 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240b4350 con 0x7fac24159890 2023-10-06T03:09:54.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac29d3d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac24158bb0 con 0x7fac2415db10 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.042+0000 7fac22ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac24158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac24158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57896/0 (socket says 172.21.15.92:57896) 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 -- 172.21.15.92:0/2548804296 learned_addr learned my addr 172.21.15.92:0/2548804296 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac237fe700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac240c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac23fff700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac240c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 -- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 msgr2=0x7fac240c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac240c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 -- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 msgr2=0x7fac240c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac240c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 -- 172.21.15.92:0/2548804296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009a20 con 0x7fac24159c60 2023-10-06T03:09:54.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac22ffd700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac24158670 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fac1800d8c0 tx=0x7fac18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180125f0 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18012750 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18012af0 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac240c8890 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.043+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac240c8e50 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.044+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2404c3c0 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.045+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fac1801e460 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.045+0000 7fac0b7fe700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac10067320 0x7fac100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.045+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fac18085050 con 0x7fac24159c60 2023-10-06T03:09:54.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.045+0000 7fac237fe700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac10067320 0x7fac100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.045+0000 7fac237fe700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac10067320 0x7fac100697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fac240b67f0 tx=0x7fac14019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:54.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.049+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac18051e80 con 0x7fac24159c60 2023-10-06T03:09:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.275+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac240b4490 con 0x7fac24159c60 2023-10-06T03:09:54.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.276+0000 7fac0b7fe700 1 -- 172.21.15.92:0/2548804296 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3534 (secure 0 0 0) 0x7fac180554a0 con 0x7fac24159c60 2023-10-06T03:09:54.277 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:54.277 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/628402547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":628402547},{"type":"v1","addr":"172.21.15.136:6834","nonce":628402547}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7225,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3905715312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3905715312},{"type":"v1","addr":"172.21.15.136:6835","nonce":3905715312}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":7989,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3296306280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3296306280},{"type":"v1","addr":"172.21.15.92:6835","nonce":3296306280}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":8286,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1272912746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":1272912746},{"type":"v1","addr":"172.21.15.92:6837","nonce":1272912746}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[]} 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac10067320 msgr2=0x7fac100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac10067320 0x7fac100697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fac240b67f0 tx=0x7fac14019000 comp rx=0 tx=0).stop 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 msgr2=0x7fac24158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac24158670 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fac1800d8c0 tx=0x7fac18008040 comp rx=0 tx=0).stop 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 shutdown_connections 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fac10067320 0x7fac100697d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac2415db10 0x7fac240c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac24159890 0x7fac240c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 --2- 172.21.15.92:0/2548804296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fac24159c60 0x7fac24158670 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 >> 172.21.15.92:0/2548804296 conn(0x7fac24068e20 msgr2=0x7fac240c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:54.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 shutdown_connections 2023-10-06T03:09:54.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.278+0000 7fac29d3d700 1 -- 172.21.15.92:0/2548804296 wait complete. 2023-10-06T03:09:54.280 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 107 2023-10-06T03:09:54.290 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:09:54.291 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:09:54.291 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:09:54.291 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:54.291 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:09:54.291 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:09:54.291 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:09:54.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:54.292 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:09:54.292 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:09:54.292 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:09:54.292 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:54.292 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:09:54.292 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:09:54.293 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:09:54.293 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:09:54.299 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:09:54.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:09:54.300 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:09:54.301 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:09:54.303 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:09:54.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:09:54.304 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:09:54.304 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:09:54.342 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:09:54.342 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:54.343 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:09:54.343 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:54.345 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:09:54.346 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:54.346 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:09:54.347 DEBUG:teuthology.parallel:result is None 2023-10-06T03:09:54.347 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:09:54.416 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:09:54.417 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:09:54.430 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:09:54.430 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:09:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.468+0000 7f94ce4a7700 1 Processor -- start 2023-10-06T03:09:54.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.468+0000 7f94ce4a7700 1 -- start start 2023-10-06T03:09:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94ce4a7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94c806bcd0 0x7f94c80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94ce4a7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f94c815df50 0x7f94c815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94ce4a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c815d7e0 con 0x7f94c806c0a0 2023-10-06T03:09:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94ce4a7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c815dac0 con 0x7f94c806bcd0 2023-10-06T03:09:54.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94ce4a7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c815aa30 con 0x7f94c815df50 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94c7fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f94c806bcd0 0x7f94c80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94c77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f94c806c0a0 0x7f94c815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60332/0 (socket says 172.21.15.92:60332) 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.469+0000 7f94c77fe700 1 -- 172.21.15.92:0/2751726814 learned_addr learned my addr 172.21.15.92:0/2751726814 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94cca44700 1 --2- 172.21.15.92:0/2751726814 >> v2:172.21.15.136:3300/0 conn(0x7f94c815df50 0x7f94c815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c7fff700 1 -- 172.21.15.92:0/2751726814 >> v2:172.21.15.136:3300/0 conn(0x7f94c815df50 msgr2=0x7f94c815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c7fff700 1 --2- 172.21.15.92:0/2751726814 >> v2:172.21.15.136:3300/0 conn(0x7f94c815df50 0x7f94c815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c7fff700 1 -- 172.21.15.92:0/2751726814 >> v1:172.21.15.92:6789/0 conn(0x7f94c806c0a0 legacy=0x7f94c815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c6ffd700 1 -- 172.21.15.92:0/2751726814 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764275304 0 0) 0x7f94c815d7e0 con 0x7f94c806c0a0 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c7fff700 1 -- 172.21.15.92:0/2751726814 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c815ad10 con 0x7f94c806bcd0 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c7fff700 1 --2- 172.21.15.92:0/2751726814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80caa90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f94c815d720 tx=0x7f94b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a5bb3810b931d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:54.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c6ffd700 1 -- 172.21.15.92:0/2751726814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800b6b0 con 0x7f94c806bcd0 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c6ffd700 1 -- 172.21.15.92:0/2751726814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800b810 con 0x7f94c806bcd0 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94c6ffd700 1 -- 172.21.15.92:0/2751726814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800bbb0 con 0x7f94c806bcd0 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94ce4a7700 1 -- 172.21.15.92:0/2751726814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 msgr2=0x7f94c80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.470+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/2751726814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80caa90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f94c815d720 tx=0x7f94b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.471+0000 7f94ce4a7700 1 -- 172.21.15.92:0/2751726814 shutdown_connections 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.471+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/2751726814 >> v2:172.21.15.136:3300/0 conn(0x7f94c815df50 0x7f94c815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.471+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/2751726814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80caa90 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.471+0000 7f94ce4a7700 1 -- 172.21.15.92:0/2751726814 >> 172.21.15.92:0/2751726814 conn(0x7f94c80ce600 msgr2=0x7f94c80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.471+0000 7f94ce4a7700 1 -- 172.21.15.92:0/2751726814 shutdown_connections 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.471+0000 7f94ce4a7700 1 -- 172.21.15.92:0/2751726814 wait complete. 2023-10-06T03:09:54.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.472+0000 7f94ce4a7700 1 Processor -- start 2023-10-06T03:09:54.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.472+0000 7f94ce4a7700 1 -- start start 2023-10-06T03:09:54.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94c806c0a0 0x7f94c80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c815df50 0x7f94c80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c816b0e0 con 0x7f94c806bcd0 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c816b3c0 con 0x7f94c815df50 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c816b6a0 con 0x7f94c806c0a0 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57946/0 (socket says 172.21.15.92:57946) 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 -- 172.21.15.92:0/3475415185 learned_addr learned my addr 172.21.15.92:0/3475415185 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c77fe700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94c806c0a0 0x7f94c80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94cca44700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c815df50 0x7f94c80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 -- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94c806c0a0 msgr2=0x7f94c80b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94c806c0a0 0x7f94c80b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 -- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c815df50 msgr2=0x7f94c80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:54.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c815df50 0x7f94c80b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 -- 172.21.15.92:0/3475415185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009420 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94c7fff700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80c5bc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f94c8069220 tx=0x7f94b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8006af0 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800b980 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c816b980 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8026430 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.473+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c80b06e0 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.474+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c804c3c0 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.474+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f94b8023030 con 0x7f94c806bcd0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.475+0000 7f94affff700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f94b4067320 0x7f94b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.475+0000 7f94c77fe700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f94b4067320 0x7f94b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:54.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.475+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f94b80861c0 con 0x7f94c806bcd0 2023-10-06T03:09:54.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.475+0000 7f94c77fe700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f94b4067320 0x7f94b40697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c8f0 tx=0x7f94bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:54.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.479+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94b8053070 con 0x7f94c806bcd0 2023-10-06T03:09:54.681 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:54.681+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f94c815aff0 con 0x7f94c806bcd0 2023-10-06T03:09:55.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.516+0000 7f94affff700 1 -- 172.21.15.92:0/3475415185 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v163) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94b8056690 con 0x7f94c806bcd0 2023-10-06T03:09:55.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f94b4067320 msgr2=0x7f94b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f94b4067320 0x7f94b40697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c8f0 tx=0x7f94bc009040 comp rx=0 tx=0).stop 2023-10-06T03:09:55.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 msgr2=0x7f94c80c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80c5bc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f94c8069220 tx=0x7f94b8017000 comp rx=0 tx=0).stop 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 shutdown_connections 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f94b4067320 0x7f94b40697d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94c806bcd0 0x7f94c80c5bc0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94c815df50 0x7f94c80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 --2- 172.21.15.92:0/3475415185 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94c806c0a0 0x7f94c80b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 >> 172.21.15.92:0/3475415185 conn(0x7f94c80ce600 msgr2=0x7f94c80c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 shutdown_connections 2023-10-06T03:09:55.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.519+0000 7f94ce4a7700 1 -- 172.21.15.92:0/3475415185 wait complete. 2023-10-06T03:09:55.522 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:09:55.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:09:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.647+0000 7f793ca24700 1 Processor -- start 2023-10-06T03:09:55.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.647+0000 7f793ca24700 1 -- start start 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f793ca24700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f79381574d0 0x7f79380cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f793ca24700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 0x7f793806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f793ca24700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793806c380 con 0x7f793806bf20 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f793ca24700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380b16a0 con 0x7f79381574d0 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f793ca24700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79380b1c60 con 0x7f79380cc3c0 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f7937fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f793806bf20 0x7f793806cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60340/0 (socket says 172.21.15.92:60340) 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f7937fff700 1 -- 172.21.15.92:0/1834203718 learned_addr learned my addr 172.21.15.92:0/1834203718 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8662024 0 0) 0x7f7930001110 con 0x7f793806bf20 2023-10-06T03:09:55.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.648+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7920003570 con 0x7f793806bf20 2023-10-06T03:09:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1273319830 0 0) 0x7f7920003570 con 0x7f793806bf20 2023-10-06T03:09:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 msgr2=0x7f793806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 0x7f793806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 >> v2:172.21.15.136:3300/0 conn(0x7f79381574d0 msgr2=0x7f79380cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 --2- 172.21.15.92:0/1834203718 >> v2:172.21.15.136:3300/0 conn(0x7f79381574d0 0x7f79380cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79381578a0 con 0x7f793806bf20 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7930001600 con 0x7f793806bf20 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 >> v1:172.21.15.92:6789/0 conn(0x7f793806bf20 legacy=0x7f793806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 0x7f7920003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7920004170 0x7f7920004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7920004e70 0x7f79200052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380b16a0 con 0x7f7920004e70 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380b1c60 con 0x7f7920004170 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792000bda0 con 0x7f79380cc3c0 2023-10-06T03:09:55.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7920004e70 0x7f79200052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79377fe700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 0x7f7920003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7936ffd700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7920004170 0x7f7920004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 -- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 msgr2=0x7f7920003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 0x7f7920003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 -- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7920004170 msgr2=0x7f7920004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7920004170 0x7f7920004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 -- 172.21.15.92:0/1834203718 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7920003570 con 0x7f7920004e70 2023-10-06T03:09:55.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7936ffd700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7920004170 0x7f7920004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79377fe700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79380cc3c0 0x7f7920003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f7937fff700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7920004e70 0x7f79200052e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f793000c5f0 tx=0x7f79380b1bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddb433714c89de3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000cdf0 con 0x7f7920004e70 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930005920 con 0x7f7920004e70 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.649+0000 7f79367fc700 1 -- 172.21.15.92:0/1834203718 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930005ce0 con 0x7f7920004e70 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f793ca24700 1 -- 172.21.15.92:0/1834203718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7920004e70 msgr2=0x7f79200052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f793ca24700 1 --2- 172.21.15.92:0/1834203718 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7920004e70 0x7f79200052e0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f793000c5f0 tx=0x7f79380b1bd0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f7937fff700 1 -- 172.21.15.92:0/1834203718 reap_dead start 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f793ca24700 1 -- 172.21.15.92:0/1834203718 shutdown_connections 2023-10-06T03:09:55.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f793ca24700 1 -- 172.21.15.92:0/1834203718 >> 172.21.15.92:0/1834203718 conn(0x7f79380ce430 msgr2=0x7f79380ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f793ca24700 1 -- 172.21.15.92:0/1834203718 shutdown_connections 2023-10-06T03:09:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.650+0000 7f793ca24700 1 -- 172.21.15.92:0/1834203718 wait complete. 2023-10-06T03:09:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.651+0000 7f793ca24700 1 Processor -- start 2023-10-06T03:09:55.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.651+0000 7f793ca24700 1 -- start start 2023-10-06T03:09:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f793ca24700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7938137cc0 0x7f79380af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f793ca24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 0x7f79380af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f793ca24700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f793815b460 0x7f793815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f793ca24700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380afe80 con 0x7f7938138090 2023-10-06T03:09:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f793ca24700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793815a3c0 con 0x7f7938137cc0 2023-10-06T03:09:55.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f793ca24700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793815a6a0 con 0x7f793815b460 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 0x7f79380af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 0x7f79380af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57976/0 (socket says 172.21.15.92:57976) 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 -- 172.21.15.92:0/2097314213 learned_addr learned my addr 172.21.15.92:0/2097314213 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 -- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f793815b460 msgr2=0x7f793815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f793815b460 0x7f793815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 -- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7938137cc0 msgr2=0x7f79380af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f79377fe700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7938137cc0 0x7f79380af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:55.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7938137cc0 0x7f79380af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 -- 172.21.15.92:0/2097314213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793815a980 con 0x7f7938138090 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f7936ffd700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 0x7f79380af940 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f792c00d800 tx=0x7f792c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.652+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c011500 con 0x7f7938138090 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.653+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c00f040 con 0x7f7938138090 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.653+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793815ac60 con 0x7f7938138090 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.653+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0118b0 con 0x7f7938138090 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.653+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793815be10 con 0x7f7938138090 2023-10-06T03:09:55.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.654+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793815b180 con 0x7f7938138090 2023-10-06T03:09:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.654+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f792c011ce0 con 0x7f7938138090 2023-10-06T03:09:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.654+0000 7f791f7fe700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7924067350 0x7f7924069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.655+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f792c084470 con 0x7f7938138090 2023-10-06T03:09:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.655+0000 7f79377fe700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7924067350 0x7f7924069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.655+0000 7f79377fe700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7924067350 0x7f7924069800 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f792800e4a0 tx=0x7f79280095c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:55.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.659+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f793815b180 con 0x7f7938138090 2023-10-06T03:09:55.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.914+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f793815b180 con 0x7f7938138090 2023-10-06T03:09:55.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.914+0000 7f791f7fe700 1 -- 172.21.15.92:0/2097314213 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f793815b180 con 0x7f7938138090 2023-10-06T03:09:55.915 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:55.915 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:09:55.915 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:09:55.915 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.916 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.917 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:09:55.918 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:09:55.919 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.920 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.921 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:09:55.922 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.923 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.924 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.925 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:09:55.926 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.927 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:09:55.929 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:09:55.930 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:09:55.931 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.932 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:09:55.933 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:09:55.934 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:09:55.934 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:09:55.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.934 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:55.934 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:09:55.934 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:09:55.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7924067350 msgr2=0x7f7924069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7924067350 0x7f7924069800 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f792800e4a0 tx=0x7f79280095c0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 msgr2=0x7f79380af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:55.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 0x7f79380af940 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f792c00d800 tx=0x7f792c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:55.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 shutdown_connections 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7924067350 0x7f7924069800 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f793815b460 0x7f793815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7938137cc0 0x7f79380af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 --2- 172.21.15.92:0/2097314213 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7938138090 0x7f79380af940 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.916+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 >> 172.21.15.92:0/2097314213 conn(0x7f79380ce430 msgr2=0x7f7938159600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.917+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 shutdown_connections 2023-10-06T03:09:55.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:55.917+0000 7f793ca24700 1 -- 172.21.15.92:0/2097314213 wait complete. 2023-10-06T03:09:55.938 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:09:56.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.051+0000 7f675cab2700 1 Processor -- start 2023-10-06T03:09:56.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.051+0000 7f675cab2700 1 -- start start 2023-10-06T03:09:56.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f675cab2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67581597f0 0x7f67580b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f675cab2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f675815d230 0x7f675815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f675cab2700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675815db70 con 0x7f6758159420 2023-10-06T03:09:56.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f675cab2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b11d0 con 0x7f675815d230 2023-10-06T03:09:56.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f675cab2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b14b0 con 0x7f67581597f0 2023-10-06T03:09:56.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f675815d230 0x7f675815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f675815d230 0x7f675815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57988/0 (socket says 172.21.15.92:57988) 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 -- 172.21.15.92:0/3997197240 learned_addr learned my addr 172.21.15.92:0/3997197240 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 -- 172.21.15.92:0/3997197240 >> v2:172.21.15.136:3300/0 conn(0x7f67581597f0 msgr2=0x7f67580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 --2- 172.21.15.92:0/3997197240 >> v2:172.21.15.136:3300/0 conn(0x7f67581597f0 0x7f67580b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 -- 172.21.15.92:0/3997197240 >> v1:172.21.15.136:6789/0 conn(0x7f6758159420 legacy=0x7f67580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 -- 172.21.15.92:0/3997197240 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580b1790 con 0x7f675815d230 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.052+0000 7f6757fff700 1 --2- 172.21.15.92:0/3997197240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f675815d630 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6758089750 tx=0x7f6750009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbfa7b9783cc303 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f67567fc700 1 -- 172.21.15.92:0/3997197240 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000b820 con 0x7f675815d230 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f67567fc700 1 -- 172.21.15.92:0/3997197240 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675000b980 con 0x7f675815d230 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f67567fc700 1 -- 172.21.15.92:0/3997197240 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675000bd20 con 0x7f675815d230 2023-10-06T03:09:56.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 -- 172.21.15.92:0/3997197240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 msgr2=0x7f675815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 --2- 172.21.15.92:0/3997197240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f675815d630 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6758089750 tx=0x7f6750009250 comp rx=0 tx=0).stop 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 -- 172.21.15.92:0/3997197240 shutdown_connections 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 --2- 172.21.15.92:0/3997197240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f675815d630 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 --2- 172.21.15.92:0/3997197240 >> v2:172.21.15.136:3300/0 conn(0x7f67581597f0 0x7f67580b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 -- 172.21.15.92:0/3997197240 >> 172.21.15.92:0/3997197240 conn(0x7f67580d0660 msgr2=0x7f67580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 -- 172.21.15.92:0/3997197240 shutdown_connections 2023-10-06T03:09:56.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.053+0000 7f675cab2700 1 -- 172.21.15.92:0/3997197240 wait complete. 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.054+0000 7f675cab2700 1 Processor -- start 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.054+0000 7f675cab2700 1 -- start start 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f675cab2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6758159420 0x7f67580cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f675cab2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67581597f0 0x7f67580c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f675cab2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f67580c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f675cab2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67580b5f90 con 0x7f675815d230 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f675cab2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675815aba0 con 0x7f67581597f0 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f675cab2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675815ae80 con 0x7f6758159420 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f67580c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f67580c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57992/0 (socket says 172.21.15.92:57992) 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 -- 172.21.15.92:0/2304717527 learned_addr learned my addr 172.21.15.92:0/2304717527 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6756ffd700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67581597f0 0x7f67580c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f67577fe700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6758159420 0x7f67580cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 -- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6758159420 msgr2=0x7f67580cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6758159420 0x7f67580cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 -- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67581597f0 msgr2=0x7f67580c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67581597f0 0x7f67580c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.055+0000 7f6757fff700 1 -- 172.21.15.92:0/2304717527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500094f0 con 0x7f675815d230 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.056+0000 7f6757fff700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f67580c5940 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6750009290 tx=0x7f6750016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.056+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675001a650 con 0x7f675815d230 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.056+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675001f070 con 0x7f675815d230 2023-10-06T03:09:56.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.056+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675815b160 con 0x7f675815d230 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.056+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675001aa20 con 0x7f675815d230 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.056+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67580cc390 con 0x7f675815d230 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.057+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675815b680 con 0x7f675815d230 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.057+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f675001b440 con 0x7f675815d230 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.057+0000 7f673f7fe700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067320 0x7f67440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.058+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6750021070 con 0x7f675815d230 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.058+0000 7f67577fe700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067320 0x7f67440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.061 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.058+0000 7f67577fe700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067320 0x7f67440697d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6748009fd0 tx=0x7f6748009620 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.062 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.062+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f675815b680 con 0x7f675815d230 2023-10-06T03:09:56.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.246+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f675815b680 con 0x7f675815d230 2023-10-06T03:09:56.247 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.247+0000 7f673f7fe700 1 -- 172.21.15.92:0/2304717527 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f675815b680 con 0x7f675815d230 2023-10-06T03:09:56.248 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:56.248 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":163,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:55.509288+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:09:56.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067320 msgr2=0x7f67440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.250 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067320 0x7f67440697d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6748009fd0 tx=0x7f6748009620 comp rx=0 tx=0).stop 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 msgr2=0x7f67580c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f67580c5940 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6750009290 tx=0x7f6750016000 comp rx=0 tx=0).stop 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 shutdown_connections 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6744067320 0x7f67440697d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f675815d230 0x7f67580c5940 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6758159420 0x7f67580cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 --2- 172.21.15.92:0/2304717527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67581597f0 0x7f67580c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 >> 172.21.15.92:0/2304717527 conn(0x7f67580d0660 msgr2=0x7f67580695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 shutdown_connections 2023-10-06T03:09:56.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.249+0000 7f675cab2700 1 -- 172.21.15.92:0/2304717527 wait complete. 2023-10-06T03:09:56.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:09:56.263 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:09:56.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.379+0000 7f7d9c62e700 1 Processor -- start 2023-10-06T03:09:56.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.379+0000 7f7d9c62e700 1 -- start start 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.380+0000 7f7d9c62e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7d94155eb0 0x7f7d9406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.380+0000 7f7d9c62e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d940c70b0 0x7f7d94069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.380+0000 7f7d9c62e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d94069cb0 0x7f7d940ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.380+0000 7f7d9c62e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9406a130 con 0x7f7d940c70b0 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.380+0000 7f7d9c62e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9406eb90 con 0x7f7d94155eb0 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.380+0000 7f7d9c62e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9406ee70 con 0x7f7d94069cb0 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d940c70b0 0x7f7d94069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7d940c70b0 0x7f7d94069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57996/0 (socket says 172.21.15.92:57996) 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 -- 172.21.15.92:0/2601024045 learned_addr learned my addr 172.21.15.92:0/2601024045 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 -- 172.21.15.92:0/2601024045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d94069cb0 msgr2=0x7f7d940ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 --2- 172.21.15.92:0/2601024045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d94069cb0 0x7f7d940ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 -- 172.21.15.92:0/2601024045 >> v2:172.21.15.136:3300/0 conn(0x7f7d94155eb0 msgr2=0x7f7d9406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 --2- 172.21.15.92:0/2601024045 >> v2:172.21.15.136:3300/0 conn(0x7f7d94155eb0 0x7f7d9406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 -- 172.21.15.92:0/2601024045 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9406f150 con 0x7f7d940c70b0 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d99bc9700 1 --2- 172.21.15.92:0/2601024045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d940c70b0 0x7f7d94069770 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7d94065160 tx=0x7f7d84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb2b1d45caae9db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d993c8700 1 -- 172.21.15.92:0/2601024045 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400b860 con 0x7f7d940c70b0 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d993c8700 1 -- 172.21.15.92:0/2601024045 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8400b9c0 con 0x7f7d940c70b0 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.381+0000 7f7d993c8700 1 -- 172.21.15.92:0/2601024045 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400bd60 con 0x7f7d940c70b0 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2601024045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d940c70b0 msgr2=0x7f7d94069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2601024045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d940c70b0 0x7f7d94069770 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7d94065160 tx=0x7f7d84009250 comp rx=0 tx=0).stop 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2601024045 shutdown_connections 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2601024045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d94069cb0 0x7f7d940ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2601024045 >> v2:172.21.15.136:3300/0 conn(0x7f7d94155eb0 0x7f7d9406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2601024045 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d940c70b0 0x7f7d94069770 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2601024045 >> 172.21.15.92:0/2601024045 conn(0x7f7d94068e10 msgr2=0x7f7d940cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:56.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2601024045 shutdown_connections 2023-10-06T03:09:56.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.382+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2601024045 wait complete. 2023-10-06T03:09:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.383+0000 7f7d9c62e700 1 Processor -- start 2023-10-06T03:09:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.383+0000 7f7d9c62e700 1 -- start start 2023-10-06T03:09:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 0x7f7d940c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d940c70b0 0x7f7d94145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94155eb0 0x7f7d940ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d94145590 con 0x7f7d94069cb0 2023-10-06T03:09:56.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940c3680 con 0x7f7d94155eb0 2023-10-06T03:09:56.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940c3960 con 0x7f7d940c70b0 2023-10-06T03:09:56.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 0x7f7d940c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 0x7f7d940c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58012/0 (socket says 172.21.15.92:58012) 2023-10-06T03:09:56.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 -- 172.21.15.92:0/2775488700 learned_addr learned my addr 172.21.15.92:0/2775488700 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:56.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9abcb700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94155eb0 0x7f7d940ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d99bc9700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d940c70b0 0x7f7d94145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 -- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d940c70b0 msgr2=0x7f7d94145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d940c70b0 0x7f7d94145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 -- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94155eb0 msgr2=0x7f7d940ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94155eb0 0x7f7d940ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 -- 172.21.15.92:0/2775488700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d840094f0 con 0x7f7d94069cb0 2023-10-06T03:09:56.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9a3ca700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 0x7f7d940c4720 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000bc30 tx=0x7f7d90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d900092e0 con 0x7f7d94069cb0 2023-10-06T03:09:56.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9000c7a0 con 0x7f7d94069cb0 2023-10-06T03:09:56.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d940c3c40 con 0x7f7d94069cb0 2023-10-06T03:09:56.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000cb80 con 0x7f7d94069cb0 2023-10-06T03:09:56.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.384+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d940c4200 con 0x7f7d94069cb0 2023-10-06T03:09:56.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.385+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d940c4c60 con 0x7f7d94069cb0 2023-10-06T03:09:56.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.386+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7d90012070 con 0x7f7d94069cb0 2023-10-06T03:09:56.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.386+0000 7f7d8a7fc700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067320 0x7f7d780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.386+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7d900846e0 con 0x7f7d94069cb0 2023-10-06T03:09:56.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.386+0000 7f7d99bc9700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067320 0x7f7d780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.386+0000 7f7d99bc9700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067320 0x7f7d780697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7d94065160 tx=0x7f7d84019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.390+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d90051560 con 0x7f7d94069cb0 2023-10-06T03:09:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.576+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d940c43f0 con 0x7f7d94069cb0 2023-10-06T03:09:56.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.576+0000 7f7d8a7fc700 1 -- 172.21.15.92:0/2775488700 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d90054b80 con 0x7f7d94069cb0 2023-10-06T03:09:56.577 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:09:56.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.578+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067320 msgr2=0x7f7d780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.580 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.578+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067320 0x7f7d780697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7d94065160 tx=0x7f7d84019000 comp rx=0 tx=0).stop 2023-10-06T03:09:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 msgr2=0x7f7d940c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 0x7f7d940c4720 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000bc30 tx=0x7f7d90009840 comp rx=0 tx=0).stop 2023-10-06T03:09:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 shutdown_connections 2023-10-06T03:09:56.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7d78067320 0x7f7d780697d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7d94069cb0 0x7f7d940c4720 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d94155eb0 0x7f7d940ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 --2- 172.21.15.92:0/2775488700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d940c70b0 0x7f7d94145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 >> 172.21.15.92:0/2775488700 conn(0x7f7d94068e10 msgr2=0x7f7d94154ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 shutdown_connections 2023-10-06T03:09:56.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.579+0000 7f7d9c62e700 1 -- 172.21.15.92:0/2775488700 wait complete. 2023-10-06T03:09:56.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:09:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.708+0000 7fa25e6fa700 1 Processor -- start 2023-10-06T03:09:56.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.708+0000 7fa25e6fa700 1 -- start start 2023-10-06T03:09:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa25e6fa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa258159890 0x7fa25815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa25e6fa700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa258069560 0x7fa2580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa25e6fa700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa258069ee0 con 0x7fa258159c60 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa25e6fa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25815a160 con 0x7fa258159890 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa25e6fa700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580d13a0 con 0x7fa258069560 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa258159890 0x7fa25815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa258159890 0x7fa25815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58022/0 (socket says 172.21.15.92:58022) 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 -- 172.21.15.92:0/1821153928 learned_addr learned my addr 172.21.15.92:0/1821153928 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa25cc97700 1 --2- 172.21.15.92:0/1821153928 >> v2:172.21.15.136:3300/0 conn(0x7fa258069560 0x7fa2580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 -- 172.21.15.92:0/1821153928 >> v2:172.21.15.136:3300/0 conn(0x7fa258069560 msgr2=0x7fa2580699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:56.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 --2- 172.21.15.92:0/1821153928 >> v2:172.21.15.136:3300/0 conn(0x7fa258069560 0x7fa2580699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 -- 172.21.15.92:0/1821153928 >> v1:172.21.15.136:6789/0 conn(0x7fa258159c60 legacy=0x7fa25815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 -- 172.21.15.92:0/1821153928 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25815e140 con 0x7fa258159890 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.709+0000 7fa257fff700 1 --2- 172.21.15.92:0/1821153928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258159890 0x7fa25815abd0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa25806a890 tx=0x7fa248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bf076f54089ae7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa256ffd700 1 -- 172.21.15.92:0/1821153928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800b820 con 0x7fa258159890 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa256ffd700 1 -- 172.21.15.92:0/1821153928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24800b980 con 0x7fa258159890 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa256ffd700 1 -- 172.21.15.92:0/1821153928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800bd20 con 0x7fa258159890 2023-10-06T03:09:56.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 -- 172.21.15.92:0/1821153928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258159890 msgr2=0x7fa25815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/1821153928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258159890 0x7fa25815abd0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa25806a890 tx=0x7fa248009250 comp rx=0 tx=0).stop 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 -- 172.21.15.92:0/1821153928 shutdown_connections 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/1821153928 >> v2:172.21.15.136:3300/0 conn(0x7fa258069560 0x7fa2580699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/1821153928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258159890 0x7fa25815abd0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 -- 172.21.15.92:0/1821153928 >> 172.21.15.92:0/1821153928 conn(0x7fa258068e20 msgr2=0x7fa2580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 -- 172.21.15.92:0/1821153928 shutdown_connections 2023-10-06T03:09:56.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.710+0000 7fa25e6fa700 1 -- 172.21.15.92:0/1821153928 wait complete. 2023-10-06T03:09:56.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.711+0000 7fa25e6fa700 1 Processor -- start 2023-10-06T03:09:56.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.711+0000 7fa25e6fa700 1 -- start start 2023-10-06T03:09:56.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25e6fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 0x7fa2580c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25e6fa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258159890 0x7fa2580b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25e6fa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa258159c60 0x7fa25816a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25e6fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580b5030 con 0x7fa258069560 2023-10-06T03:09:56.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25e6fa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580c78a0 con 0x7fa258159890 2023-10-06T03:09:56.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25e6fa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580c7b80 con 0x7fa258159c60 2023-10-06T03:09:56.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa257fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 0x7fa2580c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa257fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 0x7fa2580c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58028/0 (socket says 172.21.15.92:58028) 2023-10-06T03:09:56.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa257fff700 1 -- 172.21.15.92:0/523885282 learned_addr learned my addr 172.21.15.92:0/523885282 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:56.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa2577fe700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258159890 0x7fa2580b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.712+0000 7fa25cc97700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa258159c60 0x7fa25816a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa257fff700 1 -- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa258159c60 msgr2=0x7fa25816a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa257fff700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa258159c60 0x7fa25816a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa257fff700 1 -- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258159890 msgr2=0x7fa2580b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa257fff700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258159890 0x7fa2580b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa257fff700 1 -- 172.21.15.92:0/523885282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2480094f0 con 0x7fa258069560 2023-10-06T03:09:56.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa2577fe700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258159890 0x7fa2580b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:56.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa257fff700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 0x7fa2580c8c20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa248009290 tx=0x7fa248016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248011e20 con 0x7fa258069560 2023-10-06T03:09:56.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248020070 con 0x7fa258069560 2023-10-06T03:09:56.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24801b6a0 con 0x7fa258069560 2023-10-06T03:09:56.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2580c7e60 con 0x7fa258069560 2023-10-06T03:09:56.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.713+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2580c8420 con 0x7fa258069560 2023-10-06T03:09:56.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.714+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25804c3c0 con 0x7fa258069560 2023-10-06T03:09:56.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.714+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa248022030 con 0x7fa258069560 2023-10-06T03:09:56.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.715+0000 7fa23ffff700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa244067320 0x7fa2440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:56.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.715+0000 7fa2577fe700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa244067320 0x7fa2440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:56.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.715+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa248086970 con 0x7fa258069560 2023-10-06T03:09:56.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.715+0000 7fa2577fe700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa244067320 0x7fa2440697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa24c006fd0 tx=0x7fa24c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:56.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.719+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa248053900 con 0x7fa258069560 2023-10-06T03:09:56.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.900+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa25804c3c0 con 0x7fa258069560 2023-10-06T03:09:56.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.900+0000 7fa23ffff700 1 -- 172.21.15.92:0/523885282 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v163) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa248056f20 con 0x7fa258069560 2023-10-06T03:09:56.901 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:09:56.903 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.902+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa244067320 msgr2=0x7fa2440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.902+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa244067320 0x7fa2440697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa24c006fd0 tx=0x7fa24c008040 comp rx=0 tx=0).stop 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.902+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 msgr2=0x7fa2580c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.902+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 0x7fa2580c8c20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa248009290 tx=0x7fa248016000 comp rx=0 tx=0).stop 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 shutdown_connections 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa244067320 0x7fa2440697d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa258069560 0x7fa2580c8c20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa258159c60 0x7fa25816a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 --2- 172.21.15.92:0/523885282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258159890 0x7fa2580b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 >> 172.21.15.92:0/523885282 conn(0x7fa258068e20 msgr2=0x7fa2580d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 shutdown_connections 2023-10-06T03:09:56.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:56.903+0000 7fa25e6fa700 1 -- 172.21.15.92:0/523885282 wait complete. 2023-10-06T03:09:56.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:09:57.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.030+0000 7f02b31f1700 1 Processor -- start 2023-10-06T03:09:57.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.030+0000 7f02b31f1700 1 -- start start 2023-10-06T03:09:57.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b31f1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02ac06bcd0 0x7f02ac0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b31f1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f02ac06c0a0 0x7f02ac15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b31f1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac15d7e0 con 0x7f02ac15df50 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b31f1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac15dac0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b31f1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac15aa30 con 0x7f02ac06c0a0 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b0f8d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02ac06bcd0 0x7f02ac0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b0f8d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02ac06bcd0 0x7f02ac0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58038/0 (socket says 172.21.15.92:58038) 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.031+0000 7f02b0f8d700 1 -- 172.21.15.92:0/1917414190 learned_addr learned my addr 172.21.15.92:0/1917414190 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02a3fff700 1 --2- 172.21.15.92:0/1917414190 >> v2:172.21.15.136:3300/0 conn(0x7f02ac06c0a0 0x7f02ac15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b0f8d700 1 -- 172.21.15.92:0/1917414190 >> v2:172.21.15.136:3300/0 conn(0x7f02ac06c0a0 msgr2=0x7f02ac15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b0f8d700 1 --2- 172.21.15.92:0/1917414190 >> v2:172.21.15.136:3300/0 conn(0x7f02ac06c0a0 0x7f02ac15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b0f8d700 1 -- 172.21.15.92:0/1917414190 >> v1:172.21.15.136:6789/0 conn(0x7f02ac15df50 legacy=0x7f02ac0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b0f8d700 1 -- 172.21.15.92:0/1917414190 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac15ad10 con 0x7f02ac06bcd0 2023-10-06T03:09:57.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b0f8d700 1 --2- 172.21.15.92:0/1917414190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0caa90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f02ac06b4e0 tx=0x7f02a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993f78efc7407d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02a37fe700 1 -- 172.21.15.92:0/1917414190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800e070 con 0x7f02ac06bcd0 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02a37fe700 1 -- 172.21.15.92:0/1917414190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a800ba20 con 0x7f02ac06bcd0 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02a37fe700 1 -- 172.21.15.92:0/1917414190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a80092e0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b31f1700 1 -- 172.21.15.92:0/1917414190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 msgr2=0x7f02ac0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.032+0000 7f02b31f1700 1 --2- 172.21.15.92:0/1917414190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0caa90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f02ac06b4e0 tx=0x7f02a8009840 comp rx=0 tx=0).stop 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.033+0000 7f02b31f1700 1 -- 172.21.15.92:0/1917414190 shutdown_connections 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.033+0000 7f02b31f1700 1 --2- 172.21.15.92:0/1917414190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0caa90 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.033+0000 7f02b31f1700 1 --2- 172.21.15.92:0/1917414190 >> v2:172.21.15.136:3300/0 conn(0x7f02ac06c0a0 0x7f02ac15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.033+0000 7f02b31f1700 1 -- 172.21.15.92:0/1917414190 >> 172.21.15.92:0/1917414190 conn(0x7f02ac0ce600 msgr2=0x7f02ac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.033+0000 7f02b31f1700 1 -- 172.21.15.92:0/1917414190 shutdown_connections 2023-10-06T03:09:57.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.033+0000 7f02b31f1700 1 -- 172.21.15.92:0/1917414190 wait complete. 2023-10-06T03:09:57.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.034+0000 7f02b31f1700 1 Processor -- start 2023-10-06T03:09:57.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.034+0000 7f02b31f1700 1 -- start start 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02ac06c0a0 0x7f02ac0b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02ac15df50 0x7f02ac144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0b3700 con 0x7f02ac06bcd0 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0b2af0 con 0x7f02ac06c0a0 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0b2dd0 con 0x7f02ac15df50 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58048/0 (socket says 172.21.15.92:58048) 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 -- 172.21.15.92:0/2802612223 learned_addr learned my addr 172.21.15.92:0/2802612223 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b178e700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02ac15df50 0x7f02ac144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02a3fff700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02ac06c0a0 0x7f02ac0b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 -- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02ac15df50 msgr2=0x7f02ac144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02ac15df50 0x7f02ac144220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 -- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02ac06c0a0 msgr2=0x7f02ac0b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02ac06c0a0 0x7f02ac0b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 -- 172.21.15.92:0/2802612223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8009ae0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b178e700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02ac15df50 0x7f02ac144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b0f8d700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0b3b90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b6c0 tx=0x7f02a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8007000 con 0x7f02ac06bcd0 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a8007820 con 0x7f02ac06bcd0 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac0b30b0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8007bc0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.035+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac0b2800 con 0x7f02ac06bcd0 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.036+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac04c3c0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.037+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f02a801c410 con 0x7f02ac06bcd0 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.037+0000 7f02a0ff9700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0298067320 0x7f02980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.037+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f02a8086790 con 0x7f02ac06bcd0 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.037+0000 7f02a3fff700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0298067320 0x7f02980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.038+0000 7f02a3fff700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0298067320 0x7f02980697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c3ad0 tx=0x7f029c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:57.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.041+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02a80533a0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.225+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f02ac15aff0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.225+0000 7f02a0ff9700 1 -- 172.21.15.92:0/2802612223 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v163) v1 ==== 84+0+126 (secure 0 0 0) 0x7f02a80569c0 con 0x7f02ac06bcd0 2023-10-06T03:09:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:09:57.226 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:09:57.226 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:09:57.227 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:09:57.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0298067320 msgr2=0x7f02980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0298067320 0x7f02980697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c3ad0 tx=0x7f029c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 msgr2=0x7f02ac0b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0b3b90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f02a800b6c0 tx=0x7f02a8018000 comp rx=0 tx=0).stop 2023-10-06T03:09:57.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 shutdown_connections 2023-10-06T03:09:57.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0298067320 0x7f02980697d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02ac06bcd0 0x7f02ac0b3b90 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02ac15df50 0x7f02ac144220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 --2- 172.21.15.92:0/2802612223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02ac06c0a0 0x7f02ac0b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 >> 172.21.15.92:0/2802612223 conn(0x7f02ac0ce600 msgr2=0x7f02ac1587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:57.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 shutdown_connections 2023-10-06T03:09:57.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.227+0000 7f02b31f1700 1 -- 172.21.15.92:0/2802612223 wait complete. 2023-10-06T03:09:57.240 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:09:57.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:09:57.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.358+0000 7f5427f16700 1 Processor -- start 2023-10-06T03:09:57.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.358+0000 7f5427f16700 1 -- start start 2023-10-06T03:09:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5427f16700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54200cd910 0x7f54200ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5427f16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54200b55a0 0x7f54200ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5427f16700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200b5b00 con 0x7f5420159ec0 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5427f16700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54200b5da0 con 0x7f54200b55a0 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5427f16700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b6080 con 0x7f54200cd910 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f54264b3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f54200b55a0 0x7f54200ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5425cb2700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5420159ec0 0x7f54200b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:60354/0 (socket says 172.21.15.92:60354) 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.359+0000 7f5425cb2700 1 -- 172.21.15.92:0/2710972876 learned_addr learned my addr 172.21.15.92:0/2710972876 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54254b1700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54200cd910 0x7f54200ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54264b3700 1 -- 172.21.15.92:0/2710972876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54200cd910 msgr2=0x7f54200ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:57.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54264b3700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54200cd910 0x7f54200ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54264b3700 1 -- 172.21.15.92:0/2710972876 >> v1:172.21.15.92:6789/0 conn(0x7f5420159ec0 legacy=0x7f54200b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5424cb0700 1 -- 172.21.15.92:0/2710972876 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798261353 0 0) 0x7f54200b5b00 con 0x7f5420159ec0 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54264b3700 1 -- 172.21.15.92:0/2710972876 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200cc100 con 0x7f54200b55a0 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54254b1700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54200cd910 0x7f54200ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f54264b3700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54200b55a0 0x7f54200ce880 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5418009e60 tx=0x7f5418009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6622f7c7eae8a8d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5424cb0700 1 -- 172.21.15.92:0/2710972876 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541800e070 con 0x7f54200b55a0 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5424cb0700 1 -- 172.21.15.92:0/2710972876 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541800bc30 con 0x7f54200b55a0 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5424cb0700 1 -- 172.21.15.92:0/2710972876 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418012430 con 0x7f54200b55a0 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5427f16700 1 -- 172.21.15.92:0/2710972876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54200b55a0 msgr2=0x7f54200ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5427f16700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54200b55a0 0x7f54200ce880 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f5418009e60 tx=0x7f5418009780 comp rx=0 tx=0).stop 2023-10-06T03:09:57.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5427f16700 1 -- 172.21.15.92:0/2710972876 shutdown_connections 2023-10-06T03:09:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5427f16700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f54200b55a0 0x7f54200ce880 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.360+0000 7f5427f16700 1 --2- 172.21.15.92:0/2710972876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54200cd910 0x7f54200ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.361+0000 7f5427f16700 1 -- 172.21.15.92:0/2710972876 >> 172.21.15.92:0/2710972876 conn(0x7f54200999f0 msgr2=0x7f54200b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.361+0000 7f5427f16700 1 -- 172.21.15.92:0/2710972876 shutdown_connections 2023-10-06T03:09:57.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.361+0000 7f5427f16700 1 -- 172.21.15.92:0/2710972876 wait complete. 2023-10-06T03:09:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 Processor -- start 2023-10-06T03:09:57.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 -- start start 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54200b55a0 0x7f542014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 0x7f542014edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542014f330 0x7f54201546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420154250 con 0x7f5420159ec0 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420153640 con 0x7f54200b55a0 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f5427f16700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5420153920 con 0x7f542014f330 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f54254b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 0x7f542014edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f54254b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 0x7f542014edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58072/0 (socket says 172.21.15.92:58072) 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.362+0000 7f54254b1700 1 -- 172.21.15.92:0/3823183862 learned_addr learned my addr 172.21.15.92:0/3823183862 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f5425cb2700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54200b55a0 0x7f542014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f54254b1700 1 -- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542014f330 msgr2=0x7f54201546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f54254b1700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542014f330 0x7f54201546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f54254b1700 1 -- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54200b55a0 msgr2=0x7f542014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f54254b1700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54200b55a0 0x7f542014e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f54254b1700 1 -- 172.21.15.92:0/3823183862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009a20 con 0x7f5420159ec0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f54254b1700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 0x7f542014edf0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f541000c830 tx=0x7f5410008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000a680 con 0x7f5420159ec0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000a7e0 con 0x7f5420159ec0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000aba0 con 0x7f5420159ec0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5420153c00 con 0x7f5420159ec0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.363+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542014ea60 con 0x7f5420159ec0 2023-10-06T03:09:57.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.364+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420154400 con 0x7f5420159ec0 2023-10-06T03:09:57.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.364+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f54100113f0 con 0x7f5420159ec0 2023-10-06T03:09:57.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.365+0000 7f5415ffb700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5404067320 0x7f54040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:57.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.365+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f54100850a0 con 0x7f5420159ec0 2023-10-06T03:09:57.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.365+0000 7f5425cb2700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5404067320 0x7f54040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:57.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.365+0000 7f5425cb2700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5404067320 0x7f54040697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f541c001780 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:57.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.369+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5420154400 con 0x7f5420159ec0 2023-10-06T03:09:57.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:57.552+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5420063c90 con 0x7f5420159ec0 2023-10-06T03:09:58.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.556+0000 7f5415ffb700 1 -- 172.21.15.92:0/3823183862 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v164) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5410051e10 con 0x7f5420159ec0 2023-10-06T03:09:58.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5404067320 msgr2=0x7f54040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:58.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5404067320 0x7f54040697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f541c001780 tx=0x7f541c009040 comp rx=0 tx=0).stop 2023-10-06T03:09:58.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 msgr2=0x7f542014edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:58.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 0x7f542014edf0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f541000c830 tx=0x7f5410008040 comp rx=0 tx=0).stop 2023-10-06T03:09:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 shutdown_connections 2023-10-06T03:09:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5404067320 0x7f54040697d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f542014f330 0x7f54201546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54200b55a0 0x7f542014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 --2- 172.21.15.92:0/3823183862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5420159ec0 0x7f542014edf0 secure :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f541000c830 tx=0x7f5410008040 comp rx=0 tx=0).stop 2023-10-06T03:09:58.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.560+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 >> 172.21.15.92:0/3823183862 conn(0x7f54200999f0 msgr2=0x7f54200c5630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.561+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 shutdown_connections 2023-10-06T03:09:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.561+0000 7f5427f16700 1 -- 172.21.15.92:0/3823183862 wait complete. 2023-10-06T03:09:58.563 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:09:58.576 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:09:58.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.698+0000 7fc56f59e700 1 Processor -- start 2023-10-06T03:09:58.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.698+0000 7fc56f59e700 1 -- start start 2023-10-06T03:09:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56f59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5701595d0 0x7fc5700ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56f59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc57015d930 0x7fc5700b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56f59e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700b10f0 con 0x7fc5700cf480 2023-10-06T03:09:58.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56f59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5700cd530 con 0x7fc5701595d0 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56f59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57015e000 con 0x7fc57015d930 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5701595d0 0x7fc5700ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc5701595d0 0x7fc5700ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58080/0 (socket says 172.21.15.92:58080) 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 -- 172.21.15.92:0/3030764329 learned_addr learned my addr 172.21.15.92:0/3030764329 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56ed9d700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc57015d930 0x7fc5700b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 -- 172.21.15.92:0/3030764329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc57015d930 msgr2=0x7fc5700b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc57015d930 0x7fc5700b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 -- 172.21.15.92:0/3030764329 >> v1:172.21.15.136:6789/0 conn(0x7fc5700cf480 legacy=0x7fc57015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56e59c700 1 -- 172.21.15.92:0/3030764329 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700cf990 con 0x7fc5701595d0 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.699+0000 7fc56ed9d700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc57015d930 0x7fc5700b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:09:58.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56e59c700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5701595d0 0x7fc5700ccff0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc560009f70 tx=0x7fc560009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bd39cbbb48a21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56d59a700 1 -- 172.21.15.92:0/3030764329 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56000e070 con 0x7fc5701595d0 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56d59a700 1 -- 172.21.15.92:0/3030764329 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56000ba90 con 0x7fc5701595d0 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56d59a700 1 -- 172.21.15.92:0/3030764329 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600092e0 con 0x7fc5701595d0 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 -- 172.21.15.92:0/3030764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5701595d0 msgr2=0x7fc5700ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5701595d0 0x7fc5700ccff0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc560009f70 tx=0x7fc560009840 comp rx=0 tx=0).stop 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 -- 172.21.15.92:0/3030764329 shutdown_connections 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5701595d0 0x7fc5700ccff0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 --2- 172.21.15.92:0/3030764329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc57015d930 0x7fc5700b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 -- 172.21.15.92:0/3030764329 >> 172.21.15.92:0/3030764329 conn(0x7fc5700d05f0 msgr2=0x7fc5700d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 -- 172.21.15.92:0/3030764329 shutdown_connections 2023-10-06T03:09:58.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.700+0000 7fc56f59e700 1 -- 172.21.15.92:0/3030764329 wait complete. 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.701+0000 7fc56f59e700 1 Processor -- start 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.701+0000 7fc56f59e700 1 -- start start 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56f59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5700cf480 0x7fc57014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56f59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5701595d0 0x7fc5700b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56f59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 0x7fc5700b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56f59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57014dcf0 con 0x7fc57015d930 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56f59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57014d620 con 0x7fc5700cf480 2023-10-06T03:09:58.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56f59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc570157890 con 0x7fc5701595d0 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 0x7fc5700b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 0x7fc5700b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:58090/0 (socket says 172.21.15.92:58090) 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56ed9d700 1 -- 172.21.15.92:0/1669940737 learned_addr learned my addr 172.21.15.92:0/1669940737 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56e59c700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5700cf480 0x7fc57014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.702+0000 7fc56dd9b700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5701595d0 0x7fc5700b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56ed9d700 1 -- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5701595d0 msgr2=0x7fc5700b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56ed9d700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5701595d0 0x7fc5700b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56ed9d700 1 -- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5700cf480 msgr2=0x7fc57014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56ed9d700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5700cf480 0x7fc57014d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56ed9d700 1 -- 172.21.15.92:0/1669940737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560009ae0 con 0x7fc57015d930 2023-10-06T03:09:58.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56dd9b700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5701595d0 0x7fc5700b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56ed9d700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 0x7fc5700b4910 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc56800c850 tx=0x7fc568008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568011600 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc568011780 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc568011b40 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc570157b70 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.703+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc570158130 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.704+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57014de60 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.704+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc5680193f0 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.705+0000 7fc55e7fc700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc550067320 0x7fc5500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.705+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fc568085010 con 0x7fc57015d930 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.705+0000 7fc56e59c700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc550067320 0x7fc5500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:58.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.705+0000 7fc56e59c700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc550067320 0x7fc5500697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc560009880 tx=0x7fc560017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:58.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.709+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc57014de60 con 0x7fc57015d930 2023-10-06T03:09:58.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:58.891+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc57014de60 con 0x7fc57015d930 2023-10-06T03:09:59.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.565+0000 7fc55e7fc700 1 -- 172.21.15.92:0/1669940737 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v165) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc57014de60 con 0x7fc57015d930 2023-10-06T03:09:59.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc550067320 msgr2=0x7fc5500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:59.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc550067320 0x7fc5500697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc560009880 tx=0x7fc560017000 comp rx=0 tx=0).stop 2023-10-06T03:09:59.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 msgr2=0x7fc5700b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:59.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 0x7fc5700b4910 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc56800c850 tx=0x7fc568008040 comp rx=0 tx=0).stop 2023-10-06T03:09:59.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 shutdown_connections 2023-10-06T03:09:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc550067320 0x7fc5500697d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc57015d930 0x7fc5700b4910 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.568+0000 7fc56f59e700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5700cf480 0x7fc57014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.569+0000 7fc56f59e700 1 --2- 172.21.15.92:0/1669940737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5701595d0 0x7fc5700b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.569+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 >> 172.21.15.92:0/1669940737 conn(0x7fc5700d05f0 msgr2=0x7fc570068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:59.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.571+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 shutdown_connections 2023-10-06T03:09:59.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.576+0000 7fc56f59e700 1 -- 172.21.15.92:0/1669940737 wait complete. 2023-10-06T03:09:59.579 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:09:59.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:09:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.709+0000 7f001ffff700 1 Processor -- start 2023-10-06T03:09:59.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.709+0000 7f001ffff700 1 -- start start 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001ffff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f00200b4e70 0x7f00200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f00200b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001ffff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00200b2080 con 0x7f002015d240 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001ffff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00200cfb00 con 0x7f00200b4e70 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00200b5c40 con 0x7f00200b16d0 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f002015d240 0x7f002015d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45108/0 (socket says 172.21.15.92:45108) 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001f7fe700 1 -- 172.21.15.92:0/1953376670 learned_addr learned my addr 172.21.15.92:0/1953376670 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001effd700 1 --2- 172.21.15.92:0/1953376670 >> v2:172.21.15.136:3300/0 conn(0x7f00200b4e70 0x7f00200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725983369 0 0) 0x7f00200b2080 con 0x7f002015d240 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f000c003570 con 0x7f002015d240 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.710+0000 7f00167fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f00200b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1253137951 0 0) 0x7f000c003570 con 0x7f002015d240 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 msgr2=0x7f00200b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f00200b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 >> v2:172.21.15.136:3300/0 conn(0x7f00200b4e70 msgr2=0x7f00200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 --2- 172.21.15.92:0/1953376670 >> v2:172.21.15.136:3300/0 conn(0x7f00200b4e70 0x7f00200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00200b5f20 con 0x7f002015d240 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f00180017f0 con 0x7f002015d240 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 >> v1:172.21.15.92:6789/0 conn(0x7f002015d240 legacy=0x7f002015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000c003a60 0x7f000c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f000c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f000c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200cfb00 con 0x7f000c004810 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00200b5c40 con 0x7f000c003a60 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c00ba60 con 0x7f00200b16d0 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f000c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001effd700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000c003a60 0x7f000c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001f7fe700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f000c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 -- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 msgr2=0x7f000c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f000c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 -- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000c003a60 msgr2=0x7f000c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000c003a60 0x7f000c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 -- 172.21.15.92:0/1953376670 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c003570 con 0x7f000c004810 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001f7fe700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00200b16d0 0x7f000c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f00167fc700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f000c004c80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0008009c30 tx=0x7f00080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe477c5e155225c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800e710 con 0x7f000c004810 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000800e870 con 0x7f000c004810 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.711+0000 7f001e7fc700 1 -- 172.21.15.92:0/1953376670 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000800ec30 con 0x7f000c004810 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001ffff700 1 -- 172.21.15.92:0/1953376670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 msgr2=0x7f000c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001ffff700 1 --2- 172.21.15.92:0/1953376670 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f000c004c80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0008009c30 tx=0x7f00080091a0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001f7fe700 1 -- 172.21.15.92:0/1953376670 reap_dead start 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001ffff700 1 -- 172.21.15.92:0/1953376670 shutdown_connections 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001ffff700 1 -- 172.21.15.92:0/1953376670 >> 172.21.15.92:0/1953376670 conn(0x7f00200d10c0 msgr2=0x7f00200affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001ffff700 1 -- 172.21.15.92:0/1953376670 shutdown_connections 2023-10-06T03:09:59.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.712+0000 7f001ffff700 1 -- 172.21.15.92:0/1953376670 wait complete. 2023-10-06T03:09:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.713+0000 7f001ffff700 1 Processor -- start 2023-10-06T03:09:59.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.713+0000 7f001ffff700 1 -- start start 2023-10-06T03:09:59.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f00200ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001ffff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0020158f10 0x7f00200cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f002015fc10 0x7f00201600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001ffff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00201605f0 con 0x7f000c004810 2023-10-06T03:09:59.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001ffff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002016a9c0 con 0x7f0020158f10 2023-10-06T03:09:59.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001ffff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0020169af0 con 0x7f002015fc10 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f00200ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f00200ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45224/0 (socket says 172.21.15.92:45224) 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 -- 172.21.15.92:0/3323355230 learned_addr learned my addr 172.21.15.92:0/3323355230 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f00167fc700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0020158f10 0x7f00200cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001f7fe700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f002015fc10 0x7f00201600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 -- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f002015fc10 msgr2=0x7f00201600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f002015fc10 0x7f00201600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 -- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0020158f10 msgr2=0x7f00200cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0020158f10 0x7f00200cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 -- 172.21.15.92:0/3323355230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0020169db0 con 0x7f000c004810 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.714+0000 7f001effd700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f00200ca4b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0010011d70 tx=0x7f00100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:59.721 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.715+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001000fac0 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.715+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001000fc20 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.715+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010014440 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.715+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002016a090 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.715+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002016a650 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.716+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002016aca0 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.716+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0010014870 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.716+0000 7f00177fe700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.717+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f0010018030 con 0x7f000c004810 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.717+0000 7f00167fc700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:09:59.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.717+0000 7f00167fc700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f00080091e0 tx=0x7f000801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:09:59.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.721+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00100523e0 con 0x7f000c004810 2023-10-06T03:09:59.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:09:59.947+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f00200b14b0 con 0x7f000c004810 2023-10-06T03:10:00.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.566+0000 7f00177fe700 1 -- 172.21.15.92:0/3323355230 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v109) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0010055a00 con 0x7f000c004810 2023-10-06T03:10:00.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 msgr2=0x7f00000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f00080091e0 tx=0x7f000801a000 comp rx=0 tx=0).stop 2023-10-06T03:10:00.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 msgr2=0x7f00200ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f00200ca4b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0010011d70 tx=0x7f00100091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 shutdown_connections 2023-10-06T03:10:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 secure :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f00080091e0 tx=0x7f000801a000 comp rx=0 tx=0).stop 2023-10-06T03:10:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f000c004810 0x7f00200ca4b0 secure :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f0010011d70 tx=0x7f00100091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f002015fc10 0x7f00201600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 --2- 172.21.15.92:0/3323355230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0020158f10 0x7f00200cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.573+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 >> 172.21.15.92:0/3323355230 conn(0x7f00200d10c0 msgr2=0x7f0020069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:00.577 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.576+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 shutdown_connections 2023-10-06T03:10:00.578 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.577+0000 7f001ffff700 1 -- 172.21.15.92:0/3323355230 wait complete. 2023-10-06T03:10:00.584 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 29 and data pool 30 2023-10-06T03:10:00.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:00.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.715+0000 7fc5a8031700 1 Processor -- start 2023-10-06T03:10:00.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.715+0000 7fc5a8031700 1 -- start start 2023-10-06T03:10:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a8031700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a00b1850 0x7fc5a00b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a8031700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a00cb300 con 0x7fc5a00ae680 2023-10-06T03:10:00.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a8031700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a00cb5e0 con 0x7fc5a01579f0 2023-10-06T03:10:00.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a8031700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a006fc30 con 0x7fc5a00b1850 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a55cc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc5a00ae680 0x7fc5a01572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45124/0 (socket says 172.21.15.92:45124) 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a55cc700 1 -- 172.21.15.92:0/784679340 learned_addr learned my addr 172.21.15.92:0/784679340 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379383041 0 0) 0x7fc5a00cb300 con 0x7fc5a00ae680 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc588003570 con 0x7fc5a00ae680 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a00b1850 0x7fc5a00b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612117025 0 0) 0x7fc588003570 con 0x7fc5a00ae680 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a00b1850 msgr2=0x7fc5a00b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a00b1850 0x7fc5a00b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 >> v1:172.21.15.136:6789/0 conn(0x7fc5a01579f0 legacy=0x7fc5a00abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00cb8c0 con 0x7fc5a00ae680 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc594001470 con 0x7fc5a00ae680 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 >> v1:172.21.15.92:6789/0 conn(0x7fc5a00ae680 legacy=0x7fc5a01572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588003b10 0x7fc588003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc588004810 0x7fc588004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5880053c0 0x7fc58800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a006fc30 con 0x7fc588004810 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58800c1b0 con 0x7fc588003b10 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58800c370 con 0x7fc5880053c0 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.716+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a00b1850 0x7fc5a00b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:00.726 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc588004810 0x7fc588004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a65ce700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5880053c0 0x7fc58800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588003b10 0x7fc588003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 -- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5880053c0 msgr2=0x7fc58800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5880053c0 0x7fc58800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 -- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588003b10 msgr2=0x7fc588003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588003b10 0x7fc588003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 -- 172.21.15.92:0/784679340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc588003570 con 0x7fc588004810 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a65ce700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5880053c0 0x7fc58800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a55cc700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc588004810 0x7fc588004c80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fc59400c2a0 tx=0x7fc5a00c9ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d96eaffc7e35b31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400cb70 con 0x7fc588004810 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59400ccd0 con 0x7fc588004810 2023-10-06T03:10:00.727 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.717+0000 7fc5a4dcb700 1 -- 172.21.15.92:0/784679340 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594005b70 con 0x7fc588004810 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a8031700 1 -- 172.21.15.92:0/784679340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc588004810 msgr2=0x7fc588004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a8031700 1 --2- 172.21.15.92:0/784679340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc588004810 0x7fc588004c80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fc59400c2a0 tx=0x7fc5a00c9ac0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a65ce700 1 -- 172.21.15.92:0/784679340 reap_dead start 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a8031700 1 -- 172.21.15.92:0/784679340 shutdown_connections 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a8031700 1 -- 172.21.15.92:0/784679340 >> 172.21.15.92:0/784679340 conn(0x7fc5a006a320 msgr2=0x7fc5a00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a8031700 1 -- 172.21.15.92:0/784679340 shutdown_connections 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.718+0000 7fc5a8031700 1 -- 172.21.15.92:0/784679340 wait complete. 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.719+0000 7fc5a8031700 1 Processor -- start 2023-10-06T03:10:00.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.719+0000 7fc5a8031700 1 -- start start 2023-10-06T03:10:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588004810 0x7fc5a0149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a0157e70 0x7fc5a01582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 0x7fc5a014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a00cc260 con 0x7fc5a014f350 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a014fd30 con 0x7fc588004810 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a015a050 con 0x7fc5a0157e70 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 0x7fc5a014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 0x7fc5a014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45254/0 (socket says 172.21.15.92:45254) 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 -- 172.21.15.92:0/1498120748 learned_addr learned my addr 172.21.15.92:0/1498120748 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588004810 0x7fc5a0149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a55cc700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a0157e70 0x7fc5a01582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 -- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a0157e70 msgr2=0x7fc5a01582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a0157e70 0x7fc5a01582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 -- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588004810 msgr2=0x7fc5a0149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588004810 0x7fc5a0149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 -- 172.21.15.92:0/1498120748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a01589e0 con 0x7fc5a014f350 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a55cc700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a0157e70 0x7fc5a01582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588004810 0x7fc5a0149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a65ce700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 0x7fc5a014f7f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0157dc0 tx=0x7fc590009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000ee60 con 0x7fc5a014f350 2023-10-06T03:10:00.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5900114f0 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900118b0 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.720+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a0158cc0 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.721+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a0154130 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.722+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a0153cb0 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.722+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc590011c60 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.722+0000 7fc59dffb700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc58c067320 0x7fc58c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.722+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc5900844e0 con 0x7fc5a014f350 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.722+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc58c067320 0x7fc58c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:00.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.723+0000 7fc5a5dcd700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc58c067320 0x7fc58c0697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc598009fd0 tx=0x7fc5980098c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:00.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.726+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc59004d5a0 con 0x7fc5a014f350 2023-10-06T03:10:00.910 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.910+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a004c3c0 con 0x7fc5a014f350 2023-10-06T03:10:00.911 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.911+0000 7fc59dffb700 1 -- 172.21.15.92:0/1498120748 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fc5a0153cb0 con 0x7fc5a014f350 2023-10-06T03:10:00.911 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:00.912 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":166,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:59.948693+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:00.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.912+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc58c067320 msgr2=0x7fc58c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.912+0000 7fc5a8031700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc58c067320 0x7fc58c0697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc598009fd0 tx=0x7fc5980098c0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.912+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 msgr2=0x7fc5a014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.912+0000 7fc5a8031700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 0x7fc5a014f7f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0157dc0 tx=0x7fc590009040 comp rx=0 tx=0).stop 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 shutdown_connections 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc58c067320 0x7fc58c0697d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc5a014f350 0x7fc5a014f7f0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc588004810 0x7fc5a0149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 --2- 172.21.15.92:0/1498120748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5a0157e70 0x7fc5a01582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:00.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 >> 172.21.15.92:0/1498120748 conn(0x7fc5a006a320 msgr2=0x7fc5a014bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:00.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 shutdown_connections 2023-10-06T03:10:00.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:00.913+0000 7fc5a8031700 1 -- 172.21.15.92:0/1498120748 wait complete. 2023-10-06T03:10:00.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.039+0000 7fa27f0bf700 1 Processor -- start 2023-10-06T03:10:01.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.039+0000 7fa27f0bf700 1 -- start start 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27f0bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa27815d230 0x7fa27815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27f0bf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2780b2980 con 0x7fa278159420 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27f0bf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27815db70 con 0x7fa2781597f0 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27f0bf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2780b11d0 con 0x7fa27815d230 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e8be700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa27815d230 0x7fa27815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e0bd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa278159420 0x7fa2780b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45128/0 (socket says 172.21.15.92:45128) 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e0bd700 1 -- 172.21.15.92:0/2321978109 learned_addr learned my addr 172.21.15.92:0/2321978109 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:01.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e8be700 1 -- 172.21.15.92:0/2321978109 >> v1:172.21.15.136:6789/0 conn(0x7fa2781597f0 legacy=0x7fa2780b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e8be700 1 -- 172.21.15.92:0/2321978109 >> v1:172.21.15.92:6789/0 conn(0x7fa278159420 legacy=0x7fa2780b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e8be700 1 -- 172.21.15.92:0/2321978109 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780b14b0 con 0x7fa27815d230 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27d0bb700 1 -- 172.21.15.92:0/2321978109 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217866841 0 0) 0x7fa2780b2980 con 0x7fa278159420 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.040+0000 7fa27e8be700 1 --2- 172.21.15.92:0/2321978109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa27815d230 0x7fa27815d630 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa270009e60 tx=0x7fa270009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819b94ea942f5bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27d0bb700 1 -- 172.21.15.92:0/2321978109 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000e070 con 0x7fa27815d230 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27d0bb700 1 -- 172.21.15.92:0/2321978109 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27000bbf0 con 0x7fa27815d230 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27d0bb700 1 -- 172.21.15.92:0/2321978109 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270012430 con 0x7fa27815d230 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 -- 172.21.15.92:0/2321978109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa27815d230 msgr2=0x7fa27815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/2321978109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa27815d230 0x7fa27815d630 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa270009e60 tx=0x7fa270009780 comp rx=0 tx=0).stop 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 -- 172.21.15.92:0/2321978109 shutdown_connections 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/2321978109 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa27815d230 0x7fa27815d630 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 -- 172.21.15.92:0/2321978109 >> 172.21.15.92:0/2321978109 conn(0x7fa2780d0660 msgr2=0x7fa2780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 -- 172.21.15.92:0/2321978109 shutdown_connections 2023-10-06T03:10:01.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.041+0000 7fa27f0bf700 1 -- 172.21.15.92:0/2321978109 wait complete. 2023-10-06T03:10:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.042+0000 7fa27f0bf700 1 Processor -- start 2023-10-06T03:10:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.042+0000 7fa27f0bf700 1 -- start start 2023-10-06T03:10:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27f0bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 0x7fa278154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27f0bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2781597f0 0x7fa278159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27f0bf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa27815d230 0x7fa278169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27f0bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27814f840 con 0x7fa278159420 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27f0bf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27815f1c0 con 0x7fa27815d230 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27f0bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27815a1e0 con 0x7fa2781597f0 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 0x7fa278154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 0x7fa278154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45266/0 (socket says 172.21.15.92:45266) 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 -- 172.21.15.92:0/3585327756 learned_addr learned my addr 172.21.15.92:0/3585327756 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 -- 172.21.15.92:0/3585327756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2781597f0 msgr2=0x7fa278159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2781597f0 0x7fa278159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 -- 172.21.15.92:0/3585327756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa27815d230 msgr2=0x7fa278169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa27815d230 0x7fa278169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 -- 172.21.15.92:0/3585327756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270009a20 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.043+0000 7fa27e0bd700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 0x7fa278154d20 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c7e0 tx=0x7fa274007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.044+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274005b30 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.044+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa274005c90 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.044+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274007300 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.044+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa278155420 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.044+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27815eb70 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.044+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27804c3c0 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.045+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa27401b400 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.045+0000 7fa26e7fc700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa25c067320 0x7fa25c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.045+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa274084860 con 0x7fa278159420 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.045+0000 7fa27d8bc700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa25c067320 0x7fa25c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.046+0000 7fa27d8bc700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa25c067320 0x7fa25c0697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa268007fd0 tx=0x7fa268007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:01.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.050+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa274051490 con 0x7fa278159420 2023-10-06T03:10:01.232 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.232+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2780af710 con 0x7fa278159420 2023-10-06T03:10:01.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.233+0000 7fa26e7fc700 1 -- 172.21.15.92:0/3585327756 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fa274054ab0 con 0x7fa278159420 2023-10-06T03:10:01.234 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:01.234 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":166,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:09:59.948693+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.234+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa25c067320 msgr2=0x7fa25c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.234+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa25c067320 0x7fa25c0697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa268007fd0 tx=0x7fa268007b20 comp rx=0 tx=0).stop 2023-10-06T03:10:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.234+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 msgr2=0x7fa278154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.234+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 0x7fa278154d20 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c7e0 tx=0x7fa274007df0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 shutdown_connections 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa25c067320 0x7fa25c0697d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa278159420 0x7fa278154d20 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa27815d230 0x7fa278169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 --2- 172.21.15.92:0/3585327756 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2781597f0 0x7fa278159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 >> 172.21.15.92:0/3585327756 conn(0x7fa2780d0660 msgr2=0x7fa2780cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 shutdown_connections 2023-10-06T03:10:01.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.235+0000 7fa27f0bf700 1 -- 172.21.15.92:0/3585327756 wait complete. 2023-10-06T03:10:01.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:10:01.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.366+0000 7ff41c971700 1 Processor -- start 2023-10-06T03:10:01.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.366+0000 7ff41c971700 1 -- start start 2023-10-06T03:10:01.369 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41c971700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff4180b4e70 0x7ff4180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41c971700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff4180b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41c971700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff41815d240 0x7ff41815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41c971700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4180b2080 con 0x7ff41815d240 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41c971700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4180cfb00 con 0x7ff4180b4e70 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41c971700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4180b5c40 con 0x7ff4180b16d0 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff41815d240 0x7ff41815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff41815d240 0x7ff41815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45268/0 (socket says 172.21.15.92:45268) 2023-10-06T03:10:01.370 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 -- 172.21.15.92:0/2036051373 learned_addr learned my addr 172.21.15.92:0/2036051373 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41659c700 1 --2- 172.21.15.92:0/2036051373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff4180b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff416d9d700 1 --2- 172.21.15.92:0/2036051373 >> v2:172.21.15.136:3300/0 conn(0x7ff4180b4e70 0x7ff4180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 -- 172.21.15.92:0/2036051373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 msgr2=0x7ff4180b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 --2- 172.21.15.92:0/2036051373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff4180b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 -- 172.21.15.92:0/2036051373 >> v2:172.21.15.136:3300/0 conn(0x7ff4180b4e70 msgr2=0x7ff4180ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 --2- 172.21.15.92:0/2036051373 >> v2:172.21.15.136:3300/0 conn(0x7ff4180b4e70 0x7ff4180ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 -- 172.21.15.92:0/2036051373 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180b5f20 con 0x7ff41815d240 2023-10-06T03:10:01.371 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.367+0000 7ff41759e700 1 --2- 172.21.15.92:0/2036051373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815d6c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff418064910 tx=0x7ff408009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c7c6e228867ed6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff415d9b700 1 -- 172.21.15.92:0/2036051373 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800b640 con 0x7ff41815d240 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff415d9b700 1 -- 172.21.15.92:0/2036051373 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40800b7a0 con 0x7ff41815d240 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff415d9b700 1 -- 172.21.15.92:0/2036051373 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800bb60 con 0x7ff41815d240 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 -- 172.21.15.92:0/2036051373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 msgr2=0x7ff41815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 --2- 172.21.15.92:0/2036051373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815d6c0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff418064910 tx=0x7ff408009250 comp rx=0 tx=0).stop 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 -- 172.21.15.92:0/2036051373 shutdown_connections 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 --2- 172.21.15.92:0/2036051373 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815d6c0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 --2- 172.21.15.92:0/2036051373 >> v2:172.21.15.136:3300/0 conn(0x7ff4180b4e70 0x7ff4180ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.372 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 --2- 172.21.15.92:0/2036051373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff4180b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 -- 172.21.15.92:0/2036051373 >> 172.21.15.92:0/2036051373 conn(0x7ff4180d10c0 msgr2=0x7ff4180affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:01.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 -- 172.21.15.92:0/2036051373 shutdown_connections 2023-10-06T03:10:01.373 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.368+0000 7ff41c971700 1 -- 172.21.15.92:0/2036051373 wait complete. 2023-10-06T03:10:01.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.369+0000 7ff41c971700 1 Processor -- start 2023-10-06T03:10:01.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.369+0000 7ff41c971700 1 -- start start 2023-10-06T03:10:01.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41c971700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff418158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41c971700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4180b4e70 0x7ff418159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41c971700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41c971700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418153230 con 0x7ff41815d240 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41c971700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff418153510 con 0x7ff4180b4e70 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41c971700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4181537f0 con 0x7ff4180b16d0 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41759e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45276/0 (socket says 172.21.15.92:45276) 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41759e700 1 -- 172.21.15.92:0/836758913 learned_addr learned my addr 172.21.15.92:0/836758913 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41759e700 1 -- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 msgr2=0x7ff418158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff41659c700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4180b4e70 0x7ff418159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.370+0000 7ff416d9d700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff418158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41759e700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff418158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41759e700 1 -- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4180b4e70 msgr2=0x7ff418159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41759e700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4180b4e70 0x7ff418159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41759e700 1 -- 172.21.15.92:0/836758913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080094f0 con 0x7ff41815d240 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41659c700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4180b4e70 0x7ff418159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff416d9d700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff418158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41759e700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815e680 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff4080094c0 tx=0x7ff408017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:01.379 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408011420 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4080115c0 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff418153ad0 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408011980 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.371+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41815ed80 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.372+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff418154020 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.372+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff408011cf0 con 0x7ff41815d240 2023-10-06T03:10:01.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.373+0000 7ff40effd700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff3f8067320 0x7ff3f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.373+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff4080862d0 con 0x7ff41815d240 2023-10-06T03:10:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.373+0000 7ff416d9d700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff3f8067320 0x7ff3f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.373+0000 7ff416d9d700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff3f8067320 0x7ff3f80697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff410007780 tx=0x7ff41000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:01.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.377+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff408052f30 con 0x7ff41815d240 2023-10-06T03:10:01.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:01.605+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff41804bb80 con 0x7ff41815d240 2023-10-06T03:10:02.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.594+0000 7ff40effd700 1 -- 172.21.15.92:0/836758913 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v112) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff408056550 con 0x7ff41815d240 2023-10-06T03:10:02.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff3f8067320 msgr2=0x7ff3f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:02.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff3f8067320 0x7ff3f80697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff410007780 tx=0x7ff41000b040 comp rx=0 tx=0).stop 2023-10-06T03:10:02.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 msgr2=0x7ff41815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:02.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815e680 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff4080094c0 tx=0x7ff408017000 comp rx=0 tx=0).stop 2023-10-06T03:10:02.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 shutdown_connections 2023-10-06T03:10:02.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff3f8067320 0x7ff3f80697d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41815d240 0x7ff41815e680 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4180b16d0 0x7ff418158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 --2- 172.21.15.92:0/836758913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4180b4e70 0x7ff418159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 >> 172.21.15.92:0/836758913 conn(0x7ff4180d10c0 msgr2=0x7ff4180ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:02.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 shutdown_connections 2023-10-06T03:10:02.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.596+0000 7ff41c971700 1 -- 172.21.15.92:0/836758913 wait complete. 2023-10-06T03:10:02.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:10:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.761+0000 7f774aacb700 1 Processor -- start 2023-10-06T03:10:02.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.761+0000 7f774aacb700 1 -- start start 2023-10-06T03:10:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f774aacb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7744159890 0x7f774415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f774aacb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744069560 0x7f77440699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:02.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f774aacb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7744069ee0 con 0x7f7744159c60 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f774aacb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774415a160 con 0x7f7744159890 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f774aacb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77440d13a0 con 0x7f7744069560 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7744159890 0x7f774415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7744159890 0x7f774415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45280/0 (socket says 172.21.15.92:45280) 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 -- 172.21.15.92:0/2575449888 learned_addr learned my addr 172.21.15.92:0/2575449888 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7743fff700 1 --1- 172.21.15.92:0/2575449888 >> v1:172.21.15.92:6789/0 conn(0x7f7744159c60 0x7f774415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45140/0 (socket says 172.21.15.92:45140) 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 -- 172.21.15.92:0/2575449888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744069560 msgr2=0x7f77440699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 --2- 172.21.15.92:0/2575449888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744069560 0x7f77440699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 -- 172.21.15.92:0/2575449888 >> v1:172.21.15.92:6789/0 conn(0x7f7744159c60 legacy=0x7f774415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f7748867700 1 -- 172.21.15.92:0/2575449888 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774415e140 con 0x7f7744159890 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.762+0000 7f77437fe700 1 -- 172.21.15.92:0/2575449888 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49078932 0 0) 0x7f7744069ee0 con 0x7f7744159c60 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f7748867700 1 --2- 172.21.15.92:0/2575449888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159890 0x7f774415abd0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7734009f20 tx=0x7f7734009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b59b2dbee74f75c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f77437fe700 1 -- 172.21.15.92:0/2575449888 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400e070 con 0x7f7744159890 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f77437fe700 1 -- 172.21.15.92:0/2575449888 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773400ba80 con 0x7f7744159890 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f77437fe700 1 -- 172.21.15.92:0/2575449888 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340092e0 con 0x7f7744159890 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 -- 172.21.15.92:0/2575449888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159890 msgr2=0x7f774415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:02.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 --2- 172.21.15.92:0/2575449888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159890 0x7f774415abd0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7734009f20 tx=0x7f7734009840 comp rx=0 tx=0).stop 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 -- 172.21.15.92:0/2575449888 shutdown_connections 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 --2- 172.21.15.92:0/2575449888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744069560 0x7f77440699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 --2- 172.21.15.92:0/2575449888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159890 0x7f774415abd0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 -- 172.21.15.92:0/2575449888 >> 172.21.15.92:0/2575449888 conn(0x7f7744068e20 msgr2=0x7f77440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 -- 172.21.15.92:0/2575449888 shutdown_connections 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.763+0000 7f774aacb700 1 -- 172.21.15.92:0/2575449888 wait complete. 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.764+0000 7f774aacb700 1 Processor -- start 2023-10-06T03:10:02.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 -- start start 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7744069560 0x7f77440ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744159890 0x7f77440b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 0x7f77440b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77440cf720 con 0x7f7744159c60 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744151960 con 0x7f7744069560 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f774aacb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77440cee80 con 0x7f7744159890 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f7749068700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 0x7f77440b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:02.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f7749068700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 0x7f77440b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45282/0 (socket says 172.21.15.92:45282) 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.765+0000 7f7749068700 1 -- 172.21.15.92:0/1854027898 learned_addr learned my addr 172.21.15.92:0/1854027898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7743fff700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744159890 0x7f77440b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7748867700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7744069560 0x7f77440ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7749068700 1 -- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744159890 msgr2=0x7f77440b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7749068700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744159890 0x7f77440b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7749068700 1 -- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7744069560 msgr2=0x7f77440ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7749068700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7744069560 0x7f77440ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7749068700 1 -- 172.21.15.92:0/1854027898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734009ae0 con 0x7f7744159c60 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7748867700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7744069560 0x7f77440ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7749068700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 0x7f77440b3590 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b590 tx=0x7f773c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:02.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00f510 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c00f670 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77440cf160 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c00fa30 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.766+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7744150fa0 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.767+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7744151730 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.768+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f773c01a400 con 0x7f7744159c60 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.768+0000 7f7740ff9700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7730067320 0x7f77300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:02.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.768+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f773c084860 con 0x7f7744159c60 2023-10-06T03:10:02.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.768+0000 7f7748867700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7730067320 0x7f77300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:02.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.768+0000 7f7748867700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7730067320 0x7f77300697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7734009a60 tx=0x7f7734018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:02.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:02.772+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7744151730 con 0x7f7744159c60 2023-10-06T03:10:03.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.000+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7744151730 con 0x7f7744159c60 2023-10-06T03:10:03.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.597+0000 7f7740ff9700 1 -- 172.21.15.92:0/1854027898 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v113) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7744151730 con 0x7f7744159c60 2023-10-06T03:10:03.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.599+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7730067320 msgr2=0x7f77300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.599+0000 7f774aacb700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7730067320 0x7f77300697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7734009a60 tx=0x7f7734018000 comp rx=0 tx=0).stop 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 msgr2=0x7f77440b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 0x7f77440b3590 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b590 tx=0x7f773c006f90 comp rx=0 tx=0).stop 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 shutdown_connections 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7730067320 0x7f77300697d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7744159c60 0x7f77440b3590 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7744069560 0x7f77440ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 --2- 172.21.15.92:0/1854027898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7744159890 0x7f77440b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 >> 172.21.15.92:0/1854027898 conn(0x7f7744068e20 msgr2=0x7f774414d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:03.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 shutdown_connections 2023-10-06T03:10:03.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.600+0000 7f774aacb700 1 -- 172.21.15.92:0/1854027898 wait complete. 2023-10-06T03:10:03.614 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.726+0000 7f8bf70a1700 1 Processor -- start 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.726+0000 7f8bf70a1700 1 -- start start 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf70a1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf00c91e0 0x7f8bf00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf70a1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bf006ae10 0x7f8bf006b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf70a1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf00cc210 con 0x7f8bf006aa40 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf70a1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf00cc4f0 con 0x7f8bf006ae10 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf70a1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00cc7d0 con 0x7f8bf00c91e0 2023-10-06T03:10:03.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bf006ae10 0x7f8bf006b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bf006ae10 0x7f8bf006b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45296/0 (socket says 172.21.15.92:45296) 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 -- 172.21.15.92:0/163429487 learned_addr learned my addr 172.21.15.92:0/163429487 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8beffff700 1 --2- 172.21.15.92:0/163429487 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf00c91e0 0x7f8bf00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 -- 172.21.15.92:0/163429487 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf00c91e0 msgr2=0x7f8bf00cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 --2- 172.21.15.92:0/163429487 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf00c91e0 0x7f8bf00cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 -- 172.21.15.92:0/163429487 >> v1:172.21.15.136:6789/0 conn(0x7f8bf006aa40 legacy=0x7f8bf006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 -- 172.21.15.92:0/163429487 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00ccab0 con 0x7f8bf006ae10 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bf563e700 1 --2- 172.21.15.92:0/163429487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf006ae10 0x7f8bf006b2b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d1a0 tx=0x7f8be800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624a94181358c401 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.727+0000 7f8bef7fe700 1 -- 172.21.15.92:0/163429487 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8010070 con 0x7f8bf006ae10 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bef7fe700 1 -- 172.21.15.92:0/163429487 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800b0d0 con 0x7f8bf006ae10 2023-10-06T03:10:03.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bef7fe700 1 -- 172.21.15.92:0/163429487 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800b490 con 0x7f8bf006ae10 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 -- 172.21.15.92:0/163429487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf006ae10 msgr2=0x7f8bf006b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/163429487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf006ae10 0x7f8bf006b2b0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f8be800d1a0 tx=0x7f8be800ba50 comp rx=0 tx=0).stop 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 -- 172.21.15.92:0/163429487 shutdown_connections 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/163429487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf006ae10 0x7f8bf006b2b0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/163429487 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf00c91e0 0x7f8bf00cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 -- 172.21.15.92:0/163429487 >> 172.21.15.92:0/163429487 conn(0x7f8bf00cd670 msgr2=0x7f8bf00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 -- 172.21.15.92:0/163429487 shutdown_connections 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.728+0000 7f8bf70a1700 1 -- 172.21.15.92:0/163429487 wait complete. 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.729+0000 7f8bf70a1700 1 Processor -- start 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.729+0000 7f8bf70a1700 1 -- start start 2023-10-06T03:10:03.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf70a1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf006aa40 0x7f8bf0157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf70a1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf006ae10 0x7f8bf014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf70a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 0x7f8bf00b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf70a1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00aeae0 con 0x7f8bf00c91e0 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf70a1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00aedc0 con 0x7f8bf006aa40 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf70a1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00af0a0 con 0x7f8bf006ae10 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 0x7f8bf00b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 0x7f8bf00b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45300/0 (socket says 172.21.15.92:45300) 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 -- 172.21.15.92:0/2916142795 learned_addr learned my addr 172.21.15.92:0/2916142795 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8beffff700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf006ae10 0x7f8bf014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf4e3d700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf006aa40 0x7f8bf0157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 -- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf006ae10 msgr2=0x7f8bf014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf006ae10 0x7f8bf014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 -- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf006aa40 msgr2=0x7f8bf0157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.730+0000 7f8bf563e700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf006aa40 0x7f8bf0157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8bf563e700 1 -- 172.21.15.92:0/2916142795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be800bcf0 con 0x7f8bf00c91e0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8beffff700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf006ae10 0x7f8bf014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8bf4e3d700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf006aa40 0x7f8bf0157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8bf563e700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 0x7f8bf00b2bd0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8bf006c6a0 tx=0x7f8be801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800acf0 con 0x7f8bf00c91e0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800ae90 con 0x7f8bf00c91e0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf00af380 con 0x7f8bf00c91e0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be801e690 con 0x7f8bf00c91e0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.731+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf00ae590 con 0x7f8bf00c91e0 2023-10-06T03:10:03.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.732+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf00af5a0 con 0x7f8bf00c91e0 2023-10-06T03:10:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.732+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8be8003040 con 0x7f8bf00c91e0 2023-10-06T03:10:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.733+0000 7f8becff9700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8bd4067320 0x7f8bd40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:03.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.733+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8be8086cc0 con 0x7f8bf00c91e0 2023-10-06T03:10:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.733+0000 7f8bf4e3d700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8bd4067320 0x7f8bd40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.733+0000 7f8bf4e3d700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8bd4067320 0x7f8bd40697d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8be40018f0 tx=0x7f8be4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:03.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.737+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8be808d080 con 0x7f8bf00c91e0 2023-10-06T03:10:03.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.960+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bf00ae870 con 0x7f8bf00c91e0 2023-10-06T03:10:03.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.961+0000 7f8becff9700 1 -- 172.21.15.92:0/2916142795 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f8be8022090 con 0x7f8bf00c91e0 2023-10-06T03:10:03.962 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:03.963 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:03.593287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.962+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8bd4067320 msgr2=0x7f8bd40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.962+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8bd4067320 0x7f8bd40697d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8be40018f0 tx=0x7f8be4009040 comp rx=0 tx=0).stop 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.962+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 msgr2=0x7f8bf00b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.962+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 0x7f8bf00b2bd0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f8bf006c6a0 tx=0x7f8be801a000 comp rx=0 tx=0).stop 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 shutdown_connections 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8bd4067320 0x7f8bd40697d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bf00c91e0 0x7f8bf00b2bd0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bf006aa40 0x7f8bf0157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 --2- 172.21.15.92:0/2916142795 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bf006ae10 0x7f8bf014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 >> 172.21.15.92:0/2916142795 conn(0x7f8bf00cd670 msgr2=0x7f8bf0153550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:03.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 shutdown_connections 2023-10-06T03:10:03.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:03.963+0000 7f8bf70a1700 1 -- 172.21.15.92:0/2916142795 wait complete. 2023-10-06T03:10:03.966 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 113 2023-10-06T03:10:03.976 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:04.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.089+0000 7efd01241700 1 Processor -- start 2023-10-06T03:10:04.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.089+0000 7efd01241700 1 -- start start 2023-10-06T03:10:04.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efd01241700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcfc159db0 0x7efcfc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efd01241700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 0x7efcfc06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efd01241700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc0c96a0 con 0x7efcfc1599e0 2023-10-06T03:10:04.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efd01241700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0cb9e0 con 0x7efcfc159db0 2023-10-06T03:10:04.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efd01241700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0cab60 con 0x7efcfc06a2b0 2023-10-06T03:10:04.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efcfad9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efcfc1599e0 0x7efcfc069080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45146/0 (socket says 172.21.15.92:45146) 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efcfad9d700 1 -- 172.21.15.92:0/1154865559 learned_addr learned my addr 172.21.15.92:0/1154865559 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efcfb59e700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 0x7efcfc06a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efcf259c700 1 --2- 172.21.15.92:0/1154865559 >> v2:172.21.15.136:3300/0 conn(0x7efcfc159db0 0x7efcfc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636199201 0 0) 0x7efcfc0c96a0 con 0x7efcfc1599e0 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.090+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce8003570 con 0x7efcfc1599e0 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2678475558 0 0) 0x7efce8003570 con 0x7efcfc1599e0 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 msgr2=0x7efcfc06a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 0x7efcfc06a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 >> v2:172.21.15.136:3300/0 conn(0x7efcfc159db0 msgr2=0x7efcfc069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 --2- 172.21.15.92:0/1154865559 >> v2:172.21.15.136:3300/0 conn(0x7efcfc159db0 0x7efcfc069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.096 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0cb420 con 0x7efcfc1599e0 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7efcec001470 con 0x7efcfc1599e0 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 >> v1:172.21.15.92:6789/0 conn(0x7efcfc1599e0 legacy=0x7efcfc069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8003a60 0x7efce8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efce8004810 0x7efce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 0x7efce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0cb9e0 con 0x7efce8004810 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0cab60 con 0x7efce8003a60 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce800ba60 con 0x7efcfc06a2b0 2023-10-06T03:10:04.097 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efce8004810 0x7efce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfad9d700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8003a60 0x7efce8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfb59e700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 0x7efce80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 -- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 msgr2=0x7efce80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc06a2b0 0x7efce80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 -- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8003a60 msgr2=0x7efce8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8003a60 0x7efce8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 -- 172.21.15.92:0/1154865559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce8003570 con 0x7efce8004810 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcf259c700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efce8004810 0x7efce8004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7efce4009c00 tx=0x7efce40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fed0e8268f451ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.091+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400e710 con 0x7efce8004810 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce400e870 con 0x7efce8004810 2023-10-06T03:10:04.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efcfa59c700 1 -- 172.21.15.92:0/1154865559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400ec30 con 0x7efce8004810 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efd01241700 1 -- 172.21.15.92:0/1154865559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efce8004810 msgr2=0x7efce8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efd01241700 1 --2- 172.21.15.92:0/1154865559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efce8004810 0x7efce8004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7efce4009c00 tx=0x7efce40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efcfb59e700 1 -- 172.21.15.92:0/1154865559 reap_dead start 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efd01241700 1 -- 172.21.15.92:0/1154865559 shutdown_connections 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efd01241700 1 -- 172.21.15.92:0/1154865559 >> 172.21.15.92:0/1154865559 conn(0x7efcfc0c9860 msgr2=0x7efcfc0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efd01241700 1 -- 172.21.15.92:0/1154865559 shutdown_connections 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.092+0000 7efd01241700 1 -- 172.21.15.92:0/1154865559 wait complete. 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.093+0000 7efd01241700 1 Processor -- start 2023-10-06T03:10:04.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.093+0000 7efd01241700 1 -- start start 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efd01241700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8004810 0x7efcfc0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efd01241700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 0x7efcfc0c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efd01241700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc0c7e70 0x7efcfc0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efd01241700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0c8850 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efd01241700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0b0440 con 0x7efce8004810 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efd01241700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0c6d70 con 0x7efcfc0c7e70 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 0x7efcfc0c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 0x7efcfc0c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45328/0 (socket says 172.21.15.92:45328) 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 -- 172.21.15.92:0/572326350 learned_addr learned my addr 172.21.15.92:0/572326350 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:04.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcfb59e700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc0c7e70 0x7efcfc0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcfad9d700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8004810 0x7efcfc0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 -- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc0c7e70 msgr2=0x7efcfc0c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc0c7e70 0x7efcfc0c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 -- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8004810 msgr2=0x7efcfc0c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8004810 0x7efcfc0c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.094+0000 7efcf259c700 1 -- 172.21.15.92:0/572326350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0c6a90 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.095+0000 7efcf259c700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 0x7efcfc0c6420 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7efce4009680 tx=0x7efce401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.095+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4023070 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.095+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce400e9e0 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.095+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc0afdf0 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.095+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4006e50 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.095+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc14ca90 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.096+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc0b05f0 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.096+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efce4013680 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.096+0000 7efcf37fe700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efcdc067320 0x7efcdc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.096+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efce4086300 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.096+0000 7efcfad9d700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efcdc067320 0x7efcdc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.097+0000 7efcfad9d700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efcdc067320 0x7efcdc0697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efcec000e50 tx=0x7efcfc0cbb20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.101+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcfc0b05f0 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.285+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcfc0b05f0 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.286+0000 7efcf37fe700 1 -- 172.21.15.92:0/572326350 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+16880 (secure 0 0 0) 0x7efcfc0b05f0 con 0x7efcfc0c5fb0 2023-10-06T03:10:04.289 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:04.289 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":167,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:01.563708+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:04.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efcdc067320 msgr2=0x7efcdc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efcdc067320 0x7efcdc0697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efcec000e50 tx=0x7efcfc0cbb20 comp rx=0 tx=0).stop 2023-10-06T03:10:04.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 msgr2=0x7efcfc0c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 0x7efcfc0c6420 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7efce4009680 tx=0x7efce401a000 comp rx=0 tx=0).stop 2023-10-06T03:10:04.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 shutdown_connections 2023-10-06T03:10:04.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efcdc067320 0x7efcdc0697d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcfc0c7e70 0x7efcfc0c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efce8004810 0x7efcfc0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 --2- 172.21.15.92:0/572326350 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efcfc0c5fb0 0x7efcfc0c6420 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 >> 172.21.15.92:0/572326350 conn(0x7efcfc0c9860 msgr2=0x7efcfc14ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:04.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 shutdown_connections 2023-10-06T03:10:04.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.288+0000 7efd01241700 1 -- 172.21.15.92:0/572326350 wait complete. 2023-10-06T03:10:04.303 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:10:04.303 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:04.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.415+0000 7ff1bb9dc700 1 Processor -- start 2023-10-06T03:10:04.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.416+0000 7ff1bb9dc700 1 -- start start 2023-10-06T03:10:04.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.416+0000 7ff1bb9dc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff1b4159c60 0x7ff1b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.416+0000 7ff1bb9dc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff1b415db10 0x7ff1b4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.416+0000 7ff1bb9dc700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b415a160 con 0x7ff1b4159890 2023-10-06T03:10:04.420 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.416+0000 7ff1bb9dc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b415df50 con 0x7ff1b4159c60 2023-10-06T03:10:04.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.416+0000 7ff1bb9dc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b40d13a0 con 0x7ff1b415db10 2023-10-06T03:10:04.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff1b4159c60 0x7ff1b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff1b4159c60 0x7ff1b415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45336/0 (socket says 172.21.15.92:45336) 2023-10-06T03:10:04.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 -- 172.21.15.92:0/250996443 learned_addr learned my addr 172.21.15.92:0/250996443 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b9f79700 1 --2- 172.21.15.92:0/250996443 >> v2:172.21.15.136:3300/0 conn(0x7ff1b415db10 0x7ff1b4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 -- 172.21.15.92:0/250996443 >> v2:172.21.15.136:3300/0 conn(0x7ff1b415db10 msgr2=0x7ff1b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 --2- 172.21.15.92:0/250996443 >> v2:172.21.15.136:3300/0 conn(0x7ff1b415db10 0x7ff1b4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 -- 172.21.15.92:0/250996443 >> v1:172.21.15.136:6789/0 conn(0x7ff1b4159890 legacy=0x7ff1b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 -- 172.21.15.92:0/250996443 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b415e140 con 0x7ff1b4159c60 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1b8f77700 1 --2- 172.21.15.92:0/250996443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b415d5d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff1b406a890 tx=0x7ff1a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67356c122bf954f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1abfff700 1 -- 172.21.15.92:0/250996443 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400b860 con 0x7ff1b4159c60 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1abfff700 1 -- 172.21.15.92:0/250996443 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a400b9c0 con 0x7ff1b4159c60 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.417+0000 7ff1abfff700 1 -- 172.21.15.92:0/250996443 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400bd60 con 0x7ff1b4159c60 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/250996443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 msgr2=0x7ff1b415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/250996443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b415d5d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff1b406a890 tx=0x7ff1a4009250 comp rx=0 tx=0).stop 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/250996443 shutdown_connections 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/250996443 >> v2:172.21.15.136:3300/0 conn(0x7ff1b415db10 0x7ff1b4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/250996443 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b415d5d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/250996443 >> 172.21.15.92:0/250996443 conn(0x7ff1b4068e20 msgr2=0x7ff1b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:04.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/250996443 shutdown_connections 2023-10-06T03:10:04.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.418+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/250996443 wait complete. 2023-10-06T03:10:04.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.419+0000 7ff1bb9dc700 1 Processor -- start 2023-10-06T03:10:04.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.419+0000 7ff1bb9dc700 1 -- start start 2023-10-06T03:10:04.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1bb9dc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b4159890 0x7ff1b4140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1bb9dc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1bb9dc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1b415db10 0x7ff1b40b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1bb9dc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4141300 con 0x7ff1b4159c60 2023-10-06T03:10:04.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1bb9dc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40cd6d0 con 0x7ff1b4159890 2023-10-06T03:10:04.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1bb9dc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40cd9b0 con 0x7ff1b415db10 2023-10-06T03:10:04.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45352/0 (socket says 172.21.15.92:45352) 2023-10-06T03:10:04.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 -- 172.21.15.92:0/1345976340 learned_addr learned my addr 172.21.15.92:0/1345976340 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:04.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b9778700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b4159890 0x7ff1b4140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 -- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1b415db10 msgr2=0x7ff1b40b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1b415db10 0x7ff1b40b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 -- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b4159890 msgr2=0x7ff1b4140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b4159890 0x7ff1b4140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 -- 172.21.15.92:0/1345976340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40094f0 con 0x7ff1b4159c60 2023-10-06T03:10:04.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.420+0000 7ff1b8f77700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b40b3590 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4070e20 tx=0x7ff1a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.421+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4011e60 con 0x7ff1b4159c60 2023-10-06T03:10:04.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.421+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a401a630 con 0x7ff1b4159c60 2023-10-06T03:10:04.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.421+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b40cdc90 con 0x7ff1b4159c60 2023-10-06T03:10:04.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.421+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a401a9f0 con 0x7ff1b4159c60 2023-10-06T03:10:04.434 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.421+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b40ce250 con 0x7ff1b4159c60 2023-10-06T03:10:04.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.421+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b40ce440 con 0x7ff1b4159c60 2023-10-06T03:10:04.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.422+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff1a402d440 con 0x7ff1b4159c60 2023-10-06T03:10:04.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.422+0000 7ff1a97fa700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff198067320 0x7ff1980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.422+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1a4086aa0 con 0x7ff1b4159c60 2023-10-06T03:10:04.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.422+0000 7ff1b9778700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff198067320 0x7ff1980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.423+0000 7ff1b9778700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff198067320 0x7ff1980697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0001760 tx=0x7ff1b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.426+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1a40535d0 con 0x7ff1b4159c60 2023-10-06T03:10:04.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.606+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1b404c3c0 con 0x7ff198067320 2023-10-06T03:10:04.616 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.614+0000 7ff1a97fa700 1 -- 172.21.15.92:0/1345976340 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300724 (secure 0 0 0) 0x7ff1b404c3c0 con 0x7ff198067320 2023-10-06T03:10:04.620 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.617+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff198067320 msgr2=0x7ff1980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.617+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff198067320 0x7ff1980697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0001760 tx=0x7ff1b0009040 comp rx=0 tx=0).stop 2023-10-06T03:10:04.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.617+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 msgr2=0x7ff1b40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.617+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b40b3590 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4070e20 tx=0x7ff1a4017000 comp rx=0 tx=0).stop 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 shutdown_connections 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff198067320 0x7ff1980697d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1b415db10 0x7ff1b40b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b4159890 0x7ff1b4140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 --2- 172.21.15.92:0/1345976340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff1b4159c60 0x7ff1b40b3590 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 >> 172.21.15.92:0/1345976340 conn(0x7ff1b4068e20 msgr2=0x7ff1b4157230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 shutdown_connections 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.618+0000 7ff1bb9dc700 1 -- 172.21.15.92:0/1345976340 wait complete. 2023-10-06T03:10:04.623 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:04.631 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2023-10-06T03:10:03.284861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428464,"kb_used_data":17392,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546256,"statfs":{"total":773094113280,"available":770607366144,"internally_reserved":0,"allocated":17809408,"data_stored":9013788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.728766"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564434+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:41.356940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558240+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:12.066574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564572+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:07.780193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558510+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:59.104467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.563019+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:55.575340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558159+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:16.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558461+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:43.377430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.561578+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:11.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560088+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:43.891167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.559871+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:52.355890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560216+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:04.162723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560206+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:51.176383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562265+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:18.735651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562359+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:44.646495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562356+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:02.816419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.559839+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:53.625688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560342+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:18.997297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562848+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:21.235383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562355+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:38.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560936+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:34.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.561794+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:59.878285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558240+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:12.315062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564346+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:50.501106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560121+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:58.269921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560088+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.846420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.928374+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:09:55.928374+0000","last_peered":"2023-10-06T03:09:55.928374+0000","last_clean":"2023-10-06T03:09:55.928374+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:09:55.928374+0000","last_undegraded":"2023-10-06T03:09:55.928374+0000","last_fullsized":"2023-10-06T03:09:55.928374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558237+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:50.502006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.517244+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:09:55.517244+0000","last_peered":"2023-10-06T03:09:55.517244+0000","last_clean":"2023-10-06T03:09:55.517244+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:09:55.517244+0000","last_undegraded":"2023-10-06T03:09:55.517244+0000","last_fullsized":"2023-10-06T03:09:55.517244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564816+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:04.265844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970468+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:09:55.970468+0000","last_peered":"2023-10-06T03:09:55.970468+0000","last_clean":"2023-10-06T03:09:55.970468+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:09:55.970468+0000","last_undegraded":"2023-10-06T03:09:55.970468+0000","last_fullsized":"2023-10-06T03:09:55.970468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560161+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:22.517573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970466+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:09:55.970466+0000","last_peered":"2023-10-06T03:09:55.970466+0000","last_clean":"2023-10-06T03:09:55.970466+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:09:55.970466+0000","last_undegraded":"2023-10-06T03:09:55.970466+0000","last_fullsized":"2023-10-06T03:09:55.970466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564623+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:20.417515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":284,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970488+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:09:55.970488+0000","last_peered":"2023-10-06T03:09:55.970488+0000","last_clean":"2023-10-06T03:09:55.970488+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:09:55.970488+0000","last_undegraded":"2023-10-06T03:09:55.970488+0000","last_fullsized":"2023-10-06T03:09:55.970488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":147,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.518640+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:09:55.518640+0000","last_peered":"2023-10-06T03:09:55.518640+0000","last_clean":"2023-10-06T03:09:55.518640+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:09:55.518640+0000","last_undegraded":"2023-10-06T03:09:55.518640+0000","last_fullsized":"2023-10-06T03:09:55.518640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.561982+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:11.583823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970006+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:09:55.970006+0000","last_peered":"2023-10-06T03:09:55.970006+0000","last_clean":"2023-10-06T03:09:55.970006+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:09:55.970006+0000","last_undegraded":"2023-10-06T03:09:55.970006+0000","last_fullsized":"2023-10-06T03:09:55.970006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558832+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:53.568443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":147,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.518622+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:09:55.518622+0000","last_peered":"2023-10-06T03:09:55.518622+0000","last_clean":"2023-10-06T03:09:55.518622+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:09:55.518622+0000","last_undegraded":"2023-10-06T03:09:55.518622+0000","last_fullsized":"2023-10-06T03:09:55.518622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562399+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:23.001726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.517262+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:09:55.517262+0000","last_peered":"2023-10-06T03:09:55.517262+0000","last_clean":"2023-10-06T03:09:55.517262+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:09:55.517262+0000","last_undegraded":"2023-10-06T03:09:55.517262+0000","last_fullsized":"2023-10-06T03:09:55.517262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1564881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.625}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.625},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.311},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.27000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.378},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.439},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.46999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303996,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067844,"statfs":{"total":96636764160,"available":96325472256,"internally_reserved":0,"allocated":2674688,"data_stored":1571039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30399999999999999,"15min":0.31},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.315},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32600000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.51100000000000001,"15min":0.64100000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"last":0.26100000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.34100000000000003},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.20899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.61699999999999999},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.623}]},{"osd":6,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.55100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.26000000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.495},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":974513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.315}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.503},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.372},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.316},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.42099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:04.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:04.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.784+0000 7f4ee8611700 1 Processor -- start 2023-10-06T03:10:04.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.784+0000 7f4ee8611700 1 -- start start 2023-10-06T03:10:04.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.785+0000 7f4ee8611700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4ee0068550 0x7f4ee0068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.785+0000 7f4ee8611700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ee015e010 0x7f4ee006f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.785+0000 7f4ee8611700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee006ec70 con 0x7f4ee006f640 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.785+0000 7f4ee8611700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee00b3430 con 0x7f4ee015e010 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.785+0000 7f4ee8611700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee00b2e70 con 0x7f4ee0068550 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.785+0000 7f4ee5bac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ee015e010 0x7f4ee006f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4ee015e010 0x7f4ee006f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45366/0 (socket says 172.21.15.92:45366) 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 -- 172.21.15.92:0/2421690513 learned_addr learned my addr 172.21.15.92:0/2421690513 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee63ad700 1 --2- 172.21.15.92:0/2421690513 >> v2:172.21.15.136:3300/0 conn(0x7f4ee0068550 0x7f4ee0068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 -- 172.21.15.92:0/2421690513 >> v2:172.21.15.136:3300/0 conn(0x7f4ee0068550 msgr2=0x7f4ee0068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 --2- 172.21.15.92:0/2421690513 >> v2:172.21.15.136:3300/0 conn(0x7f4ee0068550 0x7f4ee0068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 -- 172.21.15.92:0/2421690513 >> v1:172.21.15.136:6789/0 conn(0x7f4ee006f640 legacy=0x7f4ee006a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 -- 172.21.15.92:0/2421690513 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00b3150 con 0x7f4ee015e010 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee63ad700 1 --2- 172.21.15.92:0/2421690513 >> v2:172.21.15.136:3300/0 conn(0x7f4ee0068550 0x7f4ee0068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee5bac700 1 --2- 172.21.15.92:0/2421690513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee015e010 0x7f4ee006f100 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006e170 tx=0x7f4ed0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9c2a7afae28baf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee53ab700 1 -- 172.21.15.92:0/2421690513 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000b820 con 0x7f4ee015e010 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee53ab700 1 -- 172.21.15.92:0/2421690513 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed000b980 con 0x7f4ee015e010 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee53ab700 1 -- 172.21.15.92:0/2421690513 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000bd20 con 0x7f4ee015e010 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee8611700 1 -- 172.21.15.92:0/2421690513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee015e010 msgr2=0x7f4ee006f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.786+0000 7f4ee8611700 1 --2- 172.21.15.92:0/2421690513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee015e010 0x7f4ee006f100 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006e170 tx=0x7f4ed0009250 comp rx=0 tx=0).stop 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.787+0000 7f4ee8611700 1 -- 172.21.15.92:0/2421690513 shutdown_connections 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.787+0000 7f4ee8611700 1 --2- 172.21.15.92:0/2421690513 >> v2:172.21.15.136:3300/0 conn(0x7f4ee0068550 0x7f4ee0068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.787+0000 7f4ee8611700 1 --2- 172.21.15.92:0/2421690513 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee015e010 0x7f4ee006f100 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.787+0000 7f4ee8611700 1 -- 172.21.15.92:0/2421690513 >> 172.21.15.92:0/2421690513 conn(0x7f4ee00cd480 msgr2=0x7f4ee00cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.787+0000 7f4ee8611700 1 -- 172.21.15.92:0/2421690513 shutdown_connections 2023-10-06T03:10:04.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.787+0000 7f4ee8611700 1 -- 172.21.15.92:0/2421690513 wait complete. 2023-10-06T03:10:04.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 Processor -- start 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 -- start start 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 0x7f4ee015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ee006f640 0x7f4ee015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ee015e010 0x7f4ee016d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0141710 con 0x7f4ee0068550 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee015a6d0 con 0x7f4ee015e010 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.788+0000 7f4ee8611700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0159ac0 con 0x7f4ee006f640 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 0x7f4ee015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 0x7f4ee015bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45378/0 (socket says 172.21.15.92:45378) 2023-10-06T03:10:04.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 -- 172.21.15.92:0/3474261896 learned_addr learned my addr 172.21.15.92:0/3474261896 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee5bac700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ee006f640 0x7f4ee015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee6bae700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ee015e010 0x7f4ee016d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 -- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ee006f640 msgr2=0x7f4ee015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ee006f640 0x7f4ee015c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 -- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ee015e010 msgr2=0x7f4ee016d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ee015e010 0x7f4ee016d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 -- 172.21.15.92:0/3474261896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00094f0 con 0x7f4ee0068550 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee63ad700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 0x7f4ee015bef0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c830 tx=0x7f4edc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc011600 con 0x7f4ee0068550 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc011760 con 0x7f4ee0068550 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc011b20 con 0x7f4ee0068550 2023-10-06T03:10:04.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee0159da0 con 0x7f4ee0068550 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.789+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee015a360 con 0x7f4ee0068550 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.790+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee004c3c0 con 0x7f4ee0068550 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.790+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4edc018360 con 0x7f4ee0068550 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.791+0000 7f4ed67fc700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ec4067320 0x7f4ec40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.791+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4edc084ee0 con 0x7f4ee0068550 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.791+0000 7f4ee5bac700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ec4067320 0x7f4ec40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.791+0000 7f4ee5bac700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ec4067320 0x7f4ec40697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006e170 tx=0x7f4ed0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.795+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4edc051900 con 0x7f4ee0068550 2023-10-06T03:10:04.825 INFO:tasks.ceph.mds.b.smithi136.stderr:2023-10-06T03:10:04.824+0000 7f4b2182d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:10:04.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.973+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ee015a810 con 0x7f4ec4067320 2023-10-06T03:10:04.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.981+0000 7f4ed67fc700 1 -- 172.21.15.92:0/3474261896 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300724 (secure 0 0 0) 0x7f4ed0006cf0 con 0x7f4ec4067320 2023-10-06T03:10:04.986 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ec4067320 msgr2=0x7f4ec40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ec4067320 0x7f4ec40697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006e170 tx=0x7f4ed0016000 comp rx=0 tx=0).stop 2023-10-06T03:10:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 msgr2=0x7f4ee015bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:04.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 0x7f4ee015bef0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c830 tx=0x7f4edc008040 comp rx=0 tx=0).stop 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 shutdown_connections 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4ec4067320 0x7f4ec40697d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ee0068550 0x7f4ee015bef0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ee015e010 0x7f4ee016d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 --2- 172.21.15.92:0/3474261896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ee006f640 0x7f4ee015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 >> 172.21.15.92:0/3474261896 conn(0x7f4ee00cd480 msgr2=0x7f4ee0152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.984+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 shutdown_connections 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:04.985+0000 7f4ee8611700 1 -- 172.21.15.92:0/3474261896 wait complete. 2023-10-06T03:10:04.989 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:04.998 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2023-10-06T03:10:03.284861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428464,"kb_used_data":17392,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752546256,"statfs":{"total":773094113280,"available":770607366144,"internally_reserved":0,"allocated":17809408,"data_stored":9013788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.728766"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564434+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:41.356940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558240+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:06:12.066574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564572+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:07.780193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558510+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:59.104467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.563019+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:55.575340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558159+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:16.898682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558461+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:43.377430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.561578+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:11.547444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560088+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:43.891167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.559871+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:52.355890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560216+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:04.162723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560206+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:51.176383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562265+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:18.735651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562359+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:44.646495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562356+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:02.816419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.559839+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:53.625688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560342+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:18.997297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562848+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:21.235383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562355+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:38.329796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560936+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:34.041078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.561794+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:59.878285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558240+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:43:12.315062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564346+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:50.501106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560121+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:58.269921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560088+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:24.846420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":129,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.928374+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:09:55.928374+0000","last_peered":"2023-10-06T03:09:55.928374+0000","last_clean":"2023-10-06T03:09:55.928374+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:09:55.928374+0000","last_undegraded":"2023-10-06T03:09:55.928374+0000","last_fullsized":"2023-10-06T03:09:55.928374+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558237+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:50.502006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.517244+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:09:55.517244+0000","last_peered":"2023-10-06T03:09:55.517244+0000","last_clean":"2023-10-06T03:09:55.517244+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:09:55.517244+0000","last_undegraded":"2023-10-06T03:09:55.517244+0000","last_fullsized":"2023-10-06T03:09:55.517244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564816+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:04.265844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970468+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:09:55.970468+0000","last_peered":"2023-10-06T03:09:55.970468+0000","last_clean":"2023-10-06T03:09:55.970468+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:09:55.970468+0000","last_undegraded":"2023-10-06T03:09:55.970468+0000","last_fullsized":"2023-10-06T03:09:55.970468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.560161+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:22.517573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970466+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:09:55.970466+0000","last_peered":"2023-10-06T03:09:55.970466+0000","last_clean":"2023-10-06T03:09:55.970466+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:09:55.970466+0000","last_undegraded":"2023-10-06T03:09:55.970466+0000","last_fullsized":"2023-10-06T03:09:55.970466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.564623+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:20.417515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":284,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970488+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:09:55.970488+0000","last_peered":"2023-10-06T03:09:55.970488+0000","last_clean":"2023-10-06T03:09:55.970488+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:09:55.970488+0000","last_undegraded":"2023-10-06T03:09:55.970488+0000","last_fullsized":"2023-10-06T03:09:55.970488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":147,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.518640+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:09:55.518640+0000","last_peered":"2023-10-06T03:09:55.518640+0000","last_clean":"2023-10-06T03:09:55.518640+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:09:55.518640+0000","last_undegraded":"2023-10-06T03:09:55.518640+0000","last_fullsized":"2023-10-06T03:09:55.518640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.561982+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:11.583823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.970006+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:09:55.970006+0000","last_peered":"2023-10-06T03:09:55.970006+0000","last_clean":"2023-10-06T03:09:55.970006+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:09:55.970006+0000","last_undegraded":"2023-10-06T03:09:55.970006+0000","last_fullsized":"2023-10-06T03:09:55.970006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.558832+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:53.568443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":147,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.518622+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:09:55.518622+0000","last_peered":"2023-10-06T03:09:55.518622+0000","last_clean":"2023-10-06T03:09:55.518622+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:09:55.518622+0000","last_undegraded":"2023-10-06T03:09:55.518622+0000","last_fullsized":"2023-10-06T03:09:55.518622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:09:59.555617+0000","last_change":"2023-10-06T03:09:59.555617+0000","last_active":"2023-10-06T03:09:59.555617+0000","last_peered":"2023-10-06T03:09:59.555617+0000","last_clean":"2023-10-06T03:09:59.555617+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:59.555617+0000","last_undegraded":"2023-10-06T03:09:59.555617+0000","last_fullsized":"2023-10-06T03:09:59.555617+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:09:58.550323+0000","last_change":"2023-10-06T03:09:58.562399+0000","last_active":"2023-10-06T03:09:58.550323+0000","last_peered":"2023-10-06T03:09:58.550323+0000","last_clean":"2023-10-06T03:09:58.550323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:09:58.550323+0000","last_undegraded":"2023-10-06T03:09:58.550323+0000","last_fullsized":"2023-10-06T03:09:58.550323+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:23.001726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-06T03:09:55.517262+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:09:55.517262+0000","last_peered":"2023-10-06T03:09:55.517262+0000","last_clean":"2023-10-06T03:09:55.517262+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:09:55.517262+0000","last_undegraded":"2023-10-06T03:09:55.517262+0000","last_fullsized":"2023-10-06T03:09:55.517262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303984,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067856,"statfs":{"total":96636764160,"available":96325484544,"internally_reserved":0,"allocated":2662400,"data_stored":1564881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.625}]}]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.625},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.311},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.27000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.38800000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.378},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.439},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.46999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574953,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303996,"kb_used_data":2612,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067844,"statfs":{"total":96636764160,"available":96325472256,"internally_reserved":0,"allocated":2674688,"data_stored":1571039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30399999999999999,"15min":0.31},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.315},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32600000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.51100000000000001,"15min":0.64100000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"last":0.26100000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.34100000000000003},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.20899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.61699999999999999},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.623}]},{"osd":6,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.55100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.308}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.26000000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.436},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.495},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303404,"kb_used_data":2020,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068436,"statfs":{"total":96636764160,"available":96326078464,"internally_reserved":0,"allocated":2068480,"data_stored":974513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.315}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":980671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.503},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.372},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.316},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.42099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:05.005 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.151+0000 7f9e38bf6700 1 Processor -- start 2023-10-06T03:10:05.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.151+0000 7f9e38bf6700 1 -- start start 2023-10-06T03:10:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e38bf6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e34159420 0x7f9e340b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e38bf6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e3415d230 0x7f9e3415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e38bf6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3415dbd0 con 0x7f9e341597f0 2023-10-06T03:10:05.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e38bf6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e340b2980 con 0x7f9e3415d230 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e38bf6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340b11d0 con 0x7f9e34159420 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e3415d230 0x7f9e3415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e3415d230 0x7f9e3415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45382/0 (socket says 172.21.15.92:45382) 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 -- 172.21.15.92:0/379905914 learned_addr learned my addr 172.21.15.92:0/379905914 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e337fe700 1 --2- 172.21.15.92:0/379905914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e34159420 0x7f9e340b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 -- 172.21.15.92:0/379905914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e34159420 msgr2=0x7f9e340b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 --2- 172.21.15.92:0/379905914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e34159420 0x7f9e340b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 -- 172.21.15.92:0/379905914 >> v1:172.21.15.136:6789/0 conn(0x7f9e341597f0 legacy=0x7f9e340b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.152+0000 7f9e33fff700 1 -- 172.21.15.92:0/379905914 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e340b14b0 con 0x7f9e3415d230 2023-10-06T03:10:05.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e33fff700 1 --2- 172.21.15.92:0/379905914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e3415d230 0x7f9e3415d690 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009f20 tx=0x7f9e2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb4dcca2d4435a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e327fc700 1 -- 172.21.15.92:0/379905914 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c00e070 con 0x7f9e3415d230 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e327fc700 1 -- 172.21.15.92:0/379905914 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c00b9d0 con 0x7f9e3415d230 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e327fc700 1 -- 172.21.15.92:0/379905914 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0092e0 con 0x7f9e3415d230 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 -- 172.21.15.92:0/379905914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e3415d230 msgr2=0x7f9e3415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/379905914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e3415d230 0x7f9e3415d690 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009f20 tx=0x7f9e2c009840 comp rx=0 tx=0).stop 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 -- 172.21.15.92:0/379905914 shutdown_connections 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/379905914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e3415d230 0x7f9e3415d690 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/379905914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e34159420 0x7f9e340b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 -- 172.21.15.92:0/379905914 >> 172.21.15.92:0/379905914 conn(0x7f9e340d0660 msgr2=0x7f9e340d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:05.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 -- 172.21.15.92:0/379905914 shutdown_connections 2023-10-06T03:10:05.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.153+0000 7f9e38bf6700 1 -- 172.21.15.92:0/379905914 wait complete. 2023-10-06T03:10:05.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 Processor -- start 2023-10-06T03:10:05.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 -- start start 2023-10-06T03:10:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 0x7f9e340ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e341597f0 0x7f9e340cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e3415d230 0x7f9e3416a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340caf70 con 0x7f9e34159420 2023-10-06T03:10:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340cb250 con 0x7f9e3415d230 2023-10-06T03:10:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e38bf6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e340cb530 con 0x7f9e341597f0 2023-10-06T03:10:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 0x7f9e340ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e337fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 0x7f9e340ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45394/0 (socket says 172.21.15.92:45394) 2023-10-06T03:10:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.155+0000 7f9e337fe700 1 -- 172.21.15.92:0/1760042858 learned_addr learned my addr 172.21.15.92:0/1760042858 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e33fff700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e3415d230 0x7f9e3416a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e32ffd700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e341597f0 0x7f9e340cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e337fe700 1 -- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e341597f0 msgr2=0x7f9e340cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e337fe700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e341597f0 0x7f9e340cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e337fe700 1 -- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e3415d230 msgr2=0x7f9e3416a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e337fe700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e3415d230 0x7f9e3416a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e337fe700 1 -- 172.21.15.92:0/1760042858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c009ae0 con 0x7f9e34159420 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e32ffd700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e341597f0 0x7f9e340cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e337fe700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 0x7f9e340ca990 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400ec80 tx=0x7f9e240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400c850 con 0x7f9e34159420 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2400c9b0 con 0x7f9e34159420 2023-10-06T03:10:05.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e340cb810 con 0x7f9e34159420 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400cd50 con 0x7f9e34159420 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.156+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e34155920 con 0x7f9e34159420 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.157+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e340cbe60 con 0x7f9e34159420 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.157+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9e24013680 con 0x7f9e34159420 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.158+0000 7f9e1b7fe700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e20067320 0x7f9e200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.158+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9e24085090 con 0x7f9e34159420 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.158+0000 7f9e32ffd700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e20067320 0x7f9e200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.158+0000 7f9e32ffd700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e20067320 0x7f9e200697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9e28007fd0 tx=0x7f9e28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.162+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e24051bc0 con 0x7f9e34159420 2023-10-06T03:10:05.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.340+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e3404c3c0 con 0x7f9e20067320 2023-10-06T03:10:05.350 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.348+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/1760042858 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311707 (secure 0 0 0) 0x7f9e3404c3c0 con 0x7f9e20067320 2023-10-06T03:10:05.354 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:05.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e20067320 msgr2=0x7f9e200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e20067320 0x7f9e200697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9e28007fd0 tx=0x7f9e28009040 comp rx=0 tx=0).stop 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 msgr2=0x7f9e340ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 0x7f9e340ca990 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400ec80 tx=0x7f9e240091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 shutdown_connections 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e20067320 0x7f9e200697d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e34159420 0x7f9e340ca990 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e3415d230 0x7f9e3416a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 --2- 172.21.15.92:0/1760042858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e341597f0 0x7f9e340cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.351+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 >> 172.21.15.92:0/1760042858 conn(0x7f9e340d0660 msgr2=0x7f9e341564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:05.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.352+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 shutdown_connections 2023-10-06T03:10:05.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.352+0000 7f9e38bf6700 1 -- 172.21.15.92:0/1760042858 wait complete. 2023-10-06T03:10:05.358 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:05.365 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2023-10-06T03:10:05.285151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428840,"kb_used_data":17768,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752545880,"statfs":{"total":773094113280,"available":770606981120,"internally_reserved":0,"allocated":18194432,"data_stored":9193700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000978"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592260+0000","last_change":"2023-10-06T03:10:00.590921+0000","last_active":"2023-10-06T03:10:00.592260+0000","last_peered":"2023-10-06T03:10:00.592260+0000","last_clean":"2023-10-06T03:10:00.592260+0000","last_became_active":"2023-10-06T03:10:00.590562+0000","last_became_peered":"2023-10-06T03:10:00.590562+0000","last_unstale":"2023-10-06T03:10:00.592260+0000","last_undegraded":"2023-10-06T03:10:00.592260+0000","last_fullsized":"2023-10-06T03:10:00.592260+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:09.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579997+0000","last_change":"2023-10-06T03:10:00.576176+0000","last_active":"2023-10-06T03:10:00.579997+0000","last_peered":"2023-10-06T03:10:00.579997+0000","last_clean":"2023-10-06T03:10:00.579997+0000","last_became_active":"2023-10-06T03:10:00.575892+0000","last_became_peered":"2023-10-06T03:10:00.575892+0000","last_unstale":"2023-10-06T03:10:00.579997+0000","last_undegraded":"2023-10-06T03:10:00.579997+0000","last_fullsized":"2023-10-06T03:10:00.579997+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.584497+0000","last_change":"2023-10-06T03:10:00.579899+0000","last_active":"2023-10-06T03:10:00.584497+0000","last_peered":"2023-10-06T03:10:00.584497+0000","last_clean":"2023-10-06T03:10:00.584497+0000","last_became_active":"2023-10-06T03:10:00.579648+0000","last_became_peered":"2023-10-06T03:10:00.579648+0000","last_unstale":"2023-10-06T03:10:00.584497+0000","last_undegraded":"2023-10-06T03:10:00.584497+0000","last_fullsized":"2023-10-06T03:10:00.584497+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:35.670624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585148+0000","last_change":"2023-10-06T03:10:00.576866+0000","last_active":"2023-10-06T03:10:00.585148+0000","last_peered":"2023-10-06T03:10:00.585148+0000","last_clean":"2023-10-06T03:10:00.585148+0000","last_became_active":"2023-10-06T03:10:00.576583+0000","last_became_peered":"2023-10-06T03:10:00.576583+0000","last_unstale":"2023-10-06T03:10:00.585148+0000","last_undegraded":"2023-10-06T03:10:00.585148+0000","last_fullsized":"2023-10-06T03:10:00.585148+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:25.010008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591219+0000","last_change":"2023-10-06T03:10:00.585448+0000","last_active":"2023-10-06T03:10:00.591219+0000","last_peered":"2023-10-06T03:10:00.591219+0000","last_clean":"2023-10-06T03:10:00.591219+0000","last_became_active":"2023-10-06T03:10:00.584829+0000","last_became_peered":"2023-10-06T03:10:00.584829+0000","last_unstale":"2023-10-06T03:10:00.591219+0000","last_undegraded":"2023-10-06T03:10:00.591219+0000","last_fullsized":"2023-10-06T03:10:00.591219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.911735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593134+0000","last_change":"2023-10-06T03:10:00.592217+0000","last_active":"2023-10-06T03:10:00.593134+0000","last_peered":"2023-10-06T03:10:00.593134+0000","last_clean":"2023-10-06T03:10:00.593134+0000","last_became_active":"2023-10-06T03:10:00.591948+0000","last_became_peered":"2023-10-06T03:10:00.591948+0000","last_unstale":"2023-10-06T03:10:00.593134+0000","last_undegraded":"2023-10-06T03:10:00.593134+0000","last_fullsized":"2023-10-06T03:10:00.593134+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:24.007020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585916+0000","last_change":"2023-10-06T03:10:00.582650+0000","last_active":"2023-10-06T03:10:00.585916+0000","last_peered":"2023-10-06T03:10:00.585916+0000","last_clean":"2023-10-06T03:10:00.585916+0000","last_became_active":"2023-10-06T03:10:00.581814+0000","last_became_peered":"2023-10-06T03:10:00.581814+0000","last_unstale":"2023-10-06T03:10:00.585916+0000","last_undegraded":"2023-10-06T03:10:00.585916+0000","last_fullsized":"2023-10-06T03:10:00.585916+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593113+0000","last_change":"2023-10-06T03:10:00.590222+0000","last_active":"2023-10-06T03:10:00.593113+0000","last_peered":"2023-10-06T03:10:00.593113+0000","last_clean":"2023-10-06T03:10:00.593113+0000","last_became_active":"2023-10-06T03:10:00.589982+0000","last_became_peered":"2023-10-06T03:10:00.589982+0000","last_unstale":"2023-10-06T03:10:00.593113+0000","last_undegraded":"2023-10-06T03:10:00.593113+0000","last_fullsized":"2023-10-06T03:10:00.593113+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:42.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579402+0000","last_change":"2023-10-06T03:10:00.577494+0000","last_active":"2023-10-06T03:10:00.579402+0000","last_peered":"2023-10-06T03:10:00.579402+0000","last_clean":"2023-10-06T03:10:00.579402+0000","last_became_active":"2023-10-06T03:10:00.577250+0000","last_became_peered":"2023-10-06T03:10:00.577250+0000","last_unstale":"2023-10-06T03:10:00.579402+0000","last_undegraded":"2023-10-06T03:10:00.579402+0000","last_fullsized":"2023-10-06T03:10:00.579402+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:59.275214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585983+0000","last_change":"2023-10-06T03:10:00.582063+0000","last_active":"2023-10-06T03:10:00.585983+0000","last_peered":"2023-10-06T03:10:00.585983+0000","last_clean":"2023-10-06T03:10:00.585983+0000","last_became_active":"2023-10-06T03:10:00.581786+0000","last_became_peered":"2023-10-06T03:10:00.581786+0000","last_unstale":"2023-10-06T03:10:00.585983+0000","last_undegraded":"2023-10-06T03:10:00.585983+0000","last_fullsized":"2023-10-06T03:10:00.585983+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:06.937724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586746+0000","last_change":"2023-10-06T03:10:00.583742+0000","last_active":"2023-10-06T03:10:00.586746+0000","last_peered":"2023-10-06T03:10:00.586746+0000","last_clean":"2023-10-06T03:10:00.586746+0000","last_became_active":"2023-10-06T03:10:00.583140+0000","last_became_peered":"2023-10-06T03:10:00.583140+0000","last_unstale":"2023-10-06T03:10:00.586746+0000","last_undegraded":"2023-10-06T03:10:00.586746+0000","last_fullsized":"2023-10-06T03:10:00.586746+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:26.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594983+0000","last_change":"2023-10-06T03:10:00.590922+0000","last_active":"2023-10-06T03:10:00.594983+0000","last_peered":"2023-10-06T03:10:00.594983+0000","last_clean":"2023-10-06T03:10:00.594983+0000","last_became_active":"2023-10-06T03:10:00.586978+0000","last_became_peered":"2023-10-06T03:10:00.586978+0000","last_unstale":"2023-10-06T03:10:00.594983+0000","last_undegraded":"2023-10-06T03:10:00.594983+0000","last_fullsized":"2023-10-06T03:10:00.594983+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:39.335090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592425+0000","last_change":"2023-10-06T03:10:00.590023+0000","last_active":"2023-10-06T03:10:00.592425+0000","last_peered":"2023-10-06T03:10:00.592425+0000","last_clean":"2023-10-06T03:10:00.592425+0000","last_became_active":"2023-10-06T03:10:00.589773+0000","last_became_peered":"2023-10-06T03:10:00.589773+0000","last_unstale":"2023-10-06T03:10:00.592425+0000","last_undegraded":"2023-10-06T03:10:00.592425+0000","last_fullsized":"2023-10-06T03:10:00.592425+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:39.579557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594945+0000","last_change":"2023-10-06T03:10:00.590602+0000","last_active":"2023-10-06T03:10:00.594945+0000","last_peered":"2023-10-06T03:10:00.594945+0000","last_clean":"2023-10-06T03:10:00.594945+0000","last_became_active":"2023-10-06T03:10:00.586819+0000","last_became_peered":"2023-10-06T03:10:00.586819+0000","last_unstale":"2023-10-06T03:10:00.594945+0000","last_undegraded":"2023-10-06T03:10:00.594945+0000","last_fullsized":"2023-10-06T03:10:00.594945+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:00.109760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585396+0000","last_change":"2023-10-06T03:10:00.583053+0000","last_active":"2023-10-06T03:10:00.585396+0000","last_peered":"2023-10-06T03:10:00.585396+0000","last_clean":"2023-10-06T03:10:00.585396+0000","last_became_active":"2023-10-06T03:10:00.582801+0000","last_became_peered":"2023-10-06T03:10:00.582801+0000","last_unstale":"2023-10-06T03:10:00.585396+0000","last_undegraded":"2023-10-06T03:10:00.585396+0000","last_fullsized":"2023-10-06T03:10:00.585396+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:02.664992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.581141+0000","last_change":"2023-10-06T03:10:00.577913+0000","last_active":"2023-10-06T03:10:00.581141+0000","last_peered":"2023-10-06T03:10:00.581141+0000","last_clean":"2023-10-06T03:10:00.581141+0000","last_became_active":"2023-10-06T03:10:00.577496+0000","last_became_peered":"2023-10-06T03:10:00.577496+0000","last_unstale":"2023-10-06T03:10:00.581141+0000","last_undegraded":"2023-10-06T03:10:00.581141+0000","last_fullsized":"2023-10-06T03:10:00.581141+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:39.267712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.598667+0000","last_change":"2023-10-06T03:10:00.597708+0000","last_active":"2023-10-06T03:10:00.598667+0000","last_peered":"2023-10-06T03:10:00.598667+0000","last_clean":"2023-10-06T03:10:00.598667+0000","last_became_active":"2023-10-06T03:10:00.597225+0000","last_became_peered":"2023-10-06T03:10:00.597225+0000","last_unstale":"2023-10-06T03:10:00.598667+0000","last_undegraded":"2023-10-06T03:10:00.598667+0000","last_fullsized":"2023-10-06T03:10:00.598667+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:24.007020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588183+0000","last_change":"2023-10-06T03:10:00.585502+0000","last_active":"2023-10-06T03:10:00.588183+0000","last_peered":"2023-10-06T03:10:00.588183+0000","last_clean":"2023-10-06T03:10:00.588183+0000","last_became_active":"2023-10-06T03:10:00.585258+0000","last_became_peered":"2023-10-06T03:10:00.585258+0000","last_unstale":"2023-10-06T03:10:00.588183+0000","last_undegraded":"2023-10-06T03:10:00.588183+0000","last_fullsized":"2023-10-06T03:10:00.588183+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591484+0000","last_change":"2023-10-06T03:10:00.588935+0000","last_active":"2023-10-06T03:10:00.591484+0000","last_peered":"2023-10-06T03:10:00.591484+0000","last_clean":"2023-10-06T03:10:00.591484+0000","last_became_active":"2023-10-06T03:10:00.588042+0000","last_became_peered":"2023-10-06T03:10:00.588042+0000","last_unstale":"2023-10-06T03:10:00.591484+0000","last_undegraded":"2023-10-06T03:10:00.591484+0000","last_fullsized":"2023-10-06T03:10:00.591484+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.581122+0000","last_change":"2023-10-06T03:10:00.576942+0000","last_active":"2023-10-06T03:10:00.581122+0000","last_peered":"2023-10-06T03:10:00.581122+0000","last_clean":"2023-10-06T03:10:00.581122+0000","last_became_active":"2023-10-06T03:10:00.576620+0000","last_became_peered":"2023-10-06T03:10:00.576620+0000","last_unstale":"2023-10-06T03:10:00.581122+0000","last_undegraded":"2023-10-06T03:10:00.581122+0000","last_fullsized":"2023-10-06T03:10:00.581122+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:22.240676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.584622+0000","last_change":"2023-10-06T03:10:00.579953+0000","last_active":"2023-10-06T03:10:00.584622+0000","last_peered":"2023-10-06T03:10:00.584622+0000","last_clean":"2023-10-06T03:10:00.584622+0000","last_became_active":"2023-10-06T03:10:00.579695+0000","last_became_peered":"2023-10-06T03:10:00.579695+0000","last_unstale":"2023-10-06T03:10:00.584622+0000","last_undegraded":"2023-10-06T03:10:00.584622+0000","last_fullsized":"2023-10-06T03:10:00.584622+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.911735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576451+0000","last_change":"2023-10-06T03:10:00.575164+0000","last_active":"2023-10-06T03:10:00.576451+0000","last_peered":"2023-10-06T03:10:00.576451+0000","last_clean":"2023-10-06T03:10:00.576451+0000","last_became_active":"2023-10-06T03:10:00.574927+0000","last_became_peered":"2023-10-06T03:10:00.574927+0000","last_unstale":"2023-10-06T03:10:00.576451+0000","last_undegraded":"2023-10-06T03:10:00.576451+0000","last_fullsized":"2023-10-06T03:10:00.576451+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.803607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.577631+0000","last_change":"2023-10-06T03:10:00.575464+0000","last_active":"2023-10-06T03:10:00.577631+0000","last_peered":"2023-10-06T03:10:00.577631+0000","last_clean":"2023-10-06T03:10:00.577631+0000","last_became_active":"2023-10-06T03:10:00.575233+0000","last_became_peered":"2023-10-06T03:10:00.575233+0000","last_unstale":"2023-10-06T03:10:00.577631+0000","last_undegraded":"2023-10-06T03:10:00.577631+0000","last_fullsized":"2023-10-06T03:10:00.577631+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:18.843464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591161+0000","last_change":"2023-10-06T03:10:00.586643+0000","last_active":"2023-10-06T03:10:00.591161+0000","last_peered":"2023-10-06T03:10:00.591161+0000","last_clean":"2023-10-06T03:10:00.591161+0000","last_became_active":"2023-10-06T03:10:00.586359+0000","last_became_peered":"2023-10-06T03:10:00.586359+0000","last_unstale":"2023-10-06T03:10:00.591161+0000","last_undegraded":"2023-10-06T03:10:00.591161+0000","last_fullsized":"2023-10-06T03:10:00.591161+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592278+0000","last_change":"2023-10-06T03:10:00.591267+0000","last_active":"2023-10-06T03:10:00.592278+0000","last_peered":"2023-10-06T03:10:00.592278+0000","last_clean":"2023-10-06T03:10:00.592278+0000","last_became_active":"2023-10-06T03:10:00.590664+0000","last_became_peered":"2023-10-06T03:10:00.590664+0000","last_unstale":"2023-10-06T03:10:00.592278+0000","last_undegraded":"2023-10-06T03:10:00.592278+0000","last_fullsized":"2023-10-06T03:10:00.592278+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591365+0000","last_change":"2023-10-06T03:10:00.589001+0000","last_active":"2023-10-06T03:10:00.591365+0000","last_peered":"2023-10-06T03:10:00.591365+0000","last_clean":"2023-10-06T03:10:00.591365+0000","last_became_active":"2023-10-06T03:10:00.588833+0000","last_became_peered":"2023-10-06T03:10:00.588833+0000","last_unstale":"2023-10-06T03:10:00.591365+0000","last_undegraded":"2023-10-06T03:10:00.591365+0000","last_fullsized":"2023-10-06T03:10:00.591365+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:55.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586764+0000","last_change":"2023-10-06T03:10:00.583945+0000","last_active":"2023-10-06T03:10:00.586764+0000","last_peered":"2023-10-06T03:10:00.586764+0000","last_clean":"2023-10-06T03:10:00.586764+0000","last_became_active":"2023-10-06T03:10:00.583483+0000","last_became_peered":"2023-10-06T03:10:00.583483+0000","last_unstale":"2023-10-06T03:10:00.586764+0000","last_undegraded":"2023-10-06T03:10:00.586764+0000","last_fullsized":"2023-10-06T03:10:00.586764+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.043746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595005+0000","last_change":"2023-10-06T03:10:00.591638+0000","last_active":"2023-10-06T03:10:00.595005+0000","last_peered":"2023-10-06T03:10:00.595005+0000","last_clean":"2023-10-06T03:10:00.595005+0000","last_became_active":"2023-10-06T03:10:00.591381+0000","last_became_peered":"2023-10-06T03:10:00.591381+0000","last_unstale":"2023-10-06T03:10:00.595005+0000","last_undegraded":"2023-10-06T03:10:00.595005+0000","last_fullsized":"2023-10-06T03:10:00.595005+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:03.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585883+0000","last_change":"2023-10-06T03:10:00.582573+0000","last_active":"2023-10-06T03:10:00.585883+0000","last_peered":"2023-10-06T03:10:00.585883+0000","last_clean":"2023-10-06T03:10:00.585883+0000","last_became_active":"2023-10-06T03:10:00.582310+0000","last_became_peered":"2023-10-06T03:10:00.582310+0000","last_unstale":"2023-10-06T03:10:00.585883+0000","last_undegraded":"2023-10-06T03:10:00.585883+0000","last_fullsized":"2023-10-06T03:10:00.585883+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:47.002388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595540+0000","last_change":"2023-10-06T03:10:00.594200+0000","last_active":"2023-10-06T03:10:00.595540+0000","last_peered":"2023-10-06T03:10:00.595540+0000","last_clean":"2023-10-06T03:10:00.595540+0000","last_became_active":"2023-10-06T03:10:00.593924+0000","last_became_peered":"2023-10-06T03:10:00.593924+0000","last_unstale":"2023-10-06T03:10:00.595540+0000","last_undegraded":"2023-10-06T03:10:00.595540+0000","last_fullsized":"2023-10-06T03:10:00.595540+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:24.816047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585699+0000","last_change":"2023-10-06T03:10:00.580666+0000","last_active":"2023-10-06T03:10:00.585699+0000","last_peered":"2023-10-06T03:10:00.585699+0000","last_clean":"2023-10-06T03:10:00.585699+0000","last_became_active":"2023-10-06T03:10:00.580418+0000","last_became_peered":"2023-10-06T03:10:00.580418+0000","last_unstale":"2023-10-06T03:10:00.585699+0000","last_undegraded":"2023-10-06T03:10:00.585699+0000","last_fullsized":"2023-10-06T03:10:00.585699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:11.143586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588450+0000","last_change":"2023-10-06T03:10:00.587824+0000","last_active":"2023-10-06T03:10:00.588450+0000","last_peered":"2023-10-06T03:10:00.588450+0000","last_clean":"2023-10-06T03:10:00.588450+0000","last_became_active":"2023-10-06T03:10:00.587522+0000","last_became_peered":"2023-10-06T03:10:00.587522+0000","last_unstale":"2023-10-06T03:10:00.588450+0000","last_undegraded":"2023-10-06T03:10:00.588450+0000","last_fullsized":"2023-10-06T03:10:00.588450+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:22.796834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591074+0000","last_change":"2023-10-06T03:10:00.587669+0000","last_active":"2023-10-06T03:10:00.591074+0000","last_peered":"2023-10-06T03:10:00.591074+0000","last_clean":"2023-10-06T03:10:00.591074+0000","last_became_active":"2023-10-06T03:10:00.587393+0000","last_became_peered":"2023-10-06T03:10:00.587393+0000","last_unstale":"2023-10-06T03:10:00.591074+0000","last_undegraded":"2023-10-06T03:10:00.591074+0000","last_fullsized":"2023-10-06T03:10:00.591074+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:06.249936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579297+0000","last_change":"2023-10-06T03:09:59.579556+0000","last_active":"2023-10-06T03:10:00.579297+0000","last_peered":"2023-10-06T03:10:00.579297+0000","last_clean":"2023-10-06T03:10:00.579297+0000","last_became_active":"2023-10-06T03:09:59.579270+0000","last_became_peered":"2023-10-06T03:09:59.579270+0000","last_unstale":"2023-10-06T03:10:00.579297+0000","last_undegraded":"2023-10-06T03:10:00.579297+0000","last_fullsized":"2023-10-06T03:10:00.579297+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:04.820008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579434+0000","last_change":"2023-10-06T03:10:00.576176+0000","last_active":"2023-10-06T03:10:00.579434+0000","last_peered":"2023-10-06T03:10:00.579434+0000","last_clean":"2023-10-06T03:10:00.579434+0000","last_became_active":"2023-10-06T03:10:00.575676+0000","last_became_peered":"2023-10-06T03:10:00.575676+0000","last_unstale":"2023-10-06T03:10:00.579434+0000","last_undegraded":"2023-10-06T03:10:00.579434+0000","last_fullsized":"2023-10-06T03:10:00.579434+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:08.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570781+0000","last_change":"2023-10-06T03:09:59.586249+0000","last_active":"2023-10-06T03:10:00.570781+0000","last_peered":"2023-10-06T03:10:00.570781+0000","last_clean":"2023-10-06T03:10:00.570781+0000","last_became_active":"2023-10-06T03:09:59.585984+0000","last_became_peered":"2023-10-06T03:09:59.585984+0000","last_unstale":"2023-10-06T03:10:00.570781+0000","last_undegraded":"2023-10-06T03:10:00.570781+0000","last_fullsized":"2023-10-06T03:10:00.570781+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:59.104467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585567+0000","last_change":"2023-10-06T03:10:00.581602+0000","last_active":"2023-10-06T03:10:00.585567+0000","last_peered":"2023-10-06T03:10:00.585567+0000","last_clean":"2023-10-06T03:10:00.585567+0000","last_became_active":"2023-10-06T03:10:00.581253+0000","last_became_peered":"2023-10-06T03:10:00.581253+0000","last_unstale":"2023-10-06T03:10:00.585567+0000","last_undegraded":"2023-10-06T03:10:00.585567+0000","last_fullsized":"2023-10-06T03:10:00.585567+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:09.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"166'3","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.589720+0000","last_change":"2023-10-06T03:09:59.579902+0000","last_active":"2023-10-06T03:10:00.589720+0000","last_peered":"2023-10-06T03:10:00.589720+0000","last_clean":"2023-10-06T03:10:00.589720+0000","last_became_active":"2023-10-06T03:09:59.579620+0000","last_became_peered":"2023-10-06T03:09:59.579620+0000","last_unstale":"2023-10-06T03:10:00.589720+0000","last_undegraded":"2023-10-06T03:10:00.589720+0000","last_fullsized":"2023-10-06T03:10:00.589720+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:02.798313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585756+0000","last_change":"2023-10-06T03:10:00.582773+0000","last_active":"2023-10-06T03:10:00.585756+0000","last_peered":"2023-10-06T03:10:00.585756+0000","last_clean":"2023-10-06T03:10:00.585756+0000","last_became_active":"2023-10-06T03:10:00.581895+0000","last_became_peered":"2023-10-06T03:10:00.581895+0000","last_unstale":"2023-10-06T03:10:00.585756+0000","last_undegraded":"2023-10-06T03:10:00.585756+0000","last_fullsized":"2023-10-06T03:10:00.585756+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"166'6","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597822+0000","last_change":"2023-10-06T03:09:59.588064+0000","last_active":"2023-10-06T03:10:00.597822+0000","last_peered":"2023-10-06T03:10:00.597822+0000","last_clean":"2023-10-06T03:10:00.597822+0000","last_became_active":"2023-10-06T03:09:59.587806+0000","last_became_peered":"2023-10-06T03:09:59.587806+0000","last_unstale":"2023-10-06T03:10:00.597822+0000","last_undegraded":"2023-10-06T03:10:00.597822+0000","last_fullsized":"2023-10-06T03:10:00.597822+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:53.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592520+0000","last_change":"2023-10-06T03:10:00.587580+0000","last_active":"2023-10-06T03:10:00.592520+0000","last_peered":"2023-10-06T03:10:00.592520+0000","last_clean":"2023-10-06T03:10:00.592520+0000","last_became_active":"2023-10-06T03:10:00.587352+0000","last_became_peered":"2023-10-06T03:10:00.587352+0000","last_unstale":"2023-10-06T03:10:00.592520+0000","last_undegraded":"2023-10-06T03:10:00.592520+0000","last_fullsized":"2023-10-06T03:10:00.592520+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595124+0000","last_change":"2023-10-06T03:09:59.585422+0000","last_active":"2023-10-06T03:10:00.595124+0000","last_peered":"2023-10-06T03:10:00.595124+0000","last_clean":"2023-10-06T03:10:00.595124+0000","last_became_active":"2023-10-06T03:09:59.585144+0000","last_became_peered":"2023-10-06T03:09:59.585144+0000","last_unstale":"2023-10-06T03:10:00.595124+0000","last_undegraded":"2023-10-06T03:10:00.595124+0000","last_fullsized":"2023-10-06T03:10:00.595124+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:44.098638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593645+0000","last_change":"2023-10-06T03:10:00.586192+0000","last_active":"2023-10-06T03:10:00.593645+0000","last_peered":"2023-10-06T03:10:00.593645+0000","last_clean":"2023-10-06T03:10:00.593645+0000","last_became_active":"2023-10-06T03:10:00.585916+0000","last_became_peered":"2023-10-06T03:10:00.585916+0000","last_unstale":"2023-10-06T03:10:00.593645+0000","last_undegraded":"2023-10-06T03:10:00.593645+0000","last_fullsized":"2023-10-06T03:10:00.593645+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:45.651788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568664+0000","last_change":"2023-10-06T03:09:59.577719+0000","last_active":"2023-10-06T03:10:00.568664+0000","last_peered":"2023-10-06T03:10:00.568664+0000","last_clean":"2023-10-06T03:10:00.568664+0000","last_became_active":"2023-10-06T03:09:59.577438+0000","last_became_peered":"2023-10-06T03:09:59.577438+0000","last_unstale":"2023-10-06T03:10:00.568664+0000","last_undegraded":"2023-10-06T03:10:00.568664+0000","last_fullsized":"2023-10-06T03:10:00.568664+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:48.569159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580711+0000","last_change":"2023-10-06T03:10:00.578706+0000","last_active":"2023-10-06T03:10:00.580711+0000","last_peered":"2023-10-06T03:10:00.580711+0000","last_clean":"2023-10-06T03:10:00.580711+0000","last_became_active":"2023-10-06T03:10:00.578426+0000","last_became_peered":"2023-10-06T03:10:00.578426+0000","last_unstale":"2023-10-06T03:10:00.580711+0000","last_undegraded":"2023-10-06T03:10:00.580711+0000","last_fullsized":"2023-10-06T03:10:00.580711+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:40.682371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569528+0000","last_change":"2023-10-06T03:09:59.578616+0000","last_active":"2023-10-06T03:10:00.569528+0000","last_peered":"2023-10-06T03:10:00.569528+0000","last_clean":"2023-10-06T03:10:00.569528+0000","last_became_active":"2023-10-06T03:09:59.578324+0000","last_became_peered":"2023-10-06T03:09:59.578324+0000","last_unstale":"2023-10-06T03:10:00.569528+0000","last_undegraded":"2023-10-06T03:10:00.569528+0000","last_fullsized":"2023-10-06T03:10:00.569528+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:18.392314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.583271+0000","last_change":"2023-10-06T03:10:00.581675+0000","last_active":"2023-10-06T03:10:00.583271+0000","last_peered":"2023-10-06T03:10:00.583271+0000","last_clean":"2023-10-06T03:10:00.583271+0000","last_became_active":"2023-10-06T03:10:00.580948+0000","last_became_peered":"2023-10-06T03:10:00.580948+0000","last_unstale":"2023-10-06T03:10:00.583271+0000","last_undegraded":"2023-10-06T03:10:00.583271+0000","last_fullsized":"2023-10-06T03:10:00.583271+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:05.271138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"166'8","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595055+0000","last_change":"2023-10-06T03:09:59.584996+0000","last_active":"2023-10-06T03:10:00.595055+0000","last_peered":"2023-10-06T03:10:00.595055+0000","last_clean":"2023-10-06T03:10:00.595055+0000","last_became_active":"2023-10-06T03:09:59.584702+0000","last_became_peered":"2023-10-06T03:09:59.584702+0000","last_unstale":"2023-10-06T03:10:00.595055+0000","last_undegraded":"2023-10-06T03:10:00.595055+0000","last_fullsized":"2023-10-06T03:10:00.595055+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:13.811016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592469+0000","last_change":"2023-10-06T03:10:00.587246+0000","last_active":"2023-10-06T03:10:00.592469+0000","last_peered":"2023-10-06T03:10:00.592469+0000","last_clean":"2023-10-06T03:10:00.592469+0000","last_became_active":"2023-10-06T03:10:00.586984+0000","last_became_peered":"2023-10-06T03:10:00.586984+0000","last_unstale":"2023-10-06T03:10:00.592469+0000","last_undegraded":"2023-10-06T03:10:00.592469+0000","last_fullsized":"2023-10-06T03:10:00.592469+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:25.739194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594854+0000","last_change":"2023-10-06T03:09:59.584290+0000","last_active":"2023-10-06T03:10:00.594854+0000","last_peered":"2023-10-06T03:10:00.594854+0000","last_clean":"2023-10-06T03:10:00.594854+0000","last_became_active":"2023-10-06T03:09:59.584045+0000","last_became_peered":"2023-10-06T03:09:59.584045+0000","last_unstale":"2023-10-06T03:10:00.594854+0000","last_undegraded":"2023-10-06T03:10:00.594854+0000","last_fullsized":"2023-10-06T03:10:00.594854+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:04:49.405237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591503+0000","last_change":"2023-10-06T03:10:00.589081+0000","last_active":"2023-10-06T03:10:00.591503+0000","last_peered":"2023-10-06T03:10:00.591503+0000","last_clean":"2023-10-06T03:10:00.591503+0000","last_became_active":"2023-10-06T03:10:00.588677+0000","last_became_peered":"2023-10-06T03:10:00.588677+0000","last_unstale":"2023-10-06T03:10:00.591503+0000","last_undegraded":"2023-10-06T03:10:00.591503+0000","last_fullsized":"2023-10-06T03:10:00.591503+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.582700+0000","last_change":"2023-10-06T03:09:59.574540+0000","last_active":"2023-10-06T03:10:00.582700+0000","last_peered":"2023-10-06T03:10:00.582700+0000","last_clean":"2023-10-06T03:10:00.582700+0000","last_became_active":"2023-10-06T03:09:59.574290+0000","last_became_peered":"2023-10-06T03:09:59.574290+0000","last_unstale":"2023-10-06T03:10:00.582700+0000","last_undegraded":"2023-10-06T03:10:00.582700+0000","last_fullsized":"2023-10-06T03:10:00.582700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:28.837966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585579+0000","last_change":"2023-10-06T03:10:00.581659+0000","last_active":"2023-10-06T03:10:00.585579+0000","last_peered":"2023-10-06T03:10:00.585579+0000","last_clean":"2023-10-06T03:10:00.585579+0000","last_became_active":"2023-10-06T03:10:00.579525+0000","last_became_peered":"2023-10-06T03:10:00.579525+0000","last_unstale":"2023-10-06T03:10:00.585579+0000","last_undegraded":"2023-10-06T03:10:00.585579+0000","last_fullsized":"2023-10-06T03:10:00.585579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591892+0000","last_change":"2023-10-06T03:09:59.581150+0000","last_active":"2023-10-06T03:10:00.591892+0000","last_peered":"2023-10-06T03:10:00.591892+0000","last_clean":"2023-10-06T03:10:00.591892+0000","last_became_active":"2023-10-06T03:09:59.580879+0000","last_became_peered":"2023-10-06T03:09:59.580879+0000","last_unstale":"2023-10-06T03:10:00.591892+0000","last_undegraded":"2023-10-06T03:10:00.591892+0000","last_fullsized":"2023-10-06T03:10:00.591892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:01.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597594+0000","last_change":"2023-10-06T03:10:00.594466+0000","last_active":"2023-10-06T03:10:00.597594+0000","last_peered":"2023-10-06T03:10:00.597594+0000","last_clean":"2023-10-06T03:10:00.597594+0000","last_became_active":"2023-10-06T03:10:00.594209+0000","last_became_peered":"2023-10-06T03:10:00.594209+0000","last_unstale":"2023-10-06T03:10:00.597594+0000","last_undegraded":"2023-10-06T03:10:00.597594+0000","last_fullsized":"2023-10-06T03:10:00.597594+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:26.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579636+0000","last_change":"2023-10-06T03:09:59.581085+0000","last_active":"2023-10-06T03:10:00.579636+0000","last_peered":"2023-10-06T03:10:00.579636+0000","last_clean":"2023-10-06T03:10:00.579636+0000","last_became_active":"2023-10-06T03:09:59.580800+0000","last_became_peered":"2023-10-06T03:09:59.580800+0000","last_unstale":"2023-10-06T03:10:00.579636+0000","last_undegraded":"2023-10-06T03:10:00.579636+0000","last_fullsized":"2023-10-06T03:10:00.579636+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:35.139211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597534+0000","last_change":"2023-10-06T03:10:00.596330+0000","last_active":"2023-10-06T03:10:00.597534+0000","last_peered":"2023-10-06T03:10:00.597534+0000","last_clean":"2023-10-06T03:10:00.597534+0000","last_became_active":"2023-10-06T03:10:00.596047+0000","last_became_peered":"2023-10-06T03:10:00.596047+0000","last_unstale":"2023-10-06T03:10:00.597534+0000","last_undegraded":"2023-10-06T03:10:00.597534+0000","last_fullsized":"2023-10-06T03:10:00.597534+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:25.010008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.596134+0000","last_change":"2023-10-06T03:09:59.585047+0000","last_active":"2023-10-06T03:10:00.596134+0000","last_peered":"2023-10-06T03:10:00.596134+0000","last_clean":"2023-10-06T03:10:00.596134+0000","last_became_active":"2023-10-06T03:09:59.584765+0000","last_became_peered":"2023-10-06T03:09:59.584765+0000","last_unstale":"2023-10-06T03:10:00.596134+0000","last_undegraded":"2023-10-06T03:10:00.596134+0000","last_fullsized":"2023-10-06T03:10:00.596134+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591317+0000","last_change":"2023-10-06T03:10:00.586313+0000","last_active":"2023-10-06T03:10:00.591317+0000","last_peered":"2023-10-06T03:10:00.591317+0000","last_clean":"2023-10-06T03:10:00.591317+0000","last_became_active":"2023-10-06T03:10:00.585727+0000","last_became_peered":"2023-10-06T03:10:00.585727+0000","last_unstale":"2023-10-06T03:10:00.591317+0000","last_undegraded":"2023-10-06T03:10:00.591317+0000","last_fullsized":"2023-10-06T03:10:00.591317+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:49.574453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568794+0000","last_change":"2023-10-06T03:09:59.578270+0000","last_active":"2023-10-06T03:10:00.568794+0000","last_peered":"2023-10-06T03:10:00.568794+0000","last_clean":"2023-10-06T03:10:00.568794+0000","last_became_active":"2023-10-06T03:09:59.578053+0000","last_became_peered":"2023-10-06T03:09:59.578053+0000","last_unstale":"2023-10-06T03:10:00.568794+0000","last_undegraded":"2023-10-06T03:10:00.568794+0000","last_fullsized":"2023-10-06T03:10:00.568794+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:38.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593581+0000","last_change":"2023-10-06T03:10:00.593111+0000","last_active":"2023-10-06T03:10:00.593581+0000","last_peered":"2023-10-06T03:10:00.593581+0000","last_clean":"2023-10-06T03:10:00.593581+0000","last_became_active":"2023-10-06T03:10:00.592821+0000","last_became_peered":"2023-10-06T03:10:00.592821+0000","last_unstale":"2023-10-06T03:10:00.593581+0000","last_undegraded":"2023-10-06T03:10:00.593581+0000","last_fullsized":"2023-10-06T03:10:00.593581+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:06.937724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569192+0000","last_change":"2023-10-06T03:09:59.579123+0000","last_active":"2023-10-06T03:10:00.569192+0000","last_peered":"2023-10-06T03:10:00.569192+0000","last_clean":"2023-10-06T03:10:00.569192+0000","last_became_active":"2023-10-06T03:09:59.578903+0000","last_became_peered":"2023-10-06T03:09:59.578903+0000","last_unstale":"2023-10-06T03:10:00.569192+0000","last_undegraded":"2023-10-06T03:10:00.569192+0000","last_fullsized":"2023-10-06T03:10:00.569192+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:38.262418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595574+0000","last_change":"2023-10-06T03:10:00.595104+0000","last_active":"2023-10-06T03:10:00.595574+0000","last_peered":"2023-10-06T03:10:00.595574+0000","last_clean":"2023-10-06T03:10:00.595574+0000","last_became_active":"2023-10-06T03:10:00.594906+0000","last_became_peered":"2023-10-06T03:10:00.594906+0000","last_unstale":"2023-10-06T03:10:00.595574+0000","last_undegraded":"2023-10-06T03:10:00.595574+0000","last_fullsized":"2023-10-06T03:10:00.595574+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:17.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.565462+0000","last_change":"2023-10-06T03:09:59.574352+0000","last_active":"2023-10-06T03:10:00.565462+0000","last_peered":"2023-10-06T03:10:00.565462+0000","last_clean":"2023-10-06T03:10:00.565462+0000","last_became_active":"2023-10-06T03:09:59.574058+0000","last_became_peered":"2023-10-06T03:09:59.574058+0000","last_unstale":"2023-10-06T03:10:00.565462+0000","last_undegraded":"2023-10-06T03:10:00.565462+0000","last_fullsized":"2023-10-06T03:10:00.565462+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:47.797463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579200+0000","last_change":"2023-10-06T03:10:00.574251+0000","last_active":"2023-10-06T03:10:00.579200+0000","last_peered":"2023-10-06T03:10:00.579200+0000","last_clean":"2023-10-06T03:10:00.579200+0000","last_became_active":"2023-10-06T03:10:00.574007+0000","last_became_peered":"2023-10-06T03:10:00.574007+0000","last_unstale":"2023-10-06T03:10:00.579200+0000","last_undegraded":"2023-10-06T03:10:00.579200+0000","last_fullsized":"2023-10-06T03:10:00.579200+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593738+0000","last_change":"2023-10-06T03:09:59.584422+0000","last_active":"2023-10-06T03:10:00.593738+0000","last_peered":"2023-10-06T03:10:00.593738+0000","last_clean":"2023-10-06T03:10:00.593738+0000","last_became_active":"2023-10-06T03:09:59.583935+0000","last_became_peered":"2023-10-06T03:09:59.583935+0000","last_unstale":"2023-10-06T03:10:00.593738+0000","last_undegraded":"2023-10-06T03:10:00.593738+0000","last_fullsized":"2023-10-06T03:10:00.593738+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:53.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591950+0000","last_change":"2023-10-06T03:10:00.590564+0000","last_active":"2023-10-06T03:10:00.591950+0000","last_peered":"2023-10-06T03:10:00.591950+0000","last_clean":"2023-10-06T03:10:00.591950+0000","last_became_active":"2023-10-06T03:10:00.590286+0000","last_became_peered":"2023-10-06T03:10:00.590286+0000","last_unstale":"2023-10-06T03:10:00.591950+0000","last_undegraded":"2023-10-06T03:10:00.591950+0000","last_fullsized":"2023-10-06T03:10:00.591950+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:40.682371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568230+0000","last_change":"2023-10-06T03:09:59.579097+0000","last_active":"2023-10-06T03:10:00.568230+0000","last_peered":"2023-10-06T03:10:00.568230+0000","last_clean":"2023-10-06T03:10:00.568230+0000","last_became_active":"2023-10-06T03:09:59.578839+0000","last_became_peered":"2023-10-06T03:09:59.578839+0000","last_unstale":"2023-10-06T03:10:00.568230+0000","last_undegraded":"2023-10-06T03:10:00.568230+0000","last_fullsized":"2023-10-06T03:10:00.568230+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:21.791540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588959+0000","last_change":"2023-10-06T03:10:00.587760+0000","last_active":"2023-10-06T03:10:00.588959+0000","last_peered":"2023-10-06T03:10:00.588959+0000","last_clean":"2023-10-06T03:10:00.588959+0000","last_became_active":"2023-10-06T03:10:00.587504+0000","last_became_peered":"2023-10-06T03:10:00.587504+0000","last_unstale":"2023-10-06T03:10:00.588959+0000","last_undegraded":"2023-10-06T03:10:00.588959+0000","last_fullsized":"2023-10-06T03:10:00.588959+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.573532+0000","last_change":"2023-10-06T03:09:59.575066+0000","last_active":"2023-10-06T03:10:00.573532+0000","last_peered":"2023-10-06T03:10:00.573532+0000","last_clean":"2023-10-06T03:10:00.573532+0000","last_became_active":"2023-10-06T03:09:59.574835+0000","last_became_peered":"2023-10-06T03:09:59.574835+0000","last_unstale":"2023-10-06T03:10:00.573532+0000","last_undegraded":"2023-10-06T03:10:00.573532+0000","last_fullsized":"2023-10-06T03:10:00.573532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:41.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.577571+0000","last_change":"2023-10-06T03:10:00.574835+0000","last_active":"2023-10-06T03:10:00.577571+0000","last_peered":"2023-10-06T03:10:00.577571+0000","last_clean":"2023-10-06T03:10:00.577571+0000","last_became_active":"2023-10-06T03:10:00.574665+0000","last_became_peered":"2023-10-06T03:10:00.574665+0000","last_unstale":"2023-10-06T03:10:00.577571+0000","last_undegraded":"2023-10-06T03:10:00.577571+0000","last_fullsized":"2023-10-06T03:10:00.577571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:46.345022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579849+0000","last_change":"2023-10-06T03:09:59.584117+0000","last_active":"2023-10-06T03:10:00.579849+0000","last_peered":"2023-10-06T03:10:00.579849+0000","last_clean":"2023-10-06T03:10:00.579849+0000","last_became_active":"2023-10-06T03:09:59.583892+0000","last_became_peered":"2023-10-06T03:09:59.583892+0000","last_unstale":"2023-10-06T03:10:00.579849+0000","last_undegraded":"2023-10-06T03:10:00.579849+0000","last_fullsized":"2023-10-06T03:10:00.579849+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:24.144827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586011+0000","last_change":"2023-10-06T03:10:00.584710+0000","last_active":"2023-10-06T03:10:00.586011+0000","last_peered":"2023-10-06T03:10:00.586011+0000","last_clean":"2023-10-06T03:10:00.586011+0000","last_became_active":"2023-10-06T03:10:00.584403+0000","last_became_peered":"2023-10-06T03:10:00.584403+0000","last_unstale":"2023-10-06T03:10:00.586011+0000","last_undegraded":"2023-10-06T03:10:00.586011+0000","last_fullsized":"2023-10-06T03:10:00.586011+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:58.888506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579750+0000","last_change":"2023-10-06T03:09:59.581174+0000","last_active":"2023-10-06T03:10:00.579750+0000","last_peered":"2023-10-06T03:10:00.579750+0000","last_clean":"2023-10-06T03:10:00.579750+0000","last_became_active":"2023-10-06T03:09:59.580756+0000","last_became_peered":"2023-10-06T03:09:59.580756+0000","last_unstale":"2023-10-06T03:10:00.579750+0000","last_undegraded":"2023-10-06T03:10:00.579750+0000","last_fullsized":"2023-10-06T03:10:00.579750+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:59.106481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585956+0000","last_change":"2023-10-06T03:10:00.585304+0000","last_active":"2023-10-06T03:10:00.585956+0000","last_peered":"2023-10-06T03:10:00.585956+0000","last_clean":"2023-10-06T03:10:00.585956+0000","last_became_active":"2023-10-06T03:10:00.585059+0000","last_became_peered":"2023-10-06T03:10:00.585059+0000","last_unstale":"2023-10-06T03:10:00.585956+0000","last_undegraded":"2023-10-06T03:10:00.585956+0000","last_fullsized":"2023-10-06T03:10:00.585956+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:49.574453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570927+0000","last_change":"2023-10-06T03:09:59.575259+0000","last_active":"2023-10-06T03:10:00.570927+0000","last_peered":"2023-10-06T03:10:00.570927+0000","last_clean":"2023-10-06T03:10:00.570927+0000","last_became_active":"2023-10-06T03:09:59.574967+0000","last_became_peered":"2023-10-06T03:09:59.574967+0000","last_unstale":"2023-10-06T03:10:00.570927+0000","last_undegraded":"2023-10-06T03:10:00.570927+0000","last_fullsized":"2023-10-06T03:10:00.570927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:24.973597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580453+0000","last_change":"2023-10-06T03:10:00.577813+0000","last_active":"2023-10-06T03:10:00.580453+0000","last_peered":"2023-10-06T03:10:00.580453+0000","last_clean":"2023-10-06T03:10:00.580453+0000","last_became_active":"2023-10-06T03:10:00.577551+0000","last_became_peered":"2023-10-06T03:10:00.577551+0000","last_unstale":"2023-10-06T03:10:00.580453+0000","last_undegraded":"2023-10-06T03:10:00.580453+0000","last_fullsized":"2023-10-06T03:10:00.580453+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:52.181676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575127+0000","last_change":"2023-10-06T03:09:59.580165+0000","last_active":"2023-10-06T03:10:00.575127+0000","last_peered":"2023-10-06T03:10:00.575127+0000","last_clean":"2023-10-06T03:10:00.575127+0000","last_became_active":"2023-10-06T03:09:59.579314+0000","last_became_peered":"2023-10-06T03:09:59.579314+0000","last_unstale":"2023-10-06T03:10:00.575127+0000","last_undegraded":"2023-10-06T03:10:00.575127+0000","last_fullsized":"2023-10-06T03:10:00.575127+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:05.244643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595174+0000","last_change":"2023-10-06T03:10:00.592300+0000","last_active":"2023-10-06T03:10:00.595174+0000","last_peered":"2023-10-06T03:10:00.595174+0000","last_clean":"2023-10-06T03:10:00.595174+0000","last_became_active":"2023-10-06T03:10:00.592002+0000","last_became_peered":"2023-10-06T03:10:00.592002+0000","last_unstale":"2023-10-06T03:10:00.595174+0000","last_undegraded":"2023-10-06T03:10:00.595174+0000","last_fullsized":"2023-10-06T03:10:00.595174+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.043746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.583297+0000","last_change":"2023-10-06T03:09:59.582382+0000","last_active":"2023-10-06T03:10:00.583297+0000","last_peered":"2023-10-06T03:10:00.583297+0000","last_clean":"2023-10-06T03:10:00.583297+0000","last_became_active":"2023-10-06T03:09:59.581539+0000","last_became_peered":"2023-10-06T03:09:59.581539+0000","last_unstale":"2023-10-06T03:10:00.583297+0000","last_undegraded":"2023-10-06T03:10:00.583297+0000","last_fullsized":"2023-10-06T03:10:00.583297+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:01.769794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576395+0000","last_change":"2023-10-06T03:10:00.572994+0000","last_active":"2023-10-06T03:10:00.576395+0000","last_peered":"2023-10-06T03:10:00.576395+0000","last_clean":"2023-10-06T03:10:00.576395+0000","last_became_active":"2023-10-06T03:10:00.572728+0000","last_became_peered":"2023-10-06T03:10:00.572728+0000","last_unstale":"2023-10-06T03:10:00.576395+0000","last_undegraded":"2023-10-06T03:10:00.576395+0000","last_fullsized":"2023-10-06T03:10:00.576395+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:28.150464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569869+0000","last_change":"2023-10-06T03:09:59.575701+0000","last_active":"2023-10-06T03:10:00.569869+0000","last_peered":"2023-10-06T03:10:00.569869+0000","last_clean":"2023-10-06T03:10:00.569869+0000","last_became_active":"2023-10-06T03:09:59.575456+0000","last_became_peered":"2023-10-06T03:09:59.575456+0000","last_unstale":"2023-10-06T03:10:00.569869+0000","last_undegraded":"2023-10-06T03:10:00.569869+0000","last_fullsized":"2023-10-06T03:10:00.569869+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:06.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580294+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:00.580294+0000","last_peered":"2023-10-06T03:10:00.580294+0000","last_clean":"2023-10-06T03:10:00.580294+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:00.580294+0000","last_undegraded":"2023-10-06T03:10:00.580294+0000","last_fullsized":"2023-10-06T03:10:00.580294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576433+0000","last_change":"2023-10-06T03:10:00.575092+0000","last_active":"2023-10-06T03:10:00.576433+0000","last_peered":"2023-10-06T03:10:00.576433+0000","last_clean":"2023-10-06T03:10:00.576433+0000","last_became_active":"2023-10-06T03:10:00.574839+0000","last_became_peered":"2023-10-06T03:10:00.574839+0000","last_unstale":"2023-10-06T03:10:00.576433+0000","last_undegraded":"2023-10-06T03:10:00.576433+0000","last_fullsized":"2023-10-06T03:10:00.576433+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:05.825301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570650+0000","last_change":"2023-10-06T03:09:59.585944+0000","last_active":"2023-10-06T03:10:00.570650+0000","last_peered":"2023-10-06T03:10:00.570650+0000","last_clean":"2023-10-06T03:10:00.570650+0000","last_became_active":"2023-10-06T03:09:59.585698+0000","last_became_peered":"2023-10-06T03:09:59.585698+0000","last_unstale":"2023-10-06T03:10:00.570650+0000","last_undegraded":"2023-10-06T03:10:00.570650+0000","last_fullsized":"2023-10-06T03:10:00.570650+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:43.377430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568310+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:00.568310+0000","last_peered":"2023-10-06T03:10:00.568310+0000","last_clean":"2023-10-06T03:10:00.568310+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:00.568310+0000","last_undegraded":"2023-10-06T03:10:00.568310+0000","last_fullsized":"2023-10-06T03:10:00.568310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593671+0000","last_change":"2023-10-06T03:10:00.590105+0000","last_active":"2023-10-06T03:10:00.593671+0000","last_peered":"2023-10-06T03:10:00.593671+0000","last_clean":"2023-10-06T03:10:00.593671+0000","last_became_active":"2023-10-06T03:10:00.589873+0000","last_became_peered":"2023-10-06T03:10:00.589873+0000","last_unstale":"2023-10-06T03:10:00.593671+0000","last_undegraded":"2023-10-06T03:10:00.593671+0000","last_fullsized":"2023-10-06T03:10:00.593671+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:42.817569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594988+0000","last_change":"2023-10-06T03:09:59.584721+0000","last_active":"2023-10-06T03:10:00.594988+0000","last_peered":"2023-10-06T03:10:00.594988+0000","last_clean":"2023-10-06T03:10:00.594988+0000","last_became_active":"2023-10-06T03:09:59.584460+0000","last_became_peered":"2023-10-06T03:09:59.584460+0000","last_unstale":"2023-10-06T03:10:00.594988+0000","last_undegraded":"2023-10-06T03:10:00.594988+0000","last_fullsized":"2023-10-06T03:10:00.594988+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:24.661979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576953+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:00.576953+0000","last_peered":"2023-10-06T03:10:00.576953+0000","last_clean":"2023-10-06T03:10:00.576953+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:00.576953+0000","last_undegraded":"2023-10-06T03:10:00.576953+0000","last_fullsized":"2023-10-06T03:10:00.576953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580569+0000","last_change":"2023-10-06T03:10:00.578820+0000","last_active":"2023-10-06T03:10:00.580569+0000","last_peered":"2023-10-06T03:10:00.580569+0000","last_clean":"2023-10-06T03:10:00.580569+0000","last_became_active":"2023-10-06T03:10:00.578555+0000","last_became_peered":"2023-10-06T03:10:00.578555+0000","last_unstale":"2023-10-06T03:10:00.580569+0000","last_undegraded":"2023-10-06T03:10:00.580569+0000","last_fullsized":"2023-10-06T03:10:00.580569+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:47.002388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"166'4","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591496+0000","last_change":"2023-10-06T03:09:59.576969+0000","last_active":"2023-10-06T03:10:00.591496+0000","last_peered":"2023-10-06T03:10:00.591496+0000","last_clean":"2023-10-06T03:10:00.591496+0000","last_became_active":"2023-10-06T03:09:59.576674+0000","last_became_peered":"2023-10-06T03:09:59.576674+0000","last_unstale":"2023-10-06T03:10:00.591496+0000","last_undegraded":"2023-10-06T03:10:00.591496+0000","last_fullsized":"2023-10-06T03:10:00.591496+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:20.093813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575151+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:00.575151+0000","last_peered":"2023-10-06T03:10:00.575151+0000","last_clean":"2023-10-06T03:10:00.575151+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:00.575151+0000","last_undegraded":"2023-10-06T03:10:00.575151+0000","last_fullsized":"2023-10-06T03:10:00.575151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595586+0000","last_change":"2023-10-06T03:10:00.595130+0000","last_active":"2023-10-06T03:10:00.595586+0000","last_peered":"2023-10-06T03:10:00.595586+0000","last_clean":"2023-10-06T03:10:00.595586+0000","last_became_active":"2023-10-06T03:10:00.594892+0000","last_became_peered":"2023-10-06T03:10:00.594892+0000","last_unstale":"2023-10-06T03:10:00.595586+0000","last_undegraded":"2023-10-06T03:10:00.595586+0000","last_fullsized":"2023-10-06T03:10:00.595586+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:08.545791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.589755+0000","last_change":"2023-10-06T03:09:59.580125+0000","last_active":"2023-10-06T03:10:00.589755+0000","last_peered":"2023-10-06T03:10:00.589755+0000","last_clean":"2023-10-06T03:10:00.589755+0000","last_became_active":"2023-10-06T03:09:59.579797+0000","last_became_peered":"2023-10-06T03:09:59.579797+0000","last_unstale":"2023-10-06T03:10:00.589755+0000","last_undegraded":"2023-10-06T03:10:00.589755+0000","last_fullsized":"2023-10-06T03:10:00.589755+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:01.659699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":287,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575211+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:00.575211+0000","last_peered":"2023-10-06T03:10:00.575211+0000","last_clean":"2023-10-06T03:10:00.575211+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:00.575211+0000","last_undegraded":"2023-10-06T03:10:00.575211+0000","last_fullsized":"2023-10-06T03:10:00.575211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":150,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570531+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:00.570531+0000","last_peered":"2023-10-06T03:10:00.570531+0000","last_clean":"2023-10-06T03:10:00.570531+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:00.570531+0000","last_undegraded":"2023-10-06T03:10:00.570531+0000","last_fullsized":"2023-10-06T03:10:00.570531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580687+0000","last_change":"2023-10-06T03:10:00.578960+0000","last_active":"2023-10-06T03:10:00.580687+0000","last_peered":"2023-10-06T03:10:00.580687+0000","last_clean":"2023-10-06T03:10:00.580687+0000","last_became_active":"2023-10-06T03:10:00.578467+0000","last_became_peered":"2023-10-06T03:10:00.578467+0000","last_unstale":"2023-10-06T03:10:00.580687+0000","last_undegraded":"2023-10-06T03:10:00.580687+0000","last_fullsized":"2023-10-06T03:10:00.580687+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594862+0000","last_change":"2023-10-06T03:09:59.582739+0000","last_active":"2023-10-06T03:10:00.594862+0000","last_peered":"2023-10-06T03:10:00.594862+0000","last_clean":"2023-10-06T03:10:00.594862+0000","last_became_active":"2023-10-06T03:09:59.582478+0000","last_became_peered":"2023-10-06T03:09:59.582478+0000","last_unstale":"2023-10-06T03:10:00.594862+0000","last_undegraded":"2023-10-06T03:10:00.594862+0000","last_fullsized":"2023-10-06T03:10:00.594862+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:20.649970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.565806+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:00.565806+0000","last_peered":"2023-10-06T03:10:00.565806+0000","last_clean":"2023-10-06T03:10:00.565806+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:00.565806+0000","last_undegraded":"2023-10-06T03:10:00.565806+0000","last_fullsized":"2023-10-06T03:10:00.565806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597596+0000","last_change":"2023-10-06T03:10:00.596419+0000","last_active":"2023-10-06T03:10:00.597596+0000","last_peered":"2023-10-06T03:10:00.597596+0000","last_clean":"2023-10-06T03:10:00.597596+0000","last_became_active":"2023-10-06T03:10:00.596142+0000","last_became_peered":"2023-10-06T03:10:00.596142+0000","last_unstale":"2023-10-06T03:10:00.597596+0000","last_undegraded":"2023-10-06T03:10:00.597596+0000","last_fullsized":"2023-10-06T03:10:00.597596+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:48.596753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595426+0000","last_change":"2023-10-06T03:09:59.584917+0000","last_active":"2023-10-06T03:10:00.595426+0000","last_peered":"2023-10-06T03:10:00.595426+0000","last_clean":"2023-10-06T03:10:00.595426+0000","last_became_active":"2023-10-06T03:09:59.584649+0000","last_became_peered":"2023-10-06T03:09:59.584649+0000","last_unstale":"2023-10-06T03:10:00.595426+0000","last_undegraded":"2023-10-06T03:10:00.595426+0000","last_fullsized":"2023-10-06T03:10:00.595426+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:54.965400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":150,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570365+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:00.570365+0000","last_peered":"2023-10-06T03:10:00.570365+0000","last_clean":"2023-10-06T03:10:00.570365+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:00.570365+0000","last_undegraded":"2023-10-06T03:10:00.570365+0000","last_fullsized":"2023-10-06T03:10:00.570365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580208+0000","last_change":"2023-10-06T03:10:00.575607+0000","last_active":"2023-10-06T03:10:00.580208+0000","last_peered":"2023-10-06T03:10:00.580208+0000","last_clean":"2023-10-06T03:10:00.580208+0000","last_became_active":"2023-10-06T03:10:00.575358+0000","last_became_peered":"2023-10-06T03:10:00.575358+0000","last_unstale":"2023-10-06T03:10:00.580208+0000","last_undegraded":"2023-10-06T03:10:00.580208+0000","last_fullsized":"2023-10-06T03:10:00.580208+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:25.739194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592663+0000","last_change":"2023-10-06T03:09:59.585796+0000","last_active":"2023-10-06T03:10:00.592663+0000","last_peered":"2023-10-06T03:10:00.592663+0000","last_clean":"2023-10-06T03:10:00.592663+0000","last_became_active":"2023-10-06T03:09:59.585493+0000","last_became_peered":"2023-10-06T03:09:59.585493+0000","last_unstale":"2023-10-06T03:10:00.592663+0000","last_undegraded":"2023-10-06T03:10:00.592663+0000","last_fullsized":"2023-10-06T03:10:00.592663+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:23.810753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.573930+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:00.573930+0000","last_peered":"2023-10-06T03:10:00.573930+0000","last_clean":"2023-10-06T03:10:00.573930+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:00.573930+0000","last_undegraded":"2023-10-06T03:10:00.573930+0000","last_fullsized":"2023-10-06T03:10:00.573930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304040,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067800,"statfs":{"total":96636764160,"available":96325427200,"internally_reserved":0,"allocated":2719744,"data_stored":1592056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.626},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.438},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303476,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068364,"statfs":{"total":96636764160,"available":96326004736,"internally_reserved":0,"allocated":2142208,"data_stored":1002706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.375}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.433},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.441}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303452,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068388,"statfs":{"total":96636764160,"available":96326029312,"internally_reserved":0,"allocated":2117632,"data_stored":1001044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.434},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.443},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.503}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":2711552,"data_stored":1591412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30399999999999999,"15min":0.31},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32600000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.51100000000000001,"15min":0.64100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.34100000000000003},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.20899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.61699999999999999},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.437}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":1001632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28499999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.30399999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.433},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.501},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":1001066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.498},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":1001134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.441}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.499},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303468,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068372,"statfs":{"total":96636764160,"available":96326012928,"internally_reserved":0,"allocated":2134016,"data_stored":1002650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:05.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T03:10:05.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.519+0000 7f687f530700 1 Processor -- start 2023-10-06T03:10:05.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.519+0000 7f687f530700 1 -- start start 2023-10-06T03:10:05.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687f530700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68780b55a0 0x7f6878164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687f530700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68780cfa60 con 0x7f68780cd910 2023-10-06T03:10:05.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687f530700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68780b5b00 con 0x7f6878159ec0 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687f530700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68780b5da0 con 0x7f68780b55a0 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687dacd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f68780b55a0 0x7f6878164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687cacb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f68780cd910 0x7f68780ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45154/0 (socket says 172.21.15.92:45154) 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.520+0000 7f687cacb700 1 -- 172.21.15.92:0/2760738402 learned_addr learned my addr 172.21.15.92:0/2760738402 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687dacd700 1 -- 172.21.15.92:0/2760738402 >> v1:172.21.15.136:6789/0 conn(0x7f6878159ec0 legacy=0x7f68780b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687dacd700 1 -- 172.21.15.92:0/2760738402 >> v1:172.21.15.92:6789/0 conn(0x7f68780cd910 legacy=0x7f68780ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687dacd700 1 -- 172.21.15.92:0/2760738402 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780b6080 con 0x7f68780b55a0 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f686ffff700 1 -- 172.21.15.92:0/2760738402 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715115026 0 0) 0x7f68780cfa60 con 0x7f68780cd910 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687dacd700 1 --2- 172.21.15.92:0/2760738402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f6878164400 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6870009e90 tx=0x7f6870009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2068336ab33930 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f686ffff700 1 -- 172.21.15.92:0/2760738402 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000e070 con 0x7f68780b55a0 2023-10-06T03:10:05.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f686ffff700 1 -- 172.21.15.92:0/2760738402 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687000bbf0 con 0x7f68780b55a0 2023-10-06T03:10:05.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f686ffff700 1 -- 172.21.15.92:0/2760738402 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6870012430 con 0x7f68780b55a0 2023-10-06T03:10:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687f530700 1 -- 172.21.15.92:0/2760738402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 msgr2=0x7f6878164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687f530700 1 --2- 172.21.15.92:0/2760738402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f6878164400 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6870009e90 tx=0x7f6870009780 comp rx=0 tx=0).stop 2023-10-06T03:10:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687f530700 1 -- 172.21.15.92:0/2760738402 shutdown_connections 2023-10-06T03:10:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687f530700 1 --2- 172.21.15.92:0/2760738402 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f6878164400 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687f530700 1 -- 172.21.15.92:0/2760738402 >> 172.21.15.92:0/2760738402 conn(0x7f68780999f0 msgr2=0x7f68780b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.521+0000 7f687f530700 1 -- 172.21.15.92:0/2760738402 shutdown_connections 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.522+0000 7f687f530700 1 -- 172.21.15.92:0/2760738402 wait complete. 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 Processor -- start 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 -- start start 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f68781451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6878159ec0 0x7f6878145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6878069360 0x7f68780697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68781469e0 con 0x7f68780b55a0 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6878157680 con 0x7f6878069360 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687f530700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6878145dd0 con 0x7f6878159ec0 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.523+0000 7f687d2cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f68781451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f68781451f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45406/0 (socket says 172.21.15.92:45406) 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 -- 172.21.15.92:0/441907539 learned_addr learned my addr 172.21.15.92:0/441907539 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687cacb700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6878159ec0 0x7f6878145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687dacd700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6878069360 0x7f68780697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 -- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6878159ec0 msgr2=0x7f6878145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6878159ec0 0x7f6878145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 -- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6878069360 msgr2=0x7f68780697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6878069360 0x7f68780697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 -- 172.21.15.92:0/441907539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6870009a20 con 0x7f68780b55a0 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687dacd700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6878069360 0x7f68780697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687cacb700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6878159ec0 0x7f6878145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687d2cc700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f68781451f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f687400c7c0 tx=0x7f6874007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874011590 con 0x7f68780b55a0 2023-10-06T03:10:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6874011710 con 0x7f68780b55a0 2023-10-06T03:10:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68781460b0 con 0x7f68780b55a0 2023-10-06T03:10:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874011ad0 con 0x7f68780b55a0 2023-10-06T03:10:05.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.524+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6878146670 con 0x7f68780b55a0 2023-10-06T03:10:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.525+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68780b2f00 con 0x7f68780b55a0 2023-10-06T03:10:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.525+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f687401b3b0 con 0x7f68780b55a0 2023-10-06T03:10:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.526+0000 7f686d7fa700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f685c067320 0x7f685c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.526+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f68740842b0 con 0x7f68780b55a0 2023-10-06T03:10:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.526+0000 7f687cacb700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f685c067320 0x7f685c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.526+0000 7f687cacb700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f685c067320 0x7f685c0697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f68780b2310 tx=0x7f6868009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.530+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6874050e90 con 0x7f68780b55a0 2023-10-06T03:10:05.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.792+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6878063c90 con 0x7f68780b55a0 2023-10-06T03:10:05.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.793+0000 7f686d7fa700 1 -- 172.21.15.92:0/441907539 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1441 (secure 0 0 0) 0x7f68740175c0 con 0x7f68780b55a0 2023-10-06T03:10:05.795 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:05.795 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":528,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":167,"num_osds":8,"num_up_osds":8,"osd_up_since":1696561288,"num_in_osds":8,"osd_in_since":1696561280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2487132160,"bytes_avail":770606981120,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":113,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"b","status":"up:active","gid":8361}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-06T03:09:49.282112+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f685c067320 msgr2=0x7f685c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f685c067320 0x7f685c0697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f68780b2310 tx=0x7f6868009040 comp rx=0 tx=0).stop 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 msgr2=0x7f68781451f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f68781451f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f687400c7c0 tx=0x7f6874007f70 comp rx=0 tx=0).stop 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 shutdown_connections 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f685c067320 0x7f685c0697d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f68780b55a0 0x7f68781451f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6878069360 0x7f68780697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 --2- 172.21.15.92:0/441907539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6878159ec0 0x7f6878145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 >> 172.21.15.92:0/441907539 conn(0x7f68780999f0 msgr2=0x7f6878159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:05.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 shutdown_connections 2023-10-06T03:10:05.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.795+0000 7f687f530700 1 -- 172.21.15.92:0/441907539 wait complete. 2023-10-06T03:10:05.809 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T03:10:05.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.921+0000 7f8b41b5b700 1 Processor -- start 2023-10-06T03:10:05.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.921+0000 7f8b41b5b700 1 -- start start 2023-10-06T03:10:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b41b5b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c1597f0 0x7f8b3c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b41b5b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8b3c15d230 0x7f8b3c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b41b5b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3c15dbd0 con 0x7f8b3c159420 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b41b5b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b11d0 con 0x7f8b3c15d230 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b41b5b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0b14b0 con 0x7f8b3c1597f0 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b4135a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8b3c15d230 0x7f8b3c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b40b59700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f8b3c159420 0x7f8b3c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:59602/0 (socket says 172.21.15.92:59602) 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b40b59700 1 -- 172.21.15.92:0/3855178318 learned_addr learned my addr 172.21.15.92:0/3855178318 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b33fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c1597f0 0x7f8b3c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b337fe700 1 -- 172.21.15.92:0/3855178318 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232449621 0 0) 0x7f8b38001110 con 0x7f8b3c159420 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.922+0000 7f8b337fe700 1 -- 172.21.15.92:0/3855178318 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b24003570 con 0x7f8b3c159420 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b33fff700 1 -- 172.21.15.92:0/3855178318 >> v2:172.21.15.136:3300/0 conn(0x7f8b3c15d230 msgr2=0x7f8b3c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b33fff700 1 --2- 172.21.15.92:0/3855178318 >> v2:172.21.15.136:3300/0 conn(0x7f8b3c15d230 0x7f8b3c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b33fff700 1 -- 172.21.15.92:0/3855178318 >> v1:172.21.15.136:6789/0 conn(0x7f8b3c159420 legacy=0x7f8b3c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b33fff700 1 -- 172.21.15.92:0/3855178318 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0b1790 con 0x7f8b3c1597f0 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b4135a700 1 --2- 172.21.15.92:0/3855178318 >> v2:172.21.15.136:3300/0 conn(0x7f8b3c15d230 0x7f8b3c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b33fff700 1 --2- 172.21.15.92:0/3855178318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c1597f0 0x7f8b3c0b2270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0098e0 tx=0x7f8b2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d7ac8d8da597a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b337fe700 1 -- 172.21.15.92:0/3855178318 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c011630 con 0x7f8b3c1597f0 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b337fe700 1 -- 172.21.15.92:0/3855178318 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c011790 con 0x7f8b3c1597f0 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b337fe700 1 -- 172.21.15.92:0/3855178318 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c011b30 con 0x7f8b3c1597f0 2023-10-06T03:10:05.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3855178318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c1597f0 msgr2=0x7f8b3c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3855178318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c1597f0 0x7f8b3c0b2270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0098e0 tx=0x7f8b2c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3855178318 shutdown_connections 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3855178318 >> v2:172.21.15.136:3300/0 conn(0x7f8b3c15d230 0x7f8b3c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3855178318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c1597f0 0x7f8b3c0b2270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.923+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3855178318 >> 172.21.15.92:0/3855178318 conn(0x7f8b3c0d0660 msgr2=0x7f8b3c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.924+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3855178318 shutdown_connections 2023-10-06T03:10:05.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.924+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3855178318 wait complete. 2023-10-06T03:10:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 Processor -- start 2023-10-06T03:10:05.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 -- start start 2023-10-06T03:10:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c159420 0x7f8b3c0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 0x7f8b3c16d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b3c15d230 0x7f8b3c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1552c0 con 0x7f8b3c1597f0 2023-10-06T03:10:05.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cb280 con 0x7f8b3c15d230 2023-10-06T03:10:05.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.925+0000 7f8b41b5b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0ca670 con 0x7f8b3c159420 2023-10-06T03:10:05.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 0x7f8b3c16d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 0x7f8b3c16d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45422/0 (socket says 172.21.15.92:45422) 2023-10-06T03:10:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 -- 172.21.15.92:0/3822601907 learned_addr learned my addr 172.21.15.92:0/3822601907 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b40b59700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c159420 0x7f8b3c0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b4135a700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b3c15d230 0x7f8b3c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 -- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c159420 msgr2=0x7f8b3c0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c159420 0x7f8b3c0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 -- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b3c15d230 msgr2=0x7f8b3c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b3c15d230 0x7f8b3c15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 -- 172.21.15.92:0/3822601907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34009a20 con 0x7f8b3c1597f0 2023-10-06T03:10:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b33fff700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 0x7f8b3c16d7f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0b15f0 tx=0x7f8b2c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c011b10 con 0x7f8b3c1597f0 2023-10-06T03:10:05.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c000cc0 con 0x7f8b3c1597f0 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c009420 con 0x7f8b3c1597f0 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c025620 con 0x7f8b3c1597f0 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.926+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c0cac30 con 0x7f8b3c1597f0 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.927+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c0cb3f0 con 0x7f8b3c1597f0 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.928+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b2c025a10 con 0x7f8b3c1597f0 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.928+0000 7f8b30ff9700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b28067320 0x7f8b280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:05.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.928+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b2c086530 con 0x7f8b3c1597f0 2023-10-06T03:10:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.928+0000 7f8b40b59700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b28067320 0x7f8b280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.928+0000 7f8b40b59700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b28067320 0x7f8b280697d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8b38001270 tx=0x7f8b38007bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:05.939 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:05.932+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b2c053120 con 0x7f8b3c1597f0 2023-10-06T03:10:06.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.108+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b3c0cafd0 con 0x7f8b28067320 2023-10-06T03:10:06.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.116+0000 7f8b30ff9700 1 -- 172.21.15.92:0/3822601907 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311707 (secure 0 0 0) 0x7f8b3c0cafd0 con 0x7f8b28067320 2023-10-06T03:10:06.121 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:06.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b28067320 msgr2=0x7f8b280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b28067320 0x7f8b280697d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8b38001270 tx=0x7f8b38007bf0 comp rx=0 tx=0).stop 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 msgr2=0x7f8b3c16d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 0x7f8b3c16d7f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0b15f0 tx=0x7f8b2c017000 comp rx=0 tx=0).stop 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 shutdown_connections 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b28067320 0x7f8b280697d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b3c15d230 0x7f8b3c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b3c159420 0x7f8b3c0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 --2- 172.21.15.92:0/3822601907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b3c1597f0 0x7f8b3c16d7f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.119+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 >> 172.21.15.92:0/3822601907 conn(0x7f8b3c0d0660 msgr2=0x7f8b3c156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:06.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.120+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 shutdown_connections 2023-10-06T03:10:06.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:06.120+0000 7f8b41b5b700 1 -- 172.21.15.92:0/3822601907 wait complete. 2023-10-06T03:10:06.127 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:06.133 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2023-10-06T03:10:05.285151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428840,"kb_used_data":17768,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752545880,"statfs":{"total":773094113280,"available":770606981120,"internally_reserved":0,"allocated":18194432,"data_stored":9193700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000978"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592260+0000","last_change":"2023-10-06T03:10:00.590921+0000","last_active":"2023-10-06T03:10:00.592260+0000","last_peered":"2023-10-06T03:10:00.592260+0000","last_clean":"2023-10-06T03:10:00.592260+0000","last_became_active":"2023-10-06T03:10:00.590562+0000","last_became_peered":"2023-10-06T03:10:00.590562+0000","last_unstale":"2023-10-06T03:10:00.592260+0000","last_undegraded":"2023-10-06T03:10:00.592260+0000","last_fullsized":"2023-10-06T03:10:00.592260+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:09.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579997+0000","last_change":"2023-10-06T03:10:00.576176+0000","last_active":"2023-10-06T03:10:00.579997+0000","last_peered":"2023-10-06T03:10:00.579997+0000","last_clean":"2023-10-06T03:10:00.579997+0000","last_became_active":"2023-10-06T03:10:00.575892+0000","last_became_peered":"2023-10-06T03:10:00.575892+0000","last_unstale":"2023-10-06T03:10:00.579997+0000","last_undegraded":"2023-10-06T03:10:00.579997+0000","last_fullsized":"2023-10-06T03:10:00.579997+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.584497+0000","last_change":"2023-10-06T03:10:00.579899+0000","last_active":"2023-10-06T03:10:00.584497+0000","last_peered":"2023-10-06T03:10:00.584497+0000","last_clean":"2023-10-06T03:10:00.584497+0000","last_became_active":"2023-10-06T03:10:00.579648+0000","last_became_peered":"2023-10-06T03:10:00.579648+0000","last_unstale":"2023-10-06T03:10:00.584497+0000","last_undegraded":"2023-10-06T03:10:00.584497+0000","last_fullsized":"2023-10-06T03:10:00.584497+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:35.670624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585148+0000","last_change":"2023-10-06T03:10:00.576866+0000","last_active":"2023-10-06T03:10:00.585148+0000","last_peered":"2023-10-06T03:10:00.585148+0000","last_clean":"2023-10-06T03:10:00.585148+0000","last_became_active":"2023-10-06T03:10:00.576583+0000","last_became_peered":"2023-10-06T03:10:00.576583+0000","last_unstale":"2023-10-06T03:10:00.585148+0000","last_undegraded":"2023-10-06T03:10:00.585148+0000","last_fullsized":"2023-10-06T03:10:00.585148+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:25.010008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591219+0000","last_change":"2023-10-06T03:10:00.585448+0000","last_active":"2023-10-06T03:10:00.591219+0000","last_peered":"2023-10-06T03:10:00.591219+0000","last_clean":"2023-10-06T03:10:00.591219+0000","last_became_active":"2023-10-06T03:10:00.584829+0000","last_became_peered":"2023-10-06T03:10:00.584829+0000","last_unstale":"2023-10-06T03:10:00.591219+0000","last_undegraded":"2023-10-06T03:10:00.591219+0000","last_fullsized":"2023-10-06T03:10:00.591219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.911735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593134+0000","last_change":"2023-10-06T03:10:00.592217+0000","last_active":"2023-10-06T03:10:00.593134+0000","last_peered":"2023-10-06T03:10:00.593134+0000","last_clean":"2023-10-06T03:10:00.593134+0000","last_became_active":"2023-10-06T03:10:00.591948+0000","last_became_peered":"2023-10-06T03:10:00.591948+0000","last_unstale":"2023-10-06T03:10:00.593134+0000","last_undegraded":"2023-10-06T03:10:00.593134+0000","last_fullsized":"2023-10-06T03:10:00.593134+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:24.007020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585916+0000","last_change":"2023-10-06T03:10:00.582650+0000","last_active":"2023-10-06T03:10:00.585916+0000","last_peered":"2023-10-06T03:10:00.585916+0000","last_clean":"2023-10-06T03:10:00.585916+0000","last_became_active":"2023-10-06T03:10:00.581814+0000","last_became_peered":"2023-10-06T03:10:00.581814+0000","last_unstale":"2023-10-06T03:10:00.585916+0000","last_undegraded":"2023-10-06T03:10:00.585916+0000","last_fullsized":"2023-10-06T03:10:00.585916+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593113+0000","last_change":"2023-10-06T03:10:00.590222+0000","last_active":"2023-10-06T03:10:00.593113+0000","last_peered":"2023-10-06T03:10:00.593113+0000","last_clean":"2023-10-06T03:10:00.593113+0000","last_became_active":"2023-10-06T03:10:00.589982+0000","last_became_peered":"2023-10-06T03:10:00.589982+0000","last_unstale":"2023-10-06T03:10:00.593113+0000","last_undegraded":"2023-10-06T03:10:00.593113+0000","last_fullsized":"2023-10-06T03:10:00.593113+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:42.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579402+0000","last_change":"2023-10-06T03:10:00.577494+0000","last_active":"2023-10-06T03:10:00.579402+0000","last_peered":"2023-10-06T03:10:00.579402+0000","last_clean":"2023-10-06T03:10:00.579402+0000","last_became_active":"2023-10-06T03:10:00.577250+0000","last_became_peered":"2023-10-06T03:10:00.577250+0000","last_unstale":"2023-10-06T03:10:00.579402+0000","last_undegraded":"2023-10-06T03:10:00.579402+0000","last_fullsized":"2023-10-06T03:10:00.579402+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:59.275214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585983+0000","last_change":"2023-10-06T03:10:00.582063+0000","last_active":"2023-10-06T03:10:00.585983+0000","last_peered":"2023-10-06T03:10:00.585983+0000","last_clean":"2023-10-06T03:10:00.585983+0000","last_became_active":"2023-10-06T03:10:00.581786+0000","last_became_peered":"2023-10-06T03:10:00.581786+0000","last_unstale":"2023-10-06T03:10:00.585983+0000","last_undegraded":"2023-10-06T03:10:00.585983+0000","last_fullsized":"2023-10-06T03:10:00.585983+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:06.937724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586746+0000","last_change":"2023-10-06T03:10:00.583742+0000","last_active":"2023-10-06T03:10:00.586746+0000","last_peered":"2023-10-06T03:10:00.586746+0000","last_clean":"2023-10-06T03:10:00.586746+0000","last_became_active":"2023-10-06T03:10:00.583140+0000","last_became_peered":"2023-10-06T03:10:00.583140+0000","last_unstale":"2023-10-06T03:10:00.586746+0000","last_undegraded":"2023-10-06T03:10:00.586746+0000","last_fullsized":"2023-10-06T03:10:00.586746+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:26.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594983+0000","last_change":"2023-10-06T03:10:00.590922+0000","last_active":"2023-10-06T03:10:00.594983+0000","last_peered":"2023-10-06T03:10:00.594983+0000","last_clean":"2023-10-06T03:10:00.594983+0000","last_became_active":"2023-10-06T03:10:00.586978+0000","last_became_peered":"2023-10-06T03:10:00.586978+0000","last_unstale":"2023-10-06T03:10:00.594983+0000","last_undegraded":"2023-10-06T03:10:00.594983+0000","last_fullsized":"2023-10-06T03:10:00.594983+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:39.335090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592425+0000","last_change":"2023-10-06T03:10:00.590023+0000","last_active":"2023-10-06T03:10:00.592425+0000","last_peered":"2023-10-06T03:10:00.592425+0000","last_clean":"2023-10-06T03:10:00.592425+0000","last_became_active":"2023-10-06T03:10:00.589773+0000","last_became_peered":"2023-10-06T03:10:00.589773+0000","last_unstale":"2023-10-06T03:10:00.592425+0000","last_undegraded":"2023-10-06T03:10:00.592425+0000","last_fullsized":"2023-10-06T03:10:00.592425+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:39.579557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594945+0000","last_change":"2023-10-06T03:10:00.590602+0000","last_active":"2023-10-06T03:10:00.594945+0000","last_peered":"2023-10-06T03:10:00.594945+0000","last_clean":"2023-10-06T03:10:00.594945+0000","last_became_active":"2023-10-06T03:10:00.586819+0000","last_became_peered":"2023-10-06T03:10:00.586819+0000","last_unstale":"2023-10-06T03:10:00.594945+0000","last_undegraded":"2023-10-06T03:10:00.594945+0000","last_fullsized":"2023-10-06T03:10:00.594945+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:00.109760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585396+0000","last_change":"2023-10-06T03:10:00.583053+0000","last_active":"2023-10-06T03:10:00.585396+0000","last_peered":"2023-10-06T03:10:00.585396+0000","last_clean":"2023-10-06T03:10:00.585396+0000","last_became_active":"2023-10-06T03:10:00.582801+0000","last_became_peered":"2023-10-06T03:10:00.582801+0000","last_unstale":"2023-10-06T03:10:00.585396+0000","last_undegraded":"2023-10-06T03:10:00.585396+0000","last_fullsized":"2023-10-06T03:10:00.585396+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:02.664992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.581141+0000","last_change":"2023-10-06T03:10:00.577913+0000","last_active":"2023-10-06T03:10:00.581141+0000","last_peered":"2023-10-06T03:10:00.581141+0000","last_clean":"2023-10-06T03:10:00.581141+0000","last_became_active":"2023-10-06T03:10:00.577496+0000","last_became_peered":"2023-10-06T03:10:00.577496+0000","last_unstale":"2023-10-06T03:10:00.581141+0000","last_undegraded":"2023-10-06T03:10:00.581141+0000","last_fullsized":"2023-10-06T03:10:00.581141+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:39.267712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.598667+0000","last_change":"2023-10-06T03:10:00.597708+0000","last_active":"2023-10-06T03:10:00.598667+0000","last_peered":"2023-10-06T03:10:00.598667+0000","last_clean":"2023-10-06T03:10:00.598667+0000","last_became_active":"2023-10-06T03:10:00.597225+0000","last_became_peered":"2023-10-06T03:10:00.597225+0000","last_unstale":"2023-10-06T03:10:00.598667+0000","last_undegraded":"2023-10-06T03:10:00.598667+0000","last_fullsized":"2023-10-06T03:10:00.598667+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:24.007020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588183+0000","last_change":"2023-10-06T03:10:00.585502+0000","last_active":"2023-10-06T03:10:00.588183+0000","last_peered":"2023-10-06T03:10:00.588183+0000","last_clean":"2023-10-06T03:10:00.588183+0000","last_became_active":"2023-10-06T03:10:00.585258+0000","last_became_peered":"2023-10-06T03:10:00.585258+0000","last_unstale":"2023-10-06T03:10:00.588183+0000","last_undegraded":"2023-10-06T03:10:00.588183+0000","last_fullsized":"2023-10-06T03:10:00.588183+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591484+0000","last_change":"2023-10-06T03:10:00.588935+0000","last_active":"2023-10-06T03:10:00.591484+0000","last_peered":"2023-10-06T03:10:00.591484+0000","last_clean":"2023-10-06T03:10:00.591484+0000","last_became_active":"2023-10-06T03:10:00.588042+0000","last_became_peered":"2023-10-06T03:10:00.588042+0000","last_unstale":"2023-10-06T03:10:00.591484+0000","last_undegraded":"2023-10-06T03:10:00.591484+0000","last_fullsized":"2023-10-06T03:10:00.591484+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.581122+0000","last_change":"2023-10-06T03:10:00.576942+0000","last_active":"2023-10-06T03:10:00.581122+0000","last_peered":"2023-10-06T03:10:00.581122+0000","last_clean":"2023-10-06T03:10:00.581122+0000","last_became_active":"2023-10-06T03:10:00.576620+0000","last_became_peered":"2023-10-06T03:10:00.576620+0000","last_unstale":"2023-10-06T03:10:00.581122+0000","last_undegraded":"2023-10-06T03:10:00.581122+0000","last_fullsized":"2023-10-06T03:10:00.581122+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:22.240676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.584622+0000","last_change":"2023-10-06T03:10:00.579953+0000","last_active":"2023-10-06T03:10:00.584622+0000","last_peered":"2023-10-06T03:10:00.584622+0000","last_clean":"2023-10-06T03:10:00.584622+0000","last_became_active":"2023-10-06T03:10:00.579695+0000","last_became_peered":"2023-10-06T03:10:00.579695+0000","last_unstale":"2023-10-06T03:10:00.584622+0000","last_undegraded":"2023-10-06T03:10:00.584622+0000","last_fullsized":"2023-10-06T03:10:00.584622+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.911735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576451+0000","last_change":"2023-10-06T03:10:00.575164+0000","last_active":"2023-10-06T03:10:00.576451+0000","last_peered":"2023-10-06T03:10:00.576451+0000","last_clean":"2023-10-06T03:10:00.576451+0000","last_became_active":"2023-10-06T03:10:00.574927+0000","last_became_peered":"2023-10-06T03:10:00.574927+0000","last_unstale":"2023-10-06T03:10:00.576451+0000","last_undegraded":"2023-10-06T03:10:00.576451+0000","last_fullsized":"2023-10-06T03:10:00.576451+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.803607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.577631+0000","last_change":"2023-10-06T03:10:00.575464+0000","last_active":"2023-10-06T03:10:00.577631+0000","last_peered":"2023-10-06T03:10:00.577631+0000","last_clean":"2023-10-06T03:10:00.577631+0000","last_became_active":"2023-10-06T03:10:00.575233+0000","last_became_peered":"2023-10-06T03:10:00.575233+0000","last_unstale":"2023-10-06T03:10:00.577631+0000","last_undegraded":"2023-10-06T03:10:00.577631+0000","last_fullsized":"2023-10-06T03:10:00.577631+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:18.843464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591161+0000","last_change":"2023-10-06T03:10:00.586643+0000","last_active":"2023-10-06T03:10:00.591161+0000","last_peered":"2023-10-06T03:10:00.591161+0000","last_clean":"2023-10-06T03:10:00.591161+0000","last_became_active":"2023-10-06T03:10:00.586359+0000","last_became_peered":"2023-10-06T03:10:00.586359+0000","last_unstale":"2023-10-06T03:10:00.591161+0000","last_undegraded":"2023-10-06T03:10:00.591161+0000","last_fullsized":"2023-10-06T03:10:00.591161+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592278+0000","last_change":"2023-10-06T03:10:00.591267+0000","last_active":"2023-10-06T03:10:00.592278+0000","last_peered":"2023-10-06T03:10:00.592278+0000","last_clean":"2023-10-06T03:10:00.592278+0000","last_became_active":"2023-10-06T03:10:00.590664+0000","last_became_peered":"2023-10-06T03:10:00.590664+0000","last_unstale":"2023-10-06T03:10:00.592278+0000","last_undegraded":"2023-10-06T03:10:00.592278+0000","last_fullsized":"2023-10-06T03:10:00.592278+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591365+0000","last_change":"2023-10-06T03:10:00.589001+0000","last_active":"2023-10-06T03:10:00.591365+0000","last_peered":"2023-10-06T03:10:00.591365+0000","last_clean":"2023-10-06T03:10:00.591365+0000","last_became_active":"2023-10-06T03:10:00.588833+0000","last_became_peered":"2023-10-06T03:10:00.588833+0000","last_unstale":"2023-10-06T03:10:00.591365+0000","last_undegraded":"2023-10-06T03:10:00.591365+0000","last_fullsized":"2023-10-06T03:10:00.591365+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:55.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586764+0000","last_change":"2023-10-06T03:10:00.583945+0000","last_active":"2023-10-06T03:10:00.586764+0000","last_peered":"2023-10-06T03:10:00.586764+0000","last_clean":"2023-10-06T03:10:00.586764+0000","last_became_active":"2023-10-06T03:10:00.583483+0000","last_became_peered":"2023-10-06T03:10:00.583483+0000","last_unstale":"2023-10-06T03:10:00.586764+0000","last_undegraded":"2023-10-06T03:10:00.586764+0000","last_fullsized":"2023-10-06T03:10:00.586764+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.043746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595005+0000","last_change":"2023-10-06T03:10:00.591638+0000","last_active":"2023-10-06T03:10:00.595005+0000","last_peered":"2023-10-06T03:10:00.595005+0000","last_clean":"2023-10-06T03:10:00.595005+0000","last_became_active":"2023-10-06T03:10:00.591381+0000","last_became_peered":"2023-10-06T03:10:00.591381+0000","last_unstale":"2023-10-06T03:10:00.595005+0000","last_undegraded":"2023-10-06T03:10:00.595005+0000","last_fullsized":"2023-10-06T03:10:00.595005+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:03.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585883+0000","last_change":"2023-10-06T03:10:00.582573+0000","last_active":"2023-10-06T03:10:00.585883+0000","last_peered":"2023-10-06T03:10:00.585883+0000","last_clean":"2023-10-06T03:10:00.585883+0000","last_became_active":"2023-10-06T03:10:00.582310+0000","last_became_peered":"2023-10-06T03:10:00.582310+0000","last_unstale":"2023-10-06T03:10:00.585883+0000","last_undegraded":"2023-10-06T03:10:00.585883+0000","last_fullsized":"2023-10-06T03:10:00.585883+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:47.002388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595540+0000","last_change":"2023-10-06T03:10:00.594200+0000","last_active":"2023-10-06T03:10:00.595540+0000","last_peered":"2023-10-06T03:10:00.595540+0000","last_clean":"2023-10-06T03:10:00.595540+0000","last_became_active":"2023-10-06T03:10:00.593924+0000","last_became_peered":"2023-10-06T03:10:00.593924+0000","last_unstale":"2023-10-06T03:10:00.595540+0000","last_undegraded":"2023-10-06T03:10:00.595540+0000","last_fullsized":"2023-10-06T03:10:00.595540+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:24.816047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585699+0000","last_change":"2023-10-06T03:10:00.580666+0000","last_active":"2023-10-06T03:10:00.585699+0000","last_peered":"2023-10-06T03:10:00.585699+0000","last_clean":"2023-10-06T03:10:00.585699+0000","last_became_active":"2023-10-06T03:10:00.580418+0000","last_became_peered":"2023-10-06T03:10:00.580418+0000","last_unstale":"2023-10-06T03:10:00.585699+0000","last_undegraded":"2023-10-06T03:10:00.585699+0000","last_fullsized":"2023-10-06T03:10:00.585699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:11.143586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588450+0000","last_change":"2023-10-06T03:10:00.587824+0000","last_active":"2023-10-06T03:10:00.588450+0000","last_peered":"2023-10-06T03:10:00.588450+0000","last_clean":"2023-10-06T03:10:00.588450+0000","last_became_active":"2023-10-06T03:10:00.587522+0000","last_became_peered":"2023-10-06T03:10:00.587522+0000","last_unstale":"2023-10-06T03:10:00.588450+0000","last_undegraded":"2023-10-06T03:10:00.588450+0000","last_fullsized":"2023-10-06T03:10:00.588450+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:22.796834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591074+0000","last_change":"2023-10-06T03:10:00.587669+0000","last_active":"2023-10-06T03:10:00.591074+0000","last_peered":"2023-10-06T03:10:00.591074+0000","last_clean":"2023-10-06T03:10:00.591074+0000","last_became_active":"2023-10-06T03:10:00.587393+0000","last_became_peered":"2023-10-06T03:10:00.587393+0000","last_unstale":"2023-10-06T03:10:00.591074+0000","last_undegraded":"2023-10-06T03:10:00.591074+0000","last_fullsized":"2023-10-06T03:10:00.591074+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:06.249936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579297+0000","last_change":"2023-10-06T03:09:59.579556+0000","last_active":"2023-10-06T03:10:00.579297+0000","last_peered":"2023-10-06T03:10:00.579297+0000","last_clean":"2023-10-06T03:10:00.579297+0000","last_became_active":"2023-10-06T03:09:59.579270+0000","last_became_peered":"2023-10-06T03:09:59.579270+0000","last_unstale":"2023-10-06T03:10:00.579297+0000","last_undegraded":"2023-10-06T03:10:00.579297+0000","last_fullsized":"2023-10-06T03:10:00.579297+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:04.820008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579434+0000","last_change":"2023-10-06T03:10:00.576176+0000","last_active":"2023-10-06T03:10:00.579434+0000","last_peered":"2023-10-06T03:10:00.579434+0000","last_clean":"2023-10-06T03:10:00.579434+0000","last_became_active":"2023-10-06T03:10:00.575676+0000","last_became_peered":"2023-10-06T03:10:00.575676+0000","last_unstale":"2023-10-06T03:10:00.579434+0000","last_undegraded":"2023-10-06T03:10:00.579434+0000","last_fullsized":"2023-10-06T03:10:00.579434+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:08.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570781+0000","last_change":"2023-10-06T03:09:59.586249+0000","last_active":"2023-10-06T03:10:00.570781+0000","last_peered":"2023-10-06T03:10:00.570781+0000","last_clean":"2023-10-06T03:10:00.570781+0000","last_became_active":"2023-10-06T03:09:59.585984+0000","last_became_peered":"2023-10-06T03:09:59.585984+0000","last_unstale":"2023-10-06T03:10:00.570781+0000","last_undegraded":"2023-10-06T03:10:00.570781+0000","last_fullsized":"2023-10-06T03:10:00.570781+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:59.104467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585567+0000","last_change":"2023-10-06T03:10:00.581602+0000","last_active":"2023-10-06T03:10:00.585567+0000","last_peered":"2023-10-06T03:10:00.585567+0000","last_clean":"2023-10-06T03:10:00.585567+0000","last_became_active":"2023-10-06T03:10:00.581253+0000","last_became_peered":"2023-10-06T03:10:00.581253+0000","last_unstale":"2023-10-06T03:10:00.585567+0000","last_undegraded":"2023-10-06T03:10:00.585567+0000","last_fullsized":"2023-10-06T03:10:00.585567+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:09.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"166'3","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.589720+0000","last_change":"2023-10-06T03:09:59.579902+0000","last_active":"2023-10-06T03:10:00.589720+0000","last_peered":"2023-10-06T03:10:00.589720+0000","last_clean":"2023-10-06T03:10:00.589720+0000","last_became_active":"2023-10-06T03:09:59.579620+0000","last_became_peered":"2023-10-06T03:09:59.579620+0000","last_unstale":"2023-10-06T03:10:00.589720+0000","last_undegraded":"2023-10-06T03:10:00.589720+0000","last_fullsized":"2023-10-06T03:10:00.589720+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:02.798313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585756+0000","last_change":"2023-10-06T03:10:00.582773+0000","last_active":"2023-10-06T03:10:00.585756+0000","last_peered":"2023-10-06T03:10:00.585756+0000","last_clean":"2023-10-06T03:10:00.585756+0000","last_became_active":"2023-10-06T03:10:00.581895+0000","last_became_peered":"2023-10-06T03:10:00.581895+0000","last_unstale":"2023-10-06T03:10:00.585756+0000","last_undegraded":"2023-10-06T03:10:00.585756+0000","last_fullsized":"2023-10-06T03:10:00.585756+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"166'6","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597822+0000","last_change":"2023-10-06T03:09:59.588064+0000","last_active":"2023-10-06T03:10:00.597822+0000","last_peered":"2023-10-06T03:10:00.597822+0000","last_clean":"2023-10-06T03:10:00.597822+0000","last_became_active":"2023-10-06T03:09:59.587806+0000","last_became_peered":"2023-10-06T03:09:59.587806+0000","last_unstale":"2023-10-06T03:10:00.597822+0000","last_undegraded":"2023-10-06T03:10:00.597822+0000","last_fullsized":"2023-10-06T03:10:00.597822+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:53.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592520+0000","last_change":"2023-10-06T03:10:00.587580+0000","last_active":"2023-10-06T03:10:00.592520+0000","last_peered":"2023-10-06T03:10:00.592520+0000","last_clean":"2023-10-06T03:10:00.592520+0000","last_became_active":"2023-10-06T03:10:00.587352+0000","last_became_peered":"2023-10-06T03:10:00.587352+0000","last_unstale":"2023-10-06T03:10:00.592520+0000","last_undegraded":"2023-10-06T03:10:00.592520+0000","last_fullsized":"2023-10-06T03:10:00.592520+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595124+0000","last_change":"2023-10-06T03:09:59.585422+0000","last_active":"2023-10-06T03:10:00.595124+0000","last_peered":"2023-10-06T03:10:00.595124+0000","last_clean":"2023-10-06T03:10:00.595124+0000","last_became_active":"2023-10-06T03:09:59.585144+0000","last_became_peered":"2023-10-06T03:09:59.585144+0000","last_unstale":"2023-10-06T03:10:00.595124+0000","last_undegraded":"2023-10-06T03:10:00.595124+0000","last_fullsized":"2023-10-06T03:10:00.595124+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:44.098638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593645+0000","last_change":"2023-10-06T03:10:00.586192+0000","last_active":"2023-10-06T03:10:00.593645+0000","last_peered":"2023-10-06T03:10:00.593645+0000","last_clean":"2023-10-06T03:10:00.593645+0000","last_became_active":"2023-10-06T03:10:00.585916+0000","last_became_peered":"2023-10-06T03:10:00.585916+0000","last_unstale":"2023-10-06T03:10:00.593645+0000","last_undegraded":"2023-10-06T03:10:00.593645+0000","last_fullsized":"2023-10-06T03:10:00.593645+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:45.651788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568664+0000","last_change":"2023-10-06T03:09:59.577719+0000","last_active":"2023-10-06T03:10:00.568664+0000","last_peered":"2023-10-06T03:10:00.568664+0000","last_clean":"2023-10-06T03:10:00.568664+0000","last_became_active":"2023-10-06T03:09:59.577438+0000","last_became_peered":"2023-10-06T03:09:59.577438+0000","last_unstale":"2023-10-06T03:10:00.568664+0000","last_undegraded":"2023-10-06T03:10:00.568664+0000","last_fullsized":"2023-10-06T03:10:00.568664+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:48.569159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580711+0000","last_change":"2023-10-06T03:10:00.578706+0000","last_active":"2023-10-06T03:10:00.580711+0000","last_peered":"2023-10-06T03:10:00.580711+0000","last_clean":"2023-10-06T03:10:00.580711+0000","last_became_active":"2023-10-06T03:10:00.578426+0000","last_became_peered":"2023-10-06T03:10:00.578426+0000","last_unstale":"2023-10-06T03:10:00.580711+0000","last_undegraded":"2023-10-06T03:10:00.580711+0000","last_fullsized":"2023-10-06T03:10:00.580711+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:40.682371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569528+0000","last_change":"2023-10-06T03:09:59.578616+0000","last_active":"2023-10-06T03:10:00.569528+0000","last_peered":"2023-10-06T03:10:00.569528+0000","last_clean":"2023-10-06T03:10:00.569528+0000","last_became_active":"2023-10-06T03:09:59.578324+0000","last_became_peered":"2023-10-06T03:09:59.578324+0000","last_unstale":"2023-10-06T03:10:00.569528+0000","last_undegraded":"2023-10-06T03:10:00.569528+0000","last_fullsized":"2023-10-06T03:10:00.569528+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:18.392314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.583271+0000","last_change":"2023-10-06T03:10:00.581675+0000","last_active":"2023-10-06T03:10:00.583271+0000","last_peered":"2023-10-06T03:10:00.583271+0000","last_clean":"2023-10-06T03:10:00.583271+0000","last_became_active":"2023-10-06T03:10:00.580948+0000","last_became_peered":"2023-10-06T03:10:00.580948+0000","last_unstale":"2023-10-06T03:10:00.583271+0000","last_undegraded":"2023-10-06T03:10:00.583271+0000","last_fullsized":"2023-10-06T03:10:00.583271+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:05.271138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"166'8","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595055+0000","last_change":"2023-10-06T03:09:59.584996+0000","last_active":"2023-10-06T03:10:00.595055+0000","last_peered":"2023-10-06T03:10:00.595055+0000","last_clean":"2023-10-06T03:10:00.595055+0000","last_became_active":"2023-10-06T03:09:59.584702+0000","last_became_peered":"2023-10-06T03:09:59.584702+0000","last_unstale":"2023-10-06T03:10:00.595055+0000","last_undegraded":"2023-10-06T03:10:00.595055+0000","last_fullsized":"2023-10-06T03:10:00.595055+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:13.811016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592469+0000","last_change":"2023-10-06T03:10:00.587246+0000","last_active":"2023-10-06T03:10:00.592469+0000","last_peered":"2023-10-06T03:10:00.592469+0000","last_clean":"2023-10-06T03:10:00.592469+0000","last_became_active":"2023-10-06T03:10:00.586984+0000","last_became_peered":"2023-10-06T03:10:00.586984+0000","last_unstale":"2023-10-06T03:10:00.592469+0000","last_undegraded":"2023-10-06T03:10:00.592469+0000","last_fullsized":"2023-10-06T03:10:00.592469+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:25.739194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594854+0000","last_change":"2023-10-06T03:09:59.584290+0000","last_active":"2023-10-06T03:10:00.594854+0000","last_peered":"2023-10-06T03:10:00.594854+0000","last_clean":"2023-10-06T03:10:00.594854+0000","last_became_active":"2023-10-06T03:09:59.584045+0000","last_became_peered":"2023-10-06T03:09:59.584045+0000","last_unstale":"2023-10-06T03:10:00.594854+0000","last_undegraded":"2023-10-06T03:10:00.594854+0000","last_fullsized":"2023-10-06T03:10:00.594854+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:04:49.405237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591503+0000","last_change":"2023-10-06T03:10:00.589081+0000","last_active":"2023-10-06T03:10:00.591503+0000","last_peered":"2023-10-06T03:10:00.591503+0000","last_clean":"2023-10-06T03:10:00.591503+0000","last_became_active":"2023-10-06T03:10:00.588677+0000","last_became_peered":"2023-10-06T03:10:00.588677+0000","last_unstale":"2023-10-06T03:10:00.591503+0000","last_undegraded":"2023-10-06T03:10:00.591503+0000","last_fullsized":"2023-10-06T03:10:00.591503+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.582700+0000","last_change":"2023-10-06T03:09:59.574540+0000","last_active":"2023-10-06T03:10:00.582700+0000","last_peered":"2023-10-06T03:10:00.582700+0000","last_clean":"2023-10-06T03:10:00.582700+0000","last_became_active":"2023-10-06T03:09:59.574290+0000","last_became_peered":"2023-10-06T03:09:59.574290+0000","last_unstale":"2023-10-06T03:10:00.582700+0000","last_undegraded":"2023-10-06T03:10:00.582700+0000","last_fullsized":"2023-10-06T03:10:00.582700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:28.837966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585579+0000","last_change":"2023-10-06T03:10:00.581659+0000","last_active":"2023-10-06T03:10:00.585579+0000","last_peered":"2023-10-06T03:10:00.585579+0000","last_clean":"2023-10-06T03:10:00.585579+0000","last_became_active":"2023-10-06T03:10:00.579525+0000","last_became_peered":"2023-10-06T03:10:00.579525+0000","last_unstale":"2023-10-06T03:10:00.585579+0000","last_undegraded":"2023-10-06T03:10:00.585579+0000","last_fullsized":"2023-10-06T03:10:00.585579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591892+0000","last_change":"2023-10-06T03:09:59.581150+0000","last_active":"2023-10-06T03:10:00.591892+0000","last_peered":"2023-10-06T03:10:00.591892+0000","last_clean":"2023-10-06T03:10:00.591892+0000","last_became_active":"2023-10-06T03:09:59.580879+0000","last_became_peered":"2023-10-06T03:09:59.580879+0000","last_unstale":"2023-10-06T03:10:00.591892+0000","last_undegraded":"2023-10-06T03:10:00.591892+0000","last_fullsized":"2023-10-06T03:10:00.591892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:01.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597594+0000","last_change":"2023-10-06T03:10:00.594466+0000","last_active":"2023-10-06T03:10:00.597594+0000","last_peered":"2023-10-06T03:10:00.597594+0000","last_clean":"2023-10-06T03:10:00.597594+0000","last_became_active":"2023-10-06T03:10:00.594209+0000","last_became_peered":"2023-10-06T03:10:00.594209+0000","last_unstale":"2023-10-06T03:10:00.597594+0000","last_undegraded":"2023-10-06T03:10:00.597594+0000","last_fullsized":"2023-10-06T03:10:00.597594+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:26.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579636+0000","last_change":"2023-10-06T03:09:59.581085+0000","last_active":"2023-10-06T03:10:00.579636+0000","last_peered":"2023-10-06T03:10:00.579636+0000","last_clean":"2023-10-06T03:10:00.579636+0000","last_became_active":"2023-10-06T03:09:59.580800+0000","last_became_peered":"2023-10-06T03:09:59.580800+0000","last_unstale":"2023-10-06T03:10:00.579636+0000","last_undegraded":"2023-10-06T03:10:00.579636+0000","last_fullsized":"2023-10-06T03:10:00.579636+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:35.139211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597534+0000","last_change":"2023-10-06T03:10:00.596330+0000","last_active":"2023-10-06T03:10:00.597534+0000","last_peered":"2023-10-06T03:10:00.597534+0000","last_clean":"2023-10-06T03:10:00.597534+0000","last_became_active":"2023-10-06T03:10:00.596047+0000","last_became_peered":"2023-10-06T03:10:00.596047+0000","last_unstale":"2023-10-06T03:10:00.597534+0000","last_undegraded":"2023-10-06T03:10:00.597534+0000","last_fullsized":"2023-10-06T03:10:00.597534+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:25.010008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.596134+0000","last_change":"2023-10-06T03:09:59.585047+0000","last_active":"2023-10-06T03:10:00.596134+0000","last_peered":"2023-10-06T03:10:00.596134+0000","last_clean":"2023-10-06T03:10:00.596134+0000","last_became_active":"2023-10-06T03:09:59.584765+0000","last_became_peered":"2023-10-06T03:09:59.584765+0000","last_unstale":"2023-10-06T03:10:00.596134+0000","last_undegraded":"2023-10-06T03:10:00.596134+0000","last_fullsized":"2023-10-06T03:10:00.596134+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591317+0000","last_change":"2023-10-06T03:10:00.586313+0000","last_active":"2023-10-06T03:10:00.591317+0000","last_peered":"2023-10-06T03:10:00.591317+0000","last_clean":"2023-10-06T03:10:00.591317+0000","last_became_active":"2023-10-06T03:10:00.585727+0000","last_became_peered":"2023-10-06T03:10:00.585727+0000","last_unstale":"2023-10-06T03:10:00.591317+0000","last_undegraded":"2023-10-06T03:10:00.591317+0000","last_fullsized":"2023-10-06T03:10:00.591317+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:49.574453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568794+0000","last_change":"2023-10-06T03:09:59.578270+0000","last_active":"2023-10-06T03:10:00.568794+0000","last_peered":"2023-10-06T03:10:00.568794+0000","last_clean":"2023-10-06T03:10:00.568794+0000","last_became_active":"2023-10-06T03:09:59.578053+0000","last_became_peered":"2023-10-06T03:09:59.578053+0000","last_unstale":"2023-10-06T03:10:00.568794+0000","last_undegraded":"2023-10-06T03:10:00.568794+0000","last_fullsized":"2023-10-06T03:10:00.568794+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:38.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593581+0000","last_change":"2023-10-06T03:10:00.593111+0000","last_active":"2023-10-06T03:10:00.593581+0000","last_peered":"2023-10-06T03:10:00.593581+0000","last_clean":"2023-10-06T03:10:00.593581+0000","last_became_active":"2023-10-06T03:10:00.592821+0000","last_became_peered":"2023-10-06T03:10:00.592821+0000","last_unstale":"2023-10-06T03:10:00.593581+0000","last_undegraded":"2023-10-06T03:10:00.593581+0000","last_fullsized":"2023-10-06T03:10:00.593581+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:06.937724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569192+0000","last_change":"2023-10-06T03:09:59.579123+0000","last_active":"2023-10-06T03:10:00.569192+0000","last_peered":"2023-10-06T03:10:00.569192+0000","last_clean":"2023-10-06T03:10:00.569192+0000","last_became_active":"2023-10-06T03:09:59.578903+0000","last_became_peered":"2023-10-06T03:09:59.578903+0000","last_unstale":"2023-10-06T03:10:00.569192+0000","last_undegraded":"2023-10-06T03:10:00.569192+0000","last_fullsized":"2023-10-06T03:10:00.569192+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:38.262418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595574+0000","last_change":"2023-10-06T03:10:00.595104+0000","last_active":"2023-10-06T03:10:00.595574+0000","last_peered":"2023-10-06T03:10:00.595574+0000","last_clean":"2023-10-06T03:10:00.595574+0000","last_became_active":"2023-10-06T03:10:00.594906+0000","last_became_peered":"2023-10-06T03:10:00.594906+0000","last_unstale":"2023-10-06T03:10:00.595574+0000","last_undegraded":"2023-10-06T03:10:00.595574+0000","last_fullsized":"2023-10-06T03:10:00.595574+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:17.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.565462+0000","last_change":"2023-10-06T03:09:59.574352+0000","last_active":"2023-10-06T03:10:00.565462+0000","last_peered":"2023-10-06T03:10:00.565462+0000","last_clean":"2023-10-06T03:10:00.565462+0000","last_became_active":"2023-10-06T03:09:59.574058+0000","last_became_peered":"2023-10-06T03:09:59.574058+0000","last_unstale":"2023-10-06T03:10:00.565462+0000","last_undegraded":"2023-10-06T03:10:00.565462+0000","last_fullsized":"2023-10-06T03:10:00.565462+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:47.797463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579200+0000","last_change":"2023-10-06T03:10:00.574251+0000","last_active":"2023-10-06T03:10:00.579200+0000","last_peered":"2023-10-06T03:10:00.579200+0000","last_clean":"2023-10-06T03:10:00.579200+0000","last_became_active":"2023-10-06T03:10:00.574007+0000","last_became_peered":"2023-10-06T03:10:00.574007+0000","last_unstale":"2023-10-06T03:10:00.579200+0000","last_undegraded":"2023-10-06T03:10:00.579200+0000","last_fullsized":"2023-10-06T03:10:00.579200+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593738+0000","last_change":"2023-10-06T03:09:59.584422+0000","last_active":"2023-10-06T03:10:00.593738+0000","last_peered":"2023-10-06T03:10:00.593738+0000","last_clean":"2023-10-06T03:10:00.593738+0000","last_became_active":"2023-10-06T03:09:59.583935+0000","last_became_peered":"2023-10-06T03:09:59.583935+0000","last_unstale":"2023-10-06T03:10:00.593738+0000","last_undegraded":"2023-10-06T03:10:00.593738+0000","last_fullsized":"2023-10-06T03:10:00.593738+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:53.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591950+0000","last_change":"2023-10-06T03:10:00.590564+0000","last_active":"2023-10-06T03:10:00.591950+0000","last_peered":"2023-10-06T03:10:00.591950+0000","last_clean":"2023-10-06T03:10:00.591950+0000","last_became_active":"2023-10-06T03:10:00.590286+0000","last_became_peered":"2023-10-06T03:10:00.590286+0000","last_unstale":"2023-10-06T03:10:00.591950+0000","last_undegraded":"2023-10-06T03:10:00.591950+0000","last_fullsized":"2023-10-06T03:10:00.591950+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:40.682371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568230+0000","last_change":"2023-10-06T03:09:59.579097+0000","last_active":"2023-10-06T03:10:00.568230+0000","last_peered":"2023-10-06T03:10:00.568230+0000","last_clean":"2023-10-06T03:10:00.568230+0000","last_became_active":"2023-10-06T03:09:59.578839+0000","last_became_peered":"2023-10-06T03:09:59.578839+0000","last_unstale":"2023-10-06T03:10:00.568230+0000","last_undegraded":"2023-10-06T03:10:00.568230+0000","last_fullsized":"2023-10-06T03:10:00.568230+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:21.791540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588959+0000","last_change":"2023-10-06T03:10:00.587760+0000","last_active":"2023-10-06T03:10:00.588959+0000","last_peered":"2023-10-06T03:10:00.588959+0000","last_clean":"2023-10-06T03:10:00.588959+0000","last_became_active":"2023-10-06T03:10:00.587504+0000","last_became_peered":"2023-10-06T03:10:00.587504+0000","last_unstale":"2023-10-06T03:10:00.588959+0000","last_undegraded":"2023-10-06T03:10:00.588959+0000","last_fullsized":"2023-10-06T03:10:00.588959+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.573532+0000","last_change":"2023-10-06T03:09:59.575066+0000","last_active":"2023-10-06T03:10:00.573532+0000","last_peered":"2023-10-06T03:10:00.573532+0000","last_clean":"2023-10-06T03:10:00.573532+0000","last_became_active":"2023-10-06T03:09:59.574835+0000","last_became_peered":"2023-10-06T03:09:59.574835+0000","last_unstale":"2023-10-06T03:10:00.573532+0000","last_undegraded":"2023-10-06T03:10:00.573532+0000","last_fullsized":"2023-10-06T03:10:00.573532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:41.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.577571+0000","last_change":"2023-10-06T03:10:00.574835+0000","last_active":"2023-10-06T03:10:00.577571+0000","last_peered":"2023-10-06T03:10:00.577571+0000","last_clean":"2023-10-06T03:10:00.577571+0000","last_became_active":"2023-10-06T03:10:00.574665+0000","last_became_peered":"2023-10-06T03:10:00.574665+0000","last_unstale":"2023-10-06T03:10:00.577571+0000","last_undegraded":"2023-10-06T03:10:00.577571+0000","last_fullsized":"2023-10-06T03:10:00.577571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:46.345022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579849+0000","last_change":"2023-10-06T03:09:59.584117+0000","last_active":"2023-10-06T03:10:00.579849+0000","last_peered":"2023-10-06T03:10:00.579849+0000","last_clean":"2023-10-06T03:10:00.579849+0000","last_became_active":"2023-10-06T03:09:59.583892+0000","last_became_peered":"2023-10-06T03:09:59.583892+0000","last_unstale":"2023-10-06T03:10:00.579849+0000","last_undegraded":"2023-10-06T03:10:00.579849+0000","last_fullsized":"2023-10-06T03:10:00.579849+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:24.144827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586011+0000","last_change":"2023-10-06T03:10:00.584710+0000","last_active":"2023-10-06T03:10:00.586011+0000","last_peered":"2023-10-06T03:10:00.586011+0000","last_clean":"2023-10-06T03:10:00.586011+0000","last_became_active":"2023-10-06T03:10:00.584403+0000","last_became_peered":"2023-10-06T03:10:00.584403+0000","last_unstale":"2023-10-06T03:10:00.586011+0000","last_undegraded":"2023-10-06T03:10:00.586011+0000","last_fullsized":"2023-10-06T03:10:00.586011+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:58.888506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579750+0000","last_change":"2023-10-06T03:09:59.581174+0000","last_active":"2023-10-06T03:10:00.579750+0000","last_peered":"2023-10-06T03:10:00.579750+0000","last_clean":"2023-10-06T03:10:00.579750+0000","last_became_active":"2023-10-06T03:09:59.580756+0000","last_became_peered":"2023-10-06T03:09:59.580756+0000","last_unstale":"2023-10-06T03:10:00.579750+0000","last_undegraded":"2023-10-06T03:10:00.579750+0000","last_fullsized":"2023-10-06T03:10:00.579750+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:59.106481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585956+0000","last_change":"2023-10-06T03:10:00.585304+0000","last_active":"2023-10-06T03:10:00.585956+0000","last_peered":"2023-10-06T03:10:00.585956+0000","last_clean":"2023-10-06T03:10:00.585956+0000","last_became_active":"2023-10-06T03:10:00.585059+0000","last_became_peered":"2023-10-06T03:10:00.585059+0000","last_unstale":"2023-10-06T03:10:00.585956+0000","last_undegraded":"2023-10-06T03:10:00.585956+0000","last_fullsized":"2023-10-06T03:10:00.585956+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:49.574453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570927+0000","last_change":"2023-10-06T03:09:59.575259+0000","last_active":"2023-10-06T03:10:00.570927+0000","last_peered":"2023-10-06T03:10:00.570927+0000","last_clean":"2023-10-06T03:10:00.570927+0000","last_became_active":"2023-10-06T03:09:59.574967+0000","last_became_peered":"2023-10-06T03:09:59.574967+0000","last_unstale":"2023-10-06T03:10:00.570927+0000","last_undegraded":"2023-10-06T03:10:00.570927+0000","last_fullsized":"2023-10-06T03:10:00.570927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:24.973597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580453+0000","last_change":"2023-10-06T03:10:00.577813+0000","last_active":"2023-10-06T03:10:00.580453+0000","last_peered":"2023-10-06T03:10:00.580453+0000","last_clean":"2023-10-06T03:10:00.580453+0000","last_became_active":"2023-10-06T03:10:00.577551+0000","last_became_peered":"2023-10-06T03:10:00.577551+0000","last_unstale":"2023-10-06T03:10:00.580453+0000","last_undegraded":"2023-10-06T03:10:00.580453+0000","last_fullsized":"2023-10-06T03:10:00.580453+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:52.181676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575127+0000","last_change":"2023-10-06T03:09:59.580165+0000","last_active":"2023-10-06T03:10:00.575127+0000","last_peered":"2023-10-06T03:10:00.575127+0000","last_clean":"2023-10-06T03:10:00.575127+0000","last_became_active":"2023-10-06T03:09:59.579314+0000","last_became_peered":"2023-10-06T03:09:59.579314+0000","last_unstale":"2023-10-06T03:10:00.575127+0000","last_undegraded":"2023-10-06T03:10:00.575127+0000","last_fullsized":"2023-10-06T03:10:00.575127+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:05.244643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595174+0000","last_change":"2023-10-06T03:10:00.592300+0000","last_active":"2023-10-06T03:10:00.595174+0000","last_peered":"2023-10-06T03:10:00.595174+0000","last_clean":"2023-10-06T03:10:00.595174+0000","last_became_active":"2023-10-06T03:10:00.592002+0000","last_became_peered":"2023-10-06T03:10:00.592002+0000","last_unstale":"2023-10-06T03:10:00.595174+0000","last_undegraded":"2023-10-06T03:10:00.595174+0000","last_fullsized":"2023-10-06T03:10:00.595174+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.043746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.583297+0000","last_change":"2023-10-06T03:09:59.582382+0000","last_active":"2023-10-06T03:10:00.583297+0000","last_peered":"2023-10-06T03:10:00.583297+0000","last_clean":"2023-10-06T03:10:00.583297+0000","last_became_active":"2023-10-06T03:09:59.581539+0000","last_became_peered":"2023-10-06T03:09:59.581539+0000","last_unstale":"2023-10-06T03:10:00.583297+0000","last_undegraded":"2023-10-06T03:10:00.583297+0000","last_fullsized":"2023-10-06T03:10:00.583297+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:01.769794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576395+0000","last_change":"2023-10-06T03:10:00.572994+0000","last_active":"2023-10-06T03:10:00.576395+0000","last_peered":"2023-10-06T03:10:00.576395+0000","last_clean":"2023-10-06T03:10:00.576395+0000","last_became_active":"2023-10-06T03:10:00.572728+0000","last_became_peered":"2023-10-06T03:10:00.572728+0000","last_unstale":"2023-10-06T03:10:00.576395+0000","last_undegraded":"2023-10-06T03:10:00.576395+0000","last_fullsized":"2023-10-06T03:10:00.576395+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:28.150464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569869+0000","last_change":"2023-10-06T03:09:59.575701+0000","last_active":"2023-10-06T03:10:00.569869+0000","last_peered":"2023-10-06T03:10:00.569869+0000","last_clean":"2023-10-06T03:10:00.569869+0000","last_became_active":"2023-10-06T03:09:59.575456+0000","last_became_peered":"2023-10-06T03:09:59.575456+0000","last_unstale":"2023-10-06T03:10:00.569869+0000","last_undegraded":"2023-10-06T03:10:00.569869+0000","last_fullsized":"2023-10-06T03:10:00.569869+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:06.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580294+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:00.580294+0000","last_peered":"2023-10-06T03:10:00.580294+0000","last_clean":"2023-10-06T03:10:00.580294+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:00.580294+0000","last_undegraded":"2023-10-06T03:10:00.580294+0000","last_fullsized":"2023-10-06T03:10:00.580294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576433+0000","last_change":"2023-10-06T03:10:00.575092+0000","last_active":"2023-10-06T03:10:00.576433+0000","last_peered":"2023-10-06T03:10:00.576433+0000","last_clean":"2023-10-06T03:10:00.576433+0000","last_became_active":"2023-10-06T03:10:00.574839+0000","last_became_peered":"2023-10-06T03:10:00.574839+0000","last_unstale":"2023-10-06T03:10:00.576433+0000","last_undegraded":"2023-10-06T03:10:00.576433+0000","last_fullsized":"2023-10-06T03:10:00.576433+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:05.825301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570650+0000","last_change":"2023-10-06T03:09:59.585944+0000","last_active":"2023-10-06T03:10:00.570650+0000","last_peered":"2023-10-06T03:10:00.570650+0000","last_clean":"2023-10-06T03:10:00.570650+0000","last_became_active":"2023-10-06T03:09:59.585698+0000","last_became_peered":"2023-10-06T03:09:59.585698+0000","last_unstale":"2023-10-06T03:10:00.570650+0000","last_undegraded":"2023-10-06T03:10:00.570650+0000","last_fullsized":"2023-10-06T03:10:00.570650+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:43.377430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568310+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:00.568310+0000","last_peered":"2023-10-06T03:10:00.568310+0000","last_clean":"2023-10-06T03:10:00.568310+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:00.568310+0000","last_undegraded":"2023-10-06T03:10:00.568310+0000","last_fullsized":"2023-10-06T03:10:00.568310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593671+0000","last_change":"2023-10-06T03:10:00.590105+0000","last_active":"2023-10-06T03:10:00.593671+0000","last_peered":"2023-10-06T03:10:00.593671+0000","last_clean":"2023-10-06T03:10:00.593671+0000","last_became_active":"2023-10-06T03:10:00.589873+0000","last_became_peered":"2023-10-06T03:10:00.589873+0000","last_unstale":"2023-10-06T03:10:00.593671+0000","last_undegraded":"2023-10-06T03:10:00.593671+0000","last_fullsized":"2023-10-06T03:10:00.593671+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:42.817569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594988+0000","last_change":"2023-10-06T03:09:59.584721+0000","last_active":"2023-10-06T03:10:00.594988+0000","last_peered":"2023-10-06T03:10:00.594988+0000","last_clean":"2023-10-06T03:10:00.594988+0000","last_became_active":"2023-10-06T03:09:59.584460+0000","last_became_peered":"2023-10-06T03:09:59.584460+0000","last_unstale":"2023-10-06T03:10:00.594988+0000","last_undegraded":"2023-10-06T03:10:00.594988+0000","last_fullsized":"2023-10-06T03:10:00.594988+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:24.661979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576953+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:00.576953+0000","last_peered":"2023-10-06T03:10:00.576953+0000","last_clean":"2023-10-06T03:10:00.576953+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:00.576953+0000","last_undegraded":"2023-10-06T03:10:00.576953+0000","last_fullsized":"2023-10-06T03:10:00.576953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580569+0000","last_change":"2023-10-06T03:10:00.578820+0000","last_active":"2023-10-06T03:10:00.580569+0000","last_peered":"2023-10-06T03:10:00.580569+0000","last_clean":"2023-10-06T03:10:00.580569+0000","last_became_active":"2023-10-06T03:10:00.578555+0000","last_became_peered":"2023-10-06T03:10:00.578555+0000","last_unstale":"2023-10-06T03:10:00.580569+0000","last_undegraded":"2023-10-06T03:10:00.580569+0000","last_fullsized":"2023-10-06T03:10:00.580569+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:47.002388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"166'4","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591496+0000","last_change":"2023-10-06T03:09:59.576969+0000","last_active":"2023-10-06T03:10:00.591496+0000","last_peered":"2023-10-06T03:10:00.591496+0000","last_clean":"2023-10-06T03:10:00.591496+0000","last_became_active":"2023-10-06T03:09:59.576674+0000","last_became_peered":"2023-10-06T03:09:59.576674+0000","last_unstale":"2023-10-06T03:10:00.591496+0000","last_undegraded":"2023-10-06T03:10:00.591496+0000","last_fullsized":"2023-10-06T03:10:00.591496+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:20.093813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575151+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:00.575151+0000","last_peered":"2023-10-06T03:10:00.575151+0000","last_clean":"2023-10-06T03:10:00.575151+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:00.575151+0000","last_undegraded":"2023-10-06T03:10:00.575151+0000","last_fullsized":"2023-10-06T03:10:00.575151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595586+0000","last_change":"2023-10-06T03:10:00.595130+0000","last_active":"2023-10-06T03:10:00.595586+0000","last_peered":"2023-10-06T03:10:00.595586+0000","last_clean":"2023-10-06T03:10:00.595586+0000","last_became_active":"2023-10-06T03:10:00.594892+0000","last_became_peered":"2023-10-06T03:10:00.594892+0000","last_unstale":"2023-10-06T03:10:00.595586+0000","last_undegraded":"2023-10-06T03:10:00.595586+0000","last_fullsized":"2023-10-06T03:10:00.595586+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:08.545791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.589755+0000","last_change":"2023-10-06T03:09:59.580125+0000","last_active":"2023-10-06T03:10:00.589755+0000","last_peered":"2023-10-06T03:10:00.589755+0000","last_clean":"2023-10-06T03:10:00.589755+0000","last_became_active":"2023-10-06T03:09:59.579797+0000","last_became_peered":"2023-10-06T03:09:59.579797+0000","last_unstale":"2023-10-06T03:10:00.589755+0000","last_undegraded":"2023-10-06T03:10:00.589755+0000","last_fullsized":"2023-10-06T03:10:00.589755+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:01.659699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":287,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575211+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:00.575211+0000","last_peered":"2023-10-06T03:10:00.575211+0000","last_clean":"2023-10-06T03:10:00.575211+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:00.575211+0000","last_undegraded":"2023-10-06T03:10:00.575211+0000","last_fullsized":"2023-10-06T03:10:00.575211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":150,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570531+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:00.570531+0000","last_peered":"2023-10-06T03:10:00.570531+0000","last_clean":"2023-10-06T03:10:00.570531+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:00.570531+0000","last_undegraded":"2023-10-06T03:10:00.570531+0000","last_fullsized":"2023-10-06T03:10:00.570531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580687+0000","last_change":"2023-10-06T03:10:00.578960+0000","last_active":"2023-10-06T03:10:00.580687+0000","last_peered":"2023-10-06T03:10:00.580687+0000","last_clean":"2023-10-06T03:10:00.580687+0000","last_became_active":"2023-10-06T03:10:00.578467+0000","last_became_peered":"2023-10-06T03:10:00.578467+0000","last_unstale":"2023-10-06T03:10:00.580687+0000","last_undegraded":"2023-10-06T03:10:00.580687+0000","last_fullsized":"2023-10-06T03:10:00.580687+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594862+0000","last_change":"2023-10-06T03:09:59.582739+0000","last_active":"2023-10-06T03:10:00.594862+0000","last_peered":"2023-10-06T03:10:00.594862+0000","last_clean":"2023-10-06T03:10:00.594862+0000","last_became_active":"2023-10-06T03:09:59.582478+0000","last_became_peered":"2023-10-06T03:09:59.582478+0000","last_unstale":"2023-10-06T03:10:00.594862+0000","last_undegraded":"2023-10-06T03:10:00.594862+0000","last_fullsized":"2023-10-06T03:10:00.594862+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:20.649970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.565806+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:00.565806+0000","last_peered":"2023-10-06T03:10:00.565806+0000","last_clean":"2023-10-06T03:10:00.565806+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:00.565806+0000","last_undegraded":"2023-10-06T03:10:00.565806+0000","last_fullsized":"2023-10-06T03:10:00.565806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597596+0000","last_change":"2023-10-06T03:10:00.596419+0000","last_active":"2023-10-06T03:10:00.597596+0000","last_peered":"2023-10-06T03:10:00.597596+0000","last_clean":"2023-10-06T03:10:00.597596+0000","last_became_active":"2023-10-06T03:10:00.596142+0000","last_became_peered":"2023-10-06T03:10:00.596142+0000","last_unstale":"2023-10-06T03:10:00.597596+0000","last_undegraded":"2023-10-06T03:10:00.597596+0000","last_fullsized":"2023-10-06T03:10:00.597596+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:48.596753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595426+0000","last_change":"2023-10-06T03:09:59.584917+0000","last_active":"2023-10-06T03:10:00.595426+0000","last_peered":"2023-10-06T03:10:00.595426+0000","last_clean":"2023-10-06T03:10:00.595426+0000","last_became_active":"2023-10-06T03:09:59.584649+0000","last_became_peered":"2023-10-06T03:09:59.584649+0000","last_unstale":"2023-10-06T03:10:00.595426+0000","last_undegraded":"2023-10-06T03:10:00.595426+0000","last_fullsized":"2023-10-06T03:10:00.595426+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:54.965400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":150,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570365+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:00.570365+0000","last_peered":"2023-10-06T03:10:00.570365+0000","last_clean":"2023-10-06T03:10:00.570365+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:00.570365+0000","last_undegraded":"2023-10-06T03:10:00.570365+0000","last_fullsized":"2023-10-06T03:10:00.570365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580208+0000","last_change":"2023-10-06T03:10:00.575607+0000","last_active":"2023-10-06T03:10:00.580208+0000","last_peered":"2023-10-06T03:10:00.580208+0000","last_clean":"2023-10-06T03:10:00.580208+0000","last_became_active":"2023-10-06T03:10:00.575358+0000","last_became_peered":"2023-10-06T03:10:00.575358+0000","last_unstale":"2023-10-06T03:10:00.580208+0000","last_undegraded":"2023-10-06T03:10:00.580208+0000","last_fullsized":"2023-10-06T03:10:00.580208+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:25.739194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592663+0000","last_change":"2023-10-06T03:09:59.585796+0000","last_active":"2023-10-06T03:10:00.592663+0000","last_peered":"2023-10-06T03:10:00.592663+0000","last_clean":"2023-10-06T03:10:00.592663+0000","last_became_active":"2023-10-06T03:09:59.585493+0000","last_became_peered":"2023-10-06T03:09:59.585493+0000","last_unstale":"2023-10-06T03:10:00.592663+0000","last_undegraded":"2023-10-06T03:10:00.592663+0000","last_fullsized":"2023-10-06T03:10:00.592663+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:23.810753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.573930+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:00.573930+0000","last_peered":"2023-10-06T03:10:00.573930+0000","last_clean":"2023-10-06T03:10:00.573930+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:00.573930+0000","last_undegraded":"2023-10-06T03:10:00.573930+0000","last_fullsized":"2023-10-06T03:10:00.573930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304040,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067800,"statfs":{"total":96636764160,"available":96325427200,"internally_reserved":0,"allocated":2719744,"data_stored":1592056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.626},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.438},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303476,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068364,"statfs":{"total":96636764160,"available":96326004736,"internally_reserved":0,"allocated":2142208,"data_stored":1002706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.375}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.433},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.441}]}]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303452,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068388,"statfs":{"total":96636764160,"available":96326029312,"internally_reserved":0,"allocated":2117632,"data_stored":1001044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.434},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.443},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.503}]}]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":2711552,"data_stored":1591412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30399999999999999,"15min":0.31},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32600000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36599999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.51100000000000001,"15min":0.64100000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.34100000000000003},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.20899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.61699999999999999},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.437}]}]},{"osd":2,"up_from":13,"seq":55834574954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":1001632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.28499999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.30399999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.433},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.501},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":1001066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56000000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.498},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":1001134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.441}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.499},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574956,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303468,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068372,"statfs":{"total":96636764160,"available":96326012928,"internally_reserved":0,"allocated":2134016,"data_stored":1002650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:09.141 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:09.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.265+0000 7ff77b59e700 1 Processor -- start 2023-10-06T03:10:09.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.265+0000 7ff77b59e700 1 -- start start 2023-10-06T03:10:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77b59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff77c13f620 0x7ff77c0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77b59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff77c158ad0 0x7ff77c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77b59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0b2080 con 0x7ff77c0c8120 2023-10-06T03:10:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77b59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0c8ae0 con 0x7ff77c158ad0 2023-10-06T03:10:09.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77b59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c06eb10 con 0x7ff77c13f620 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77ad9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff77c158ad0 0x7ff77c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff779d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff77c0c8120 0x7ff77c156270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45170/0 (socket says 172.21.15.92:45170) 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff779d9b700 1 -- 172.21.15.92:0/2012203275 learned_addr learned my addr 172.21.15.92:0/2012203275 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77a59c700 1 --2- 172.21.15.92:0/2012203275 >> v2:172.21.15.136:3300/0 conn(0x7ff77c13f620 0x7ff77c0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77ad9d700 1 -- 172.21.15.92:0/2012203275 >> v2:172.21.15.136:3300/0 conn(0x7ff77c13f620 msgr2=0x7ff77c0af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77ad9d700 1 --2- 172.21.15.92:0/2012203275 >> v2:172.21.15.136:3300/0 conn(0x7ff77c13f620 0x7ff77c0af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77ad9d700 1 -- 172.21.15.92:0/2012203275 >> v1:172.21.15.92:6789/0 conn(0x7ff77c0c8120 legacy=0x7ff77c156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77959a700 1 -- 172.21.15.92:0/2012203275 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077143220 0 0) 0x7ff77c0b2080 con 0x7ff77c0c8120 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.266+0000 7ff77ad9d700 1 -- 172.21.15.92:0/2012203275 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0b2320 con 0x7ff77c158ad0 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77ad9d700 1 --2- 172.21.15.92:0/2012203275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c158ad0 0x7ff77c0c85a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff77c06daa0 tx=0x7ff774009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae0641a3817f363 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77959a700 1 -- 172.21.15.92:0/2012203275 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400e070 con 0x7ff77c158ad0 2023-10-06T03:10:09.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77959a700 1 -- 172.21.15.92:0/2012203275 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77400bb20 con 0x7ff77c158ad0 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77959a700 1 -- 172.21.15.92:0/2012203275 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774012430 con 0x7ff77c158ad0 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 -- 172.21.15.92:0/2012203275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c158ad0 msgr2=0x7ff77c0c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 --2- 172.21.15.92:0/2012203275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c158ad0 0x7ff77c0c85a0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff77c06daa0 tx=0x7ff774009780 comp rx=0 tx=0).stop 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 -- 172.21.15.92:0/2012203275 shutdown_connections 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 --2- 172.21.15.92:0/2012203275 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c158ad0 0x7ff77c0c85a0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 --2- 172.21.15.92:0/2012203275 >> v2:172.21.15.136:3300/0 conn(0x7ff77c13f620 0x7ff77c0af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 -- 172.21.15.92:0/2012203275 >> 172.21.15.92:0/2012203275 conn(0x7ff77c06ddb0 msgr2=0x7ff77c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.267+0000 7ff77b59e700 1 -- 172.21.15.92:0/2012203275 shutdown_connections 2023-10-06T03:10:09.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.268+0000 7ff77b59e700 1 -- 172.21.15.92:0/2012203275 wait complete. 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 Processor -- start 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 -- start start 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 0x7ff77c16d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff77c13f620 0x7ff77c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77c158ad0 0x7ff77c06aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0698c0 con 0x7ff77c0c8120 2023-10-06T03:10:09.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c06a250 con 0x7ff77c13f620 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77b59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c06a810 con 0x7ff77c158ad0 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 0x7ff77c16d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 0x7ff77c16d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45448/0 (socket says 172.21.15.92:45448) 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.269+0000 7ff77a59c700 1 -- 172.21.15.92:0/1863876135 learned_addr learned my addr 172.21.15.92:0/1863876135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77ad9d700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77c158ad0 0x7ff77c06aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff779d9b700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff77c13f620 0x7ff77c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77a59c700 1 -- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77c158ad0 msgr2=0x7ff77c06aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77a59c700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77c158ad0 0x7ff77c06aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77a59c700 1 -- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff77c13f620 msgr2=0x7ff77c1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77a59c700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff77c13f620 0x7ff77c1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77a59c700 1 -- 172.21.15.92:0/1863876135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009a20 con 0x7ff77c0c8120 2023-10-06T03:10:09.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77ad9d700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77c158ad0 0x7ff77c06aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff779d9b700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff77c13f620 0x7ff77c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77a59c700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 0x7ff77c16d960 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00ecb0 tx=0x7ff76c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00c9d0 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00cb30 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c16dea0 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00ced0 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.270+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c069df0 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.271+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c04c3c0 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.272+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff76c013680 con 0x7ff77c0c8120 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.272+0000 7ff76a7fc700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff75c067320 0x7ff75c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.272+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff76c085190 con 0x7ff77c0c8120 2023-10-06T03:10:09.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.272+0000 7ff779d9b700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff75c067320 0x7ff75c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.272+0000 7ff779d9b700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff75c067320 0x7ff75c0697d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff77c13ddb0 tx=0x7ff770009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:09.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.276+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff76c051cc0 con 0x7ff77c0c8120 2023-10-06T03:10:09.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.457+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff77c06a950 con 0x7ff75c067320 2023-10-06T03:10:09.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.466+0000 7ff76a7fc700 1 -- 172.21.15.92:0/1863876135 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311665 (secure 0 0 0) 0x7ff7700058a0 con 0x7ff75c067320 2023-10-06T03:10:09.472 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:09.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff75c067320 msgr2=0x7ff75c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff75c067320 0x7ff75c0697d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff77c13ddb0 tx=0x7ff770009040 comp rx=0 tx=0).stop 2023-10-06T03:10:09.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 msgr2=0x7ff77c16d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 0x7ff77c16d960 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00ecb0 tx=0x7ff76c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 shutdown_connections 2023-10-06T03:10:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff75c067320 0x7ff75c0697d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff77c0c8120 0x7ff77c16d960 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77c158ad0 0x7ff77c06aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 --2- 172.21.15.92:0/1863876135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff77c13f620 0x7ff77c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 >> 172.21.15.92:0/1863876135 conn(0x7ff77c06ddb0 msgr2=0x7ff77c0aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 shutdown_connections 2023-10-06T03:10:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.469+0000 7ff77b59e700 1 -- 172.21.15.92:0/1863876135 wait complete. 2023-10-06T03:10:09.478 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:09.483 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2023-10-06T03:10:09.286123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2428840,"kb_used_data":17768,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752545880,"statfs":{"total":773094113280,"available":770606981120,"internally_reserved":0,"allocated":18194432,"data_stored":9193700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.721864"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592260+0000","last_change":"2023-10-06T03:10:00.590921+0000","last_active":"2023-10-06T03:10:00.592260+0000","last_peered":"2023-10-06T03:10:00.592260+0000","last_clean":"2023-10-06T03:10:00.592260+0000","last_became_active":"2023-10-06T03:10:00.590562+0000","last_became_peered":"2023-10-06T03:10:00.590562+0000","last_unstale":"2023-10-06T03:10:00.592260+0000","last_undegraded":"2023-10-06T03:10:00.592260+0000","last_fullsized":"2023-10-06T03:10:00.592260+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:09.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579997+0000","last_change":"2023-10-06T03:10:00.576176+0000","last_active":"2023-10-06T03:10:00.579997+0000","last_peered":"2023-10-06T03:10:00.579997+0000","last_clean":"2023-10-06T03:10:00.579997+0000","last_became_active":"2023-10-06T03:10:00.575892+0000","last_became_peered":"2023-10-06T03:10:00.575892+0000","last_unstale":"2023-10-06T03:10:00.579997+0000","last_undegraded":"2023-10-06T03:10:00.579997+0000","last_fullsized":"2023-10-06T03:10:00.579997+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.584497+0000","last_change":"2023-10-06T03:10:00.579899+0000","last_active":"2023-10-06T03:10:00.584497+0000","last_peered":"2023-10-06T03:10:00.584497+0000","last_clean":"2023-10-06T03:10:00.584497+0000","last_became_active":"2023-10-06T03:10:00.579648+0000","last_became_peered":"2023-10-06T03:10:00.579648+0000","last_unstale":"2023-10-06T03:10:00.584497+0000","last_undegraded":"2023-10-06T03:10:00.584497+0000","last_fullsized":"2023-10-06T03:10:00.584497+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:11:35.670624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585148+0000","last_change":"2023-10-06T03:10:00.576866+0000","last_active":"2023-10-06T03:10:00.585148+0000","last_peered":"2023-10-06T03:10:00.585148+0000","last_clean":"2023-10-06T03:10:00.585148+0000","last_became_active":"2023-10-06T03:10:00.576583+0000","last_became_peered":"2023-10-06T03:10:00.576583+0000","last_unstale":"2023-10-06T03:10:00.585148+0000","last_undegraded":"2023-10-06T03:10:00.585148+0000","last_fullsized":"2023-10-06T03:10:00.585148+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:25.010008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591219+0000","last_change":"2023-10-06T03:10:00.585448+0000","last_active":"2023-10-06T03:10:00.591219+0000","last_peered":"2023-10-06T03:10:00.591219+0000","last_clean":"2023-10-06T03:10:00.591219+0000","last_became_active":"2023-10-06T03:10:00.584829+0000","last_became_peered":"2023-10-06T03:10:00.584829+0000","last_unstale":"2023-10-06T03:10:00.591219+0000","last_undegraded":"2023-10-06T03:10:00.591219+0000","last_fullsized":"2023-10-06T03:10:00.591219+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.911735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593134+0000","last_change":"2023-10-06T03:10:00.592217+0000","last_active":"2023-10-06T03:10:00.593134+0000","last_peered":"2023-10-06T03:10:00.593134+0000","last_clean":"2023-10-06T03:10:00.593134+0000","last_became_active":"2023-10-06T03:10:00.591948+0000","last_became_peered":"2023-10-06T03:10:00.591948+0000","last_unstale":"2023-10-06T03:10:00.593134+0000","last_undegraded":"2023-10-06T03:10:00.593134+0000","last_fullsized":"2023-10-06T03:10:00.593134+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:24.007020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585916+0000","last_change":"2023-10-06T03:10:00.582650+0000","last_active":"2023-10-06T03:10:00.585916+0000","last_peered":"2023-10-06T03:10:00.585916+0000","last_clean":"2023-10-06T03:10:00.585916+0000","last_became_active":"2023-10-06T03:10:00.581814+0000","last_became_peered":"2023-10-06T03:10:00.581814+0000","last_unstale":"2023-10-06T03:10:00.585916+0000","last_undegraded":"2023-10-06T03:10:00.585916+0000","last_fullsized":"2023-10-06T03:10:00.585916+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593113+0000","last_change":"2023-10-06T03:10:00.590222+0000","last_active":"2023-10-06T03:10:00.593113+0000","last_peered":"2023-10-06T03:10:00.593113+0000","last_clean":"2023-10-06T03:10:00.593113+0000","last_became_active":"2023-10-06T03:10:00.589982+0000","last_became_peered":"2023-10-06T03:10:00.589982+0000","last_unstale":"2023-10-06T03:10:00.593113+0000","last_undegraded":"2023-10-06T03:10:00.593113+0000","last_fullsized":"2023-10-06T03:10:00.593113+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:42.554540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579402+0000","last_change":"2023-10-06T03:10:00.577494+0000","last_active":"2023-10-06T03:10:00.579402+0000","last_peered":"2023-10-06T03:10:00.579402+0000","last_clean":"2023-10-06T03:10:00.579402+0000","last_became_active":"2023-10-06T03:10:00.577250+0000","last_became_peered":"2023-10-06T03:10:00.577250+0000","last_unstale":"2023-10-06T03:10:00.579402+0000","last_undegraded":"2023-10-06T03:10:00.579402+0000","last_fullsized":"2023-10-06T03:10:00.579402+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:24:59.275214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585983+0000","last_change":"2023-10-06T03:10:00.582063+0000","last_active":"2023-10-06T03:10:00.585983+0000","last_peered":"2023-10-06T03:10:00.585983+0000","last_clean":"2023-10-06T03:10:00.585983+0000","last_became_active":"2023-10-06T03:10:00.581786+0000","last_became_peered":"2023-10-06T03:10:00.581786+0000","last_unstale":"2023-10-06T03:10:00.585983+0000","last_undegraded":"2023-10-06T03:10:00.585983+0000","last_fullsized":"2023-10-06T03:10:00.585983+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:06.937724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586746+0000","last_change":"2023-10-06T03:10:00.583742+0000","last_active":"2023-10-06T03:10:00.586746+0000","last_peered":"2023-10-06T03:10:00.586746+0000","last_clean":"2023-10-06T03:10:00.586746+0000","last_became_active":"2023-10-06T03:10:00.583140+0000","last_became_peered":"2023-10-06T03:10:00.583140+0000","last_unstale":"2023-10-06T03:10:00.586746+0000","last_undegraded":"2023-10-06T03:10:00.586746+0000","last_fullsized":"2023-10-06T03:10:00.586746+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:26.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594983+0000","last_change":"2023-10-06T03:10:00.590922+0000","last_active":"2023-10-06T03:10:00.594983+0000","last_peered":"2023-10-06T03:10:00.594983+0000","last_clean":"2023-10-06T03:10:00.594983+0000","last_became_active":"2023-10-06T03:10:00.586978+0000","last_became_peered":"2023-10-06T03:10:00.586978+0000","last_unstale":"2023-10-06T03:10:00.594983+0000","last_undegraded":"2023-10-06T03:10:00.594983+0000","last_fullsized":"2023-10-06T03:10:00.594983+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:39.335090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592425+0000","last_change":"2023-10-06T03:10:00.590023+0000","last_active":"2023-10-06T03:10:00.592425+0000","last_peered":"2023-10-06T03:10:00.592425+0000","last_clean":"2023-10-06T03:10:00.592425+0000","last_became_active":"2023-10-06T03:10:00.589773+0000","last_became_peered":"2023-10-06T03:10:00.589773+0000","last_unstale":"2023-10-06T03:10:00.592425+0000","last_undegraded":"2023-10-06T03:10:00.592425+0000","last_fullsized":"2023-10-06T03:10:00.592425+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:39.579557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594945+0000","last_change":"2023-10-06T03:10:00.590602+0000","last_active":"2023-10-06T03:10:00.594945+0000","last_peered":"2023-10-06T03:10:00.594945+0000","last_clean":"2023-10-06T03:10:00.594945+0000","last_became_active":"2023-10-06T03:10:00.586819+0000","last_became_peered":"2023-10-06T03:10:00.586819+0000","last_unstale":"2023-10-06T03:10:00.594945+0000","last_undegraded":"2023-10-06T03:10:00.594945+0000","last_fullsized":"2023-10-06T03:10:00.594945+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:00.109760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585396+0000","last_change":"2023-10-06T03:10:00.583053+0000","last_active":"2023-10-06T03:10:00.585396+0000","last_peered":"2023-10-06T03:10:00.585396+0000","last_clean":"2023-10-06T03:10:00.585396+0000","last_became_active":"2023-10-06T03:10:00.582801+0000","last_became_peered":"2023-10-06T03:10:00.582801+0000","last_unstale":"2023-10-06T03:10:00.585396+0000","last_undegraded":"2023-10-06T03:10:00.585396+0000","last_fullsized":"2023-10-06T03:10:00.585396+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:02.664992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.581141+0000","last_change":"2023-10-06T03:10:00.577913+0000","last_active":"2023-10-06T03:10:00.581141+0000","last_peered":"2023-10-06T03:10:00.581141+0000","last_clean":"2023-10-06T03:10:00.581141+0000","last_became_active":"2023-10-06T03:10:00.577496+0000","last_became_peered":"2023-10-06T03:10:00.577496+0000","last_unstale":"2023-10-06T03:10:00.581141+0000","last_undegraded":"2023-10-06T03:10:00.581141+0000","last_fullsized":"2023-10-06T03:10:00.581141+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:39.267712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.598667+0000","last_change":"2023-10-06T03:10:00.597708+0000","last_active":"2023-10-06T03:10:00.598667+0000","last_peered":"2023-10-06T03:10:00.598667+0000","last_clean":"2023-10-06T03:10:00.598667+0000","last_became_active":"2023-10-06T03:10:00.597225+0000","last_became_peered":"2023-10-06T03:10:00.597225+0000","last_unstale":"2023-10-06T03:10:00.598667+0000","last_undegraded":"2023-10-06T03:10:00.598667+0000","last_fullsized":"2023-10-06T03:10:00.598667+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:24.007020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588183+0000","last_change":"2023-10-06T03:10:00.585502+0000","last_active":"2023-10-06T03:10:00.588183+0000","last_peered":"2023-10-06T03:10:00.588183+0000","last_clean":"2023-10-06T03:10:00.588183+0000","last_became_active":"2023-10-06T03:10:00.585258+0000","last_became_peered":"2023-10-06T03:10:00.585258+0000","last_unstale":"2023-10-06T03:10:00.588183+0000","last_undegraded":"2023-10-06T03:10:00.588183+0000","last_fullsized":"2023-10-06T03:10:00.588183+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591484+0000","last_change":"2023-10-06T03:10:00.588935+0000","last_active":"2023-10-06T03:10:00.591484+0000","last_peered":"2023-10-06T03:10:00.591484+0000","last_clean":"2023-10-06T03:10:00.591484+0000","last_became_active":"2023-10-06T03:10:00.588042+0000","last_became_peered":"2023-10-06T03:10:00.588042+0000","last_unstale":"2023-10-06T03:10:00.591484+0000","last_undegraded":"2023-10-06T03:10:00.591484+0000","last_fullsized":"2023-10-06T03:10:00.591484+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.581122+0000","last_change":"2023-10-06T03:10:00.576942+0000","last_active":"2023-10-06T03:10:00.581122+0000","last_peered":"2023-10-06T03:10:00.581122+0000","last_clean":"2023-10-06T03:10:00.581122+0000","last_became_active":"2023-10-06T03:10:00.576620+0000","last_became_peered":"2023-10-06T03:10:00.576620+0000","last_unstale":"2023-10-06T03:10:00.581122+0000","last_undegraded":"2023-10-06T03:10:00.581122+0000","last_fullsized":"2023-10-06T03:10:00.581122+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:22.240676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.584622+0000","last_change":"2023-10-06T03:10:00.579953+0000","last_active":"2023-10-06T03:10:00.584622+0000","last_peered":"2023-10-06T03:10:00.584622+0000","last_clean":"2023-10-06T03:10:00.584622+0000","last_became_active":"2023-10-06T03:10:00.579695+0000","last_became_peered":"2023-10-06T03:10:00.579695+0000","last_unstale":"2023-10-06T03:10:00.584622+0000","last_undegraded":"2023-10-06T03:10:00.584622+0000","last_fullsized":"2023-10-06T03:10:00.584622+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:29.911735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576451+0000","last_change":"2023-10-06T03:10:00.575164+0000","last_active":"2023-10-06T03:10:00.576451+0000","last_peered":"2023-10-06T03:10:00.576451+0000","last_clean":"2023-10-06T03:10:00.576451+0000","last_became_active":"2023-10-06T03:10:00.574927+0000","last_became_peered":"2023-10-06T03:10:00.574927+0000","last_unstale":"2023-10-06T03:10:00.576451+0000","last_undegraded":"2023-10-06T03:10:00.576451+0000","last_fullsized":"2023-10-06T03:10:00.576451+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:03.803607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.577631+0000","last_change":"2023-10-06T03:10:00.575464+0000","last_active":"2023-10-06T03:10:00.577631+0000","last_peered":"2023-10-06T03:10:00.577631+0000","last_clean":"2023-10-06T03:10:00.577631+0000","last_became_active":"2023-10-06T03:10:00.575233+0000","last_became_peered":"2023-10-06T03:10:00.575233+0000","last_unstale":"2023-10-06T03:10:00.577631+0000","last_undegraded":"2023-10-06T03:10:00.577631+0000","last_fullsized":"2023-10-06T03:10:00.577631+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:18.843464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591161+0000","last_change":"2023-10-06T03:10:00.586643+0000","last_active":"2023-10-06T03:10:00.591161+0000","last_peered":"2023-10-06T03:10:00.591161+0000","last_clean":"2023-10-06T03:10:00.591161+0000","last_became_active":"2023-10-06T03:10:00.586359+0000","last_became_peered":"2023-10-06T03:10:00.586359+0000","last_unstale":"2023-10-06T03:10:00.591161+0000","last_undegraded":"2023-10-06T03:10:00.591161+0000","last_fullsized":"2023-10-06T03:10:00.591161+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592278+0000","last_change":"2023-10-06T03:10:00.591267+0000","last_active":"2023-10-06T03:10:00.592278+0000","last_peered":"2023-10-06T03:10:00.592278+0000","last_clean":"2023-10-06T03:10:00.592278+0000","last_became_active":"2023-10-06T03:10:00.590664+0000","last_became_peered":"2023-10-06T03:10:00.590664+0000","last_unstale":"2023-10-06T03:10:00.592278+0000","last_undegraded":"2023-10-06T03:10:00.592278+0000","last_fullsized":"2023-10-06T03:10:00.592278+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591365+0000","last_change":"2023-10-06T03:10:00.589001+0000","last_active":"2023-10-06T03:10:00.591365+0000","last_peered":"2023-10-06T03:10:00.591365+0000","last_clean":"2023-10-06T03:10:00.591365+0000","last_became_active":"2023-10-06T03:10:00.588833+0000","last_became_peered":"2023-10-06T03:10:00.588833+0000","last_unstale":"2023-10-06T03:10:00.591365+0000","last_undegraded":"2023-10-06T03:10:00.591365+0000","last_fullsized":"2023-10-06T03:10:00.591365+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:55.353349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586764+0000","last_change":"2023-10-06T03:10:00.583945+0000","last_active":"2023-10-06T03:10:00.586764+0000","last_peered":"2023-10-06T03:10:00.586764+0000","last_clean":"2023-10-06T03:10:00.586764+0000","last_became_active":"2023-10-06T03:10:00.583483+0000","last_became_peered":"2023-10-06T03:10:00.583483+0000","last_unstale":"2023-10-06T03:10:00.586764+0000","last_undegraded":"2023-10-06T03:10:00.586764+0000","last_fullsized":"2023-10-06T03:10:00.586764+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.043746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595005+0000","last_change":"2023-10-06T03:10:00.591638+0000","last_active":"2023-10-06T03:10:00.595005+0000","last_peered":"2023-10-06T03:10:00.595005+0000","last_clean":"2023-10-06T03:10:00.595005+0000","last_became_active":"2023-10-06T03:10:00.591381+0000","last_became_peered":"2023-10-06T03:10:00.591381+0000","last_unstale":"2023-10-06T03:10:00.595005+0000","last_undegraded":"2023-10-06T03:10:00.595005+0000","last_fullsized":"2023-10-06T03:10:00.595005+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:03.821712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585883+0000","last_change":"2023-10-06T03:10:00.582573+0000","last_active":"2023-10-06T03:10:00.585883+0000","last_peered":"2023-10-06T03:10:00.585883+0000","last_clean":"2023-10-06T03:10:00.585883+0000","last_became_active":"2023-10-06T03:10:00.582310+0000","last_became_peered":"2023-10-06T03:10:00.582310+0000","last_unstale":"2023-10-06T03:10:00.585883+0000","last_undegraded":"2023-10-06T03:10:00.585883+0000","last_fullsized":"2023-10-06T03:10:00.585883+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:47.002388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595540+0000","last_change":"2023-10-06T03:10:00.594200+0000","last_active":"2023-10-06T03:10:00.595540+0000","last_peered":"2023-10-06T03:10:00.595540+0000","last_clean":"2023-10-06T03:10:00.595540+0000","last_became_active":"2023-10-06T03:10:00.593924+0000","last_became_peered":"2023-10-06T03:10:00.593924+0000","last_unstale":"2023-10-06T03:10:00.595540+0000","last_undegraded":"2023-10-06T03:10:00.595540+0000","last_fullsized":"2023-10-06T03:10:00.595540+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:24.816047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585699+0000","last_change":"2023-10-06T03:10:00.580666+0000","last_active":"2023-10-06T03:10:00.585699+0000","last_peered":"2023-10-06T03:10:00.585699+0000","last_clean":"2023-10-06T03:10:00.585699+0000","last_became_active":"2023-10-06T03:10:00.580418+0000","last_became_peered":"2023-10-06T03:10:00.580418+0000","last_unstale":"2023-10-06T03:10:00.585699+0000","last_undegraded":"2023-10-06T03:10:00.585699+0000","last_fullsized":"2023-10-06T03:10:00.585699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:18:11.143586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588450+0000","last_change":"2023-10-06T03:10:00.587824+0000","last_active":"2023-10-06T03:10:00.588450+0000","last_peered":"2023-10-06T03:10:00.588450+0000","last_clean":"2023-10-06T03:10:00.588450+0000","last_became_active":"2023-10-06T03:10:00.587522+0000","last_became_peered":"2023-10-06T03:10:00.587522+0000","last_unstale":"2023-10-06T03:10:00.588450+0000","last_undegraded":"2023-10-06T03:10:00.588450+0000","last_fullsized":"2023-10-06T03:10:00.588450+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:22.796834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591074+0000","last_change":"2023-10-06T03:10:00.587669+0000","last_active":"2023-10-06T03:10:00.591074+0000","last_peered":"2023-10-06T03:10:00.591074+0000","last_clean":"2023-10-06T03:10:00.591074+0000","last_became_active":"2023-10-06T03:10:00.587393+0000","last_became_peered":"2023-10-06T03:10:00.587393+0000","last_unstale":"2023-10-06T03:10:00.591074+0000","last_undegraded":"2023-10-06T03:10:00.591074+0000","last_fullsized":"2023-10-06T03:10:00.591074+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:06.249936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579297+0000","last_change":"2023-10-06T03:09:59.579556+0000","last_active":"2023-10-06T03:10:00.579297+0000","last_peered":"2023-10-06T03:10:00.579297+0000","last_clean":"2023-10-06T03:10:00.579297+0000","last_became_active":"2023-10-06T03:09:59.579270+0000","last_became_peered":"2023-10-06T03:09:59.579270+0000","last_unstale":"2023-10-06T03:10:00.579297+0000","last_undegraded":"2023-10-06T03:10:00.579297+0000","last_fullsized":"2023-10-06T03:10:00.579297+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:04.820008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579434+0000","last_change":"2023-10-06T03:10:00.576176+0000","last_active":"2023-10-06T03:10:00.579434+0000","last_peered":"2023-10-06T03:10:00.579434+0000","last_clean":"2023-10-06T03:10:00.579434+0000","last_became_active":"2023-10-06T03:10:00.575676+0000","last_became_peered":"2023-10-06T03:10:00.575676+0000","last_unstale":"2023-10-06T03:10:00.579434+0000","last_undegraded":"2023-10-06T03:10:00.579434+0000","last_fullsized":"2023-10-06T03:10:00.579434+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:11:08.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570781+0000","last_change":"2023-10-06T03:09:59.586249+0000","last_active":"2023-10-06T03:10:00.570781+0000","last_peered":"2023-10-06T03:10:00.570781+0000","last_clean":"2023-10-06T03:10:00.570781+0000","last_became_active":"2023-10-06T03:09:59.585984+0000","last_became_peered":"2023-10-06T03:09:59.585984+0000","last_unstale":"2023-10-06T03:10:00.570781+0000","last_undegraded":"2023-10-06T03:10:00.570781+0000","last_fullsized":"2023-10-06T03:10:00.570781+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:48:59.104467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585567+0000","last_change":"2023-10-06T03:10:00.581602+0000","last_active":"2023-10-06T03:10:00.585567+0000","last_peered":"2023-10-06T03:10:00.585567+0000","last_clean":"2023-10-06T03:10:00.585567+0000","last_became_active":"2023-10-06T03:10:00.581253+0000","last_became_peered":"2023-10-06T03:10:00.581253+0000","last_unstale":"2023-10-06T03:10:00.585567+0000","last_undegraded":"2023-10-06T03:10:00.585567+0000","last_fullsized":"2023-10-06T03:10:00.585567+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:09.623831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"166'3","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.589720+0000","last_change":"2023-10-06T03:09:59.579902+0000","last_active":"2023-10-06T03:10:00.589720+0000","last_peered":"2023-10-06T03:10:00.589720+0000","last_clean":"2023-10-06T03:10:00.589720+0000","last_became_active":"2023-10-06T03:09:59.579620+0000","last_became_peered":"2023-10-06T03:09:59.579620+0000","last_unstale":"2023-10-06T03:10:00.589720+0000","last_undegraded":"2023-10-06T03:10:00.589720+0000","last_fullsized":"2023-10-06T03:10:00.589720+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:02.798313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585756+0000","last_change":"2023-10-06T03:10:00.582773+0000","last_active":"2023-10-06T03:10:00.585756+0000","last_peered":"2023-10-06T03:10:00.585756+0000","last_clean":"2023-10-06T03:10:00.585756+0000","last_became_active":"2023-10-06T03:10:00.581895+0000","last_became_peered":"2023-10-06T03:10:00.581895+0000","last_unstale":"2023-10-06T03:10:00.585756+0000","last_undegraded":"2023-10-06T03:10:00.585756+0000","last_fullsized":"2023-10-06T03:10:00.585756+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:40:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"166'6","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597822+0000","last_change":"2023-10-06T03:09:59.588064+0000","last_active":"2023-10-06T03:10:00.597822+0000","last_peered":"2023-10-06T03:10:00.597822+0000","last_clean":"2023-10-06T03:10:00.597822+0000","last_became_active":"2023-10-06T03:09:59.587806+0000","last_became_peered":"2023-10-06T03:09:59.587806+0000","last_unstale":"2023-10-06T03:10:00.597822+0000","last_undegraded":"2023-10-06T03:10:00.597822+0000","last_fullsized":"2023-10-06T03:10:00.597822+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:53.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592520+0000","last_change":"2023-10-06T03:10:00.587580+0000","last_active":"2023-10-06T03:10:00.592520+0000","last_peered":"2023-10-06T03:10:00.592520+0000","last_clean":"2023-10-06T03:10:00.592520+0000","last_became_active":"2023-10-06T03:10:00.587352+0000","last_became_peered":"2023-10-06T03:10:00.587352+0000","last_unstale":"2023-10-06T03:10:00.592520+0000","last_undegraded":"2023-10-06T03:10:00.592520+0000","last_fullsized":"2023-10-06T03:10:00.592520+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595124+0000","last_change":"2023-10-06T03:09:59.585422+0000","last_active":"2023-10-06T03:10:00.595124+0000","last_peered":"2023-10-06T03:10:00.595124+0000","last_clean":"2023-10-06T03:10:00.595124+0000","last_became_active":"2023-10-06T03:09:59.585144+0000","last_became_peered":"2023-10-06T03:09:59.585144+0000","last_unstale":"2023-10-06T03:10:00.595124+0000","last_undegraded":"2023-10-06T03:10:00.595124+0000","last_fullsized":"2023-10-06T03:10:00.595124+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:27:44.098638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593645+0000","last_change":"2023-10-06T03:10:00.586192+0000","last_active":"2023-10-06T03:10:00.593645+0000","last_peered":"2023-10-06T03:10:00.593645+0000","last_clean":"2023-10-06T03:10:00.593645+0000","last_became_active":"2023-10-06T03:10:00.585916+0000","last_became_peered":"2023-10-06T03:10:00.585916+0000","last_unstale":"2023-10-06T03:10:00.593645+0000","last_undegraded":"2023-10-06T03:10:00.593645+0000","last_fullsized":"2023-10-06T03:10:00.593645+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:45.651788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568664+0000","last_change":"2023-10-06T03:09:59.577719+0000","last_active":"2023-10-06T03:10:00.568664+0000","last_peered":"2023-10-06T03:10:00.568664+0000","last_clean":"2023-10-06T03:10:00.568664+0000","last_became_active":"2023-10-06T03:09:59.577438+0000","last_became_peered":"2023-10-06T03:09:59.577438+0000","last_unstale":"2023-10-06T03:10:00.568664+0000","last_undegraded":"2023-10-06T03:10:00.568664+0000","last_fullsized":"2023-10-06T03:10:00.568664+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:48.569159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580711+0000","last_change":"2023-10-06T03:10:00.578706+0000","last_active":"2023-10-06T03:10:00.580711+0000","last_peered":"2023-10-06T03:10:00.580711+0000","last_clean":"2023-10-06T03:10:00.580711+0000","last_became_active":"2023-10-06T03:10:00.578426+0000","last_became_peered":"2023-10-06T03:10:00.578426+0000","last_unstale":"2023-10-06T03:10:00.580711+0000","last_undegraded":"2023-10-06T03:10:00.580711+0000","last_fullsized":"2023-10-06T03:10:00.580711+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:40.682371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569528+0000","last_change":"2023-10-06T03:09:59.578616+0000","last_active":"2023-10-06T03:10:00.569528+0000","last_peered":"2023-10-06T03:10:00.569528+0000","last_clean":"2023-10-06T03:10:00.569528+0000","last_became_active":"2023-10-06T03:09:59.578324+0000","last_became_peered":"2023-10-06T03:09:59.578324+0000","last_unstale":"2023-10-06T03:10:00.569528+0000","last_undegraded":"2023-10-06T03:10:00.569528+0000","last_fullsized":"2023-10-06T03:10:00.569528+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:18.392314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.583271+0000","last_change":"2023-10-06T03:10:00.581675+0000","last_active":"2023-10-06T03:10:00.583271+0000","last_peered":"2023-10-06T03:10:00.583271+0000","last_clean":"2023-10-06T03:10:00.583271+0000","last_became_active":"2023-10-06T03:10:00.580948+0000","last_became_peered":"2023-10-06T03:10:00.580948+0000","last_unstale":"2023-10-06T03:10:00.583271+0000","last_undegraded":"2023-10-06T03:10:00.583271+0000","last_fullsized":"2023-10-06T03:10:00.583271+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:05.271138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"166'8","reported_seq":18,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595055+0000","last_change":"2023-10-06T03:09:59.584996+0000","last_active":"2023-10-06T03:10:00.595055+0000","last_peered":"2023-10-06T03:10:00.595055+0000","last_clean":"2023-10-06T03:10:00.595055+0000","last_became_active":"2023-10-06T03:09:59.584702+0000","last_became_peered":"2023-10-06T03:09:59.584702+0000","last_unstale":"2023-10-06T03:10:00.595055+0000","last_undegraded":"2023-10-06T03:10:00.595055+0000","last_fullsized":"2023-10-06T03:10:00.595055+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:06:13.811016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592469+0000","last_change":"2023-10-06T03:10:00.587246+0000","last_active":"2023-10-06T03:10:00.592469+0000","last_peered":"2023-10-06T03:10:00.592469+0000","last_clean":"2023-10-06T03:10:00.592469+0000","last_became_active":"2023-10-06T03:10:00.586984+0000","last_became_peered":"2023-10-06T03:10:00.586984+0000","last_unstale":"2023-10-06T03:10:00.592469+0000","last_undegraded":"2023-10-06T03:10:00.592469+0000","last_fullsized":"2023-10-06T03:10:00.592469+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:25.739194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594854+0000","last_change":"2023-10-06T03:09:59.584290+0000","last_active":"2023-10-06T03:10:00.594854+0000","last_peered":"2023-10-06T03:10:00.594854+0000","last_clean":"2023-10-06T03:10:00.594854+0000","last_became_active":"2023-10-06T03:09:59.584045+0000","last_became_peered":"2023-10-06T03:09:59.584045+0000","last_unstale":"2023-10-06T03:10:00.594854+0000","last_undegraded":"2023-10-06T03:10:00.594854+0000","last_fullsized":"2023-10-06T03:10:00.594854+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:04:49.405237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591503+0000","last_change":"2023-10-06T03:10:00.589081+0000","last_active":"2023-10-06T03:10:00.591503+0000","last_peered":"2023-10-06T03:10:00.591503+0000","last_clean":"2023-10-06T03:10:00.591503+0000","last_became_active":"2023-10-06T03:10:00.588677+0000","last_became_peered":"2023-10-06T03:10:00.588677+0000","last_unstale":"2023-10-06T03:10:00.591503+0000","last_undegraded":"2023-10-06T03:10:00.591503+0000","last_fullsized":"2023-10-06T03:10:00.591503+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:28:58.355683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.582700+0000","last_change":"2023-10-06T03:09:59.574540+0000","last_active":"2023-10-06T03:10:00.582700+0000","last_peered":"2023-10-06T03:10:00.582700+0000","last_clean":"2023-10-06T03:10:00.582700+0000","last_became_active":"2023-10-06T03:09:59.574290+0000","last_became_peered":"2023-10-06T03:09:59.574290+0000","last_unstale":"2023-10-06T03:10:00.582700+0000","last_undegraded":"2023-10-06T03:10:00.582700+0000","last_fullsized":"2023-10-06T03:10:00.582700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:31:28.837966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585579+0000","last_change":"2023-10-06T03:10:00.581659+0000","last_active":"2023-10-06T03:10:00.585579+0000","last_peered":"2023-10-06T03:10:00.585579+0000","last_clean":"2023-10-06T03:10:00.585579+0000","last_became_active":"2023-10-06T03:10:00.579525+0000","last_became_peered":"2023-10-06T03:10:00.579525+0000","last_unstale":"2023-10-06T03:10:00.585579+0000","last_undegraded":"2023-10-06T03:10:00.585579+0000","last_fullsized":"2023-10-06T03:10:00.585579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:21.596880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591892+0000","last_change":"2023-10-06T03:09:59.581150+0000","last_active":"2023-10-06T03:10:00.591892+0000","last_peered":"2023-10-06T03:10:00.591892+0000","last_clean":"2023-10-06T03:10:00.591892+0000","last_became_active":"2023-10-06T03:09:59.580879+0000","last_became_peered":"2023-10-06T03:09:59.580879+0000","last_unstale":"2023-10-06T03:10:00.591892+0000","last_undegraded":"2023-10-06T03:10:00.591892+0000","last_fullsized":"2023-10-06T03:10:00.591892+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:01.384563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597594+0000","last_change":"2023-10-06T03:10:00.594466+0000","last_active":"2023-10-06T03:10:00.597594+0000","last_peered":"2023-10-06T03:10:00.597594+0000","last_clean":"2023-10-06T03:10:00.597594+0000","last_became_active":"2023-10-06T03:10:00.594209+0000","last_became_peered":"2023-10-06T03:10:00.594209+0000","last_unstale":"2023-10-06T03:10:00.597594+0000","last_undegraded":"2023-10-06T03:10:00.597594+0000","last_fullsized":"2023-10-06T03:10:00.597594+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:26.337949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579636+0000","last_change":"2023-10-06T03:09:59.581085+0000","last_active":"2023-10-06T03:10:00.579636+0000","last_peered":"2023-10-06T03:10:00.579636+0000","last_clean":"2023-10-06T03:10:00.579636+0000","last_became_active":"2023-10-06T03:09:59.580800+0000","last_became_peered":"2023-10-06T03:09:59.580800+0000","last_unstale":"2023-10-06T03:10:00.579636+0000","last_undegraded":"2023-10-06T03:10:00.579636+0000","last_fullsized":"2023-10-06T03:10:00.579636+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:35.139211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597534+0000","last_change":"2023-10-06T03:10:00.596330+0000","last_active":"2023-10-06T03:10:00.597534+0000","last_peered":"2023-10-06T03:10:00.597534+0000","last_clean":"2023-10-06T03:10:00.597534+0000","last_became_active":"2023-10-06T03:10:00.596047+0000","last_became_peered":"2023-10-06T03:10:00.596047+0000","last_unstale":"2023-10-06T03:10:00.597534+0000","last_undegraded":"2023-10-06T03:10:00.597534+0000","last_fullsized":"2023-10-06T03:10:00.597534+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:25.010008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.596134+0000","last_change":"2023-10-06T03:09:59.585047+0000","last_active":"2023-10-06T03:10:00.596134+0000","last_peered":"2023-10-06T03:10:00.596134+0000","last_clean":"2023-10-06T03:10:00.596134+0000","last_became_active":"2023-10-06T03:09:59.584765+0000","last_became_peered":"2023-10-06T03:09:59.584765+0000","last_unstale":"2023-10-06T03:10:00.596134+0000","last_undegraded":"2023-10-06T03:10:00.596134+0000","last_fullsized":"2023-10-06T03:10:00.596134+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:44.822460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591317+0000","last_change":"2023-10-06T03:10:00.586313+0000","last_active":"2023-10-06T03:10:00.591317+0000","last_peered":"2023-10-06T03:10:00.591317+0000","last_clean":"2023-10-06T03:10:00.591317+0000","last_became_active":"2023-10-06T03:10:00.585727+0000","last_became_peered":"2023-10-06T03:10:00.585727+0000","last_unstale":"2023-10-06T03:10:00.591317+0000","last_undegraded":"2023-10-06T03:10:00.591317+0000","last_fullsized":"2023-10-06T03:10:00.591317+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:49.574453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568794+0000","last_change":"2023-10-06T03:09:59.578270+0000","last_active":"2023-10-06T03:10:00.568794+0000","last_peered":"2023-10-06T03:10:00.568794+0000","last_clean":"2023-10-06T03:10:00.568794+0000","last_became_active":"2023-10-06T03:09:59.578053+0000","last_became_peered":"2023-10-06T03:09:59.578053+0000","last_unstale":"2023-10-06T03:10:00.568794+0000","last_undegraded":"2023-10-06T03:10:00.568794+0000","last_fullsized":"2023-10-06T03:10:00.568794+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:38.574264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593581+0000","last_change":"2023-10-06T03:10:00.593111+0000","last_active":"2023-10-06T03:10:00.593581+0000","last_peered":"2023-10-06T03:10:00.593581+0000","last_clean":"2023-10-06T03:10:00.593581+0000","last_became_active":"2023-10-06T03:10:00.592821+0000","last_became_peered":"2023-10-06T03:10:00.592821+0000","last_unstale":"2023-10-06T03:10:00.593581+0000","last_undegraded":"2023-10-06T03:10:00.593581+0000","last_fullsized":"2023-10-06T03:10:00.593581+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:06.937724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569192+0000","last_change":"2023-10-06T03:09:59.579123+0000","last_active":"2023-10-06T03:10:00.569192+0000","last_peered":"2023-10-06T03:10:00.569192+0000","last_clean":"2023-10-06T03:10:00.569192+0000","last_became_active":"2023-10-06T03:09:59.578903+0000","last_became_peered":"2023-10-06T03:09:59.578903+0000","last_unstale":"2023-10-06T03:10:00.569192+0000","last_undegraded":"2023-10-06T03:10:00.569192+0000","last_fullsized":"2023-10-06T03:10:00.569192+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:38.262418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595574+0000","last_change":"2023-10-06T03:10:00.595104+0000","last_active":"2023-10-06T03:10:00.595574+0000","last_peered":"2023-10-06T03:10:00.595574+0000","last_clean":"2023-10-06T03:10:00.595574+0000","last_became_active":"2023-10-06T03:10:00.594906+0000","last_became_peered":"2023-10-06T03:10:00.594906+0000","last_unstale":"2023-10-06T03:10:00.595574+0000","last_undegraded":"2023-10-06T03:10:00.595574+0000","last_fullsized":"2023-10-06T03:10:00.595574+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:17.220542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.565462+0000","last_change":"2023-10-06T03:09:59.574352+0000","last_active":"2023-10-06T03:10:00.565462+0000","last_peered":"2023-10-06T03:10:00.565462+0000","last_clean":"2023-10-06T03:10:00.565462+0000","last_became_active":"2023-10-06T03:09:59.574058+0000","last_became_peered":"2023-10-06T03:09:59.574058+0000","last_unstale":"2023-10-06T03:10:00.565462+0000","last_undegraded":"2023-10-06T03:10:00.565462+0000","last_fullsized":"2023-10-06T03:10:00.565462+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:47.797463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579200+0000","last_change":"2023-10-06T03:10:00.574251+0000","last_active":"2023-10-06T03:10:00.579200+0000","last_peered":"2023-10-06T03:10:00.579200+0000","last_clean":"2023-10-06T03:10:00.579200+0000","last_became_active":"2023-10-06T03:10:00.574007+0000","last_became_peered":"2023-10-06T03:10:00.574007+0000","last_unstale":"2023-10-06T03:10:00.579200+0000","last_undegraded":"2023-10-06T03:10:00.579200+0000","last_fullsized":"2023-10-06T03:10:00.579200+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:20.706311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593738+0000","last_change":"2023-10-06T03:09:59.584422+0000","last_active":"2023-10-06T03:10:00.593738+0000","last_peered":"2023-10-06T03:10:00.593738+0000","last_clean":"2023-10-06T03:10:00.593738+0000","last_became_active":"2023-10-06T03:09:59.583935+0000","last_became_peered":"2023-10-06T03:09:59.583935+0000","last_unstale":"2023-10-06T03:10:00.593738+0000","last_undegraded":"2023-10-06T03:10:00.593738+0000","last_fullsized":"2023-10-06T03:10:00.593738+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:53.328309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591950+0000","last_change":"2023-10-06T03:10:00.590564+0000","last_active":"2023-10-06T03:10:00.591950+0000","last_peered":"2023-10-06T03:10:00.591950+0000","last_clean":"2023-10-06T03:10:00.591950+0000","last_became_active":"2023-10-06T03:10:00.590286+0000","last_became_peered":"2023-10-06T03:10:00.590286+0000","last_unstale":"2023-10-06T03:10:00.591950+0000","last_undegraded":"2023-10-06T03:10:00.591950+0000","last_fullsized":"2023-10-06T03:10:00.591950+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:40.682371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"167'1","reported_seq":11,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-06T03:10:01.567132+0000","last_change":"2023-10-06T03:09:59.579097+0000","last_active":"2023-10-06T03:10:01.567132+0000","last_peered":"2023-10-06T03:10:01.567132+0000","last_clean":"2023-10-06T03:10:01.567132+0000","last_became_active":"2023-10-06T03:09:59.578839+0000","last_became_peered":"2023-10-06T03:09:59.578839+0000","last_unstale":"2023-10-06T03:10:01.567132+0000","last_undegraded":"2023-10-06T03:10:01.567132+0000","last_fullsized":"2023-10-06T03:10:01.567132+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:37:21.791540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.588959+0000","last_change":"2023-10-06T03:10:00.587760+0000","last_active":"2023-10-06T03:10:00.588959+0000","last_peered":"2023-10-06T03:10:00.588959+0000","last_clean":"2023-10-06T03:10:00.588959+0000","last_became_active":"2023-10-06T03:10:00.587504+0000","last_became_peered":"2023-10-06T03:10:00.587504+0000","last_unstale":"2023-10-06T03:10:00.588959+0000","last_undegraded":"2023-10-06T03:10:00.588959+0000","last_fullsized":"2023-10-06T03:10:00.588959+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:25.851713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.573532+0000","last_change":"2023-10-06T03:09:59.575066+0000","last_active":"2023-10-06T03:10:00.573532+0000","last_peered":"2023-10-06T03:10:00.573532+0000","last_clean":"2023-10-06T03:10:00.573532+0000","last_became_active":"2023-10-06T03:09:59.574835+0000","last_became_peered":"2023-10-06T03:09:59.574835+0000","last_unstale":"2023-10-06T03:10:00.573532+0000","last_undegraded":"2023-10-06T03:10:00.573532+0000","last_fullsized":"2023-10-06T03:10:00.573532+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:41.549247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.577571+0000","last_change":"2023-10-06T03:10:00.574835+0000","last_active":"2023-10-06T03:10:00.577571+0000","last_peered":"2023-10-06T03:10:00.577571+0000","last_clean":"2023-10-06T03:10:00.577571+0000","last_became_active":"2023-10-06T03:10:00.574665+0000","last_became_peered":"2023-10-06T03:10:00.574665+0000","last_unstale":"2023-10-06T03:10:00.577571+0000","last_undegraded":"2023-10-06T03:10:00.577571+0000","last_fullsized":"2023-10-06T03:10:00.577571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:46.345022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579849+0000","last_change":"2023-10-06T03:09:59.584117+0000","last_active":"2023-10-06T03:10:00.579849+0000","last_peered":"2023-10-06T03:10:00.579849+0000","last_clean":"2023-10-06T03:10:00.579849+0000","last_became_active":"2023-10-06T03:09:59.583892+0000","last_became_peered":"2023-10-06T03:09:59.583892+0000","last_unstale":"2023-10-06T03:10:00.579849+0000","last_undegraded":"2023-10-06T03:10:00.579849+0000","last_fullsized":"2023-10-06T03:10:00.579849+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:24.144827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.586011+0000","last_change":"2023-10-06T03:10:00.584710+0000","last_active":"2023-10-06T03:10:00.586011+0000","last_peered":"2023-10-06T03:10:00.586011+0000","last_clean":"2023-10-06T03:10:00.586011+0000","last_became_active":"2023-10-06T03:10:00.584403+0000","last_became_peered":"2023-10-06T03:10:00.584403+0000","last_unstale":"2023-10-06T03:10:00.586011+0000","last_undegraded":"2023-10-06T03:10:00.586011+0000","last_fullsized":"2023-10-06T03:10:00.586011+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:58.888506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.579750+0000","last_change":"2023-10-06T03:09:59.581174+0000","last_active":"2023-10-06T03:10:00.579750+0000","last_peered":"2023-10-06T03:10:00.579750+0000","last_clean":"2023-10-06T03:10:00.579750+0000","last_became_active":"2023-10-06T03:09:59.580756+0000","last_became_peered":"2023-10-06T03:09:59.580756+0000","last_unstale":"2023-10-06T03:10:00.579750+0000","last_undegraded":"2023-10-06T03:10:00.579750+0000","last_fullsized":"2023-10-06T03:10:00.579750+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:19:59.106481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.585956+0000","last_change":"2023-10-06T03:10:00.585304+0000","last_active":"2023-10-06T03:10:00.585956+0000","last_peered":"2023-10-06T03:10:00.585956+0000","last_clean":"2023-10-06T03:10:00.585956+0000","last_became_active":"2023-10-06T03:10:00.585059+0000","last_became_peered":"2023-10-06T03:10:00.585059+0000","last_unstale":"2023-10-06T03:10:00.585956+0000","last_undegraded":"2023-10-06T03:10:00.585956+0000","last_fullsized":"2023-10-06T03:10:00.585956+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:49.574453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570927+0000","last_change":"2023-10-06T03:09:59.575259+0000","last_active":"2023-10-06T03:10:00.570927+0000","last_peered":"2023-10-06T03:10:00.570927+0000","last_clean":"2023-10-06T03:10:00.570927+0000","last_became_active":"2023-10-06T03:09:59.574967+0000","last_became_peered":"2023-10-06T03:09:59.574967+0000","last_unstale":"2023-10-06T03:10:00.570927+0000","last_undegraded":"2023-10-06T03:10:00.570927+0000","last_fullsized":"2023-10-06T03:10:00.570927+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:24.973597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580453+0000","last_change":"2023-10-06T03:10:00.577813+0000","last_active":"2023-10-06T03:10:00.580453+0000","last_peered":"2023-10-06T03:10:00.580453+0000","last_clean":"2023-10-06T03:10:00.580453+0000","last_became_active":"2023-10-06T03:10:00.577551+0000","last_became_peered":"2023-10-06T03:10:00.577551+0000","last_unstale":"2023-10-06T03:10:00.580453+0000","last_undegraded":"2023-10-06T03:10:00.580453+0000","last_fullsized":"2023-10-06T03:10:00.580453+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:52.181676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575127+0000","last_change":"2023-10-06T03:09:59.580165+0000","last_active":"2023-10-06T03:10:00.575127+0000","last_peered":"2023-10-06T03:10:00.575127+0000","last_clean":"2023-10-06T03:10:00.575127+0000","last_became_active":"2023-10-06T03:09:59.579314+0000","last_became_peered":"2023-10-06T03:09:59.579314+0000","last_unstale":"2023-10-06T03:10:00.575127+0000","last_undegraded":"2023-10-06T03:10:00.575127+0000","last_fullsized":"2023-10-06T03:10:00.575127+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:05.244643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595174+0000","last_change":"2023-10-06T03:10:00.592300+0000","last_active":"2023-10-06T03:10:00.595174+0000","last_peered":"2023-10-06T03:10:00.595174+0000","last_clean":"2023-10-06T03:10:00.595174+0000","last_became_active":"2023-10-06T03:10:00.592002+0000","last_became_peered":"2023-10-06T03:10:00.592002+0000","last_unstale":"2023-10-06T03:10:00.595174+0000","last_undegraded":"2023-10-06T03:10:00.595174+0000","last_fullsized":"2023-10-06T03:10:00.595174+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:09.043746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.583297+0000","last_change":"2023-10-06T03:09:59.582382+0000","last_active":"2023-10-06T03:10:00.583297+0000","last_peered":"2023-10-06T03:10:00.583297+0000","last_clean":"2023-10-06T03:10:00.583297+0000","last_became_active":"2023-10-06T03:09:59.581539+0000","last_became_peered":"2023-10-06T03:09:59.581539+0000","last_unstale":"2023-10-06T03:10:00.583297+0000","last_undegraded":"2023-10-06T03:10:00.583297+0000","last_fullsized":"2023-10-06T03:10:00.583297+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:01.769794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576395+0000","last_change":"2023-10-06T03:10:00.572994+0000","last_active":"2023-10-06T03:10:00.576395+0000","last_peered":"2023-10-06T03:10:00.576395+0000","last_clean":"2023-10-06T03:10:00.576395+0000","last_became_active":"2023-10-06T03:10:00.572728+0000","last_became_peered":"2023-10-06T03:10:00.572728+0000","last_unstale":"2023-10-06T03:10:00.576395+0000","last_undegraded":"2023-10-06T03:10:00.576395+0000","last_fullsized":"2023-10-06T03:10:00.576395+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:28.150464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.569869+0000","last_change":"2023-10-06T03:09:59.575701+0000","last_active":"2023-10-06T03:10:00.569869+0000","last_peered":"2023-10-06T03:10:00.569869+0000","last_clean":"2023-10-06T03:10:00.569869+0000","last_became_active":"2023-10-06T03:09:59.575456+0000","last_became_peered":"2023-10-06T03:09:59.575456+0000","last_unstale":"2023-10-06T03:10:00.569869+0000","last_undegraded":"2023-10-06T03:10:00.569869+0000","last_fullsized":"2023-10-06T03:10:00.569869+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:04:06.901285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580294+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:00.580294+0000","last_peered":"2023-10-06T03:10:00.580294+0000","last_clean":"2023-10-06T03:10:00.580294+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:00.580294+0000","last_undegraded":"2023-10-06T03:10:00.580294+0000","last_fullsized":"2023-10-06T03:10:00.580294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576433+0000","last_change":"2023-10-06T03:10:00.575092+0000","last_active":"2023-10-06T03:10:00.576433+0000","last_peered":"2023-10-06T03:10:00.576433+0000","last_clean":"2023-10-06T03:10:00.576433+0000","last_became_active":"2023-10-06T03:10:00.574839+0000","last_became_peered":"2023-10-06T03:10:00.574839+0000","last_unstale":"2023-10-06T03:10:00.576433+0000","last_undegraded":"2023-10-06T03:10:00.576433+0000","last_fullsized":"2023-10-06T03:10:00.576433+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:49:05.825301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570650+0000","last_change":"2023-10-06T03:09:59.585944+0000","last_active":"2023-10-06T03:10:00.570650+0000","last_peered":"2023-10-06T03:10:00.570650+0000","last_clean":"2023-10-06T03:10:00.570650+0000","last_became_active":"2023-10-06T03:09:59.585698+0000","last_became_peered":"2023-10-06T03:09:59.585698+0000","last_unstale":"2023-10-06T03:10:00.570650+0000","last_undegraded":"2023-10-06T03:10:00.570650+0000","last_fullsized":"2023-10-06T03:10:00.570650+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:43.377430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.568310+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:00.568310+0000","last_peered":"2023-10-06T03:10:00.568310+0000","last_clean":"2023-10-06T03:10:00.568310+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:00.568310+0000","last_undegraded":"2023-10-06T03:10:00.568310+0000","last_fullsized":"2023-10-06T03:10:00.568310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.593671+0000","last_change":"2023-10-06T03:10:00.590105+0000","last_active":"2023-10-06T03:10:00.593671+0000","last_peered":"2023-10-06T03:10:00.593671+0000","last_clean":"2023-10-06T03:10:00.593671+0000","last_became_active":"2023-10-06T03:10:00.589873+0000","last_became_peered":"2023-10-06T03:10:00.589873+0000","last_unstale":"2023-10-06T03:10:00.593671+0000","last_undegraded":"2023-10-06T03:10:00.593671+0000","last_fullsized":"2023-10-06T03:10:00.593671+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:42.817569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594988+0000","last_change":"2023-10-06T03:09:59.584721+0000","last_active":"2023-10-06T03:10:00.594988+0000","last_peered":"2023-10-06T03:10:00.594988+0000","last_clean":"2023-10-06T03:10:00.594988+0000","last_became_active":"2023-10-06T03:09:59.584460+0000","last_became_peered":"2023-10-06T03:09:59.584460+0000","last_unstale":"2023-10-06T03:10:00.594988+0000","last_undegraded":"2023-10-06T03:10:00.594988+0000","last_fullsized":"2023-10-06T03:10:00.594988+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:26:24.661979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.576953+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:00.576953+0000","last_peered":"2023-10-06T03:10:00.576953+0000","last_clean":"2023-10-06T03:10:00.576953+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:00.576953+0000","last_undegraded":"2023-10-06T03:10:00.576953+0000","last_fullsized":"2023-10-06T03:10:00.576953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580569+0000","last_change":"2023-10-06T03:10:00.578820+0000","last_active":"2023-10-06T03:10:00.580569+0000","last_peered":"2023-10-06T03:10:00.580569+0000","last_clean":"2023-10-06T03:10:00.580569+0000","last_became_active":"2023-10-06T03:10:00.578555+0000","last_became_peered":"2023-10-06T03:10:00.578555+0000","last_unstale":"2023-10-06T03:10:00.580569+0000","last_undegraded":"2023-10-06T03:10:00.580569+0000","last_fullsized":"2023-10-06T03:10:00.580569+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:51:47.002388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"166'4","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.591496+0000","last_change":"2023-10-06T03:09:59.576969+0000","last_active":"2023-10-06T03:10:00.591496+0000","last_peered":"2023-10-06T03:10:00.591496+0000","last_clean":"2023-10-06T03:10:00.591496+0000","last_became_active":"2023-10-06T03:09:59.576674+0000","last_became_peered":"2023-10-06T03:09:59.576674+0000","last_unstale":"2023-10-06T03:10:00.591496+0000","last_undegraded":"2023-10-06T03:10:00.591496+0000","last_fullsized":"2023-10-06T03:10:00.591496+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:20.093813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575151+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:00.575151+0000","last_peered":"2023-10-06T03:10:00.575151+0000","last_clean":"2023-10-06T03:10:00.575151+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:00.575151+0000","last_undegraded":"2023-10-06T03:10:00.575151+0000","last_fullsized":"2023-10-06T03:10:00.575151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595586+0000","last_change":"2023-10-06T03:10:00.595130+0000","last_active":"2023-10-06T03:10:00.595586+0000","last_peered":"2023-10-06T03:10:00.595586+0000","last_clean":"2023-10-06T03:10:00.595586+0000","last_became_active":"2023-10-06T03:10:00.594892+0000","last_became_peered":"2023-10-06T03:10:00.594892+0000","last_unstale":"2023-10-06T03:10:00.595586+0000","last_undegraded":"2023-10-06T03:10:00.595586+0000","last_fullsized":"2023-10-06T03:10:00.595586+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:08.545791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.589755+0000","last_change":"2023-10-06T03:09:59.580125+0000","last_active":"2023-10-06T03:10:00.589755+0000","last_peered":"2023-10-06T03:10:00.589755+0000","last_clean":"2023-10-06T03:10:00.589755+0000","last_became_active":"2023-10-06T03:09:59.579797+0000","last_became_peered":"2023-10-06T03:09:59.579797+0000","last_unstale":"2023-10-06T03:10:00.589755+0000","last_undegraded":"2023-10-06T03:10:00.589755+0000","last_fullsized":"2023-10-06T03:10:00.589755+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:01.659699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":287,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.575211+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:00.575211+0000","last_peered":"2023-10-06T03:10:00.575211+0000","last_clean":"2023-10-06T03:10:00.575211+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:00.575211+0000","last_undegraded":"2023-10-06T03:10:00.575211+0000","last_fullsized":"2023-10-06T03:10:00.575211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":150,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570531+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:00.570531+0000","last_peered":"2023-10-06T03:10:00.570531+0000","last_clean":"2023-10-06T03:10:00.570531+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:00.570531+0000","last_undegraded":"2023-10-06T03:10:00.570531+0000","last_fullsized":"2023-10-06T03:10:00.570531+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580687+0000","last_change":"2023-10-06T03:10:00.578960+0000","last_active":"2023-10-06T03:10:00.580687+0000","last_peered":"2023-10-06T03:10:00.580687+0000","last_clean":"2023-10-06T03:10:00.580687+0000","last_became_active":"2023-10-06T03:10:00.578467+0000","last_became_peered":"2023-10-06T03:10:00.578467+0000","last_unstale":"2023-10-06T03:10:00.580687+0000","last_undegraded":"2023-10-06T03:10:00.580687+0000","last_fullsized":"2023-10-06T03:10:00.580687+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:04:34.884260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.594862+0000","last_change":"2023-10-06T03:09:59.582739+0000","last_active":"2023-10-06T03:10:00.594862+0000","last_peered":"2023-10-06T03:10:00.594862+0000","last_clean":"2023-10-06T03:10:00.594862+0000","last_became_active":"2023-10-06T03:09:59.582478+0000","last_became_peered":"2023-10-06T03:09:59.582478+0000","last_unstale":"2023-10-06T03:10:00.594862+0000","last_undegraded":"2023-10-06T03:10:00.594862+0000","last_fullsized":"2023-10-06T03:10:00.594862+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:20.649970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.565806+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:00.565806+0000","last_peered":"2023-10-06T03:10:00.565806+0000","last_clean":"2023-10-06T03:10:00.565806+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:00.565806+0000","last_undegraded":"2023-10-06T03:10:00.565806+0000","last_fullsized":"2023-10-06T03:10:00.565806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.597596+0000","last_change":"2023-10-06T03:10:00.596419+0000","last_active":"2023-10-06T03:10:00.597596+0000","last_peered":"2023-10-06T03:10:00.597596+0000","last_clean":"2023-10-06T03:10:00.597596+0000","last_became_active":"2023-10-06T03:10:00.596142+0000","last_became_peered":"2023-10-06T03:10:00.596142+0000","last_unstale":"2023-10-06T03:10:00.597596+0000","last_undegraded":"2023-10-06T03:10:00.597596+0000","last_fullsized":"2023-10-06T03:10:00.597596+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:48.596753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"166'2","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.595426+0000","last_change":"2023-10-06T03:09:59.584917+0000","last_active":"2023-10-06T03:10:00.595426+0000","last_peered":"2023-10-06T03:10:00.595426+0000","last_clean":"2023-10-06T03:10:00.595426+0000","last_became_active":"2023-10-06T03:09:59.584649+0000","last_became_peered":"2023-10-06T03:09:59.584649+0000","last_unstale":"2023-10-06T03:10:00.595426+0000","last_undegraded":"2023-10-06T03:10:00.595426+0000","last_fullsized":"2023-10-06T03:10:00.595426+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:54.965400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":150,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.570365+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:00.570365+0000","last_peered":"2023-10-06T03:10:00.570365+0000","last_clean":"2023-10-06T03:10:00.570365+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:00.570365+0000","last_undegraded":"2023-10-06T03:10:00.570365+0000","last_fullsized":"2023-10-06T03:10:00.570365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.580208+0000","last_change":"2023-10-06T03:10:00.575607+0000","last_active":"2023-10-06T03:10:00.580208+0000","last_peered":"2023-10-06T03:10:00.580208+0000","last_clean":"2023-10-06T03:10:00.580208+0000","last_became_active":"2023-10-06T03:10:00.575358+0000","last_became_peered":"2023-10-06T03:10:00.575358+0000","last_unstale":"2023-10-06T03:10:00.580208+0000","last_undegraded":"2023-10-06T03:10:00.580208+0000","last_fullsized":"2023-10-06T03:10:00.580208+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:59.555617+0000","last_clean_scrub_stamp":"2023-10-06T03:09:59.555617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:25.739194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"166'1","reported_seq":11,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.592663+0000","last_change":"2023-10-06T03:09:59.585796+0000","last_active":"2023-10-06T03:10:00.592663+0000","last_peered":"2023-10-06T03:10:00.592663+0000","last_clean":"2023-10-06T03:10:00.592663+0000","last_became_active":"2023-10-06T03:09:59.585493+0000","last_became_peered":"2023-10-06T03:09:59.585493+0000","last_unstale":"2023-10-06T03:10:00.592663+0000","last_undegraded":"2023-10-06T03:10:00.592663+0000","last_fullsized":"2023-10-06T03:10:00.592663+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:09:58.550323+0000","last_clean_scrub_stamp":"2023-10-06T03:09:58.550323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:07:23.810753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":166,"state":"active+clean","last_fresh":"2023-10-06T03:10:00.573930+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:00.573930+0000","last_peered":"2023-10-06T03:10:00.573930+0000","last_clean":"2023-10-06T03:10:00.573930+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:00.573930+0000","last_undegraded":"2023-10-06T03:10:00.573930+0000","last_fullsized":"2023-10-06T03:10:00.573930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304040,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067800,"statfs":{"total":96636764160,"available":96325427200,"internally_reserved":0,"allocated":2719744,"data_stored":1592056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.746}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.66400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303476,"kb_used_data":2092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068364,"statfs":{"total":96636764160,"available":96326004736,"internally_reserved":0,"allocated":2142208,"data_stored":1002706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.627}]}]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303452,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068388,"statfs":{"total":96636764160,"available":96326029312,"internally_reserved":0,"allocated":2117632,"data_stored":1001044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.434},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.443},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.503}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304032,"kb_used_data":2648,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067808,"statfs":{"total":96636764160,"available":96325435392,"internally_reserved":0,"allocated":2711552,"data_stored":1591412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.30399999999999999,"15min":0.31},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.50600000000000001,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.32600000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.33800000000000002,"15min":0.33500000000000002},"min":{"1min":0.27900000000000003,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.505,"5min":0.51100000000000001,"15min":0.64100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24199999999999999},"max":{"1min":0.52500000000000002,"5min":0.56100000000000005,"15min":0.57199999999999995},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.34100000000000003},"min":{"1min":0.255,"5min":0.23100000000000001,"15min":0.20899999999999999},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.372},{"interface":"front","average":{"1min":0.377,"5min":0.36699999999999999,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.61699999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57699999999999996,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.99199999999999999},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58499999999999996,"15min":0.58999999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.79400000000000004,"5min":0.85899999999999999,"15min":0.92100000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58599999999999997,"15min":0.60399999999999998},"min":{"1min":0.436,"5min":0.375,"15min":0.375},"max":{"1min":0.747,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81000000000000005,"5min":0.878,"15min":1.0089999999999999},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:09:37 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.58899999999999997,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.47499999999999998,"5min":0.41799999999999998,"15min":0.39000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.65100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303460,"kb_used_data":2076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068380,"statfs":{"total":96636764160,"available":96326021120,"internally_reserved":0,"allocated":2125824,"data_stored":1001632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.37}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.497},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.41999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":1001066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35799999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38700000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.623}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65100000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303456,"kb_used_data":2072,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068384,"statfs":{"total":96636764160,"available":96326025216,"internally_reserved":0,"allocated":2121728,"data_stored":1001134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.68999999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.437},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303468,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068372,"statfs":{"total":96636764160,"available":96326012928,"internally_reserved":0,"allocated":2134016,"data_stored":1002650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:09.492 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:10:09.492 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:10:09.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.644+0000 7fa2ba8fe700 1 Processor -- start 2023-10-06T03:10:09.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.644+0000 7fa2ba8fe700 1 -- start start 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba8fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 0x7fa2b406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba8fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa2b41577c0 0x7fa2b406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba8fe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b415ad70 con 0x7fa2b4068390 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba8fe700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b40c8610 con 0x7fa2b41577c0 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba8fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b415a160 con 0x7fa2b41573f0 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba0fd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa2b4068390 0x7fa2b415d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47222/0 (socket says 172.21.15.92:47222) 2023-10-06T03:10:09.653 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/890012184 learned_addr learned my addr 172.21.15.92:0/890012184 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b90fb700 1 --2- 172.21.15.92:0/890012184 >> v2:172.21.15.136:3300/0 conn(0x7fa2b41577c0 0x7fa2b406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b98fc700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 0x7fa2b406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487950666 0 0) 0x7fa2b415ad70 con 0x7fa2b4068390 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa29c003570 con 0x7fa2b4068390 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 857270370 0 0) 0x7fa29c003570 con 0x7fa2b4068390 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 msgr2=0x7fa2b406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 0x7fa2b406c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 >> v2:172.21.15.136:3300/0 conn(0x7fa2b41577c0 msgr2=0x7fa2b406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 --2- 172.21.15.92:0/890012184 >> v2:172.21.15.136:3300/0 conn(0x7fa2b41577c0 0x7fa2b406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b415a440 con 0x7fa2b4068390 2023-10-06T03:10:09.654 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fa2a4001470 con 0x7fa2b4068390 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.645+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 >> v1:172.21.15.92:6789/0 conn(0x7fa2b4068390 legacy=0x7fa2b415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa29c003a60 0x7fa29c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 0x7fa29c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa29c004d50 0x7fa29c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40c8610 con 0x7fa29c004d50 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b415a160 con 0x7fa29c003a60 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c00ba60 con 0x7fa2b41573f0 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa29c004d50 0x7fa29c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b98fc700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa29c003a60 0x7fa29c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b90fb700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 0x7fa29c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 msgr2=0x7fa29c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:09.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b41573f0 0x7fa29c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa29c003a60 msgr2=0x7fa29c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa29c003a60 0x7fa29c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/890012184 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c003570 con 0x7fa29c004d50 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa29c004d50 0x7fa29c0051c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c3c0 tx=0x7fa2b415ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e933022d82dfaafe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cc30 con 0x7fa29c004d50 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a400cd90 con 0x7fa29c004d50 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.646+0000 7fa2b88fa700 1 -- 172.21.15.92:0/890012184 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4005b70 con 0x7fa29c004d50 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/890012184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa29c004d50 msgr2=0x7fa29c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/890012184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa29c004d50 0x7fa29c0051c0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400c3c0 tx=0x7fa2b415ace0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/890012184 reap_dead start 2023-10-06T03:10:09.656 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/890012184 shutdown_connections 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/890012184 >> 172.21.15.92:0/890012184 conn(0x7fa2b40cddc0 msgr2=0x7fa2b40c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/890012184 shutdown_connections 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.647+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/890012184 wait complete. 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.648+0000 7fa2ba8fe700 1 Processor -- start 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.648+0000 7fa2ba8fe700 1 -- start start 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b4149f60 0x7fa2b40c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2b414a330 0x7fa2b40c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 0x7fa2b415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b406aca0 con 0x7fa2b40a98d0 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b406b7e0 con 0x7fa2b414a330 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b406aeb0 con 0x7fa2b4149f60 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 0x7fa2b415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 0x7fa2b415f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45968/0 (socket says 172.21.15.92:45968) 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/3151038761 learned_addr learned my addr 172.21.15.92:0/3151038761 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2b90fb700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2b414a330 0x7fa2b40c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2b98fc700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b4149f60 0x7fa2b40c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b4149f60 msgr2=0x7fa2b40c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b4149f60 0x7fa2b40c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2b414a330 msgr2=0x7fa2b40c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2b414a330 0x7fa2b40c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 -- 172.21.15.92:0/3151038761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b406b190 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2b90fb700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2b414a330 0x7fa2b40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba0fd700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 0x7fa2b415f610 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4011000 tx=0x7fa2a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cc30 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4001430 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b406b470 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.649+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40074f0 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.650+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b40a9d50 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.650+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b40aa000 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.651+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa2a401e430 con 0x7fa2b40a98d0 2023-10-06T03:10:09.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.651+0000 7fa2b1ffb700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2a0067320 0x7fa2a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:09.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.651+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa2a4085960 con 0x7fa2b40a98d0 2023-10-06T03:10:09.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.651+0000 7fa2b98fc700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2a0067320 0x7fa2a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:09.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.652+0000 7fa2b98fc700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2a0067320 0x7fa2a00697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009fd0 tx=0x7fa2ac00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:09.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.655+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2a4052490 con 0x7fa2b40a98d0 2023-10-06T03:10:09.915 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.913+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa2b40c3d10 con 0x7fa2b40a98d0 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.917+0000 7fa2b1ffb700 1 -- 172.21.15.92:0/3151038761 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa2a40170e0 con 0x7fa2b40a98d0 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2a0067320 msgr2=0x7fa2a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2a0067320 0x7fa2a00697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009fd0 tx=0x7fa2ac00b040 comp rx=0 tx=0).stop 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 msgr2=0x7fa2b415f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 0x7fa2b415f610 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4011000 tx=0x7fa2a4015000 comp rx=0 tx=0).stop 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 shutdown_connections 2023-10-06T03:10:09.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa2a0067320 0x7fa2a00697d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2b40a98d0 0x7fa2b415f610 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa2b4149f60 0x7fa2b40c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 --2- 172.21.15.92:0/3151038761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2b414a330 0x7fa2b40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 >> 172.21.15.92:0/3151038761 conn(0x7fa2b40cddc0 msgr2=0x7fa2b40c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 shutdown_connections 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:09.919+0000 7fa2ba8fe700 1 -- 172.21.15.92:0/3151038761 wait complete. 2023-10-06T03:10:09.951 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:10:09.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 Processor -- start 2023-10-06T03:10:10.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 -- start start 2023-10-06T03:10:10.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30b8150180 0x7f30b8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b806e2a0 con 0x7f30b80c7240 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80c76c0 con 0x7f30b8150180 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.065+0000 7f30bcb9f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b806b2e0 con 0x7f30b8150550 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30b8150180 0x7f30b8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30b8150180 0x7f30b8157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45974/0 (socket says 172.21.15.92:45974) 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 -- 172.21.15.92:0/1841375222 learned_addr learned my addr 172.21.15.92:0/1841375222 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b6ffd700 1 --2- 172.21.15.92:0/1841375222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 -- 172.21.15.92:0/1841375222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 msgr2=0x7f30b80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 --2- 172.21.15.92:0/1841375222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 -- 172.21.15.92:0/1841375222 >> v1:172.21.15.136:6789/0 conn(0x7f30b80c7240 legacy=0x7f30b8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 -- 172.21.15.92:0/1841375222 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80c78e0 con 0x7f30b8150180 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b77fe700 1 --2- 172.21.15.92:0/1841375222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b8157770 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009f80 tx=0x7f30a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb78027139f4ef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b67fc700 1 -- 172.21.15.92:0/1841375222 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a800e070 con 0x7f30b8150180 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b67fc700 1 -- 172.21.15.92:0/1841375222 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a800bab0 con 0x7f30b8150180 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.066+0000 7f30b67fc700 1 -- 172.21.15.92:0/1841375222 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80092e0 con 0x7f30b8150180 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1841375222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 msgr2=0x7f30b8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1841375222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b8157770 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009f80 tx=0x7f30a8009840 comp rx=0 tx=0).stop 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1841375222 shutdown_connections 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1841375222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b8157770 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1841375222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1841375222 >> 172.21.15.92:0/1841375222 conn(0x7f30b80cc1f0 msgr2=0x7f30b80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1841375222 shutdown_connections 2023-10-06T03:10:10.071 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.067+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1841375222 wait complete. 2023-10-06T03:10:10.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.068+0000 7f30bcb9f700 1 Processor -- start 2023-10-06T03:10:10.073 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.068+0000 7f30bcb9f700 1 -- start start 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b80c7240 0x7f30b814ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b0510 con 0x7f30b8150180 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80af900 con 0x7f30b80c7240 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80afbe0 con 0x7f30b8150550 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b80aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45986/0 (socket says 172.21.15.92:45986) 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 -- 172.21.15.92:0/1863063141 learned_addr learned my addr 172.21.15.92:0/1863063141 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b77fe700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b80c7240 0x7f30b814ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b7fff700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 -- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 msgr2=0x7f30b80c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 -- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b80c7240 msgr2=0x7f30b814ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b80c7240 0x7f30b814ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 -- 172.21.15.92:0/1863063141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8009ae0 con 0x7f30b8150180 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30b6ffd700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b80aef90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00d890 tx=0x7f30ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00b600 con 0x7f30b8150180 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac00b760 con 0x7f30b8150180 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b80afec0 con 0x7f30b8150180 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.069+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00bae0 con 0x7f30b8150180 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.070+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b814f360 con 0x7f30b8150180 2023-10-06T03:10:10.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.070+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8157770 con 0x7f30b8150180 2023-10-06T03:10:10.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.071+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f30ac01e460 con 0x7f30b8150180 2023-10-06T03:10:10.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.071+0000 7f309f7fe700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30a4067320 0x7f30a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.071+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f30ac085480 con 0x7f30b8150180 2023-10-06T03:10:10.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.071+0000 7f30b77fe700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30a4067320 0x7f30a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.072+0000 7f30b77fe700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30a4067320 0x7f30a40697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009880 tx=0x7f30a8018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.075+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30ac051fb0 con 0x7f30b8150180 2023-10-06T03:10:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.300+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30b804c3c0 con 0x7f30b8150180 2023-10-06T03:10:10.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.301+0000 7f309f7fe700 1 -- 172.21.15.92:0/1863063141 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4604 (secure 0 0 0) 0x7f30ac0555d0 con 0x7f30b8150180 2023-10-06T03:10:10.302 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:10.303 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":113,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:03.593287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30a4067320 msgr2=0x7f30a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30a4067320 0x7f30a40697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009880 tx=0x7f30a8018000 comp rx=0 tx=0).stop 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 msgr2=0x7f30b80aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b80aef90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00d890 tx=0x7f30ac008040 comp rx=0 tx=0).stop 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 shutdown_connections 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30a4067320 0x7f30a40697d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b8150550 0x7f30b80c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b80c7240 0x7f30b814ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 --2- 172.21.15.92:0/1863063141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30b8150180 0x7f30b80aef90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 >> 172.21.15.92:0/1863063141 conn(0x7f30b80cc1f0 msgr2=0x7f30b80c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 shutdown_connections 2023-10-06T03:10:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.303+0000 7f30bcb9f700 1 -- 172.21.15.92:0/1863063141 wait complete. 2023-10-06T03:10:10.307 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 113 2023-10-06T03:10:10.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:09:59.949008+0000', 'modified': '2023-10-06T03:10:03.593287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8361}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8361': {'gid': 8361, 'name': 'b', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.136:6834/1055679258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 1055679258}, {'type': 'v1', 'addr': '172.21.15.136:6834', 'nonce': 1055679258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:10:10.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:10:10.720 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T03:10:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.837+0000 7f91c8ff9700 1 Processor -- start 2023-10-06T03:10:10.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.837+0000 7f91c8ff9700 1 -- start start 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91c40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c8ff9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40b2980 con 0x7f91c415d230 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c8ff9700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40b11d0 con 0x7f91c41597f0 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c8ff9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40b14b0 con 0x7f91c4159420 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f91c415d230 0x7f91c40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47230/0 (socket says 172.21.15.92:47230) 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c3fff700 1 -- 172.21.15.92:0/3594316775 learned_addr learned my addr 172.21.15.92:0/3594316775 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91c40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90602170 0 0) 0x7f91c40b2980 con 0x7f91c415d230 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0003570 con 0x7f91c415d230 2023-10-06T03:10:10.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705774916 0 0) 0x7f91c40b11d0 con 0x7f91c41597f0 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c40b2980 con 0x7f91c41597f0 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519673162 0 0) 0x7f91b0003570 con 0x7f91c415d230 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 msgr2=0x7f91c40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91c40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 >> v1:172.21.15.136:6789/0 conn(0x7f91c41597f0 legacy=0x7f91c40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40b1790 con 0x7f91c415d230 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f91b4001470 con 0x7f91c415d230 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91c40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.838+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 >> v1:172.21.15.92:6789/0 conn(0x7f91c415d230 legacy=0x7f91c40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91b0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004160 0x7f91b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91b0004d50 0x7f91b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40b14b0 con 0x7f91b0004d50 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000bba0 con 0x7f91b0004160 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000bd60 con 0x7f91c4159420 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91b0004d50 0x7f91b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91b0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004160 0x7f91b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 -- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 msgr2=0x7f91b0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c4159420 0x7f91b0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 -- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004160 msgr2=0x7f91b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004160 0x7f91b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 -- 172.21.15.92:0/3594316775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc009a20 con 0x7f91b0004d50 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91b0004d50 0x7f91b00051c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f91b400c2d0 tx=0x7f91b4001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0dcab5bab7cb144 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b400cbc0 con 0x7f91b0004d50 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b400cd20 con 0x7f91b0004d50 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.839+0000 7f91c27fc700 1 -- 172.21.15.92:0/3594316775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4005b70 con 0x7f91b0004d50 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3594316775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91b0004d50 msgr2=0x7f91b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3594316775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91b0004d50 0x7f91b00051c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f91b400c2d0 tx=0x7f91b4001e10 comp rx=0 tx=0).stop 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c3fff700 1 -- 172.21.15.92:0/3594316775 reap_dead start 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3594316775 shutdown_connections 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3594316775 >> 172.21.15.92:0/3594316775 conn(0x7f91c40d0660 msgr2=0x7f91c40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3594316775 shutdown_connections 2023-10-06T03:10:10.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.840+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3594316775 wait complete. 2023-10-06T03:10:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.841+0000 7f91c8ff9700 1 Processor -- start 2023-10-06T03:10:10.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.841+0000 7f91c8ff9700 1 -- start start 2023-10-06T03:10:10.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004d50 0x7f91c416d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c414e360 0x7f91c414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c8ff9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 0x7f91c414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c8ff9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c416dd20 con 0x7f91c414e7d0 2023-10-06T03:10:10.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c8ff9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c415aa80 con 0x7f91b0004d50 2023-10-06T03:10:10.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c8ff9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c415ad20 con 0x7f91c414e360 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 0x7f91c414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 0x7f91c414d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46004/0 (socket says 172.21.15.92:46004) 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 -- 172.21.15.92:0/3913716926 learned_addr learned my addr 172.21.15.92:0/3913716926 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004d50 0x7f91c416d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 -- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c414e360 msgr2=0x7f91c414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c414e360 0x7f91c414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 -- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004d50 msgr2=0x7f91c416d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004d50 0x7f91c416d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 -- 172.21.15.92:0/3913716926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c415b000 con 0x7f91c414e7d0 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 0x7f91c414d4b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f91b4011000 tx=0x7f91b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91bb7fe700 1 -- 172.21.15.92:0/3913716926 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40070d0 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.842+0000 7f91bb7fe700 1 -- 172.21.15.92:0/3913716926 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4007270 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.843+0000 7f91bb7fe700 1 -- 172.21.15.92:0/3913716926 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4007630 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.843+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c415b2e0 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.843+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c415b8a0 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.844+0000 7f91bb7fe700 1 -- 172.21.15.92:0/3913716926 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f91b4019070 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.844+0000 7f91bb7fe700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f91a4067320 0x7f91a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.844+0000 7f91bb7fe700 1 -- 172.21.15.92:0/3913716926 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91b4086200 con 0x7f91c414e7d0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.844+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f91a4067320 0x7f91a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.845+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f91a4067320 0x7f91a40697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0094d0 tx=0x7f91bc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.846+0000 7f91d3553b80 1 Processor -- start 2023-10-06T03:10:10.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.846+0000 7f91d3553b80 1 -- start start 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae5a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5aa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91d3553b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae504a70 con 0x55b3ae5af2c0 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae502270 con 0x55b3ae5b0a10 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae4fe060 con 0x55b3ae5cee50 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5aa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46014/0 (socket says 172.21.15.92:46014) 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 -- 172.21.15.92:0/2683982784 learned_addr learned my addr 172.21.15.92:0/2683982784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 -- 172.21.15.92:0/2683982784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 msgr2=0x55b3ae5a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae5a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 -- 172.21.15.92:0/2683982784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 msgr2=0x55b3ae5aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.847+0000 7f91c2ffd700 1 -- 172.21.15.92:0/2683982784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ae398270 con 0x55b3ae5af2c0 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5aa200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f91ac007a80 tx=0x7f91ac007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c32722ef70473c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91b8ff9700 1 -- 172.21.15.92:0/2683982784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac002ad0 con 0x55b3ae5af2c0 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91b8ff9700 1 -- 172.21.15.92:0/2683982784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac002c50 con 0x55b3ae5af2c0 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91b8ff9700 1 -- 172.21.15.92:0/2683982784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0133f0 con 0x55b3ae5af2c0 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91d3553b80 1 -- 172.21.15.92:0/2683982784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 msgr2=0x55b3ae5aa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91d3553b80 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5aa200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f91ac007a80 tx=0x7f91ac007d30 comp rx=0 tx=0).stop 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91d3553b80 1 -- 172.21.15.92:0/2683982784 shutdown_connections 2023-10-06T03:10:10.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.848+0000 7f91d3553b80 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae5a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 --2- 172.21.15.92:0/2683982784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5aa200 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 -- 172.21.15.92:0/2683982784 >> 172.21.15.92:0/2683982784 conn(0x55b3ae5aee80 msgr2=0x55b3ae5af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 -- 172.21.15.92:0/2683982784 shutdown_connections 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 -- 172.21.15.92:0/2683982784 wait complete. 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 -- start start 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.849+0000 7f91d3553b80 1 Processor -- start 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae5be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae4a9890 con 0x55b3ae5af2c0 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae4c4ea0 con 0x55b3ae5b0a10 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae380e30 con 0x55b3ae5cee50 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91d3553b80 10 client.0 _ll_register_callbacks cb 0x55b3ae571020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46030/0 (socket says 172.21.15.92:46030) 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 -- 172.21.15.92:0/7357949 learned_addr learned my addr 172.21.15.92:0/7357949 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 -- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 msgr2=0x55b3ae5be040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae5be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 -- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 msgr2=0x55b3ae5f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.850+0000 7f91c37fe700 1 -- 172.21.15.92:0/7357949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c40b2980 con 0x55b3ae5af2c0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91c37fe700 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5cfa70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f91bc016550 tx=0x7f91bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5075c89607b2167d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91a1ffb700 1 -- 172.21.15.92:0/7357949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00e040 con 0x55b3ae5af2c0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91a1ffb700 1 -- 172.21.15.92:0/7357949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91bc016d80 con 0x55b3ae5af2c0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3ae5a2940 con 0x55b3ae5af2c0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91a1ffb700 1 -- 172.21.15.92:0/7357949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91bc009c70 con 0x55b3ae5af2c0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 10 client.8529 fetch_fsmap learned FSMap version 113 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55b3ae33afe0 con 0x55b3ae5af2c0 2023-10-06T03:10:10.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91a1ffb700 1 -- 172.21.15.92:0/7357949 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3ae5a2940 con 0x55b3ae5af2c0 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91a1ffb700 1 -- 172.21.15.92:0/7357949 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 113) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f91bc023650 con 0x55b3ae5af2c0 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 10 client.8529 fetch_fsmap finished waiting for FSMap version 113 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 10 client.8529 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 20 client.8529 populate_metadata read hostname 'smithi092' 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5f7170 0x55b3ae5dc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 4 client.8529 mds_command: new command op to 8361 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 --> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3ae57f870 con 0x55b3ae5f7170 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.851+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5f7170 0x55b3ae5dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.852+0000 7f91a1ffb700 10 client.8529 ms_handle_connect on v2:172.21.15.136:6832/1055679258 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.852+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5f7170 0x55b3ae5dc880 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=122cd9cd346b308d server_cookie=8a6d4a66f0ac904f in_seq=0 out_seq=0 2023-10-06T03:10:10.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.852+0000 7f91a1ffb700 1 -- 172.21.15.92:0/7357949 <== mds.0 v2:172.21.15.136:6832/1055679258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55b3ae57f870 con 0x55b3ae5f7170 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.852+0000 7f91a1ffb700 10 client.8529 handle_command_reply: tid=0 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.852+0000 7f91d3553b80 1 client.8529 shutdown 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 msgr2=0x55b3ae5cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5cfa70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f91bc016550 tx=0x7f91bc019000 comp rx=0 tx=0).stop 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 shutdown_connections 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5f7170 0x55b3ae5dc880 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5cfa70 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91a1ffb700 0 client.8529 ms_handle_reset on v2:172.21.15.136:6832/1055679258 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae5be040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 --2- 172.21.15.92:0/7357949 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 >> 172.21.15.92:0/7357949 conn(0x55b3ae5aee80 msgr2=0x55b3ae5afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 shutdown_connections 2023-10-06T03:10:10.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 1 -- 172.21.15.92:0/7357949 wait complete. 2023-10-06T03:10:10.865 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.853+0000 7f91d3553b80 20 client.8529 trim_cache size 0 max 16384 2023-10-06T03:10:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 Processor -- start 2023-10-06T03:10:10.871 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 -- start start 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5af2c0 0x55b3ae60dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 0x55b3ae5c0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae5f8500 con 0x55b3ae5cee50 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae5a2700 con 0x55b3ae5b0a10 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.870+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae5f82b0 con 0x55b3ae5af2c0 2023-10-06T03:10:10.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 0x55b3ae5c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 0x55b3ae5c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46044/0 (socket says 172.21.15.92:46044) 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 -- 172.21.15.92:0/1817555240 learned_addr learned my addr 172.21.15.92:0/1817555240 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c37fe700 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5af2c0 0x55b3ae60dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 -- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5af2c0 msgr2=0x55b3ae60dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5af2c0 0x55b3ae60dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 -- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 msgr2=0x55b3ae5f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 -- 172.21.15.92:0/1817555240 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ae4cc780 con 0x55b3ae5cee50 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91c3fff700 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 0x55b3ae5c0930 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f91b4083f70 tx=0x7f91b4007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d3bf07bdf7b100 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91b97fa700 1 -- 172.21.15.92:0/1817555240 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b4086030 con 0x55b3ae5cee50 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91b97fa700 1 -- 172.21.15.92:0/1817555240 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b4011360 con 0x55b3ae5cee50 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91b97fa700 1 -- 172.21.15.92:0/1817555240 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40570f0 con 0x55b3ae5cee50 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91d3553b80 1 -- 172.21.15.92:0/1817555240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 msgr2=0x55b3ae5c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.871+0000 7f91d3553b80 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 0x55b3ae5c0930 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f91b4083f70 tx=0x7f91b4007060 comp rx=0 tx=0).stop 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 -- 172.21.15.92:0/1817555240 shutdown_connections 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5cee50 0x55b3ae5c0930 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5af2c0 0x55b3ae60dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 --2- 172.21.15.92:0/1817555240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 -- 172.21.15.92:0/1817555240 >> 172.21.15.92:0/1817555240 conn(0x55b3ae5aee80 msgr2=0x55b3ae5d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 -- 172.21.15.92:0/1817555240 shutdown_connections 2023-10-06T03:10:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 -- 172.21.15.92:0/1817555240 wait complete. 2023-10-06T03:10:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 -- start start 2023-10-06T03:10:10.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.872+0000 7f91d3553b80 1 Processor -- start 2023-10-06T03:10:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae65a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae5aa870 con 0x55b3ae5b0a10 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae659c10 con 0x55b3ae5af2c0 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae5aa010 con 0x55b3ae5cee50 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91d3553b80 10 client.0 _ll_register_callbacks cb 0x55b3ae571020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46056/0 (socket says 172.21.15.92:46056) 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 -- 172.21.15.92:0/3136732692 learned_addr learned my addr 172.21.15.92:0/3136732692 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c3fff700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae65a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 -- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 msgr2=0x55b3ae65a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae65a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 -- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5af2c0 msgr2=0x55b3ae5d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 -- 172.21.15.92:0/3136732692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c415b8a0 con 0x55b3ae5b0a10 2023-10-06T03:10:10.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91c2ffd700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5a9ad0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x55b3ae5dbae0 tx=0x7f91ac026250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4ef0561dd08be1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.873+0000 7f91a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91a1ffb700 1 -- 172.21.15.92:0/3136732692 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0133f0 con 0x55b3ae5b0a10 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91a1ffb700 1 -- 172.21.15.92:0/3136732692 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91ac013760 con 0x55b3ae5b0a10 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91ac002c50 con 0x55b3ae5b0a10 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91a1ffb700 1 -- 172.21.15.92:0/3136732692 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91ac002ad0 con 0x55b3ae5b0a10 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 10 client.8535 fetch_fsmap learned FSMap version 113 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55b3ae4b3470 con 0x55b3ae5b0a10 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91a1ffb700 1 -- 172.21.15.92:0/3136732692 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac002c50 con 0x55b3ae5b0a10 2023-10-06T03:10:10.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91a1ffb700 1 -- 172.21.15.92:0/3136732692 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 113) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f91ac012070 con 0x55b3ae5b0a10 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 10 client.8535 fetch_fsmap finished waiting for FSMap version 113 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 10 client.8535 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 20 client.8535 populate_metadata read hostname 'smithi092' 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5ccef0 0x55b3ae5cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 4 client.8535 mds_command: new command op to 8361 tid=0[{"prefix": "status"}] 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 --> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b3ae4a0e00 con 0x55b3ae5ccef0 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.874+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5ccef0 0x55b3ae5cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91a1ffb700 10 client.8535 ms_handle_connect on v2:172.21.15.136:6832/1055679258 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91c37fe700 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5ccef0 0x55b3ae5cd2c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf244fcd55580b28 server_cookie=f5dbd4468f314b7 in_seq=0 out_seq=0 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91a1ffb700 1 -- 172.21.15.92:0/3136732692 <== mds.0 v2:172.21.15.136:6832/1055679258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55b3ae4a0e00 con 0x55b3ae5ccef0 2023-10-06T03:10:10.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91a1ffb700 10 client.8535 handle_command_reply: tid=0 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 client.8535 shutdown 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 msgr2=0x55b3ae5a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5a9ad0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x55b3ae5dbae0 tx=0x7f91ac026250 comp rx=0 tx=0).stop 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 shutdown_connections 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:6832/1055679258,v1:172.21.15.136:6834/1055679258] conn(0x55b3ae5ccef0 0x55b3ae5cd2c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55b3ae5cee50 0x55b3ae65a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91a1ffb700 0 client.8535 ms_handle_reset on v2:172.21.15.136:6832/1055679258 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3ae5af2c0 0x55b3ae5d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 --2- 172.21.15.92:0/3136732692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55b3ae5b0a10 0x55b3ae5a9ad0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 >> 172.21.15.92:0/3136732692 conn(0x55b3ae5aee80 msgr2=0x55b3ae5d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 shutdown_connections 2023-10-06T03:10:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 1 -- 172.21.15.92:0/3136732692 wait complete. 2023-10-06T03:10:10.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91d3553b80 20 client.8535 trim_cache size 0 max 16384 2023-10-06T03:10:10.886 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:10:10.886 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:10:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:10:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8361, 2023-10-06T03:10:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:10:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:10:10.887 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:10:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 10.312373515000001, 2023-10-06T03:10:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 113, 2023-10-06T03:10:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 167, 2023-10-06T03:10:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 167, 2023-10-06T03:10:10.888 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 16.445254900999998 2023-10-06T03:10:10.888 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:10:10.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f91a4067320 msgr2=0x7f91a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f91a4067320 0x7f91a40697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0094d0 tx=0x7f91bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 msgr2=0x7f91c414d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.875+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 0x7f91c414d4b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f91b4011000 tx=0x7f91b4015000 comp rx=0 tx=0).stop 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 shutdown_connections 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f91a4067320 0x7f91a40697d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f91c414e7d0 0x7f91c414d4b0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91b0004d50 0x7f91c416d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 --2- 172.21.15.92:0/3913716926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91c414e360 0x7f91c414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 >> 172.21.15.92:0/3913716926 conn(0x7f91c40d0660 msgr2=0x7f91c40b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 shutdown_connections 2023-10-06T03:10:10.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:10.876+0000 7f91c8ff9700 1 -- 172.21.15.92:0/3913716926 wait complete. 2023-10-06T03:10:10.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:10:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.004+0000 7f729f8bf700 1 Processor -- start 2023-10-06T03:10:11.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.004+0000 7f729f8bf700 1 -- start start 2023-10-06T03:10:11.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729f8bf700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f729806c0a0 0x7f72980caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:11.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729f8bf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f729815df50 0x7f729815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:11.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729f8bf700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729815d7e0 con 0x7f729806bcd0 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729f8bf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729815dac0 con 0x7f729815df50 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729f8bf700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729815aa30 con 0x7f729806c0a0 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729de5c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f729815df50 0x7f729815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729d65b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f729806bcd0 0x7f729815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47234/0 (socket says 172.21.15.92:47234) 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.005+0000 7f729d65b700 1 -- 172.21.15.92:0/2025357659 learned_addr learned my addr 172.21.15.92:0/2025357659 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729ce5a700 1 --2- 172.21.15.92:0/2025357659 >> v2:172.21.15.136:3300/0 conn(0x7f729806c0a0 0x7f72980caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729de5c700 1 -- 172.21.15.92:0/2025357659 >> v2:172.21.15.136:3300/0 conn(0x7f729806c0a0 msgr2=0x7f72980caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729de5c700 1 --2- 172.21.15.92:0/2025357659 >> v2:172.21.15.136:3300/0 conn(0x7f729806c0a0 0x7f72980caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729de5c700 1 -- 172.21.15.92:0/2025357659 >> v1:172.21.15.92:6789/0 conn(0x7f729806bcd0 legacy=0x7f729815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:11.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729de5c700 1 -- 172.21.15.92:0/2025357659 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729815ad10 con 0x7f729815df50 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f728ffff700 1 -- 172.21.15.92:0/2025357659 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641240908 0 0) 0x7f729815d7e0 con 0x7f729806bcd0 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729de5c700 1 --2- 172.21.15.92:0/2025357659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729815df50 0x7f729815e3b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f729806b4e0 tx=0x7f72900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9053ee6c2b468ece server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f728ffff700 1 -- 172.21.15.92:0/2025357659 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000b6b0 con 0x7f729815df50 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f728ffff700 1 -- 172.21.15.92:0/2025357659 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729000b810 con 0x7f729815df50 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f728ffff700 1 -- 172.21.15.92:0/2025357659 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729000bbb0 con 0x7f729815df50 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729f8bf700 1 -- 172.21.15.92:0/2025357659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729815df50 msgr2=0x7f729815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2025357659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729815df50 0x7f729815e3b0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f729806b4e0 tx=0x7f72900091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729f8bf700 1 -- 172.21.15.92:0/2025357659 shutdown_connections 2023-10-06T03:10:11.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2025357659 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729815df50 0x7f729815e3b0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.006+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2025357659 >> v2:172.21.15.136:3300/0 conn(0x7f729806c0a0 0x7f72980caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.007+0000 7f729f8bf700 1 -- 172.21.15.92:0/2025357659 >> 172.21.15.92:0/2025357659 conn(0x7f72980ce600 msgr2=0x7f72980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:11.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.007+0000 7f729f8bf700 1 -- 172.21.15.92:0/2025357659 shutdown_connections 2023-10-06T03:10:11.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.007+0000 7f729f8bf700 1 -- 172.21.15.92:0/2025357659 wait complete. 2023-10-06T03:10:11.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.008+0000 7f729f8bf700 1 Processor -- start 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.008+0000 7f729f8bf700 1 -- start start 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.008+0000 7f729f8bf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 0x7f72980b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f729806c0a0 0x7f7298157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f729815df50 0x7f729816a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298159230 con 0x7f729806bcd0 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981596c0 con 0x7f729806c0a0 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7298158620 con 0x7f729815df50 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 0x7f72980b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 0x7f72980b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46068/0 (socket says 172.21.15.92:46068) 2023-10-06T03:10:11.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 -- 172.21.15.92:0/2624348009 learned_addr learned my addr 172.21.15.92:0/2624348009 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729ce5a700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f729806c0a0 0x7f7298157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 -- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f729815df50 msgr2=0x7f729816a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f729815df50 0x7f729816a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 -- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f729806c0a0 msgr2=0x7f7298157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f729806c0a0 0x7f7298157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 -- 172.21.15.92:0/2624348009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290009420 con 0x7f729806bcd0 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729d65b700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 0x7f72980b3bb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f729400cc70 tx=0x7f7294001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:11.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400ce80 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729400ab70 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298158900 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72940070f0 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.009+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7298158ec0 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.010+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7298159510 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.011+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f72940074e0 con 0x7f729806bcd0 2023-10-06T03:10:11.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.011+0000 7f728d7fa700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f727c067320 0x7f727c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.011+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f72940851f0 con 0x7f729806bcd0 2023-10-06T03:10:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.011+0000 7f729ce5a700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f727c067320 0x7f727c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.011+0000 7f729ce5a700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f727c067320 0x7f727c0697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:11.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.015+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7294051da0 con 0x7f729806bcd0 2023-10-06T03:10:11.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.238+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f729804c3c0 con 0x7f729806bcd0 2023-10-06T03:10:11.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.239+0000 7f728d7fa700 1 -- 172.21.15.92:0/2624348009 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7f72940553c0 con 0x7f729806bcd0 2023-10-06T03:10:11.240 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:10:11.242 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.240+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f727c067320 msgr2=0x7f727c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.240+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f727c067320 0x7f727c0697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).stop 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.240+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 msgr2=0x7f72980b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.240+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 0x7f72980b3bb0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f729400cc70 tx=0x7f7294001a30 comp rx=0 tx=0).stop 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 shutdown_connections 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f727c067320 0x7f727c0697d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f729806bcd0 0x7f72980b3bb0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f729815df50 0x7f729816a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 --2- 172.21.15.92:0/2624348009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f729806c0a0 0x7f7298157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 >> 172.21.15.92:0/2624348009 conn(0x7f72980ce600 msgr2=0x7f72980c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:11.243 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 shutdown_connections 2023-10-06T03:10:11.244 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:11.241+0000 7f729f8bf700 1 -- 172.21.15.92:0/2624348009 wait complete. 2023-10-06T03:10:11.254 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:10:11.254 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:10:11.254 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:10:11.254 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:10:11.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:11.255 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:10:11.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:10:11.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:10:11.255 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:10:11.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.255 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:10:11.270 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:10:11.270 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:10:11.270 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:10:11.270 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:10:11.271 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41995sec preferred_lft 41995sec 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.272 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:11.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.273 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:10:11.336 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:10:11.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.337 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:10:11.393 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:10:11.394 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:10:11.394 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:10:11.394 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:10:11.394 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:10:11.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.394 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:10:11.394 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:10:11.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:10:11.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:10:11.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:10:11.395 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:10:11.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.493 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:10:11.508 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:10:11.508 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:10:11.509 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:10:11.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.509 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:10:11.573 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:11.573 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:11.628 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:10:11.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:11.629 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:11.684 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:10:11.684 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:10:11.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:11.764 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:11.764 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:11.783 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:11.784 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:11.784 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:11.784 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:11.806 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:11.807 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:11.807 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:11.807 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:11.807 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:11.807 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:11.807 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:11.808 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:11.808 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:10:11.808 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:10:11.808 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:10:11.810 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:10:11.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:11.810 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:11.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:10:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.027+0000 7f961eb06700 1 Processor -- start 2023-10-06T03:10:12.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.027+0000 7f961eb06700 1 -- start start 2023-10-06T03:10:12.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961eb06700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 0x7f961815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961eb06700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9618159c60 0x7f961815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961eb06700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961815a160 con 0x7f961815db10 2023-10-06T03:10:12.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961eb06700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180d13a0 con 0x7f9618159c60 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961eb06700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961815e140 con 0x7f9618159890 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961d0a3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f961815db10 0x7f9618069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47248/0 (socket says 172.21.15.92:47248) 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.028+0000 7f961d0a3700 1 -- 172.21.15.92:0/3613423600 learned_addr learned my addr 172.21.15.92:0/3613423600 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 --2- 172.21.15.92:0/3613423600 >> v2:172.21.15.136:3300/0 conn(0x7f9618159c60 0x7f961815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f961c8a2700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 0x7f961815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309959434 0 0) 0x7f961815a160 con 0x7f961815db10 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9600003570 con 0x7f961815db10 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1680621378 0 0) 0x7f9600003570 con 0x7f961815db10 2023-10-06T03:10:12.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 msgr2=0x7f961815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 0x7f961815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 >> v2:172.21.15.136:3300/0 conn(0x7f9618159c60 msgr2=0x7f961815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 --2- 172.21.15.92:0/3613423600 >> v2:172.21.15.136:3300/0 conn(0x7f9618159c60 0x7f961815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961815a3d0 con 0x7f961815db10 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f9610001470 con 0x7f961815db10 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 >> v1:172.21.15.92:6789/0 conn(0x7f961815db10 legacy=0x7f9618069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:12.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 0x7f9600003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9600003fe0 0x7f9600004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600004d50 0x7f96000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180d13a0 con 0x7f9600003fe0 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961815e140 con 0x7f9600004d50 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960000ba90 con 0x7f9618159890 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9600003fe0 0x7f9600004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f961c8a2700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 0x7f9600003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f961d0a3700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600004d50 0x7f96000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 -- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 msgr2=0x7f9600003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9618159890 0x7f9600003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 -- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600004d50 msgr2=0x7f96000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600004d50 0x7f96000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 -- 172.21.15.92:0/3613423600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600003570 con 0x7f9600003fe0 2023-10-06T03:10:12.039 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f961d0a3700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600004d50 0x7f96000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.029+0000 7f960ffff700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9600003fe0 0x7f9600004810 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f961815a090 tx=0x7f96080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f465f36351081217 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800e6b0 con 0x7f9600003fe0 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960800e810 con 0x7f9600003fe0 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f960f7fe700 1 -- 172.21.15.92:0/3613423600 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800ebd0 con 0x7f9600003fe0 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961eb06700 1 -- 172.21.15.92:0/3613423600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9600003fe0 msgr2=0x7f9600004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961eb06700 1 --2- 172.21.15.92:0/3613423600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9600003fe0 0x7f9600004810 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f961815a090 tx=0x7f96080091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961d0a3700 1 -- 172.21.15.92:0/3613423600 reap_dead start 2023-10-06T03:10:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961eb06700 1 -- 172.21.15.92:0/3613423600 shutdown_connections 2023-10-06T03:10:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961eb06700 1 -- 172.21.15.92:0/3613423600 >> 172.21.15.92:0/3613423600 conn(0x7f9618068e20 msgr2=0x7f96180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961eb06700 1 -- 172.21.15.92:0/3613423600 shutdown_connections 2023-10-06T03:10:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.030+0000 7f961eb06700 1 -- 172.21.15.92:0/3613423600 wait complete. 2023-10-06T03:10:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.031+0000 7f961eb06700 1 Processor -- start 2023-10-06T03:10:12.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 -- start start 2023-10-06T03:10:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600003fe0 0x7f9618151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96180b17b0 0x7f961814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 0x7f961816d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961816d830 con 0x7f961816ce50 2023-10-06T03:10:12.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180b36c0 con 0x7f9600003fe0 2023-10-06T03:10:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961eb06700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96180b1c20 con 0x7f96180b17b0 2023-10-06T03:10:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961d0a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 0x7f961816d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961d0a3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 0x7f961816d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46092/0 (socket says 172.21.15.92:46092) 2023-10-06T03:10:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961d0a3700 1 -- 172.21.15.92:0/4252933812 learned_addr learned my addr 172.21.15.92:0/4252933812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f961c8a2700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600003fe0 0x7f9618151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.032+0000 7f960ffff700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96180b17b0 0x7f961814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961d0a3700 1 -- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96180b17b0 msgr2=0x7f961814d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961d0a3700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96180b17b0 0x7f961814d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961d0a3700 1 -- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600003fe0 msgr2=0x7f9618151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961d0a3700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600003fe0 0x7f9618151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961d0a3700 1 -- 172.21.15.92:0/4252933812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9618152280 con 0x7f961816ce50 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961d0a3700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 0x7f961816d2f0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9610000c00 tx=0x7f9610000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:12.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100014b0 con 0x7f961816ce50 2023-10-06T03:10:12.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610012590 con 0x7f961816ce50 2023-10-06T03:10:12.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9618152560 con 0x7f961816ce50 2023-10-06T03:10:12.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610012930 con 0x7f961816ce50 2023-10-06T03:10:12.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.033+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9618152b20 con 0x7f961816ce50 2023-10-06T03:10:12.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.034+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96180b3850 con 0x7f961816ce50 2023-10-06T03:10:12.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.034+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9610012ca0 con 0x7f961816ce50 2023-10-06T03:10:12.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.035+0000 7f960cff9700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9604067320 0x7f96040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:12.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.035+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9610084b80 con 0x7f961816ce50 2023-10-06T03:10:12.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.035+0000 7f961c8a2700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9604067320 0x7f96040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:12.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.035+0000 7f961c8a2700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9604067320 0x7f96040697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f96181534f0 tx=0x7f961400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:12.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.039+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96100516b0 con 0x7f961816ce50 2023-10-06T03:10:12.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.266+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f961804c3c0 con 0x7f961816ce50 2023-10-06T03:10:12.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.267+0000 7f960cff9700 1 -- 172.21.15.92:0/4252933812 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9610084e60 con 0x7f961816ce50 2023-10-06T03:10:12.269 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:10:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.268+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9604067320 msgr2=0x7f96040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.268+0000 7f961eb06700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9604067320 0x7f96040697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f96181534f0 tx=0x7f961400b040 comp rx=0 tx=0).stop 2023-10-06T03:10:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.268+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 msgr2=0x7f961816d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.268+0000 7f961eb06700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 0x7f961816d2f0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9610000c00 tx=0x7f9610000f20 comp rx=0 tx=0).stop 2023-10-06T03:10:12.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 shutdown_connections 2023-10-06T03:10:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9604067320 0x7f96040697d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f961816ce50 0x7f961816d2f0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9600003fe0 0x7f9618151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 --2- 172.21.15.92:0/4252933812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96180b17b0 0x7f961814d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 >> 172.21.15.92:0/4252933812 conn(0x7f9618068e20 msgr2=0x7f96180cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:12.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 shutdown_connections 2023-10-06T03:10:12.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:12.269+0000 7f961eb06700 1 -- 172.21.15.92:0/4252933812 wait complete. 2023-10-06T03:10:12.282 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:10:12.283 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:10:12.283 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:10:12.283 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:10:12.283 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:12.283 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:10:12.283 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:10:12.284 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:10:12.284 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:10:12.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.284 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:10:12.303 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:10:12.303 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:10:12.303 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:10:12.303 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:12.304 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 41995sec preferred_lft 41995sec 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.305 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:10:12.306 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:12.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.307 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:10:12.373 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:10:12.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.375 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:10:12.434 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:10:12.434 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:10:12.434 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:10:12.434 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:10:12.434 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:10:12.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.435 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:10:12.435 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:10:12.435 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:10:12.435 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:10:12.436 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:10:12.436 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:10:12.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.540 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:10:12.555 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:10:12.555 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:10:12.556 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:10:12.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.556 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:10:12.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:12.620 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:12.675 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:10:12.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:12.676 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:12.730 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:10:12.730 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:10:12.730 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:12.810 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:12.810 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:12.829 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:12.850 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:12.850 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:12.850 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:12.850 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:10:12.851 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:10:12.852 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:10:12.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:12.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:12.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:10:12.939 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:10:12.963 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:10:13.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.045+0000 7f96d63fb700 1 Processor -- start 2023-10-06T03:10:13.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.045+0000 7f96d63fb700 1 -- start start 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.045+0000 7f96d63fb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f96d0159ec0 0x7f96d00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96d63fb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00b5b00 con 0x7f96d00cf350 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96d63fb700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d00b5850 con 0x7f96d00b53d0 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96d63fb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d00b5da0 con 0x7f96d0159ec0 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96d4998700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f96d00cf350 0x7f96d00cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47256/0 (socket says 172.21.15.92:47256) 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96d4998700 1 -- 172.21.15.92:0/446891976 learned_addr learned my addr 172.21.15.92:0/446891976 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:13.049 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96cffff700 1 --2- 172.21.15.92:0/446891976 >> v2:172.21.15.136:3300/0 conn(0x7f96d0159ec0 0x7f96d00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96ceffd700 1 -- 172.21.15.92:0/446891976 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057672723 0 0) 0x7f96d00b5b00 con 0x7f96d00cf350 2023-10-06T03:10:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96ceffd700 1 -- 172.21.15.92:0/446891976 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b8003570 con 0x7f96d00cf350 2023-10-06T03:10:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96ceffd700 1 -- 172.21.15.92:0/446891976 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174044213 0 0) 0x7f96b8003570 con 0x7f96d00cf350 2023-10-06T03:10:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96cffff700 1 -- 172.21.15.92:0/446891976 >> v1:172.21.15.136:6789/0 conn(0x7f96d00b53d0 legacy=0x7f96d00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96cffff700 1 -- 172.21.15.92:0/446891976 >> v1:172.21.15.92:6789/0 conn(0x7f96d00cf350 legacy=0x7f96d00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.050 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96cffff700 1 -- 172.21.15.92:0/446891976 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00b6080 con 0x7f96d0159ec0 2023-10-06T03:10:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96cffff700 1 --2- 172.21.15.92:0/446891976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00b4e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f96c00098d0 tx=0x7f96c0009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5152cb82e27c2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.046+0000 7f96ceffd700 1 -- 172.21.15.92:0/446891976 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c80017f0 con 0x7f96d0159ec0 2023-10-06T03:10:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96ceffd700 1 -- 172.21.15.92:0/446891976 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0011420 con 0x7f96d0159ec0 2023-10-06T03:10:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96ceffd700 1 -- 172.21.15.92:0/446891976 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c00117c0 con 0x7f96d0159ec0 2023-10-06T03:10:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 -- 172.21.15.92:0/446891976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 msgr2=0x7f96d00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 --2- 172.21.15.92:0/446891976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00b4e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f96c00098d0 tx=0x7f96c0009120 comp rx=0 tx=0).stop 2023-10-06T03:10:13.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 -- 172.21.15.92:0/446891976 shutdown_connections 2023-10-06T03:10:13.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 --2- 172.21.15.92:0/446891976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00b4e90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 -- 172.21.15.92:0/446891976 >> 172.21.15.92:0/446891976 conn(0x7f96d00999f0 msgr2=0x7f96d00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:13.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 -- 172.21.15.92:0/446891976 shutdown_connections 2023-10-06T03:10:13.052 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.047+0000 7f96d63fb700 1 -- 172.21.15.92:0/446891976 wait complete. 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.048+0000 7f96d63fb700 1 Processor -- start 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.048+0000 7f96d63fb700 1 -- start start 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d63fb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 0x7f96d014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d63fb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96d00cf350 0x7f96d009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d63fb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00d0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d63fb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d009ad90 con 0x7f96d00b53d0 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d63fb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d014b770 con 0x7f96d0159ec0 2023-10-06T03:10:13.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d63fb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d009d440 con 0x7f96d00cf350 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 0x7f96d014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 0x7f96d014b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46100/0 (socket says 172.21.15.92:46100) 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 -- 172.21.15.92:0/2148867514 learned_addr learned my addr 172.21.15.92:0/2148867514 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96d4998700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cf7fe700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96d00cf350 0x7f96d009ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 -- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96d00cf350 msgr2=0x7f96d009ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96d00cf350 0x7f96d009ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 -- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 msgr2=0x7f96d00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.049+0000 7f96cffff700 1 -- 172.21.15.92:0/2148867514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00093c0 con 0x7f96d00b53d0 2023-10-06T03:10:13.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.050+0000 7f96cffff700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 0x7f96d014b230 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009390 tx=0x7f96c0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.050+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c001cb30 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.050+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c001cc90 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.050+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d009a740 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.050+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c001a680 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.050+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d009d720 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.051+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0063c90 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.051+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f96c001aa30 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.052+0000 7f96b7fff700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96bc067320 0x7f96bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.052+0000 7f96cf7fe700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96bc067320 0x7f96bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.052+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96c0086980 con 0x7f96d00b53d0 2023-10-06T03:10:13.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.052+0000 7f96cf7fe700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96bc067320 0x7f96bc0697d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f96d00b2460 tx=0x7f96c4007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:13.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.056+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96c00534b0 con 0x7f96d00b53d0 2023-10-06T03:10:13.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.279+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f96d00718b0 con 0x7f96d00b53d0 2023-10-06T03:10:13.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.624+0000 7f96b7fff700 1 -- 172.21.15.92:0/2148867514 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 136+0+0 (secure 0 0 0) 0x7f96c0056ad0 con 0x7f96d00b53d0 2023-10-06T03:10:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96bc067320 msgr2=0x7f96bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96bc067320 0x7f96bc0697d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f96d00b2460 tx=0x7f96c4007ec0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 msgr2=0x7f96d014b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 0x7f96d014b230 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009390 tx=0x7f96c0016000 comp rx=0 tx=0).stop 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 shutdown_connections 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f96bc067320 0x7f96bc0697d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f96d00b53d0 0x7f96d014b230 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f96d0159ec0 0x7f96d00d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 --2- 172.21.15.92:0/2148867514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f96d00cf350 0x7f96d009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 >> 172.21.15.92:0/2148867514 conn(0x7f96d00999f0 msgr2=0x7f96d00c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 shutdown_connections 2023-10-06T03:10:13.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.626+0000 7f96d63fb700 1 -- 172.21.15.92:0/2148867514 wait complete. 2023-10-06T03:10:13.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:10:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.794+0000 7f304be77700 1 Processor -- start 2023-10-06T03:10:13.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.794+0000 7f304be77700 1 -- start start 2023-10-06T03:10:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304be77700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3044159890 0x7f304415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304be77700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3044159c60 0x7f304415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304be77700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304415a160 con 0x7f304415db10 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304be77700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30440d13a0 con 0x7f3044159890 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304be77700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304415e140 con 0x7f3044159c60 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049c13700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3044159890 0x7f304415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304a414700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f304415db10 0x7f3044069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47270/0 (socket says 172.21.15.92:47270) 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049c13700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3044159890 0x7f304415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46110/0 (socket says 172.21.15.92:46110) 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f304a414700 1 -- 172.21.15.92:0/484510662 learned_addr learned my addr 172.21.15.92:0/484510662 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049412700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3044159c60 0x7f304415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049c13700 1 -- 172.21.15.92:0/484510662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3044159c60 msgr2=0x7f304415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049c13700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3044159c60 0x7f304415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049c13700 1 -- 172.21.15.92:0/484510662 >> v1:172.21.15.92:6789/0 conn(0x7f304415db10 legacy=0x7f3044069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3048c11700 1 -- 172.21.15.92:0/484510662 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871413526 0 0) 0x7f304415a160 con 0x7f304415db10 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049c13700 1 -- 172.21.15.92:0/484510662 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304415a3d0 con 0x7f3044159890 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.795+0000 7f3049412700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3044159c60 0x7f304415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f3049c13700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304415abd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f3040009e60 tx=0x7f3040009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a76e8939f6363fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f3048c11700 1 -- 172.21.15.92:0/484510662 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304000e070 con 0x7f3044159890 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f3048c11700 1 -- 172.21.15.92:0/484510662 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304000bc30 con 0x7f3044159890 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f3048c11700 1 -- 172.21.15.92:0/484510662 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040012430 con 0x7f3044159890 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 -- 172.21.15.92:0/484510662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 msgr2=0x7f304415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304415abd0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f3040009e60 tx=0x7f3040009780 comp rx=0 tx=0).stop 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 -- 172.21.15.92:0/484510662 shutdown_connections 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304415abd0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 --2- 172.21.15.92:0/484510662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3044159c60 0x7f304415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 -- 172.21.15.92:0/484510662 >> 172.21.15.92:0/484510662 conn(0x7f3044068e20 msgr2=0x7f30440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 -- 172.21.15.92:0/484510662 shutdown_connections 2023-10-06T03:10:13.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.796+0000 7f304be77700 1 -- 172.21.15.92:0/484510662 wait complete. 2023-10-06T03:10:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.797+0000 7f304be77700 1 Processor -- start 2023-10-06T03:10:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.797+0000 7f304be77700 1 -- start start 2023-10-06T03:10:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304be77700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304414a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304be77700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3044159c60 0x7f304415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304be77700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f304415db10 0x7f3044154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304be77700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304415cca0 con 0x7f3044159890 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304be77700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440cf880 con 0x7f3044159c60 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304be77700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304414a8d0 con 0x7f304415db10 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f3049c13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304414a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f3049c13700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304414a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46116/0 (socket says 172.21.15.92:46116) 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f3049c13700 1 -- 172.21.15.92:0/1550945048 learned_addr learned my addr 172.21.15.92:0/1550945048 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f3049412700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3044159c60 0x7f304415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.798+0000 7f304a414700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f304415db10 0x7f3044154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3049c13700 1 -- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f304415db10 msgr2=0x7f3044154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3049c13700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f304415db10 0x7f3044154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3049c13700 1 -- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3044159c60 msgr2=0x7f304415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3049c13700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3044159c60 0x7f304415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3049c13700 1 -- 172.21.15.92:0/1550945048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3040009a20 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3049c13700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304414a390 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f30400097c0 tx=0x7f3040019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040012e80 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040027470 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040027830 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30440cec70 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.799+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30440cf230 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.800+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304404c3c0 con 0x7f3044159890 2023-10-06T03:10:13.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.800+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3040024070 con 0x7f3044159890 2023-10-06T03:10:13.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.801+0000 7f3039ffb700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3028067320 0x7f30280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:13.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.801+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f30400877d0 con 0x7f3044159890 2023-10-06T03:10:13.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.801+0000 7f3049412700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3028067320 0x7f30280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:13.806 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.801+0000 7f3049412700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3028067320 0x7f30280697d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3034006f00 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:13.807 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:13.805+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3040054300 con 0x7f3044159890 2023-10-06T03:10:14.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.013+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f304415d8f0 con 0x7f3028067320 2023-10-06T03:10:14.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.014+0000 7f3039ffb700 1 -- 172.21.15.92:0/1550945048 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f304415d8f0 con 0x7f3028067320 2023-10-06T03:10:14.016 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:10:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:10:14.016 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:14.016 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:10:14.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3028067320 msgr2=0x7f30280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3028067320 0x7f30280697d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3034006f00 tx=0x7f3034008040 comp rx=0 tx=0).stop 2023-10-06T03:10:14.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 msgr2=0x7f304414a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304414a390 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f30400097c0 tx=0x7f3040019000 comp rx=0 tx=0).stop 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 shutdown_connections 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3028067320 0x7f30280697d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3044159890 0x7f304414a390 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f304415db10 0x7f3044154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 --2- 172.21.15.92:0/1550945048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3044159c60 0x7f304415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 >> 172.21.15.92:0/1550945048 conn(0x7f3044068e20 msgr2=0x7f30440d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 shutdown_connections 2023-10-06T03:10:14.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.016+0000 7f304be77700 1 -- 172.21.15.92:0/1550945048 wait complete. 2023-10-06T03:10:14.030 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:10:14.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.145+0000 7f954a05c700 1 Processor -- start 2023-10-06T03:10:14.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.145+0000 7f954a05c700 1 -- start start 2023-10-06T03:10:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954a05c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f95440b4e70 0x7f95440ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954a05c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f954415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954a05c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95440cfb00 con 0x7f95440b16d0 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954a05c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95440b5c40 con 0x7f95440b4e70 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954a05c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440b5f20 con 0x7f95440b1b40 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954985b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f954415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f9548859700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f95440b16d0 0x7f954415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:50840/0 (socket says 172.21.15.92:50840) 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f9548859700 1 -- 172.21.15.92:0/1846504446 learned_addr learned my addr 172.21.15.92:0/1846504446 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954905a700 1 --2- 172.21.15.92:0/1846504446 >> v2:172.21.15.136:3300/0 conn(0x7f95440b4e70 0x7f95440ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f953bfff700 1 -- 172.21.15.92:0/1846504446 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402231310 0 0) 0x7f95440cfb00 con 0x7f95440b16d0 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f953bfff700 1 -- 172.21.15.92:0/1846504446 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9530003570 con 0x7f95440b16d0 2023-10-06T03:10:14.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954985b700 1 -- 172.21.15.92:0/1846504446 >> v2:172.21.15.136:3300/0 conn(0x7f95440b4e70 msgr2=0x7f95440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954985b700 1 --2- 172.21.15.92:0/1846504446 >> v2:172.21.15.136:3300/0 conn(0x7f95440b4e70 0x7f95440ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954985b700 1 -- 172.21.15.92:0/1846504446 >> v1:172.21.15.136:6789/0 conn(0x7f95440b16d0 legacy=0x7f954415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954985b700 1 -- 172.21.15.92:0/1846504446 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440b6200 con 0x7f95440b1b40 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.146+0000 7f954905a700 1 --2- 172.21.15.92:0/1846504446 >> v2:172.21.15.136:3300/0 conn(0x7f95440b4e70 0x7f95440ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954985b700 1 --2- 172.21.15.92:0/1846504446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f954415d950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95400098e0 tx=0x7f95400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284d6955c347dfda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f953bfff700 1 -- 172.21.15.92:0/1846504446 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540011630 con 0x7f95440b1b40 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f953bfff700 1 -- 172.21.15.92:0/1846504446 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9540011790 con 0x7f95440b1b40 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f953bfff700 1 -- 172.21.15.92:0/1846504446 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540011b30 con 0x7f95440b1b40 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 -- 172.21.15.92:0/1846504446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 msgr2=0x7f954415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 --2- 172.21.15.92:0/1846504446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f954415d950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95400098e0 tx=0x7f95400091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 -- 172.21.15.92:0/1846504446 shutdown_connections 2023-10-06T03:10:14.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 --2- 172.21.15.92:0/1846504446 >> v2:172.21.15.136:3300/0 conn(0x7f95440b4e70 0x7f95440ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 --2- 172.21.15.92:0/1846504446 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f954415d950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 -- 172.21.15.92:0/1846504446 >> 172.21.15.92:0/1846504446 conn(0x7f95440d10c0 msgr2=0x7f95440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:14.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 -- 172.21.15.92:0/1846504446 shutdown_connections 2023-10-06T03:10:14.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.147+0000 7f954a05c700 1 -- 172.21.15.92:0/1846504446 wait complete. 2023-10-06T03:10:14.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.148+0000 7f954a05c700 1 Processor -- start 2023-10-06T03:10:14.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.148+0000 7f954a05c700 1 -- start start 2023-10-06T03:10:14.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954a05c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 0x7f95441459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954a05c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f9544068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954a05c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95440b4e70 0x7f95440688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954a05c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cb2c0 con 0x7f95440b16d0 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954a05c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440c6c70 con 0x7f95440b4e70 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954a05c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440c6360 con 0x7f95440b1b40 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954905a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 0x7f95441459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954905a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 0x7f95441459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46120/0 (socket says 172.21.15.92:46120) 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.149+0000 7f954905a700 1 -- 172.21.15.92:0/688825569 learned_addr learned my addr 172.21.15.92:0/688825569 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954985b700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95440b4e70 0x7f95440688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f9548859700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f9544068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954905a700 1 -- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 msgr2=0x7f9544068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954905a700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f9544068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954905a700 1 -- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95440b4e70 msgr2=0x7f95440688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954905a700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95440b4e70 0x7f95440688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954905a700 1 -- 172.21.15.92:0/688825569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c009a20 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954905a700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 0x7f95441459e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f953c00c1a0 tx=0x7f953c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00cce0 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c00f040 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9540009420 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c014690 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.150+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95440c6620 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.151+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95440ccbe0 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.151+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f953c014a00 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.152+0000 7f95397fa700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9528067320 0x7f95280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.152+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f953c085c80 con 0x7f95440b16d0 2023-10-06T03:10:14.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.152+0000 7f9548859700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9528067320 0x7f95280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.152+0000 7f9548859700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9528067320 0x7f95280697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f954416cfa0 tx=0x7f9534009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:14.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.156+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f953c0526a0 con 0x7f95440b16d0 2023-10-06T03:10:14.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.340+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f954404bb80 con 0x7f95440b16d0 2023-10-06T03:10:14.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.340+0000 7f95397fa700 1 -- 172.21.15.92:0/688825569 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f953c014cb0 con 0x7f95440b16d0 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9528067320 msgr2=0x7f95280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9528067320 0x7f95280697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f954416cfa0 tx=0x7f9534009040 comp rx=0 tx=0).stop 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 msgr2=0x7f95441459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 0x7f95441459e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f953c00c1a0 tx=0x7f953c00b040 comp rx=0 tx=0).stop 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 shutdown_connections 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9528067320 0x7f95280697d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.344 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95440b16d0 0x7f95441459e0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95440b4e70 0x7f95440688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 --2- 172.21.15.92:0/688825569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95440b1b40 0x7f9544068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 >> 172.21.15.92:0/688825569 conn(0x7f95440d10c0 msgr2=0x7f95440b54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:14.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 shutdown_connections 2023-10-06T03:10:14.345 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.342+0000 7f954a05c700 1 -- 172.21.15.92:0/688825569 wait complete. 2023-10-06T03:10:14.356 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:10:14.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:10:14.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:14.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:10:14.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:14.399 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:10:14.454 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:10:14.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:10:14.455 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:14.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:10:14.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:14.499 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:10:14.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:10:14.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:14.556 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T03:10:14.570 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T03:10:14.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:10:14.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:10:14.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:14.572 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T03:10:14.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:10:14.612 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T03:10:14.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create another-data-pool --pg_num_min 64 2023-10-06T03:10:14.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.728+0000 7fba344a6700 1 Processor -- start 2023-10-06T03:10:14.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.728+0000 7fba344a6700 1 -- start start 2023-10-06T03:10:14.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba344a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fba2c155360 0x7fba2c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba344a6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fba2c0ced30 0x7fba2c0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba344a6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c15c320 con 0x7fba2c0b40f0 2023-10-06T03:10:14.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba344a6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c0b4590 con 0x7fba2c155360 2023-10-06T03:10:14.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba344a6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2c15b710 con 0x7fba2c0ced30 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba32242700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fba2c155360 0x7fba2c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba32242700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fba2c155360 0x7fba2c0b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46134/0 (socket says 172.21.15.92:46134) 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.729+0000 7fba32242700 1 -- 172.21.15.92:0/1252933941 learned_addr learned my addr 172.21.15.92:0/1252933941 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32a43700 1 --2- 172.21.15.92:0/1252933941 >> v2:172.21.15.136:3300/0 conn(0x7fba2c0ced30 0x7fba2c0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32242700 1 -- 172.21.15.92:0/1252933941 >> v2:172.21.15.136:3300/0 conn(0x7fba2c0ced30 msgr2=0x7fba2c0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32242700 1 --2- 172.21.15.92:0/1252933941 >> v2:172.21.15.136:3300/0 conn(0x7fba2c0ced30 0x7fba2c0cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32242700 1 -- 172.21.15.92:0/1252933941 >> v1:172.21.15.136:6789/0 conn(0x7fba2c0b40f0 legacy=0x7fba2c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32242700 1 -- 172.21.15.92:0/1252933941 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c15b9f0 con 0x7fba2c155360 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32a43700 1 --2- 172.21.15.92:0/1252933941 >> v2:172.21.15.136:3300/0 conn(0x7fba2c0ced30 0x7fba2c0cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba32242700 1 --2- 172.21.15.92:0/1252933941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c155360 0x7fba2c0b3bb0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fba2c06f720 tx=0x7fba28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06b6114f1ba9139 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba31240700 1 -- 172.21.15.92:0/1252933941 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800b820 con 0x7fba2c155360 2023-10-06T03:10:14.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba31240700 1 -- 172.21.15.92:0/1252933941 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2800b980 con 0x7fba2c155360 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba31240700 1 -- 172.21.15.92:0/1252933941 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800bd20 con 0x7fba2c155360 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba344a6700 1 -- 172.21.15.92:0/1252933941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c155360 msgr2=0x7fba2c0b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.730+0000 7fba344a6700 1 --2- 172.21.15.92:0/1252933941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c155360 0x7fba2c0b3bb0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fba2c06f720 tx=0x7fba28009250 comp rx=0 tx=0).stop 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.731+0000 7fba344a6700 1 -- 172.21.15.92:0/1252933941 shutdown_connections 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.731+0000 7fba344a6700 1 --2- 172.21.15.92:0/1252933941 >> v2:172.21.15.136:3300/0 conn(0x7fba2c0ced30 0x7fba2c0cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.731+0000 7fba344a6700 1 --2- 172.21.15.92:0/1252933941 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c155360 0x7fba2c0b3bb0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.731+0000 7fba344a6700 1 -- 172.21.15.92:0/1252933941 >> 172.21.15.92:0/1252933941 conn(0x7fba2c06baf0 msgr2=0x7fba2c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.731+0000 7fba344a6700 1 -- 172.21.15.92:0/1252933941 shutdown_connections 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.731+0000 7fba344a6700 1 -- 172.21.15.92:0/1252933941 wait complete. 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 Processor -- start 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 -- start start 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 0x7fba2c14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba2c0ced30 0x7fba2c0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba2c155360 0x7fba2c069c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0694a0 con 0x7fba2c0b40f0 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c069780 con 0x7fba2c155360 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.732+0000 7fba344a6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c14e430 con 0x7fba2c0ced30 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 0x7fba2c14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 0x7fba2c14def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46148/0 (socket says 172.21.15.92:46148) 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 -- 172.21.15.92:0/3580257065 learned_addr learned my addr 172.21.15.92:0/3580257065 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba31a41700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba2c0ced30 0x7fba2c0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32a43700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba2c155360 0x7fba2c069c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 -- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba2c0ced30 msgr2=0x7fba2c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba2c0ced30 0x7fba2c0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 -- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba2c155360 msgr2=0x7fba2c069c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba2c155360 0x7fba2c069c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 -- 172.21.15.92:0/3580257065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba280094f0 con 0x7fba2c0b40f0 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32a43700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba2c155360 0x7fba2c069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba32242700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 0x7fba2c14def0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fba280030c0 tx=0x7fba28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28011db0 con 0x7fba2c0b40f0 2023-10-06T03:10:14.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba28020070 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c0c9ad0 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2801b6a0 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.733+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c0ca090 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.734+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c069a60 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.734+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fba28022030 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.735+0000 7fba227fc700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fba10067320 0x7fba100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.735+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba28086af0 con 0x7fba2c0b40f0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.735+0000 7fba31a41700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fba10067320 0x7fba100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:14.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.735+0000 7fba31a41700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fba10067320 0x7fba100697d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fba2c154bf0 tx=0x7fba1c007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:14.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.739+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba28053730 con 0x7fba2c0b40f0 2023-10-06T03:10:14.923 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:14.922+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64} v 0) v1 -- 0x7fba2c069a60 con 0x7fba2c0b40f0 2023-10-06T03:10:15.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.632+0000 7fba227fc700 1 -- 172.21.15.92:0/3580257065 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64}]=0 pool 'another-data-pool' created v168) v1 ==== 142+0+0 (secure 0 0 0) 0x7fba28056d50 con 0x7fba2c0b40f0 2023-10-06T03:10:15.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fba10067320 msgr2=0x7fba100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fba10067320 0x7fba100697d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fba2c154bf0 tx=0x7fba1c007ea0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 msgr2=0x7fba2c14def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 0x7fba2c14def0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fba280030c0 tx=0x7fba28016000 comp rx=0 tx=0).stop 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 shutdown_connections 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fba10067320 0x7fba100697d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba2c0b40f0 0x7fba2c14def0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba2c155360 0x7fba2c069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 --2- 172.21.15.92:0/3580257065 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba2c0ced30 0x7fba2c0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 >> 172.21.15.92:0/3580257065 conn(0x7fba2c06baf0 msgr2=0x7fba2c15a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:15.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 shutdown_connections 2023-10-06T03:10:15.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.636+0000 7fba344a6700 1 -- 172.21.15.92:0/3580257065 wait complete. 2023-10-06T03:10:15.644 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'another-data-pool' created 2023-10-06T03:10:15.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs another-data-pool 2023-10-06T03:10:15.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.767+0000 7f76b2783700 1 Processor -- start 2023-10-06T03:10:15.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.767+0000 7f76b2783700 1 -- start start 2023-10-06T03:10:15.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b2783700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac141500 0x7f76ac0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b2783700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f76ac06c0a0 0x7f76ac0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b2783700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac15c2a0 con 0x7f76ac06bcd0 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b2783700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac0c9880 con 0x7f76ac06c0a0 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b2783700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac15b690 con 0x7f76ac141500 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76ab7fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac141500 0x7f76ac0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b0d20700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f76ac06c0a0 0x7f76ac0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76abfff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f76ac06bcd0 0x7f76ac140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:50858/0 (socket says 172.21.15.92:50858) 2023-10-06T03:10:15.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76abfff700 1 -- 172.21.15.92:0/1482442599 learned_addr learned my addr 172.21.15.92:0/1482442599 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76ab7fe700 1 -- 172.21.15.92:0/1482442599 >> v2:172.21.15.136:3300/0 conn(0x7f76ac06c0a0 msgr2=0x7f76ac0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76ab7fe700 1 --2- 172.21.15.92:0/1482442599 >> v2:172.21.15.136:3300/0 conn(0x7f76ac06c0a0 0x7f76ac0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76ab7fe700 1 -- 172.21.15.92:0/1482442599 >> v1:172.21.15.136:6789/0 conn(0x7f76ac06bcd0 legacy=0x7f76ac140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76ab7fe700 1 -- 172.21.15.92:0/1482442599 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac15b970 con 0x7f76ac141500 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.768+0000 7f76b0d20700 1 --2- 172.21.15.92:0/1482442599 >> v2:172.21.15.136:3300/0 conn(0x7f76ac06c0a0 0x7f76ac0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76ab7fe700 1 --2- 172.21.15.92:0/1482442599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac141500 0x7f76ac0c8e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f769c0098e0 tx=0x7f769c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07c69004db4afee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:15.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76aaffd700 1 -- 172.21.15.92:0/1482442599 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00b640 con 0x7f76ac141500 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76aaffd700 1 -- 172.21.15.92:0/1482442599 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00b7a0 con 0x7f76ac141500 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76aaffd700 1 -- 172.21.15.92:0/1482442599 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00bb40 con 0x7f76ac141500 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 -- 172.21.15.92:0/1482442599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac141500 msgr2=0x7f76ac0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 --2- 172.21.15.92:0/1482442599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac141500 0x7f76ac0c8e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f769c0098e0 tx=0x7f769c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 -- 172.21.15.92:0/1482442599 shutdown_connections 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 --2- 172.21.15.92:0/1482442599 >> v2:172.21.15.136:3300/0 conn(0x7f76ac06c0a0 0x7f76ac0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 --2- 172.21.15.92:0/1482442599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac141500 0x7f76ac0c8e00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 -- 172.21.15.92:0/1482442599 >> 172.21.15.92:0/1482442599 conn(0x7f76ac0ce600 msgr2=0x7f76ac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 -- 172.21.15.92:0/1482442599 shutdown_connections 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.769+0000 7f76b2783700 1 -- 172.21.15.92:0/1482442599 wait complete. 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.770+0000 7f76b2783700 1 Processor -- start 2023-10-06T03:10:15.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 -- start start 2023-10-06T03:10:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 0x7f76ac13c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:15.775 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac06c0a0 0x7f76ac0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76ac141500 0x7f76ac0c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac13c7d0 con 0x7f76ac06bcd0 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac13d800 con 0x7f76ac141500 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76b2783700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac13cbf0 con 0x7f76ac06c0a0 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.771+0000 7f76abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 0x7f76ac13c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 0x7f76ac13c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46164/0 (socket says 172.21.15.92:46164) 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 -- 172.21.15.92:0/1782772763 learned_addr learned my addr 172.21.15.92:0/1782772763 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76b0d20700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76ac141500 0x7f76ac0c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76ab7fe700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac06c0a0 0x7f76ac0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 -- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac06c0a0 msgr2=0x7f76ac0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:15.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac06c0a0 0x7f76ac0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 -- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76ac141500 msgr2=0x7f76ac0c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76ac141500 0x7f76ac0c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 -- 172.21.15.92:0/1782772763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4009a20 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76abfff700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 0x7f76ac13c290 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f769400c7c0 tx=0x7f7694007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694005b30 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7694005c90 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c009420 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694004a80 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.772+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac13d1b0 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.773+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac04c3c0 con 0x7f76ac06bcd0 2023-10-06T03:10:15.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.774+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f76940070d0 con 0x7f76ac06bcd0 2023-10-06T03:10:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.774+0000 7f76a3fff700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f768c067320 0x7f768c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.774+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6430+0+0 (secure 0 0 0) 0x7f7694084ec0 con 0x7f76ac06bcd0 2023-10-06T03:10:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.774+0000 7f76ab7fe700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f768c067320 0x7f768c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:15.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.775+0000 7f76ab7fe700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f768c067320 0x7f768c0697d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f769c0093f0 tx=0x7f769c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:15.780 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:15.778+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7694051980 con 0x7f76ac06bcd0 2023-10-06T03:10:16.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.005+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"} v 0) v1 -- 0x7f76ac06c9d0 con 0x7f76ac06bcd0 2023-10-06T03:10:16.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.642+0000 7f76a3fff700 1 -- 172.21.15.92:0/1782772763 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"}]=0 added data pool 31 to fsmap v115) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76940210b0 con 0x7f76ac06bcd0 2023-10-06T03:10:16.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f768c067320 msgr2=0x7f768c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f768c067320 0x7f768c0697d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f769c0093f0 tx=0x7f769c017000 comp rx=0 tx=0).stop 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 msgr2=0x7f76ac13c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 0x7f76ac13c290 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f769400c7c0 tx=0x7f7694007f70 comp rx=0 tx=0).stop 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 shutdown_connections 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f768c067320 0x7f768c0697d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76ac06bcd0 0x7f76ac13c290 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76ac141500 0x7f76ac0c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 --2- 172.21.15.92:0/1782772763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76ac06c0a0 0x7f76ac0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 >> 172.21.15.92:0/1782772763 conn(0x7f76ac0ce600 msgr2=0x7f76ac0c4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 shutdown_connections 2023-10-06T03:10:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.648+0000 7f76b2783700 1 -- 172.21.15.92:0/1782772763 wait complete. 2023-10-06T03:10:16.651 INFO:teuthology.orchestra.run.smithi092.stderr:added data pool 31 to fsmap 2023-10-06T03:10:16.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:16.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.776+0000 7faba335d700 1 Processor -- start 2023-10-06T03:10:16.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.776+0000 7faba335d700 1 -- start start 2023-10-06T03:10:16.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba335d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 0x7fab9c157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba335d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c06ea60 con 0x7fab9c0c6b30 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba335d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c06ed40 con 0x7fab9c0ae840 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba335d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0afe60 con 0x7fab9c157380 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba1b5a700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fab9c0c6b30 0x7fab9c15bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47286/0 (socket says 172.21.15.92:47286) 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba1b5a700 1 -- 172.21.15.92:0/3549320590 learned_addr learned my addr 172.21.15.92:0/3549320590 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250588479 0 0) 0x7fab9c06ea60 con 0x7fab9c0c6b30 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab88003570 con 0x7fab9c0c6b30 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.777+0000 7faba235b700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 0x7fab9c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4124181820 0 0) 0x7fab88003570 con 0x7fab9c0c6b30 2023-10-06T03:10:16.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 msgr2=0x7fab9c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 0x7fab9c157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 >> v1:172.21.15.136:6789/0 conn(0x7fab9c0ae840 legacy=0x7fab9c0aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0b0140 con 0x7fab9c0c6b30 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fab8c001470 con 0x7fab9c0c6b30 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 >> v1:172.21.15.92:6789/0 conn(0x7fab9c0c6b30 legacy=0x7fab9c15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 0x7fab88003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88003fd0 0x7fab88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.783 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004d50 0x7fab880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0afe60 con 0x7fab88003fd0 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800bb40 con 0x7fab88004d50 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8800bd00 con 0x7fab9c157380 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88003fd0 0x7fab88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba235b700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 0x7fab88003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba2b5c700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004d50 0x7fab880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 -- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 msgr2=0x7fab88003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c157380 0x7fab88003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 -- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004d50 msgr2=0x7fab880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:16.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88004d50 0x7fab880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 -- 172.21.15.92:0/3549320590 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88003570 con 0x7fab88003fd0 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1b5a700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88003fd0 0x7fab88004810 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00c3c0 tx=0x7fab8c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4882f6e7fa35680a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00cc90 con 0x7fab88003fd0 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00cdf0 con 0x7fab88003fd0 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.778+0000 7faba1359700 1 -- 172.21.15.92:0/3549320590 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c005b70 con 0x7fab88003fd0 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba335d700 1 -- 172.21.15.92:0/3549320590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88003fd0 msgr2=0x7fab88004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba335d700 1 --2- 172.21.15.92:0/3549320590 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab88003fd0 0x7fab88004810 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00c3c0 tx=0x7fab8c001e70 comp rx=0 tx=0).stop 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba2b5c700 1 -- 172.21.15.92:0/3549320590 reap_dead start 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba335d700 1 -- 172.21.15.92:0/3549320590 shutdown_connections 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba335d700 1 -- 172.21.15.92:0/3549320590 >> 172.21.15.92:0/3549320590 conn(0x7fab9c06ddb0 msgr2=0x7fab9c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba335d700 1 -- 172.21.15.92:0/3549320590 shutdown_connections 2023-10-06T03:10:16.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.779+0000 7faba335d700 1 -- 172.21.15.92:0/3549320590 wait complete. 2023-10-06T03:10:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.780+0000 7faba335d700 1 Processor -- start 2023-10-06T03:10:16.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.780+0000 7faba335d700 1 -- start start 2023-10-06T03:10:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba335d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003fd0 0x7fab9c16a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba335d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c06b7b0 0x7fab9c16ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba335d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 0x7fab9c164860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba335d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c13c070 con 0x7fab9c1643e0 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba335d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c14fc40 con 0x7fab88003fd0 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba335d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c164da0 con 0x7fab9c06b7b0 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 0x7fab9c164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 0x7fab9c164860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46186/0 (socket says 172.21.15.92:46186) 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 -- 172.21.15.92:0/3155093252 learned_addr learned my addr 172.21.15.92:0/3155093252 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba235b700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003fd0 0x7fab9c16a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba1b5a700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c06b7b0 0x7fab9c16ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 -- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c06b7b0 msgr2=0x7fab9c16ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c06b7b0 0x7fab9c16ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 -- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003fd0 msgr2=0x7fab9c16a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:16.789 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003fd0 0x7fab9c16a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 -- 172.21.15.92:0/3155093252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c14f030 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba1b5a700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c06b7b0 0x7fab9c16ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba235b700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003fd0 0x7fab9c16a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.781+0000 7faba2b5c700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 0x7fab9c164860 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0684b0 tx=0x7fab94009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.782+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94012590 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.782+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab94012710 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.782+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c14f310 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.782+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94012ad0 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.782+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c14f8d0 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.782+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c04c3c0 con 0x7fab9c1643e0 2023-10-06T03:10:16.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.783+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fab9401d460 con 0x7fab9c1643e0 2023-10-06T03:10:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.783+0000 7fab927fc700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab80067320 0x7fab800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.783+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fab94084f80 con 0x7fab9c1643e0 2023-10-06T03:10:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.783+0000 7faba235b700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab80067320 0x7fab800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.784+0000 7faba235b700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab80067320 0x7fab800697d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab980099f0 tx=0x7fab9800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:16.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:16.787+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab94051920 con 0x7fab9c1643e0 2023-10-06T03:10:17.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.011+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab9c0b1670 con 0x7fab9c1643e0 2023-10-06T03:10:17.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.011+0000 7fab927fc700 1 -- 172.21.15.92:0/3155093252 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fab94054f40 con 0x7fab9c1643e0 2023-10-06T03:10:17.013 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:17.013 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:16.631396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab80067320 msgr2=0x7fab800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab80067320 0x7fab800697d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab980099f0 tx=0x7fab9800b040 comp rx=0 tx=0).stop 2023-10-06T03:10:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 msgr2=0x7fab9c164860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 0x7fab9c164860 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0684b0 tx=0x7fab94009040 comp rx=0 tx=0).stop 2023-10-06T03:10:17.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 shutdown_connections 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fab80067320 0x7fab800697d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab9c1643e0 0x7fab9c164860 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab88003fd0 0x7fab9c16a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 --2- 172.21.15.92:0/3155093252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab9c06b7b0 0x7fab9c16ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 >> 172.21.15.92:0/3155093252 conn(0x7fab9c06ddb0 msgr2=0x7fab9c154e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 shutdown_connections 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.013+0000 7faba335d700 1 -- 172.21.15.92:0/3155093252 wait complete. 2023-10-06T03:10:17.016 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 115 2023-10-06T03:10:17.027 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:17.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.139+0000 7fbd3a6de700 1 Processor -- start 2023-10-06T03:10:17.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.139+0000 7fbd3a6de700 1 -- start start 2023-10-06T03:10:17.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd3a6de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd3415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd3a6de700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3415a160 con 0x7fbd3415db10 2023-10-06T03:10:17.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd3a6de700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd340d13a0 con 0x7fbd34159890 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd3a6de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3415e140 con 0x7fbd34159c60 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd38c7b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbd3415db10 0x7fbd34069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47290/0 (socket says 172.21.15.92:47290) 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd38c7b700 1 -- 172.21.15.92:0/3242359330 learned_addr learned my addr 172.21.15.92:0/3242359330 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.140+0000 7fbd337fe700 1 --2- 172.21.15.92:0/3242359330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd3415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd32ffd700 1 -- 172.21.15.92:0/3242359330 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054063391 0 0) 0x7fbd3415a160 con 0x7fbd3415db10 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd32ffd700 1 -- 172.21.15.92:0/3242359330 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd1c003570 con 0x7fbd3415db10 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd32ffd700 1 -- 172.21.15.92:0/3242359330 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018195548 0 0) 0x7fbd340d13a0 con 0x7fbd34159890 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd337fe700 1 -- 172.21.15.92:0/3242359330 >> v1:172.21.15.136:6789/0 conn(0x7fbd34159890 legacy=0x7fbd3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd337fe700 1 -- 172.21.15.92:0/3242359330 >> v1:172.21.15.92:6789/0 conn(0x7fbd3415db10 legacy=0x7fbd34069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd337fe700 1 -- 172.21.15.92:0/3242359330 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3415a3d0 con 0x7fbd34159c60 2023-10-06T03:10:17.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd337fe700 1 --2- 172.21.15.92:0/3242359330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd3415d5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009910 tx=0x7fbd280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=491757c992acce51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd32ffd700 1 -- 172.21.15.92:0/3242359330 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28011630 con 0x7fbd34159c60 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd32ffd700 1 -- 172.21.15.92:0/3242359330 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28011790 con 0x7fbd34159c60 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.141+0000 7fbd32ffd700 1 -- 172.21.15.92:0/3242359330 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28011b50 con 0x7fbd34159c60 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 -- 172.21.15.92:0/3242359330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 msgr2=0x7fbd3415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/3242359330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd3415d5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009910 tx=0x7fbd280091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 -- 172.21.15.92:0/3242359330 shutdown_connections 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/3242359330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd3415d5d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 -- 172.21.15.92:0/3242359330 >> 172.21.15.92:0/3242359330 conn(0x7fbd34068e20 msgr2=0x7fbd340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 -- 172.21.15.92:0/3242359330 shutdown_connections 2023-10-06T03:10:17.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.142+0000 7fbd3a6de700 1 -- 172.21.15.92:0/3242359330 wait complete. 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.143+0000 7fbd3a6de700 1 Processor -- start 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.143+0000 7fbd3a6de700 1 -- start start 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 0x7fbd3416de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd34169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3415db10 0x7fbd34140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3416adc0 con 0x7fbd34159890 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3416e3d0 con 0x7fbd3415db10 2023-10-06T03:10:17.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3416a340 con 0x7fbd34159c60 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 0x7fbd3416de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 0x7fbd3416de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46188/0 (socket says 172.21.15.92:46188) 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 -- 172.21.15.92:0/2516592039 learned_addr learned my addr 172.21.15.92:0/2516592039 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd337fe700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd34169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd38c7b700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3415db10 0x7fbd34140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 -- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 msgr2=0x7fbd34169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd34169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 -- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3415db10 msgr2=0x7fbd34140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3415db10 0x7fbd34140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 -- 172.21.15.92:0/2516592039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd28009420 con 0x7fbd34159890 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd337fe700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd34169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:17.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd33fff700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 0x7fbd3416de90 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c7e0 tx=0x7fbd24007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24012700 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd24012880 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3416a800 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.144+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24012c40 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.145+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd340b0700 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.146+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3404c3c0 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.146+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd2401b400 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.146+0000 7fbd1bfff700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd20067320 0x7fbd200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.146+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fbd24084b30 con 0x7fbd34159890 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.146+0000 7fbd337fe700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd20067320 0x7fbd200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.147+0000 7fbd337fe700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd20067320 0x7fbd200697d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbd3415a090 tx=0x7fbd28016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.150+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd240514d0 con 0x7fbd34159890 2023-10-06T03:10:17.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.335+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd3406a5b0 con 0x7fbd34159890 2023-10-06T03:10:17.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.336+0000 7fbd1bfff700 1 -- 172.21.15.92:0/2516592039 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+18463 (secure 0 0 0) 0x7fbd24054af0 con 0x7fbd34159890 2023-10-06T03:10:17.338 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:17.338 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":169,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:16.007461+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-10-06T03:10:14.941237+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd20067320 msgr2=0x7fbd200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd20067320 0x7fbd200697d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbd3415a090 tx=0x7fbd28016000 comp rx=0 tx=0).stop 2023-10-06T03:10:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 msgr2=0x7fbd3416de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 0x7fbd3416de90 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c7e0 tx=0x7fbd24007df0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 shutdown_connections 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd20067320 0x7fbd200697d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd34159890 0x7fbd3416de90 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3415db10 0x7fbd34140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 --2- 172.21.15.92:0/2516592039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd34159c60 0x7fbd34169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 >> 172.21.15.92:0/2516592039 conn(0x7fbd34068e20 msgr2=0x7fbd34153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 shutdown_connections 2023-10-06T03:10:17.341 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.338+0000 7fbd3a6de700 1 -- 172.21.15.92:0/2516592039 wait complete. 2023-10-06T03:10:17.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.465+0000 7fea663ec700 1 Processor -- start 2023-10-06T03:10:17.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.465+0000 7fea663ec700 1 -- start start 2023-10-06T03:10:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.466+0000 7fea663ec700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fea600b1420 0x7fea600ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.466+0000 7fea663ec700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fea600ccf90 0x7fea600c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.466+0000 7fea663ec700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea600cc300 con 0x7fea600cbf30 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.466+0000 7fea663ec700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea60156bf0 con 0x7fea600ccf90 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.466+0000 7fea663ec700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea600ae950 con 0x7fea600b1420 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.466+0000 7fea64989700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fea600ccf90 0x7fea600c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fea600ccf90 0x7fea600c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46202/0 (socket says 172.21.15.92:46202) 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 -- 172.21.15.92:0/1186226630 learned_addr learned my addr 172.21.15.92:0/1186226630 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea5f7fe700 1 --2- 172.21.15.92:0/1186226630 >> v2:172.21.15.136:3300/0 conn(0x7fea600b1420 0x7fea600ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 -- 172.21.15.92:0/1186226630 >> v2:172.21.15.136:3300/0 conn(0x7fea600b1420 msgr2=0x7fea600ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:17.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 --2- 172.21.15.92:0/1186226630 >> v2:172.21.15.136:3300/0 conn(0x7fea600b1420 0x7fea600ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 -- 172.21.15.92:0/1186226630 >> v1:172.21.15.136:6789/0 conn(0x7fea600cbf30 legacy=0x7fea600b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 -- 172.21.15.92:0/1186226630 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea600ad370 con 0x7fea600ccf90 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea5f7fe700 1 --2- 172.21.15.92:0/1186226630 >> v2:172.21.15.136:3300/0 conn(0x7fea600b1420 0x7fea600ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea64989700 1 --2- 172.21.15.92:0/1186226630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600ccf90 0x7fea600c9740 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fea600c9280 tx=0x7fea5800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b527f799d1de05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea5effd700 1 -- 172.21.15.92:0/1186226630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58010070 con 0x7fea600ccf90 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea5effd700 1 -- 172.21.15.92:0/1186226630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5800de80 con 0x7fea600ccf90 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.467+0000 7fea5effd700 1 -- 172.21.15.92:0/1186226630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800b320 con 0x7fea600ccf90 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 -- 172.21.15.92:0/1186226630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600ccf90 msgr2=0x7fea600c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 --2- 172.21.15.92:0/1186226630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600ccf90 0x7fea600c9740 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fea600c9280 tx=0x7fea5800ba50 comp rx=0 tx=0).stop 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 -- 172.21.15.92:0/1186226630 shutdown_connections 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 --2- 172.21.15.92:0/1186226630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600ccf90 0x7fea600c9740 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 --2- 172.21.15.92:0/1186226630 >> v2:172.21.15.136:3300/0 conn(0x7fea600b1420 0x7fea600ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 -- 172.21.15.92:0/1186226630 >> 172.21.15.92:0/1186226630 conn(0x7fea6006a320 msgr2=0x7fea600cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:17.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 -- 172.21.15.92:0/1186226630 shutdown_connections 2023-10-06T03:10:17.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.468+0000 7fea663ec700 1 -- 172.21.15.92:0/1186226630 wait complete. 2023-10-06T03:10:17.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.469+0000 7fea663ec700 1 Processor -- start 2023-10-06T03:10:17.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.469+0000 7fea663ec700 1 -- start start 2023-10-06T03:10:17.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea600b1420 0x7fea60144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 0x7fea60159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea600ccf90 0x7fea6015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60145cc0 con 0x7fea600cbf30 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601450b0 con 0x7fea600ccf90 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea60145390 con 0x7fea600b1420 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 0x7fea60159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 0x7fea60159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46204/0 (socket says 172.21.15.92:46204) 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 -- 172.21.15.92:0/2322089261 learned_addr learned my addr 172.21.15.92:0/2322089261 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 -- 172.21.15.92:0/2322089261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea600b1420 msgr2=0x7fea60144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea600b1420 0x7fea60144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 -- 172.21.15.92:0/2322089261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea600ccf90 msgr2=0x7fea6015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea600ccf90 0x7fea6015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 -- 172.21.15.92:0/2322089261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5800bcf0 con 0x7fea600cbf30 2023-10-06T03:10:17.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea5f7fe700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 0x7fea60159d10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c800 tx=0x7fea54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54011600 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54011760 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54011b20 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea60145670 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.470+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea600c8080 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.471+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6004c3c0 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.472+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fea540193b0 con 0x7fea600cbf30 2023-10-06T03:10:17.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.472+0000 7fea47fff700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea4c067320 0x7fea4c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.477 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.472+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fea54085120 con 0x7fea600cbf30 2023-10-06T03:10:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.472+0000 7fea5ffff700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea4c067320 0x7fea4c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.473+0000 7fea5ffff700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea4c067320 0x7fea4c0697d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fea50001730 tx=0x7fea50007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.476+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea540519b0 con 0x7fea600cbf30 2023-10-06T03:10:17.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.704+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea600ad650 con 0x7fea600cbf30 2023-10-06T03:10:17.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.705+0000 7fea47fff700 1 -- 172.21.15.92:0/2322089261 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+4607 (secure 0 0 0) 0x7fea54054fd0 con 0x7fea600cbf30 2023-10-06T03:10:17.707 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:17.707 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":115,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:16.631396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea4c067320 msgr2=0x7fea4c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea4c067320 0x7fea4c0697d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fea50001730 tx=0x7fea50007b40 comp rx=0 tx=0).stop 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 msgr2=0x7fea60159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 0x7fea60159d10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c800 tx=0x7fea54008040 comp rx=0 tx=0).stop 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 shutdown_connections 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea4c067320 0x7fea4c0697d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea600ccf90 0x7fea6015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea600b1420 0x7fea60144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 --2- 172.21.15.92:0/2322089261 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea600cbf30 0x7fea60159d10 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 >> 172.21.15.92:0/2322089261 conn(0x7fea6006a320 msgr2=0x7fea600afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:17.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 shutdown_connections 2023-10-06T03:10:17.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.707+0000 7fea663ec700 1 -- 172.21.15.92:0/2322089261 wait complete. 2023-10-06T03:10:17.710 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 115 2023-10-06T03:10:17.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.834+0000 7f9b1cbab700 1 Processor -- start 2023-10-06T03:10:17.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.834+0000 7f9b1cbab700 1 -- start start 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1cbab700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9b18159890 0x7f9b1815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1cbab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 0x7f9b180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1cbab700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b18069ee0 con 0x7f9b18159c60 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1cbab700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1815a160 con 0x7f9b18159890 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1cbab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180d13a0 con 0x7f9b18069560 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1659c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9b18159890 0x7f9b1815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 0x7f9b180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1659c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9b18159890 0x7f9b1815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:40866/0 (socket says 172.21.15.92:40866) 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b16d9d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 0x7f9b180699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:44794/0 (socket says 172.21.15.92:44794) 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b1659c700 1 -- 172.21.15.92:0/1562575112 learned_addr learned my addr 172.21.15.92:0/1562575112 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:17.839 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.835+0000 7f9b15d9b700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f9b18159c60 0x7f9b1815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:50910/0 (socket says 172.21.15.92:50910) 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1659c700 1 -- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 msgr2=0x7f9b180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1659c700 1 --2- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 0x7f9b180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1659c700 1 -- 172.21.15.92:0/1562575112 >> v1:172.21.15.136:6789/0 conn(0x7f9b18159c60 legacy=0x7f9b1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1659c700 1 -- 172.21.15.92:0/1562575112 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1815e140 con 0x7f9b18159890 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b16d9d700 1 --2- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 0x7f9b180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1659c700 1 --2- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18159890 0x7f9b1815abd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009f20 tx=0x7f9b08009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef7a103aafc05c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1559a700 1 -- 172.21.15.92:0/1562575112 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0800e070 con 0x7f9b18159890 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1559a700 1 -- 172.21.15.92:0/1562575112 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0800b9d0 con 0x7f9b18159890 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.836+0000 7f9b1559a700 1 -- 172.21.15.92:0/1562575112 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080092e0 con 0x7f9b18159890 2023-10-06T03:10:17.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 -- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18159890 msgr2=0x7f9b1815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18159890 0x7f9b1815abd0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009f20 tx=0x7f9b08009840 comp rx=0 tx=0).stop 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 -- 172.21.15.92:0/1562575112 shutdown_connections 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18159890 0x7f9b1815abd0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/1562575112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18069560 0x7f9b180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 -- 172.21.15.92:0/1562575112 >> 172.21.15.92:0/1562575112 conn(0x7f9b18068e20 msgr2=0x7f9b180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 -- 172.21.15.92:0/1562575112 shutdown_connections 2023-10-06T03:10:17.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.837+0000 7f9b1cbab700 1 -- 172.21.15.92:0/1562575112 wait complete. 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.838+0000 7f9b1cbab700 1 Processor -- start 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.838+0000 7f9b1cbab700 1 -- start start 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18069560 0x7f9b18145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 0x7f9b180cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18159c60 0x7f9b180cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b181457a0 con 0x7f9b18159890 2023-10-06T03:10:17.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180c9950 con 0x7f9b18069560 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b180c9c30 con 0x7f9b18159c60 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 0x7f9b180cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 0x7f9b180cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46206/0 (socket says 172.21.15.92:46206) 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 -- 172.21.15.92:0/956681393 learned_addr learned my addr 172.21.15.92:0/956681393 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1659c700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18069560 0x7f9b18145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b16d9d700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18159c60 0x7f9b180cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 -- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18159c60 msgr2=0x7f9b180cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18159c60 0x7f9b180cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 -- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18069560 msgr2=0x7f9b18145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18069560 0x7f9b18145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 -- 172.21.15.92:0/956681393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b10009420 con 0x7f9b18159890 2023-10-06T03:10:17.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b16d9d700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18159c60 0x7f9b180cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b15d9b700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 0x7f9b180cf210 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c7c0 tx=0x7f9b0c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c005a70 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c005bd0 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b08009ae0 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0070d0 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.839+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b180ca1f0 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.840+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1804c3c0 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.841+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9b0c007480 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.841+0000 7f9b067fc700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9af8067320 0x7f9af80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.841+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6462+0+0 (secure 0 0 0) 0x7f9b0c084500 con 0x7f9b18159890 2023-10-06T03:10:17.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.841+0000 7f9b1659c700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9af8067320 0x7f9af80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.841+0000 7f9b1659c700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9af8067320 0x7f9af80697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009880 tx=0x7f9b08018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:17.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:17.845+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b0c050ea0 con 0x7f9b18159890 2023-10-06T03:10:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.028+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b1806a5b0 con 0x7f9b18159890 2023-10-06T03:10:18.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.029+0000 7f9b067fc700 1 -- 172.21.15.92:0/956681393 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18450 (secure 0 0 0) 0x7f9b0c017090 con 0x7f9b18159890 2023-10-06T03:10:18.032 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:18.032 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":170,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:17.639650+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-10-06T03:10:14.941237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9af8067320 msgr2=0x7f9af80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9af8067320 0x7f9af80697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009880 tx=0x7f9b08018000 comp rx=0 tx=0).stop 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 msgr2=0x7f9b180cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 0x7f9b180cf210 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c7c0 tx=0x7f9b0c007f70 comp rx=0 tx=0).stop 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 shutdown_connections 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9af8067320 0x7f9af80697d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b18159c60 0x7f9b180cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b18069560 0x7f9b18145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 --2- 172.21.15.92:0/956681393 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b18159890 0x7f9b180cf210 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.031+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 >> 172.21.15.92:0/956681393 conn(0x7f9b18068e20 msgr2=0x7f9b18154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:18.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.032+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 shutdown_connections 2023-10-06T03:10:18.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.032+0000 7f9b1cbab700 1 -- 172.21.15.92:0/956681393 wait complete. 2023-10-06T03:10:18.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000126469 --yes-i-really-mean-it 2023-10-06T03:10:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.165+0000 7fe38aa91700 1 Processor -- start 2023-10-06T03:10:18.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.165+0000 7fe38aa91700 1 -- start start 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38aa91700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe384159ec0 0x7fe3840b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38aa91700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3840b53d0 0x7fe3840cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38aa91700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840b5b00 con 0x7fe3840cce40 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38aa91700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840b5850 con 0x7fe384159ec0 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38aa91700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840b5da0 con 0x7fe3840b53d0 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38882d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe384159ec0 0x7fe3840b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38902e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe3840cce40 0x7fe3840cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47300/0 (socket says 172.21.15.92:47300) 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.166+0000 7fe38902e700 1 -- 172.21.15.92:0/1797963841 learned_addr learned my addr 172.21.15.92:0/1797963841 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38882d700 1 -- 172.21.15.92:0/1797963841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3840b53d0 msgr2=0x7fe3840cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:18.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe383fff700 1 --2- 172.21.15.92:0/1797963841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3840b53d0 0x7fe3840cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38882d700 1 --2- 172.21.15.92:0/1797963841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3840b53d0 0x7fe3840cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe3837fe700 1 -- 172.21.15.92:0/1797963841 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846804347 0 0) 0x7fe3840b5b00 con 0x7fe3840cce40 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38882d700 1 -- 172.21.15.92:0/1797963841 >> v1:172.21.15.92:6789/0 conn(0x7fe3840cce40 legacy=0x7fe3840cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38882d700 1 -- 172.21.15.92:0/1797963841 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840b6080 con 0x7fe384159ec0 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38882d700 1 --2- 172.21.15.92:0/1797963841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe384159ec0 0x7fe3840b4e90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe3740098d0 tx=0x7fe374009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492d1467387d839 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe3837fe700 1 -- 172.21.15.92:0/1797963841 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374009e70 con 0x7fe384159ec0 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe3837fe700 1 -- 172.21.15.92:0/1797963841 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374005c10 con 0x7fe384159ec0 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe3837fe700 1 -- 172.21.15.92:0/1797963841 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400b650 con 0x7fe384159ec0 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38aa91700 1 -- 172.21.15.92:0/1797963841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe384159ec0 msgr2=0x7fe3840b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.167+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1797963841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe384159ec0 0x7fe3840b4e90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe3740098d0 tx=0x7fe374009120 comp rx=0 tx=0).stop 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.168+0000 7fe38aa91700 1 -- 172.21.15.92:0/1797963841 shutdown_connections 2023-10-06T03:10:18.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.168+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1797963841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe384159ec0 0x7fe3840b4e90 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.168+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1797963841 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3840b53d0 0x7fe3840cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.168+0000 7fe38aa91700 1 -- 172.21.15.92:0/1797963841 >> 172.21.15.92:0/1797963841 conn(0x7fe3840999f0 msgr2=0x7fe3840b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.168+0000 7fe38aa91700 1 -- 172.21.15.92:0/1797963841 shutdown_connections 2023-10-06T03:10:18.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.168+0000 7fe38aa91700 1 -- 172.21.15.92:0/1797963841 wait complete. 2023-10-06T03:10:18.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 Processor -- start 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 -- start start 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 0x7fe38406aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3840cce40 0x7fe384156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe384159ec0 0x7fe38416a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe384157120 con 0x7fe3840b53d0 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38416a750 con 0x7fe3840cce40 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.169+0000 7fe38aa91700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe384069f70 con 0x7fe384159ec0 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 0x7fe38406aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 0x7fe38406aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46236/0 (socket says 172.21.15.92:46236) 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 -- 172.21.15.92:0/1756150445 learned_addr learned my addr 172.21.15.92:0/1756150445 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:18.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38902e700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe384159ec0 0x7fe38416a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe383fff700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3840cce40 0x7fe384156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 -- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe384159ec0 msgr2=0x7fe38416a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe384159ec0 0x7fe38416a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 -- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3840cce40 msgr2=0x7fe384156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3840cce40 0x7fe384156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 -- 172.21.15.92:0/1756150445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740093c0 con 0x7fe3840b53d0 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38882d700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 0x7fe38406aeb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe374009fd0 tx=0x7fe374016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:18.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374006460 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3740065e0 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe384069360 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374026430 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.170+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe384069920 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.171+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe384063c90 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.171+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe37401a420 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.172+0000 7fe380ff9700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe364067320 0x7fe3640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.172+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6462+0+0 (secure 0 0 0) 0x7fe374086950 con 0x7fe3840b53d0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.172+0000 7fe383fff700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe364067320 0x7fe3640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:18.178 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.172+0000 7fe383fff700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe364067320 0x7fe3640697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006fd0 tx=0x7fe36c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:18.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.176+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3740532f0 con 0x7fe3840b53d0 2023-10-06T03:10:18.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:18.385+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000126469", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe38406a0b0 con 0x7fe364067320 2023-10-06T03:10:19.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.659+0000 7fe380ff9700 1 -- 172.21.15.92:0/1756150445 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+127 (secure 0 0 0) 0x7fe36c00f630 con 0x7fe364067320 2023-10-06T03:10:19.661 INFO:teuthology.orchestra.run.smithi092.stdout:FS volume 'cephfs' renamed to 'volume_0000000000126469'. But failed to rename data pools as more than one data pool was found. 2023-10-06T03:10:19.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.661+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe364067320 msgr2=0x7fe3640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:19.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.661+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe364067320 0x7fe3640697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006fd0 tx=0x7fe36c008040 comp rx=0 tx=0).stop 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.661+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 msgr2=0x7fe38406aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.661+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 0x7fe38406aeb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe374009fd0 tx=0x7fe374016000 comp rx=0 tx=0).stop 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 shutdown_connections 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe364067320 0x7fe3640697d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3840b53d0 0x7fe38406aeb0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe384159ec0 0x7fe38416a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 --2- 172.21.15.92:0/1756150445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3840cce40 0x7fe384156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 >> 172.21.15.92:0/1756150445 conn(0x7fe3840999f0 msgr2=0x7fe3840c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 shutdown_connections 2023-10-06T03:10:19.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.662+0000 7fe38aa91700 1 -- 172.21.15.92:0/1756150445 wait complete. 2023-10-06T03:10:19.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:10:19.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.799+0000 7f51b6f0a700 1 Processor -- start 2023-10-06T03:10:19.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.799+0000 7f51b6f0a700 1 -- start start 2023-10-06T03:10:19.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b6f0a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f51b0159420 0x7f51b00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:19.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b6f0a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b00b2980 con 0x7f51b01597f0 2023-10-06T03:10:19.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b6f0a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b00b11d0 con 0x7f51b015d230 2023-10-06T03:10:19.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b6f0a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b00b14b0 con 0x7f51b0159420 2023-10-06T03:10:19.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b5f08700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f51b0159420 0x7f51b00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b5707700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f51b01597f0 0x7f51b00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37846/0 (socket says 172.21.15.92:37846) 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b5707700 1 -- 172.21.15.92:0/3227368622 learned_addr learned my addr 172.21.15.92:0/3227368622 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b5f08700 1 -- 172.21.15.92:0/3227368622 >> v1:172.21.15.136:6789/0 conn(0x7f51b015d230 legacy=0x7f51b015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b5f08700 1 -- 172.21.15.92:0/3227368622 >> v1:172.21.15.92:6789/0 conn(0x7f51b01597f0 legacy=0x7f51b00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b5f08700 1 -- 172.21.15.92:0/3227368622 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00b1790 con 0x7f51b0159420 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.800+0000 7f51b4f06700 1 -- 172.21.15.92:0/3227368622 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181868087 0 0) 0x7f51b00b2980 con 0x7f51b01597f0 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b5f08700 1 --2- 172.21.15.92:0/3227368622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0159420 0x7f51b00b5f40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009e60 tx=0x7f51a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667b6501445c66a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b4f06700 1 -- 172.21.15.92:0/3227368622 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000e070 con 0x7f51b0159420 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b4f06700 1 -- 172.21.15.92:0/3227368622 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a000bbf0 con 0x7f51b0159420 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b4f06700 1 -- 172.21.15.92:0/3227368622 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0012430 con 0x7f51b0159420 2023-10-06T03:10:19.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 -- 172.21.15.92:0/3227368622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0159420 msgr2=0x7f51b00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:19.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/3227368622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0159420 0x7f51b00b5f40 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009e60 tx=0x7f51a0009780 comp rx=0 tx=0).stop 2023-10-06T03:10:19.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 -- 172.21.15.92:0/3227368622 shutdown_connections 2023-10-06T03:10:19.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/3227368622 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b0159420 0x7f51b00b5f40 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 -- 172.21.15.92:0/3227368622 >> 172.21.15.92:0/3227368622 conn(0x7f51b00d0660 msgr2=0x7f51b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:19.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 -- 172.21.15.92:0/3227368622 shutdown_connections 2023-10-06T03:10:19.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.801+0000 7f51b6f0a700 1 -- 172.21.15.92:0/3227368622 wait complete. 2023-10-06T03:10:19.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.802+0000 7f51b6f0a700 1 Processor -- start 2023-10-06T03:10:19.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.802+0000 7f51b6f0a700 1 -- start start 2023-10-06T03:10:19.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6f0a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51b0159420 0x7f51b0149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6f0a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51b01597f0 0x7f51b0068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6f0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 0x7f51b00690d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6f0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00c5400 con 0x7f51b015d230 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6f0a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00c5ca0 con 0x7f51b0159420 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6f0a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00b56f0 con 0x7f51b01597f0 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 0x7f51b00690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 0x7f51b00690d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:35998/0 (socket says 172.21.15.92:35998) 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 -- 172.21.15.92:0/813392176 learned_addr learned my addr 172.21.15.92:0/813392176 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b5707700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51b01597f0 0x7f51b0068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b5f08700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51b0159420 0x7f51b0149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:19.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 -- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51b01597f0 msgr2=0x7f51b0068b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51b01597f0 0x7f51b0068b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 -- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51b0159420 msgr2=0x7f51b0149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.803+0000 7f51b6709700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51b0159420 0x7f51b0149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51b6709700 1 -- 172.21.15.92:0/813392176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0009a20 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51b6709700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 0x7f51b00690d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f51a800b4e0 tx=0x7f51a8009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800f510 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800f670 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b00c5650 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800fa30 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.804+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b00c9b40 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.805+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b015dd70 con 0x7f51b015d230 2023-10-06T03:10:19.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.805+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f51a801a400 con 0x7f51b015d230 2023-10-06T03:10:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.806+0000 7f51adffb700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f519c067320 0x7f519c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.806+0000 7f51b5f08700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f519c067320 0x7f519c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.806+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f51a80849a0 con 0x7f51b015d230 2023-10-06T03:10:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.806+0000 7f51b5f08700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f519c067320 0x7f519c0697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f51a00099f0 tx=0x7f51a0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:19.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:19.810+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51a8051370 con 0x7f51b015d230 2023-10-06T03:10:20.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.017+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51b004c3c0 con 0x7f519c067320 2023-10-06T03:10:20.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.018+0000 7f51adffb700 1 -- 172.21.15.92:0/813392176 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f51b004c3c0 con 0x7f519c067320 2023-10-06T03:10:20.020 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:10:20.020 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:20.020 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volume_0000000000126469" 2023-10-06T03:10:20.020 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:20.020 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f519c067320 msgr2=0x7f519c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f519c067320 0x7f519c0697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f51a00099f0 tx=0x7f51a0018000 comp rx=0 tx=0).stop 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 msgr2=0x7f51b00690d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 0x7f51b00690d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f51a800b4e0 tx=0x7f51a8009060 comp rx=0 tx=0).stop 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 shutdown_connections 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f519c067320 0x7f519c0697d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51b015d230 0x7f51b00690d0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51b0159420 0x7f51b0149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 --2- 172.21.15.92:0/813392176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51b01597f0 0x7f51b0068b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 >> 172.21.15.92:0/813392176 conn(0x7f51b00d0660 msgr2=0x7f51b0069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 shutdown_connections 2023-10-06T03:10:20.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.020+0000 7f51b6f0a700 1 -- 172.21.15.92:0/813392176 wait complete. 2023-10-06T03:10:20.034 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:20.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.146+0000 7f24e4e0b700 1 Processor -- start 2023-10-06T03:10:20.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.146+0000 7f24e4e0b700 1 -- start start 2023-10-06T03:10:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24e4e0b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f24e0159890 0x7f24e015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24e4e0b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e0159c60 0x7f24e015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24e4e0b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e015a160 con 0x7f24e015db10 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24e4e0b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e015df50 con 0x7f24e0159c60 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24e4e0b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e00d13a0 con 0x7f24e0159890 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24ddd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e0159c60 0x7f24e015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24ddd9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f24e0159c60 0x7f24e015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36012/0 (socket says 172.21.15.92:36012) 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.147+0000 7f24ddd9b700 1 -- 172.21.15.92:0/3348626332 learned_addr learned my addr 172.21.15.92:0/3348626332 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24de59c700 1 --2- 172.21.15.92:0/3348626332 >> v2:172.21.15.136:3300/0 conn(0x7f24e0159890 0x7f24e015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24ddd9b700 1 -- 172.21.15.92:0/3348626332 >> v2:172.21.15.136:3300/0 conn(0x7f24e0159890 msgr2=0x7f24e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/3348626332 >> v2:172.21.15.136:3300/0 conn(0x7f24e0159890 0x7f24e015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24ddd9b700 1 -- 172.21.15.92:0/3348626332 >> v1:172.21.15.136:6789/0 conn(0x7f24e015db10 legacy=0x7f24e0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24ddd9b700 1 -- 172.21.15.92:0/3348626332 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e015e140 con 0x7f24e0159c60 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/3348626332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e0159c60 0x7f24e015d5d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f24e00d1090 tx=0x7f24c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d94bebb8a9a6ae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24dd59a700 1 -- 172.21.15.92:0/3348626332 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800b860 con 0x7f24e0159c60 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24dd59a700 1 -- 172.21.15.92:0/3348626332 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c800b9c0 con 0x7f24e0159c60 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.148+0000 7f24dd59a700 1 -- 172.21.15.92:0/3348626332 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800bd60 con 0x7f24e0159c60 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 -- 172.21.15.92:0/3348626332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e0159c60 msgr2=0x7f24e015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/3348626332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e0159c60 0x7f24e015d5d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f24e00d1090 tx=0x7f24c8009250 comp rx=0 tx=0).stop 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 -- 172.21.15.92:0/3348626332 shutdown_connections 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/3348626332 >> v2:172.21.15.136:3300/0 conn(0x7f24e0159890 0x7f24e015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/3348626332 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e0159c60 0x7f24e015d5d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 -- 172.21.15.92:0/3348626332 >> 172.21.15.92:0/3348626332 conn(0x7f24e0068e20 msgr2=0x7f24e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 -- 172.21.15.92:0/3348626332 shutdown_connections 2023-10-06T03:10:20.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.149+0000 7f24e4e0b700 1 -- 172.21.15.92:0/3348626332 wait complete. 2023-10-06T03:10:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.150+0000 7f24e4e0b700 1 Processor -- start 2023-10-06T03:10:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.150+0000 7f24e4e0b700 1 -- start start 2023-10-06T03:10:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.150+0000 7f24e4e0b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0159890 0x7f24e016c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e0159c60 0x7f24e00c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 0x7f24e00c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e016cf20 con 0x7f24e015db10 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00cef30 con 0x7f24e0159890 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e00cf210 con 0x7f24e0159c60 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 0x7f24e00c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 0x7f24e00c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36016/0 (socket says 172.21.15.92:36016) 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 -- 172.21.15.92:0/2399107805 learned_addr learned my addr 172.21.15.92:0/2399107805 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ddd9b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e0159c60 0x7f24e00c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24de59c700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0159890 0x7f24e016c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 -- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e0159c60 msgr2=0x7f24e00c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e0159c60 0x7f24e00c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.156 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 -- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0159890 msgr2=0x7f24e016c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0159890 0x7f24e016c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 -- 172.21.15.92:0/2399107805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80094f0 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24de59c700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0159890 0x7f24e016c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24ded9d700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 0x7f24e00c7fa0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f24d800c4c0 tx=0x7f24d8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8011590 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d8011710 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e00cf4f0 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8011ad0 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.151+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e00cfab0 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.152+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e004c3c0 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.153+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f24d801b3b0 con 0x7f24e015db10 2023-10-06T03:10:20.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.153+0000 7f24d67fc700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c0067320 0x7f24c00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.153+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f24d8084360 con 0x7f24e015db10 2023-10-06T03:10:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.153+0000 7f24de59c700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c0067320 0x7f24c00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.158 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.154+0000 7f24de59c700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c0067320 0x7f24c00697d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f24d0007540 tx=0x7f24d000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.157+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24d8050cb0 con 0x7f24e015db10 2023-10-06T03:10:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.382+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24e006a5b0 con 0x7f24e015db10 2023-10-06T03:10:20.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.383+0000 7f24d67fc700 1 -- 172.21.15.92:0/2399107805 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f24d8017090 con 0x7f24e015db10 2023-10-06T03:10:20.385 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:20.385 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:18.647577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000000126469","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c0067320 msgr2=0x7f24c00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c0067320 0x7f24c00697d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f24d0007540 tx=0x7f24d000b040 comp rx=0 tx=0).stop 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 msgr2=0x7f24e00c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 0x7f24e00c7fa0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f24d800c4c0 tx=0x7f24d8007f50 comp rx=0 tx=0).stop 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 shutdown_connections 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c0067320 0x7f24c00697d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.387 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e015db10 0x7f24e00c7fa0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e0159890 0x7f24e016c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 --2- 172.21.15.92:0/2399107805 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e0159c60 0x7f24e00c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 >> 172.21.15.92:0/2399107805 conn(0x7f24e0068e20 msgr2=0x7f24e01530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:20.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 shutdown_connections 2023-10-06T03:10:20.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.385+0000 7f24e4e0b700 1 -- 172.21.15.92:0/2399107805 wait complete. 2023-10-06T03:10:20.389 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 116 2023-10-06T03:10:20.399 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:20.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.512+0000 7f2182f91700 1 Processor -- start 2023-10-06T03:10:20.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.512+0000 7f2182f91700 1 -- start start 2023-10-06T03:10:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f2182f91700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f217c069560 0x7f217c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f2182f91700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c069ee0 con 0x7f217c159890 2023-10-06T03:10:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f2182f91700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c15a160 con 0x7f217c159c60 2023-10-06T03:10:20.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f2182f91700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c0d13a0 con 0x7f217c069560 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f2180d2d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f217c159890 0x7f217c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37860/0 (socket says 172.21.15.92:37860) 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f2180d2d700 1 -- 172.21.15.92:0/2257355068 learned_addr learned my addr 172.21.15.92:0/2257355068 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.513+0000 7f218152e700 1 --2- 172.21.15.92:0/2257355068 >> v2:172.21.15.136:3300/0 conn(0x7f217c069560 0x7f217c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f217b7fe700 1 -- 172.21.15.92:0/2257355068 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364313708 0 0) 0x7f217c069ee0 con 0x7f217c159890 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f217b7fe700 1 -- 172.21.15.92:0/2257355068 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2164003570 con 0x7f217c159890 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f218152e700 1 -- 172.21.15.92:0/2257355068 >> v1:172.21.15.136:6789/0 conn(0x7f217c159c60 legacy=0x7f217c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f217b7fe700 1 -- 172.21.15.92:0/2257355068 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2433317290 0 0) 0x7f2164003570 con 0x7f217c159890 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f218152e700 1 -- 172.21.15.92:0/2257355068 >> v1:172.21.15.92:6789/0 conn(0x7f217c159890 legacy=0x7f217c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f218152e700 1 -- 172.21.15.92:0/2257355068 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c15e140 con 0x7f217c069560 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f218152e700 1 --2- 172.21.15.92:0/2257355068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c0699a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f21740098e0 tx=0x7f21740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f80beaa9cd28dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f217b7fe700 1 -- 172.21.15.92:0/2257355068 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0017f0 con 0x7f217c069560 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f217b7fe700 1 -- 172.21.15.92:0/2257355068 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174011420 con 0x7f217c069560 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.514+0000 7f217b7fe700 1 -- 172.21.15.92:0/2257355068 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740117a0 con 0x7f217c069560 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 -- 172.21.15.92:0/2257355068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 msgr2=0x7f217c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 --2- 172.21.15.92:0/2257355068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c0699a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f21740098e0 tx=0x7f21740091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 -- 172.21.15.92:0/2257355068 shutdown_connections 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 --2- 172.21.15.92:0/2257355068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c0699a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 -- 172.21.15.92:0/2257355068 >> 172.21.15.92:0/2257355068 conn(0x7f217c068e20 msgr2=0x7f217c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 -- 172.21.15.92:0/2257355068 shutdown_connections 2023-10-06T03:10:20.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.515+0000 7f2182f91700 1 -- 172.21.15.92:0/2257355068 wait complete. 2023-10-06T03:10:20.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.516+0000 7f2182f91700 1 Processor -- start 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.516+0000 7f2182f91700 1 -- start start 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f217c159890 0x7f217c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 0x7f217c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b4510 con 0x7f217c159c60 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b47f0 con 0x7f217c069560 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b4ad0 con 0x7f217c159890 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 0x7f217c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 0x7f217c0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36026/0 (socket says 172.21.15.92:36026) 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 -- 172.21.15.92:0/1951370976 learned_addr learned my addr 172.21.15.92:0/1951370976 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2180d2d700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f217bfff700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f217c159890 0x7f217c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 -- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f217c159890 msgr2=0x7f217c0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f217c159890 0x7f217c0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 -- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 msgr2=0x7f217c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 -- 172.21.15.92:0/1951370976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174009420 con 0x7f217c159c60 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f218152e700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 0x7f217c0cb330 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f217c06a070 tx=0x7f2174017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174010e40 con 0x7f217c159c60 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217401d430 con 0x7f217c159c60 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c0b4db0 con 0x7f217c159c60 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217401d7b0 con 0x7f217c159c60 2023-10-06T03:10:20.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.517+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c154980 con 0x7f217c159c60 2023-10-06T03:10:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.518+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c0b52d0 con 0x7f217c159c60 2023-10-06T03:10:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.519+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f217401dba0 con 0x7f217c159c60 2023-10-06T03:10:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.519+0000 7f2178ff9700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2168067320 0x7f21680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.519+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f21740870e0 con 0x7f217c159c60 2023-10-06T03:10:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.519+0000 7f2180d2d700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2168067320 0x7f21680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.520+0000 7f2180d2d700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2168067320 0x7f21680697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f216c001920 tx=0x7f216c0017e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.523+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f217c0b52d0 con 0x7f217c159c60 2023-10-06T03:10:20.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.710+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f217c0b52d0 con 0x7f217c159c60 2023-10-06T03:10:20.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.711+0000 7f2178ff9700 1 -- 172.21.15.92:0/1951370976 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18521 (secure 0 0 0) 0x7f217c0b52d0 con 0x7f217c159c60 2023-10-06T03:10:20.712 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:20.713 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":172,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:19.655516+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs.volume_0000000000126469.meta","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000126469"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-10-06T03:10:14.941237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2168067320 msgr2=0x7f21680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2168067320 0x7f21680697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f216c001920 tx=0x7f216c0017e0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 msgr2=0x7f217c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 0x7f217c0cb330 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f217c06a070 tx=0x7f2174017000 comp rx=0 tx=0).stop 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 shutdown_connections 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2168067320 0x7f21680697d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f217c159c60 0x7f217c0cb330 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f217c069560 0x7f217c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 --2- 172.21.15.92:0/1951370976 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f217c159890 0x7f217c0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 >> 172.21.15.92:0/1951370976 conn(0x7f217c068e20 msgr2=0x7f217c155280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 shutdown_connections 2023-10-06T03:10:20.716 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.713+0000 7f2182f91700 1 -- 172.21.15.92:0/1951370976 wait complete. 2023-10-06T03:10:20.727 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:10:20.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:10:20.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:10:20.809 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:10:20.809 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:20.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.922+0000 7f7016ee9700 1 Processor -- start 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.922+0000 7f7016ee9700 1 -- start start 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7016ee9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7010159420 0x7f70100b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7016ee9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70101597f0 0x7f70100b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7016ee9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100b11d0 con 0x7f701015d230 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7016ee9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70100b14b0 con 0x7f7010159420 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7016ee9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70100b1790 con 0x7f70101597f0 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7015ee7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7010159420 0x7f70100b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f70166e8700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f701015d230 0x7f70100b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37876/0 (socket says 172.21.15.92:37876) 2023-10-06T03:10:20.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f70166e8700 1 -- 172.21.15.92:0/2832129461 learned_addr learned my addr 172.21.15.92:0/2832129461 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7015ee7700 1 -- 172.21.15.92:0/2832129461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70101597f0 msgr2=0x7f70100b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7015ee7700 1 --2- 172.21.15.92:0/2832129461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70101597f0 0x7f70100b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7015ee7700 1 -- 172.21.15.92:0/2832129461 >> v1:172.21.15.92:6789/0 conn(0x7f701015d230 legacy=0x7f70100b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7014ee5700 1 -- 172.21.15.92:0/2832129461 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320132235 0 0) 0x7f70100b11d0 con 0x7f701015d230 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.923+0000 7f7015ee7700 1 -- 172.21.15.92:0/2832129461 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100b1a70 con 0x7f7010159420 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7015ee7700 1 --2- 172.21.15.92:0/2832129461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7010159420 0x7f70100b5f40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e60 tx=0x7f700c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6993eb5536307bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7014ee5700 1 -- 172.21.15.92:0/2832129461 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00e070 con 0x7f7010159420 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7014ee5700 1 -- 172.21.15.92:0/2832129461 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00bc30 con 0x7f7010159420 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7014ee5700 1 -- 172.21.15.92:0/2832129461 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c012430 con 0x7f7010159420 2023-10-06T03:10:20.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 -- 172.21.15.92:0/2832129461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7010159420 msgr2=0x7f70100b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 --2- 172.21.15.92:0/2832129461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7010159420 0x7f70100b5f40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e60 tx=0x7f700c009780 comp rx=0 tx=0).stop 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 -- 172.21.15.92:0/2832129461 shutdown_connections 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 --2- 172.21.15.92:0/2832129461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7010159420 0x7f70100b5f40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 --2- 172.21.15.92:0/2832129461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70101597f0 0x7f70100b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 -- 172.21.15.92:0/2832129461 >> 172.21.15.92:0/2832129461 conn(0x7f70100d0660 msgr2=0x7f70100d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 -- 172.21.15.92:0/2832129461 shutdown_connections 2023-10-06T03:10:20.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.924+0000 7f7016ee9700 1 -- 172.21.15.92:0/2832129461 wait complete. 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.925+0000 7f7016ee9700 1 Processor -- start 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.925+0000 7f7016ee9700 1 -- start start 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7016ee9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010159420 0x7f7010157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7016ee9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 0x7f701015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7016ee9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701015d230 0x7f70100c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7016ee9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701015b120 con 0x7f70101597f0 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7016ee9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701015a230 con 0x7f7010159420 2023-10-06T03:10:20.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7016ee9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701015a510 con 0x7f701015d230 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 0x7f701015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 0x7f701015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36034/0 (socket says 172.21.15.92:36034) 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 -- 172.21.15.92:0/3409588864 learned_addr learned my addr 172.21.15.92:0/3409588864 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70166e8700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701015d230 0x7f70100c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f7015ee7700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010159420 0x7f7010157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 -- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701015d230 msgr2=0x7f70100c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701015d230 0x7f70100c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 -- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010159420 msgr2=0x7f7010157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:20.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.926+0000 7f70156e6700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010159420 0x7f7010157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f70156e6700 1 -- 172.21.15.92:0/3409588864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c009a20 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f70156e6700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 0x7f701015b5b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f700000b810 tx=0x7f7000009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000f5f0 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700000f770 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701015a7f0 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000fb30 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.927+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701015adb0 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.928+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701004c3c0 con 0x7f70101597f0 2023-10-06T03:10:20.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.928+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7000010480 con 0x7f70101597f0 2023-10-06T03:10:20.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.929+0000 7f7005ffb700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ff4067320 0x7f6ff40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:20.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.929+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f7000084f10 con 0x7f70101597f0 2023-10-06T03:10:20.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.929+0000 7f7015ee7700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ff4067320 0x7f6ff40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:20.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.929+0000 7f7015ee7700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ff4067320 0x7f6ff40697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e60 tx=0x7f700c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:20.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:20.933+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70000518e0 con 0x7f70101597f0 2023-10-06T03:10:21.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.159+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70100b1d50 con 0x7f70101597f0 2023-10-06T03:10:21.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.159+0000 7f7005ffb700 1 -- 172.21.15.92:0/3409588864 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f7000054f00 con 0x7f70101597f0 2023-10-06T03:10:21.161 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:21.161 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:18.647577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000000126469","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.161+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ff4067320 msgr2=0x7f6ff40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.161+0000 7f7016ee9700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ff4067320 0x7f6ff40697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f700c009e60 tx=0x7f700c019000 comp rx=0 tx=0).stop 2023-10-06T03:10:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.161+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 msgr2=0x7f701015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.161+0000 7f7016ee9700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 0x7f701015b5b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f700000b810 tx=0x7f7000009250 comp rx=0 tx=0).stop 2023-10-06T03:10:21.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 shutdown_connections 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6ff4067320 0x7f6ff40697d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701015d230 0x7f70100c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7010159420 0x7f7010157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 --2- 172.21.15.92:0/3409588864 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f70101597f0 0x7f701015b5b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 >> 172.21.15.92:0/3409588864 conn(0x7f70100d0660 msgr2=0x7f7010069670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 shutdown_connections 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.162+0000 7f7016ee9700 1 -- 172.21.15.92:0/3409588864 wait complete. 2023-10-06T03:10:21.168 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 116 2023-10-06T03:10:21.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:21.289 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.288+0000 7f269899f700 1 Processor -- start 2023-10-06T03:10:21.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.288+0000 7f269899f700 1 -- start start 2023-10-06T03:10:21.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269899f700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2694159c60 0x7f269415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269899f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269415a160 con 0x7f269415db10 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269899f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26940d13a0 con 0x7f2694159890 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269899f700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269415df50 con 0x7f2694159c60 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f2692d9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f269415db10 0x7f2694069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37890/0 (socket says 172.21.15.92:37890) 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f2692d9d700 1 -- 172.21.15.92:0/2806122647 learned_addr learned my addr 172.21.15.92:0/2806122647 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f2691d9b700 1 --2- 172.21.15.92:0/2806122647 >> v2:172.21.15.136:3300/0 conn(0x7f2694159c60 0x7f269415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152621606 0 0) 0x7f269415a160 con 0x7f269415db10 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f267c003570 con 0x7f269415db10 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665840902 0 0) 0x7f26940d13a0 con 0x7f2694159890 2023-10-06T03:10:21.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269415a160 con 0x7f2694159890 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2469199170 0 0) 0x7f267c003570 con 0x7f269415db10 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 >> v2:172.21.15.136:3300/0 conn(0x7f2694159c60 msgr2=0x7f269415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 --2- 172.21.15.92:0/2806122647 >> v2:172.21.15.136:3300/0 conn(0x7f2694159c60 0x7f269415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 >> v1:172.21.15.136:6789/0 conn(0x7f2694159890 legacy=0x7f269415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269415e140 con 0x7f269415db10 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f268c0017f0 con 0x7f269415db10 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 >> v1:172.21.15.92:6789/0 conn(0x7f269415db10 legacy=0x7f2694069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c003c40 0x7f267c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c005210 0x7f267c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c00c130 con 0x7f267c005210 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c00c270 con 0x7f267c003c40 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.289+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c00c400 con 0x7f267c004810 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c005210 0x7f267c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f269259c700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c003c40 0x7f267c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2691d9b700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 -- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f267c004810 msgr2=0x7f267c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 -- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c003c40 msgr2=0x7f267c004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:21.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c003c40 0x7f267c004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 -- 172.21.15.92:0/2806122647 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c003570 con 0x7f267c005210 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2691d9b700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f267c004810 0x7f267c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 -- 172.21.15.92:0/2806122647 reap_dead start 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f2692d9d700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c005210 0x7f267c00ba60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f268c00a830 tx=0x7f268c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7449663a2c88fa4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0071a0 con 0x7f267c005210 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c007300 con 0x7f267c005210 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.290+0000 7f269159a700 1 -- 172.21.15.92:0/2806122647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0076c0 con 0x7f267c005210 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 -- 172.21.15.92:0/2806122647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c005210 msgr2=0x7f267c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c005210 0x7f267c00ba60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f268c00a830 tx=0x7f268c0020c0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 -- 172.21.15.92:0/2806122647 shutdown_connections 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 --2- 172.21.15.92:0/2806122647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f267c005210 0x7f267c00ba60 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 -- 172.21.15.92:0/2806122647 >> 172.21.15.92:0/2806122647 conn(0x7f2694068e20 msgr2=0x7f26940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 -- 172.21.15.92:0/2806122647 shutdown_connections 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.291+0000 7f269899f700 1 -- 172.21.15.92:0/2806122647 wait complete. 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.292+0000 7f269899f700 1 Processor -- start 2023-10-06T03:10:21.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.292+0000 7f269899f700 1 -- start start 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c005210 0x7f26940b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 0x7f2694151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940b4210 0x7f2694151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26940ce3a0 con 0x7f26940b3da0 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26940cdf20 con 0x7f267c005210 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26940c78a0 con 0x7f26940b4210 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 0x7f2694151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 0x7f2694151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36046/0 (socket says 172.21.15.92:36046) 2023-10-06T03:10:21.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 -- 172.21.15.92:0/631368464 learned_addr learned my addr 172.21.15.92:0/631368464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269259c700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c005210 0x7f26940b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 -- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940b4210 msgr2=0x7f2694151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940b4210 0x7f2694151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 -- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c005210 msgr2=0x7f26940b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c005210 0x7f26940b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 -- 172.21.15.92:0/631368464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940ca3d0 con 0x7f26940b3da0 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f2691d9b700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 0x7f2694151500 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2688009fd0 tx=0x7f2688008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688015740 con 0x7f26940b3da0 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26880158a0 con 0x7f26940b3da0 2023-10-06T03:10:21.302 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688015c40 con 0x7f26940b3da0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26940ca6b0 con 0x7f26940b3da0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.293+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26940ce5f0 con 0x7f26940b3da0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.294+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269404c3c0 con 0x7f26940b3da0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.295+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2688013410 con 0x7f26940b3da0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.295+0000 7f26827fc700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2674067320 0x7f26740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.295+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f2688085690 con 0x7f26940b3da0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.295+0000 7f269259c700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2674067320 0x7f26740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.296+0000 7f269259c700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2674067320 0x7f26740697d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2684007e40 tx=0x7f2684009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.299+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2688051fe0 con 0x7f26940b3da0 2023-10-06T03:10:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.483+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f269406a5b0 con 0x7f26940b3da0 2023-10-06T03:10:21.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.484+0000 7f26827fc700 1 -- 172.21.15.92:0/631368464 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18521 (secure 0 0 0) 0x7f2688055600 con 0x7f26940b3da0 2023-10-06T03:10:21.486 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:21.486 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":172,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:19.655516+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs.volume_0000000000126469.meta","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000126469"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-10-06T03:10:14.941237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2674067320 msgr2=0x7f26740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2674067320 0x7f26740697d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2684007e40 tx=0x7f2684009040 comp rx=0 tx=0).stop 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 msgr2=0x7f2694151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 0x7f2694151500 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2688009fd0 tx=0x7f2688008040 comp rx=0 tx=0).stop 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 shutdown_connections 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2674067320 0x7f26740697d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940b4210 0x7f2694151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f267c005210 0x7f26940b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 --2- 172.21.15.92:0/631368464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f26940b3da0 0x7f2694151500 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 >> 172.21.15.92:0/631368464 conn(0x7f2694068e20 msgr2=0x7f26940d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 shutdown_connections 2023-10-06T03:10:21.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.486+0000 7f269899f700 1 -- 172.21.15.92:0/631368464 wait complete. 2023-10-06T03:10:21.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs.volume_0000000000126469.meta', 'create_time': '2023-10-06T03:09:57.571177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000126469'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:09:58.909445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000126469'}}}, {'pool': 31, 'pool_name': 'another-data-pool', 'create_time': '2023-10-06T03:10:14.941237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000126469'}}}] 2023-10-06T03:10:21.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:10:21.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.619+0000 7f544e6a7700 1 Processor -- start 2023-10-06T03:10:21.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.619+0000 7f544e6a7700 1 -- start start 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544e6a7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5448159890 0x7f544815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544e6a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159c60 0x7f544815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544e6a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544815a160 con 0x7f544815db10 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544e6a7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54480d13a0 con 0x7f5448159890 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544e6a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544815e140 con 0x7f5448159c60 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f5447fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5448159890 0x7f544815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544cc44700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f544815db10 0x7f5448069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37892/0 (socket says 172.21.15.92:37892) 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f544cc44700 1 -- 172.21.15.92:0/2613305943 learned_addr learned my addr 172.21.15.92:0/2613305943 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.620+0000 7f54477fe700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159c60 0x7f544815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5447fff700 1 -- 172.21.15.92:0/2613305943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159c60 msgr2=0x7f544815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:21.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5447fff700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159c60 0x7f544815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5447fff700 1 -- 172.21.15.92:0/2613305943 >> v1:172.21.15.92:6789/0 conn(0x7f544815db10 legacy=0x7f5448069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5446ffd700 1 -- 172.21.15.92:0/2613305943 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603915883 0 0) 0x7f544815a160 con 0x7f544815db10 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5447fff700 1 -- 172.21.15.92:0/2613305943 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544815a3d0 con 0x7f5448159890 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f54477fe700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159c60 0x7f544815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5447fff700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159890 0x7f544815abd0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f54380098e0 tx=0x7f54380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfba579a86b519c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5446ffd700 1 -- 172.21.15.92:0/2613305943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438009e80 con 0x7f5448159890 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5446ffd700 1 -- 172.21.15.92:0/2613305943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543800b860 con 0x7f5448159890 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f5446ffd700 1 -- 172.21.15.92:0/2613305943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543800bbe0 con 0x7f5448159890 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f544e6a7700 1 -- 172.21.15.92:0/2613305943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159890 msgr2=0x7f544815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f544e6a7700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159890 0x7f544815abd0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f54380098e0 tx=0x7f54380091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f544e6a7700 1 -- 172.21.15.92:0/2613305943 shutdown_connections 2023-10-06T03:10:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f544e6a7700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159890 0x7f544815abd0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f544e6a7700 1 --2- 172.21.15.92:0/2613305943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159c60 0x7f544815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.621+0000 7f544e6a7700 1 -- 172.21.15.92:0/2613305943 >> 172.21.15.92:0/2613305943 conn(0x7f5448068e20 msgr2=0x7f54480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.622+0000 7f544e6a7700 1 -- 172.21.15.92:0/2613305943 shutdown_connections 2023-10-06T03:10:21.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.622+0000 7f544e6a7700 1 -- 172.21.15.92:0/2613305943 wait complete. 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 Processor -- start 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 -- start start 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159890 0x7f54480b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 0x7f544815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f544815db10 0x7f544815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480ce880 con 0x7f5448159c60 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544815c590 con 0x7f544815db10 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.623+0000 7f544e6a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54480ceb40 con 0x7f5448159890 2023-10-06T03:10:21.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 0x7f544815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 0x7f544815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36062/0 (socket says 172.21.15.92:36062) 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 -- 172.21.15.92:0/3170191875 learned_addr learned my addr 172.21.15.92:0/3170191875 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f5447fff700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159890 0x7f54480b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f544cc44700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f544815db10 0x7f544815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 -- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159890 msgr2=0x7f54480b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159890 0x7f54480b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 -- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f544815db10 msgr2=0x7f544815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f544815db10 0x7f544815b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 -- 172.21.15.92:0/3170191875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438009420 con 0x7f5448159c60 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f544cc44700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f544815db10 0x7f544815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f54477fe700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 0x7f544815c980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f543c00e8d0 tx=0x7f543c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00ee70 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c00c5e0 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00c9a0 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544815d5b0 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.624+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544815bfd0 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.625+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544804c3c0 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.625+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f543c00cd50 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.626+0000 7f542ffff700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5434067320 0x7f54340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.626+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f543c084e90 con 0x7f5448159c60 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.626+0000 7f5447fff700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5434067320 0x7f54340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.631 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.626+0000 7f5447fff700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5434067320 0x7f54340697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f54380091e0 tx=0x7f5438017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.630+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f543c055160 con 0x7f5448159c60 2023-10-06T03:10:21.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.814+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f544806a5b0 con 0x7f5448159c60 2023-10-06T03:10:21.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.814+0000 7f542ffff700 1 -- 172.21.15.92:0/3170191875 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7f543c054cf0 con 0x7f5448159c60 2023-10-06T03:10:21.816 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5434067320 msgr2=0x7f54340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5434067320 0x7f54340697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f54380091e0 tx=0x7f5438017000 comp rx=0 tx=0).stop 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 msgr2=0x7f544815c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 0x7f544815c980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f543c00e8d0 tx=0x7f543c008040 comp rx=0 tx=0).stop 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 shutdown_connections 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5434067320 0x7f54340697d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f544815db10 0x7f544815b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5448159890 0x7f54480b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 --2- 172.21.15.92:0/3170191875 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5448159c60 0x7f544815c980 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 >> 172.21.15.92:0/3170191875 conn(0x7f5448068e20 msgr2=0x7f54480b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:21.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 shutdown_connections 2023-10-06T03:10:21.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.817+0000 7f544e6a7700 1 -- 172.21.15.92:0/3170191875 wait complete. 2023-10-06T03:10:21.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:10:21.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.946+0000 7fd26ec7c700 1 Processor -- start 2023-10-06T03:10:21.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.946+0000 7fd26ec7c700 1 -- start start 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd26ec7c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd268159890 0x7fd26815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd26ec7c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26815a160 con 0x7fd268159c60 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd26ec7c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2680d13a0 con 0x7fd268069560 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd26ec7c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26815e140 con 0x7fd268159890 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd267fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd268159c60 0x7fd26815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37898/0 (socket says 172.21.15.92:37898) 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd267fff700 1 -- 172.21.15.92:0/1963987008 learned_addr learned my addr 172.21.15.92:0/1963987008 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:21.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd26ca18700 1 --2- 172.21.15.92:0/1963987008 >> v2:172.21.15.136:3300/0 conn(0x7fd268159890 0x7fd26815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd2677fe700 1 -- 172.21.15.92:0/1963987008 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938807519 0 0) 0x7fd26815a160 con 0x7fd268159c60 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd2677fe700 1 -- 172.21.15.92:0/1963987008 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd254003570 con 0x7fd268159c60 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.947+0000 7fd26ca18700 1 -- 172.21.15.92:0/1963987008 >> v1:172.21.15.136:6789/0 conn(0x7fd268069560 legacy=0x7fd2680ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ca18700 1 -- 172.21.15.92:0/1963987008 >> v1:172.21.15.92:6789/0 conn(0x7fd268159c60 legacy=0x7fd26815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd2677fe700 1 -- 172.21.15.92:0/1963987008 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318098864 0 0) 0x7fd254003570 con 0x7fd268159c60 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ca18700 1 -- 172.21.15.92:0/1963987008 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26815a3d0 con 0x7fd268159890 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ca18700 1 --2- 172.21.15.92:0/1963987008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159890 0x7fd26815abd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd258009e60 tx=0x7fd258009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604daf491728b121 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd2677fe700 1 -- 172.21.15.92:0/1963987008 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800e070 con 0x7fd268159890 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd2677fe700 1 -- 172.21.15.92:0/1963987008 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25800b850 con 0x7fd268159890 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd2677fe700 1 -- 172.21.15.92:0/1963987008 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250001470 con 0x7fd268159890 2023-10-06T03:10:21.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ec7c700 1 -- 172.21.15.92:0/1963987008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159890 msgr2=0x7fd26815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/1963987008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159890 0x7fd26815abd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd258009e60 tx=0x7fd258009780 comp rx=0 tx=0).stop 2023-10-06T03:10:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ec7c700 1 -- 172.21.15.92:0/1963987008 shutdown_connections 2023-10-06T03:10:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/1963987008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159890 0x7fd26815abd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ec7c700 1 -- 172.21.15.92:0/1963987008 >> 172.21.15.92:0/1963987008 conn(0x7fd268068e20 msgr2=0x7fd2680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.948+0000 7fd26ec7c700 1 -- 172.21.15.92:0/1963987008 shutdown_connections 2023-10-06T03:10:21.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.949+0000 7fd26ec7c700 1 -- 172.21.15.92:0/1963987008 wait complete. 2023-10-06T03:10:21.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 Processor -- start 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 -- start start 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd268069560 0x7fd26814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159c60 0x7fd26815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 0x7fd2680b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680b38d0 con 0x7fd26815d070 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26815d510 con 0x7fd268159c60 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.950+0000 7fd26ec7c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2680cd6b0 con 0x7fd268069560 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 0x7fd2680b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 0x7fd2680b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36066/0 (socket says 172.21.15.92:36066) 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 -- 172.21.15.92:0/3125638101 learned_addr learned my addr 172.21.15.92:0/3125638101 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd267fff700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159c60 0x7fd26815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26ca18700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd268069560 0x7fd26814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 -- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd268069560 msgr2=0x7fd26814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd268069560 0x7fd26814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 -- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159c60 msgr2=0x7fd26815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159c60 0x7fd26815cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 -- 172.21.15.92:0/3125638101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258009a20 con 0x7fd26815d070 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26d219700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 0x7fd2680b4a00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd26000c560 tx=0x7fd260007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260011590 con 0x7fd26815d070 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2600116f0 con 0x7fd26815d070 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26814d820 con 0x7fd26815d070 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd260011ab0 con 0x7fd26815d070 2023-10-06T03:10:21.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.951+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26815d7c0 con 0x7fd26815d070 2023-10-06T03:10:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.952+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26815ddf0 con 0x7fd26815d070 2023-10-06T03:10:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.953+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd26001b3b0 con 0x7fd26815d070 2023-10-06T03:10:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.953+0000 7fd264ff9700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd248067320 0x7fd2480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.953+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fd260084430 con 0x7fd26815d070 2023-10-06T03:10:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.953+0000 7fd26ca18700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd248067320 0x7fd2480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:21.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.953+0000 7fd26ca18700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd248067320 0x7fd2480697d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd268069370 tx=0x7fd258017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:21.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:21.957+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd260050d80 con 0x7fd26815d070 2023-10-06T03:10:22.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.140+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd26804c3c0 con 0x7fd26815d070 2023-10-06T03:10:22.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.140+0000 7fd264ff9700 1 -- 172.21.15.92:0/3125638101 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v172) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2600170a0 con 0x7fd26815d070 2023-10-06T03:10:22.142 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd248067320 msgr2=0x7fd2480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd248067320 0x7fd2480697d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd268069370 tx=0x7fd258017000 comp rx=0 tx=0).stop 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 msgr2=0x7fd2680b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 0x7fd2680b4a00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd26000c560 tx=0x7fd260007ef0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 shutdown_connections 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd248067320 0x7fd2480697d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd26815d070 0x7fd2680b4a00 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd268069560 0x7fd26814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 --2- 172.21.15.92:0/3125638101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd268159c60 0x7fd26815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 >> 172.21.15.92:0/3125638101 conn(0x7fd268068e20 msgr2=0x7fd2680b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 shutdown_connections 2023-10-06T03:10:22.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.142+0000 7fd26ec7c700 1 -- 172.21.15.92:0/3125638101 wait complete. 2023-10-06T03:10:22.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000126469.meta pg_num 2023-10-06T03:10:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.271+0000 7fe4dee16700 1 Processor -- start 2023-10-06T03:10:22.273 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.271+0000 7fe4dee16700 1 -- start start 2023-10-06T03:10:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4dee16700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4d8159890 0x7fe4d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4dee16700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d815a160 con 0x7fe4d8159c60 2023-10-06T03:10:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4dee16700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d80d13a0 con 0x7fe4d8069560 2023-10-06T03:10:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4dee16700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d815e140 con 0x7fe4d8159890 2023-10-06T03:10:22.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4dcbb2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe4d8159890 0x7fe4d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4cffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4d8159c60 0x7fe4d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37902/0 (socket says 172.21.15.92:37902) 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4cffff700 1 -- 172.21.15.92:0/4267967977 learned_addr learned my addr 172.21.15.92:0/4267967977 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4dcbb2700 1 -- 172.21.15.92:0/4267967977 >> v1:172.21.15.136:6789/0 conn(0x7fe4d8069560 legacy=0x7fe4d80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dcbb2700 1 -- 172.21.15.92:0/4267967977 >> v1:172.21.15.92:6789/0 conn(0x7fe4d8159c60 legacy=0x7fe4d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dcbb2700 1 -- 172.21.15.92:0/4267967977 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d815a3d0 con 0x7fe4d8159890 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.272+0000 7fe4cf7fe700 1 -- 172.21.15.92:0/4267967977 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570554830 0 0) 0x7fe4d815a160 con 0x7fe4d8159c60 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dcbb2700 1 --2- 172.21.15.92:0/4267967977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d815abd0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40098e0 tx=0x7fe4d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657fa882aa232d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4cf7fe700 1 -- 172.21.15.92:0/4267967977 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4009e80 con 0x7fe4d8159890 2023-10-06T03:10:22.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4cf7fe700 1 -- 172.21.15.92:0/4267967977 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d400b820 con 0x7fe4d8159890 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4cf7fe700 1 -- 172.21.15.92:0/4267967977 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400bba0 con 0x7fe4d8159890 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 -- 172.21.15.92:0/4267967977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 msgr2=0x7fe4d815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 --2- 172.21.15.92:0/4267967977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d815abd0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40098e0 tx=0x7fe4d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 -- 172.21.15.92:0/4267967977 shutdown_connections 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 --2- 172.21.15.92:0/4267967977 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d815abd0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 -- 172.21.15.92:0/4267967977 >> 172.21.15.92:0/4267967977 conn(0x7fe4d8068e20 msgr2=0x7fe4d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 -- 172.21.15.92:0/4267967977 shutdown_connections 2023-10-06T03:10:22.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.273+0000 7fe4dee16700 1 -- 172.21.15.92:0/4267967977 wait complete. 2023-10-06T03:10:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.274+0000 7fe4dee16700 1 Processor -- start 2023-10-06T03:10:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 -- start start 2023-10-06T03:10:22.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4d8069560 0x7fe4d80c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d80b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d8159c60 0x7fe4d80c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d815c740 con 0x7fe4d8159890 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d815ca20 con 0x7fe4d8159c60 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4dee16700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d815cd00 con 0x7fe4d8069560 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.275+0000 7fe4cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d80b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d80b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36090/0 (socket says 172.21.15.92:36090) 2023-10-06T03:10:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 -- 172.21.15.92:0/2340610961 learned_addr learned my addr 172.21.15.92:0/2340610961 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4dcbb2700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4d8069560 0x7fe4d80c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4dd3b3700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d8159c60 0x7fe4d80c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 -- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4d8069560 msgr2=0x7fe4d80c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.283 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4d8069560 0x7fe4d80c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 -- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d8159c60 msgr2=0x7fe4d80c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d8159c60 0x7fe4d80c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.284 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 -- 172.21.15.92:0/2340610961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d4009420 con 0x7fe4d8159890 2023-10-06T03:10:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4cffff700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d80b5e50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c7e0 tx=0x7fe4c8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8005b30 con 0x7fe4d8159890 2023-10-06T03:10:22.285 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8005c90 con 0x7fe4d8159890 2023-10-06T03:10:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d815cfe0 con 0x7fe4d8159890 2023-10-06T03:10:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8007300 con 0x7fe4d8159890 2023-10-06T03:10:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.276+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d815d5a0 con 0x7fe4d8159890 2023-10-06T03:10:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.277+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d815d760 con 0x7fe4d8159890 2023-10-06T03:10:22.286 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.277+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe4c801b3b0 con 0x7fe4d8159890 2023-10-06T03:10:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.278+0000 7fe4ccff9700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4c4067320 0x7fe4c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.278+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fe4c80844d0 con 0x7fe4d8159890 2023-10-06T03:10:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.278+0000 7fe4dcbb2700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4c4067320 0x7fe4c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.278+0000 7fe4dcbb2700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4c4067320 0x7fe4c40697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40030c0 tx=0x7fe4d4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.287 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.282+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4c8050e20 con 0x7fe4d8159890 2023-10-06T03:10:22.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.463+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000126469.meta", "var": "pg_num"} v 0) v1 -- 0x7fe4d804c3c0 con 0x7fe4d8159890 2023-10-06T03:10:22.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.463+0000 7fe4ccff9700 1 -- 172.21.15.92:0/2340610961 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000126469.meta", "var": "pg_num"}]=0 v172) v1 ==== 124+0+11 (secure 0 0 0) 0x7fe4c8017020 con 0x7fe4d8159890 2023-10-06T03:10:22.466 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:10:22.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4c4067320 msgr2=0x7fe4c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4c4067320 0x7fe4c40697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40030c0 tx=0x7fe4d4016000 comp rx=0 tx=0).stop 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 msgr2=0x7fe4d80b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d80b5e50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c7e0 tx=0x7fe4c8007df0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 shutdown_connections 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4c4067320 0x7fe4c40697d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4d8159c60 0x7fe4d80c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4d8069560 0x7fe4d80c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 --2- 172.21.15.92:0/2340610961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d8159890 0x7fe4d80b5e50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 >> 172.21.15.92:0/2340610961 conn(0x7fe4d8068e20 msgr2=0x7fe4d8152e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:22.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 shutdown_connections 2023-10-06T03:10:22.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.466+0000 7fe4dee16700 1 -- 172.21.15.92:0/2340610961 wait complete. 2023-10-06T03:10:22.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:10:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.597+0000 7f1e14755700 1 Processor -- start 2023-10-06T03:10:22.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.597+0000 7f1e14755700 1 -- start start 2023-10-06T03:10:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.598+0000 7f1e14755700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1e0c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.598+0000 7f1e14755700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c15a160 con 0x7f1e0c159c60 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.598+0000 7f1e14755700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0d13a0 con 0x7f1e0c069560 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.598+0000 7f1e14755700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15e140 con 0x7f1e0c159890 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.598+0000 7f1e11cf0700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1e0c159c60 0x7f1e0c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37912/0 (socket says 172.21.15.92:37912) 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.598+0000 7f1e11cf0700 1 -- 172.21.15.92:0/1090143869 learned_addr learned my addr 172.21.15.92:0/1090143869 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e124f1700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1e0c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530038125 0 0) 0x7f1e0c15a160 con 0x7f1e0c159c60 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df8003570 con 0x7f1e0c159c60 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3150651588 0 0) 0x7f1df8003570 con 0x7f1e0c159c60 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 msgr2=0x7f1e0c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1e0c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 >> v1:172.21.15.136:6789/0 conn(0x7f1e0c069560 legacy=0x7f1e0c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c15a3d0 con 0x7f1e0c159c60 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f1dfc001470 con 0x7f1e0c159c60 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 >> v1:172.21.15.92:6789/0 conn(0x7f1e0c159c60 legacy=0x7f1e0c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df8003a90 0x7f1df8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1df8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df8004d50 0x7f1df80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15e140 con 0x7f1df8004d50 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df800bb10 con 0x7f1df8003a90 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df800bcd0 con 0x7f1e0c159890 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e124f1700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df8003a90 0x7f1df8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df8004d50 0x7f1df80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e11cf0700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1df8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e12cf2700 1 -- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 msgr2=0x7f1df8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1df8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e12cf2700 1 -- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df8003a90 msgr2=0x7f1df8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df8003a90 0x7f1df8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e12cf2700 1 -- 172.21.15.92:0/1090143869 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8003570 con 0x7f1df8004d50 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e11cf0700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e0c159890 0x7f1df8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.599+0000 7f1e124f1700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df8003a90 0x7f1df8003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df8004d50 0x7f1df80051c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f1e040019a0 tx=0x7f1e04007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e9c8f19f694ec8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400c5d0 con 0x7f1df8004d50 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400c750 con 0x7f1df8004d50 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e114ef700 1 -- 172.21.15.92:0/1090143869 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400cb10 con 0x7f1df8004d50 2023-10-06T03:10:22.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e14755700 1 -- 172.21.15.92:0/1090143869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df8004d50 msgr2=0x7f1df80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e14755700 1 --2- 172.21.15.92:0/1090143869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1df8004d50 0x7f1df80051c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f1e040019a0 tx=0x7f1e04007d10 comp rx=0 tx=0).stop 2023-10-06T03:10:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e12cf2700 1 -- 172.21.15.92:0/1090143869 reap_dead start 2023-10-06T03:10:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e14755700 1 -- 172.21.15.92:0/1090143869 shutdown_connections 2023-10-06T03:10:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e14755700 1 -- 172.21.15.92:0/1090143869 >> 172.21.15.92:0/1090143869 conn(0x7f1e0c068e20 msgr2=0x7f1e0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e14755700 1 -- 172.21.15.92:0/1090143869 shutdown_connections 2023-10-06T03:10:22.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.600+0000 7f1e14755700 1 -- 172.21.15.92:0/1090143869 wait complete. 2023-10-06T03:10:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.601+0000 7f1e14755700 1 Processor -- start 2023-10-06T03:10:22.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.601+0000 7f1e14755700 1 -- start start 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e14755700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df8004d50 0x7f1e0c14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e14755700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c0cd6c0 0x7f1e0c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e14755700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 0x7f1e0c0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e14755700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0ce070 con 0x7f1e0c0b3740 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e14755700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15ed90 con 0x7f1e0c0cd6c0 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e14755700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15fbf0 con 0x7f1df8004d50 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e12cf2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 0x7f1e0c0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e12cf2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 0x7f1e0c0b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36112/0 (socket says 172.21.15.92:36112) 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e12cf2700 1 -- 172.21.15.92:0/135348493 learned_addr learned my addr 172.21.15.92:0/135348493 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:22.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.602+0000 7f1e124f1700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df8004d50 0x7f1e0c14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e11cf0700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c0cd6c0 0x7f1e0c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e12cf2700 1 -- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df8004d50 msgr2=0x7f1e0c14f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df8004d50 0x7f1e0c14f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e12cf2700 1 -- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c0cd6c0 msgr2=0x7f1e0c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c0cd6c0 0x7f1e0c0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e12cf2700 1 -- 172.21.15.92:0/135348493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c15efe0 con 0x7f1e0c0b3740 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e11cf0700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c0cd6c0 0x7f1e0c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e12cf2700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 0x7f1e0c0b3bb0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1e04003000 tx=0x7f1e04015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04023070 con 0x7f1e0c0b3740 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400c5d0 con 0x7f1e0c0b3740 2023-10-06T03:10:22.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c15f2c0 con 0x7f1e0c0b3740 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04007af0 con 0x7f1e0c0b3740 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.603+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c15f880 con 0x7f1e0c0b3740 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.604+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c04c3c0 con 0x7f1e0c0b3740 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.604+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e04019070 con 0x7f1e0c0b3740 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.605+0000 7f1e027fc700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1df0067320 0x7f1df00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.605+0000 7f1e124f1700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1df0067320 0x7f1df00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.605+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f1e04085f60 con 0x7f1e0c0b3740 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.605+0000 7f1e124f1700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1df0067320 0x7f1df00697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1e080094a0 tx=0x7f1e0800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.609+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e04052970 con 0x7f1e0c0b3740 2023-10-06T03:10:22.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.791+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e0c0b2f40 con 0x7f1e0c0b3740 2023-10-06T03:10:22.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.791+0000 7f1e027fc700 1 -- 172.21.15.92:0/135348493 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e04017030 con 0x7f1e0c0b3740 2023-10-06T03:10:22.793 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:10:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1df0067320 msgr2=0x7f1df00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1df0067320 0x7f1df00697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1e080094a0 tx=0x7f1e0800b040 comp rx=0 tx=0).stop 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 msgr2=0x7f1e0c0b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 0x7f1e0c0b3bb0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1e04003000 tx=0x7f1e04015000 comp rx=0 tx=0).stop 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 shutdown_connections 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1df0067320 0x7f1df00697d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e0c0b3740 0x7f1e0c0b3bb0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df8004d50 0x7f1e0c14f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 --2- 172.21.15.92:0/135348493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c0cd6c0 0x7f1e0c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.793+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 >> 172.21.15.92:0/135348493 conn(0x7f1e0c068e20 msgr2=0x7f1e0c0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.794+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 shutdown_connections 2023-10-06T03:10:22.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.794+0000 7f1e14755700 1 -- 172.21.15.92:0/135348493 wait complete. 2023-10-06T03:10:22.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get another-data-pool pg_num 2023-10-06T03:10:22.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.923+0000 7f67ebbca700 1 Processor -- start 2023-10-06T03:10:22.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.923+0000 7f67ebbca700 1 -- start start 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.924+0000 7f67ebbca700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 0x7f67e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.924+0000 7f67ebbca700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e415a160 con 0x7f67e4069560 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.924+0000 7f67ebbca700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40d13a0 con 0x7f67e4159c60 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.924+0000 7f67ebbca700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e415e140 con 0x7f67e4159890 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.924+0000 7f67ea167700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f67e4069560 0x7f67e40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37916/0 (socket says 172.21.15.92:37916) 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.924+0000 7f67ea167700 1 -- 172.21.15.92:0/466038361 learned_addr learned my addr 172.21.15.92:0/466038361 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 0x7f67e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596606079 0 0) 0x7f67e415a160 con 0x7f67e4069560 2023-10-06T03:10:22.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d0003570 con 0x7f67e4069560 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308838930 0 0) 0x7f67e40d13a0 con 0x7f67e4159c60 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e415a160 con 0x7f67e4159c60 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926498383 0 0) 0x7f67d0003570 con 0x7f67e4069560 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 msgr2=0x7f67e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 0x7f67e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 >> v1:172.21.15.136:6789/0 conn(0x7f67e4159c60 legacy=0x7f67e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e415a3d0 con 0x7f67e4069560 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f67dc0017f0 con 0x7f67e4069560 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 >> v1:172.21.15.92:6789/0 conn(0x7f67e4069560 legacy=0x7f67e40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67d0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 0x7f67d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67d0004d50 0x7f67d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e415e140 con 0x7f67d0003c20 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d000bb40 con 0x7f67d0004d50 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d000bd00 con 0x7f67e4159890 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67d0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67ea167700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67d0004d50 0x7f67d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9165700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 0x7f67d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 -- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 msgr2=0x7f67d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e4159890 0x7f67d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 -- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67d0004d50 msgr2=0x7f67d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67d0004d50 0x7f67d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.925+0000 7f67e9966700 1 -- 172.21.15.92:0/466038361 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0003570 con 0x7f67d0003c20 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67e9966700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67d0004010 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009c00 tx=0x7f67e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b41c9418b36e94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e000e710 con 0x7f67d0003c20 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e000e870 con 0x7f67d0003c20 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67e8964700 1 -- 172.21.15.92:0/466038361 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e000ec30 con 0x7f67d0003c20 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ebbca700 1 -- 172.21.15.92:0/466038361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 msgr2=0x7f67d0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ebbca700 1 --2- 172.21.15.92:0/466038361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67d0004010 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009c00 tx=0x7f67e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ea167700 1 -- 172.21.15.92:0/466038361 reap_dead start 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ebbca700 1 -- 172.21.15.92:0/466038361 shutdown_connections 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ebbca700 1 -- 172.21.15.92:0/466038361 >> 172.21.15.92:0/466038361 conn(0x7f67e4068e20 msgr2=0x7f67e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ebbca700 1 -- 172.21.15.92:0/466038361 shutdown_connections 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.926+0000 7f67ebbca700 1 -- 172.21.15.92:0/466038361 wait complete. 2023-10-06T03:10:22.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.927+0000 7f67ebbca700 1 Processor -- start 2023-10-06T03:10:22.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.927+0000 7f67ebbca700 1 -- start start 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ebbca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67e40b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ebbca700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67e40b12d0 0x7f67e4149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ebbca700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e40b3590 0x7f67e40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ebbca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b3f40 con 0x7f67d0003c20 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ebbca700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b2450 con 0x7f67e40b12d0 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ebbca700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b62a0 con 0x7f67e40b3590 2023-10-06T03:10:22.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67e9966700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67e40b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67e9966700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67e40b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36138/0 (socket says 172.21.15.92:36138) 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67e9966700 1 -- 172.21.15.92:0/2559996229 learned_addr learned my addr 172.21.15.92:0/2559996229 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67e9165700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67e40b12d0 0x7f67e4149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67ea167700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e40b3590 0x7f67e40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.928+0000 7f67e9966700 1 -- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e40b3590 msgr2=0x7f67e40b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67e9966700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e40b3590 0x7f67e40b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67e9966700 1 -- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67e40b12d0 msgr2=0x7f67e4149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67e9966700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67e40b12d0 0x7f67e4149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67e9966700 1 -- 172.21.15.92:0/2559996229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40b1840 con 0x7f67d0003c20 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67ea167700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e40b3590 0x7f67e40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67e9165700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67e40b12d0 0x7f67e4149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:22.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67e9966700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67e40b28e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e4b0 tx=0x7f67e001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0022070 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e000ebf0 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e40b1b20 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e001c6c0 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.929+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e40b20e0 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.930+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e404c3c0 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.930+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f67e001ca70 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.931+0000 7f67d9ffb700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f67c8067320 0x7f67c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.931+0000 7f67e9165700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f67c8067320 0x7f67c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.931+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f67e00868f0 con 0x7f67d0003c20 2023-10-06T03:10:22.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.931+0000 7f67e9165700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f67c8067320 0x7f67c80697d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f67d40010e0 tx=0x7f67d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:22.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:22.935+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67e0053280 con 0x7f67d0003c20 2023-10-06T03:10:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.119+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"} v 0) v1 -- 0x7f67e40b2730 con 0x7f67d0003c20 2023-10-06T03:10:23.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.120+0000 7f67d9ffb700 1 -- 172.21.15.92:0/2559996229 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"}]=0 v172) v1 ==== 106+0+11 (secure 0 0 0) 0x7f67e0056860 con 0x7f67d0003c20 2023-10-06T03:10:23.122 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f67c8067320 msgr2=0x7f67c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f67c8067320 0x7f67c80697d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f67d40010e0 tx=0x7f67d4009040 comp rx=0 tx=0).stop 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 msgr2=0x7f67e40b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67e40b28e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f67e000e4b0 tx=0x7f67e001a000 comp rx=0 tx=0).stop 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 shutdown_connections 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f67c8067320 0x7f67c80697d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67d0003c20 0x7f67e40b28e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67e40b3590 0x7f67e40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 --2- 172.21.15.92:0/2559996229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67e40b12d0 0x7f67e4149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 >> 172.21.15.92:0/2559996229 conn(0x7f67e4068e20 msgr2=0x7f67e4149d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 shutdown_connections 2023-10-06T03:10:23.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.122+0000 7f67ebbca700 1 -- 172.21.15.92:0/2559996229 wait complete. 2023-10-06T03:10:23.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 Processor -- start 2023-10-06T03:10:23.251 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 -- start start 2023-10-06T03:10:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdeb8159890 0x7fdeb815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.253 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb8069ee0 con 0x7fdeb8159c60 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb815a160 con 0x7fdeb8159890 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.250+0000 7fdebed82700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80d13a0 con 0x7fdeb8069560 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdeb8159890 0x7fdeb815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdeb8159890 0x7fdeb815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36148/0 (socket says 172.21.15.92:36148) 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 -- 172.21.15.92:0/2492061776 learned_addr learned my addr 172.21.15.92:0/2492061776 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebd31f700 1 --2- 172.21.15.92:0/2492061776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 -- 172.21.15.92:0/2492061776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 msgr2=0x7fdeb80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 --2- 172.21.15.92:0/2492061776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 -- 172.21.15.92:0/2492061776 >> v1:172.21.15.136:6789/0 conn(0x7fdeb8159c60 legacy=0x7fdeb815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 -- 172.21.15.92:0/2492061776 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb815e140 con 0x7fdeb8159890 2023-10-06T03:10:23.254 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdebcb1e700 1 --2- 172.21.15.92:0/2492061776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb815abd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0099b0 tx=0x7fdeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df07dd6f7e9985fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdeb77fe700 1 -- 172.21.15.92:0/2492061776 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac009ea0 con 0x7fdeb8159890 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdeb77fe700 1 -- 172.21.15.92:0/2492061776 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00b860 con 0x7fdeb8159890 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.251+0000 7fdeb77fe700 1 -- 172.21.15.92:0/2492061776 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00bc00 con 0x7fdeb8159890 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 -- 172.21.15.92:0/2492061776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 msgr2=0x7fdeb815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 --2- 172.21.15.92:0/2492061776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb815abd0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0099b0 tx=0x7fdeac009250 comp rx=0 tx=0).stop 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 -- 172.21.15.92:0/2492061776 shutdown_connections 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 --2- 172.21.15.92:0/2492061776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb815abd0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 --2- 172.21.15.92:0/2492061776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 -- 172.21.15.92:0/2492061776 >> 172.21.15.92:0/2492061776 conn(0x7fdeb8068e20 msgr2=0x7fdeb80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 -- 172.21.15.92:0/2492061776 shutdown_connections 2023-10-06T03:10:23.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.252+0000 7fdebed82700 1 -- 172.21.15.92:0/2492061776 wait complete. 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.253+0000 7fdebed82700 1 Processor -- start 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.253+0000 7fdebed82700 1 -- start start 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebed82700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebed82700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb80c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebed82700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb8159c60 0x7fdeb81524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebed82700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb814d330 con 0x7fdeb8159890 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebed82700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80ca380 con 0x7fdeb8159c60 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebed82700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80c8b00 con 0x7fdeb8069560 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb80c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb80c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36158/0 (socket says 172.21.15.92:36158) 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 -- 172.21.15.92:0/3636928591 learned_addr learned my addr 172.21.15.92:0/3636928591 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebcb1e700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdebd31f700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb8159c60 0x7fdeb81524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 -- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 msgr2=0x7fdeb814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb814cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 -- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb8159c60 msgr2=0x7fdeb81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb8159c60 0x7fdeb81524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 -- 172.21.15.92:0/3636928591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0094f0 con 0x7fdeb8159890 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.254+0000 7fdeb7fff700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb80c8eb0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c590 tx=0x7fdea400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.255+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011590 con 0x7fdeb8159890 2023-10-06T03:10:23.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.255+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4011710 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.255+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb80c9770 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.255+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4011ad0 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.255+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb80c9d30 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.256+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb80ca530 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.256+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdea401c440 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.256+0000 7fdeb4ff9700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde9c067320 0x7fde9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.256+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fdea4084450 con 0x7fdeb8159890 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.256+0000 7fdebcb1e700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde9c067320 0x7fde9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.257+0000 7fdebcb1e700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde9c067320 0x7fde9c0697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0099b0 tx=0x7fdeac017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.261+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeb80ca530 con 0x7fdeb8159890 2023-10-06T03:10:23.487 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.486+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdeb80ca6f0 con 0x7fdeb8159890 2023-10-06T03:10:23.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.486+0000 7fdeb4ff9700 1 -- 172.21.15.92:0/3636928591 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4624 (secure 0 0 0) 0x7fdea4017090 con 0x7fdeb8159890 2023-10-06T03:10:23.488 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:23.488 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:18.647577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000000126469","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:23.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde9c067320 msgr2=0x7fde9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde9c067320 0x7fde9c0697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0099b0 tx=0x7fdeac017000 comp rx=0 tx=0).stop 2023-10-06T03:10:23.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 msgr2=0x7fdeb80c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb80c8eb0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fdea400c590 tx=0x7fdea400c8a0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 shutdown_connections 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fde9c067320 0x7fde9c0697d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeb8159c60 0x7fdeb81524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdeb8069560 0x7fdeb814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 --2- 172.21.15.92:0/3636928591 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdeb8159890 0x7fdeb80c8eb0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 >> 172.21.15.92:0/3636928591 conn(0x7fdeb8068e20 msgr2=0x7fdeb815c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 shutdown_connections 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.488+0000 7fdebed82700 1 -- 172.21.15.92:0/3636928591 wait complete. 2023-10-06T03:10:23.491 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 116 2023-10-06T03:10:23.502 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 Processor -- start 2023-10-06T03:10:23.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 -- start start 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb3dc159890 0x7fb3dc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159c60 0x7fb3dc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3dc15a160 con 0x7fb3dc15db10 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3dc0d13a0 con 0x7fb3dc159890 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.616+0000 7fb3e0892700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc15e140 con 0x7fb3dc159c60 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3da59c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb3dc159890 0x7fb3dc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3dad9d700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb3dc15db10 0x7fb3dc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:50864/0 (socket says 172.21.15.92:50864) 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3dad9d700 1 -- 172.21.15.92:0/2728138528 learned_addr learned my addr 172.21.15.92:0/2728138528 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3d1d9b700 1 --2- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159c60 0x7fb3dc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3da59c700 1 -- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159c60 msgr2=0x7fb3dc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3da59c700 1 --2- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159c60 0x7fb3dc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3da59c700 1 -- 172.21.15.92:0/2728138528 >> v1:172.21.15.136:6789/0 conn(0x7fb3dc15db10 legacy=0x7fb3dc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3d9d9b700 1 -- 172.21.15.92:0/2728138528 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343712818 0 0) 0x7fb3dc15a160 con 0x7fb3dc15db10 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3da59c700 1 -- 172.21.15.92:0/2728138528 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc15a3d0 con 0x7fb3dc159890 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3da59c700 1 --2- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159890 0x7fb3dc15abd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0098e0 tx=0x7fb3cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8dff93a8d49c53c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3d9d9b700 1 -- 172.21.15.92:0/2728138528 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00b640 con 0x7fb3dc159890 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3d9d9b700 1 -- 172.21.15.92:0/2728138528 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc00b7a0 con 0x7fb3dc159890 2023-10-06T03:10:23.622 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.617+0000 7fb3d9d9b700 1 -- 172.21.15.92:0/2728138528 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00bb40 con 0x7fb3dc159890 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 -- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159890 msgr2=0x7fb3dc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 --2- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159890 0x7fb3dc15abd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0098e0 tx=0x7fb3cc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 -- 172.21.15.92:0/2728138528 shutdown_connections 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 --2- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159890 0x7fb3dc15abd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 --2- 172.21.15.92:0/2728138528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159c60 0x7fb3dc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 -- 172.21.15.92:0/2728138528 >> 172.21.15.92:0/2728138528 conn(0x7fb3dc068e20 msgr2=0x7fb3dc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 -- 172.21.15.92:0/2728138528 shutdown_connections 2023-10-06T03:10:23.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.618+0000 7fb3e0892700 1 -- 172.21.15.92:0/2728138528 wait complete. 2023-10-06T03:10:23.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.619+0000 7fb3e0892700 1 Processor -- start 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.619+0000 7fb3e0892700 1 -- start start 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3e0892700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159890 0x7fb3dc0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3e0892700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159c60 0x7fb3dc14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3e0892700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 0x7fb3dc0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3e0892700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc14d800 con 0x7fb3dc15db10 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3e0892700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0cf3c0 con 0x7fb3dc159c60 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3e0892700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0cf5f0 con 0x7fb3dc159890 2023-10-06T03:10:23.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 0x7fb3dc0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 0x7fb3dc0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36164/0 (socket says 172.21.15.92:36164) 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 -- 172.21.15.92:0/4050305471 learned_addr learned my addr 172.21.15.92:0/4050305471 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 -- 172.21.15.92:0/4050305471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159890 msgr2=0x7fb3dc0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159890 0x7fb3dc0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 -- 172.21.15.92:0/4050305471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159c60 msgr2=0x7fb3dc14d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159c60 0x7fb3dc14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 -- 172.21.15.92:0/4050305471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009420 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.620+0000 7fb3dad9d700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 0x7fb3dc0cee80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400cbf0 tx=0x7fb3d40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.621+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d400a9c0 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.621+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d400ab20 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.621+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3dc0cf8d0 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.621+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40070f0 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.621+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc0cfe90 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.622+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3dc04c3c0 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.622+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb3d40074e0 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.622+0000 7fb3d2ffd700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb3bc067320 0x7fb3bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.622+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fb3d40853d0 con 0x7fb3dc15db10 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.622+0000 7fb3da59c700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb3bc067320 0x7fb3bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.623+0000 7fb3da59c700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb3bc067320 0x7fb3bc0697d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0b0330 tx=0x7fb3cc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.627+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3d4051d60 con 0x7fb3dc15db10 2023-10-06T03:10:23.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.816+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3dc06a5b0 con 0x7fb3dc15db10 2023-10-06T03:10:23.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.817+0000 7fb3d2ffd700 1 -- 172.21.15.92:0/4050305471 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18521 (secure 0 0 0) 0x7fb3d4055380 con 0x7fb3dc15db10 2023-10-06T03:10:23.819 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:23.819 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":172,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:19.655516+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs.volume_0000000000126469.meta","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000126469"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-10-06T03:10:14.941237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:23.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb3bc067320 msgr2=0x7fb3bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb3bc067320 0x7fb3bc0697d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0b0330 tx=0x7fb3cc017000 comp rx=0 tx=0).stop 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 msgr2=0x7fb3dc0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 0x7fb3dc0cee80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400cbf0 tx=0x7fb3d40019b0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 shutdown_connections 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb3bc067320 0x7fb3bc0697d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb3dc15db10 0x7fb3dc0cee80 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3dc159890 0x7fb3dc0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 --2- 172.21.15.92:0/4050305471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3dc159c60 0x7fb3dc14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 >> 172.21.15.92:0/4050305471 conn(0x7fb3dc068e20 msgr2=0x7fb3dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 shutdown_connections 2023-10-06T03:10:23.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.819+0000 7fb3e0892700 1 -- 172.21.15.92:0/4050305471 wait complete. 2023-10-06T03:10:23.833 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000126469 and related pools 2023-10-06T03:10:23.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:23.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 Processor -- start 2023-10-06T03:10:23.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 -- start start 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3eb0159420 0x7f3eb00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3eb015d230 0x7f3eb015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00b2980 con 0x7f3eb01597f0 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb015db70 con 0x7f3eb015d230 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaf59e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb00b11d0 con 0x7f3eb0159420 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3eb015d230 0x7f3eb015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eadd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3eb01597f0 0x7f3eb00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37932/0 (socket says 172.21.15.92:37932) 2023-10-06T03:10:23.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eadd9b700 1 -- 172.21.15.92:0/1774502368 learned_addr learned my addr 172.21.15.92:0/1774502368 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eae59c700 1 --2- 172.21.15.92:0/1774502368 >> v2:172.21.15.136:3300/0 conn(0x7f3eb0159420 0x7f3eb00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaed9d700 1 -- 172.21.15.92:0/1774502368 >> v2:172.21.15.136:3300/0 conn(0x7f3eb0159420 msgr2=0x7f3eb00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaed9d700 1 --2- 172.21.15.92:0/1774502368 >> v2:172.21.15.136:3300/0 conn(0x7f3eb0159420 0x7f3eb00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaed9d700 1 -- 172.21.15.92:0/1774502368 >> v1:172.21.15.92:6789/0 conn(0x7f3eb01597f0 legacy=0x7f3eb00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaed9d700 1 -- 172.21.15.92:0/1774502368 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00b14b0 con 0x7f3eb015d230 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3ead59a700 1 -- 172.21.15.92:0/1774502368 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496071263 0 0) 0x7f3eb00b2980 con 0x7f3eb01597f0 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eae59c700 1 --2- 172.21.15.92:0/1774502368 >> v2:172.21.15.136:3300/0 conn(0x7f3eb0159420 0x7f3eb00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3eaed9d700 1 --2- 172.21.15.92:0/1774502368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb015d630 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00b2f20 tx=0x7f3ea8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f660d24ac3ad217 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.948+0000 7f3ead59a700 1 -- 172.21.15.92:0/1774502368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800e070 con 0x7f3eb015d230 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3ead59a700 1 -- 172.21.15.92:0/1774502368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea800bc30 con 0x7f3eb015d230 2023-10-06T03:10:23.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3ead59a700 1 -- 172.21.15.92:0/1774502368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8012430 con 0x7f3eb015d230 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 -- 172.21.15.92:0/1774502368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 msgr2=0x7f3eb015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/1774502368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb015d630 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00b2f20 tx=0x7f3ea8009780 comp rx=0 tx=0).stop 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 -- 172.21.15.92:0/1774502368 shutdown_connections 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/1774502368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb015d630 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/1774502368 >> v2:172.21.15.136:3300/0 conn(0x7f3eb0159420 0x7f3eb00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 -- 172.21.15.92:0/1774502368 >> 172.21.15.92:0/1774502368 conn(0x7f3eb00d0660 msgr2=0x7f3eb00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 -- 172.21.15.92:0/1774502368 shutdown_connections 2023-10-06T03:10:23.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.949+0000 7f3eaf59e700 1 -- 172.21.15.92:0/1774502368 wait complete. 2023-10-06T03:10:23.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.950+0000 7f3eaf59e700 1 Processor -- start 2023-10-06T03:10:23.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.950+0000 7f3eaf59e700 1 -- start start 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaf59e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eb0159420 0x7f3eb016a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaf59e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eb01597f0 0x7f3eb016c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaf59e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaf59e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb015a240 con 0x7f3eb015d230 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaf59e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb015a520 con 0x7f3eb01597f0 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaf59e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb015a800 con 0x7f3eb0159420 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36190/0 (socket says 172.21.15.92:36190) 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 -- 172.21.15.92:0/2509522741 learned_addr learned my addr 172.21.15.92:0/2509522741 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:23.956 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 -- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eb0159420 msgr2=0x7f3eb016a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eb0159420 0x7f3eb016a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eadd9b700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eb01597f0 0x7f3eb016c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 -- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eb01597f0 msgr2=0x7f3eb016c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eb01597f0 0x7f3eb016c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 -- 172.21.15.92:0/2509522741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009a20 con 0x7f3eb015d230 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3eaed9d700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8012580 tx=0x7f3ea8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.951+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea801e630 con 0x7f3eb015d230 2023-10-06T03:10:23.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.952+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea801e7d0 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.952+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb015aae0 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.952+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea801eb90 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.952+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb016a5b0 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.953+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb015ba10 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.953+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3ea8024070 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.953+0000 7f3e9e7fc700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e90067320 0x7f3e900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.953+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f3ea8087920 con 0x7f3eb015d230 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.953+0000 7f3eae59c700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e90067320 0x7f3e900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:23.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.954+0000 7f3eae59c700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e90067320 0x7f3e900697d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3eb006a360 tx=0x7f3ea0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:23.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:23.958+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ea8054270 con 0x7f3eb015d230 2023-10-06T03:10:24.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.184+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eb004c3c0 con 0x7f3eb015d230 2023-10-06T03:10:24.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.184+0000 7f3e9e7fc700 1 -- 172.21.15.92:0/2509522741 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f3ea8057890 con 0x7f3eb015d230 2023-10-06T03:10:24.186 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:24.186 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:18.647577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000000126469","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:24.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e90067320 msgr2=0x7f3e900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e90067320 0x7f3e900697d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3eb006a360 tx=0x7f3ea0008040 comp rx=0 tx=0).stop 2023-10-06T03:10:24.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 msgr2=0x7f3eb00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8012580 tx=0x7f3ea8019000 comp rx=0 tx=0).stop 2023-10-06T03:10:24.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 shutdown_connections 2023-10-06T03:10:24.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3e90067320 0x7f3e900697d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3eb015d230 0x7f3eb00b4e20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eb0159420 0x7f3eb016a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 --2- 172.21.15.92:0/2509522741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eb01597f0 0x7f3eb016c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.190 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.186+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 >> 172.21.15.92:0/2509522741 conn(0x7f3eb00d0660 msgr2=0x7f3eb0157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:24.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.187+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 shutdown_connections 2023-10-06T03:10:24.191 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.187+0000 7f3eaf59e700 1 -- 172.21.15.92:0/2509522741 wait complete. 2023-10-06T03:10:24.191 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 116 2023-10-06T03:10:24.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:24.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.315+0000 7f6d94924700 1 Processor -- start 2023-10-06T03:10:24.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.315+0000 7f6d94924700 1 -- start start 2023-10-06T03:10:24.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d94924700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6d90069560 0x7f6d900699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d94924700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d90069ee0 con 0x7f6d90159890 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d94924700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9015a160 con 0x7f6d90159c60 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d94924700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900d13a0 con 0x7f6d90069560 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8e59c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6d90159890 0x7f6d9015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37936/0 (socket says 172.21.15.92:37936) 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8e59c700 1 -- 172.21.15.92:0/2384462234 learned_addr learned my addr 172.21.15.92:0/2384462234 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148560482 0 0) 0x7f6d90069ee0 con 0x7f6d90159890 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d78003570 con 0x7f6d90159890 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2384462234 >> v2:172.21.15.136:3300/0 conn(0x7f6d90069560 0x7f6d900699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823630334 0 0) 0x7f6d78003570 con 0x7f6d90159890 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 >> v2:172.21.15.136:3300/0 conn(0x7f6d90069560 msgr2=0x7f6d900699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 --2- 172.21.15.92:0/2384462234 >> v2:172.21.15.136:3300/0 conn(0x7f6d90069560 0x7f6d900699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 >> v1:172.21.15.136:6789/0 conn(0x7f6d90159c60 legacy=0x7f6d9015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9015e140 con 0x7f6d90159890 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f6d880017f0 con 0x7f6d90159890 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 >> v1:172.21.15.92:6789/0 conn(0x7f6d90159890 legacy=0x7f6d9015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d78003a60 0x7f6d78003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d7800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900d13a0 con 0x7f6d780051c0 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7800c0d0 con 0x7f6d78004810 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7800c290 con 0x7f6d78003a60 2023-10-06T03:10:24.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d7800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8e59c700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d78003a60 0x7f6d78003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8dd9b700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 -- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d78003a60 msgr2=0x7f6d78003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d78003a60 0x7f6d78003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 -- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d78004810 msgr2=0x7f6d78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 -- 172.21.15.92:0/2384462234 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78003570 con 0x7f6d780051c0 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 -- 172.21.15.92:0/2384462234 reap_dead start 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d7800ba60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009c00 tx=0x7f6d800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c2697e3e2e9e59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000e710 con 0x7f6d780051c0 2023-10-06T03:10:24.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000e890 con 0x7f6d780051c0 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.316+0000 7f6d8d59a700 1 -- 172.21.15.92:0/2384462234 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000ec50 con 0x7f6d780051c0 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 -- 172.21.15.92:0/2384462234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 msgr2=0x7f6d7800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d7800ba60 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009c00 tx=0x7f6d800091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 -- 172.21.15.92:0/2384462234 shutdown_connections 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 --2- 172.21.15.92:0/2384462234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d7800ba60 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 -- 172.21.15.92:0/2384462234 >> 172.21.15.92:0/2384462234 conn(0x7f6d90068e20 msgr2=0x7f6d900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 -- 172.21.15.92:0/2384462234 shutdown_connections 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.317+0000 7f6d94924700 1 -- 172.21.15.92:0/2384462234 wait complete. 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.318+0000 7f6d94924700 1 Processor -- start 2023-10-06T03:10:24.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.318+0000 7f6d94924700 1 -- start start 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d900ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9015c610 0x7f6d9015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d9016d220 0x7f6d9016d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9016dbd0 con 0x7f6d780051c0 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9015c180 con 0x7f6d9015c610 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900ca820 con 0x7f6d9016d220 2023-10-06T03:10:24.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d900ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d900ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36208/0 (socket says 172.21.15.92:36208) 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 -- 172.21.15.92:0/2206336504 learned_addr learned my addr 172.21.15.92:0/2206336504 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8dd9b700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9015c610 0x7f6d9015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 -- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d9016d220 msgr2=0x7f6d9016d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d9016d220 0x7f6d9016d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d9016d220 0x7f6d9016d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 -- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9015c610 msgr2=0x7f6d9015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9015c610 0x7f6d9015ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 -- 172.21.15.92:0/2206336504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9015b570 con 0x7f6d780051c0 2023-10-06T03:10:24.327 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8dd9b700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9015c610 0x7f6d9015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8ed9d700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d9016d220 0x7f6d9016d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d8e59c700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d900ca2e0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800a0c0 tx=0x7f6d880021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800ad10 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800ae70 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9015b850 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88013620 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.319+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9015be10 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.320+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9004c3c0 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.321+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6d88013990 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.321+0000 7f6d7e7fc700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d70067320 0x7f6d700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.321+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f6d88085d30 con 0x7f6d780051c0 2023-10-06T03:10:24.328 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.321+0000 7f6d8dd9b700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d70067320 0x7f6d700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.321+0000 7f6d8dd9b700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d70067320 0x7f6d700697d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000f60 tx=0x7f6d84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:24.329 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.326+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d88052680 con 0x7f6d780051c0 2023-10-06T03:10:24.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.550+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d900caba0 con 0x7f6d780051c0 2023-10-06T03:10:24.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.550+0000 7f6d7e7fc700 1 -- 172.21.15.92:0/2206336504 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f6d88017090 con 0x7f6d780051c0 2023-10-06T03:10:24.551 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:24.551 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":116,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:09:59.949008+0000","modified":"2023-10-06T03:10:18.647577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8361":{"gid":8361,"name":"b","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.136:6834/1055679258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1055679258},{"type":"v1","addr":"172.21.15.136:6834","nonce":1055679258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[30,31],"metadata_pool":29,"enabled":true,"fs_name":"volume_0000000000126469","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-06T03:10:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d70067320 msgr2=0x7f6d700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d70067320 0x7f6d700697d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000f60 tx=0x7f6d84009040 comp rx=0 tx=0).stop 2023-10-06T03:10:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 msgr2=0x7f6d900ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d900ca2e0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800a0c0 tx=0x7f6d880021d0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 shutdown_connections 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d70067320 0x7f6d700697d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d780051c0 0x7f6d900ca2e0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d9016d220 0x7f6d9016d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 --2- 172.21.15.92:0/2206336504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d9015c610 0x7f6d9015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 >> 172.21.15.92:0/2206336504 conn(0x7f6d90068e20 msgr2=0x7f6d901536a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 shutdown_connections 2023-10-06T03:10:24.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.551+0000 7f6d94924700 1 -- 172.21.15.92:0/2206336504 wait complete. 2023-10-06T03:10:24.555 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 116 2023-10-06T03:10:24.565 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.683+0000 7f78adfe9700 1 Processor -- start 2023-10-06T03:10:24.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.683+0000 7f78adfe9700 1 -- start start 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78adfe9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78a8159890 0x7f78a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78adfe9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159c60 0x7f78a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78adfe9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f78a815db10 0x7f78a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78adfe9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a815a160 con 0x7f78a8159890 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78adfe9700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80d13a0 con 0x7f78a815db10 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78adfe9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a815e140 con 0x7f78a8159c60 2023-10-06T03:10:24.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.684+0000 7f78a77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78a8159890 0x7f78a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78a8159890 0x7f78a815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36214/0 (socket says 172.21.15.92:36214) 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 -- 172.21.15.92:0/286339063 learned_addr learned my addr 172.21.15.92:0/286339063 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a6ffd700 1 --2- 172.21.15.92:0/286339063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159c60 0x7f78a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a7fff700 1 --2- 172.21.15.92:0/286339063 >> v2:172.21.15.136:3300/0 conn(0x7f78a815db10 0x7f78a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 -- 172.21.15.92:0/286339063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159c60 msgr2=0x7f78a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 --2- 172.21.15.92:0/286339063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159c60 0x7f78a815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 -- 172.21.15.92:0/286339063 >> v2:172.21.15.136:3300/0 conn(0x7f78a815db10 msgr2=0x7f78a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 --2- 172.21.15.92:0/286339063 >> v2:172.21.15.136:3300/0 conn(0x7f78a815db10 0x7f78a8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 -- 172.21.15.92:0/286339063 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a815a3d0 con 0x7f78a8159890 2023-10-06T03:10:24.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.685+0000 7f78a77fe700 1 --2- 172.21.15.92:0/286339063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8159890 0x7f78a815abd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f78a8089b50 tx=0x7f7898009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db3e34e23514050 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.689+0000 7f78a67fc700 1 -- 172.21.15.92:0/286339063 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800e070 con 0x7f78a8159890 2023-10-06T03:10:24.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.689+0000 7f78a67fc700 1 -- 172.21.15.92:0/286339063 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800ba50 con 0x7f78a8159890 2023-10-06T03:10:24.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.689+0000 7f78a67fc700 1 -- 172.21.15.92:0/286339063 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78980092e0 con 0x7f78a8159890 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 -- 172.21.15.92:0/286339063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8159890 msgr2=0x7f78a815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 --2- 172.21.15.92:0/286339063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8159890 0x7f78a815abd0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f78a8089b50 tx=0x7f7898009840 comp rx=0 tx=0).stop 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 -- 172.21.15.92:0/286339063 shutdown_connections 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 --2- 172.21.15.92:0/286339063 >> v2:172.21.15.136:3300/0 conn(0x7f78a815db10 0x7f78a8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 --2- 172.21.15.92:0/286339063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a8159890 0x7f78a815abd0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 --2- 172.21.15.92:0/286339063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159c60 0x7f78a815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 -- 172.21.15.92:0/286339063 >> 172.21.15.92:0/286339063 conn(0x7f78a8068e20 msgr2=0x7f78a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 -- 172.21.15.92:0/286339063 shutdown_connections 2023-10-06T03:10:24.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.690+0000 7f78adfe9700 1 -- 172.21.15.92:0/286339063 wait complete. 2023-10-06T03:10:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.691+0000 7f78adfe9700 1 Processor -- start 2023-10-06T03:10:24.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.691+0000 7f78adfe9700 1 -- start start 2023-10-06T03:10:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78adfe9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159890 0x7f78a813bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78adfe9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8159c60 0x7f78a814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78adfe9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 0x7f78a814e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78adfe9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80cd100 con 0x7f78a815db10 2023-10-06T03:10:24.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78adfe9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a813c130 con 0x7f78a8159c60 2023-10-06T03:10:24.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78adfe9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80cd3a0 con 0x7f78a8159890 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 0x7f78a814e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 0x7f78a814e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36222/0 (socket says 172.21.15.92:36222) 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 -- 172.21.15.92:0/3204399102 learned_addr learned my addr 172.21.15.92:0/3204399102 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a77fe700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159890 0x7f78a813bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a6ffd700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8159c60 0x7f78a814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 -- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159890 msgr2=0x7f78a813bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159890 0x7f78a813bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 -- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8159c60 msgr2=0x7f78a814df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8159c60 0x7f78a814df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 -- 172.21.15.92:0/3204399102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898009ae0 con 0x7f78a815db10 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a77fe700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159890 0x7f78a813bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.692+0000 7f78a7fff700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 0x7f78a814e440 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d890 tx=0x7f78a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.693+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a00125f0 con 0x7f78a815db10 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.693+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0012770 con 0x7f78a815db10 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.693+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a814d6d0 con 0x7f78a815db10 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.693+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0012b30 con 0x7f78a815db10 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.693+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a8150de0 con 0x7f78a815db10 2023-10-06T03:10:24.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.694+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a80cd510 con 0x7f78a815db10 2023-10-06T03:10:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.694+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f78a001e460 con 0x7f78a815db10 2023-10-06T03:10:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.695+0000 7f789f7fe700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7888067320 0x7f78880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.695+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f78a0086100 con 0x7f78a815db10 2023-10-06T03:10:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.695+0000 7f78a77fe700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7888067320 0x7f78880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:24.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.695+0000 7f78a77fe700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7888067320 0x7f78880697d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7898009880 tx=0x7f7898018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:24.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.698+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78a0052a60 con 0x7f78a815db10 2023-10-06T03:10:24.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.882+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78a804c3c0 con 0x7f78a815db10 2023-10-06T03:10:24.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.883+0000 7f789f7fe700 1 -- 172.21.15.92:0/3204399102 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+18712 (secure 0 0 0) 0x7f78a0056080 con 0x7f78a815db10 2023-10-06T03:10:24.884 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:24.884 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":173,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:24.673673+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs.volume_0000000000126469.meta","create_time":"2023-10-06T03:09:57.571177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000126469"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2023-10-06T03:09:58.909445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}},{"pool":31,"pool_name":"another-data-pool","create_time":"2023-10-06T03:10:14.941237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000126469"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.4","mappings":[{"from":6,"to":0}]},{"pgid":"29.d","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.16","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:24.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7888067320 msgr2=0x7f78880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7888067320 0x7f78880697d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7898009880 tx=0x7f7898018000 comp rx=0 tx=0).stop 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 msgr2=0x7f78a814e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 0x7f78a814e440 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d890 tx=0x7f78a0008040 comp rx=0 tx=0).stop 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 shutdown_connections 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7888067320 0x7f78880697d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78a815db10 0x7f78a814e440 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78a8159890 0x7f78a813bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 --2- 172.21.15.92:0/3204399102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a8159c60 0x7f78a814df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:24.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 >> 172.21.15.92:0/3204399102 conn(0x7f78a8068e20 msgr2=0x7f78a80d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:24.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 shutdown_connections 2023-10-06T03:10:24.888 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:24.886+0000 7f78adfe9700 1 -- 172.21.15.92:0/3204399102 wait complete. 2023-10-06T03:10:24.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000126469 2023-10-06T03:10:25.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.015+0000 7f372a2d4700 1 Processor -- start 2023-10-06T03:10:25.016 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.015+0000 7f372a2d4700 1 -- start start 2023-10-06T03:10:25.017 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.015+0000 7f372a2d4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3724159c60 0x7f372415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f372a2d4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f372415db10 0x7f3724069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f372a2d4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f372415a160 con 0x7f3724159890 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f372a2d4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37240d13a0 con 0x7f372415db10 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f372a2d4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372415e140 con 0x7f3724159c60 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f372415db10 0x7f3724069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3723fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3724159890 0x7f372415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37948/0 (socket says 172.21.15.92:37948) 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f372415db10 0x7f3724069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36236/0 (socket says 172.21.15.92:36236) 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3723fff700 1 -- 172.21.15.92:0/3297034505 learned_addr learned my addr 172.21.15.92:0/3297034505 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:25.018 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f37237fe700 1 --2- 172.21.15.92:0/3297034505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3724159c60 0x7f372415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 -- 172.21.15.92:0/3297034505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3724159c60 msgr2=0x7f372415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 --2- 172.21.15.92:0/3297034505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3724159c60 0x7f372415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 -- 172.21.15.92:0/3297034505 >> v1:172.21.15.92:6789/0 conn(0x7f3724159890 legacy=0x7f372415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 -- 172.21.15.92:0/3297034505 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372415a3d0 con 0x7f372415db10 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3722ffd700 1 -- 172.21.15.92:0/3297034505 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926451495 0 0) 0x7f372415a160 con 0x7f3724159890 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3728871700 1 --2- 172.21.15.92:0/3297034505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f372415db10 0x7f3724069560 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f371c009e60 tx=0x7f371c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fbcf0123baafbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3722ffd700 1 -- 172.21.15.92:0/3297034505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c00e070 con 0x7f372415db10 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3722ffd700 1 -- 172.21.15.92:0/3297034505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c00bb60 con 0x7f372415db10 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.016+0000 7f3722ffd700 1 -- 172.21.15.92:0/3297034505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371c012430 con 0x7f372415db10 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 -- 172.21.15.92:0/3297034505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f372415db10 msgr2=0x7f3724069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.019 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 --2- 172.21.15.92:0/3297034505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f372415db10 0x7f3724069560 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f371c009e60 tx=0x7f371c009780 comp rx=0 tx=0).stop 2023-10-06T03:10:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 -- 172.21.15.92:0/3297034505 shutdown_connections 2023-10-06T03:10:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 --2- 172.21.15.92:0/3297034505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f372415db10 0x7f3724069560 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 --2- 172.21.15.92:0/3297034505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3724159c60 0x7f372415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 -- 172.21.15.92:0/3297034505 >> 172.21.15.92:0/3297034505 conn(0x7f3724068e20 msgr2=0x7f37240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 -- 172.21.15.92:0/3297034505 shutdown_connections 2023-10-06T03:10:25.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.017+0000 7f372a2d4700 1 -- 172.21.15.92:0/3297034505 wait complete. 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.018+0000 7f372a2d4700 1 Processor -- start 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.018+0000 7f372a2d4700 1 -- start start 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f372a2d4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3724159890 0x7f37240b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f372a2d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 0x7f372414a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f372a2d4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f372415db10 0x7f3724151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f372a2d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372414ad30 con 0x7f3724159c60 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f372a2d4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37240b2340 con 0x7f3724159890 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f372a2d4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37241523e0 con 0x7f372415db10 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 0x7f372414a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 0x7f372414a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36238/0 (socket says 172.21.15.92:36238) 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 -- 172.21.15.92:0/2372082656 learned_addr learned my addr 172.21.15.92:0/2372082656 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 -- 172.21.15.92:0/2372082656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f372415db10 msgr2=0x7f3724151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f372415db10 0x7f3724151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 -- 172.21.15.92:0/2372082656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3724159890 msgr2=0x7f37240b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3724159890 0x7f37240b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 -- 172.21.15.92:0/2372082656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c009a20 con 0x7f3724159c60 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.019+0000 7f37237fe700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 0x7f372414a7f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f371800ec80 tx=0x7f37180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.020+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371800c9d0 con 0x7f3724159c60 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.020+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371800cb30 con 0x7f3724159c60 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.020+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372414b060 con 0x7f3724159c60 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.020+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718013470 con 0x7f3724159c60 2023-10-06T03:10:25.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.020+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37240b1b20 con 0x7f3724159c60 2023-10-06T03:10:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.021+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372404c3c0 con 0x7f3724159c60 2023-10-06T03:10:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.021+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3718013860 con 0x7f3724159c60 2023-10-06T03:10:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.021+0000 7f370bfff700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3710067320 0x7f37100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.021+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f3718085370 con 0x7f3724159c60 2023-10-06T03:10:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.021+0000 7f3723fff700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3710067320 0x7f37100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.022+0000 7f3723fff700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3710067320 0x7f37100697d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f37140011e0 tx=0x7f37140079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:25.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.025+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3718051c50 con 0x7f3724159c60 2023-10-06T03:10:25.252 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.251+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000126469"} v 0) v1 -- 0x7f372406a5b0 con 0x7f3724159c60 2023-10-06T03:10:25.252 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:10:25.252+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:10:25.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.256+0000 7f370bfff700 1 -- 172.21.15.92:0/2372082656 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000126469"}]=0 volume_0000000000126469 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 195+0+0 (secure 0 0 0) 0x7f3718055270 con 0x7f3724159c60 2023-10-06T03:10:25.258 INFO:tasks.ceph.mds.b.smithi136.stderr: -943> 2023-10-06T03:10:04.824+0000 7f4b2182d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:10:25.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3710067320 msgr2=0x7f37100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3710067320 0x7f37100697d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f37140011e0 tx=0x7f37140079d0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 msgr2=0x7f372414a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 0x7f372414a7f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f371800ec80 tx=0x7f37180091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 shutdown_connections 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3710067320 0x7f37100697d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f372415db10 0x7f3724151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3724159890 0x7f37240b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 --2- 172.21.15.92:0/2372082656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3724159c60 0x7f372414a7f0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 >> 172.21.15.92:0/2372082656 conn(0x7f3724068e20 msgr2=0x7f3724156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 shutdown_connections 2023-10-06T03:10:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.259+0000 7f372a2d4700 1 -- 172.21.15.92:0/2372082656 wait complete. 2023-10-06T03:10:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:volume_0000000000126469 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:10:25.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000126469 --yes-i-really-mean-it 2023-10-06T03:10:25.282 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:10:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.389+0000 7fa999b70700 1 Processor -- start 2023-10-06T03:10:25.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.389+0000 7fa999b70700 1 -- start start 2023-10-06T03:10:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa999b70700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa994069560 0x7fa9940699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa999b70700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa994069ee0 con 0x7fa994159c60 2023-10-06T03:10:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa999b70700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99415a160 con 0x7fa994159890 2023-10-06T03:10:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa999b70700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9940d13a0 con 0x7fa994069560 2023-10-06T03:10:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa994069560 0x7fa9940699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa994069560 0x7fa9940699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36240/0 (socket says 172.21.15.92:36240) 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 -- 172.21.15.92:0/1896917427 learned_addr learned my addr 172.21.15.92:0/1896917427 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa992ffd700 1 --1- 172.21.15.92:0/1896917427 >> v1:172.21.15.92:6789/0 conn(0x7fa994159c60 0x7fa99415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37950/0 (socket says 172.21.15.92:37950) 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 -- 172.21.15.92:0/1896917427 >> v1:172.21.15.136:6789/0 conn(0x7fa994159890 legacy=0x7fa99415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 -- 172.21.15.92:0/1896917427 >> v1:172.21.15.92:6789/0 conn(0x7fa994159c60 legacy=0x7fa99415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 -- 172.21.15.92:0/1896917427 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99415e140 con 0x7fa994069560 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa993fff700 1 --2- 172.21.15.92:0/1896917427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994069560 0x7fa9940699a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0099b0 tx=0x7fa98c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78d9a7064f9f5ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.390+0000 7fa9927fc700 1 -- 172.21.15.92:0/1896917427 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00b820 con 0x7fa994069560 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa9927fc700 1 -- 172.21.15.92:0/1896917427 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c00b980 con 0x7fa994069560 2023-10-06T03:10:25.392 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa9927fc700 1 -- 172.21.15.92:0/1896917427 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00bd20 con 0x7fa994069560 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 -- 172.21.15.92:0/1896917427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994069560 msgr2=0x7fa9940699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 --2- 172.21.15.92:0/1896917427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994069560 0x7fa9940699a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0099b0 tx=0x7fa98c009250 comp rx=0 tx=0).stop 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 -- 172.21.15.92:0/1896917427 shutdown_connections 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 --2- 172.21.15.92:0/1896917427 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994069560 0x7fa9940699a0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 -- 172.21.15.92:0/1896917427 >> 172.21.15.92:0/1896917427 conn(0x7fa994068e20 msgr2=0x7fa9940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 -- 172.21.15.92:0/1896917427 shutdown_connections 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.391+0000 7fa999b70700 1 -- 172.21.15.92:0/1896917427 wait complete. 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.392+0000 7fa999b70700 1 Processor -- start 2023-10-06T03:10:25.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.392+0000 7fa999b70700 1 -- start start 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa999b70700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994069560 0x7fa9940b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa999b70700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 0x7fa99415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa999b70700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa994159c60 0x7fa9940c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa999b70700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415e380 con 0x7fa994159890 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa999b70700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415ec20 con 0x7fa994069560 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa999b70700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940cfd60 con 0x7fa994159c60 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 0x7fa99415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 0x7fa99415f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36246/0 (socket says 172.21.15.92:36246) 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 -- 172.21.15.92:0/202932438 learned_addr learned my addr 172.21.15.92:0/202932438 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa9937fe700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994069560 0x7fa9940b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 -- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa994159c60 msgr2=0x7fa9940c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa994159c60 0x7fa9940c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 -- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994069560 msgr2=0x7fa9940b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994069560 0x7fa9940b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.393+0000 7fa992ffd700 1 -- 172.21.15.92:0/202932438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0094f0 con 0x7fa994159890 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.394+0000 7fa992ffd700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 0x7fa99415f0b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c560 tx=0x7fa988007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.394+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988005a70 con 0x7fa994159890 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.394+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988005bd0 con 0x7fa994159890 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.394+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99415e5d0 con 0x7fa994159890 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.394+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9880070d0 con 0x7fa994159890 2023-10-06T03:10:25.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.394+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940c6c50 con 0x7fa994159890 2023-10-06T03:10:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.395+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99404c3c0 con 0x7fa994159890 2023-10-06T03:10:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.395+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa988007480 con 0x7fa994159890 2023-10-06T03:10:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.395+0000 7fa97b7fe700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa980067320 0x7fa9800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.396+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa988084550 con 0x7fa994159890 2023-10-06T03:10:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.396+0000 7fa9937fe700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa980067320 0x7fa9800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:25.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.396+0000 7fa9937fe700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa980067320 0x7fa9800697d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa984007fd0 tx=0x7fa984009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:25.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.400+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa988050de0 con 0x7fa994159890 2023-10-06T03:10:25.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:25.629+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000126469", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa99406a5b0 con 0x7fa994159890 2023-10-06T03:10:26.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.259+0000 7fa97b7fe700 1 -- 172.21.15.92:0/202932438 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000126469", "yes_i_really_mean_it": true}]=0 v118) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa988054400 con 0x7fa994159890 2023-10-06T03:10:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa980067320 msgr2=0x7fa9800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa980067320 0x7fa9800697d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa984007fd0 tx=0x7fa984009040 comp rx=0 tx=0).stop 2023-10-06T03:10:26.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 msgr2=0x7fa99415f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 0x7fa99415f0b0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c560 tx=0x7fa988007f70 comp rx=0 tx=0).stop 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 shutdown_connections 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa980067320 0x7fa9800697d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa994159c60 0x7fa9940c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa994069560 0x7fa9940b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 --2- 172.21.15.92:0/202932438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa994159890 0x7fa99415f0b0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 >> 172.21.15.92:0/202932438 conn(0x7fa994068e20 msgr2=0x7fa994157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 shutdown_connections 2023-10-06T03:10:26.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.261+0000 7fa999b70700 1 -- 172.21.15.92:0/202932438 wait complete. 2023-10-06T03:10:26.275 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000126469.meta cephfs.volume_0000000000126469.meta --yes-i-really-really-mean-it 2023-10-06T03:10:26.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.395+0000 7f014dc81700 1 Processor -- start 2023-10-06T03:10:26.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.395+0000 7f014dc81700 1 -- start start 2023-10-06T03:10:26.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f014dc81700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0148159ec0 0x7f01480b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f014dc81700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01480b53d0 0x7f01480cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f014dc81700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480cce40 0x7f01480cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f014dc81700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01480b5b00 con 0x7f0148159ec0 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f014dc81700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01480b5850 con 0x7f01480b53d0 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f014dc81700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01480b5da0 con 0x7f01480cce40 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f01477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0148159ec0 0x7f01480b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f01477fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0148159ec0 0x7f01480b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36248/0 (socket says 172.21.15.92:36248) 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.396+0000 7f01477fe700 1 -- 172.21.15.92:0/3775502667 learned_addr learned my addr 172.21.15.92:0/3775502667 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01477fe700 1 -- 172.21.15.92:0/3775502667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480cce40 msgr2=0x7f01480cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01477fe700 1 --2- 172.21.15.92:0/3775502667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480cce40 0x7f01480cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01477fe700 1 -- 172.21.15.92:0/3775502667 >> v2:172.21.15.136:3300/0 conn(0x7f01480b53d0 msgr2=0x7f01480cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01477fe700 1 --2- 172.21.15.92:0/3775502667 >> v2:172.21.15.136:3300/0 conn(0x7f01480b53d0 0x7f01480cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01477fe700 1 -- 172.21.15.92:0/3775502667 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01480b6080 con 0x7f0148159ec0 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01477fe700 1 --2- 172.21.15.92:0/3775502667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f01480b4e90 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f01480b64d0 tx=0x7f0140009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1649c36830514e39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01467fc700 1 -- 172.21.15.92:0/3775502667 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014000e070 con 0x7f0148159ec0 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01467fc700 1 -- 172.21.15.92:0/3775502667 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014000bad0 con 0x7f0148159ec0 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f01467fc700 1 -- 172.21.15.92:0/3775502667 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400092e0 con 0x7f0148159ec0 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 -- 172.21.15.92:0/3775502667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 msgr2=0x7f01480b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 --2- 172.21.15.92:0/3775502667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f01480b4e90 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f01480b64d0 tx=0x7f0140009840 comp rx=0 tx=0).stop 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 -- 172.21.15.92:0/3775502667 shutdown_connections 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 --2- 172.21.15.92:0/3775502667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480cce40 0x7f01480cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 --2- 172.21.15.92:0/3775502667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f01480b4e90 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 --2- 172.21.15.92:0/3775502667 >> v2:172.21.15.136:3300/0 conn(0x7f01480b53d0 0x7f01480cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 -- 172.21.15.92:0/3775502667 >> 172.21.15.92:0/3775502667 conn(0x7f01480999f0 msgr2=0x7f01480b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 -- 172.21.15.92:0/3775502667 shutdown_connections 2023-10-06T03:10:26.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.397+0000 7f014dc81700 1 -- 172.21.15.92:0/3775502667 wait complete. 2023-10-06T03:10:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 Processor -- start 2023-10-06T03:10:26.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 -- start start 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480b53d0 0x7f014814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480cce40 0x7f01480c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f0148154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014814e820 con 0x7f0148159ec0 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0148069250 con 0x7f01480cce40 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f014dc81700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0148068f00 con 0x7f01480b53d0 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.399+0000 7f0147fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f0148154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f0148154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36262/0 (socket says 172.21.15.92:36262) 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 -- 172.21.15.92:0/2270012822 learned_addr learned my addr 172.21.15.92:0/2270012822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:26.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f01477fe700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480b53d0 0x7f014814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0146ffd700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480cce40 0x7f01480c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 -- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480b53d0 msgr2=0x7f014814e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480b53d0 0x7f014814e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 -- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480cce40 msgr2=0x7f01480c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480cce40 0x7f01480c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 -- 172.21.15.92:0/2270012822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009ae0 con 0x7f0148159ec0 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f01477fe700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480b53d0 0x7f014814e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0146ffd700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480cce40 0x7f01480c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:26.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f0147fff700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f0148154410 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f013800b5b0 tx=0x7f01380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380059b0 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138005b10 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01480c93f0 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138006030 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.400+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01480c99b0 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.401+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01480718b0 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.401+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0138006460 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.402+0000 7f013f7fe700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0128067350 0x7f0128069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.402+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0138084520 con 0x7f0148159ec0 2023-10-06T03:10:26.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.402+0000 7f01477fe700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0128067350 0x7f0128069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.402+0000 7f01477fe700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0128067350 0x7f0128069800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0140009fd0 tx=0x7f0140018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:26.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.406+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0138050e30 con 0x7f0148159ec0 2023-10-06T03:10:26.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:26.590+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000126469.meta", "pool2": "cephfs.volume_0000000000126469.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0148063c40 con 0x7f0148159ec0 2023-10-06T03:10:27.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.262+0000 7f013f7fe700 1 -- 172.21.15.92:0/2270012822 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000126469.meta", "pool2": "cephfs.volume_0000000000126469.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000126469.meta' removed v175) v1 ==== 241+0+0 (secure 0 0 0) 0x7f0138054450 con 0x7f0148159ec0 2023-10-06T03:10:27.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0128067350 msgr2=0x7f0128069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0128067350 0x7f0128069800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0140009fd0 tx=0x7f0140018000 comp rx=0 tx=0).stop 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 msgr2=0x7f0148154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f0148154410 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f013800b5b0 tx=0x7f01380091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 shutdown_connections 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0128067350 0x7f0128069800 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0148159ec0 0x7f0148154410 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01480b53d0 0x7f014814e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 --2- 172.21.15.92:0/2270012822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480cce40 0x7f01480c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 >> 172.21.15.92:0/2270012822 conn(0x7f01480999f0 msgr2=0x7f01480cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:27.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 shutdown_connections 2023-10-06T03:10:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.265+0000 7f014dc81700 1 -- 172.21.15.92:0/2270012822 wait complete. 2023-10-06T03:10:27.267 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs.volume_0000000000126469.meta' removed 2023-10-06T03:10:27.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:10:27.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.397+0000 7f78fd685700 1 Processor -- start 2023-10-06T03:10:27.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.397+0000 7f78fd685700 1 -- start start 2023-10-06T03:10:27.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78fd685700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78f8159ec0 0x7f78f80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78fd685700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f78f80b53d0 0x7f78f80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78fd685700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 0x7f78f80cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78fd685700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f80b5b00 con 0x7f78f8159ec0 2023-10-06T03:10:27.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78fd685700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f80b5850 con 0x7f78f80b53d0 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78fd685700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80b5da0 con 0x7f78f80cce40 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78f8159ec0 0x7f78f80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f78f8159ec0 0x7f78f80b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36276/0 (socket says 172.21.15.92:36276) 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 -- 172.21.15.92:0/4238643846 learned_addr learned my addr 172.21.15.92:0/4238643846 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 -- 172.21.15.92:0/4238643846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 msgr2=0x7f78f80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 --2- 172.21.15.92:0/4238643846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 0x7f78f80cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 -- 172.21.15.92:0/4238643846 >> v2:172.21.15.136:3300/0 conn(0x7f78f80b53d0 msgr2=0x7f78f80cc900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 --2- 172.21.15.92:0/4238643846 >> v2:172.21.15.136:3300/0 conn(0x7f78f80b53d0 0x7f78f80cc900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.398+0000 7f78f6ffd700 1 -- 172.21.15.92:0/4238643846 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80b6080 con 0x7f78f8159ec0 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78f6ffd700 1 --2- 172.21.15.92:0/4238643846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8159ec0 0x7f78f80b4e90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f78f80b64d0 tx=0x7f78e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bc480bc34e601f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:27.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78f67fc700 1 -- 172.21.15.92:0/4238643846 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e800e070 con 0x7f78f8159ec0 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78f67fc700 1 -- 172.21.15.92:0/4238643846 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e800bad0 con 0x7f78f8159ec0 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78f67fc700 1 -- 172.21.15.92:0/4238643846 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e80092e0 con 0x7f78f8159ec0 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 -- 172.21.15.92:0/4238643846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8159ec0 msgr2=0x7f78f80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 --2- 172.21.15.92:0/4238643846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8159ec0 0x7f78f80b4e90 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f78f80b64d0 tx=0x7f78e8009840 comp rx=0 tx=0).stop 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 -- 172.21.15.92:0/4238643846 shutdown_connections 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 --2- 172.21.15.92:0/4238643846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 0x7f78f80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 --2- 172.21.15.92:0/4238643846 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f8159ec0 0x7f78f80b4e90 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 --2- 172.21.15.92:0/4238643846 >> v2:172.21.15.136:3300/0 conn(0x7f78f80b53d0 0x7f78f80cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 -- 172.21.15.92:0/4238643846 >> 172.21.15.92:0/4238643846 conn(0x7f78f80999f0 msgr2=0x7f78f80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 -- 172.21.15.92:0/4238643846 shutdown_connections 2023-10-06T03:10:27.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.399+0000 7f78fd685700 1 -- 172.21.15.92:0/4238643846 wait complete. 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.400+0000 7f78fd685700 1 Processor -- start 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 -- start start 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 0x7f78f81448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 0x7f78f809b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f8159ec0 0x7f78f809d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f809c7c0 con 0x7f78f80b53d0 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f809bbb0 con 0x7f78f8159ec0 2023-10-06T03:10:27.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78fd685700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f809be90 con 0x7f78f80cce40 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78f6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 0x7f78f81448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78f6ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 0x7f78f81448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36282/0 (socket says 172.21.15.92:36282) 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.401+0000 7f78f6ffd700 1 -- 172.21.15.92:0/3174284423 learned_addr learned my addr 172.21.15.92:0/3174284423 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78f6ffd700 1 -- 172.21.15.92:0/3174284423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 msgr2=0x7f78f809b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78f6ffd700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 0x7f78f809b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78f6ffd700 1 -- 172.21.15.92:0/3174284423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f8159ec0 msgr2=0x7f78f809d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78f6ffd700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f8159ec0 0x7f78f809d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78f6ffd700 1 -- 172.21.15.92:0/3174284423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8009ae0 con 0x7f78f80b53d0 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78f6ffd700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 0x7f78f81448f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f78f8089390 tx=0x7f78e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8008bb0 con 0x7f78f80b53d0 2023-10-06T03:10:27.406 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8008d30 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f809c170 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e8008ea0 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.402+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f8069c30 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.403+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f8063c90 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.403+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f78e8003040 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.403+0000 7f78ef7fe700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78d8067320 0x7f78d80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.404+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f78e8086a60 con 0x7f78f80b53d0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.404+0000 7f78ee7fc700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78d8067320 0x7f78d80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:27.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.404+0000 7f78ee7fc700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78d8067320 0x7f78d80697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f78f815f6b0 tx=0x7f78e0006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:27.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.408+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78e8053430 con 0x7f78f80b53d0 2023-10-06T03:10:27.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:27.591+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f78f80718b0 con 0x7f78f80b53d0 2023-10-06T03:10:28.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.554+0000 7f78ef7fe700 1 -- 172.21.15.92:0/3174284423 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f78e80220d0 con 0x7f78f80b53d0 2023-10-06T03:10:28.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78d8067320 msgr2=0x7f78d80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78d8067320 0x7f78d80697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f78f815f6b0 tx=0x7f78e0006c60 comp rx=0 tx=0).stop 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 msgr2=0x7f78f81448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 0x7f78f81448f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f78f8089390 tx=0x7f78e8018000 comp rx=0 tx=0).stop 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 shutdown_connections 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f78d8067320 0x7f78d80697d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f78f80b53d0 0x7f78f81448f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78f8159ec0 0x7f78f809d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 --2- 172.21.15.92:0/3174284423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78f80cce40 0x7f78f809b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 >> 172.21.15.92:0/3174284423 conn(0x7f78f80999f0 msgr2=0x7f78f80c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:28.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 shutdown_connections 2023-10-06T03:10:28.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.558+0000 7f78fd685700 1 -- 172.21.15.92:0/3174284423 wait complete. 2023-10-06T03:10:28.560 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:10:28.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm another-data-pool another-data-pool --yes-i-really-really-mean-it 2023-10-06T03:10:28.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.703+0000 7fa5b22c0700 1 Processor -- start 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5b22c0700 1 -- start start 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5b22c0700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5ac159ec0 0x7fa5ac0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5b22c0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa5ac0cf350 0x7fa5ac0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5b22c0700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac0b5b00 con 0x7fa5ac0b53d0 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5b22c0700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0b5850 con 0x7fa5ac159ec0 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5b22c0700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac0b5da0 con 0x7fa5ac0cf350 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5abfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa5ac159ec0 0x7fa5ac0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5ab7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa5ac0b53d0 0x7fa5ac0cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37960/0 (socket says 172.21.15.92:37960) 2023-10-06T03:10:28.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.704+0000 7fa5ab7fe700 1 -- 172.21.15.92:0/3299858807 learned_addr learned my addr 172.21.15.92:0/3299858807 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5b085d700 1 --2- 172.21.15.92:0/3299858807 >> v2:172.21.15.136:3300/0 conn(0x7fa5ac0cf350 0x7fa5ac0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5abfff700 1 -- 172.21.15.92:0/3299858807 >> v2:172.21.15.136:3300/0 conn(0x7fa5ac0cf350 msgr2=0x7fa5ac0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5abfff700 1 --2- 172.21.15.92:0/3299858807 >> v2:172.21.15.136:3300/0 conn(0x7fa5ac0cf350 0x7fa5ac0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5abfff700 1 -- 172.21.15.92:0/3299858807 >> v1:172.21.15.92:6789/0 conn(0x7fa5ac0b53d0 legacy=0x7fa5ac0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5aaffd700 1 -- 172.21.15.92:0/3299858807 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302423298 0 0) 0x7fa5ac0b5b00 con 0x7fa5ac0b53d0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5abfff700 1 -- 172.21.15.92:0/3299858807 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0b6080 con 0x7fa5ac159ec0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5abfff700 1 --2- 172.21.15.92:0/3299858807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac159ec0 0x7fa5ac0b4e90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0cb4a0 tx=0x7fa59c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6adc2f600fe5879b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5aaffd700 1 -- 172.21.15.92:0/3299858807 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c005c10 con 0x7fa5ac159ec0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5aaffd700 1 -- 172.21.15.92:0/3299858807 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c005d70 con 0x7fa5ac159ec0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.705+0000 7fa5aaffd700 1 -- 172.21.15.92:0/3299858807 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c006260 con 0x7fa5ac159ec0 2023-10-06T03:10:28.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3299858807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac159ec0 msgr2=0x7fa5ac0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3299858807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac159ec0 0x7fa5ac0b4e90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0cb4a0 tx=0x7fa59c009120 comp rx=0 tx=0).stop 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3299858807 shutdown_connections 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3299858807 >> v2:172.21.15.136:3300/0 conn(0x7fa5ac0cf350 0x7fa5ac0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3299858807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac159ec0 0x7fa5ac0b4e90 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3299858807 >> 172.21.15.92:0/3299858807 conn(0x7fa5ac0999f0 msgr2=0x7fa5ac0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3299858807 shutdown_connections 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.706+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3299858807 wait complete. 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.707+0000 7fa5b22c0700 1 Processor -- start 2023-10-06T03:10:28.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.707+0000 7fa5b22c0700 1 -- start start 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5ac0b53d0 0x7fa5ac09a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 0x7fa5ac09c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5ac159ec0 0x7fa5ac156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac09b850 con 0x7fa5ac0cf350 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac09a9a0 con 0x7fa5ac0b53d0 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac09ac40 con 0x7fa5ac159ec0 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 0x7fa5ac09c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 0x7fa5ac09c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36292/0 (socket says 172.21.15.92:36292) 2023-10-06T03:10:28.713 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 -- 172.21.15.92:0/3144126374 learned_addr learned my addr 172.21.15.92:0/3144126374 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5abfff700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5ac0b53d0 0x7fa5ac09a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b085d700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5ac159ec0 0x7fa5ac156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 -- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5ac159ec0 msgr2=0x7fa5ac156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5ac159ec0 0x7fa5ac156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 -- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5ac0b53d0 msgr2=0x7fa5ac09a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5ac0b53d0 0x7fa5ac09a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 -- 172.21.15.92:0/3144126374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c0093c0 con 0x7fa5ac0cf350 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5ab7fe700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 0x7fa5ac09c260 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c5b0 tx=0x7fa5a0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0005b30 con 0x7fa5ac0cf350 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0005c90 con 0x7fa5ac0cf350 2023-10-06T03:10:28.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac09af20 con 0x7fa5ac0cf350 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0007300 con 0x7fa5ac0cf350 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.708+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac09b4e0 con 0x7fa5ac0cf350 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.709+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac063c90 con 0x7fa5ac0cf350 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.710+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa5a001b3b0 con 0x7fa5ac0cf350 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.710+0000 7fa593fff700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa598067320 0x7fa5980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.710+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5713+0+0 (secure 0 0 0) 0x7fa5a0084160 con 0x7fa5ac0cf350 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.710+0000 7fa5abfff700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa598067320 0x7fa5980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.710+0000 7fa5abfff700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa598067320 0x7fa5980697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009160 tx=0x7fa59c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:28.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.714+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5a0050de0 con 0x7fa5ac0cf350 2023-10-06T03:10:28.899 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:28.898+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5ac0718b0 con 0x7fa5ac0cf350 2023-10-06T03:10:29.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.559+0000 7fa593fff700 1 -- 172.21.15.92:0/3144126374 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true}]=0 pool 'another-data-pool' removed v177) v1 ==== 187+0+0 (secure 0 0 0) 0x7fa5a0054400 con 0x7fa5ac0cf350 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa598067320 msgr2=0x7fa5980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa598067320 0x7fa5980697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009160 tx=0x7fa59c016000 comp rx=0 tx=0).stop 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 msgr2=0x7fa5ac09c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 0x7fa5ac09c260 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c5b0 tx=0x7fa5a0001070 comp rx=0 tx=0).stop 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 shutdown_connections 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa598067320 0x7fa5980697d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5ac159ec0 0x7fa5ac156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5ac0b53d0 0x7fa5ac09a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 --2- 172.21.15.92:0/3144126374 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa5ac0cf350 0x7fa5ac09c260 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 >> 172.21.15.92:0/3144126374 conn(0x7fa5ac0999f0 msgr2=0x7fa5ac159020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 shutdown_connections 2023-10-06T03:10:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.562+0000 7fa5b22c0700 1 -- 172.21.15.92:0/3144126374 wait complete. 2023-10-06T03:10:29.564 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'another-data-pool' removed 2023-10-06T03:10:29.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:10:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.692+0000 7f4a29046700 1 Processor -- start 2023-10-06T03:10:29.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.692+0000 7f4a29046700 1 -- start start 2023-10-06T03:10:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a29046700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 0x7f4a240ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a29046700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4a240b16d0 0x7f4a240b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a29046700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240b2080 con 0x7f4a2415d240 2023-10-06T03:10:29.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a29046700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240cfb00 con 0x7f4a240b16d0 2023-10-06T03:10:29.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a29046700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240b5c40 con 0x7f4a240b4e70 2023-10-06T03:10:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a28845700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4a2415d240 0x7f4a2415d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47126/0 (socket says 172.21.15.92:47126) 2023-10-06T03:10:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a28845700 1 -- 172.21.15.92:0/2757953685 learned_addr learned my addr 172.21.15.92:0/2757953685 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028181101 0 0) 0x7f4a240b2080 con 0x7f4a2415d240 2023-10-06T03:10:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.693+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a0c003570 con 0x7f4a2415d240 2023-10-06T03:10:29.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a237fe700 1 --2- 172.21.15.92:0/2757953685 >> v2:172.21.15.136:3300/0 conn(0x7f4a240b16d0 0x7f4a240b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 0x7f4a240ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3867794010 0 0) 0x7f4a0c003570 con 0x7f4a2415d240 2023-10-06T03:10:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 msgr2=0x7f4a240ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 0x7f4a240ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 >> v2:172.21.15.136:3300/0 conn(0x7f4a240b16d0 msgr2=0x7f4a240b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 --2- 172.21.15.92:0/2757953685 >> v2:172.21.15.136:3300/0 conn(0x7f4a240b16d0 0x7f4a240b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240b5f20 con 0x7f4a2415d240 2023-10-06T03:10:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f4a1c0017f0 con 0x7f4a2415d240 2023-10-06T03:10:29.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 >> v1:172.21.15.92:6789/0 conn(0x7f4a2415d240 legacy=0x7f4a2415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a0c003a60 0x7f4a0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a0c004810 0x7f4a0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 0x7f4a0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cfb00 con 0x7f4a0c003a60 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240b5c40 con 0x7f4a0c004810 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c00bb90 con 0x7f4a240b4e70 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a0c003a60 0x7f4a0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a28845700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 0x7f4a0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 -- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 msgr2=0x7f4a0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a240b4e70 0x7f4a0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 -- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a0c004810 msgr2=0x7f4a0c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:10:29.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a0c004810 0x7f4a0c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 -- 172.21.15.92:0/2757953685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c003570 con 0x7f4a0c003a60 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a23fff700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a0c003a60 0x7f4a0c003e70 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400bb90 tx=0x7f4a14008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a637f0baace7f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400d5d0 con 0x7f4a0c003a60 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1400d730 con 0x7f4a0c003a60 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.694+0000 7f4a22ffd700 1 -- 172.21.15.92:0/2757953685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1400daf0 con 0x7f4a0c003a60 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a29046700 1 -- 172.21.15.92:0/2757953685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a0c003a60 msgr2=0x7f4a0c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a29046700 1 --2- 172.21.15.92:0/2757953685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a0c003a60 0x7f4a0c003e70 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400bb90 tx=0x7f4a14008040 comp rx=0 tx=0).stop 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a28845700 1 -- 172.21.15.92:0/2757953685 reap_dead start 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a29046700 1 -- 172.21.15.92:0/2757953685 shutdown_connections 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a29046700 1 -- 172.21.15.92:0/2757953685 >> 172.21.15.92:0/2757953685 conn(0x7f4a240d10c0 msgr2=0x7f4a240affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a29046700 1 -- 172.21.15.92:0/2757953685 shutdown_connections 2023-10-06T03:10:29.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.695+0000 7f4a29046700 1 -- 172.21.15.92:0/2757953685 wait complete. 2023-10-06T03:10:29.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.696+0000 7f4a29046700 1 Processor -- start 2023-10-06T03:10:29.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.696+0000 7f4a29046700 1 -- start start 2023-10-06T03:10:29.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a29046700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a241459c0 0x7f4a2415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a29046700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 0x7f4a2415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a29046700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a2416d770 0x7f4a2416dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a29046700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c8b30 con 0x7f4a24145d90 2023-10-06T03:10:29.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a29046700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c8e10 con 0x7f4a2416d770 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a29046700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c90f0 con 0x7f4a241459c0 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 0x7f4a2415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 0x7f4a2415e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47702/0 (socket says 172.21.15.92:47702) 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 -- 172.21.15.92:0/1473475099 learned_addr learned my addr 172.21.15.92:0/1473475099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 -- 172.21.15.92:0/1473475099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a241459c0 msgr2=0x7f4a2415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a241459c0 0x7f4a2415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 -- 172.21.15.92:0/1473475099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a2416d770 msgr2=0x7f4a2416dc10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a2416d770 0x7f4a2416dc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.697+0000 7f4a237fe700 1 -- 172.21.15.92:0/1473475099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240c93d0 con 0x7f4a24145d90 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.698+0000 7f4a237fe700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 0x7f4a2415e120 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800d890 tx=0x7f4a18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:29.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.698+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180125f0 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.698+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18012770 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.698+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a240c96b0 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.698+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18012b30 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.698+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a240c73a0 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.699+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a240c98d0 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.699+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4a1801e460 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.700+0000 7f4a0bfff700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a10067320 0x7f4a100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.700+0000 7f4a23fff700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a10067320 0x7f4a100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.700+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4a180850c0 con 0x7f4a24145d90 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.700+0000 7f4a23fff700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a10067320 0x7f4a100697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400bfd0 tx=0x7f4a1401b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:29.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.704+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a180b65a0 con 0x7f4a24145d90 2023-10-06T03:10:29.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.886+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4a2406a160 con 0x7f4a24145d90 2023-10-06T03:10:29.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.890+0000 7f4a0bfff700 1 -- 172.21.15.92:0/1473475099 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4a18051eb0 con 0x7f4a24145d90 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a10067320 msgr2=0x7f4a100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a10067320 0x7f4a100697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400bfd0 tx=0x7f4a1401b000 comp rx=0 tx=0).stop 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 msgr2=0x7f4a2415e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 0x7f4a2415e120 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800d890 tx=0x7f4a18008040 comp rx=0 tx=0).stop 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 shutdown_connections 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a10067320 0x7f4a100697d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a2416d770 0x7f4a2416dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a241459c0 0x7f4a2415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 --2- 172.21.15.92:0/1473475099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a24145d90 0x7f4a2415e120 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:29.893 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 >> 172.21.15.92:0/1473475099 conn(0x7f4a240d10c0 msgr2=0x7f4a24159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 shutdown_connections 2023-10-06T03:10:29.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:29.892+0000 7f4a29046700 1 -- 172.21.15.92:0/1473475099 wait complete. 2023-10-06T03:10:29.904 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-06T03:10:30.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.021+0000 7facff33f700 1 Processor -- start 2023-10-06T03:10:30.022 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.021+0000 7facff33f700 1 -- start start 2023-10-06T03:10:30.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.022+0000 7facff33f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf806bcd0 0x7facf80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7facff33f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf815d7e0 con 0x7facf806c0a0 2023-10-06T03:10:30.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7facff33f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf815dac0 con 0x7facf815df50 2023-10-06T03:10:30.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7facff33f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf815aa30 con 0x7facf806bcd0 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7facfc8da700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7facf806c0a0 0x7facf815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47134/0 (socket says 172.21.15.92:47134) 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7facfc8da700 1 -- 172.21.15.92:0/3441577698 learned_addr learned my addr 172.21.15.92:0/3441577698 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7facfd0db700 1 --2- 172.21.15.92:0/3441577698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf806bcd0 0x7facf80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.023+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923942666 0 0) 0x7facf815dac0 con 0x7facf815df50 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face4003570 con 0x7facf815df50 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105624708 0 0) 0x7facf815d7e0 con 0x7facf806c0a0 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf815dac0 con 0x7facf806c0a0 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7facfd0db700 1 -- 172.21.15.92:0/3441577698 >> v1:172.21.15.136:6789/0 conn(0x7facf815df50 legacy=0x7facf80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7facfd0db700 1 -- 172.21.15.92:0/3441577698 >> v1:172.21.15.92:6789/0 conn(0x7facf806c0a0 legacy=0x7facf815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1502045914 0 0) 0x7facf815dac0 con 0x7facf806c0a0 2023-10-06T03:10:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7facfd0db700 1 -- 172.21.15.92:0/3441577698 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf815ad10 con 0x7facf806bcd0 2023-10-06T03:10:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7facfd0db700 1 --2- 172.21.15.92:0/3441577698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf806bcd0 0x7facf80caa90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7facf4009910 tx=0x7facf40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849232e3224f3225 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face80013f0 con 0x7facf806bcd0 2023-10-06T03:10:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4011420 con 0x7facf806bcd0 2023-10-06T03:10:30.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.024+0000 7faceffff700 1 -- 172.21.15.92:0/3441577698 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40117c0 con 0x7facf806bcd0 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 -- 172.21.15.92:0/3441577698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf806bcd0 msgr2=0x7facf80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 --2- 172.21.15.92:0/3441577698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf806bcd0 0x7facf80caa90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7facf4009910 tx=0x7facf40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 -- 172.21.15.92:0/3441577698 shutdown_connections 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 --2- 172.21.15.92:0/3441577698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf806bcd0 0x7facf80caa90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 -- 172.21.15.92:0/3441577698 >> 172.21.15.92:0/3441577698 conn(0x7facf80ce600 msgr2=0x7facf80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 -- 172.21.15.92:0/3441577698 shutdown_connections 2023-10-06T03:10:30.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.025+0000 7facff33f700 1 -- 172.21.15.92:0/3441577698 wait complete. 2023-10-06T03:10:30.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.026+0000 7facff33f700 1 Processor -- start 2023-10-06T03:10:30.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.026+0000 7facff33f700 1 -- start start 2023-10-06T03:10:30.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facff33f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf806bcd0 0x7facf80b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facff33f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 0x7facf8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facff33f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf815df50 0x7facf80c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facff33f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b3760 con 0x7facf806c0a0 2023-10-06T03:10:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facff33f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80c6500 con 0x7facf806bcd0 2023-10-06T03:10:30.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facff33f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b2d90 con 0x7facf815df50 2023-10-06T03:10:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facfc8da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 0x7facf8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facfc8da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 0x7facf8155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47710/0 (socket says 172.21.15.92:47710) 2023-10-06T03:10:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facfc8da700 1 -- 172.21.15.92:0/621195928 learned_addr learned my addr 172.21.15.92:0/621195928 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:30.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facfd0db700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf806bcd0 0x7facf80b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.027+0000 7facfd8dc700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf815df50 0x7facf80c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfc8da700 1 -- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf815df50 msgr2=0x7facf80c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfc8da700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf815df50 0x7facf80c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfc8da700 1 -- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf806bcd0 msgr2=0x7facf80b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfc8da700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf806bcd0 0x7facf80b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfc8da700 1 -- 172.21.15.92:0/621195928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4009420 con 0x7facf806c0a0 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfd0db700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf806bcd0 0x7facf80b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facfc8da700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 0x7facf8155aa0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7face800cb90 tx=0x7face8001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8007150 con 0x7facf806c0a0 2023-10-06T03:10:30.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face80072b0 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf80b2460 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8007670 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.028+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf80b2a20 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.029+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf804c3c0 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.029+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7face8004a40 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.030+0000 7faced7fa700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facdc067320 0x7facdc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.030+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5292+0+0 (secure 0 0 0) 0x7face8084220 con 0x7facf806c0a0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.030+0000 7facfd0db700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facdc067320 0x7facdc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.030+0000 7facfd0db700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facdc067320 0x7facdc0697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7facf40030c0 tx=0x7facf4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:30.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.034+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7face80510d0 con 0x7facf806c0a0 2023-10-06T03:10:30.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.296+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7facf806c9d0 con 0x7facf806c0a0 2023-10-06T03:10:30.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.560+0000 7faced7fa700 1 -- 172.21.15.92:0/621195928 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v453) v1 ==== 163+0+0 (secure 0 0 0) 0x7face80546f0 con 0x7facf806c0a0 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facdc067320 msgr2=0x7facdc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facdc067320 0x7facdc0697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7facf40030c0 tx=0x7facf4016000 comp rx=0 tx=0).stop 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 msgr2=0x7facf8155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 0x7facf8155aa0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7face800cb90 tx=0x7face8001a60 comp rx=0 tx=0).stop 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 shutdown_connections 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7facdc067320 0x7facdc0697d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facf815df50 0x7facf80c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facf806bcd0 0x7facf80b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 --2- 172.21.15.92:0/621195928 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7facf806c0a0 0x7facf8155aa0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 >> 172.21.15.92:0/621195928 conn(0x7facf80ce600 msgr2=0x7facf8157530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 shutdown_connections 2023-10-06T03:10:30.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.563+0000 7facff33f700 1 -- 172.21.15.92:0/621195928 wait complete. 2023-10-06T03:10:30.576 INFO:tasks.cephfs_test_runner:That renaming a volume with more than one data pool does not change ... ok 2023-10-06T03:10:30.576 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) That volume rename is idempotent. 2023-10-06T03:10:30.577 INFO:tasks.cephfs_test_runner:test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:10:30.577 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-06T03:10:30.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.691+0000 7f95d186d700 1 Processor -- start 2023-10-06T03:10:30.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.691+0000 7f95d186d700 1 -- start start 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d186d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f95cc15d230 0x7f95cc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d186d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc0b2980 con 0x7f95cc159420 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d186d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc15db70 con 0x7f95cc1597f0 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d186d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc0b11d0 con 0x7f95cc15d230 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d086b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f95cc159420 0x7f95cc0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47138/0 (socket says 172.21.15.92:47138) 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d086b700 1 -- 172.21.15.92:0/799390689 learned_addr learned my addr 172.21.15.92:0/799390689 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:30.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d106c700 1 --2- 172.21.15.92:0/799390689 >> v2:172.21.15.92:3300/0 conn(0x7f95cc15d230 0x7f95cc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d106c700 1 -- 172.21.15.92:0/799390689 >> v1:172.21.15.136:6789/0 conn(0x7f95cc1597f0 legacy=0x7f95cc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d106c700 1 -- 172.21.15.92:0/799390689 >> v1:172.21.15.92:6789/0 conn(0x7f95cc159420 legacy=0x7f95cc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.692+0000 7f95d106c700 1 -- 172.21.15.92:0/799390689 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0b14b0 con 0x7f95cc15d230 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d106c700 1 --2- 172.21.15.92:0/799390689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc15d230 0x7f95cc15d630 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f95c40098e0 tx=0x7f95c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89556f106abe350 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95c37fe700 1 -- 172.21.15.92:0/799390689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4009e80 con 0x7f95cc15d230 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95c37fe700 1 -- 172.21.15.92:0/799390689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c400b820 con 0x7f95cc15d230 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95c37fe700 1 -- 172.21.15.92:0/799390689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c400bba0 con 0x7f95cc15d230 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 -- 172.21.15.92:0/799390689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc15d230 msgr2=0x7f95cc15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 --2- 172.21.15.92:0/799390689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc15d230 0x7f95cc15d630 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f95c40098e0 tx=0x7f95c40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 -- 172.21.15.92:0/799390689 shutdown_connections 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 --2- 172.21.15.92:0/799390689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc15d230 0x7f95cc15d630 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 -- 172.21.15.92:0/799390689 >> 172.21.15.92:0/799390689 conn(0x7f95cc0d0660 msgr2=0x7f95cc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:30.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 -- 172.21.15.92:0/799390689 shutdown_connections 2023-10-06T03:10:30.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.693+0000 7f95d186d700 1 -- 172.21.15.92:0/799390689 wait complete. 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.694+0000 7f95d186d700 1 Processor -- start 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 -- start start 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc159420 0x7f95cc15a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 0x7f95cc15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc15d230 0x7f95cc0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc0af7f0 con 0x7f95cc1597f0 2023-10-06T03:10:30.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc155700 con 0x7f95cc15d230 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95d186d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc15b5a0 con 0x7f95cc159420 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 0x7f95cc15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95c3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 0x7f95cc15c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47714/0 (socket says 172.21.15.92:47714) 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.695+0000 7f95c3fff700 1 -- 172.21.15.92:0/1310703685 learned_addr learned my addr 172.21.15.92:0/1310703685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c3fff700 1 -- 172.21.15.92:0/1310703685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc159420 msgr2=0x7f95cc15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c3fff700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc159420 0x7f95cc15a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c3fff700 1 -- 172.21.15.92:0/1310703685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc15d230 msgr2=0x7f95cc0c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c3fff700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc15d230 0x7f95cc0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c3fff700 1 -- 172.21.15.92:0/1310703685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009420 con 0x7f95cc1597f0 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c3fff700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 0x7f95cc15c850 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b810 tx=0x7f95bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:30.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00f510 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc00f670 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00fa10 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc15a990 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.696+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc15af50 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.697+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc04c3c0 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.697+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f95bc01a400 con 0x7f95cc1597f0 2023-10-06T03:10:30.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.698+0000 7f95c0ff9700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95b8067320 0x7f95b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:30.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.698+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f95bc084560 con 0x7f95cc1597f0 2023-10-06T03:10:30.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.698+0000 7f95d086b700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95b8067320 0x7f95b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:30.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.698+0000 7f95d086b700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95b8067320 0x7f95b80697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f95c80020e0 tx=0x7f95c8001690 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:30.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.702+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95bc054f20 con 0x7f95cc1597f0 2023-10-06T03:10:30.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:30.962+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7f95cc0b1790 con 0x7f95cc1597f0 2023-10-06T03:10:31.568 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.567+0000 7f95c0ff9700 1 -- 172.21.15.92:0/1310703685 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v454) v1 ==== 150+0+0 (secure 0 0 0) 0x7f95bc054ab0 con 0x7f95cc1597f0 2023-10-06T03:10:31.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95b8067320 msgr2=0x7f95b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95b8067320 0x7f95b80697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f95c80020e0 tx=0x7f95c8001690 comp rx=0 tx=0).stop 2023-10-06T03:10:31.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 msgr2=0x7f95cc15c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 0x7f95cc15c850 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00b810 tx=0x7f95bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 shutdown_connections 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f95b8067320 0x7f95b80697d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc15d230 0x7f95cc0c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc159420 0x7f95cc15a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 --2- 172.21.15.92:0/1310703685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f95cc1597f0 0x7f95cc15c850 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 >> 172.21.15.92:0/1310703685 conn(0x7f95cc0d0660 msgr2=0x7f95cc0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 shutdown_connections 2023-10-06T03:10:31.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.569+0000 7f95d186d700 1 -- 172.21.15.92:0/1310703685 wait complete. 2023-10-06T03:10:31.583 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:10:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.700+0000 7efca96ce700 1 Processor -- start 2023-10-06T03:10:31.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.700+0000 7efca96ce700 1 -- start start 2023-10-06T03:10:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca96ce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca96ce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7efca415db10 0x7efca4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca96ce700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca415a160 con 0x7efca4159890 2023-10-06T03:10:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca96ce700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca40d13a0 con 0x7efca415db10 2023-10-06T03:10:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca96ce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca415e140 con 0x7efca4159c60 2023-10-06T03:10:31.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7efca4159890 0x7efca415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:47152/0 (socket says 172.21.15.92:47152) 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca2ffd700 1 -- 172.21.15.92:0/1006546141 learned_addr learned my addr 172.21.15.92:0/1006546141 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.701+0000 7efca37fe700 1 --2- 172.21.15.92:0/1006546141 >> v2:172.21.15.92:3300/0 conn(0x7efca415db10 0x7efca4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca27fc700 1 -- 172.21.15.92:0/1006546141 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77060238 0 0) 0x7efca415a160 con 0x7efca4159890 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efc9a7fc700 1 --2- 172.21.15.92:0/1006546141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca27fc700 1 -- 172.21.15.92:0/1006546141 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc90003570 con 0x7efca4159890 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca37fe700 1 -- 172.21.15.92:0/1006546141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 msgr2=0x7efca415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca37fe700 1 --2- 172.21.15.92:0/1006546141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca37fe700 1 -- 172.21.15.92:0/1006546141 >> v1:172.21.15.92:6789/0 conn(0x7efca4159890 legacy=0x7efca415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca37fe700 1 -- 172.21.15.92:0/1006546141 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca415a3d0 con 0x7efca415db10 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca37fe700 1 --2- 172.21.15.92:0/1006546141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca415db10 0x7efca4069560 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e60 tx=0x7efc9c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5c8c31387ff5b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca27fc700 1 -- 172.21.15.92:0/1006546141 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00e070 con 0x7efca415db10 2023-10-06T03:10:31.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca27fc700 1 -- 172.21.15.92:0/1006546141 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c00bb60 con 0x7efca415db10 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.702+0000 7efca27fc700 1 -- 172.21.15.92:0/1006546141 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c012430 con 0x7efca415db10 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 -- 172.21.15.92:0/1006546141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca415db10 msgr2=0x7efca4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 --2- 172.21.15.92:0/1006546141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca415db10 0x7efca4069560 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009e60 tx=0x7efc9c009780 comp rx=0 tx=0).stop 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 -- 172.21.15.92:0/1006546141 shutdown_connections 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 --2- 172.21.15.92:0/1006546141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca415db10 0x7efca4069560 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 --2- 172.21.15.92:0/1006546141 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 -- 172.21.15.92:0/1006546141 >> 172.21.15.92:0/1006546141 conn(0x7efca4068e20 msgr2=0x7efca40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:31.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 -- 172.21.15.92:0/1006546141 shutdown_connections 2023-10-06T03:10:31.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.703+0000 7efca96ce700 1 -- 172.21.15.92:0/1006546141 wait complete. 2023-10-06T03:10:31.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.704+0000 7efca96ce700 1 Processor -- start 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.704+0000 7efca96ce700 1 -- start start 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 0x7efca40cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca4151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca415db10 0x7efca4153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40b05c0 con 0x7efca4159890 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40b08a0 con 0x7efca415db10 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca40cd160 con 0x7efca4159c60 2023-10-06T03:10:31.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 0x7efca40cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 0x7efca40cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47724/0 (socket says 172.21.15.92:47724) 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 -- 172.21.15.92:0/803211384 learned_addr learned my addr 172.21.15.92:0/803211384 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efc9a7fc700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca4151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca37fe700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca415db10 0x7efca4153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 -- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 msgr2=0x7efca4151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca4151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 -- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca415db10 msgr2=0x7efca4153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca415db10 0x7efca4153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 -- 172.21.15.92:0/803211384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c009a20 con 0x7efca4159890 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca37fe700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca415db10 0x7efca4153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca2ffd700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 0x7efca40cf850 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c7c0 tx=0x7efc94007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94005b30 con 0x7efca4159890 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc94005c90 con 0x7efca4159890 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca40cd440 con 0x7efca4159890 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94007340 con 0x7efca4159890 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.705+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40cfd90 con 0x7efca4159890 2023-10-06T03:10:31.712 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.706+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40cd8e0 con 0x7efca4159890 2023-10-06T03:10:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.707+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efc9401b400 con 0x7efca4159890 2023-10-06T03:10:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.707+0000 7efc9b7fe700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc84067320 0x7efc840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.707+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5292+0+0 (secure 0 0 0) 0x7efc94084560 con 0x7efca4159890 2023-10-06T03:10:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.707+0000 7efc9a7fc700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc84067320 0x7efc840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.707+0000 7efc9a7fc700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc84067320 0x7efc840697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efca40c6f30 tx=0x7efc8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:31.714 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.711+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efca40cd8e0 con 0x7efca4159890 2023-10-06T03:10:31.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.893+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efca40cd8e0 con 0x7efca4159890 2023-10-06T03:10:31.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.897+0000 7efc9b7fe700 1 -- 172.21.15.92:0/803211384 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7efca40cd8e0 con 0x7efca4159890 2023-10-06T03:10:31.900 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc84067320 msgr2=0x7efc840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc84067320 0x7efc840697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efca40c6f30 tx=0x7efc8c008040 comp rx=0 tx=0).stop 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 msgr2=0x7efca40cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 0x7efca40cf850 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c7c0 tx=0x7efc94007e00 comp rx=0 tx=0).stop 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 shutdown_connections 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7efc84067320 0x7efc840697d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efca4159890 0x7efca40cf850 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efca415db10 0x7efca4153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 --2- 172.21.15.92:0/803211384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efca4159c60 0x7efca4151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:31.901 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 >> 172.21.15.92:0/803211384 conn(0x7efca4068e20 msgr2=0x7efca40c59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:31.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 shutdown_connections 2023-10-06T03:10:31.902 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:31.899+0000 7efca96ce700 1 -- 172.21.15.92:0/803211384 wait complete. 2023-10-06T03:10:31.912 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:10:31.913 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:10:31.961 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:10:32.006 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:10:32.006 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:10:32.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.120+0000 7f9ae53d7700 1 Processor -- start 2023-10-06T03:10:32.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.120+0000 7f9ae53d7700 1 -- start start 2023-10-06T03:10:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9ae53d7700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ae0159890 0x7f9ae015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9ae53d7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9ae0159c60 0x7f9ae015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9ae53d7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9ae53d7700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae015df50 con 0x7f9ae0159890 2023-10-06T03:10:32.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9ae53d7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae015a160 con 0x7f9ae0159c60 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9ae53d7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae00d13a0 con 0x7f9ae015db10 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9adeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ae0159890 0x7f9ae015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9adeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ae0159890 0x7f9ae015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47730/0 (socket says 172.21.15.92:47730) 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.121+0000 7f9adeffd700 1 -- 172.21.15.92:0/92976898 learned_addr learned my addr 172.21.15.92:0/92976898 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ad67fc700 1 --2- 172.21.15.92:0/92976898 >> v2:172.21.15.136:3300/0 conn(0x7f9ae0159c60 0x7f9ae015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adf7fe700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adeffd700 1 -- 172.21.15.92:0/92976898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 msgr2=0x7f9ae0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:32.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adeffd700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adeffd700 1 -- 172.21.15.92:0/92976898 >> v2:172.21.15.136:3300/0 conn(0x7f9ae0159c60 msgr2=0x7f9ae015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adeffd700 1 --2- 172.21.15.92:0/92976898 >> v2:172.21.15.136:3300/0 conn(0x7f9ae0159c60 0x7f9ae015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adeffd700 1 -- 172.21.15.92:0/92976898 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae015e140 con 0x7f9ae0159890 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adf7fe700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9adeffd700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae015abd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00b67f0 tx=0x7f9ad0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ef3e3437fff49a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ade7fc700 1 -- 172.21.15.92:0/92976898 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000b860 con 0x7f9ae0159890 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ade7fc700 1 -- 172.21.15.92:0/92976898 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000b9c0 con 0x7f9ae0159890 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ade7fc700 1 -- 172.21.15.92:0/92976898 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000bd60 con 0x7f9ae0159890 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 -- 172.21.15.92:0/92976898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 msgr2=0x7f9ae015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae015abd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00b67f0 tx=0x7f9ad0009250 comp rx=0 tx=0).stop 2023-10-06T03:10:32.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 -- 172.21.15.92:0/92976898 shutdown_connections 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/92976898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae015abd0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/92976898 >> v2:172.21.15.136:3300/0 conn(0x7f9ae0159c60 0x7f9ae015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.122+0000 7f9ae53d7700 1 -- 172.21.15.92:0/92976898 >> 172.21.15.92:0/92976898 conn(0x7f9ae0068e20 msgr2=0x7f9ae00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.123+0000 7f9ae53d7700 1 -- 172.21.15.92:0/92976898 shutdown_connections 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.123+0000 7f9ae53d7700 1 -- 172.21.15.92:0/92976898 wait complete. 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 Processor -- start 2023-10-06T03:10:32.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 -- start start 2023-10-06T03:10:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae0142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ae0159c60 0x7f9ae0156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae01577d0 con 0x7f9ae0159890 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae01430f0 con 0x7f9ae0159c60 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.124+0000 7f9ae53d7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0156bc0 con 0x7f9ae015db10 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae0142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae0142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47740/0 (socket says 172.21.15.92:47740) 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 -- 172.21.15.92:0/840761822 learned_addr learned my addr 172.21.15.92:0/840761822 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:32.131 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9ad67fc700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ae0159c60 0x7f9ae0156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adf7fe700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 -- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 msgr2=0x7f9ae00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae00b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 -- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ae0159c60 msgr2=0x7f9ae0156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ae0159c60 0x7f9ae0156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 -- 172.21.15.92:0/840761822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00094f0 con 0x7f9ae0159890 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adf7fe700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9adeffd700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae0142bb0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006fd0 tx=0x7f9ad0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:32.132 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0011e30 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0024440 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae0156ea0 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0024800 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.125+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae0157460 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.126+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae004c3c0 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.126+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9ad001a420 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.127+0000 7f9ad77fe700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ac0067320 0x7f9ac00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.127+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9ad0086910 con 0x7f9ae0159890 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.127+0000 7f9ad67fc700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ac0067320 0x7f9ac00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:32.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.127+0000 7f9ad67fc700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ac0067320 0x7f9ac00697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9ae016d150 tx=0x7f9ac8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:32.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.131+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ad0053740 con 0x7f9ae0159890 2023-10-06T03:10:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.358+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ae006a5b0 con 0x7f9ae0159890 2023-10-06T03:10:32.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.359+0000 7f9ad77fe700 1 -- 172.21.15.92:0/840761822 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+3531 (secure 0 0 0) 0x7f9ad0015de0 con 0x7f9ae0159890 2023-10-06T03:10:32.360 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:32.360 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6596,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/8461306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":8461306},{"type":"v1","addr":"172.21.15.136:6835","nonce":8461306}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":7702,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/1754985264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":1754985264},{"type":"v1","addr":"172.21.15.136:6834","nonce":1754985264}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":118},{"gid":8349,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4120444907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4120444907},{"type":"v1","addr":"172.21.15.92:6835","nonce":4120444907}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108},{"gid":8352,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2781455315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2781455315},{"type":"v1","addr":"172.21.15.92:6837","nonce":2781455315}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":108}],"filesystems":[]} 2023-10-06T03:10:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ac0067320 msgr2=0x7f9ac00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:32.362 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ac0067320 0x7f9ac00697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9ae016d150 tx=0x7f9ac8008040 comp rx=0 tx=0).stop 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 msgr2=0x7f9ae0142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae0142bb0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006fd0 tx=0x7f9ad0017000 comp rx=0 tx=0).stop 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 shutdown_connections 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ac0067320 0x7f9ac00697d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae0159890 0x7f9ae0142bb0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ae015db10 0x7f9ae00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 --2- 172.21.15.92:0/840761822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ae0159c60 0x7f9ae0156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 >> 172.21.15.92:0/840761822 conn(0x7f9ae0068e20 msgr2=0x7f9ae00cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 shutdown_connections 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:32.361+0000 7f9ae53d7700 1 -- 172.21.15.92:0/840761822 wait complete. 2023-10-06T03:10:32.363 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 118 2023-10-06T03:10:32.374 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:10:32.374 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:10:32.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:10:32.375 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:10:32.375 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:10:32.375 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:10:32.375 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:10:32.375 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:10:32.376 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:10:32.376 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:10:32.376 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:10:32.376 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:10:32.376 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:10:32.376 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:10:32.376 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:10:32.377 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:10:32.383 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:10:32.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:10:32.384 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:10:32.384 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:10:32.388 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:10:32.388 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:10:32.426 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:10:32.430 DEBUG:teuthology.parallel:result is None 2023-10-06T03:10:32.431 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:10:32.431 DEBUG:teuthology.parallel:result is None 2023-10-06T03:10:32.432 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:10:32.432 DEBUG:teuthology.parallel:result is None 2023-10-06T03:10:32.499 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:10:32.500 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:10:32.513 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:10:33.867 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T03:10:38.388 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:10:38.389 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:10:38.391 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:10:38.392 DEBUG:teuthology.parallel:result is None 2023-10-06T03:10:38.392 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:10:38.475 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:10:38.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.518+0000 7f082a981700 1 Processor -- start 2023-10-06T03:10:38.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.518+0000 7f082a981700 1 -- start start 2023-10-06T03:10:38.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.519+0000 7f082a981700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f0824140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.519+0000 7f082a981700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f082406c0a0 0x7f0824141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.519+0000 7f082a981700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082415c2a0 con 0x7f08240c8e00 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.519+0000 7f082a981700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082415b690 con 0x7f082406c0a0 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.519+0000 7f082a981700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082415b970 con 0x7f082406bcd0 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0828f1e700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f08240c8e00 0x7f08240c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:45976/0 (socket says 172.21.15.92:45976) 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f08237fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f082406c0a0 0x7f0824141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0823fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f0824140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0828f1e700 1 -- 172.21.15.92:0/3662626022 learned_addr learned my addr 172.21.15.92:0/3662626022 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0822ffd700 1 -- 172.21.15.92:0/3662626022 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118863912 0 0) 0x7f082415c2a0 con 0x7f08240c8e00 2023-10-06T03:10:38.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0823fff700 1 -- 172.21.15.92:0/3662626022 >> v2:172.21.15.136:3300/0 conn(0x7f082406c0a0 msgr2=0x7f0824141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0823fff700 1 --2- 172.21.15.92:0/3662626022 >> v2:172.21.15.136:3300/0 conn(0x7f082406c0a0 0x7f0824141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0823fff700 1 -- 172.21.15.92:0/3662626022 >> v1:172.21.15.136:6789/0 conn(0x7f08240c8e00 legacy=0x7f08240c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0823fff700 1 -- 172.21.15.92:0/3662626022 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082415bc50 con 0x7f082406bcd0 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0823fff700 1 --2- 172.21.15.92:0/3662626022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f0824140df0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08140098e0 tx=0x7f08140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa6f96305e1ef789 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0822ffd700 1 -- 172.21.15.92:0/3662626022 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400b640 con 0x7f082406bcd0 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0822ffd700 1 -- 172.21.15.92:0/3662626022 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081400b7a0 con 0x7f082406bcd0 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.520+0000 7f0822ffd700 1 -- 172.21.15.92:0/3662626022 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400bb40 con 0x7f082406bcd0 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 -- 172.21.15.92:0/3662626022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 msgr2=0x7f0824140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 --2- 172.21.15.92:0/3662626022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f0824140df0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08140098e0 tx=0x7f08140091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 -- 172.21.15.92:0/3662626022 shutdown_connections 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 --2- 172.21.15.92:0/3662626022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f0824140df0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:38.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 --2- 172.21.15.92:0/3662626022 >> v2:172.21.15.136:3300/0 conn(0x7f082406c0a0 0x7f0824141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 -- 172.21.15.92:0/3662626022 >> 172.21.15.92:0/3662626022 conn(0x7f08240ce600 msgr2=0x7f08240cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 -- 172.21.15.92:0/3662626022 shutdown_connections 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.521+0000 7f082a981700 1 -- 172.21.15.92:0/3662626022 wait complete. 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.522+0000 7f082a981700 1 Processor -- start 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.522+0000 7f082a981700 1 -- start start 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f082a981700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f082413c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f082a981700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 0x7f082413f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f082a981700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240c8e00 0x7f08240cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f082a981700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082413d870 con 0x7f082406c0a0 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f082a981700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0824155e40 con 0x7f08240c8e00 2023-10-06T03:10:38.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f082a981700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082413d540 con 0x7f082406bcd0 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 0x7f082413f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 0x7f082413f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:47792/0 (socket says 172.21.15.92:47792) 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 -- 172.21.15.92:0/2933374603 learned_addr learned my addr 172.21.15.92:0/2933374603 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f0828f1e700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240c8e00 0x7f08240cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f0823fff700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f082413c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 -- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 msgr2=0x7f082413c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f082413c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 -- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240c8e00 msgr2=0x7f08240cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240c8e00 0x7f08240cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 -- 172.21.15.92:0/2933374603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009420 con 0x7f082406c0a0 2023-10-06T03:10:38.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.523+0000 7f08237fe700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 0x7f082413f420 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f081800ec80 tx=0x7f08180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.524+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800c850 con 0x7f082406c0a0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.524+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0824155880 con 0x7f082406c0a0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.524+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800c9b0 con 0x7f082406c0a0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.524+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800cd50 con 0x7f082406c0a0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.524+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0824155570 con 0x7f082406c0a0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.525+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082404c3c0 con 0x7f082406c0a0 2023-10-06T03:10:38.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.525+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0818013680 con 0x7f082406c0a0 2023-10-06T03:10:38.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.525+0000 7f080bfff700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0810067320 0x7f08100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:38.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.525+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0818084db0 con 0x7f082406c0a0 2023-10-06T03:10:38.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.526+0000 7f0823fff700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0810067320 0x7f08100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:38.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.526+0000 7f0823fff700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0810067320 0x7f08100697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f08140093f0 tx=0x7f0814017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:38.534 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.530+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0818089070 con 0x7f082406c0a0 2023-10-06T03:10:38.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:38.731+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f082406c9d0 con 0x7f082406c0a0 2023-10-06T03:10:39.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.569+0000 7f080bfff700 1 -- 172.21.15.92:0/2933374603 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v182) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0818017080 con 0x7f082406c0a0 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0810067320 msgr2=0x7f08100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0810067320 0x7f08100697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f08140093f0 tx=0x7f0814017000 comp rx=0 tx=0).stop 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 msgr2=0x7f082413f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 0x7f082413f420 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f081800ec80 tx=0x7f08180091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 shutdown_connections 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0810067320 0x7f08100697d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08240c8e00 0x7f08240cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f082406bcd0 0x7f082413c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 --2- 172.21.15.92:0/2933374603 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f082406c0a0 0x7f082413f420 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 >> 172.21.15.92:0/2933374603 conn(0x7f08240ce600 msgr2=0x7f082415ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 shutdown_connections 2023-10-06T03:10:39.574 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.571+0000 7f082a981700 1 -- 172.21.15.92:0/2933374603 wait complete. 2023-10-06T03:10:39.574 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:10:39.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:10:39.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.698+0000 7fa1917dd700 1 Processor -- start 2023-10-06T03:10:39.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa1917dd700 1 -- start start 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa1917dd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa18c159c60 0x7fa18c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa1917dd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa18c15db10 0x7fa18c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa1917dd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18c15a160 con 0x7fa18c159890 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa1917dd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c15df50 con 0x7fa18c15db10 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa1917dd700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18c0d13a0 con 0x7fa18c159c60 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa18b7fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa18c15db10 0x7fa18c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.699+0000 7fa18affd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa18c159890 0x7fa18c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49348/0 (socket says 172.21.15.92:49348) 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18affd700 1 -- 172.21.15.92:0/2140219630 learned_addr learned my addr 172.21.15.92:0/2140219630 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:39.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18b7fe700 1 -- 172.21.15.92:0/2140219630 >> v2:172.21.15.136:3300/0 conn(0x7fa18c159c60 msgr2=0x7fa18c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa1827fc700 1 --2- 172.21.15.92:0/2140219630 >> v2:172.21.15.136:3300/0 conn(0x7fa18c159c60 0x7fa18c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18b7fe700 1 --2- 172.21.15.92:0/2140219630 >> v2:172.21.15.136:3300/0 conn(0x7fa18c159c60 0x7fa18c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18a7fc700 1 -- 172.21.15.92:0/2140219630 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454631972 0 0) 0x7fa18c15a160 con 0x7fa18c159890 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18b7fe700 1 -- 172.21.15.92:0/2140219630 >> v1:172.21.15.92:6789/0 conn(0x7fa18c159890 legacy=0x7fa18c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18b7fe700 1 -- 172.21.15.92:0/2140219630 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c15e140 con 0x7fa18c15db10 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa1827fc700 1 --2- 172.21.15.92:0/2140219630 >> v2:172.21.15.136:3300/0 conn(0x7fa18c159c60 0x7fa18c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18b7fe700 1 --2- 172.21.15.92:0/2140219630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c15db10 0x7fa18c069560 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0b67f0 tx=0x7fa184009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caf20fd48835bfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18a7fc700 1 -- 172.21.15.92:0/2140219630 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18400e070 con 0x7fa18c15db10 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18a7fc700 1 -- 172.21.15.92:0/2140219630 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18400bc30 con 0x7fa18c15db10 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.700+0000 7fa18a7fc700 1 -- 172.21.15.92:0/2140219630 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184012430 con 0x7fa18c15db10 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 -- 172.21.15.92:0/2140219630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c15db10 msgr2=0x7fa18c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.704 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 --2- 172.21.15.92:0/2140219630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c15db10 0x7fa18c069560 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0b67f0 tx=0x7fa184009780 comp rx=0 tx=0).stop 2023-10-06T03:10:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 -- 172.21.15.92:0/2140219630 shutdown_connections 2023-10-06T03:10:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 --2- 172.21.15.92:0/2140219630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c15db10 0x7fa18c069560 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 --2- 172.21.15.92:0/2140219630 >> v2:172.21.15.136:3300/0 conn(0x7fa18c159c60 0x7fa18c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 -- 172.21.15.92:0/2140219630 >> 172.21.15.92:0/2140219630 conn(0x7fa18c068e20 msgr2=0x7fa18c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 -- 172.21.15.92:0/2140219630 shutdown_connections 2023-10-06T03:10:39.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.701+0000 7fa1917dd700 1 -- 172.21.15.92:0/2140219630 wait complete. 2023-10-06T03:10:39.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.702+0000 7fa1917dd700 1 Processor -- start 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.702+0000 7fa1917dd700 1 -- start start 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 0x7fa18c14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa18c159c60 0x7fa18c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c15db10 0x7fa18c0c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c154620 con 0x7fa18c159890 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c14fd10 con 0x7fa18c15db10 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c153a10 con 0x7fa18c159c60 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 0x7fa18c14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 0x7fa18c14f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44854/0 (socket says 172.21.15.92:44854) 2023-10-06T03:10:39.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 -- 172.21.15.92:0/1680677095 learned_addr learned my addr 172.21.15.92:0/1680677095 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 -- 172.21.15.92:0/1680677095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa18c159c60 msgr2=0x7fa18c1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa18c159c60 0x7fa18c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 -- 172.21.15.92:0/1680677095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c15db10 msgr2=0x7fa18c0c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c15db10 0x7fa18c0c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 -- 172.21.15.92:0/1680677095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184009a20 con 0x7fa18c159890 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa18affd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 0x7fa18c14f7d0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c810 tx=0x7fa17c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c005b30 con 0x7fa18c159890 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c005c90 con 0x7fa18c159890 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c007300 con 0x7fa18c159890 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c153cf0 con 0x7fa18c159890 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.703+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18c1542b0 con 0x7fa18c159890 2023-10-06T03:10:39.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.704+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18c04c3c0 con 0x7fa18c159890 2023-10-06T03:10:39.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.705+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa17c01b400 con 0x7fa18c159890 2023-10-06T03:10:39.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.705+0000 7fa1837fe700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa16c067320 0x7fa16c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:39.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.705+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa17c084540 con 0x7fa18c159890 2023-10-06T03:10:39.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.705+0000 7fa1827fc700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa16c067320 0x7fa16c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:39.710 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.705+0000 7fa1827fc700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa16c067320 0x7fa16c0697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa18c152070 tx=0x7fa174008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:39.711 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.709+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa17c0514c0 con 0x7fa18c159890 2023-10-06T03:10:39.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.965+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa18c1549b0 con 0x7fa18c159890 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.965+0000 7fa1837fe700 1 -- 172.21.15.92:0/1680677095 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa17c054ae0 con 0x7fa18c159890 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:10:39.967 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.968 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.969 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:10:39.970 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.971 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:10:39.972 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.973 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.974 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.975 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.976 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.977 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:10:39.978 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:10:39.979 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.980 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:10:39.981 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:10:39.982 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa16c067320 msgr2=0x7fa16c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa16c067320 0x7fa16c0697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa18c152070 tx=0x7fa174008040 comp rx=0 tx=0).stop 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 msgr2=0x7fa18c14f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 0x7fa18c14f7d0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c810 tx=0x7fa17c007df0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 shutdown_connections 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa16c067320 0x7fa16c0697d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa18c159890 0x7fa18c14f7d0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa18c15db10 0x7fa18c0c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 --2- 172.21.15.92:0/1680677095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa18c159c60 0x7fa18c1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 >> 172.21.15.92:0/1680677095 conn(0x7fa18c068e20 msgr2=0x7fa18c15d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:39.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 shutdown_connections 2023-10-06T03:10:39.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:39.967+0000 7fa1917dd700 1 -- 172.21.15.92:0/1680677095 wait complete. 2023-10-06T03:10:39.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 Processor -- start 2023-10-06T03:10:40.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 -- start start 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb08157940 0x7ffb0806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffb0806c1e0 0x7ffb080cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0806c750 con 0x7ffb080cbc10 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0806ca30 con 0x7ffb08157940 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.098+0000 7ffb0e4fd700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0806cd10 con 0x7ffb0806c1e0 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb08157940 0x7ffb0806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb08157940 0x7ffb0806f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44864/0 (socket says 172.21.15.92:44864) 2023-10-06T03:10:40.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 -- 172.21.15.92:0/4205410105 learned_addr learned my addr 172.21.15.92:0/4205410105 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb077fe700 1 --2- 172.21.15.92:0/4205410105 >> v2:172.21.15.136:3300/0 conn(0x7ffb0806c1e0 0x7ffb080cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 -- 172.21.15.92:0/4205410105 >> v2:172.21.15.136:3300/0 conn(0x7ffb0806c1e0 msgr2=0x7ffb080cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 --2- 172.21.15.92:0/4205410105 >> v2:172.21.15.136:3300/0 conn(0x7ffb0806c1e0 0x7ffb080cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 -- 172.21.15.92:0/4205410105 >> v1:172.21.15.136:6789/0 conn(0x7ffb080cbc10 legacy=0x7ffb0815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 -- 172.21.15.92:0/4205410105 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0806cff0 con 0x7ffb08157940 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb07fff700 1 --2- 172.21.15.92:0/4205410105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08157940 0x7ffb0806f5a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7ffb0806b470 tx=0x7ffaf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc38d0acbb6675 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb06ffd700 1 -- 172.21.15.92:0/4205410105 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8009e70 con 0x7ffb08157940 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb06ffd700 1 -- 172.21.15.92:0/4205410105 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf800b860 con 0x7ffb08157940 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.099+0000 7ffb06ffd700 1 -- 172.21.15.92:0/4205410105 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800bc00 con 0x7ffb08157940 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/4205410105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08157940 msgr2=0x7ffb0806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/4205410105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08157940 0x7ffb0806f5a0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7ffb0806b470 tx=0x7ffaf8009250 comp rx=0 tx=0).stop 2023-10-06T03:10:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/4205410105 shutdown_connections 2023-10-06T03:10:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/4205410105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb08157940 0x7ffb0806f5a0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/4205410105 >> v2:172.21.15.136:3300/0 conn(0x7ffb0806c1e0 0x7ffb080cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/4205410105 >> 172.21.15.92:0/4205410105 conn(0x7ffb080cd480 msgr2=0x7ffb080cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/4205410105 shutdown_connections 2023-10-06T03:10:40.104 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.100+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/4205410105 wait complete. 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.101+0000 7ffb0e4fd700 1 Processor -- start 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.101+0000 7ffb0e4fd700 1 -- start start 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0e4fd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb0806c1e0 0x7ffb08151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0e4fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 0x7ffb081561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0e4fd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb08157940 0x7ffb080c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0e4fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb080c2e50 con 0x7ffb080cbc10 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0e4fd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08155170 con 0x7ffb08157940 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0e4fd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08155450 con 0x7ffb0806c1e0 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 0x7ffb081561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.107 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 0x7ffb081561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44872/0 (socket says 172.21.15.92:44872) 2023-10-06T03:10:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 -- 172.21.15.92:0/705131044 learned_addr learned my addr 172.21.15.92:0/705131044 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0ca9a700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb08157940 0x7ffb080c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb07fff700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb0806c1e0 0x7ffb08151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 -- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb0806c1e0 msgr2=0x7ffb08151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.108 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb0806c1e0 0x7ffb08151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 -- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb08157940 msgr2=0x7ffb080c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb08157940 0x7ffb080c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 -- 172.21.15.92:0/705131044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80094f0 con 0x7ffb080cbc10 2023-10-06T03:10:40.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb0ca9a700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb08157940 0x7ffb080c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.102+0000 7ffb077fe700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 0x7ffb081561f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00ec80 tx=0x7ffafc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.103+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00c9d0 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.103+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc00cb30 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.103+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb08155730 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.103+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc013470 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.103+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb08155cf0 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.104+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0804c3c0 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.104+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ffafc013820 con 0x7ffb080cbc10 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.104+0000 7ffaeffff700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffaf4067320 0x7ffaf40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.105+0000 7ffb07fff700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffaf4067320 0x7ffaf40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.105+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ffafc084e70 con 0x7ffb080cbc10 2023-10-06T03:10:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.105+0000 7ffb07fff700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffaf4067320 0x7ffaf40697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffb0806b470 tx=0x7ffaf8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.109+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffafc051cf0 con 0x7ffb080cbc10 2023-10-06T03:10:40.294 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.293+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb0815b580 con 0x7ffb080cbc10 2023-10-06T03:10:40.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.294+0000 7ffaeffff700 1 -- 172.21.15.92:0/705131044 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+13660 (secure 0 0 0) 0x7ffafc055310 con 0x7ffb080cbc10 2023-10-06T03:10:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:40.295 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":182,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:39.561166+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffaf4067320 msgr2=0x7ffaf40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffaf4067320 0x7ffaf40697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffb0806b470 tx=0x7ffaf8017000 comp rx=0 tx=0).stop 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 msgr2=0x7ffb081561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 0x7ffb081561f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00ec80 tx=0x7ffafc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 shutdown_connections 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ffaf4067320 0x7ffaf40697d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb08157940 0x7ffb080c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb0806c1e0 0x7ffb08151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 --2- 172.21.15.92:0/705131044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb080cbc10 0x7ffb081561f0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 >> 172.21.15.92:0/705131044 conn(0x7ffb080cd480 msgr2=0x7ffb080b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:40.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 shutdown_connections 2023-10-06T03:10:40.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.296+0000 7ffb0e4fd700 1 -- 172.21.15.92:0/705131044 wait complete. 2023-10-06T03:10:40.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:10:40.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:10:40.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.426+0000 7f8dd0d40700 1 Processor -- start 2023-10-06T03:10:40.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.426+0000 7f8dd0d40700 1 -- start start 2023-10-06T03:10:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dd0d40700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8dcc159890 0x7f8dcc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.430 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dd0d40700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dd0d40700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dcc069ee0 con 0x7f8dcc159c60 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dd0d40700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc15a160 con 0x7f8dcc159890 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dd0d40700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0d13a0 con 0x7f8dcc069560 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dc9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8dcc159c60 0x7f8dcc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49354/0 (socket says 172.21.15.92:49354) 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dc9d9b700 1 -- 172.21.15.92:0/1943975792 learned_addr learned my addr 172.21.15.92:0/1943975792 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dca59c700 1 --2- 172.21.15.92:0/1943975792 >> v2:172.21.15.136:3300/0 conn(0x7f8dcc159890 0x7f8dcc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dcad9d700 1 --2- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.427+0000 7f8dc959a700 1 -- 172.21.15.92:0/1943975792 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211737696 0 0) 0x7f8dcc069ee0 con 0x7f8dcc159c60 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dc959a700 1 -- 172.21.15.92:0/1943975792 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db4003570 con 0x7f8dcc159c60 2023-10-06T03:10:40.431 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dca59c700 1 -- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 msgr2=0x7f8dcc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dca59c700 1 --2- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dca59c700 1 -- 172.21.15.92:0/1943975792 >> v1:172.21.15.92:6789/0 conn(0x7f8dcc159c60 legacy=0x7f8dcc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dca59c700 1 -- 172.21.15.92:0/1943975792 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc15e140 con 0x7f8dcc159890 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dca59c700 1 --2- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc15abd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009e60 tx=0x7f8dbc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9979edf5c23aa61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dc959a700 1 -- 172.21.15.92:0/1943975792 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00e070 con 0x7f8dcc159890 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dc959a700 1 -- 172.21.15.92:0/1943975792 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc00bab0 con 0x7f8dcc159890 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.428+0000 7f8dc959a700 1 -- 172.21.15.92:0/1943975792 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00be30 con 0x7f8dcc159890 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 -- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 msgr2=0x7f8dcc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc15abd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009e60 tx=0x7f8dbc009780 comp rx=0 tx=0).stop 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 -- 172.21.15.92:0/1943975792 shutdown_connections 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/1943975792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc15abd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 -- 172.21.15.92:0/1943975792 >> 172.21.15.92:0/1943975792 conn(0x7f8dcc068e20 msgr2=0x7f8dcc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:40.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 -- 172.21.15.92:0/1943975792 shutdown_connections 2023-10-06T03:10:40.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.429+0000 7f8dd0d40700 1 -- 172.21.15.92:0/1943975792 wait complete. 2023-10-06T03:10:40.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.430+0000 7f8dd0d40700 1 Processor -- start 2023-10-06T03:10:40.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.430+0000 7f8dd0d40700 1 -- start start 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.430+0000 7f8dd0d40700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc0b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 0x7f8dcc0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c8db0 con 0x7f8dcc159c60 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c81a0 con 0x7f8dcc159890 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0c8480 con 0x7f8dcc069560 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 0x7f8dcc0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 0x7f8dcc0b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44876/0 (socket says 172.21.15.92:44876) 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 -- 172.21.15.92:0/943892263 learned_addr learned my addr 172.21.15.92:0/943892263 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dc9d9b700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc0b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dca59c700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 -- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 msgr2=0x7f8dcc0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 -- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 msgr2=0x7f8dcc0b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc0b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 -- 172.21.15.92:0/943892263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc009a20 con 0x7f8dcc159c60 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dca59c700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dcad9d700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 0x7f8dcc0b1330 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e8d0 tx=0x7f8dc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4013680 con 0x7f8dcc159c60 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc40137e0 con 0x7f8dcc159c60 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc0c8760 con 0x7f8dcc159c60 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4013ba0 con 0x7f8dcc159c60 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.431+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc0afea0 con 0x7f8dcc159c60 2023-10-06T03:10:40.437 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.432+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc0c8f60 con 0x7f8dcc159c60 2023-10-06T03:10:40.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.433+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8dc40123d0 con 0x7f8dcc159c60 2023-10-06T03:10:40.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.433+0000 7f8dba7fc700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8dac067320 0x7f8dac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.433+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8dc4084ba0 con 0x7f8dcc159c60 2023-10-06T03:10:40.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.433+0000 7f8dca59c700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8dac067320 0x7f8dac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.438 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.434+0000 7f8dca59c700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8dac067320 0x7f8dac0697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0097c0 tx=0x7f8dbc019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.437+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8dc4051a20 con 0x7f8dcc159c60 2023-10-06T03:10:40.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T03:10:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.622+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dcc04c3c0 con 0x7f8dcc159c60 2023-10-06T03:10:40.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.623+0000 7f8dba7fc700 1 -- 172.21.15.92:0/943892263 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dc4055040 con 0x7f8dcc159c60 2023-10-06T03:10:40.625 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:10:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8dac067320 msgr2=0x7f8dac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.626 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8dac067320 0x7f8dac0697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0097c0 tx=0x7f8dbc019000 comp rx=0 tx=0).stop 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 msgr2=0x7f8dcc0b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 0x7f8dcc0b1330 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400e8d0 tx=0x7f8dc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 shutdown_connections 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8dac067320 0x7f8dac0697d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8dcc159c60 0x7f8dcc0b1330 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dcc069560 0x7f8dcc0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 --2- 172.21.15.92:0/943892263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dcc159890 0x7f8dcc0b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 >> 172.21.15.92:0/943892263 conn(0x7f8dcc068e20 msgr2=0x7f8dcc1588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 shutdown_connections 2023-10-06T03:10:40.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.625+0000 7f8dd0d40700 1 -- 172.21.15.92:0/943892263 wait complete. 2023-10-06T03:10:40.638 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.753+0000 7f530b91e700 1 Processor -- start 2023-10-06T03:10:40.755 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.754+0000 7f530b91e700 1 -- start start 2023-10-06T03:10:40.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.754+0000 7f530b91e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 0x7f530415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f530b91e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5304159c60 0x7f530415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f530b91e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530415a160 con 0x7f530415db10 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f530b91e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53040d13a0 con 0x7f5304159c60 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f530b91e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530415e140 con 0x7f5304159890 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f5309ebb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f530415db10 0x7f5304069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49366/0 (socket says 172.21.15.92:49366) 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f5309ebb700 1 -- 172.21.15.92:0/943921989 learned_addr learned my addr 172.21.15.92:0/943921989 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f5308eb9700 1 --2- 172.21.15.92:0/943921989 >> v2:172.21.15.136:3300/0 conn(0x7f5304159c60 0x7f530415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f53096ba700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 0x7f530415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360163670 0 0) 0x7f530415a160 con 0x7f530415db10 2023-10-06T03:10:40.759 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f0003570 con 0x7f530415db10 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1805852343 0 0) 0x7f52f0003570 con 0x7f530415db10 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 msgr2=0x7f530415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 0x7f530415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 >> v2:172.21.15.136:3300/0 conn(0x7f5304159c60 msgr2=0x7f530415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 --2- 172.21.15.92:0/943921989 >> v2:172.21.15.136:3300/0 conn(0x7f5304159c60 0x7f530415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530415a3d0 con 0x7f530415db10 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f52fc001470 con 0x7f530415db10 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 >> v1:172.21.15.92:6789/0 conn(0x7f530415db10 legacy=0x7f5304069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 0x7f52f0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f0003fe0 0x7f52f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f0004d50 0x7f52f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040d13a0 con 0x7f52f0004d50 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530415e140 con 0x7f52f0003fe0 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f000ba90 con 0x7f5304159890 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f5309ebb700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f0004d50 0x7f52f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f53096ba700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 0x7f52f0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.755+0000 7f5308eb9700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f0003fe0 0x7f52f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 -- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 msgr2=0x7f52f0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5304159890 0x7f52f0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 -- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f0003fe0 msgr2=0x7f52f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52f0003fe0 0x7f52f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 -- 172.21.15.92:0/943921989 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0003570 con 0x7f52f0004d50 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f0004d50 0x7f52f00051c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00c3c0 tx=0x7f52fc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6d217b5bb14054 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00cc30 con 0x7f52f0004d50 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc00cdb0 con 0x7f52f0004d50 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f52fbfff700 1 -- 172.21.15.92:0/943921989 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc005b70 con 0x7f52f0004d50 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f530b91e700 1 -- 172.21.15.92:0/943921989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f0004d50 msgr2=0x7f52f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f530b91e700 1 --2- 172.21.15.92:0/943921989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52f0004d50 0x7f52f00051c0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00c3c0 tx=0x7f52fc001e90 comp rx=0 tx=0).stop 2023-10-06T03:10:40.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f5309ebb700 1 -- 172.21.15.92:0/943921989 reap_dead start 2023-10-06T03:10:40.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f530b91e700 1 -- 172.21.15.92:0/943921989 shutdown_connections 2023-10-06T03:10:40.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.756+0000 7f530b91e700 1 -- 172.21.15.92:0/943921989 >> 172.21.15.92:0/943921989 conn(0x7f5304068e20 msgr2=0x7f53040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:40.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.757+0000 7f530b91e700 1 -- 172.21.15.92:0/943921989 shutdown_connections 2023-10-06T03:10:40.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.757+0000 7f530b91e700 1 -- 172.21.15.92:0/943921989 wait complete. 2023-10-06T03:10:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 Processor -- start 2023-10-06T03:10:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 -- start start 2023-10-06T03:10:40.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5304140dc0 0x7f530416d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 0x7f53040b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53040b1c20 0x7f53040b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304152880 con 0x7f530416d6f0 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530416db60 con 0x7f5304140dc0 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.758+0000 7f530b91e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5304156570 con 0x7f53040b1c20 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 0x7f53040b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 0x7f53040b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44886/0 (socket says 172.21.15.92:44886) 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 -- 172.21.15.92:0/2917368668 learned_addr learned my addr 172.21.15.92:0/2917368668 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f53096ba700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5304140dc0 0x7f530416d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5309ebb700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53040b1c20 0x7f53040b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 -- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53040b1c20 msgr2=0x7f53040b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53040b1c20 0x7f53040b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 -- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5304140dc0 msgr2=0x7f530416d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5304140dc0 0x7f530416d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 -- 172.21.15.92:0/2917368668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304158a10 con 0x7f530416d6f0 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f5308eb9700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 0x7f53040b16e0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009fd0 tx=0x7f52f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400f790 con 0x7f530416d6f0 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f400f8f0 con 0x7f530416d6f0 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53040b2440 con 0x7f530416d6f0 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400fc70 con 0x7f530416d6f0 2023-10-06T03:10:40.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.759+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5304155f20 con 0x7f530416d6f0 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.760+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530404c3c0 con 0x7f530416d6f0 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.760+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f52f4014440 con 0x7f530416d6f0 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.761+0000 7f52f97fa700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f52e8067320 0x7f52e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.761+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f52f4018030 con 0x7f530416d6f0 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.761+0000 7f53096ba700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f52e8067320 0x7f52e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.761+0000 7f53096ba700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f52e8067320 0x7f52e80697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f53000094d0 tx=0x7f530000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:40.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.765+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52f4055c40 con 0x7f530416d6f0 2023-10-06T03:10:40.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.947+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f53041566b0 con 0x7f530416d6f0 2023-10-06T03:10:40.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.947+0000 7f52f97fa700 1 -- 172.21.15.92:0/2917368668 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v182) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52f40557d0 con 0x7f530416d6f0 2023-10-06T03:10:40.949 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:10:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.949+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f52e8067320 msgr2=0x7f52e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.949+0000 7f530b91e700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f52e8067320 0x7f52e80697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f53000094d0 tx=0x7f530000b040 comp rx=0 tx=0).stop 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.949+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 msgr2=0x7f53040b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.949+0000 7f530b91e700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 0x7f53040b16e0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009fd0 tx=0x7f52f40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 shutdown_connections 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f52e8067320 0x7f52e80697d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53040b1c20 0x7f53040b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5304140dc0 0x7f530416d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 --2- 172.21.15.92:0/2917368668 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f530416d6f0 0x7f53040b16e0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 >> 172.21.15.92:0/2917368668 conn(0x7f5304068e20 msgr2=0x7f5304152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 shutdown_connections 2023-10-06T03:10:40.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:40.950+0000 7f530b91e700 1 -- 172.21.15.92:0/2917368668 wait complete. 2023-10-06T03:10:40.964 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:10:41.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.076+0000 7f6a0fe7c700 1 Processor -- start 2023-10-06T03:10:41.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.077+0000 7f6a0fe7c700 1 -- start start 2023-10-06T03:10:41.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.077+0000 7f6a0fe7c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f6a0815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.077+0000 7f6a0fe7c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0815a160 con 0x7f6a08069560 2023-10-06T03:10:41.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.077+0000 7f6a0fe7c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a080d13a0 con 0x7f6a08159c60 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.077+0000 7f6a0fe7c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0815e140 con 0x7f6a08159890 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0e419700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6a08069560 0x7f6a080ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49378/0 (socket says 172.21.15.92:49378) 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0e419700 1 -- 172.21.15.92:0/1928806044 learned_addr learned my addr 172.21.15.92:0/1928806044 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0dc18700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f6a0815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225624963 0 0) 0x7f6a0815a160 con 0x7f6a08069560 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f0003570 con 0x7f6a08069560 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067906143 0 0) 0x7f69f0003570 con 0x7f6a08069560 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 msgr2=0x7f6a0815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:41.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f6a0815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 >> v1:172.21.15.136:6789/0 conn(0x7f6a08159c60 legacy=0x7f6a0815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0815a3d0 con 0x7f6a08069560 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f6a000017f0 con 0x7f6a08069560 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 >> v1:172.21.15.92:6789/0 conn(0x7f6a08069560 legacy=0x7f6a080ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f69f0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69f0004010 0x7f69f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f0004d50 0x7f69f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0815e140 con 0x7f69f0004010 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f000bbf0 con 0x7f69f0004d50 2023-10-06T03:10:41.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f000bdb0 con 0x7f6a08159890 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0d417700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69f0004010 0x7f69f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0dc18700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f69f0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.078+0000 7f6a0e419700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f0004d50 0x7f69f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0d417700 1 -- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 msgr2=0x7f69f0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0d417700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f69f0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0d417700 1 -- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f0004d50 msgr2=0x7f69f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0d417700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f0004d50 0x7f69f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0d417700 1 -- 172.21.15.92:0/1928806044 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0003570 con 0x7f69f0004010 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0dc18700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a08159890 0x7f69f0003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0e419700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69f0004d50 0x7f69f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0d417700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69f0004010 0x7f69f0004810 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6a0815a090 tx=0x7f69fc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58d91a24c75bc6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:41.087 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc012420 con 0x7f69f0004010 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc012580 con 0x7f69f0004010 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0cc16700 1 -- 172.21.15.92:0/1928806044 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc012920 con 0x7f69f0004010 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/1928806044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69f0004010 msgr2=0x7f69f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/1928806044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69f0004010 0x7f69f0004810 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f6a0815a090 tx=0x7f69fc007d30 comp rx=0 tx=0).stop 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0e419700 1 -- 172.21.15.92:0/1928806044 reap_dead start 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/1928806044 shutdown_connections 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/1928806044 >> 172.21.15.92:0/1928806044 conn(0x7f6a08068e20 msgr2=0x7f6a080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/1928806044 shutdown_connections 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.079+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/1928806044 wait complete. 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.080+0000 7f6a0fe7c700 1 Processor -- start 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 -- start start 2023-10-06T03:10:41.088 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69f0004010 0x7f6a080cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 0x7f6a080b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a080b2120 0x7f6a080b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080b4950 con 0x7f6a080b3590 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080b6430 con 0x7f6a080b2120 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0fe7c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a080b1470 con 0x7f69f0004010 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0d417700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 0x7f6a080b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0d417700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 0x7f6a080b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44892/0 (socket says 172.21.15.92:44892) 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.081+0000 7f6a0d417700 1 -- 172.21.15.92:0/2279621473 learned_addr learned my addr 172.21.15.92:0/2279621473 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0e419700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a080b2120 0x7f6a080b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0dc18700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69f0004010 0x7f6a080cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0d417700 1 -- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69f0004010 msgr2=0x7f6a080cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0d417700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69f0004010 0x7f6a080cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0d417700 1 -- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a080b2120 msgr2=0x7f6a080b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0d417700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a080b2120 0x7f6a080b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0d417700 1 -- 172.21.15.92:0/2279621473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080b1750 con 0x7f6a080b3590 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0e419700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a080b2120 0x7f6a080b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0d417700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 0x7f6a080b1be0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f69fc001e40 tx=0x7f69fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc023070 con 0x7f6a080b3590 2023-10-06T03:10:41.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc011ba0 con 0x7f6a080b3590 2023-10-06T03:10:41.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc006820 con 0x7f6a080b3590 2023-10-06T03:10:41.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a08144740 con 0x7f6a080b3590 2023-10-06T03:10:41.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.082+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a080cfd60 con 0x7f6a080b3590 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.083+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a080b18e0 con 0x7f6a080b3590 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.083+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f69fc019070 con 0x7f6a080b3590 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.084+0000 7f6a05ffb700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f69f4067320 0x7f69f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.084+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f69fc085ad0 con 0x7f6a080b3590 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.084+0000 7f6a0dc18700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f69f4067320 0x7f69f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.084+0000 7f6a0dc18700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f69f4067320 0x7f69f40697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006f00 tx=0x7f69f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:41.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.088+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69fc052a20 con 0x7f6a080b3590 2023-10-06T03:10:41.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.272+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6a0804c3c0 con 0x7f6a080b3590 2023-10-06T03:10:41.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.273+0000 7f6a05ffb700 1 -- 172.21.15.92:0/2279621473 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v182) v1 ==== 84+0+126 (secure 0 0 0) 0x7f69fc017030 con 0x7f6a080b3590 2023-10-06T03:10:41.275 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:41.276 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:10:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:10:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:10:41.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:10:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:10:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:10:41.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:41.277 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:10:41.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f69f4067320 msgr2=0x7f69f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f69f4067320 0x7f69f40697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006f00 tx=0x7f69f8008040 comp rx=0 tx=0).stop 2023-10-06T03:10:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 msgr2=0x7f6a080b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 0x7f6a080b1be0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f69fc001e40 tx=0x7f69fc015000 comp rx=0 tx=0).stop 2023-10-06T03:10:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 shutdown_connections 2023-10-06T03:10:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f69f4067320 0x7f69f40697d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.279 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a080b2120 0x7f6a080b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69f0004010 0x7f6a080cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 --2- 172.21.15.92:0/2279621473 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a080b3590 0x7f6a080b1be0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 >> 172.21.15.92:0/2279621473 conn(0x7f6a08068e20 msgr2=0x7f6a080c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 shutdown_connections 2023-10-06T03:10:41.280 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.275+0000 7f6a0fe7c700 1 -- 172.21.15.92:0/2279621473 wait complete. 2023-10-06T03:10:41.289 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:10:41.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:10:41.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.405+0000 7fe534bee700 1 Processor -- start 2023-10-06T03:10:41.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.405+0000 7fe534bee700 1 -- start start 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe534bee700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe530159ec0 0x7fe5300b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe534bee700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5300cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe534bee700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5300b5b00 con 0x7fe5300b53d0 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe534bee700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5300b5850 con 0x7fe530159ec0 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe534bee700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300b5da0 con 0x7fe5300cf350 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe52e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe530159ec0 0x7fe5300b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe52e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe530159ec0 0x7fe5300b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44900/0 (socket says 172.21.15.92:44900) 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.406+0000 7fe52e59c700 1 -- 172.21.15.92:0/706582862 learned_addr learned my addr 172.21.15.92:0/706582862 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52ed9d700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5300cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52e59c700 1 -- 172.21.15.92:0/706582862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 msgr2=0x7fe5300cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52e59c700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5300cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52e59c700 1 -- 172.21.15.92:0/706582862 >> v1:172.21.15.136:6789/0 conn(0x7fe5300b53d0 legacy=0x7fe5300cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52e59c700 1 -- 172.21.15.92:0/706582862 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300b6080 con 0x7fe530159ec0 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52ed9d700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5300cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52e59c700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe530159ec0 0x7fe5300b4e90 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fe520009f70 tx=0x7fe520009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f63af2455984ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52d59a700 1 -- 172.21.15.92:0/706582862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52000e070 con 0x7fe530159ec0 2023-10-06T03:10:41.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52d59a700 1 -- 172.21.15.92:0/706582862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52000ba90 con 0x7fe530159ec0 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe52d59a700 1 -- 172.21.15.92:0/706582862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5200092e0 con 0x7fe530159ec0 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe534bee700 1 -- 172.21.15.92:0/706582862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe530159ec0 msgr2=0x7fe5300b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.407+0000 7fe534bee700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe530159ec0 0x7fe5300b4e90 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fe520009f70 tx=0x7fe520009840 comp rx=0 tx=0).stop 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.408+0000 7fe534bee700 1 -- 172.21.15.92:0/706582862 shutdown_connections 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.408+0000 7fe534bee700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe530159ec0 0x7fe5300b4e90 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.408+0000 7fe534bee700 1 --2- 172.21.15.92:0/706582862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5300cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.408+0000 7fe534bee700 1 -- 172.21.15.92:0/706582862 >> 172.21.15.92:0/706582862 conn(0x7fe5300999f0 msgr2=0x7fe5300b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.408+0000 7fe534bee700 1 -- 172.21.15.92:0/706582862 shutdown_connections 2023-10-06T03:10:41.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.408+0000 7fe534bee700 1 -- 172.21.15.92:0/706582862 wait complete. 2023-10-06T03:10:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 Processor -- start 2023-10-06T03:10:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 -- start start 2023-10-06T03:10:41.415 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 0x7fe53013e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5301512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe530159ec0 0x7fe530151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530153ef0 con 0x7fe5300b53d0 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53013eea0 con 0x7fe530159ec0 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe534bee700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53016d390 con 0x7fe5300cf350 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe52e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 0x7fe53013e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe52e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 0x7fe53013e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44906/0 (socket says 172.21.15.92:44906) 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.409+0000 7fe52e59c700 1 -- 172.21.15.92:0/737431673 learned_addr learned my addr 172.21.15.92:0/737431673 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52ed9d700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe530159ec0 0x7fe530151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.416 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52dd9b700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5301512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52e59c700 1 -- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 msgr2=0x7fe5301512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52e59c700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5301512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52e59c700 1 -- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe530159ec0 msgr2=0x7fe530151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52e59c700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe530159ec0 0x7fe530151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52e59c700 1 -- 172.21.15.92:0/737431673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520009ae0 con 0x7fe5300b53d0 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52ed9d700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe530159ec0 0x7fe530151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe52e59c700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 0x7fe53013e960 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe530089390 tx=0x7fe520017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520012b90 con 0x7fe5300b53d0 2023-10-06T03:10:41.417 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe520012d10 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52001d670 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53016d670 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.410+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53016dc30 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.411+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe530154060 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.411+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe52001daa0 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.412+0000 7fe51e7fc700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe510067320 0x7fe5100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.412+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe5200862b0 con 0x7fe5300b53d0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.412+0000 7fe52dd9b700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe510067320 0x7fe5100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:41.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.412+0000 7fe52dd9b700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe510067320 0x7fe5100697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:41.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.416+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe530154060 con 0x7fe5300b53d0 2023-10-06T03:10:41.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:41.597+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe530063c90 con 0x7fe5300b53d0 2023-10-06T03:10:42.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.588+0000 7fe51e7fc700 1 -- 172.21.15.92:0/737431673 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v183) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe520053020 con 0x7fe5300b53d0 2023-10-06T03:10:42.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe510067320 msgr2=0x7fe5100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:42.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe510067320 0x7fe5100697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).stop 2023-10-06T03:10:42.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 msgr2=0x7fe53013e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:42.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 0x7fe53013e960 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe530089390 tx=0x7fe520017000 comp rx=0 tx=0).stop 2023-10-06T03:10:42.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 shutdown_connections 2023-10-06T03:10:42.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe510067320 0x7fe5100697d0 secure :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe524006fd0 tx=0x7fe524008040 comp rx=0 tx=0).stop 2023-10-06T03:10:42.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5300b53d0 0x7fe53013e960 secure :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe530089390 tx=0x7fe520017000 comp rx=0 tx=0).stop 2023-10-06T03:10:42.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe530159ec0 0x7fe530151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:42.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 --2- 172.21.15.92:0/737431673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5300cf350 0x7fe5301512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:42.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.592+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 >> 172.21.15.92:0/737431673 conn(0x7fe5300999f0 msgr2=0x7fe5300d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:42.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.594+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 shutdown_connections 2023-10-06T03:10:42.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.594+0000 7fe534bee700 1 -- 172.21.15.92:0/737431673 wait complete. 2023-10-06T03:10:42.596 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:10:42.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:10:42.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.732+0000 7f8d5ac4b700 1 Processor -- start 2023-10-06T03:10:42.734 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.732+0000 7f8d5ac4b700 1 -- start start 2023-10-06T03:10:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d5ac4b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d54069c70 0x7f8d540cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d5ac4b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540ce380 con 0x7f8d54159950 2023-10-06T03:10:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d5ac4b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540cec20 con 0x7f8d540af320 2023-10-06T03:10:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d589e7700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d54159950 0x7f8d54069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49386/0 (socket says 172.21.15.92:49386) 2023-10-06T03:10:42.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d589e7700 1 -- 172.21.15.92:0/2323103320 learned_addr learned my addr 172.21.15.92:0/2323103320 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/2323103320 --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540afa90 con 0x7f8d54069c70 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d53fff700 1 --2- 172.21.15.92:0/2323103320 >> v2:172.21.15.92:3300/0 conn(0x7f8d54069c70 0x7f8d540cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d537fe700 1 -- 172.21.15.92:0/2323103320 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748388840 0 0) 0x7f8d540ce380 con 0x7f8d54159950 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d537fe700 1 -- 172.21.15.92:0/2323103320 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d3c003570 con 0x7f8d54159950 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d53fff700 1 -- 172.21.15.92:0/2323103320 >> v1:172.21.15.136:6789/0 conn(0x7f8d540af320 legacy=0x7f8d540cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d53fff700 1 -- 172.21.15.92:0/2323103320 >> v1:172.21.15.92:6789/0 conn(0x7f8d54159950 legacy=0x7f8d54069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.733+0000 7f8d53fff700 1 -- 172.21.15.92:0/2323103320 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540ce5d0 con 0x7f8d54069c70 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d53fff700 1 --2- 172.21.15.92:0/2323103320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54069c70 0x7f8d540cbb90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8d480098e0 tx=0x7f8d480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5068b7a8d9093 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d537fe700 1 -- 172.21.15.92:0/2323103320 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48011630 con 0x7f8d54069c70 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d537fe700 1 -- 172.21.15.92:0/2323103320 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48011790 con 0x7f8d54069c70 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d537fe700 1 -- 172.21.15.92:0/2323103320 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48011b30 con 0x7f8d54069c70 2023-10-06T03:10:42.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/2323103320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54069c70 msgr2=0x7f8d540cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/2323103320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54069c70 0x7f8d540cbb90 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8d480098e0 tx=0x7f8d480091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/2323103320 shutdown_connections 2023-10-06T03:10:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/2323103320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54069c70 0x7f8d540cbb90 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/2323103320 >> 172.21.15.92:0/2323103320 conn(0x7f8d54068e00 msgr2=0x7f8d540d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/2323103320 shutdown_connections 2023-10-06T03:10:42.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.734+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/2323103320 wait complete. 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.735+0000 7f8d5ac4b700 1 Processor -- start 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 -- start start 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d54069c70 0x7f8d540b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d540af320 0x7f8d5414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 0x7f8d540b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b4b40 con 0x7f8d54159950 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b3f30 con 0x7f8d540af320 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d5ac4b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b4210 con 0x7f8d54069c70 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d591e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 0x7f8d540b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:42.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d591e8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 0x7f8d540b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44914/0 (socket says 172.21.15.92:44914) 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.736+0000 7f8d591e8700 1 -- 172.21.15.92:0/48268709 learned_addr learned my addr 172.21.15.92:0/48268709 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d591e8700 1 -- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d54069c70 msgr2=0x7f8d540b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d53fff700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d540af320 0x7f8d5414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d591e8700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d54069c70 0x7f8d540b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d591e8700 1 -- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d540af320 msgr2=0x7f8d5414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d591e8700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d540af320 0x7f8d5414d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d591e8700 1 -- 172.21.15.92:0/48268709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009420 con 0x7f8d54159950 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d591e8700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 0x7f8d540b3550 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400b5e0 tx=0x7f8d44009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:42.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400f510 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4400f690 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400fa50 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540b44f0 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.737+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5414d780 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.738+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540718b0 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.739+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8d4401a400 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.739+0000 7f8d50ff9700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d40067320 0x7f8d400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.739+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f8d44084620 con 0x7f8d54159950 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.739+0000 7f8d589e7700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d40067320 0x7f8d400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:42.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.739+0000 7f8d589e7700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d40067320 0x7f8d400697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c005c50 tx=0x7f8d4c001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:42.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.743+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d440513b0 con 0x7f8d54159950 2023-10-06T03:10:42.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:42.927+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8d540b4c80 con 0x7f8d54159950 2023-10-06T03:10:43.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:10:43.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:10:43.593 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.591+0000 7f8d50ff9700 1 -- 172.21.15.92:0/48268709 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v184) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8d440549d0 con 0x7f8d54159950 2023-10-06T03:10:43.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.598+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d40067320 msgr2=0x7f8d400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.598+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d40067320 0x7f8d400697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c005c50 tx=0x7f8d4c001640 comp rx=0 tx=0).stop 2023-10-06T03:10:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.598+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 msgr2=0x7f8d540b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.598+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 0x7f8d540b3550 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400b5e0 tx=0x7f8d44009120 comp rx=0 tx=0).stop 2023-10-06T03:10:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.599+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 shutdown_connections 2023-10-06T03:10:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.599+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8d40067320 0x7f8d400697d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.599+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d54159950 0x7f8d540b3550 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.599+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d54069c70 0x7f8d540b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.599+0000 7f8d5ac4b700 1 --2- 172.21.15.92:0/48268709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d540af320 0x7f8d5414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.600+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 >> 172.21.15.92:0/48268709 conn(0x7f8d54068e00 msgr2=0x7f8d540c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.600+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 shutdown_connections 2023-10-06T03:10:43.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.600+0000 7f8d5ac4b700 1 -- 172.21.15.92:0/48268709 wait complete. 2023-10-06T03:10:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:10:43.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:10:43.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.735+0000 7f4250ef9700 1 Processor -- start 2023-10-06T03:10:43.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.735+0000 7f4250ef9700 1 -- start start 2023-10-06T03:10:43.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4250ef9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4250ef9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f424c159c60 0x7f424c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4250ef9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f424c15db10 0x7f424c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4250ef9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c15a160 con 0x7f424c159c60 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4250ef9700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c0d13a0 con 0x7f424c15db10 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4250ef9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c15e140 con 0x7f424c159890 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f424c159c60 0x7f424c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f424c159c60 0x7f424c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44930/0 (socket says 172.21.15.92:44930) 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 -- 172.21.15.92:0/3809404385 learned_addr learned my addr 172.21.15.92:0/3809404385 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f424a59c700 1 --2- 172.21.15.92:0/3809404385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f424ad9d700 1 --2- 172.21.15.92:0/3809404385 >> v2:172.21.15.136:3300/0 conn(0x7f424c15db10 0x7f424c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 -- 172.21.15.92:0/3809404385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 msgr2=0x7f424c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 --2- 172.21.15.92:0/3809404385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 -- 172.21.15.92:0/3809404385 >> v2:172.21.15.136:3300/0 conn(0x7f424c15db10 msgr2=0x7f424c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 --2- 172.21.15.92:0/3809404385 >> v2:172.21.15.136:3300/0 conn(0x7f424c15db10 0x7f424c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.736+0000 7f4241d9b700 1 -- 172.21.15.92:0/3809404385 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c15a3d0 con 0x7f424c159c60 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4241d9b700 1 --2- 172.21.15.92:0/3809404385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c15d5d0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f424c089b50 tx=0x7f4234009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aec7e314d14549 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4249d9b700 1 -- 172.21.15.92:0/3809404385 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400b860 con 0x7f424c159c60 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4249d9b700 1 -- 172.21.15.92:0/3809404385 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400b9c0 con 0x7f424c159c60 2023-10-06T03:10:43.741 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4249d9b700 1 -- 172.21.15.92:0/3809404385 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400bd60 con 0x7f424c159c60 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 -- 172.21.15.92:0/3809404385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 msgr2=0x7f424c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 --2- 172.21.15.92:0/3809404385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c15d5d0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f424c089b50 tx=0x7f4234009250 comp rx=0 tx=0).stop 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 -- 172.21.15.92:0/3809404385 shutdown_connections 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 --2- 172.21.15.92:0/3809404385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 --2- 172.21.15.92:0/3809404385 >> v2:172.21.15.136:3300/0 conn(0x7f424c15db10 0x7f424c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 --2- 172.21.15.92:0/3809404385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c15d5d0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 -- 172.21.15.92:0/3809404385 >> 172.21.15.92:0/3809404385 conn(0x7f424c068e20 msgr2=0x7f424c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 -- 172.21.15.92:0/3809404385 shutdown_connections 2023-10-06T03:10:43.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.737+0000 7f4250ef9700 1 -- 172.21.15.92:0/3809404385 wait complete. 2023-10-06T03:10:43.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.738+0000 7f4250ef9700 1 Processor -- start 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.738+0000 7f4250ef9700 1 -- start start 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4250ef9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c1474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4250ef9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4250ef9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f424c15db10 0x7f424c0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4250ef9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c169d00 con 0x7f424c159c60 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4250ef9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0ce6c0 con 0x7f424c15db10 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4250ef9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c16a860 con 0x7f424c159890 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4241d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4241d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44946/0 (socket says 172.21.15.92:44946) 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f4241d9b700 1 -- 172.21.15.92:0/539414680 learned_addr learned my addr 172.21.15.92:0/539414680 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f424a59c700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c1474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.739+0000 7f424ad9d700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f424c15db10 0x7f424c0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4241d9b700 1 -- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 msgr2=0x7f424c1474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4241d9b700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c1474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4241d9b700 1 -- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f424c15db10 msgr2=0x7f424c0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4241d9b700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f424c15db10 0x7f424c0b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4241d9b700 1 -- 172.21.15.92:0/539414680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340094f0 con 0x7f424c159c60 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4241d9b700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c0ce180 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f424c070e20 tx=0x7f4234017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:43.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234011dc0 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234024440 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c169f30 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234024800 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.740+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c16a4f0 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.741+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c0b4c60 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.741+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f423401a420 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.742+0000 7f4242ffd700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f422c067320 0x7f422c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.742+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f4234086cc0 con 0x7f424c159c60 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.742+0000 7f424a59c700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f422c067320 0x7f422c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:43.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.742+0000 7f424a59c700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f422c067320 0x7f422c0697d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f423c006fd0 tx=0x7f423c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:43.749 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.746+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f423408d080 con 0x7f424c159c60 2023-10-06T03:10:43.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:43.972+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f424c04c3c0 con 0x7f424c159c60 2023-10-06T03:10:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.597+0000 7f4242ffd700 1 -- 172.21.15.92:0/539414680 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 32 and data pool 33 v121) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4234053850 con 0x7f424c159c60 2023-10-06T03:10:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f422c067320 msgr2=0x7f422c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f422c067320 0x7f422c0697d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f423c006fd0 tx=0x7f423c008040 comp rx=0 tx=0).stop 2023-10-06T03:10:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 msgr2=0x7f424c0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c0ce180 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f424c070e20 tx=0x7f4234017000 comp rx=0 tx=0).stop 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 shutdown_connections 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f422c067320 0x7f422c0697d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f424c15db10 0x7f424c0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f424c159890 0x7f424c1474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 --2- 172.21.15.92:0/539414680 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f424c159c60 0x7f424c0ce180 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.601+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 >> 172.21.15.92:0/539414680 conn(0x7f424c068e20 msgr2=0x7f424c153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.602+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 shutdown_connections 2023-10-06T03:10:44.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.602+0000 7f4250ef9700 1 -- 172.21.15.92:0/539414680 wait complete. 2023-10-06T03:10:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 32 and data pool 33 2023-10-06T03:10:44.626 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:44.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.738+0000 7fd77e40a700 1 Processor -- start 2023-10-06T03:10:44.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.738+0000 7fd77e40a700 1 -- start start 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77e40a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 0x7fd7780cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77e40a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd77806bbe0 0x7fd77806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77e40a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7780adcf0 con 0x7fd77806eab0 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77e40a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77806cbf0 con 0x7fd77806bbe0 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77e40a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780aceb0 con 0x7fd778156950 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77dc09700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd77806eab0 0x7fd7780c78a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49402/0 (socket says 172.21.15.92:49402) 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd77dc09700 1 -- 172.21.15.92:0/3037405243 learned_addr learned my addr 172.21.15.92:0/3037405243 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840538784 0 0) 0x7fd7780adcf0 con 0x7fd77806eab0 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.739+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd764003570 con 0x7fd77806eab0 2023-10-06T03:10:44.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77d408700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 0x7fd7780cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 --2- 172.21.15.92:0/3037405243 >> v2:172.21.15.136:3300/0 conn(0x7fd77806bbe0 0x7fd77806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2646822979 0 0) 0x7fd764003570 con 0x7fd77806eab0 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 msgr2=0x7fd7780cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 0x7fd7780cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 >> v2:172.21.15.136:3300/0 conn(0x7fd77806bbe0 msgr2=0x7fd77806e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 --2- 172.21.15.92:0/3037405243 >> v2:172.21.15.136:3300/0 conn(0x7fd77806bbe0 0x7fd77806e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780adfb0 con 0x7fd77806eab0 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd770001470 con 0x7fd77806eab0 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 >> v1:172.21.15.92:6789/0 conn(0x7fd77806eab0 legacy=0x7fd7780c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd764003a90 0x7fd764003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.745 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd764004810 0x7fd764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 0x7fd7640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77806cbf0 con 0x7fd764004810 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780aceb0 con 0x7fd764003a90 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76400ba60 con 0x7fd778156950 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd764004810 0x7fd764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77d408700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd764003a90 0x7fd764003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77dc09700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 0x7fd7640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 -- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 msgr2=0x7fd7640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd778156950 0x7fd7640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 -- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd764003a90 msgr2=0x7fd764003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:44.746 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd764003a90 0x7fd764003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 -- 172.21.15.92:0/3037405243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764003570 con 0x7fd764004810 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.740+0000 7fd77cc07700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd764004810 0x7fd764004c80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd768009c30 tx=0x7fd7680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525b8ef3289f0651 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76800e710 con 0x7fd764004810 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76800e870 con 0x7fd764004810 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd76ffff700 1 -- 172.21.15.92:0/3037405243 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76800ec30 con 0x7fd764004810 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77e40a700 1 -- 172.21.15.92:0/3037405243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd764004810 msgr2=0x7fd764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77e40a700 1 --2- 172.21.15.92:0/3037405243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd764004810 0x7fd764004c80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd768009c30 tx=0x7fd7680091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77dc09700 1 -- 172.21.15.92:0/3037405243 reap_dead start 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77e40a700 1 -- 172.21.15.92:0/3037405243 shutdown_connections 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77e40a700 1 -- 172.21.15.92:0/3037405243 >> 172.21.15.92:0/3037405243 conn(0x7fd77806b490 msgr2=0x7fd7780cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:44.747 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77e40a700 1 -- 172.21.15.92:0/3037405243 shutdown_connections 2023-10-06T03:10:44.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.741+0000 7fd77e40a700 1 -- 172.21.15.92:0/3037405243 wait complete. 2023-10-06T03:10:44.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.742+0000 7fd77e40a700 1 Processor -- start 2023-10-06T03:10:44.748 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.742+0000 7fd77e40a700 1 -- start start 2023-10-06T03:10:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77e40a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd764004810 0x7fd778159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77e40a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd778156e90 0x7fd778157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77e40a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 0x7fd778153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77e40a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781584f0 con 0x7fd778153750 2023-10-06T03:10:44.750 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77e40a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781540f0 con 0x7fd778156e90 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77e40a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7781578e0 con 0x7fd764004810 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77dc09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 0x7fd778153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77dc09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 0x7fd778153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44958/0 (socket says 172.21.15.92:44958) 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77dc09700 1 -- 172.21.15.92:0/2162692623 learned_addr learned my addr 172.21.15.92:0/2162692623 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77d408700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd764004810 0x7fd778159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.743+0000 7fd77cc07700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd778156e90 0x7fd778157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77dc09700 1 -- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd764004810 msgr2=0x7fd778159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77dc09700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd764004810 0x7fd778159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77dc09700 1 -- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd778156e90 msgr2=0x7fd778157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77dc09700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd778156e90 0x7fd778157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.751 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77dc09700 1 -- 172.21.15.92:0/2162692623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd778157bc0 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77cc07700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd778156e90 0x7fd778157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77dc09700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 0x7fd778153bb0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd77806b980 tx=0x7fd770001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000f8d0 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000fa30 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd778157ea0 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000fdb0 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.744+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7780c5b00 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.745+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7781586a0 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.745+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd7700135c0 con 0x7fd778153750 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.746+0000 7fd76d7fa700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd75c067320 0x7fd75c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:44.752 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.746+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd770084760 con 0x7fd778153750 2023-10-06T03:10:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.746+0000 7fd77d408700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd75c067320 0x7fd75c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.746+0000 7fd77d408700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd75c067320 0x7fd75c0697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd7740094d0 tx=0x7fd77400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:44.753 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.750+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd77004e5a0 con 0x7fd778153750 2023-10-06T03:10:44.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.935+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd77804c3c0 con 0x7fd778153750 2023-10-06T03:10:44.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.936+0000 7fd76d7fa700 1 -- 172.21.15.92:0/2162692623 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fd77001a020 con 0x7fd778153750 2023-10-06T03:10:44.938 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:44.938 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":185,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:43.973921+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-10-06T03:10:42.946799+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd75c067320 msgr2=0x7fd75c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd75c067320 0x7fd75c0697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd7740094d0 tx=0x7fd77400b040 comp rx=0 tx=0).stop 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 msgr2=0x7fd778153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 0x7fd778153bb0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fd77806b980 tx=0x7fd770001e50 comp rx=0 tx=0).stop 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 shutdown_connections 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd75c067320 0x7fd75c0697d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd778153750 0x7fd778153bb0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd764004810 0x7fd778159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 --2- 172.21.15.92:0/2162692623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd778156e90 0x7fd778157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:44.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 >> 172.21.15.92:0/2162692623 conn(0x7fd77806b490 msgr2=0x7fd778154960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:44.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 shutdown_connections 2023-10-06T03:10:44.941 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:44.938+0000 7fd77e40a700 1 -- 172.21.15.92:0/2162692623 wait complete. 2023-10-06T03:10:44.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.065+0000 7fd7f6ca4700 1 Processor -- start 2023-10-06T03:10:45.067 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.065+0000 7fd7f6ca4700 1 -- start start 2023-10-06T03:10:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f6ca4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd7f015d230 0x7fd7f015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f6ca4700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f00b2980 con 0x7fd7f01597f0 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f6ca4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f015db70 con 0x7fd7f0159420 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f6ca4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00b11d0 con 0x7fd7f015d230 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f64a3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd7f015d230 0x7fd7f015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f64a3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd7f015d230 0x7fd7f015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44962/0 (socket says 172.21.15.92:44962) 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f54a1700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd7f01597f0 0x7fd7f00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49404/0 (socket says 172.21.15.92:49404) 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f64a3700 1 -- 172.21.15.92:0/2892835313 learned_addr learned my addr 172.21.15.92:0/2892835313 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f5ca2700 1 --1- 172.21.15.92:0/2892835313 >> v1:172.21.15.136:6789/0 conn(0x7fd7f0159420 0x7fd7f00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:38318/0 (socket says 172.21.15.92:38318) 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f64a3700 1 -- 172.21.15.92:0/2892835313 >> v1:172.21.15.136:6789/0 conn(0x7fd7f0159420 legacy=0x7fd7f00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f64a3700 1 -- 172.21.15.92:0/2892835313 >> v1:172.21.15.92:6789/0 conn(0x7fd7f01597f0 legacy=0x7fd7f00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f64a3700 1 -- 172.21.15.92:0/2892835313 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00b14b0 con 0x7fd7f015d230 2023-10-06T03:10:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.066+0000 7fd7f4ca0700 1 -- 172.21.15.92:0/2892835313 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872648656 0 0) 0x7fd7f00b2980 con 0x7fd7f01597f0 2023-10-06T03:10:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f64a3700 1 --2- 172.21.15.92:0/2892835313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f015d230 0x7fd7f015d630 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80099b0 tx=0x7fd7e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4b4fdbc1a4725c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f4ca0700 1 -- 172.21.15.92:0/2892835313 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800b860 con 0x7fd7f015d230 2023-10-06T03:10:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f4ca0700 1 -- 172.21.15.92:0/2892835313 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800b9c0 con 0x7fd7f015d230 2023-10-06T03:10:45.070 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f4ca0700 1 -- 172.21.15.92:0/2892835313 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800bd60 con 0x7fd7f015d230 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/2892835313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f015d230 msgr2=0x7fd7f015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/2892835313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f015d230 0x7fd7f015d630 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80099b0 tx=0x7fd7e8009250 comp rx=0 tx=0).stop 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/2892835313 shutdown_connections 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/2892835313 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f015d230 0x7fd7f015d630 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/2892835313 >> 172.21.15.92:0/2892835313 conn(0x7fd7f00d0660 msgr2=0x7fd7f00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/2892835313 shutdown_connections 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.067+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/2892835313 wait complete. 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.068+0000 7fd7f6ca4700 1 Processor -- start 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 -- start start 2023-10-06T03:10:45.074 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 0x7fd7f0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f01597f0 0x7fd7f016a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f015d230 0x7fd7f00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0154ff0 con 0x7fd7f0159420 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f016b2e0 con 0x7fd7f015d230 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f6ca4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c63d0 con 0x7fd7f01597f0 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f5ca2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 0x7fd7f0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f5ca2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 0x7fd7f0154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44972/0 (socket says 172.21.15.92:44972) 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f5ca2700 1 -- 172.21.15.92:0/1533547083 learned_addr learned my addr 172.21.15.92:0/1533547083 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f64a3700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f015d230 0x7fd7f00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.069+0000 7fd7f54a1700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f01597f0 0x7fd7f016a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f5ca2700 1 -- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f01597f0 msgr2=0x7fd7f016a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f5ca2700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f01597f0 0x7fd7f016a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f5ca2700 1 -- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f015d230 msgr2=0x7fd7f00b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f5ca2700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f015d230 0x7fd7f00b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f5ca2700 1 -- 172.21.15.92:0/1533547083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e80094f0 con 0x7fd7f0159420 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f54a1700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f01597f0 0x7fd7f016a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f5ca2700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 0x7fd7f0154ab0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00cd80 tx=0x7fd7ec001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec005df0 con 0x7fd7f0159420 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec00a960 con 0x7fd7f0159420 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f016ac90 con 0x7fd7f0159420 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00ad20 con 0x7fd7f0159420 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.070+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f016a990 con 0x7fd7f0159420 2023-10-06T03:10:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.071+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f00d1080 con 0x7fd7f0159420 2023-10-06T03:10:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.071+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd7ec007300 con 0x7fd7f0159420 2023-10-06T03:10:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.072+0000 7fd7e5ffb700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd7d4067320 0x7fd7d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.072+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd7ec084fa0 con 0x7fd7f0159420 2023-10-06T03:10:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.072+0000 7fd7f54a1700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd7d4067320 0x7fd7d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.072+0000 7fd7f54a1700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd7d4067320 0x7fd7d40697d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00c79e0 tx=0x7fd7e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.076+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7ec051b50 con 0x7fd7f0159420 2023-10-06T03:10:45.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.259+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f004c3c0 con 0x7fd7f0159420 2023-10-06T03:10:45.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.260+0000 7fd7e5ffb700 1 -- 172.21.15.92:0/1533547083 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16893 (secure 0 0 0) 0x7fd7ec055170 con 0x7fd7f0159420 2023-10-06T03:10:45.262 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:45.262 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":185,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:43.973921+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-10-06T03:10:42.946799+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd7d4067320 msgr2=0x7fd7d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd7d4067320 0x7fd7d40697d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00c79e0 tx=0x7fd7e0009040 comp rx=0 tx=0).stop 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 msgr2=0x7fd7f0154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 0x7fd7f0154ab0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00cd80 tx=0x7fd7ec001b90 comp rx=0 tx=0).stop 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 shutdown_connections 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd7d4067320 0x7fd7d40697d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd7f0159420 0x7fd7f0154ab0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f015d230 0x7fd7f00b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 --2- 172.21.15.92:0/1533547083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f01597f0 0x7fd7f016a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 >> 172.21.15.92:0/1533547083 conn(0x7fd7f00d0660 msgr2=0x7fd7f0155750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 shutdown_connections 2023-10-06T03:10:45.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.262+0000 7fd7f6ca4700 1 -- 172.21.15.92:0/1533547083 wait complete. 2023-10-06T03:10:45.276 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:10:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.392+0000 7fe5c45a6700 1 Processor -- start 2023-10-06T03:10:45.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.392+0000 7fe5c45a6700 1 -- start start 2023-10-06T03:10:45.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.393+0000 7fe5c45a6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06c0a0 0x7fe5bc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.393+0000 7fe5c45a6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5bc15df50 0x7fe5bc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.393+0000 7fe5c45a6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5bc15d7e0 con 0x7fe5bc06bcd0 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.393+0000 7fe5c45a6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5bc15dac0 con 0x7fe5bc15df50 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.393+0000 7fe5c45a6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc15aa30 con 0x7fe5bc06c0a0 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5bc15df50 0x7fe5bc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe5bc15df50 0x7fe5bc15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44988/0 (socket says 172.21.15.92:44988) 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 -- 172.21.15.92:0/1515676685 learned_addr learned my addr 172.21.15.92:0/1515676685 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c1b41700 1 --2- 172.21.15.92:0/1515676685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06c0a0 0x7fe5bc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 -- 172.21.15.92:0/1515676685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06c0a0 msgr2=0x7fe5bc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 --2- 172.21.15.92:0/1515676685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06c0a0 0x7fe5bc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 -- 172.21.15.92:0/1515676685 >> v1:172.21.15.136:6789/0 conn(0x7fe5bc06bcd0 legacy=0x7fe5bc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 -- 172.21.15.92:0/1515676685 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc15ad10 con 0x7fe5bc15df50 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c2b43700 1 --2- 172.21.15.92:0/1515676685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc15e3c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40099b0 tx=0x7fe5b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63aa7c748cf945a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c1340700 1 -- 172.21.15.92:0/1515676685 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400b860 con 0x7fe5bc15df50 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c1340700 1 -- 172.21.15.92:0/1515676685 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b400b9c0 con 0x7fe5bc15df50 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.394+0000 7fe5c1340700 1 -- 172.21.15.92:0/1515676685 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400bd80 con 0x7fe5bc15df50 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 -- 172.21.15.92:0/1515676685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 msgr2=0x7fe5bc15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/1515676685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc15e3c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40099b0 tx=0x7fe5b4009250 comp rx=0 tx=0).stop 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 -- 172.21.15.92:0/1515676685 shutdown_connections 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/1515676685 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc15e3c0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/1515676685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06c0a0 0x7fe5bc0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 -- 172.21.15.92:0/1515676685 >> 172.21.15.92:0/1515676685 conn(0x7fe5bc0ce600 msgr2=0x7fe5bc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:45.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 -- 172.21.15.92:0/1515676685 shutdown_connections 2023-10-06T03:10:45.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.395+0000 7fe5c45a6700 1 -- 172.21.15.92:0/1515676685 wait complete. 2023-10-06T03:10:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.396+0000 7fe5c45a6700 1 Processor -- start 2023-10-06T03:10:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.396+0000 7fe5c45a6700 1 -- start start 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06bcd0 0x7fe5bc14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5bc06c0a0 0x7fe5bc16d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc16de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0c85e0 con 0x7fe5bc15df50 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0b3670 con 0x7fe5bc06c0a0 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5bc0b3eb0 con 0x7fe5bc06bcd0 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc16de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc16de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:44990/0 (socket says 172.21.15.92:44990) 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 -- 172.21.15.92:0/3390543942 learned_addr learned my addr 172.21.15.92:0/3390543942 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2342700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06bcd0 0x7fe5bc14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c1b41700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5bc06c0a0 0x7fe5bc16d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 -- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06bcd0 msgr2=0x7fe5bc14a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06bcd0 0x7fe5bc14a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 -- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5bc06c0a0 msgr2=0x7fe5bc16d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5bc06c0a0 0x7fe5bc16d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 -- 172.21.15.92:0/3390543942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40094f0 con 0x7fe5bc15df50 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c1b41700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5bc06c0a0 0x7fe5bc16d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c2b43700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc16de70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4006fd0 tx=0x7fe5b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b401f070 con 0x7fe5bc15df50 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b401a420 con 0x7fe5bc15df50 2023-10-06T03:10:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5bc0b4340 con 0x7fe5bc15df50 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b400b860 con 0x7fe5bc15df50 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.397+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5bc0b3860 con 0x7fe5bc15df50 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.398+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5bc04c3c0 con 0x7fe5bc15df50 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.399+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe5b401aa30 con 0x7fe5bc15df50 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.399+0000 7fe5b27fc700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a0067320 0x7fe5a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.399+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe5b4086460 con 0x7fe5bc15df50 2023-10-06T03:10:45.404 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.399+0000 7fe5c2342700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a0067320 0x7fe5a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.400+0000 7fe5c2342700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a0067320 0x7fe5a00697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8007fd0 tx=0x7fe5b8001650 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:45.405 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.403+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5b4053010 con 0x7fe5bc15df50 2023-10-06T03:10:45.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:45.636+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe5bc0c51f0 con 0x7fe5bc15df50 2023-10-06T03:10:46.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.622+0000 7fe5b27fc700 1 -- 172.21.15.92:0/3390543942 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v124) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe5b4056630 con 0x7fe5bc15df50 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a0067320 msgr2=0x7fe5a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a0067320 0x7fe5a00697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8007fd0 tx=0x7fe5b8001650 comp rx=0 tx=0).stop 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 msgr2=0x7fe5bc16de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc16de70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4006fd0 tx=0x7fe5b4017000 comp rx=0 tx=0).stop 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 shutdown_connections 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe5a0067320 0x7fe5a00697d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe5bc15df50 0x7fe5bc16de70 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5bc06bcd0 0x7fe5bc14a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.627 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 --2- 172.21.15.92:0/3390543942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5bc06c0a0 0x7fe5bc16d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 >> 172.21.15.92:0/3390543942 conn(0x7fe5bc0ce600 msgr2=0x7fe5bc06d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:46.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 shutdown_connections 2023-10-06T03:10:46.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.625+0000 7fe5c45a6700 1 -- 172.21.15.92:0/3390543942 wait complete. 2023-10-06T03:10:46.639 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:10:46.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.756+0000 7fa9b911a700 1 Processor -- start 2023-10-06T03:10:46.758 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b911a700 1 -- start start 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b911a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b40b4e70 0x7fa9b40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b911a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9b40b16d0 0x7fa9b40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b911a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b40b2080 con 0x7fa9b415d240 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b911a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40cfb00 con 0x7fa9b40b16d0 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b911a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b40b5c40 con 0x7fa9b40b4e70 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.757+0000 7fa9b37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9b40b16d0 0x7fa9b40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa9b40b16d0 0x7fa9b40b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45002/0 (socket says 172.21.15.92:45002) 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 -- 172.21.15.92:0/1646955331 learned_addr learned my addr 172.21.15.92:0/1646955331 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 -- 172.21.15.92:0/1646955331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b40b4e70 msgr2=0x7fa9b40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:46.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b3fff700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b40b4e70 0x7fa9b40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b40b4e70 0x7fa9b40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 -- 172.21.15.92:0/1646955331 >> v1:172.21.15.136:6789/0 conn(0x7fa9b415d240 legacy=0x7fa9b415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 -- 172.21.15.92:0/1646955331 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b40b5f20 con 0x7fa9b40b16d0 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b3fff700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b40b4e70 0x7fa9b40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b37fe700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b40b1b40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80099b0 tx=0x7fa9a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2172a05355b1a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b2ffd700 1 -- 172.21.15.92:0/1646955331 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800b820 con 0x7fa9b40b16d0 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b2ffd700 1 -- 172.21.15.92:0/1646955331 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a800b980 con 0x7fa9b40b16d0 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.758+0000 7fa9b2ffd700 1 -- 172.21.15.92:0/1646955331 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a800bd20 con 0x7fa9b40b16d0 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 -- 172.21.15.92:0/1646955331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 msgr2=0x7fa9b40b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b40b1b40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80099b0 tx=0x7fa9a8009250 comp rx=0 tx=0).stop 2023-10-06T03:10:46.762 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 -- 172.21.15.92:0/1646955331 shutdown_connections 2023-10-06T03:10:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b40b4e70 0x7fa9b40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1646955331 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b40b1b40 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 -- 172.21.15.92:0/1646955331 >> 172.21.15.92:0/1646955331 conn(0x7fa9b40d10c0 msgr2=0x7fa9b40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 -- 172.21.15.92:0/1646955331 shutdown_connections 2023-10-06T03:10:46.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.759+0000 7fa9b911a700 1 -- 172.21.15.92:0/1646955331 wait complete. 2023-10-06T03:10:46.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.760+0000 7fa9b911a700 1 Processor -- start 2023-10-06T03:10:46.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.760+0000 7fa9b911a700 1 -- start start 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b41459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9b40b4e70 0x7fa9b4158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b415d240 0x7fa9b4159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4145f20 con 0x7fa9b40b16d0 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4149680 con 0x7fa9b40b4e70 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4158550 con 0x7fa9b415d240 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b41459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b41459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45006/0 (socket says 172.21.15.92:45006) 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 -- 172.21.15.92:0/1818072943 learned_addr learned my addr 172.21.15.92:0/1818072943 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:46.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 -- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b415d240 msgr2=0x7fa9b4159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b37fe700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9b40b4e70 0x7fa9b4158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b8919700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b415d240 0x7fa9b4159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b415d240 0x7fa9b4159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 -- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9b40b4e70 msgr2=0x7fa9b4158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9b40b4e70 0x7fa9b4158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 -- 172.21.15.92:0/1818072943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a80094f0 con 0x7fa9b40b16d0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b3fff700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b41459e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c830 tx=0x7fa9a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011600 con 0x7fa9b40b16d0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a4011760 con 0x7fa9b40b16d0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011b20 con 0x7fa9b40b16d0 2023-10-06T03:10:46.767 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b4148710 con 0x7fa9b40b16d0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.761+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b4148cd0 con 0x7fa9b40b16d0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.762+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b4088990 con 0x7fa9b40b16d0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.763+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa9a400a3e0 con 0x7fa9b40b16d0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.763+0000 7fa99bfff700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa9a0067320 0x7fa9a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.763+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa9a40853a0 con 0x7fa9b40b16d0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.763+0000 7fa9b37fe700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa9a0067320 0x7fa9a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.763+0000 7fa9b37fe700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa9a0067320 0x7fa9a00697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80099b0 tx=0x7fa9a8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:46.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.767+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9a4051dc0 con 0x7fa9b40b16d0 2023-10-06T03:10:46.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:46.992+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa9b404bb80 con 0x7fa9b40b16d0 2023-10-06T03:10:47.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.623+0000 7fa99bfff700 1 -- 172.21.15.92:0/1818072943 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v125) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa9a40553e0 con 0x7fa9b40b16d0 2023-10-06T03:10:47.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa9a0067320 msgr2=0x7fa9a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa9a0067320 0x7fa9a00697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80099b0 tx=0x7fa9a8016000 comp rx=0 tx=0).stop 2023-10-06T03:10:47.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 msgr2=0x7fa9b41459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b41459e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400c830 tx=0x7fa9a4008040 comp rx=0 tx=0).stop 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 shutdown_connections 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa9a0067320 0x7fa9a00697d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9b40b16d0 0x7fa9b41459e0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9b415d240 0x7fa9b4159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 --2- 172.21.15.92:0/1818072943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9b40b4e70 0x7fa9b4158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.626+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 >> 172.21.15.92:0/1818072943 conn(0x7fa9b40d10c0 msgr2=0x7fa9b415c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.627+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 shutdown_connections 2023-10-06T03:10:47.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.627+0000 7fa9b911a700 1 -- 172.21.15.92:0/1818072943 wait complete. 2023-10-06T03:10:47.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:47.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.759+0000 7f8e1e91e700 1 Processor -- start 2023-10-06T03:10:47.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.759+0000 7f8e1e91e700 1 -- start start 2023-10-06T03:10:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1e91e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8e18159890 0x7f8e1815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1e91e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159c60 0x7f8e1815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1e91e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e1815db10 0x7f8e18069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.763 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1e91e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1815a160 con 0x7f8e1815db10 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1e91e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e180d13a0 con 0x7f8e18159890 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1e91e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1815e140 con 0x7f8e18159c60 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1cebb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e1815db10 0x7f8e18069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1cebb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8e1815db10 0x7f8e18069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45020/0 (socket says 172.21.15.92:45020) 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e1cebb700 1 -- 172.21.15.92:0/483097113 learned_addr learned my addr 172.21.15.92:0/483097113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e177fe700 1 --2- 172.21.15.92:0/483097113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159c60 0x7f8e1815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.760+0000 7f8e17fff700 1 --2- 172.21.15.92:0/483097113 >> v2:172.21.15.136:3300/0 conn(0x7f8e18159890 0x7f8e1815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1cebb700 1 -- 172.21.15.92:0/483097113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159c60 msgr2=0x7f8e1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1cebb700 1 --2- 172.21.15.92:0/483097113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159c60 0x7f8e1815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1cebb700 1 -- 172.21.15.92:0/483097113 >> v2:172.21.15.136:3300/0 conn(0x7f8e18159890 msgr2=0x7f8e1815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1cebb700 1 --2- 172.21.15.92:0/483097113 >> v2:172.21.15.136:3300/0 conn(0x7f8e18159890 0x7f8e1815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.764 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1cebb700 1 -- 172.21.15.92:0/483097113 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1815a3d0 con 0x7f8e1815db10 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1cebb700 1 --2- 172.21.15.92:0/483097113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e18069560 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8e180d1090 tx=0x7f8e10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a9490c4352d7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e16ffd700 1 -- 172.21.15.92:0/483097113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1000e070 con 0x7f8e1815db10 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e16ffd700 1 -- 172.21.15.92:0/483097113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1000ba50 con 0x7f8e1815db10 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e16ffd700 1 -- 172.21.15.92:0/483097113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100092e0 con 0x7f8e1815db10 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 -- 172.21.15.92:0/483097113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 msgr2=0x7f8e18069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/483097113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e18069560 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8e180d1090 tx=0x7f8e10009840 comp rx=0 tx=0).stop 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 -- 172.21.15.92:0/483097113 shutdown_connections 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/483097113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e18069560 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/483097113 >> v2:172.21.15.136:3300/0 conn(0x7f8e18159890 0x7f8e1815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/483097113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159c60 0x7f8e1815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 -- 172.21.15.92:0/483097113 >> 172.21.15.92:0/483097113 conn(0x7f8e18068e20 msgr2=0x7f8e180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:47.765 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 -- 172.21.15.92:0/483097113 shutdown_connections 2023-10-06T03:10:47.766 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.761+0000 7f8e1e91e700 1 -- 172.21.15.92:0/483097113 wait complete. 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.762+0000 7f8e1e91e700 1 Processor -- start 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 -- start start 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159890 0x7f8e180b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e18159c60 0x7f8e180cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e180cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18153e70 con 0x7f8e1815db10 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18153860 con 0x7f8e18159c60 2023-10-06T03:10:47.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.763+0000 7f8e1e91e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e180cfbc0 con 0x7f8e18159890 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e180cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e180cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45036/0 (socket says 172.21.15.92:45036) 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 -- 172.21.15.92:0/303613099 learned_addr learned my addr 172.21.15.92:0/303613099 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e17fff700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159890 0x7f8e180b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e177fe700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e18159c60 0x7f8e180cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 -- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159890 msgr2=0x7f8e180b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159890 0x7f8e180b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 -- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e18159c60 msgr2=0x7f8e180cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e18159c60 0x7f8e180cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 -- 172.21.15.92:0/303613099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009ae0 con 0x7f8e1815db10 2023-10-06T03:10:47.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1cebb700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e180cade0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8e18070e20 tx=0x7f8e10018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100032f0 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10008d20 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e18153b00 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100256b0 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.764+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e18151010 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.765+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1804c3c0 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.765+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8e10003040 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.766+0000 7f8dfffff700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8e04067320 0x7f8e040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.766+0000 7f8e17fff700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8e04067320 0x7f8e040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.766+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8e10086770 con 0x7f8e1815db10 2023-10-06T03:10:47.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.766+0000 7f8e17fff700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8e04067320 0x7f8e040697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e08006fd0 tx=0x7f8e08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:47.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.770+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e10053240 con 0x7f8e1815db10 2023-10-06T03:10:47.903 INFO:tasks.ceph.mds.a.smithi092.stderr:2023-10-06T03:10:47.901+0000 7f7bbc5ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:10:47.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.994+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e18153fb0 con 0x7f8e1815db10 2023-10-06T03:10:47.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.994+0000 7f8dfffff700 1 -- 172.21.15.92:0/303613099 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f8e10009940 con 0x7f8e1815db10 2023-10-06T03:10:47.996 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:47.996 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:10:47.622584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8e04067320 msgr2=0x7f8e040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8e04067320 0x7f8e040697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8e08006fd0 tx=0x7f8e08008040 comp rx=0 tx=0).stop 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 msgr2=0x7f8e180cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e180cade0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8e18070e20 tx=0x7f8e10018000 comp rx=0 tx=0).stop 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 shutdown_connections 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8e04067320 0x7f8e040697d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e1815db10 0x7f8e180cade0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e18159890 0x7f8e180b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 --2- 172.21.15.92:0/303613099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e18159c60 0x7f8e180cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 >> 172.21.15.92:0/303613099 conn(0x7f8e18068e20 msgr2=0x7f8e18152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 shutdown_connections 2023-10-06T03:10:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:47.996+0000 7f8e1e91e700 1 -- 172.21.15.92:0/303613099 wait complete. 2023-10-06T03:10:47.999 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 125 2023-10-06T03:10:48.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:10:48.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.122+0000 7eff64db9700 1 Processor -- start 2023-10-06T03:10:48.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.122+0000 7eff64db9700 1 -- start start 2023-10-06T03:10:48.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff64db9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7eff60159890 0x7eff6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff64db9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159c60 0x7eff6015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff64db9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff6015db10 0x7eff60069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff64db9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6015a160 con 0x7eff6015db10 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff64db9700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff600d13a0 con 0x7eff60159890 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff64db9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6015e140 con 0x7eff60159c60 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff5ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff6015db10 0x7eff60069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff5ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7eff6015db10 0x7eff60069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45048/0 (socket says 172.21.15.92:45048) 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.123+0000 7eff5ed9d700 1 -- 172.21.15.92:0/745035387 learned_addr learned my addr 172.21.15.92:0/745035387 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5e59c700 1 --2- 172.21.15.92:0/745035387 >> v2:172.21.15.136:3300/0 conn(0x7eff60159890 0x7eff6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5dd9b700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159c60 0x7eff6015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5ed9d700 1 -- 172.21.15.92:0/745035387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159c60 msgr2=0x7eff6015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5ed9d700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159c60 0x7eff6015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5ed9d700 1 -- 172.21.15.92:0/745035387 >> v2:172.21.15.136:3300/0 conn(0x7eff60159890 msgr2=0x7eff6015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5ed9d700 1 --2- 172.21.15.92:0/745035387 >> v2:172.21.15.136:3300/0 conn(0x7eff60159890 0x7eff6015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5ed9d700 1 -- 172.21.15.92:0/745035387 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6015a3d0 con 0x7eff6015db10 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5dd9b700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159c60 0x7eff6015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5ed9d700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6015db10 0x7eff60069560 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7eff600d1090 tx=0x7eff58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33011d05502cebb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5d59a700 1 -- 172.21.15.92:0/745035387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800b860 con 0x7eff6015db10 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5d59a700 1 -- 172.21.15.92:0/745035387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5800b9c0 con 0x7eff6015db10 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff5d59a700 1 -- 172.21.15.92:0/745035387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800bd60 con 0x7eff6015db10 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff64db9700 1 -- 172.21.15.92:0/745035387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6015db10 msgr2=0x7eff60069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.124+0000 7eff64db9700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6015db10 0x7eff60069560 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7eff600d1090 tx=0x7eff58009250 comp rx=0 tx=0).stop 2023-10-06T03:10:48.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 -- 172.21.15.92:0/745035387 shutdown_connections 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff6015db10 0x7eff60069560 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 --2- 172.21.15.92:0/745035387 >> v2:172.21.15.136:3300/0 conn(0x7eff60159890 0x7eff6015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 --2- 172.21.15.92:0/745035387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159c60 0x7eff6015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 -- 172.21.15.92:0/745035387 >> 172.21.15.92:0/745035387 conn(0x7eff60068e20 msgr2=0x7eff600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 -- 172.21.15.92:0/745035387 shutdown_connections 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.125+0000 7eff64db9700 1 -- 172.21.15.92:0/745035387 wait complete. 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 Processor -- start 2023-10-06T03:10:48.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 -- start start 2023-10-06T03:10:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159890 0x7eff60169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 0x7eff6016cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6015db10 0x7eff6016d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff60160420 con 0x7eff60159c60 2023-10-06T03:10:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff60169970 con 0x7eff6015db10 2023-10-06T03:10:48.133 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.126+0000 7eff64db9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff60169390 con 0x7eff60159890 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 0x7eff6016cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 0x7eff6016cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45062/0 (socket says 172.21.15.92:45062) 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 -- 172.21.15.92:0/1805522432 learned_addr learned my addr 172.21.15.92:0/1805522432 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5e59c700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159890 0x7eff60169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5ed9d700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6015db10 0x7eff6016d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 -- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159890 msgr2=0x7eff60169e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159890 0x7eff60169e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 -- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6015db10 msgr2=0x7eff6016d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6015db10 0x7eff6016d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.134 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 -- 172.21.15.92:0/1805522432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580094f0 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff5dd9b700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 0x7eff6016cfa0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c8b0 tx=0x7eff54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011600 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54011760 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff600c7a50 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011b00 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.127+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff60160140 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.128+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6004c3c0 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.129+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7eff540193b0 con 0x7eff60159c60 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.129+0000 7eff4e7fc700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff40067320 0x7eff400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.129+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7eff54088120 con 0x7eff60159c60 2023-10-06T03:10:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.129+0000 7eff5e59c700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff40067320 0x7eff400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.129+0000 7eff5e59c700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff40067320 0x7eff400697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7eff50009fd0 tx=0x7eff50009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.136 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.133+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff54051850 con 0x7eff60159c60 2023-10-06T03:10:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.315+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff6006a5b0 con 0x7eff60159c60 2023-10-06T03:10:48.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.316+0000 7eff4e7fc700 1 -- 172.21.15.92:0/1805522432 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16880 (secure 0 0 0) 0x7eff5401f090 con 0x7eff60159c60 2023-10-06T03:10:48.318 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:48.318 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":186,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:10:45.592368+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-10-06T03:10:42.946799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:10:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff40067320 msgr2=0x7eff400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff40067320 0x7eff400697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7eff50009fd0 tx=0x7eff50009880 comp rx=0 tx=0).stop 2023-10-06T03:10:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 msgr2=0x7eff6016cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 0x7eff6016cfa0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c8b0 tx=0x7eff54008040 comp rx=0 tx=0).stop 2023-10-06T03:10:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 shutdown_connections 2023-10-06T03:10:48.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff40067320 0x7eff400697d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff6015db10 0x7eff6016d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff60159890 0x7eff60169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 --2- 172.21.15.92:0/1805522432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff60159c60 0x7eff6016cfa0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 >> 172.21.15.92:0/1805522432 conn(0x7eff60068e20 msgr2=0x7eff6015b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 shutdown_connections 2023-10-06T03:10:48.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.318+0000 7eff64db9700 1 -- 172.21.15.92:0/1805522432 wait complete. 2023-10-06T03:10:48.332 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:10:48.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.448+0000 7f200aa1e700 1 Processor -- start 2023-10-06T03:10:48.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.448+0000 7f200aa1e700 1 -- start start 2023-10-06T03:10:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.449+0000 7f200aa1e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2004159890 0x7f200415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.449+0000 7f200aa1e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200415a160 con 0x7f2004159c60 2023-10-06T03:10:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.449+0000 7f200aa1e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20040d13a0 con 0x7f2004069560 2023-10-06T03:10:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.449+0000 7f200aa1e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200415e140 con 0x7f2004159890 2023-10-06T03:10:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.449+0000 7f20037fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2004159c60 0x7f200415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:49416/0 (socket says 172.21.15.92:49416) 2023-10-06T03:10:48.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.449+0000 7f20037fe700 1 -- 172.21.15.92:0/1297695188 learned_addr learned my addr 172.21.15.92:0/1297695188 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137621812 0 0) 0x7f200415a160 con 0x7f2004159c60 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2003fff700 1 --2- 172.21.15.92:0/1297695188 >> v2:172.21.15.136:3300/0 conn(0x7f2004159890 0x7f200415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fec003570 con 0x7f2004159c60 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889967558 0 0) 0x7f1fec003570 con 0x7f2004159c60 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 >> v2:172.21.15.136:3300/0 conn(0x7f2004159890 msgr2=0x7f200415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 --2- 172.21.15.92:0/1297695188 >> v2:172.21.15.136:3300/0 conn(0x7f2004159890 0x7f200415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 >> v1:172.21.15.136:6789/0 conn(0x7f2004069560 legacy=0x7f20040ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200415a3d0 con 0x7f2004159c60 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f1ff8001470 con 0x7f2004159c60 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 >> v1:172.21.15.92:6789/0 conn(0x7f2004159c60 legacy=0x7f200415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fec003a60 0x7f1fec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f1fec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fec0051c0 0x7f1fec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200415e140 con 0x7f1fec004810 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec00c0d0 con 0x7f1fec0051c0 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec00c290 con 0x7f1fec003a60 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f1fec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2003fff700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fec003a60 0x7f1fec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2008fbb700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fec0051c0 0x7f1fec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 -- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fec003a60 msgr2=0x7f1fec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fec003a60 0x7f1fec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 -- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fec0051c0 msgr2=0x7f1fec00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fec0051c0 0x7f1fec00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 -- 172.21.15.92:0/1297695188 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec003570 con 0x7f1fec004810 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2008fbb700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fec0051c0 0x7f1fec00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f2008fbb700 1 -- 172.21.15.92:0/1297695188 reap_dead start 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.450+0000 7f20037fe700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f1fec004c80 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c3c0 tx=0x7f1ff8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2411dd9d91cf9a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800cc90 con 0x7f1fec004810 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800cdf0 con 0x7f1fec004810 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f2002ffd700 1 -- 172.21.15.92:0/1297695188 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8005b70 con 0x7f1fec004810 2023-10-06T03:10:48.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 -- 172.21.15.92:0/1297695188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 msgr2=0x7f1fec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f1fec004c80 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800c3c0 tx=0x7f1ff8001e50 comp rx=0 tx=0).stop 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 -- 172.21.15.92:0/1297695188 shutdown_connections 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 --2- 172.21.15.92:0/1297695188 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f1fec004c80 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 -- 172.21.15.92:0/1297695188 >> 172.21.15.92:0/1297695188 conn(0x7f2004068e20 msgr2=0x7f20040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 -- 172.21.15.92:0/1297695188 shutdown_connections 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.451+0000 7f200aa1e700 1 -- 172.21.15.92:0/1297695188 wait complete. 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.452+0000 7f200aa1e700 1 Processor -- start 2023-10-06T03:10:48.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.452+0000 7f200aa1e700 1 -- start start 2023-10-06T03:10:48.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f200aa1e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f20041582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f200aa1e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20040b5710 0x7f20040b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f200aa1e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20040b3600 0x7f20040b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f200aa1e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20040cd8b0 con 0x7f1fec004810 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f200aa1e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20040cdb00 con 0x7f20040b3600 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f200aa1e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20040cdde0 con 0x7f20040b5710 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f2003fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f20041582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f2003fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f20041582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45078/0 (socket says 172.21.15.92:45078) 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f2003fff700 1 -- 172.21.15.92:0/3543325225 learned_addr learned my addr 172.21.15.92:0/3543325225 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f2008fbb700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20040b3600 0x7f20040b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.453+0000 7f20037fe700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20040b5710 0x7f20040b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f2003fff700 1 -- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20040b5710 msgr2=0x7f20040b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f2003fff700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20040b5710 0x7f20040b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f2003fff700 1 -- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20040b3600 msgr2=0x7f20040b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f2003fff700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20040b3600 0x7f20040b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f2003fff700 1 -- 172.21.15.92:0/3543325225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20040ce0c0 con 0x7f1fec004810 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f20037fe700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20040b5710 0x7f20040b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f2003fff700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f20041582f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4011d40 tx=0x7f1ff40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff400fac0 con 0x7f1fec004810 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff400fc20 con 0x7f1fec004810 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20040ce3a0 con 0x7f1fec004810 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4014440 con 0x7f1fec004810 2023-10-06T03:10:48.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.454+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2004158830 con 0x7f1fec004810 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.455+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20040ce710 con 0x7f1fec004810 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.455+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1ff4014a80 con 0x7f1fec004810 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.456+0000 7f1febfff700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ff0067320 0x7f1ff00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.456+0000 7f20037fe700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ff0067320 0x7f1ff00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.456+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1ff4018030 con 0x7f1fec004810 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.456+0000 7f20037fe700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ff0067320 0x7f1ff00697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000c00 tx=0x7f1ff8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.460+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ff4052680 con 0x7f1fec004810 2023-10-06T03:10:48.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.641+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f200404c3c0 con 0x7f1ff0067320 2023-10-06T03:10:48.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.649+0000 7f1febfff700 1 -- 172.21.15.92:0/3543325225 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308753 (secure 0 0 0) 0x7f200404c3c0 con 0x7f1ff0067320 2023-10-06T03:10:48.654 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ff0067320 msgr2=0x7f1ff00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ff0067320 0x7f1ff00697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000c00 tx=0x7f1ff8015000 comp rx=0 tx=0).stop 2023-10-06T03:10:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 msgr2=0x7f20041582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f20041582f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4011d40 tx=0x7f1ff40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 shutdown_connections 2023-10-06T03:10:48.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1ff0067320 0x7f1ff00697d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1fec004810 0x7f20041582f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20040b3600 0x7f20040b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 --2- 172.21.15.92:0/3543325225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20040b5710 0x7f20040b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 >> 172.21.15.92:0/3543325225 conn(0x7f2004068e20 msgr2=0x7f200415d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 shutdown_connections 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.652+0000 7f200aa1e700 1 -- 172.21.15.92:0/3543325225 wait complete. 2023-10-06T03:10:48.658 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:48.666 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2023-10-06T03:10:47.293430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430456,"kb_used_data":19384,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544264,"statfs":{"total":773094113280,"available":770605326336,"internally_reserved":0,"allocated":19849216,"data_stored":10035880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.704982"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.605719+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602195+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:10.291594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599051+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:21.853236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598023+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:19.895234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597713+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:05.486771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.600915+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:37.400072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.603877+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.605691+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599170+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:42.270836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.604237+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601336+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:56.604102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.603010+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597439+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:24.712832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599339+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:50.968184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599777+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:39.383810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597680+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:15.499378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597969+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:00.320153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.603154+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.604417+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596961+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:20.132651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599279+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:55.567941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599743+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:14.184117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.600719+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:10.535161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595639+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:07.998111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601687+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:21.853236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.602925+0000","last_change":"2023-10-06T03:10:43.602925+0000","last_active":"2023-10-06T03:10:43.602925+0000","last_peered":"2023-10-06T03:10:43.602925+0000","last_clean":"2023-10-06T03:10:43.602925+0000","last_became_active":"2023-10-06T03:10:43.602673+0000","last_became_peered":"2023-10-06T03:10:43.602673+0000","last_unstale":"2023-10-06T03:10:43.602925+0000","last_undegraded":"2023-10-06T03:10:43.602925+0000","last_fullsized":"2023-10-06T03:10:43.602925+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596304+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:19.701085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.603054+0000","last_change":"2023-10-06T03:10:43.603054+0000","last_active":"2023-10-06T03:10:43.603054+0000","last_peered":"2023-10-06T03:10:43.603054+0000","last_clean":"2023-10-06T03:10:43.603054+0000","last_became_active":"2023-10-06T03:10:43.602849+0000","last_became_peered":"2023-10-06T03:10:43.602849+0000","last_unstale":"2023-10-06T03:10:43.603054+0000","last_undegraded":"2023-10-06T03:10:43.603054+0000","last_fullsized":"2023-10-06T03:10:43.603054+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:44.214194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601734+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:42.363268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.604672+0000","last_change":"2023-10-06T03:10:43.604672+0000","last_active":"2023-10-06T03:10:43.604672+0000","last_peered":"2023-10-06T03:10:43.604672+0000","last_clean":"2023-10-06T03:10:43.604672+0000","last_became_active":"2023-10-06T03:10:43.602513+0000","last_became_peered":"2023-10-06T03:10:43.602513+0000","last_unstale":"2023-10-06T03:10:43.604672+0000","last_undegraded":"2023-10-06T03:10:43.604672+0000","last_fullsized":"2023-10-06T03:10:43.604672+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:36.206746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602384+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609088+0000","last_change":"2023-10-06T03:10:43.609088+0000","last_active":"2023-10-06T03:10:43.609088+0000","last_peered":"2023-10-06T03:10:43.609088+0000","last_clean":"2023-10-06T03:10:43.609088+0000","last_became_active":"2023-10-06T03:10:43.608821+0000","last_became_peered":"2023-10-06T03:10:43.608821+0000","last_unstale":"2023-10-06T03:10:43.609088+0000","last_undegraded":"2023-10-06T03:10:43.609088+0000","last_fullsized":"2023-10-06T03:10:43.609088+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:36.735333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599238+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:30.013331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.610186+0000","last_change":"2023-10-06T03:10:43.610186+0000","last_active":"2023-10-06T03:10:43.610186+0000","last_peered":"2023-10-06T03:10:43.610186+0000","last_clean":"2023-10-06T03:10:43.610186+0000","last_became_active":"2023-10-06T03:10:43.609855+0000","last_became_peered":"2023-10-06T03:10:43.609855+0000","last_unstale":"2023-10-06T03:10:43.610186+0000","last_undegraded":"2023-10-06T03:10:43.610186+0000","last_fullsized":"2023-10-06T03:10:43.610186+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601825+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:10.291594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.606849+0000","last_change":"2023-10-06T03:10:43.606849+0000","last_active":"2023-10-06T03:10:43.606849+0000","last_peered":"2023-10-06T03:10:43.606849+0000","last_clean":"2023-10-06T03:10:43.606849+0000","last_became_active":"2023-10-06T03:10:43.606628+0000","last_became_peered":"2023-10-06T03:10:43.606628+0000","last_unstale":"2023-10-06T03:10:43.606849+0000","last_undegraded":"2023-10-06T03:10:43.606849+0000","last_fullsized":"2023-10-06T03:10:43.606849+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:37.755076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597226+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:50.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.617619+0000","last_change":"2023-10-06T03:10:43.617619+0000","last_active":"2023-10-06T03:10:43.617619+0000","last_peered":"2023-10-06T03:10:43.617619+0000","last_clean":"2023-10-06T03:10:43.617619+0000","last_became_active":"2023-10-06T03:10:43.617376+0000","last_became_peered":"2023-10-06T03:10:43.617376+0000","last_unstale":"2023-10-06T03:10:43.617619+0000","last_undegraded":"2023-10-06T03:10:43.617619+0000","last_fullsized":"2023-10-06T03:10:43.617619+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:25.490995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.600177+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:32.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608993+0000","last_change":"2023-10-06T03:10:43.608993+0000","last_active":"2023-10-06T03:10:43.608993+0000","last_peered":"2023-10-06T03:10:43.608993+0000","last_clean":"2023-10-06T03:10:43.608993+0000","last_became_active":"2023-10-06T03:10:43.608716+0000","last_became_peered":"2023-10-06T03:10:43.608716+0000","last_unstale":"2023-10-06T03:10:43.608993+0000","last_undegraded":"2023-10-06T03:10:43.608993+0000","last_fullsized":"2023-10-06T03:10:43.608993+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:13.569451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601210+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:19.895234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608400+0000","last_change":"2023-10-06T03:10:43.608400+0000","last_active":"2023-10-06T03:10:43.608400+0000","last_peered":"2023-10-06T03:10:43.608400+0000","last_clean":"2023-10-06T03:10:43.608400+0000","last_became_active":"2023-10-06T03:10:43.607925+0000","last_became_peered":"2023-10-06T03:10:43.607925+0000","last_unstale":"2023-10-06T03:10:43.608400+0000","last_undegraded":"2023-10-06T03:10:43.608400+0000","last_fullsized":"2023-10-06T03:10:43.608400+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:36.739791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595417+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:55.567941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611484+0000","last_change":"2023-10-06T03:10:43.611484+0000","last_active":"2023-10-06T03:10:43.611484+0000","last_peered":"2023-10-06T03:10:43.611484+0000","last_clean":"2023-10-06T03:10:43.611484+0000","last_became_active":"2023-10-06T03:10:43.611227+0000","last_became_peered":"2023-10-06T03:10:43.611227+0000","last_unstale":"2023-10-06T03:10:43.611484+0000","last_undegraded":"2023-10-06T03:10:43.611484+0000","last_fullsized":"2023-10-06T03:10:43.611484+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:43.590879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601952+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:42.918967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.610252+0000","last_change":"2023-10-06T03:10:43.610252+0000","last_active":"2023-10-06T03:10:43.610252+0000","last_peered":"2023-10-06T03:10:43.610252+0000","last_clean":"2023-10-06T03:10:43.610252+0000","last_became_active":"2023-10-06T03:10:43.609979+0000","last_became_peered":"2023-10-06T03:10:43.609979+0000","last_unstale":"2023-10-06T03:10:43.610252+0000","last_undegraded":"2023-10-06T03:10:43.610252+0000","last_fullsized":"2023-10-06T03:10:43.610252+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:26.682895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595512+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:33.614656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.604507+0000","last_change":"2023-10-06T03:10:43.604507+0000","last_active":"2023-10-06T03:10:43.604507+0000","last_peered":"2023-10-06T03:10:43.604507+0000","last_clean":"2023-10-06T03:10:43.604507+0000","last_became_active":"2023-10-06T03:10:43.604261+0000","last_became_peered":"2023-10-06T03:10:43.604261+0000","last_unstale":"2023-10-06T03:10:43.604507+0000","last_undegraded":"2023-10-06T03:10:43.604507+0000","last_fullsized":"2023-10-06T03:10:43.604507+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:12.780687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595384+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:15.658864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609477+0000","last_change":"2023-10-06T03:10:43.609477+0000","last_active":"2023-10-06T03:10:43.609477+0000","last_peered":"2023-10-06T03:10:43.609477+0000","last_clean":"2023-10-06T03:10:43.609477+0000","last_became_active":"2023-10-06T03:10:43.609213+0000","last_became_peered":"2023-10-06T03:10:43.609213+0000","last_unstale":"2023-10-06T03:10:43.609477+0000","last_undegraded":"2023-10-06T03:10:43.609477+0000","last_fullsized":"2023-10-06T03:10:43.609477+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:20.842732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601118+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:47.320743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611154+0000","last_change":"2023-10-06T03:10:43.611154+0000","last_active":"2023-10-06T03:10:43.611154+0000","last_peered":"2023-10-06T03:10:43.611154+0000","last_clean":"2023-10-06T03:10:43.611154+0000","last_became_active":"2023-10-06T03:10:43.610907+0000","last_became_peered":"2023-10-06T03:10:43.610907+0000","last_unstale":"2023-10-06T03:10:43.611154+0000","last_undegraded":"2023-10-06T03:10:43.611154+0000","last_fullsized":"2023-10-06T03:10:43.611154+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595146+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:42.270836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608802+0000","last_change":"2023-10-06T03:10:43.608802+0000","last_active":"2023-10-06T03:10:43.608802+0000","last_peered":"2023-10-06T03:10:43.608802+0000","last_clean":"2023-10-06T03:10:43.608802+0000","last_became_active":"2023-10-06T03:10:43.608524+0000","last_became_peered":"2023-10-06T03:10:43.608524+0000","last_unstale":"2023-10-06T03:10:43.608802+0000","last_undegraded":"2023-10-06T03:10:43.608802+0000","last_fullsized":"2023-10-06T03:10:43.608802+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:28.221106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595178+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:30.013331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.617468+0000","last_change":"2023-10-06T03:10:43.617468+0000","last_active":"2023-10-06T03:10:43.617468+0000","last_peered":"2023-10-06T03:10:43.617468+0000","last_clean":"2023-10-06T03:10:43.617468+0000","last_became_active":"2023-10-06T03:10:43.617179+0000","last_became_peered":"2023-10-06T03:10:43.617179+0000","last_unstale":"2023-10-06T03:10:43.617468+0000","last_undegraded":"2023-10-06T03:10:43.617468+0000","last_fullsized":"2023-10-06T03:10:43.617468+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:31.017050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599372+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:09.769655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.607558+0000","last_change":"2023-10-06T03:10:43.607558+0000","last_active":"2023-10-06T03:10:43.607558+0000","last_peered":"2023-10-06T03:10:43.607558+0000","last_clean":"2023-10-06T03:10:43.607558+0000","last_became_active":"2023-10-06T03:10:43.607274+0000","last_became_peered":"2023-10-06T03:10:43.607274+0000","last_unstale":"2023-10-06T03:10:43.607558+0000","last_undegraded":"2023-10-06T03:10:43.607558+0000","last_fullsized":"2023-10-06T03:10:43.607558+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:55.593599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602458+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:04.349135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608774+0000","last_change":"2023-10-06T03:10:43.608774+0000","last_active":"2023-10-06T03:10:43.608774+0000","last_peered":"2023-10-06T03:10:43.608774+0000","last_clean":"2023-10-06T03:10:43.608774+0000","last_became_active":"2023-10-06T03:10:43.608468+0000","last_became_peered":"2023-10-06T03:10:43.608468+0000","last_unstale":"2023-10-06T03:10:43.608774+0000","last_undegraded":"2023-10-06T03:10:43.608774+0000","last_fullsized":"2023-10-06T03:10:43.608774+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:49.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601522+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:13.942196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609053+0000","last_change":"2023-10-06T03:10:43.609053+0000","last_active":"2023-10-06T03:10:43.609053+0000","last_peered":"2023-10-06T03:10:43.609053+0000","last_clean":"2023-10-06T03:10:43.609053+0000","last_became_active":"2023-10-06T03:10:43.608789+0000","last_became_peered":"2023-10-06T03:10:43.608789+0000","last_unstale":"2023-10-06T03:10:43.609053+0000","last_undegraded":"2023-10-06T03:10:43.609053+0000","last_fullsized":"2023-10-06T03:10:43.609053+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:36.389568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597064+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:14.579954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611124+0000","last_change":"2023-10-06T03:10:43.611124+0000","last_active":"2023-10-06T03:10:43.611124+0000","last_peered":"2023-10-06T03:10:43.611124+0000","last_clean":"2023-10-06T03:10:43.611124+0000","last_became_active":"2023-10-06T03:10:43.610877+0000","last_became_peered":"2023-10-06T03:10:43.610877+0000","last_unstale":"2023-10-06T03:10:43.611124+0000","last_undegraded":"2023-10-06T03:10:43.611124+0000","last_fullsized":"2023-10-06T03:10:43.611124+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:20:00.923932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597687+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:29.231610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.607875+0000","last_change":"2023-10-06T03:10:43.607875+0000","last_active":"2023-10-06T03:10:43.607875+0000","last_peered":"2023-10-06T03:10:43.607875+0000","last_clean":"2023-10-06T03:10:43.607875+0000","last_became_active":"2023-10-06T03:10:43.607570+0000","last_became_peered":"2023-10-06T03:10:43.607570+0000","last_unstale":"2023-10-06T03:10:43.607875+0000","last_undegraded":"2023-10-06T03:10:43.607875+0000","last_fullsized":"2023-10-06T03:10:43.607875+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598001+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:21.632793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611649+0000","last_change":"2023-10-06T03:10:43.611649+0000","last_active":"2023-10-06T03:10:43.611649+0000","last_peered":"2023-10-06T03:10:43.611649+0000","last_clean":"2023-10-06T03:10:43.611649+0000","last_became_active":"2023-10-06T03:10:43.611297+0000","last_became_peered":"2023-10-06T03:10:43.611297+0000","last_unstale":"2023-10-06T03:10:43.611649+0000","last_undegraded":"2023-10-06T03:10:43.611649+0000","last_fullsized":"2023-10-06T03:10:43.611649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.884730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.594586+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:53.466758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.610052+0000","last_change":"2023-10-06T03:10:43.610052+0000","last_active":"2023-10-06T03:10:43.610052+0000","last_peered":"2023-10-06T03:10:43.610052+0000","last_clean":"2023-10-06T03:10:43.610052+0000","last_became_active":"2023-10-06T03:10:43.609757+0000","last_became_peered":"2023-10-06T03:10:43.609757+0000","last_unstale":"2023-10-06T03:10:43.610052+0000","last_undegraded":"2023-10-06T03:10:43.610052+0000","last_fullsized":"2023-10-06T03:10:43.610052+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:20.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596804+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:48.515082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609073+0000","last_change":"2023-10-06T03:10:43.609073+0000","last_active":"2023-10-06T03:10:43.609073+0000","last_peered":"2023-10-06T03:10:43.609073+0000","last_clean":"2023-10-06T03:10:43.609073+0000","last_became_active":"2023-10-06T03:10:43.608677+0000","last_became_peered":"2023-10-06T03:10:43.608677+0000","last_unstale":"2023-10-06T03:10:43.609073+0000","last_undegraded":"2023-10-06T03:10:43.609073+0000","last_fullsized":"2023-10-06T03:10:43.609073+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:09.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598980+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:18.914721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.603495+0000","last_change":"2023-10-06T03:10:43.603495+0000","last_active":"2023-10-06T03:10:43.603495+0000","last_peered":"2023-10-06T03:10:43.603495+0000","last_clean":"2023-10-06T03:10:43.603495+0000","last_became_active":"2023-10-06T03:10:43.603249+0000","last_became_peered":"2023-10-06T03:10:43.603249+0000","last_unstale":"2023-10-06T03:10:43.603495+0000","last_undegraded":"2023-10-06T03:10:43.603495+0000","last_fullsized":"2023-10-06T03:10:43.603495+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:49.332478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.594592+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:48.515082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.617649+0000","last_change":"2023-10-06T03:10:43.617649+0000","last_active":"2023-10-06T03:10:43.617649+0000","last_peered":"2023-10-06T03:10:43.617649+0000","last_clean":"2023-10-06T03:10:43.617649+0000","last_became_active":"2023-10-06T03:10:43.617403+0000","last_became_peered":"2023-10-06T03:10:43.617403+0000","last_unstale":"2023-10-06T03:10:43.617649+0000","last_undegraded":"2023-10-06T03:10:43.617649+0000","last_fullsized":"2023-10-06T03:10:43.617649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:17.771738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597534+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:01.675210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609223+0000","last_change":"2023-10-06T03:10:43.609223+0000","last_active":"2023-10-06T03:10:43.609223+0000","last_peered":"2023-10-06T03:10:43.609223+0000","last_clean":"2023-10-06T03:10:43.609223+0000","last_became_active":"2023-10-06T03:10:43.606876+0000","last_became_peered":"2023-10-06T03:10:43.606876+0000","last_unstale":"2023-10-06T03:10:43.609223+0000","last_undegraded":"2023-10-06T03:10:43.609223+0000","last_fullsized":"2023-10-06T03:10:43.609223+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:04.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598597+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:56.604102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611147+0000","last_change":"2023-10-06T03:10:43.611147+0000","last_active":"2023-10-06T03:10:43.611147+0000","last_peered":"2023-10-06T03:10:43.611147+0000","last_clean":"2023-10-06T03:10:43.611147+0000","last_became_active":"2023-10-06T03:10:43.610865+0000","last_became_peered":"2023-10-06T03:10:43.610865+0000","last_unstale":"2023-10-06T03:10:43.611147+0000","last_undegraded":"2023-10-06T03:10:43.611147+0000","last_fullsized":"2023-10-06T03:10:43.611147+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598337+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:04.736772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611002+0000","last_change":"2023-10-06T03:10:43.611002+0000","last_active":"2023-10-06T03:10:43.611002+0000","last_peered":"2023-10-06T03:10:43.611002+0000","last_clean":"2023-10-06T03:10:43.611002+0000","last_became_active":"2023-10-06T03:10:43.610692+0000","last_became_peered":"2023-10-06T03:10:43.610692+0000","last_unstale":"2023-10-06T03:10:43.611002+0000","last_undegraded":"2023-10-06T03:10:43.611002+0000","last_fullsized":"2023-10-06T03:10:43.611002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:15.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598294+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:53.654291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.607559+0000","last_change":"2023-10-06T03:10:43.607559+0000","last_active":"2023-10-06T03:10:43.607559+0000","last_peered":"2023-10-06T03:10:43.607559+0000","last_clean":"2023-10-06T03:10:43.607559+0000","last_became_active":"2023-10-06T03:10:43.607302+0000","last_became_peered":"2023-10-06T03:10:43.607302+0000","last_unstale":"2023-10-06T03:10:43.607559+0000","last_undegraded":"2023-10-06T03:10:43.607559+0000","last_fullsized":"2023-10-06T03:10:43.607559+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596291+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:41.713908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.605635+0000","last_change":"2023-10-06T03:10:43.605635+0000","last_active":"2023-10-06T03:10:43.605635+0000","last_peered":"2023-10-06T03:10:43.605635+0000","last_clean":"2023-10-06T03:10:43.605635+0000","last_became_active":"2023-10-06T03:10:43.605202+0000","last_became_peered":"2023-10-06T03:10:43.605202+0000","last_unstale":"2023-10-06T03:10:43.605635+0000","last_undegraded":"2023-10-06T03:10:43.605635+0000","last_fullsized":"2023-10-06T03:10:43.605635+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:32.594410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597608+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:32.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608350+0000","last_change":"2023-10-06T03:10:43.608350+0000","last_active":"2023-10-06T03:10:43.608350+0000","last_peered":"2023-10-06T03:10:43.608350+0000","last_clean":"2023-10-06T03:10:43.608350+0000","last_became_active":"2023-10-06T03:10:43.608121+0000","last_became_peered":"2023-10-06T03:10:43.608121+0000","last_unstale":"2023-10-06T03:10:43.608350+0000","last_undegraded":"2023-10-06T03:10:43.608350+0000","last_fullsized":"2023-10-06T03:10:43.608350+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:14.031939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602346+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:01.251002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594241+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:42.594241+0000","last_peered":"2023-10-06T03:10:42.594241+0000","last_clean":"2023-10-06T03:10:42.594241+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:42.594241+0000","last_undegraded":"2023-10-06T03:10:42.594241+0000","last_fullsized":"2023-10-06T03:10:42.594241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602161+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.596084+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:42.596084+0000","last_peered":"2023-10-06T03:10:42.596084+0000","last_clean":"2023-10-06T03:10:42.596084+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:42.596084+0000","last_undegraded":"2023-10-06T03:10:42.596084+0000","last_fullsized":"2023-10-06T03:10:42.596084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597657+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:10.535161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594454+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:42.594454+0000","last_peered":"2023-10-06T03:10:42.594454+0000","last_clean":"2023-10-06T03:10:42.594454+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:42.594454+0000","last_undegraded":"2023-10-06T03:10:42.594454+0000","last_fullsized":"2023-10-06T03:10:42.594454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595405+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:14.184117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594260+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:42.594260+0000","last_peered":"2023-10-06T03:10:42.594260+0000","last_clean":"2023-10-06T03:10:42.594260+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:42.594260+0000","last_undegraded":"2023-10-06T03:10:42.594260+0000","last_fullsized":"2023-10-06T03:10:42.594260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602492+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":301,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594282+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:42.594282+0000","last_peered":"2023-10-06T03:10:42.594282+0000","last_clean":"2023-10-06T03:10:42.594282+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:42.594282+0000","last_undegraded":"2023-10-06T03:10:42.594282+0000","last_fullsized":"2023-10-06T03:10:42.594282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.592696+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:42.592696+0000","last_peered":"2023-10-06T03:10:42.592696+0000","last_clean":"2023-10-06T03:10:42.592696+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:42.592696+0000","last_undegraded":"2023-10-06T03:10:42.592696+0000","last_fullsized":"2023-10-06T03:10:42.592696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598892+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:05.627341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":149,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.589735+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:42.589735+0000","last_peered":"2023-10-06T03:10:42.589735+0000","last_clean":"2023-10-06T03:10:42.589735+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:42.589735+0000","last_undegraded":"2023-10-06T03:10:42.589735+0000","last_fullsized":"2023-10-06T03:10:42.589735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602492+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.592681+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:42.592681+0000","last_peered":"2023-10-06T03:10:42.592681+0000","last_clean":"2023-10-06T03:10:42.592681+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:42.592681+0000","last_undegraded":"2023-10-06T03:10:42.592681+0000","last_fullsized":"2023-10-06T03:10:42.592681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597856+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:42.386144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.596943+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:42.596943+0000","last_peered":"2023-10-06T03:10:42.596943+0000","last_clean":"2023-10-06T03:10:42.596943+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:42.596943+0000","last_undegraded":"2023-10-06T03:10:42.596943+0000","last_fullsized":"2023-10-06T03:10:42.596943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304248,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067592,"statfs":{"total":96636764160,"available":96325214208,"internally_reserved":0,"allocated":2932736,"data_stored":1700569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.623},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.498},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.70299999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.371},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.40999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.439}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1110201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.80800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304248,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067592,"statfs":{"total":96636764160,"available":96325214208,"internally_reserved":0,"allocated":2932736,"data_stored":1700569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.248},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.30299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.26200000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.376},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1110201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.627},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.38600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.316},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.374},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.747}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:48.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:48.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.820+0000 7fbe5e68b700 1 Processor -- start 2023-10-06T03:10:48.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.820+0000 7fbe5e68b700 1 -- start start 2023-10-06T03:10:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5e68b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe58159890 0x7fbe5815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5e68b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbe58159c60 0x7fbe5815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5e68b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbe5815db10 0x7fbe58069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5e68b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5815a160 con 0x7fbe5815db10 2023-10-06T03:10:48.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5e68b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe580d13a0 con 0x7fbe58159c60 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5e68b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5815e140 con 0x7fbe58159890 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbe5815db10 0x7fbe58069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbe5815db10 0x7fbe58069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45094/0 (socket says 172.21.15.92:45094) 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 -- 172.21.15.92:0/2440041736 learned_addr learned my addr 172.21.15.92:0/2440041736 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe57fff700 1 --2- 172.21.15.92:0/2440041736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe58159890 0x7fbe5815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe577fe700 1 --2- 172.21.15.92:0/2440041736 >> v2:172.21.15.136:3300/0 conn(0x7fbe58159c60 0x7fbe5815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 -- 172.21.15.92:0/2440041736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe58159890 msgr2=0x7fbe5815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 --2- 172.21.15.92:0/2440041736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe58159890 0x7fbe5815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 -- 172.21.15.92:0/2440041736 >> v2:172.21.15.136:3300/0 conn(0x7fbe58159c60 msgr2=0x7fbe5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 --2- 172.21.15.92:0/2440041736 >> v2:172.21.15.136:3300/0 conn(0x7fbe58159c60 0x7fbe5815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe5cc28700 1 -- 172.21.15.92:0/2440041736 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5815a3d0 con 0x7fbe5815db10 2023-10-06T03:10:48.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.821+0000 7fbe577fe700 1 --2- 172.21.15.92:0/2440041736 >> v2:172.21.15.136:3300/0 conn(0x7fbe58159c60 0x7fbe5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5cc28700 1 --2- 172.21.15.92:0/2440041736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5815db10 0x7fbe58069560 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fbe580650f0 tx=0x7fbe50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b528b025afe92400 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe56ffd700 1 -- 172.21.15.92:0/2440041736 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000e070 con 0x7fbe5815db10 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe56ffd700 1 -- 172.21.15.92:0/2440041736 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000ba20 con 0x7fbe5815db10 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe56ffd700 1 -- 172.21.15.92:0/2440041736 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500092e0 con 0x7fbe5815db10 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 -- 172.21.15.92:0/2440041736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5815db10 msgr2=0x7fbe58069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/2440041736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5815db10 0x7fbe58069560 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fbe580650f0 tx=0x7fbe50009840 comp rx=0 tx=0).stop 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 -- 172.21.15.92:0/2440041736 shutdown_connections 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/2440041736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe58159890 0x7fbe5815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/2440041736 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe5815db10 0x7fbe58069560 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/2440041736 >> v2:172.21.15.136:3300/0 conn(0x7fbe58159c60 0x7fbe5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 -- 172.21.15.92:0/2440041736 >> 172.21.15.92:0/2440041736 conn(0x7fbe58068e20 msgr2=0x7fbe580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:48.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 -- 172.21.15.92:0/2440041736 shutdown_connections 2023-10-06T03:10:48.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.822+0000 7fbe5e68b700 1 -- 172.21.15.92:0/2440041736 wait complete. 2023-10-06T03:10:48.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.823+0000 7fbe5e68b700 1 Processor -- start 2023-10-06T03:10:48.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 -- start start 2023-10-06T03:10:48.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe58159890 0x7fbe58152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 0x7fbe5814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe5815db10 0x7fbe5815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58152870 con 0x7fbe58159c60 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58152b50 con 0x7fbe58159890 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe5e68b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58152e30 con 0x7fbe5815db10 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 0x7fbe5814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe577fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 0x7fbe5814c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45106/0 (socket says 172.21.15.92:45106) 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.824+0000 7fbe577fe700 1 -- 172.21.15.92:0/1215424800 learned_addr learned my addr 172.21.15.92:0/1215424800 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe57fff700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe58159890 0x7fbe58152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe5cc28700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe5815db10 0x7fbe5815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe577fe700 1 -- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe5815db10 msgr2=0x7fbe5815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe577fe700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe5815db10 0x7fbe5815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe577fe700 1 -- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe58159890 msgr2=0x7fbe58152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe577fe700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe58159890 0x7fbe58152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe577fe700 1 -- 172.21.15.92:0/1215424800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe50009ae0 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe577fe700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 0x7fbe5814c3f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e8a0 tx=0x7fbe4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00c430 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00c590 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe58153110 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00c950 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.825+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5815d8c0 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.826+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5804c3c0 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.826+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbe4c00cd00 con 0x7fbe58159c60 2023-10-06T03:10:48.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.827+0000 7fbe3ffff700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe44067320 0x7fbe440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:48.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.827+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbe4c084c50 con 0x7fbe58159c60 2023-10-06T03:10:48.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.827+0000 7fbe57fff700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe44067320 0x7fbe440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:48.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.827+0000 7fbe57fff700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe44067320 0x7fbe440697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbe480093f0 tx=0x7fbe48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:48.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:48.831+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe4c051670 con 0x7fbe58159c60 2023-10-06T03:10:49.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.013+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe5806a5b0 con 0x7fbe44067320 2023-10-06T03:10:49.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.022+0000 7fbe3ffff700 1 -- 172.21.15.92:0/1215424800 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308753 (secure 0 0 0) 0x7fbe5806a5b0 con 0x7fbe44067320 2023-10-06T03:10:49.027 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:49.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe44067320 msgr2=0x7fbe440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe44067320 0x7fbe440697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbe480093f0 tx=0x7fbe48008040 comp rx=0 tx=0).stop 2023-10-06T03:10:49.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 msgr2=0x7fbe5814c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 0x7fbe5814c3f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e8a0 tx=0x7fbe4c008040 comp rx=0 tx=0).stop 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 shutdown_connections 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbe44067320 0x7fbe440697d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe5815db10 0x7fbe5815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe58159890 0x7fbe58152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 --2- 172.21.15.92:0/1215424800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbe58159c60 0x7fbe5814c3f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 >> 172.21.15.92:0/1215424800 conn(0x7fbe58068e20 msgr2=0x7fbe580b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 shutdown_connections 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.025+0000 7fbe5e68b700 1 -- 172.21.15.92:0/1215424800 wait complete. 2023-10-06T03:10:49.030 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:49.038 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2023-10-06T03:10:47.293430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430456,"kb_used_data":19384,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752544264,"statfs":{"total":773094113280,"available":770605326336,"internally_reserved":0,"allocated":19849216,"data_stored":10035880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.704982"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.605719+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602195+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:10.291594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599051+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:21.853236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598023+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:19.895234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597713+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:05.486771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.600915+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:37.400072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.603877+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.605691+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599170+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:42.270836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.604237+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601336+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:56.604102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.603010+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597439+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:24.712832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599339+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:50.968184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599777+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:39.383810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597680+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:15.499378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597969+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:00.320153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.603154+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.604417+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596961+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:20.132651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599279+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:55.567941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599743+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:14.184117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.600719+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:10.535161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595639+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:07.998111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601687+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:21.853236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.602925+0000","last_change":"2023-10-06T03:10:43.602925+0000","last_active":"2023-10-06T03:10:43.602925+0000","last_peered":"2023-10-06T03:10:43.602925+0000","last_clean":"2023-10-06T03:10:43.602925+0000","last_became_active":"2023-10-06T03:10:43.602673+0000","last_became_peered":"2023-10-06T03:10:43.602673+0000","last_unstale":"2023-10-06T03:10:43.602925+0000","last_undegraded":"2023-10-06T03:10:43.602925+0000","last_fullsized":"2023-10-06T03:10:43.602925+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596304+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:19.701085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.603054+0000","last_change":"2023-10-06T03:10:43.603054+0000","last_active":"2023-10-06T03:10:43.603054+0000","last_peered":"2023-10-06T03:10:43.603054+0000","last_clean":"2023-10-06T03:10:43.603054+0000","last_became_active":"2023-10-06T03:10:43.602849+0000","last_became_peered":"2023-10-06T03:10:43.602849+0000","last_unstale":"2023-10-06T03:10:43.603054+0000","last_undegraded":"2023-10-06T03:10:43.603054+0000","last_fullsized":"2023-10-06T03:10:43.603054+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:44.214194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601734+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:42.363268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.604672+0000","last_change":"2023-10-06T03:10:43.604672+0000","last_active":"2023-10-06T03:10:43.604672+0000","last_peered":"2023-10-06T03:10:43.604672+0000","last_clean":"2023-10-06T03:10:43.604672+0000","last_became_active":"2023-10-06T03:10:43.602513+0000","last_became_peered":"2023-10-06T03:10:43.602513+0000","last_unstale":"2023-10-06T03:10:43.604672+0000","last_undegraded":"2023-10-06T03:10:43.604672+0000","last_fullsized":"2023-10-06T03:10:43.604672+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:36.206746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602384+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609088+0000","last_change":"2023-10-06T03:10:43.609088+0000","last_active":"2023-10-06T03:10:43.609088+0000","last_peered":"2023-10-06T03:10:43.609088+0000","last_clean":"2023-10-06T03:10:43.609088+0000","last_became_active":"2023-10-06T03:10:43.608821+0000","last_became_peered":"2023-10-06T03:10:43.608821+0000","last_unstale":"2023-10-06T03:10:43.609088+0000","last_undegraded":"2023-10-06T03:10:43.609088+0000","last_fullsized":"2023-10-06T03:10:43.609088+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:36.735333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599238+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:30.013331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.610186+0000","last_change":"2023-10-06T03:10:43.610186+0000","last_active":"2023-10-06T03:10:43.610186+0000","last_peered":"2023-10-06T03:10:43.610186+0000","last_clean":"2023-10-06T03:10:43.610186+0000","last_became_active":"2023-10-06T03:10:43.609855+0000","last_became_peered":"2023-10-06T03:10:43.609855+0000","last_unstale":"2023-10-06T03:10:43.610186+0000","last_undegraded":"2023-10-06T03:10:43.610186+0000","last_fullsized":"2023-10-06T03:10:43.610186+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601825+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:10.291594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.606849+0000","last_change":"2023-10-06T03:10:43.606849+0000","last_active":"2023-10-06T03:10:43.606849+0000","last_peered":"2023-10-06T03:10:43.606849+0000","last_clean":"2023-10-06T03:10:43.606849+0000","last_became_active":"2023-10-06T03:10:43.606628+0000","last_became_peered":"2023-10-06T03:10:43.606628+0000","last_unstale":"2023-10-06T03:10:43.606849+0000","last_undegraded":"2023-10-06T03:10:43.606849+0000","last_fullsized":"2023-10-06T03:10:43.606849+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:37.755076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597226+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:24:50.638916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.617619+0000","last_change":"2023-10-06T03:10:43.617619+0000","last_active":"2023-10-06T03:10:43.617619+0000","last_peered":"2023-10-06T03:10:43.617619+0000","last_clean":"2023-10-06T03:10:43.617619+0000","last_became_active":"2023-10-06T03:10:43.617376+0000","last_became_peered":"2023-10-06T03:10:43.617376+0000","last_unstale":"2023-10-06T03:10:43.617619+0000","last_undegraded":"2023-10-06T03:10:43.617619+0000","last_fullsized":"2023-10-06T03:10:43.617619+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:25.490995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.600177+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:32.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608993+0000","last_change":"2023-10-06T03:10:43.608993+0000","last_active":"2023-10-06T03:10:43.608993+0000","last_peered":"2023-10-06T03:10:43.608993+0000","last_clean":"2023-10-06T03:10:43.608993+0000","last_became_active":"2023-10-06T03:10:43.608716+0000","last_became_peered":"2023-10-06T03:10:43.608716+0000","last_unstale":"2023-10-06T03:10:43.608993+0000","last_undegraded":"2023-10-06T03:10:43.608993+0000","last_fullsized":"2023-10-06T03:10:43.608993+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:13.569451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601210+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:19.895234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608400+0000","last_change":"2023-10-06T03:10:43.608400+0000","last_active":"2023-10-06T03:10:43.608400+0000","last_peered":"2023-10-06T03:10:43.608400+0000","last_clean":"2023-10-06T03:10:43.608400+0000","last_became_active":"2023-10-06T03:10:43.607925+0000","last_became_peered":"2023-10-06T03:10:43.607925+0000","last_unstale":"2023-10-06T03:10:43.608400+0000","last_undegraded":"2023-10-06T03:10:43.608400+0000","last_fullsized":"2023-10-06T03:10:43.608400+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:36.739791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595417+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:55.567941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611484+0000","last_change":"2023-10-06T03:10:43.611484+0000","last_active":"2023-10-06T03:10:43.611484+0000","last_peered":"2023-10-06T03:10:43.611484+0000","last_clean":"2023-10-06T03:10:43.611484+0000","last_became_active":"2023-10-06T03:10:43.611227+0000","last_became_peered":"2023-10-06T03:10:43.611227+0000","last_unstale":"2023-10-06T03:10:43.611484+0000","last_undegraded":"2023-10-06T03:10:43.611484+0000","last_fullsized":"2023-10-06T03:10:43.611484+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:43.590879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601952+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:42.918967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.610252+0000","last_change":"2023-10-06T03:10:43.610252+0000","last_active":"2023-10-06T03:10:43.610252+0000","last_peered":"2023-10-06T03:10:43.610252+0000","last_clean":"2023-10-06T03:10:43.610252+0000","last_became_active":"2023-10-06T03:10:43.609979+0000","last_became_peered":"2023-10-06T03:10:43.609979+0000","last_unstale":"2023-10-06T03:10:43.610252+0000","last_undegraded":"2023-10-06T03:10:43.610252+0000","last_fullsized":"2023-10-06T03:10:43.610252+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:26.682895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595512+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:33.614656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.604507+0000","last_change":"2023-10-06T03:10:43.604507+0000","last_active":"2023-10-06T03:10:43.604507+0000","last_peered":"2023-10-06T03:10:43.604507+0000","last_clean":"2023-10-06T03:10:43.604507+0000","last_became_active":"2023-10-06T03:10:43.604261+0000","last_became_peered":"2023-10-06T03:10:43.604261+0000","last_unstale":"2023-10-06T03:10:43.604507+0000","last_undegraded":"2023-10-06T03:10:43.604507+0000","last_fullsized":"2023-10-06T03:10:43.604507+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:12.780687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595384+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:15.658864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609477+0000","last_change":"2023-10-06T03:10:43.609477+0000","last_active":"2023-10-06T03:10:43.609477+0000","last_peered":"2023-10-06T03:10:43.609477+0000","last_clean":"2023-10-06T03:10:43.609477+0000","last_became_active":"2023-10-06T03:10:43.609213+0000","last_became_peered":"2023-10-06T03:10:43.609213+0000","last_unstale":"2023-10-06T03:10:43.609477+0000","last_undegraded":"2023-10-06T03:10:43.609477+0000","last_fullsized":"2023-10-06T03:10:43.609477+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:20.842732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601118+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:47.320743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611154+0000","last_change":"2023-10-06T03:10:43.611154+0000","last_active":"2023-10-06T03:10:43.611154+0000","last_peered":"2023-10-06T03:10:43.611154+0000","last_clean":"2023-10-06T03:10:43.611154+0000","last_became_active":"2023-10-06T03:10:43.610907+0000","last_became_peered":"2023-10-06T03:10:43.610907+0000","last_unstale":"2023-10-06T03:10:43.611154+0000","last_undegraded":"2023-10-06T03:10:43.611154+0000","last_fullsized":"2023-10-06T03:10:43.611154+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595146+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:42.270836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608802+0000","last_change":"2023-10-06T03:10:43.608802+0000","last_active":"2023-10-06T03:10:43.608802+0000","last_peered":"2023-10-06T03:10:43.608802+0000","last_clean":"2023-10-06T03:10:43.608802+0000","last_became_active":"2023-10-06T03:10:43.608524+0000","last_became_peered":"2023-10-06T03:10:43.608524+0000","last_unstale":"2023-10-06T03:10:43.608802+0000","last_undegraded":"2023-10-06T03:10:43.608802+0000","last_fullsized":"2023-10-06T03:10:43.608802+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:28.221106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595178+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:30.013331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.617468+0000","last_change":"2023-10-06T03:10:43.617468+0000","last_active":"2023-10-06T03:10:43.617468+0000","last_peered":"2023-10-06T03:10:43.617468+0000","last_clean":"2023-10-06T03:10:43.617468+0000","last_became_active":"2023-10-06T03:10:43.617179+0000","last_became_peered":"2023-10-06T03:10:43.617179+0000","last_unstale":"2023-10-06T03:10:43.617468+0000","last_undegraded":"2023-10-06T03:10:43.617468+0000","last_fullsized":"2023-10-06T03:10:43.617468+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:31.017050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.599372+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:00:09.769655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.607558+0000","last_change":"2023-10-06T03:10:43.607558+0000","last_active":"2023-10-06T03:10:43.607558+0000","last_peered":"2023-10-06T03:10:43.607558+0000","last_clean":"2023-10-06T03:10:43.607558+0000","last_became_active":"2023-10-06T03:10:43.607274+0000","last_became_peered":"2023-10-06T03:10:43.607274+0000","last_unstale":"2023-10-06T03:10:43.607558+0000","last_undegraded":"2023-10-06T03:10:43.607558+0000","last_fullsized":"2023-10-06T03:10:43.607558+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:55.593599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602458+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:04.349135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608774+0000","last_change":"2023-10-06T03:10:43.608774+0000","last_active":"2023-10-06T03:10:43.608774+0000","last_peered":"2023-10-06T03:10:43.608774+0000","last_clean":"2023-10-06T03:10:43.608774+0000","last_became_active":"2023-10-06T03:10:43.608468+0000","last_became_peered":"2023-10-06T03:10:43.608468+0000","last_unstale":"2023-10-06T03:10:43.608774+0000","last_undegraded":"2023-10-06T03:10:43.608774+0000","last_fullsized":"2023-10-06T03:10:43.608774+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:49.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.601522+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:28:13.942196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609053+0000","last_change":"2023-10-06T03:10:43.609053+0000","last_active":"2023-10-06T03:10:43.609053+0000","last_peered":"2023-10-06T03:10:43.609053+0000","last_clean":"2023-10-06T03:10:43.609053+0000","last_became_active":"2023-10-06T03:10:43.608789+0000","last_became_peered":"2023-10-06T03:10:43.608789+0000","last_unstale":"2023-10-06T03:10:43.609053+0000","last_undegraded":"2023-10-06T03:10:43.609053+0000","last_fullsized":"2023-10-06T03:10:43.609053+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:36.389568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597064+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:14.579954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611124+0000","last_change":"2023-10-06T03:10:43.611124+0000","last_active":"2023-10-06T03:10:43.611124+0000","last_peered":"2023-10-06T03:10:43.611124+0000","last_clean":"2023-10-06T03:10:43.611124+0000","last_became_active":"2023-10-06T03:10:43.610877+0000","last_became_peered":"2023-10-06T03:10:43.610877+0000","last_unstale":"2023-10-06T03:10:43.611124+0000","last_undegraded":"2023-10-06T03:10:43.611124+0000","last_fullsized":"2023-10-06T03:10:43.611124+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:20:00.923932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597687+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:29.231610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.607875+0000","last_change":"2023-10-06T03:10:43.607875+0000","last_active":"2023-10-06T03:10:43.607875+0000","last_peered":"2023-10-06T03:10:43.607875+0000","last_clean":"2023-10-06T03:10:43.607875+0000","last_became_active":"2023-10-06T03:10:43.607570+0000","last_became_peered":"2023-10-06T03:10:43.607570+0000","last_unstale":"2023-10-06T03:10:43.607875+0000","last_undegraded":"2023-10-06T03:10:43.607875+0000","last_fullsized":"2023-10-06T03:10:43.607875+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598001+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:21.632793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611649+0000","last_change":"2023-10-06T03:10:43.611649+0000","last_active":"2023-10-06T03:10:43.611649+0000","last_peered":"2023-10-06T03:10:43.611649+0000","last_clean":"2023-10-06T03:10:43.611649+0000","last_became_active":"2023-10-06T03:10:43.611297+0000","last_became_peered":"2023-10-06T03:10:43.611297+0000","last_unstale":"2023-10-06T03:10:43.611649+0000","last_undegraded":"2023-10-06T03:10:43.611649+0000","last_fullsized":"2023-10-06T03:10:43.611649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.884730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.594586+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:53.466758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.610052+0000","last_change":"2023-10-06T03:10:43.610052+0000","last_active":"2023-10-06T03:10:43.610052+0000","last_peered":"2023-10-06T03:10:43.610052+0000","last_clean":"2023-10-06T03:10:43.610052+0000","last_became_active":"2023-10-06T03:10:43.609757+0000","last_became_peered":"2023-10-06T03:10:43.609757+0000","last_unstale":"2023-10-06T03:10:43.610052+0000","last_undegraded":"2023-10-06T03:10:43.610052+0000","last_fullsized":"2023-10-06T03:10:43.610052+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:20.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596804+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:48.515082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609073+0000","last_change":"2023-10-06T03:10:43.609073+0000","last_active":"2023-10-06T03:10:43.609073+0000","last_peered":"2023-10-06T03:10:43.609073+0000","last_clean":"2023-10-06T03:10:43.609073+0000","last_became_active":"2023-10-06T03:10:43.608677+0000","last_became_peered":"2023-10-06T03:10:43.608677+0000","last_unstale":"2023-10-06T03:10:43.609073+0000","last_undegraded":"2023-10-06T03:10:43.609073+0000","last_fullsized":"2023-10-06T03:10:43.609073+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:09.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598980+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:18.914721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.603495+0000","last_change":"2023-10-06T03:10:43.603495+0000","last_active":"2023-10-06T03:10:43.603495+0000","last_peered":"2023-10-06T03:10:43.603495+0000","last_clean":"2023-10-06T03:10:43.603495+0000","last_became_active":"2023-10-06T03:10:43.603249+0000","last_became_peered":"2023-10-06T03:10:43.603249+0000","last_unstale":"2023-10-06T03:10:43.603495+0000","last_undegraded":"2023-10-06T03:10:43.603495+0000","last_fullsized":"2023-10-06T03:10:43.603495+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:49.332478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.594592+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:48.515082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.617649+0000","last_change":"2023-10-06T03:10:43.617649+0000","last_active":"2023-10-06T03:10:43.617649+0000","last_peered":"2023-10-06T03:10:43.617649+0000","last_clean":"2023-10-06T03:10:43.617649+0000","last_became_active":"2023-10-06T03:10:43.617403+0000","last_became_peered":"2023-10-06T03:10:43.617403+0000","last_unstale":"2023-10-06T03:10:43.617649+0000","last_undegraded":"2023-10-06T03:10:43.617649+0000","last_fullsized":"2023-10-06T03:10:43.617649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:17.771738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597534+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:01.675210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.609223+0000","last_change":"2023-10-06T03:10:43.609223+0000","last_active":"2023-10-06T03:10:43.609223+0000","last_peered":"2023-10-06T03:10:43.609223+0000","last_clean":"2023-10-06T03:10:43.609223+0000","last_became_active":"2023-10-06T03:10:43.606876+0000","last_became_peered":"2023-10-06T03:10:43.606876+0000","last_unstale":"2023-10-06T03:10:43.609223+0000","last_undegraded":"2023-10-06T03:10:43.609223+0000","last_fullsized":"2023-10-06T03:10:43.609223+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:04.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598597+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:56.604102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611147+0000","last_change":"2023-10-06T03:10:43.611147+0000","last_active":"2023-10-06T03:10:43.611147+0000","last_peered":"2023-10-06T03:10:43.611147+0000","last_clean":"2023-10-06T03:10:43.611147+0000","last_became_active":"2023-10-06T03:10:43.610865+0000","last_became_peered":"2023-10-06T03:10:43.610865+0000","last_unstale":"2023-10-06T03:10:43.611147+0000","last_undegraded":"2023-10-06T03:10:43.611147+0000","last_fullsized":"2023-10-06T03:10:43.611147+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598337+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:31:04.736772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.611002+0000","last_change":"2023-10-06T03:10:43.611002+0000","last_active":"2023-10-06T03:10:43.611002+0000","last_peered":"2023-10-06T03:10:43.611002+0000","last_clean":"2023-10-06T03:10:43.611002+0000","last_became_active":"2023-10-06T03:10:43.610692+0000","last_became_peered":"2023-10-06T03:10:43.610692+0000","last_unstale":"2023-10-06T03:10:43.611002+0000","last_undegraded":"2023-10-06T03:10:43.611002+0000","last_fullsized":"2023-10-06T03:10:43.611002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:15.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598294+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:07:53.654291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.607559+0000","last_change":"2023-10-06T03:10:43.607559+0000","last_active":"2023-10-06T03:10:43.607559+0000","last_peered":"2023-10-06T03:10:43.607559+0000","last_clean":"2023-10-06T03:10:43.607559+0000","last_became_active":"2023-10-06T03:10:43.607302+0000","last_became_peered":"2023-10-06T03:10:43.607302+0000","last_unstale":"2023-10-06T03:10:43.607559+0000","last_undegraded":"2023-10-06T03:10:43.607559+0000","last_fullsized":"2023-10-06T03:10:43.607559+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.596291+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:41.713908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.605635+0000","last_change":"2023-10-06T03:10:43.605635+0000","last_active":"2023-10-06T03:10:43.605635+0000","last_peered":"2023-10-06T03:10:43.605635+0000","last_clean":"2023-10-06T03:10:43.605635+0000","last_became_active":"2023-10-06T03:10:43.605202+0000","last_became_peered":"2023-10-06T03:10:43.605202+0000","last_unstale":"2023-10-06T03:10:43.605635+0000","last_undegraded":"2023-10-06T03:10:43.605635+0000","last_fullsized":"2023-10-06T03:10:43.605635+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:32.594410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597608+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:15:32.244859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":8,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-06T03:10:43.608350+0000","last_change":"2023-10-06T03:10:43.608350+0000","last_active":"2023-10-06T03:10:43.608350+0000","last_peered":"2023-10-06T03:10:43.608350+0000","last_clean":"2023-10-06T03:10:43.608350+0000","last_became_active":"2023-10-06T03:10:43.608121+0000","last_became_peered":"2023-10-06T03:10:43.608121+0000","last_unstale":"2023-10-06T03:10:43.608350+0000","last_undegraded":"2023-10-06T03:10:43.608350+0000","last_fullsized":"2023-10-06T03:10:43.608350+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:14.031939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602346+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:01.251002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594241+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:42.594241+0000","last_peered":"2023-10-06T03:10:42.594241+0000","last_clean":"2023-10-06T03:10:42.594241+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:42.594241+0000","last_undegraded":"2023-10-06T03:10:42.594241+0000","last_fullsized":"2023-10-06T03:10:42.594241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602161+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.596084+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:42.596084+0000","last_peered":"2023-10-06T03:10:42.596084+0000","last_clean":"2023-10-06T03:10:42.596084+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:42.596084+0000","last_undegraded":"2023-10-06T03:10:42.596084+0000","last_fullsized":"2023-10-06T03:10:42.596084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597657+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:10.535161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":148,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594454+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:42.594454+0000","last_peered":"2023-10-06T03:10:42.594454+0000","last_clean":"2023-10-06T03:10:42.594454+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:42.594454+0000","last_undegraded":"2023-10-06T03:10:42.594454+0000","last_fullsized":"2023-10-06T03:10:42.594454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.595405+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:14.184117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594260+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:42.594260+0000","last_peered":"2023-10-06T03:10:42.594260+0000","last_clean":"2023-10-06T03:10:42.594260+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:42.594260+0000","last_undegraded":"2023-10-06T03:10:42.594260+0000","last_fullsized":"2023-10-06T03:10:42.594260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602492+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":301,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.594282+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:42.594282+0000","last_peered":"2023-10-06T03:10:42.594282+0000","last_clean":"2023-10-06T03:10:42.594282+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:42.594282+0000","last_undegraded":"2023-10-06T03:10:42.594282+0000","last_fullsized":"2023-10-06T03:10:42.594282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.592696+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:42.592696+0000","last_peered":"2023-10-06T03:10:42.592696+0000","last_clean":"2023-10-06T03:10:42.592696+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:42.592696+0000","last_undegraded":"2023-10-06T03:10:42.592696+0000","last_fullsized":"2023-10-06T03:10:42.592696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.598892+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:05.627341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":149,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.589735+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:42.589735+0000","last_peered":"2023-10-06T03:10:42.589735+0000","last_clean":"2023-10-06T03:10:42.589735+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:42.589735+0000","last_undegraded":"2023-10-06T03:10:42.589735+0000","last_fullsized":"2023-10-06T03:10:42.589735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.602492+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.592681+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:42.592681+0000","last_peered":"2023-10-06T03:10:42.592681+0000","last_clean":"2023-10-06T03:10:42.592681+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:42.592681+0000","last_undegraded":"2023-10-06T03:10:42.592681+0000","last_fullsized":"2023-10-06T03:10:42.592681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:10:43.586077+0000","last_change":"2023-10-06T03:10:43.597856+0000","last_active":"2023-10-06T03:10:43.586077+0000","last_peered":"2023-10-06T03:10:43.586077+0000","last_clean":"2023-10-06T03:10:43.586077+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:10:43.586077+0000","last_undegraded":"2023-10-06T03:10:43.586077+0000","last_fullsized":"2023-10-06T03:10:43.586077+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:42.386144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-06T03:10:42.596943+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:42.596943+0000","last_peered":"2023-10-06T03:10:42.596943+0000","last_clean":"2023-10-06T03:10:42.596943+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:42.596943+0000","last_undegraded":"2023-10-06T03:10:42.596943+0000","last_fullsized":"2023-10-06T03:10:42.596943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304248,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067592,"statfs":{"total":96636764160,"available":96325214208,"internally_reserved":0,"allocated":2932736,"data_stored":1700569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.66800000000000004},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.72699999999999998,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.623},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.38,"5min":0.33100000000000002,"15min":0.316},"max":{"1min":0.70699999999999996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.53100000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61399999999999999,"5min":1.264,"15min":1.264},"last":0.498},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53800000000000003,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65400000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58899999999999997,"15min":0.56000000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":2.286,"15min":2.286},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.72099999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.70299999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.55000000000000004,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.64000000000000001,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.371},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72599999999999998,"5min":2.504,"15min":2.504},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55100000000000005,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.316},"max":{"1min":0.76800000000000002,"5min":1.141,"15min":1.141},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.36399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":1.639,"15min":1.639},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71199999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.40999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.439}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1110201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.80800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304248,"kb_used_data":2864,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067592,"statfs":{"total":96636764160,"available":96325214208,"internally_reserved":0,"allocated":2932736,"data_stored":1700569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.248},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.30299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.26200000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.495},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.29999999999999999,"15min":0.317},"min":{"1min":0.217,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.39800000000000002,"5min":0.498,"15min":0.53100000000000003},"last":0.376},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.248,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.45100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.29399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.34699999999999998},"min":{"1min":0.29499999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.42299999999999999,"5min":0.504,"15min":0.56599999999999995},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32400000000000001},"min":{"1min":0.254,"5min":0.222,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.376},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.439,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":0.85599999999999998,"15min":0.86299999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58199999999999996,"15min":0.54800000000000004},"min":{"1min":0.45400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.80900000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.53200000000000003},"min":{"1min":0.42599999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.44400000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76900000000000002,"5min":0.82399999999999995,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58799999999999997,"15min":0.56100000000000005},"min":{"1min":0.432,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59499999999999997,"15min":0.59199999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":1.8740000000000001},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58699999999999997,"15min":0.61099999999999999},"min":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.75800000000000001,"5min":0.90200000000000002,"15min":1.903},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32700000000000001,"15min":0.311},"min":{"1min":0.22700000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.45100000000000001,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33100000000000002},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.433,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.312,"15min":0.309},"min":{"1min":0.23599999999999999,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.42499999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.32500000000000001,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.216,"15min":0.216},"max":{"1min":0.442,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35299999999999998,"15min":0.34799999999999998},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.437,"5min":0.63400000000000001,"15min":0.67500000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35099999999999998,"15min":0.35399999999999998},"min":{"1min":0.253,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.438,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.41399999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77300000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.35399999999999998,"15min":0.34799999999999998},"max":{"1min":0.81699999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.44900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.56000000000000005,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77300000000000002,"5min":0.873,"15min":5.5220000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80800000000000005,"5min":0.997,"15min":0.997},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.39700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.78400000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76300000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1110201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.627},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.38600000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.40899999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303656,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068184,"statfs":{"total":96636764160,"available":96325820416,"internally_reserved":0,"allocated":2326528,"data_stored":1103585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71199999999999997,"5min":1.143,"15min":2.3599999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.59699999999999998},"min":{"1min":0.371,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83499999999999996,"5min":0.86299999999999999,"15min":2.371},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.53600000000000003},"min":{"1min":0.39700000000000002,"5min":0.32500000000000001,"15min":0.30599999999999999},"max":{"1min":0.67800000000000005,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.316},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.55200000000000005,"15min":0.58399999999999996},"min":{"1min":0.41399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.64600000000000002,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":6.532,"15min":6.532},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58099999999999996,"15min":0.69499999999999995},"min":{"1min":0.37,"5min":0.315,"15min":0.315},"max":{"1min":0.74299999999999999,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.66400000000000003,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.374},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58599999999999997,"15min":0.56999999999999995},"min":{"1min":0.39600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":5.524,"15min":5.524},"last":0.496}]},{"osd":4,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.504,"15min":0.52100000000000002},"min":{"1min":0.31900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.73699999999999999,"5min":0.86699999999999999,"15min":0.92900000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.57299999999999995},"min":{"1min":0.29399999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75,"5min":0.81499999999999995,"15min":0.97299999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52000000000000002,"15min":0.55600000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68200000000000005,"5min":0.81000000000000005,"15min":1.1990000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80700000000000005,"5min":0.84199999999999997,"15min":1.052},"last":0.747}]},{"osd":7,"last update":"Fri Oct 6 03:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.61899999999999999},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":0.878,"15min":1.175},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56499999999999995,"15min":0.60399999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71199999999999997,"5min":0.75600000000000001,"15min":1.145},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:49.045 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:49.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.190+0000 7f02ea25f700 1 Processor -- start 2023-10-06T03:10:49.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.190+0000 7f02ea25f700 1 -- start start 2023-10-06T03:10:49.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02ea25f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159890 0x7f02e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02ea25f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02e4159c60 0x7f02e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02ea25f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e415a160 con 0x7f02e415db10 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02ea25f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40d13a0 con 0x7f02e4159c60 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02ea25f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e415e140 con 0x7f02e4159890 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02e2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02e4159c60 0x7f02e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02e2ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02e4159c60 0x7f02e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45110/0 (socket says 172.21.15.92:45110) 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.191+0000 7f02e2ffd700 1 -- 172.21.15.92:0/1244881404 learned_addr learned my addr 172.21.15.92:0/1244881404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e2ffd700 1 -- 172.21.15.92:0/1244881404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159890 msgr2=0x7f02e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e37fe700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159890 0x7f02e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e2ffd700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159890 0x7f02e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e2ffd700 1 -- 172.21.15.92:0/1244881404 >> v1:172.21.15.136:6789/0 conn(0x7f02e415db10 legacy=0x7f02e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e2ffd700 1 -- 172.21.15.92:0/1244881404 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e415a3d0 con 0x7f02e4159c60 2023-10-06T03:10:49.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e37fe700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159890 0x7f02e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e2ffd700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159c60 0x7f02e415d5d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f02d80099b0 tx=0x7f02d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62c4fb1359a5a66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e27fc700 1 -- 172.21.15.92:0/1244881404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800b820 con 0x7f02e4159c60 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e27fc700 1 -- 172.21.15.92:0/1244881404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d800b980 con 0x7f02e4159c60 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02e27fc700 1 -- 172.21.15.92:0/1244881404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d800bd20 con 0x7f02e4159c60 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02ea25f700 1 -- 172.21.15.92:0/1244881404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159c60 msgr2=0x7f02e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02ea25f700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159c60 0x7f02e415d5d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f02d80099b0 tx=0x7f02d8009250 comp rx=0 tx=0).stop 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02ea25f700 1 -- 172.21.15.92:0/1244881404 shutdown_connections 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02ea25f700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159890 0x7f02e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02ea25f700 1 --2- 172.21.15.92:0/1244881404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159c60 0x7f02e415d5d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.192+0000 7f02ea25f700 1 -- 172.21.15.92:0/1244881404 >> 172.21.15.92:0/1244881404 conn(0x7f02e4068e20 msgr2=0x7f02e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:49.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.193+0000 7f02ea25f700 1 -- 172.21.15.92:0/1244881404 shutdown_connections 2023-10-06T03:10:49.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.193+0000 7f02ea25f700 1 -- 172.21.15.92:0/1244881404 wait complete. 2023-10-06T03:10:49.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.194+0000 7f02ea25f700 1 Processor -- start 2023-10-06T03:10:49.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.194+0000 7f02ea25f700 1 -- start start 2023-10-06T03:10:49.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.194+0000 7f02ea25f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 0x7f02e40cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.194+0000 7f02ea25f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159c60 0x7f02e4152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.194+0000 7f02ea25f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e415db10 0x7f02e40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.194+0000 7f02ea25f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40cfcd0 con 0x7f02e4159890 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02ea25f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40cff00 con 0x7f02e415db10 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02ea25f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4152950 con 0x7f02e4159c60 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 0x7f02e40cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 0x7f02e40cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:45114/0 (socket says 172.21.15.92:45114) 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 -- 172.21.15.92:0/3309607618 learned_addr learned my addr 172.21.15.92:0/3309607618 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e2ffd700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159c60 0x7f02e4152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e3fff700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e415db10 0x7f02e40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 -- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159c60 msgr2=0x7f02e4152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159c60 0x7f02e4152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 -- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e415db10 msgr2=0x7f02e40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e415db10 0x7f02e40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 -- 172.21.15.92:0/3309607618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80094f0 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02e37fe700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 0x7f02e40cf790 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f02d400a430 tx=0x7f02d4007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4015440 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d40155a0 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e416a0c0 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4015940 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.195+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e416a680 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.196+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e404c3c0 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.197+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f02d4015cf0 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.197+0000 7f02cb7fe700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02d0067320 0x7f02d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.197+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f02d4085550 con 0x7f02e4159890 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.197+0000 7f02e2ffd700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02d0067320 0x7f02d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.198+0000 7f02e2ffd700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02d0067320 0x7f02d00697d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f02d80099b0 tx=0x7f02d8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.201+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02d404e5a0 con 0x7f02e4159890 2023-10-06T03:10:49.383 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.382+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02e406a5b0 con 0x7f02d0067320 2023-10-06T03:10:49.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.390+0000 7f02cb7fe700 1 -- 172.21.15.92:0/3309607618 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311825 (secure 0 0 0) 0x7f02e406a5b0 con 0x7f02d0067320 2023-10-06T03:10:49.395 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:49.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02d0067320 msgr2=0x7f02d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02d0067320 0x7f02d00697d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f02d80099b0 tx=0x7f02d8016000 comp rx=0 tx=0).stop 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 msgr2=0x7f02e40cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 0x7f02e40cf790 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f02d400a430 tx=0x7f02d4007910 comp rx=0 tx=0).stop 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 shutdown_connections 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f02d0067320 0x7f02d00697d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02e4159890 0x7f02e40cf790 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e415db10 0x7f02e40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 --2- 172.21.15.92:0/3309607618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02e4159c60 0x7f02e4152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 >> 172.21.15.92:0/3309607618 conn(0x7f02e4068e20 msgr2=0x7f02e4153190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:49.398 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 shutdown_connections 2023-10-06T03:10:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.393+0000 7f02ea25f700 1 -- 172.21.15.92:0/3309607618 wait complete. 2023-10-06T03:10:49.399 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:49.407 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2023-10-06T03:10:49.293882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430760,"kb_used_data":19688,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752543960,"statfs":{"total":773094113280,"available":770605015040,"internally_reserved":0,"allocated":20160512,"data_stored":10177012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001246"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611079+0000","last_change":"2023-10-06T03:10:44.608143+0000","last_active":"2023-10-06T03:10:44.611079+0000","last_peered":"2023-10-06T03:10:44.611079+0000","last_clean":"2023-10-06T03:10:44.611079+0000","last_became_active":"2023-10-06T03:10:44.607875+0000","last_became_peered":"2023-10-06T03:10:44.607875+0000","last_unstale":"2023-10-06T03:10:44.611079+0000","last_undegraded":"2023-10-06T03:10:44.611079+0000","last_fullsized":"2023-10-06T03:10:44.611079+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:15.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617809+0000","last_change":"2023-10-06T03:10:44.616908+0000","last_active":"2023-10-06T03:10:44.617809+0000","last_peered":"2023-10-06T03:10:44.617809+0000","last_clean":"2023-10-06T03:10:44.617809+0000","last_became_active":"2023-10-06T03:10:44.616555+0000","last_became_peered":"2023-10-06T03:10:44.616555+0000","last_unstale":"2023-10-06T03:10:44.617809+0000","last_undegraded":"2023-10-06T03:10:44.617809+0000","last_fullsized":"2023-10-06T03:10:44.617809+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:23.309599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611589+0000","last_change":"2023-10-06T03:10:44.605922+0000","last_active":"2023-10-06T03:10:44.611589+0000","last_peered":"2023-10-06T03:10:44.611589+0000","last_clean":"2023-10-06T03:10:44.611589+0000","last_became_active":"2023-10-06T03:10:44.605623+0000","last_became_peered":"2023-10-06T03:10:44.605623+0000","last_unstale":"2023-10-06T03:10:44.611589+0000","last_undegraded":"2023-10-06T03:10:44.611589+0000","last_fullsized":"2023-10-06T03:10:44.611589+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612282+0000","last_change":"2023-10-06T03:10:44.610178+0000","last_active":"2023-10-06T03:10:44.612282+0000","last_peered":"2023-10-06T03:10:44.612282+0000","last_clean":"2023-10-06T03:10:44.612282+0000","last_became_active":"2023-10-06T03:10:44.609892+0000","last_became_peered":"2023-10-06T03:10:44.609892+0000","last_unstale":"2023-10-06T03:10:44.612282+0000","last_undegraded":"2023-10-06T03:10:44.612282+0000","last_fullsized":"2023-10-06T03:10:44.612282+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612326+0000","last_change":"2023-10-06T03:10:44.608872+0000","last_active":"2023-10-06T03:10:44.612326+0000","last_peered":"2023-10-06T03:10:44.612326+0000","last_clean":"2023-10-06T03:10:44.612326+0000","last_became_active":"2023-10-06T03:10:44.608412+0000","last_became_peered":"2023-10-06T03:10:44.608412+0000","last_unstale":"2023-10-06T03:10:44.612326+0000","last_undegraded":"2023-10-06T03:10:44.612326+0000","last_fullsized":"2023-10-06T03:10:44.612326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:42.363268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.609968+0000","last_change":"2023-10-06T03:10:44.605533+0000","last_active":"2023-10-06T03:10:44.609968+0000","last_peered":"2023-10-06T03:10:44.609968+0000","last_clean":"2023-10-06T03:10:44.609968+0000","last_became_active":"2023-10-06T03:10:44.605254+0000","last_became_peered":"2023-10-06T03:10:44.605254+0000","last_unstale":"2023-10-06T03:10:44.609968+0000","last_undegraded":"2023-10-06T03:10:44.609968+0000","last_fullsized":"2023-10-06T03:10:44.609968+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:22.407090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614874+0000","last_change":"2023-10-06T03:10:44.613032+0000","last_active":"2023-10-06T03:10:44.614874+0000","last_peered":"2023-10-06T03:10:44.614874+0000","last_clean":"2023-10-06T03:10:44.614874+0000","last_became_active":"2023-10-06T03:10:44.612535+0000","last_became_peered":"2023-10-06T03:10:44.612535+0000","last_unstale":"2023-10-06T03:10:44.614874+0000","last_undegraded":"2023-10-06T03:10:44.614874+0000","last_fullsized":"2023-10-06T03:10:44.614874+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:27.693399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619444+0000","last_change":"2023-10-06T03:10:44.612966+0000","last_active":"2023-10-06T03:10:44.619444+0000","last_peered":"2023-10-06T03:10:44.619444+0000","last_clean":"2023-10-06T03:10:44.619444+0000","last_became_active":"2023-10-06T03:10:44.612484+0000","last_became_peered":"2023-10-06T03:10:44.612484+0000","last_unstale":"2023-10-06T03:10:44.619444+0000","last_undegraded":"2023-10-06T03:10:44.619444+0000","last_fullsized":"2023-10-06T03:10:44.619444+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:32.111948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615003+0000","last_change":"2023-10-06T03:10:44.609587+0000","last_active":"2023-10-06T03:10:44.615003+0000","last_peered":"2023-10-06T03:10:44.615003+0000","last_clean":"2023-10-06T03:10:44.615003+0000","last_became_active":"2023-10-06T03:10:44.609390+0000","last_became_peered":"2023-10-06T03:10:44.609390+0000","last_unstale":"2023-10-06T03:10:44.615003+0000","last_undegraded":"2023-10-06T03:10:44.615003+0000","last_fullsized":"2023-10-06T03:10:44.615003+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:33.614656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618712+0000","last_change":"2023-10-06T03:10:44.618093+0000","last_active":"2023-10-06T03:10:44.618712+0000","last_peered":"2023-10-06T03:10:44.618712+0000","last_clean":"2023-10-06T03:10:44.618712+0000","last_became_active":"2023-10-06T03:10:44.617856+0000","last_became_peered":"2023-10-06T03:10:44.617856+0000","last_unstale":"2023-10-06T03:10:44.618712+0000","last_undegraded":"2023-10-06T03:10:44.618712+0000","last_fullsized":"2023-10-06T03:10:44.618712+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.931574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.623816+0000","last_change":"2023-10-06T03:10:44.623316+0000","last_active":"2023-10-06T03:10:44.623816+0000","last_peered":"2023-10-06T03:10:44.623816+0000","last_clean":"2023-10-06T03:10:44.623816+0000","last_became_active":"2023-10-06T03:10:44.623058+0000","last_became_peered":"2023-10-06T03:10:44.623058+0000","last_unstale":"2023-10-06T03:10:44.623816+0000","last_undegraded":"2023-10-06T03:10:44.623816+0000","last_fullsized":"2023-10-06T03:10:44.623816+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:37.745837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606653+0000","last_change":"2023-10-06T03:10:44.604123+0000","last_active":"2023-10-06T03:10:44.606653+0000","last_peered":"2023-10-06T03:10:44.606653+0000","last_clean":"2023-10-06T03:10:44.606653+0000","last_became_active":"2023-10-06T03:10:44.603822+0000","last_became_peered":"2023-10-06T03:10:44.603822+0000","last_unstale":"2023-10-06T03:10:44.606653+0000","last_undegraded":"2023-10-06T03:10:44.606653+0000","last_fullsized":"2023-10-06T03:10:44.606653+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:50.256975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624806+0000","last_change":"2023-10-06T03:10:44.612705+0000","last_active":"2023-10-06T03:10:44.624806+0000","last_peered":"2023-10-06T03:10:44.624806+0000","last_clean":"2023-10-06T03:10:44.624806+0000","last_became_active":"2023-10-06T03:10:44.612459+0000","last_became_peered":"2023-10-06T03:10:44.612459+0000","last_unstale":"2023-10-06T03:10:44.624806+0000","last_undegraded":"2023-10-06T03:10:44.624806+0000","last_fullsized":"2023-10-06T03:10:44.624806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:31.032848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625100+0000","last_change":"2023-10-06T03:10:44.608588+0000","last_active":"2023-10-06T03:10:44.625100+0000","last_peered":"2023-10-06T03:10:44.625100+0000","last_clean":"2023-10-06T03:10:44.625100+0000","last_became_active":"2023-10-06T03:10:44.607707+0000","last_became_peered":"2023-10-06T03:10:44.607707+0000","last_unstale":"2023-10-06T03:10:44.625100+0000","last_undegraded":"2023-10-06T03:10:44.625100+0000","last_fullsized":"2023-10-06T03:10:44.625100+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:11.796662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624888+0000","last_change":"2023-10-06T03:10:44.612883+0000","last_active":"2023-10-06T03:10:44.624888+0000","last_peered":"2023-10-06T03:10:44.624888+0000","last_clean":"2023-10-06T03:10:44.624888+0000","last_became_active":"2023-10-06T03:10:44.612648+0000","last_became_peered":"2023-10-06T03:10:44.612648+0000","last_unstale":"2023-10-06T03:10:44.624888+0000","last_undegraded":"2023-10-06T03:10:44.624888+0000","last_fullsized":"2023-10-06T03:10:44.624888+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:21.015787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622176+0000","last_change":"2023-10-06T03:10:44.621118+0000","last_active":"2023-10-06T03:10:44.622176+0000","last_peered":"2023-10-06T03:10:44.622176+0000","last_clean":"2023-10-06T03:10:44.622176+0000","last_became_active":"2023-10-06T03:10:44.614783+0000","last_became_peered":"2023-10-06T03:10:44.614783+0000","last_unstale":"2023-10-06T03:10:44.622176+0000","last_undegraded":"2023-10-06T03:10:44.622176+0000","last_fullsized":"2023-10-06T03:10:44.622176+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:04.349135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616832+0000","last_change":"2023-10-06T03:10:44.607117+0000","last_active":"2023-10-06T03:10:44.616832+0000","last_peered":"2023-10-06T03:10:44.616832+0000","last_clean":"2023-10-06T03:10:44.616832+0000","last_became_active":"2023-10-06T03:10:44.606719+0000","last_became_peered":"2023-10-06T03:10:44.606719+0000","last_unstale":"2023-10-06T03:10:44.616832+0000","last_undegraded":"2023-10-06T03:10:44.616832+0000","last_fullsized":"2023-10-06T03:10:44.616832+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:20.569599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614899+0000","last_change":"2023-10-06T03:10:44.613168+0000","last_active":"2023-10-06T03:10:44.614899+0000","last_peered":"2023-10-06T03:10:44.614899+0000","last_clean":"2023-10-06T03:10:44.614899+0000","last_became_active":"2023-10-06T03:10:44.612843+0000","last_became_peered":"2023-10-06T03:10:44.612843+0000","last_unstale":"2023-10-06T03:10:44.614899+0000","last_undegraded":"2023-10-06T03:10:44.614899+0000","last_fullsized":"2023-10-06T03:10:44.614899+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:37.217250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619764+0000","last_change":"2023-10-06T03:10:44.619199+0000","last_active":"2023-10-06T03:10:44.619764+0000","last_peered":"2023-10-06T03:10:44.619764+0000","last_clean":"2023-10-06T03:10:44.619764+0000","last_became_active":"2023-10-06T03:10:44.618882+0000","last_became_peered":"2023-10-06T03:10:44.618882+0000","last_unstale":"2023-10-06T03:10:44.619764+0000","last_undegraded":"2023-10-06T03:10:44.619764+0000","last_fullsized":"2023-10-06T03:10:44.619764+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:15.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622081+0000","last_change":"2023-10-06T03:10:44.613713+0000","last_active":"2023-10-06T03:10:44.622081+0000","last_peered":"2023-10-06T03:10:44.622081+0000","last_clean":"2023-10-06T03:10:44.622081+0000","last_became_active":"2023-10-06T03:10:44.610828+0000","last_became_peered":"2023-10-06T03:10:44.610828+0000","last_unstale":"2023-10-06T03:10:44.622081+0000","last_undegraded":"2023-10-06T03:10:44.622081+0000","last_fullsized":"2023-10-06T03:10:44.622081+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:23.629899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615084+0000","last_change":"2023-10-06T03:10:44.612712+0000","last_active":"2023-10-06T03:10:44.615084+0000","last_peered":"2023-10-06T03:10:44.615084+0000","last_clean":"2023-10-06T03:10:44.615084+0000","last_became_active":"2023-10-06T03:10:44.612456+0000","last_became_peered":"2023-10-06T03:10:44.612456+0000","last_unstale":"2023-10-06T03:10:44.615084+0000","last_undegraded":"2023-10-06T03:10:44.615084+0000","last_fullsized":"2023-10-06T03:10:44.615084+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:19.393529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608598+0000","last_change":"2023-10-06T03:10:44.606764+0000","last_active":"2023-10-06T03:10:44.608598+0000","last_peered":"2023-10-06T03:10:44.608598+0000","last_clean":"2023-10-06T03:10:44.608598+0000","last_became_active":"2023-10-06T03:10:44.606218+0000","last_became_peered":"2023-10-06T03:10:44.606218+0000","last_unstale":"2023-10-06T03:10:44.608598+0000","last_undegraded":"2023-10-06T03:10:44.608598+0000","last_fullsized":"2023-10-06T03:10:44.608598+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:00.320153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613821+0000","last_change":"2023-10-06T03:10:44.610117+0000","last_active":"2023-10-06T03:10:44.613821+0000","last_peered":"2023-10-06T03:10:44.613821+0000","last_clean":"2023-10-06T03:10:44.613821+0000","last_became_active":"2023-10-06T03:10:44.609777+0000","last_became_peered":"2023-10-06T03:10:44.609777+0000","last_unstale":"2023-10-06T03:10:44.613821+0000","last_undegraded":"2023-10-06T03:10:44.613821+0000","last_fullsized":"2023-10-06T03:10:44.613821+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617323+0000","last_change":"2023-10-06T03:10:44.611318+0000","last_active":"2023-10-06T03:10:44.617323+0000","last_peered":"2023-10-06T03:10:44.617323+0000","last_clean":"2023-10-06T03:10:44.617323+0000","last_became_active":"2023-10-06T03:10:44.610495+0000","last_became_peered":"2023-10-06T03:10:44.610495+0000","last_unstale":"2023-10-06T03:10:44.617323+0000","last_undegraded":"2023-10-06T03:10:44.617323+0000","last_fullsized":"2023-10-06T03:10:44.617323+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:00.253800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615621+0000","last_change":"2023-10-06T03:10:44.613771+0000","last_active":"2023-10-06T03:10:44.615621+0000","last_peered":"2023-10-06T03:10:44.615621+0000","last_clean":"2023-10-06T03:10:44.615621+0000","last_became_active":"2023-10-06T03:10:44.613387+0000","last_became_peered":"2023-10-06T03:10:44.613387+0000","last_unstale":"2023-10-06T03:10:44.615621+0000","last_undegraded":"2023-10-06T03:10:44.615621+0000","last_fullsized":"2023-10-06T03:10:44.615621+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:15.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"185'4","reported_seq":14,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626960+0000","last_change":"2023-10-06T03:10:43.602925+0000","last_active":"2023-10-06T03:10:44.626960+0000","last_peered":"2023-10-06T03:10:44.626960+0000","last_clean":"2023-10-06T03:10:44.626960+0000","last_became_active":"2023-10-06T03:10:43.602673+0000","last_became_peered":"2023-10-06T03:10:43.602673+0000","last_unstale":"2023-10-06T03:10:44.626960+0000","last_undegraded":"2023-10-06T03:10:44.626960+0000","last_fullsized":"2023-10-06T03:10:44.626960+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624836+0000","last_change":"2023-10-06T03:10:44.612058+0000","last_active":"2023-10-06T03:10:44.624836+0000","last_peered":"2023-10-06T03:10:44.624836+0000","last_clean":"2023-10-06T03:10:44.624836+0000","last_became_active":"2023-10-06T03:10:44.611816+0000","last_became_peered":"2023-10-06T03:10:44.611816+0000","last_unstale":"2023-10-06T03:10:44.624836+0000","last_undegraded":"2023-10-06T03:10:44.624836+0000","last_fullsized":"2023-10-06T03:10:44.624836+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:07.945301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626677+0000","last_change":"2023-10-06T03:10:43.603054+0000","last_active":"2023-10-06T03:10:44.626677+0000","last_peered":"2023-10-06T03:10:44.626677+0000","last_clean":"2023-10-06T03:10:44.626677+0000","last_became_active":"2023-10-06T03:10:43.602849+0000","last_became_peered":"2023-10-06T03:10:43.602849+0000","last_unstale":"2023-10-06T03:10:44.626677+0000","last_undegraded":"2023-10-06T03:10:44.626677+0000","last_fullsized":"2023-10-06T03:10:44.626677+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:44.214194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612802+0000","last_change":"2023-10-06T03:10:44.608006+0000","last_active":"2023-10-06T03:10:44.612802+0000","last_peered":"2023-10-06T03:10:44.612802+0000","last_clean":"2023-10-06T03:10:44.612802+0000","last_became_active":"2023-10-06T03:10:44.607700+0000","last_became_peered":"2023-10-06T03:10:44.607700+0000","last_unstale":"2023-10-06T03:10:44.612802+0000","last_undegraded":"2023-10-06T03:10:44.612802+0000","last_fullsized":"2023-10-06T03:10:44.612802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:29.630492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622258+0000","last_change":"2023-10-06T03:10:43.604672+0000","last_active":"2023-10-06T03:10:44.622258+0000","last_peered":"2023-10-06T03:10:44.622258+0000","last_clean":"2023-10-06T03:10:44.622258+0000","last_became_active":"2023-10-06T03:10:43.602513+0000","last_became_peered":"2023-10-06T03:10:43.602513+0000","last_unstale":"2023-10-06T03:10:44.622258+0000","last_undegraded":"2023-10-06T03:10:44.622258+0000","last_fullsized":"2023-10-06T03:10:44.622258+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:36.206746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606511+0000","last_change":"2023-10-06T03:10:44.603518+0000","last_active":"2023-10-06T03:10:44.606511+0000","last_peered":"2023-10-06T03:10:44.606511+0000","last_clean":"2023-10-06T03:10:44.606511+0000","last_became_active":"2023-10-06T03:10:44.603255+0000","last_became_peered":"2023-10-06T03:10:44.603255+0000","last_unstale":"2023-10-06T03:10:44.606511+0000","last_undegraded":"2023-10-06T03:10:44.606511+0000","last_fullsized":"2023-10-06T03:10:44.606511+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626714+0000","last_change":"2023-10-06T03:10:43.609088+0000","last_active":"2023-10-06T03:10:44.626714+0000","last_peered":"2023-10-06T03:10:44.626714+0000","last_clean":"2023-10-06T03:10:44.626714+0000","last_became_active":"2023-10-06T03:10:43.608821+0000","last_became_peered":"2023-10-06T03:10:43.608821+0000","last_unstale":"2023-10-06T03:10:44.626714+0000","last_undegraded":"2023-10-06T03:10:44.626714+0000","last_fullsized":"2023-10-06T03:10:44.626714+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:36.735333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608618+0000","last_change":"2023-10-06T03:10:44.606922+0000","last_active":"2023-10-06T03:10:44.608618+0000","last_peered":"2023-10-06T03:10:44.608618+0000","last_clean":"2023-10-06T03:10:44.608618+0000","last_became_active":"2023-10-06T03:10:44.606522+0000","last_became_peered":"2023-10-06T03:10:44.606522+0000","last_unstale":"2023-10-06T03:10:44.608618+0000","last_undegraded":"2023-10-06T03:10:44.608618+0000","last_fullsized":"2023-10-06T03:10:44.608618+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:15.658864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.620526+0000","last_change":"2023-10-06T03:10:43.610186+0000","last_active":"2023-10-06T03:10:44.620526+0000","last_peered":"2023-10-06T03:10:44.620526+0000","last_clean":"2023-10-06T03:10:44.620526+0000","last_became_active":"2023-10-06T03:10:43.609855+0000","last_became_peered":"2023-10-06T03:10:43.609855+0000","last_unstale":"2023-10-06T03:10:44.620526+0000","last_undegraded":"2023-10-06T03:10:44.620526+0000","last_fullsized":"2023-10-06T03:10:44.620526+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613890+0000","last_change":"2023-10-06T03:10:44.608127+0000","last_active":"2023-10-06T03:10:44.613890+0000","last_peered":"2023-10-06T03:10:44.613890+0000","last_clean":"2023-10-06T03:10:44.613890+0000","last_became_active":"2023-10-06T03:10:44.607790+0000","last_became_peered":"2023-10-06T03:10:44.607790+0000","last_unstale":"2023-10-06T03:10:44.613890+0000","last_undegraded":"2023-10-06T03:10:44.613890+0000","last_fullsized":"2023-10-06T03:10:44.613890+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:03.796183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619316+0000","last_change":"2023-10-06T03:10:43.606849+0000","last_active":"2023-10-06T03:10:44.619316+0000","last_peered":"2023-10-06T03:10:44.619316+0000","last_clean":"2023-10-06T03:10:44.619316+0000","last_became_active":"2023-10-06T03:10:43.606628+0000","last_became_peered":"2023-10-06T03:10:43.606628+0000","last_unstale":"2023-10-06T03:10:44.619316+0000","last_undegraded":"2023-10-06T03:10:44.619316+0000","last_fullsized":"2023-10-06T03:10:44.619316+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:37.755076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615572+0000","last_change":"2023-10-06T03:10:44.609219+0000","last_active":"2023-10-06T03:10:44.615572+0000","last_peered":"2023-10-06T03:10:44.615572+0000","last_clean":"2023-10-06T03:10:44.615572+0000","last_became_active":"2023-10-06T03:10:44.608960+0000","last_became_peered":"2023-10-06T03:10:44.608960+0000","last_unstale":"2023-10-06T03:10:44.615572+0000","last_undegraded":"2023-10-06T03:10:44.615572+0000","last_fullsized":"2023-10-06T03:10:44.615572+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:58.313141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597171+0000","last_change":"2023-10-06T03:10:43.617619+0000","last_active":"2023-10-06T03:10:44.597171+0000","last_peered":"2023-10-06T03:10:44.597171+0000","last_clean":"2023-10-06T03:10:44.597171+0000","last_became_active":"2023-10-06T03:10:43.617376+0000","last_became_peered":"2023-10-06T03:10:43.617376+0000","last_unstale":"2023-10-06T03:10:44.597171+0000","last_undegraded":"2023-10-06T03:10:44.597171+0000","last_fullsized":"2023-10-06T03:10:44.597171+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:25.490995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614923+0000","last_change":"2023-10-06T03:10:44.613231+0000","last_active":"2023-10-06T03:10:44.614923+0000","last_peered":"2023-10-06T03:10:44.614923+0000","last_clean":"2023-10-06T03:10:44.614923+0000","last_became_active":"2023-10-06T03:10:44.612945+0000","last_became_peered":"2023-10-06T03:10:44.612945+0000","last_unstale":"2023-10-06T03:10:44.614923+0000","last_undegraded":"2023-10-06T03:10:44.614923+0000","last_fullsized":"2023-10-06T03:10:44.614923+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:50.256975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602634+0000","last_change":"2023-10-06T03:10:43.608993+0000","last_active":"2023-10-06T03:10:44.602634+0000","last_peered":"2023-10-06T03:10:44.602634+0000","last_clean":"2023-10-06T03:10:44.602634+0000","last_became_active":"2023-10-06T03:10:43.608716+0000","last_became_peered":"2023-10-06T03:10:43.608716+0000","last_unstale":"2023-10-06T03:10:44.602634+0000","last_undegraded":"2023-10-06T03:10:44.602634+0000","last_fullsized":"2023-10-06T03:10:44.602634+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:13.569451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614842+0000","last_change":"2023-10-06T03:10:44.612005+0000","last_active":"2023-10-06T03:10:44.614842+0000","last_peered":"2023-10-06T03:10:44.614842+0000","last_clean":"2023-10-06T03:10:44.614842+0000","last_became_active":"2023-10-06T03:10:44.611698+0000","last_became_peered":"2023-10-06T03:10:44.611698+0000","last_unstale":"2023-10-06T03:10:44.614842+0000","last_undegraded":"2023-10-06T03:10:44.614842+0000","last_fullsized":"2023-10-06T03:10:44.614842+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:44.601383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603232+0000","last_change":"2023-10-06T03:10:43.608400+0000","last_active":"2023-10-06T03:10:44.603232+0000","last_peered":"2023-10-06T03:10:44.603232+0000","last_clean":"2023-10-06T03:10:44.603232+0000","last_became_active":"2023-10-06T03:10:43.607925+0000","last_became_peered":"2023-10-06T03:10:43.607925+0000","last_unstale":"2023-10-06T03:10:44.603232+0000","last_undegraded":"2023-10-06T03:10:44.603232+0000","last_fullsized":"2023-10-06T03:10:44.603232+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:36.739791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616814+0000","last_change":"2023-10-06T03:10:44.607010+0000","last_active":"2023-10-06T03:10:44.616814+0000","last_peered":"2023-10-06T03:10:44.616814+0000","last_clean":"2023-10-06T03:10:44.616814+0000","last_became_active":"2023-10-06T03:10:44.606570+0000","last_became_peered":"2023-10-06T03:10:44.606570+0000","last_unstale":"2023-10-06T03:10:44.616814+0000","last_undegraded":"2023-10-06T03:10:44.616814+0000","last_fullsized":"2023-10-06T03:10:44.616814+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:59.437351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597622+0000","last_change":"2023-10-06T03:10:43.611484+0000","last_active":"2023-10-06T03:10:44.597622+0000","last_peered":"2023-10-06T03:10:44.597622+0000","last_clean":"2023-10-06T03:10:44.597622+0000","last_became_active":"2023-10-06T03:10:43.611227+0000","last_became_peered":"2023-10-06T03:10:43.611227+0000","last_unstale":"2023-10-06T03:10:44.597622+0000","last_undegraded":"2023-10-06T03:10:44.597622+0000","last_fullsized":"2023-10-06T03:10:44.597622+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:43.590879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625995+0000","last_change":"2023-10-06T03:10:44.613128+0000","last_active":"2023-10-06T03:10:44.625995+0000","last_peered":"2023-10-06T03:10:44.625995+0000","last_clean":"2023-10-06T03:10:44.625995+0000","last_became_active":"2023-10-06T03:10:44.612674+0000","last_became_peered":"2023-10-06T03:10:44.612674+0000","last_unstale":"2023-10-06T03:10:44.625995+0000","last_undegraded":"2023-10-06T03:10:44.625995+0000","last_fullsized":"2023-10-06T03:10:44.625995+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:04.843808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"185'3","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622581+0000","last_change":"2023-10-06T03:10:43.610252+0000","last_active":"2023-10-06T03:10:44.622581+0000","last_peered":"2023-10-06T03:10:44.622581+0000","last_clean":"2023-10-06T03:10:44.622581+0000","last_became_active":"2023-10-06T03:10:43.609979+0000","last_became_peered":"2023-10-06T03:10:43.609979+0000","last_unstale":"2023-10-06T03:10:44.622581+0000","last_undegraded":"2023-10-06T03:10:44.622581+0000","last_fullsized":"2023-10-06T03:10:44.622581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:26.682895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618618+0000","last_change":"2023-10-06T03:10:44.617246+0000","last_active":"2023-10-06T03:10:44.618618+0000","last_peered":"2023-10-06T03:10:44.618618+0000","last_clean":"2023-10-06T03:10:44.618618+0000","last_became_active":"2023-10-06T03:10:44.616958+0000","last_became_peered":"2023-10-06T03:10:44.616958+0000","last_unstale":"2023-10-06T03:10:44.618618+0000","last_undegraded":"2023-10-06T03:10:44.618618+0000","last_fullsized":"2023-10-06T03:10:44.618618+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.761168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"185'6","reported_seq":16,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625574+0000","last_change":"2023-10-06T03:10:43.604507+0000","last_active":"2023-10-06T03:10:44.625574+0000","last_peered":"2023-10-06T03:10:44.625574+0000","last_clean":"2023-10-06T03:10:44.625574+0000","last_became_active":"2023-10-06T03:10:43.604261+0000","last_became_peered":"2023-10-06T03:10:43.604261+0000","last_unstale":"2023-10-06T03:10:44.625574+0000","last_undegraded":"2023-10-06T03:10:44.625574+0000","last_fullsized":"2023-10-06T03:10:44.625574+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:12.780687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617308+0000","last_change":"2023-10-06T03:10:44.612460+0000","last_active":"2023-10-06T03:10:44.617308+0000","last_peered":"2023-10-06T03:10:44.617308+0000","last_clean":"2023-10-06T03:10:44.617308+0000","last_became_active":"2023-10-06T03:10:44.612210+0000","last_became_peered":"2023-10-06T03:10:44.612210+0000","last_unstale":"2023-10-06T03:10:44.617308+0000","last_undegraded":"2023-10-06T03:10:44.617308+0000","last_fullsized":"2023-10-06T03:10:44.617308+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:42.518427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626985+0000","last_change":"2023-10-06T03:10:43.609477+0000","last_active":"2023-10-06T03:10:44.626985+0000","last_peered":"2023-10-06T03:10:44.626985+0000","last_clean":"2023-10-06T03:10:44.626985+0000","last_became_active":"2023-10-06T03:10:43.609213+0000","last_became_peered":"2023-10-06T03:10:43.609213+0000","last_unstale":"2023-10-06T03:10:44.626985+0000","last_undegraded":"2023-10-06T03:10:44.626985+0000","last_fullsized":"2023-10-06T03:10:44.626985+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:20.842732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613908+0000","last_change":"2023-10-06T03:10:44.609257+0000","last_active":"2023-10-06T03:10:44.613908+0000","last_peered":"2023-10-06T03:10:44.613908+0000","last_clean":"2023-10-06T03:10:44.613908+0000","last_became_active":"2023-10-06T03:10:44.609084+0000","last_became_peered":"2023-10-06T03:10:44.609084+0000","last_unstale":"2023-10-06T03:10:44.613908+0000","last_undegraded":"2023-10-06T03:10:44.613908+0000","last_fullsized":"2023-10-06T03:10:44.613908+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:42.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.594785+0000","last_change":"2023-10-06T03:10:43.611154+0000","last_active":"2023-10-06T03:10:44.594785+0000","last_peered":"2023-10-06T03:10:44.594785+0000","last_clean":"2023-10-06T03:10:44.594785+0000","last_became_active":"2023-10-06T03:10:43.610907+0000","last_became_peered":"2023-10-06T03:10:43.610907+0000","last_unstale":"2023-10-06T03:10:44.594785+0000","last_undegraded":"2023-10-06T03:10:44.594785+0000","last_fullsized":"2023-10-06T03:10:44.594785+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617353+0000","last_change":"2023-10-06T03:10:44.611635+0000","last_active":"2023-10-06T03:10:44.617353+0000","last_peered":"2023-10-06T03:10:44.617353+0000","last_clean":"2023-10-06T03:10:44.617353+0000","last_became_active":"2023-10-06T03:10:44.611050+0000","last_became_peered":"2023-10-06T03:10:44.611050+0000","last_unstale":"2023-10-06T03:10:44.617353+0000","last_undegraded":"2023-10-06T03:10:44.617353+0000","last_fullsized":"2023-10-06T03:10:44.617353+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:40.043675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602572+0000","last_change":"2023-10-06T03:10:43.608802+0000","last_active":"2023-10-06T03:10:44.602572+0000","last_peered":"2023-10-06T03:10:44.602572+0000","last_clean":"2023-10-06T03:10:44.602572+0000","last_became_active":"2023-10-06T03:10:43.608524+0000","last_became_peered":"2023-10-06T03:10:43.608524+0000","last_unstale":"2023-10-06T03:10:44.602572+0000","last_undegraded":"2023-10-06T03:10:44.602572+0000","last_fullsized":"2023-10-06T03:10:44.602572+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:28.221106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616261+0000","last_change":"2023-10-06T03:10:44.605209+0000","last_active":"2023-10-06T03:10:44.616261+0000","last_peered":"2023-10-06T03:10:44.616261+0000","last_clean":"2023-10-06T03:10:44.616261+0000","last_became_active":"2023-10-06T03:10:44.604955+0000","last_became_peered":"2023-10-06T03:10:44.604955+0000","last_unstale":"2023-10-06T03:10:44.616261+0000","last_undegraded":"2023-10-06T03:10:44.616261+0000","last_fullsized":"2023-10-06T03:10:44.616261+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:38.258343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"185'8","reported_seq":18,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.629219+0000","last_change":"2023-10-06T03:10:43.617468+0000","last_active":"2023-10-06T03:10:44.629219+0000","last_peered":"2023-10-06T03:10:44.629219+0000","last_clean":"2023-10-06T03:10:44.629219+0000","last_became_active":"2023-10-06T03:10:43.617179+0000","last_became_peered":"2023-10-06T03:10:43.617179+0000","last_unstale":"2023-10-06T03:10:44.629219+0000","last_undegraded":"2023-10-06T03:10:44.629219+0000","last_fullsized":"2023-10-06T03:10:44.629219+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:31.017050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613036+0000","last_change":"2023-10-06T03:10:44.606315+0000","last_active":"2023-10-06T03:10:44.613036+0000","last_peered":"2023-10-06T03:10:44.613036+0000","last_clean":"2023-10-06T03:10:44.613036+0000","last_became_active":"2023-10-06T03:10:44.605927+0000","last_became_peered":"2023-10-06T03:10:44.605927+0000","last_unstale":"2023-10-06T03:10:44.613036+0000","last_undegraded":"2023-10-06T03:10:44.613036+0000","last_fullsized":"2023-10-06T03:10:44.613036+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:03.205188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.621050+0000","last_change":"2023-10-06T03:10:43.607558+0000","last_active":"2023-10-06T03:10:44.621050+0000","last_peered":"2023-10-06T03:10:44.621050+0000","last_clean":"2023-10-06T03:10:44.621050+0000","last_became_active":"2023-10-06T03:10:43.607274+0000","last_became_peered":"2023-10-06T03:10:43.607274+0000","last_unstale":"2023-10-06T03:10:44.621050+0000","last_undegraded":"2023-10-06T03:10:44.621050+0000","last_fullsized":"2023-10-06T03:10:44.621050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:55.593599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618323+0000","last_change":"2023-10-06T03:10:44.614485+0000","last_active":"2023-10-06T03:10:44.618323+0000","last_peered":"2023-10-06T03:10:44.618323+0000","last_clean":"2023-10-06T03:10:44.618323+0000","last_became_active":"2023-10-06T03:10:44.614187+0000","last_became_peered":"2023-10-06T03:10:44.614187+0000","last_unstale":"2023-10-06T03:10:44.618323+0000","last_undegraded":"2023-10-06T03:10:44.618323+0000","last_fullsized":"2023-10-06T03:10:44.618323+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:23.629899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.621860+0000","last_change":"2023-10-06T03:10:43.608774+0000","last_active":"2023-10-06T03:10:44.621860+0000","last_peered":"2023-10-06T03:10:44.621860+0000","last_clean":"2023-10-06T03:10:44.621860+0000","last_became_active":"2023-10-06T03:10:43.608468+0000","last_became_peered":"2023-10-06T03:10:43.608468+0000","last_unstale":"2023-10-06T03:10:44.621860+0000","last_undegraded":"2023-10-06T03:10:44.621860+0000","last_fullsized":"2023-10-06T03:10:44.621860+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:49.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626701+0000","last_change":"2023-10-06T03:10:44.625520+0000","last_active":"2023-10-06T03:10:44.626701+0000","last_peered":"2023-10-06T03:10:44.626701+0000","last_clean":"2023-10-06T03:10:44.626701+0000","last_became_active":"2023-10-06T03:10:44.625234+0000","last_became_peered":"2023-10-06T03:10:44.625234+0000","last_unstale":"2023-10-06T03:10:44.626701+0000","last_undegraded":"2023-10-06T03:10:44.626701+0000","last_fullsized":"2023-10-06T03:10:44.626701+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:06.054091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622246+0000","last_change":"2023-10-06T03:10:43.609053+0000","last_active":"2023-10-06T03:10:44.622246+0000","last_peered":"2023-10-06T03:10:44.622246+0000","last_clean":"2023-10-06T03:10:44.622246+0000","last_became_active":"2023-10-06T03:10:43.608789+0000","last_became_peered":"2023-10-06T03:10:43.608789+0000","last_unstale":"2023-10-06T03:10:44.622246+0000","last_undegraded":"2023-10-06T03:10:44.622246+0000","last_fullsized":"2023-10-06T03:10:44.622246+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:36.389568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.604674+0000","last_change":"2023-10-06T03:10:44.602617+0000","last_active":"2023-10-06T03:10:44.604674+0000","last_peered":"2023-10-06T03:10:44.604674+0000","last_clean":"2023-10-06T03:10:44.604674+0000","last_became_active":"2023-10-06T03:10:44.602342+0000","last_became_peered":"2023-10-06T03:10:44.602342+0000","last_unstale":"2023-10-06T03:10:44.604674+0000","last_undegraded":"2023-10-06T03:10:44.604674+0000","last_fullsized":"2023-10-06T03:10:44.604674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.598698+0000","last_change":"2023-10-06T03:10:43.611124+0000","last_active":"2023-10-06T03:10:44.598698+0000","last_peered":"2023-10-06T03:10:44.598698+0000","last_clean":"2023-10-06T03:10:44.598698+0000","last_became_active":"2023-10-06T03:10:43.610877+0000","last_became_peered":"2023-10-06T03:10:43.610877+0000","last_unstale":"2023-10-06T03:10:44.598698+0000","last_undegraded":"2023-10-06T03:10:44.598698+0000","last_fullsized":"2023-10-06T03:10:44.598698+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:20:00.923932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.620606+0000","last_change":"2023-10-06T03:10:44.610074+0000","last_active":"2023-10-06T03:10:44.620606+0000","last_peered":"2023-10-06T03:10:44.620606+0000","last_clean":"2023-10-06T03:10:44.620606+0000","last_became_active":"2023-10-06T03:10:44.609795+0000","last_became_peered":"2023-10-06T03:10:44.609795+0000","last_unstale":"2023-10-06T03:10:44.620606+0000","last_undegraded":"2023-10-06T03:10:44.620606+0000","last_fullsized":"2023-10-06T03:10:44.620606+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:23.309599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618997+0000","last_change":"2023-10-06T03:10:43.607875+0000","last_active":"2023-10-06T03:10:44.618997+0000","last_peered":"2023-10-06T03:10:44.618997+0000","last_clean":"2023-10-06T03:10:44.618997+0000","last_became_active":"2023-10-06T03:10:43.607570+0000","last_became_peered":"2023-10-06T03:10:43.607570+0000","last_unstale":"2023-10-06T03:10:44.618997+0000","last_undegraded":"2023-10-06T03:10:44.618997+0000","last_fullsized":"2023-10-06T03:10:44.618997+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622108+0000","last_change":"2023-10-06T03:10:44.614535+0000","last_active":"2023-10-06T03:10:44.622108+0000","last_peered":"2023-10-06T03:10:44.622108+0000","last_clean":"2023-10-06T03:10:44.622108+0000","last_became_active":"2023-10-06T03:10:44.614151+0000","last_became_peered":"2023-10-06T03:10:44.614151+0000","last_unstale":"2023-10-06T03:10:44.622108+0000","last_undegraded":"2023-10-06T03:10:44.622108+0000","last_fullsized":"2023-10-06T03:10:44.622108+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:42.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595616+0000","last_change":"2023-10-06T03:10:43.611649+0000","last_active":"2023-10-06T03:10:44.595616+0000","last_peered":"2023-10-06T03:10:44.595616+0000","last_clean":"2023-10-06T03:10:44.595616+0000","last_became_active":"2023-10-06T03:10:43.611297+0000","last_became_peered":"2023-10-06T03:10:43.611297+0000","last_unstale":"2023-10-06T03:10:44.595616+0000","last_undegraded":"2023-10-06T03:10:44.595616+0000","last_fullsized":"2023-10-06T03:10:44.595616+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.884730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617340+0000","last_change":"2023-10-06T03:10:44.611469+0000","last_active":"2023-10-06T03:10:44.617340+0000","last_peered":"2023-10-06T03:10:44.617340+0000","last_clean":"2023-10-06T03:10:44.617340+0000","last_became_active":"2023-10-06T03:10:44.610791+0000","last_became_peered":"2023-10-06T03:10:44.610791+0000","last_unstale":"2023-10-06T03:10:44.617340+0000","last_undegraded":"2023-10-06T03:10:44.617340+0000","last_fullsized":"2023-10-06T03:10:44.617340+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:19.393529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602668+0000","last_change":"2023-10-06T03:10:43.610052+0000","last_active":"2023-10-06T03:10:44.602668+0000","last_peered":"2023-10-06T03:10:44.602668+0000","last_clean":"2023-10-06T03:10:44.602668+0000","last_became_active":"2023-10-06T03:10:43.609757+0000","last_became_peered":"2023-10-06T03:10:43.609757+0000","last_unstale":"2023-10-06T03:10:44.602668+0000","last_undegraded":"2023-10-06T03:10:44.602668+0000","last_fullsized":"2023-10-06T03:10:44.602668+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:20.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614717+0000","last_change":"2023-10-06T03:10:44.608535+0000","last_active":"2023-10-06T03:10:44.614717+0000","last_peered":"2023-10-06T03:10:44.614717+0000","last_clean":"2023-10-06T03:10:44.614717+0000","last_became_active":"2023-10-06T03:10:44.607437+0000","last_became_peered":"2023-10-06T03:10:44.607437+0000","last_unstale":"2023-10-06T03:10:44.614717+0000","last_undegraded":"2023-10-06T03:10:44.614717+0000","last_fullsized":"2023-10-06T03:10:44.614717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:07.998111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.596401+0000","last_change":"2023-10-06T03:10:43.609073+0000","last_active":"2023-10-06T03:10:44.596401+0000","last_peered":"2023-10-06T03:10:44.596401+0000","last_clean":"2023-10-06T03:10:44.596401+0000","last_became_active":"2023-10-06T03:10:43.608677+0000","last_became_peered":"2023-10-06T03:10:43.608677+0000","last_unstale":"2023-10-06T03:10:44.596401+0000","last_undegraded":"2023-10-06T03:10:44.596401+0000","last_fullsized":"2023-10-06T03:10:44.596401+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:09.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613017+0000","last_change":"2023-10-06T03:10:44.606162+0000","last_active":"2023-10-06T03:10:44.613017+0000","last_peered":"2023-10-06T03:10:44.613017+0000","last_clean":"2023-10-06T03:10:44.613017+0000","last_became_active":"2023-10-06T03:10:44.605602+0000","last_became_peered":"2023-10-06T03:10:44.605602+0000","last_unstale":"2023-10-06T03:10:44.613017+0000","last_undegraded":"2023-10-06T03:10:44.613017+0000","last_fullsized":"2023-10-06T03:10:44.613017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:54.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.629090+0000","last_change":"2023-10-06T03:10:43.603495+0000","last_active":"2023-10-06T03:10:44.629090+0000","last_peered":"2023-10-06T03:10:44.629090+0000","last_clean":"2023-10-06T03:10:44.629090+0000","last_became_active":"2023-10-06T03:10:43.603249+0000","last_became_peered":"2023-10-06T03:10:43.603249+0000","last_unstale":"2023-10-06T03:10:44.629090+0000","last_undegraded":"2023-10-06T03:10:44.629090+0000","last_fullsized":"2023-10-06T03:10:44.629090+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:49.332478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616794+0000","last_change":"2023-10-06T03:10:44.606893+0000","last_active":"2023-10-06T03:10:44.616794+0000","last_peered":"2023-10-06T03:10:44.616794+0000","last_clean":"2023-10-06T03:10:44.616794+0000","last_became_active":"2023-10-06T03:10:44.606341+0000","last_became_peered":"2023-10-06T03:10:44.606341+0000","last_unstale":"2023-10-06T03:10:44.616794+0000","last_undegraded":"2023-10-06T03:10:44.616794+0000","last_fullsized":"2023-10-06T03:10:44.616794+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:27.791364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"186'1","reported_seq":11,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-06T03:10:45.874675+0000","last_change":"2023-10-06T03:10:43.617649+0000","last_active":"2023-10-06T03:10:45.874675+0000","last_peered":"2023-10-06T03:10:45.874675+0000","last_clean":"2023-10-06T03:10:45.874675+0000","last_became_active":"2023-10-06T03:10:43.617403+0000","last_became_peered":"2023-10-06T03:10:43.617403+0000","last_unstale":"2023-10-06T03:10:45.874675+0000","last_undegraded":"2023-10-06T03:10:45.874675+0000","last_fullsized":"2023-10-06T03:10:45.874675+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:17.771738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612300+0000","last_change":"2023-10-06T03:10:44.610899+0000","last_active":"2023-10-06T03:10:44.612300+0000","last_peered":"2023-10-06T03:10:44.612300+0000","last_clean":"2023-10-06T03:10:44.612300+0000","last_became_active":"2023-10-06T03:10:44.610620+0000","last_became_peered":"2023-10-06T03:10:44.610620+0000","last_unstale":"2023-10-06T03:10:44.612300+0000","last_undegraded":"2023-10-06T03:10:44.612300+0000","last_fullsized":"2023-10-06T03:10:44.612300+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608467+0000","last_change":"2023-10-06T03:10:43.609223+0000","last_active":"2023-10-06T03:10:44.608467+0000","last_peered":"2023-10-06T03:10:44.608467+0000","last_clean":"2023-10-06T03:10:44.608467+0000","last_became_active":"2023-10-06T03:10:43.606876+0000","last_became_peered":"2023-10-06T03:10:43.606876+0000","last_unstale":"2023-10-06T03:10:44.608467+0000","last_undegraded":"2023-10-06T03:10:44.608467+0000","last_fullsized":"2023-10-06T03:10:44.608467+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:04.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611431+0000","last_change":"2023-10-06T03:10:44.605653+0000","last_active":"2023-10-06T03:10:44.611431+0000","last_peered":"2023-10-06T03:10:44.611431+0000","last_clean":"2023-10-06T03:10:44.611431+0000","last_became_active":"2023-10-06T03:10:44.605289+0000","last_became_peered":"2023-10-06T03:10:44.605289+0000","last_unstale":"2023-10-06T03:10:44.611431+0000","last_undegraded":"2023-10-06T03:10:44.611431+0000","last_fullsized":"2023-10-06T03:10:44.611431+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603121+0000","last_change":"2023-10-06T03:10:43.611147+0000","last_active":"2023-10-06T03:10:44.603121+0000","last_peered":"2023-10-06T03:10:44.603121+0000","last_clean":"2023-10-06T03:10:44.603121+0000","last_became_active":"2023-10-06T03:10:43.610865+0000","last_became_peered":"2023-10-06T03:10:43.610865+0000","last_unstale":"2023-10-06T03:10:44.603121+0000","last_undegraded":"2023-10-06T03:10:44.603121+0000","last_fullsized":"2023-10-06T03:10:44.603121+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625117+0000","last_change":"2023-10-06T03:10:44.608741+0000","last_active":"2023-10-06T03:10:44.625117+0000","last_peered":"2023-10-06T03:10:44.625117+0000","last_clean":"2023-10-06T03:10:44.625117+0000","last_became_active":"2023-10-06T03:10:44.608014+0000","last_became_peered":"2023-10-06T03:10:44.608014+0000","last_unstale":"2023-10-06T03:10:44.625117+0000","last_undegraded":"2023-10-06T03:10:44.625117+0000","last_fullsized":"2023-10-06T03:10:44.625117+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:52.576251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595192+0000","last_change":"2023-10-06T03:10:43.611002+0000","last_active":"2023-10-06T03:10:44.595192+0000","last_peered":"2023-10-06T03:10:44.595192+0000","last_clean":"2023-10-06T03:10:44.595192+0000","last_became_active":"2023-10-06T03:10:43.610692+0000","last_became_peered":"2023-10-06T03:10:43.610692+0000","last_unstale":"2023-10-06T03:10:44.595192+0000","last_undegraded":"2023-10-06T03:10:44.595192+0000","last_fullsized":"2023-10-06T03:10:44.595192+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:15.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624786+0000","last_change":"2023-10-06T03:10:44.607396+0000","last_active":"2023-10-06T03:10:44.624786+0000","last_peered":"2023-10-06T03:10:44.624786+0000","last_clean":"2023-10-06T03:10:44.624786+0000","last_became_active":"2023-10-06T03:10:44.607134+0000","last_became_peered":"2023-10-06T03:10:44.607134+0000","last_unstale":"2023-10-06T03:10:44.624786+0000","last_undegraded":"2023-10-06T03:10:44.624786+0000","last_fullsized":"2023-10-06T03:10:44.624786+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:11.054676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.594876+0000","last_change":"2023-10-06T03:10:43.607559+0000","last_active":"2023-10-06T03:10:44.594876+0000","last_peered":"2023-10-06T03:10:44.594876+0000","last_clean":"2023-10-06T03:10:44.594876+0000","last_became_active":"2023-10-06T03:10:43.607302+0000","last_became_peered":"2023-10-06T03:10:43.607302+0000","last_unstale":"2023-10-06T03:10:44.594876+0000","last_undegraded":"2023-10-06T03:10:44.594876+0000","last_fullsized":"2023-10-06T03:10:44.594876+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626242+0000","last_change":"2023-10-06T03:10:44.624362+0000","last_active":"2023-10-06T03:10:44.626242+0000","last_peered":"2023-10-06T03:10:44.626242+0000","last_clean":"2023-10-06T03:10:44.626242+0000","last_became_active":"2023-10-06T03:10:44.624101+0000","last_became_peered":"2023-10-06T03:10:44.624101+0000","last_unstale":"2023-10-06T03:10:44.626242+0000","last_undegraded":"2023-10-06T03:10:44.626242+0000","last_fullsized":"2023-10-06T03:10:44.626242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:05.039778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.599608+0000","last_change":"2023-10-06T03:10:43.605635+0000","last_active":"2023-10-06T03:10:44.599608+0000","last_peered":"2023-10-06T03:10:44.599608+0000","last_clean":"2023-10-06T03:10:44.599608+0000","last_became_active":"2023-10-06T03:10:43.605202+0000","last_became_peered":"2023-10-06T03:10:43.605202+0000","last_unstale":"2023-10-06T03:10:44.599608+0000","last_undegraded":"2023-10-06T03:10:44.599608+0000","last_fullsized":"2023-10-06T03:10:44.599608+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:32.594410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606406+0000","last_change":"2023-10-06T03:10:44.604594+0000","last_active":"2023-10-06T03:10:44.606406+0000","last_peered":"2023-10-06T03:10:44.606406+0000","last_clean":"2023-10-06T03:10:44.606406+0000","last_became_active":"2023-10-06T03:10:44.604267+0000","last_became_peered":"2023-10-06T03:10:44.604267+0000","last_unstale":"2023-10-06T03:10:44.606406+0000","last_undegraded":"2023-10-06T03:10:44.606406+0000","last_fullsized":"2023-10-06T03:10:44.606406+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.601579+0000","last_change":"2023-10-06T03:10:43.608350+0000","last_active":"2023-10-06T03:10:44.601579+0000","last_peered":"2023-10-06T03:10:44.601579+0000","last_clean":"2023-10-06T03:10:44.601579+0000","last_became_active":"2023-10-06T03:10:43.608121+0000","last_became_peered":"2023-10-06T03:10:43.608121+0000","last_unstale":"2023-10-06T03:10:44.601579+0000","last_undegraded":"2023-10-06T03:10:44.601579+0000","last_fullsized":"2023-10-06T03:10:44.601579+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:14.031939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626216+0000","last_change":"2023-10-06T03:10:44.613649+0000","last_active":"2023-10-06T03:10:44.626216+0000","last_peered":"2023-10-06T03:10:44.626216+0000","last_clean":"2023-10-06T03:10:44.626216+0000","last_became_active":"2023-10-06T03:10:44.613383+0000","last_became_peered":"2023-10-06T03:10:44.613383+0000","last_unstale":"2023-10-06T03:10:44.626216+0000","last_undegraded":"2023-10-06T03:10:44.626216+0000","last_fullsized":"2023-10-06T03:10:44.626216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:01.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":147,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.604838+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:44.604838+0000","last_peered":"2023-10-06T03:10:44.604838+0000","last_clean":"2023-10-06T03:10:44.604838+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:44.604838+0000","last_undegraded":"2023-10-06T03:10:44.604838+0000","last_fullsized":"2023-10-06T03:10:44.604838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615564+0000","last_change":"2023-10-06T03:10:44.613667+0000","last_active":"2023-10-06T03:10:44.615564+0000","last_peered":"2023-10-06T03:10:44.615564+0000","last_clean":"2023-10-06T03:10:44.615564+0000","last_became_active":"2023-10-06T03:10:44.613322+0000","last_became_peered":"2023-10-06T03:10:44.613322+0000","last_unstale":"2023-10-06T03:10:44.615564+0000","last_undegraded":"2023-10-06T03:10:44.615564+0000","last_fullsized":"2023-10-06T03:10:44.615564+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:01.094720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597815+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:44.597815+0000","last_peered":"2023-10-06T03:10:44.597815+0000","last_clean":"2023-10-06T03:10:44.597815+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:44.597815+0000","last_undegraded":"2023-10-06T03:10:44.597815+0000","last_fullsized":"2023-10-06T03:10:44.597815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618962+0000","last_change":"2023-10-06T03:10:44.615738+0000","last_active":"2023-10-06T03:10:44.618962+0000","last_peered":"2023-10-06T03:10:44.618962+0000","last_clean":"2023-10-06T03:10:44.618962+0000","last_became_active":"2023-10-06T03:10:44.615457+0000","last_became_peered":"2023-10-06T03:10:44.615457+0000","last_unstale":"2023-10-06T03:10:44.618962+0000","last_undegraded":"2023-10-06T03:10:44.618962+0000","last_fullsized":"2023-10-06T03:10:44.618962+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":150,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603180+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:44.603180+0000","last_peered":"2023-10-06T03:10:44.603180+0000","last_clean":"2023-10-06T03:10:44.603180+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:44.603180+0000","last_undegraded":"2023-10-06T03:10:44.603180+0000","last_fullsized":"2023-10-06T03:10:44.603180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616994+0000","last_change":"2023-10-06T03:10:44.610773+0000","last_active":"2023-10-06T03:10:44.616994+0000","last_peered":"2023-10-06T03:10:44.616994+0000","last_clean":"2023-10-06T03:10:44.616994+0000","last_became_active":"2023-10-06T03:10:44.610515+0000","last_became_peered":"2023-10-06T03:10:44.610515+0000","last_unstale":"2023-10-06T03:10:44.616994+0000","last_undegraded":"2023-10-06T03:10:44.616994+0000","last_fullsized":"2023-10-06T03:10:44.616994+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:37.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602214+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:44.602214+0000","last_peered":"2023-10-06T03:10:44.602214+0000","last_clean":"2023-10-06T03:10:44.602214+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:44.602214+0000","last_undegraded":"2023-10-06T03:10:44.602214+0000","last_fullsized":"2023-10-06T03:10:44.602214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615899+0000","last_change":"2023-10-06T03:10:44.614523+0000","last_active":"2023-10-06T03:10:44.615899+0000","last_peered":"2023-10-06T03:10:44.615899+0000","last_clean":"2023-10-06T03:10:44.615899+0000","last_became_active":"2023-10-06T03:10:44.614265+0000","last_became_peered":"2023-10-06T03:10:44.614265+0000","last_unstale":"2023-10-06T03:10:44.615899+0000","last_undegraded":"2023-10-06T03:10:44.615899+0000","last_fullsized":"2023-10-06T03:10:44.615899+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:32.111948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":303,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602337+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:44.602337+0000","last_peered":"2023-10-06T03:10:44.602337+0000","last_clean":"2023-10-06T03:10:44.602337+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:44.602337+0000","last_undegraded":"2023-10-06T03:10:44.602337+0000","last_fullsized":"2023-10-06T03:10:44.602337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597322+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:44.597322+0000","last_peered":"2023-10-06T03:10:44.597322+0000","last_clean":"2023-10-06T03:10:44.597322+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:44.597322+0000","last_undegraded":"2023-10-06T03:10:44.597322+0000","last_fullsized":"2023-10-06T03:10:44.597322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625133+0000","last_change":"2023-10-06T03:10:44.608933+0000","last_active":"2023-10-06T03:10:44.625133+0000","last_peered":"2023-10-06T03:10:44.625133+0000","last_clean":"2023-10-06T03:10:44.625133+0000","last_became_active":"2023-10-06T03:10:44.608337+0000","last_became_peered":"2023-10-06T03:10:44.608337+0000","last_unstale":"2023-10-06T03:10:44.625133+0000","last_undegraded":"2023-10-06T03:10:44.625133+0000","last_fullsized":"2023-10-06T03:10:44.625133+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:12.581813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.599681+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:44.599681+0000","last_peered":"2023-10-06T03:10:44.599681+0000","last_clean":"2023-10-06T03:10:44.599681+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:44.599681+0000","last_undegraded":"2023-10-06T03:10:44.599681+0000","last_fullsized":"2023-10-06T03:10:44.599681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611631+0000","last_change":"2023-10-06T03:10:44.610553+0000","last_active":"2023-10-06T03:10:44.611631+0000","last_peered":"2023-10-06T03:10:44.611631+0000","last_clean":"2023-10-06T03:10:44.611631+0000","last_became_active":"2023-10-06T03:10:44.610247+0000","last_became_peered":"2023-10-06T03:10:44.610247+0000","last_unstale":"2023-10-06T03:10:44.611631+0000","last_undegraded":"2023-10-06T03:10:44.611631+0000","last_fullsized":"2023-10-06T03:10:44.611631+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595601+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:44.595601+0000","last_peered":"2023-10-06T03:10:44.595601+0000","last_clean":"2023-10-06T03:10:44.595601+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:44.595601+0000","last_undegraded":"2023-10-06T03:10:44.595601+0000","last_fullsized":"2023-10-06T03:10:44.595601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626189+0000","last_change":"2023-10-06T03:10:44.623258+0000","last_active":"2023-10-06T03:10:44.626189+0000","last_peered":"2023-10-06T03:10:44.626189+0000","last_clean":"2023-10-06T03:10:44.626189+0000","last_became_active":"2023-10-06T03:10:44.622785+0000","last_became_peered":"2023-10-06T03:10:44.622785+0000","last_unstale":"2023-10-06T03:10:44.626189+0000","last_undegraded":"2023-10-06T03:10:44.626189+0000","last_fullsized":"2023-10-06T03:10:44.626189+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:48.634468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.598553+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:44.598553+0000","last_peered":"2023-10-06T03:10:44.598553+0000","last_clean":"2023-10-06T03:10:44.598553+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:44.598553+0000","last_undegraded":"2023-10-06T03:10:44.598553+0000","last_fullsized":"2023-10-06T03:10:44.598553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304276,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067564,"statfs":{"total":96636764160,"available":96325185536,"internally_reserved":0,"allocated":2961408,"data_stored":1714396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.745,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64100000000000001,"5min":1.264,"15min":1.264},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":2.286,"15min":2.286},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":2.504,"15min":2.504},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":1.141,"15min":1.141},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":1.639,"15min":1.639},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82199999999999995,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.307,"15min":0.307},"max":{"1min":0.77800000000000002,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.376}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303700,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068140,"statfs":{"total":96636764160,"available":96325775360,"internally_reserved":0,"allocated":2371584,"data_stored":1124070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303708,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068132,"statfs":{"total":96636764160,"available":96325767168,"internally_reserved":0,"allocated":2379776,"data_stored":1124706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.27600000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.505},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.54200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2957312,"data_stored":1714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.25}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303692,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068148,"statfs":{"total":96636764160,"available":96325783552,"internally_reserved":0,"allocated":2363392,"data_stored":1123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.317},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.33400000000000002,"15min":0.32100000000000001},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.374,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33100000000000002,"15min":0.33000000000000002},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.504,"15min":0.56599999999999995},"last":0.25700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.435,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.35899999999999999,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.86299999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.374,"15min":0.374},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.442},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.903},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303708,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068132,"statfs":{"total":96636764160,"available":96325767168,"internally_reserved":0,"allocated":2379776,"data_stored":1126048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.436,"5min":0.503,"15min":0.65200000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.49299999999999999,"15min":0.72299999999999998},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.313},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.59599999999999997,"15min":0.69699999999999995},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.47399999999999998,"5min":0.63,"15min":0.67600000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78800000000000003,"5min":0.79700000000000004,"15min":0.94399999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.44900000000000001,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.997},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.98499999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":1.0660000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303712,"kb_used_data":2328,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068128,"statfs":{"total":96636764160,"available":96325763072,"internally_reserved":0,"allocated":2383872,"data_stored":1125550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.37},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.625},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.434},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.34799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303692,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068148,"statfs":{"total":96636764160,"available":96325783552,"internally_reserved":0,"allocated":2363392,"data_stored":1123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":1.143,"15min":2.3599999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.59199999999999997},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.28100000000000003},"max":{"1min":0.73199999999999998,"5min":0.86299999999999999,"15min":2.371},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.82499999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":6.532,"15min":6.532},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.67600000000000005},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72899999999999998,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.65900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":5.524,"15min":5.524},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64100000000000001,"5min":0.78200000000000003,"15min":1.1990000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.44900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.052},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.175},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.145},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:49.414 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T03:10:49.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.559+0000 7f351e2ea700 1 Processor -- start 2023-10-06T03:10:49.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.559+0000 7f351e2ea700 1 -- start start 2023-10-06T03:10:49.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351e2ea700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35180ccc40 0x7f35180ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351e2ea700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180cd010 0x7f35180acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351e2ea700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f35181564a0 0x7f35181568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351e2ea700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180ad9d0 con 0x7f35180ccc40 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351e2ea700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180aed00 con 0x7f35181564a0 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351e2ea700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3518153f10 con 0x7f35180cd010 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f3517fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35180ccc40 0x7f35180ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f3517fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f35180ccc40 0x7f35180ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37562/0 (socket says 172.21.15.92:37562) 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f3517fff700 1 -- 172.21.15.92:0/3370956496 learned_addr learned my addr 172.21.15.92:0/3370956496 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:49.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f351c887700 1 --2- 172.21.15.92:0/3370956496 >> v2:172.21.15.136:3300/0 conn(0x7f35181564a0 0x7f35181568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.560+0000 7f35177fe700 1 --2- 172.21.15.92:0/3370956496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180cd010 0x7f35180acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3517fff700 1 -- 172.21.15.92:0/3370956496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180cd010 msgr2=0x7f35180acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:49.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3517fff700 1 --2- 172.21.15.92:0/3370956496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180cd010 0x7f35180acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3517fff700 1 -- 172.21.15.92:0/3370956496 >> v2:172.21.15.136:3300/0 conn(0x7f35181564a0 msgr2=0x7f35181568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:49.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3517fff700 1 --2- 172.21.15.92:0/3370956496 >> v2:172.21.15.136:3300/0 conn(0x7f35181564a0 0x7f35181568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3517fff700 1 -- 172.21.15.92:0/3370956496 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180af800 con 0x7f35180ccc40 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3517fff700 1 --2- 172.21.15.92:0/3370956496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180ccc40 0x7f35180ac800 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f35180cd480 tx=0x7f3508009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7ededffcb9d027 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3516ffd700 1 -- 172.21.15.92:0/3370956496 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800b860 con 0x7f35180ccc40 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3516ffd700 1 -- 172.21.15.92:0/3370956496 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800b9c0 con 0x7f35180ccc40 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f3516ffd700 1 -- 172.21.15.92:0/3370956496 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800bd60 con 0x7f35180ccc40 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 -- 172.21.15.92:0/3370956496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180ccc40 msgr2=0x7f35180ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 --2- 172.21.15.92:0/3370956496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180ccc40 0x7f35180ac800 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f35180cd480 tx=0x7f3508009250 comp rx=0 tx=0).stop 2023-10-06T03:10:49.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 -- 172.21.15.92:0/3370956496 shutdown_connections 2023-10-06T03:10:49.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 --2- 172.21.15.92:0/3370956496 >> v2:172.21.15.136:3300/0 conn(0x7f35181564a0 0x7f35181568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 --2- 172.21.15.92:0/3370956496 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180ccc40 0x7f35180ac800 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 --2- 172.21.15.92:0/3370956496 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180cd010 0x7f35180acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 -- 172.21.15.92:0/3370956496 >> 172.21.15.92:0/3370956496 conn(0x7f351806a310 msgr2=0x7f35180cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:49.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 -- 172.21.15.92:0/3370956496 shutdown_connections 2023-10-06T03:10:49.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.561+0000 7f351e2ea700 1 -- 172.21.15.92:0/3370956496 wait complete. 2023-10-06T03:10:49.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.562+0000 7f351e2ea700 1 Processor -- start 2023-10-06T03:10:49.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 -- start start 2023-10-06T03:10:49.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180ccc40 0x7f35181523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 0x7f35180ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35181564a0 0x7f35180cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351815ad50 con 0x7f35180cd010 2023-10-06T03:10:49.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351816a160 con 0x7f35181564a0 2023-10-06T03:10:49.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f351e2ea700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351816a440 con 0x7f35180ccc40 2023-10-06T03:10:49.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.563+0000 7f35177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 0x7f35180ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 0x7f35180ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37564/0 (socket says 172.21.15.92:37564) 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 -- 172.21.15.92:0/4195886035 learned_addr learned my addr 172.21.15.92:0/4195886035 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f3517fff700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180ccc40 0x7f35181523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f351c887700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35181564a0 0x7f35180cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 -- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180ccc40 msgr2=0x7f35181523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180ccc40 0x7f35181523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 -- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35181564a0 msgr2=0x7f35180cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35181564a0 0x7f35180cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 -- 172.21.15.92:0/4195886035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080094f0 con 0x7f35180cd010 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f35177fe700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 0x7f35180ca8e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f350c00fe20 tx=0x7f350c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00de10 con 0x7f35180cd010 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c007b00 con 0x7f35180cd010 2023-10-06T03:10:49.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351816a720 con 0x7f35180cd010 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c01a460 con 0x7f35180cd010 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.564+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3518152920 con 0x7f35180cd010 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.565+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351815aec0 con 0x7f35180cd010 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.565+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f350c01a7d0 con 0x7f35180cd010 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.566+0000 7f34fffff700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3504067320 0x7f35040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.566+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f350c087670 con 0x7f35180cd010 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.566+0000 7f3517fff700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3504067320 0x7f35040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.566+0000 7f3517fff700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3504067320 0x7f35040697d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f35180cd480 tx=0x7f3508017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.573 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.570+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f350c054090 con 0x7f35180cd010 2023-10-06T03:10:49.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.835+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f351804c3c0 con 0x7f35180cd010 2023-10-06T03:10:49.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.836+0000 7f34fffff700 1 -- 172.21.15.92:0/4195886035 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1441 (secure 0 0 0) 0x7f350c0576b0 con 0x7f35180cd010 2023-10-06T03:10:49.838 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:49.838 INFO:teuthology.orchestra.run.smithi092.stdout:{"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":572,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":186,"num_osds":8,"num_up_osds":8,"osd_up_since":1696561288,"num_in_osds":8,"osd_in_since":1696561280,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592674,"bytes_used":2489098240,"bytes_avail":770605015040,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":125,"id":15,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":15,"rank":0,"name":"a","status":"up:active","gid":8802}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-10-06T03:10:25.289642+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T03:10:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3504067320 msgr2=0x7f35040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3504067320 0x7f35040697d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f35180cd480 tx=0x7f3508017000 comp rx=0 tx=0).stop 2023-10-06T03:10:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 msgr2=0x7f35180ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 0x7f35180ca8e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f350c00fe20 tx=0x7f350c008040 comp rx=0 tx=0).stop 2023-10-06T03:10:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 shutdown_connections 2023-10-06T03:10:49.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3504067320 0x7f35040697d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35181564a0 0x7f35180cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f35180ccc40 0x7f35181523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 --2- 172.21.15.92:0/4195886035 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35180cd010 0x7f35180ca8e0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 >> 172.21.15.92:0/4195886035 conn(0x7f351806a310 msgr2=0x7f35181568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:49.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 shutdown_connections 2023-10-06T03:10:49.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.838+0000 7f351e2ea700 1 -- 172.21.15.92:0/4195886035 wait complete. 2023-10-06T03:10:49.851 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T03:10:49.852 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.964+0000 7f6d6ef29700 1 Processor -- start 2023-10-06T03:10:49.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.964+0000 7f6d6ef29700 1 -- start start 2023-10-06T03:10:49.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ef29700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d68068550 0x7f6d68068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ef29700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6d6815e010 0x7f6d6806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ef29700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6806ec70 con 0x7f6d6806f640 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ef29700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d680b3430 con 0x7f6d68068550 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ef29700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d680b2e70 con 0x7f6d6815e010 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ccc5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d68068550 0x7f6d68068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ccc5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6d68068550 0x7f6d68068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37566/0 (socket says 172.21.15.92:37566) 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.965+0000 7f6d6ccc5700 1 -- 172.21.15.92:0/2054563085 learned_addr learned my addr 172.21.15.92:0/2054563085 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d5ffff700 1 --2- 172.21.15.92:0/2054563085 >> v2:172.21.15.136:3300/0 conn(0x7f6d6815e010 0x7f6d6806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ccc5700 1 -- 172.21.15.92:0/2054563085 >> v2:172.21.15.136:3300/0 conn(0x7f6d6815e010 msgr2=0x7f6d6806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:49.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ccc5700 1 --2- 172.21.15.92:0/2054563085 >> v2:172.21.15.136:3300/0 conn(0x7f6d6815e010 0x7f6d6806f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ccc5700 1 -- 172.21.15.92:0/2054563085 >> v1:172.21.15.136:6789/0 conn(0x7f6d6806f640 legacy=0x7f6d6806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ccc5700 1 -- 172.21.15.92:0/2054563085 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680b3150 con 0x7f6d68068550 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ccc5700 1 --2- 172.21.15.92:0/2054563085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68068550 0x7f6d68068920 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6d6806d310 tx=0x7f6d64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3889c5fed82577 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d5f7fe700 1 -- 172.21.15.92:0/2054563085 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400b820 con 0x7f6d68068550 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d5f7fe700 1 -- 172.21.15.92:0/2054563085 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400b980 con 0x7f6d68068550 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d5f7fe700 1 -- 172.21.15.92:0/2054563085 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400bd20 con 0x7f6d68068550 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2054563085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68068550 msgr2=0x7f6d68068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.966+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2054563085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68068550 0x7f6d68068920 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f6d6806d310 tx=0x7f6d64009250 comp rx=0 tx=0).stop 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.967+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2054563085 shutdown_connections 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.967+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2054563085 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d68068550 0x7f6d68068920 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.967+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2054563085 >> v2:172.21.15.136:3300/0 conn(0x7f6d6815e010 0x7f6d6806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.967+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2054563085 >> 172.21.15.92:0/2054563085 conn(0x7f6d680cd480 msgr2=0x7f6d680cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.967+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2054563085 shutdown_connections 2023-10-06T03:10:49.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.967+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2054563085 wait complete. 2023-10-06T03:10:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.968+0000 7f6d6ef29700 1 Processor -- start 2023-10-06T03:10:49.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.968+0000 7f6d6ef29700 1 -- start start 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d68068550 0x7f6d6815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 0x7f6d68152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d6815e010 0x7f6d680af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6815aa30 con 0x7f6d6806f640 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d68159e20 con 0x7f6d68068550 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6815a100 con 0x7f6d6815e010 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 0x7f6d68152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 0x7f6d68152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37578/0 (socket says 172.21.15.92:37578) 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 -- 172.21.15.92:0/2695936407 learned_addr learned my addr 172.21.15.92:0/2695936407 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:49.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ccc5700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d68068550 0x7f6d6815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6d4c6700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d6815e010 0x7f6d680af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 -- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d6815e010 msgr2=0x7f6d680af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d6815e010 0x7f6d680af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 -- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d68068550 msgr2=0x7f6d6815aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d68068550 0x7f6d6815aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 -- 172.21.15.92:0/2695936407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640094f0 con 0x7f6d6806f640 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6d4c6700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d6815e010 0x7f6d680af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5ffff700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 0x7f6d68152720 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800d860 tx=0x7f6d58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d580125f0 con 0x7f6d6806f640 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58012770 con 0x7f6d6806f640 2023-10-06T03:10:49.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6815a3e0 con 0x7f6d6806f640 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58012b30 con 0x7f6d6806f640 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.969+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6815b400 con 0x7f6d6806f640 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.970+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6815abe0 con 0x7f6d6806f640 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.971+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6d5801e460 con 0x7f6d6806f640 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.971+0000 7f6d5cff9700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d54067320 0x7f6d540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.971+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d58085420 con 0x7f6d6806f640 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.971+0000 7f6d6ccc5700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d54067320 0x7f6d540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.971+0000 7f6d6ccc5700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d54067320 0x7f6d540697d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6d6806d310 tx=0x7f6d64016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:49.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:49.975+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d6815abe0 con 0x7f6d6806f640 2023-10-06T03:10:50.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.153+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d6804c3c0 con 0x7f6d54067320 2023-10-06T03:10:50.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.161+0000 7f6d5cff9700 1 -- 172.21.15.92:0/2695936407 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311825 (secure 0 0 0) 0x7f6d6804c3c0 con 0x7f6d54067320 2023-10-06T03:10:50.166 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d54067320 msgr2=0x7f6d540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d54067320 0x7f6d540697d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6d6806d310 tx=0x7f6d64016000 comp rx=0 tx=0).stop 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 msgr2=0x7f6d68152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 0x7f6d68152720 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800d860 tx=0x7f6d58008040 comp rx=0 tx=0).stop 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 shutdown_connections 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6d54067320 0x7f6d540697d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d6815e010 0x7f6d680af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d68068550 0x7f6d6815aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:50.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 --2- 172.21.15.92:0/2695936407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6d6806f640 0x7f6d68152720 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:50.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 >> 172.21.15.92:0/2695936407 conn(0x7f6d680cd480 msgr2=0x7f6d68156d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:50.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 shutdown_connections 2023-10-06T03:10:50.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:50.165+0000 7f6d6ef29700 1 -- 172.21.15.92:0/2695936407 wait complete. 2023-10-06T03:10:50.170 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:50.178 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2023-10-06T03:10:49.293882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430760,"kb_used_data":19688,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752543960,"statfs":{"total":773094113280,"available":770605015040,"internally_reserved":0,"allocated":20160512,"data_stored":10177012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001246"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611079+0000","last_change":"2023-10-06T03:10:44.608143+0000","last_active":"2023-10-06T03:10:44.611079+0000","last_peered":"2023-10-06T03:10:44.611079+0000","last_clean":"2023-10-06T03:10:44.611079+0000","last_became_active":"2023-10-06T03:10:44.607875+0000","last_became_peered":"2023-10-06T03:10:44.607875+0000","last_unstale":"2023-10-06T03:10:44.611079+0000","last_undegraded":"2023-10-06T03:10:44.611079+0000","last_fullsized":"2023-10-06T03:10:44.611079+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:15.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617809+0000","last_change":"2023-10-06T03:10:44.616908+0000","last_active":"2023-10-06T03:10:44.617809+0000","last_peered":"2023-10-06T03:10:44.617809+0000","last_clean":"2023-10-06T03:10:44.617809+0000","last_became_active":"2023-10-06T03:10:44.616555+0000","last_became_peered":"2023-10-06T03:10:44.616555+0000","last_unstale":"2023-10-06T03:10:44.617809+0000","last_undegraded":"2023-10-06T03:10:44.617809+0000","last_fullsized":"2023-10-06T03:10:44.617809+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:23.309599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611589+0000","last_change":"2023-10-06T03:10:44.605922+0000","last_active":"2023-10-06T03:10:44.611589+0000","last_peered":"2023-10-06T03:10:44.611589+0000","last_clean":"2023-10-06T03:10:44.611589+0000","last_became_active":"2023-10-06T03:10:44.605623+0000","last_became_peered":"2023-10-06T03:10:44.605623+0000","last_unstale":"2023-10-06T03:10:44.611589+0000","last_undegraded":"2023-10-06T03:10:44.611589+0000","last_fullsized":"2023-10-06T03:10:44.611589+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612282+0000","last_change":"2023-10-06T03:10:44.610178+0000","last_active":"2023-10-06T03:10:44.612282+0000","last_peered":"2023-10-06T03:10:44.612282+0000","last_clean":"2023-10-06T03:10:44.612282+0000","last_became_active":"2023-10-06T03:10:44.609892+0000","last_became_peered":"2023-10-06T03:10:44.609892+0000","last_unstale":"2023-10-06T03:10:44.612282+0000","last_undegraded":"2023-10-06T03:10:44.612282+0000","last_fullsized":"2023-10-06T03:10:44.612282+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612326+0000","last_change":"2023-10-06T03:10:44.608872+0000","last_active":"2023-10-06T03:10:44.612326+0000","last_peered":"2023-10-06T03:10:44.612326+0000","last_clean":"2023-10-06T03:10:44.612326+0000","last_became_active":"2023-10-06T03:10:44.608412+0000","last_became_peered":"2023-10-06T03:10:44.608412+0000","last_unstale":"2023-10-06T03:10:44.612326+0000","last_undegraded":"2023-10-06T03:10:44.612326+0000","last_fullsized":"2023-10-06T03:10:44.612326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:42.363268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.609968+0000","last_change":"2023-10-06T03:10:44.605533+0000","last_active":"2023-10-06T03:10:44.609968+0000","last_peered":"2023-10-06T03:10:44.609968+0000","last_clean":"2023-10-06T03:10:44.609968+0000","last_became_active":"2023-10-06T03:10:44.605254+0000","last_became_peered":"2023-10-06T03:10:44.605254+0000","last_unstale":"2023-10-06T03:10:44.609968+0000","last_undegraded":"2023-10-06T03:10:44.609968+0000","last_fullsized":"2023-10-06T03:10:44.609968+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:22.407090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614874+0000","last_change":"2023-10-06T03:10:44.613032+0000","last_active":"2023-10-06T03:10:44.614874+0000","last_peered":"2023-10-06T03:10:44.614874+0000","last_clean":"2023-10-06T03:10:44.614874+0000","last_became_active":"2023-10-06T03:10:44.612535+0000","last_became_peered":"2023-10-06T03:10:44.612535+0000","last_unstale":"2023-10-06T03:10:44.614874+0000","last_undegraded":"2023-10-06T03:10:44.614874+0000","last_fullsized":"2023-10-06T03:10:44.614874+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:27.693399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619444+0000","last_change":"2023-10-06T03:10:44.612966+0000","last_active":"2023-10-06T03:10:44.619444+0000","last_peered":"2023-10-06T03:10:44.619444+0000","last_clean":"2023-10-06T03:10:44.619444+0000","last_became_active":"2023-10-06T03:10:44.612484+0000","last_became_peered":"2023-10-06T03:10:44.612484+0000","last_unstale":"2023-10-06T03:10:44.619444+0000","last_undegraded":"2023-10-06T03:10:44.619444+0000","last_fullsized":"2023-10-06T03:10:44.619444+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:32.111948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615003+0000","last_change":"2023-10-06T03:10:44.609587+0000","last_active":"2023-10-06T03:10:44.615003+0000","last_peered":"2023-10-06T03:10:44.615003+0000","last_clean":"2023-10-06T03:10:44.615003+0000","last_became_active":"2023-10-06T03:10:44.609390+0000","last_became_peered":"2023-10-06T03:10:44.609390+0000","last_unstale":"2023-10-06T03:10:44.615003+0000","last_undegraded":"2023-10-06T03:10:44.615003+0000","last_fullsized":"2023-10-06T03:10:44.615003+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:33.614656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618712+0000","last_change":"2023-10-06T03:10:44.618093+0000","last_active":"2023-10-06T03:10:44.618712+0000","last_peered":"2023-10-06T03:10:44.618712+0000","last_clean":"2023-10-06T03:10:44.618712+0000","last_became_active":"2023-10-06T03:10:44.617856+0000","last_became_peered":"2023-10-06T03:10:44.617856+0000","last_unstale":"2023-10-06T03:10:44.618712+0000","last_undegraded":"2023-10-06T03:10:44.618712+0000","last_fullsized":"2023-10-06T03:10:44.618712+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.931574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.623816+0000","last_change":"2023-10-06T03:10:44.623316+0000","last_active":"2023-10-06T03:10:44.623816+0000","last_peered":"2023-10-06T03:10:44.623816+0000","last_clean":"2023-10-06T03:10:44.623816+0000","last_became_active":"2023-10-06T03:10:44.623058+0000","last_became_peered":"2023-10-06T03:10:44.623058+0000","last_unstale":"2023-10-06T03:10:44.623816+0000","last_undegraded":"2023-10-06T03:10:44.623816+0000","last_fullsized":"2023-10-06T03:10:44.623816+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:37.745837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606653+0000","last_change":"2023-10-06T03:10:44.604123+0000","last_active":"2023-10-06T03:10:44.606653+0000","last_peered":"2023-10-06T03:10:44.606653+0000","last_clean":"2023-10-06T03:10:44.606653+0000","last_became_active":"2023-10-06T03:10:44.603822+0000","last_became_peered":"2023-10-06T03:10:44.603822+0000","last_unstale":"2023-10-06T03:10:44.606653+0000","last_undegraded":"2023-10-06T03:10:44.606653+0000","last_fullsized":"2023-10-06T03:10:44.606653+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:50.256975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624806+0000","last_change":"2023-10-06T03:10:44.612705+0000","last_active":"2023-10-06T03:10:44.624806+0000","last_peered":"2023-10-06T03:10:44.624806+0000","last_clean":"2023-10-06T03:10:44.624806+0000","last_became_active":"2023-10-06T03:10:44.612459+0000","last_became_peered":"2023-10-06T03:10:44.612459+0000","last_unstale":"2023-10-06T03:10:44.624806+0000","last_undegraded":"2023-10-06T03:10:44.624806+0000","last_fullsized":"2023-10-06T03:10:44.624806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:31.032848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625100+0000","last_change":"2023-10-06T03:10:44.608588+0000","last_active":"2023-10-06T03:10:44.625100+0000","last_peered":"2023-10-06T03:10:44.625100+0000","last_clean":"2023-10-06T03:10:44.625100+0000","last_became_active":"2023-10-06T03:10:44.607707+0000","last_became_peered":"2023-10-06T03:10:44.607707+0000","last_unstale":"2023-10-06T03:10:44.625100+0000","last_undegraded":"2023-10-06T03:10:44.625100+0000","last_fullsized":"2023-10-06T03:10:44.625100+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:11.796662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624888+0000","last_change":"2023-10-06T03:10:44.612883+0000","last_active":"2023-10-06T03:10:44.624888+0000","last_peered":"2023-10-06T03:10:44.624888+0000","last_clean":"2023-10-06T03:10:44.624888+0000","last_became_active":"2023-10-06T03:10:44.612648+0000","last_became_peered":"2023-10-06T03:10:44.612648+0000","last_unstale":"2023-10-06T03:10:44.624888+0000","last_undegraded":"2023-10-06T03:10:44.624888+0000","last_fullsized":"2023-10-06T03:10:44.624888+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:21.015787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622176+0000","last_change":"2023-10-06T03:10:44.621118+0000","last_active":"2023-10-06T03:10:44.622176+0000","last_peered":"2023-10-06T03:10:44.622176+0000","last_clean":"2023-10-06T03:10:44.622176+0000","last_became_active":"2023-10-06T03:10:44.614783+0000","last_became_peered":"2023-10-06T03:10:44.614783+0000","last_unstale":"2023-10-06T03:10:44.622176+0000","last_undegraded":"2023-10-06T03:10:44.622176+0000","last_fullsized":"2023-10-06T03:10:44.622176+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:04.349135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616832+0000","last_change":"2023-10-06T03:10:44.607117+0000","last_active":"2023-10-06T03:10:44.616832+0000","last_peered":"2023-10-06T03:10:44.616832+0000","last_clean":"2023-10-06T03:10:44.616832+0000","last_became_active":"2023-10-06T03:10:44.606719+0000","last_became_peered":"2023-10-06T03:10:44.606719+0000","last_unstale":"2023-10-06T03:10:44.616832+0000","last_undegraded":"2023-10-06T03:10:44.616832+0000","last_fullsized":"2023-10-06T03:10:44.616832+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:20.569599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614899+0000","last_change":"2023-10-06T03:10:44.613168+0000","last_active":"2023-10-06T03:10:44.614899+0000","last_peered":"2023-10-06T03:10:44.614899+0000","last_clean":"2023-10-06T03:10:44.614899+0000","last_became_active":"2023-10-06T03:10:44.612843+0000","last_became_peered":"2023-10-06T03:10:44.612843+0000","last_unstale":"2023-10-06T03:10:44.614899+0000","last_undegraded":"2023-10-06T03:10:44.614899+0000","last_fullsized":"2023-10-06T03:10:44.614899+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:37.217250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619764+0000","last_change":"2023-10-06T03:10:44.619199+0000","last_active":"2023-10-06T03:10:44.619764+0000","last_peered":"2023-10-06T03:10:44.619764+0000","last_clean":"2023-10-06T03:10:44.619764+0000","last_became_active":"2023-10-06T03:10:44.618882+0000","last_became_peered":"2023-10-06T03:10:44.618882+0000","last_unstale":"2023-10-06T03:10:44.619764+0000","last_undegraded":"2023-10-06T03:10:44.619764+0000","last_fullsized":"2023-10-06T03:10:44.619764+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:15.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622081+0000","last_change":"2023-10-06T03:10:44.613713+0000","last_active":"2023-10-06T03:10:44.622081+0000","last_peered":"2023-10-06T03:10:44.622081+0000","last_clean":"2023-10-06T03:10:44.622081+0000","last_became_active":"2023-10-06T03:10:44.610828+0000","last_became_peered":"2023-10-06T03:10:44.610828+0000","last_unstale":"2023-10-06T03:10:44.622081+0000","last_undegraded":"2023-10-06T03:10:44.622081+0000","last_fullsized":"2023-10-06T03:10:44.622081+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:23.629899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615084+0000","last_change":"2023-10-06T03:10:44.612712+0000","last_active":"2023-10-06T03:10:44.615084+0000","last_peered":"2023-10-06T03:10:44.615084+0000","last_clean":"2023-10-06T03:10:44.615084+0000","last_became_active":"2023-10-06T03:10:44.612456+0000","last_became_peered":"2023-10-06T03:10:44.612456+0000","last_unstale":"2023-10-06T03:10:44.615084+0000","last_undegraded":"2023-10-06T03:10:44.615084+0000","last_fullsized":"2023-10-06T03:10:44.615084+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:19.393529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608598+0000","last_change":"2023-10-06T03:10:44.606764+0000","last_active":"2023-10-06T03:10:44.608598+0000","last_peered":"2023-10-06T03:10:44.608598+0000","last_clean":"2023-10-06T03:10:44.608598+0000","last_became_active":"2023-10-06T03:10:44.606218+0000","last_became_peered":"2023-10-06T03:10:44.606218+0000","last_unstale":"2023-10-06T03:10:44.608598+0000","last_undegraded":"2023-10-06T03:10:44.608598+0000","last_fullsized":"2023-10-06T03:10:44.608598+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:00.320153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613821+0000","last_change":"2023-10-06T03:10:44.610117+0000","last_active":"2023-10-06T03:10:44.613821+0000","last_peered":"2023-10-06T03:10:44.613821+0000","last_clean":"2023-10-06T03:10:44.613821+0000","last_became_active":"2023-10-06T03:10:44.609777+0000","last_became_peered":"2023-10-06T03:10:44.609777+0000","last_unstale":"2023-10-06T03:10:44.613821+0000","last_undegraded":"2023-10-06T03:10:44.613821+0000","last_fullsized":"2023-10-06T03:10:44.613821+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617323+0000","last_change":"2023-10-06T03:10:44.611318+0000","last_active":"2023-10-06T03:10:44.617323+0000","last_peered":"2023-10-06T03:10:44.617323+0000","last_clean":"2023-10-06T03:10:44.617323+0000","last_became_active":"2023-10-06T03:10:44.610495+0000","last_became_peered":"2023-10-06T03:10:44.610495+0000","last_unstale":"2023-10-06T03:10:44.617323+0000","last_undegraded":"2023-10-06T03:10:44.617323+0000","last_fullsized":"2023-10-06T03:10:44.617323+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:00.253800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615621+0000","last_change":"2023-10-06T03:10:44.613771+0000","last_active":"2023-10-06T03:10:44.615621+0000","last_peered":"2023-10-06T03:10:44.615621+0000","last_clean":"2023-10-06T03:10:44.615621+0000","last_became_active":"2023-10-06T03:10:44.613387+0000","last_became_peered":"2023-10-06T03:10:44.613387+0000","last_unstale":"2023-10-06T03:10:44.615621+0000","last_undegraded":"2023-10-06T03:10:44.615621+0000","last_fullsized":"2023-10-06T03:10:44.615621+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:15.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"185'4","reported_seq":14,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626960+0000","last_change":"2023-10-06T03:10:43.602925+0000","last_active":"2023-10-06T03:10:44.626960+0000","last_peered":"2023-10-06T03:10:44.626960+0000","last_clean":"2023-10-06T03:10:44.626960+0000","last_became_active":"2023-10-06T03:10:43.602673+0000","last_became_peered":"2023-10-06T03:10:43.602673+0000","last_unstale":"2023-10-06T03:10:44.626960+0000","last_undegraded":"2023-10-06T03:10:44.626960+0000","last_fullsized":"2023-10-06T03:10:44.626960+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624836+0000","last_change":"2023-10-06T03:10:44.612058+0000","last_active":"2023-10-06T03:10:44.624836+0000","last_peered":"2023-10-06T03:10:44.624836+0000","last_clean":"2023-10-06T03:10:44.624836+0000","last_became_active":"2023-10-06T03:10:44.611816+0000","last_became_peered":"2023-10-06T03:10:44.611816+0000","last_unstale":"2023-10-06T03:10:44.624836+0000","last_undegraded":"2023-10-06T03:10:44.624836+0000","last_fullsized":"2023-10-06T03:10:44.624836+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:07.945301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626677+0000","last_change":"2023-10-06T03:10:43.603054+0000","last_active":"2023-10-06T03:10:44.626677+0000","last_peered":"2023-10-06T03:10:44.626677+0000","last_clean":"2023-10-06T03:10:44.626677+0000","last_became_active":"2023-10-06T03:10:43.602849+0000","last_became_peered":"2023-10-06T03:10:43.602849+0000","last_unstale":"2023-10-06T03:10:44.626677+0000","last_undegraded":"2023-10-06T03:10:44.626677+0000","last_fullsized":"2023-10-06T03:10:44.626677+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:44.214194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612802+0000","last_change":"2023-10-06T03:10:44.608006+0000","last_active":"2023-10-06T03:10:44.612802+0000","last_peered":"2023-10-06T03:10:44.612802+0000","last_clean":"2023-10-06T03:10:44.612802+0000","last_became_active":"2023-10-06T03:10:44.607700+0000","last_became_peered":"2023-10-06T03:10:44.607700+0000","last_unstale":"2023-10-06T03:10:44.612802+0000","last_undegraded":"2023-10-06T03:10:44.612802+0000","last_fullsized":"2023-10-06T03:10:44.612802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:29.630492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622258+0000","last_change":"2023-10-06T03:10:43.604672+0000","last_active":"2023-10-06T03:10:44.622258+0000","last_peered":"2023-10-06T03:10:44.622258+0000","last_clean":"2023-10-06T03:10:44.622258+0000","last_became_active":"2023-10-06T03:10:43.602513+0000","last_became_peered":"2023-10-06T03:10:43.602513+0000","last_unstale":"2023-10-06T03:10:44.622258+0000","last_undegraded":"2023-10-06T03:10:44.622258+0000","last_fullsized":"2023-10-06T03:10:44.622258+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:36.206746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606511+0000","last_change":"2023-10-06T03:10:44.603518+0000","last_active":"2023-10-06T03:10:44.606511+0000","last_peered":"2023-10-06T03:10:44.606511+0000","last_clean":"2023-10-06T03:10:44.606511+0000","last_became_active":"2023-10-06T03:10:44.603255+0000","last_became_peered":"2023-10-06T03:10:44.603255+0000","last_unstale":"2023-10-06T03:10:44.606511+0000","last_undegraded":"2023-10-06T03:10:44.606511+0000","last_fullsized":"2023-10-06T03:10:44.606511+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626714+0000","last_change":"2023-10-06T03:10:43.609088+0000","last_active":"2023-10-06T03:10:44.626714+0000","last_peered":"2023-10-06T03:10:44.626714+0000","last_clean":"2023-10-06T03:10:44.626714+0000","last_became_active":"2023-10-06T03:10:43.608821+0000","last_became_peered":"2023-10-06T03:10:43.608821+0000","last_unstale":"2023-10-06T03:10:44.626714+0000","last_undegraded":"2023-10-06T03:10:44.626714+0000","last_fullsized":"2023-10-06T03:10:44.626714+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:36.735333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608618+0000","last_change":"2023-10-06T03:10:44.606922+0000","last_active":"2023-10-06T03:10:44.608618+0000","last_peered":"2023-10-06T03:10:44.608618+0000","last_clean":"2023-10-06T03:10:44.608618+0000","last_became_active":"2023-10-06T03:10:44.606522+0000","last_became_peered":"2023-10-06T03:10:44.606522+0000","last_unstale":"2023-10-06T03:10:44.608618+0000","last_undegraded":"2023-10-06T03:10:44.608618+0000","last_fullsized":"2023-10-06T03:10:44.608618+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:15.658864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.620526+0000","last_change":"2023-10-06T03:10:43.610186+0000","last_active":"2023-10-06T03:10:44.620526+0000","last_peered":"2023-10-06T03:10:44.620526+0000","last_clean":"2023-10-06T03:10:44.620526+0000","last_became_active":"2023-10-06T03:10:43.609855+0000","last_became_peered":"2023-10-06T03:10:43.609855+0000","last_unstale":"2023-10-06T03:10:44.620526+0000","last_undegraded":"2023-10-06T03:10:44.620526+0000","last_fullsized":"2023-10-06T03:10:44.620526+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613890+0000","last_change":"2023-10-06T03:10:44.608127+0000","last_active":"2023-10-06T03:10:44.613890+0000","last_peered":"2023-10-06T03:10:44.613890+0000","last_clean":"2023-10-06T03:10:44.613890+0000","last_became_active":"2023-10-06T03:10:44.607790+0000","last_became_peered":"2023-10-06T03:10:44.607790+0000","last_unstale":"2023-10-06T03:10:44.613890+0000","last_undegraded":"2023-10-06T03:10:44.613890+0000","last_fullsized":"2023-10-06T03:10:44.613890+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:03.796183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619316+0000","last_change":"2023-10-06T03:10:43.606849+0000","last_active":"2023-10-06T03:10:44.619316+0000","last_peered":"2023-10-06T03:10:44.619316+0000","last_clean":"2023-10-06T03:10:44.619316+0000","last_became_active":"2023-10-06T03:10:43.606628+0000","last_became_peered":"2023-10-06T03:10:43.606628+0000","last_unstale":"2023-10-06T03:10:44.619316+0000","last_undegraded":"2023-10-06T03:10:44.619316+0000","last_fullsized":"2023-10-06T03:10:44.619316+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:37.755076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615572+0000","last_change":"2023-10-06T03:10:44.609219+0000","last_active":"2023-10-06T03:10:44.615572+0000","last_peered":"2023-10-06T03:10:44.615572+0000","last_clean":"2023-10-06T03:10:44.615572+0000","last_became_active":"2023-10-06T03:10:44.608960+0000","last_became_peered":"2023-10-06T03:10:44.608960+0000","last_unstale":"2023-10-06T03:10:44.615572+0000","last_undegraded":"2023-10-06T03:10:44.615572+0000","last_fullsized":"2023-10-06T03:10:44.615572+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:58.313141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597171+0000","last_change":"2023-10-06T03:10:43.617619+0000","last_active":"2023-10-06T03:10:44.597171+0000","last_peered":"2023-10-06T03:10:44.597171+0000","last_clean":"2023-10-06T03:10:44.597171+0000","last_became_active":"2023-10-06T03:10:43.617376+0000","last_became_peered":"2023-10-06T03:10:43.617376+0000","last_unstale":"2023-10-06T03:10:44.597171+0000","last_undegraded":"2023-10-06T03:10:44.597171+0000","last_fullsized":"2023-10-06T03:10:44.597171+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:25.490995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614923+0000","last_change":"2023-10-06T03:10:44.613231+0000","last_active":"2023-10-06T03:10:44.614923+0000","last_peered":"2023-10-06T03:10:44.614923+0000","last_clean":"2023-10-06T03:10:44.614923+0000","last_became_active":"2023-10-06T03:10:44.612945+0000","last_became_peered":"2023-10-06T03:10:44.612945+0000","last_unstale":"2023-10-06T03:10:44.614923+0000","last_undegraded":"2023-10-06T03:10:44.614923+0000","last_fullsized":"2023-10-06T03:10:44.614923+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:50.256975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602634+0000","last_change":"2023-10-06T03:10:43.608993+0000","last_active":"2023-10-06T03:10:44.602634+0000","last_peered":"2023-10-06T03:10:44.602634+0000","last_clean":"2023-10-06T03:10:44.602634+0000","last_became_active":"2023-10-06T03:10:43.608716+0000","last_became_peered":"2023-10-06T03:10:43.608716+0000","last_unstale":"2023-10-06T03:10:44.602634+0000","last_undegraded":"2023-10-06T03:10:44.602634+0000","last_fullsized":"2023-10-06T03:10:44.602634+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:13.569451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614842+0000","last_change":"2023-10-06T03:10:44.612005+0000","last_active":"2023-10-06T03:10:44.614842+0000","last_peered":"2023-10-06T03:10:44.614842+0000","last_clean":"2023-10-06T03:10:44.614842+0000","last_became_active":"2023-10-06T03:10:44.611698+0000","last_became_peered":"2023-10-06T03:10:44.611698+0000","last_unstale":"2023-10-06T03:10:44.614842+0000","last_undegraded":"2023-10-06T03:10:44.614842+0000","last_fullsized":"2023-10-06T03:10:44.614842+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:44.601383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603232+0000","last_change":"2023-10-06T03:10:43.608400+0000","last_active":"2023-10-06T03:10:44.603232+0000","last_peered":"2023-10-06T03:10:44.603232+0000","last_clean":"2023-10-06T03:10:44.603232+0000","last_became_active":"2023-10-06T03:10:43.607925+0000","last_became_peered":"2023-10-06T03:10:43.607925+0000","last_unstale":"2023-10-06T03:10:44.603232+0000","last_undegraded":"2023-10-06T03:10:44.603232+0000","last_fullsized":"2023-10-06T03:10:44.603232+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:36.739791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616814+0000","last_change":"2023-10-06T03:10:44.607010+0000","last_active":"2023-10-06T03:10:44.616814+0000","last_peered":"2023-10-06T03:10:44.616814+0000","last_clean":"2023-10-06T03:10:44.616814+0000","last_became_active":"2023-10-06T03:10:44.606570+0000","last_became_peered":"2023-10-06T03:10:44.606570+0000","last_unstale":"2023-10-06T03:10:44.616814+0000","last_undegraded":"2023-10-06T03:10:44.616814+0000","last_fullsized":"2023-10-06T03:10:44.616814+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:59.437351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597622+0000","last_change":"2023-10-06T03:10:43.611484+0000","last_active":"2023-10-06T03:10:44.597622+0000","last_peered":"2023-10-06T03:10:44.597622+0000","last_clean":"2023-10-06T03:10:44.597622+0000","last_became_active":"2023-10-06T03:10:43.611227+0000","last_became_peered":"2023-10-06T03:10:43.611227+0000","last_unstale":"2023-10-06T03:10:44.597622+0000","last_undegraded":"2023-10-06T03:10:44.597622+0000","last_fullsized":"2023-10-06T03:10:44.597622+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:43.590879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625995+0000","last_change":"2023-10-06T03:10:44.613128+0000","last_active":"2023-10-06T03:10:44.625995+0000","last_peered":"2023-10-06T03:10:44.625995+0000","last_clean":"2023-10-06T03:10:44.625995+0000","last_became_active":"2023-10-06T03:10:44.612674+0000","last_became_peered":"2023-10-06T03:10:44.612674+0000","last_unstale":"2023-10-06T03:10:44.625995+0000","last_undegraded":"2023-10-06T03:10:44.625995+0000","last_fullsized":"2023-10-06T03:10:44.625995+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:04.843808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"185'3","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622581+0000","last_change":"2023-10-06T03:10:43.610252+0000","last_active":"2023-10-06T03:10:44.622581+0000","last_peered":"2023-10-06T03:10:44.622581+0000","last_clean":"2023-10-06T03:10:44.622581+0000","last_became_active":"2023-10-06T03:10:43.609979+0000","last_became_peered":"2023-10-06T03:10:43.609979+0000","last_unstale":"2023-10-06T03:10:44.622581+0000","last_undegraded":"2023-10-06T03:10:44.622581+0000","last_fullsized":"2023-10-06T03:10:44.622581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:26.682895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618618+0000","last_change":"2023-10-06T03:10:44.617246+0000","last_active":"2023-10-06T03:10:44.618618+0000","last_peered":"2023-10-06T03:10:44.618618+0000","last_clean":"2023-10-06T03:10:44.618618+0000","last_became_active":"2023-10-06T03:10:44.616958+0000","last_became_peered":"2023-10-06T03:10:44.616958+0000","last_unstale":"2023-10-06T03:10:44.618618+0000","last_undegraded":"2023-10-06T03:10:44.618618+0000","last_fullsized":"2023-10-06T03:10:44.618618+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.761168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"185'6","reported_seq":16,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625574+0000","last_change":"2023-10-06T03:10:43.604507+0000","last_active":"2023-10-06T03:10:44.625574+0000","last_peered":"2023-10-06T03:10:44.625574+0000","last_clean":"2023-10-06T03:10:44.625574+0000","last_became_active":"2023-10-06T03:10:43.604261+0000","last_became_peered":"2023-10-06T03:10:43.604261+0000","last_unstale":"2023-10-06T03:10:44.625574+0000","last_undegraded":"2023-10-06T03:10:44.625574+0000","last_fullsized":"2023-10-06T03:10:44.625574+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:12.780687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617308+0000","last_change":"2023-10-06T03:10:44.612460+0000","last_active":"2023-10-06T03:10:44.617308+0000","last_peered":"2023-10-06T03:10:44.617308+0000","last_clean":"2023-10-06T03:10:44.617308+0000","last_became_active":"2023-10-06T03:10:44.612210+0000","last_became_peered":"2023-10-06T03:10:44.612210+0000","last_unstale":"2023-10-06T03:10:44.617308+0000","last_undegraded":"2023-10-06T03:10:44.617308+0000","last_fullsized":"2023-10-06T03:10:44.617308+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:42.518427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626985+0000","last_change":"2023-10-06T03:10:43.609477+0000","last_active":"2023-10-06T03:10:44.626985+0000","last_peered":"2023-10-06T03:10:44.626985+0000","last_clean":"2023-10-06T03:10:44.626985+0000","last_became_active":"2023-10-06T03:10:43.609213+0000","last_became_peered":"2023-10-06T03:10:43.609213+0000","last_unstale":"2023-10-06T03:10:44.626985+0000","last_undegraded":"2023-10-06T03:10:44.626985+0000","last_fullsized":"2023-10-06T03:10:44.626985+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:20.842732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613908+0000","last_change":"2023-10-06T03:10:44.609257+0000","last_active":"2023-10-06T03:10:44.613908+0000","last_peered":"2023-10-06T03:10:44.613908+0000","last_clean":"2023-10-06T03:10:44.613908+0000","last_became_active":"2023-10-06T03:10:44.609084+0000","last_became_peered":"2023-10-06T03:10:44.609084+0000","last_unstale":"2023-10-06T03:10:44.613908+0000","last_undegraded":"2023-10-06T03:10:44.613908+0000","last_fullsized":"2023-10-06T03:10:44.613908+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:42.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.594785+0000","last_change":"2023-10-06T03:10:43.611154+0000","last_active":"2023-10-06T03:10:44.594785+0000","last_peered":"2023-10-06T03:10:44.594785+0000","last_clean":"2023-10-06T03:10:44.594785+0000","last_became_active":"2023-10-06T03:10:43.610907+0000","last_became_peered":"2023-10-06T03:10:43.610907+0000","last_unstale":"2023-10-06T03:10:44.594785+0000","last_undegraded":"2023-10-06T03:10:44.594785+0000","last_fullsized":"2023-10-06T03:10:44.594785+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617353+0000","last_change":"2023-10-06T03:10:44.611635+0000","last_active":"2023-10-06T03:10:44.617353+0000","last_peered":"2023-10-06T03:10:44.617353+0000","last_clean":"2023-10-06T03:10:44.617353+0000","last_became_active":"2023-10-06T03:10:44.611050+0000","last_became_peered":"2023-10-06T03:10:44.611050+0000","last_unstale":"2023-10-06T03:10:44.617353+0000","last_undegraded":"2023-10-06T03:10:44.617353+0000","last_fullsized":"2023-10-06T03:10:44.617353+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:40.043675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602572+0000","last_change":"2023-10-06T03:10:43.608802+0000","last_active":"2023-10-06T03:10:44.602572+0000","last_peered":"2023-10-06T03:10:44.602572+0000","last_clean":"2023-10-06T03:10:44.602572+0000","last_became_active":"2023-10-06T03:10:43.608524+0000","last_became_peered":"2023-10-06T03:10:43.608524+0000","last_unstale":"2023-10-06T03:10:44.602572+0000","last_undegraded":"2023-10-06T03:10:44.602572+0000","last_fullsized":"2023-10-06T03:10:44.602572+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:28.221106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616261+0000","last_change":"2023-10-06T03:10:44.605209+0000","last_active":"2023-10-06T03:10:44.616261+0000","last_peered":"2023-10-06T03:10:44.616261+0000","last_clean":"2023-10-06T03:10:44.616261+0000","last_became_active":"2023-10-06T03:10:44.604955+0000","last_became_peered":"2023-10-06T03:10:44.604955+0000","last_unstale":"2023-10-06T03:10:44.616261+0000","last_undegraded":"2023-10-06T03:10:44.616261+0000","last_fullsized":"2023-10-06T03:10:44.616261+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:38.258343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"185'8","reported_seq":18,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.629219+0000","last_change":"2023-10-06T03:10:43.617468+0000","last_active":"2023-10-06T03:10:44.629219+0000","last_peered":"2023-10-06T03:10:44.629219+0000","last_clean":"2023-10-06T03:10:44.629219+0000","last_became_active":"2023-10-06T03:10:43.617179+0000","last_became_peered":"2023-10-06T03:10:43.617179+0000","last_unstale":"2023-10-06T03:10:44.629219+0000","last_undegraded":"2023-10-06T03:10:44.629219+0000","last_fullsized":"2023-10-06T03:10:44.629219+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:31.017050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613036+0000","last_change":"2023-10-06T03:10:44.606315+0000","last_active":"2023-10-06T03:10:44.613036+0000","last_peered":"2023-10-06T03:10:44.613036+0000","last_clean":"2023-10-06T03:10:44.613036+0000","last_became_active":"2023-10-06T03:10:44.605927+0000","last_became_peered":"2023-10-06T03:10:44.605927+0000","last_unstale":"2023-10-06T03:10:44.613036+0000","last_undegraded":"2023-10-06T03:10:44.613036+0000","last_fullsized":"2023-10-06T03:10:44.613036+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:03.205188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.621050+0000","last_change":"2023-10-06T03:10:43.607558+0000","last_active":"2023-10-06T03:10:44.621050+0000","last_peered":"2023-10-06T03:10:44.621050+0000","last_clean":"2023-10-06T03:10:44.621050+0000","last_became_active":"2023-10-06T03:10:43.607274+0000","last_became_peered":"2023-10-06T03:10:43.607274+0000","last_unstale":"2023-10-06T03:10:44.621050+0000","last_undegraded":"2023-10-06T03:10:44.621050+0000","last_fullsized":"2023-10-06T03:10:44.621050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:55.593599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618323+0000","last_change":"2023-10-06T03:10:44.614485+0000","last_active":"2023-10-06T03:10:44.618323+0000","last_peered":"2023-10-06T03:10:44.618323+0000","last_clean":"2023-10-06T03:10:44.618323+0000","last_became_active":"2023-10-06T03:10:44.614187+0000","last_became_peered":"2023-10-06T03:10:44.614187+0000","last_unstale":"2023-10-06T03:10:44.618323+0000","last_undegraded":"2023-10-06T03:10:44.618323+0000","last_fullsized":"2023-10-06T03:10:44.618323+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:23.629899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.621860+0000","last_change":"2023-10-06T03:10:43.608774+0000","last_active":"2023-10-06T03:10:44.621860+0000","last_peered":"2023-10-06T03:10:44.621860+0000","last_clean":"2023-10-06T03:10:44.621860+0000","last_became_active":"2023-10-06T03:10:43.608468+0000","last_became_peered":"2023-10-06T03:10:43.608468+0000","last_unstale":"2023-10-06T03:10:44.621860+0000","last_undegraded":"2023-10-06T03:10:44.621860+0000","last_fullsized":"2023-10-06T03:10:44.621860+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:49.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626701+0000","last_change":"2023-10-06T03:10:44.625520+0000","last_active":"2023-10-06T03:10:44.626701+0000","last_peered":"2023-10-06T03:10:44.626701+0000","last_clean":"2023-10-06T03:10:44.626701+0000","last_became_active":"2023-10-06T03:10:44.625234+0000","last_became_peered":"2023-10-06T03:10:44.625234+0000","last_unstale":"2023-10-06T03:10:44.626701+0000","last_undegraded":"2023-10-06T03:10:44.626701+0000","last_fullsized":"2023-10-06T03:10:44.626701+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:06.054091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622246+0000","last_change":"2023-10-06T03:10:43.609053+0000","last_active":"2023-10-06T03:10:44.622246+0000","last_peered":"2023-10-06T03:10:44.622246+0000","last_clean":"2023-10-06T03:10:44.622246+0000","last_became_active":"2023-10-06T03:10:43.608789+0000","last_became_peered":"2023-10-06T03:10:43.608789+0000","last_unstale":"2023-10-06T03:10:44.622246+0000","last_undegraded":"2023-10-06T03:10:44.622246+0000","last_fullsized":"2023-10-06T03:10:44.622246+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:36.389568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.604674+0000","last_change":"2023-10-06T03:10:44.602617+0000","last_active":"2023-10-06T03:10:44.604674+0000","last_peered":"2023-10-06T03:10:44.604674+0000","last_clean":"2023-10-06T03:10:44.604674+0000","last_became_active":"2023-10-06T03:10:44.602342+0000","last_became_peered":"2023-10-06T03:10:44.602342+0000","last_unstale":"2023-10-06T03:10:44.604674+0000","last_undegraded":"2023-10-06T03:10:44.604674+0000","last_fullsized":"2023-10-06T03:10:44.604674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.598698+0000","last_change":"2023-10-06T03:10:43.611124+0000","last_active":"2023-10-06T03:10:44.598698+0000","last_peered":"2023-10-06T03:10:44.598698+0000","last_clean":"2023-10-06T03:10:44.598698+0000","last_became_active":"2023-10-06T03:10:43.610877+0000","last_became_peered":"2023-10-06T03:10:43.610877+0000","last_unstale":"2023-10-06T03:10:44.598698+0000","last_undegraded":"2023-10-06T03:10:44.598698+0000","last_fullsized":"2023-10-06T03:10:44.598698+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:20:00.923932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.620606+0000","last_change":"2023-10-06T03:10:44.610074+0000","last_active":"2023-10-06T03:10:44.620606+0000","last_peered":"2023-10-06T03:10:44.620606+0000","last_clean":"2023-10-06T03:10:44.620606+0000","last_became_active":"2023-10-06T03:10:44.609795+0000","last_became_peered":"2023-10-06T03:10:44.609795+0000","last_unstale":"2023-10-06T03:10:44.620606+0000","last_undegraded":"2023-10-06T03:10:44.620606+0000","last_fullsized":"2023-10-06T03:10:44.620606+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:23.309599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618997+0000","last_change":"2023-10-06T03:10:43.607875+0000","last_active":"2023-10-06T03:10:44.618997+0000","last_peered":"2023-10-06T03:10:44.618997+0000","last_clean":"2023-10-06T03:10:44.618997+0000","last_became_active":"2023-10-06T03:10:43.607570+0000","last_became_peered":"2023-10-06T03:10:43.607570+0000","last_unstale":"2023-10-06T03:10:44.618997+0000","last_undegraded":"2023-10-06T03:10:44.618997+0000","last_fullsized":"2023-10-06T03:10:44.618997+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622108+0000","last_change":"2023-10-06T03:10:44.614535+0000","last_active":"2023-10-06T03:10:44.622108+0000","last_peered":"2023-10-06T03:10:44.622108+0000","last_clean":"2023-10-06T03:10:44.622108+0000","last_became_active":"2023-10-06T03:10:44.614151+0000","last_became_peered":"2023-10-06T03:10:44.614151+0000","last_unstale":"2023-10-06T03:10:44.622108+0000","last_undegraded":"2023-10-06T03:10:44.622108+0000","last_fullsized":"2023-10-06T03:10:44.622108+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:42.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595616+0000","last_change":"2023-10-06T03:10:43.611649+0000","last_active":"2023-10-06T03:10:44.595616+0000","last_peered":"2023-10-06T03:10:44.595616+0000","last_clean":"2023-10-06T03:10:44.595616+0000","last_became_active":"2023-10-06T03:10:43.611297+0000","last_became_peered":"2023-10-06T03:10:43.611297+0000","last_unstale":"2023-10-06T03:10:44.595616+0000","last_undegraded":"2023-10-06T03:10:44.595616+0000","last_fullsized":"2023-10-06T03:10:44.595616+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.884730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617340+0000","last_change":"2023-10-06T03:10:44.611469+0000","last_active":"2023-10-06T03:10:44.617340+0000","last_peered":"2023-10-06T03:10:44.617340+0000","last_clean":"2023-10-06T03:10:44.617340+0000","last_became_active":"2023-10-06T03:10:44.610791+0000","last_became_peered":"2023-10-06T03:10:44.610791+0000","last_unstale":"2023-10-06T03:10:44.617340+0000","last_undegraded":"2023-10-06T03:10:44.617340+0000","last_fullsized":"2023-10-06T03:10:44.617340+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:19.393529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602668+0000","last_change":"2023-10-06T03:10:43.610052+0000","last_active":"2023-10-06T03:10:44.602668+0000","last_peered":"2023-10-06T03:10:44.602668+0000","last_clean":"2023-10-06T03:10:44.602668+0000","last_became_active":"2023-10-06T03:10:43.609757+0000","last_became_peered":"2023-10-06T03:10:43.609757+0000","last_unstale":"2023-10-06T03:10:44.602668+0000","last_undegraded":"2023-10-06T03:10:44.602668+0000","last_fullsized":"2023-10-06T03:10:44.602668+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:20.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614717+0000","last_change":"2023-10-06T03:10:44.608535+0000","last_active":"2023-10-06T03:10:44.614717+0000","last_peered":"2023-10-06T03:10:44.614717+0000","last_clean":"2023-10-06T03:10:44.614717+0000","last_became_active":"2023-10-06T03:10:44.607437+0000","last_became_peered":"2023-10-06T03:10:44.607437+0000","last_unstale":"2023-10-06T03:10:44.614717+0000","last_undegraded":"2023-10-06T03:10:44.614717+0000","last_fullsized":"2023-10-06T03:10:44.614717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:07.998111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.596401+0000","last_change":"2023-10-06T03:10:43.609073+0000","last_active":"2023-10-06T03:10:44.596401+0000","last_peered":"2023-10-06T03:10:44.596401+0000","last_clean":"2023-10-06T03:10:44.596401+0000","last_became_active":"2023-10-06T03:10:43.608677+0000","last_became_peered":"2023-10-06T03:10:43.608677+0000","last_unstale":"2023-10-06T03:10:44.596401+0000","last_undegraded":"2023-10-06T03:10:44.596401+0000","last_fullsized":"2023-10-06T03:10:44.596401+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:09.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613017+0000","last_change":"2023-10-06T03:10:44.606162+0000","last_active":"2023-10-06T03:10:44.613017+0000","last_peered":"2023-10-06T03:10:44.613017+0000","last_clean":"2023-10-06T03:10:44.613017+0000","last_became_active":"2023-10-06T03:10:44.605602+0000","last_became_peered":"2023-10-06T03:10:44.605602+0000","last_unstale":"2023-10-06T03:10:44.613017+0000","last_undegraded":"2023-10-06T03:10:44.613017+0000","last_fullsized":"2023-10-06T03:10:44.613017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:54.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.629090+0000","last_change":"2023-10-06T03:10:43.603495+0000","last_active":"2023-10-06T03:10:44.629090+0000","last_peered":"2023-10-06T03:10:44.629090+0000","last_clean":"2023-10-06T03:10:44.629090+0000","last_became_active":"2023-10-06T03:10:43.603249+0000","last_became_peered":"2023-10-06T03:10:43.603249+0000","last_unstale":"2023-10-06T03:10:44.629090+0000","last_undegraded":"2023-10-06T03:10:44.629090+0000","last_fullsized":"2023-10-06T03:10:44.629090+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:49.332478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616794+0000","last_change":"2023-10-06T03:10:44.606893+0000","last_active":"2023-10-06T03:10:44.616794+0000","last_peered":"2023-10-06T03:10:44.616794+0000","last_clean":"2023-10-06T03:10:44.616794+0000","last_became_active":"2023-10-06T03:10:44.606341+0000","last_became_peered":"2023-10-06T03:10:44.606341+0000","last_unstale":"2023-10-06T03:10:44.616794+0000","last_undegraded":"2023-10-06T03:10:44.616794+0000","last_fullsized":"2023-10-06T03:10:44.616794+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:27.791364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"186'1","reported_seq":11,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-06T03:10:45.874675+0000","last_change":"2023-10-06T03:10:43.617649+0000","last_active":"2023-10-06T03:10:45.874675+0000","last_peered":"2023-10-06T03:10:45.874675+0000","last_clean":"2023-10-06T03:10:45.874675+0000","last_became_active":"2023-10-06T03:10:43.617403+0000","last_became_peered":"2023-10-06T03:10:43.617403+0000","last_unstale":"2023-10-06T03:10:45.874675+0000","last_undegraded":"2023-10-06T03:10:45.874675+0000","last_fullsized":"2023-10-06T03:10:45.874675+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:17.771738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612300+0000","last_change":"2023-10-06T03:10:44.610899+0000","last_active":"2023-10-06T03:10:44.612300+0000","last_peered":"2023-10-06T03:10:44.612300+0000","last_clean":"2023-10-06T03:10:44.612300+0000","last_became_active":"2023-10-06T03:10:44.610620+0000","last_became_peered":"2023-10-06T03:10:44.610620+0000","last_unstale":"2023-10-06T03:10:44.612300+0000","last_undegraded":"2023-10-06T03:10:44.612300+0000","last_fullsized":"2023-10-06T03:10:44.612300+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608467+0000","last_change":"2023-10-06T03:10:43.609223+0000","last_active":"2023-10-06T03:10:44.608467+0000","last_peered":"2023-10-06T03:10:44.608467+0000","last_clean":"2023-10-06T03:10:44.608467+0000","last_became_active":"2023-10-06T03:10:43.606876+0000","last_became_peered":"2023-10-06T03:10:43.606876+0000","last_unstale":"2023-10-06T03:10:44.608467+0000","last_undegraded":"2023-10-06T03:10:44.608467+0000","last_fullsized":"2023-10-06T03:10:44.608467+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:04.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611431+0000","last_change":"2023-10-06T03:10:44.605653+0000","last_active":"2023-10-06T03:10:44.611431+0000","last_peered":"2023-10-06T03:10:44.611431+0000","last_clean":"2023-10-06T03:10:44.611431+0000","last_became_active":"2023-10-06T03:10:44.605289+0000","last_became_peered":"2023-10-06T03:10:44.605289+0000","last_unstale":"2023-10-06T03:10:44.611431+0000","last_undegraded":"2023-10-06T03:10:44.611431+0000","last_fullsized":"2023-10-06T03:10:44.611431+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603121+0000","last_change":"2023-10-06T03:10:43.611147+0000","last_active":"2023-10-06T03:10:44.603121+0000","last_peered":"2023-10-06T03:10:44.603121+0000","last_clean":"2023-10-06T03:10:44.603121+0000","last_became_active":"2023-10-06T03:10:43.610865+0000","last_became_peered":"2023-10-06T03:10:43.610865+0000","last_unstale":"2023-10-06T03:10:44.603121+0000","last_undegraded":"2023-10-06T03:10:44.603121+0000","last_fullsized":"2023-10-06T03:10:44.603121+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625117+0000","last_change":"2023-10-06T03:10:44.608741+0000","last_active":"2023-10-06T03:10:44.625117+0000","last_peered":"2023-10-06T03:10:44.625117+0000","last_clean":"2023-10-06T03:10:44.625117+0000","last_became_active":"2023-10-06T03:10:44.608014+0000","last_became_peered":"2023-10-06T03:10:44.608014+0000","last_unstale":"2023-10-06T03:10:44.625117+0000","last_undegraded":"2023-10-06T03:10:44.625117+0000","last_fullsized":"2023-10-06T03:10:44.625117+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:52.576251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595192+0000","last_change":"2023-10-06T03:10:43.611002+0000","last_active":"2023-10-06T03:10:44.595192+0000","last_peered":"2023-10-06T03:10:44.595192+0000","last_clean":"2023-10-06T03:10:44.595192+0000","last_became_active":"2023-10-06T03:10:43.610692+0000","last_became_peered":"2023-10-06T03:10:43.610692+0000","last_unstale":"2023-10-06T03:10:44.595192+0000","last_undegraded":"2023-10-06T03:10:44.595192+0000","last_fullsized":"2023-10-06T03:10:44.595192+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:15.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624786+0000","last_change":"2023-10-06T03:10:44.607396+0000","last_active":"2023-10-06T03:10:44.624786+0000","last_peered":"2023-10-06T03:10:44.624786+0000","last_clean":"2023-10-06T03:10:44.624786+0000","last_became_active":"2023-10-06T03:10:44.607134+0000","last_became_peered":"2023-10-06T03:10:44.607134+0000","last_unstale":"2023-10-06T03:10:44.624786+0000","last_undegraded":"2023-10-06T03:10:44.624786+0000","last_fullsized":"2023-10-06T03:10:44.624786+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:11.054676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.594876+0000","last_change":"2023-10-06T03:10:43.607559+0000","last_active":"2023-10-06T03:10:44.594876+0000","last_peered":"2023-10-06T03:10:44.594876+0000","last_clean":"2023-10-06T03:10:44.594876+0000","last_became_active":"2023-10-06T03:10:43.607302+0000","last_became_peered":"2023-10-06T03:10:43.607302+0000","last_unstale":"2023-10-06T03:10:44.594876+0000","last_undegraded":"2023-10-06T03:10:44.594876+0000","last_fullsized":"2023-10-06T03:10:44.594876+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626242+0000","last_change":"2023-10-06T03:10:44.624362+0000","last_active":"2023-10-06T03:10:44.626242+0000","last_peered":"2023-10-06T03:10:44.626242+0000","last_clean":"2023-10-06T03:10:44.626242+0000","last_became_active":"2023-10-06T03:10:44.624101+0000","last_became_peered":"2023-10-06T03:10:44.624101+0000","last_unstale":"2023-10-06T03:10:44.626242+0000","last_undegraded":"2023-10-06T03:10:44.626242+0000","last_fullsized":"2023-10-06T03:10:44.626242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:05.039778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.599608+0000","last_change":"2023-10-06T03:10:43.605635+0000","last_active":"2023-10-06T03:10:44.599608+0000","last_peered":"2023-10-06T03:10:44.599608+0000","last_clean":"2023-10-06T03:10:44.599608+0000","last_became_active":"2023-10-06T03:10:43.605202+0000","last_became_peered":"2023-10-06T03:10:43.605202+0000","last_unstale":"2023-10-06T03:10:44.599608+0000","last_undegraded":"2023-10-06T03:10:44.599608+0000","last_fullsized":"2023-10-06T03:10:44.599608+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:32.594410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606406+0000","last_change":"2023-10-06T03:10:44.604594+0000","last_active":"2023-10-06T03:10:44.606406+0000","last_peered":"2023-10-06T03:10:44.606406+0000","last_clean":"2023-10-06T03:10:44.606406+0000","last_became_active":"2023-10-06T03:10:44.604267+0000","last_became_peered":"2023-10-06T03:10:44.604267+0000","last_unstale":"2023-10-06T03:10:44.606406+0000","last_undegraded":"2023-10-06T03:10:44.606406+0000","last_fullsized":"2023-10-06T03:10:44.606406+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.601579+0000","last_change":"2023-10-06T03:10:43.608350+0000","last_active":"2023-10-06T03:10:44.601579+0000","last_peered":"2023-10-06T03:10:44.601579+0000","last_clean":"2023-10-06T03:10:44.601579+0000","last_became_active":"2023-10-06T03:10:43.608121+0000","last_became_peered":"2023-10-06T03:10:43.608121+0000","last_unstale":"2023-10-06T03:10:44.601579+0000","last_undegraded":"2023-10-06T03:10:44.601579+0000","last_fullsized":"2023-10-06T03:10:44.601579+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:14.031939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626216+0000","last_change":"2023-10-06T03:10:44.613649+0000","last_active":"2023-10-06T03:10:44.626216+0000","last_peered":"2023-10-06T03:10:44.626216+0000","last_clean":"2023-10-06T03:10:44.626216+0000","last_became_active":"2023-10-06T03:10:44.613383+0000","last_became_peered":"2023-10-06T03:10:44.613383+0000","last_unstale":"2023-10-06T03:10:44.626216+0000","last_undegraded":"2023-10-06T03:10:44.626216+0000","last_fullsized":"2023-10-06T03:10:44.626216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:01.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":147,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.604838+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:44.604838+0000","last_peered":"2023-10-06T03:10:44.604838+0000","last_clean":"2023-10-06T03:10:44.604838+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:44.604838+0000","last_undegraded":"2023-10-06T03:10:44.604838+0000","last_fullsized":"2023-10-06T03:10:44.604838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615564+0000","last_change":"2023-10-06T03:10:44.613667+0000","last_active":"2023-10-06T03:10:44.615564+0000","last_peered":"2023-10-06T03:10:44.615564+0000","last_clean":"2023-10-06T03:10:44.615564+0000","last_became_active":"2023-10-06T03:10:44.613322+0000","last_became_peered":"2023-10-06T03:10:44.613322+0000","last_unstale":"2023-10-06T03:10:44.615564+0000","last_undegraded":"2023-10-06T03:10:44.615564+0000","last_fullsized":"2023-10-06T03:10:44.615564+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:01.094720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597815+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:44.597815+0000","last_peered":"2023-10-06T03:10:44.597815+0000","last_clean":"2023-10-06T03:10:44.597815+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:44.597815+0000","last_undegraded":"2023-10-06T03:10:44.597815+0000","last_fullsized":"2023-10-06T03:10:44.597815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618962+0000","last_change":"2023-10-06T03:10:44.615738+0000","last_active":"2023-10-06T03:10:44.618962+0000","last_peered":"2023-10-06T03:10:44.618962+0000","last_clean":"2023-10-06T03:10:44.618962+0000","last_became_active":"2023-10-06T03:10:44.615457+0000","last_became_peered":"2023-10-06T03:10:44.615457+0000","last_unstale":"2023-10-06T03:10:44.618962+0000","last_undegraded":"2023-10-06T03:10:44.618962+0000","last_fullsized":"2023-10-06T03:10:44.618962+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":150,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603180+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:44.603180+0000","last_peered":"2023-10-06T03:10:44.603180+0000","last_clean":"2023-10-06T03:10:44.603180+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:44.603180+0000","last_undegraded":"2023-10-06T03:10:44.603180+0000","last_fullsized":"2023-10-06T03:10:44.603180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616994+0000","last_change":"2023-10-06T03:10:44.610773+0000","last_active":"2023-10-06T03:10:44.616994+0000","last_peered":"2023-10-06T03:10:44.616994+0000","last_clean":"2023-10-06T03:10:44.616994+0000","last_became_active":"2023-10-06T03:10:44.610515+0000","last_became_peered":"2023-10-06T03:10:44.610515+0000","last_unstale":"2023-10-06T03:10:44.616994+0000","last_undegraded":"2023-10-06T03:10:44.616994+0000","last_fullsized":"2023-10-06T03:10:44.616994+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:37.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602214+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:44.602214+0000","last_peered":"2023-10-06T03:10:44.602214+0000","last_clean":"2023-10-06T03:10:44.602214+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:44.602214+0000","last_undegraded":"2023-10-06T03:10:44.602214+0000","last_fullsized":"2023-10-06T03:10:44.602214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615899+0000","last_change":"2023-10-06T03:10:44.614523+0000","last_active":"2023-10-06T03:10:44.615899+0000","last_peered":"2023-10-06T03:10:44.615899+0000","last_clean":"2023-10-06T03:10:44.615899+0000","last_became_active":"2023-10-06T03:10:44.614265+0000","last_became_peered":"2023-10-06T03:10:44.614265+0000","last_unstale":"2023-10-06T03:10:44.615899+0000","last_undegraded":"2023-10-06T03:10:44.615899+0000","last_fullsized":"2023-10-06T03:10:44.615899+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:32.111948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":303,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602337+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:44.602337+0000","last_peered":"2023-10-06T03:10:44.602337+0000","last_clean":"2023-10-06T03:10:44.602337+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:44.602337+0000","last_undegraded":"2023-10-06T03:10:44.602337+0000","last_fullsized":"2023-10-06T03:10:44.602337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597322+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:44.597322+0000","last_peered":"2023-10-06T03:10:44.597322+0000","last_clean":"2023-10-06T03:10:44.597322+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:44.597322+0000","last_undegraded":"2023-10-06T03:10:44.597322+0000","last_fullsized":"2023-10-06T03:10:44.597322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625133+0000","last_change":"2023-10-06T03:10:44.608933+0000","last_active":"2023-10-06T03:10:44.625133+0000","last_peered":"2023-10-06T03:10:44.625133+0000","last_clean":"2023-10-06T03:10:44.625133+0000","last_became_active":"2023-10-06T03:10:44.608337+0000","last_became_peered":"2023-10-06T03:10:44.608337+0000","last_unstale":"2023-10-06T03:10:44.625133+0000","last_undegraded":"2023-10-06T03:10:44.625133+0000","last_fullsized":"2023-10-06T03:10:44.625133+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:12.581813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.599681+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:44.599681+0000","last_peered":"2023-10-06T03:10:44.599681+0000","last_clean":"2023-10-06T03:10:44.599681+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:44.599681+0000","last_undegraded":"2023-10-06T03:10:44.599681+0000","last_fullsized":"2023-10-06T03:10:44.599681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611631+0000","last_change":"2023-10-06T03:10:44.610553+0000","last_active":"2023-10-06T03:10:44.611631+0000","last_peered":"2023-10-06T03:10:44.611631+0000","last_clean":"2023-10-06T03:10:44.611631+0000","last_became_active":"2023-10-06T03:10:44.610247+0000","last_became_peered":"2023-10-06T03:10:44.610247+0000","last_unstale":"2023-10-06T03:10:44.611631+0000","last_undegraded":"2023-10-06T03:10:44.611631+0000","last_fullsized":"2023-10-06T03:10:44.611631+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595601+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:44.595601+0000","last_peered":"2023-10-06T03:10:44.595601+0000","last_clean":"2023-10-06T03:10:44.595601+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:44.595601+0000","last_undegraded":"2023-10-06T03:10:44.595601+0000","last_fullsized":"2023-10-06T03:10:44.595601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626189+0000","last_change":"2023-10-06T03:10:44.623258+0000","last_active":"2023-10-06T03:10:44.626189+0000","last_peered":"2023-10-06T03:10:44.626189+0000","last_clean":"2023-10-06T03:10:44.626189+0000","last_became_active":"2023-10-06T03:10:44.622785+0000","last_became_peered":"2023-10-06T03:10:44.622785+0000","last_unstale":"2023-10-06T03:10:44.626189+0000","last_undegraded":"2023-10-06T03:10:44.626189+0000","last_fullsized":"2023-10-06T03:10:44.626189+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:48.634468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.598553+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:44.598553+0000","last_peered":"2023-10-06T03:10:44.598553+0000","last_clean":"2023-10-06T03:10:44.598553+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:44.598553+0000","last_undegraded":"2023-10-06T03:10:44.598553+0000","last_fullsized":"2023-10-06T03:10:44.598553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304276,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067564,"statfs":{"total":96636764160,"available":96325185536,"internally_reserved":0,"allocated":2961408,"data_stored":1714396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.745,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64100000000000001,"5min":1.264,"15min":1.264},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":2.286,"15min":2.286},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":2.504,"15min":2.504},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":1.141,"15min":1.141},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":1.639,"15min":1.639},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82199999999999995,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.307,"15min":0.307},"max":{"1min":0.77800000000000002,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.376}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303700,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068140,"statfs":{"total":96636764160,"available":96325775360,"internally_reserved":0,"allocated":2371584,"data_stored":1124070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303708,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068132,"statfs":{"total":96636764160,"available":96325767168,"internally_reserved":0,"allocated":2379776,"data_stored":1124706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.27600000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.505},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.54200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2957312,"data_stored":1714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.25}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303692,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068148,"statfs":{"total":96636764160,"available":96325783552,"internally_reserved":0,"allocated":2363392,"data_stored":1123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.317},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.33400000000000002,"15min":0.32100000000000001},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.374,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33100000000000002,"15min":0.33000000000000002},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.504,"15min":0.56599999999999995},"last":0.25700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.435,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.35899999999999999,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.86299999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.374,"15min":0.374},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.442},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.903},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303708,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068132,"statfs":{"total":96636764160,"available":96325767168,"internally_reserved":0,"allocated":2379776,"data_stored":1126048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.436,"5min":0.503,"15min":0.65200000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.49299999999999999,"15min":0.72299999999999998},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.313},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.59599999999999997,"15min":0.69699999999999995},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.47399999999999998,"5min":0.63,"15min":0.67600000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78800000000000003,"5min":0.79700000000000004,"15min":0.94399999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.44900000000000001,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.997},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.98499999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":1.0660000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303712,"kb_used_data":2328,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068128,"statfs":{"total":96636764160,"available":96325763072,"internally_reserved":0,"allocated":2383872,"data_stored":1125550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.37},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.625},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.434},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.34799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303692,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068148,"statfs":{"total":96636764160,"available":96325783552,"internally_reserved":0,"allocated":2363392,"data_stored":1123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":1.143,"15min":2.3599999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.59199999999999997},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.28100000000000003},"max":{"1min":0.73199999999999998,"5min":0.86299999999999999,"15min":2.371},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.82499999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":6.532,"15min":6.532},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.67600000000000005},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72899999999999998,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.65900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":5.524,"15min":5.524},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64100000000000001,"5min":0.78200000000000003,"15min":1.1990000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.44900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.052},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.175},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.145},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:53.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:10:53.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.305+0000 7ff279675700 1 Processor -- start 2023-10-06T03:10:53.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.306+0000 7ff279675700 1 -- start start 2023-10-06T03:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.306+0000 7ff279675700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff274068920 0x7ff27406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.306+0000 7ff279675700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27406ec70 con 0x7ff274068550 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.306+0000 7ff279675700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740b2e70 con 0x7ff27406f640 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.306+0000 7ff279675700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff274068d90 con 0x7ff274068920 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.306+0000 7ff26a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff274068920 0x7ff27406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff26a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff274068920 0x7ff27406f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37588/0 (socket says 172.21.15.92:37588) 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff272ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7ff274068550 0x7ff27415e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56228/0 (socket says 172.21.15.92:56228) 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff26a7fc700 1 -- 172.21.15.92:0/3379016100 learned_addr learned my addr 172.21.15.92:0/3379016100 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff26a7fc700 1 -- 172.21.15.92:0/3379016100 >> v1:172.21.15.136:6789/0 conn(0x7ff27406f640 legacy=0x7ff274159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff26a7fc700 1 -- 172.21.15.92:0/3379016100 >> v1:172.21.15.92:6789/0 conn(0x7ff274068550 legacy=0x7ff27415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff26a7fc700 1 -- 172.21.15.92:0/3379016100 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740b3150 con 0x7ff274068920 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff2727fc700 1 -- 172.21.15.92:0/3379016100 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646467347 0 0) 0x7ff27406ec70 con 0x7ff274068550 2023-10-06T03:10:53.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff26a7fc700 1 --2- 172.21.15.92:0/3379016100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27406f100 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0099b0 tx=0x7ff25c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea1e131710b4422 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff2727fc700 1 -- 172.21.15.92:0/3379016100 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00b860 con 0x7ff274068920 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff2727fc700 1 -- 172.21.15.92:0/3379016100 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00b9c0 con 0x7ff274068920 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.307+0000 7ff2727fc700 1 -- 172.21.15.92:0/3379016100 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00bd60 con 0x7ff274068920 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 -- 172.21.15.92:0/3379016100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 msgr2=0x7ff27406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 --2- 172.21.15.92:0/3379016100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27406f100 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0099b0 tx=0x7ff25c009250 comp rx=0 tx=0).stop 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 -- 172.21.15.92:0/3379016100 shutdown_connections 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 --2- 172.21.15.92:0/3379016100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27406f100 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 -- 172.21.15.92:0/3379016100 >> 172.21.15.92:0/3379016100 conn(0x7ff2740cd480 msgr2=0x7ff2740cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 -- 172.21.15.92:0/3379016100 shutdown_connections 2023-10-06T03:10:53.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.308+0000 7ff279675700 1 -- 172.21.15.92:0/3379016100 wait complete. 2023-10-06T03:10:53.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.309+0000 7ff279675700 1 Processor -- start 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.309+0000 7ff279675700 1 -- start start 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff274068550 0x7ff2740ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27416a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff27406f640 0x7ff27416ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff274152360 con 0x7ff274068920 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740abd10 con 0x7ff274068550 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27415cc50 con 0x7ff27406f640 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27416a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27416a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37602/0 (socket says 172.21.15.92:37602) 2023-10-06T03:10:53.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 -- 172.21.15.92:0/1879994706 learned_addr learned my addr 172.21.15.92:0/1879994706 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff272ffd700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff274068550 0x7ff2740ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff2737fe700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff27406f640 0x7ff27416ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 -- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff27406f640 msgr2=0x7ff27416ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff27406f640 0x7ff27416ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 -- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff274068550 msgr2=0x7ff2740ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff274068550 0x7ff2740ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 -- 172.21.15.92:0/1879994706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0094f0 con 0x7ff274068920 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26a7fc700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27416a610 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff25c006f50 tx=0x7ff25c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c01f070 con 0x7ff274068920 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c01a420 con 0x7ff274068920 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27415cf30 con 0x7ff274068920 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.310+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c01a800 con 0x7ff274068920 2023-10-06T03:10:53.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.311+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27415d210 con 0x7ff274068920 2023-10-06T03:10:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.311+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27404c3c0 con 0x7ff274068920 2023-10-06T03:10:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.312+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff25c01ab70 con 0x7ff274068920 2023-10-06T03:10:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.312+0000 7ff26b7fe700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff254067320 0x7ff2540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.312+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff25c086260 con 0x7ff274068920 2023-10-06T03:10:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.312+0000 7ff272ffd700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff254067320 0x7ff2540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.313+0000 7ff272ffd700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff254067320 0x7ff2540697d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff264007fd0 tx=0x7ff264007db0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:53.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.317+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff25c052d90 con 0x7ff274068920 2023-10-06T03:10:53.495 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.494+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff27406c990 con 0x7ff254067320 2023-10-06T03:10:53.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.502+0000 7ff26b7fe700 1 -- 172.21.15.92:0/1879994706 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311825 (secure 0 0 0) 0x7ff27406c990 con 0x7ff254067320 2023-10-06T03:10:53.507 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:53.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff254067320 msgr2=0x7ff2540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff254067320 0x7ff2540697d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff264007fd0 tx=0x7ff264007db0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 msgr2=0x7ff27416a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27416a610 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff25c006f50 tx=0x7ff25c017000 comp rx=0 tx=0).stop 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 shutdown_connections 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff254067320 0x7ff2540697d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff27406f640 0x7ff27416ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff274068550 0x7ff2740ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 --2- 172.21.15.92:0/1879994706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff274068920 0x7ff27416a610 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.505+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 >> 172.21.15.92:0/1879994706 conn(0x7ff2740cd480 msgr2=0x7ff2740aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.506+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 shutdown_connections 2023-10-06T03:10:53.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.506+0000 7ff279675700 1 -- 172.21.15.92:0/1879994706 wait complete. 2023-10-06T03:10:53.511 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:10:53.519 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2023-10-06T03:10:53.294951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2430760,"kb_used_data":19688,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752543960,"statfs":{"total":773094113280,"available":770605015040,"internally_reserved":0,"allocated":20160512,"data_stored":10177012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.701961"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611079+0000","last_change":"2023-10-06T03:10:44.608143+0000","last_active":"2023-10-06T03:10:44.611079+0000","last_peered":"2023-10-06T03:10:44.611079+0000","last_clean":"2023-10-06T03:10:44.611079+0000","last_became_active":"2023-10-06T03:10:44.607875+0000","last_became_peered":"2023-10-06T03:10:44.607875+0000","last_unstale":"2023-10-06T03:10:44.611079+0000","last_undegraded":"2023-10-06T03:10:44.611079+0000","last_fullsized":"2023-10-06T03:10:44.611079+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:15.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617809+0000","last_change":"2023-10-06T03:10:44.616908+0000","last_active":"2023-10-06T03:10:44.617809+0000","last_peered":"2023-10-06T03:10:44.617809+0000","last_clean":"2023-10-06T03:10:44.617809+0000","last_became_active":"2023-10-06T03:10:44.616555+0000","last_became_peered":"2023-10-06T03:10:44.616555+0000","last_unstale":"2023-10-06T03:10:44.617809+0000","last_undegraded":"2023-10-06T03:10:44.617809+0000","last_fullsized":"2023-10-06T03:10:44.617809+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:23.309599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611589+0000","last_change":"2023-10-06T03:10:44.605922+0000","last_active":"2023-10-06T03:10:44.611589+0000","last_peered":"2023-10-06T03:10:44.611589+0000","last_clean":"2023-10-06T03:10:44.611589+0000","last_became_active":"2023-10-06T03:10:44.605623+0000","last_became_peered":"2023-10-06T03:10:44.605623+0000","last_unstale":"2023-10-06T03:10:44.611589+0000","last_undegraded":"2023-10-06T03:10:44.611589+0000","last_fullsized":"2023-10-06T03:10:44.611589+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612282+0000","last_change":"2023-10-06T03:10:44.610178+0000","last_active":"2023-10-06T03:10:44.612282+0000","last_peered":"2023-10-06T03:10:44.612282+0000","last_clean":"2023-10-06T03:10:44.612282+0000","last_became_active":"2023-10-06T03:10:44.609892+0000","last_became_peered":"2023-10-06T03:10:44.609892+0000","last_unstale":"2023-10-06T03:10:44.612282+0000","last_undegraded":"2023-10-06T03:10:44.612282+0000","last_fullsized":"2023-10-06T03:10:44.612282+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612326+0000","last_change":"2023-10-06T03:10:44.608872+0000","last_active":"2023-10-06T03:10:44.612326+0000","last_peered":"2023-10-06T03:10:44.612326+0000","last_clean":"2023-10-06T03:10:44.612326+0000","last_became_active":"2023-10-06T03:10:44.608412+0000","last_became_peered":"2023-10-06T03:10:44.608412+0000","last_unstale":"2023-10-06T03:10:44.612326+0000","last_undegraded":"2023-10-06T03:10:44.612326+0000","last_fullsized":"2023-10-06T03:10:44.612326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:42.363268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.609968+0000","last_change":"2023-10-06T03:10:44.605533+0000","last_active":"2023-10-06T03:10:44.609968+0000","last_peered":"2023-10-06T03:10:44.609968+0000","last_clean":"2023-10-06T03:10:44.609968+0000","last_became_active":"2023-10-06T03:10:44.605254+0000","last_became_peered":"2023-10-06T03:10:44.605254+0000","last_unstale":"2023-10-06T03:10:44.609968+0000","last_undegraded":"2023-10-06T03:10:44.609968+0000","last_fullsized":"2023-10-06T03:10:44.609968+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:22.407090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614874+0000","last_change":"2023-10-06T03:10:44.613032+0000","last_active":"2023-10-06T03:10:44.614874+0000","last_peered":"2023-10-06T03:10:44.614874+0000","last_clean":"2023-10-06T03:10:44.614874+0000","last_became_active":"2023-10-06T03:10:44.612535+0000","last_became_peered":"2023-10-06T03:10:44.612535+0000","last_unstale":"2023-10-06T03:10:44.614874+0000","last_undegraded":"2023-10-06T03:10:44.614874+0000","last_fullsized":"2023-10-06T03:10:44.614874+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:27.693399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619444+0000","last_change":"2023-10-06T03:10:44.612966+0000","last_active":"2023-10-06T03:10:44.619444+0000","last_peered":"2023-10-06T03:10:44.619444+0000","last_clean":"2023-10-06T03:10:44.619444+0000","last_became_active":"2023-10-06T03:10:44.612484+0000","last_became_peered":"2023-10-06T03:10:44.612484+0000","last_unstale":"2023-10-06T03:10:44.619444+0000","last_undegraded":"2023-10-06T03:10:44.619444+0000","last_fullsized":"2023-10-06T03:10:44.619444+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:32.111948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615003+0000","last_change":"2023-10-06T03:10:44.609587+0000","last_active":"2023-10-06T03:10:44.615003+0000","last_peered":"2023-10-06T03:10:44.615003+0000","last_clean":"2023-10-06T03:10:44.615003+0000","last_became_active":"2023-10-06T03:10:44.609390+0000","last_became_peered":"2023-10-06T03:10:44.609390+0000","last_unstale":"2023-10-06T03:10:44.615003+0000","last_undegraded":"2023-10-06T03:10:44.615003+0000","last_fullsized":"2023-10-06T03:10:44.615003+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:33.614656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618712+0000","last_change":"2023-10-06T03:10:44.618093+0000","last_active":"2023-10-06T03:10:44.618712+0000","last_peered":"2023-10-06T03:10:44.618712+0000","last_clean":"2023-10-06T03:10:44.618712+0000","last_became_active":"2023-10-06T03:10:44.617856+0000","last_became_peered":"2023-10-06T03:10:44.617856+0000","last_unstale":"2023-10-06T03:10:44.618712+0000","last_undegraded":"2023-10-06T03:10:44.618712+0000","last_fullsized":"2023-10-06T03:10:44.618712+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:52.931574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.623816+0000","last_change":"2023-10-06T03:10:44.623316+0000","last_active":"2023-10-06T03:10:44.623816+0000","last_peered":"2023-10-06T03:10:44.623816+0000","last_clean":"2023-10-06T03:10:44.623816+0000","last_became_active":"2023-10-06T03:10:44.623058+0000","last_became_peered":"2023-10-06T03:10:44.623058+0000","last_unstale":"2023-10-06T03:10:44.623816+0000","last_undegraded":"2023-10-06T03:10:44.623816+0000","last_fullsized":"2023-10-06T03:10:44.623816+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:37.745837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606653+0000","last_change":"2023-10-06T03:10:44.604123+0000","last_active":"2023-10-06T03:10:44.606653+0000","last_peered":"2023-10-06T03:10:44.606653+0000","last_clean":"2023-10-06T03:10:44.606653+0000","last_became_active":"2023-10-06T03:10:44.603822+0000","last_became_peered":"2023-10-06T03:10:44.603822+0000","last_unstale":"2023-10-06T03:10:44.606653+0000","last_undegraded":"2023-10-06T03:10:44.606653+0000","last_fullsized":"2023-10-06T03:10:44.606653+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:50.256975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624806+0000","last_change":"2023-10-06T03:10:44.612705+0000","last_active":"2023-10-06T03:10:44.624806+0000","last_peered":"2023-10-06T03:10:44.624806+0000","last_clean":"2023-10-06T03:10:44.624806+0000","last_became_active":"2023-10-06T03:10:44.612459+0000","last_became_peered":"2023-10-06T03:10:44.612459+0000","last_unstale":"2023-10-06T03:10:44.624806+0000","last_undegraded":"2023-10-06T03:10:44.624806+0000","last_fullsized":"2023-10-06T03:10:44.624806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:52:31.032848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625100+0000","last_change":"2023-10-06T03:10:44.608588+0000","last_active":"2023-10-06T03:10:44.625100+0000","last_peered":"2023-10-06T03:10:44.625100+0000","last_clean":"2023-10-06T03:10:44.625100+0000","last_became_active":"2023-10-06T03:10:44.607707+0000","last_became_peered":"2023-10-06T03:10:44.607707+0000","last_unstale":"2023-10-06T03:10:44.625100+0000","last_undegraded":"2023-10-06T03:10:44.625100+0000","last_fullsized":"2023-10-06T03:10:44.625100+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:11.796662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624888+0000","last_change":"2023-10-06T03:10:44.612883+0000","last_active":"2023-10-06T03:10:44.624888+0000","last_peered":"2023-10-06T03:10:44.624888+0000","last_clean":"2023-10-06T03:10:44.624888+0000","last_became_active":"2023-10-06T03:10:44.612648+0000","last_became_peered":"2023-10-06T03:10:44.612648+0000","last_unstale":"2023-10-06T03:10:44.624888+0000","last_undegraded":"2023-10-06T03:10:44.624888+0000","last_fullsized":"2023-10-06T03:10:44.624888+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:40:21.015787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622176+0000","last_change":"2023-10-06T03:10:44.621118+0000","last_active":"2023-10-06T03:10:44.622176+0000","last_peered":"2023-10-06T03:10:44.622176+0000","last_clean":"2023-10-06T03:10:44.622176+0000","last_became_active":"2023-10-06T03:10:44.614783+0000","last_became_peered":"2023-10-06T03:10:44.614783+0000","last_unstale":"2023-10-06T03:10:44.622176+0000","last_undegraded":"2023-10-06T03:10:44.622176+0000","last_fullsized":"2023-10-06T03:10:44.622176+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:04.349135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616832+0000","last_change":"2023-10-06T03:10:44.607117+0000","last_active":"2023-10-06T03:10:44.616832+0000","last_peered":"2023-10-06T03:10:44.616832+0000","last_clean":"2023-10-06T03:10:44.616832+0000","last_became_active":"2023-10-06T03:10:44.606719+0000","last_became_peered":"2023-10-06T03:10:44.606719+0000","last_unstale":"2023-10-06T03:10:44.616832+0000","last_undegraded":"2023-10-06T03:10:44.616832+0000","last_fullsized":"2023-10-06T03:10:44.616832+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:20.569599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614899+0000","last_change":"2023-10-06T03:10:44.613168+0000","last_active":"2023-10-06T03:10:44.614899+0000","last_peered":"2023-10-06T03:10:44.614899+0000","last_clean":"2023-10-06T03:10:44.614899+0000","last_became_active":"2023-10-06T03:10:44.612843+0000","last_became_peered":"2023-10-06T03:10:44.612843+0000","last_unstale":"2023-10-06T03:10:44.614899+0000","last_undegraded":"2023-10-06T03:10:44.614899+0000","last_fullsized":"2023-10-06T03:10:44.614899+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:37.217250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619764+0000","last_change":"2023-10-06T03:10:44.619199+0000","last_active":"2023-10-06T03:10:44.619764+0000","last_peered":"2023-10-06T03:10:44.619764+0000","last_clean":"2023-10-06T03:10:44.619764+0000","last_became_active":"2023-10-06T03:10:44.618882+0000","last_became_peered":"2023-10-06T03:10:44.618882+0000","last_unstale":"2023-10-06T03:10:44.619764+0000","last_undegraded":"2023-10-06T03:10:44.619764+0000","last_fullsized":"2023-10-06T03:10:44.619764+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:15.442772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622081+0000","last_change":"2023-10-06T03:10:44.613713+0000","last_active":"2023-10-06T03:10:44.622081+0000","last_peered":"2023-10-06T03:10:44.622081+0000","last_clean":"2023-10-06T03:10:44.622081+0000","last_became_active":"2023-10-06T03:10:44.610828+0000","last_became_peered":"2023-10-06T03:10:44.610828+0000","last_unstale":"2023-10-06T03:10:44.622081+0000","last_undegraded":"2023-10-06T03:10:44.622081+0000","last_fullsized":"2023-10-06T03:10:44.622081+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:23.629899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615084+0000","last_change":"2023-10-06T03:10:44.612712+0000","last_active":"2023-10-06T03:10:44.615084+0000","last_peered":"2023-10-06T03:10:44.615084+0000","last_clean":"2023-10-06T03:10:44.615084+0000","last_became_active":"2023-10-06T03:10:44.612456+0000","last_became_peered":"2023-10-06T03:10:44.612456+0000","last_unstale":"2023-10-06T03:10:44.615084+0000","last_undegraded":"2023-10-06T03:10:44.615084+0000","last_fullsized":"2023-10-06T03:10:44.615084+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:19.393529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608598+0000","last_change":"2023-10-06T03:10:44.606764+0000","last_active":"2023-10-06T03:10:44.608598+0000","last_peered":"2023-10-06T03:10:44.608598+0000","last_clean":"2023-10-06T03:10:44.608598+0000","last_became_active":"2023-10-06T03:10:44.606218+0000","last_became_peered":"2023-10-06T03:10:44.606218+0000","last_unstale":"2023-10-06T03:10:44.608598+0000","last_undegraded":"2023-10-06T03:10:44.608598+0000","last_fullsized":"2023-10-06T03:10:44.608598+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:00.320153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613821+0000","last_change":"2023-10-06T03:10:44.610117+0000","last_active":"2023-10-06T03:10:44.613821+0000","last_peered":"2023-10-06T03:10:44.613821+0000","last_clean":"2023-10-06T03:10:44.613821+0000","last_became_active":"2023-10-06T03:10:44.609777+0000","last_became_peered":"2023-10-06T03:10:44.609777+0000","last_unstale":"2023-10-06T03:10:44.613821+0000","last_undegraded":"2023-10-06T03:10:44.613821+0000","last_fullsized":"2023-10-06T03:10:44.613821+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:04.949522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617323+0000","last_change":"2023-10-06T03:10:44.611318+0000","last_active":"2023-10-06T03:10:44.617323+0000","last_peered":"2023-10-06T03:10:44.617323+0000","last_clean":"2023-10-06T03:10:44.617323+0000","last_became_active":"2023-10-06T03:10:44.610495+0000","last_became_peered":"2023-10-06T03:10:44.610495+0000","last_unstale":"2023-10-06T03:10:44.617323+0000","last_undegraded":"2023-10-06T03:10:44.617323+0000","last_fullsized":"2023-10-06T03:10:44.617323+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:00.253800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615621+0000","last_change":"2023-10-06T03:10:44.613771+0000","last_active":"2023-10-06T03:10:44.615621+0000","last_peered":"2023-10-06T03:10:44.615621+0000","last_clean":"2023-10-06T03:10:44.615621+0000","last_became_active":"2023-10-06T03:10:44.613387+0000","last_became_peered":"2023-10-06T03:10:44.613387+0000","last_unstale":"2023-10-06T03:10:44.615621+0000","last_undegraded":"2023-10-06T03:10:44.615621+0000","last_fullsized":"2023-10-06T03:10:44.615621+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:40:15.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"185'4","reported_seq":14,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626960+0000","last_change":"2023-10-06T03:10:43.602925+0000","last_active":"2023-10-06T03:10:44.626960+0000","last_peered":"2023-10-06T03:10:44.626960+0000","last_clean":"2023-10-06T03:10:44.626960+0000","last_became_active":"2023-10-06T03:10:43.602673+0000","last_became_peered":"2023-10-06T03:10:43.602673+0000","last_unstale":"2023-10-06T03:10:44.626960+0000","last_undegraded":"2023-10-06T03:10:44.626960+0000","last_fullsized":"2023-10-06T03:10:44.626960+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624836+0000","last_change":"2023-10-06T03:10:44.612058+0000","last_active":"2023-10-06T03:10:44.624836+0000","last_peered":"2023-10-06T03:10:44.624836+0000","last_clean":"2023-10-06T03:10:44.624836+0000","last_became_active":"2023-10-06T03:10:44.611816+0000","last_became_peered":"2023-10-06T03:10:44.611816+0000","last_unstale":"2023-10-06T03:10:44.624836+0000","last_undegraded":"2023-10-06T03:10:44.624836+0000","last_fullsized":"2023-10-06T03:10:44.624836+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:07.945301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626677+0000","last_change":"2023-10-06T03:10:43.603054+0000","last_active":"2023-10-06T03:10:44.626677+0000","last_peered":"2023-10-06T03:10:44.626677+0000","last_clean":"2023-10-06T03:10:44.626677+0000","last_became_active":"2023-10-06T03:10:43.602849+0000","last_became_peered":"2023-10-06T03:10:43.602849+0000","last_unstale":"2023-10-06T03:10:44.626677+0000","last_undegraded":"2023-10-06T03:10:44.626677+0000","last_fullsized":"2023-10-06T03:10:44.626677+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:44.214194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612802+0000","last_change":"2023-10-06T03:10:44.608006+0000","last_active":"2023-10-06T03:10:44.612802+0000","last_peered":"2023-10-06T03:10:44.612802+0000","last_clean":"2023-10-06T03:10:44.612802+0000","last_became_active":"2023-10-06T03:10:44.607700+0000","last_became_peered":"2023-10-06T03:10:44.607700+0000","last_unstale":"2023-10-06T03:10:44.612802+0000","last_undegraded":"2023-10-06T03:10:44.612802+0000","last_fullsized":"2023-10-06T03:10:44.612802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:29.630492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622258+0000","last_change":"2023-10-06T03:10:43.604672+0000","last_active":"2023-10-06T03:10:44.622258+0000","last_peered":"2023-10-06T03:10:44.622258+0000","last_clean":"2023-10-06T03:10:44.622258+0000","last_became_active":"2023-10-06T03:10:43.602513+0000","last_became_peered":"2023-10-06T03:10:43.602513+0000","last_unstale":"2023-10-06T03:10:44.622258+0000","last_undegraded":"2023-10-06T03:10:44.622258+0000","last_fullsized":"2023-10-06T03:10:44.622258+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:36.206746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606511+0000","last_change":"2023-10-06T03:10:44.603518+0000","last_active":"2023-10-06T03:10:44.606511+0000","last_peered":"2023-10-06T03:10:44.606511+0000","last_clean":"2023-10-06T03:10:44.606511+0000","last_became_active":"2023-10-06T03:10:44.603255+0000","last_became_peered":"2023-10-06T03:10:44.603255+0000","last_unstale":"2023-10-06T03:10:44.606511+0000","last_undegraded":"2023-10-06T03:10:44.606511+0000","last_fullsized":"2023-10-06T03:10:44.606511+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626714+0000","last_change":"2023-10-06T03:10:43.609088+0000","last_active":"2023-10-06T03:10:44.626714+0000","last_peered":"2023-10-06T03:10:44.626714+0000","last_clean":"2023-10-06T03:10:44.626714+0000","last_became_active":"2023-10-06T03:10:43.608821+0000","last_became_peered":"2023-10-06T03:10:43.608821+0000","last_unstale":"2023-10-06T03:10:44.626714+0000","last_undegraded":"2023-10-06T03:10:44.626714+0000","last_fullsized":"2023-10-06T03:10:44.626714+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:21:36.735333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608618+0000","last_change":"2023-10-06T03:10:44.606922+0000","last_active":"2023-10-06T03:10:44.608618+0000","last_peered":"2023-10-06T03:10:44.608618+0000","last_clean":"2023-10-06T03:10:44.608618+0000","last_became_active":"2023-10-06T03:10:44.606522+0000","last_became_peered":"2023-10-06T03:10:44.606522+0000","last_unstale":"2023-10-06T03:10:44.608618+0000","last_undegraded":"2023-10-06T03:10:44.608618+0000","last_fullsized":"2023-10-06T03:10:44.608618+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:15.658864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.620526+0000","last_change":"2023-10-06T03:10:43.610186+0000","last_active":"2023-10-06T03:10:44.620526+0000","last_peered":"2023-10-06T03:10:44.620526+0000","last_clean":"2023-10-06T03:10:44.620526+0000","last_became_active":"2023-10-06T03:10:43.609855+0000","last_became_peered":"2023-10-06T03:10:43.609855+0000","last_unstale":"2023-10-06T03:10:44.620526+0000","last_undegraded":"2023-10-06T03:10:44.620526+0000","last_fullsized":"2023-10-06T03:10:44.620526+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613890+0000","last_change":"2023-10-06T03:10:44.608127+0000","last_active":"2023-10-06T03:10:44.613890+0000","last_peered":"2023-10-06T03:10:44.613890+0000","last_clean":"2023-10-06T03:10:44.613890+0000","last_became_active":"2023-10-06T03:10:44.607790+0000","last_became_peered":"2023-10-06T03:10:44.607790+0000","last_unstale":"2023-10-06T03:10:44.613890+0000","last_undegraded":"2023-10-06T03:10:44.613890+0000","last_fullsized":"2023-10-06T03:10:44.613890+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:03.796183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.619316+0000","last_change":"2023-10-06T03:10:43.606849+0000","last_active":"2023-10-06T03:10:44.619316+0000","last_peered":"2023-10-06T03:10:44.619316+0000","last_clean":"2023-10-06T03:10:44.619316+0000","last_became_active":"2023-10-06T03:10:43.606628+0000","last_became_peered":"2023-10-06T03:10:43.606628+0000","last_unstale":"2023-10-06T03:10:44.619316+0000","last_undegraded":"2023-10-06T03:10:44.619316+0000","last_fullsized":"2023-10-06T03:10:44.619316+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:37.755076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615572+0000","last_change":"2023-10-06T03:10:44.609219+0000","last_active":"2023-10-06T03:10:44.615572+0000","last_peered":"2023-10-06T03:10:44.615572+0000","last_clean":"2023-10-06T03:10:44.615572+0000","last_became_active":"2023-10-06T03:10:44.608960+0000","last_became_peered":"2023-10-06T03:10:44.608960+0000","last_unstale":"2023-10-06T03:10:44.615572+0000","last_undegraded":"2023-10-06T03:10:44.615572+0000","last_fullsized":"2023-10-06T03:10:44.615572+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:58.313141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597171+0000","last_change":"2023-10-06T03:10:43.617619+0000","last_active":"2023-10-06T03:10:44.597171+0000","last_peered":"2023-10-06T03:10:44.597171+0000","last_clean":"2023-10-06T03:10:44.597171+0000","last_became_active":"2023-10-06T03:10:43.617376+0000","last_became_peered":"2023-10-06T03:10:43.617376+0000","last_unstale":"2023-10-06T03:10:44.597171+0000","last_undegraded":"2023-10-06T03:10:44.597171+0000","last_fullsized":"2023-10-06T03:10:44.597171+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:25.490995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614923+0000","last_change":"2023-10-06T03:10:44.613231+0000","last_active":"2023-10-06T03:10:44.614923+0000","last_peered":"2023-10-06T03:10:44.614923+0000","last_clean":"2023-10-06T03:10:44.614923+0000","last_became_active":"2023-10-06T03:10:44.612945+0000","last_became_peered":"2023-10-06T03:10:44.612945+0000","last_unstale":"2023-10-06T03:10:44.614923+0000","last_undegraded":"2023-10-06T03:10:44.614923+0000","last_fullsized":"2023-10-06T03:10:44.614923+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:50.256975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602634+0000","last_change":"2023-10-06T03:10:43.608993+0000","last_active":"2023-10-06T03:10:44.602634+0000","last_peered":"2023-10-06T03:10:44.602634+0000","last_clean":"2023-10-06T03:10:44.602634+0000","last_became_active":"2023-10-06T03:10:43.608716+0000","last_became_peered":"2023-10-06T03:10:43.608716+0000","last_unstale":"2023-10-06T03:10:44.602634+0000","last_undegraded":"2023-10-06T03:10:44.602634+0000","last_fullsized":"2023-10-06T03:10:44.602634+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:13.569451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614842+0000","last_change":"2023-10-06T03:10:44.612005+0000","last_active":"2023-10-06T03:10:44.614842+0000","last_peered":"2023-10-06T03:10:44.614842+0000","last_clean":"2023-10-06T03:10:44.614842+0000","last_became_active":"2023-10-06T03:10:44.611698+0000","last_became_peered":"2023-10-06T03:10:44.611698+0000","last_unstale":"2023-10-06T03:10:44.614842+0000","last_undegraded":"2023-10-06T03:10:44.614842+0000","last_fullsized":"2023-10-06T03:10:44.614842+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:44.601383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603232+0000","last_change":"2023-10-06T03:10:43.608400+0000","last_active":"2023-10-06T03:10:44.603232+0000","last_peered":"2023-10-06T03:10:44.603232+0000","last_clean":"2023-10-06T03:10:44.603232+0000","last_became_active":"2023-10-06T03:10:43.607925+0000","last_became_peered":"2023-10-06T03:10:43.607925+0000","last_unstale":"2023-10-06T03:10:44.603232+0000","last_undegraded":"2023-10-06T03:10:44.603232+0000","last_fullsized":"2023-10-06T03:10:44.603232+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:36.739791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616814+0000","last_change":"2023-10-06T03:10:44.607010+0000","last_active":"2023-10-06T03:10:44.616814+0000","last_peered":"2023-10-06T03:10:44.616814+0000","last_clean":"2023-10-06T03:10:44.616814+0000","last_became_active":"2023-10-06T03:10:44.606570+0000","last_became_peered":"2023-10-06T03:10:44.606570+0000","last_unstale":"2023-10-06T03:10:44.616814+0000","last_undegraded":"2023-10-06T03:10:44.616814+0000","last_fullsized":"2023-10-06T03:10:44.616814+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:59.437351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597622+0000","last_change":"2023-10-06T03:10:43.611484+0000","last_active":"2023-10-06T03:10:44.597622+0000","last_peered":"2023-10-06T03:10:44.597622+0000","last_clean":"2023-10-06T03:10:44.597622+0000","last_became_active":"2023-10-06T03:10:43.611227+0000","last_became_peered":"2023-10-06T03:10:43.611227+0000","last_unstale":"2023-10-06T03:10:44.597622+0000","last_undegraded":"2023-10-06T03:10:44.597622+0000","last_fullsized":"2023-10-06T03:10:44.597622+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:43.590879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625995+0000","last_change":"2023-10-06T03:10:44.613128+0000","last_active":"2023-10-06T03:10:44.625995+0000","last_peered":"2023-10-06T03:10:44.625995+0000","last_clean":"2023-10-06T03:10:44.625995+0000","last_became_active":"2023-10-06T03:10:44.612674+0000","last_became_peered":"2023-10-06T03:10:44.612674+0000","last_unstale":"2023-10-06T03:10:44.625995+0000","last_undegraded":"2023-10-06T03:10:44.625995+0000","last_fullsized":"2023-10-06T03:10:44.625995+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:04.843808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"185'3","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622581+0000","last_change":"2023-10-06T03:10:43.610252+0000","last_active":"2023-10-06T03:10:44.622581+0000","last_peered":"2023-10-06T03:10:44.622581+0000","last_clean":"2023-10-06T03:10:44.622581+0000","last_became_active":"2023-10-06T03:10:43.609979+0000","last_became_peered":"2023-10-06T03:10:43.609979+0000","last_unstale":"2023-10-06T03:10:44.622581+0000","last_undegraded":"2023-10-06T03:10:44.622581+0000","last_fullsized":"2023-10-06T03:10:44.622581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:01:26.682895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618618+0000","last_change":"2023-10-06T03:10:44.617246+0000","last_active":"2023-10-06T03:10:44.618618+0000","last_peered":"2023-10-06T03:10:44.618618+0000","last_clean":"2023-10-06T03:10:44.618618+0000","last_became_active":"2023-10-06T03:10:44.616958+0000","last_became_peered":"2023-10-06T03:10:44.616958+0000","last_unstale":"2023-10-06T03:10:44.618618+0000","last_undegraded":"2023-10-06T03:10:44.618618+0000","last_fullsized":"2023-10-06T03:10:44.618618+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.761168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"185'6","reported_seq":16,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625574+0000","last_change":"2023-10-06T03:10:43.604507+0000","last_active":"2023-10-06T03:10:44.625574+0000","last_peered":"2023-10-06T03:10:44.625574+0000","last_clean":"2023-10-06T03:10:44.625574+0000","last_became_active":"2023-10-06T03:10:43.604261+0000","last_became_peered":"2023-10-06T03:10:43.604261+0000","last_unstale":"2023-10-06T03:10:44.625574+0000","last_undegraded":"2023-10-06T03:10:44.625574+0000","last_fullsized":"2023-10-06T03:10:44.625574+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:12.780687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617308+0000","last_change":"2023-10-06T03:10:44.612460+0000","last_active":"2023-10-06T03:10:44.617308+0000","last_peered":"2023-10-06T03:10:44.617308+0000","last_clean":"2023-10-06T03:10:44.617308+0000","last_became_active":"2023-10-06T03:10:44.612210+0000","last_became_peered":"2023-10-06T03:10:44.612210+0000","last_unstale":"2023-10-06T03:10:44.617308+0000","last_undegraded":"2023-10-06T03:10:44.617308+0000","last_fullsized":"2023-10-06T03:10:44.617308+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:10:42.518427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626985+0000","last_change":"2023-10-06T03:10:43.609477+0000","last_active":"2023-10-06T03:10:44.626985+0000","last_peered":"2023-10-06T03:10:44.626985+0000","last_clean":"2023-10-06T03:10:44.626985+0000","last_became_active":"2023-10-06T03:10:43.609213+0000","last_became_peered":"2023-10-06T03:10:43.609213+0000","last_unstale":"2023-10-06T03:10:44.626985+0000","last_undegraded":"2023-10-06T03:10:44.626985+0000","last_fullsized":"2023-10-06T03:10:44.626985+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:54:20.842732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613908+0000","last_change":"2023-10-06T03:10:44.609257+0000","last_active":"2023-10-06T03:10:44.613908+0000","last_peered":"2023-10-06T03:10:44.613908+0000","last_clean":"2023-10-06T03:10:44.613908+0000","last_became_active":"2023-10-06T03:10:44.609084+0000","last_became_peered":"2023-10-06T03:10:44.609084+0000","last_unstale":"2023-10-06T03:10:44.613908+0000","last_undegraded":"2023-10-06T03:10:44.613908+0000","last_fullsized":"2023-10-06T03:10:44.613908+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:55:42.637941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.594785+0000","last_change":"2023-10-06T03:10:43.611154+0000","last_active":"2023-10-06T03:10:44.594785+0000","last_peered":"2023-10-06T03:10:44.594785+0000","last_clean":"2023-10-06T03:10:44.594785+0000","last_became_active":"2023-10-06T03:10:43.610907+0000","last_became_peered":"2023-10-06T03:10:43.610907+0000","last_unstale":"2023-10-06T03:10:44.594785+0000","last_undegraded":"2023-10-06T03:10:44.594785+0000","last_fullsized":"2023-10-06T03:10:44.594785+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617353+0000","last_change":"2023-10-06T03:10:44.611635+0000","last_active":"2023-10-06T03:10:44.617353+0000","last_peered":"2023-10-06T03:10:44.617353+0000","last_clean":"2023-10-06T03:10:44.617353+0000","last_became_active":"2023-10-06T03:10:44.611050+0000","last_became_peered":"2023-10-06T03:10:44.611050+0000","last_unstale":"2023-10-06T03:10:44.617353+0000","last_undegraded":"2023-10-06T03:10:44.617353+0000","last_fullsized":"2023-10-06T03:10:44.617353+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:22:40.043675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602572+0000","last_change":"2023-10-06T03:10:43.608802+0000","last_active":"2023-10-06T03:10:44.602572+0000","last_peered":"2023-10-06T03:10:44.602572+0000","last_clean":"2023-10-06T03:10:44.602572+0000","last_became_active":"2023-10-06T03:10:43.608524+0000","last_became_peered":"2023-10-06T03:10:43.608524+0000","last_unstale":"2023-10-06T03:10:44.602572+0000","last_undegraded":"2023-10-06T03:10:44.602572+0000","last_fullsized":"2023-10-06T03:10:44.602572+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:28.221106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616261+0000","last_change":"2023-10-06T03:10:44.605209+0000","last_active":"2023-10-06T03:10:44.616261+0000","last_peered":"2023-10-06T03:10:44.616261+0000","last_clean":"2023-10-06T03:10:44.616261+0000","last_became_active":"2023-10-06T03:10:44.604955+0000","last_became_peered":"2023-10-06T03:10:44.604955+0000","last_unstale":"2023-10-06T03:10:44.616261+0000","last_undegraded":"2023-10-06T03:10:44.616261+0000","last_fullsized":"2023-10-06T03:10:44.616261+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:38.258343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"185'8","reported_seq":18,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.629219+0000","last_change":"2023-10-06T03:10:43.617468+0000","last_active":"2023-10-06T03:10:44.629219+0000","last_peered":"2023-10-06T03:10:44.629219+0000","last_clean":"2023-10-06T03:10:44.629219+0000","last_became_active":"2023-10-06T03:10:43.617179+0000","last_became_peered":"2023-10-06T03:10:43.617179+0000","last_unstale":"2023-10-06T03:10:44.629219+0000","last_undegraded":"2023-10-06T03:10:44.629219+0000","last_fullsized":"2023-10-06T03:10:44.629219+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:31.017050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613036+0000","last_change":"2023-10-06T03:10:44.606315+0000","last_active":"2023-10-06T03:10:44.613036+0000","last_peered":"2023-10-06T03:10:44.613036+0000","last_clean":"2023-10-06T03:10:44.613036+0000","last_became_active":"2023-10-06T03:10:44.605927+0000","last_became_peered":"2023-10-06T03:10:44.605927+0000","last_unstale":"2023-10-06T03:10:44.613036+0000","last_undegraded":"2023-10-06T03:10:44.613036+0000","last_fullsized":"2023-10-06T03:10:44.613036+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:03.205188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.621050+0000","last_change":"2023-10-06T03:10:43.607558+0000","last_active":"2023-10-06T03:10:44.621050+0000","last_peered":"2023-10-06T03:10:44.621050+0000","last_clean":"2023-10-06T03:10:44.621050+0000","last_became_active":"2023-10-06T03:10:43.607274+0000","last_became_peered":"2023-10-06T03:10:43.607274+0000","last_unstale":"2023-10-06T03:10:44.621050+0000","last_undegraded":"2023-10-06T03:10:44.621050+0000","last_fullsized":"2023-10-06T03:10:44.621050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:55.593599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618323+0000","last_change":"2023-10-06T03:10:44.614485+0000","last_active":"2023-10-06T03:10:44.618323+0000","last_peered":"2023-10-06T03:10:44.618323+0000","last_clean":"2023-10-06T03:10:44.618323+0000","last_became_active":"2023-10-06T03:10:44.614187+0000","last_became_peered":"2023-10-06T03:10:44.614187+0000","last_unstale":"2023-10-06T03:10:44.618323+0000","last_undegraded":"2023-10-06T03:10:44.618323+0000","last_fullsized":"2023-10-06T03:10:44.618323+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:23.629899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.621860+0000","last_change":"2023-10-06T03:10:43.608774+0000","last_active":"2023-10-06T03:10:44.621860+0000","last_peered":"2023-10-06T03:10:44.621860+0000","last_clean":"2023-10-06T03:10:44.621860+0000","last_became_active":"2023-10-06T03:10:43.608468+0000","last_became_peered":"2023-10-06T03:10:43.608468+0000","last_unstale":"2023-10-06T03:10:44.621860+0000","last_undegraded":"2023-10-06T03:10:44.621860+0000","last_fullsized":"2023-10-06T03:10:44.621860+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:49.246471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626701+0000","last_change":"2023-10-06T03:10:44.625520+0000","last_active":"2023-10-06T03:10:44.626701+0000","last_peered":"2023-10-06T03:10:44.626701+0000","last_clean":"2023-10-06T03:10:44.626701+0000","last_became_active":"2023-10-06T03:10:44.625234+0000","last_became_peered":"2023-10-06T03:10:44.625234+0000","last_unstale":"2023-10-06T03:10:44.626701+0000","last_undegraded":"2023-10-06T03:10:44.626701+0000","last_fullsized":"2023-10-06T03:10:44.626701+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:06.054091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"185'1","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622246+0000","last_change":"2023-10-06T03:10:43.609053+0000","last_active":"2023-10-06T03:10:44.622246+0000","last_peered":"2023-10-06T03:10:44.622246+0000","last_clean":"2023-10-06T03:10:44.622246+0000","last_became_active":"2023-10-06T03:10:43.608789+0000","last_became_peered":"2023-10-06T03:10:43.608789+0000","last_unstale":"2023-10-06T03:10:44.622246+0000","last_undegraded":"2023-10-06T03:10:44.622246+0000","last_fullsized":"2023-10-06T03:10:44.622246+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:36.389568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.604674+0000","last_change":"2023-10-06T03:10:44.602617+0000","last_active":"2023-10-06T03:10:44.604674+0000","last_peered":"2023-10-06T03:10:44.604674+0000","last_clean":"2023-10-06T03:10:44.604674+0000","last_became_active":"2023-10-06T03:10:44.602342+0000","last_became_peered":"2023-10-06T03:10:44.602342+0000","last_unstale":"2023-10-06T03:10:44.604674+0000","last_undegraded":"2023-10-06T03:10:44.604674+0000","last_fullsized":"2023-10-06T03:10:44.604674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:59.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.598698+0000","last_change":"2023-10-06T03:10:43.611124+0000","last_active":"2023-10-06T03:10:44.598698+0000","last_peered":"2023-10-06T03:10:44.598698+0000","last_clean":"2023-10-06T03:10:44.598698+0000","last_became_active":"2023-10-06T03:10:43.610877+0000","last_became_peered":"2023-10-06T03:10:43.610877+0000","last_unstale":"2023-10-06T03:10:44.598698+0000","last_undegraded":"2023-10-06T03:10:44.598698+0000","last_fullsized":"2023-10-06T03:10:44.598698+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:20:00.923932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.620606+0000","last_change":"2023-10-06T03:10:44.610074+0000","last_active":"2023-10-06T03:10:44.620606+0000","last_peered":"2023-10-06T03:10:44.620606+0000","last_clean":"2023-10-06T03:10:44.620606+0000","last_became_active":"2023-10-06T03:10:44.609795+0000","last_became_peered":"2023-10-06T03:10:44.609795+0000","last_unstale":"2023-10-06T03:10:44.620606+0000","last_undegraded":"2023-10-06T03:10:44.620606+0000","last_fullsized":"2023-10-06T03:10:44.620606+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:23.309599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618997+0000","last_change":"2023-10-06T03:10:43.607875+0000","last_active":"2023-10-06T03:10:44.618997+0000","last_peered":"2023-10-06T03:10:44.618997+0000","last_clean":"2023-10-06T03:10:44.618997+0000","last_became_active":"2023-10-06T03:10:43.607570+0000","last_became_peered":"2023-10-06T03:10:43.607570+0000","last_unstale":"2023-10-06T03:10:44.618997+0000","last_undegraded":"2023-10-06T03:10:44.618997+0000","last_fullsized":"2023-10-06T03:10:44.618997+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:05.143398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.622108+0000","last_change":"2023-10-06T03:10:44.614535+0000","last_active":"2023-10-06T03:10:44.622108+0000","last_peered":"2023-10-06T03:10:44.622108+0000","last_clean":"2023-10-06T03:10:44.622108+0000","last_became_active":"2023-10-06T03:10:44.614151+0000","last_became_peered":"2023-10-06T03:10:44.614151+0000","last_unstale":"2023-10-06T03:10:44.622108+0000","last_undegraded":"2023-10-06T03:10:44.622108+0000","last_fullsized":"2023-10-06T03:10:44.622108+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:42.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595616+0000","last_change":"2023-10-06T03:10:43.611649+0000","last_active":"2023-10-06T03:10:44.595616+0000","last_peered":"2023-10-06T03:10:44.595616+0000","last_clean":"2023-10-06T03:10:44.595616+0000","last_became_active":"2023-10-06T03:10:43.611297+0000","last_became_peered":"2023-10-06T03:10:43.611297+0000","last_unstale":"2023-10-06T03:10:44.595616+0000","last_undegraded":"2023-10-06T03:10:44.595616+0000","last_fullsized":"2023-10-06T03:10:44.595616+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:18.884730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.617340+0000","last_change":"2023-10-06T03:10:44.611469+0000","last_active":"2023-10-06T03:10:44.617340+0000","last_peered":"2023-10-06T03:10:44.617340+0000","last_clean":"2023-10-06T03:10:44.617340+0000","last_became_active":"2023-10-06T03:10:44.610791+0000","last_became_peered":"2023-10-06T03:10:44.610791+0000","last_unstale":"2023-10-06T03:10:44.617340+0000","last_undegraded":"2023-10-06T03:10:44.617340+0000","last_fullsized":"2023-10-06T03:10:44.617340+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:19.393529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602668+0000","last_change":"2023-10-06T03:10:43.610052+0000","last_active":"2023-10-06T03:10:44.602668+0000","last_peered":"2023-10-06T03:10:44.602668+0000","last_clean":"2023-10-06T03:10:44.602668+0000","last_became_active":"2023-10-06T03:10:43.609757+0000","last_became_peered":"2023-10-06T03:10:43.609757+0000","last_unstale":"2023-10-06T03:10:44.602668+0000","last_undegraded":"2023-10-06T03:10:44.602668+0000","last_fullsized":"2023-10-06T03:10:44.602668+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:20.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.614717+0000","last_change":"2023-10-06T03:10:44.608535+0000","last_active":"2023-10-06T03:10:44.614717+0000","last_peered":"2023-10-06T03:10:44.614717+0000","last_clean":"2023-10-06T03:10:44.614717+0000","last_became_active":"2023-10-06T03:10:44.607437+0000","last_became_peered":"2023-10-06T03:10:44.607437+0000","last_unstale":"2023-10-06T03:10:44.614717+0000","last_undegraded":"2023-10-06T03:10:44.614717+0000","last_fullsized":"2023-10-06T03:10:44.614717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:07.998111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.596401+0000","last_change":"2023-10-06T03:10:43.609073+0000","last_active":"2023-10-06T03:10:44.596401+0000","last_peered":"2023-10-06T03:10:44.596401+0000","last_clean":"2023-10-06T03:10:44.596401+0000","last_became_active":"2023-10-06T03:10:43.608677+0000","last_became_peered":"2023-10-06T03:10:43.608677+0000","last_unstale":"2023-10-06T03:10:44.596401+0000","last_undegraded":"2023-10-06T03:10:44.596401+0000","last_fullsized":"2023-10-06T03:10:44.596401+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:09.524657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.613017+0000","last_change":"2023-10-06T03:10:44.606162+0000","last_active":"2023-10-06T03:10:44.613017+0000","last_peered":"2023-10-06T03:10:44.613017+0000","last_clean":"2023-10-06T03:10:44.613017+0000","last_became_active":"2023-10-06T03:10:44.605602+0000","last_became_peered":"2023-10-06T03:10:44.605602+0000","last_unstale":"2023-10-06T03:10:44.613017+0000","last_undegraded":"2023-10-06T03:10:44.613017+0000","last_fullsized":"2023-10-06T03:10:44.613017+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:54.904132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"185'2","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.629090+0000","last_change":"2023-10-06T03:10:43.603495+0000","last_active":"2023-10-06T03:10:44.629090+0000","last_peered":"2023-10-06T03:10:44.629090+0000","last_clean":"2023-10-06T03:10:44.629090+0000","last_became_active":"2023-10-06T03:10:43.603249+0000","last_became_peered":"2023-10-06T03:10:43.603249+0000","last_unstale":"2023-10-06T03:10:44.629090+0000","last_undegraded":"2023-10-06T03:10:44.629090+0000","last_fullsized":"2023-10-06T03:10:44.629090+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:49.332478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616794+0000","last_change":"2023-10-06T03:10:44.606893+0000","last_active":"2023-10-06T03:10:44.616794+0000","last_peered":"2023-10-06T03:10:44.616794+0000","last_clean":"2023-10-06T03:10:44.616794+0000","last_became_active":"2023-10-06T03:10:44.606341+0000","last_became_peered":"2023-10-06T03:10:44.606341+0000","last_unstale":"2023-10-06T03:10:44.616794+0000","last_undegraded":"2023-10-06T03:10:44.616794+0000","last_fullsized":"2023-10-06T03:10:44.616794+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:27.791364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"186'1","reported_seq":11,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-06T03:10:45.874675+0000","last_change":"2023-10-06T03:10:43.617649+0000","last_active":"2023-10-06T03:10:45.874675+0000","last_peered":"2023-10-06T03:10:45.874675+0000","last_clean":"2023-10-06T03:10:45.874675+0000","last_became_active":"2023-10-06T03:10:43.617403+0000","last_became_peered":"2023-10-06T03:10:43.617403+0000","last_unstale":"2023-10-06T03:10:45.874675+0000","last_undegraded":"2023-10-06T03:10:45.874675+0000","last_fullsized":"2023-10-06T03:10:45.874675+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:17.771738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.612300+0000","last_change":"2023-10-06T03:10:44.610899+0000","last_active":"2023-10-06T03:10:44.612300+0000","last_peered":"2023-10-06T03:10:44.612300+0000","last_clean":"2023-10-06T03:10:44.612300+0000","last_became_active":"2023-10-06T03:10:44.610620+0000","last_became_peered":"2023-10-06T03:10:44.610620+0000","last_unstale":"2023-10-06T03:10:44.612300+0000","last_undegraded":"2023-10-06T03:10:44.612300+0000","last_fullsized":"2023-10-06T03:10:44.612300+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:38:24.072681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.608467+0000","last_change":"2023-10-06T03:10:43.609223+0000","last_active":"2023-10-06T03:10:44.608467+0000","last_peered":"2023-10-06T03:10:44.608467+0000","last_clean":"2023-10-06T03:10:44.608467+0000","last_became_active":"2023-10-06T03:10:43.606876+0000","last_became_peered":"2023-10-06T03:10:43.606876+0000","last_unstale":"2023-10-06T03:10:44.608467+0000","last_undegraded":"2023-10-06T03:10:44.608467+0000","last_fullsized":"2023-10-06T03:10:44.608467+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:07:04.476267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611431+0000","last_change":"2023-10-06T03:10:44.605653+0000","last_active":"2023-10-06T03:10:44.611431+0000","last_peered":"2023-10-06T03:10:44.611431+0000","last_clean":"2023-10-06T03:10:44.611431+0000","last_became_active":"2023-10-06T03:10:44.605289+0000","last_became_peered":"2023-10-06T03:10:44.605289+0000","last_unstale":"2023-10-06T03:10:44.611431+0000","last_undegraded":"2023-10-06T03:10:44.611431+0000","last_fullsized":"2023-10-06T03:10:44.611431+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:43.691771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603121+0000","last_change":"2023-10-06T03:10:43.611147+0000","last_active":"2023-10-06T03:10:44.603121+0000","last_peered":"2023-10-06T03:10:44.603121+0000","last_clean":"2023-10-06T03:10:44.603121+0000","last_became_active":"2023-10-06T03:10:43.610865+0000","last_became_peered":"2023-10-06T03:10:43.610865+0000","last_unstale":"2023-10-06T03:10:44.603121+0000","last_undegraded":"2023-10-06T03:10:44.603121+0000","last_fullsized":"2023-10-06T03:10:44.603121+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:58.825102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625117+0000","last_change":"2023-10-06T03:10:44.608741+0000","last_active":"2023-10-06T03:10:44.625117+0000","last_peered":"2023-10-06T03:10:44.625117+0000","last_clean":"2023-10-06T03:10:44.625117+0000","last_became_active":"2023-10-06T03:10:44.608014+0000","last_became_peered":"2023-10-06T03:10:44.608014+0000","last_unstale":"2023-10-06T03:10:44.625117+0000","last_undegraded":"2023-10-06T03:10:44.625117+0000","last_fullsized":"2023-10-06T03:10:44.625117+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:52.576251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595192+0000","last_change":"2023-10-06T03:10:43.611002+0000","last_active":"2023-10-06T03:10:44.595192+0000","last_peered":"2023-10-06T03:10:44.595192+0000","last_clean":"2023-10-06T03:10:44.595192+0000","last_became_active":"2023-10-06T03:10:43.610692+0000","last_became_peered":"2023-10-06T03:10:43.610692+0000","last_unstale":"2023-10-06T03:10:44.595192+0000","last_undegraded":"2023-10-06T03:10:44.595192+0000","last_fullsized":"2023-10-06T03:10:44.595192+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:20:15.647883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.624786+0000","last_change":"2023-10-06T03:10:44.607396+0000","last_active":"2023-10-06T03:10:44.624786+0000","last_peered":"2023-10-06T03:10:44.624786+0000","last_clean":"2023-10-06T03:10:44.624786+0000","last_became_active":"2023-10-06T03:10:44.607134+0000","last_became_peered":"2023-10-06T03:10:44.607134+0000","last_unstale":"2023-10-06T03:10:44.624786+0000","last_undegraded":"2023-10-06T03:10:44.624786+0000","last_fullsized":"2023-10-06T03:10:44.624786+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:49:11.054676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.594876+0000","last_change":"2023-10-06T03:10:43.607559+0000","last_active":"2023-10-06T03:10:44.594876+0000","last_peered":"2023-10-06T03:10:44.594876+0000","last_clean":"2023-10-06T03:10:44.594876+0000","last_became_active":"2023-10-06T03:10:43.607302+0000","last_became_peered":"2023-10-06T03:10:43.607302+0000","last_unstale":"2023-10-06T03:10:44.594876+0000","last_undegraded":"2023-10-06T03:10:44.594876+0000","last_fullsized":"2023-10-06T03:10:44.594876+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:29:31.540562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626242+0000","last_change":"2023-10-06T03:10:44.624362+0000","last_active":"2023-10-06T03:10:44.626242+0000","last_peered":"2023-10-06T03:10:44.626242+0000","last_clean":"2023-10-06T03:10:44.626242+0000","last_became_active":"2023-10-06T03:10:44.624101+0000","last_became_peered":"2023-10-06T03:10:44.624101+0000","last_unstale":"2023-10-06T03:10:44.626242+0000","last_undegraded":"2023-10-06T03:10:44.626242+0000","last_fullsized":"2023-10-06T03:10:44.626242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:05.039778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.599608+0000","last_change":"2023-10-06T03:10:43.605635+0000","last_active":"2023-10-06T03:10:44.599608+0000","last_peered":"2023-10-06T03:10:44.599608+0000","last_clean":"2023-10-06T03:10:44.599608+0000","last_became_active":"2023-10-06T03:10:43.605202+0000","last_became_peered":"2023-10-06T03:10:43.605202+0000","last_unstale":"2023-10-06T03:10:44.599608+0000","last_undegraded":"2023-10-06T03:10:44.599608+0000","last_fullsized":"2023-10-06T03:10:44.599608+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:13:32.594410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.606406+0000","last_change":"2023-10-06T03:10:44.604594+0000","last_active":"2023-10-06T03:10:44.606406+0000","last_peered":"2023-10-06T03:10:44.606406+0000","last_clean":"2023-10-06T03:10:44.606406+0000","last_became_active":"2023-10-06T03:10:44.604267+0000","last_became_peered":"2023-10-06T03:10:44.604267+0000","last_unstale":"2023-10-06T03:10:44.606406+0000","last_undegraded":"2023-10-06T03:10:44.606406+0000","last_fullsized":"2023-10-06T03:10:44.606406+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:03.816949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":10,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.601579+0000","last_change":"2023-10-06T03:10:43.608350+0000","last_active":"2023-10-06T03:10:44.601579+0000","last_peered":"2023-10-06T03:10:44.601579+0000","last_clean":"2023-10-06T03:10:44.601579+0000","last_became_active":"2023-10-06T03:10:43.608121+0000","last_became_peered":"2023-10-06T03:10:43.608121+0000","last_unstale":"2023-10-06T03:10:44.601579+0000","last_undegraded":"2023-10-06T03:10:44.601579+0000","last_fullsized":"2023-10-06T03:10:44.601579+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:42.575574+0000","last_clean_scrub_stamp":"2023-10-06T03:10:42.575574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:14.031939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626216+0000","last_change":"2023-10-06T03:10:44.613649+0000","last_active":"2023-10-06T03:10:44.626216+0000","last_peered":"2023-10-06T03:10:44.626216+0000","last_clean":"2023-10-06T03:10:44.626216+0000","last_became_active":"2023-10-06T03:10:44.613383+0000","last_became_peered":"2023-10-06T03:10:44.613383+0000","last_unstale":"2023-10-06T03:10:44.626216+0000","last_undegraded":"2023-10-06T03:10:44.626216+0000","last_fullsized":"2023-10-06T03:10:44.626216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:01.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":147,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.604838+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:10:44.604838+0000","last_peered":"2023-10-06T03:10:44.604838+0000","last_clean":"2023-10-06T03:10:44.604838+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:10:44.604838+0000","last_undegraded":"2023-10-06T03:10:44.604838+0000","last_fullsized":"2023-10-06T03:10:44.604838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615564+0000","last_change":"2023-10-06T03:10:44.613667+0000","last_active":"2023-10-06T03:10:44.615564+0000","last_peered":"2023-10-06T03:10:44.615564+0000","last_clean":"2023-10-06T03:10:44.615564+0000","last_became_active":"2023-10-06T03:10:44.613322+0000","last_became_peered":"2023-10-06T03:10:44.613322+0000","last_unstale":"2023-10-06T03:10:44.615564+0000","last_undegraded":"2023-10-06T03:10:44.615564+0000","last_fullsized":"2023-10-06T03:10:44.615564+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:01.094720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597815+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:10:44.597815+0000","last_peered":"2023-10-06T03:10:44.597815+0000","last_clean":"2023-10-06T03:10:44.597815+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:10:44.597815+0000","last_undegraded":"2023-10-06T03:10:44.597815+0000","last_fullsized":"2023-10-06T03:10:44.597815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.618962+0000","last_change":"2023-10-06T03:10:44.615738+0000","last_active":"2023-10-06T03:10:44.618962+0000","last_peered":"2023-10-06T03:10:44.618962+0000","last_clean":"2023-10-06T03:10:44.618962+0000","last_became_active":"2023-10-06T03:10:44.615457+0000","last_became_peered":"2023-10-06T03:10:44.615457+0000","last_unstale":"2023-10-06T03:10:44.618962+0000","last_undegraded":"2023-10-06T03:10:44.618962+0000","last_fullsized":"2023-10-06T03:10:44.618962+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:32.027554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":150,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.603180+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:10:44.603180+0000","last_peered":"2023-10-06T03:10:44.603180+0000","last_clean":"2023-10-06T03:10:44.603180+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:10:44.603180+0000","last_undegraded":"2023-10-06T03:10:44.603180+0000","last_fullsized":"2023-10-06T03:10:44.603180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.616994+0000","last_change":"2023-10-06T03:10:44.610773+0000","last_active":"2023-10-06T03:10:44.616994+0000","last_peered":"2023-10-06T03:10:44.616994+0000","last_clean":"2023-10-06T03:10:44.616994+0000","last_became_active":"2023-10-06T03:10:44.610515+0000","last_became_peered":"2023-10-06T03:10:44.610515+0000","last_unstale":"2023-10-06T03:10:44.616994+0000","last_undegraded":"2023-10-06T03:10:44.616994+0000","last_fullsized":"2023-10-06T03:10:44.616994+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:37.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":150,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602214+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:10:44.602214+0000","last_peered":"2023-10-06T03:10:44.602214+0000","last_clean":"2023-10-06T03:10:44.602214+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:10:44.602214+0000","last_undegraded":"2023-10-06T03:10:44.602214+0000","last_fullsized":"2023-10-06T03:10:44.602214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.615899+0000","last_change":"2023-10-06T03:10:44.614523+0000","last_active":"2023-10-06T03:10:44.615899+0000","last_peered":"2023-10-06T03:10:44.615899+0000","last_clean":"2023-10-06T03:10:44.615899+0000","last_became_active":"2023-10-06T03:10:44.614265+0000","last_became_peered":"2023-10-06T03:10:44.614265+0000","last_unstale":"2023-10-06T03:10:44.615899+0000","last_undegraded":"2023-10-06T03:10:44.615899+0000","last_fullsized":"2023-10-06T03:10:44.615899+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:32.111948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":303,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.602337+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:10:44.602337+0000","last_peered":"2023-10-06T03:10:44.602337+0000","last_clean":"2023-10-06T03:10:44.602337+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:10:44.602337+0000","last_undegraded":"2023-10-06T03:10:44.602337+0000","last_fullsized":"2023-10-06T03:10:44.602337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.597322+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:10:44.597322+0000","last_peered":"2023-10-06T03:10:44.597322+0000","last_clean":"2023-10-06T03:10:44.597322+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:10:44.597322+0000","last_undegraded":"2023-10-06T03:10:44.597322+0000","last_fullsized":"2023-10-06T03:10:44.597322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.625133+0000","last_change":"2023-10-06T03:10:44.608933+0000","last_active":"2023-10-06T03:10:44.625133+0000","last_peered":"2023-10-06T03:10:44.625133+0000","last_clean":"2023-10-06T03:10:44.625133+0000","last_became_active":"2023-10-06T03:10:44.608337+0000","last_became_peered":"2023-10-06T03:10:44.608337+0000","last_unstale":"2023-10-06T03:10:44.625133+0000","last_undegraded":"2023-10-06T03:10:44.625133+0000","last_fullsized":"2023-10-06T03:10:44.625133+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:12.581813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.599681+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:10:44.599681+0000","last_peered":"2023-10-06T03:10:44.599681+0000","last_clean":"2023-10-06T03:10:44.599681+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:10:44.599681+0000","last_undegraded":"2023-10-06T03:10:44.599681+0000","last_fullsized":"2023-10-06T03:10:44.599681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.611631+0000","last_change":"2023-10-06T03:10:44.610553+0000","last_active":"2023-10-06T03:10:44.611631+0000","last_peered":"2023-10-06T03:10:44.611631+0000","last_clean":"2023-10-06T03:10:44.611631+0000","last_became_active":"2023-10-06T03:10:44.610247+0000","last_became_peered":"2023-10-06T03:10:44.610247+0000","last_unstale":"2023-10-06T03:10:44.611631+0000","last_undegraded":"2023-10-06T03:10:44.611631+0000","last_fullsized":"2023-10-06T03:10:44.611631+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:28.278681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.595601+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:10:44.595601+0000","last_peered":"2023-10-06T03:10:44.595601+0000","last_clean":"2023-10-06T03:10:44.595601+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:10:44.595601+0000","last_undegraded":"2023-10-06T03:10:44.595601+0000","last_fullsized":"2023-10-06T03:10:44.595601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":9,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.626189+0000","last_change":"2023-10-06T03:10:44.623258+0000","last_active":"2023-10-06T03:10:44.626189+0000","last_peered":"2023-10-06T03:10:44.626189+0000","last_clean":"2023-10-06T03:10:44.626189+0000","last_became_active":"2023-10-06T03:10:44.622785+0000","last_became_peered":"2023-10-06T03:10:44.622785+0000","last_unstale":"2023-10-06T03:10:44.626189+0000","last_undegraded":"2023-10-06T03:10:44.626189+0000","last_fullsized":"2023-10-06T03:10:44.626189+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:10:43.586077+0000","last_clean_scrub_stamp":"2023-10-06T03:10:43.586077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:48.634468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-06T03:10:44.598553+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:10:44.598553+0000","last_peered":"2023-10-06T03:10:44.598553+0000","last_clean":"2023-10-06T03:10:44.598553+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:10:44.598553+0000","last_undegraded":"2023-10-06T03:10:44.598553+0000","last_fullsized":"2023-10-06T03:10:44.598553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304276,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067564,"statfs":{"total":96636764160,"available":96325185536,"internally_reserved":0,"allocated":2961408,"data_stored":1714396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.745,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64100000000000001,"5min":1.264,"15min":1.264},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":2.286,"15min":2.286},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":2.504,"15min":2.504},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":1.141,"15min":1.141},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":1.639,"15min":1.639},"last":0.40400000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82199999999999995,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.307,"15min":0.307},"max":{"1min":0.77800000000000002,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.376}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303700,"kb_used_data":2316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068140,"statfs":{"total":96636764160,"available":96325775360,"internally_reserved":0,"allocated":2371584,"data_stored":1124070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.72899999999999998,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.495,"15min":0.504},"min":{"1min":0.38600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.68899999999999995,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.755,"15min":0.755},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.52400000000000002},"min":{"1min":0.35199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.70299999999999996,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53300000000000003,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.72799999999999998,"5min":0.747,"15min":0.79000000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53400000000000003,"15min":0.55300000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.66900000000000004,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.68899999999999995,"5min":0.77400000000000002,"15min":0.78200000000000003},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.51000000000000001,"15min":0.53500000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.629}]},{"osd":5,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52200000000000002,"15min":0.54300000000000004},"min":{"1min":0.313,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.58799999999999997},"min":{"1min":0.40600000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.85199999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:09:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.58599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":0.86799999999999999},"last":0.64300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574963,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303708,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068132,"statfs":{"total":96636764160,"available":96325767168,"internally_reserved":0,"allocated":2379776,"data_stored":1124706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.33500000000000002},"min":{"1min":0.223,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55600000000000005,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.377,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.50800000000000001,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34100000000000003,"15min":0.33400000000000002},"min":{"1min":0.23799999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.57999999999999996,"5min":1.282,"15min":1.282},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33300000000000002,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48499999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.27600000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.33700000000000002},"min":{"1min":0.223,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.498,"5min":1.25,"15min":1.25},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.54700000000000004,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.57299999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.79400000000000004,"5min":1.302,"15min":1.302},"last":0.505},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56999999999999995,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.96299999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":1.363,"15min":5.0270000000000001},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59299999999999997},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.747,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.86499999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.499},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":1.379,"15min":1.379},"last":0.54200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304272,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067568,"statfs":{"total":96636764160,"available":96325189632,"internally_reserved":0,"allocated":2957312,"data_stored":1714326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.25}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.47499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.502},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.58499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303692,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068148,"statfs":{"total":96636764160,"available":96325783552,"internally_reserved":0,"allocated":2363392,"data_stored":1123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.317},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.33400000000000002,"15min":0.32100000000000001},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.374,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33100000000000002,"15min":0.33000000000000002},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.504,"15min":0.56599999999999995},"last":0.25700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.435,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.35899999999999999,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.86299999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.374,"15min":0.374},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.442},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.903},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303708,"kb_used_data":2324,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068132,"statfs":{"total":96636764160,"available":96325767168,"internally_reserved":0,"allocated":2379776,"data_stored":1126048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.436,"5min":0.503,"15min":0.65200000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.49299999999999999,"15min":0.72299999999999998},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.313},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.59599999999999997,"15min":0.69699999999999995},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.47399999999999998,"5min":0.63,"15min":0.67600000000000005},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78800000000000003,"5min":0.79700000000000004,"15min":0.94399999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.44900000000000001,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.997},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.98499999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":1.0660000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303712,"kb_used_data":2328,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068128,"statfs":{"total":96636764160,"available":96325763072,"internally_reserved":0,"allocated":2383872,"data_stored":1125550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51400000000000001,"15min":0.503},"min":{"1min":0.38,"5min":0.30299999999999999,"15min":0.30099999999999999},"max":{"1min":0.66700000000000004,"5min":0.73499999999999999,"15min":0.73599999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.56899999999999995},"min":{"1min":0.433,"5min":0.307,"15min":0.307},"max":{"1min":0.70699999999999996,"5min":0.78100000000000003,"15min":1.4410000000000001},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.501,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71899999999999997,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.37},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":1.0009999999999999},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.50900000000000001,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.69999999999999996,"5min":0.746,"15min":3},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51900000000000002,"15min":0.73899999999999999},"min":{"1min":0.42699999999999999,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.754,"5min":0.754,"15min":10.084},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54100000000000004,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.74299999999999999,"5min":0.77800000000000002,"15min":1.5029999999999999},"last":0.625},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.65700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.82199999999999995,"15min":4.9939999999999998},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.48199999999999998,"15min":0.50900000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.58999999999999997,"5min":0.752,"15min":0.83999999999999997},"last":0.434},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.57199999999999995},"min":{"1min":0.312,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70799999999999996,"5min":0.84899999999999998,"15min":1.54},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.88200000000000001,"15min":0.90500000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.54400000000000004,"15min":0.56599999999999995},"min":{"1min":0.34100000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.80700000000000005,"15min":0.90300000000000002},"last":0.34799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56299999999999994,"15min":0.58399999999999996},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.68400000000000005,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57699999999999996,"15min":0.60699999999999998},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.44800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303692,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068148,"statfs":{"total":96636764160,"available":96325783552,"internally_reserved":0,"allocated":2363392,"data_stored":1123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":1.143,"15min":2.3599999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.59199999999999997},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.28100000000000003},"max":{"1min":0.73199999999999998,"5min":0.86299999999999999,"15min":2.371},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.82499999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":6.532,"15min":6.532},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.67600000000000005},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72899999999999998,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.65900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":5.524,"15min":5.524},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64100000000000001,"5min":0.78200000000000003,"15min":1.1990000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.44900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.052},"last":0.95399999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.175},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.145},"last":0.93899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:10:53.526 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:10:53.526 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:10:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.678+0000 7fa14ba7d700 1 Processor -- start 2023-10-06T03:10:53.680 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.678+0000 7fa14ba7d700 1 -- start start 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.679+0000 7fa14ba7d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa144159950 0x7fa1440cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.679+0000 7fa14ba7d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa144069c70 0x7fa1440cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.679+0000 7fa14ba7d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1440ce380 con 0x7fa1440af320 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.679+0000 7fa14ba7d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1440cec20 con 0x7fa144069c70 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.679+0000 7fa14ba7d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440afa90 con 0x7fa144159950 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.679+0000 7fa14a01a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa144069c70 0x7fa1440cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa144069c70 0x7fa1440cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37610/0 (socket says 172.21.15.92:37610) 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 -- 172.21.15.92:0/767587288 learned_addr learned my addr 172.21.15.92:0/767587288 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:53.682 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa149819700 1 --2- 172.21.15.92:0/767587288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa144159950 0x7fa1440cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 -- 172.21.15.92:0/767587288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa144159950 msgr2=0x7fa1440cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 --2- 172.21.15.92:0/767587288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa144159950 0x7fa1440cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 -- 172.21.15.92:0/767587288 >> v1:172.21.15.136:6789/0 conn(0x7fa1440af320 legacy=0x7fa144069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 -- 172.21.15.92:0/767587288 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1440ce5d0 con 0x7fa144069c70 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa14a01a700 1 --2- 172.21.15.92:0/767587288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144069c70 0x7fa1440cf0b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0099b0 tx=0x7fa13c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608b5f6e3a208b12 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa148817700 1 -- 172.21.15.92:0/767587288 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c009ea0 con 0x7fa144069c70 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa148817700 1 -- 172.21.15.92:0/767587288 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c00b860 con 0x7fa144069c70 2023-10-06T03:10:53.683 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.680+0000 7fa148817700 1 -- 172.21.15.92:0/767587288 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00bc00 con 0x7fa144069c70 2023-10-06T03:10:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 -- 172.21.15.92:0/767587288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144069c70 msgr2=0x7fa1440cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/767587288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144069c70 0x7fa1440cf0b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0099b0 tx=0x7fa13c009250 comp rx=0 tx=0).stop 2023-10-06T03:10:53.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 -- 172.21.15.92:0/767587288 shutdown_connections 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/767587288 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144069c70 0x7fa1440cf0b0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/767587288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa144159950 0x7fa1440cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 -- 172.21.15.92:0/767587288 >> 172.21.15.92:0/767587288 conn(0x7fa144068e00 msgr2=0x7fa1440d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 -- 172.21.15.92:0/767587288 shutdown_connections 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.681+0000 7fa14ba7d700 1 -- 172.21.15.92:0/767587288 wait complete. 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.682+0000 7fa14ba7d700 1 Processor -- start 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.682+0000 7fa14ba7d700 1 -- start start 2023-10-06T03:10:53.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.682+0000 7fa14ba7d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa144069c70 0x7fa1440b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:54.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.682+0000 7fa14ba7d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1440af320 0x7fa144151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:54.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.682+0000 7fa14ba7d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 0x7fa144151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:54.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14ba7d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa144157fe0 con 0x7fa144159950 2023-10-06T03:10:54.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14ba7d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441571f0 con 0x7fa144069c70 2023-10-06T03:10:54.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14ba7d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441581f0 con 0x7fa1440af320 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 0x7fa144151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 0x7fa144151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37624/0 (socket says 172.21.15.92:37624) 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 -- 172.21.15.92:0/2661124315 learned_addr learned my addr 172.21.15.92:0/2661124315 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa149819700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa144069c70 0x7fa1440b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa149018700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1440af320 0x7fa144151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 -- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1440af320 msgr2=0x7fa144151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1440af320 0x7fa144151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 -- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa144069c70 msgr2=0x7fa1440b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa144069c70 0x7fa1440b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 -- 172.21.15.92:0/2661124315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0094f0 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa149819700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa144069c70 0x7fa1440b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14a01a700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 0x7fa144151ff0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa144070e20 tx=0x7fa13c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c025420 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c0255a0 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1441584d0 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c025960 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.683+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa144156e00 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.684+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1440c9df0 con 0x7fa144159950 2023-10-06T03:10:54.567 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.685+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa13c025d50 con 0x7fa144159950 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.685+0000 7fa139ffb700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa128067320 0x7fa1280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.685+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa13c086760 con 0x7fa144159950 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.685+0000 7fa149819700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa128067320 0x7fa1280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.686+0000 7fa149819700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa128067320 0x7fa1280697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa140007620 tx=0x7fa14000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.689+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1440c9df0 con 0x7fa144159950 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.948+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa1440c9df0 con 0x7fa144159950 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.951+0000 7fa139ffb700 1 -- 172.21.15.92:0/2661124315 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7fa1440c9df0 con 0x7fa144159950 2023-10-06T03:10:54.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa128067320 msgr2=0x7fa1280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa128067320 0x7fa1280697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa140007620 tx=0x7fa14000b040 comp rx=0 tx=0).stop 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 msgr2=0x7fa144151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 0x7fa144151ff0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fa144070e20 tx=0x7fa13c017000 comp rx=0 tx=0).stop 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 shutdown_connections 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa128067320 0x7fa1280697d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa144159950 0x7fa144151ff0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa144069c70 0x7fa1440b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 --2- 172.21.15.92:0/2661124315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1440af320 0x7fa144151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 >> 172.21.15.92:0/2661124315 conn(0x7fa144068e00 msgr2=0x7fa1440c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 shutdown_connections 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:53.954+0000 7fa14ba7d700 1 -- 172.21.15.92:0/2661124315 wait complete. 2023-10-06T03:10:54.571 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:10:54.572 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:10:54.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.687+0000 7f5e9896e700 1 Processor -- start 2023-10-06T03:10:54.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.687+0000 7f5e9896e700 1 -- start start 2023-10-06T03:10:54.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9896e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5e94159c60 0x7f5e9415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:54.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9896e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e9415db10 0x7f5e94069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:54.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9896e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9415a160 con 0x7f5e94159890 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9896e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9415df50 con 0x7f5e9415db10 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9896e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e940d13a0 con 0x7f5e94159c60 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e92d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5e9415db10 0x7f5e94069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5e94159890 0x7f5e9415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56244/0 (socket says 172.21.15.92:56244) 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e9259c700 1 -- 172.21.15.92:0/3668327280 learned_addr learned my addr 172.21.15.92:0/3668327280 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e89d9b700 1 --2- 172.21.15.92:0/3668327280 >> v2:172.21.15.136:3300/0 conn(0x7f5e94159c60 0x7f5e9415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e92d9d700 1 -- 172.21.15.92:0/3668327280 >> v2:172.21.15.136:3300/0 conn(0x7f5e94159c60 msgr2=0x7f5e9415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e92d9d700 1 --2- 172.21.15.92:0/3668327280 >> v2:172.21.15.136:3300/0 conn(0x7f5e94159c60 0x7f5e9415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e92d9d700 1 -- 172.21.15.92:0/3668327280 >> v1:172.21.15.92:6789/0 conn(0x7f5e94159890 legacy=0x7f5e9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e92d9d700 1 -- 172.21.15.92:0/3668327280 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9415e140 con 0x7f5e9415db10 2023-10-06T03:10:54.729 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.688+0000 7f5e91d9b700 1 -- 172.21.15.92:0/3668327280 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091407787 0 0) 0x7f5e9415a160 con 0x7f5e94159890 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e92d9d700 1 --2- 172.21.15.92:0/3668327280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9415db10 0x7f5e94069560 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5e940b67f0 tx=0x7f5e8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b030237f90174345 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e91d9b700 1 -- 172.21.15.92:0/3668327280 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c00e070 con 0x7f5e9415db10 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e91d9b700 1 -- 172.21.15.92:0/3668327280 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8c00bb60 con 0x7f5e9415db10 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e91d9b700 1 -- 172.21.15.92:0/3668327280 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c012430 con 0x7f5e9415db10 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 -- 172.21.15.92:0/3668327280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9415db10 msgr2=0x7f5e94069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 --2- 172.21.15.92:0/3668327280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9415db10 0x7f5e94069560 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5e940b67f0 tx=0x7f5e8c009780 comp rx=0 tx=0).stop 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 -- 172.21.15.92:0/3668327280 shutdown_connections 2023-10-06T03:10:54.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 --2- 172.21.15.92:0/3668327280 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e9415db10 0x7f5e94069560 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 --2- 172.21.15.92:0/3668327280 >> v2:172.21.15.136:3300/0 conn(0x7f5e94159c60 0x7f5e9415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 -- 172.21.15.92:0/3668327280 >> 172.21.15.92:0/3668327280 conn(0x7f5e94068e20 msgr2=0x7f5e940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 -- 172.21.15.92:0/3668327280 shutdown_connections 2023-10-06T03:10:55.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.689+0000 7f5e9896e700 1 -- 172.21.15.92:0/3668327280 wait complete. 2023-10-06T03:10:55.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.690+0000 7f5e9896e700 1 Processor -- start 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.690+0000 7f5e9896e700 1 -- start start 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9896e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e94159890 0x7f5e940cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9896e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 0x7f5e940cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9896e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9415db10 0x7f5e94148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9896e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940b0540 con 0x7f5e94159c60 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9896e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9416d490 con 0x7f5e9415db10 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9896e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9416d770 con 0x7f5e94159890 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e89d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 0x7f5e940cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e89d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 0x7f5e940cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37640/0 (socket says 172.21.15.92:37640) 2023-10-06T03:10:55.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e89d9b700 1 -- 172.21.15.92:0/2119888560 learned_addr learned my addr 172.21.15.92:0/2119888560 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e9259c700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e94159890 0x7f5e940cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.691+0000 7f5e92d9d700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9415db10 0x7f5e94148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e89d9b700 1 -- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e94159890 msgr2=0x7f5e940cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e89d9b700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e94159890 0x7f5e940cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e89d9b700 1 -- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9415db10 msgr2=0x7f5e94148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e89d9b700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9415db10 0x7f5e94148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e89d9b700 1 -- 172.21.15.92:0/2119888560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c009a20 con 0x7f5e94159c60 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e92d9d700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9415db10 0x7f5e94148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e89d9b700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 0x7f5e940cb040 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00ec80 tx=0x7f5e7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00c850 con 0x7f5e94159c60 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c00c9b0 con 0x7f5e94159c60 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9416da50 con 0x7f5e94159c60 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c00cd50 con 0x7f5e94159c60 2023-10-06T03:10:55.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.692+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e940cfce0 con 0x7f5e94159c60 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.693+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e9404c3c0 con 0x7f5e94159c60 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.693+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e7c013680 con 0x7f5e94159c60 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.694+0000 7f5e8affd700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e74067320 0x7f5e740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.694+0000 7f5e9259c700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e74067320 0x7f5e740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.694+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e7c085090 con 0x7f5e94159c60 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.694+0000 7f5e9259c700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e74067320 0x7f5e740697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5e84007fd0 tx=0x7f5e84007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.698+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e7c051bc0 con 0x7f5e94159c60 2023-10-06T03:10:55.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.922+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e9406a5b0 con 0x7f5e94159c60 2023-10-06T03:10:55.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.922+0000 7f5e8affd700 1 -- 172.21.15.92:0/2119888560 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4607 (secure 0 0 0) 0x7f5e7c0551e0 con 0x7f5e94159c60 2023-10-06T03:10:55.207 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:10:55.207 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:10:47.622584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:10:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e74067320 msgr2=0x7f5e740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e74067320 0x7f5e740697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5e84007fd0 tx=0x7f5e84007d30 comp rx=0 tx=0).stop 2023-10-06T03:10:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 msgr2=0x7f5e940cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.208 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 0x7f5e940cb040 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00ec80 tx=0x7f5e7c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.209 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 shutdown_connections 2023-10-06T03:10:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5e74067320 0x7f5e740697d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e9415db10 0x7f5e94148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e94159890 0x7f5e940cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 --2- 172.21.15.92:0/2119888560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5e94159c60 0x7f5e940cb040 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 >> 172.21.15.92:0/2119888560 conn(0x7f5e94068e20 msgr2=0x7f5e941552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 shutdown_connections 2023-10-06T03:10:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:54.924+0000 7f5e9896e700 1 -- 172.21.15.92:0/2119888560 wait complete. 2023-10-06T03:10:55.277 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 125 2023-10-06T03:10:55.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:10:43.974246+0000', 'modified': '2023-10-06T03:10:47.622584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8802}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8802': {'gid': 8802, 'name': 'a', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6837/427612348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 427612348}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 427612348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:10:55.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:10:55.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:10:55.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.799+0000 7fce37e6a700 1 Processor -- start 2023-10-06T03:10:55.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.799+0000 7fce37e6a700 1 -- start start 2023-10-06T03:10:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce37e6a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce3015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce37e6a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce3015db10 0x7fce30069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce37e6a700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3015a160 con 0x7fce30159890 2023-10-06T03:10:55.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce37e6a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300d13a0 con 0x7fce3015db10 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce37e6a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3015e140 con 0x7fce30159c60 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce36407700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce3015db10 0x7fce30069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce36407700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce3015db10 0x7fce30069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37654/0 (socket says 172.21.15.92:37654) 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.800+0000 7fce36407700 1 -- 172.21.15.92:0/452046143 learned_addr learned my addr 172.21.15.92:0/452046143 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce35405700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce3015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce36407700 1 -- 172.21.15.92:0/452046143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 msgr2=0x7fce3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce36407700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce3015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce36407700 1 -- 172.21.15.92:0/452046143 >> v1:172.21.15.136:6789/0 conn(0x7fce30159890 legacy=0x7fce3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce36407700 1 -- 172.21.15.92:0/452046143 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3015a3d0 con 0x7fce3015db10 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce35405700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce3015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:55.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce36407700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce30069560 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fce280099b0 tx=0x7fce28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57af3ba40465f4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce34c04700 1 -- 172.21.15.92:0/452046143 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800b820 con 0x7fce3015db10 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce34c04700 1 -- 172.21.15.92:0/452046143 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2800b980 con 0x7fce3015db10 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce34c04700 1 -- 172.21.15.92:0/452046143 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2800bd20 con 0x7fce3015db10 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 -- 172.21.15.92:0/452046143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 msgr2=0x7fce30069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce30069560 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fce280099b0 tx=0x7fce28009250 comp rx=0 tx=0).stop 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 -- 172.21.15.92:0/452046143 shutdown_connections 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce30069560 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 --2- 172.21.15.92:0/452046143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce3015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 -- 172.21.15.92:0/452046143 >> 172.21.15.92:0/452046143 conn(0x7fce30068e20 msgr2=0x7fce300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 -- 172.21.15.92:0/452046143 shutdown_connections 2023-10-06T03:10:55.805 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.801+0000 7fce37e6a700 1 -- 172.21.15.92:0/452046143 wait complete. 2023-10-06T03:10:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.802+0000 7fce37e6a700 1 Processor -- start 2023-10-06T03:10:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 -- start start 2023-10-06T03:10:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce30159890 0x7fce300ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce300c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.808 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce300b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300cdde0 con 0x7fce3015db10 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300ce080 con 0x7fce30159890 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce37e6a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300ce360 con 0x7fce30159c60 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce36407700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce300b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce36407700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce300b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37664/0 (socket says 172.21.15.92:37664) 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce36407700 1 -- 172.21.15.92:0/3138040608 learned_addr learned my addr 172.21.15.92:0/3138040608 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce35c06700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce30159890 0x7fce300ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.803+0000 7fce35405700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce300c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce36407700 1 -- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 msgr2=0x7fce300c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce36407700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce300c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce36407700 1 -- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce30159890 msgr2=0x7fce300ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.809 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce36407700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce30159890 0x7fce300ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce36407700 1 -- 172.21.15.92:0/3138040608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce280094f0 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce35405700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce300c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce36407700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce300b1600 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fce280094c0 tx=0x7fce28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce25ffb700 1 -- 172.21.15.92:0/3138040608 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2801b450 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce25ffb700 1 -- 172.21.15.92:0/3138040608 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce28020070 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300ce640 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce25ffb700 1 -- 172.21.15.92:0/3138040608 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2801b800 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.804+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300b2050 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.805+0000 7fce25ffb700 1 -- 172.21.15.92:0/3138040608 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fce28022030 con 0x7fce3015db10 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.806+0000 7fce25ffb700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fce14067320 0x7fce140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.810 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.806+0000 7fce25ffb700 1 -- 172.21.15.92:0/3138040608 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce28086cb0 con 0x7fce3015db10 2023-10-06T03:10:55.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.806+0000 7fce35c06700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fce14067320 0x7fce140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.811 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.806+0000 7fce35c06700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fce14067320 0x7fce140697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fce2c007fd0 tx=0x7fce2c0026a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 Processor -- start 2023-10-06T03:10:55.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 -- start start 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6ca5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6cf56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6cf6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6bfca70 con 0x55f3c6ca5840 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6bfa270 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.808+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6bf6060 con 0x55f3c6cf5c00 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6ca5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6ca5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37666/0 (socket says 172.21.15.92:37666) 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 -- 172.21.15.92:0/3759911028 learned_addr learned my addr 172.21.15.92:0/3759911028 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce36407700 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6cf6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35405700 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6cf56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 -- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 msgr2=0x55f3c6cf6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6cf6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 -- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 msgr2=0x55f3c6cf56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6cf56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 -- 172.21.15.92:0/3759911028 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3c6a90270 con 0x55f3c6ca5840 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35405700 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6cf56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce35c06700 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6ca5c10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00fe50 tx=0x7fce2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c31c1255c505e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce1affd700 1 -- 172.21.15.92:0/3759911028 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c003e00 con 0x55f3c6ca5840 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce1affd700 1 -- 172.21.15.92:0/3759911028 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c019070 con 0x55f3c6ca5840 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.809+0000 7fce1affd700 1 -- 172.21.15.92:0/3759911028 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c014d00 con 0x55f3c6ca5840 2023-10-06T03:10:55.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 -- 172.21.15.92:0/3759911028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 msgr2=0x55f3c6ca5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6ca5c10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00fe50 tx=0x7fce2c016000 comp rx=0 tx=0).stop 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 -- 172.21.15.92:0/3759911028 shutdown_connections 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6ca5c10 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6cf6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 --2- 172.21.15.92:0/3759911028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6cf56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 -- 172.21.15.92:0/3759911028 >> 172.21.15.92:0/3759911028 conn(0x55f3c6ca7160 msgr2=0x55f3c6ca7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 -- 172.21.15.92:0/3759911028 shutdown_connections 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 -- 172.21.15.92:0/3759911028 wait complete. 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 -- start start 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.810+0000 7fce41162b80 1 Processor -- start 2023-10-06T03:10:55.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca6cc0 0x55f3c6ce3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6ce3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6ba1890 con 0x55f3c6ca5840 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6bbcea0 con 0x55f3c6cf5c00 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a78e30 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce41162b80 10 client.0 _ll_register_callbacks cb 0x55f3c6c68e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.811+0000 7fce35c06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37672/0 (socket says 172.21.15.92:37672) 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 -- 172.21.15.92:0/3633018404 learned_addr learned my addr 172.21.15.92:0/3633018404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35405700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca6cc0 0x55f3c6ce3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce36407700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6ce3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 -- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca6cc0 msgr2=0x55f3c6ce3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca6cc0 0x55f3c6ce3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 -- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 msgr2=0x55f3c6ce3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6ce3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 -- 172.21.15.92:0/3633018404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c00fb00 con 0x55f3c6ca5840 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce35c06700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf4c10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fce2c014730 tx=0x7fce2c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5759d6dc38ec5170 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce0ffff700 1 -- 172.21.15.92:0/3633018404 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c020650 con 0x55f3c6ca5840 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce0ffff700 1 -- 172.21.15.92:0/3633018404 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce2c035c50 con 0x55f3c6ca5840 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3c6c9a940 con 0x55f3c6ca5840 2023-10-06T03:10:55.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce0ffff700 1 -- 172.21.15.92:0/3633018404 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce2c035e10 con 0x55f3c6ca5840 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce41162b80 10 client.8982 fetch_fsmap learned FSMap version 125 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55f3c6a32fe0 con 0x55f3c6ca5840 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce0ffff700 1 -- 172.21.15.92:0/3633018404 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3c6c9a940 con 0x55f3c6ca5840 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce0ffff700 1 -- 172.21.15.92:0/3633018404 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 125) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce2c0209c0 con 0x55f3c6ca5840 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.812+0000 7fce41162b80 10 client.8982 fetch_fsmap finished waiting for FSMap version 125 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce41162b80 10 client.8982 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce41162b80 20 client.8982 populate_metadata read hostname 'smithi092' 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce41162b80 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ce5f60 0x55f3c6ce6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce41162b80 4 client.8982 mds_command: new command op to 8802 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 --> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3c6c77870 con 0x55f3c6ce5f60 2023-10-06T03:10:55.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce35405700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ce5f60 0x55f3c6ce6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce0ffff700 10 client.8982 ms_handle_connect on v2:172.21.15.92:6835/427612348 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce35405700 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ce5f60 0x55f3c6ce6330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb01c2dd553d6fb9 server_cookie=7af82edf0bbb6fd7 in_seq=0 out_seq=0 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce0ffff700 1 -- 172.21.15.92:0/3633018404 <== mds.0 v2:172.21.15.92:6835/427612348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55f3c6c77870 con 0x55f3c6ce5f60 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce0ffff700 10 client.8982 handle_command_reply: tid=0 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.813+0000 7fce41162b80 1 client.8982 shutdown 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 msgr2=0x55f3c6cf4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf4c10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fce2c014730 tx=0x7fce2c020000 comp rx=0 tx=0).stop 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 shutdown_connections 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ce5f60 0x55f3c6ce6330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf4c10 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce0ffff700 0 client.8982 ms_handle_reset on v2:172.21.15.92:6835/427612348 2023-10-06T03:10:55.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6ce3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 --2- 172.21.15.92:0/3633018404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca6cc0 0x55f3c6ce3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 >> 172.21.15.92:0/3633018404 conn(0x55f3c6ca7160 msgr2=0x55f3c6ccea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 shutdown_connections 2023-10-06T03:10:55.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 1 -- 172.21.15.92:0/3633018404 wait complete. 2023-10-06T03:10:55.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.814+0000 7fce41162b80 20 client.8982 trim_cache size 0 max 16384 2023-10-06T03:10:55.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 Processor -- start 2023-10-06T03:10:55.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 -- start start 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ca1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6ca1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6ce6d50 con 0x55f3c6ca5840 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6cd5840 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6ca0c60 con 0x55f3c6cf5c00 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce35c06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce35c06700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37680/0 (socket says 172.21.15.92:37680) 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.831+0000 7fce35c06700 1 -- 172.21.15.92:0/1299700572 learned_addr learned my addr 172.21.15.92:0/1299700572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35405700 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ca1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35c06700 1 -- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 msgr2=0x55f3c6ca1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35c06700 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6ca1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35c06700 1 -- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 msgr2=0x55f3c6ca1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35c06700 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ca1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35c06700 1 -- 172.21.15.92:0/1299700572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3c6bc4780 con 0x55f3c6ca5840 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35405700 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ca1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce35c06700 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf00e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fce2c040050 tx=0x7fce2c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13108eb9fbd2c4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce1b7fe700 1 -- 172.21.15.92:0/1299700572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c014860 con 0x55f3c6ca5840 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce1b7fe700 1 -- 172.21.15.92:0/1299700572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c02f070 con 0x55f3c6ca5840 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce1b7fe700 1 -- 172.21.15.92:0/1299700572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c03e470 con 0x55f3c6ca5840 2023-10-06T03:10:55.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 -- 172.21.15.92:0/1299700572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 msgr2=0x55f3c6cf00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf00e0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fce2c040050 tx=0x7fce2c02d070 comp rx=0 tx=0).stop 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 -- 172.21.15.92:0/1299700572 shutdown_connections 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca5840 0x55f3c6cf00e0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6cf5c00 0x55f3c6ca1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 --2- 172.21.15.92:0/1299700572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ca1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 -- 172.21.15.92:0/1299700572 >> 172.21.15.92:0/1299700572 conn(0x55f3c6ca7160 msgr2=0x55f3c6ce4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 -- 172.21.15.92:0/1299700572 shutdown_connections 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.832+0000 7fce41162b80 1 -- 172.21.15.92:0/1299700572 wait complete. 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.833+0000 7fce41162b80 1 -- start start 2023-10-06T03:10:55.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.833+0000 7fce41162b80 1 Processor -- start 2023-10-06T03:10:55.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca5840 0x55f3c6ceb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ceb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6cebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6d0ae50 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6d0d400 con 0x55f3c6cf5c00 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6d099f0 con 0x55f3c6ca5840 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce41162b80 10 client.0 _ll_register_callbacks cb 0x55f3c6c68e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ceb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ceb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37688/0 (socket says 172.21.15.92:37688) 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 -- 172.21.15.92:0/745212314 learned_addr learned my addr 172.21.15.92:0/745212314 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce36407700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6cebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35c06700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca5840 0x55f3c6ceb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 -- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca5840 msgr2=0x55f3c6ceb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca5840 0x55f3c6ceb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 -- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 msgr2=0x55f3c6cebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6cebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 -- 172.21.15.92:0/745212314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3c6a32fe0 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce36407700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6cebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce35405700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ceb910 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x55f3c6ccf7e0 tx=0x7fce2001c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21c87b615b52717 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.834+0000 7fce0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce0ffff700 1 -- 172.21.15.92:0/745212314 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000ad90 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce0ffff700 1 -- 172.21.15.92:0/745212314 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce2001f460 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce2c019070 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce0ffff700 1 -- 172.21.15.92:0/745212314 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce2001f850 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 10 client.8988 fetch_fsmap learned FSMap version 125 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55f3c6bab470 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce0ffff700 1 -- 172.21.15.92:0/745212314 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c019070 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce0ffff700 1 -- 172.21.15.92:0/745212314 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 125) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce2001f9b0 con 0x55f3c6ca6cc0 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 10 client.8988 fetch_fsmap finished waiting for FSMap version 125 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 10 client.8988 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 20 client.8988 populate_metadata read hostname 'smithi092' 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ccaeb0 0x55f3c6ccb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 4 client.8988 mds_command: new command op to 8802 tid=0[{"prefix": "status"}] 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 --> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f3c6b98e00 con 0x55f3c6ccaeb0 2023-10-06T03:10:55.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce35c06700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ccaeb0 0x55f3c6ccb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce0ffff700 10 client.8988 ms_handle_connect on v2:172.21.15.92:6835/427612348 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.835+0000 7fce35c06700 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ccaeb0 0x55f3c6ccb280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c105f743594418d9 server_cookie=6bfc572cd78ef105 in_seq=0 out_seq=0 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce0ffff700 1 -- 172.21.15.92:0/745212314 <== mds.0 v2:172.21.15.92:6835/427612348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55f3c6b98e00 con 0x55f3c6ccaeb0 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce0ffff700 10 client.8988 handle_command_reply: tid=0 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 client.8988 shutdown 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 msgr2=0x55f3c6ceb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ceb910 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x55f3c6ccf7e0 tx=0x7fce2001c250 comp rx=0 tx=0).stop 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 shutdown_connections 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:6835/427612348,v1:172.21.15.92:6837/427612348] conn(0x55f3c6ccaeb0 0x55f3c6ccb280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c6cf5c00 0x55f3c6cebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce0ffff700 0 client.8988 ms_handle_reset on v2:172.21.15.92:6835/427612348 2023-10-06T03:10:55.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55f3c6ca5840 0x55f3c6ceb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 --2- 172.21.15.92:0/745212314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f3c6ca6cc0 0x55f3c6ceb910 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 >> 172.21.15.92:0/745212314 conn(0x55f3c6ca7160 msgr2=0x55f3c6cf02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 shutdown_connections 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 1 -- 172.21.15.92:0/745212314 wait complete. 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce41162b80 20 client.8988 trim_cache size 0 max 16384 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:10:55.845 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 8802, 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 11.244620388, 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 125, 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 186, 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 186, 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 23.336377921 2023-10-06T03:10:55.846 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:10:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fce14067320 msgr2=0x7fce140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fce14067320 0x7fce140697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fce2c007fd0 tx=0x7fce2c0026a0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 msgr2=0x7fce300b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce300b1600 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fce280094c0 tx=0x7fce28016000 comp rx=0 tx=0).stop 2023-10-06T03:10:55.847 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 shutdown_connections 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fce14067320 0x7fce140697d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce3015db10 0x7fce300b1600 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce30159890 0x7fce300ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 --2- 172.21.15.92:0/3138040608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce30159c60 0x7fce300c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.836+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 >> 172.21.15.92:0/3138040608 conn(0x7fce30068e20 msgr2=0x7fce300cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.837+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 shutdown_connections 2023-10-06T03:10:55.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.837+0000 7fce37e6a700 1 -- 172.21.15.92:0/3138040608 wait complete. 2023-10-06T03:10:55.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:10:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.963+0000 7f8305db1700 1 Processor -- start 2023-10-06T03:10:55.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.963+0000 7f8305db1700 1 -- start start 2023-10-06T03:10:55.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.964+0000 7f8305db1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83001599e0 0x7f8300069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.964+0000 7f8305db1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f8300069080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.964+0000 7f8305db1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000c96a0 con 0x7f830006a2b0 2023-10-06T03:10:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.964+0000 7f8305db1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000cb9e0 con 0x7f83001599e0 2023-10-06T03:10:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.964+0000 7f8305db1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000cab60 con 0x7f8300159db0 2023-10-06T03:10:55.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83001599e0 0x7f8300069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82feffd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f8300069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83001599e0 0x7f8300069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:44992/0 (socket says 172.21.15.92:44992) 2023-10-06T03:10:55.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 -- 172.21.15.92:0/1277199230 learned_addr learned my addr 172.21.15.92:0/1277199230 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82fffff700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f830006a2b0 0x7f830006a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:54452/0 (socket says 172.21.15.92:54452) 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 -- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 msgr2=0x7f8300069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 --2- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f8300069080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 -- 172.21.15.92:0/1277199230 >> v1:172.21.15.136:6789/0 conn(0x7f830006a2b0 legacy=0x7f830006a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 -- 172.21.15.92:0/1277199230 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000cb420 con 0x7f83001599e0 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82fe7fc700 1 -- 172.21.15.92:0/1277199230 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855090446 0 0) 0x7f83000c96a0 con 0x7f830006a2b0 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82ff7fe700 1 --2- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f8300069ac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009f20 tx=0x7f82f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13019381ccf07eba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82fe7fc700 1 -- 172.21.15.92:0/1277199230 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000e070 con 0x7f83001599e0 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82fe7fc700 1 -- 172.21.15.92:0/1277199230 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f000b9d0 con 0x7f83001599e0 2023-10-06T03:10:55.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.965+0000 7f82fe7fc700 1 -- 172.21.15.92:0/1277199230 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00092e0 con 0x7f83001599e0 2023-10-06T03:10:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 -- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 msgr2=0x7f8300069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 --2- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f8300069ac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009f20 tx=0x7f82f0009840 comp rx=0 tx=0).stop 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 -- 172.21.15.92:0/1277199230 shutdown_connections 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 --2- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f8300069ac0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 --2- 172.21.15.92:0/1277199230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f8300069080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 -- 172.21.15.92:0/1277199230 >> 172.21.15.92:0/1277199230 conn(0x7f83000c9860 msgr2=0x7f83000c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 -- 172.21.15.92:0/1277199230 shutdown_connections 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.966+0000 7f8305db1700 1 -- 172.21.15.92:0/1277199230 wait complete. 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.967+0000 7f8305db1700 1 Processor -- start 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.967+0000 7f8305db1700 1 -- start start 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f8305db1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 0x7f83000c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f8305db1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f830014e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f8305db1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f830014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f8305db1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c64d0 con 0x7f830006a2b0 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f8305db1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c5850 con 0x7f83001599e0 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f8305db1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c5b30 con 0x7f8300159db0 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 0x7f83000c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 0x7f83000c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37702/0 (socket says 172.21.15.92:37702) 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 -- 172.21.15.92:0/2848669814 learned_addr learned my addr 172.21.15.92:0/2848669814 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82feffd700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f830014e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82fffff700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f830014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 -- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 msgr2=0x7f830014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f830014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 -- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 msgr2=0x7f830014e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f830014e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:55.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82ff7fe700 1 -- 172.21.15.92:0/2848669814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0009ae0 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82feffd700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f830014e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.968+0000 7f82fffff700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f830014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.969+0000 7f82ff7fe700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 0x7f83000c4e70 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f82f000b610 tx=0x7f82f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.969+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0008a50 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.969+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000c5e10 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.969+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0008bb0 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.969+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0026460 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.969+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830016adf0 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.970+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830004c3c0 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.970+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f82f0003040 con 0x7f830006a2b0 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.970+0000 7f82e77fe700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f82ec067320 0x7f82ec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:55.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.971+0000 7f82feffd700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f82ec067320 0x7f82ec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.971+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f82f0086780 con 0x7f830006a2b0 2023-10-06T03:10:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.971+0000 7f82feffd700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f82ec067320 0x7f82ec0697d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f830006b3e0 tx=0x7f82f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:55.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:55.975+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82f0053250 con 0x7f830006a2b0 2023-10-06T03:10:56.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.198+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83000cb700 con 0x7f830006a2b0 2023-10-06T03:10:56.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.199+0000 7f82e77fe700 1 -- 172.21.15.92:0/2848669814 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v125) v1 ==== 53+0+73 (secure 0 0 0) 0x7f82f0056870 con 0x7f830006a2b0 2023-10-06T03:10:56.201 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f82ec067320 msgr2=0x7f82ec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f82ec067320 0x7f82ec0697d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f830006b3e0 tx=0x7f82f4008040 comp rx=0 tx=0).stop 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 msgr2=0x7f83000c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 0x7f83000c4e70 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f82f000b610 tx=0x7f82f0018000 comp rx=0 tx=0).stop 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 shutdown_connections 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f82ec067320 0x7f82ec0697d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f830006a2b0 0x7f83000c4e70 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8300159db0 0x7f830014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 --2- 172.21.15.92:0/2848669814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83001599e0 0x7f830014e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 >> 172.21.15.92:0/2848669814 conn(0x7f83000c9860 msgr2=0x7f83000c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:56.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 shutdown_connections 2023-10-06T03:10:56.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.201+0000 7f8305db1700 1 -- 172.21.15.92:0/2848669814 wait complete. 2023-10-06T03:10:56.214 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:10:56.214 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:10:56.215 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:10:56.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.215 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:10:56.230 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:56.231 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41950sec preferred_lft 41950sec 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:10:56.232 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:56.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.234 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:10:56.294 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:10:56.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.296 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:10:56.352 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:10:56.352 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:10:56.352 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:10:56.353 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:10:56.353 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:10:56.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.353 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:10:56.353 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:10:56.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:10:56.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:10:56.353 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:10:56.354 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:10:56.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.453 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:10:56.467 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:10:56.467 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:10:56.467 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:10:56.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.467 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:10:56.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:56.532 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:56.587 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:10:56.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:56.587 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:56.642 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:10:56.642 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:10:56.642 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:56.721 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:56.721 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:56.740 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:56.762 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:56.763 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:56.763 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:10:56.763 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:10:56.763 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:10:56.764 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:10:56.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:56.764 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:56.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:10:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.983+0000 7f638359e700 1 Processor -- start 2023-10-06T03:10:56.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.983+0000 7f638359e700 1 -- start start 2023-10-06T03:10:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f638359e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f638415d230 0x7f638415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f638359e700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63840b2980 con 0x7f63841597f0 2023-10-06T03:10:56.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f638359e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f638415db70 con 0x7f6384159420 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f638359e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63840b11d0 con 0x7f638415d230 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6379d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f63841597f0 0x7f63840b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56256/0 (socket says 172.21.15.92:56256) 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6379d9b700 1 -- 172.21.15.92:0/1438959209 learned_addr learned my addr 172.21.15.92:0/1438959209 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089654749 0 0) 0x7f63840b2980 con 0x7f63841597f0 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6382d9d700 1 --2- 172.21.15.92:0/1438959209 >> v2:172.21.15.136:3300/0 conn(0x7f638415d230 0x7f638415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6370003570 con 0x7f63841597f0 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3125787646 0 0) 0x7f6370003570 con 0x7f63841597f0 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 >> v2:172.21.15.136:3300/0 conn(0x7f638415d230 msgr2=0x7f638415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 --2- 172.21.15.92:0/1438959209 >> v2:172.21.15.136:3300/0 conn(0x7f638415d230 0x7f638415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 >> v1:172.21.15.136:6789/0 conn(0x7f6384159420 legacy=0x7f63840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63840b14b0 con 0x7f63841597f0 2023-10-06T03:10:56.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.984+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f636c001470 con 0x7f63841597f0 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 >> v1:172.21.15.92:6789/0 conn(0x7f63841597f0 legacy=0x7f63840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6370003a60 0x7f6370003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6370004810 0x7f6370004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63700052e0 0x7f637000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840b11d0 con 0x7f6370004810 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637000c190 con 0x7f6370003a60 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637000c350 con 0x7f63700052e0 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6370004810 0x7f6370004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f638259c700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6370003a60 0x7f6370003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6382d9d700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63700052e0 0x7f637000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 -- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63700052e0 msgr2=0x7f637000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63700052e0 0x7f637000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 -- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6370003a60 msgr2=0x7f6370003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6370003a60 0x7f6370003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 -- 172.21.15.92:0/1438959209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370003570 con 0x7f6370004810 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6382d9d700 1 -- 172.21.15.92:0/1438959209 reap_dead start 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6379d9b700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6370004810 0x7f6370004c80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f6384159c70 tx=0x7f638415dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baebc6ed2ef5d614 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c00cbb0 con 0x7f6370004810 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c00cd10 con 0x7f6370004810 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.985+0000 7f6381d9b700 1 -- 172.21.15.92:0/1438959209 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c005b70 con 0x7f6370004810 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 -- 172.21.15.92:0/1438959209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6370004810 msgr2=0x7f6370004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6370004810 0x7f6370004c80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f6384159c70 tx=0x7f638415dd00 comp rx=0 tx=0).stop 2023-10-06T03:10:56.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 -- 172.21.15.92:0/1438959209 shutdown_connections 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 --2- 172.21.15.92:0/1438959209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6370004810 0x7f6370004c80 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 -- 172.21.15.92:0/1438959209 >> 172.21.15.92:0/1438959209 conn(0x7f63840d0660 msgr2=0x7f63840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 -- 172.21.15.92:0/1438959209 shutdown_connections 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.986+0000 7f638359e700 1 -- 172.21.15.92:0/1438959209 wait complete. 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.987+0000 7f638359e700 1 Processor -- start 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.987+0000 7f638359e700 1 -- start start 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6370004810 0x7f638416c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f638416a5d0 0x7f638415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 0x7f63841693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63840c6d70 con 0x7f638415caf0 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638415a350 con 0x7f638416a5d0 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638415cf40 con 0x7f6370004810 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 0x7f63841693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 0x7f63841693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37706/0 (socket says 172.21.15.92:37706) 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 -- 172.21.15.92:0/399221378 learned_addr learned my addr 172.21.15.92:0/399221378 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6379d9b700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f638416a5d0 0x7f638415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638259c700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6370004810 0x7f638416c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 -- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6370004810 msgr2=0x7f638416c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6370004810 0x7f638416c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 -- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f638416a5d0 msgr2=0x7f638415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f638416a5d0 0x7f638415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 -- 172.21.15.92:0/399221378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384159d40 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638259c700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6370004810 0x7f638416c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f6382d9d700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 0x7f63841693c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f637c009fd0 tx=0x7f637c009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c002d10 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637c00f050 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638416a050 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637c014660 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.988+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638416a330 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.989+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638415a4e0 con 0x7f638415caf0 2023-10-06T03:10:56.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.990+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f637c003040 con 0x7f638415caf0 2023-10-06T03:10:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.990+0000 7f637affd700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6364067320 0x7f63640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.990+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f637c085220 con 0x7f638415caf0 2023-10-06T03:10:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.990+0000 7f638259c700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6364067320 0x7f63640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.991+0000 7f638259c700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6364067320 0x7f63640697d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6374000c00 tx=0x7f6374009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:56.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:56.994+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f637c0556d0 con 0x7f638415caf0 2023-10-06T03:10:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.218+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f638404c3c0 con 0x7f638415caf0 2023-10-06T03:10:57.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.218+0000 7f637affd700 1 -- 172.21.15.92:0/399221378 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v125) v1 ==== 53+0+73 (secure 0 0 0) 0x7f637c055260 con 0x7f638415caf0 2023-10-06T03:10:57.220 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:10:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6364067320 msgr2=0x7f63640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:57.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6364067320 0x7f63640697d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6374000c00 tx=0x7f6374009040 comp rx=0 tx=0).stop 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 msgr2=0x7f63841693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 0x7f63841693c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f637c009fd0 tx=0x7f637c009870 comp rx=0 tx=0).stop 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 shutdown_connections 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6364067320 0x7f63640697d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f638415caf0 0x7f63841693c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6370004810 0x7f638416c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 --2- 172.21.15.92:0/399221378 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f638416a5d0 0x7f638415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.223 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 >> 172.21.15.92:0/399221378 conn(0x7f63840d0660 msgr2=0x7f6384068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 shutdown_connections 2023-10-06T03:10:57.224 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.220+0000 7f638359e700 1 -- 172.21.15.92:0/399221378 wait complete. 2023-10-06T03:10:57.233 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:10:57.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:10:57.235 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:10:57.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.235 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:57.255 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:10:57.256 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 41950sec preferred_lft 41950sec 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:10:57.257 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:10:57.258 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:10:57.259 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:10:57.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.259 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:10:57.323 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:10:57.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.325 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:10:57.383 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:10:57.383 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:10:57.383 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:10:57.383 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:10:57.383 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:10:57.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.384 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:10:57.384 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:10:57.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:10:57.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:10:57.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:10:57.384 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:10:57.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.489 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:10:57.503 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:10:57.503 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:10:57.504 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:10:57.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.504 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:10:57.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:57.568 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:57.623 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:10:57.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:10:57.623 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:57.678 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:10:57.678 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:10:57.678 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:57.757 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:57.757 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:57.776 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:57.776 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:57.776 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:57.776 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:57.776 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:57.776 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:57.777 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:57.777 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:57.798 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:10:57.798 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:10:57.798 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:10:57.798 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:10:57.798 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:10:57.798 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:10:57.799 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:10:57.799 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:10:57.799 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:10:57.799 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:10:57.799 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:10:57.800 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:10:57.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:57.801 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:57.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:10:57.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.986+0000 7f12e36de700 1 Processor -- start 2023-10-06T03:10:57.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.986+0000 7f12e36de700 1 -- start start 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e36de700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f12dc1564e0 0x7f12dc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e36de700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12dc06b000 0x7f12dc06f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e36de700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc06a0b0 con 0x7f12dc1568b0 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e36de700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc0694a0 con 0x7f12dc06b000 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e36de700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc069780 con 0x7f12dc1564e0 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e1c7b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12dc06b000 0x7f12dc06f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e0c79700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f12dc1568b0 0x7f12dc06a8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56258/0 (socket says 172.21.15.92:56258) 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e0c79700 1 -- 172.21.15.92:0/163878160 learned_addr learned my addr 172.21.15.92:0/163878160 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:57.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e147a700 1 --2- 172.21.15.92:0/163878160 >> v2:172.21.15.136:3300/0 conn(0x7f12dc1564e0 0x7f12dc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e1c7b700 1 -- 172.21.15.92:0/163878160 >> v2:172.21.15.136:3300/0 conn(0x7f12dc1564e0 msgr2=0x7f12dc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e1c7b700 1 --2- 172.21.15.92:0/163878160 >> v2:172.21.15.136:3300/0 conn(0x7f12dc1564e0 0x7f12dc0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e1c7b700 1 -- 172.21.15.92:0/163878160 >> v1:172.21.15.92:6789/0 conn(0x7f12dc1568b0 legacy=0x7f12dc06a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12d3fff700 1 -- 172.21.15.92:0/163878160 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980463728 0 0) 0x7f12dc06a0b0 con 0x7f12dc1568b0 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e1c7b700 1 -- 172.21.15.92:0/163878160 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc069a60 con 0x7f12dc06b000 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12e1c7b700 1 --2- 172.21.15.92:0/163878160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc06b000 0x7f12dc06f0d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f12dc06b9a0 tx=0x7f12d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6c9031652cba45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12d3fff700 1 -- 172.21.15.92:0/163878160 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011630 con 0x7f12dc06b000 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12d3fff700 1 -- 172.21.15.92:0/163878160 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4011790 con 0x7f12dc06b000 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.987+0000 7f12d3fff700 1 -- 172.21.15.92:0/163878160 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011b30 con 0x7f12dc06b000 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 -- 172.21.15.92:0/163878160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc06b000 msgr2=0x7f12dc06f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:57.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 --2- 172.21.15.92:0/163878160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc06b000 0x7f12dc06f0d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f12dc06b9a0 tx=0x7f12d40091a0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 -- 172.21.15.92:0/163878160 shutdown_connections 2023-10-06T03:10:57.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 --2- 172.21.15.92:0/163878160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc06b000 0x7f12dc06f0d0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 --2- 172.21.15.92:0/163878160 >> v2:172.21.15.136:3300/0 conn(0x7f12dc1564e0 0x7f12dc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 -- 172.21.15.92:0/163878160 >> 172.21.15.92:0/163878160 conn(0x7f12dc0cd640 msgr2=0x7f12dc0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:57.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 -- 172.21.15.92:0/163878160 shutdown_connections 2023-10-06T03:10:57.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.988+0000 7f12e36de700 1 -- 172.21.15.92:0/163878160 wait complete. 2023-10-06T03:10:57.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.989+0000 7f12e36de700 1 Processor -- start 2023-10-06T03:10:57.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.989+0000 7f12e36de700 1 -- start start 2023-10-06T03:10:57.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e36de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc06b000 0x7f12dc159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:57.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e36de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 0x7f12dc157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:57.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e36de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc1568b0 0x7f12dc0c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e36de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1587f0 con 0x7f12dc1564e0 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e36de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc166bb0 con 0x7f12dc1568b0 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e36de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc157c80 con 0x7f12dc06b000 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 0x7f12dc157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 0x7f12dc157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37732/0 (socket says 172.21.15.92:37732) 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 -- 172.21.15.92:0/2382638687 learned_addr learned my addr 172.21.15.92:0/2382638687 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e1c7b700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc1568b0 0x7f12dc0c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e147a700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc06b000 0x7f12dc159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 -- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc06b000 msgr2=0x7f12dc159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc06b000 0x7f12dc159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 -- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc1568b0 msgr2=0x7f12dc0c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc1568b0 0x7f12dc0c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e0c79700 1 -- 172.21.15.92:0/2382638687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4009420 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.990+0000 7f12e147a700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc06b000 0x7f12dc159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.991+0000 7f12e0c79700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 0x7f12dc157740 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00ea50 tx=0x7f12cc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.991+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc005b30 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.991+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc005c90 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.991+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc157f50 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.991+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007340 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.991+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc158510 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.992+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc0718b0 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.992+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f12cc0014b0 con 0x7f12dc1564e0 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.992+0000 7f12d17fa700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12c0067320 0x7f12c00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:57.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.992+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f12cc0871e0 con 0x7f12dc1564e0 2023-10-06T03:10:57.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.992+0000 7f12e147a700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12c0067320 0x7f12c00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:57.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.993+0000 7f12e147a700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12c0067320 0x7f12c00697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f12d80076c0 tx=0x7f12d800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:57.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:57.997+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12cc053c00 con 0x7f12dc1564e0 2023-10-06T03:10:58.227 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.226+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12dc166cf0 con 0x7f12dc1564e0 2023-10-06T03:10:58.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.654+0000 7f12d17fa700 1 -- 172.21.15.92:0/2382638687 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v126) v1 ==== 136+0+0 (secure 0 0 0) 0x7f12cc057220 con 0x7f12dc1564e0 2023-10-06T03:10:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12c0067320 msgr2=0x7f12c00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12c0067320 0x7f12c00697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f12d80076c0 tx=0x7f12d800b040 comp rx=0 tx=0).stop 2023-10-06T03:10:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 msgr2=0x7f12dc157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 0x7f12dc157740 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00ea50 tx=0x7f12cc00c330 comp rx=0 tx=0).stop 2023-10-06T03:10:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 shutdown_connections 2023-10-06T03:10:58.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f12c0067320 0x7f12c00697d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc1568b0 0x7f12dc0c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc06b000 0x7f12dc159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 --2- 172.21.15.92:0/2382638687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12dc1564e0 0x7f12dc157740 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 >> 172.21.15.92:0/2382638687 conn(0x7f12dc0cd640 msgr2=0x7f12dc0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 shutdown_connections 2023-10-06T03:10:58.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.656+0000 7f12e36de700 1 -- 172.21.15.92:0/2382638687 wait complete. 2023-10-06T03:10:58.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:10:58.787 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.786+0000 7f84ab354700 1 Processor -- start 2023-10-06T03:10:58.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.786+0000 7f84ab354700 1 -- start start 2023-10-06T03:10:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84ab354700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f84a41597f0 0x7f84a40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84ab354700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84a415d230 0x7f84a415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84ab354700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a415db70 con 0x7f84a4159420 2023-10-06T03:10:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84ab354700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a40b11d0 con 0x7f84a415d230 2023-10-06T03:10:58.790 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84ab354700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a40b14b0 con 0x7f84a41597f0 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84a415d230 0x7f84a415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f84a415d230 0x7f84a415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37736/0 (socket says 172.21.15.92:37736) 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 -- 172.21.15.92:0/49144961 learned_addr learned my addr 172.21.15.92:0/49144961 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84a9b51700 1 --2- 172.21.15.92:0/49144961 >> v2:172.21.15.136:3300/0 conn(0x7f84a41597f0 0x7f84a40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 -- 172.21.15.92:0/49144961 >> v2:172.21.15.136:3300/0 conn(0x7f84a41597f0 msgr2=0x7f84a40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 --2- 172.21.15.92:0/49144961 >> v2:172.21.15.136:3300/0 conn(0x7f84a41597f0 0x7f84a40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 -- 172.21.15.92:0/49144961 >> v1:172.21.15.136:6789/0 conn(0x7f84a4159420 legacy=0x7f84a40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.787+0000 7f84aab53700 1 -- 172.21.15.92:0/49144961 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40b1790 con 0x7f84a415d230 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84aab53700 1 --2- 172.21.15.92:0/49144961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a415d630 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f84a4089750 tx=0x7f849c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f06bdda0b54172 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84a9350700 1 -- 172.21.15.92:0/49144961 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c00e070 con 0x7f84a415d230 2023-10-06T03:10:58.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84a9350700 1 -- 172.21.15.92:0/49144961 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c00ba20 con 0x7f84a415d230 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84a9350700 1 -- 172.21.15.92:0/49144961 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0092e0 con 0x7f84a415d230 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 -- 172.21.15.92:0/49144961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 msgr2=0x7f84a415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 --2- 172.21.15.92:0/49144961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a415d630 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f84a4089750 tx=0x7f849c009840 comp rx=0 tx=0).stop 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 -- 172.21.15.92:0/49144961 shutdown_connections 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 --2- 172.21.15.92:0/49144961 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a415d630 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 --2- 172.21.15.92:0/49144961 >> v2:172.21.15.136:3300/0 conn(0x7f84a41597f0 0x7f84a40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 -- 172.21.15.92:0/49144961 >> 172.21.15.92:0/49144961 conn(0x7f84a40d0660 msgr2=0x7f84a40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 -- 172.21.15.92:0/49144961 shutdown_connections 2023-10-06T03:10:58.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.788+0000 7f84ab354700 1 -- 172.21.15.92:0/49144961 wait complete. 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.789+0000 7f84ab354700 1 Processor -- start 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 -- start start 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4159420 0x7f84a415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84a41597f0 0x7f84a415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a415b650 con 0x7f84a415d230 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a415b090 con 0x7f84a4159420 2023-10-06T03:10:58.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84ab354700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a415b370 con 0x7f84a41597f0 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84aab53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84aab53700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a4153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37750/0 (socket says 172.21.15.92:37750) 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84aab53700 1 -- 172.21.15.92:0/3375697327 learned_addr learned my addr 172.21.15.92:0/3375697327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84aa352700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4159420 0x7f84a415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.790+0000 7f84a9b51700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84a41597f0 0x7f84a415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84aab53700 1 -- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84a41597f0 msgr2=0x7f84a415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84aab53700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84a41597f0 0x7f84a415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84aab53700 1 -- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4159420 msgr2=0x7f84a415c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84aab53700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4159420 0x7f84a415c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84aab53700 1 -- 172.21.15.92:0/3375697327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c009ae0 con 0x7f84a415d230 2023-10-06T03:10:58.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84a9b51700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84a41597f0 0x7f84a415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84aab53700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a4153e40 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f849c0072f0 tx=0x7f849c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c008ae0 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c008c40 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a40c55a0 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c025460 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.791+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a40c7430 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.792+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a40c7620 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.792+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f849c003040 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.792+0000 7f849a7fc700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8488067320 0x7f84880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.793+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f849c086b20 con 0x7f84a415d230 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.793+0000 7f84aa352700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8488067320 0x7f84880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:58.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.793+0000 7f84aa352700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8488067320 0x7f84880697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f84a0003a80 tx=0x7f84a0007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:58.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:58.797+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84a40c7620 con 0x7f84a415d230 2023-10-06T03:10:59.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.009+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f84a40c5400 con 0x7f8488067320 2023-10-06T03:10:59.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.010+0000 7f849a7fc700 1 -- 172.21.15.92:0/3375697327 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f84a40c5400 con 0x7f8488067320 2023-10-06T03:10:59.012 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:10:59.012 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:10:59.012 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:10:59.012 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:10:59.012 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:10:59.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8488067320 msgr2=0x7f84880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8488067320 0x7f84880697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f84a0003a80 tx=0x7f84a0007ec0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 msgr2=0x7f84a4153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a4153e40 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f849c0072f0 tx=0x7f849c018000 comp rx=0 tx=0).stop 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 shutdown_connections 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8488067320 0x7f84880697d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84a415d230 0x7f84a4153e40 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84a4159420 0x7f84a415c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 --2- 172.21.15.92:0/3375697327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84a41597f0 0x7f84a415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 >> 172.21.15.92:0/3375697327 conn(0x7f84a40d0660 msgr2=0x7f84a40b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 shutdown_connections 2023-10-06T03:10:59.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.012+0000 7f84ab354700 1 -- 172.21.15.92:0/3375697327 wait complete. 2023-10-06T03:10:59.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:10:59.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.141+0000 7f5b3868c700 1 Processor -- start 2023-10-06T03:10:59.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.141+0000 7f5b3868c700 1 -- start start 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.142+0000 7f5b3868c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b300c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.142+0000 7f5b3868c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3015c2a0 con 0x7f5b3006bcd0 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.142+0000 7f5b3868c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3015b690 con 0x7f5b30141500 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.142+0000 7f5b3868c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3015b970 con 0x7f5b300cd1e0 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.142+0000 7f5b36428700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5b3006bcd0 0x7f5b30140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:56268/0 (socket says 172.21.15.92:56268) 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.142+0000 7f5b36428700 1 -- 172.21.15.92:0/665359077 learned_addr learned my addr 172.21.15.92:0/665359077 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242235280 0 0) 0x7f5b3015c2a0 con 0x7f5b3006bcd0 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b1c003570 con 0x7f5b3006bcd0 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36c29700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b300c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031726569 0 0) 0x7f5b1c003570 con 0x7f5b3006bcd0 2023-10-06T03:10:59.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 msgr2=0x7f5b300c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b300c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 >> v1:172.21.15.136:6789/0 conn(0x7f5b30141500 legacy=0x7f5b300c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3015bc50 con 0x7f5b3006bcd0 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f5b2c0017f0 con 0x7f5b3006bcd0 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 >> v1:172.21.15.92:6789/0 conn(0x7f5b3006bcd0 legacy=0x7f5b30140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b1c003a90 0x7f5b1c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c004da0 0x7f5b1c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3015b970 con 0x7f5b1c003a90 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c00bc20 con 0x7f5b1c004da0 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c00bde0 con 0x7f5b300cd1e0 2023-10-06T03:10:59.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b1c003a90 0x7f5b1c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35c27700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36c29700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c004da0 0x7f5b1c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 -- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 msgr2=0x7f5b1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 -- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c004da0 msgr2=0x7f5b1c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c004da0 0x7f5b1c0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 -- 172.21.15.92:0/665359077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c003570 con 0x7f5b1c003a90 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36c29700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c004da0 0x7f5b1c0051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b35c27700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300cd1e0 0x7f5b1c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.143+0000 7f5b36428700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b1c003a90 0x7f5b1c003ea0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c002250 tx=0x7f5b3006e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a301817ee67d7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0070d0 con 0x7f5b1c003a90 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c007230 con 0x7f5b1c003a90 2023-10-06T03:10:59.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b35426700 1 -- 172.21.15.92:0/665359077 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0075f0 con 0x7f5b1c003a90 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b3868c700 1 -- 172.21.15.92:0/665359077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b1c003a90 msgr2=0x7f5b1c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b3868c700 1 --2- 172.21.15.92:0/665359077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b1c003a90 0x7f5b1c003ea0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c002250 tx=0x7f5b3006e470 comp rx=0 tx=0).stop 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b36c29700 1 -- 172.21.15.92:0/665359077 reap_dead start 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b3868c700 1 -- 172.21.15.92:0/665359077 shutdown_connections 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b3868c700 1 -- 172.21.15.92:0/665359077 >> 172.21.15.92:0/665359077 conn(0x7f5b300ce600 msgr2=0x7f5b300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b3868c700 1 -- 172.21.15.92:0/665359077 shutdown_connections 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.144+0000 7f5b3868c700 1 -- 172.21.15.92:0/665359077 wait complete. 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.145+0000 7f5b3868c700 1 Processor -- start 2023-10-06T03:10:59.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.145+0000 7f5b3868c700 1 -- start start 2023-10-06T03:10:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b3868c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c003a90 0x7f5b300b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b3868c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 0x7f5b300c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b3868c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300c8520 0x7f5b300afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b3868c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300b25b0 con 0x7f5b30168ad0 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b3868c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301692e0 con 0x7f5b1c003a90 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b3868c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300c89a0 con 0x7f5b300c8520 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 0x7f5b300c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 0x7f5b300c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:37770/0 (socket says 172.21.15.92:37770) 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 -- 172.21.15.92:0/372589305 learned_addr learned my addr 172.21.15.92:0/372589305 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 -- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300c8520 msgr2=0x7f5b300afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300c8520 0x7f5b300afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b36428700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c003a90 0x7f5b300b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 -- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c003a90 msgr2=0x7f5b300b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c003a90 0x7f5b300b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.146+0000 7f5b35c27700 1 -- 172.21.15.92:0/372589305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b300b27a0 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.147+0000 7f5b35c27700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 0x7f5b300c7fe0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5b20000c00 tx=0x7f5b20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.147+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20017070 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.147+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b20006820 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.147+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20006bc0 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.147+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b300b2a80 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.147+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b30168f40 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.148+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b30169450 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.148+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5b20007020 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.148+0000 7f5b267fc700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5b14067320 0x7f5b140697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.149+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5b20084a10 con 0x7f5b30168ad0 2023-10-06T03:10:59.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.149+0000 7f5b36428700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5b14067320 0x7f5b140697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.149+0000 7f5b36428700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5b14067320 0x7f5b140697d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5b300b3360 tx=0x7f5b2c015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:59.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.153+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b30169450 con 0x7f5b30168ad0 2023-10-06T03:10:59.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.338+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5b30169450 con 0x7f5b30168ad0 2023-10-06T03:10:59.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.338+0000 7f5b267fc700 1 -- 172.21.15.92:0/372589305 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b30169450 con 0x7f5b30168ad0 2023-10-06T03:10:59.342 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5b14067320 msgr2=0x7f5b140697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5b14067320 0x7f5b140697d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5b300b3360 tx=0x7f5b2c015000 comp rx=0 tx=0).stop 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 msgr2=0x7f5b300c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 0x7f5b300c7fe0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5b20000c00 tx=0x7f5b20009040 comp rx=0 tx=0).stop 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 shutdown_connections 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5b14067320 0x7f5b140697d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b300c8520 0x7f5b300afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1c003a90 0x7f5b300b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 --2- 172.21.15.92:0/372589305 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b30168ad0 0x7f5b300c7fe0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 >> 172.21.15.92:0/372589305 conn(0x7f5b300ce600 msgr2=0x7f5b3013fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 shutdown_connections 2023-10-06T03:10:59.343 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.340+0000 7f5b3868c700 1 -- 172.21.15.92:0/372589305 wait complete. 2023-10-06T03:10:59.354 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:10:59.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:10:59.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:10:59.391 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:10:59.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:59.391 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:10:59.446 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:10:59.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:10:59.447 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:10:59.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:10:59.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:59.494 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:10:59.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:10:59.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:59.511 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T03:10:59.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:10:59.525 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T03:10:59.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:10:59.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:10:59.526 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T03:10:59.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:10:59.568 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T03:10:59.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000445601 --yes-i-really-mean-it 2023-10-06T03:10:59.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.687+0000 7fa129ccc700 1 Processor -- start 2023-10-06T03:10:59.689 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.687+0000 7fa129ccc700 1 -- start start 2023-10-06T03:10:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa129ccc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa124156350 0x7fa124157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa129ccc700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa1240b0760 0x7fa1240b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa129ccc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa124159f20 0x7fa12415a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa129ccc700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1240b1140 con 0x7fa124156350 2023-10-06T03:10:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa129ccc700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1240b0570 con 0x7fa1240b0760 2023-10-06T03:10:59.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa129ccc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1241578b0 con 0x7fa124159f20 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa124156350 0x7fa124157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa124156350 0x7fa124157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33682/0 (socket says 172.21.15.92:33682) 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 -- 172.21.15.92:0/661324625 learned_addr learned my addr 172.21.15.92:0/661324625 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa11bfff700 1 --2- 172.21.15.92:0/661324625 >> v2:172.21.15.136:3300/0 conn(0x7fa1240b0760 0x7fa1240b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa1294cb700 1 --2- 172.21.15.92:0/661324625 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa124159f20 0x7fa12415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 -- 172.21.15.92:0/661324625 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa124159f20 msgr2=0x7fa12415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 --2- 172.21.15.92:0/661324625 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa124159f20 0x7fa12415a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.693 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 -- 172.21.15.92:0/661324625 >> v2:172.21.15.136:3300/0 conn(0x7fa1240b0760 msgr2=0x7fa1240b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 --2- 172.21.15.92:0/661324625 >> v2:172.21.15.136:3300/0 conn(0x7fa1240b0760 0x7fa1240b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 -- 172.21.15.92:0/661324625 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1240af4d0 con 0x7fa124156350 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.688+0000 7fa128cca700 1 --2- 172.21.15.92:0/661324625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124157370 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa1240642a0 tx=0x7fa120009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd0f74a20f50a83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa11b7fe700 1 -- 172.21.15.92:0/661324625 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12000e070 con 0x7fa124156350 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa11b7fe700 1 -- 172.21.15.92:0/661324625 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12000ba50 con 0x7fa124156350 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa11b7fe700 1 -- 172.21.15.92:0/661324625 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200092e0 con 0x7fa124156350 2023-10-06T03:10:59.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 -- 172.21.15.92:0/661324625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 msgr2=0x7fa124157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 --2- 172.21.15.92:0/661324625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124157370 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa1240642a0 tx=0x7fa120009840 comp rx=0 tx=0).stop 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 -- 172.21.15.92:0/661324625 shutdown_connections 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 --2- 172.21.15.92:0/661324625 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa124159f20 0x7fa12415a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 --2- 172.21.15.92:0/661324625 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124157370 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 --2- 172.21.15.92:0/661324625 >> v2:172.21.15.136:3300/0 conn(0x7fa1240b0760 0x7fa1240b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 -- 172.21.15.92:0/661324625 >> 172.21.15.92:0/661324625 conn(0x7fa124068e20 msgr2=0x7fa1240cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 -- 172.21.15.92:0/661324625 shutdown_connections 2023-10-06T03:10:59.695 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.689+0000 7fa129ccc700 1 -- 172.21.15.92:0/661324625 wait complete. 2023-10-06T03:10:59.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.690+0000 7fa129ccc700 1 Processor -- start 2023-10-06T03:10:59.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.690+0000 7fa129ccc700 1 -- start start 2023-10-06T03:10:59.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa129ccc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1240b0760 0x7fa12414a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa129ccc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa129ccc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa124159f20 0x7fa124152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa129ccc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12414a760 con 0x7fa124156350 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa129ccc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1241686a0 con 0x7fa124159f20 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa129ccc700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa124167900 con 0x7fa1240b0760 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa11bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.699 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa11bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33696/0 (socket says 172.21.15.92:33696) 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa11bfff700 1 -- 172.21.15.92:0/3849539047 learned_addr learned my addr 172.21.15.92:0/3849539047 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa128cca700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1240b0760 0x7fa12414a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.691+0000 7fa1294cb700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa124159f20 0x7fa124152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa11bfff700 1 -- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1240b0760 msgr2=0x7fa12414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa11bfff700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1240b0760 0x7fa12414a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa11bfff700 1 -- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa124159f20 msgr2=0x7fa124152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa11bfff700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa124159f20 0x7fa124152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:10:59.700 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa11bfff700 1 -- 172.21.15.92:0/3849539047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120009ae0 con 0x7fa124156350 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa11bfff700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124152850 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa11400d860 tx=0x7fa114008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114009610 con 0x7fa124156350 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa114009770 con 0x7fa124156350 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa124167be0 con 0x7fa124156350 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114009b30 con 0x7fa124156350 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.692+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa124168330 con 0x7fa124156350 2023-10-06T03:10:59.701 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.693+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa124168850 con 0x7fa124156350 2023-10-06T03:10:59.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.693+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa11401e460 con 0x7fa124156350 2023-10-06T03:10:59.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.694+0000 7fa118ff9700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa110067320 0x7fa1100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:10:59.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.694+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1140853e0 con 0x7fa124156350 2023-10-06T03:10:59.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.694+0000 7fa128cca700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa110067320 0x7fa1100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:10:59.702 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.694+0000 7fa128cca700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa110067320 0x7fa1100697d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa120009880 tx=0x7fa120018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:10:59.703 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.698+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa124168850 con 0x7fa124156350 2023-10-06T03:10:59.909 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:10:59.907+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000445601", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa124168810 con 0x7fa110067320 2023-10-06T03:11:02.679 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.678+0000 7fa118ff9700 1 -- 172.21.15.92:0/3849539047 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7fa124168810 con 0x7fa110067320 2023-10-06T03:11:02.679 INFO:teuthology.orchestra.run.smithi092.stdout:FS volume 'cephfs' renamed to 'volume_0000000000445601' 2023-10-06T03:11:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.682+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa110067320 msgr2=0x7fa1100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.682+0000 7fa129ccc700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa110067320 0x7fa1100697d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa120009880 tx=0x7fa120018000 comp rx=0 tx=0).stop 2023-10-06T03:11:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.682+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 msgr2=0x7fa124152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.682+0000 7fa129ccc700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124152850 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa11400d860 tx=0x7fa114008040 comp rx=0 tx=0).stop 2023-10-06T03:11:02.684 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 shutdown_connections 2023-10-06T03:11:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa110067320 0x7fa1100697d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa124159f20 0x7fa124152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1240b0760 0x7fa12414a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 --2- 172.21.15.92:0/3849539047 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa124156350 0x7fa124152850 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 >> 172.21.15.92:0/3849539047 conn(0x7fa124068e20 msgr2=0x7fa124069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:02.685 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 shutdown_connections 2023-10-06T03:11:02.686 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.683+0000 7fa129ccc700 1 -- 172.21.15.92:0/3849539047 wait complete. 2023-10-06T03:11:02.698 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000445601 --yes-i-really-mean-it 2023-10-06T03:11:02.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.822+0000 7f594b983700 1 Processor -- start 2023-10-06T03:11:02.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.822+0000 7f594b983700 1 -- start start 2023-10-06T03:11:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594b983700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f59440b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594b983700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440b5b00 con 0x7f59440cf350 2023-10-06T03:11:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594b983700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440b5da0 con 0x7f59440b53d0 2023-10-06T03:11:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594b983700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59440b6080 con 0x7f5944159ec0 2023-10-06T03:11:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f5949f20700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f59440cf350 0x7f59440cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45254/0 (socket says 172.21.15.92:45254) 2023-10-06T03:11:02.825 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f5949f20700 1 -- 172.21.15.92:0/2648501149 learned_addr learned my addr 172.21.15.92:0/2648501149 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594971f700 1 --2- 172.21.15.92:0/2648501149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f59440b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925520854 0 0) 0x7f59440b5b00 con 0x7f59440cf350 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5930003570 con 0x7f59440cf350 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119282189 0 0) 0x7f59440b5da0 con 0x7f59440b53d0 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59440b5b00 con 0x7f59440b53d0 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594971f700 1 -- 172.21.15.92:0/2648501149 >> v1:172.21.15.136:6789/0 conn(0x7f59440b53d0 legacy=0x7f59440cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228533006 0 0) 0x7f5930003570 con 0x7f59440cf350 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594971f700 1 -- 172.21.15.92:0/2648501149 >> v1:172.21.15.92:6789/0 conn(0x7f59440cf350 legacy=0x7f59440cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.823+0000 7f594971f700 1 -- 172.21.15.92:0/2648501149 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440cc100 con 0x7f5944159ec0 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594971f700 1 --2- 172.21.15.92:0/2648501149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f59440b4e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5940009900 tx=0x7f5940009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97ba0dbc1109394 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:02.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0017f0 con 0x7f5944159ec0 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940011420 con 0x7f5944159ec0 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f593bfff700 1 -- 172.21.15.92:0/2648501149 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400117c0 con 0x7f5944159ec0 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 -- 172.21.15.92:0/2648501149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 msgr2=0x7f59440b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 --2- 172.21.15.92:0/2648501149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f59440b4e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5940009900 tx=0x7f5940009120 comp rx=0 tx=0).stop 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 -- 172.21.15.92:0/2648501149 shutdown_connections 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 --2- 172.21.15.92:0/2648501149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f59440b4e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 -- 172.21.15.92:0/2648501149 >> 172.21.15.92:0/2648501149 conn(0x7f59440999f0 msgr2=0x7f59440b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 -- 172.21.15.92:0/2648501149 shutdown_connections 2023-10-06T03:11:02.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.824+0000 7f594b983700 1 -- 172.21.15.92:0/2648501149 wait complete. 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.825+0000 7f594b983700 1 Processor -- start 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.825+0000 7f594b983700 1 -- start start 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594b983700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59440b53d0 0x7f5944144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594b983700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 0x7f5944154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594b983700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f5944155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594b983700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5944156e10 con 0x7f59440cf350 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594b983700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59441570b0 con 0x7f59440b53d0 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594b983700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5944157390 con 0x7f5944159ec0 2023-10-06T03:11:02.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f5948f1e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 0x7f5944154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f5948f1e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 0x7f5944154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33706/0 (socket says 172.21.15.92:33706) 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f5948f1e700 1 -- 172.21.15.92:0/1053187780 learned_addr learned my addr 172.21.15.92:0/1053187780 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f594971f700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59440b53d0 0x7f5944144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.826+0000 7f5949f20700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f5944155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5948f1e700 1 -- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 msgr2=0x7f5944155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5948f1e700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f5944155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5948f1e700 1 -- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59440b53d0 msgr2=0x7f5944144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5948f1e700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59440b53d0 0x7f5944144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5948f1e700 1 -- 172.21.15.92:0/1053187780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400093c0 con 0x7f59440cf350 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5949f20700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f5944155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:02.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f5948f1e700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 0x7f5944154c00 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f593400c580 tx=0x7f5934007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934010510 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934010670 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5944157670 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934010a30 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.827+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59441445d0 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.828+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944157b90 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.828+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f593401b400 con 0x7f59440cf350 2023-10-06T03:11:02.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.829+0000 7f59397fa700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5928067320 0x7f59280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:02.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.829+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f5934084ac0 con 0x7f59440cf350 2023-10-06T03:11:02.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.829+0000 7f594971f700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5928067320 0x7f59280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:02.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.829+0000 7f594971f700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5928067320 0x7f59280697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5940009160 tx=0x7f5940016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:02.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:02.833+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5944157b90 con 0x7f59440cf350 2023-10-06T03:11:03.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.045+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000445601", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5944157b90 con 0x7f5928067320 2023-10-06T03:11:03.687 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.686+0000 7f59397fa700 1 -- 172.21.15.92:0/1053187780 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f5944157b90 con 0x7f5928067320 2023-10-06T03:11:03.688 INFO:teuthology.orchestra.run.smithi092.stdout:FS volume 'cephfs' renamed to 'volume_0000000000445601' 2023-10-06T03:11:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.688+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5928067320 msgr2=0x7f59280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5928067320 0x7f59280697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5940009160 tx=0x7f5940016000 comp rx=0 tx=0).stop 2023-10-06T03:11:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 msgr2=0x7f5944154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 0x7f5944154c00 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f593400c580 tx=0x7f5934007e10 comp rx=0 tx=0).stop 2023-10-06T03:11:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 shutdown_connections 2023-10-06T03:11:03.690 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5928067320 0x7f59280697d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5944159ec0 0x7f5944155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59440b53d0 0x7f5944144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 --2- 172.21.15.92:0/1053187780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59440cf350 0x7f5944154c00 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 >> 172.21.15.92:0/1053187780 conn(0x7f59440999f0 msgr2=0x7f59441522f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:03.691 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 shutdown_connections 2023-10-06T03:11:03.692 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.689+0000 7f594b983700 1 -- 172.21.15.92:0/1053187780 wait complete. 2023-10-06T03:11:03.702 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:11:03.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.819+0000 7f6634f0f700 1 Processor -- start 2023-10-06T03:11:03.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.819+0000 7f6634f0f700 1 -- start start 2023-10-06T03:11:03.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6634f0f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6630140f70 0x7f663015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:03.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6634f0f700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66300cb040 0x7f663006d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:03.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6634f0f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66300ccd80 con 0x7f663015b930 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6634f0f700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6630158ee0 con 0x7f66300cb040 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6634f0f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66300cc790 con 0x7f6630140f70 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6625d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66300cb040 0x7f663006d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6625d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f66300cb040 0x7f663006d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33722/0 (socket says 172.21.15.92:33722) 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6625d9b700 1 -- 172.21.15.92:0/2669731003 learned_addr learned my addr 172.21.15.92:0/2669731003 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f662e59c700 1 --2- 172.21.15.92:0/2669731003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6630140f70 0x7f663015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6625d9b700 1 -- 172.21.15.92:0/2669731003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6630140f70 msgr2=0x7f663015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.820+0000 7f6625d9b700 1 --2- 172.21.15.92:0/2669731003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6630140f70 0x7f663015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6625d9b700 1 -- 172.21.15.92:0/2669731003 >> v1:172.21.15.136:6789/0 conn(0x7f663015b930 legacy=0x7f663006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6625d9b700 1 -- 172.21.15.92:0/2669731003 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663015bdb0 con 0x7f66300cb040 2023-10-06T03:11:03.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6625d9b700 1 --2- 172.21.15.92:0/2669731003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66300cb040 0x7f663006d2b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f66180099b0 tx=0x7f6618009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d70effb0d74da1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f662dd9b700 1 -- 172.21.15.92:0/2669731003 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800b860 con 0x7f66300cb040 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f662dd9b700 1 -- 172.21.15.92:0/2669731003 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661800b9c0 con 0x7f66300cb040 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f662dd9b700 1 -- 172.21.15.92:0/2669731003 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661800bd60 con 0x7f66300cb040 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 -- 172.21.15.92:0/2669731003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66300cb040 msgr2=0x7f663006d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 --2- 172.21.15.92:0/2669731003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66300cb040 0x7f663006d2b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f66180099b0 tx=0x7f6618009250 comp rx=0 tx=0).stop 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 -- 172.21.15.92:0/2669731003 shutdown_connections 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 --2- 172.21.15.92:0/2669731003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6630140f70 0x7f663015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 --2- 172.21.15.92:0/2669731003 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66300cb040 0x7f663006d2b0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 -- 172.21.15.92:0/2669731003 >> 172.21.15.92:0/2669731003 conn(0x7f66300ce540 msgr2=0x7f66300ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 -- 172.21.15.92:0/2669731003 shutdown_connections 2023-10-06T03:11:03.824 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.821+0000 7f6634f0f700 1 -- 172.21.15.92:0/2669731003 wait complete. 2023-10-06T03:11:03.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 Processor -- start 2023-10-06T03:11:03.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 -- start start 2023-10-06T03:11:03.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 0x7f6630156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:03.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f663013b9a0 0x7f6630156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6630139cf0 0x7f663013a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663013ac40 con 0x7f6630140f70 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663013af20 con 0x7f6630139cf0 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f6634f0f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663013b200 con 0x7f663013b9a0 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f662e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 0x7f6630156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f662e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 0x7f6630156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33732/0 (socket says 172.21.15.92:33732) 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.823+0000 7f662e59c700 1 -- 172.21.15.92:0/396583485 learned_addr learned my addr 172.21.15.92:0/396583485 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6625d9b700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f663013b9a0 0x7f6630156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662ed9d700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6630139cf0 0x7f663013a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662e59c700 1 -- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f663013b9a0 msgr2=0x7f6630156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:03.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662e59c700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f663013b9a0 0x7f6630156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662e59c700 1 -- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6630139cf0 msgr2=0x7f663013a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662e59c700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6630139cf0 0x7f663013a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662e59c700 1 -- 172.21.15.92:0/396583485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180094f0 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6625d9b700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f663013b9a0 0x7f6630156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f662e59c700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 0x7f6630156120 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f662000d890 tx=0x7f6620008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200125f0 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6620012750 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663013b4e0 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620012ad0 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.824+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663013a860 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.825+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66300c34a0 con 0x7f6630140f70 2023-10-06T03:11:03.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.825+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f662001e460 con 0x7f6630140f70 2023-10-06T03:11:03.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.826+0000 7f6626ffd700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6610067320 0x7f66100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:03.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.826+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f6620085410 con 0x7f6630140f70 2023-10-06T03:11:03.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.826+0000 7f6625d9b700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6610067320 0x7f66100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:03.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.826+0000 7f6625d9b700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6610067320 0x7f66100697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f66180099b0 tx=0x7f6618017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:03.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:03.830+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6620051ef0 con 0x7f6630140f70 2023-10-06T03:11:04.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.041+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f663004c3c0 con 0x7f6610067320 2023-10-06T03:11:04.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.043+0000 7f6626ffd700 1 -- 172.21.15.92:0/396583485 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7f663004c3c0 con 0x7f6610067320 2023-10-06T03:11:04.043 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:11:04.043 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "volume_0000000000445601" 2023-10-06T03:11:04.044 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:04.044 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:11:04.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6610067320 msgr2=0x7f66100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6610067320 0x7f66100697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f66180099b0 tx=0x7f6618017000 comp rx=0 tx=0).stop 2023-10-06T03:11:04.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 msgr2=0x7f6630156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 0x7f6630156120 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f662000d890 tx=0x7f6620008040 comp rx=0 tx=0).stop 2023-10-06T03:11:04.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 shutdown_connections 2023-10-06T03:11:04.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6610067320 0x7f66100697d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6630140f70 0x7f6630156120 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6630139cf0 0x7f663013a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 --2- 172.21.15.92:0/396583485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f663013b9a0 0x7f6630156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 >> 172.21.15.92:0/396583485 conn(0x7f66300ce540 msgr2=0x7f663013db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:04.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 shutdown_connections 2023-10-06T03:11:04.047 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.045+0000 7f6634f0f700 1 -- 172.21.15.92:0/396583485 wait complete. 2023-10-06T03:11:04.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:04.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.171+0000 7fd390a8b700 1 Processor -- start 2023-10-06T03:11:04.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.171+0000 7fd390a8b700 1 -- start start 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd390a8b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 0x7fd38c0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd390a8b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 0x7fd38c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd390a8b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0c74b0 con 0x7fd38c159ee0 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd390a8b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0c8700 con 0x7fd38c159b10 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd390a8b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0c9070 con 0x7fd38c15d730 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd381d9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 0x7fd38c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd38a59c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 0x7fd38c0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd381d9b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 0x7fd38c15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:35614/0 (socket says 172.21.15.92:35614) 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd38a59c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 0x7fd38c0cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38892/0 (socket says 172.21.15.92:38892) 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd381d9b700 1 -- 172.21.15.92:0/2815109146 learned_addr learned my addr 172.21.15.92:0/2815109146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.172+0000 7fd38ad9d700 1 --1- 172.21.15.92:0/2815109146 >> v1:172.21.15.136:6789/0 conn(0x7fd38c159ee0 0x7fd38c15a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:56636/0 (socket says 172.21.15.92:56636) 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd381d9b700 1 -- 172.21.15.92:0/2815109146 >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 msgr2=0x7fd38c0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd381d9b700 1 --2- 172.21.15.92:0/2815109146 >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 0x7fd38c0cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd381d9b700 1 -- 172.21.15.92:0/2815109146 >> v1:172.21.15.136:6789/0 conn(0x7fd38c159ee0 legacy=0x7fd38c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd381d9b700 1 -- 172.21.15.92:0/2815109146 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c15e0e0 con 0x7fd38c15d730 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd389d9b700 1 -- 172.21.15.92:0/2815109146 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725139053 0 0) 0x7fd38c0c74b0 con 0x7fd38c159ee0 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd38a59c700 1 --2- 172.21.15.92:0/2815109146 >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 0x7fd38c0cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd381d9b700 1 --2- 172.21.15.92:0/2815109146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 0x7fd38c15dba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd3740099b0 tx=0x7fd374009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d467a235edf9eff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd389d9b700 1 -- 172.21.15.92:0/2815109146 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd374009ea0 con 0x7fd38c15d730 2023-10-06T03:11:04.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd389d9b700 1 -- 172.21.15.92:0/2815109146 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400b8f0 con 0x7fd38c15d730 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.173+0000 7fd389d9b700 1 -- 172.21.15.92:0/2815109146 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bc90 con 0x7fd38c15d730 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 -- 172.21.15.92:0/2815109146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 msgr2=0x7fd38c15dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 --2- 172.21.15.92:0/2815109146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 0x7fd38c15dba0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd3740099b0 tx=0x7fd374009250 comp rx=0 tx=0).stop 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 -- 172.21.15.92:0/2815109146 shutdown_connections 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 --2- 172.21.15.92:0/2815109146 >> v2:172.21.15.136:3300/0 conn(0x7fd38c159b10 0x7fd38c0cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 --2- 172.21.15.92:0/2815109146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c15d730 0x7fd38c15dba0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 -- 172.21.15.92:0/2815109146 >> 172.21.15.92:0/2815109146 conn(0x7fd38c0c9870 msgr2=0x7fd38c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 -- 172.21.15.92:0/2815109146 shutdown_connections 2023-10-06T03:11:04.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.174+0000 7fd390a8b700 1 -- 172.21.15.92:0/2815109146 wait complete. 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.175+0000 7fd390a8b700 1 Processor -- start 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.175+0000 7fd390a8b700 1 -- start start 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd390a8b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c159b10 0x7fd38c0c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd390a8b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 0x7fd38c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd390a8b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38c15d730 0x7fd38c155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd390a8b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c15a3f0 con 0x7fd38c159ee0 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd390a8b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0c2180 con 0x7fd38c15d730 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd390a8b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0c2460 con 0x7fd38c159b10 2023-10-06T03:11:04.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 0x7fd38c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 0x7fd38c152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33740/0 (socket says 172.21.15.92:33740) 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 -- 172.21.15.92:0/3744960026 learned_addr learned my addr 172.21.15.92:0/3744960026 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd38a59c700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c159b10 0x7fd38c0c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd38ad9d700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38c15d730 0x7fd38c155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 -- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c159b10 msgr2=0x7fd38c0c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c159b10 0x7fd38c0c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 -- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38c15d730 msgr2=0x7fd38c155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38c15d730 0x7fd38c155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 -- 172.21.15.92:0/3744960026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009420 con 0x7fd38c159ee0 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd381d9b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 0x7fd38c152ee0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd374006fd0 tx=0x7fd374017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.176+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bb30 con 0x7fd38c159ee0 2023-10-06T03:11:04.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.177+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374026430 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.177+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3740094f0 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.177+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bc90 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.177+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c0c2a20 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.177+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c15a560 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.178+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd374023030 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.178+0000 7fd382ffd700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd36c067320 0x7fd36c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.179+0000 7fd38a59c700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd36c067320 0x7fd36c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.179+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fd374086600 con 0x7fd38c159ee0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.179+0000 7fd38a59c700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd36c067320 0x7fd36c0697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0093f0 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.183+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3740530e0 con 0x7fd38c159ee0 2023-10-06T03:11:04.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.410+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd38c04c3c0 con 0x7fd38c159ee0 2023-10-06T03:11:04.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.410+0000 7fd382ffd700 1 -- 172.21.15.92:0/3744960026 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4624 (secure 0 0 0) 0x7fd374056700 con 0x7fd38c159ee0 2023-10-06T03:11:04.411 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:04.411 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:11:00.658708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"volume_0000000000445601","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:11:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd36c067320 msgr2=0x7fd36c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd36c067320 0x7fd36c0697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0093f0 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2023-10-06T03:11:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 msgr2=0x7fd38c152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 0x7fd38c152ee0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd374006fd0 tx=0x7fd374017000 comp rx=0 tx=0).stop 2023-10-06T03:11:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 shutdown_connections 2023-10-06T03:11:04.413 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd36c067320 0x7fd36c0697d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd38c15d730 0x7fd38c155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38c159b10 0x7fd38c0c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 --2- 172.21.15.92:0/3744960026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38c159ee0 0x7fd38c152ee0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 >> 172.21.15.92:0/3744960026 conn(0x7fd38c0c9870 msgr2=0x7fd38c154850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 shutdown_connections 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.412+0000 7fd390a8b700 1 -- 172.21.15.92:0/3744960026 wait complete. 2023-10-06T03:11:04.414 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 128 2023-10-06T03:11:04.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:04.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.539+0000 7f6e0a078700 1 Processor -- start 2023-10-06T03:11:04.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.539+0000 7f6e0a078700 1 -- start start 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e0a078700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6e04159890 0x7f6e0415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e0a078700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 0x7f6e0415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e0a078700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0415a160 con 0x7f6e0415db10 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e0a078700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e040d13a0 con 0x7f6e04159890 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e0a078700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415e140 con 0x7f6e04159c60 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e037fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6e04159890 0x7f6e0415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.540+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 0x7f6e0415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 0x7f6e0415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:35642/0 (socket says 172.21.15.92:35642) 2023-10-06T03:11:04.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2809708111 learned_addr learned my addr 172.21.15.92:0/2809708111 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e03fff700 1 --1- 172.21.15.92:0/2809708111 >> v1:172.21.15.136:6789/0 conn(0x7f6e0415db10 0x7f6e04069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:56644/0 (socket says 172.21.15.92:56644) 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2809708111 >> v2:172.21.15.136:3300/0 conn(0x7f6e04159890 msgr2=0x7f6e0415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 --2- 172.21.15.92:0/2809708111 >> v2:172.21.15.136:3300/0 conn(0x7f6e04159890 0x7f6e0415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2809708111 >> v1:172.21.15.136:6789/0 conn(0x7f6e0415db10 legacy=0x7f6e04069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2809708111 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0415a3d0 con 0x7f6e04159c60 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e02ffd700 1 --2- 172.21.15.92:0/2809708111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 0x7f6e0415d5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6df80099b0 tx=0x7f6df8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe951ef5e63165e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e027fc700 1 -- 172.21.15.92:0/2809708111 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8009ea0 con 0x7f6e04159c60 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e027fc700 1 -- 172.21.15.92:0/2809708111 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800b860 con 0x7f6e04159c60 2023-10-06T03:11:04.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.541+0000 7f6e027fc700 1 -- 172.21.15.92:0/2809708111 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800bc00 con 0x7f6e04159c60 2023-10-06T03:11:04.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 -- 172.21.15.92:0/2809708111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 msgr2=0x7f6e0415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2809708111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 0x7f6e0415d5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6df80099b0 tx=0x7f6df8009250 comp rx=0 tx=0).stop 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 -- 172.21.15.92:0/2809708111 shutdown_connections 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2809708111 >> v2:172.21.15.136:3300/0 conn(0x7f6e04159890 0x7f6e0415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2809708111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159c60 0x7f6e0415d5d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 -- 172.21.15.92:0/2809708111 >> 172.21.15.92:0/2809708111 conn(0x7f6e04068e20 msgr2=0x7f6e040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 -- 172.21.15.92:0/2809708111 shutdown_connections 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.542+0000 7f6e0a078700 1 -- 172.21.15.92:0/2809708111 wait complete. 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.543+0000 7f6e0a078700 1 Processor -- start 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.543+0000 7f6e0a078700 1 -- start start 2023-10-06T03:11:04.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159890 0x7f6e040ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 0x7f6e040cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e0415db10 0x7f6e04169f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0416a450 con 0x7f6e04159c60 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0416a6f0 con 0x7f6e0415db10 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0416a9d0 con 0x7f6e04159890 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 0x7f6e040cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 0x7f6e040cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33754/0 (socket says 172.21.15.92:33754) 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2045523821 learned_addr learned my addr 172.21.15.92:0/2045523821 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e037fe700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159890 0x7f6e040ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e03fff700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e0415db10 0x7f6e04169f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159890 msgr2=0x7f6e040ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159890 0x7f6e040ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e0415db10 msgr2=0x7f6e04169f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e0415db10 0x7f6e04169f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 -- 172.21.15.92:0/2045523821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df80094f0 con 0x7f6e04159c60 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e02ffd700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 0x7f6e040cb340 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f6e04070e20 tx=0x7f6df8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8000d10 con 0x7f6e04159c60 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df800bbe0 con 0x7f6e04159c60 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0416acb0 con 0x7f6e04159c60 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.544+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8011670 con 0x7f6e04159c60 2023-10-06T03:11:04.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.545+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0416b0e0 con 0x7f6e04159c60 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.545+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0416aef0 con 0x7f6e04159c60 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.546+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6df8011aa0 con 0x7f6e04159c60 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.546+0000 7f6deb7fe700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6df0067320 0x7f6df00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.546+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f6df80866c0 con 0x7f6e04159c60 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.546+0000 7f6e037fe700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6df0067320 0x7f6df00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.547+0000 7f6e037fe700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6df0067320 0x7f6df00697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6df400f570 tx=0x7f6df400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.551 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.550+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e0416aef0 con 0x7f6e04159c60 2023-10-06T03:11:04.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.738+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e040cd0c0 con 0x7f6e04159c60 2023-10-06T03:11:04.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.739+0000 7f6deb7fe700 1 -- 172.21.15.92:0/2045523821 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f6e040cd0c0 con 0x7f6e04159c60 2023-10-06T03:11:04.740 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:04.740 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":189,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:02.673835+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs.volume_0000000000445601.meta","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000445601"}}},{"pool":33,"pool_name":"cephfs.volume_0000000000445601.data","create_time":"2023-10-06T03:10:42.946799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000445601"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6df0067320 msgr2=0x7f6df00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6df0067320 0x7f6df00697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6df400f570 tx=0x7f6df400b040 comp rx=0 tx=0).stop 2023-10-06T03:11:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 msgr2=0x7f6e040cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 0x7f6e040cb340 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f6e04070e20 tx=0x7f6df8017000 comp rx=0 tx=0).stop 2023-10-06T03:11:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 shutdown_connections 2023-10-06T03:11:04.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6df0067320 0x7f6df00697d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e0415db10 0x7f6e04169f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6e04159890 0x7f6e040ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 --2- 172.21.15.92:0/2045523821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6e04159c60 0x7f6e040cb340 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 >> 172.21.15.92:0/2045523821 conn(0x7f6e04068e20 msgr2=0x7f6e040d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 shutdown_connections 2023-10-06T03:11:04.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.741+0000 7f6e0a078700 1 -- 172.21.15.92:0/2045523821 wait complete. 2023-10-06T03:11:04.754 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:11:04.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:11:04.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:11:04.846 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:11:04.846 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:04.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.959+0000 7fbd64fd8700 1 Processor -- start 2023-10-06T03:11:04.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.959+0000 7fbd64fd8700 1 -- start start 2023-10-06T03:11:04.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd64fd8700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbd60159890 0x7fbd6015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd64fd8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbd60159c60 0x7fbd6015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd64fd8700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6015a160 con 0x7fbd6015db10 2023-10-06T03:11:04.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd64fd8700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6015df50 con 0x7fbd60159c60 2023-10-06T03:11:04.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd64fd8700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600d13a0 con 0x7fbd60159890 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd55d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbd60159c60 0x7fbd6015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd5ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbd6015db10 0x7fbd60069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45256/0 (socket says 172.21.15.92:45256) 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd5ed9d700 1 -- 172.21.15.92:0/3894050467 learned_addr learned my addr 172.21.15.92:0/3894050467 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd5e59c700 1 --2- 172.21.15.92:0/3894050467 >> v2:172.21.15.136:3300/0 conn(0x7fbd60159890 0x7fbd6015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd55d9b700 1 -- 172.21.15.92:0/3894050467 >> v2:172.21.15.136:3300/0 conn(0x7fbd60159890 msgr2=0x7fbd6015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd55d9b700 1 --2- 172.21.15.92:0/3894050467 >> v2:172.21.15.136:3300/0 conn(0x7fbd60159890 0x7fbd6015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd55d9b700 1 -- 172.21.15.92:0/3894050467 >> v1:172.21.15.92:6789/0 conn(0x7fbd6015db10 legacy=0x7fbd60069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd5dd9b700 1 -- 172.21.15.92:0/3894050467 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910158313 0 0) 0x7fbd6015a160 con 0x7fbd6015db10 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.960+0000 7fbd55d9b700 1 -- 172.21.15.92:0/3894050467 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6015e140 con 0x7fbd60159c60 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd55d9b700 1 --2- 172.21.15.92:0/3894050467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd60159c60 0x7fbd6015d5d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbd600d1090 tx=0x7fbd480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c457e45bbc0be6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd5dd9b700 1 -- 172.21.15.92:0/3894050467 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800b6b0 con 0x7fbd60159c60 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd5dd9b700 1 -- 172.21.15.92:0/3894050467 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4800b810 con 0x7fbd60159c60 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd5dd9b700 1 -- 172.21.15.92:0/3894050467 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800bbb0 con 0x7fbd60159c60 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 -- 172.21.15.92:0/3894050467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd60159c60 msgr2=0x7fbd6015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/3894050467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd60159c60 0x7fbd6015d5d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbd600d1090 tx=0x7fbd480091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 -- 172.21.15.92:0/3894050467 shutdown_connections 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/3894050467 >> v2:172.21.15.136:3300/0 conn(0x7fbd60159890 0x7fbd6015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/3894050467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd60159c60 0x7fbd6015d5d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 -- 172.21.15.92:0/3894050467 >> 172.21.15.92:0/3894050467 conn(0x7fbd60068e20 msgr2=0x7fbd600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 -- 172.21.15.92:0/3894050467 shutdown_connections 2023-10-06T03:11:04.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.961+0000 7fbd64fd8700 1 -- 172.21.15.92:0/3894050467 wait complete. 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.962+0000 7fbd64fd8700 1 Processor -- start 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 -- start start 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd60159890 0x7fbd600cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd60159c60 0x7fbd6014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 0x7fbd60152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60151ff0 con 0x7fbd6015db10 2023-10-06T03:11:04.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600cdc20 con 0x7fbd60159c60 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd64fd8700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600cdec0 con 0x7fbd60159890 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd5ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 0x7fbd60152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd5ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 0x7fbd60152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33782/0 (socket says 172.21.15.92:33782) 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.963+0000 7fbd5ed9d700 1 -- 172.21.15.92:0/1490333760 learned_addr learned my addr 172.21.15.92:0/1490333760 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd55d9b700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd60159c60 0x7fbd6014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5e59c700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd60159890 0x7fbd600cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5ed9d700 1 -- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd60159890 msgr2=0x7fbd600cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5ed9d700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd60159890 0x7fbd600cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5ed9d700 1 -- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd60159c60 msgr2=0x7fbd6014d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5ed9d700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd60159c60 0x7fbd6014d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:04.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5ed9d700 1 -- 172.21.15.92:0/1490333760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48009420 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd5ed9d700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 0x7fbd60152300 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800cbf0 tx=0x7fbd580019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800a9c0 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5800ab20 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd60152e50 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580070f0 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.964+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd600ce720 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.965+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd600ce080 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.965+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd58007520 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.966+0000 7fbd56ffd700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd40067320 0x7fbd400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.966+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fbd58085200 con 0x7fbd6015db10 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.966+0000 7fbd5e59c700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd40067320 0x7fbd400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.966+0000 7fbd5e59c700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd40067320 0x7fbd400697d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000ab30 tx=0x7fbd50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:04.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:04.970+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd58051d60 con 0x7fbd6015db10 2023-10-06T03:11:05.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.195+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd60152130 con 0x7fbd6015db10 2023-10-06T03:11:05.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.196+0000 7fbd56ffd700 1 -- 172.21.15.92:0/1490333760 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4624 (secure 0 0 0) 0x7fbd58055380 con 0x7fbd6015db10 2023-10-06T03:11:05.197 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:05.197 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:11:00.658708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"volume_0000000000445601","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd40067320 msgr2=0x7fbd400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd40067320 0x7fbd400697d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000ab30 tx=0x7fbd50008040 comp rx=0 tx=0).stop 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 msgr2=0x7fbd60152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 0x7fbd60152300 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800cbf0 tx=0x7fbd580019b0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 shutdown_connections 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbd40067320 0x7fbd400697d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd6015db10 0x7fbd60152300 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd60159890 0x7fbd600cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 --2- 172.21.15.92:0/1490333760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd60159c60 0x7fbd6014d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 >> 172.21.15.92:0/1490333760 conn(0x7fbd60068e20 msgr2=0x7fbd60069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:05.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 shutdown_connections 2023-10-06T03:11:05.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.198+0000 7fbd64fd8700 1 -- 172.21.15.92:0/1490333760 wait complete. 2023-10-06T03:11:05.200 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 128 2023-10-06T03:11:05.211 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:05.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.325+0000 7fdadd29c700 1 Processor -- start 2023-10-06T03:11:05.326 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.325+0000 7fdadd29c700 1 -- start start 2023-10-06T03:11:05.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.326+0000 7fdadd29c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 0x7fdad815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.326+0000 7fdadd29c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad815a160 con 0x7fdad8069560 2023-10-06T03:11:05.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.326+0000 7fdadd29c700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad80d13a0 con 0x7fdad8159c60 2023-10-06T03:11:05.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.326+0000 7fdadd29c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad815e140 con 0x7fdad8159890 2023-10-06T03:11:05.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.326+0000 7fdad77fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdad8069560 0x7fdad80ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45266/0 (socket says 172.21.15.92:45266) 2023-10-06T03:11:05.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.326+0000 7fdad77fe700 1 -- 172.21.15.92:0/963583095 learned_addr learned my addr 172.21.15.92:0/963583095 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 0x7fdad815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130114541 0 0) 0x7fdad815a160 con 0x7fdad8069560 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac4003570 con 0x7fdad8069560 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650228753 0 0) 0x7fdac4003570 con 0x7fdad8069560 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 msgr2=0x7fdad815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 0x7fdad815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 >> v1:172.21.15.136:6789/0 conn(0x7fdad8159c60 legacy=0x7fdad815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad815a3d0 con 0x7fdad8069560 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fdad00017f0 con 0x7fdad8069560 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 >> v1:172.21.15.92:6789/0 conn(0x7fdad8069560 legacy=0x7fdad80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac4003a90 0x7fdac4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.331 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4004810 0x7fdac4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 0x7fdac40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad815e140 con 0x7fdac4003a90 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac400bb10 con 0x7fdac4004810 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac400bcd0 con 0x7fdad8159890 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac4003a90 0x7fdac4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdace7fc700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4004810 0x7fdac4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad77fe700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 0x7fdac40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 -- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 msgr2=0x7fdac40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad8159890 0x7fdac40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 -- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4004810 msgr2=0x7fdac4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4004810 0x7fdac4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 -- 172.21.15.92:0/963583095 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac4003570 con 0x7fdac4003a90 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.327+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac4003a90 0x7fdac4003e60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009c00 tx=0x7fdac80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4068ba8435f1298 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800e710 con 0x7fdac4003a90 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac800e870 con 0x7fdac4003a90 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdad67fc700 1 -- 172.21.15.92:0/963583095 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800ec30 con 0x7fdac4003a90 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdadd29c700 1 -- 172.21.15.92:0/963583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac4003a90 msgr2=0x7fdac4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdadd29c700 1 --2- 172.21.15.92:0/963583095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdac4003a90 0x7fdac4003e60 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009c00 tx=0x7fdac80091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdad77fe700 1 -- 172.21.15.92:0/963583095 reap_dead start 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdadd29c700 1 -- 172.21.15.92:0/963583095 shutdown_connections 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdadd29c700 1 -- 172.21.15.92:0/963583095 >> 172.21.15.92:0/963583095 conn(0x7fdad8068e20 msgr2=0x7fdad80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdadd29c700 1 -- 172.21.15.92:0/963583095 shutdown_connections 2023-10-06T03:11:05.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.328+0000 7fdadd29c700 1 -- 172.21.15.92:0/963583095 wait complete. 2023-10-06T03:11:05.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.329+0000 7fdadd29c700 1 Processor -- start 2023-10-06T03:11:05.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.329+0000 7fdadd29c700 1 -- start start 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdadd29c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4003a90 0x7fdad80c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdadd29c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 0x7fdad8151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdadd29c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80cd0c0 0x7fdad80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdadd29c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad80cda80 con 0x7fdad80b15a0 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdadd29c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8151970 con 0x7fdac4003a90 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdadd29c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad80b1d50 con 0x7fdad80cd0c0 2023-10-06T03:11:05.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdace7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 0x7fdad8151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdace7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 0x7fdad8151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33796/0 (socket says 172.21.15.92:33796) 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdace7fc700 1 -- 172.21.15.92:0/3013153834 learned_addr learned my addr 172.21.15.92:0/3013153834 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4003a90 0x7fdad80c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.330+0000 7fdad77fe700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80cd0c0 0x7fdad80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdace7fc700 1 -- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80cd0c0 msgr2=0x7fdad80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdace7fc700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80cd0c0 0x7fdad80cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdace7fc700 1 -- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4003a90 msgr2=0x7fdad80c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdace7fc700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4003a90 0x7fdad80c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdace7fc700 1 -- 172.21.15.92:0/3013153834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad80b2030 con 0x7fdad80b15a0 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdace7fc700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 0x7fdad8151430 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e860 tx=0x7fdac0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0012590 con 0x7fdad80b15a0 2023-10-06T03:11:05.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0012710 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad80b2310 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0012ad0 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.331+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad80b1a10 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.332+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad80b2550 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.332+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdac001d460 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.333+0000 7fdacf7fe700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.333+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fdac0084fe0 con 0x7fdad80b15a0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.333+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.333+0000 7fdad6ffd700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdac80091e0 tx=0x7fdac801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.337+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdad80b2550 con 0x7fdad80b15a0 2023-10-06T03:11:05.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.520+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdad80c8740 con 0x7fdad80b15a0 2023-10-06T03:11:05.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.521+0000 7fdacf7fe700 1 -- 172.21.15.92:0/3013153834 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fdad80c8740 con 0x7fdad80b15a0 2023-10-06T03:11:05.522 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:05.522 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":189,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:02.673835+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs.volume_0000000000445601.meta","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000445601"}}},{"pool":33,"pool_name":"cephfs.volume_0000000000445601.data","create_time":"2023-10-06T03:10:42.946799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000445601"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:05.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 msgr2=0x7fdab80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdac80091e0 tx=0x7fdac801a000 comp rx=0 tx=0).stop 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 msgr2=0x7fdad8151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 0x7fdad8151430 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fdac000e860 tx=0x7fdac0009040 comp rx=0 tx=0).stop 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 shutdown_connections 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdab8067320 0x7fdab80697d0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdad80cd0c0 0x7fdad80cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac4003a90 0x7fdad80c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.523+0000 7fdadd29c700 1 --2- 172.21.15.92:0/3013153834 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdad80b15a0 0x7fdad8151430 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.524+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 >> 172.21.15.92:0/3013153834 conn(0x7fdad8068e20 msgr2=0x7fdad8152480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.524+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 shutdown_connections 2023-10-06T03:11:05.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.524+0000 7fdadd29c700 1 -- 172.21.15.92:0/3013153834 wait complete. 2023-10-06T03:11:05.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 32, 'pool_name': 'cephfs.volume_0000000000445601.meta', 'create_time': '2023-10-06T03:10:41.617289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000445601'}}}, {'pool': 33, 'pool_name': 'cephfs.volume_0000000000445601.data', 'create_time': '2023-10-06T03:10:42.946799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000445601'}}}] 2023-10-06T03:11:05.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:11:05.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.654+0000 7f33c7fe1700 1 Processor -- start 2023-10-06T03:11:05.655 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.655+0000 7f33c7fe1700 1 -- start start 2023-10-06T03:11:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.655+0000 7f33c7fe1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f33c006b160 0x7f33c006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.655+0000 7f33c7fe1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 0x7f33c006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.655+0000 7f33c7fe1700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c00ac450 con 0x7f33c00637d0 2023-10-06T03:11:05.658 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.655+0000 7f33c7fe1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c00693d0 con 0x7f33c006b160 2023-10-06T03:11:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.655+0000 7f33c7fe1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0069d00 con 0x7f33c006a2b0 2023-10-06T03:11:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 0x7f33c006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c5d7d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f33c006b160 0x7f33c006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 0x7f33c006a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:35700/0 (socket says 172.21.15.92:35700) 2023-10-06T03:11:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c5d7d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f33c006b160 0x7f33c006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:38984/0 (socket says 172.21.15.92:38984) 2023-10-06T03:11:05.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 -- 172.21.15.92:0/2345452169 learned_addr learned my addr 172.21.15.92:0/2345452169 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c557c700 1 --1- 172.21.15.92:0/2345452169 >> v1:172.21.15.136:6789/0 conn(0x7f33c00637d0 0x7f33c0068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:56668/0 (socket says 172.21.15.92:56668) 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 -- 172.21.15.92:0/2345452169 >> v2:172.21.15.136:3300/0 conn(0x7f33c006b160 msgr2=0x7f33c006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 --2- 172.21.15.92:0/2345452169 >> v2:172.21.15.136:3300/0 conn(0x7f33c006b160 0x7f33c006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 -- 172.21.15.92:0/2345452169 >> v1:172.21.15.136:6789/0 conn(0x7f33c00637d0 legacy=0x7f33c0068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 -- 172.21.15.92:0/2345452169 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c0068830 con 0x7f33c006a2b0 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c657e700 1 --2- 172.21.15.92:0/2345452169 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 0x7f33c006a6f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f33b80099b0 tx=0x7f33b8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16da1054318cd38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c4d7b700 1 -- 172.21.15.92:0/2345452169 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8009ea0 con 0x7f33c006a2b0 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c4d7b700 1 -- 172.21.15.92:0/2345452169 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b800b860 con 0x7f33c006a2b0 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.656+0000 7f33c4d7b700 1 -- 172.21.15.92:0/2345452169 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b800bc20 con 0x7f33c006a2b0 2023-10-06T03:11:05.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 -- 172.21.15.92:0/2345452169 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 msgr2=0x7f33c006a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/2345452169 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 0x7f33c006a6f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f33b80099b0 tx=0x7f33b8009250 comp rx=0 tx=0).stop 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 -- 172.21.15.92:0/2345452169 shutdown_connections 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/2345452169 >> v2:172.21.15.136:3300/0 conn(0x7f33c006b160 0x7f33c006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/2345452169 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c006a2b0 0x7f33c006a6f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 -- 172.21.15.92:0/2345452169 >> 172.21.15.92:0/2345452169 conn(0x7f33c006ded0 msgr2=0x7f33c006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 -- 172.21.15.92:0/2345452169 shutdown_connections 2023-10-06T03:11:05.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.657+0000 7f33c7fe1700 1 -- 172.21.15.92:0/2345452169 wait complete. 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.658+0000 7f33c7fe1700 1 Processor -- start 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.658+0000 7f33c7fe1700 1 -- start start 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c7fe1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c00637d0 0x7f33c014ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c7fe1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c006a2b0 0x7f33c00b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c7fe1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 0x7f33c00b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c7fe1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0155f10 con 0x7f33c006b160 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c7fe1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c014f260 con 0x7f33c006a2b0 2023-10-06T03:11:05.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c7fe1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c0155320 con 0x7f33c00637d0 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 0x7f33c00b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 0x7f33c00b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33798/0 (socket says 172.21.15.92:33798) 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 -- 172.21.15.92:0/1336912242 learned_addr learned my addr 172.21.15.92:0/1336912242 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c5d7d700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c00637d0 0x7f33c014ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c557c700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c006a2b0 0x7f33c00b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 -- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c00637d0 msgr2=0x7f33c014ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c00637d0 0x7f33c014ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 -- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c006a2b0 msgr2=0x7f33c00b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c006a2b0 0x7f33c00b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 -- 172.21.15.92:0/1336912242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80094f0 con 0x7f33c006b160 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.659+0000 7f33c657e700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 0x7f33c00b2130 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006fd0 tx=0x7f33b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.660+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b801f070 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.660+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b800bbe0 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.660+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c01555e0 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.660+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8009ea0 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.660+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c0155ba0 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.661+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c004c3c0 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.661+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f33b8011840 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.662+0000 7f33b5ffb700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33a4067320 0x7f33a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.662+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f33b80862c0 con 0x7f33c006b160 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.662+0000 7f33c5d7d700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33a4067320 0x7f33a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.662+0000 7f33c5d7d700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33a4067320 0x7f33a40697d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00f570 tx=0x7f33bc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.666+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33b8052ed0 con 0x7f33c006b160 2023-10-06T03:11:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.851+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33c0069740 con 0x7f33c006b160 2023-10-06T03:11:05.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.851+0000 7f33b5ffb700 1 -- 172.21.15.92:0/1336912242 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33b80564f0 con 0x7f33c006b160 2023-10-06T03:11:05.852 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:11:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33a4067320 msgr2=0x7f33a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33a4067320 0x7f33a40697d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00f570 tx=0x7f33bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:11:05.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 msgr2=0x7f33c00b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 0x7f33c00b2130 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f33b8006fd0 tx=0x7f33b8017000 comp rx=0 tx=0).stop 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 shutdown_connections 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33a4067320 0x7f33a40697d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c006b160 0x7f33c00b2130 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33c00637d0 0x7f33c014ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 --2- 172.21.15.92:0/1336912242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f33c006a2b0 0x7f33c00b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 >> 172.21.15.92:0/1336912242 conn(0x7f33c006ded0 msgr2=0x7f33c006e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 shutdown_connections 2023-10-06T03:11:05.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.853+0000 7f33c7fe1700 1 -- 172.21.15.92:0/1336912242 wait complete. 2023-10-06T03:11:05.866 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:11:05.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.982+0000 7fd396f04700 1 Processor -- start 2023-10-06T03:11:05.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.982+0000 7fd396f04700 1 -- start start 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.983+0000 7fd396f04700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd39006bcd0 0x7fd3900caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.983+0000 7fd396f04700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 0x7fd39015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.983+0000 7fd396f04700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39015d7e0 con 0x7fd39015df50 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.983+0000 7fd396f04700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39015dac0 con 0x7fd39006bcd0 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.983+0000 7fd396f04700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39015aa30 con 0x7fd39006c0a0 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 0x7fd39015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd39006bcd0 0x7fd3900caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 0x7fd39015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:35708/0 (socket says 172.21.15.92:35708) 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd39006bcd0 0x7fd3900caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:39004/0 (socket says 172.21.15.92:39004) 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd38ffff700 1 -- 172.21.15.92:0/1589711967 learned_addr learned my addr 172.21.15.92:0/1589711967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd3954a1700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd39015df50 0x7fd3900c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:56682/0 (socket says 172.21.15.92:56682) 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 -- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 msgr2=0x7fd39015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 --2- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 0x7fd39015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 -- 172.21.15.92:0/1589711967 >> v1:172.21.15.136:6789/0 conn(0x7fd39015df50 legacy=0x7fd3900c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd38f7fe700 1 -- 172.21.15.92:0/1589711967 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638638484 0 0) 0x7fd39015d7e0 con 0x7fd39015df50 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 -- 172.21.15.92:0/1589711967 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39015ad10 con 0x7fd39006bcd0 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd38ffff700 1 --2- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 0x7fd39015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.984+0000 7fd394ca0700 1 --2- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006bcd0 0x7fd3900caa90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd380009e60 tx=0x7fd380009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3345e1d114c1f14e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd38f7fe700 1 -- 172.21.15.92:0/1589711967 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000e070 con 0x7fd39006bcd0 2023-10-06T03:11:05.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd38f7fe700 1 -- 172.21.15.92:0/1589711967 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38000bb60 con 0x7fd39006bcd0 2023-10-06T03:11:05.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd38f7fe700 1 -- 172.21.15.92:0/1589711967 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380012430 con 0x7fd39006bcd0 2023-10-06T03:11:05.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 -- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006bcd0 msgr2=0x7fd3900caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 --2- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006bcd0 0x7fd3900caa90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd380009e60 tx=0x7fd380009780 comp rx=0 tx=0).stop 2023-10-06T03:11:05.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 -- 172.21.15.92:0/1589711967 shutdown_connections 2023-10-06T03:11:05.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 --2- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006bcd0 0x7fd3900caa90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 --2- 172.21.15.92:0/1589711967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006c0a0 0x7fd39015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 -- 172.21.15.92:0/1589711967 >> 172.21.15.92:0/1589711967 conn(0x7fd3900ce600 msgr2=0x7fd3900cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 -- 172.21.15.92:0/1589711967 shutdown_connections 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.985+0000 7fd396f04700 1 -- 172.21.15.92:0/1589711967 wait complete. 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.986+0000 7fd396f04700 1 Processor -- start 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.986+0000 7fd396f04700 1 -- start start 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd396f04700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006bcd0 0x7fd3900b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd396f04700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006c0a0 0x7fd39015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd396f04700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 0x7fd39015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd396f04700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39016e7f0 con 0x7fd39015df50 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd396f04700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39016e360 con 0x7fd39006c0a0 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd396f04700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39016b260 con 0x7fd39006bcd0 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd3954a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 0x7fd39015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd3954a1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 0x7fd39015b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33814/0 (socket says 172.21.15.92:33814) 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd3954a1700 1 -- 172.21.15.92:0/1526166135 learned_addr learned my addr 172.21.15.92:0/1526166135 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd394ca0700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006bcd0 0x7fd3900b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.987+0000 7fd38ffff700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006c0a0 0x7fd39015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd3954a1700 1 -- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006bcd0 msgr2=0x7fd3900b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd3954a1700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006bcd0 0x7fd3900b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd3954a1700 1 -- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006c0a0 msgr2=0x7fd39015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd3954a1700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006c0a0 0x7fd39015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd3954a1700 1 -- 172.21.15.92:0/1526166135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840094f0 con 0x7fd39015df50 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd38ffff700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006c0a0 0x7fd39015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd394ca0700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006bcd0 0x7fd3900b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd3954a1700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 0x7fd39015b8a0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c5b0 tx=0x7fd388007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388005a70 con 0x7fd39015df50 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd388005bd0 con 0x7fd39015df50 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380009a20 con 0x7fd39015df50 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880070d0 con 0x7fd39015df50 2023-10-06T03:11:05.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.988+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39016dae0 con 0x7fd39015df50 2023-10-06T03:11:05.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.989+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39016e510 con 0x7fd39015df50 2023-10-06T03:11:05.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.989+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd388007440 con 0x7fd39015df50 2023-10-06T03:11:05.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.990+0000 7fd38cff9700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd37c067320 0x7fd37c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:05.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.990+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fd388084390 con 0x7fd39015df50 2023-10-06T03:11:05.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.990+0000 7fd394ca0700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd37c067320 0x7fd37c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:05.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.990+0000 7fd394ca0700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd37c067320 0x7fd37c0697d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd390157d80 tx=0x7fd380019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:05.994 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:05.994+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd39016e510 con 0x7fd39015df50 2023-10-06T03:11:06.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.178+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd39016e510 con 0x7fd39015df50 2023-10-06T03:11:06.179 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.179+0000 7fd38cff9700 1 -- 172.21.15.92:0/1526166135 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v189) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd388017020 con 0x7fd39015df50 2023-10-06T03:11:06.179 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd37c067320 msgr2=0x7fd37c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd37c067320 0x7fd37c0697d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd390157d80 tx=0x7fd380019000 comp rx=0 tx=0).stop 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 msgr2=0x7fd39015b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 0x7fd39015b8a0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fd38800c5b0 tx=0x7fd388007d10 comp rx=0 tx=0).stop 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 shutdown_connections 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd37c067320 0x7fd37c0697d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd39015df50 0x7fd39015b8a0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39006bcd0 0x7fd3900b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 --2- 172.21.15.92:0/1526166135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd39006c0a0 0x7fd39015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 >> 172.21.15.92:0/1526166135 conn(0x7fd3900ce600 msgr2=0x7fd390156960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 shutdown_connections 2023-10-06T03:11:06.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.181+0000 7fd396f04700 1 -- 172.21.15.92:0/1526166135 wait complete. 2023-10-06T03:11:06.194 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000445601.meta pg_num 2023-10-06T03:11:06.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.310+0000 7f1acd339700 1 Processor -- start 2023-10-06T03:11:06.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.310+0000 7f1acd339700 1 -- start start 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1acd339700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8159890 0x7f1ac815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1acd339700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ac8069560 0x7f1ac80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1acd339700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac815a160 con 0x7f1ac8159c60 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1acd339700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac8069f20 con 0x7f1ac8069560 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1acd339700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80d13a0 con 0x7f1ac8159890 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ac8069560 0x7f1ac80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ac8069560 0x7f1ac80699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33830/0 (socket says 172.21.15.92:33830) 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1abe7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1ac8159c60 0x7f1ac815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45270/0 (socket says 172.21.15.92:45270) 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 -- 172.21.15.92:0/3303529445 learned_addr learned my addr 172.21.15.92:0/3303529445 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac6ffd700 1 --2- 172.21.15.92:0/3303529445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8159890 0x7f1ac815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 -- 172.21.15.92:0/3303529445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8159890 msgr2=0x7f1ac815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 --2- 172.21.15.92:0/3303529445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8159890 0x7f1ac815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 -- 172.21.15.92:0/3303529445 >> v1:172.21.15.92:6789/0 conn(0x7f1ac8159c60 legacy=0x7f1ac815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 -- 172.21.15.92:0/3303529445 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac815e140 con 0x7f1ac8069560 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac67fc700 1 -- 172.21.15.92:0/3303529445 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653721875 0 0) 0x7f1ac815a160 con 0x7f1ac8159c60 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac77fe700 1 --2- 172.21.15.92:0/3303529445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8069560 0x7f1ac80699e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009f20 tx=0x7f1ac0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a226b716aefe8d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac67fc700 1 -- 172.21.15.92:0/3303529445 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000e070 con 0x7f1ac8069560 2023-10-06T03:11:06.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac67fc700 1 -- 172.21.15.92:0/3303529445 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac000ba80 con 0x7f1ac8069560 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.311+0000 7f1ac67fc700 1 -- 172.21.15.92:0/3303529445 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00092e0 con 0x7f1ac8069560 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 -- 172.21.15.92:0/3303529445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8069560 msgr2=0x7f1ac80699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 --2- 172.21.15.92:0/3303529445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8069560 0x7f1ac80699e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009f20 tx=0x7f1ac0009840 comp rx=0 tx=0).stop 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 -- 172.21.15.92:0/3303529445 shutdown_connections 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 --2- 172.21.15.92:0/3303529445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8159890 0x7f1ac815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 --2- 172.21.15.92:0/3303529445 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8069560 0x7f1ac80699e0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 -- 172.21.15.92:0/3303529445 >> 172.21.15.92:0/3303529445 conn(0x7f1ac8068e20 msgr2=0x7f1ac80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 -- 172.21.15.92:0/3303529445 shutdown_connections 2023-10-06T03:11:06.315 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.312+0000 7f1acd339700 1 -- 172.21.15.92:0/3303529445 wait complete. 2023-10-06T03:11:06.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.313+0000 7f1acd339700 1 Processor -- start 2023-10-06T03:11:06.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.313+0000 7f1acd339700 1 -- start start 2023-10-06T03:11:06.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1acd339700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8069560 0x7f1ac8142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1acd339700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8159890 0x7f1ac80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1acd339700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 0x7f1ac815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1acd339700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac815c940 con 0x7f1ac8159c60 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1acd339700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac815cf00 con 0x7f1ac8159890 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1acd339700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8154900 con 0x7f1ac8069560 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 0x7f1ac815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 0x7f1ac815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33836/0 (socket says 172.21.15.92:33836) 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 -- 172.21.15.92:0/1294375232 learned_addr learned my addr 172.21.15.92:0/1294375232 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 -- 172.21.15.92:0/1294375232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8069560 msgr2=0x7f1ac8142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8069560 0x7f1ac8142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 -- 172.21.15.92:0/1294375232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8159890 msgr2=0x7f1ac80c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8159890 0x7f1ac80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.314+0000 7f1ac77fe700 1 -- 172.21.15.92:0/1294375232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0009ae0 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.315+0000 7f1ac77fe700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 0x7f1ac815b4d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000b6c0 tx=0x7f1ac0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.315+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00032f0 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.315+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0025460 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.315+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac815c090 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.315+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0025820 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.315+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac815cb90 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.316+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac804c3c0 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.316+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1ac0003040 con 0x7f1ac8159c60 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.316+0000 7f1abf7fe700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1aa8067320 0x7f1aa80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.316+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f1ac0087030 con 0x7f1ac8159c60 2023-10-06T03:11:06.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.317+0000 7f1ac6ffd700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1aa8067320 0x7f1aa80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.317+0000 7f1ac6ffd700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1aa8067320 0x7f1aa80697d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009a60 tx=0x7f1ab8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.321 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.321+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ac0053a00 con 0x7f1ac8159c60 2023-10-06T03:11:06.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.500+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000445601.meta", "var": "pg_num"} v 0) v1 -- 0x7f1ac815d040 con 0x7f1ac8159c60 2023-10-06T03:11:06.501 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.501+0000 7f1abf7fe700 1 -- 172.21.15.92:0/1294375232 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000445601.meta", "var": "pg_num"}]=0 v189) v1 ==== 124+0+11 (secure 0 0 0) 0x7f1ac815d040 con 0x7f1ac8159c60 2023-10-06T03:11:06.502 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1aa8067320 msgr2=0x7f1aa80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1aa8067320 0x7f1aa80697d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009a60 tx=0x7f1ab8008040 comp rx=0 tx=0).stop 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 msgr2=0x7f1ac815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 0x7f1ac815b4d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000b6c0 tx=0x7f1ac0018000 comp rx=0 tx=0).stop 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 shutdown_connections 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1aa8067320 0x7f1aa80697d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ac8159c60 0x7f1ac815b4d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac8069560 0x7f1ac8142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 --2- 172.21.15.92:0/1294375232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8159890 0x7f1ac80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 >> 172.21.15.92:0/1294375232 conn(0x7f1ac8068e20 msgr2=0x7f1ac80c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:06.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 shutdown_connections 2023-10-06T03:11:06.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.503+0000 7f1acd339700 1 -- 172.21.15.92:0/1294375232 wait complete. 2023-10-06T03:11:06.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000445601.data pg_num 2023-10-06T03:11:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 Processor -- start 2023-10-06T03:11:06.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 -- start start 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f30fc159890 0x7f30fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30fc159c60 0x7f30fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc15a160 con 0x7f30fc15db10 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc15df50 con 0x7f30fc159c60 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.633+0000 7f3102b28700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0d13a0 con 0x7f30fc159890 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30fc159c60 0x7f30fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f30fc159c60 0x7f30fc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33842/0 (socket says 172.21.15.92:33842) 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 -- 172.21.15.92:0/2942269010 learned_addr learned my addr 172.21.15.92:0/2942269010 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:06.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f31008c4700 1 --2- 172.21.15.92:0/2942269010 >> v2:172.21.15.136:3300/0 conn(0x7f30fc159890 0x7f30fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 -- 172.21.15.92:0/2942269010 >> v2:172.21.15.136:3300/0 conn(0x7f30fc159890 msgr2=0x7f30fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 --2- 172.21.15.92:0/2942269010 >> v2:172.21.15.136:3300/0 conn(0x7f30fc159890 0x7f30fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 -- 172.21.15.92:0/2942269010 >> v1:172.21.15.136:6789/0 conn(0x7f30fc15db10 legacy=0x7f30fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 -- 172.21.15.92:0/2942269010 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc15e140 con 0x7f30fc159c60 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f3fff700 1 --2- 172.21.15.92:0/2942269010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159c60 0x7f30fc15d5d0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0d1090 tx=0x7f30ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fffa66c121109ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f37fe700 1 -- 172.21.15.92:0/2942269010 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00b860 con 0x7f30fc159c60 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f37fe700 1 -- 172.21.15.92:0/2942269010 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec00b9c0 con 0x7f30fc159c60 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.634+0000 7f30f37fe700 1 -- 172.21.15.92:0/2942269010 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00bd60 con 0x7f30fc159c60 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 -- 172.21.15.92:0/2942269010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159c60 msgr2=0x7f30fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 --2- 172.21.15.92:0/2942269010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159c60 0x7f30fc15d5d0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0d1090 tx=0x7f30ec009250 comp rx=0 tx=0).stop 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 -- 172.21.15.92:0/2942269010 shutdown_connections 2023-10-06T03:11:06.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 --2- 172.21.15.92:0/2942269010 >> v2:172.21.15.136:3300/0 conn(0x7f30fc159890 0x7f30fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 --2- 172.21.15.92:0/2942269010 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159c60 0x7f30fc15d5d0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 -- 172.21.15.92:0/2942269010 >> 172.21.15.92:0/2942269010 conn(0x7f30fc068e20 msgr2=0x7f30fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 -- 172.21.15.92:0/2942269010 shutdown_connections 2023-10-06T03:11:06.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.635+0000 7f3102b28700 1 -- 172.21.15.92:0/2942269010 wait complete. 2023-10-06T03:11:06.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.636+0000 7f3102b28700 1 Processor -- start 2023-10-06T03:11:06.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.636+0000 7f3102b28700 1 -- start start 2023-10-06T03:11:06.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 0x7f30fc0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc159c60 0x7f30fc0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc15db10 0x7f30fc0b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc153e60 con 0x7f30fc159890 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc169f30 con 0x7f30fc159c60 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc16aa50 con 0x7f30fc15db10 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 0x7f30fc0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 0x7f30fc0b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33850/0 (socket says 172.21.15.92:33850) 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 -- 172.21.15.92:0/3762132600 learned_addr learned my addr 172.21.15.92:0/3762132600 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f30f3fff700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc159c60 0x7f30fc0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31010c5700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc15db10 0x7f30fc0b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 -- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc15db10 msgr2=0x7f30fc0b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc15db10 0x7f30fc0b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 -- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc159c60 msgr2=0x7f30fc0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc159c60 0x7f30fc0b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 -- 172.21.15.92:0/3762132600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec0094f0 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f31008c4700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 0x7f30fc0b4f70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e190 tx=0x7f30f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f800ee30 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f8014420 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f80147e0 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc16a120 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.637+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc16a6e0 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.638+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc04c3c0 con 0x7f30fc159890 2023-10-06T03:11:06.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.639+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f30f8014b90 con 0x7f30fc159890 2023-10-06T03:11:06.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.639+0000 7f30f0ff9700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30e8067320 0x7f30e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.639+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f30f80856e0 con 0x7f30fc159890 2023-10-06T03:11:06.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.639+0000 7f30f3fff700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30e8067320 0x7f30e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.640+0000 7f30f3fff700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30e8067320 0x7f30e80697d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0d1090 tx=0x7f30ec017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.643+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30f8052180 con 0x7f30fc159890 2023-10-06T03:11:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.827+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000445601.data", "var": "pg_num"} v 0) v1 -- 0x7f30fc16ad30 con 0x7f30fc159890 2023-10-06T03:11:06.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.827+0000 7f30f0ff9700 1 -- 172.21.15.92:0/3762132600 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000445601.data", "var": "pg_num"}]=0 v189) v1 ==== 124+0+11 (secure 0 0 0) 0x7f30f8009a60 con 0x7f30fc159890 2023-10-06T03:11:06.828 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:11:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30e8067320 msgr2=0x7f30e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30e8067320 0x7f30e80697d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0d1090 tx=0x7f30ec017000 comp rx=0 tx=0).stop 2023-10-06T03:11:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 msgr2=0x7f30fc0b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 0x7f30fc0b4f70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e190 tx=0x7f30f8009920 comp rx=0 tx=0).stop 2023-10-06T03:11:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 shutdown_connections 2023-10-06T03:11:06.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f30e8067320 0x7f30e80697d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30fc159890 0x7f30fc0b4f70 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc15db10 0x7f30fc0b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 --2- 172.21.15.92:0/3762132600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc159c60 0x7f30fc0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 >> 172.21.15.92:0/3762132600 conn(0x7f30fc068e20 msgr2=0x7f30fc0d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 shutdown_connections 2023-10-06T03:11:06.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.829+0000 7f3102b28700 1 -- 172.21.15.92:0/3762132600 wait complete. 2023-10-06T03:11:06.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.954+0000 7f2692f9e700 1 Processor -- start 2023-10-06T03:11:06.955 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2692f9e700 1 -- start start 2023-10-06T03:11:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2692f9e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f268c159420 0x7f268c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.957 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2692f9e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2692f9e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0b11d0 con 0x7f268c15d230 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2692f9e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0b14b0 con 0x7f268c159420 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2692f9e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c0b1790 con 0x7f268c1597f0 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.955+0000 7f2691f9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f268c159420 0x7f268c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f268c159420 0x7f268c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33860/0 (socket says 172.21.15.92:33860) 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 -- 172.21.15.92:0/2787203454 learned_addr learned my addr 172.21.15.92:0/2787203454 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f269179b700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 -- 172.21.15.92:0/2787203454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 msgr2=0x7f268c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 -- 172.21.15.92:0/2787203454 >> v1:172.21.15.136:6789/0 conn(0x7f268c15d230 legacy=0x7f268c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:06.958 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 -- 172.21.15.92:0/2787203454 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c0b1a70 con 0x7f268c159420 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f269179b700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2691f9c700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159420 0x7f268c0b5f40 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2688009f20 tx=0x7f2688009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f941d5babe4693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2690f9a700 1 -- 172.21.15.92:0/2787203454 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800e070 con 0x7f268c159420 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2690f9a700 1 -- 172.21.15.92:0/2787203454 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268800ba40 con 0x7f268c159420 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.956+0000 7f2690f9a700 1 -- 172.21.15.92:0/2787203454 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880092e0 con 0x7f268c159420 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 -- 172.21.15.92:0/2787203454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159420 msgr2=0x7f268c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159420 0x7f268c0b5f40 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2688009f20 tx=0x7f2688009840 comp rx=0 tx=0).stop 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 -- 172.21.15.92:0/2787203454 shutdown_connections 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c159420 0x7f268c0b5f40 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 --2- 172.21.15.92:0/2787203454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 -- 172.21.15.92:0/2787203454 >> 172.21.15.92:0/2787203454 conn(0x7f268c0d0660 msgr2=0x7f268c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:06.959 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 -- 172.21.15.92:0/2787203454 shutdown_connections 2023-10-06T03:11:06.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.957+0000 7f2692f9e700 1 -- 172.21.15.92:0/2787203454 wait complete. 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 Processor -- start 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 -- start start 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f268c159420 0x7f268c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c15ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 0x7f268c15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c15a910 con 0x7f268c15d230 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.958+0000 7f2692f9e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c159d00 con 0x7f268c159420 2023-10-06T03:11:06.962 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2692f9e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c159fe0 con 0x7f268c1597f0 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 0x7f268c15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 0x7f268c15b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33866/0 (socket says 172.21.15.92:33866) 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 -- 172.21.15.92:0/3470255218 learned_addr learned my addr 172.21.15.92:0/3470255218 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269179b700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c15ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2691f9c700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f268c159420 0x7f268c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 -- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 msgr2=0x7f268c15ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c15ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 -- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f268c159420 msgr2=0x7f268c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f268c159420 0x7f268c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 -- 172.21.15.92:0/3470255218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688009ae0 con 0x7f268c15d230 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f269279d700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 0x7f268c15b2e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f268400b4e0 tx=0x7f2684006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400be90 con 0x7f268c15d230 2023-10-06T03:11:06.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c15a2c0 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268400f720 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400f890 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.959+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c157970 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.960+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c15aac0 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.961+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f268400fcc0 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.961+0000 7f2681ffb700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2670067320 0x7f26700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.961+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f2684084620 con 0x7f268c15d230 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.961+0000 7f2691f9c700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2670067320 0x7f26700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:06.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.962+0000 7f2691f9c700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2670067320 0x7f26700697d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2688009880 tx=0x7f2688017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:06.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:06.965+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2684051100 con 0x7f268c15d230 2023-10-06T03:11:07.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.194+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f268c04c3c0 con 0x7f268c15d230 2023-10-06T03:11:07.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.195+0000 7f2681ffb700 1 -- 172.21.15.92:0/3470255218 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f2684054720 con 0x7f268c15d230 2023-10-06T03:11:07.196 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:07.196 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:11:00.658708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"volume_0000000000445601","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:11:07.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2670067320 msgr2=0x7f26700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2670067320 0x7f26700697d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2688009880 tx=0x7f2688017000 comp rx=0 tx=0).stop 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 msgr2=0x7f268c15b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 0x7f268c15b2e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f268400b4e0 tx=0x7f2684006f90 comp rx=0 tx=0).stop 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 shutdown_connections 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2670067320 0x7f26700697d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f268c15d230 0x7f268c15b2e0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f268c159420 0x7f268c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 --2- 172.21.15.92:0/3470255218 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f268c1597f0 0x7f268c15ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 >> 172.21.15.92:0/3470255218 conn(0x7f268c0d0660 msgr2=0x7f268c15d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 shutdown_connections 2023-10-06T03:11:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.197+0000 7f2692f9e700 1 -- 172.21.15.92:0/3470255218 wait complete. 2023-10-06T03:11:07.200 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 128 2023-10-06T03:11:07.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.324+0000 7f8bb6ff6700 1 Processor -- start 2023-10-06T03:11:07.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.324+0000 7f8bb6ff6700 1 -- start start 2023-10-06T03:11:07.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb6ff6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01574e0 0x7f8bb00af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb6ff6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bb01578b0 0x7f8bb00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb6ff6700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8bb006e8c0 0x7f8bb00aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb6ff6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb00b0bc0 con 0x7f8bb01578b0 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb6ff6700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb006c700 con 0x7f8bb006e8c0 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb6ff6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb00affb0 con 0x7f8bb01574e0 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb57f3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bb01578b0 0x7f8bb00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb57f3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8bb01578b0 0x7f8bb00ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33878/0 (socket says 172.21.15.92:33878) 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb57f3700 1 -- 172.21.15.92:0/476395948 learned_addr learned my addr 172.21.15.92:0/476395948 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb67f5700 1 --2- 172.21.15.92:0/476395948 >> v2:172.21.15.136:3300/0 conn(0x7f8bb006e8c0 0x7f8bb00aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.325+0000 7f8bb5ff4700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01574e0 0x7f8bb00af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.333 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb57f3700 1 -- 172.21.15.92:0/476395948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01574e0 msgr2=0x7f8bb00af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb57f3700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01574e0 0x7f8bb00af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb57f3700 1 -- 172.21.15.92:0/476395948 >> v2:172.21.15.136:3300/0 conn(0x7f8bb006e8c0 msgr2=0x7f8bb00aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb57f3700 1 --2- 172.21.15.92:0/476395948 >> v2:172.21.15.136:3300/0 conn(0x7f8bb006e8c0 0x7f8bb00aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb57f3700 1 -- 172.21.15.92:0/476395948 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00b0290 con 0x7f8bb01578b0 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb5ff4700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01574e0 0x7f8bb00af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb57f3700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb01578b0 0x7f8bb00ae840 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8bb006ef20 tx=0x7f8ba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570c70fc076afcbb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb4ff2700 1 -- 172.21.15.92:0/476395948 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0009e70 con 0x7f8bb01578b0 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb4ff2700 1 -- 172.21.15.92:0/476395948 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba000b860 con 0x7f8bb01578b0 2023-10-06T03:11:07.334 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb4ff2700 1 -- 172.21.15.92:0/476395948 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000bc00 con 0x7f8bb01578b0 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/476395948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb01578b0 msgr2=0x7f8bb00ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.326+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb01578b0 0x7f8bb00ae840 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f8bb006ef20 tx=0x7f8ba0009250 comp rx=0 tx=0).stop 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/476395948 shutdown_connections 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01574e0 0x7f8bb00af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/476395948 >> v2:172.21.15.136:3300/0 conn(0x7f8bb006e8c0 0x7f8bb00aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/476395948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb01578b0 0x7f8bb00ae840 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/476395948 >> 172.21.15.92:0/476395948 conn(0x7f8bb006ddb0 msgr2=0x7f8bb006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:07.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/476395948 shutdown_connections 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.327+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/476395948 wait complete. 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 Processor -- start 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 -- start start 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 0x7f8bb013a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bb01574e0 0x7f8bb0069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01578b0 0x7f8bb006a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0158e40 con 0x7f8bb006e8c0 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb006a5b0 con 0x7f8bb01574e0 2023-10-06T03:11:07.336 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.328+0000 7f8bb6ff6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb0158230 con 0x7f8bb01578b0 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 0x7f8bb013a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 0x7f8bb013a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33890/0 (socket says 172.21.15.92:33890) 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 -- 172.21.15.92:0/6274282 learned_addr learned my addr 172.21.15.92:0/6274282 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb57f3700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bb01574e0 0x7f8bb0069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb67f5700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01578b0 0x7f8bb006a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 -- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01578b0 msgr2=0x7f8bb006a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01578b0 0x7f8bb006a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 -- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bb01574e0 msgr2=0x7f8bb0069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bb01574e0 0x7f8bb0069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 -- 172.21.15.92:0/6274282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00094f0 con 0x7f8bb006e8c0 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb5ff4700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 0x7f8bb013a5e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00d1d0 tx=0x7f8bac009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00de70 con 0x7f8bb006e8c0 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00f040 con 0x7f8bb006e8c0 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb0158510 con 0x7f8bb006e8c0 2023-10-06T03:11:07.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac015670 con 0x7f8bb006e8c0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.329+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb0158ad0 con 0x7f8bb006e8c0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.330+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb004c3c0 con 0x7f8bb006e8c0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.330+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8bac015aa0 con 0x7f8bb006e8c0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.331+0000 7f8ba5ffb700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b94067320 0x7f8b940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.331+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f8bac085370 con 0x7f8bb006e8c0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.331+0000 7f8bb57f3700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b94067320 0x7f8b940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.331+0000 7f8bb57f3700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b94067320 0x7f8b940697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8bb006ef20 tx=0x7f8ba0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:07.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.335+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bac04e5a0 con 0x7f8bb006e8c0 2023-10-06T03:11:07.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.520+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bb00b0570 con 0x7f8bb006e8c0 2023-10-06T03:11:07.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.520+0000 7f8ba5ffb700 1 -- 172.21.15.92:0/6274282 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f8bac051df0 con 0x7f8bb006e8c0 2023-10-06T03:11:07.521 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:07.522 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":189,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:02.673835+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs.volume_0000000000445601.meta","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000445601"}}},{"pool":33,"pool_name":"cephfs.volume_0000000000445601.data","create_time":"2023-10-06T03:10:42.946799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000445601"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:07.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b94067320 msgr2=0x7f8b940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b94067320 0x7f8b940697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8bb006ef20 tx=0x7f8ba0017000 comp rx=0 tx=0).stop 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 msgr2=0x7f8bb013a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 0x7f8bb013a5e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00d1d0 tx=0x7f8bac009a90 comp rx=0 tx=0).stop 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 shutdown_connections 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b94067320 0x7f8b940697d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8bb006e8c0 0x7f8bb013a5e0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8bb01578b0 0x7f8bb006a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 --2- 172.21.15.92:0/6274282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8bb01574e0 0x7f8bb0069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 >> 172.21.15.92:0/6274282 conn(0x7f8bb006ddb0 msgr2=0x7f8bb00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:07.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 shutdown_connections 2023-10-06T03:11:07.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.523+0000 7f8bb6ff6700 1 -- 172.21.15.92:0/6274282 wait complete. 2023-10-06T03:11:07.536 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000445601 and related pools 2023-10-06T03:11:07.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:07.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.651+0000 7f7d11812700 1 Processor -- start 2023-10-06T03:11:07.652 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.651+0000 7f7d11812700 1 -- start start 2023-10-06T03:11:07.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.652+0000 7f7d11812700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 0x7f7d0c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.659 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.652+0000 7f7d11812700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c0cc210 con 0x7f7d0c0c91e0 2023-10-06T03:11:07.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.652+0000 7f7d11812700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0c0cc4f0 con 0x7f7d0c06bac0 2023-10-06T03:11:07.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.652+0000 7f7d11812700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0cc7d0 con 0x7f7d0c06aa40 2023-10-06T03:11:07.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.652+0000 7f7d0b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7d0c0c91e0 0x7f7d0c160390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45276/0 (socket says 172.21.15.92:45276) 2023-10-06T03:11:07.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.652+0000 7f7d0b7fe700 1 -- 172.21.15.92:0/3229799423 learned_addr learned my addr 172.21.15.92:0/3229799423 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:07.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0affd700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 0x7f7d0c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531736427 0 0) 0x7f7d0c0cc210 con 0x7f7d0c0c91e0 2023-10-06T03:11:07.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cf8003570 con 0x7f7d0c0c91e0 2023-10-06T03:11:07.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4048498153 0 0) 0x7f7cf8003570 con 0x7f7d0c0c91e0 2023-10-06T03:11:07.661 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 msgr2=0x7f7d0c0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 0x7f7d0c0cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 >> v1:172.21.15.136:6789/0 conn(0x7f7d0c06bac0 legacy=0x7f7d0c06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0ccab0 con 0x7f7d0c0c91e0 2023-10-06T03:11:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f7d040017f0 con 0x7f7d0c0c91e0 2023-10-06T03:11:07.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 >> v1:172.21.15.92:6789/0 conn(0x7f7d0c0c91e0 legacy=0x7f7d0c160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 0x7f7cf8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cf8003fd0 0x7f7cf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7cf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0cc7d0 con 0x7f7cf8004d50 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf800bb40 con 0x7f7cf8003fd0 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf800bd00 con 0x7f7d0c06aa40 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7cf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0affd700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 0x7f7cf8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d027fc700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cf8003fd0 0x7f7cf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 -- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 msgr2=0x7f7cf8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c06aa40 0x7f7cf8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 -- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cf8003fd0 msgr2=0x7f7cf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cf8003fd0 0x7f7cf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 -- 172.21.15.92:0/3229799423 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8003570 con 0x7f7cf8004d50 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.653+0000 7f7d0b7fe700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7cf80051c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400a950 tx=0x7f7d040021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc02d549cc26759 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d040070d0 con 0x7f7cf8004d50 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d04007230 con 0x7f7cf8004d50 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d0a7fc700 1 -- 172.21.15.92:0/3229799423 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d040075f0 con 0x7f7cf8004d50 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d11812700 1 -- 172.21.15.92:0/3229799423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 msgr2=0x7f7cf80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d11812700 1 --2- 172.21.15.92:0/3229799423 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7cf80051c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400a950 tx=0x7f7d040021f0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d0b7fe700 1 -- 172.21.15.92:0/3229799423 reap_dead start 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d11812700 1 -- 172.21.15.92:0/3229799423 shutdown_connections 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d11812700 1 -- 172.21.15.92:0/3229799423 >> 172.21.15.92:0/3229799423 conn(0x7f7d0c0cd670 msgr2=0x7f7d0c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:07.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d11812700 1 -- 172.21.15.92:0/3229799423 shutdown_connections 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.654+0000 7f7d11812700 1 -- 172.21.15.92:0/3229799423 wait complete. 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.655+0000 7f7d11812700 1 Processor -- start 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 -- start start 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7d0c142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d0c0b2ad0 0x7f7d0c1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c149b20 0x7f7d0c0c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0b2640 con 0x7f7cf8004d50 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0b1d10 con 0x7f7d0c0b2ad0 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d11812700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c0b1ff0 con 0x7f7d0c149b20 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7d0c142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.665 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d0affd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7d0c142860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33906/0 (socket says 172.21.15.92:33906) 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.656+0000 7f7d0affd700 1 -- 172.21.15.92:0/1416170531 learned_addr learned my addr 172.21.15.92:0/1416170531 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d0affd700 1 -- 172.21.15.92:0/1416170531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c149b20 msgr2=0x7f7d0c0c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d0affd700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c149b20 0x7f7d0c0c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d0affd700 1 -- 172.21.15.92:0/1416170531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d0c0b2ad0 msgr2=0x7f7d0c1495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d0affd700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d0c0b2ad0 0x7f7d0c1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d0affd700 1 -- 172.21.15.92:0/1416170531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0b22d0 con 0x7f7cf8004d50 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d0affd700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7d0c142860 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc011b10 tx=0x7f7cfc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc005c10 con 0x7f7cf8004d50 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc005d70 con 0x7f7cf8004d50 2023-10-06T03:11:07.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00fa10 con 0x7f7cf8004d50 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c148a50 con 0x7f7cf8004d50 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.657+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c0c3b60 con 0x7f7cf8004d50 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.658+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c0b2460 con 0x7f7cf8004d50 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.658+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7cfc006050 con 0x7f7cf8004d50 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.659+0000 7f7d037fe700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7cec067320 0x7f7cec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.659+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7f7cfc085850 con 0x7f7cf8004d50 2023-10-06T03:11:07.667 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.659+0000 7f7d027fc700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7cec067320 0x7f7cec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.659+0000 7f7d027fc700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7cec067320 0x7f7cec0697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4007e20 tx=0x7f7cf4007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:07.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.663+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cfc052330 con 0x7f7cf8004d50 2023-10-06T03:11:07.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.890+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d0c0b2920 con 0x7f7cf8004d50 2023-10-06T03:11:07.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.891+0000 7f7d037fe700 1 -- 172.21.15.92:0/1416170531 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4624 (secure 0 0 0) 0x7f7cfc055950 con 0x7f7cf8004d50 2023-10-06T03:11:07.891 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:07.892 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:11:00.658708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"volume_0000000000445601","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:11:07.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7cec067320 msgr2=0x7f7cec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7cec067320 0x7f7cec0697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4007e20 tx=0x7f7cf4007d10 comp rx=0 tx=0).stop 2023-10-06T03:11:07.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 msgr2=0x7f7d0c142860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:07.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7d0c142860 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc011b10 tx=0x7f7cfc009120 comp rx=0 tx=0).stop 2023-10-06T03:11:07.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 shutdown_connections 2023-10-06T03:11:07.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7cec067320 0x7f7cec0697d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cf8004d50 0x7f7d0c142860 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d0c149b20 0x7f7d0c0c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 --2- 172.21.15.92:0/1416170531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d0c0b2ad0 0x7f7d0c1495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 >> 172.21.15.92:0/1416170531 conn(0x7f7d0c0cd670 msgr2=0x7f7d0c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 shutdown_connections 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:07.893+0000 7f7d11812700 1 -- 172.21.15.92:0/1416170531 wait complete. 2023-10-06T03:11:07.895 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 128 2023-10-06T03:11:07.906 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.019+0000 7fdc058fe700 1 Processor -- start 2023-10-06T03:11:08.020 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.019+0000 7fdc058fe700 1 -- start start 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc058fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 0x7fdc000cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc058fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdc000cc3c0 0x7fdc0006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc058fe700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0006c380 con 0x7fdc0006bf20 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc058fe700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000b16a0 con 0x7fdc000cc3c0 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc058fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000b1c60 con 0x7fdc001574d0 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc050fd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc0006bf20 0x7fdc0006cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45280/0 (socket says 172.21.15.92:45280) 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc050fd700 1 -- 172.21.15.92:0/2743826704 learned_addr learned my addr 172.21.15.92:0/2743826704 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315020999 0 0) 0x7fdbf8001490 con 0x7fdc0006bf20 2023-10-06T03:11:08.024 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdc048fc700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 0x7fdc000cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbfffff700 1 --2- 172.21.15.92:0/2743826704 >> v2:172.21.15.136:3300/0 conn(0x7fdc000cc3c0 0x7fdc0006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbf0003570 con 0x7fdc0006bf20 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3712540531 0 0) 0x7fdbf0003570 con 0x7fdc0006bf20 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 msgr2=0x7fdc000cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 0x7fdc000cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 >> v2:172.21.15.136:3300/0 conn(0x7fdc000cc3c0 msgr2=0x7fdc0006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 --2- 172.21.15.92:0/2743826704 >> v2:172.21.15.136:3300/0 conn(0x7fdc000cc3c0 0x7fdc0006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.025 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc001578a0 con 0x7fdc0006bf20 2023-10-06T03:11:08.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.020+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fdbf8001980 con 0x7fdc0006bf20 2023-10-06T03:11:08.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 >> v1:172.21.15.92:6789/0 conn(0x7fdc0006bf20 legacy=0x7fdc0006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 0x7fdbf0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdbf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf0004d50 0x7fdbf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000b16a0 con 0x7fdbf0003fe0 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000b1c60 con 0x7fdbf0004d50 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf000ba90 con 0x7fdc001574d0 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdbf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdc048fc700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 0x7fdbf0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdc050fd700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf0004d50 0x7fdbf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 -- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 msgr2=0x7fdbf0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc001574d0 0x7fdbf0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 -- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf0004d50 msgr2=0x7fdbf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:08.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf0004d50 0x7fdbf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 -- 172.21.15.92:0/2743826704 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf0003570 con 0x7fdbf0003fe0 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbfffff700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdbf0004810 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009c00 tx=0x7fdbec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfd460f524a10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00e6b0 con 0x7fdbf0003fe0 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec00e810 con 0x7fdbf0003fe0 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.021+0000 7fdbff7fe700 1 -- 172.21.15.92:0/2743826704 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00ebd0 con 0x7fdbf0003fe0 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc058fe700 1 -- 172.21.15.92:0/2743826704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 msgr2=0x7fdbf0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc058fe700 1 --2- 172.21.15.92:0/2743826704 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdbf0004810 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009c00 tx=0x7fdbec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc050fd700 1 -- 172.21.15.92:0/2743826704 reap_dead start 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc058fe700 1 -- 172.21.15.92:0/2743826704 shutdown_connections 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc058fe700 1 -- 172.21.15.92:0/2743826704 >> 172.21.15.92:0/2743826704 conn(0x7fdc000ce430 msgr2=0x7fdc000ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:08.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc058fe700 1 -- 172.21.15.92:0/2743826704 shutdown_connections 2023-10-06T03:11:08.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.022+0000 7fdc058fe700 1 -- 172.21.15.92:0/2743826704 wait complete. 2023-10-06T03:11:08.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.023+0000 7fdc058fe700 1 Processor -- start 2023-10-06T03:11:08.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.023+0000 7fdc058fe700 1 -- start start 2023-10-06T03:11:08.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdc0006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.031 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc000c75e0 0x7fdc000b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0014c000 0x7fdc0014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0014c9b0 con 0x7fdbf0003fe0 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000c8760 con 0x7fdc0014c000 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0006ad10 con 0x7fdc000c75e0 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdc0006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdc0006a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33934/0 (socket says 172.21.15.92:33934) 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 -- 172.21.15.92:0/3818853563 learned_addr learned my addr 172.21.15.92:0/3818853563 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 -- 172.21.15.92:0/3818853563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc000c75e0 msgr2=0x7fdc000b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc000c75e0 0x7fdc000b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 -- 172.21.15.92:0/3818853563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0014c000 msgr2=0x7fdc0014c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0014c000 0x7fdc0014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 -- 172.21.15.92:0/3818853563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000c7b50 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc048fc700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdc0006a7d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4011b60 tx=0x7fdbf4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4005c10 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4005d70 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc000c7e30 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.024+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400fa60 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.025+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc000c83f0 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.025+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0004c3c0 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.026+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdbf4006050 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.026+0000 7fdbfcff9700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4067320 0x7fdbe40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.026+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fdbf4085580 con 0x7fdbf0003fe0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.026+0000 7fdbfffff700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4067320 0x7fdbe40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.026+0000 7fdbfffff700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4067320 0x7fdbe40697d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc000ca210 tx=0x7fdbec01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.030+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbf4052060 con 0x7fdbf0003fe0 2023-10-06T03:11:08.255 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.255+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc000c8a40 con 0x7fdbf0003fe0 2023-10-06T03:11:08.256 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.255+0000 7fdbfcff9700 1 -- 172.21.15.92:0/3818853563 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4624 (secure 0 0 0) 0x7fdbf4055680 con 0x7fdbf0003fe0 2023-10-06T03:11:08.256 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:08.256 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":128,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:10:43.974246+0000","modified":"2023-10-06T03:11:00.658708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8802":{"gid":8802,"name":"a","rank":0,"incarnation":121,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6837/427612348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":427612348},{"type":"v1","addr":"172.21.15.92:6837","nonce":427612348}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"volume_0000000000445601","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-06T03:11:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4067320 msgr2=0x7fdbe40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.258 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4067320 0x7fdbe40697d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc000ca210 tx=0x7fdbec01a000 comp rx=0 tx=0).stop 2023-10-06T03:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 msgr2=0x7fdc0006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdc0006a7d0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4011b60 tx=0x7fdbf4009120 comp rx=0 tx=0).stop 2023-10-06T03:11:08.259 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 shutdown_connections 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdbe4067320 0x7fdbe40697d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdbf0003fe0 0x7fdc0006a7d0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc0014c000 0x7fdc0014c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 --2- 172.21.15.92:0/3818853563 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc000c75e0 0x7fdc000b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.257+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 >> 172.21.15.92:0/3818853563 conn(0x7fdc000ce430 msgr2=0x7fdc000b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.258+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 shutdown_connections 2023-10-06T03:11:08.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.258+0000 7fdc058fe700 1 -- 172.21.15.92:0/3818853563 wait complete. 2023-10-06T03:11:08.261 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 128 2023-10-06T03:11:08.270 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.385+0000 7fcc3b195700 1 Processor -- start 2023-10-06T03:11:08.386 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.385+0000 7fcc3b195700 1 -- start start 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc3b195700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc34159c60 0x7fcc3415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc3b195700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcc3415db10 0x7fcc34069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc3b195700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3415a160 con 0x7fcc34159890 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc3b195700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3415df50 con 0x7fcc34159c60 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc3b195700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc340d13a0 con 0x7fcc3415db10 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc38f31700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fcc34159890 0x7fcc3415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45288/0 (socket says 172.21.15.92:45288) 2023-10-06T03:11:08.388 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc2bfff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcc34159c60 0x7fcc3415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc38f31700 1 -- 172.21.15.92:0/14530041 learned_addr learned my addr 172.21.15.92:0/14530041 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc39732700 1 --2- 172.21.15.92:0/14530041 >> v2:172.21.15.136:3300/0 conn(0x7fcc3415db10 0x7fcc34069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc2bfff700 1 -- 172.21.15.92:0/14530041 >> v2:172.21.15.136:3300/0 conn(0x7fcc3415db10 msgr2=0x7fcc34069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc2bfff700 1 --2- 172.21.15.92:0/14530041 >> v2:172.21.15.136:3300/0 conn(0x7fcc3415db10 0x7fcc34069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc2bfff700 1 -- 172.21.15.92:0/14530041 >> v1:172.21.15.92:6789/0 conn(0x7fcc34159890 legacy=0x7fcc3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc2bfff700 1 -- 172.21.15.92:0/14530041 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3415e140 con 0x7fcc34159c60 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.386+0000 7fcc2b7fe700 1 -- 172.21.15.92:0/14530041 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516588729 0 0) 0x7fcc3415a160 con 0x7fcc34159890 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc2bfff700 1 --2- 172.21.15.92:0/14530041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc3415d5d0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcc3406a890 tx=0x7fcc240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809f90d5d53f6db1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc2b7fe700 1 -- 172.21.15.92:0/14530041 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24011630 con 0x7fcc34159c60 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc2b7fe700 1 -- 172.21.15.92:0/14530041 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24011790 con 0x7fcc34159c60 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc2b7fe700 1 -- 172.21.15.92:0/14530041 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24011b30 con 0x7fcc34159c60 2023-10-06T03:11:08.389 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 -- 172.21.15.92:0/14530041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 msgr2=0x7fcc3415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 --2- 172.21.15.92:0/14530041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc3415d5d0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcc3406a890 tx=0x7fcc240091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 -- 172.21.15.92:0/14530041 shutdown_connections 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 --2- 172.21.15.92:0/14530041 >> v2:172.21.15.136:3300/0 conn(0x7fcc3415db10 0x7fcc34069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 --2- 172.21.15.92:0/14530041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc3415d5d0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 -- 172.21.15.92:0/14530041 >> 172.21.15.92:0/14530041 conn(0x7fcc34068e20 msgr2=0x7fcc340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 -- 172.21.15.92:0/14530041 shutdown_connections 2023-10-06T03:11:08.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.387+0000 7fcc3b195700 1 -- 172.21.15.92:0/14530041 wait complete. 2023-10-06T03:11:08.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.388+0000 7fcc3b195700 1 Processor -- start 2023-10-06T03:11:08.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 -- start start 2023-10-06T03:11:08.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc34159890 0x7fcc340b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc340b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc3415db10 0x7fcc34153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340b2270 con 0x7fcc34159c60 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340b11d0 con 0x7fcc3415db10 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc3b195700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340b14b0 con 0x7fcc34159890 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc340b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc2bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc340b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33954/0 (socket says 172.21.15.92:33954) 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.389+0000 7fcc2bfff700 1 -- 172.21.15.92:0/1302346043 learned_addr learned my addr 172.21.15.92:0/1302346043 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc39732700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc3415db10 0x7fcc34153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc38f31700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc34159890 0x7fcc340b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc2bfff700 1 -- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc34159890 msgr2=0x7fcc340b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc2bfff700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc34159890 0x7fcc340b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc2bfff700 1 -- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc3415db10 msgr2=0x7fcc34153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc2bfff700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc3415db10 0x7fcc34153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc2bfff700 1 -- 172.21.15.92:0/1302346043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc24009420 con 0x7fcc34159c60 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc38f31700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc34159890 0x7fcc340b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc2bfff700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc340b5e10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fcc34070e20 tx=0x7fcc24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24011b10 con 0x7fcc34159c60 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc240253f0 con 0x7fcc34159c60 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc340b1790 con 0x7fcc34159c60 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24025790 con 0x7fcc34159c60 2023-10-06T03:11:08.395 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.390+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc340b1d50 con 0x7fcc34159c60 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.391+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc340b1f40 con 0x7fcc34159c60 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.391+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcc24025b80 con 0x7fcc34159c60 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.392+0000 7fcc28ff9700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc20067320 0x7fcc200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.392+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fcc240866f0 con 0x7fcc34159c60 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.392+0000 7fcc38f31700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc20067320 0x7fcc200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.392+0000 7fcc38f31700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc20067320 0x7fcc200697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcc30001560 tx=0x7fcc30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.396 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.396+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc24053190 con 0x7fcc34159c60 2023-10-06T03:11:08.581 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.581+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc3404c3c0 con 0x7fcc34159c60 2023-10-06T03:11:08.582 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.582+0000 7fcc28ff9700 1 -- 172.21.15.92:0/1302346043 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fcc240567b0 con 0x7fcc34159c60 2023-10-06T03:11:08.583 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:08.583 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":189,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:02.673835+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":32,"pool_name":"cephfs.volume_0000000000445601.meta","create_time":"2023-10-06T03:10:41.617289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000445601"}}},{"pool":33,"pool_name":"cephfs.volume_0000000000445601.data","create_time":"2023-10-06T03:10:42.946799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000445601"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc20067320 msgr2=0x7fcc200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc20067320 0x7fcc200697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcc30001560 tx=0x7fcc30009040 comp rx=0 tx=0).stop 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 msgr2=0x7fcc340b5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc340b5e10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fcc34070e20 tx=0x7fcc24017000 comp rx=0 tx=0).stop 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 shutdown_connections 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fcc20067320 0x7fcc200697d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc3415db10 0x7fcc34153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc34159890 0x7fcc340b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 --2- 172.21.15.92:0/1302346043 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcc34159c60 0x7fcc340b5e10 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 >> 172.21.15.92:0/1302346043 conn(0x7fcc34068e20 msgr2=0x7fcc34157390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:08.585 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 shutdown_connections 2023-10-06T03:11:08.586 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.584+0000 7fcc3b195700 1 -- 172.21.15.92:0/1302346043 wait complete. 2023-10-06T03:11:08.597 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000445601 2023-10-06T03:11:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.714+0000 7fd38601b700 1 Processor -- start 2023-10-06T03:11:08.715 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.714+0000 7fd38601b700 1 -- start start 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38601b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd380159420 0x7fd3800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38601b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd38015d230 0x7fd38015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38601b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38015dbd0 con 0x7fd3801597f0 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38601b700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3800b2980 con 0x7fd38015d230 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38601b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800b11d0 con 0x7fd380159420 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38581a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd38015d230 0x7fd38015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd384818700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd3801597f0 0x7fd3800b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45302/0 (socket says 172.21.15.92:45302) 2023-10-06T03:11:08.717 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38581a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd38015d230 0x7fd38015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33962/0 (socket says 172.21.15.92:33962) 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd384818700 1 -- 172.21.15.92:0/3250038721 learned_addr learned my addr 172.21.15.92:0/3250038721 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd385019700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd380159420 0x7fd3800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38581a700 1 -- 172.21.15.92:0/3250038721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd380159420 msgr2=0x7fd3800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38581a700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd380159420 0x7fd3800b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38581a700 1 -- 172.21.15.92:0/3250038721 >> v1:172.21.15.92:6789/0 conn(0x7fd3801597f0 legacy=0x7fd3800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd377fff700 1 -- 172.21.15.92:0/3250038721 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127643729 0 0) 0x7fd370001110 con 0x7fd3801597f0 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd38581a700 1 -- 172.21.15.92:0/3250038721 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800b14b0 con 0x7fd38015d230 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.715+0000 7fd385019700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd380159420 0x7fd3800b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38581a700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38015d230 0x7fd38015d690 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fd378009e60 tx=0x7fd378009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b269c0ddb060c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd377fff700 1 -- 172.21.15.92:0/3250038721 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800e070 con 0x7fd38015d230 2023-10-06T03:11:08.718 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd377fff700 1 -- 172.21.15.92:0/3250038721 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800bc30 con 0x7fd38015d230 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd377fff700 1 -- 172.21.15.92:0/3250038721 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378012430 con 0x7fd38015d230 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 -- 172.21.15.92:0/3250038721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38015d230 msgr2=0x7fd38015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38015d230 0x7fd38015d690 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fd378009e60 tx=0x7fd378009780 comp rx=0 tx=0).stop 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 -- 172.21.15.92:0/3250038721 shutdown_connections 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd380159420 0x7fd3800b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 --2- 172.21.15.92:0/3250038721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd38015d230 0x7fd38015d690 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 -- 172.21.15.92:0/3250038721 >> 172.21.15.92:0/3250038721 conn(0x7fd3800d0660 msgr2=0x7fd3800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 -- 172.21.15.92:0/3250038721 shutdown_connections 2023-10-06T03:11:08.719 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.716+0000 7fd38601b700 1 -- 172.21.15.92:0/3250038721 wait complete. 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.717+0000 7fd38601b700 1 Processor -- start 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.717+0000 7fd38601b700 1 -- start start 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd38601b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd380159420 0x7fd3800c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd38601b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 0x7fd38015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd38601b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38015d230 0x7fd380152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd38601b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38016a0e0 con 0x7fd3801597f0 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd38601b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800b6360 con 0x7fd380159420 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd38601b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38015caa0 con 0x7fd38015d230 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd384818700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 0x7fd38015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.718+0000 7fd384818700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 0x7fd38015c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33970/0 (socket says 172.21.15.92:33970) 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 -- 172.21.15.92:0/1370741921 learned_addr learned my addr 172.21.15.92:0/1370741921 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd38581a700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38015d230 0x7fd380152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 -- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38015d230 msgr2=0x7fd380152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38015d230 0x7fd380152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 -- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd380159420 msgr2=0x7fd3800c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd380159420 0x7fd3800c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 -- 172.21.15.92:0/1370741921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378009a20 con 0x7fd3801597f0 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd384818700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 0x7fd38015c3a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c810 tx=0x7fd370007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370005b30 con 0x7fd3801597f0 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd370005c90 con 0x7fd3801597f0 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370007300 con 0x7fd3801597f0 2023-10-06T03:11:08.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3801697f0 con 0x7fd3801597f0 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.719+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd380169a90 con 0x7fd3801597f0 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.720+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38004c3c0 con 0x7fd3801597f0 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.720+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd3700076f0 con 0x7fd3801597f0 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.721+0000 7fd3757fa700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd364067320 0x7fd3640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.721+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6136+0+0 (secure 0 0 0) 0x7fd370084830 con 0x7fd3801597f0 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.721+0000 7fd385019700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd364067320 0x7fd3640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:08.724 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.721+0000 7fd385019700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd364067320 0x7fd3640697d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd380069790 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:08.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.725+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd370054da0 con 0x7fd3801597f0 2023-10-06T03:11:08.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.951+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000445601"} v 0) v1 -- 0x7fd38015dbd0 con 0x7fd3801597f0 2023-10-06T03:11:08.952 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:11:08.951+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:11:08.958 INFO:tasks.ceph.mds.a.smithi092.stderr: -924> 2023-10-06T03:10:47.901+0000 7f7bbc5ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:11:08.960 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.959+0000 7fd3757fa700 1 -- 172.21.15.92:0/1370741921 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000445601"}]=0 volume_0000000000445601 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v129) v1 ==== 195+0+0 (secure 0 0 0) 0x7fd38015dbd0 con 0x7fd3801597f0 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd364067320 msgr2=0x7fd3640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd364067320 0x7fd3640697d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd380069790 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 msgr2=0x7fd38015c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 0x7fd38015c3a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd37000c810 tx=0x7fd370007c50 comp rx=0 tx=0).stop 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 shutdown_connections 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd364067320 0x7fd3640697d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd38015d230 0x7fd380152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd380159420 0x7fd3800c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 --2- 172.21.15.92:0/1370741921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3801597f0 0x7fd38015c3a0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 >> 172.21.15.92:0/1370741921 conn(0x7fd3800d0660 msgr2=0x7fd380069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 shutdown_connections 2023-10-06T03:11:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:08.962+0000 7fd38601b700 1 -- 172.21.15.92:0/1370741921 wait complete. 2023-10-06T03:11:08.964 INFO:teuthology.orchestra.run.smithi092.stderr:volume_0000000000445601 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:11:08.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000445601 --yes-i-really-mean-it 2023-10-06T03:11:08.979 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:11:09.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.091+0000 7fea53fff700 1 Processor -- start 2023-10-06T03:11:09.092 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.091+0000 7fea53fff700 1 -- start start 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea53fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea540c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea53fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fea5406bfa0 0x7fea540c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea53fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540ca920 con 0x7fea54068390 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea53fff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5406af00 con 0x7fea5406bfa0 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea53fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5406b1e0 con 0x7fea54159a70 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea537fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fea54068390 0x7fea540caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:45328/0 (socket says 172.21.15.92:45328) 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea537fe700 1 -- 172.21.15.92:0/3921696158 learned_addr learned my addr 172.21.15.92:0/3921696158 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea4a7fc700 1 --2- 172.21.15.92:0/3921696158 >> v2:172.21.15.136:3300/0 conn(0x7fea5406bfa0 0x7fea540c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea52ffd700 1 --2- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea540c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:09.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea527fc700 1 -- 172.21.15.92:0/3921696158 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554486800 0 0) 0x7fea540ca920 con 0x7fea54068390 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.092+0000 7fea527fc700 1 -- 172.21.15.92:0/3921696158 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea40003570 con 0x7fea54068390 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea4a7fc700 1 -- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 msgr2=0x7fea540c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea4a7fc700 1 --2- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea540c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea527fc700 1 -- 172.21.15.92:0/3921696158 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832335406 0 0) 0x7fea40003570 con 0x7fea54068390 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea4a7fc700 1 -- 172.21.15.92:0/3921696158 >> v1:172.21.15.92:6789/0 conn(0x7fea54068390 legacy=0x7fea540caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea4a7fc700 1 -- 172.21.15.92:0/3921696158 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5406b4c0 con 0x7fea5406bfa0 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea4a7fc700 1 --2- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 0x7fea540c8830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0098d0 tx=0x7fea3c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b522958c8e88e71a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea527fc700 1 -- 172.21.15.92:0/3921696158 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c0017f0 con 0x7fea5406bfa0 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea527fc700 1 -- 172.21.15.92:0/3921696158 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c011420 con 0x7fea5406bfa0 2023-10-06T03:11:09.095 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.093+0000 7fea527fc700 1 -- 172.21.15.92:0/3921696158 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0117e0 con 0x7fea5406bfa0 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 -- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 msgr2=0x7fea540c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 --2- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 0x7fea540c8830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0098d0 tx=0x7fea3c009120 comp rx=0 tx=0).stop 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 -- 172.21.15.92:0/3921696158 shutdown_connections 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 --2- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea540c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 --2- 172.21.15.92:0/3921696158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 0x7fea540c8830 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 -- 172.21.15.92:0/3921696158 >> 172.21.15.92:0/3921696158 conn(0x7fea540cddc0 msgr2=0x7fea540c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 -- 172.21.15.92:0/3921696158 shutdown_connections 2023-10-06T03:11:09.098 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.094+0000 7fea53fff700 1 -- 172.21.15.92:0/3921696158 wait complete. 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.095+0000 7fea53fff700 1 Processor -- start 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.095+0000 7fea53fff700 1 -- start start 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 0x7fea540b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 0x7fea5416a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea54151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5416aed0 con 0x7fea54068390 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540b30b0 con 0x7fea5406bfa0 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5415e4d0 con 0x7fea54159a70 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 0x7fea540b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:09.099 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 0x7fea540b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:33990/0 (socket says 172.21.15.92:33990) 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 -- 172.21.15.92:0/3757662523 learned_addr learned my addr 172.21.15.92:0/3757662523 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 -- 172.21.15.92:0/3757662523 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 msgr2=0x7fea54151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea54151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 -- 172.21.15.92:0/3757662523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 msgr2=0x7fea5416a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 0x7fea5416a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 -- 172.21.15.92:0/3757662523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0093c0 con 0x7fea54068390 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea52ffd700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 0x7fea540b29b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fea4400ec80 tx=0x7fea440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4400c9d0 con 0x7fea54068390 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4400cb30 con 0x7fea54068390 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea540b0250 con 0x7fea54068390 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44013470 con 0x7fea54068390 2023-10-06T03:11:09.100 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.096+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5415e1f0 con 0x7fea54068390 2023-10-06T03:11:09.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.097+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5406d560 con 0x7fea54068390 2023-10-06T03:11:09.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.098+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fea44013820 con 0x7fea54068390 2023-10-06T03:11:09.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.098+0000 7fea4b7fe700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea34067320 0x7fea340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:09.101 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.098+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6222+0+0 (secure 0 0 0) 0x7fea440851e0 con 0x7fea54068390 2023-10-06T03:11:09.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.098+0000 7fea4a7fc700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea34067320 0x7fea340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:09.102 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.099+0000 7fea4a7fc700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea34067320 0x7fea340697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fea54152cb0 tx=0x7fea3c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:09.103 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.102+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea44051c70 con 0x7fea54068390 2023-10-06T03:11:09.330 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.329+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000445601", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fea5406b7a0 con 0x7fea54068390 2023-10-06T03:11:09.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.965+0000 7fea4b7fe700 1 -- 172.21.15.92:0/3757662523 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000445601", "yes_i_really_mean_it": true}]=0 v130) v1 ==== 121+0+0 (secure 0 0 0) 0x7fea44055290 con 0x7fea54068390 2023-10-06T03:11:09.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.967+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea34067320 msgr2=0x7fea340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:09.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea34067320 0x7fea340697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fea54152cb0 tx=0x7fea3c016000 comp rx=0 tx=0).stop 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 msgr2=0x7fea540b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 0x7fea540b29b0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fea4400ec80 tx=0x7fea440091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 shutdown_connections 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fea34067320 0x7fea340697d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fea54068390 0x7fea540b29b0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea54159a70 0x7fea54151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 --2- 172.21.15.92:0/3757662523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea5406bfa0 0x7fea5416a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 >> 172.21.15.92:0/3757662523 conn(0x7fea540cddc0 msgr2=0x7fea54153370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 shutdown_connections 2023-10-06T03:11:09.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:09.968+0000 7fea53fff700 1 -- 172.21.15.92:0/3757662523 wait complete. 2023-10-06T03:11:09.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000445601.meta cephfs.volume_0000000000445601.meta --yes-i-really-really-mean-it 2023-10-06T03:11:10.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.144+0000 7f71e9dae700 1 Processor -- start 2023-10-06T03:11:10.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.144+0000 7f71e9dae700 1 -- start start 2023-10-06T03:11:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e9dae700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71e4159ec0 0x7f71e40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e9dae700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f71e40b53d0 0x7f71e40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e9dae700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e40b5b00 con 0x7f71e40cce40 2023-10-06T03:11:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e9dae700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40b5850 con 0x7f71e4159ec0 2023-10-06T03:11:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e9dae700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40b5da0 con 0x7f71e40b53d0 2023-10-06T03:11:10.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71e4159ec0 0x7f71e40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f71e4159ec0 0x7f71e40b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43462/0 (socket says 172.21.15.92:43462) 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 -- 172.21.15.92:0/3847206936 learned_addr learned my addr 172.21.15.92:0/3847206936 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 -- 172.21.15.92:0/3847206936 >> v2:172.21.15.136:3300/0 conn(0x7f71e40b53d0 msgr2=0x7f71e40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 --2- 172.21.15.92:0/3847206936 >> v2:172.21.15.136:3300/0 conn(0x7f71e40b53d0 0x7f71e40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 -- 172.21.15.92:0/3847206936 >> v1:172.21.15.136:6789/0 conn(0x7f71e40cce40 legacy=0x7f71e40cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 -- 172.21.15.92:0/3847206936 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40b6080 con 0x7f71e4159ec0 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e37fe700 1 --2- 172.21.15.92:0/3847206936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e4159ec0 0x7f71e40b4e90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f71e40b64d0 tx=0x7f71d4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456bc1383506cffe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e27fc700 1 -- 172.21.15.92:0/3847206936 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4005c10 con 0x7f71e4159ec0 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e27fc700 1 -- 172.21.15.92:0/3847206936 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4005d70 con 0x7f71e4159ec0 2023-10-06T03:11:10.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.145+0000 7f71e27fc700 1 -- 172.21.15.92:0/3847206936 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d400b840 con 0x7f71e4159ec0 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 -- 172.21.15.92:0/3847206936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e4159ec0 msgr2=0x7f71e40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 --2- 172.21.15.92:0/3847206936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e4159ec0 0x7f71e40b4e90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f71e40b64d0 tx=0x7f71d4009220 comp rx=0 tx=0).stop 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 -- 172.21.15.92:0/3847206936 shutdown_connections 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 --2- 172.21.15.92:0/3847206936 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e4159ec0 0x7f71e40b4e90 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 --2- 172.21.15.92:0/3847206936 >> v2:172.21.15.136:3300/0 conn(0x7f71e40b53d0 0x7f71e40cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 -- 172.21.15.92:0/3847206936 >> 172.21.15.92:0/3847206936 conn(0x7f71e40999f0 msgr2=0x7f71e40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 -- 172.21.15.92:0/3847206936 shutdown_connections 2023-10-06T03:11:10.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.146+0000 7f71e9dae700 1 -- 172.21.15.92:0/3847206936 wait complete. 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.147+0000 7f71e9dae700 1 Processor -- start 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.147+0000 7f71e9dae700 1 -- start start 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e9dae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 0x7f71e414b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e9dae700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71e40cce40 0x7f71e4069260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e9dae700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71e4159ec0 0x7f71e40697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e9dae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e4069e20 con 0x7f71e40b53d0 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e9dae700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e409aec0 con 0x7f71e4159ec0 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e9dae700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e4068630 con 0x7f71e40cce40 2023-10-06T03:11:10.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 0x7f71e414b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 0x7f71e414b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43468/0 (socket says 172.21.15.92:43468) 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 -- 172.21.15.92:0/344522124 learned_addr learned my addr 172.21.15.92:0/344522124 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 -- 172.21.15.92:0/344522124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71e40cce40 msgr2=0x7f71e4069260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71e40cce40 0x7f71e4069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 -- 172.21.15.92:0/344522124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71e4159ec0 msgr2=0x7f71e40697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71e4159ec0 0x7f71e40697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 -- 172.21.15.92:0/344522124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4009480 con 0x7f71e40b53d0 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.148+0000 7f71e37fe700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 0x7f71e414b1a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f71d4003080 tx=0x7f71d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.149+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011aa0 con 0x7f71e40b53d0 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.149+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4011c40 con 0x7f71e40b53d0 2023-10-06T03:11:10.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.149+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e409a590 con 0x7f71e40b53d0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.149+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d401d430 con 0x7f71e40b53d0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.149+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e409ab50 con 0x7f71e40b53d0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.150+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e4063c90 con 0x7f71e40b53d0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.150+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f71d401d7e0 con 0x7f71e40b53d0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.150+0000 7f71cb7fe700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f71d0067320 0x7f71d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.151+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6222+0+0 (secure 0 0 0) 0x7f71d40870c0 con 0x7f71e40b53d0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.151+0000 7f71e2ffd700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f71d0067320 0x7f71d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:10.154 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.151+0000 7f71e2ffd700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f71d0067320 0x7f71d00697d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f71e409c630 tx=0x7f71d8006d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:10.155 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.155+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71d4053b50 con 0x7f71e40b53d0 2023-10-06T03:11:10.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.338+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000445601.meta", "pool2": "cephfs.volume_0000000000445601.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71e40718b0 con 0x7f71e40b53d0 2023-10-06T03:11:10.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.967+0000 7f71cb7fe700 1 -- 172.21.15.92:0/344522124 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000445601.meta", "pool2": "cephfs.volume_0000000000445601.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000445601.meta' removed v191) v1 ==== 241+0+0 (secure 0 0 0) 0x7f71d4057170 con 0x7f71e40b53d0 2023-10-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f71d0067320 msgr2=0x7f71d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f71d0067320 0x7f71d00697d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f71e409c630 tx=0x7f71d8006d10 comp rx=0 tx=0).stop 2023-10-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 msgr2=0x7f71e414b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 0x7f71e414b1a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f71d4003080 tx=0x7f71d4015000 comp rx=0 tx=0).stop 2023-10-06T03:11:10.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 shutdown_connections 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f71d0067320 0x7f71d00697d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71e40b53d0 0x7f71e414b1a0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71e4159ec0 0x7f71e40697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 --2- 172.21.15.92:0/344522124 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71e40cce40 0x7f71e4069260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 >> 172.21.15.92:0/344522124 conn(0x7f71e40999f0 msgr2=0x7f71e4159390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 shutdown_connections 2023-10-06T03:11:10.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:10.970+0000 7f71e9dae700 1 -- 172.21.15.92:0/344522124 wait complete. 2023-10-06T03:11:10.973 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs.volume_0000000000445601.meta' removed 2023-10-06T03:11:10.985 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000445601.data cephfs.volume_0000000000445601.data --yes-i-really-really-mean-it 2023-10-06T03:11:11.105 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.105+0000 7f17a4e3d700 1 Processor -- start 2023-10-06T03:11:11.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.105+0000 7f17a4e3d700 1 -- start start 2023-10-06T03:11:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f17a4e3d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f17a0159ec0 0x7f17a00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.109 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f17a4e3d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 0x7f17a00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f17a4e3d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a00b5b00 con 0x7f17a00cce40 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f17a4e3d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a00b5850 con 0x7f17a0159ec0 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f17a4e3d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00b5da0 con 0x7f17a00b53d0 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179ed9d700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f17a00cce40 0x7f17a00cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41730/0 (socket says 172.21.15.92:41730) 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179ed9d700 1 -- 172.21.15.92:0/1230109343 learned_addr learned my addr 172.21.15.92:0/1230109343 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179dd9b700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 0x7f17a00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179e59c700 1 --2- 172.21.15.92:0/1230109343 >> v2:172.21.15.136:3300/0 conn(0x7f17a0159ec0 0x7f17a00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213041615 0 0) 0x7f17a00b5b00 con 0x7f17a00cce40 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1788003570 con 0x7f17a00cce40 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855030728 0 0) 0x7f1788003570 con 0x7f17a00cce40 2023-10-06T03:11:11.110 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 msgr2=0x7f17a00cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 0x7f17a00cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 >> v2:172.21.15.136:3300/0 conn(0x7f17a0159ec0 msgr2=0x7f17a00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.106+0000 7f179d59a700 1 --2- 172.21.15.92:0/1230109343 >> v2:172.21.15.136:3300/0 conn(0x7f17a0159ec0 0x7f17a00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00b6080 con 0x7f17a00cce40 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f17980017f0 con 0x7f17a00cce40 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 --2- 172.21.15.92:0/1230109343 >> v2:172.21.15.136:3300/0 conn(0x7f17a0159ec0 0x7f17a00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 >> v1:172.21.15.92:6789/0 conn(0x7f17a00cce40 legacy=0x7f17a00cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f1788003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 0x7f1788004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1788004d50 0x7f17880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00b5850 con 0x7f1788003a60 2023-10-06T03:11:11.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00b5da0 con 0x7f1788004d50 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178800ba90 con 0x7f17a00b53d0 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f1788003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179dd9b700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 0x7f1788004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179ed9d700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1788004d50 0x7f17880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 -- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 msgr2=0x7f1788004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a00b53d0 0x7f1788004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 -- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1788004d50 msgr2=0x7f17880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1788004d50 0x7f17880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 -- 172.21.15.92:0/1230109343 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788003570 con 0x7f1788003a60 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179ed9d700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1788004d50 0x7f17880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179e59c700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f1788003e70 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f1790009de0 tx=0x7f1790009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1218e684f92905a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:11.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790005a00 con 0x7f1788003a60 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1790005b60 con 0x7f1788003a60 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.107+0000 7f179d59a700 1 -- 172.21.15.92:0/1230109343 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179000e8c0 con 0x7f1788003a60 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f17a4e3d700 1 -- 172.21.15.92:0/1230109343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 msgr2=0x7f1788003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/1230109343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f1788003e70 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f1790009de0 tx=0x7f1790009120 comp rx=0 tx=0).stop 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f179ed9d700 1 -- 172.21.15.92:0/1230109343 reap_dead start 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f17a4e3d700 1 -- 172.21.15.92:0/1230109343 shutdown_connections 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f17a4e3d700 1 -- 172.21.15.92:0/1230109343 >> 172.21.15.92:0/1230109343 conn(0x7f17a00999f0 msgr2=0x7f17a00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f17a4e3d700 1 -- 172.21.15.92:0/1230109343 shutdown_connections 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.108+0000 7f17a4e3d700 1 -- 172.21.15.92:0/1230109343 wait complete. 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.109+0000 7f17a4e3d700 1 Processor -- start 2023-10-06T03:11:11.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.109+0000 7f17a4e3d700 1 -- start start 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f17a015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a0069e50 0x7f17a006a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a01699d0 0x7f17a0169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a016abf0 con 0x7f1788003a60 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a006a800 con 0x7f17a01699d0 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a016b420 con 0x7f17a0069e50 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f17a015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f17a015a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43486/0 (socket says 172.21.15.92:43486) 2023-10-06T03:11:11.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 -- 172.21.15.92:0/914394023 learned_addr learned my addr 172.21.15.92:0/914394023 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179ed9d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a01699d0 0x7f17a0169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179dd9b700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a0069e50 0x7f17a006a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 -- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a0069e50 msgr2=0x7f17a006a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a0069e50 0x7f17a006a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 -- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a01699d0 msgr2=0x7f17a0169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a01699d0 0x7f17a0169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 -- 172.21.15.92:0/914394023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a015ae80 con 0x7f1788003a60 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f179e59c700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f17a015a940 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f1790009160 tx=0x7f1790019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1790023070 con 0x7f1788003a60 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17900062c0 con 0x7f1788003a60 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.110+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a016a7c0 con 0x7f1788003a60 2023-10-06T03:11:11.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.111+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17900066e0 con 0x7f1788003a60 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.111+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a016b0b0 con 0x7f1788003a60 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.111+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a0063c90 con 0x7f1788003a60 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.112+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1790013680 con 0x7f1788003a60 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.112+0000 7f178e7fc700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1780067320 0x7f17800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.112+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5747+0+0 (secure 0 0 0) 0x7f1790086400 con 0x7f1788003a60 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.112+0000 7f179dd9b700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1780067320 0x7f17800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.113+0000 7f179dd9b700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1780067320 0x7f17800697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f17a009e5d0 tx=0x7f1794008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:11.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.117+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1790053020 con 0x7f1788003a60 2023-10-06T03:11:11.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.297+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000445601.data", "pool2": "cephfs.volume_0000000000445601.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17a00718b0 con 0x7f1788003a60 2023-10-06T03:11:11.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.974+0000 7f178e7fc700 1 -- 172.21.15.92:0/914394023 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000445601.data", "pool2": "cephfs.volume_0000000000445601.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000445601.data' removed v192) v1 ==== 241+0+0 (secure 0 0 0) 0x7f1790056640 con 0x7f1788003a60 2023-10-06T03:11:11.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1780067320 msgr2=0x7f17800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:11.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1780067320 0x7f17800697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f17a009e5d0 tx=0x7f1794008040 comp rx=0 tx=0).stop 2023-10-06T03:11:11.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 msgr2=0x7f17a015a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f17a015a940 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f1790009160 tx=0x7f1790019000 comp rx=0 tx=0).stop 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 shutdown_connections 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1780067320 0x7f17800697d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1788003a60 0x7f17a015a940 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a01699d0 0x7f17a0169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 --2- 172.21.15.92:0/914394023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a0069e50 0x7f17a006a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 >> 172.21.15.92:0/914394023 conn(0x7f17a00999f0 msgr2=0x7f17a0158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 shutdown_connections 2023-10-06T03:11:11.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:11.978+0000 7f17a4e3d700 1 -- 172.21.15.92:0/914394023 wait complete. 2023-10-06T03:11:11.980 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs.volume_0000000000445601.data' removed 2023-10-06T03:11:11.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:11:12.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.110+0000 7f1335e30700 1 Processor -- start 2023-10-06T03:11:12.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.110+0000 7f1335e30700 1 -- start start 2023-10-06T03:11:12.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1335e30700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1330159c60 0x7f133015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1335e30700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f133015db10 0x7f1330069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1335e30700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133015a160 con 0x7f1330159890 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1335e30700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133015df50 con 0x7f1330159c60 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1335e30700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13300d13a0 con 0x7f133015db10 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1327fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1330159c60 0x7f133015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1327fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1330159c60 0x7f133015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43496/0 (socket says 172.21.15.92:43496) 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f132f7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1330159890 0x7f133015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41738/0 (socket says 172.21.15.92:41738) 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.111+0000 7f1327fff700 1 -- 172.21.15.92:0/1944141400 learned_addr learned my addr 172.21.15.92:0/1944141400 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f132ffff700 1 --2- 172.21.15.92:0/1944141400 >> v2:172.21.15.136:3300/0 conn(0x7f133015db10 0x7f1330069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1327fff700 1 -- 172.21.15.92:0/1944141400 >> v2:172.21.15.136:3300/0 conn(0x7f133015db10 msgr2=0x7f1330069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1327fff700 1 --2- 172.21.15.92:0/1944141400 >> v2:172.21.15.136:3300/0 conn(0x7f133015db10 0x7f1330069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1327fff700 1 -- 172.21.15.92:0/1944141400 >> v1:172.21.15.92:6789/0 conn(0x7f1330159890 legacy=0x7f133015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1327fff700 1 -- 172.21.15.92:0/1944141400 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133015e140 con 0x7f1330159c60 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f132effd700 1 -- 172.21.15.92:0/1944141400 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816548300 0 0) 0x7f133015a160 con 0x7f1330159890 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1327fff700 1 --2- 172.21.15.92:0/1944141400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159c60 0x7f133015d5d0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f133006a890 tx=0x7f1318009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412df462db58556d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f132effd700 1 -- 172.21.15.92:0/1944141400 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131800b860 con 0x7f1330159c60 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f132effd700 1 -- 172.21.15.92:0/1944141400 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131800b9c0 con 0x7f1330159c60 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f132effd700 1 -- 172.21.15.92:0/1944141400 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131800bd60 con 0x7f1330159c60 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1335e30700 1 -- 172.21.15.92:0/1944141400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159c60 msgr2=0x7f133015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1335e30700 1 --2- 172.21.15.92:0/1944141400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159c60 0x7f133015d5d0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f133006a890 tx=0x7f1318009250 comp rx=0 tx=0).stop 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1335e30700 1 -- 172.21.15.92:0/1944141400 shutdown_connections 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1335e30700 1 --2- 172.21.15.92:0/1944141400 >> v2:172.21.15.136:3300/0 conn(0x7f133015db10 0x7f1330069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1335e30700 1 --2- 172.21.15.92:0/1944141400 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159c60 0x7f133015d5d0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.112+0000 7f1335e30700 1 -- 172.21.15.92:0/1944141400 >> 172.21.15.92:0/1944141400 conn(0x7f1330068e20 msgr2=0x7f13300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:12.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.113+0000 7f1335e30700 1 -- 172.21.15.92:0/1944141400 shutdown_connections 2023-10-06T03:11:12.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.113+0000 7f1335e30700 1 -- 172.21.15.92:0/1944141400 wait complete. 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 Processor -- start 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 -- start start 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 0x7f13300b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1330159c60 0x7f133014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f133015db10 0x7f1330158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1330152ac0 con 0x7f1330159890 2023-10-06T03:11:12.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133014cc40 con 0x7f1330159c60 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f1335e30700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13300b2380 con 0x7f133015db10 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f132f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 0x7f13300b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f132f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 0x7f13300b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43502/0 (socket says 172.21.15.92:43502) 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.114+0000 7f132f7fe700 1 -- 172.21.15.92:0/1348715151 learned_addr learned my addr 172.21.15.92:0/1348715151 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f1327fff700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1330159c60 0x7f133014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132ffff700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f133015db10 0x7f1330158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132f7fe700 1 -- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f133015db10 msgr2=0x7f1330158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132f7fe700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f133015db10 0x7f1330158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132f7fe700 1 -- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1330159c60 msgr2=0x7f133014c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132f7fe700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1330159c60 0x7f133014c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132f7fe700 1 -- 172.21.15.92:0/1348715151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180094f0 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f1327fff700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1330159c60 0x7f133014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f132f7fe700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 0x7f13300b1d10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f13200011e0 tx=0x7f1320007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320007300 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320007460 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1330152470 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320004830 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.115+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13300b2660 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.116+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133004c3c0 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.116+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1320004be0 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.117+0000 7f13277fe700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1310067320 0x7f13100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.117+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f1320084070 con 0x7f1330159890 2023-10-06T03:11:12.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.117+0000 7f1327fff700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1310067320 0x7f13100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.117+0000 7f1327fff700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1310067320 0x7f13100697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f133006a890 tx=0x7f1318017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:12.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.121+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1320050ea0 con 0x7f1330159890 2023-10-06T03:11:12.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.305+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f133006a5b0 con 0x7f1330159890 2023-10-06T03:11:12.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.309+0000 7f13277fe700 1 -- 172.21.15.92:0/1348715151 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f13200180a0 con 0x7f1330159890 2023-10-06T03:11:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1310067320 msgr2=0x7f13100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1310067320 0x7f13100697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f133006a890 tx=0x7f1318017000 comp rx=0 tx=0).stop 2023-10-06T03:11:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 msgr2=0x7f13300b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 0x7f13300b1d10 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f13200011e0 tx=0x7f1320007e10 comp rx=0 tx=0).stop 2023-10-06T03:11:12.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 shutdown_connections 2023-10-06T03:11:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1310067320 0x7f13100697d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1330159890 0x7f13300b1d10 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f133015db10 0x7f1330158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 --2- 172.21.15.92:0/1348715151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1330159c60 0x7f133014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 >> 172.21.15.92:0/1348715151 conn(0x7f1330068e20 msgr2=0x7f133014d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:12.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 shutdown_connections 2023-10-06T03:11:12.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.311+0000 7f1335e30700 1 -- 172.21.15.92:0/1348715151 wait complete. 2023-10-06T03:11:12.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-06T03:11:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.438+0000 7fb527fff700 1 Processor -- start 2023-10-06T03:11:12.439 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.438+0000 7fb527fff700 1 -- start start 2023-10-06T03:11:12.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb527fff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb528143050 0x7fb528159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb527fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 0x7fb52806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb527fff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52806c390 con 0x7fb52806b9e0 2023-10-06T03:11:12.443 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb527fff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb528143420 con 0x7fb528143050 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb527fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52806a1e0 con 0x7fb5280b3f20 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb51e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb52806b9e0 0x7fb5280b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41744/0 (socket says 172.21.15.92:41744) 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb51e7fc700 1 -- 172.21.15.92:0/2116635431 learned_addr learned my addr 172.21.15.92:0/2116635431 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.439+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683000950 0 0) 0x7fb52806c390 con 0x7fb52806b9e0 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb514003570 con 0x7fb52806b9e0 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb526ffd700 1 --2- 172.21.15.92:0/2116635431 >> v2:172.21.15.136:3300/0 conn(0x7fb528143050 0x7fb528159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5277fe700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 0x7fb52806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493124518 0 0) 0x7fb514003570 con 0x7fb52806b9e0 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 msgr2=0x7fb52806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 0x7fb52806be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.444 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 >> v2:172.21.15.136:3300/0 conn(0x7fb528143050 msgr2=0x7fb528159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 --2- 172.21.15.92:0/2116635431 >> v2:172.21.15.136:3300/0 conn(0x7fb528143050 0x7fb528159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280695d0 con 0x7fb52806b9e0 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fb510001470 con 0x7fb52806b9e0 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 >> v1:172.21.15.92:6789/0 conn(0x7fb52806b9e0 legacy=0x7fb5280b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb514003a60 0x7fb514003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 0x7fb5140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528143420 con 0x7fb514004810 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52806a1e0 con 0x7fb514003a60 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51400bae0 con 0x7fb5280b3f20 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb514004810 0x7fb514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.445 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb526ffd700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb514003a60 0x7fb514003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb5277fe700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 0x7fb5140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 -- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 msgr2=0x7fb5140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5280b3f20 0x7fb5140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 -- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb514003a60 msgr2=0x7fb514003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb514003a60 0x7fb514003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 -- 172.21.15.92:0/2116635431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514003570 con 0x7fb514004810 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.440+0000 7fb51e7fc700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb514004810 0x7fb514004c80 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c3c0 tx=0x7fb52806a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d438260a13cc9b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51000cc90 con 0x7fb514004810 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51000cdf0 con 0x7fb514004810 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb5267fc700 1 -- 172.21.15.92:0/2116635431 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510005b70 con 0x7fb514004810 2023-10-06T03:11:12.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb527fff700 1 -- 172.21.15.92:0/2116635431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb514004810 msgr2=0x7fb514004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb527fff700 1 --2- 172.21.15.92:0/2116635431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb514004810 0x7fb514004c80 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c3c0 tx=0x7fb52806a150 comp rx=0 tx=0).stop 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb5277fe700 1 -- 172.21.15.92:0/2116635431 reap_dead start 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb527fff700 1 -- 172.21.15.92:0/2116635431 shutdown_connections 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb527fff700 1 -- 172.21.15.92:0/2116635431 >> 172.21.15.92:0/2116635431 conn(0x7fb5280ce430 msgr2=0x7fb5280ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb527fff700 1 -- 172.21.15.92:0/2116635431 shutdown_connections 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.441+0000 7fb527fff700 1 -- 172.21.15.92:0/2116635431 wait complete. 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.442+0000 7fb527fff700 1 Processor -- start 2023-10-06T03:11:12.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.442+0000 7fb527fff700 1 -- start start 2023-10-06T03:11:12.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb527fff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb514004810 0x7fb528139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb527fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281570a0 0x7fb52813c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb527fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 0x7fb528141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb527fff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815be20 con 0x7fb528141160 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb527fff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528141b40 con 0x7fb5281570a0 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb527fff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb528157510 con 0x7fb514004810 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 0x7fb528141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 0x7fb528141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43532/0 (socket says 172.21.15.92:43532) 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 -- 172.21.15.92:0/2789511215 learned_addr learned my addr 172.21.15.92:0/2789511215 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb526ffd700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb514004810 0x7fb528139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb51e7fc700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281570a0 0x7fb52813c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.450 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 -- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb514004810 msgr2=0x7fb528139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb514004810 0x7fb528139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 -- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281570a0 msgr2=0x7fb52813c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.443+0000 7fb5277fe700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281570a0 0x7fb52813c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb5277fe700 1 -- 172.21.15.92:0/2789511215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52813c910 con 0x7fb528141160 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb5277fe700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 0x7fb528141600 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb5200094d0 tx=0x7fb52000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000e040 con 0x7fb528141160 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52000f040 con 0x7fb528141160 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb528139c90 con 0x7fb528141160 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52000fda0 con 0x7fb528141160 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.444+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52815c000 con 0x7fb528141160 2023-10-06T03:11:12.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.445+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52813a1b0 con 0x7fb528141160 2023-10-06T03:11:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.445+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb520015640 con 0x7fb528141160 2023-10-06T03:11:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.445+0000 7fb51f7fe700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb508067320 0x7fb5080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.446+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb520085220 con 0x7fb528141160 2023-10-06T03:11:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.446+0000 7fb526ffd700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb508067320 0x7fb5080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.446+0000 7fb526ffd700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb508067320 0x7fb5080697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb518006ea0 tx=0x7fb518009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:12.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.450+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb52813a1b0 con 0x7fb528141160 2023-10-06T03:11:12.707 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.707+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7fb52813a1b0 con 0x7fb528141160 2023-10-06T03:11:12.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.975+0000 7fb51f7fe700 1 -- 172.21.15.92:0/2789511215 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v491) v1 ==== 147+0+0 (secure 0 0 0) 0x7fb52813a1b0 con 0x7fb528141160 2023-10-06T03:11:12.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb508067320 msgr2=0x7fb5080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb508067320 0x7fb5080697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb518006ea0 tx=0x7fb518009120 comp rx=0 tx=0).stop 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 msgr2=0x7fb528141600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 0x7fb528141600 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb5200094d0 tx=0x7fb52000b040 comp rx=0 tx=0).stop 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 shutdown_connections 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb508067320 0x7fb5080697d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb528141160 0x7fb528141600 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb514004810 0x7fb528139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 --2- 172.21.15.92:0/2789511215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281570a0 0x7fb52813c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 >> 172.21.15.92:0/2789511215 conn(0x7fb5280ce430 msgr2=0x7fb5280adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 shutdown_connections 2023-10-06T03:11:12.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:12.978+0000 7fb527fff700 1 -- 172.21.15.92:0/2789511215 wait complete. 2023-10-06T03:11:12.991 INFO:tasks.cephfs_test_runner:That volume rename is idempotent. ... ok 2023-10-06T03:11:12.991 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-06T03:11:12.991 INFO:tasks.cephfs_test_runner:test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:11:12.992 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-06T03:11:13.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.105+0000 7fa884a55700 1 Processor -- start 2023-10-06T03:11:13.106 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 -- start start 2023-10-06T03:11:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159890 0x7fa88015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.111 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa880159c60 0x7fa88015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa88015db10 0x7fa880069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88015a160 con 0x7fa88015db10 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800d13a0 con 0x7fa880159c60 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa884a55700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88015e140 con 0x7fa880159890 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.106+0000 7fa87ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa88015db10 0x7fa880069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa88015db10 0x7fa880069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43544/0 (socket says 172.21.15.92:43544) 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2210127157 learned_addr learned my addr 172.21.15.92:0/2210127157 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa875d9b700 1 --2- 172.21.15.92:0/2210127157 >> v2:172.21.15.136:3300/0 conn(0x7fa880159c60 0x7fa88015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87e59c700 1 --2- 172.21.15.92:0/2210127157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159890 0x7fa88015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.112 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2210127157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159890 msgr2=0x7fa88015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 --2- 172.21.15.92:0/2210127157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159890 0x7fa88015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2210127157 >> v2:172.21.15.136:3300/0 conn(0x7fa880159c60 msgr2=0x7fa88015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 --2- 172.21.15.92:0/2210127157 >> v2:172.21.15.136:3300/0 conn(0x7fa880159c60 0x7fa88015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2210127157 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88015a3d0 con 0x7fa88015db10 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87ed9d700 1 --2- 172.21.15.92:0/2210127157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa880069560 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa8800650f0 tx=0x7fa878009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60a45d860b9c898 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87dd9b700 1 -- 172.21.15.92:0/2210127157 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800e070 con 0x7fa88015db10 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87dd9b700 1 -- 172.21.15.92:0/2210127157 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87800ba50 con 0x7fa88015db10 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.107+0000 7fa87dd9b700 1 -- 172.21.15.92:0/2210127157 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780092e0 con 0x7fa88015db10 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 -- 172.21.15.92:0/2210127157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 msgr2=0x7fa880069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 --2- 172.21.15.92:0/2210127157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa880069560 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa8800650f0 tx=0x7fa878009840 comp rx=0 tx=0).stop 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 -- 172.21.15.92:0/2210127157 shutdown_connections 2023-10-06T03:11:13.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 --2- 172.21.15.92:0/2210127157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159890 0x7fa88015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 --2- 172.21.15.92:0/2210127157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa880069560 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 --2- 172.21.15.92:0/2210127157 >> v2:172.21.15.136:3300/0 conn(0x7fa880159c60 0x7fa88015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 -- 172.21.15.92:0/2210127157 >> 172.21.15.92:0/2210127157 conn(0x7fa880068e20 msgr2=0x7fa8800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 -- 172.21.15.92:0/2210127157 shutdown_connections 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.108+0000 7fa884a55700 1 -- 172.21.15.92:0/2210127157 wait complete. 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.109+0000 7fa884a55700 1 Processor -- start 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.109+0000 7fa884a55700 1 -- start start 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa880159890 0x7fa880158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159c60 0x7fa8800b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa8800b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800b2d80 con 0x7fa88015db10 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800b28f0 con 0x7fa880159890 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800b1fc0 con 0x7fa880159c60 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa8800b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa8800b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43558/0 (socket says 172.21.15.92:43558) 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2101319789 learned_addr learned my addr 172.21.15.92:0/2101319789 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87e59c700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa880159890 0x7fa880158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa875d9b700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159c60 0x7fa8800b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159c60 msgr2=0x7fa8800b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159c60 0x7fa8800b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa880159890 msgr2=0x7fa880158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa880159890 0x7fa880158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 -- 172.21.15.92:0/2101319789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878009ae0 con 0x7fa88015db10 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa87ed9d700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa8800b3ad0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa880070e20 tx=0x7fa878018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780032f0 con 0x7fa88015db10 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa878008e80 con 0x7fa88015db10 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8800b22a0 con 0x7fa88015db10 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878025690 con 0x7fa88015db10 2023-10-06T03:11:13.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.110+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8800b1cb0 con 0x7fa88015db10 2023-10-06T03:11:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.111+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88004c3c0 con 0x7fa88015db10 2023-10-06T03:11:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.112+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa878003040 con 0x7fa88015db10 2023-10-06T03:11:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.112+0000 7fa876ffd700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa860067320 0x7fa8600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:13.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.112+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa878086750 con 0x7fa88015db10 2023-10-06T03:11:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.112+0000 7fa87e59c700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa860067320 0x7fa8600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.112+0000 7fa87e59c700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa860067320 0x7fa8600697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa87000ab30 tx=0x7fa870008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:13.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.116+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa878053520 con 0x7fa88015db10 2023-10-06T03:11:13.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:11:13.134 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:11:13.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.391+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7fa88006a5b0 con 0x7fa88015db10 2023-10-06T03:11:13.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.981+0000 7fa876ffd700 1 -- 172.21.15.92:0/2101319789 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v492) v1 ==== 134+0+0 (secure 0 0 0) 0x7fa878056b40 con 0x7fa88015db10 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa860067320 msgr2=0x7fa8600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa860067320 0x7fa8600697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa87000ab30 tx=0x7fa870008040 comp rx=0 tx=0).stop 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 msgr2=0x7fa8800b3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa8800b3ad0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa880070e20 tx=0x7fa878018000 comp rx=0 tx=0).stop 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 shutdown_connections 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa860067320 0x7fa8600697d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa88015db10 0x7fa8800b3ad0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa880159890 0x7fa880158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 --2- 172.21.15.92:0/2101319789 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa880159c60 0x7fa8800b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 >> 172.21.15.92:0/2101319789 conn(0x7fa880068e20 msgr2=0x7fa88015cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 shutdown_connections 2023-10-06T03:11:13.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:13.984+0000 7fa884a55700 1 -- 172.21.15.92:0/2101319789 wait complete. 2023-10-06T03:11:13.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:11:14.118 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.117+0000 7f9cfe2e5700 1 Processor -- start 2023-10-06T03:11:14.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.117+0000 7f9cfe2e5700 1 -- start start 2023-10-06T03:11:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cfe2e5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9cf8141580 0x7f9cf80cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cfe2e5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9cf80caf80 0x7f9cf80b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cfe2e5700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80ccfb0 con 0x7f9cf81411b0 2023-10-06T03:11:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cfe2e5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80b3e50 con 0x7f9cf8141580 2023-10-06T03:11:14.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cfe2e5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80b40d0 con 0x7f9cf80caf80 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9cf8141580 0x7f9cf80cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf7fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9cf81411b0 0x7f9cf815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41760/0 (socket says 172.21.15.92:41760) 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9cf8141580 0x7f9cf80cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43562/0 (socket says 172.21.15.92:43562) 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf7fff700 1 -- 172.21.15.92:0/367828782 learned_addr learned my addr 172.21.15.92:0/367828782 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 -- 172.21.15.92:0/367828782 >> v2:172.21.15.136:3300/0 conn(0x7f9cf80caf80 msgr2=0x7f9cf80b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 --2- 172.21.15.92:0/367828782 >> v2:172.21.15.136:3300/0 conn(0x7f9cf80caf80 0x7f9cf80b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 -- 172.21.15.92:0/367828782 >> v1:172.21.15.92:6789/0 conn(0x7f9cf81411b0 legacy=0x7f9cf815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 -- 172.21.15.92:0/367828782 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80b43b0 con 0x7f9cf8141580 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf6ffd700 1 -- 172.21.15.92:0/367828782 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879067347 0 0) 0x7f9cf80ccfb0 con 0x7f9cf81411b0 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.118+0000 7f9cf77fe700 1 --2- 172.21.15.92:0/367828782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80cd440 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f9cf806f720 tx=0x7f9cec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de261c73b79709e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cf6ffd700 1 -- 172.21.15.92:0/367828782 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00b6b0 con 0x7f9cf8141580 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cf6ffd700 1 -- 172.21.15.92:0/367828782 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00b810 con 0x7f9cf8141580 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cf6ffd700 1 -- 172.21.15.92:0/367828782 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00bbb0 con 0x7f9cf8141580 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/367828782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 msgr2=0x7f9cf80cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/367828782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80cd440 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f9cf806f720 tx=0x7f9cec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/367828782 shutdown_connections 2023-10-06T03:11:14.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/367828782 >> v2:172.21.15.136:3300/0 conn(0x7f9cf80caf80 0x7f9cf80b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/367828782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80cd440 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/367828782 >> 172.21.15.92:0/367828782 conn(0x7f9cf806bb50 msgr2=0x7f9cf806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:14.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/367828782 shutdown_connections 2023-10-06T03:11:14.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.119+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/367828782 wait complete. 2023-10-06T03:11:14.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.120+0000 7f9cfe2e5700 1 Processor -- start 2023-10-06T03:11:14.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.120+0000 7f9cfe2e5700 1 -- start start 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfe2e5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf80caf80 0x7f9cf8158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfe2e5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9cf81411b0 0x7f9cf80c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfe2e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfe2e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf816bb70 con 0x7f9cf8141580 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfe2e5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf816be50 con 0x7f9cf80caf80 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfe2e5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf816c130 con 0x7f9cf81411b0 2023-10-06T03:11:14.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfc882700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfc882700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43574/0 (socket says 172.21.15.92:43574) 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cfc882700 1 -- 172.21.15.92:0/1674314444 learned_addr learned my addr 172.21.15.92:0/1674314444 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cf7fff700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf80caf80 0x7f9cf8158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.121+0000 7f9cf77fe700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9cf81411b0 0x7f9cf80c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfc882700 1 -- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9cf81411b0 msgr2=0x7f9cf80c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfc882700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9cf81411b0 0x7f9cf80c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfc882700 1 -- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf80caf80 msgr2=0x7f9cf8158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfc882700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf80caf80 0x7f9cf8158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfc882700 1 -- 172.21.15.92:0/1674314444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009420 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfc882700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80c5250 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000bb60 tx=0x7f9cf0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000c7e0 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000c960 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000cd20 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf816c410 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.122+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80c9c10 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.123+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf804c3c0 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.123+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9cf0014620 con 0x7f9cf8141580 2023-10-06T03:11:14.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.124+0000 7f9cdffff700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ce4067320 0x7f9ce40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.124+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9cf0084a10 con 0x7f9cf8141580 2023-10-06T03:11:14.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.124+0000 7f9cf7fff700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ce4067320 0x7f9ce40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.124+0000 7f9cf7fff700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ce4067320 0x7f9ce40697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8007fd0 tx=0x7f9ce8007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.130 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.128+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cf0051840 con 0x7f9cf8141580 2023-10-06T03:11:14.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.311+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9cf80ccc40 con 0x7f9cf8141580 2023-10-06T03:11:14.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.316+0000 7f9cdffff700 1 -- 172.21.15.92:0/1674314444 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9cf0054e60 con 0x7f9cf8141580 2023-10-06T03:11:14.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.318+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ce4067320 msgr2=0x7f9ce40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.318+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ce4067320 0x7f9ce40697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8007fd0 tx=0x7f9ce8007d30 comp rx=0 tx=0).stop 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.318+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 msgr2=0x7f9cf80c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.318+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80c5250 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000bb60 tx=0x7f9cf0009810 comp rx=0 tx=0).stop 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 shutdown_connections 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9ce4067320 0x7f9ce40697d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9cf8141580 0x7f9cf80c5250 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9cf80caf80 0x7f9cf8158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 --2- 172.21.15.92:0/1674314444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9cf81411b0 0x7f9cf80c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 >> 172.21.15.92:0/1674314444 conn(0x7f9cf806bb50 msgr2=0x7f9cf80b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 shutdown_connections 2023-10-06T03:11:14.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.319+0000 7f9cfe2e5700 1 -- 172.21.15.92:0/1674314444 wait complete. 2023-10-06T03:11:14.331 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:11:14.332 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:11:14.376 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:11:14.419 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:11:14.419 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:11:14.419 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:14.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.532+0000 7fedb58fa700 1 Processor -- start 2023-10-06T03:11:14.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.532+0000 7fedb58fa700 1 -- start start 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.532+0000 7fedb58fa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedb0159c10 0x7fedb015b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.532+0000 7fedb58fa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb00b5ea0 0x7fedb0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.532+0000 7fedb58fa700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb015c480 con 0x7fedb0068bb0 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedb58fa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb00b2a40 con 0x7fedb0159c10 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedb58fa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb015b5d0 con 0x7fedb00b5ea0 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedb0159c10 0x7fedb015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fedb0159c10 0x7fedb015b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43582/0 (socket says 172.21.15.92:43582) 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 -- 172.21.15.92:0/2863941534 learned_addr learned my addr 172.21.15.92:0/2863941534 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:14.535 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedae7fc700 1 --2- 172.21.15.92:0/2863941534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb00b5ea0 0x7fedb0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 -- 172.21.15.92:0/2863941534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb00b5ea0 msgr2=0x7fedb0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 --2- 172.21.15.92:0/2863941534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb00b5ea0 0x7fedb0068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 -- 172.21.15.92:0/2863941534 >> v1:172.21.15.136:6789/0 conn(0x7fedb0068bb0 legacy=0x7fedb015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 -- 172.21.15.92:0/2863941534 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb015b870 con 0x7fedb0159c10 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedaeffd700 1 --2- 172.21.15.92:0/2863941534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb0159c10 0x7fedb015b090 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7feda8009f20 tx=0x7feda8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb6eeda9d1c9478 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedadffb700 1 -- 172.21.15.92:0/2863941534 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800e070 con 0x7fedb0159c10 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedadffb700 1 -- 172.21.15.92:0/2863941534 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda800ba40 con 0x7fedb0159c10 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.533+0000 7fedadffb700 1 -- 172.21.15.92:0/2863941534 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda80092e0 con 0x7fedb0159c10 2023-10-06T03:11:14.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 -- 172.21.15.92:0/2863941534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb0159c10 msgr2=0x7fedb015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2863941534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb0159c10 0x7fedb015b090 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7feda8009f20 tx=0x7feda8009840 comp rx=0 tx=0).stop 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 -- 172.21.15.92:0/2863941534 shutdown_connections 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2863941534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb0159c10 0x7fedb015b090 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2863941534 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb00b5ea0 0x7fedb0068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 -- 172.21.15.92:0/2863941534 >> 172.21.15.92:0/2863941534 conn(0x7fedb0099a20 msgr2=0x7fedb00af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 -- 172.21.15.92:0/2863941534 shutdown_connections 2023-10-06T03:11:14.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.534+0000 7fedb58fa700 1 -- 172.21.15.92:0/2863941534 wait complete. 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.535+0000 7fedb58fa700 1 Processor -- start 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.535+0000 7fedb58fa700 1 -- start start 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedb58fa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb0068bb0 0x7fedb00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedb58fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 0x7fedb00b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedb58fa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedb0159c10 0x7fedb014a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedb58fa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb015e180 con 0x7fedb00b5ea0 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedb58fa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00b0bc0 con 0x7fedb0159c10 2023-10-06T03:11:14.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedb58fa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00b0ea0 con 0x7fedb0068bb0 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 0x7fedb00b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 0x7fedb00b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43592/0 (socket says 172.21.15.92:43592) 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 -- 172.21.15.92:0/2698464505 learned_addr learned my addr 172.21.15.92:0/2698464505 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedaeffd700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb0068bb0 0x7fedb00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedaf7fe700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedb0159c10 0x7fedb014a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 -- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb0068bb0 msgr2=0x7fedb00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb0068bb0 0x7fedb00c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 -- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedb0159c10 msgr2=0x7fedb014a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedb0159c10 0x7fedb014a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 -- 172.21.15.92:0/2698464505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8009ae0 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.536+0000 7fedae7fc700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 0x7fedb00b0370 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fed9800c880 tx=0x7fed98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.537+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98011600 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.537+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed98011760 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.537+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb00b1180 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.537+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98011b20 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.537+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb00b08b0 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.538+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb015e5f0 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.538+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fed98018360 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.538+0000 7feda6ffd700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed90067320 0x7fed900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.538+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fed98084ae0 con 0x7fedb00b5ea0 2023-10-06T03:11:14.542 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.538+0000 7fedaeffd700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed90067320 0x7fed900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.539+0000 7fedaeffd700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed90067320 0x7fed900697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7feda8009880 tx=0x7feda8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.543 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.542+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed98051800 con 0x7fedb00b5ea0 2023-10-06T03:11:14.768 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.768+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedb004c3c0 con 0x7fedb00b5ea0 2023-10-06T03:11:14.769 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.768+0000 7feda6ffd700 1 -- 172.21.15.92:0/2698464505 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+3537 (secure 0 0 0) 0x7fed98015090 con 0x7fedb00b5ea0 2023-10-06T03:11:14.769 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:14.769 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":130,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7744,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/836010014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":836010014},{"type":"v1","addr":"172.21.15.136:6833","nonce":836010014}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":7750,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1689073605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1689073605},{"type":"v1","addr":"172.21.15.136:6835","nonce":1689073605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/2593228109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":2593228109},{"type":"v1","addr":"172.21.15.92:6836","nonce":2593228109}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":119},{"gid":9129,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/1154539988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":1154539988},{"type":"v1","addr":"172.21.15.92:6837","nonce":1154539988}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[]} 2023-10-06T03:11:14.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed90067320 msgr2=0x7fed900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed90067320 0x7fed900697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7feda8009880 tx=0x7feda8017000 comp rx=0 tx=0).stop 2023-10-06T03:11:14.771 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 msgr2=0x7fedb00b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 0x7fedb00b0370 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fed9800c880 tx=0x7fed98008040 comp rx=0 tx=0).stop 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 shutdown_connections 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fed90067320 0x7fed900697d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedb0159c10 0x7fedb014a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedb0068bb0 0x7fedb00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 --2- 172.21.15.92:0/2698464505 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedb00b5ea0 0x7fedb00b0370 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.770+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 >> 172.21.15.92:0/2698464505 conn(0x7fedb0099a20 msgr2=0x7fedb009a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.771+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 shutdown_connections 2023-10-06T03:11:14.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.771+0000 7fedb58fa700 1 -- 172.21.15.92:0/2698464505 wait complete. 2023-10-06T03:11:14.773 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 130 2023-10-06T03:11:14.783 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:11:14.783 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:11:14.784 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:11:14.784 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:11:14.784 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:11:14.784 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:11:14.784 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:11:14.784 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:11:14.784 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:11:14.785 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:11:14.785 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:11:14.785 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:11:14.785 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:11:14.785 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:11:14.785 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:11:14.785 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:11:14.792 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:11:14.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:11:14.793 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:11:14.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:11:14.798 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:11:14.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:11:14.799 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:11:14.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:11:14.835 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:11:14.835 DEBUG:teuthology.parallel:result is None 2023-10-06T03:11:14.836 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:11:14.836 DEBUG:teuthology.parallel:result is None 2023-10-06T03:11:14.841 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:11:14.841 DEBUG:teuthology.parallel:result is None 2023-10-06T03:11:14.841 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:11:14.842 DEBUG:teuthology.parallel:result is None 2023-10-06T03:11:14.842 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:11:14.908 INFO:tasks.ceph.mds.a.smithi092.stdout:starting mds.a at 2023-10-06T03:11:14.908 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:11:14.922 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2023-10-06T03:11:14.923 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2023-10-06T03:11:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.961+0000 7f3cca311700 1 Processor -- start 2023-10-06T03:11:14.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.961+0000 7f3cca311700 1 -- start start 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cca311700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc4159890 0x7f3cc415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cca311700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cc4159c60 0x7f3cc415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cca311700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3cc415db10 0x7f3cc4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cca311700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc415a160 con 0x7f3cc4159c60 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cca311700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40d13a0 con 0x7f3cc415db10 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cca311700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc415e140 con 0x7f3cc4159890 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.962+0000 7f3cc37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cc4159c60 0x7f3cc415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3cc4159c60 0x7f3cc415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43632/0 (socket says 172.21.15.92:43632) 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 -- 172.21.15.92:0/316571453 learned_addr learned my addr 172.21.15.92:0/316571453 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:14.966 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc88ae700 1 --2- 172.21.15.92:0/316571453 >> v2:172.21.15.136:3300/0 conn(0x7f3cc415db10 0x7f3cc4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc3fff700 1 --2- 172.21.15.92:0/316571453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc4159890 0x7f3cc415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 -- 172.21.15.92:0/316571453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc4159890 msgr2=0x7f3cc415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 --2- 172.21.15.92:0/316571453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc4159890 0x7f3cc415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 -- 172.21.15.92:0/316571453 >> v2:172.21.15.136:3300/0 conn(0x7f3cc415db10 msgr2=0x7f3cc4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 --2- 172.21.15.92:0/316571453 >> v2:172.21.15.136:3300/0 conn(0x7f3cc415db10 0x7f3cc4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 -- 172.21.15.92:0/316571453 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc415a3d0 con 0x7f3cc4159c60 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc37fe700 1 --2- 172.21.15.92:0/316571453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159c60 0x7f3cc415d5d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4089b50 tx=0x7f3cb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a6d1231f3ebb92 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc2ffd700 1 -- 172.21.15.92:0/316571453 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800b860 con 0x7f3cc4159c60 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc2ffd700 1 -- 172.21.15.92:0/316571453 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb800b9c0 con 0x7f3cc4159c60 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.963+0000 7f3cc2ffd700 1 -- 172.21.15.92:0/316571453 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800bd60 con 0x7f3cc4159c60 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 -- 172.21.15.92:0/316571453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159c60 msgr2=0x7f3cc415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 --2- 172.21.15.92:0/316571453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159c60 0x7f3cc415d5d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4089b50 tx=0x7f3cb8009250 comp rx=0 tx=0).stop 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 -- 172.21.15.92:0/316571453 shutdown_connections 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 --2- 172.21.15.92:0/316571453 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc4159890 0x7f3cc415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 --2- 172.21.15.92:0/316571453 >> v2:172.21.15.136:3300/0 conn(0x7f3cc415db10 0x7f3cc4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 --2- 172.21.15.92:0/316571453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159c60 0x7f3cc415d5d0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 -- 172.21.15.92:0/316571453 >> 172.21.15.92:0/316571453 conn(0x7f3cc4068e20 msgr2=0x7f3cc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 -- 172.21.15.92:0/316571453 shutdown_connections 2023-10-06T03:11:14.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.964+0000 7f3cca311700 1 -- 172.21.15.92:0/316571453 wait complete. 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.965+0000 7f3cca311700 1 Processor -- start 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.965+0000 7f3cca311700 1 -- start start 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 0x7f3cc40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cc4159c60 0x7f3cc4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc415db10 0x7f3cc40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40ce9d0 con 0x7f3cc4159890 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c9150 con 0x7f3cc4159c60 2023-10-06T03:11:14.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c7bf0 con 0x7f3cc415db10 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 0x7f3cc40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 0x7f3cc40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43638/0 (socket says 172.21.15.92:43638) 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 -- 172.21.15.92:0/398919992 learned_addr learned my addr 172.21.15.92:0/398919992 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc88ae700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc415db10 0x7f3cc40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc37fe700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cc4159c60 0x7f3cc4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 -- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc415db10 msgr2=0x7f3cc40cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc415db10 0x7f3cc40cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 -- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cc4159c60 msgr2=0x7f3cc4151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cc4159c60 0x7f3cc4151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:14.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 -- 172.21.15.92:0/398919992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb80094f0 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc88ae700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc415db10 0x7f3cc40cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cc3fff700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 0x7f3cc40b3590 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400d8c0 tx=0x7f3cb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400de60 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb400b780 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400bb20 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc40c96e0 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.966+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc40ce770 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.967+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc404c3c0 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.968+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3cb401e460 con 0x7f3cc4159890 2023-10-06T03:11:14.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.968+0000 7f3cabfff700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cb0067320 0x7f3cb00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.968+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3cb4085170 con 0x7f3cc4159890 2023-10-06T03:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.968+0000 7f3cc37fe700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cb0067320 0x7f3cb00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.968+0000 7f3cc37fe700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cb0067320 0x7f3cb00697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4089b50 tx=0x7f3cb8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:14.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:14.972+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cb4051fa0 con 0x7f3cc4159890 2023-10-06T03:11:15.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:15.174+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3cc406a5b0 con 0x7f3cc4159890 2023-10-06T03:11:16.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.009+0000 7f3cabfff700 1 -- 172.21.15.92:0/398919992 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v197) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3cb40555c0 con 0x7f3cc4159890 2023-10-06T03:11:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cb0067320 msgr2=0x7f3cb00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cb0067320 0x7f3cb00697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4089b50 tx=0x7f3cb8017000 comp rx=0 tx=0).stop 2023-10-06T03:11:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 msgr2=0x7f3cc40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 0x7f3cc40b3590 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400d8c0 tx=0x7f3cb4008040 comp rx=0 tx=0).stop 2023-10-06T03:11:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 shutdown_connections 2023-10-06T03:11:16.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3cb0067320 0x7f3cb00697d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3cc4159890 0x7f3cc40b3590 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cc415db10 0x7f3cc40cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 --2- 172.21.15.92:0/398919992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cc4159c60 0x7f3cc4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 >> 172.21.15.92:0/398919992 conn(0x7f3cc4068e20 msgr2=0x7f3cc4154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 shutdown_connections 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.011+0000 7f3cca311700 1 -- 172.21.15.92:0/398919992 wait complete. 2023-10-06T03:11:16.014 INFO:teuthology.orchestra.run.smithi092.stderr: removed all blocklist entries 2023-10-06T03:11:16.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:11:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.139+0000 7ff457359700 1 Processor -- start 2023-10-06T03:11:16.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.139+0000 7ff457359700 1 -- start start 2023-10-06T03:11:16.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff457359700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff45006bcd0 0x7ff450140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff457359700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff45006c0a0 0x7ff450141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff457359700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45015c2a0 con 0x7ff4500c8e00 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff457359700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4500c9900 con 0x7ff45006bcd0 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff457359700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45015b690 con 0x7ff45006c0a0 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff4550f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff45006bcd0 0x7ff450140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff4550f5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff45006bcd0 0x7ff450140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43644/0 (socket says 172.21.15.92:43644) 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.140+0000 7ff4550f5700 1 -- 172.21.15.92:0/158491808 learned_addr learned my addr 172.21.15.92:0/158491808 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff4550f5700 1 -- 172.21.15.92:0/158491808 >> v2:172.21.15.136:3300/0 conn(0x7ff45006c0a0 msgr2=0x7ff450141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff4550f5700 1 --2- 172.21.15.92:0/158491808 >> v2:172.21.15.136:3300/0 conn(0x7ff45006c0a0 0x7ff450141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff4550f5700 1 -- 172.21.15.92:0/158491808 >> v1:172.21.15.136:6789/0 conn(0x7ff4500c8e00 legacy=0x7ff4500c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff4550f5700 1 -- 172.21.15.92:0/158491808 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45015b970 con 0x7ff45006bcd0 2023-10-06T03:11:16.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff4550f5700 1 --2- 172.21.15.92:0/158491808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff45006bcd0 0x7ff450140df0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff45006c6e0 tx=0x7ff44c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81bd1a6b4567d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff447fff700 1 -- 172.21.15.92:0/158491808 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c00e070 con 0x7ff45006bcd0 2023-10-06T03:11:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff447fff700 1 -- 172.21.15.92:0/158491808 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44c00ba10 con 0x7ff45006bcd0 2023-10-06T03:11:16.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff447fff700 1 -- 172.21.15.92:0/158491808 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44c0092e0 con 0x7ff45006bcd0 2023-10-06T03:11:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff457359700 1 -- 172.21.15.92:0/158491808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff45006bcd0 msgr2=0x7ff450140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.141+0000 7ff457359700 1 --2- 172.21.15.92:0/158491808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff45006bcd0 0x7ff450140df0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff45006c6e0 tx=0x7ff44c009840 comp rx=0 tx=0).stop 2023-10-06T03:11:16.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.142+0000 7ff457359700 1 -- 172.21.15.92:0/158491808 shutdown_connections 2023-10-06T03:11:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.142+0000 7ff457359700 1 --2- 172.21.15.92:0/158491808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff45006bcd0 0x7ff450140df0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.142+0000 7ff457359700 1 --2- 172.21.15.92:0/158491808 >> v2:172.21.15.136:3300/0 conn(0x7ff45006c0a0 0x7ff450141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.142+0000 7ff457359700 1 -- 172.21.15.92:0/158491808 >> 172.21.15.92:0/158491808 conn(0x7ff4500ce600 msgr2=0x7ff4500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.142+0000 7ff457359700 1 -- 172.21.15.92:0/158491808 shutdown_connections 2023-10-06T03:11:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.142+0000 7ff457359700 1 -- 172.21.15.92:0/158491808 wait complete. 2023-10-06T03:11:16.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.143+0000 7ff457359700 1 Processor -- start 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.143+0000 7ff457359700 1 -- start start 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff45006bcd0 0x7ff4500c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff45006c0a0 0x7ff4501553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 0x7ff4500b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500b35a0 con 0x7ff4500c8e00 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45013e9d0 con 0x7ff45006c0a0 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500b2fe0 con 0x7ff45006bcd0 2023-10-06T03:11:16.148 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 0x7ff4500b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 0x7ff4500b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43650/0 (socket says 172.21.15.92:43650) 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 -- 172.21.15.92:0/676529326 learned_addr learned my addr 172.21.15.92:0/676529326 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4550f5700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff45006bcd0 0x7ff4500c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4548f4700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff45006c0a0 0x7ff4501553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 -- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff45006bcd0 msgr2=0x7ff4500c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff45006bcd0 0x7ff4500c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 -- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff45006c0a0 msgr2=0x7ff4501553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff45006c0a0 0x7ff4501553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 -- 172.21.15.92:0/676529326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c009ae0 con 0x7ff4500c8e00 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4558f6700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 0x7ff4500b2520 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff44800b4e0 tx=0x7ff448009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800f5e0 con 0x7ff4500c8e00 2023-10-06T03:11:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44800f760 con 0x7ff4500c8e00 2023-10-06T03:11:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4500b32c0 con 0x7ff4500c8e00 2023-10-06T03:11:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800fb20 con 0x7ff4500c8e00 2023-10-06T03:11:16.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.144+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45013ef50 con 0x7ff4500c8e00 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.145+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45013f170 con 0x7ff4500c8e00 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.146+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff44801a400 con 0x7ff4500c8e00 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.146+0000 7ff4457fa700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff4340672f0 0x7ff4340697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.146+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff448084540 con 0x7ff4500c8e00 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.146+0000 7ff4550f5700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff4340672f0 0x7ff4340697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.146+0000 7ff4550f5700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff4340672f0 0x7ff4340697a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009880 tx=0x7ff44c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.150+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4480514c0 con 0x7ff4500c8e00 2023-10-06T03:11:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.406+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff45004c3c0 con 0x7ff4500c8e00 2023-10-06T03:11:16.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.406+0000 7ff4457fa700 1 -- 172.21.15.92:0/676529326 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff44800fed0 con 0x7ff4500c8e00 2023-10-06T03:11:16.408 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "auth_dump": [ 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.a", 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lYaIEBxAAWQY4kLxy3GEx0qT0z8kvww==", 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.409 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.b", 2023-10-06T03:11:16.410 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l3MpBEhAAHfvSeP8tolmT69n7XqdBMA==", 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.c", 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lWcfBDRAABYDFUK5Op7BBnhL5a1xvQQ==", 2023-10-06T03:11:16.411 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mds.d", 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lhkkaGRAAryxdOqefJCX3c9j4JsAchQ==", 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.412 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.0", 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBgeB9lvYLrABAA+8rSfwnzrkKwdOG2coJwjw==", 2023-10-06T03:11:16.413 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.1", 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBieB9l0BGROhAA5vX/fdDCEtZ3LAdDD/4aoQ==", 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.2", 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBleB9lAOhiOBAA8Fy97IWVaPGgS+VCuHUqaA==", 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.415 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.3", 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBoeB9lWVbXOBAArXQVjiBIjJYJe0PnSRNuGA==", 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.416 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.4", 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBueB9lw3wKOhAAfRLrVLpVOnyfjRurarhVgg==", 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.5", 2023-10-06T03:11:16.417 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBxeB9lac/RNxAA3jn73m7F4bMR4GRIpXTugA==", 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.418 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.6", 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB0eB9ljbHdOBAAf0pBG1el46LcZ2PDOXJiFA==", 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "osd.7", 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQB3eB9lK0QpNhAAplyKkCxAzmzbmwo1ntEHjA==", 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.419 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow profile osd", 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile osd", 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.0", 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lNb4IHRAAD7h+UaV7ixJND9iqPSIYvw==", 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow r", 2023-10-06T03:11:16.420 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.1", 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lUtBkJRAAIkopI2OgDJwkHI8m8qO0yg==", 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.421 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.2", 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9l58ViIhAA0Jj1bDSLrq6NBGCg0dVWVQ==", 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.422 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.3", 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9lBKy6KhAAFmNq2jfJK/q7+qy5xwb6wg==", 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow", 2023-10-06T03:11:16.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow r", 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow rw", 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow rwx" 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "client.admin", 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9l00+UIhAA05CDqEZXHNOjx21CHsAcAQ==", 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mgr": "allow *", 2023-10-06T03:11:16.424 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow *", 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.x", 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBceB9la7IABBAAhnSSaUHcNfMBfWfNLvL45g==", 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.425 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "entity": "mgr.y", 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "key": "AQBbeB9lP2tQNxAAS7Jp2UkJCs/5Cxc+2JNRVg==", 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": { 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": "allow *", 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "mon": "allow profile mgr", 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: "osd": "allow *" 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2023-10-06T03:11:16.426 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:11:16.428 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff4340672f0 msgr2=0x7ff4340697a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff4340672f0 0x7ff4340697a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009880 tx=0x7ff44c017000 comp rx=0 tx=0).stop 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 msgr2=0x7ff4500b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 0x7ff4500b2520 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff44800b4e0 tx=0x7ff448009060 comp rx=0 tx=0).stop 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 shutdown_connections 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff4340672f0 0x7ff4340697a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4500c8e00 0x7ff4500b2520 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff45006bcd0 0x7ff4500c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 --2- 172.21.15.92:0/676529326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff45006c0a0 0x7ff4501553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 >> 172.21.15.92:0/676529326 conn(0x7ff4500ce600 msgr2=0x7ff4500c49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 shutdown_connections 2023-10-06T03:11:16.429 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.408+0000 7ff457359700 1 -- 172.21.15.92:0/676529326 wait complete. 2023-10-06T03:11:16.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 Processor -- start 2023-10-06T03:11:16.544 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 -- start start 2023-10-06T03:11:16.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6ff8159c60 0x7f6ff815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ff815db10 0x7f6ff8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.547 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff815a160 con 0x7f6ff8159890 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff815df50 con 0x7f6ff815db10 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6fff09a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff80d13a0 con 0x7f6ff8159c60 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6ffce36700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6ff8159890 0x7f6ff815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41778/0 (socket says 172.21.15.92:41778) 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6ffce36700 1 -- 172.21.15.92:0/2159136480 learned_addr learned my addr 172.21.15.92:0/2159136480 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.543+0000 7f6ffd637700 1 --2- 172.21.15.92:0/2159136480 >> v2:172.21.15.92:3300/0 conn(0x7f6ff815db10 0x7f6ff8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ff7fff700 1 --2- 172.21.15.92:0/2159136480 >> v2:172.21.15.136:3300/0 conn(0x7f6ff8159c60 0x7f6ff815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ffd637700 1 -- 172.21.15.92:0/2159136480 >> v2:172.21.15.136:3300/0 conn(0x7f6ff8159c60 msgr2=0x7f6ff815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:16.548 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ffd637700 1 --2- 172.21.15.92:0/2159136480 >> v2:172.21.15.136:3300/0 conn(0x7f6ff8159c60 0x7f6ff815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ffd637700 1 -- 172.21.15.92:0/2159136480 >> v1:172.21.15.92:6789/0 conn(0x7f6ff8159890 legacy=0x7f6ff815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ffd637700 1 -- 172.21.15.92:0/2159136480 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff815e140 con 0x7f6ff815db10 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ff77fe700 1 -- 172.21.15.92:0/2159136480 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339218271 0 0) 0x7f6ff815a160 con 0x7f6ff8159890 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ffd637700 1 --2- 172.21.15.92:0/2159136480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff815db10 0x7f6ff8069560 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80b67f0 tx=0x7f6ff0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ad78c550f265fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ff77fe700 1 -- 172.21.15.92:0/2159136480 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000e070 con 0x7f6ff815db10 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ff77fe700 1 -- 172.21.15.92:0/2159136480 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff000bc30 con 0x7f6ff815db10 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.544+0000 7f6ff77fe700 1 -- 172.21.15.92:0/2159136480 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0012430 con 0x7f6ff815db10 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 -- 172.21.15.92:0/2159136480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff815db10 msgr2=0x7f6ff8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 --2- 172.21.15.92:0/2159136480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff815db10 0x7f6ff8069560 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6ff80b67f0 tx=0x7f6ff0009780 comp rx=0 tx=0).stop 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 -- 172.21.15.92:0/2159136480 shutdown_connections 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 --2- 172.21.15.92:0/2159136480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff815db10 0x7f6ff8069560 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.549 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 --2- 172.21.15.92:0/2159136480 >> v2:172.21.15.136:3300/0 conn(0x7f6ff8159c60 0x7f6ff815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 -- 172.21.15.92:0/2159136480 >> 172.21.15.92:0/2159136480 conn(0x7f6ff8068e20 msgr2=0x7f6ff80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:16.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 -- 172.21.15.92:0/2159136480 shutdown_connections 2023-10-06T03:11:16.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.545+0000 7f6fff09a700 1 -- 172.21.15.92:0/2159136480 wait complete. 2023-10-06T03:11:16.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.546+0000 7f6fff09a700 1 Processor -- start 2023-10-06T03:11:16.550 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.546+0000 7f6fff09a700 1 -- start start 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 0x7f6ff80cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ff8159c60 0x7f6ff80cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ff815db10 0x7f6ff8148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80b0540 con 0x7f6ff8159890 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff816d490 con 0x7f6ff8159c60 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff816d770 con 0x7f6ff815db10 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 0x7f6ff80cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 0x7f6ff80cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43680/0 (socket says 172.21.15.92:43680) 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 -- 172.21.15.92:0/691041723 learned_addr learned my addr 172.21.15.92:0/691041723 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ff7fff700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ff8159c60 0x7f6ff80cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffd637700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ff815db10 0x7f6ff8148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 -- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ff815db10 msgr2=0x7f6ff8148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ff815db10 0x7f6ff8148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 -- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ff8159c60 msgr2=0x7f6ff80cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ff8159c60 0x7f6ff80cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 -- 172.21.15.92:0/691041723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0009a20 con 0x7f6ff8159890 2023-10-06T03:11:16.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ffce36700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 0x7f6ff80cf7a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c7e0 tx=0x7f6fec007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec005b30 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec005c90 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff816da50 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec007300 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.547+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff80cfce0 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.548+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff80b06b0 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.549+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6fec01b400 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.549+0000 7f6ff4ff9700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6fdc067320 0x7f6fdc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.549+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6fec084510 con 0x7f6ff8159890 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.549+0000 7f6ff7fff700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6fdc067320 0x7f6fdc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.549+0000 7f6ff7fff700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6fdc067320 0x7f6fdc0697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8150f60 tx=0x7f6fe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.553+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fec051490 con 0x7f6ff8159890 2023-10-06T03:11:16.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.738+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ff804c3c0 con 0x7f6ff8159890 2023-10-06T03:11:16.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.739+0000 7f6ff4ff9700 1 -- 172.21.15.92:0/691041723 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+13660 (secure 0 0 0) 0x7f6fec054ab0 con 0x7f6ff8159890 2023-10-06T03:11:16.741 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:16.741 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":197,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:16.005644+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6fdc067320 msgr2=0x7f6fdc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6fdc067320 0x7f6fdc0697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8150f60 tx=0x7f6fe4008040 comp rx=0 tx=0).stop 2023-10-06T03:11:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 msgr2=0x7f6ff80cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 0x7f6ff80cf7a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00c7e0 tx=0x7f6fec007df0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 shutdown_connections 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6fdc067320 0x7f6fdc0697d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ff8159890 0x7f6ff80cf7a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ff815db10 0x7f6ff8148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 --2- 172.21.15.92:0/691041723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ff8159c60 0x7f6ff80cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 >> 172.21.15.92:0/691041723 conn(0x7f6ff8068e20 msgr2=0x7f6ff81552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 shutdown_connections 2023-10-06T03:11:16.744 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.741+0000 7f6fff09a700 1 -- 172.21.15.92:0/691041723 wait complete. 2023-10-06T03:11:16.755 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T03:11:16.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:11:16.872 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.871+0000 7f0f82f4b700 1 Processor -- start 2023-10-06T03:11:16.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.871+0000 7f0f82f4b700 1 -- start start 2023-10-06T03:11:16.874 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f82f4b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 0x7f0f7c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f82f4b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b1b40 0x7f0f7c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f82f4b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c0cfb00 con 0x7f0f7c0b16d0 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f82f4b700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c0b5c40 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f82f4b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0b5f20 con 0x7f0f7c0b1b40 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f81f49700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 0x7f0f7c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f81f49700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 0x7f0f7c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:50140/0 (socket says 172.21.15.92:50140) 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f8274a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b1b40 0x7f0f7c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f81748700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f0f7c0b16d0 0x7f0f7c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:43476/0 (socket says 172.21.15.92:43476) 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.872+0000 7f0f81f49700 1 -- 172.21.15.92:0/3437507404 learned_addr learned my addr 172.21.15.92:0/3437507404 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:16.875 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f8274a700 1 -- 172.21.15.92:0/3437507404 >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 msgr2=0x7f0f7c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f8274a700 1 --2- 172.21.15.92:0/3437507404 >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 0x7f0f7c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f8274a700 1 -- 172.21.15.92:0/3437507404 >> v1:172.21.15.136:6789/0 conn(0x7f0f7c0b16d0 legacy=0x7f0f7c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f8274a700 1 -- 172.21.15.92:0/3437507404 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0b6200 con 0x7f0f7c0b1b40 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f81f49700 1 --2- 172.21.15.92:0/3437507404 >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 0x7f0f7c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f8274a700 1 --2- 172.21.15.92:0/3437507404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b1b40 0x7f0f7c15d950 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009e60 tx=0x7f0f74009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d9e718fa3ffafd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f80f47700 1 -- 172.21.15.92:0/3437507404 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400e070 con 0x7f0f7c0b1b40 2023-10-06T03:11:16.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f80f47700 1 -- 172.21.15.92:0/3437507404 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7400bb60 con 0x7f0f7c0b1b40 2023-10-06T03:11:16.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f80f47700 1 -- 172.21.15.92:0/3437507404 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74012430 con 0x7f0f7c0b1b40 2023-10-06T03:11:16.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f82f4b700 1 -- 172.21.15.92:0/3437507404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b1b40 msgr2=0x7f0f7c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.873+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/3437507404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b1b40 0x7f0f7c15d950 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f74009e60 tx=0x7f0f74009780 comp rx=0 tx=0).stop 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.874+0000 7f0f82f4b700 1 -- 172.21.15.92:0/3437507404 shutdown_connections 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.874+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/3437507404 >> v2:172.21.15.136:3300/0 conn(0x7f0f7c0b4e70 0x7f0f7c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.874+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/3437507404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b1b40 0x7f0f7c15d950 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.874+0000 7f0f82f4b700 1 -- 172.21.15.92:0/3437507404 >> 172.21.15.92:0/3437507404 conn(0x7f0f7c0d10c0 msgr2=0x7f0f7c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.874+0000 7f0f82f4b700 1 -- 172.21.15.92:0/3437507404 shutdown_connections 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.874+0000 7f0f82f4b700 1 -- 172.21.15.92:0/3437507404 wait complete. 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 Processor -- start 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 -- start start 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b16d0 0x7f0f7c14c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.880 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7c0b1b40 0x7f0f7c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 0x7f0f7c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cab60 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c14c680 con 0x7f0f7c0b1b40 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.875+0000 7f0f82f4b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0c7d70 con 0x7f0f7c0b16d0 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 0x7f0f7c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 0x7f0f7c159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43682/0 (socket says 172.21.15.92:43682) 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 -- 172.21.15.92:0/77355812 learned_addr learned my addr 172.21.15.92:0/77355812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f81748700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7c0b1b40 0x7f0f7c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f81f49700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b16d0 0x7f0f7c14c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 -- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b16d0 msgr2=0x7f0f7c14c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.881 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b16d0 0x7f0f7c14c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 -- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7c0b1b40 msgr2=0x7f0f7c158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7c0b1b40 0x7f0f7c158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 -- 172.21.15.92:0/77355812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f780094f0 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f8274a700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 0x7f0f7c159130 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b6c0 tx=0x7f0f74019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74012e90 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f74027470 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f74009a20 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74027850 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.876+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c0c7720 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.877+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c0c7580 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.878+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0f74024070 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.878+0000 7f0f71ffb700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0f60067320 0x7f0f600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.878+0000 7f0f81f49700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0f60067320 0x7f0f600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.878+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0f740873f0 con 0x7f0f7c0b4e70 2023-10-06T03:11:16.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.879+0000 7f0f81f49700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0f60067320 0x7f0f600697d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0f78006d20 tx=0x7f0f78009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:16.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:16.882+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f7c0c7580 con 0x7f0f7c0b4e70 2023-10-06T03:11:17.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.064+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f7c0c7580 con 0x7f0f7c0b4e70 2023-10-06T03:11:17.066 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.064+0000 7f0f71ffb700 1 -- 172.21.15.92:0/77355812 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v197) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f7c0c7580 con 0x7f0f7c0b4e70 2023-10-06T03:11:17.066 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:11:17.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0f60067320 msgr2=0x7f0f600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0f60067320 0x7f0f600697d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0f78006d20 tx=0x7f0f78009250 comp rx=0 tx=0).stop 2023-10-06T03:11:17.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 msgr2=0x7f0f7c159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 0x7f0f7c159130 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400b6c0 tx=0x7f0f74019000 comp rx=0 tx=0).stop 2023-10-06T03:11:17.068 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 shutdown_connections 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0f60067320 0x7f0f600697d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f7c0b4e70 0x7f0f7c159130 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f7c0b16d0 0x7f0f7c14c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 --2- 172.21.15.92:0/77355812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f7c0b1b40 0x7f0f7c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 >> 172.21.15.92:0/77355812 conn(0x7f0f7c0d10c0 msgr2=0x7f0f7c0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 shutdown_connections 2023-10-06T03:11:17.069 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.066+0000 7f0f82f4b700 1 -- 172.21.15.92:0/77355812 wait complete. 2023-10-06T03:11:17.080 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:11:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.195+0000 7f90ea735700 1 Processor -- start 2023-10-06T03:11:17.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.195+0000 7f90ea735700 1 -- start start 2023-10-06T03:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90ea735700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159890 0x7f90e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90ea735700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f90e4159c60 0x7f90e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90ea735700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90e415db10 0x7f90e4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90ea735700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e415a160 con 0x7f90e415db10 2023-10-06T03:11:17.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90ea735700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e40d13a0 con 0x7f90e4159c60 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90ea735700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e415e140 con 0x7f90e4159890 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90e415db10 0x7f90e4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f90e415db10 0x7f90e4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43692/0 (socket says 172.21.15.92:43692) 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 -- 172.21.15.92:0/2909365701 learned_addr learned my addr 172.21.15.92:0/2909365701 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e3fff700 1 --2- 172.21.15.92:0/2909365701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159890 0x7f90e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e37fe700 1 --2- 172.21.15.92:0/2909365701 >> v2:172.21.15.136:3300/0 conn(0x7f90e4159c60 0x7f90e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 -- 172.21.15.92:0/2909365701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159890 msgr2=0x7f90e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 --2- 172.21.15.92:0/2909365701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159890 0x7f90e415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 -- 172.21.15.92:0/2909365701 >> v2:172.21.15.136:3300/0 conn(0x7f90e4159c60 msgr2=0x7f90e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 --2- 172.21.15.92:0/2909365701 >> v2:172.21.15.136:3300/0 conn(0x7f90e4159c60 0x7f90e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.196+0000 7f90e8cd2700 1 -- 172.21.15.92:0/2909365701 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e415a3d0 con 0x7f90e415db10 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90e8cd2700 1 --2- 172.21.15.92:0/2909365701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e4069560 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f90e40650f0 tx=0x7f90dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2339715e951364 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90e2ffd700 1 -- 172.21.15.92:0/2909365701 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00e070 con 0x7f90e415db10 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90e2ffd700 1 -- 172.21.15.92:0/2909365701 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00ba50 con 0x7f90e415db10 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90e2ffd700 1 -- 172.21.15.92:0/2909365701 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0092e0 con 0x7f90e415db10 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 -- 172.21.15.92:0/2909365701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 msgr2=0x7f90e4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 --2- 172.21.15.92:0/2909365701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e4069560 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f90e40650f0 tx=0x7f90dc009840 comp rx=0 tx=0).stop 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 -- 172.21.15.92:0/2909365701 shutdown_connections 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 --2- 172.21.15.92:0/2909365701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159890 0x7f90e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 --2- 172.21.15.92:0/2909365701 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e4069560 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 --2- 172.21.15.92:0/2909365701 >> v2:172.21.15.136:3300/0 conn(0x7f90e4159c60 0x7f90e415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 -- 172.21.15.92:0/2909365701 >> 172.21.15.92:0/2909365701 conn(0x7f90e4068e20 msgr2=0x7f90e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:17.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 -- 172.21.15.92:0/2909365701 shutdown_connections 2023-10-06T03:11:17.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.197+0000 7f90ea735700 1 -- 172.21.15.92:0/2909365701 wait complete. 2023-10-06T03:11:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.198+0000 7f90ea735700 1 Processor -- start 2023-10-06T03:11:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.198+0000 7f90ea735700 1 -- start start 2023-10-06T03:11:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90ea735700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e4159890 0x7f90e4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90ea735700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159c60 0x7f90e40c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90ea735700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e40b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90ea735700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40ce820 con 0x7f90e415db10 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90ea735700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40cdc50 con 0x7f90e4159890 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90ea735700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e40cdef0 con 0x7f90e4159c60 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90e8cd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e40b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90e8cd2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e40b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43702/0 (socket says 172.21.15.92:43702) 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.199+0000 7f90e8cd2700 1 -- 172.21.15.92:0/3431778623 learned_addr learned my addr 172.21.15.92:0/3431778623 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e8cd2700 1 -- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159c60 msgr2=0x7f90e40c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e8cd2700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159c60 0x7f90e40c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e3fff700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e4159890 0x7f90e4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e8cd2700 1 -- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e4159890 msgr2=0x7f90e4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e8cd2700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e4159890 0x7f90e4151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e8cd2700 1 -- 172.21.15.92:0/3431778623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90dc009ae0 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90e8cd2700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e40b5970 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f90e4070e20 tx=0x7f90dc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0032f0 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc008e80 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e40ce1d0 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc01c640 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.200+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e4152290 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.201+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e404c3c0 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.201+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f90dc003040 con 0x7f90e415db10 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.202+0000 7f90cbfff700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d0067320 0x7f90d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.206 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.202+0000 7f90e3fff700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d0067320 0x7f90d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.202+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f90dc086600 con 0x7f90e415db10 2023-10-06T03:11:17.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.202+0000 7f90e3fff700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d0067320 0x7f90d00697d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f90d40089b0 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.207 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.206+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90dc053370 con 0x7f90e415db10 2023-10-06T03:11:17.390 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.389+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90e40ce960 con 0x7f90e415db10 2023-10-06T03:11:17.391 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.389+0000 7f90cbfff700 1 -- 172.21.15.92:0/3431778623 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v197) v1 ==== 92+0+10 (secure 0 0 0) 0x7f90dc056990 con 0x7f90e415db10 2023-10-06T03:11:17.391 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d0067320 msgr2=0x7f90d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d0067320 0x7f90d00697d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f90d40089b0 tx=0x7f90d4008040 comp rx=0 tx=0).stop 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 msgr2=0x7f90e40b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e40b5970 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f90e4070e20 tx=0x7f90dc018000 comp rx=0 tx=0).stop 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 shutdown_connections 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f90d0067320 0x7f90d00697d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90e415db10 0x7f90e40b5970 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90e4159890 0x7f90e4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.393 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 --2- 172.21.15.92:0/3431778623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90e4159c60 0x7f90e40c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 >> 172.21.15.92:0/3431778623 conn(0x7f90e4068e20 msgr2=0x7f90e415b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:17.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 shutdown_connections 2023-10-06T03:11:17.394 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.391+0000 7f90ea735700 1 -- 172.21.15.92:0/3431778623 wait complete. 2023-10-06T03:11:17.405 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:11:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.518+0000 7f0871f3a700 1 Processor -- start 2023-10-06T03:11:17.520 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.518+0000 7f0871f3a700 1 -- start start 2023-10-06T03:11:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f0871f3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f086c0cb5a0 0x7f086c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f0871f3a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f086c0c85a0 0x7f086c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f0871f3a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086c069d30 con 0x7f086c156fb0 2023-10-06T03:11:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f0871f3a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086c0c8110 con 0x7f086c0c85a0 2023-10-06T03:11:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f0871f3a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0cbf30 con 0x7f086c0cb5a0 2023-10-06T03:11:17.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f086b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f086c156fb0 0x7f086c06a5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41792/0 (socket says 172.21.15.92:41792) 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.519+0000 7f086b7fe700 1 -- 172.21.15.92:0/136976601 learned_addr learned my addr 172.21.15.92:0/136976601 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f0863fff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f086c0cb5a0 0x7f086c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 --2- 172.21.15.92:0/136976601 >> v2:172.21.15.136:3300/0 conn(0x7f086c0c85a0 0x7f086c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662004692 0 0) 0x7f086c069d30 con 0x7f086c156fb0 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0858003570 con 0x7f086c156fb0 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2513526560 0 0) 0x7f0858003570 con 0x7f086c156fb0 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f086c0cb5a0 msgr2=0x7f086c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f086c0cb5a0 0x7f086c0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 >> v2:172.21.15.136:3300/0 conn(0x7f086c0c85a0 msgr2=0x7f086c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:17.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 --2- 172.21.15.92:0/136976601 >> v2:172.21.15.136:3300/0 conn(0x7f086c0c85a0 0x7f086c0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c1568e0 con 0x7f086c156fb0 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f085c001470 con 0x7f086c156fb0 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 >> v1:172.21.15.92:6789/0 conn(0x7f086c156fb0 legacy=0x7f086c06a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0858003a60 0x7f0858003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0858004810 0x7f0858004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08580051c0 0x7f085800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0c8110 con 0x7f08580051c0 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0cbf30 con 0x7f0858003a60 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085800e1b0 con 0x7f0858004810 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 --2- 172.21.15.92:0/136976601 >> v2:172.21.15.136:3300/0 conn(0x7f086c0c85a0 0x7f086c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f0863fff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f086c0cb5a0 0x7f086c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:17.526 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08580051c0 0x7f085800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086b7fe700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0858003a60 0x7f0858003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f0863fff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0858004810 0x7f0858004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 -- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0858004810 msgr2=0x7f0858004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0858004810 0x7f0858004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 -- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0858003a60 msgr2=0x7f0858003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0858003a60 0x7f0858003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.520+0000 7f086bfff700 1 -- 172.21.15.92:0/136976601 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858003570 con 0x7f08580051c0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f086bfff700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08580051c0 0x7f085800ba60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f086400e3c0 tx=0x7f086400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2dee063217a214 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086400ec70 con 0x7f08580051c0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086400f040 con 0x7f08580051c0 2023-10-06T03:11:17.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f086affd700 1 -- 172.21.15.92:0/136976601 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864014480 con 0x7f08580051c0 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f0871f3a700 1 -- 172.21.15.92:0/136976601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08580051c0 msgr2=0x7f085800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f0871f3a700 1 --2- 172.21.15.92:0/136976601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08580051c0 0x7f085800ba60 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f086400e3c0 tx=0x7f086400b040 comp rx=0 tx=0).stop 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f086bfff700 1 -- 172.21.15.92:0/136976601 reap_dead start 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f0871f3a700 1 -- 172.21.15.92:0/136976601 shutdown_connections 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f0871f3a700 1 -- 172.21.15.92:0/136976601 >> 172.21.15.92:0/136976601 conn(0x7f086c0c9870 msgr2=0x7f086c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f0871f3a700 1 -- 172.21.15.92:0/136976601 shutdown_connections 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.521+0000 7f0871f3a700 1 -- 172.21.15.92:0/136976601 wait complete. 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.522+0000 7f0871f3a700 1 Processor -- start 2023-10-06T03:11:17.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.522+0000 7f0871f3a700 1 -- start start 2023-10-06T03:11:17.530 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0871f3a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08580051c0 0x7f086c14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0871f3a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f086c0b2630 0x7f086c0c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0871f3a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 0x7f086c1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0871f3a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c0c6400 con 0x7f086c0c5f80 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0871f3a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c14e0e0 con 0x7f086c0b2630 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0871f3a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c14d7b0 con 0x7f08580051c0 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f086bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 0x7f086c1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f086bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 0x7f086c1530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43716/0 (socket says 172.21.15.92:43716) 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f086bfff700 1 -- 172.21.15.92:0/473680921 learned_addr learned my addr 172.21.15.92:0/473680921 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f086bfff700 1 -- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08580051c0 msgr2=0x7f086c14c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.531 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f086bfff700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08580051c0 0x7f086c14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f086bfff700 1 -- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f086c0b2630 msgr2=0x7f086c0c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.523+0000 7f0863fff700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f086c0b2630 0x7f086c0c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f086bfff700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f086c0b2630 0x7f086c0c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f086bfff700 1 -- 172.21.15.92:0/473680921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c14da90 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f086bfff700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 0x7f086c1530e0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0864009750 tx=0x7f0864019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0864014e00 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086401f660 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086401fa20 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c14dd70 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.524+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c153620 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.525+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c04c3c0 con 0x7f086c0c5f80 2023-10-06T03:11:17.532 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.525+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0864025070 con 0x7f086c0c5f80 2023-10-06T03:11:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.526+0000 7f08637fe700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f084c067320 0x7f084c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.526+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0864086ba0 con 0x7f086c0c5f80 2023-10-06T03:11:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.526+0000 7f086b7fe700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f084c067320 0x7f084c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.526+0000 7f086b7fe700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f084c067320 0x7f084c0697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f085c005bd0 tx=0x7f086c0c8250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.533 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.530+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0864053a20 con 0x7f086c0c5f80 2023-10-06T03:11:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.717+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f086c14e250 con 0x7f086c0c5f80 2023-10-06T03:11:17.720 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.717+0000 7f08637fe700 1 -- 172.21.15.92:0/473680921 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v197) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0864057040 con 0x7f086c0c5f80 2023-10-06T03:11:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:17.720 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:11:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:17.720 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 1, 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": ".mgr" 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "poolnum": 2, 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: "poolname": "rbd" 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:17.721 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:11:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f084c067320 msgr2=0x7f084c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f084c067320 0x7f084c0697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f085c005bd0 tx=0x7f086c0c8250 comp rx=0 tx=0).stop 2023-10-06T03:11:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 msgr2=0x7f086c1530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 0x7f086c1530e0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f0864009750 tx=0x7f0864019000 comp rx=0 tx=0).stop 2023-10-06T03:11:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 shutdown_connections 2023-10-06T03:11:17.722 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f084c067320 0x7f084c0697d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.719+0000 7f0871f3a700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f086c0c5f80 0x7f086c1530e0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.720+0000 7f0871f3a700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08580051c0 0x7f086c14c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.720+0000 7f0871f3a700 1 --2- 172.21.15.92:0/473680921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f086c0b2630 0x7f086c0c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.720+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 >> 172.21.15.92:0/473680921 conn(0x7f086c0c9870 msgr2=0x7f086c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:17.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.720+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 shutdown_connections 2023-10-06T03:11:17.723 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.720+0000 7f0871f3a700 1 -- 172.21.15.92:0/473680921 wait complete. 2023-10-06T03:11:17.733 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:11:17.734 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:11:17.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.849+0000 7eff800a7700 1 Processor -- start 2023-10-06T03:11:17.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.849+0000 7eff800a7700 1 -- start start 2023-10-06T03:11:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff800a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff781430c0 0x7eff780cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff800a7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff780cf410 con 0x7eff78069bb0 2023-10-06T03:11:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff800a7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff780cf6f0 con 0x7eff78143490 2023-10-06T03:11:17.853 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff800a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780cf9d0 con 0x7eff781430c0 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff7e644700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7eff78069bb0 0x7eff78163540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:41800/0 (socket says 172.21.15.92:41800) 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff7e644700 1 -- 172.21.15.92:0/3205083644 learned_addr learned my addr 172.21.15.92:0/3205083644 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.850+0000 7eff7de43700 1 --2- 172.21.15.92:0/3205083644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff781430c0 0x7eff780cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724378649 0 0) 0x7eff780cf410 con 0x7eff78069bb0 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff64003570 con 0x7eff78069bb0 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413226297 0 0) 0x7eff780cf6f0 con 0x7eff78143490 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff780cf410 con 0x7eff78143490 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7de43700 1 -- 172.21.15.92:0/3205083644 >> v1:172.21.15.136:6789/0 conn(0x7eff78143490 legacy=0x7eff780694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7de43700 1 -- 172.21.15.92:0/3205083644 >> v1:172.21.15.92:6789/0 conn(0x7eff78069bb0 legacy=0x7eff78163540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7de43700 1 -- 172.21.15.92:0/3205083644 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780cfcb0 con 0x7eff781430c0 2023-10-06T03:11:17.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7de43700 1 --2- 172.21.15.92:0/3205083644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff781430c0 0x7eff780cc7d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff74009910 tx=0x7eff740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88eefd2ee0a69497 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74011630 con 0x7eff781430c0 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74011790 con 0x7eff781430c0 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.851+0000 7eff7ce41700 1 -- 172.21.15.92:0/3205083644 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74011b30 con 0x7eff781430c0 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 -- 172.21.15.92:0/3205083644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff781430c0 msgr2=0x7eff780cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 --2- 172.21.15.92:0/3205083644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff781430c0 0x7eff780cc7d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff74009910 tx=0x7eff740091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 -- 172.21.15.92:0/3205083644 shutdown_connections 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 --2- 172.21.15.92:0/3205083644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff781430c0 0x7eff780cc7d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 -- 172.21.15.92:0/3205083644 >> 172.21.15.92:0/3205083644 conn(0x7eff78068e60 msgr2=0x7eff780b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 -- 172.21.15.92:0/3205083644 shutdown_connections 2023-10-06T03:11:17.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.852+0000 7eff800a7700 1 -- 172.21.15.92:0/3205083644 wait complete. 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.853+0000 7eff800a7700 1 Processor -- start 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.853+0000 7eff800a7700 1 -- start start 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.853+0000 7eff800a7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff78069bb0 0x7eff78158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff800a7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff781430c0 0x7eff7813b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff800a7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 0x7eff7813bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff800a7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7813ed80 con 0x7eff78143490 2023-10-06T03:11:17.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff800a7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff78159400 con 0x7eff781430c0 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff800a7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815c940 con 0x7eff78069bb0 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 0x7eff7813bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 0x7eff7813bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43722/0 (socket says 172.21.15.92:43722) 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 -- 172.21.15.92:0/423943170 learned_addr learned my addr 172.21.15.92:0/423943170 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7de43700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff78069bb0 0x7eff78158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7d642700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff781430c0 0x7eff7813b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 -- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff78069bb0 msgr2=0x7eff78158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff78069bb0 0x7eff78158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 -- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff781430c0 msgr2=0x7eff7813b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff781430c0 0x7eff7813b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 -- 172.21.15.92:0/423943170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74009420 con 0x7eff78143490 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7de43700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff78069bb0 0x7eff78158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7d642700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff781430c0 0x7eff7813b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff7e644700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 0x7eff7813bee0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7eff7000cd20 tx=0x7eff70001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000ab30 con 0x7eff78143490 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7000acb0 con 0x7eff78143490 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70007340 con 0x7eff78143490 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.854+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7815e5f0 con 0x7eff78143490 2023-10-06T03:11:17.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.855+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7815dfa0 con 0x7eff78143490 2023-10-06T03:11:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.855+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff78064550 con 0x7eff78143490 2023-10-06T03:11:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.856+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7eff700076b0 con 0x7eff78143490 2023-10-06T03:11:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.856+0000 7eff6dffb700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff5c067320 0x7eff5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.856+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5206+0+0 (secure 0 0 0) 0x7eff70084e20 con 0x7eff78143490 2023-10-06T03:11:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.856+0000 7eff7de43700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff5c067320 0x7eff5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:17.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.856+0000 7eff7de43700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff5c067320 0x7eff5c0697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7eff74009860 tx=0x7eff74017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:17.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:17.860+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff70051d20 con 0x7eff78143490 2023-10-06T03:11:18.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:18.045+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff7804c3c0 con 0x7eff78143490 2023-10-06T03:11:19.021 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.020+0000 7eff6dffb700 1 -- 172.21.15.92:0/423943170 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v198) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff70055340 con 0x7eff78143490 2023-10-06T03:11:19.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.025+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff5c067320 msgr2=0x7eff5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:19.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.025+0000 7eff800a7700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff5c067320 0x7eff5c0697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7eff74009860 tx=0x7eff74017000 comp rx=0 tx=0).stop 2023-10-06T03:11:19.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.025+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 msgr2=0x7eff7813bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:19.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.025+0000 7eff800a7700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 0x7eff7813bee0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7eff7000cd20 tx=0x7eff70001ac0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.026+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 shutdown_connections 2023-10-06T03:11:19.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.026+0000 7eff800a7700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7eff5c067320 0x7eff5c0697d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.026+0000 7eff800a7700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7eff78143490 0x7eff7813bee0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.026+0000 7eff800a7700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff78069bb0 0x7eff78158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.026+0000 7eff800a7700 1 --2- 172.21.15.92:0/423943170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff781430c0 0x7eff7813b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.026+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 >> 172.21.15.92:0/423943170 conn(0x7eff78068e60 msgr2=0x7eff780b60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:19.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.027+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 shutdown_connections 2023-10-06T03:11:19.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.027+0000 7eff800a7700 1 -- 172.21.15.92:0/423943170 wait complete. 2023-10-06T03:11:19.029 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' created 2023-10-06T03:11:19.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:11:19.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.162+0000 7f28c33aa700 1 Processor -- start 2023-10-06T03:11:19.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 -- start start 2023-10-06T03:11:19.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28bc06bcd0 0x7f28bc0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc158d40 0x7f28bc140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f28bc06c0a0 0x7f28bc1412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc15b790 con 0x7f28bc06bcd0 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc15ba70 con 0x7f28bc06c0a0 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c33aa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc15bd50 con 0x7f28bc158d40 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.163+0000 7f28c1146700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28bc06bcd0 0x7f28bc0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f28bc06bcd0 0x7f28bc0c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43728/0 (socket says 172.21.15.92:43728) 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 -- 172.21.15.92:0/336937812 learned_addr learned my addr 172.21.15.92:0/336937812 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c0945700 1 --2- 172.21.15.92:0/336937812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc158d40 0x7f28bc140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1947700 1 --2- 172.21.15.92:0/336937812 >> v2:172.21.15.136:3300/0 conn(0x7f28bc06c0a0 0x7f28bc1412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 -- 172.21.15.92:0/336937812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc158d40 msgr2=0x7f28bc140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 --2- 172.21.15.92:0/336937812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc158d40 0x7f28bc140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 -- 172.21.15.92:0/336937812 >> v2:172.21.15.136:3300/0 conn(0x7f28bc06c0a0 msgr2=0x7f28bc1412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 --2- 172.21.15.92:0/336937812 >> v2:172.21.15.136:3300/0 conn(0x7f28bc06c0a0 0x7f28bc1412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 -- 172.21.15.92:0/336937812 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc15c030 con 0x7f28bc06bcd0 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28c1146700 1 --2- 172.21.15.92:0/336937812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc06bcd0 0x7f28bc0c9230 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b10d0 tx=0x7f28b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2731a36abaf967c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28b3fff700 1 -- 172.21.15.92:0/336937812 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800b860 con 0x7f28bc06bcd0 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28b3fff700 1 -- 172.21.15.92:0/336937812 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b800b9c0 con 0x7f28bc06bcd0 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.164+0000 7f28b3fff700 1 -- 172.21.15.92:0/336937812 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800bd60 con 0x7f28bc06bcd0 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 -- 172.21.15.92:0/336937812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc06bcd0 msgr2=0x7f28bc0c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 --2- 172.21.15.92:0/336937812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc06bcd0 0x7f28bc0c9230 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b10d0 tx=0x7f28b8009250 comp rx=0 tx=0).stop 2023-10-06T03:11:19.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 -- 172.21.15.92:0/336937812 shutdown_connections 2023-10-06T03:11:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 --2- 172.21.15.92:0/336937812 >> v2:172.21.15.136:3300/0 conn(0x7f28bc06c0a0 0x7f28bc1412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 --2- 172.21.15.92:0/336937812 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc06bcd0 0x7f28bc0c9230 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 --2- 172.21.15.92:0/336937812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc158d40 0x7f28bc140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 -- 172.21.15.92:0/336937812 >> 172.21.15.92:0/336937812 conn(0x7f28bc0ce5c0 msgr2=0x7f28bc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 -- 172.21.15.92:0/336937812 shutdown_connections 2023-10-06T03:11:19.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.165+0000 7f28c33aa700 1 -- 172.21.15.92:0/336937812 wait complete. 2023-10-06T03:11:19.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.166+0000 7f28c33aa700 1 Processor -- start 2023-10-06T03:11:19.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.166+0000 7f28c33aa700 1 -- start start 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c33aa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc06bcd0 0x7f28bc155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c33aa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc06c0a0 0x7f28bc0cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c33aa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 0x7f28bc0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c33aa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0b30c0 con 0x7f28bc158d40 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c33aa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0b2440 con 0x7f28bc06c0a0 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c33aa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0b2720 con 0x7f28bc06bcd0 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 0x7f28bc0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 0x7f28bc0cd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:43730/0 (socket says 172.21.15.92:43730) 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 -- 172.21.15.92:0/371687222 learned_addr learned my addr 172.21.15.92:0/371687222 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:19.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1146700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc06bcd0 0x7f28bc155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c0945700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc06c0a0 0x7f28bc0cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 -- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc06bcd0 msgr2=0x7f28bc155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc06bcd0 0x7f28bc155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 -- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc06c0a0 msgr2=0x7f28bc0cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc06c0a0 0x7f28bc0cd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 -- 172.21.15.92:0/371687222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80094f0 con 0x7f28bc158d40 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1146700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc06bcd0 0x7f28bc155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28c1947700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 0x7f28bc0cd950 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009fa0 tx=0x7f28b4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.167+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400ed20 con 0x7f28bc158d40 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.168+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400ee80 con 0x7f28bc158d40 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.168+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4014670 con 0x7f28bc158d40 2023-10-06T03:11:19.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.168+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc0b2a00 con 0x7f28bc158d40 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.168+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc0b3730 con 0x7f28bc158d40 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.168+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc155ac0 con 0x7f28bc158d40 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.169+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f28b40149e0 con 0x7f28bc158d40 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.169+0000 7f28b17fa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28a0067320 0x7f28a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.169+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f28b40853e0 con 0x7f28bc158d40 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.169+0000 7f28c1146700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28a0067320 0x7f28a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:19.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.170+0000 7f28c1146700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28a0067320 0x7f28a00697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b10d0 tx=0x7f28b8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:19.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.174+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28b40520f0 con 0x7f28bc158d40 2023-10-06T03:11:19.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:19.359+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f28bc0718b0 con 0x7f28bc158d40 2023-10-06T03:11:20.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.026+0000 7f28b17fa700 1 -- 172.21.15.92:0/371687222 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v199) v1 ==== 170+0+0 (secure 0 0 0) 0x7f28b4055710 con 0x7f28bc158d40 2023-10-06T03:11:20.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.030+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28a0067320 msgr2=0x7f28a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:20.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.030+0000 7f28c33aa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28a0067320 0x7f28a00697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b10d0 tx=0x7f28b8017000 comp rx=0 tx=0).stop 2023-10-06T03:11:20.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 msgr2=0x7f28bc0cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:20.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 0x7f28bc0cd950 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009fa0 tx=0x7f28b4009900 comp rx=0 tx=0).stop 2023-10-06T03:11:20.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 shutdown_connections 2023-10-06T03:11:20.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f28a0067320 0x7f28a00697d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28bc158d40 0x7f28bc0cd950 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc06bcd0 0x7f28bc155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.031+0000 7f28c33aa700 1 --2- 172.21.15.92:0/371687222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc06c0a0 0x7f28bc0cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.032+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 >> 172.21.15.92:0/371687222 conn(0x7f28bc0ce5c0 msgr2=0x7f28bc1410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:20.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.033+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 shutdown_connections 2023-10-06T03:11:20.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.034+0000 7f28c33aa700 1 -- 172.21.15.92:0/371687222 wait complete. 2023-10-06T03:11:20.038 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' created 2023-10-06T03:11:20.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:11:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.171+0000 7fb8725de700 1 Processor -- start 2023-10-06T03:11:20.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.171+0000 7fb8725de700 1 -- start start 2023-10-06T03:11:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb8725de700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb86c0b16d0 0x7fb86c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb8725de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b33b0 0x7fb86c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb8725de700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c0b2080 con 0x7fb86c0b4e70 2023-10-06T03:11:20.175 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb8725de700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86c0cfb00 con 0x7fb86c0b16d0 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb8725de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0b5c40 con 0x7fb86c0b33b0 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb870ddb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb86c0b16d0 0x7fb86c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb870ddb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb86c0b16d0 0x7fb86c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59592/0 (socket says 172.21.15.92:59592) 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb870ddb700 1 -- 172.21.15.92:0/1682292090 learned_addr learned my addr 172.21.15.92:0/1682292090 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb871ddd700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b33b0 0x7fb86c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb870ddb700 1 -- 172.21.15.92:0/1682292090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b33b0 msgr2=0x7fb86c0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.172+0000 7fb870ddb700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b33b0 0x7fb86c0b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb870ddb700 1 -- 172.21.15.92:0/1682292090 >> v1:172.21.15.136:6789/0 conn(0x7fb86c0b4e70 legacy=0x7fb86c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb870ddb700 1 -- 172.21.15.92:0/1682292090 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c0b5f20 con 0x7fb86c0b16d0 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb871ddd700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b33b0 0x7fb86c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:20.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb870ddb700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b16d0 0x7fb86c0b1b40 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0099b0 tx=0x7fb85c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe4c459e09d9df0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb863fff700 1 -- 172.21.15.92:0/1682292090 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00b860 con 0x7fb86c0b16d0 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb863fff700 1 -- 172.21.15.92:0/1682292090 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c00b9c0 con 0x7fb86c0b16d0 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb863fff700 1 -- 172.21.15.92:0/1682292090 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00bd80 con 0x7fb86c0b16d0 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb8725de700 1 -- 172.21.15.92:0/1682292090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b16d0 msgr2=0x7fb86c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb8725de700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b16d0 0x7fb86c0b1b40 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0099b0 tx=0x7fb85c009250 comp rx=0 tx=0).stop 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb8725de700 1 -- 172.21.15.92:0/1682292090 shutdown_connections 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb8725de700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b33b0 0x7fb86c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb8725de700 1 --2- 172.21.15.92:0/1682292090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b16d0 0x7fb86c0b1b40 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.173+0000 7fb8725de700 1 -- 172.21.15.92:0/1682292090 >> 172.21.15.92:0/1682292090 conn(0x7fb86c0d10c0 msgr2=0x7fb86c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.174+0000 7fb8725de700 1 -- 172.21.15.92:0/1682292090 shutdown_connections 2023-10-06T03:11:20.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.174+0000 7fb8725de700 1 -- 172.21.15.92:0/1682292090 wait complete. 2023-10-06T03:11:20.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.175+0000 7fb8725de700 1 Processor -- start 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.175+0000 7fb8725de700 1 -- start start 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8725de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb86c0b16d0 0x7fb86c16ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8725de700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 0x7fb86c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8725de700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b4e70 0x7fb86c0a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8725de700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0afa60 con 0x7fb86c0b33b0 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8725de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c8fb0 con 0x7fb86c0b16d0 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8725de700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c83a0 con 0x7fb86c0b4e70 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 0x7fb86c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:20.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 0x7fb86c0a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59596/0 (socket says 172.21.15.92:59596) 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 -- 172.21.15.92:0/2961266760 learned_addr learned my addr 172.21.15.92:0/2961266760 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb871ddd700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b4e70 0x7fb86c0a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8715dc700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb86c0b16d0 0x7fb86c16ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 -- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b4e70 msgr2=0x7fb86c0a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b4e70 0x7fb86c0a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 -- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb86c0b16d0 msgr2=0x7fb86c16ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb86c0b16d0 0x7fb86c16ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 -- 172.21.15.92:0/2961266760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0094f0 con 0x7fb86c0b33b0 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb870ddb700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 0x7fb86c0a6f00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb86c070dd0 tx=0x7fb85c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.176+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c006b60 con 0x7fb86c0b33b0 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.177+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c011d70 con 0x7fb86c0b33b0 2023-10-06T03:11:20.183 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.177+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c0c8680 con 0x7fb86c0b33b0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.177+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0266c0 con 0x7fb86c0b33b0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.177+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c0c8c40 con 0x7fb86c0b33b0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.177+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c0ca4b0 con 0x7fb86c0b33b0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.178+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb85c01a420 con 0x7fb86c0b33b0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.178+0000 7fb8617fa700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb850067320 0x7fb8500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.178+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fb85c0866c0 con 0x7fb86c0b33b0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.179+0000 7fb8715dc700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb850067320 0x7fb8500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.179+0000 7fb8715dc700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb850067320 0x7fb8500697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb868003a80 tx=0x7fb868007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:20.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.183+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb86c0ca4b0 con 0x7fb86c0b33b0 2023-10-06T03:11:20.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:20.407+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb86c0ca4b0 con 0x7fb86c0b33b0 2023-10-06T03:11:21.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.028+0000 7fb8617fa700 1 -- 172.21.15.92:0/2961266760 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 34 and data pool 35 v132) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb86c0ca4b0 con 0x7fb86c0b33b0 2023-10-06T03:11:21.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.033+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb850067320 msgr2=0x7fb8500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.033+0000 7fb8725de700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb850067320 0x7fb8500697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb868003a80 tx=0x7fb868007ec0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.033+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 msgr2=0x7fb86c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.033+0000 7fb8725de700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 0x7fb86c0a6f00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb86c070dd0 tx=0x7fb85c017000 comp rx=0 tx=0).stop 2023-10-06T03:11:21.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.039+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 shutdown_connections 2023-10-06T03:11:21.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.039+0000 7fb8725de700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb850067320 0x7fb8500697d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.039+0000 7fb8725de700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb86c0b4e70 0x7fb86c0a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.039+0000 7fb8725de700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb86c0b16d0 0x7fb86c16ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.039+0000 7fb8725de700 1 --2- 172.21.15.92:0/2961266760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb86c0b33b0 0x7fb86c0a6f00 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.039+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 >> 172.21.15.92:0/2961266760 conn(0x7fb86c0d10c0 msgr2=0x7fb86c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:21.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.050+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 shutdown_connections 2023-10-06T03:11:21.051 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.050+0000 7fb8725de700 1 -- 172.21.15.92:0/2961266760 wait complete. 2023-10-06T03:11:21.052 INFO:teuthology.orchestra.run.smithi092.stderr:new fs with metadata pool 34 and data pool 35 2023-10-06T03:11:21.066 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:21.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.180+0000 7f1763b64700 1 Processor -- start 2023-10-06T03:11:21.182 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.180+0000 7f1763b64700 1 -- start start 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.181+0000 7f1763b64700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f175c0caa90 0x7f175c0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.181+0000 7f1763b64700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c15d7e0 con 0x7f175c06c0a0 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.181+0000 7f1763b64700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c15dac0 con 0x7f175c06bcd0 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.181+0000 7f1763b64700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175c0caef0 con 0x7f175c0caa90 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.181+0000 7f17610ff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f175c06c0a0 0x7f175c15df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50432/0 (socket says 172.21.15.92:50432) 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.181+0000 7f17610ff700 1 -- 172.21.15.92:0/316409190 learned_addr learned my addr 172.21.15.92:0/316409190 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 --2- 172.21.15.92:0/316409190 >> v2:172.21.15.136:3300/0 conn(0x7f175c0caa90 0x7f175c0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181568369 0 0) 0x7f175c15d7e0 con 0x7f175c06c0a0 2023-10-06T03:11:21.186 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1748003570 con 0x7f175c06c0a0 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4170107844 0 0) 0x7f1748003570 con 0x7f175c06c0a0 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 >> v2:172.21.15.136:3300/0 conn(0x7f175c0caa90 msgr2=0x7f175c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 --2- 172.21.15.92:0/316409190 >> v2:172.21.15.136:3300/0 conn(0x7f175c0caa90 0x7f175c0c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 >> v1:172.21.15.136:6789/0 conn(0x7f175c06bcd0 legacy=0x7f175c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c15aa30 con 0x7f175c06c0a0 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f174c001470 con 0x7f175c06c0a0 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 >> v1:172.21.15.92:6789/0 conn(0x7f175c06c0a0 legacy=0x7f175c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1748003a60 0x7f1748003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748004810 0x7f1748004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17480052e0 0x7f174800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0caef0 con 0x7f17480052e0 2023-10-06T03:11:21.187 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174800c2a0 con 0x7f1748004810 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174800c460 con 0x7f1748003a60 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17480052e0 0x7f174800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f17610ff700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748004810 0x7f1748004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1761900700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1748003a60 0x7f1748003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 -- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1748003a60 msgr2=0x7f1748003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1748003a60 0x7f1748003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 -- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748004810 msgr2=0x7f1748004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1748004810 0x7f1748004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.182+0000 7f1762101700 1 -- 172.21.15.92:0/316409190 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748003570 con 0x7f17480052e0 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1762101700 1 -- 172.21.15.92:0/316409190 reap_dead start 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1762101700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17480052e0 0x7f174800ba60 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f1754009c00 tx=0x7f17540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e32bfc9b2e3ee0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.188 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400e710 con 0x7f17480052e0 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175400e870 con 0x7f17480052e0 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f17608fe700 1 -- 172.21.15.92:0/316409190 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400ec30 con 0x7f17480052e0 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 -- 172.21.15.92:0/316409190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17480052e0 msgr2=0x7f174800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17480052e0 0x7f174800ba60 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f1754009c00 tx=0x7f17540091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 -- 172.21.15.92:0/316409190 shutdown_connections 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 --2- 172.21.15.92:0/316409190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17480052e0 0x7f174800ba60 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 -- 172.21.15.92:0/316409190 >> 172.21.15.92:0/316409190 conn(0x7f175c0ce600 msgr2=0x7f175c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 -- 172.21.15.92:0/316409190 shutdown_connections 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.183+0000 7f1763b64700 1 -- 172.21.15.92:0/316409190 wait complete. 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.184+0000 7f1763b64700 1 Processor -- start 2023-10-06T03:11:21.189 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.184+0000 7f1763b64700 1 -- start start 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1763b64700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17480052e0 0x7f175c16e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1763b64700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 0x7f175c0c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1763b64700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f175c0b39d0 0x7f175c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1763b64700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c16e620 con 0x7f175c0c7fd0 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1763b64700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0b4380 con 0x7f175c0b39d0 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1763b64700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0c95e0 con 0x7f17480052e0 2023-10-06T03:11:21.192 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 0x7f175c0c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 0x7f175c0c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59614/0 (socket says 172.21.15.92:59614) 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 -- 172.21.15.92:0/2956966819 learned_addr learned my addr 172.21.15.92:0/2956966819 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1761900700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17480052e0 0x7f175c16e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1762101700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f175c0b39d0 0x7f175c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 -- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17480052e0 msgr2=0x7f175c16e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17480052e0 0x7f175c16e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 -- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f175c0b39d0 msgr2=0x7f175c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f175c0b39d0 0x7f175c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f17610ff700 1 -- 172.21.15.92:0/2956966819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0c8540 con 0x7f175c0c7fd0 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1761900700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17480052e0 0x7f175c16e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.185+0000 7f1762101700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f175c0b39d0 0x7f175c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:21.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.186+0000 7f17610ff700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 0x7f175c0c6140 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f174c0073c0 tx=0x7f175c15da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.186+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c018030 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.186+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c007830 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.186+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c0c8820 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.186+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c00fb00 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.186+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c0c8de0 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.187+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c04c3c0 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.187+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f174c01b070 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.188+0000 7f1751ffb700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1740067320 0x7f17400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.188+0000 7f1761900700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1740067320 0x7f17400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.188+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f174c019070 con 0x7f175c0c7fd0 2023-10-06T03:11:21.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.188+0000 7f1761900700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1740067320 0x7f17400697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1758004290 tx=0x7f1758001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.192+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f174c051840 con 0x7f175c0c7fd0 2023-10-06T03:11:21.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.379+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f175c15ad10 con 0x7f175c0c7fd0 2023-10-06T03:11:21.381 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.380+0000 7f1751ffb700 1 -- 172.21.15.92:0/2956966819 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f174c054e60 con 0x7f175c0c7fd0 2023-10-06T03:11:21.382 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:21.382 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":200,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:20.409259+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:11:18.064170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-06T03:11:19.378751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1740067320 msgr2=0x7f17400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1740067320 0x7f17400697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1758004290 tx=0x7f1758001640 comp rx=0 tx=0).stop 2023-10-06T03:11:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 msgr2=0x7f175c0c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 0x7f175c0c6140 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f174c0073c0 tx=0x7f175c15da30 comp rx=0 tx=0).stop 2023-10-06T03:11:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 shutdown_connections 2023-10-06T03:11:21.384 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f1740067320 0x7f17400697d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f175c0b39d0 0x7f175c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17480052e0 0x7f175c16e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 --2- 172.21.15.92:0/2956966819 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175c0c7fd0 0x7f175c0c6140 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 >> 172.21.15.92:0/2956966819 conn(0x7f175c0ce600 msgr2=0x7f175c15b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 shutdown_connections 2023-10-06T03:11:21.385 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.382+0000 7f1763b64700 1 -- 172.21.15.92:0/2956966819 wait complete. 2023-10-06T03:11:21.396 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:21.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.509+0000 7f1635885700 1 Processor -- start 2023-10-06T03:11:21.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.509+0000 7f1635885700 1 -- start start 2023-10-06T03:11:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1635885700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 0x7f16300b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1635885700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16300b11d0 con 0x7f1630159420 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1635885700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16300b14b0 con 0x7f163015d230 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1635885700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b1790 con 0x7f16301597f0 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1634883700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1630159420 0x7f16300b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50446/0 (socket says 172.21.15.92:50446) 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1634883700 1 -- 172.21.15.92:0/24414775 learned_addr learned my addr 172.21.15.92:0/24414775 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f1627fff700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 0x7f16300b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042412930 0 0) 0x7f16300b11d0 con 0x7f1630159420 2023-10-06T03:11:21.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.510+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1618003570 con 0x7f1630159420 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3375282071 0 0) 0x7f1618003570 con 0x7f1630159420 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 msgr2=0x7f16300b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 0x7f16300b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 >> v1:172.21.15.136:6789/0 conn(0x7f163015d230 legacy=0x7f16300b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16300b1a70 con 0x7f1630159420 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f162c001470 con 0x7f1630159420 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 >> v1:172.21.15.92:6789/0 conn(0x7f1630159420 legacy=0x7f16300b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618003a90 0x7f1618003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1618004810 0x7f1618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 0x7f16180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b1790 con 0x7f1618004810 2023-10-06T03:11:21.516 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161800bb10 con 0x7f1618003a90 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161800bcd0 con 0x7f16301597f0 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1618004810 0x7f1618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1634883700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618003a90 0x7f1618003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1635084700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 0x7f16180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 -- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 msgr2=0x7f16180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16301597f0 0x7f16180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 -- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618003a90 msgr2=0x7f1618003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.517 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618003a90 0x7f1618003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 -- 172.21.15.92:0/24414775 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618003570 con 0x7f1618004810 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f1627fff700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1618004810 0x7f1618004c80 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f1620009c00 tx=0x7f16200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3e7c4c59ae84fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000e710 con 0x7f1618004810 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162000e870 con 0x7f1618004810 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.511+0000 7f16277fe700 1 -- 172.21.15.92:0/24414775 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162000ec30 con 0x7f1618004810 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635885700 1 -- 172.21.15.92:0/24414775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1618004810 msgr2=0x7f1618004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635885700 1 --2- 172.21.15.92:0/24414775 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1618004810 0x7f1618004c80 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f1620009c00 tx=0x7f16200091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635084700 1 -- 172.21.15.92:0/24414775 reap_dead start 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635885700 1 -- 172.21.15.92:0/24414775 shutdown_connections 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635885700 1 -- 172.21.15.92:0/24414775 >> 172.21.15.92:0/24414775 conn(0x7f16300d0660 msgr2=0x7f16300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:21.518 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635885700 1 -- 172.21.15.92:0/24414775 shutdown_connections 2023-10-06T03:11:21.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.512+0000 7f1635885700 1 -- 172.21.15.92:0/24414775 wait complete. 2023-10-06T03:11:21.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.513+0000 7f1635885700 1 Processor -- start 2023-10-06T03:11:21.519 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.513+0000 7f1635885700 1 -- start start 2023-10-06T03:11:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635885700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618004810 0x7f1630157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635885700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 0x7f163014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635885700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16300c5dc0 0x7f16300c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.521 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635885700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630159bc0 con 0x7f16300c9790 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635885700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163015a180 con 0x7f1618004810 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635885700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300c6660 con 0x7f16300c5dc0 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 0x7f163014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 0x7f163014a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59626/0 (socket says 172.21.15.92:59626) 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 -- 172.21.15.92:0/326777501 learned_addr learned my addr 172.21.15.92:0/326777501 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1635084700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16300c5dc0 0x7f16300c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1634883700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618004810 0x7f1630157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 -- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16300c5dc0 msgr2=0x7f16300c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16300c5dc0 0x7f16300c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 -- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618004810 msgr2=0x7f1630157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618004810 0x7f1630157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.522 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 -- 172.21.15.92:0/326777501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16300c6940 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1634883700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618004810 0x7f1630157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.514+0000 7f1627fff700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 0x7f163014a0b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1618004770 tx=0x7f162001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.515+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620022070 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.515+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162000ebf0 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.515+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162001c6a0 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.515+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1630159e10 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.515+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16300c6230 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.516+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16300c6420 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.516+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f162001ca90 con 0x7f16300c9790 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.516+0000 7f1624ff9700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f161c067320 0x7f161c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.523 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.516+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1620086650 con 0x7f16300c9790 2023-10-06T03:11:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.516+0000 7f1634883700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f161c067320 0x7f161c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.517+0000 7f1634883700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f161c067320 0x7f161c0697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f162c0015a0 tx=0x7f162c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.521+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16300c6420 con 0x7f16300c9790 2023-10-06T03:11:21.705 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.704+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16300c6420 con 0x7f16300c9790 2023-10-06T03:11:21.706 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.705+0000 7f1624ff9700 1 -- 172.21.15.92:0/326777501 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16893 (secure 0 0 0) 0x7f16300c6420 con 0x7f16300c9790 2023-10-06T03:11:21.706 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:21.707 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":200,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:20.409259+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:11:18.064170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-06T03:11:19.378751+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:21.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f161c067320 msgr2=0x7f161c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.708 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f161c067320 0x7f161c0697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f162c0015a0 tx=0x7f162c009040 comp rx=0 tx=0).stop 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 msgr2=0x7f163014a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 0x7f163014a0b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1618004770 tx=0x7f162001a000 comp rx=0 tx=0).stop 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 shutdown_connections 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f161c067320 0x7f161c0697d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16300c5dc0 0x7f16300c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618004810 0x7f1630157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 --2- 172.21.15.92:0/326777501 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16300c9790 0x7f163014a0b0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 >> 172.21.15.92:0/326777501 conn(0x7f16300d0660 msgr2=0x7f16300b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 shutdown_connections 2023-10-06T03:11:21.709 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.707+0000 7f1635885700 1 -- 172.21.15.92:0/326777501 wait complete. 2023-10-06T03:11:21.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:11:21.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.836+0000 7f32c7f5c700 1 Processor -- start 2023-10-06T03:11:21.838 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.836+0000 7f32c7f5c700 1 -- start start 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c7f5c700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f32c0159890 0x7f32c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c7f5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 0x7f32c00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c7f5c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c0069ee0 con 0x7f32c0159c60 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c7f5c700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c015a160 con 0x7f32c0159890 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c7f5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00d13a0 con 0x7f32c0069560 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c54f7700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f32c0159c60 0x7f32c015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50458/0 (socket says 172.21.15.92:50458) 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c54f7700 1 -- 172.21.15.92:0/3792958325 learned_addr learned my addr 172.21.15.92:0/3792958325 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:21.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c64f9700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 0x7f32c00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c5cf8700 1 --2- 172.21.15.92:0/3792958325 >> v2:172.21.15.136:3300/0 conn(0x7f32c0159890 0x7f32c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503857231 0 0) 0x7f32c0069ee0 con 0x7f32c0159c60 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.837+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32ac003570 con 0x7f32c0159c60 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723607273 0 0) 0x7f32ac003570 con 0x7f32c0159c60 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 msgr2=0x7f32c00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 0x7f32c00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 >> v2:172.21.15.136:3300/0 conn(0x7f32c0159890 msgr2=0x7f32c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 --2- 172.21.15.92:0/3792958325 >> v2:172.21.15.136:3300/0 conn(0x7f32c0159890 0x7f32c015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c015e140 con 0x7f32c0159c60 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f32b0001470 con 0x7f32c0159c60 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 >> v1:172.21.15.92:6789/0 conn(0x7f32c0159c60 legacy=0x7f32c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32ac003a60 0x7f32ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32ac004810 0x7f32ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 0x7f32ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c015a160 con 0x7f32ac004810 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c00d13a0 con 0x7f32ac003a60 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac00ba60 con 0x7f32c0069560 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32ac004810 0x7f32ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c5cf8700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32ac003a60 0x7f32ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c64f9700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 0x7f32ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 -- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 msgr2=0x7f32ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32c0069560 0x7f32ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 -- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32ac003a60 msgr2=0x7f32ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:21.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32ac003a60 0x7f32ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 -- 172.21.15.92:0/3792958325 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac003570 con 0x7f32ac004810 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c54f7700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32ac004810 0x7f32ac004c80 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c3f0 tx=0x7f32b0001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a106542bc8e9496f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000cc80 con 0x7f32ac004810 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b000ce00 con 0x7f32ac004810 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.838+0000 7f32c4cf6700 1 -- 172.21.15.92:0/3792958325 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0005b70 con 0x7f32ac004810 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c7f5c700 1 -- 172.21.15.92:0/3792958325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32ac004810 msgr2=0x7f32ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/3792958325 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32ac004810 0x7f32ac004c80 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c3f0 tx=0x7f32b0001e90 comp rx=0 tx=0).stop 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c64f9700 1 -- 172.21.15.92:0/3792958325 reap_dead start 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c7f5c700 1 -- 172.21.15.92:0/3792958325 shutdown_connections 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c7f5c700 1 -- 172.21.15.92:0/3792958325 >> 172.21.15.92:0/3792958325 conn(0x7f32c0068e20 msgr2=0x7f32c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c7f5c700 1 -- 172.21.15.92:0/3792958325 shutdown_connections 2023-10-06T03:11:21.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.839+0000 7f32c7f5c700 1 -- 172.21.15.92:0/3792958325 wait complete. 2023-10-06T03:11:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.840+0000 7f32c7f5c700 1 Processor -- start 2023-10-06T03:11:21.846 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.840+0000 7f32c7f5c700 1 -- start start 2023-10-06T03:11:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c7f5c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32ac004810 0x7f32c016aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c7f5c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 0x7f32c00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c7f5c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32c016d5e0 0x7f32c00d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c7f5c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c016e4a0 con 0x7f32c016a0e0 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c7f5c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c016dee0 con 0x7f32c016d5e0 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c7f5c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c016e1c0 con 0x7f32ac004810 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 0x7f32c00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 0x7f32c00cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59638/0 (socket says 172.21.15.92:59638) 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 -- 172.21.15.92:0/4128107836 learned_addr learned my addr 172.21.15.92:0/4128107836 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c5cf8700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32ac004810 0x7f32c016aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c64f9700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32c016d5e0 0x7f32c00d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 -- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32ac004810 msgr2=0x7f32c016aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32ac004810 0x7f32c016aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 -- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32c016d5e0 msgr2=0x7f32c00d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32c016d5e0 0x7f32c00d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 -- 172.21.15.92:0/4128107836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00c93d0 con 0x7f32c016a0e0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c5cf8700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32ac004810 0x7f32c016aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c64f9700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32c016d5e0 0x7f32c00d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.841+0000 7f32c54f7700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 0x7f32c00cd0c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f32b0007d10 tx=0x7f32b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.842+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0001430 con 0x7f32c016a0e0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.842+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00072e0 con 0x7f32c016a0e0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.842+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c00c96b0 con 0x7f32c016a0e0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.842+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b00076e0 con 0x7f32c016a0e0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.842+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c016da80 con 0x7f32c016a0e0 2023-10-06T03:11:21.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.843+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c004c3c0 con 0x7f32c016a0e0 2023-10-06T03:11:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.843+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f32b0019030 con 0x7f32c016a0e0 2023-10-06T03:11:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.843+0000 7f32b5ffb700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32a4067320 0x7f32a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.844+0000 7f32c5cf8700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32a4067320 0x7f32a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.844+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32b0085760 con 0x7f32c016a0e0 2023-10-06T03:11:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.844+0000 7f32c5cf8700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32a4067320 0x7f32a40697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0099f0 tx=0x7f32bc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:21.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:21.848+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32b0052390 con 0x7f32c016a0e0 2023-10-06T03:11:22.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:22.077+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32c00b6410 con 0x7f32c016a0e0 2023-10-06T03:11:23.054 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.052+0000 7f32b5ffb700 1 -- 172.21.15.92:0/4128107836 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v135) v1 ==== 118+0+0 (secure 0 0 0) 0x7f32b00559b0 con 0x7f32c016a0e0 2023-10-06T03:11:23.056 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32a4067320 msgr2=0x7f32a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32a4067320 0x7f32a40697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0099f0 tx=0x7f32bc00b040 comp rx=0 tx=0).stop 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 msgr2=0x7f32c00cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 0x7f32c00cd0c0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f32b0007d10 tx=0x7f32b0015000 comp rx=0 tx=0).stop 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 shutdown_connections 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f32a4067320 0x7f32a40697d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32c016d5e0 0x7f32c00d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32ac004810 0x7f32c016aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 --2- 172.21.15.92:0/4128107836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32c016a0e0 0x7f32c00cd0c0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 >> 172.21.15.92:0/4128107836 conn(0x7f32c0068e20 msgr2=0x7f32c00b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:23.057 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 shutdown_connections 2023-10-06T03:11:23.058 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.055+0000 7f32c7f5c700 1 -- 172.21.15.92:0/4128107836 wait complete. 2023-10-06T03:11:23.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:11:23.193 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.192+0000 7f07a8dc9700 1 Processor -- start 2023-10-06T03:11:23.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.192+0000 7f07a8dc9700 1 -- start start 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a8dc9700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a8dc9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a8dc9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a4069ee0 con 0x7f07a4159c60 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a8dc9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a415a160 con 0x7f07a4069560 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a8dc9700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a40d13a0 con 0x7f07a4159890 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59644/0 (socket says 172.21.15.92:59644) 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 -- 172.21.15.92:0/3604337793 learned_addr learned my addr 172.21.15.92:0/3604337793 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f079bfff700 1 --1- 172.21.15.92:0/3604337793 >> v1:172.21.15.92:6789/0 conn(0x7f07a4159c60 0x7f07a415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50472/0 (socket says 172.21.15.92:50472) 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a259c700 1 --2- 172.21.15.92:0/3604337793 >> v2:172.21.15.136:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:23.196 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 -- 172.21.15.92:0/3604337793 >> v2:172.21.15.136:3300/0 conn(0x7f07a4159890 msgr2=0x7f07a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 --2- 172.21.15.92:0/3604337793 >> v2:172.21.15.136:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 -- 172.21.15.92:0/3604337793 >> v1:172.21.15.92:6789/0 conn(0x7f07a4159c60 legacy=0x7f07a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 -- 172.21.15.92:0/3604337793 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a415e140 con 0x7f07a4069560 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a1d9b700 1 -- 172.21.15.92:0/3604337793 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569739972 0 0) 0x7f07a4069ee0 con 0x7f07a4159c60 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a259c700 1 --2- 172.21.15.92:0/3604337793 >> v2:172.21.15.136:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a2d9d700 1 --2- 172.21.15.92:0/3604337793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40699a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f07a40d1090 tx=0x7f079c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bfabd9de599992 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a1d9b700 1 -- 172.21.15.92:0/3604337793 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00e070 con 0x7f07a4069560 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a1d9b700 1 -- 172.21.15.92:0/3604337793 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c00ba50 con 0x7f07a4069560 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.193+0000 7f07a1d9b700 1 -- 172.21.15.92:0/3604337793 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c0092e0 con 0x7f07a4069560 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3604337793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 msgr2=0x7f07a40699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3604337793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40699a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f07a40d1090 tx=0x7f079c009840 comp rx=0 tx=0).stop 2023-10-06T03:11:23.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3604337793 shutdown_connections 2023-10-06T03:11:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3604337793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40699a0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3604337793 >> v2:172.21.15.136:3300/0 conn(0x7f07a4159890 0x7f07a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3604337793 >> 172.21.15.92:0/3604337793 conn(0x7f07a4068e20 msgr2=0x7f07a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3604337793 shutdown_connections 2023-10-06T03:11:23.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.194+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3604337793 wait complete. 2023-10-06T03:11:23.200 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.195+0000 7f07a8dc9700 1 Processor -- start 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.195+0000 7f07a8dc9700 1 -- start start 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07a4159890 0x7f07a40b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07a4159c60 0x7f07a415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40ce860 con 0x7f07a4069560 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40cd970 con 0x7f07a4159c60 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40cdc50 con 0x7f07a4159890 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59660/0 (socket says 172.21.15.92:59660) 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 -- 172.21.15.92:0/3038146353 learned_addr learned my addr 172.21.15.92:0/3038146353 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:23.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 -- 172.21.15.92:0/3038146353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07a4159890 msgr2=0x7f07a40b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07a4159890 0x7f07a40b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 -- 172.21.15.92:0/3038146353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07a4159c60 msgr2=0x7f07a415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07a4159c60 0x7f07a415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 -- 172.21.15.92:0/3038146353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c009ae0 con 0x7f07a4069560 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a259c700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40cecf0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f079400e8d0 tx=0x7f0794008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079400ee70 con 0x7f07a4069560 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079400c5e0 con 0x7f07a4069560 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079400c980 con 0x7f07a4069560 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.196+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a40cdf30 con 0x7f07a4069560 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.197+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a40ce4f0 con 0x7f07a4069560 2023-10-06T03:11:23.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.197+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a404c3c0 con 0x7f07a4069560 2023-10-06T03:11:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.198+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0794013410 con 0x7f07a4069560 2023-10-06T03:11:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.198+0000 7f079a7fc700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0784067320 0x7f07840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.198+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0794084d60 con 0x7f07a4069560 2023-10-06T03:11:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.198+0000 7f079bfff700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0784067320 0x7f07840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:23.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.199+0000 7f079bfff700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0784067320 0x7f07840697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f078c001660 tx=0x7f078c007b70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:23.204 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.203+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0794055210 con 0x7f07a4069560 2023-10-06T03:11:23.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:23.430+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f07a406a5b0 con 0x7f07a4069560 2023-10-06T03:11:24.055 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.054+0000 7f079a7fc700 1 -- 172.21.15.92:0/3038146353 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v136) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0794054da0 con 0x7f07a4069560 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0784067320 msgr2=0x7f07840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0784067320 0x7f07840697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f078c001660 tx=0x7f078c007b70 comp rx=0 tx=0).stop 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 msgr2=0x7f07a40cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40cecf0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f079400e8d0 tx=0x7f0794008040 comp rx=0 tx=0).stop 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 shutdown_connections 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0784067320 0x7f07840697d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f07a4069560 0x7f07a40cecf0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07a4159c60 0x7f07a415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 --2- 172.21.15.92:0/3038146353 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07a4159890 0x7f07a40b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 >> 172.21.15.92:0/3038146353 conn(0x7f07a4068e20 msgr2=0x7f07a40cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:24.059 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 shutdown_connections 2023-10-06T03:11:24.060 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.057+0000 7f07a8dc9700 1 -- 172.21.15.92:0/3038146353 wait complete. 2023-10-06T03:11:24.073 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:24.194 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.193+0000 7f0cdcd23700 1 Processor -- start 2023-10-06T03:11:24.195 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.193+0000 7f0cdcd23700 1 -- start start 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cdcd23700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0cd8159890 0x7f0cd815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cdcd23700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0cd8159c60 0x7f0cd815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cdcd23700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd815a160 con 0x7f0cd815db10 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cdcd23700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd815df50 con 0x7f0cd8159890 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cdcd23700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd80d13a0 con 0x7f0cd8159c60 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0cd8159890 0x7f0cd815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0cd8159890 0x7f0cd815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59670/0 (socket says 172.21.15.92:59670) 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 -- 172.21.15.92:0/3833438399 learned_addr learned my addr 172.21.15.92:0/3833438399 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:24.197 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 -- 172.21.15.92:0/3833438399 >> v2:172.21.15.136:3300/0 conn(0x7f0cd8159c60 msgr2=0x7f0cd815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 --2- 172.21.15.92:0/3833438399 >> v2:172.21.15.136:3300/0 conn(0x7f0cd8159c60 0x7f0cd815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 -- 172.21.15.92:0/3833438399 >> v1:172.21.15.136:6789/0 conn(0x7f0cd815db10 legacy=0x7f0cd8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 -- 172.21.15.92:0/3833438399 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd815e140 con 0x7f0cd8159890 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd659c700 1 --2- 172.21.15.92:0/3833438399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd8159890 0x7f0cd815abd0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b67f0 tx=0x7f0cd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156f719439562a11 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.194+0000 7f0cd559a700 1 -- 172.21.15.92:0/3833438399 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000e070 con 0x7f0cd8159890 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cd559a700 1 -- 172.21.15.92:0/3833438399 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd000ba10 con 0x7f0cd8159890 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cd559a700 1 -- 172.21.15.92:0/3833438399 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00092e0 con 0x7f0cd8159890 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 -- 172.21.15.92:0/3833438399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd8159890 msgr2=0x7f0cd815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/3833438399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd8159890 0x7f0cd815abd0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80b67f0 tx=0x7f0cd0009840 comp rx=0 tx=0).stop 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 -- 172.21.15.92:0/3833438399 shutdown_connections 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/3833438399 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd8159890 0x7f0cd815abd0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.198 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/3833438399 >> v2:172.21.15.136:3300/0 conn(0x7f0cd8159c60 0x7f0cd815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 -- 172.21.15.92:0/3833438399 >> 172.21.15.92:0/3833438399 conn(0x7f0cd8068e20 msgr2=0x7f0cd80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:24.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 -- 172.21.15.92:0/3833438399 shutdown_connections 2023-10-06T03:11:24.199 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.195+0000 7f0cdcd23700 1 -- 172.21.15.92:0/3833438399 wait complete. 2023-10-06T03:11:24.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.196+0000 7f0cdcd23700 1 Processor -- start 2023-10-06T03:11:24.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.196+0000 7f0cdcd23700 1 -- start start 2023-10-06T03:11:24.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd8159890 0x7f0cd80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0cd8159c60 0x7f0cd816e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.201 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 0x7f0cd80ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80b3d80 con 0x7f0cd815db10 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd816af00 con 0x7f0cd8159890 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd80b3730 con 0x7f0cd8159c60 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 0x7f0cd80ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 0x7f0cd80ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59672/0 (socket says 172.21.15.92:59672) 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 -- 172.21.15.92:0/1125160052 learned_addr learned my addr 172.21.15.92:0/1125160052 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 -- 172.21.15.92:0/1125160052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0cd8159c60 msgr2=0x7f0cd816e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0cd8159c60 0x7f0cd816e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 -- 172.21.15.92:0/1125160052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd8159890 msgr2=0x7f0cd80b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd8159890 0x7f0cd80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.202 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 -- 172.21.15.92:0/1125160052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0009ae0 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cd6d9d700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 0x7f0cd80ce8c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800b4e0 tx=0x7f0cc8009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800f5e0 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc800f760 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd80b3a10 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.197+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800fb20 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.198+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd80cee00 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.198+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd80b3590 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.199+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0cc801a400 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.199+0000 7f0cc67fc700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0cb8067320 0x7f0cb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.199+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0cc8084990 con 0x7f0cd815db10 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.199+0000 7f0cd659c700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0cb8067320 0x7f0cb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.203 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.200+0000 7f0cd659c700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0cb8067320 0x7f0cb80697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80cf6a0 tx=0x7f0cd0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.205 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.203+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cd80b3590 con 0x7f0cd815db10 2023-10-06T03:11:24.432 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.431+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cd80b3590 con 0x7f0cd815db10 2023-10-06T03:11:24.433 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.431+0000 7f0cc67fc700 1 -- 172.21.15.92:0/1125160052 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f0cd80b3590 con 0x7f0cd815db10 2023-10-06T03:11:24.433 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:24.433 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":136,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:11:20.409587+0000","modified":"2023-10-06T03:11:24.052115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3455480640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3455480640},{"type":"v1","addr":"172.21.15.92:6837","nonce":3455480640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T03:11:24.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0cb8067320 msgr2=0x7f0cb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0cb8067320 0x7f0cb80697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80cf6a0 tx=0x7f0cd0017000 comp rx=0 tx=0).stop 2023-10-06T03:11:24.435 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 msgr2=0x7f0cd80ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 0x7f0cd80ce8c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800b4e0 tx=0x7f0cc8009060 comp rx=0 tx=0).stop 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 shutdown_connections 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0cb8067320 0x7f0cb80697d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0cd815db10 0x7f0cd80ce8c0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0cd8159890 0x7f0cd80b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 --2- 172.21.15.92:0/1125160052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0cd8159c60 0x7f0cd816e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.433+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 >> 172.21.15.92:0/1125160052 conn(0x7f0cd8068e20 msgr2=0x7f0cd80b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.434+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 shutdown_connections 2023-10-06T03:11:24.436 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.434+0000 7f0cdcd23700 1 -- 172.21.15.92:0/1125160052 wait complete. 2023-10-06T03:11:24.437 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 136 2023-10-06T03:11:24.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:11:24.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.560+0000 7f09bcfe6700 1 Processor -- start 2023-10-06T03:11:24.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.560+0000 7f09bcfe6700 1 -- start start 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09bcfe6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09b8159420 0x7f09b80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09bcfe6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b81597f0 0x7f09b80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09bcfe6700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b80b11d0 con 0x7f09b815d230 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09bcfe6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b80b14b0 con 0x7f09b8159420 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09bcfe6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80b1790 con 0x7f09b81597f0 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09b8159420 0x7f09b80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.564 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09b8159420 0x7f09b80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59678/0 (socket says 172.21.15.92:59678) 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 -- 172.21.15.92:0/3341864447 learned_addr learned my addr 172.21.15.92:0/3341864447 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b6ffd700 1 --2- 172.21.15.92:0/3341864447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b81597f0 0x7f09b80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 -- 172.21.15.92:0/3341864447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b81597f0 msgr2=0x7f09b80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 --2- 172.21.15.92:0/3341864447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b81597f0 0x7f09b80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 -- 172.21.15.92:0/3341864447 >> v1:172.21.15.136:6789/0 conn(0x7f09b815d230 legacy=0x7f09b80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.561+0000 7f09b77fe700 1 -- 172.21.15.92:0/3341864447 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80b1a70 con 0x7f09b8159420 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09b77fe700 1 --2- 172.21.15.92:0/3341864447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80b5f40 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009f20 tx=0x7f09a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1f34120ead1055 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09b67fc700 1 -- 172.21.15.92:0/3341864447 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800e070 con 0x7f09b8159420 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09b67fc700 1 -- 172.21.15.92:0/3341864447 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800ba40 con 0x7f09b8159420 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09b67fc700 1 -- 172.21.15.92:0/3341864447 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a80092e0 con 0x7f09b8159420 2023-10-06T03:11:24.565 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3341864447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 msgr2=0x7f09b80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3341864447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80b5f40 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009f20 tx=0x7f09a8009840 comp rx=0 tx=0).stop 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3341864447 shutdown_connections 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3341864447 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80b5f40 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3341864447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b81597f0 0x7f09b80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3341864447 >> 172.21.15.92:0/3341864447 conn(0x7f09b80d0660 msgr2=0x7f09b80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3341864447 shutdown_connections 2023-10-06T03:11:24.566 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.562+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3341864447 wait complete. 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 Processor -- start 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 -- start start 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b81597f0 0x7f09b80cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b815d230 0x7f09b816a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80cc3e0 con 0x7f09b8159420 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b815ae70 con 0x7f09b81597f0 2023-10-06T03:11:24.569 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09bcfe6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80cbb40 con 0x7f09b815d230 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09b77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09b77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59692/0 (socket says 172.21.15.92:59692) 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.564+0000 7f09b77fe700 1 -- 172.21.15.92:0/3546587113 learned_addr learned my addr 172.21.15.92:0/3546587113 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b6ffd700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b81597f0 0x7f09b80cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b7fff700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b815d230 0x7f09b816a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b77fe700 1 -- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b815d230 msgr2=0x7f09b816a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b77fe700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b815d230 0x7f09b816a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b77fe700 1 -- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b81597f0 msgr2=0x7f09b80cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b77fe700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b81597f0 0x7f09b80cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.570 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b77fe700 1 -- 172.21.15.92:0/3546587113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009ae0 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09b77fe700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80cadd0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f09b8089340 tx=0x7f09a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8012aa0 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a8012c00 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b80cbe20 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a801d420 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.565+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b80cb850 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.566+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b804c3c0 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.566+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f09a801d810 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.567+0000 7f099f7fe700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09a4067320 0x7f09a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.567+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09a8086610 con 0x7f09b8159420 2023-10-06T03:11:24.571 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.567+0000 7f09b6ffd700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09a4067320 0x7f09a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.567+0000 7f09b6ffd700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09a4067320 0x7f09a40697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f09b80c7f30 tx=0x7f09ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.572 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.571+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09a8053030 con 0x7f09b8159420 2023-10-06T03:11:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.755+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09b80b1d50 con 0x7f09b8159420 2023-10-06T03:11:24.757 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.756+0000 7f099f7fe700 1 -- 172.21.15.92:0/3546587113 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+16880 (secure 0 0 0) 0x7f09a8056650 con 0x7f09b8159420 2023-10-06T03:11:24.758 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:24.758 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":201,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:11:22.026490+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:11:18.064170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-06T03:11:19.378751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:11:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09a4067320 msgr2=0x7f09a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09a4067320 0x7f09a40697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f09b80c7f30 tx=0x7f09ac008040 comp rx=0 tx=0).stop 2023-10-06T03:11:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 msgr2=0x7f09b80cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80cadd0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f09b8089340 tx=0x7f09a8017000 comp rx=0 tx=0).stop 2023-10-06T03:11:24.760 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 shutdown_connections 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f09a4067320 0x7f09a40697d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b8159420 0x7f09b80cadd0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09b815d230 0x7f09b816a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 --2- 172.21.15.92:0/3546587113 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09b81597f0 0x7f09b80cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 >> 172.21.15.92:0/3546587113 conn(0x7f09b80d0660 msgr2=0x7f09b80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 shutdown_connections 2023-10-06T03:11:24.761 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.758+0000 7f09bcfe6700 1 -- 172.21.15.92:0/3546587113 wait complete. 2023-10-06T03:11:24.772 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:11:24.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:11:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.885+0000 7f326fdfb700 1 Processor -- start 2023-10-06T03:11:24.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.885+0000 7f326fdfb700 1 -- start start 2023-10-06T03:11:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326fdfb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f326806bcd0 0x7f3268140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326fdfb700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f326806c0a0 0x7f3268141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.889 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326fdfb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32680c8e00 0x7f32680c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326fdfb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326815c2a0 con 0x7f326806bcd0 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326fdfb700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326815b690 con 0x7f326806c0a0 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326fdfb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326815b970 con 0x7f32680c8e00 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326db97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f326806bcd0 0x7f3268140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326db97700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f326806bcd0 0x7f3268140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59706/0 (socket says 172.21.15.92:59706) 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.886+0000 7f326db97700 1 -- 172.21.15.92:0/1548822508 learned_addr learned my addr 172.21.15.92:0/1548822508 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326d396700 1 --2- 172.21.15.92:0/1548822508 >> v2:172.21.15.136:3300/0 conn(0x7f326806c0a0 0x7f3268141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326e398700 1 --2- 172.21.15.92:0/1548822508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32680c8e00 0x7f32680c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326db97700 1 -- 172.21.15.92:0/1548822508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32680c8e00 msgr2=0x7f32680c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326db97700 1 --2- 172.21.15.92:0/1548822508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32680c8e00 0x7f32680c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.890 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326db97700 1 -- 172.21.15.92:0/1548822508 >> v2:172.21.15.136:3300/0 conn(0x7f326806c0a0 msgr2=0x7f3268141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326db97700 1 --2- 172.21.15.92:0/1548822508 >> v2:172.21.15.136:3300/0 conn(0x7f326806c0a0 0x7f3268141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326db97700 1 -- 172.21.15.92:0/1548822508 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326815bc50 con 0x7f326806bcd0 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326db97700 1 --2- 172.21.15.92:0/1548822508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f326806bcd0 0x7f3268140df0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f326806c6e0 tx=0x7f3264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761e5f782fa403bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326cb95700 1 -- 172.21.15.92:0/1548822508 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400b860 con 0x7f326806bcd0 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326cb95700 1 -- 172.21.15.92:0/1548822508 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326400b9c0 con 0x7f326806bcd0 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326cb95700 1 -- 172.21.15.92:0/1548822508 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400bd60 con 0x7f326806bcd0 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326fdfb700 1 -- 172.21.15.92:0/1548822508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f326806bcd0 msgr2=0x7f3268140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.887+0000 7f326fdfb700 1 --2- 172.21.15.92:0/1548822508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f326806bcd0 0x7f3268140df0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f326806c6e0 tx=0x7f3264009250 comp rx=0 tx=0).stop 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 -- 172.21.15.92:0/1548822508 shutdown_connections 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 --2- 172.21.15.92:0/1548822508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32680c8e00 0x7f32680c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.891 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 --2- 172.21.15.92:0/1548822508 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f326806bcd0 0x7f3268140df0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 --2- 172.21.15.92:0/1548822508 >> v2:172.21.15.136:3300/0 conn(0x7f326806c0a0 0x7f3268141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 -- 172.21.15.92:0/1548822508 >> 172.21.15.92:0/1548822508 conn(0x7f32680ce600 msgr2=0x7f32680cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 -- 172.21.15.92:0/1548822508 shutdown_connections 2023-10-06T03:11:24.892 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.888+0000 7f326fdfb700 1 -- 172.21.15.92:0/1548822508 wait complete. 2023-10-06T03:11:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.889+0000 7f326fdfb700 1 Processor -- start 2023-10-06T03:11:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.889+0000 7f326fdfb700 1 -- start start 2023-10-06T03:11:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.889+0000 7f326fdfb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f326806bcd0 0x7f32680c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.889+0000 7f326fdfb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f326806c0a0 0x7f32680c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.889+0000 7f326fdfb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 0x7f3268159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326fdfb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268168d50 con 0x7f32680c8e00 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326fdfb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32680c8a10 con 0x7f326806c0a0 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326fdfb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3268168140 con 0x7f326806bcd0 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 0x7f3268159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 0x7f3268159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59708/0 (socket says 172.21.15.92:59708) 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 -- 172.21.15.92:0/4109088555 learned_addr learned my addr 172.21.15.92:0/4109088555 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326d396700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f326806c0a0 0x7f32680c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326db97700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f326806bcd0 0x7f32680c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 -- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f326806bcd0 msgr2=0x7f32680c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f326806bcd0 0x7f32680c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 -- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f326806c0a0 msgr2=0x7f32680c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f326806c0a0 0x7f32680c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 -- 172.21.15.92:0/4109088555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640094f0 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326d396700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f326806c0a0 0x7f32680c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326e398700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 0x7f3268159210 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f3260007f20 tx=0x7f3260009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000eca0 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000ee20 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260014670 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3268168420 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.890+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32681689e0 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.891+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326804c3c0 con 0x7f32680c8e00 2023-10-06T03:11:24.896 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.891+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f32600149e0 con 0x7f32680c8e00 2023-10-06T03:11:24.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.892+0000 7f325dffb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f324c067350 0x7f324c069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:24.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.892+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3260085440 con 0x7f32680c8e00 2023-10-06T03:11:24.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.892+0000 7f326db97700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f324c067350 0x7f324c069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:24.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.892+0000 7f326db97700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f324c067350 0x7f324c069800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f326806c6e0 tx=0x7f3264017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:24.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:24.896+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3260051f70 con 0x7f32680c8e00 2023-10-06T03:11:25.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.077+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f326806c9d0 con 0x7f324c067350 2023-10-06T03:11:25.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.085+0000 7f325dffb700 1 -- 172.21.15.92:0/4109088555 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300773 (secure 0 0 0) 0x7f326806c9d0 con 0x7f324c067350 2023-10-06T03:11:25.090 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f324c067350 msgr2=0x7f324c069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f324c067350 0x7f324c069800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f326806c6e0 tx=0x7f3264017000 comp rx=0 tx=0).stop 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 msgr2=0x7f3268159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 0x7f3268159210 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f3260007f20 tx=0x7f3260009860 comp rx=0 tx=0).stop 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 shutdown_connections 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f324c067350 0x7f324c069800 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f32680c8e00 0x7f3268159210 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f326806bcd0 0x7f32680c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.093 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 --2- 172.21.15.92:0/4109088555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f326806c0a0 0x7f32680c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 >> 172.21.15.92:0/4109088555 conn(0x7f32680ce600 msgr2=0x7f32680c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:25.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 shutdown_connections 2023-10-06T03:11:25.094 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.088+0000 7f326fdfb700 1 -- 172.21.15.92:0/4109088555 wait complete. 2023-10-06T03:11:25.094 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:11:25.102 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2023-10-06T03:11:23.301003+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2431840,"kb_used_data":20768,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542880,"statfs":{"total":773094113280,"available":770603909120,"internally_reserved":0,"allocated":21266432,"data_stored":10713668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.280812"},"pg_stats":[{"pgid":"35.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025735+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:39.628700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022624+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:22.749417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022621+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:56:12.828746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025729+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:12.738881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.024441+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:45.233344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.023187+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:44:43.426785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.027181+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:36.476796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.026449+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:04:52.288771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.023437+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:13.687018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025345+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:20.440052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022097+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:57.061467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025782+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:00.992425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025208+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:27.771279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025176+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:20.440052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.024220+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:17.947102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025726+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:08.198002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022929+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:00:04.189843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.024738+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:47.397141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022770+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:23:15.472349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022774+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:03.220038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025438+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:21:37.290745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.024791+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:27.212158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022669+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:34.866025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025294+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:37:47.856493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025066+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:15.789415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.022669+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:09.043331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025100+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:37.599963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.024967+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:34.012798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025696+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:11:17.947102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025471+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:39.702153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025344+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:38:19.728247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:11:19.014752+0000","last_change":"2023-10-06T03:11:19.025608+0000","last_active":"2023-10-06T03:11:19.014752+0000","last_peered":"2023-10-06T03:11:19.014752+0000","last_clean":"2023-10-06T03:11:19.014752+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:19.014752+0000","last_undegraded":"2023-10-06T03:11:19.014752+0000","last_fullsized":"2023-10-06T03:11:19.014752+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:44.018450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.028366+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:11:16.028366+0000","last_peered":"2023-10-06T03:11:16.028366+0000","last_clean":"2023-10-06T03:11:16.028366+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:11:16.028366+0000","last_undegraded":"2023-10-06T03:11:16.028366+0000","last_fullsized":"2023-10-06T03:11:16.028366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.027699+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:11:16.027699+0000","last_peered":"2023-10-06T03:11:16.027699+0000","last_clean":"2023-10-06T03:11:16.027699+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:11:16.027699+0000","last_undegraded":"2023-10-06T03:11:16.027699+0000","last_fullsized":"2023-10-06T03:11:16.027699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":160,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.013146+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:11:16.013146+0000","last_peered":"2023-10-06T03:11:16.013146+0000","last_clean":"2023-10-06T03:11:16.013146+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:11:16.013146+0000","last_undegraded":"2023-10-06T03:11:16.013146+0000","last_fullsized":"2023-10-06T03:11:16.013146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":160,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.013128+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:11:16.013128+0000","last_peered":"2023-10-06T03:11:16.013128+0000","last_clean":"2023-10-06T03:11:16.013128+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:11:16.013128+0000","last_undegraded":"2023-10-06T03:11:16.013128+0000","last_fullsized":"2023-10-06T03:11:16.013128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":313,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.013140+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:11:16.013140+0000","last_peered":"2023-10-06T03:11:16.013140+0000","last_clean":"2023-10-06T03:11:16.013140+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:11:16.013140+0000","last_undegraded":"2023-10-06T03:11:16.013140+0000","last_fullsized":"2023-10-06T03:11:16.013140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":180,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.027753+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:11:16.027753+0000","last_peered":"2023-10-06T03:11:16.027753+0000","last_clean":"2023-10-06T03:11:16.027753+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:11:16.027753+0000","last_undegraded":"2023-10-06T03:11:16.027753+0000","last_fullsized":"2023-10-06T03:11:16.027753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.028148+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:11:16.028148+0000","last_peered":"2023-10-06T03:11:16.028148+0000","last_clean":"2023-10-06T03:11:16.028148+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:11:16.028148+0000","last_undegraded":"2023-10-06T03:11:16.028148+0000","last_fullsized":"2023-10-06T03:11:16.028148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":180,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.027755+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:11:16.027755+0000","last_peered":"2023-10-06T03:11:16.027755+0000","last_clean":"2023-10-06T03:11:16.027755+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:11:16.027755+0000","last_undegraded":"2023-10-06T03:11:16.027755+0000","last_fullsized":"2023-10-06T03:11:16.027755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:11:20.020393+0000","last_change":"2023-10-06T03:11:20.020393+0000","last_active":"2023-10-06T03:11:20.020393+0000","last_peered":"2023-10-06T03:11:20.020393+0000","last_clean":"2023-10-06T03:11:20.020393+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:11:20.020393+0000","last_undegraded":"2023-10-06T03:11:20.020393+0000","last_fullsized":"2023-10-06T03:11:20.020393+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-06T03:11:16.027601+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:11:16.027601+0000","last_peered":"2023-10-06T03:11:16.027601+0000","last_clean":"2023-10-06T03:11:16.027601+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:11:16.027601+0000","last_undegraded":"2023-10-06T03:11:16.027601+0000","last_fullsized":"2023-10-06T03:11:16.027601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":35,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304412,"kb_used_data":3028,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067428,"statfs":{"total":96636764160,"available":96325046272,"internally_reserved":0,"allocated":3100672,"data_stored":1780445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.745,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64100000000000001,"5min":1.264,"15min":1.264},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":2.286,"15min":2.286},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":2.504,"15min":2.504},"last":0.77700000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":1.141,"15min":1.141},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":1.639,"15min":1.639},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82199999999999995,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.747},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.307,"15min":0.307},"max":{"1min":0.77800000000000002,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.80900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303832,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068008,"statfs":{"total":96636764160,"available":96325640192,"internally_reserved":0,"allocated":2506752,"data_stored":1190077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.91400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.436},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57099999999999995},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.505},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.70799999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.72299999999999998,"5min":0.755,"15min":0.755},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79000000000000004},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.873,"15min":0.873},"last":0.34300000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72099999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.377}]},{"osd":7,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78900000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.495},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.36699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":2460,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":2519040,"data_stored":1196235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35099999999999998,"15min":0.33400000000000002},"min":{"1min":0.23300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48199999999999998,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.432,"5min":1.282,"15min":1.282},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33300000000000002,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48599999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.309}]},{"osd":3,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.27000000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.45100000000000001,"5min":1.25,"15min":1.25},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.374,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":1.1060000000000001,"5min":1.302,"15min":1.302},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.35099999999999998},"max":{"1min":0.83099999999999996,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.96299999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":1.0629999999999999,"5min":1.363,"15min":5.0270000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.052,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.56699999999999995},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":0.745,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.442},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":1.0820000000000001,"5min":1.379,"15min":1.379},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304424,"kb_used_data":3040,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067416,"statfs":{"total":96636764160,"available":96325033984,"internally_reserved":0,"allocated":3112960,"data_stored":1786603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.309},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.28799999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.498},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.47699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303832,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068008,"statfs":{"total":96636764160,"available":96325640192,"internally_reserved":0,"allocated":2506752,"data_stored":1190077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.317},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.307,"5min":0.33400000000000002,"15min":0.32100000000000001},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.374,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33100000000000002,"15min":0.33000000000000002},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.504,"15min":0.56599999999999995},"last":0.28499999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.435,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.314},{"interface":"front","average":{"1min":0.376,"5min":0.35899999999999999,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.39200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.86299999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.374,"15min":0.374},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.903},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.57299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303832,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068008,"statfs":{"total":96636764160,"available":96325640192,"internally_reserved":0,"allocated":2506752,"data_stored":1190077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.436,"5min":0.503,"15min":0.65200000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.49299999999999999,"15min":0.72299999999999998},"last":0.37}]},{"osd":1,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.313},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.59599999999999997,"15min":0.69699999999999995},"last":0.313}]},{"osd":2,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.47399999999999998,"5min":0.63,"15min":0.67600000000000005},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78800000000000003,"5min":0.79700000000000004,"15min":0.94399999999999995},"last":0.498},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.44900000000000001,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.997},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.98499999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":1.0660000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303832,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068008,"statfs":{"total":96636764160,"available":96325640192,"internally_reserved":0,"allocated":2506752,"data_stored":1190077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.73599999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.57199999999999995},"min":{"1min":0.441,"5min":0.307,"15min":0.307},"max":{"1min":1.069,"5min":1.069,"15min":1.4410000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.53300000000000003},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.31900000000000001},"max":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":1.0009999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":3},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53400000000000003,"15min":0.71599999999999997},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":10.084},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56200000000000006,"15min":0.65000000000000002},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.749,"5min":0.752,"15min":0.83999999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54400000000000004,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.54},"last":0.39600000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.46600000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59099999999999997,"15min":0.60799999999999998},"min":{"1min":0.44600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.69199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303832,"kb_used_data":2448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068008,"statfs":{"total":96636764160,"available":96325640192,"internally_reserved":0,"allocated":2506752,"data_stored":1190077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":1.143,"15min":2.3599999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.59199999999999997},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.28100000000000003},"max":{"1min":0.73199999999999998,"5min":0.86299999999999999,"15min":2.371},"last":0.495}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":6.532,"15min":6.532},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.67600000000000005},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72899999999999998,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.65900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":5.524,"15min":5.524},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64100000000000001,"5min":0.78200000000000003,"15min":1.1990000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.44900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.052},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.175},"last":0.625},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.145},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:11:25.111 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:11:25.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 Processor -- start 2023-10-06T03:11:25.257 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 -- start start 2023-10-06T03:11:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf7415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf7415db10 0x7fbf74069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7415a160 con 0x7fbf74159890 2023-10-06T03:11:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf740d13a0 con 0x7fbf7415db10 2023-10-06T03:11:25.260 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.256+0000 7fbf799ca700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7415e140 con 0x7fbf74159c60 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf6a7fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf7415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf737fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf7415db10 0x7fbf74069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf72ffd700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fbf74159890 0x7fbf7415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:41666/0 (socket says 172.21.15.92:41666) 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf72ffd700 1 -- 172.21.15.92:0/4165110678 learned_addr learned my addr 172.21.15.92:0/4165110678 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf727fc700 1 -- 172.21.15.92:0/4165110678 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336278009 0 0) 0x7fbf7415a160 con 0x7fbf74159890 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf727fc700 1 -- 172.21.15.92:0/4165110678 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf60003570 con 0x7fbf74159890 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf737fe700 1 -- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 msgr2=0x7fbf7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf737fe700 1 --2- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf7415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf737fe700 1 -- 172.21.15.92:0/4165110678 >> v1:172.21.15.136:6789/0 conn(0x7fbf74159890 legacy=0x7fbf7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf737fe700 1 -- 172.21.15.92:0/4165110678 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7415a3d0 con 0x7fbf7415db10 2023-10-06T03:11:25.261 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.257+0000 7fbf6a7fc700 1 --2- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf7415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf737fe700 1 --2- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf74069560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009e60 tx=0x7fbf6c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab52196b08bfd6da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf727fc700 1 -- 172.21.15.92:0/4165110678 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00e070 con 0x7fbf7415db10 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf727fc700 1 -- 172.21.15.92:0/4165110678 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c00bab0 con 0x7fbf7415db10 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf727fc700 1 -- 172.21.15.92:0/4165110678 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00be70 con 0x7fbf7415db10 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 -- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 msgr2=0x7fbf74069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 --2- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf74069560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c009e60 tx=0x7fbf6c009780 comp rx=0 tx=0).stop 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 -- 172.21.15.92:0/4165110678 shutdown_connections 2023-10-06T03:11:25.262 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 --2- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf74069560 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 --2- 172.21.15.92:0/4165110678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf7415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 -- 172.21.15.92:0/4165110678 >> 172.21.15.92:0/4165110678 conn(0x7fbf74068e20 msgr2=0x7fbf740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 -- 172.21.15.92:0/4165110678 shutdown_connections 2023-10-06T03:11:25.263 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.258+0000 7fbf799ca700 1 -- 172.21.15.92:0/4165110678 wait complete. 2023-10-06T03:11:25.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.259+0000 7fbf799ca700 1 Processor -- start 2023-10-06T03:11:25.265 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 -- start start 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 0x7fbf74168df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf740cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf740ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7415cc40 con 0x7fbf74159890 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7415cf20 con 0x7fbf7415db10 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf799ca700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7415d200 con 0x7fbf74159c60 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 0x7fbf74168df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf72ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 0x7fbf74168df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59720/0 (socket says 172.21.15.92:59720) 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf72ffd700 1 -- 172.21.15.92:0/1109185512 learned_addr learned my addr 172.21.15.92:0/1109185512 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf6a7fc700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf740cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.260+0000 7fbf737fe700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf740ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf72ffd700 1 -- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 msgr2=0x7fbf740cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf72ffd700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf740cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf72ffd700 1 -- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 msgr2=0x7fbf740ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf72ffd700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf740ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf72ffd700 1 -- 172.21.15.92:0/1109185512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009420 con 0x7fbf74159890 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf737fe700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf740ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf72ffd700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 0x7fbf74168df0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c7c0 tx=0x7fbf64007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64005b30 con 0x7fbf74159890 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf64005cb0 con 0x7fbf74159890 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64007320 con 0x7fbf74159890 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c009a20 con 0x7fbf74159890 2023-10-06T03:11:25.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.261+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf741512d0 con 0x7fbf74159890 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.262+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7415cd80 con 0x7fbf74159890 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.262+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbf6401b3b0 con 0x7fbf74159890 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.262+0000 7fbf6b7fe700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbf54067320 0x7fbf540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.263+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbf640842c0 con 0x7fbf74159890 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.263+0000 7fbf6a7fc700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbf54067320 0x7fbf540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.263+0000 7fbf6a7fc700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbf54067320 0x7fbf540697d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbf741530a0 tx=0x7fbf5c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:25.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.267+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf7415cd80 con 0x7fbf74159890 2023-10-06T03:11:25.324 INFO:tasks.ceph.mds.c.smithi092.stderr:2023-10-06T03:11:25.322+0000 7ff9c7f28700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:11:25.446 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.444+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf7415cd80 con 0x7fbf54067320 2023-10-06T03:11:25.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.452+0000 7fbf6b7fe700 1 -- 172.21.15.92:0/1109185512 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311793 (secure 0 0 0) 0x7fbf7415cd80 con 0x7fbf54067320 2023-10-06T03:11:25.458 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbf54067320 msgr2=0x7fbf540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbf54067320 0x7fbf540697d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbf741530a0 tx=0x7fbf5c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 msgr2=0x7fbf74168df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 0x7fbf74168df0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fbf6400c7c0 tx=0x7fbf64007e00 comp rx=0 tx=0).stop 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 shutdown_connections 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fbf54067320 0x7fbf540697d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74159890 0x7fbf74168df0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf7415db10 0x7fbf740ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 --2- 172.21.15.92:0/1109185512 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf74159c60 0x7fbf740cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 >> 172.21.15.92:0/1109185512 conn(0x7fbf74068e20 msgr2=0x7fbf740b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:25.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 shutdown_connections 2023-10-06T03:11:25.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.456+0000 7fbf799ca700 1 -- 172.21.15.92:0/1109185512 wait complete. 2023-10-06T03:11:25.462 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:11:25.471 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2023-10-06T03:11:25.301401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":197,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432264,"kb_used_data":21192,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542456,"statfs":{"total":773094113280,"available":770603474944,"internally_reserved":0,"allocated":21700608,"data_stored":10918212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001194"},"pg_stats":[{"pgid":"35.3f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050825+0000","last_change":"2023-10-06T03:11:21.043689+0000","last_active":"2023-10-06T03:11:21.050825+0000","last_peered":"2023-10-06T03:11:21.050825+0000","last_clean":"2023-10-06T03:11:21.050825+0000","last_became_active":"2023-10-06T03:11:21.043447+0000","last_became_peered":"2023-10-06T03:11:21.043447+0000","last_unstale":"2023-10-06T03:11:21.050825+0000","last_undegraded":"2023-10-06T03:11:21.050825+0000","last_fullsized":"2023-10-06T03:11:21.050825+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:58:30.832952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.3e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.047929+0000","last_change":"2023-10-06T03:11:21.041889+0000","last_active":"2023-10-06T03:11:21.047929+0000","last_peered":"2023-10-06T03:11:21.047929+0000","last_clean":"2023-10-06T03:11:21.047929+0000","last_became_active":"2023-10-06T03:11:21.041530+0000","last_became_peered":"2023-10-06T03:11:21.041530+0000","last_unstale":"2023-10-06T03:11:21.047929+0000","last_undegraded":"2023-10-06T03:11:21.047929+0000","last_fullsized":"2023-10-06T03:11:21.047929+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:24.241876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055729+0000","last_change":"2023-10-06T03:11:21.051151+0000","last_active":"2023-10-06T03:11:21.055729+0000","last_peered":"2023-10-06T03:11:21.055729+0000","last_clean":"2023-10-06T03:11:21.055729+0000","last_became_active":"2023-10-06T03:11:21.050902+0000","last_became_peered":"2023-10-06T03:11:21.050902+0000","last_unstale":"2023-10-06T03:11:21.055729+0000","last_undegraded":"2023-10-06T03:11:21.055729+0000","last_fullsized":"2023-10-06T03:11:21.055729+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:49.190192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.052338+0000","last_change":"2023-10-06T03:11:21.045741+0000","last_active":"2023-10-06T03:11:21.052338+0000","last_peered":"2023-10-06T03:11:21.052338+0000","last_clean":"2023-10-06T03:11:21.052338+0000","last_became_active":"2023-10-06T03:11:21.038884+0000","last_became_peered":"2023-10-06T03:11:21.038884+0000","last_unstale":"2023-10-06T03:11:21.052338+0000","last_undegraded":"2023-10-06T03:11:21.052338+0000","last_fullsized":"2023-10-06T03:11:21.052338+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:28.776920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.3b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.052813+0000","last_change":"2023-10-06T03:11:21.046116+0000","last_active":"2023-10-06T03:11:21.052813+0000","last_peered":"2023-10-06T03:11:21.052813+0000","last_clean":"2023-10-06T03:11:21.052813+0000","last_became_active":"2023-10-06T03:11:21.045829+0000","last_became_peered":"2023-10-06T03:11:21.045829+0000","last_unstale":"2023-10-06T03:11:21.052813+0000","last_undegraded":"2023-10-06T03:11:21.052813+0000","last_fullsized":"2023-10-06T03:11:21.052813+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:00.985257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.3a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.052898+0000","last_change":"2023-10-06T03:11:21.046221+0000","last_active":"2023-10-06T03:11:21.052898+0000","last_peered":"2023-10-06T03:11:21.052898+0000","last_clean":"2023-10-06T03:11:21.052898+0000","last_became_active":"2023-10-06T03:11:21.045935+0000","last_became_peered":"2023-10-06T03:11:21.045935+0000","last_unstale":"2023-10-06T03:11:21.052898+0000","last_undegraded":"2023-10-06T03:11:21.052898+0000","last_fullsized":"2023-10-06T03:11:21.052898+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:30:42.517850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.39","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056303+0000","last_change":"2023-10-06T03:11:21.054797+0000","last_active":"2023-10-06T03:11:21.056303+0000","last_peered":"2023-10-06T03:11:21.056303+0000","last_clean":"2023-10-06T03:11:21.056303+0000","last_became_active":"2023-10-06T03:11:21.054377+0000","last_became_peered":"2023-10-06T03:11:21.054377+0000","last_unstale":"2023-10-06T03:11:21.056303+0000","last_undegraded":"2023-10-06T03:11:21.056303+0000","last_fullsized":"2023-10-06T03:11:21.056303+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:08:31.997173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.38","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055853+0000","last_change":"2023-10-06T03:11:21.051523+0000","last_active":"2023-10-06T03:11:21.055853+0000","last_peered":"2023-10-06T03:11:21.055853+0000","last_clean":"2023-10-06T03:11:21.055853+0000","last_became_active":"2023-10-06T03:11:21.051056+0000","last_became_peered":"2023-10-06T03:11:21.051056+0000","last_unstale":"2023-10-06T03:11:21.055853+0000","last_undegraded":"2023-10-06T03:11:21.055853+0000","last_fullsized":"2023-10-06T03:11:21.055853+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:29.325931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.37","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.042266+0000","last_change":"2023-10-06T03:11:21.038185+0000","last_active":"2023-10-06T03:11:21.042266+0000","last_peered":"2023-10-06T03:11:21.042266+0000","last_clean":"2023-10-06T03:11:21.042266+0000","last_became_active":"2023-10-06T03:11:21.037930+0000","last_became_peered":"2023-10-06T03:11:21.037930+0000","last_unstale":"2023-10-06T03:11:21.042266+0000","last_undegraded":"2023-10-06T03:11:21.042266+0000","last_fullsized":"2023-10-06T03:11:21.042266+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:50.225506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.36","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056136+0000","last_change":"2023-10-06T03:11:21.052085+0000","last_active":"2023-10-06T03:11:21.056136+0000","last_peered":"2023-10-06T03:11:21.056136+0000","last_clean":"2023-10-06T03:11:21.056136+0000","last_became_active":"2023-10-06T03:11:21.051824+0000","last_became_peered":"2023-10-06T03:11:21.051824+0000","last_unstale":"2023-10-06T03:11:21.056136+0000","last_undegraded":"2023-10-06T03:11:21.056136+0000","last_fullsized":"2023-10-06T03:11:21.056136+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:09.203643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.35","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053605+0000","last_change":"2023-10-06T03:11:21.051514+0000","last_active":"2023-10-06T03:11:21.053605+0000","last_peered":"2023-10-06T03:11:21.053605+0000","last_clean":"2023-10-06T03:11:21.053605+0000","last_became_active":"2023-10-06T03:11:21.051175+0000","last_became_peered":"2023-10-06T03:11:21.051175+0000","last_unstale":"2023-10-06T03:11:21.053605+0000","last_undegraded":"2023-10-06T03:11:21.053605+0000","last_fullsized":"2023-10-06T03:11:21.053605+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:33.573560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.34","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.051176+0000","last_change":"2023-10-06T03:11:21.045216+0000","last_active":"2023-10-06T03:11:21.051176+0000","last_peered":"2023-10-06T03:11:21.051176+0000","last_clean":"2023-10-06T03:11:21.051176+0000","last_became_active":"2023-10-06T03:11:21.044949+0000","last_became_peered":"2023-10-06T03:11:21.044949+0000","last_unstale":"2023-10-06T03:11:21.051176+0000","last_undegraded":"2023-10-06T03:11:21.051176+0000","last_fullsized":"2023-10-06T03:11:21.051176+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:36.062314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.33","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.057073+0000","last_change":"2023-10-06T03:11:21.048735+0000","last_active":"2023-10-06T03:11:21.057073+0000","last_peered":"2023-10-06T03:11:21.057073+0000","last_clean":"2023-10-06T03:11:21.057073+0000","last_became_active":"2023-10-06T03:11:21.048498+0000","last_became_peered":"2023-10-06T03:11:21.048498+0000","last_unstale":"2023-10-06T03:11:21.057073+0000","last_undegraded":"2023-10-06T03:11:21.057073+0000","last_fullsized":"2023-10-06T03:11:21.057073+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:14:46.792520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.32","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053204+0000","last_change":"2023-10-06T03:11:21.047968+0000","last_active":"2023-10-06T03:11:21.053204+0000","last_peered":"2023-10-06T03:11:21.053204+0000","last_clean":"2023-10-06T03:11:21.053204+0000","last_became_active":"2023-10-06T03:11:21.047571+0000","last_became_peered":"2023-10-06T03:11:21.047571+0000","last_unstale":"2023-10-06T03:11:21.053204+0000","last_undegraded":"2023-10-06T03:11:21.053204+0000","last_fullsized":"2023-10-06T03:11:21.053204+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:21.445693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.31","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055382+0000","last_change":"2023-10-06T03:11:21.049119+0000","last_active":"2023-10-06T03:11:21.055382+0000","last_peered":"2023-10-06T03:11:21.055382+0000","last_clean":"2023-10-06T03:11:21.055382+0000","last_became_active":"2023-10-06T03:11:21.048702+0000","last_became_peered":"2023-10-06T03:11:21.048702+0000","last_unstale":"2023-10-06T03:11:21.055382+0000","last_undegraded":"2023-10-06T03:11:21.055382+0000","last_fullsized":"2023-10-06T03:11:21.055382+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:25.648397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.30","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055833+0000","last_change":"2023-10-06T03:11:21.048916+0000","last_active":"2023-10-06T03:11:21.055833+0000","last_peered":"2023-10-06T03:11:21.055833+0000","last_clean":"2023-10-06T03:11:21.055833+0000","last_became_active":"2023-10-06T03:11:21.048624+0000","last_became_peered":"2023-10-06T03:11:21.048624+0000","last_unstale":"2023-10-06T03:11:21.055833+0000","last_undegraded":"2023-10-06T03:11:21.055833+0000","last_fullsized":"2023-10-06T03:11:21.055833+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:48.664875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.038084+0000","last_change":"2023-10-06T03:11:21.035171+0000","last_active":"2023-10-06T03:11:21.038084+0000","last_peered":"2023-10-06T03:11:21.038084+0000","last_clean":"2023-10-06T03:11:21.038084+0000","last_became_active":"2023-10-06T03:11:21.034904+0000","last_became_peered":"2023-10-06T03:11:21.034904+0000","last_unstale":"2023-10-06T03:11:21.038084+0000","last_undegraded":"2023-10-06T03:11:21.038084+0000","last_fullsized":"2023-10-06T03:11:21.038084+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:19.440060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.043362+0000","last_change":"2023-10-06T03:11:21.039665+0000","last_active":"2023-10-06T03:11:21.043362+0000","last_peered":"2023-10-06T03:11:21.043362+0000","last_clean":"2023-10-06T03:11:21.043362+0000","last_became_active":"2023-10-06T03:11:21.039403+0000","last_became_peered":"2023-10-06T03:11:21.039403+0000","last_unstale":"2023-10-06T03:11:21.043362+0000","last_undegraded":"2023-10-06T03:11:21.043362+0000","last_fullsized":"2023-10-06T03:11:21.043362+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:55.402061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.2d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056320+0000","last_change":"2023-10-06T03:11:21.054482+0000","last_active":"2023-10-06T03:11:21.056320+0000","last_peered":"2023-10-06T03:11:21.056320+0000","last_clean":"2023-10-06T03:11:21.056320+0000","last_became_active":"2023-10-06T03:11:21.054235+0000","last_became_peered":"2023-10-06T03:11:21.054235+0000","last_unstale":"2023-10-06T03:11:21.056320+0000","last_undegraded":"2023-10-06T03:11:21.056320+0000","last_fullsized":"2023-10-06T03:11:21.056320+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:18.805499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055625+0000","last_change":"2023-10-06T03:11:21.052202+0000","last_active":"2023-10-06T03:11:21.055625+0000","last_peered":"2023-10-06T03:11:21.055625+0000","last_clean":"2023-10-06T03:11:21.055625+0000","last_became_active":"2023-10-06T03:11:21.051921+0000","last_became_peered":"2023-10-06T03:11:21.051921+0000","last_unstale":"2023-10-06T03:11:21.055625+0000","last_undegraded":"2023-10-06T03:11:21.055625+0000","last_fullsized":"2023-10-06T03:11:21.055625+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:01.928404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.2b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050319+0000","last_change":"2023-10-06T03:11:21.042324+0000","last_active":"2023-10-06T03:11:21.050319+0000","last_peered":"2023-10-06T03:11:21.050319+0000","last_clean":"2023-10-06T03:11:21.050319+0000","last_became_active":"2023-10-06T03:11:21.042075+0000","last_became_peered":"2023-10-06T03:11:21.042075+0000","last_unstale":"2023-10-06T03:11:21.050319+0000","last_undegraded":"2023-10-06T03:11:21.050319+0000","last_fullsized":"2023-10-06T03:11:21.050319+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:11.083269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.2a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053257+0000","last_change":"2023-10-06T03:11:21.047242+0000","last_active":"2023-10-06T03:11:21.053257+0000","last_peered":"2023-10-06T03:11:21.053257+0000","last_clean":"2023-10-06T03:11:21.053257+0000","last_became_active":"2023-10-06T03:11:21.046995+0000","last_became_peered":"2023-10-06T03:11:21.046995+0000","last_unstale":"2023-10-06T03:11:21.053257+0000","last_undegraded":"2023-10-06T03:11:21.053257+0000","last_fullsized":"2023-10-06T03:11:21.053257+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:01.511840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.29","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050695+0000","last_change":"2023-10-06T03:11:21.047536+0000","last_active":"2023-10-06T03:11:21.050695+0000","last_peered":"2023-10-06T03:11:21.050695+0000","last_clean":"2023-10-06T03:11:21.050695+0000","last_became_active":"2023-10-06T03:11:21.047268+0000","last_became_peered":"2023-10-06T03:11:21.047268+0000","last_unstale":"2023-10-06T03:11:21.050695+0000","last_undegraded":"2023-10-06T03:11:21.050695+0000","last_fullsized":"2023-10-06T03:11:21.050695+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:33.573560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.28","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053809+0000","last_change":"2023-10-06T03:11:21.049419+0000","last_active":"2023-10-06T03:11:21.053809+0000","last_peered":"2023-10-06T03:11:21.053809+0000","last_clean":"2023-10-06T03:11:21.053809+0000","last_became_active":"2023-10-06T03:11:21.049098+0000","last_became_peered":"2023-10-06T03:11:21.049098+0000","last_unstale":"2023-10-06T03:11:21.053809+0000","last_undegraded":"2023-10-06T03:11:21.053809+0000","last_fullsized":"2023-10-06T03:11:21.053809+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:29.894230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056518+0000","last_change":"2023-10-06T03:11:21.047134+0000","last_active":"2023-10-06T03:11:21.056518+0000","last_peered":"2023-10-06T03:11:21.056518+0000","last_clean":"2023-10-06T03:11:21.056518+0000","last_became_active":"2023-10-06T03:11:21.046833+0000","last_became_peered":"2023-10-06T03:11:21.046833+0000","last_unstale":"2023-10-06T03:11:21.056518+0000","last_undegraded":"2023-10-06T03:11:21.056518+0000","last_fullsized":"2023-10-06T03:11:21.056518+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:43:25.652053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1f","version":"200'4","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.057127+0000","last_change":"2023-10-06T03:11:20.042683+0000","last_active":"2023-10-06T03:11:21.057127+0000","last_peered":"2023-10-06T03:11:21.057127+0000","last_clean":"2023-10-06T03:11:21.057127+0000","last_became_active":"2023-10-06T03:11:20.042443+0000","last_became_peered":"2023-10-06T03:11:20.042443+0000","last_unstale":"2023-10-06T03:11:21.057127+0000","last_undegraded":"2023-10-06T03:11:21.057127+0000","last_fullsized":"2023-10-06T03:11:21.057127+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:08:58.213891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053841+0000","last_change":"2023-10-06T03:11:21.052434+0000","last_active":"2023-10-06T03:11:21.053841+0000","last_peered":"2023-10-06T03:11:21.053841+0000","last_clean":"2023-10-06T03:11:21.053841+0000","last_became_active":"2023-10-06T03:11:21.051375+0000","last_became_peered":"2023-10-06T03:11:21.051375+0000","last_unstale":"2023-10-06T03:11:21.053841+0000","last_undegraded":"2023-10-06T03:11:21.053841+0000","last_fullsized":"2023-10-06T03:11:21.053841+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:01.928404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.1e","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053024+0000","last_change":"2023-10-06T03:11:20.037368+0000","last_active":"2023-10-06T03:11:21.053024+0000","last_peered":"2023-10-06T03:11:21.053024+0000","last_clean":"2023-10-06T03:11:21.053024+0000","last_became_active":"2023-10-06T03:11:20.036593+0000","last_became_peered":"2023-10-06T03:11:20.036593+0000","last_unstale":"2023-10-06T03:11:21.053024+0000","last_undegraded":"2023-10-06T03:11:21.053024+0000","last_fullsized":"2023-10-06T03:11:21.053024+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:39.777809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.049458+0000","last_change":"2023-10-06T03:11:21.040090+0000","last_active":"2023-10-06T03:11:21.049458+0000","last_peered":"2023-10-06T03:11:21.049458+0000","last_clean":"2023-10-06T03:11:21.049458+0000","last_became_active":"2023-10-06T03:11:21.039838+0000","last_became_peered":"2023-10-06T03:11:21.039838+0000","last_unstale":"2023-10-06T03:11:21.049458+0000","last_undegraded":"2023-10-06T03:11:21.049458+0000","last_fullsized":"2023-10-06T03:11:21.049458+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:24:15.791890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1d","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053635+0000","last_change":"2023-10-06T03:11:20.036705+0000","last_active":"2023-10-06T03:11:21.053635+0000","last_peered":"2023-10-06T03:11:21.053635+0000","last_clean":"2023-10-06T03:11:21.053635+0000","last_became_active":"2023-10-06T03:11:20.036442+0000","last_became_peered":"2023-10-06T03:11:20.036442+0000","last_unstale":"2023-10-06T03:11:21.053635+0000","last_undegraded":"2023-10-06T03:11:21.053635+0000","last_fullsized":"2023-10-06T03:11:21.053635+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:08:45.720269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053639+0000","last_change":"2023-10-06T03:11:21.047640+0000","last_active":"2023-10-06T03:11:21.053639+0000","last_peered":"2023-10-06T03:11:21.053639+0000","last_clean":"2023-10-06T03:11:21.053639+0000","last_became_active":"2023-10-06T03:11:21.047209+0000","last_became_peered":"2023-10-06T03:11:21.047209+0000","last_unstale":"2023-10-06T03:11:21.053639+0000","last_undegraded":"2023-10-06T03:11:21.053639+0000","last_fullsized":"2023-10-06T03:11:21.053639+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:22.971620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.046938+0000","last_change":"2023-10-06T03:11:20.045469+0000","last_active":"2023-10-06T03:11:21.046938+0000","last_peered":"2023-10-06T03:11:21.046938+0000","last_clean":"2023-10-06T03:11:21.046938+0000","last_became_active":"2023-10-06T03:11:20.044949+0000","last_became_peered":"2023-10-06T03:11:20.044949+0000","last_unstale":"2023-10-06T03:11:21.046938+0000","last_undegraded":"2023-10-06T03:11:21.046938+0000","last_fullsized":"2023-10-06T03:11:21.046938+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:55.998273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.047603+0000","last_change":"2023-10-06T03:11:21.040332+0000","last_active":"2023-10-06T03:11:21.047603+0000","last_peered":"2023-10-06T03:11:21.047603+0000","last_clean":"2023-10-06T03:11:21.047603+0000","last_became_active":"2023-10-06T03:11:21.040094+0000","last_became_peered":"2023-10-06T03:11:21.040094+0000","last_unstale":"2023-10-06T03:11:21.047603+0000","last_undegraded":"2023-10-06T03:11:21.047603+0000","last_fullsized":"2023-10-06T03:11:21.047603+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:48.664875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1b","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050463+0000","last_change":"2023-10-06T03:11:20.046099+0000","last_active":"2023-10-06T03:11:21.050463+0000","last_peered":"2023-10-06T03:11:21.050463+0000","last_clean":"2023-10-06T03:11:21.050463+0000","last_became_active":"2023-10-06T03:11:20.045841+0000","last_became_peered":"2023-10-06T03:11:20.045841+0000","last_unstale":"2023-10-06T03:11:21.050463+0000","last_undegraded":"2023-10-06T03:11:21.050463+0000","last_fullsized":"2023-10-06T03:11:21.050463+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:59.537612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.1b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053658+0000","last_change":"2023-10-06T03:11:21.048568+0000","last_active":"2023-10-06T03:11:21.053658+0000","last_peered":"2023-10-06T03:11:21.053658+0000","last_clean":"2023-10-06T03:11:21.053658+0000","last_became_active":"2023-10-06T03:11:21.048275+0000","last_became_peered":"2023-10-06T03:11:21.048275+0000","last_unstale":"2023-10-06T03:11:21.053658+0000","last_undegraded":"2023-10-06T03:11:21.053658+0000","last_fullsized":"2023-10-06T03:11:21.053658+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:33.573560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1a","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.057314+0000","last_change":"2023-10-06T03:11:20.043339+0000","last_active":"2023-10-06T03:11:21.057314+0000","last_peered":"2023-10-06T03:11:21.057314+0000","last_clean":"2023-10-06T03:11:21.057314+0000","last_became_active":"2023-10-06T03:11:20.043060+0000","last_became_peered":"2023-10-06T03:11:20.043060+0000","last_unstale":"2023-10-06T03:11:21.057314+0000","last_undegraded":"2023-10-06T03:11:21.057314+0000","last_fullsized":"2023-10-06T03:11:21.057314+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:35.748827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.042788+0000","last_change":"2023-10-06T03:11:21.038562+0000","last_active":"2023-10-06T03:11:21.042788+0000","last_peered":"2023-10-06T03:11:21.042788+0000","last_clean":"2023-10-06T03:11:21.042788+0000","last_became_active":"2023-10-06T03:11:21.037584+0000","last_became_peered":"2023-10-06T03:11:21.037584+0000","last_unstale":"2023-10-06T03:11:21.042788+0000","last_undegraded":"2023-10-06T03:11:21.042788+0000","last_fullsized":"2023-10-06T03:11:21.042788+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:51.476758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.034203+0000","last_change":"2023-10-06T03:11:20.045291+0000","last_active":"2023-10-06T03:11:21.034203+0000","last_peered":"2023-10-06T03:11:21.034203+0000","last_clean":"2023-10-06T03:11:21.034203+0000","last_became_active":"2023-10-06T03:11:20.045100+0000","last_became_peered":"2023-10-06T03:11:20.045100+0000","last_unstale":"2023-10-06T03:11:21.034203+0000","last_undegraded":"2023-10-06T03:11:21.034203+0000","last_fullsized":"2023-10-06T03:11:21.034203+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:44.018450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053777+0000","last_change":"2023-10-06T03:11:21.047968+0000","last_active":"2023-10-06T03:11:21.053777+0000","last_peered":"2023-10-06T03:11:21.053777+0000","last_clean":"2023-10-06T03:11:21.053777+0000","last_became_active":"2023-10-06T03:11:21.047725+0000","last_became_peered":"2023-10-06T03:11:21.047725+0000","last_unstale":"2023-10-06T03:11:21.053777+0000","last_undegraded":"2023-10-06T03:11:21.053777+0000","last_fullsized":"2023-10-06T03:11:21.053777+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:38.793743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.033841+0000","last_change":"2023-10-06T03:11:20.042276+0000","last_active":"2023-10-06T03:11:21.033841+0000","last_peered":"2023-10-06T03:11:21.033841+0000","last_clean":"2023-10-06T03:11:21.033841+0000","last_became_active":"2023-10-06T03:11:20.041819+0000","last_became_peered":"2023-10-06T03:11:20.041819+0000","last_unstale":"2023-10-06T03:11:21.033841+0000","last_undegraded":"2023-10-06T03:11:21.033841+0000","last_fullsized":"2023-10-06T03:11:21.033841+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:45.233344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.046576+0000","last_change":"2023-10-06T03:11:21.043279+0000","last_active":"2023-10-06T03:11:21.046576+0000","last_peered":"2023-10-06T03:11:21.046576+0000","last_clean":"2023-10-06T03:11:21.046576+0000","last_became_active":"2023-10-06T03:11:21.043027+0000","last_became_peered":"2023-10-06T03:11:21.043027+0000","last_unstale":"2023-10-06T03:11:21.046576+0000","last_undegraded":"2023-10-06T03:11:21.046576+0000","last_fullsized":"2023-10-06T03:11:21.046576+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:52:50.581583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.033702+0000","last_change":"2023-10-06T03:11:20.036163+0000","last_active":"2023-10-06T03:11:21.033702+0000","last_peered":"2023-10-06T03:11:21.033702+0000","last_clean":"2023-10-06T03:11:21.033702+0000","last_became_active":"2023-10-06T03:11:20.035852+0000","last_became_peered":"2023-10-06T03:11:20.035852+0000","last_unstale":"2023-10-06T03:11:21.033702+0000","last_undegraded":"2023-10-06T03:11:21.033702+0000","last_fullsized":"2023-10-06T03:11:21.033702+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:17.366769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053190+0000","last_change":"2023-10-06T03:11:21.048172+0000","last_active":"2023-10-06T03:11:21.053190+0000","last_peered":"2023-10-06T03:11:21.053190+0000","last_clean":"2023-10-06T03:11:21.053190+0000","last_became_active":"2023-10-06T03:11:21.047917+0000","last_became_peered":"2023-10-06T03:11:21.047917+0000","last_unstale":"2023-10-06T03:11:21.053190+0000","last_undegraded":"2023-10-06T03:11:21.053190+0000","last_fullsized":"2023-10-06T03:11:21.053190+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:38.605604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.035436+0000","last_change":"2023-10-06T03:11:20.042314+0000","last_active":"2023-10-06T03:11:21.035436+0000","last_peered":"2023-10-06T03:11:21.035436+0000","last_clean":"2023-10-06T03:11:21.035436+0000","last_became_active":"2023-10-06T03:11:20.041971+0000","last_became_peered":"2023-10-06T03:11:20.041971+0000","last_unstale":"2023-10-06T03:11:21.035436+0000","last_undegraded":"2023-10-06T03:11:21.035436+0000","last_fullsized":"2023-10-06T03:11:21.035436+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:52.512838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.14","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053654+0000","last_change":"2023-10-06T03:11:21.051314+0000","last_active":"2023-10-06T03:11:21.053654+0000","last_peered":"2023-10-06T03:11:21.053654+0000","last_clean":"2023-10-06T03:11:21.053654+0000","last_became_active":"2023-10-06T03:11:21.050837+0000","last_became_peered":"2023-10-06T03:11:21.050837+0000","last_unstale":"2023-10-06T03:11:21.053654+0000","last_undegraded":"2023-10-06T03:11:21.053654+0000","last_fullsized":"2023-10-06T03:11:21.053654+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:34:48.664875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.15","version":"200'3","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055417+0000","last_change":"2023-10-06T03:11:20.043928+0000","last_active":"2023-10-06T03:11:21.055417+0000","last_peered":"2023-10-06T03:11:21.055417+0000","last_clean":"2023-10-06T03:11:21.055417+0000","last_became_active":"2023-10-06T03:11:20.043681+0000","last_became_peered":"2023-10-06T03:11:20.043681+0000","last_unstale":"2023-10-06T03:11:21.055417+0000","last_undegraded":"2023-10-06T03:11:21.055417+0000","last_fullsized":"2023-10-06T03:11:21.055417+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:18:59.058573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.046322+0000","last_change":"2023-10-06T03:11:21.037712+0000","last_active":"2023-10-06T03:11:21.046322+0000","last_peered":"2023-10-06T03:11:21.046322+0000","last_clean":"2023-10-06T03:11:21.046322+0000","last_became_active":"2023-10-06T03:11:21.037466+0000","last_became_peered":"2023-10-06T03:11:21.037466+0000","last_unstale":"2023-10-06T03:11:21.046322+0000","last_undegraded":"2023-10-06T03:11:21.046322+0000","last_fullsized":"2023-10-06T03:11:21.046322+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:35.018439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.14","version":"200'6","reported_seq":16,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.059020+0000","last_change":"2023-10-06T03:11:20.038495+0000","last_active":"2023-10-06T03:11:21.059020+0000","last_peered":"2023-10-06T03:11:21.059020+0000","last_clean":"2023-10-06T03:11:21.059020+0000","last_became_active":"2023-10-06T03:11:20.038236+0000","last_became_peered":"2023-10-06T03:11:20.038236+0000","last_unstale":"2023-10-06T03:11:21.059020+0000","last_undegraded":"2023-10-06T03:11:21.059020+0000","last_fullsized":"2023-10-06T03:11:21.059020+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:11.514392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055602+0000","last_change":"2023-10-06T03:11:21.051651+0000","last_active":"2023-10-06T03:11:21.055602+0000","last_peered":"2023-10-06T03:11:21.055602+0000","last_clean":"2023-10-06T03:11:21.055602+0000","last_became_active":"2023-10-06T03:11:21.051382+0000","last_became_peered":"2023-10-06T03:11:21.051382+0000","last_unstale":"2023-10-06T03:11:21.055602+0000","last_undegraded":"2023-10-06T03:11:21.055602+0000","last_fullsized":"2023-10-06T03:11:21.055602+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:33.615677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.13","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050466+0000","last_change":"2023-10-06T03:11:20.045996+0000","last_active":"2023-10-06T03:11:21.050466+0000","last_peered":"2023-10-06T03:11:21.050466+0000","last_clean":"2023-10-06T03:11:21.050466+0000","last_became_active":"2023-10-06T03:11:20.045693+0000","last_became_peered":"2023-10-06T03:11:20.045693+0000","last_unstale":"2023-10-06T03:11:21.050466+0000","last_undegraded":"2023-10-06T03:11:21.050466+0000","last_fullsized":"2023-10-06T03:11:21.050466+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:32.610035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.13","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056042+0000","last_change":"2023-10-06T03:11:21.053240+0000","last_active":"2023-10-06T03:11:21.056042+0000","last_peered":"2023-10-06T03:11:21.056042+0000","last_clean":"2023-10-06T03:11:21.056042+0000","last_became_active":"2023-10-06T03:11:21.052989+0000","last_became_peered":"2023-10-06T03:11:21.052989+0000","last_unstale":"2023-10-06T03:11:21.056042+0000","last_undegraded":"2023-10-06T03:11:21.056042+0000","last_fullsized":"2023-10-06T03:11:21.056042+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:35.018439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.033296+0000","last_change":"2023-10-06T03:11:20.035138+0000","last_active":"2023-10-06T03:11:21.033296+0000","last_peered":"2023-10-06T03:11:21.033296+0000","last_clean":"2023-10-06T03:11:21.033296+0000","last_became_active":"2023-10-06T03:11:20.034836+0000","last_became_peered":"2023-10-06T03:11:20.034836+0000","last_unstale":"2023-10-06T03:11:21.033296+0000","last_undegraded":"2023-10-06T03:11:21.033296+0000","last_fullsized":"2023-10-06T03:11:21.033296+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:17.799858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.10","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.044421+0000","last_change":"2023-10-06T03:11:21.041415+0000","last_active":"2023-10-06T03:11:21.044421+0000","last_peered":"2023-10-06T03:11:21.044421+0000","last_clean":"2023-10-06T03:11:21.044421+0000","last_became_active":"2023-10-06T03:11:21.041143+0000","last_became_peered":"2023-10-06T03:11:21.041143+0000","last_unstale":"2023-10-06T03:11:21.044421+0000","last_undegraded":"2023-10-06T03:11:21.044421+0000","last_fullsized":"2023-10-06T03:11:21.044421+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:02.996034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.030258+0000","last_change":"2023-10-06T03:11:20.043964+0000","last_active":"2023-10-06T03:11:21.030258+0000","last_peered":"2023-10-06T03:11:21.030258+0000","last_clean":"2023-10-06T03:11:21.030258+0000","last_became_active":"2023-10-06T03:11:20.043488+0000","last_became_peered":"2023-10-06T03:11:20.043488+0000","last_unstale":"2023-10-06T03:11:21.030258+0000","last_undegraded":"2023-10-06T03:11:21.030258+0000","last_fullsized":"2023-10-06T03:11:21.030258+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:49:00.232222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.054179+0000","last_change":"2023-10-06T03:11:21.048243+0000","last_active":"2023-10-06T03:11:21.054179+0000","last_peered":"2023-10-06T03:11:21.054179+0000","last_clean":"2023-10-06T03:11:21.054179+0000","last_became_active":"2023-10-06T03:11:21.047912+0000","last_became_peered":"2023-10-06T03:11:21.047912+0000","last_unstale":"2023-10-06T03:11:21.054179+0000","last_undegraded":"2023-10-06T03:11:21.054179+0000","last_fullsized":"2023-10-06T03:11:21.054179+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:53.518479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.10","version":"200'8","reported_seq":18,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.049478+0000","last_change":"2023-10-06T03:11:20.046112+0000","last_active":"2023-10-06T03:11:21.049478+0000","last_peered":"2023-10-06T03:11:21.049478+0000","last_clean":"2023-10-06T03:11:21.049478+0000","last_became_active":"2023-10-06T03:11:20.045834+0000","last_became_peered":"2023-10-06T03:11:20.045834+0000","last_unstale":"2023-10-06T03:11:21.049478+0000","last_undegraded":"2023-10-06T03:11:21.049478+0000","last_fullsized":"2023-10-06T03:11:21.049478+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:52.512838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.047581+0000","last_change":"2023-10-06T03:11:21.039536+0000","last_active":"2023-10-06T03:11:21.047581+0000","last_peered":"2023-10-06T03:11:21.047581+0000","last_clean":"2023-10-06T03:11:21.047581+0000","last_became_active":"2023-10-06T03:11:21.039295+0000","last_became_peered":"2023-10-06T03:11:21.039295+0000","last_unstale":"2023-10-06T03:11:21.047581+0000","last_undegraded":"2023-10-06T03:11:21.047581+0000","last_fullsized":"2023-10-06T03:11:21.047581+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:02.996034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.057154+0000","last_change":"2023-10-06T03:11:20.044647+0000","last_active":"2023-10-06T03:11:21.057154+0000","last_peered":"2023-10-06T03:11:21.057154+0000","last_clean":"2023-10-06T03:11:21.057154+0000","last_became_active":"2023-10-06T03:11:20.043846+0000","last_became_peered":"2023-10-06T03:11:20.043846+0000","last_unstale":"2023-10-06T03:11:21.057154+0000","last_undegraded":"2023-10-06T03:11:21.057154+0000","last_fullsized":"2023-10-06T03:11:21.057154+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:37.677517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.046244+0000","last_change":"2023-10-06T03:11:21.042163+0000","last_active":"2023-10-06T03:11:21.046244+0000","last_peered":"2023-10-06T03:11:21.046244+0000","last_clean":"2023-10-06T03:11:21.046244+0000","last_became_active":"2023-10-06T03:11:21.041897+0000","last_became_peered":"2023-10-06T03:11:21.041897+0000","last_unstale":"2023-10-06T03:11:21.046244+0000","last_undegraded":"2023-10-06T03:11:21.046244+0000","last_fullsized":"2023-10-06T03:11:21.046244+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:01.698753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.e","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056186+0000","last_change":"2023-10-06T03:11:20.039425+0000","last_active":"2023-10-06T03:11:21.056186+0000","last_peered":"2023-10-06T03:11:21.056186+0000","last_clean":"2023-10-06T03:11:21.056186+0000","last_became_active":"2023-10-06T03:11:20.039171+0000","last_became_peered":"2023-10-06T03:11:20.039171+0000","last_unstale":"2023-10-06T03:11:21.056186+0000","last_undegraded":"2023-10-06T03:11:21.056186+0000","last_fullsized":"2023-10-06T03:11:21.056186+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:36.517676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.040306+0000","last_change":"2023-10-06T03:11:21.038313+0000","last_active":"2023-10-06T03:11:21.040306+0000","last_peered":"2023-10-06T03:11:21.040306+0000","last_clean":"2023-10-06T03:11:21.040306+0000","last_became_active":"2023-10-06T03:11:21.038054+0000","last_became_peered":"2023-10-06T03:11:21.038054+0000","last_unstale":"2023-10-06T03:11:21.040306+0000","last_undegraded":"2023-10-06T03:11:21.040306+0000","last_fullsized":"2023-10-06T03:11:21.040306+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:08.888271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.d","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.054495+0000","last_change":"2023-10-06T03:11:20.035326+0000","last_active":"2023-10-06T03:11:21.054495+0000","last_peered":"2023-10-06T03:11:21.054495+0000","last_clean":"2023-10-06T03:11:21.054495+0000","last_became_active":"2023-10-06T03:11:20.034878+0000","last_became_peered":"2023-10-06T03:11:20.034878+0000","last_unstale":"2023-10-06T03:11:21.054495+0000","last_undegraded":"2023-10-06T03:11:21.054495+0000","last_fullsized":"2023-10-06T03:11:21.054495+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:35.682474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053597+0000","last_change":"2023-10-06T03:11:21.046596+0000","last_active":"2023-10-06T03:11:21.053597+0000","last_peered":"2023-10-06T03:11:21.053597+0000","last_clean":"2023-10-06T03:11:21.053597+0000","last_became_active":"2023-10-06T03:11:21.046350+0000","last_became_peered":"2023-10-06T03:11:21.046350+0000","last_unstale":"2023-10-06T03:11:21.053597+0000","last_undegraded":"2023-10-06T03:11:21.053597+0000","last_fullsized":"2023-10-06T03:11:21.053597+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:50.491209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.032287+0000","last_change":"2023-10-06T03:11:20.044402+0000","last_active":"2023-10-06T03:11:21.032287+0000","last_peered":"2023-10-06T03:11:21.032287+0000","last_clean":"2023-10-06T03:11:21.032287+0000","last_became_active":"2023-10-06T03:11:20.043937+0000","last_became_peered":"2023-10-06T03:11:20.043937+0000","last_unstale":"2023-10-06T03:11:21.032287+0000","last_undegraded":"2023-10-06T03:11:21.032287+0000","last_fullsized":"2023-10-06T03:11:21.032287+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:55.998273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.038727+0000","last_change":"2023-10-06T03:11:21.037280+0000","last_active":"2023-10-06T03:11:21.038727+0000","last_peered":"2023-10-06T03:11:21.038727+0000","last_clean":"2023-10-06T03:11:21.038727+0000","last_became_active":"2023-10-06T03:11:21.036720+0000","last_became_peered":"2023-10-06T03:11:21.036720+0000","last_unstale":"2023-10-06T03:11:21.038727+0000","last_undegraded":"2023-10-06T03:11:21.038727+0000","last_fullsized":"2023-10-06T03:11:21.038727+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:29:01.699636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.b","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.052838+0000","last_change":"2023-10-06T03:11:20.043397+0000","last_active":"2023-10-06T03:11:21.052838+0000","last_peered":"2023-10-06T03:11:21.052838+0000","last_clean":"2023-10-06T03:11:21.052838+0000","last_became_active":"2023-10-06T03:11:20.043145+0000","last_became_peered":"2023-10-06T03:11:20.043145+0000","last_unstale":"2023-10-06T03:11:21.052838+0000","last_undegraded":"2023-10-06T03:11:21.052838+0000","last_fullsized":"2023-10-06T03:11:21.052838+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:12:47.397141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.044275+0000","last_change":"2023-10-06T03:11:21.040506+0000","last_active":"2023-10-06T03:11:21.044275+0000","last_peered":"2023-10-06T03:11:21.044275+0000","last_clean":"2023-10-06T03:11:21.044275+0000","last_became_active":"2023-10-06T03:11:21.040222+0000","last_became_peered":"2023-10-06T03:11:21.040222+0000","last_unstale":"2023-10-06T03:11:21.044275+0000","last_undegraded":"2023-10-06T03:11:21.044275+0000","last_fullsized":"2023-10-06T03:11:21.044275+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:11:38.683158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.036638+0000","last_change":"2023-10-06T03:11:20.038718+0000","last_active":"2023-10-06T03:11:21.036638+0000","last_peered":"2023-10-06T03:11:21.036638+0000","last_clean":"2023-10-06T03:11:21.036638+0000","last_became_active":"2023-10-06T03:11:20.038437+0000","last_became_peered":"2023-10-06T03:11:20.038437+0000","last_unstale":"2023-10-06T03:11:21.036638+0000","last_undegraded":"2023-10-06T03:11:21.036638+0000","last_fullsized":"2023-10-06T03:11:21.036638+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:22:01.674468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.058720+0000","last_change":"2023-10-06T03:11:21.054207+0000","last_active":"2023-10-06T03:11:21.058720+0000","last_peered":"2023-10-06T03:11:21.058720+0000","last_clean":"2023-10-06T03:11:21.058720+0000","last_became_active":"2023-10-06T03:11:21.053337+0000","last_became_peered":"2023-10-06T03:11:21.053337+0000","last_unstale":"2023-10-06T03:11:21.058720+0000","last_undegraded":"2023-10-06T03:11:21.058720+0000","last_fullsized":"2023-10-06T03:11:21.058720+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:39.920845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.030161+0000","last_change":"2023-10-06T03:11:20.037538+0000","last_active":"2023-10-06T03:11:21.030161+0000","last_peered":"2023-10-06T03:11:21.030161+0000","last_clean":"2023-10-06T03:11:21.030161+0000","last_became_active":"2023-10-06T03:11:20.037227+0000","last_became_peered":"2023-10-06T03:11:20.037227+0000","last_unstale":"2023-10-06T03:11:21.030161+0000","last_undegraded":"2023-10-06T03:11:21.030161+0000","last_fullsized":"2023-10-06T03:11:21.030161+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:12.738881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056619+0000","last_change":"2023-10-06T03:11:21.047396+0000","last_active":"2023-10-06T03:11:21.056619+0000","last_peered":"2023-10-06T03:11:21.056619+0000","last_clean":"2023-10-06T03:11:21.056619+0000","last_became_active":"2023-10-06T03:11:21.047151+0000","last_became_peered":"2023-10-06T03:11:21.047151+0000","last_unstale":"2023-10-06T03:11:21.056619+0000","last_undegraded":"2023-10-06T03:11:21.056619+0000","last_fullsized":"2023-10-06T03:11:21.056619+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:36:15.980473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.033941+0000","last_change":"2023-10-06T03:11:20.044579+0000","last_active":"2023-10-06T03:11:21.033941+0000","last_peered":"2023-10-06T03:11:21.033941+0000","last_clean":"2023-10-06T03:11:21.033941+0000","last_became_active":"2023-10-06T03:11:20.044307+0000","last_became_peered":"2023-10-06T03:11:20.044307+0000","last_unstale":"2023-10-06T03:11:21.033941+0000","last_undegraded":"2023-10-06T03:11:21.033941+0000","last_fullsized":"2023-10-06T03:11:21.033941+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:53.714071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.055580+0000","last_change":"2023-10-06T03:11:21.051633+0000","last_active":"2023-10-06T03:11:21.055580+0000","last_peered":"2023-10-06T03:11:21.055580+0000","last_clean":"2023-10-06T03:11:21.055580+0000","last_became_active":"2023-10-06T03:11:21.051273+0000","last_became_peered":"2023-10-06T03:11:21.051273+0000","last_unstale":"2023-10-06T03:11:21.055580+0000","last_undegraded":"2023-10-06T03:11:21.055580+0000","last_fullsized":"2023-10-06T03:11:21.055580+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:35.539575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.7","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050541+0000","last_change":"2023-10-06T03:11:20.042262+0000","last_active":"2023-10-06T03:11:21.050541+0000","last_peered":"2023-10-06T03:11:21.050541+0000","last_clean":"2023-10-06T03:11:21.050541+0000","last_became_active":"2023-10-06T03:11:20.041996+0000","last_became_peered":"2023-10-06T03:11:20.041996+0000","last_unstale":"2023-10-06T03:11:21.050541+0000","last_undegraded":"2023-10-06T03:11:21.050541+0000","last_fullsized":"2023-10-06T03:11:21.050541+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:08.198002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.7","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053709+0000","last_change":"2023-10-06T03:11:21.049216+0000","last_active":"2023-10-06T03:11:21.053709+0000","last_peered":"2023-10-06T03:11:21.053709+0000","last_clean":"2023-10-06T03:11:21.053709+0000","last_became_active":"2023-10-06T03:11:21.048980+0000","last_became_peered":"2023-10-06T03:11:21.048980+0000","last_unstale":"2023-10-06T03:11:21.053709+0000","last_undegraded":"2023-10-06T03:11:21.053709+0000","last_fullsized":"2023-10-06T03:11:21.053709+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:25.137779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.031682+0000","last_change":"2023-10-06T03:11:20.033261+0000","last_active":"2023-10-06T03:11:21.031682+0000","last_peered":"2023-10-06T03:11:21.031682+0000","last_clean":"2023-10-06T03:11:21.031682+0000","last_became_active":"2023-10-06T03:11:20.032998+0000","last_became_peered":"2023-10-06T03:11:20.032998+0000","last_unstale":"2023-10-06T03:11:21.031682+0000","last_undegraded":"2023-10-06T03:11:21.031682+0000","last_fullsized":"2023-10-06T03:11:21.031682+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:54.822203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053631+0000","last_change":"2023-10-06T03:11:21.047410+0000","last_active":"2023-10-06T03:11:21.053631+0000","last_peered":"2023-10-06T03:11:21.053631+0000","last_clean":"2023-10-06T03:11:21.053631+0000","last_became_active":"2023-10-06T03:11:21.047129+0000","last_became_peered":"2023-10-06T03:11:21.047129+0000","last_unstale":"2023-10-06T03:11:21.053631+0000","last_undegraded":"2023-10-06T03:11:21.053631+0000","last_fullsized":"2023-10-06T03:11:21.053631+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:39.849451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.036770+0000","last_change":"2023-10-06T03:11:20.047448+0000","last_active":"2023-10-06T03:11:21.036770+0000","last_peered":"2023-10-06T03:11:21.036770+0000","last_clean":"2023-10-06T03:11:21.036770+0000","last_became_active":"2023-10-06T03:11:20.047135+0000","last_became_peered":"2023-10-06T03:11:20.047135+0000","last_unstale":"2023-10-06T03:11:21.036770+0000","last_undegraded":"2023-10-06T03:11:21.036770+0000","last_fullsized":"2023-10-06T03:11:21.036770+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:17.799858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053620+0000","last_change":"2023-10-06T03:11:21.047477+0000","last_active":"2023-10-06T03:11:21.053620+0000","last_peered":"2023-10-06T03:11:21.053620+0000","last_clean":"2023-10-06T03:11:21.053620+0000","last_became_active":"2023-10-06T03:11:21.046903+0000","last_became_peered":"2023-10-06T03:11:21.046903+0000","last_unstale":"2023-10-06T03:11:21.053620+0000","last_undegraded":"2023-10-06T03:11:21.053620+0000","last_fullsized":"2023-10-06T03:11:21.053620+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:22.312681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.035567+0000","last_change":"2023-10-06T03:11:20.042690+0000","last_active":"2023-10-06T03:11:21.035567+0000","last_peered":"2023-10-06T03:11:21.035567+0000","last_clean":"2023-10-06T03:11:21.035567+0000","last_became_active":"2023-10-06T03:11:20.042135+0000","last_became_peered":"2023-10-06T03:11:20.042135+0000","last_unstale":"2023-10-06T03:11:21.035567+0000","last_undegraded":"2023-10-06T03:11:21.035567+0000","last_fullsized":"2023-10-06T03:11:21.035567+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:27.771279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.038745+0000","last_change":"2023-10-06T03:11:21.037350+0000","last_active":"2023-10-06T03:11:21.038745+0000","last_peered":"2023-10-06T03:11:21.038745+0000","last_clean":"2023-10-06T03:11:21.038745+0000","last_became_active":"2023-10-06T03:11:21.037030+0000","last_became_peered":"2023-10-06T03:11:21.037030+0000","last_unstale":"2023-10-06T03:11:21.038745+0000","last_undegraded":"2023-10-06T03:11:21.038745+0000","last_fullsized":"2023-10-06T03:11:21.038745+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:07.513733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.040329+0000","last_change":"2023-10-06T03:11:20.047137+0000","last_active":"2023-10-06T03:11:21.040329+0000","last_peered":"2023-10-06T03:11:21.040329+0000","last_clean":"2023-10-06T03:11:21.040329+0000","last_became_active":"2023-10-06T03:11:20.045500+0000","last_became_peered":"2023-10-06T03:11:20.045500+0000","last_unstale":"2023-10-06T03:11:21.040329+0000","last_undegraded":"2023-10-06T03:11:21.040329+0000","last_fullsized":"2023-10-06T03:11:21.040329+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:17.366769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.040482+0000","last_change":"2023-10-06T03:11:21.037118+0000","last_active":"2023-10-06T03:11:21.040482+0000","last_peered":"2023-10-06T03:11:21.040482+0000","last_clean":"2023-10-06T03:11:21.040482+0000","last_became_active":"2023-10-06T03:11:21.036830+0000","last_became_peered":"2023-10-06T03:11:21.036830+0000","last_unstale":"2023-10-06T03:11:21.040482+0000","last_undegraded":"2023-10-06T03:11:21.040482+0000","last_fullsized":"2023-10-06T03:11:21.040482+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:18.372410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.039728+0000","last_change":"2023-10-06T03:11:20.047297+0000","last_active":"2023-10-06T03:11:21.039728+0000","last_peered":"2023-10-06T03:11:21.039728+0000","last_clean":"2023-10-06T03:11:21.039728+0000","last_became_active":"2023-10-06T03:11:20.046991+0000","last_became_peered":"2023-10-06T03:11:20.046991+0000","last_unstale":"2023-10-06T03:11:21.039728+0000","last_undegraded":"2023-10-06T03:11:21.039728+0000","last_fullsized":"2023-10-06T03:11:21.039728+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:00.992425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.058708+0000","last_change":"2023-10-06T03:11:21.054024+0000","last_active":"2023-10-06T03:11:21.058708+0000","last_peered":"2023-10-06T03:11:21.058708+0000","last_clean":"2023-10-06T03:11:21.058708+0000","last_became_active":"2023-10-06T03:11:21.053561+0000","last_became_peered":"2023-10-06T03:11:21.053561+0000","last_unstale":"2023-10-06T03:11:21.058708+0000","last_undegraded":"2023-10-06T03:11:21.058708+0000","last_fullsized":"2023-10-06T03:11:21.058708+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:11.866052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.039824+0000","last_change":"2023-10-06T03:11:20.047369+0000","last_active":"2023-10-06T03:11:21.039824+0000","last_peered":"2023-10-06T03:11:21.039824+0000","last_clean":"2023-10-06T03:11:21.039824+0000","last_became_active":"2023-10-06T03:11:20.047070+0000","last_became_peered":"2023-10-06T03:11:20.047070+0000","last_unstale":"2023-10-06T03:11:21.039824+0000","last_undegraded":"2023-10-06T03:11:21.039824+0000","last_fullsized":"2023-10-06T03:11:21.039824+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:27.212158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053232+0000","last_change":"2023-10-06T03:11:21.048663+0000","last_active":"2023-10-06T03:11:21.053232+0000","last_peered":"2023-10-06T03:11:21.053232+0000","last_clean":"2023-10-06T03:11:21.053232+0000","last_became_active":"2023-10-06T03:11:21.048402+0000","last_became_peered":"2023-10-06T03:11:21.048402+0000","last_unstale":"2023-10-06T03:11:21.053232+0000","last_undegraded":"2023-10-06T03:11:21.053232+0000","last_fullsized":"2023-10-06T03:11:21.053232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:53.518479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.032323+0000","last_change":"2023-10-06T03:11:20.037224+0000","last_active":"2023-10-06T03:11:21.032323+0000","last_peered":"2023-10-06T03:11:21.032323+0000","last_clean":"2023-10-06T03:11:21.032323+0000","last_became_active":"2023-10-06T03:11:20.036975+0000","last_became_peered":"2023-10-06T03:11:20.036975+0000","last_unstale":"2023-10-06T03:11:21.032323+0000","last_undegraded":"2023-10-06T03:11:21.032323+0000","last_fullsized":"2023-10-06T03:11:21.032323+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:19.014752+0000","last_clean_scrub_stamp":"2023-10-06T03:11:19.014752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:59.762780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"35.26","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.050544+0000","last_change":"2023-10-06T03:11:21.044424+0000","last_active":"2023-10-06T03:11:21.050544+0000","last_peered":"2023-10-06T03:11:21.050544+0000","last_clean":"2023-10-06T03:11:21.050544+0000","last_became_active":"2023-10-06T03:11:21.041040+0000","last_became_peered":"2023-10-06T03:11:21.041040+0000","last_unstale":"2023-10-06T03:11:21.050544+0000","last_undegraded":"2023-10-06T03:11:21.050544+0000","last_fullsized":"2023-10-06T03:11:21.050544+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:05.912082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.039757+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:11:21.039757+0000","last_peered":"2023-10-06T03:11:21.039757+0000","last_clean":"2023-10-06T03:11:21.039757+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:11:21.039757+0000","last_undegraded":"2023-10-06T03:11:21.039757+0000","last_fullsized":"2023-10-06T03:11:21.039757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.27","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.048199+0000","last_change":"2023-10-06T03:11:21.041474+0000","last_active":"2023-10-06T03:11:21.048199+0000","last_peered":"2023-10-06T03:11:21.048199+0000","last_clean":"2023-10-06T03:11:21.048199+0000","last_became_active":"2023-10-06T03:11:21.041222+0000","last_became_peered":"2023-10-06T03:11:21.041222+0000","last_unstale":"2023-10-06T03:11:21.048199+0000","last_undegraded":"2023-10-06T03:11:21.048199+0000","last_fullsized":"2023-10-06T03:11:21.048199+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:01.928404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.033811+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:11:21.033811+0000","last_peered":"2023-10-06T03:11:21.033811+0000","last_clean":"2023-10-06T03:11:21.033811+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:11:21.033811+0000","last_undegraded":"2023-10-06T03:11:21.033811+0000","last_fullsized":"2023-10-06T03:11:21.033811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.24","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.036633+0000","last_change":"2023-10-06T03:11:21.034359+0000","last_active":"2023-10-06T03:11:21.036633+0000","last_peered":"2023-10-06T03:11:21.036633+0000","last_clean":"2023-10-06T03:11:21.036633+0000","last_became_active":"2023-10-06T03:11:21.034086+0000","last_became_peered":"2023-10-06T03:11:21.034086+0000","last_unstale":"2023-10-06T03:11:21.036633+0000","last_undegraded":"2023-10-06T03:11:21.036633+0000","last_fullsized":"2023-10-06T03:11:21.036633+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:28:16.795057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.036867+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:11:21.036867+0000","last_peered":"2023-10-06T03:11:21.036867+0000","last_clean":"2023-10-06T03:11:21.036867+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:11:21.036867+0000","last_undegraded":"2023-10-06T03:11:21.036867+0000","last_fullsized":"2023-10-06T03:11:21.036867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.21","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056258+0000","last_change":"2023-10-06T03:11:21.054421+0000","last_active":"2023-10-06T03:11:21.056258+0000","last_peered":"2023-10-06T03:11:21.056258+0000","last_clean":"2023-10-06T03:11:21.056258+0000","last_became_active":"2023-10-06T03:11:21.053997+0000","last_became_peered":"2023-10-06T03:11:21.053997+0000","last_unstale":"2023-10-06T03:11:21.056258+0000","last_undegraded":"2023-10-06T03:11:21.056258+0000","last_fullsized":"2023-10-06T03:11:21.056258+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:28.776920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.037128+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:11:21.037128+0000","last_peered":"2023-10-06T03:11:21.037128+0000","last_clean":"2023-10-06T03:11:21.037128+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:11:21.037128+0000","last_undegraded":"2023-10-06T03:11:21.037128+0000","last_fullsized":"2023-10-06T03:11:21.037128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.22","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.044564+0000","last_change":"2023-10-06T03:11:21.041320+0000","last_active":"2023-10-06T03:11:21.044564+0000","last_peered":"2023-10-06T03:11:21.044564+0000","last_clean":"2023-10-06T03:11:21.044564+0000","last_became_active":"2023-10-06T03:11:21.041050+0000","last_became_peered":"2023-10-06T03:11:21.041050+0000","last_unstale":"2023-10-06T03:11:21.044564+0000","last_undegraded":"2023-10-06T03:11:21.044564+0000","last_fullsized":"2023-10-06T03:11:21.044564+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:39.849451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":316,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.037196+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:11:21.037196+0000","last_peered":"2023-10-06T03:11:21.037196+0000","last_clean":"2023-10-06T03:11:21.037196+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:11:21.037196+0000","last_undegraded":"2023-10-06T03:11:21.037196+0000","last_fullsized":"2023-10-06T03:11:21.037196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":183,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.032073+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:11:21.032073+0000","last_peered":"2023-10-06T03:11:21.032073+0000","last_clean":"2023-10-06T03:11:21.032073+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:11:21.032073+0000","last_undegraded":"2023-10-06T03:11:21.032073+0000","last_fullsized":"2023-10-06T03:11:21.032073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.20","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.042768+0000","last_change":"2023-10-06T03:11:21.038363+0000","last_active":"2023-10-06T03:11:21.042768+0000","last_peered":"2023-10-06T03:11:21.042768+0000","last_clean":"2023-10-06T03:11:21.042768+0000","last_became_active":"2023-10-06T03:11:21.037574+0000","last_became_peered":"2023-10-06T03:11:21.037574+0000","last_unstale":"2023-10-06T03:11:21.042768+0000","last_undegraded":"2023-10-06T03:11:21.042768+0000","last_fullsized":"2023-10-06T03:11:21.042768+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:14.184611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.029336+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:11:21.029336+0000","last_peered":"2023-10-06T03:11:21.029336+0000","last_clean":"2023-10-06T03:11:21.029336+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:11:21.029336+0000","last_undegraded":"2023-10-06T03:11:21.029336+0000","last_fullsized":"2023-10-06T03:11:21.029336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.23","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.056273+0000","last_change":"2023-10-06T03:11:21.054455+0000","last_active":"2023-10-06T03:11:21.056273+0000","last_peered":"2023-10-06T03:11:21.056273+0000","last_clean":"2023-10-06T03:11:21.056273+0000","last_became_active":"2023-10-06T03:11:21.054193+0000","last_became_peered":"2023-10-06T03:11:21.054193+0000","last_unstale":"2023-10-06T03:11:21.056273+0000","last_undegraded":"2023-10-06T03:11:21.056273+0000","last_fullsized":"2023-10-06T03:11:21.056273+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:25:02.996034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":183,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.030715+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:11:21.030715+0000","last_peered":"2023-10-06T03:11:21.030715+0000","last_clean":"2023-10-06T03:11:21.030715+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:11:21.030715+0000","last_undegraded":"2023-10-06T03:11:21.030715+0000","last_fullsized":"2023-10-06T03:11:21.030715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"35.25","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.053616+0000","last_change":"2023-10-06T03:11:21.047660+0000","last_active":"2023-10-06T03:11:21.053616+0000","last_peered":"2023-10-06T03:11:21.053616+0000","last_clean":"2023-10-06T03:11:21.053616+0000","last_became_active":"2023-10-06T03:11:21.047384+0000","last_became_peered":"2023-10-06T03:11:21.047384+0000","last_unstale":"2023-10-06T03:11:21.053616+0000","last_undegraded":"2023-10-06T03:11:21.053616+0000","last_fullsized":"2023-10-06T03:11:21.053616+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:11:20.020393+0000","last_clean_scrub_stamp":"2023-10-06T03:11:20.020393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:45.177185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-06T03:11:21.033488+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:11:21.033488+0000","last_peered":"2023-10-06T03:11:21.033488+0000","last_clean":"2023-10-06T03:11:21.033488+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:11:21.033488+0000","last_undegraded":"2023-10-06T03:11:21.033488+0000","last_fullsized":"2023-10-06T03:11:21.033488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":35,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304460,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067380,"statfs":{"total":96636764160,"available":96324997120,"internally_reserved":0,"allocated":3149824,"data_stored":1806976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51800000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.76700000000000002,"15min":6.0110000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52000000000000002},"min":{"1min":0.39000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.504}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.51000000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.745,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53200000000000003},"min":{"1min":0.35599999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.746,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.439}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.64100000000000001,"5min":1.264,"15min":1.264},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79800000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.58599999999999997,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.103,"5min":2.286,"15min":2.286},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0349999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54400000000000004,"15min":0.52200000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.73199999999999998,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":2.504,"15min":2.504},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88500000000000001,"5min":1.141,"15min":1.141},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69899999999999995,"5min":1.639,"15min":1.639},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.59399999999999997},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82199999999999995,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.376,"5min":0.307,"15min":0.307},"max":{"1min":0.77800000000000002,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1216698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.91400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.442},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57399999999999995,"15min":0.57099999999999995},"min":{"1min":0.45500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.5,"15min":0.505},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.70799999999999996,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.372,"5min":0.34599999999999997,"15min":0.32100000000000001},"max":{"1min":0.72299999999999998,"5min":0.755,"15min":0.755},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.75,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53600000000000003,"15min":0.52200000000000002},"min":{"1min":0.39500000000000002,"5min":0.33100000000000002,"15min":0.315},"max":{"1min":0.67600000000000005,"5min":0.747,"15min":0.79000000000000004},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.85099999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41199999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.622}]},{"osd":4,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.77600000000000002,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74299999999999999,"5min":0.873,"15min":0.873},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52300000000000002,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72099999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78900000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.41899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1216642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35099999999999998,"15min":0.33400000000000002},"min":{"1min":0.23300000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48199999999999998,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.40200000000000002,"15min":0.38200000000000001},"min":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.497,"5min":3.6520000000000001,"15min":3.6520000000000001},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.34200000000000003,"15min":0.33400000000000002},"min":{"1min":0.25900000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.432,"5min":1.282,"15min":1.282},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33300000000000002,"15min":0.33400000000000002},"min":{"1min":0.247,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.48599999999999999,"5min":0.58199999999999996,"15min":0.60999999999999999},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.27000000000000002,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.45100000000000001,"5min":1.25,"15min":1.25},"last":0.38},{"interface":"front","average":{"1min":0.374,"5min":0.37,"15min":0.36799999999999999},"min":{"1min":0.29099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.47199999999999998,"5min":0.57999999999999996,"15min":0.66000000000000003},"last":0.42299999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":1.1060000000000001,"5min":1.302,"15min":1.302},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.35099999999999998},"max":{"1min":0.83099999999999996,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.378,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.96299999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.36599999999999999},"max":{"1min":1.0629999999999999,"5min":1.363,"15min":5.0270000000000001},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.40799999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.052,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:10:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55800000000000005,"15min":0.56699999999999995},"min":{"1min":0.42499999999999999,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":0.745,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":1.0820000000000001,"5min":1.379,"15min":1.379},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304472,"kb_used_data":3088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067368,"statfs":{"total":96636764160,"available":96324984832,"internally_reserved":0,"allocated":3162112,"data_stored":1807586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31,"15min":0.31},"min":{"1min":0.193,"5min":0.187,"15min":0.187},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.315},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.32700000000000001},"min":{"1min":0.23499999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.33100000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23699999999999999,"15min":0.23200000000000001},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.64100000000000001},"last":0.376},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.35099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.38500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34100000000000003},"min":{"1min":0.27100000000000002,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.37,"15min":0.38300000000000001},"min":{"1min":0.252,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.39700000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35499999999999998},"max":{"1min":0.83799999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.59199999999999997},"min":{"1min":0.377,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.99199999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57199999999999995,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78900000000000003,"5min":0.82899999999999996,"15min":0.92100000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71099999999999997,"5min":0.82199999999999995,"15min":0.94799999999999995},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.93700000000000006},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.59999999999999998},"min":{"1min":0.44400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:10:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58599999999999997,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.57299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1216698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.30499999999999999,"15min":0.317},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.307,"5min":0.33400000000000002,"15min":0.32100000000000001},"min":{"1min":0.25900000000000001,"5min":0.219,"15min":0.20499999999999999},"max":{"1min":0.374,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33600000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.33100000000000002,"15min":0.33000000000000002},"min":{"1min":0.24199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.42099999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.377},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.34699999999999998},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.44500000000000001,"5min":0.504,"15min":0.56599999999999995},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23799999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.435,"5min":0.48499999999999999,"15min":0.54800000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.35899999999999999,"15min":0.375},"min":{"1min":0.28799999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.57699999999999996},"last":0.442}]},{"osd":4,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.69899999999999995,"5min":0.79000000000000004,"15min":0.86299999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58499999999999996,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.374,"15min":0.374},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.57299999999999995,"15min":0.53600000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58499999999999996,"15min":0.55600000000000005},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.56399999999999995},"min":{"1min":0.42199999999999999,"5min":0.38300000000000001,"15min":0.34399999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.60999999999999999},"min":{"1min":0.41299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.903},"last":0.62},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58799999999999997,"15min":0.56200000000000006},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.56599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303892,"kb_used_data":2508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067948,"statfs":{"total":96636764160,"available":96325578752,"internally_reserved":0,"allocated":2568192,"data_stored":1218144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.32500000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.436,"5min":0.503,"15min":0.65200000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33500000000000002,"15min":0.33200000000000002},"min":{"1min":0.29099999999999998,"5min":0.25600000000000001,"15min":0.216},"max":{"1min":0.46000000000000002,"5min":0.49299999999999999,"15min":0.72299999999999998},"last":0.309}]},{"osd":1,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.32400000000000001,"15min":0.313},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.42299999999999999,"5min":0.59599999999999997,"15min":0.69699999999999995},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.35099999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35499999999999998},"min":{"1min":0.27300000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.47399999999999998,"5min":0.63,"15min":0.67600000000000005},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54600000000000004,"15min":0.54500000000000004},"min":{"1min":0.39100000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78800000000000003,"5min":0.79700000000000004,"15min":0.94399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56100000000000005,"15min":0.55300000000000005},"min":{"1min":0.44900000000000001,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.58199999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":0.80800000000000005,"15min":0.997},"last":0.626}]},{"osd":6,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.442,"5min":0.315,"15min":0.315},"max":{"1min":0.67700000000000005,"5min":0.78600000000000003,"15min":0.86499999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":0.82899999999999996,"15min":0.98499999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.45800000000000002,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.75900000000000001,"5min":0.79600000000000004,"15min":1.0660000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.56799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303888,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067952,"statfs":{"total":96636764160,"available":96325582848,"internally_reserved":0,"allocated":2564096,"data_stored":1216700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.50800000000000001},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.73599999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56499999999999995,"15min":0.57199999999999995},"min":{"1min":0.441,"5min":0.307,"15min":0.307},"max":{"1min":1.069,"5min":1.069,"15min":1.4410000000000001},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.53300000000000003},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.31900000000000001},"max":{"1min":0.69999999999999996,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":1.0009999999999999},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51900000000000002,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.315},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":3},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53400000000000003,"15min":0.71599999999999997},"min":{"1min":0.40100000000000002,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":10.084},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.432,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56200000000000006,"15min":0.65000000000000002},"min":{"1min":0.45600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.499,"15min":0.51000000000000001},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.749,"5min":0.752,"15min":0.83999999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54400000000000004,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.54},"last":0.68899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.34799999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.51100000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59099999999999997,"15min":0.60799999999999998},"min":{"1min":0.44600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574972,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1218768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.57099999999999995},"min":{"1min":0.36499999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":1.143,"15min":2.3599999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.59199999999999997},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.28100000000000003},"max":{"1min":0.73199999999999998,"5min":0.86299999999999999,"15min":2.371},"last":0.628}]},{"osd":1,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51300000000000001,"15min":0.53300000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82499999999999996,"15min":2.3410000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.43099999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":0.83999999999999997,"5min":4.1219999999999999,"15min":4.1219999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.59299999999999997},"min":{"1min":0.39500000000000002,"5min":0.35099999999999998,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":6.532,"15min":6.532},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.58599999999999997,"15min":0.67600000000000005},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.72899999999999998,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.65900000000000003},"min":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":0.85299999999999998,"5min":0.90000000000000002,"15min":5.4630000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.38100000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.86099999999999999,"5min":5.524,"15min":5.524},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50600000000000001,"15min":0.51800000000000002},"min":{"1min":0.316,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.54200000000000004,"15min":0.57099999999999995},"min":{"1min":0.43099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51700000000000002,"15min":0.55000000000000004},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64100000000000001,"5min":0.78200000000000003,"15min":1.1990000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.44900000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.052},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57599999999999996,"15min":0.61199999999999999},"min":{"1min":0.373,"5min":0.371,"15min":0.34699999999999998},"max":{"1min":0.77000000000000002,"5min":0.878,"15min":1.175},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56399999999999995,"15min":0.59799999999999998},"min":{"1min":0.39600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.145},"last":0.52300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:11:25.479 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:11:25.479 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:11:25.633 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.632+0000 7f0a5359e700 1 Processor -- start 2023-10-06T03:11:25.634 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.632+0000 7f0a5359e700 1 -- start start 2023-10-06T03:11:25.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 0x7f0a5406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.635 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5359e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0a541577c0 0x7f0a5406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5359e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5415ad70 con 0x7f0a54068390 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5359e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540c8610 con 0x7f0a541577c0 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5415a160 con 0x7f0a541573f0 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 0x7f0a5406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a51d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0a541577c0 0x7f0a5406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 0x7f0a5406c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:33950/0 (socket says 172.21.15.92:33950) 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a51d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0a541577c0 0x7f0a5406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:33740/0 (socket says 172.21.15.92:33740) 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a52d9d700 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f0a54068390 0x7f0a5415d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:41674/0 (socket says 172.21.15.92:41674) 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 -- 172.21.15.92:0/3867695660 learned_addr learned my addr 172.21.15.92:0/3867695660 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 -- 172.21.15.92:0/3867695660 >> v2:172.21.15.136:3300/0 conn(0x7f0a541577c0 msgr2=0x7f0a5406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:25.636 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 --2- 172.21.15.92:0/3867695660 >> v2:172.21.15.136:3300/0 conn(0x7f0a541577c0 0x7f0a5406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 -- 172.21.15.92:0/3867695660 >> v1:172.21.15.136:6789/0 conn(0x7f0a54068390 legacy=0x7f0a5415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.633+0000 7f0a5259c700 1 -- 172.21.15.92:0/3867695660 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5415a440 con 0x7f0a541573f0 2023-10-06T03:11:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.634+0000 7f0a5259c700 1 --2- 172.21.15.92:0/3867695660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 0x7f0a5406c6b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a440099b0 tx=0x7f0a44009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c684086af44609 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.634+0000 7f0a5159a700 1 -- 172.21.15.92:0/3867695660 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44009ea0 con 0x7f0a541573f0 2023-10-06T03:11:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.634+0000 7f0a5159a700 1 -- 172.21.15.92:0/3867695660 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400b860 con 0x7f0a541573f0 2023-10-06T03:11:25.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.634+0000 7f0a5159a700 1 -- 172.21.15.92:0/3867695660 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400bc00 con 0x7f0a541573f0 2023-10-06T03:11:25.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.634+0000 7f0a5359e700 1 -- 172.21.15.92:0/3867695660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 msgr2=0x7f0a5406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.634+0000 7f0a5359e700 1 --2- 172.21.15.92:0/3867695660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 0x7f0a5406c6b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0a440099b0 tx=0x7f0a44009250 comp rx=0 tx=0).stop 2023-10-06T03:11:25.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.635+0000 7f0a5359e700 1 -- 172.21.15.92:0/3867695660 shutdown_connections 2023-10-06T03:11:25.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.635+0000 7f0a5359e700 1 --2- 172.21.15.92:0/3867695660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541573f0 0x7f0a5406c6b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.635+0000 7f0a5359e700 1 --2- 172.21.15.92:0/3867695660 >> v2:172.21.15.136:3300/0 conn(0x7f0a541577c0 0x7f0a5406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.635+0000 7f0a5359e700 1 -- 172.21.15.92:0/3867695660 >> 172.21.15.92:0/3867695660 conn(0x7f0a540cddc0 msgr2=0x7f0a540c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.635+0000 7f0a5359e700 1 -- 172.21.15.92:0/3867695660 shutdown_connections 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.635+0000 7f0a5359e700 1 -- 172.21.15.92:0/3867695660 wait complete. 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 Processor -- start 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 -- start start 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a54068390 0x7f0a54169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 0x7f0a541532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541577c0 0x7f0a540ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540a98d0 con 0x7f0a541573f0 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a541537f0 con 0x7f0a54068390 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.636+0000 7f0a5359e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540abc90 con 0x7f0a541577c0 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 0x7f0a541532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 0x7f0a541532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59734/0 (socket says 172.21.15.92:59734) 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 -- 172.21.15.92:0/1654804665 learned_addr learned my addr 172.21.15.92:0/1654804665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:25.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a52d9d700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541577c0 0x7f0a540ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a5259c700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a54068390 0x7f0a54169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 -- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541577c0 msgr2=0x7f0a540ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541577c0 0x7f0a540ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 -- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a54068390 msgr2=0x7f0a54169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a54068390 0x7f0a54169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 -- 172.21.15.92:0/1654804665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440094f0 con 0x7f0a541573f0 2023-10-06T03:11:25.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a5259c700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a54068390 0x7f0a54169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a51d9b700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 0x7f0a541532b0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800e900 tx=0x7f0a480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48013680 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a540abf70 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a480137e0 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48013b80 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.637+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a540ac530 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.638+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a540a9a40 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.638+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0a4800c460 con 0x7f0a541573f0 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.639+0000 7f0a427fc700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a34067320 0x7f0a340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:25.645 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.639+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0a48084ed0 con 0x7f0a541573f0 2023-10-06T03:11:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.639+0000 7f0a5259c700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a34067320 0x7f0a340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.639+0000 7f0a5259c700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a34067320 0x7f0a340697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0a440099b0 tx=0x7f0a44017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:25.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.643+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a48051a00 con 0x7f0a541573f0 2023-10-06T03:11:25.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.903+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0a5404c3c0 con 0x7f0a541573f0 2023-10-06T03:11:25.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.907+0000 7f0a427fc700 1 -- 172.21.15.92:0/1654804665 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0a480170e0 con 0x7f0a541573f0 2023-10-06T03:11:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a34067320 msgr2=0x7f0a340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a34067320 0x7f0a340697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0a440099b0 tx=0x7f0a44017000 comp rx=0 tx=0).stop 2023-10-06T03:11:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 msgr2=0x7f0a541532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 0x7f0a541532b0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800e900 tx=0x7f0a480091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.912 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 shutdown_connections 2023-10-06T03:11:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0a34067320 0x7f0a340697d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a541577c0 0x7f0a540ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a54068390 0x7f0a54169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 --2- 172.21.15.92:0/1654804665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a541573f0 0x7f0a541532b0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.909+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 >> 172.21.15.92:0/1654804665 conn(0x7f0a540cddc0 msgr2=0x7f0a541563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:25.913 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.910+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 shutdown_connections 2023-10-06T03:11:25.914 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:25.910+0000 7f0a5359e700 1 -- 172.21.15.92:0/1654804665 wait complete. 2023-10-06T03:11:25.914 INFO:teuthology.orchestra.run.smithi092.stderr:updated caps for client.0 2023-10-06T03:11:25.923 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:11:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.036+0000 7f08f7a48700 1 Processor -- start 2023-10-06T03:11:26.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.036+0000 7f08f7a48700 1 -- start start 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f7a48700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08f0159890 0x7f08f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f7a48700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f015a160 con 0x7f08f0159c60 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f7a48700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f00d13a0 con 0x7f08f0069560 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f7a48700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f015e140 con 0x7f08f0159890 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f57e4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08f0159890 0x7f08f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f4fe3700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f08f0159c60 0x7f08f015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50482/0 (socket says 172.21.15.92:50482) 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f57e4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f08f0159890 0x7f08f015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59750/0 (socket says 172.21.15.92:59750) 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f4fe3700 1 -- 172.21.15.92:0/1655170892 learned_addr learned my addr 172.21.15.92:0/1655170892 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:26.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08e7fff700 1 -- 172.21.15.92:0/1655170892 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342115690 0 0) 0x7f08f015a160 con 0x7f08f0159c60 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08e7fff700 1 -- 172.21.15.92:0/1655170892 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc003570 con 0x7f08f0159c60 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f57e4700 1 -- 172.21.15.92:0/1655170892 >> v1:172.21.15.136:6789/0 conn(0x7f08f0069560 legacy=0x7f08f00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f57e4700 1 -- 172.21.15.92:0/1655170892 >> v1:172.21.15.92:6789/0 conn(0x7f08f0159c60 legacy=0x7f08f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f57e4700 1 -- 172.21.15.92:0/1655170892 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f015a3d0 con 0x7f08f0159890 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08f57e4700 1 --2- 172.21.15.92:0/1655170892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f015abd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009e60 tx=0x7f08ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dfa80963a42ce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.037+0000 7f08e7fff700 1 -- 172.21.15.92:0/1655170892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00e070 con 0x7f08f0159890 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08e7fff700 1 -- 172.21.15.92:0/1655170892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec00bb60 con 0x7f08f0159890 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08e7fff700 1 -- 172.21.15.92:0/1655170892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec012430 con 0x7f08f0159890 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 -- 172.21.15.92:0/1655170892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 msgr2=0x7f08f015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 --2- 172.21.15.92:0/1655170892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f015abd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009e60 tx=0x7f08ec009780 comp rx=0 tx=0).stop 2023-10-06T03:11:26.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 -- 172.21.15.92:0/1655170892 shutdown_connections 2023-10-06T03:11:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 --2- 172.21.15.92:0/1655170892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f015abd0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 -- 172.21.15.92:0/1655170892 >> 172.21.15.92:0/1655170892 conn(0x7f08f0068e20 msgr2=0x7f08f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 -- 172.21.15.92:0/1655170892 shutdown_connections 2023-10-06T03:11:26.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.038+0000 7f08f7a48700 1 -- 172.21.15.92:0/1655170892 wait complete. 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.039+0000 7f08f7a48700 1 Processor -- start 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.039+0000 7f08f7a48700 1 -- start start 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f7a48700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08f0069560 0x7f08f015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f7a48700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f7a48700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f0159c60 0x7f08f014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f7a48700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b4980 con 0x7f08f0159890 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f7a48700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b3ad0 con 0x7f08f0159c60 2023-10-06T03:11:26.044 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f7a48700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f00b3d70 con 0x7f08f0069560 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f4fe3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f4fe3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59754/0 (socket says 172.21.15.92:59754) 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.040+0000 7f08f4fe3700 1 -- 172.21.15.92:0/3872816490 learned_addr learned my addr 172.21.15.92:0/3872816490 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f4fe3700 1 -- 172.21.15.92:0/3872816490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08f0069560 msgr2=0x7f08f015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f4fe3700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08f0069560 0x7f08f015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f4fe3700 1 -- 172.21.15.92:0/3872816490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f0159c60 msgr2=0x7f08f014d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f4fe3700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f0159c60 0x7f08f014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f4fe3700 1 -- 172.21.15.92:0/3872816490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec009a20 con 0x7f08f0159890 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f4fe3700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f00b3590 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c590 tx=0x7f08e0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0005b30 con 0x7f08f0159890 2023-10-06T03:11:26.045 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0005cb0 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f00b4050 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0007320 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.041+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f00b4610 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.042+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f004c3c0 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.042+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f08e001b3b0 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.042+0000 7f08e57fa700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08d4067320 0x7f08d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.043+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f08e00843f0 con 0x7f08f0159890 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.043+0000 7f08f57e4700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08d4067320 0x7f08d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.046 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.043+0000 7f08f57e4700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08d4067320 0x7f08d40697d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097c0 tx=0x7f08ec019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.048 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.047+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08e0050f20 con 0x7f08f0159890 2023-10-06T03:11:26.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.276+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08f00b4ac0 con 0x7f08f0159890 2023-10-06T03:11:26.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.277+0000 7f08e57fa700 1 -- 172.21.15.92:0/3872816490 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f08e0054540 con 0x7f08f0159890 2023-10-06T03:11:26.278 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:11:26.279 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":136,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:11:20.409587+0000","modified":"2023-10-06T03:11:24.052115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3455480640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3455480640},{"type":"v1","addr":"172.21.15.92:6837","nonce":3455480640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T03:11:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08d4067320 msgr2=0x7f08d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08d4067320 0x7f08d40697d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097c0 tx=0x7f08ec019000 comp rx=0 tx=0).stop 2023-10-06T03:11:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 msgr2=0x7f08f00b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f00b3590 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c590 tx=0x7f08e0007e00 comp rx=0 tx=0).stop 2023-10-06T03:11:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 shutdown_connections 2023-10-06T03:11:26.281 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f08d4067320 0x7f08d40697d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f0159c60 0x7f08f014d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08f0069560 0x7f08f015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 --2- 172.21.15.92:0/3872816490 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08f0159890 0x7f08f00b3590 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 >> 172.21.15.92:0/3872816490 conn(0x7f08f0068e20 msgr2=0x7f08f00cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 shutdown_connections 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.279+0000 7f08f7a48700 1 -- 172.21.15.92:0/3872816490 wait complete. 2023-10-06T03:11:26.282 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 136 2023-10-06T03:11:26.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:11:20.409587+0000', 'modified': '2023-10-06T03:11:24.052115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9198}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9198': {'gid': 9198, 'name': 'c', 'rank': 0, 'incarnation': 132, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.92:6837/3455480640', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6835', 'nonce': 3455480640}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 3455480640}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [35], 'metadata_pool': 34, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:11:26.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:11:26.695 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:11:26.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.811+0000 7fe13d3ac700 1 Processor -- start 2023-10-06T03:11:26.813 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.812+0000 7fe13d3ac700 1 -- start start 2023-10-06T03:11:26.815 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.812+0000 7fe13d3ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe1380cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.812+0000 7fe13d3ac700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1380cbb30 0x7fe1380ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.812+0000 7fe13d3ac700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13806a220 con 0x7fe1381599c0 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.812+0000 7fe13d3ac700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1380cbf60 con 0x7fe1380cbb30 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.812+0000 7fe13d3ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe138069c70 con 0x7fe13815ab90 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1380cbb30 0x7fe1380ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe12e7fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe1380cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1380cbb30 0x7fe1380ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:33768/0 (socket says 172.21.15.92:33768) 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe12e7fc700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe1380cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:33988/0 (socket says 172.21.15.92:33988) 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 -- 172.21.15.92:0/1311384651 learned_addr learned my addr 172.21.15.92:0/1311384651 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe136ffd700 1 --1- 172.21.15.92:0/1311384651 >> v1:172.21.15.136:6789/0 conn(0x7fe1381599c0 0x7fe138069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:41700/0 (socket says 172.21.15.92:41700) 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 -- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 msgr2=0x7fe1380cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 --2- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe1380cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 -- 172.21.15.92:0/1311384651 >> v1:172.21.15.136:6789/0 conn(0x7fe1381599c0 legacy=0x7fe138069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 -- 172.21.15.92:0/1311384651 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe138069f40 con 0x7fe1380cbb30 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1367fc700 1 -- 172.21.15.92:0/1311384651 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608561133 0 0) 0x7fe13806a220 con 0x7fe1381599c0 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1377fe700 1 --2- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1380cbb30 0x7fe1380ce2e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe130009f20 tx=0x7fe130009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b26c059cdcd2ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1367fc700 1 -- 172.21.15.92:0/1311384651 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000e070 con 0x7fe1380cbb30 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1367fc700 1 -- 172.21.15.92:0/1311384651 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000b9d0 con 0x7fe1380cbb30 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.813+0000 7fe1367fc700 1 -- 172.21.15.92:0/1311384651 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300092e0 con 0x7fe1380cbb30 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 -- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1380cbb30 msgr2=0x7fe1380ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1380cbb30 0x7fe1380ce2e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe130009f20 tx=0x7fe130009840 comp rx=0 tx=0).stop 2023-10-06T03:11:26.817 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 -- 172.21.15.92:0/1311384651 shutdown_connections 2023-10-06T03:11:26.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1380cbb30 0x7fe1380ce2e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/1311384651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe1380cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 -- 172.21.15.92:0/1311384651 >> 172.21.15.92:0/1311384651 conn(0x7fe138068e10 msgr2=0x7fe1380d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 -- 172.21.15.92:0/1311384651 shutdown_connections 2023-10-06T03:11:26.818 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.814+0000 7fe13d3ac700 1 -- 172.21.15.92:0/1311384651 wait complete. 2023-10-06T03:11:26.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.815+0000 7fe13d3ac700 1 Processor -- start 2023-10-06T03:11:26.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.815+0000 7fe13d3ac700 1 -- start start 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe13d3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 0x7fe138152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe13d3ac700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1381599c0 0x7fe1380c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe13d3ac700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe138154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe13d3ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13816dd30 con 0x7fe1380cbb30 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe13d3ac700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13816d120 con 0x7fe1381599c0 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe13d3ac700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13816d400 con 0x7fe13815ab90 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 0x7fe138152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 0x7fe138152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59758/0 (socket says 172.21.15.92:59758) 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 -- 172.21.15.92:0/3116511910 learned_addr learned my addr 172.21.15.92:0/3116511910 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1381599c0 0x7fe1380c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe1377fe700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe138154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 -- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 msgr2=0x7fe138154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe138154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 -- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1381599c0 msgr2=0x7fe1380c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1381599c0 0x7fe1380c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe136ffd700 1 -- 172.21.15.92:0/3116511910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009ae0 con 0x7fe1380cbb30 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.816+0000 7fe1377fe700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe138154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.817+0000 7fe136ffd700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 0x7fe138152610 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe1280011e0 tx=0x7fe128007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.817+0000 7fe12f7fe700 1 -- 172.21.15.92:0/3116511910 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128007300 con 0x7fe1380cbb30 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.817+0000 7fe12f7fe700 1 -- 172.21.15.92:0/3116511910 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe128007460 con 0x7fe1380cbb30 2023-10-06T03:11:26.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.817+0000 7fe12f7fe700 1 -- 172.21.15.92:0/3116511910 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128004830 con 0x7fe1380cbb30 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.817+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13816d6e0 con 0x7fe1380cbb30 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.817+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe138152b50 con 0x7fe1380cbb30 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.818+0000 7fe12f7fe700 1 -- 172.21.15.92:0/3116511910 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe128004be0 con 0x7fe1380cbb30 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.818+0000 7fe12f7fe700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe118067320 0x7fe1180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.819+0000 7fe12f7fe700 1 -- 172.21.15.92:0/3116511910 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe128084310 con 0x7fe1380cbb30 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.819+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe118067320 0x7fe1180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.819+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe118067320 0x7fe1180697d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1380af2a0 tx=0x7fe120008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.820+0000 7fe1466a4b80 1 Processor -- start 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.820+0000 7fe1466a4b80 1 -- start start 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea29c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea7a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8e980a70 con 0x55ca8ea79c00 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8e97e270 con 0x55ca8ea2acc0 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca8e97a060 con 0x55ca8ea29840 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea7a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea7a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59768/0 (socket says 172.21.15.92:59768) 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe1377fe700 1 -- 172.21.15.92:0/1756133327 learned_addr learned my addr 172.21.15.92:0/1756133327 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.821+0000 7fe136ffd700 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea29c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1377fe700 1 -- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 msgr2=0x55ca8ea29c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1377fe700 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea29c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1377fe700 1 -- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 msgr2=0x55ca8ea796c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1377fe700 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea796c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1377fe700 1 -- 172.21.15.92:0/1756133327 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8e814270 con 0x55ca8ea79c00 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe136ffd700 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea29c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1377fe700 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea7a050 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe138067b80 tx=0x7fe130018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59e4ea9e2073729 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe117fff700 1 -- 172.21.15.92:0/1756133327 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300032f0 con 0x55ca8ea79c00 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe117fff700 1 -- 172.21.15.92:0/1756133327 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000bb40 con 0x55ca8ea79c00 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe117fff700 1 -- 172.21.15.92:0/1756133327 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13001e640 con 0x55ca8ea79c00 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1466a4b80 1 -- 172.21.15.92:0/1756133327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 msgr2=0x55ca8ea7a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.822+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea7a050 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe138067b80 tx=0x7fe130018000 comp rx=0 tx=0).stop 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 -- 172.21.15.92:0/1756133327 shutdown_connections 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea7a050 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea29c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/1756133327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea796c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 -- 172.21.15.92:0/1756133327 >> 172.21.15.92:0/1756133327 conn(0x55ca8ea2b160 msgr2=0x55ca8ea2b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 -- 172.21.15.92:0/1756133327 shutdown_connections 2023-10-06T03:11:26.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 -- 172.21.15.92:0/1756133327 wait complete. 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 -- start start 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.823+0000 7fe1466a4b80 1 Processor -- start 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea29840 0x55ca8ea4eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea79c00 0x55ca8ea69990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8e925890 con 0x55ca8ea2acc0 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8e940ea0 con 0x55ca8ea29840 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca8e7fce30 con 0x55ca8ea79c00 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1466a4b80 10 client.0 _ll_register_callbacks cb 0x55ca8e9ece70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe12e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe12e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59772/0 (socket says 172.21.15.92:59772) 2023-10-06T03:11:26.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe12e7fc700 1 -- 172.21.15.92:0/2358913464 learned_addr learned my addr 172.21.15.92:0/2358913464 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe136ffd700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea29840 0x55ca8ea4eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.824+0000 7fe1377fe700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea79c00 0x55ca8ea69990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe12e7fc700 1 -- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea79c00 msgr2=0x55ca8ea69990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea79c00 0x55ca8ea69990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe12e7fc700 1 -- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea29840 msgr2=0x55ca8ea4eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea29840 0x55ca8ea4eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe12e7fc700 1 -- 172.21.15.92:0/2358913464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe138069f40 con 0x55ca8ea2acc0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fe1380cb160 tx=0x7fe120018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bf3f833ac90086 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1157fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1157fa700 1 -- 172.21.15.92:0/2358913464 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120022420 con 0x55ca8ea2acc0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1157fa700 1 -- 172.21.15.92:0/2358913464 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe120022580 con 0x55ca8ea2acc0 2023-10-06T03:11:26.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca8ea1e940 con 0x55ca8ea2acc0 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1157fa700 1 -- 172.21.15.92:0/2358913464 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe120022950 con 0x55ca8ea2acc0 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 10 client.9333 fetch_fsmap learned FSMap version 136 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55ca8e7b6fe0 con 0x55ca8ea2acc0 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1157fa700 1 -- 172.21.15.92:0/2358913464 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8ea1e940 con 0x55ca8ea2acc0 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1157fa700 1 -- 172.21.15.92:0/2358913464 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 136) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe120022ab0 con 0x55ca8ea2acc0 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 10 client.9333 fetch_fsmap finished waiting for FSMap version 136 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 10 client.9333 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 20 client.9333 populate_metadata read hostname 'smithi092' 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea6c370 0x55ca8ea6c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 4 client.9333 mds_command: new command op to 9198 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:11:26.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.825+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 --> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca8e9fb870 con 0x55ca8ea6c370 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe136ffd700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea6c370 0x55ca8ea6c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe1157fa700 10 client.9333 ms_handle_connect on v2:172.21.15.92:6835/3455480640 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe136ffd700 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea6c370 0x55ca8ea6c740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f9d73581594b80 server_cookie=f6d597b6a6661ad in_seq=0 out_seq=0 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe1157fa700 1 -- 172.21.15.92:0/2358913464 <== mds.0 v2:172.21.15.92:6835/3455480640 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ca8e9fb870 con 0x55ca8ea6c370 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe1157fa700 10 client.9333 handle_command_reply: tid=0 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe1466a4b80 1 client.9333 shutdown 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 msgr2=0x55ca8ea4e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.826+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fe1380cb160 tx=0x7fe120018000 comp rx=0 tx=0).stop 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 shutdown_connections 2023-10-06T03:11:26.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea79c00 0x55ca8ea69990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea29840 0x55ca8ea4eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea6c370 0x55ca8ea6c740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2358913464 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1157fa700 0 client.9333 ms_handle_reset on v2:172.21.15.92:6835/3455480640 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 >> 172.21.15.92:0/2358913464 conn(0x55ca8ea2b160 msgr2=0x55ca8ea2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 shutdown_connections 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2358913464 wait complete. 2023-10-06T03:11:26.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.827+0000 7fe1466a4b80 20 client.9333 trim_cache size 0 max 16384 2023-10-06T03:11:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.843+0000 7fe1466a4b80 1 Processor -- start 2023-10-06T03:11:26.845 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.843+0000 7fe1466a4b80 1 -- start start 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea6a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea4eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8ea29f70 con 0x55ca8ea79c00 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8ea092e0 con 0x55ca8ea2acc0 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca8ea22310 con 0x55ca8ea29840 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea4eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea4eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59788/0 (socket says 172.21.15.92:59788) 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1377fe700 1 -- 172.21.15.92:0/3665077410 learned_addr learned my addr 172.21.15.92:0/3665077410 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe136ffd700 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea6a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1377fe700 1 -- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 msgr2=0x55ca8ea6a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1377fe700 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea6a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.844+0000 7fe1377fe700 1 -- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 msgr2=0x55ca8ea4e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1377fe700 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1377fe700 1 -- 172.21.15.92:0/3665077410 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8e948780 con 0x55ca8ea79c00 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1377fe700 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea4eaf0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe13000b610 tx=0x7fe13001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef854f5c74648c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe12cff9700 1 -- 172.21.15.92:0/3665077410 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000bd80 con 0x55ca8ea79c00 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe12cff9700 1 -- 172.21.15.92:0/3665077410 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13002e040 con 0x55ca8ea79c00 2023-10-06T03:11:26.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe12cff9700 1 -- 172.21.15.92:0/3665077410 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000bee0 con 0x55ca8ea79c00 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 -- 172.21.15.92:0/3665077410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 msgr2=0x55ca8ea4eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea4eaf0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fe13000b610 tx=0x7fe13001f000 comp rx=0 tx=0).stop 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 -- 172.21.15.92:0/3665077410 shutdown_connections 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ea4eaf0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea6a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/3665077410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 -- 172.21.15.92:0/3665077410 >> 172.21.15.92:0/3665077410 conn(0x55ca8ea2b160 msgr2=0x55ca8ea68360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 -- 172.21.15.92:0/3665077410 shutdown_connections 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.845+0000 7fe1466a4b80 1 -- 172.21.15.92:0/3665077410 wait complete. 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.846+0000 7fe1466a4b80 1 -- start start 2023-10-06T03:11:26.850 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.846+0000 7fe1466a4b80 1 Processor -- start 2023-10-06T03:11:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ead5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.854 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8ead5fb0 con 0x55ca8ea79c00 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8ea6bf40 con 0x55ca8ea2acc0 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca8ea6c180 con 0x55ca8ea29840 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1466a4b80 10 client.0 _ll_register_callbacks cb 0x55ca8e9ece70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ead5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ead5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59802/0 (socket says 172.21.15.92:59802) 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 -- 172.21.15.92:0/2497744120 learned_addr learned my addr 172.21.15.92:0/2497744120 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe12e7fc700 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 -- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 msgr2=0x55ca8ea4e5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea4e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 -- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 msgr2=0x55ca8ea4eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.855 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.847+0000 7fe1377fe700 1 -- 172.21.15.92:0/2497744120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca8e814270 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1377fe700 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ead5390 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe130009880 tx=0x7fe13002a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62198981ecc0b0cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1157fa700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1157fa700 1 -- 172.21.15.92:0/2497744120 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130034ae0 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1157fa700 1 -- 172.21.15.92:0/2497744120 <== mon.0 v2:172.21.15.92:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe130034c40 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe13000bb40 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1157fa700 1 -- 172.21.15.92:0/2497744120 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe130034e00 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 10 client.9339 fetch_fsmap learned FSMap version 136 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x55ca8e92f470 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1157fa700 1 -- 172.21.15.92:0/2497744120 <== mon.0 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13000bb40 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1157fa700 1 -- 172.21.15.92:0/2497744120 <== mon.0 v2:172.21.15.92:3300/0 5 ==== fsmap(e 136) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe130024030 con 0x55ca8ea79c00 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 10 client.9339 fetch_fsmap finished waiting for FSMap version 136 2023-10-06T03:11:26.856 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 10 client.9339 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 20 client.9339 populate_metadata read hostname 'smithi092' 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea4c370 0x55ca8ea4c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 4 client.9339 mds_command: new command op to 9198 tid=0[{"prefix": "status"}] 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 --> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca8e91ce00 con 0x55ca8ea4c370 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.848+0000 7fe136ffd700 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea4c370 0x55ca8ea4c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1157fa700 10 client.9339 ms_handle_connect on v2:172.21.15.92:6835/3455480640 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe136ffd700 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea4c370 0x55ca8ea4c740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5733ef7ac8e111 server_cookie=8f1c505f5b527872 in_seq=0 out_seq=0 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1157fa700 1 -- 172.21.15.92:0/2497744120 <== mds.0 v2:172.21.15.92:6835/3455480640 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55ca8e91ce00 con 0x55ca8ea4c370 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1157fa700 10 client.9339 handle_command_reply: tid=0 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 client.9339 shutdown 2023-10-06T03:11:26.857 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 msgr2=0x55ca8ead5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ead5390 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe130009880 tx=0x7fe13002a070 comp rx=0 tx=0).stop 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 shutdown_connections 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:6835/3455480640,v1:172.21.15.92:6837/3455480640] conn(0x55ca8ea4c370 0x55ca8ea4c740 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ca8ea79c00 0x55ca8ead5390 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55ca8ea29840 0x55ca8ea4e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 --2- 172.21.15.92:0/2497744120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ca8ea2acc0 0x55ca8ea4eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1157fa700 0 client.9339 ms_handle_reset on v2:172.21.15.92:6835/3455480640 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 >> 172.21.15.92:0/2497744120 conn(0x55ca8ea2b160 msgr2=0x55ca8ea2cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 shutdown_connections 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 1 -- 172.21.15.92:0/2497744120 wait complete. 2023-10-06T03:11:26.858 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.849+0000 7fe1466a4b80 20 client.9339 trim_cache size 0 max 16384 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "cluster_fsid": "e5cba154-9a64-4fb6-add9-5855ed5b5b5a", 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "whoami": 0, 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "id": 9198, 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "want_state": "up:active", 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "state": "up:active", 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "fs_name": "cephfs", 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "rank_uptime": 5.8236795600000004, 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "mdsmap_epoch": 136, 2023-10-06T03:11:26.859 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch": 201, 2023-10-06T03:11:26.860 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_epoch_barrier": 201, 2023-10-06T03:11:26.860 INFO:teuthology.orchestra.run.smithi092.stdout: "uptime": 11.941130241 2023-10-06T03:11:26.860 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe118067320 msgr2=0x7fe1180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe118067320 0x7fe1180697d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe1380af2a0 tx=0x7fe120008040 comp rx=0 tx=0).stop 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 msgr2=0x7fe138152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 0x7fe138152610 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe1280011e0 tx=0x7fe128007e10 comp rx=0 tx=0).stop 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 shutdown_connections 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe118067320 0x7fe1180697d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1380cbb30 0x7fe138152610 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe13815ab90 0x7fe138154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.861 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 --2- 172.21.15.92:0/3116511910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1381599c0 0x7fe1380c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 >> 172.21.15.92:0/3116511910 conn(0x7fe138068e10 msgr2=0x7fe138152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 shutdown_connections 2023-10-06T03:11:26.862 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.850+0000 7fe13d3ac700 1 -- 172.21.15.92:0/3116511910 wait complete. 2023-10-06T03:11:26.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:11:26.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.976+0000 7f939a4ce700 1 Processor -- start 2023-10-06T03:11:26.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.976+0000 7f939a4ce700 1 -- start start 2023-10-06T03:11:26.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f939a4ce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f939a4ce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93940ae710 0x7f93940c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f939a4ce700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93940ca860 con 0x7f93941566e0 2023-10-06T03:11:26.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f939a4ce700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93940cab40 con 0x7f93940ae710 2023-10-06T03:11:26.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f939a4ce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93940cae20 con 0x7f9394156310 2023-10-06T03:11:26.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93940ae710 0x7f93940c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9398ccb700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f93941566e0 0x7f93940ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50484/0 (socket says 172.21.15.92:50484) 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93940ae710 0x7f93940c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59812/0 (socket says 172.21.15.92:59812) 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9398ccb700 1 -- 172.21.15.92:0/4230572947 learned_addr learned my addr 172.21.15.92:0/4230572947 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f93994cc700 1 --2- 172.21.15.92:0/4230572947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 -- 172.21.15.92:0/4230572947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 msgr2=0x7f93940c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 --2- 172.21.15.92:0/4230572947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 -- 172.21.15.92:0/4230572947 >> v1:172.21.15.92:6789/0 conn(0x7f93941566e0 legacy=0x7f93940ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f938bfff700 1 -- 172.21.15.92:0/4230572947 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292949157 0 0) 0x7f93940ca860 con 0x7f93941566e0 2023-10-06T03:11:26.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 -- 172.21.15.92:0/4230572947 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940cb100 con 0x7f93940ae710 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f9399ccd700 1 --2- 172.21.15.92:0/4230572947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93940ae710 0x7f93940c6830 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f93900098e0 tx=0x7f93900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214c75bac0591cdd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.977+0000 7f938bfff700 1 -- 172.21.15.92:0/4230572947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390009e80 con 0x7f93940ae710 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f938bfff700 1 -- 172.21.15.92:0/4230572947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939000b860 con 0x7f93940ae710 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f938bfff700 1 -- 172.21.15.92:0/4230572947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939000bbe0 con 0x7f93940ae710 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 -- 172.21.15.92:0/4230572947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93940ae710 msgr2=0x7f93940c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 --2- 172.21.15.92:0/4230572947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93940ae710 0x7f93940c6830 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f93900098e0 tx=0x7f93900091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 -- 172.21.15.92:0/4230572947 shutdown_connections 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 --2- 172.21.15.92:0/4230572947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 --2- 172.21.15.92:0/4230572947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93940ae710 0x7f93940c6830 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 -- 172.21.15.92:0/4230572947 >> 172.21.15.92:0/4230572947 conn(0x7f93940cc1f0 msgr2=0x7f93940cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:26.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 -- 172.21.15.92:0/4230572947 shutdown_connections 2023-10-06T03:11:26.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.978+0000 7f939a4ce700 1 -- 172.21.15.92:0/4230572947 wait complete. 2023-10-06T03:11:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.979+0000 7f939a4ce700 1 Processor -- start 2023-10-06T03:11:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.979+0000 7f939a4ce700 1 -- start start 2023-10-06T03:11:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f939a4ce700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93940ae710 0x7f939413e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f939a4ce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f939a4ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 0x7f93940b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f939a4ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939414d240 con 0x7f93941566e0 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f939a4ce700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93940b1e30 con 0x7f93940ae710 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f939a4ce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939414b020 con 0x7f9394156310 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 0x7f93940b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 0x7f93940b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59828/0 (socket says 172.21.15.92:59828) 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 -- 172.21.15.92:0/2906504017 learned_addr learned my addr 172.21.15.92:0/2906504017 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f93994cc700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93940ae710 0x7f939413e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9398ccb700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 -- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 msgr2=0x7f93940b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 -- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93940ae710 msgr2=0x7f939413e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:26.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93940ae710 0x7f939413e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f9399ccd700 1 -- 172.21.15.92:0/2906504017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9390009420 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.980+0000 7f93994cc700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93940ae710 0x7f939413e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.981+0000 7f9399ccd700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 0x7f93940b17b0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f939000b3c0 tx=0x7f9390017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.981+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939000bba0 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.981+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390011630 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.981+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939413eaf0 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.981+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390011a10 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.981+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939414a9d0 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.982+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939404c3c0 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.982+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f93900253f0 con 0x7f93941566e0 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.983+0000 7f93897fa700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9378067320 0x7f93780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:26.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.983+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f93900867d0 con 0x7f93941566e0 2023-10-06T03:11:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.983+0000 7f93994cc700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9378067320 0x7f93780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.983+0000 7f93994cc700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9378067320 0x7f93780697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f938c0075e0 tx=0x7f938c0097e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:26.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:26.987+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9390053400 con 0x7f93941566e0 2023-10-06T03:11:27.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.213+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f939414b160 con 0x7f93941566e0 2023-10-06T03:11:27.215 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.214+0000 7f93897fa700 1 -- 172.21.15.92:0/2906504017 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v136) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9390056a20 con 0x7f93941566e0 2023-10-06T03:11:27.216 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:11:27.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9378067320 msgr2=0x7f93780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9378067320 0x7f93780697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f938c0075e0 tx=0x7f938c0097e0 comp rx=0 tx=0).stop 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 msgr2=0x7f93940b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 0x7f93940b17b0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f939000b3c0 tx=0x7f9390017000 comp rx=0 tx=0).stop 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 shutdown_connections 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9378067320 0x7f93780697d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93941566e0 0x7f93940b17b0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93940ae710 0x7f939413e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 --2- 172.21.15.92:0/2906504017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9394156310 0x7f93940b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 >> 172.21.15.92:0/2906504017 conn(0x7f93940cc1f0 msgr2=0x7f93940c1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:27.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 shutdown_connections 2023-10-06T03:11:27.219 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.216+0000 7f939a4ce700 1 -- 172.21.15.92:0/2906504017 wait complete. 2023-10-06T03:11:27.229 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:11:27.229 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:11:27.229 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi092.front.sepia.ceph.com 2023-10-06T03:11:27.229 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:11:27.230 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:11:27.230 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:11:27.230 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:11:27.230 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:11:27.230 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:11:27.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.230 DEBUG:teuthology.orchestra.run.smithi092:> ip addr 2023-10-06T03:11:27.245 INFO:teuthology.orchestra.run.smithi092.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:11:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:11:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:11:27.245 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 ::1/128 scope host 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f0 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:d8:a1:b3 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: altname enp7s0f1 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:44 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:27.246 INFO:teuthology.orchestra.run.smithi092.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 0c:c4:7a:8f:cd:45 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: inet 172.21.15.92/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft 41919sec preferred_lft 41919sec 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::ec4:7aff:fe8f:cd45/64 scope link 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::58d6:d2ff:fef5:8a68/64 scope link 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.247 INFO:teuthology.orchestra.run.smithi092.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 06:90:68:e7:22:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::490:68ff:fee7:22ed/64 scope link 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: link/ether 22:a8:d9:c5:e3:50 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: inet6 fe80::20a8:d9ff:fec5:e350/64 scope link 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run.smithi092.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:27.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.248 DEBUG:teuthology.orchestra.run.smithi092:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:11:27.308 INFO:teuthology.orchestra.run.smithi092.stdout:1 2023-10-06T03:11:27.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.310 DEBUG:teuthology.orchestra.run.smithi092:> route 2023-10-06T03:11:27.366 INFO:teuthology.orchestra.run.smithi092.stdout:Kernel IP routing table 2023-10-06T03:11:27.367 INFO:teuthology.orchestra.run.smithi092.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:11:27.367 INFO:teuthology.orchestra.run.smithi092.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:11:27.367 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:11:27.367 INFO:teuthology.orchestra.run.smithi092.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:11:27.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.367 DEBUG:teuthology.orchestra.run.smithi092:> (cd / && exec bash -c ' 2023-10-06T03:11:27.367 DEBUG:teuthology.orchestra.run.smithi092:> set -e 2023-10-06T03:11:27.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:11:27.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:11:27.368 DEBUG:teuthology.orchestra.run.smithi092:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:11:27.368 DEBUG:teuthology.orchestra.run.smithi092:> ') 2023-10-06T03:11:27.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.466 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:11:27.480 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:11:27.480 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:11:27.481 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:11:27.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link set brx.0 up 2023-10-06T03:11:27.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:11:27.545 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:11:27.599 INFO:teuthology.orchestra.run.smithi092.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:11:27.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:11:27.600 DEBUG:teuthology.orchestra.run.smithi092:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:11:27.653 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-06T03:11:27.654 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:11:27.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:27.733 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:27.733 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:11:27.751 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:27.751 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:11:27.751 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:11:27.752 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:11:27.752 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:11:27.752 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:11:27.752 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:11:27.752 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:11:27.773 INFO:teuthology.orchestra.run.smithi092.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:27.773 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with new device syntax: 0@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:11:27.774 INFO:teuthology.orchestra.run.smithi092.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:11:27.775 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:11:27.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:27.776 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:11:27.841 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:11:27.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.994+0000 7fdee9fef700 1 Processor -- start 2023-10-06T03:11:27.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.994+0000 7fdee9fef700 1 -- start start 2023-10-06T03:11:27.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee9fef700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdee4159890 0x7fdee415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:27.998 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee9fef700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee4159c60 0x7fdee415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee9fef700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee415a160 con 0x7fdee415db10 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee9fef700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee40d13a0 con 0x7fdee4159890 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee9fef700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee415e140 con 0x7fdee4159c60 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdee4159890 0x7fdee415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee3fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdee415db10 0x7fdee4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:50498/0 (socket says 172.21.15.92:50498) 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee3fff700 1 -- 172.21.15.92:0/2070950524 learned_addr learned my addr 172.21.15.92:0/2070950524 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.995+0000 7fdee37fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdee4159890 0x7fdee415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59836/0 (socket says 172.21.15.92:59836) 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee2ffd700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee4159c60 0x7fdee415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee37fe700 1 -- 172.21.15.92:0/2070950524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee4159c60 msgr2=0x7fdee415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee37fe700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee4159c60 0x7fdee415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee37fe700 1 -- 172.21.15.92:0/2070950524 >> v1:172.21.15.92:6789/0 conn(0x7fdee415db10 legacy=0x7fdee4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:27.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee37fe700 1 -- 172.21.15.92:0/2070950524 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee415a3d0 con 0x7fdee4159890 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee27fc700 1 -- 172.21.15.92:0/2070950524 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942217136 0 0) 0x7fdee415a160 con 0x7fdee415db10 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee2ffd700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee4159c60 0x7fdee415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee37fe700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159890 0x7fdee415abd0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fded4009e60 tx=0x7fded4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c468cd8a69cbbc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee27fc700 1 -- 172.21.15.92:0/2070950524 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded400e070 con 0x7fdee4159890 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee27fc700 1 -- 172.21.15.92:0/2070950524 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400bc30 con 0x7fdee4159890 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee27fc700 1 -- 172.21.15.92:0/2070950524 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4012430 con 0x7fdee4159890 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 -- 172.21.15.92:0/2070950524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159890 msgr2=0x7fdee415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159890 0x7fdee415abd0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fded4009e60 tx=0x7fded4009780 comp rx=0 tx=0).stop 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 -- 172.21.15.92:0/2070950524 shutdown_connections 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159890 0x7fdee415abd0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.000 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 --2- 172.21.15.92:0/2070950524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee4159c60 0x7fdee415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 -- 172.21.15.92:0/2070950524 >> 172.21.15.92:0/2070950524 conn(0x7fdee4068e20 msgr2=0x7fdee40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:28.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 -- 172.21.15.92:0/2070950524 shutdown_connections 2023-10-06T03:11:28.001 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.996+0000 7fdee9fef700 1 -- 172.21.15.92:0/2070950524 wait complete. 2023-10-06T03:11:28.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 Processor -- start 2023-10-06T03:11:28.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 -- start start 2023-10-06T03:11:28.003 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdee4159890 0x7fdee415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 0x7fdee40b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee415db10 0x7fdee40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee4152390 con 0x7fdee4159c60 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee4152670 con 0x7fdee4159890 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.998+0000 7fdee9fef700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee4152950 con 0x7fdee415db10 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 0x7fdee40b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 0x7fdee40b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59842/0 (socket says 172.21.15.92:59842) 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 -- 172.21.15.92:0/1439193994 learned_addr learned my addr 172.21.15.92:0/1439193994 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee37fe700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdee4159890 0x7fdee415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee3fff700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee415db10 0x7fdee40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:28.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 -- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee415db10 msgr2=0x7fdee40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee415db10 0x7fdee40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 -- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdee4159890 msgr2=0x7fdee415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdee4159890 0x7fdee415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 -- 172.21.15.92:0/1439193994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded4009a20 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee2ffd700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 0x7fdee40b5ed0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fded800c880 tx=0x7fded8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8011600 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8011760 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee4152c30 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8011b00 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:27.999+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee40b6410 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.000+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee404c3c0 con 0x7fdee4159c60 2023-10-06T03:11:28.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.000+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fded8018360 con 0x7fdee4159c60 2023-10-06T03:11:28.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.001+0000 7fdecb7fe700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fded0067320 0x7fded00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:28.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.001+0000 7fdee37fe700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fded0067320 0x7fded00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:28.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.001+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fded8084eb0 con 0x7fdee4159c60 2023-10-06T03:11:28.006 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.001+0000 7fdee37fe700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fded0067320 0x7fded00697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fded4009e60 tx=0x7fded4019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:28.007 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.005+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fded8055360 con 0x7fdee4159c60 2023-10-06T03:11:28.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.235+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdee406a5b0 con 0x7fdee4159c60 2023-10-06T03:11:28.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.236+0000 7fdecb7fe700 1 -- 172.21.15.92:0/1439193994 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v136) v1 ==== 53+0+73 (secure 0 0 0) 0x7fded8015070 con 0x7fdee4159c60 2023-10-06T03:11:28.237 INFO:teuthology.orchestra.run.smithi092.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:11:28.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fded0067320 msgr2=0x7fded00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:28.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fded0067320 0x7fded00697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fded4009e60 tx=0x7fded4019000 comp rx=0 tx=0).stop 2023-10-06T03:11:28.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 msgr2=0x7fdee40b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 0x7fdee40b5ed0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fded800c880 tx=0x7fded8008040 comp rx=0 tx=0).stop 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 shutdown_connections 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fded0067320 0x7fded00697d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdee415db10 0x7fdee40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdee4159890 0x7fdee415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 --2- 172.21.15.92:0/1439193994 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdee4159c60 0x7fdee40b5ed0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.237+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 >> 172.21.15.92:0/1439193994 conn(0x7fdee4068e20 msgr2=0x7fdee414d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.238+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 shutdown_connections 2023-10-06T03:11:28.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:28.238+0000 7fdee9fef700 1 -- 172.21.15.92:0/1439193994 wait complete. 2023-10-06T03:11:28.251 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:11:28.251 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:11:28.251 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi136.front.sepia.ceph.com 2023-10-06T03:11:28.251 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:11:28.251 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:11:28.251 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:11:28.252 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:11:28.252 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:11:28.252 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:11:28.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.252 DEBUG:teuthology.orchestra.run.smithi136:> ip addr 2023-10-06T03:11:28.271 INFO:teuthology.orchestra.run.smithi136.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:11:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:11:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:11:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 ::1/128 scope host 2023-10-06T03:11:28.272 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.273 INFO:teuthology.orchestra.run.smithi136.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:11:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:28.273 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f0 2023-10-06T03:11:28.273 INFO:teuthology.orchestra.run.smithi136.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:11:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:d9:3b:1b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: altname enp7s0f1 2023-10-06T03:11:28.274 INFO:teuthology.orchestra.run.smithi136.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:11:28.274 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:42 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:28.275 INFO:teuthology.orchestra.run.smithi136.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:11:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 0c:c4:7a:88:6e:43 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: inet 172.21.15.136/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:11:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft 41919sec preferred_lft 41919sec 2023-10-06T03:11:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::ec4:7aff:fe88:6e43/64 scope link 2023-10-06T03:11:28.275 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.276 INFO:teuthology.orchestra.run.smithi136.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:11:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:11:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:11:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.276 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::500a:9ff:fe99:ca47/64 scope link 2023-10-06T03:11:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.277 INFO:teuthology.orchestra.run.smithi136.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:11:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 8a:ca:b0:14:30:d8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:11:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::88ca:b0ff:fe14:30d8/64 scope link 2023-10-06T03:11:28.277 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.277 INFO:teuthology.orchestra.run.smithi136.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:11:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: link/ether 72:8f:58:52:bb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:11:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: inet6 fe80::708f:58ff:fe52:bbb2/64 scope link 2023-10-06T03:11:28.278 INFO:teuthology.orchestra.run.smithi136.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:11:28.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.279 DEBUG:teuthology.orchestra.run.smithi136:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:11:28.338 INFO:teuthology.orchestra.run.smithi136.stdout:1 2023-10-06T03:11:28.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.340 DEBUG:teuthology.orchestra.run.smithi136:> route 2023-10-06T03:11:28.398 INFO:teuthology.orchestra.run.smithi136.stdout:Kernel IP routing table 2023-10-06T03:11:28.398 INFO:teuthology.orchestra.run.smithi136.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:11:28.399 INFO:teuthology.orchestra.run.smithi136.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:11:28.399 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:11:28.399 INFO:teuthology.orchestra.run.smithi136.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:11:28.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.399 DEBUG:teuthology.orchestra.run.smithi136:> (cd / && exec bash -c ' 2023-10-06T03:11:28.399 DEBUG:teuthology.orchestra.run.smithi136:> set -e 2023-10-06T03:11:28.399 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:11:28.399 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:11:28.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:11:28.400 DEBUG:teuthology.orchestra.run.smithi136:> ') 2023-10-06T03:11:28.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.505 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:11:28.520 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:11:28.520 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:11:28.521 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:11:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.521 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link set brx.0 up 2023-10-06T03:11:28.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:11:28.585 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:11:28.640 INFO:teuthology.orchestra.run.smithi136.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:11:28.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:11:28.641 DEBUG:teuthology.orchestra.run.smithi136:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:11:28.695 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-06T03:11:28.695 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:11:28.695 DEBUG:teuthology.orchestra.run.smithi136:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:28.774 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:28.775 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:11:28.793 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:28.793 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:11:28.793 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:11:28.793 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:11:28.794 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:11:28.794 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:11:28.794 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:11:28.794 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:11:28.815 INFO:teuthology.orchestra.run.smithi136.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-06T03:11:28.815 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-06T03:11:28.815 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:3300,172.21.15.136:3300,172.21.15.136:3301" 2023-10-06T03:11:28.815 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with new device syntax: 1@e5cba154-9a64-4fb6-add9-5855ed5b5b5a.cephfs=/ 2023-10-06T03:11:28.815 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.92:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2023-10-06T03:11:28.815 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: v2 mount support check returned 2 2023-10-06T03:11:28.816 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-06T03:11:28.816 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:11:28.816 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: resolved to: "172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790" 2023-10-06T03:11:28.816 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.92:6789,172.21.15.136:6789,172.21.15.136:6790:/ 2023-10-06T03:11:28.816 INFO:teuthology.orchestra.run.smithi136.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=e5cba154-9a64-4fb6-add9-5855ed5b5b5a" will pass to kernel 2023-10-06T03:11:28.817 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:11:28.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:28.817 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:11:28.881 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T03:11:29.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 Processor -- start 2023-10-06T03:11:29.002 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 -- start start 2023-10-06T03:11:29.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 0x7f2c740b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:29.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2c740b53d0 0x7f2c740cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:29.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c740b5b00 con 0x7f2c740cce40 2023-10-06T03:11:29.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c740b5da0 con 0x7f2c740b53d0 2023-10-06T03:11:29.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.001+0000 7f2c78baa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740b6080 con 0x7f2c74159ec0 2023-10-06T03:11:29.004 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 0x7f2c740b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c69d9b700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2c740b53d0 0x7f2c740cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 0x7f2c740b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:34062/0 (socket says 172.21.15.92:34062) 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 -- 172.21.15.92:0/2227894294 learned_addr learned my addr 172.21.15.92:0/2227894294 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 -- 172.21.15.92:0/2227894294 >> v2:172.21.15.136:3300/0 conn(0x7f2c740b53d0 msgr2=0x7f2c740cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 --2- 172.21.15.92:0/2227894294 >> v2:172.21.15.136:3300/0 conn(0x7f2c740b53d0 0x7f2c740cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 -- 172.21.15.92:0/2227894294 >> v1:172.21.15.136:6789/0 conn(0x7f2c740cce40 legacy=0x7f2c740cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 -- 172.21.15.92:0/2227894294 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740cc100 con 0x7f2c74159ec0 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.002+0000 7f2c7259c700 1 --2- 172.21.15.92:0/2227894294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 0x7f2c740b4e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009f70 tx=0x7f2c64009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3ea1dad4f1b242 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c71d9b700 1 -- 172.21.15.92:0/2227894294 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400e070 con 0x7f2c74159ec0 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c71d9b700 1 -- 172.21.15.92:0/2227894294 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400ba20 con 0x7f2c74159ec0 2023-10-06T03:11:29.005 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c71d9b700 1 -- 172.21.15.92:0/2227894294 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640092e0 con 0x7f2c74159ec0 2023-10-06T03:11:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 -- 172.21.15.92:0/2227894294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 msgr2=0x7f2c740b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2227894294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 0x7f2c740b4e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009f70 tx=0x7f2c64009840 comp rx=0 tx=0).stop 2023-10-06T03:11:29.008 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 -- 172.21.15.92:0/2227894294 shutdown_connections 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2227894294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c74159ec0 0x7f2c740b4e90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2227894294 >> v2:172.21.15.136:3300/0 conn(0x7f2c740b53d0 0x7f2c740cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 -- 172.21.15.92:0/2227894294 >> 172.21.15.92:0/2227894294 conn(0x7f2c740999f0 msgr2=0x7f2c740b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 -- 172.21.15.92:0/2227894294 shutdown_connections 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.003+0000 7f2c78baa700 1 -- 172.21.15.92:0/2227894294 wait complete. 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.004+0000 7f2c78baa700 1 Processor -- start 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.004+0000 7f2c78baa700 1 -- start start 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c78baa700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c740b53d0 0x7f2c740c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c78baa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c740cce40 0x7f2c7413eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:29.009 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c78baa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 0x7f2c74154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c78baa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740c9d40 con 0x7f2c74159ec0 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c78baa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7413f020 con 0x7f2c740cce40 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c78baa700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c740c9130 con 0x7f2c740b53d0 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c72d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 0x7f2c74154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c72d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 0x7f2c74154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:59854/0 (socket says 172.21.15.92:59854) 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c72d9d700 1 -- 172.21.15.92:0/2745113021 learned_addr learned my addr 172.21.15.92:0/2745113021 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c7259c700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c740b53d0 0x7f2c740c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.005+0000 7f2c69d9b700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c740cce40 0x7f2c7413eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c72d9d700 1 -- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c740b53d0 msgr2=0x7f2c740c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c72d9d700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c740b53d0 0x7f2c740c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:29.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c72d9d700 1 -- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c740cce40 msgr2=0x7f2c7413eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c72d9d700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c740cce40 0x7f2c7413eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c72d9d700 1 -- 172.21.15.92:0/2745113021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009ae0 con 0x7f2c74159ec0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c69d9b700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c740cce40 0x7f2c7413eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c7259c700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c740b53d0 0x7f2c740c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c72d9d700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 0x7f2c74154cb0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00c590 tx=0x7f2c6c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c011590 con 0x7f2c74159ec0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c0116f0 con 0x7f2c74159ec0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c740c9410 con 0x7f2c74159ec0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c011ab0 con 0x7f2c74159ec0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.006+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c740c99d0 con 0x7f2c74159ec0 2023-10-06T03:11:29.011 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.007+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c740c9ef0 con 0x7f2c74159ec0 2023-10-06T03:11:29.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.007+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2c6c01b3b0 con 0x7f2c74159ec0 2023-10-06T03:11:29.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.008+0000 7f2c6affd700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c54067320 0x7f2c540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:29.012 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.008+0000 7f2c7259c700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c54067320 0x7f2c540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:29.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.008+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c6c0841b0 con 0x7f2c74159ec0 2023-10-06T03:11:29.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.008+0000 7f2c7259c700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c54067320 0x7f2c540697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009880 tx=0x7f2c64018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:29.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.012+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c740c9ef0 con 0x7f2c74159ec0 2023-10-06T03:11:29.248 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:29.246+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c740d11f0 con 0x7f2c74159ec0 2023-10-06T03:11:30.086 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.084+0000 7f2c6affd700 1 -- 172.21.15.92:0/2745113021 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v137) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2c6c050ce0 con 0x7f2c74159ec0 2023-10-06T03:11:30.089 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c54067320 msgr2=0x7f2c540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c54067320 0x7f2c540697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009880 tx=0x7f2c64018000 comp rx=0 tx=0).stop 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 msgr2=0x7f2c74154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 0x7f2c74154cb0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00c590 tx=0x7f2c6c007ef0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 shutdown_connections 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2c54067320 0x7f2c540697d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2c74159ec0 0x7f2c74154cb0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c740b53d0 0x7f2c740c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.090 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 --2- 172.21.15.92:0/2745113021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c740cce40 0x7f2c7413eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.087+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 >> 172.21.15.92:0/2745113021 conn(0x7f2c740999f0 msgr2=0x7f2c74158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.088+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 shutdown_connections 2023-10-06T03:11:30.091 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.088+0000 7f2c78baa700 1 -- 172.21.15.92:0/2745113021 wait complete. 2023-10-06T03:11:30.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:11:30.233 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.223+0000 7fb508b18700 1 Processor -- start 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.223+0000 7fb508b18700 1 -- start start 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb508b18700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb50406bcd0 0x7fb5040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb508b18700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb50415df50 0x7fb50415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb508b18700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50415d7e0 con 0x7fb50406c0a0 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb508b18700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50415dac0 con 0x7fb50406bcd0 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb508b18700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50415aa30 con 0x7fb50415df50 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb50259c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb50406bcd0 0x7fb5040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb4f9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fb50406c0a0 0x7fb50415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57216/0 (socket says 172.21.15.92:57216) 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.224+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/4134483689 learned_addr learned my addr 172.21.15.92:0/4134483689 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb502d9d700 1 --2- 172.21.15.92:0/4134483689 >> v2:172.21.15.136:3300/0 conn(0x7fb50415df50 0x7fb50415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.234 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb50259c700 1 -- 172.21.15.92:0/4134483689 >> v2:172.21.15.136:3300/0 conn(0x7fb50415df50 msgr2=0x7fb50415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb50259c700 1 --2- 172.21.15.92:0/4134483689 >> v2:172.21.15.136:3300/0 conn(0x7fb50415df50 0x7fb50415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb50259c700 1 -- 172.21.15.92:0/4134483689 >> v1:172.21.15.92:6789/0 conn(0x7fb50406c0a0 legacy=0x7fb50415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb50259c700 1 -- 172.21.15.92:0/4134483689 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50415ad10 con 0x7fb50406bcd0 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb502d9d700 1 --2- 172.21.15.92:0/4134483689 >> v2:172.21.15.136:3300/0 conn(0x7fb50415df50 0x7fb50415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb50259c700 1 --2- 172.21.15.92:0/4134483689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406bcd0 0x7fb5040caa90 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fb50415d720 tx=0x7fb4f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03aaa0655f9efdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb501d9b700 1 -- 172.21.15.92:0/4134483689 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011630 con 0x7fb50406bcd0 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb501d9b700 1 -- 172.21.15.92:0/4134483689 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f4011790 con 0x7fb50406bcd0 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb501d9b700 1 -- 172.21.15.92:0/4134483689 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4011b30 con 0x7fb50406bcd0 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb508b18700 1 -- 172.21.15.92:0/4134483689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406bcd0 msgr2=0x7fb5040caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb508b18700 1 --2- 172.21.15.92:0/4134483689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406bcd0 0x7fb5040caa90 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fb50415d720 tx=0x7fb4f40091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.235 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb508b18700 1 -- 172.21.15.92:0/4134483689 shutdown_connections 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb508b18700 1 --2- 172.21.15.92:0/4134483689 >> v2:172.21.15.136:3300/0 conn(0x7fb50415df50 0x7fb50415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb508b18700 1 --2- 172.21.15.92:0/4134483689 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406bcd0 0x7fb5040caa90 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.225+0000 7fb508b18700 1 -- 172.21.15.92:0/4134483689 >> 172.21.15.92:0/4134483689 conn(0x7fb5040ce600 msgr2=0x7fb5040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.226+0000 7fb508b18700 1 -- 172.21.15.92:0/4134483689 shutdown_connections 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.226+0000 7fb508b18700 1 -- 172.21.15.92:0/4134483689 wait complete. 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.227+0000 7fb508b18700 1 Processor -- start 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.227+0000 7fb508b18700 1 -- start start 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.227+0000 7fb508b18700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb50406bcd0 0x7fb5040b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.227+0000 7fb508b18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 0x7fb5040c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.227+0000 7fb508b18700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb50415df50 0x7fb5040c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.236 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb508b18700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040c63c0 con 0x7fb50406c0a0 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb508b18700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040c66a0 con 0x7fb50406bcd0 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb508b18700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040c6980 con 0x7fb50415df50 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 0x7fb5040c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 0x7fb5040c33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52120/0 (socket says 172.21.15.92:52120) 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1107892196 learned_addr learned my addr 172.21.15.92:0/1107892196 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb50259c700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb50406bcd0 0x7fb5040b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb502d9d700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb50415df50 0x7fb5040c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb50415df50 msgr2=0x7fb5040c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb50415df50 0x7fb5040c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb50406bcd0 msgr2=0x7fb5040b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.237 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb50406bcd0 0x7fb5040b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 -- 172.21.15.92:0/1107892196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4009420 con 0x7fb50406c0a0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4f9d9b700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 0x7fb5040c33a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c810 tx=0x7fb4ec007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec005b30 con 0x7fb50406c0a0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec005c90 con 0x7fb50406c0a0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec007340 con 0x7fb50406c0a0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5040c6c60 con 0x7fb50406c0a0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.228+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb504148080 con 0x7fb50406c0a0 2023-10-06T03:11:30.238 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.229+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50404c3c0 con 0x7fb50406c0a0 2023-10-06T03:11:30.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.230+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb4ec01b400 con 0x7fb50406c0a0 2023-10-06T03:11:30.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.230+0000 7fb4faffd700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4067320 0x7fb4e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.230+0000 7fb50259c700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4067320 0x7fb4e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.230+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb4ec084970 con 0x7fb50406c0a0 2023-10-06T03:11:30.239 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.231+0000 7fb50259c700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4067320 0x7fb4e40697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40030c0 tx=0x7fb4f4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:30.240 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.234+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4ec051550 con 0x7fb50406c0a0 2023-10-06T03:11:30.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.449+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb50415aff0 con 0x7fb4e4067320 2023-10-06T03:11:30.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.450+0000 7fb4faffd700 1 -- 172.21.15.92:0/1107892196 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb50415aff0 con 0x7fb4e4067320 2023-10-06T03:11:30.452 INFO:teuthology.orchestra.run.smithi092.stdout:[ 2023-10-06T03:11:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: { 2023-10-06T03:11:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: "name": "cephfs" 2023-10-06T03:11:30.452 INFO:teuthology.orchestra.run.smithi092.stdout: } 2023-10-06T03:11:30.452 INFO:teuthology.orchestra.run.smithi092.stdout:] 2023-10-06T03:11:30.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4067320 msgr2=0x7fb4e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4067320 0x7fb4e40697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40030c0 tx=0x7fb4f4016000 comp rx=0 tx=0).stop 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 msgr2=0x7fb5040c33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 0x7fb5040c33a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00c810 tx=0x7fb4ec007f30 comp rx=0 tx=0).stop 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 shutdown_connections 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb4e4067320 0x7fb4e40697d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb50415df50 0x7fb5040c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb50406bcd0 0x7fb5040b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 --2- 172.21.15.92:0/1107892196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb50406c0a0 0x7fb5040c33a0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 >> 172.21.15.92:0/1107892196 conn(0x7fb5040ce600 msgr2=0x7fb50406d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 shutdown_connections 2023-10-06T03:11:30.455 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.452+0000 7fb508b18700 1 -- 172.21.15.92:0/1107892196 wait complete. 2023-10-06T03:11:30.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:11:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.581+0000 7fe4b3876700 1 Processor -- start 2023-10-06T03:11:30.583 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.581+0000 7fe4b3876700 1 -- start start 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b3876700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe4ac159890 0x7fe4ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b3876700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac159c60 0x7fe4ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b3876700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac15a160 con 0x7fe4ac15db10 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b3876700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ac0d13a0 con 0x7fe4ac159890 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b3876700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac15e140 con 0x7fe4ac159c60 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b1e13700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4ac15db10 0x7fe4ac069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57228/0 (socket says 172.21.15.92:57228) 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.582+0000 7fe4b1e13700 1 -- 172.21.15.92:0/3233294385 learned_addr learned my addr 172.21.15.92:0/3233294385 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b0e11700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac159c60 0x7fe4ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.587 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 --2- 172.21.15.92:0/3233294385 >> v2:172.21.15.136:3300/0 conn(0x7fe4ac159890 0x7fe4ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441049187 0 0) 0x7fe4ac15a160 con 0x7fe4ac15db10 2023-10-06T03:11:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe498003570 con 0x7fe4ac15db10 2023-10-06T03:11:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91086888 0 0) 0x7fe498003570 con 0x7fe4ac15db10 2023-10-06T03:11:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac159c60 msgr2=0x7fe4ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:30.588 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac159c60 0x7fe4ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 >> v2:172.21.15.136:3300/0 conn(0x7fe4ac159890 msgr2=0x7fe4ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/3233294385 >> v2:172.21.15.136:3300/0 conn(0x7fe4ac159890 0x7fe4ac15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac15a3d0 con 0x7fe4ac15db10 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fe4a4001470 con 0x7fe4ac15db10 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 --2- 172.21.15.92:0/3233294385 >> v2:172.21.15.136:3300/0 conn(0x7fe4ac159890 0x7fe4ac15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 >> v1:172.21.15.92:6789/0 conn(0x7fe4ac15db10 legacy=0x7fe4ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe498003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe498004810 0x7fe498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4980051c0 0x7fe49800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0d13a0 con 0x7fe498003a60 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac15e140 con 0x7fe498004810 2023-10-06T03:11:30.589 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49800bfa0 con 0x7fe4980051c0 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b0e11700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac159c60 0x7fe4ac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe498003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1e13700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4980051c0 0x7fe49800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b0e11700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe498004810 0x7fe498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 -- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4980051c0 msgr2=0x7fe49800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4980051c0 0x7fe49800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 -- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe498004810 msgr2=0x7fe498004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe498004810 0x7fe498004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1612700 1 -- 172.21.15.92:0/3233294385 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498003570 con 0x7fe498003a60 2023-10-06T03:11:30.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.583+0000 7fe4b1e13700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4980051c0 0x7fe49800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b1612700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe498003e70 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800e3c0 tx=0x7fe4a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b76b24d2d8f9dea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800ec70 con 0x7fe498003a60 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a800f040 con 0x7fe498003a60 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4a3fff700 1 -- 172.21.15.92:0/3233294385 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8014480 con 0x7fe498003a60 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b3876700 1 -- 172.21.15.92:0/3233294385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 msgr2=0x7fe498003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b3876700 1 --2- 172.21.15.92:0/3233294385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe498003e70 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800e3c0 tx=0x7fe4a800b040 comp rx=0 tx=0).stop 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b1e13700 1 -- 172.21.15.92:0/3233294385 reap_dead start 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b3876700 1 -- 172.21.15.92:0/3233294385 shutdown_connections 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b3876700 1 -- 172.21.15.92:0/3233294385 >> 172.21.15.92:0/3233294385 conn(0x7fe4ac068e20 msgr2=0x7fe4ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b3876700 1 -- 172.21.15.92:0/3233294385 shutdown_connections 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.584+0000 7fe4b3876700 1 -- 172.21.15.92:0/3233294385 wait complete. 2023-10-06T03:11:30.591 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.585+0000 7fe4b3876700 1 Processor -- start 2023-10-06T03:11:30.592 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.585+0000 7fe4b3876700 1 -- start start 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b3876700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe4ac0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b3876700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac0cd0d0 0x7fe4ac155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b3876700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4ac14c570 0x7fe4ac1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b3876700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0cd620 con 0x7fe498003a60 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b3876700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0cd900 con 0x7fe4ac14c570 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b3876700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0cdbe0 con 0x7fe4ac0cd0d0 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b1612700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe4ac0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.594 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b1612700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe4ac0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52148/0 (socket says 172.21.15.92:52148) 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b1612700 1 -- 172.21.15.92:0/467417572 learned_addr learned my addr 172.21.15.92:0/467417572 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b0e11700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac0cd0d0 0x7fe4ac155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.586+0000 7fe4b1e13700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4ac14c570 0x7fe4ac1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b1612700 1 -- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac0cd0d0 msgr2=0x7fe4ac155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b1612700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac0cd0d0 0x7fe4ac155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b1612700 1 -- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4ac14c570 msgr2=0x7fe4ac1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b1612700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4ac14c570 0x7fe4ac1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b1612700 1 -- 172.21.15.92:0/467417572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac0cdec0 con 0x7fe498003a60 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b1612700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe4ac0ce6c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe4980046c0 tx=0x7fe4a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8014e80 con 0x7fe498003a60 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a801d660 con 0x7fe498003a60 2023-10-06T03:11:30.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a801da20 con 0x7fe498003a60 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac0ce1a0 con 0x7fe498003a60 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.587+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac156720 con 0x7fe498003a60 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.588+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac04c3c0 con 0x7fe498003a60 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.588+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe4a8003040 con 0x7fe498003a60 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.588+0000 7fe4a17fa700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe490067350 0x7fe490069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.589+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe4a8086ec0 con 0x7fe498003a60 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.589+0000 7fe4b0e11700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe490067350 0x7fe490069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.589+0000 7fe4b0e11700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe490067350 0x7fe490069800 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0093f0 tx=0x7fe49c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:30.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.593+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4a80538a0 con 0x7fe498003a60 2023-10-06T03:11:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.779+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe4ac06a5b0 con 0x7fe498003a60 2023-10-06T03:11:30.781 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.779+0000 7fe4a17fa700 1 -- 172.21.15.92:0/467417572 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4a80180c0 con 0x7fe498003a60 2023-10-06T03:11:30.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe490067350 msgr2=0x7fe490069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.784 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe490067350 0x7fe490069800 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0093f0 tx=0x7fe49c008040 comp rx=0 tx=0).stop 2023-10-06T03:11:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 msgr2=0x7fe4ac0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe4ac0ce6c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe4980046c0 tx=0x7fe4a801a000 comp rx=0 tx=0).stop 2023-10-06T03:11:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 shutdown_connections 2023-10-06T03:11:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe490067350 0x7fe490069800 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.785 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe498003a60 0x7fe4ac0ce6c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4ac14c570 0x7fe4ac1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 --2- 172.21.15.92:0/467417572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4ac0cd0d0 0x7fe4ac155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 >> 172.21.15.92:0/467417572 conn(0x7fe4ac068e20 msgr2=0x7fe4ac0c6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 shutdown_connections 2023-10-06T03:11:30.786 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:30.782+0000 7fe4b3876700 1 -- 172.21.15.92:0/467417572 wait complete. 2023-10-06T03:11:30.796 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:11:30.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:11:30.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:11:30.834 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:11:30.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:30.835 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:11:30.890 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:11:30.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:11:30.890 DEBUG:teuthology.orchestra.run.smithi136:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-06T03:11:30.927 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:11:30.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:30.928 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:11:30.984 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:11:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:30.984 DEBUG:teuthology.orchestra.run.smithi092:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T03:11:30.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:11:31.000 INFO:teuthology.orchestra.run.smithi092.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T03:11:31.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:11:31.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:11:31.001 DEBUG:teuthology.orchestra.run.smithi136:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T03:11:31.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:11:31.041 INFO:teuthology.orchestra.run.smithi136.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T03:11:31.041 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs 2023-10-06T03:11:31.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.157+0000 7fdc8bfff700 1 Processor -- start 2023-10-06T03:11:31.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.157+0000 7fdc8bfff700 1 -- start start 2023-10-06T03:11:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8bfff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdc8c0b4e70 0x7fdc8c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b16d0 0x7fdc8c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.161 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8bfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0b2080 con 0x7fdc8c15d240 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8bfff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c0cfb00 con 0x7fdc8c0b4e70 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8bfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b5c40 con 0x7fdc8c0b16d0 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fdc8c15d240 0x7fdc8c15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57240/0 (socket says 172.21.15.92:57240) 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8b7fe700 1 -- 172.21.15.92:0/776557414 learned_addr learned my addr 172.21.15.92:0/776557414 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8affd700 1 --2- 172.21.15.92:0/776557414 >> v2:172.21.15.136:3300/0 conn(0x7fdc8c0b4e70 0x7fdc8c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc827fc700 1 --2- 172.21.15.92:0/776557414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b16d0 0x7fdc8c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8a7fc700 1 -- 172.21.15.92:0/776557414 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708733500 0 0) 0x7fdc8c0b2080 con 0x7fdc8c15d240 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.158+0000 7fdc8a7fc700 1 -- 172.21.15.92:0/776557414 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc78003570 con 0x7fdc8c15d240 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc827fc700 1 -- 172.21.15.92:0/776557414 >> v2:172.21.15.136:3300/0 conn(0x7fdc8c0b4e70 msgr2=0x7fdc8c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:31.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8a7fc700 1 -- 172.21.15.92:0/776557414 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 306067748 0 0) 0x7fdc78003570 con 0x7fdc8c15d240 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc827fc700 1 --2- 172.21.15.92:0/776557414 >> v2:172.21.15.136:3300/0 conn(0x7fdc8c0b4e70 0x7fdc8c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc827fc700 1 -- 172.21.15.92:0/776557414 >> v1:172.21.15.92:6789/0 conn(0x7fdc8c15d240 legacy=0x7fdc8c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc827fc700 1 -- 172.21.15.92:0/776557414 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0b5f20 con 0x7fdc8c0b16d0 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8affd700 1 --2- 172.21.15.92:0/776557414 >> v2:172.21.15.136:3300/0 conn(0x7fdc8c0b4e70 0x7fdc8c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc827fc700 1 --2- 172.21.15.92:0/776557414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b16d0 0x7fdc8c0b1b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098e0 tx=0x7fdc740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17bcb1f5eb9cad1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8a7fc700 1 -- 172.21.15.92:0/776557414 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840017f0 con 0x7fdc8c0b16d0 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8a7fc700 1 -- 172.21.15.92:0/776557414 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74011420 con 0x7fdc8c0b16d0 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8a7fc700 1 -- 172.21.15.92:0/776557414 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740117c0 con 0x7fdc8c0b16d0 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8bfff700 1 -- 172.21.15.92:0/776557414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b16d0 msgr2=0x7fdc8c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.159+0000 7fdc8bfff700 1 --2- 172.21.15.92:0/776557414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b16d0 0x7fdc8c0b1b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098e0 tx=0x7fdc740091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.163 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.160+0000 7fdc8bfff700 1 -- 172.21.15.92:0/776557414 shutdown_connections 2023-10-06T03:11:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.160+0000 7fdc8bfff700 1 --2- 172.21.15.92:0/776557414 >> v2:172.21.15.136:3300/0 conn(0x7fdc8c0b4e70 0x7fdc8c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.160+0000 7fdc8bfff700 1 --2- 172.21.15.92:0/776557414 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b16d0 0x7fdc8c0b1b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.160+0000 7fdc8bfff700 1 -- 172.21.15.92:0/776557414 >> 172.21.15.92:0/776557414 conn(0x7fdc8c0d10c0 msgr2=0x7fdc8c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.160+0000 7fdc8bfff700 1 -- 172.21.15.92:0/776557414 shutdown_connections 2023-10-06T03:11:31.164 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.160+0000 7fdc8bfff700 1 -- 172.21.15.92:0/776557414 wait complete. 2023-10-06T03:11:31.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.161+0000 7fdc8bfff700 1 Processor -- start 2023-10-06T03:11:31.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.161+0000 7fdc8bfff700 1 -- start start 2023-10-06T03:11:31.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc8c0b16d0 0x7fdc8c1459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b4e70 0x7fdc8c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc8c15d240 0x7fdc8c0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c7370 con 0x7fdc8c15d240 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c6360 con 0x7fdc8c0b16d0 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c6ee0 con 0x7fdc8c0b4e70 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc8c15d240 0x7fdc8c0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc8c15d240 0x7fdc8c0688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52160/0 (socket says 172.21.15.92:52160) 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 -- 172.21.15.92:0/994306407 learned_addr learned my addr 172.21.15.92:0/994306407 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc827fc700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b4e70 0x7fdc8c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8affd700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc8c0b16d0 0x7fdc8c1459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 -- 172.21.15.92:0/994306407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b4e70 msgr2=0x7fdc8c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc8c0b4e70 0x7fdc8c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 -- 172.21.15.92:0/994306407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc8c0b16d0 msgr2=0x7fdc8c1459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc8c0b16d0 0x7fdc8c1459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 -- 172.21.15.92:0/994306407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c009420 con 0x7fdc8c15d240 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8b7fe700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc8c15d240 0x7fdc8c0688d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400cbe0 tx=0x7fdc84001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:31.169 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84004dc0 con 0x7fdc8c15d240 2023-10-06T03:11:31.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc84007150 con 0x7fdc8c15d240 2023-10-06T03:11:31.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 -- 172.21.15.92:0/994306407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc74009420 con 0x7fdc8c15d240 2023-10-06T03:11:31.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84007510 con 0x7fdc8c15d240 2023-10-06T03:11:31.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.162+0000 7fdc8bfff700 1 -- 172.21.15.92:0/994306407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c0c6890 con 0x7fdc8c15d240 2023-10-06T03:11:31.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.163+0000 7fdc8bfff700 1 -- 172.21.15.92:0/994306407 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c0c7050 con 0x7fdc8c15d240 2023-10-06T03:11:31.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.164+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdc840078c0 con 0x7fdc8c15d240 2023-10-06T03:11:31.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.164+0000 7fdc837fe700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc6c067320 0x7fdc6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.164+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdc840846b0 con 0x7fdc8c15d240 2023-10-06T03:11:31.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.164+0000 7fdc8affd700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc6c067320 0x7fdc6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.164+0000 7fdc8affd700 1 --2- 172.21.15.92:0/994306407 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc6c067320 0x7fdc6c0697d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009f40 tx=0x7fdc7c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:31.172 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.168+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc84054cf0 con 0x7fdc8c15d240 2023-10-06T03:11:31.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.377+0000 7fdc8bfff700 1 -- 172.21.15.92:0/994306407 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc8c0cf780 con 0x7fdc6c067320 2023-10-06T03:11:31.379 INFO:tasks.ceph.mgr.y.smithi092.stderr:2023-10-06T03:11:31.378+0000 7fdd0eb80700 -1 mgr.server reply reply (1) Operation not permitted WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-06T03:11:31.380 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.378+0000 7fdc837fe700 1 -- 172.21.15.92:0/994306407 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: -1 WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it.) v1 ==== 204+0+0 (secure 0 0 0) 0x7fdc8c0cf780 con 0x7fdc6c067320 2023-10-06T03:11:31.381 INFO:teuthology.orchestra.run.smithi092.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-06T03:11:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:11:31.385 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2023-10-06T03:11:31.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.501+0000 7fedcaa6a700 1 Processor -- start 2023-10-06T03:11:31.503 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.501+0000 7fedcaa6a700 1 -- start start 2023-10-06T03:11:31.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedcaa6a700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fedc4157a50 0x7fedc40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedcaa6a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc40cb590 0x7fedc40c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedcaa6a700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40cd240 con 0x7fedc4157680 2023-10-06T03:11:31.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedcaa6a700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc40cc700 con 0x7fedc4157a50 2023-10-06T03:11:31.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedcaa6a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40c9a20 con 0x7fedc40cb590 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9a68700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fedc4157680 0x7fedc40cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:57256/0 (socket says 172.21.15.92:57256) 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9a68700 1 -- 172.21.15.92:0/1505205025 learned_addr learned my addr 172.21.15.92:0/1505205025 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9267700 1 --2- 172.21.15.92:0/1505205025 >> v2:172.21.15.136:3300/0 conn(0x7fedc4157a50 0x7fedc40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedca269700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc40cb590 0x7fedc40c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc8a66700 1 -- 172.21.15.92:0/1505205025 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267011714 0 0) 0x7fedc40cd240 con 0x7fedc4157680 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc8a66700 1 -- 172.21.15.92:0/1505205025 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb0003570 con 0x7fedc4157680 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9267700 1 -- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc40cb590 msgr2=0x7fedc40c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9267700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc40cb590 0x7fedc40c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9267700 1 -- 172.21.15.92:0/1505205025 >> v1:172.21.15.92:6789/0 conn(0x7fedc4157680 legacy=0x7fedc40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedc9267700 1 -- 172.21.15.92:0/1505205025 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc40c9750 con 0x7fedc4157a50 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.502+0000 7fedca269700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc40cb590 0x7fedc40c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:11:31.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedc9267700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 0x7fedc40c8cd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098e0 tx=0x7fedb40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ec08b90ab18690 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedc8a66700 1 -- 172.21.15.92:0/1505205025 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4011630 con 0x7fedc4157a50 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedc8a66700 1 -- 172.21.15.92:0/1505205025 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb4011790 con 0x7fedc4157a50 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedc8a66700 1 -- 172.21.15.92:0/1505205025 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4011b30 con 0x7fedc4157a50 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 -- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 msgr2=0x7fedc40c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 0x7fedc40c8cd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098e0 tx=0x7fedb40091a0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 -- 172.21.15.92:0/1505205025 shutdown_connections 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc40cb590 0x7fedc40c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 --2- 172.21.15.92:0/1505205025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 0x7fedc40c8cd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 -- 172.21.15.92:0/1505205025 >> 172.21.15.92:0/1505205025 conn(0x7fedc406baa0 msgr2=0x7fedc406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 -- 172.21.15.92:0/1505205025 shutdown_connections 2023-10-06T03:11:31.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.503+0000 7fedcaa6a700 1 -- 172.21.15.92:0/1505205025 wait complete. 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.504+0000 7fedcaa6a700 1 Processor -- start 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.504+0000 7fedcaa6a700 1 -- start start 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedcaa6a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40cb590 0x7fedc4151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedcaa6a700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc4157680 0x7fedc4152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedcaa6a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 0x7fedc40b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedcaa6a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc416c240 con 0x7fedc40cb590 2023-10-06T03:11:31.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedcaa6a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4168db0 con 0x7fedc4157a50 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedcaa6a700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc416b5c0 con 0x7fedc4157680 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedc9a68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40cb590 0x7fedc4151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedc9a68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40cb590 0x7fedc4151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:52162/0 (socket says 172.21.15.92:52162) 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedc9a68700 1 -- 172.21.15.92:0/3426247434 learned_addr learned my addr 172.21.15.92:0/3426247434 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedca269700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 0x7fedc40b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.505+0000 7fedc9267700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc4157680 0x7fedc4152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9a68700 1 -- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc4157680 msgr2=0x7fedc4152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9a68700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc4157680 0x7fedc4152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9a68700 1 -- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 msgr2=0x7fedc40b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9a68700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedc4157a50 0x7fedc40b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:11:31.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9a68700 1 -- 172.21.15.92:0/3426247434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc009a20 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9267700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedc4157680 0x7fedc4152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedc9a68700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fedc40cb590 0x7fedc4151cd0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c7c0 tx=0x7fedc0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedb9ffb700 1 -- 172.21.15.92:0/3426247434 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0005b30 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedb9ffb700 1 -- 172.21.15.92:0/3426247434 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0005c90 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedcaa6a700 1 -- 172.21.15.92:0/3426247434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb4009420 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedb9ffb700 1 -- 172.21.15.92:0/3426247434 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0007340 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.506+0000 7fedcaa6a700 1 -- 172.21.15.92:0/3426247434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc416bb80 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.507+0000 7fedcaa6a700 1 -- 172.21.15.92:0/3426247434 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc40ca4f0 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.507+0000 7fedb9ffb700 1 -- 172.21.15.92:0/3426247434 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fedc001b400 con 0x7fedc40cb590 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.508+0000 7fedb9ffb700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feda8067320 0x7feda80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:11:31.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.508+0000 7fedb9ffb700 1 -- 172.21.15.92:0/3426247434 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fedc0084750 con 0x7fedc40cb590 2023-10-06T03:11:31.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.508+0000 7fedc9267700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feda8067320 0x7feda80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:11:31.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.508+0000 7fedc9267700 1 --2- 172.21.15.92:0/3426247434 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7feda8067320 0x7feda80697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fedb40098e0 tx=0x7fedb4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:11:31.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.512+0000 7fedb9ffb700 1 -- 172.21.15.92:0/3426247434 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedc0051440 con 0x7fedc40cb590 2023-10-06T03:11:31.725 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:11:31.723+0000 7fedcaa6a700 1 -- 172.21.15.92:0/3426247434 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fedc4152750 con 0x7feda8067320 2023-10-06T03:11:43.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:11:43.206 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:12:13.274 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:12:13.277 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:12:43.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:12:43.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:13:13.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:13:13.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:13:31.414 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-06T03:13:31.415 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:13:31.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:13:31.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:13:31.500 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:13:31.500 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.620+0000 7f2307a20700 1 Processor -- start 2023-10-06T03:13:31.621 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2307a20700 1 -- start start 2023-10-06T03:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2307a20700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2307a20700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f230006c0a0 0x7f230015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2307a20700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230015d7e0 con 0x7f230015df50 2023-10-06T03:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2307a20700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230015dac0 con 0x7f230006c0a0 2023-10-06T03:13:31.623 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2307a20700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230015aa30 con 0x7f230006bcd0 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2304fbb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f230006c0a0 0x7f230015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2305fbd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f230015df50 0x7f23000c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37082/0 (socket says 172.21.15.92:37082) 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.621+0000 7f2305fbd700 1 -- 172.21.15.92:0/3188891365 learned_addr learned my addr 172.21.15.92:0/3188891365 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f23057bc700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2304fbb700 1 -- 172.21.15.92:0/3188891365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 msgr2=0x7f23000caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2304fbb700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2304fbb700 1 -- 172.21.15.92:0/3188891365 >> v1:172.21.15.92:6789/0 conn(0x7f230015df50 legacy=0x7f23000c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2304fbb700 1 -- 172.21.15.92:0/3188891365 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230015ad10 con 0x7f230006c0a0 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f23057bc700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2304fbb700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230015a1e0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f22f00098e0 tx=0x7f22f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9992a2306eff71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f22f7fff700 1 -- 172.21.15.92:0/3188891365 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0009e80 con 0x7f230006c0a0 2023-10-06T03:13:31.624 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f22f7fff700 1 -- 172.21.15.92:0/3188891365 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f000b860 con 0x7f230006c0a0 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f22f7fff700 1 -- 172.21.15.92:0/3188891365 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f000bbe0 con 0x7f230006c0a0 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2307a20700 1 -- 172.21.15.92:0/3188891365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 msgr2=0x7f230015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.622+0000 7f2307a20700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230015a1e0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f22f00098e0 tx=0x7f22f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.623+0000 7f2307a20700 1 -- 172.21.15.92:0/3188891365 shutdown_connections 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.623+0000 7f2307a20700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.623+0000 7f2307a20700 1 --2- 172.21.15.92:0/3188891365 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230015a1e0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.623+0000 7f2307a20700 1 -- 172.21.15.92:0/3188891365 >> 172.21.15.92:0/3188891365 conn(0x7f23000ce600 msgr2=0x7f23000cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.623+0000 7f2307a20700 1 -- 172.21.15.92:0/3188891365 shutdown_connections 2023-10-06T03:13:31.625 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.623+0000 7f2307a20700 1 -- 172.21.15.92:0/3188891365 wait complete. 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.624+0000 7f2307a20700 1 Processor -- start 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.624+0000 7f2307a20700 1 -- start start 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f230015df50 0x7f23000b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230016e4a0 con 0x7f230006c0a0 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230016d820 con 0x7f230015df50 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230016db00 con 0x7f230006bcd0 2023-10-06T03:13:31.628 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41246/0 (socket says 172.21.15.92:41246) 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 -- 172.21.15.92:0/4212728279 learned_addr learned my addr 172.21.15.92:0/4212728279 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f23057bc700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 -- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 msgr2=0x7f23000c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 -- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f230015df50 msgr2=0x7f23000b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f230015df50 0x7f23000b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 -- 172.21.15.92:0/4212728279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0009420 con 0x7f230006c0a0 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2304fbb700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230006d1f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f22f000b3c0 tx=0x7f22f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011420 con 0x7f230006c0a0 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f00115a0 con 0x7f230006c0a0 2023-10-06T03:13:31.629 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230016dde0 con 0x7f230006c0a0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.625+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011960 con 0x7f230006c0a0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.626+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000c6590 con 0x7f230006c0a0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.626+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230016e5e0 con 0x7f230006c0a0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.627+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f22f0011d50 con 0x7f230006c0a0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.627+0000 7f22f57fa700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22e40673a0 0x7f22e4069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.627+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f22f0086ac0 con 0x7f230006c0a0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.627+0000 7f23057bc700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22e40673a0 0x7f22e4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.630 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.628+0000 7f23057bc700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22e40673a0 0x7f22e4069850 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00d380 tx=0x7f22fc007500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:31.632 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.632+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f230016e5e0 con 0x7f230006c0a0 2023-10-06T03:13:31.859 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.859+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f230016e5e0 con 0x7f230006c0a0 2023-10-06T03:13:31.860 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.859+0000 7f22f57fa700 1 -- 172.21.15.92:0/4212728279 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f230016e5e0 con 0x7f230006c0a0 2023-10-06T03:13:31.860 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":137,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:11:20.409587+0000","modified":"2023-10-06T03:11:24.052115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3455480640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3455480640},{"type":"v1","addr":"172.21.15.92:6837","nonce":3455480640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22e40673a0 msgr2=0x7f22e4069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22e40673a0 0x7f22e4069850 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00d380 tx=0x7f22fc007500 comp rx=0 tx=0).stop 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 msgr2=0x7f230006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230006d1f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f22f000b3c0 tx=0x7f22f0017000 comp rx=0 tx=0).stop 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 shutdown_connections 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f22e40673a0 0x7f22e4069850 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f230015df50 0x7f23000b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.863 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f230006bcd0 0x7f23000c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 --2- 172.21.15.92:0/4212728279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f230006c0a0 0x7f230006d1f0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 >> 172.21.15.92:0/4212728279 conn(0x7f23000ce600 msgr2=0x7f23001563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.861+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 shutdown_connections 2023-10-06T03:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.862+0000 7f2307a20700 1 -- 172.21.15.92:0/4212728279 wait complete. 2023-10-06T03:13:31.864 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 137 2023-10-06T03:13:31.874 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:13:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.987+0000 7fe507515700 1 Processor -- start 2023-10-06T03:13:31.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.988+0000 7fe507515700 1 -- start start 2023-10-06T03:13:31.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.988+0000 7fe507515700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe500159c60 0x7fe50015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.988+0000 7fe507515700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe50015db10 0x7fe500069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.988+0000 7fe507515700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50015a160 con 0x7fe500159890 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.988+0000 7fe507515700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50015df50 con 0x7fe50015db10 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.988+0000 7fe507515700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5000d13a0 con 0x7fe500159c60 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe50015db10 0x7fe500069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe50015db10 0x7fe500069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41258/0 (socket says 172.21.15.92:41258) 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 -- 172.21.15.92:0/3657845359 learned_addr learned my addr 172.21.15.92:0/3657845359 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe504ab0700 1 --2- 172.21.15.92:0/3657845359 >> v2:172.21.15.136:3300/0 conn(0x7fe500159c60 0x7fe50015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 -- 172.21.15.92:0/3657845359 >> v2:172.21.15.136:3300/0 conn(0x7fe500159c60 msgr2=0x7fe50015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 --2- 172.21.15.92:0/3657845359 >> v2:172.21.15.136:3300/0 conn(0x7fe500159c60 0x7fe50015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 -- 172.21.15.92:0/3657845359 >> v1:172.21.15.136:6789/0 conn(0x7fe500159890 legacy=0x7fe50015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 -- 172.21.15.92:0/3657845359 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50015e140 con 0x7fe50015db10 2023-10-06T03:13:31.991 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe505ab2700 1 --2- 172.21.15.92:0/3657845359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe50015db10 0x7fe500069560 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe5000b67f0 tx=0x7fe4f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca287f9b290493a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe4f7fff700 1 -- 172.21.15.92:0/3657845359 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800b860 con 0x7fe50015db10 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe4f7fff700 1 -- 172.21.15.92:0/3657845359 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f800b9c0 con 0x7fe50015db10 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.989+0000 7fe4f7fff700 1 -- 172.21.15.92:0/3657845359 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f800bd60 con 0x7fe50015db10 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 -- 172.21.15.92:0/3657845359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe50015db10 msgr2=0x7fe500069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 --2- 172.21.15.92:0/3657845359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe50015db10 0x7fe500069560 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fe5000b67f0 tx=0x7fe4f8009250 comp rx=0 tx=0).stop 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 -- 172.21.15.92:0/3657845359 shutdown_connections 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 --2- 172.21.15.92:0/3657845359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe50015db10 0x7fe500069560 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 --2- 172.21.15.92:0/3657845359 >> v2:172.21.15.136:3300/0 conn(0x7fe500159c60 0x7fe50015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 -- 172.21.15.92:0/3657845359 >> 172.21.15.92:0/3657845359 conn(0x7fe500068e20 msgr2=0x7fe5000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 -- 172.21.15.92:0/3657845359 shutdown_connections 2023-10-06T03:13:31.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.990+0000 7fe507515700 1 -- 172.21.15.92:0/3657845359 wait complete. 2023-10-06T03:13:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.991+0000 7fe507515700 1 Processor -- start 2023-10-06T03:13:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.991+0000 7fe507515700 1 -- start start 2023-10-06T03:13:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe507515700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 0x7fe5000cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe507515700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe500159c60 0x7fe5000b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.995 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe507515700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50015db10 0x7fe5000b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe507515700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000cd260 con 0x7fe500159890 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe507515700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe500151f10 con 0x7fe50015db10 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe507515700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000b1d70 con 0x7fe500159c60 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 0x7fe5000cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 0x7fe5000cb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41274/0 (socket says 172.21.15.92:41274) 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 -- 172.21.15.92:0/3158408916 learned_addr learned my addr 172.21.15.92:0/3158408916 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe505ab2700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50015db10 0x7fe5000b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe504ab0700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe500159c60 0x7fe5000b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 -- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe500159c60 msgr2=0x7fe5000b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe500159c60 0x7fe5000b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 -- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50015db10 msgr2=0x7fe5000b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50015db10 0x7fe5000b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:31.996 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 -- 172.21.15.92:0/3158408916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f80094f0 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe5052b1700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 0x7fe5000cb040 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00cb40 tx=0x7fe4fc001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.992+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00a7e0 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.993+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc00a940 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.993+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5000b2050 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.993+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00ad00 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.993+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5000b2610 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.994+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5000cd0c0 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.994+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe4fc003030 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.994+0000 7fe4f57fa700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4e4067320 0x7fe4e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.994+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fe4fc084eb0 con 0x7fe500159890 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.994+0000 7fe504ab0700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4e4067320 0x7fe4e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:31.997 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.995+0000 7fe504ab0700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4e4067320 0x7fe4e40697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:31.999 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:31.998+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5000cd0c0 con 0x7fe500159890 2023-10-06T03:13:32.180 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.180+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe50014dc30 con 0x7fe500159890 2023-10-06T03:13:32.181 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.181+0000 7fe4f57fa700 1 -- 172.21.15.92:0/3158408916 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+17072 (secure 0 0 0) 0x7fe4fc051970 con 0x7fe500159890 2023-10-06T03:13:32.182 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:32.182 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":203,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:12:26.653173+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:11:18.064170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-06T03:11:19.378751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"35.1","mappings":[{"from":4,"to":1}]},{"pgid":"35.5","mappings":[{"from":4,"to":1}]},{"pgid":"35.1b","mappings":[{"from":6,"to":1}]},{"pgid":"35.24","mappings":[{"from":4,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4e4067320 msgr2=0x7fe4e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4e4067320 0x7fe4e40697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0008040 comp rx=0 tx=0).stop 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 msgr2=0x7fe5000cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 0x7fe5000cb040 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00cb40 tx=0x7fe4fc001770 comp rx=0 tx=0).stop 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 shutdown_connections 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe4e4067320 0x7fe4e40697d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe500159890 0x7fe5000cb040 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe50015db10 0x7fe5000b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.184 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 --2- 172.21.15.92:0/3158408916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe500159c60 0x7fe5000b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 >> 172.21.15.92:0/3158408916 conn(0x7fe500068e20 msgr2=0x7fe500153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:32.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 shutdown_connections 2023-10-06T03:13:32.185 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.183+0000 7fe507515700 1 -- 172.21.15.92:0/3158408916 wait complete. 2023-10-06T03:13:32.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:01:29.224553+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:01:35.968472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 34, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:11:18.064170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 35, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:11:19.378751+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:13:32.196 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:13:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.312+0000 7fd1a1e23700 1 Processor -- start 2023-10-06T03:13:32.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.312+0000 7fd1a1e23700 1 -- start start 2023-10-06T03:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd1a1e23700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd19c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd1a1e23700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c15a160 con 0x7fd19c15db10 2023-10-06T03:13:32.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd1a1e23700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c0d13a0 con 0x7fd19c159890 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd1a1e23700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c15e140 con 0x7fd19c159c60 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd19bfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd19c15db10 0x7fd19c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37088/0 (socket says 172.21.15.92:37088) 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd19bfff700 1 -- 172.21.15.92:0/1650428163 learned_addr learned my addr 172.21.15.92:0/1650428163 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd19affd700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd19c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598021301 0 0) 0x7fd19c15a160 con 0x7fd19c15db10 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.313+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd184003570 con 0x7fd19c15db10 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226255213 0 0) 0x7fd184003570 con 0x7fd19c15db10 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 msgr2=0x7fd19c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd19c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 >> v1:172.21.15.136:6789/0 conn(0x7fd19c159890 legacy=0x7fd19c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c15a3d0 con 0x7fd19c15db10 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fd1940017f0 con 0x7fd19c15db10 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 >> v1:172.21.15.92:6789/0 conn(0x7fd19c15db10 legacy=0x7fd19c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184003a90 0x7fd184003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd184004810 0x7fd184004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd184005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c15e140 con 0x7fd184003a90 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18400bc20 con 0x7fd184004810 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18400bde0 con 0x7fd19c159c60 2023-10-06T03:13:32.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184003a90 0x7fd184003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19affd700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd184004810 0x7fd184004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19bfff700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd184005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 -- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 msgr2=0x7fd184005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd184005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 -- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd184004810 msgr2=0x7fd184004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd184004810 0x7fd184004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 -- 172.21.15.92:0/1650428163 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184003570 con 0x7fd184003a90 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19bfff700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd19c159c60 0x7fd184005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184003a90 0x7fd184003ea0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001880 tx=0x7fd18c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2e7d94bbb6aa4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c012420 con 0x7fd184003a90 2023-10-06T03:13:32.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c012580 con 0x7fd184003a90 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.314+0000 7fd19a7fc700 1 -- 172.21.15.92:0/1650428163 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c012920 con 0x7fd184003a90 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd1a1e23700 1 -- 172.21.15.92:0/1650428163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184003a90 msgr2=0x7fd184003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/1650428163 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184003a90 0x7fd184003ea0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001880 tx=0x7fd18c007d30 comp rx=0 tx=0).stop 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd19bfff700 1 -- 172.21.15.92:0/1650428163 reap_dead start 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd1a1e23700 1 -- 172.21.15.92:0/1650428163 shutdown_connections 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd1a1e23700 1 -- 172.21.15.92:0/1650428163 >> 172.21.15.92:0/1650428163 conn(0x7fd19c068e20 msgr2=0x7fd19c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd1a1e23700 1 -- 172.21.15.92:0/1650428163 shutdown_connections 2023-10-06T03:13:32.320 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.315+0000 7fd1a1e23700 1 -- 172.21.15.92:0/1650428163 wait complete. 2023-10-06T03:13:32.322 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.316+0000 7fd1a1e23700 1 Processor -- start 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 -- start start 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd184003a90 0x7fd19c16d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c0c88e0 0x7fd19c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 0x7fd19c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b55d0 con 0x7fd19c15b720 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b60a0 con 0x7fd19c0c88e0 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd1a1e23700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c0b36c0 con 0x7fd184003a90 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 0x7fd19c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd19bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 0x7fd19c151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41298/0 (socket says 172.21.15.92:41298) 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd19bfff700 1 -- 172.21.15.92:0/2807445310 learned_addr learned my addr 172.21.15.92:0/2807445310 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:32.323 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd184003a90 0x7fd19c16d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.317+0000 7fd19affd700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c0c88e0 0x7fd19c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd19bfff700 1 -- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd184003a90 msgr2=0x7fd19c16d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd19bfff700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd184003a90 0x7fd19c16d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd19bfff700 1 -- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c0c88e0 msgr2=0x7fd19c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd19bfff700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c0c88e0 0x7fd19c0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd19bfff700 1 -- 172.21.15.92:0/2807445310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c0c8d50 con 0x7fd19c15b720 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd19bfff700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 0x7fd19c151b20 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fd194000f80 tx=0x7fd194002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194017070 con 0x7fd19c15b720 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400ec70 con 0x7fd19c15b720 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c0b5800 con 0x7fd19c15b720 2023-10-06T03:13:32.324 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194013420 con 0x7fd19c15b720 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.318+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c0b5dc0 con 0x7fd19c15b720 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.319+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c04c3c0 con 0x7fd19c15b720 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.319+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd1940137d0 con 0x7fd19c15b720 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.319+0000 7fd1837fe700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd188067320 0x7fd1880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.320+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fd194085020 con 0x7fd19c15b720 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.320+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd188067320 0x7fd1880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.320+0000 7fd19b7fe700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd188067320 0x7fd1880697d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001010 tx=0x7fd18c015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.325 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.324+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd194051ae0 con 0x7fd19c15b720 2023-10-06T03:13:32.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.506+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd19c0d1730 con 0x7fd19c15b720 2023-10-06T03:13:32.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.506+0000 7fd1837fe700 1 -- 172.21.15.92:0/2807445310 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v203) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1940017b0 con 0x7fd19c15b720 2023-10-06T03:13:32.507 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2023-10-06T03:13:32.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd188067320 msgr2=0x7fd1880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd188067320 0x7fd1880697d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd18c001010 tx=0x7fd18c015000 comp rx=0 tx=0).stop 2023-10-06T03:13:32.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 msgr2=0x7fd19c151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.509 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 0x7fd19c151b20 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fd194000f80 tx=0x7fd194002110 comp rx=0 tx=0).stop 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 shutdown_connections 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fd188067320 0x7fd1880697d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd19c15b720 0x7fd19c151b20 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd184003a90 0x7fd19c16d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 --2- 172.21.15.92:0/2807445310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c0c88e0 0x7fd19c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 >> 172.21.15.92:0/2807445310 conn(0x7fd19c068e20 msgr2=0x7fd19c15d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 shutdown_connections 2023-10-06T03:13:32.510 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.508+0000 7fd1a1e23700 1 -- 172.21.15.92:0/2807445310 wait complete. 2023-10-06T03:13:32.521 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:13:32.639 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.639+0000 7f7a688f6700 1 Processor -- start 2023-10-06T03:13:32.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.639+0000 7f7a688f6700 1 -- start start 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a688f6700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a64159c60 0x7f7a6415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a688f6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a6415db10 0x7f7a64069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a688f6700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6415a160 con 0x7f7a64159890 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a688f6700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6415df50 con 0x7f7a64159c60 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a688f6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640d13a0 con 0x7f7a6415db10 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a59d9b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7a64159c60 0x7f7a6415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a6259c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7a64159890 0x7f7a6415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37104/0 (socket says 172.21.15.92:37104) 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a6259c700 1 -- 172.21.15.92:0/1414521604 learned_addr learned my addr 172.21.15.92:0/1414521604 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a59d9b700 1 -- 172.21.15.92:0/1414521604 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a6415db10 msgr2=0x7f7a64069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:32.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a59d9b700 1 --2- 172.21.15.92:0/1414521604 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a6415db10 0x7f7a64069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a59d9b700 1 -- 172.21.15.92:0/1414521604 >> v1:172.21.15.92:6789/0 conn(0x7f7a64159890 legacy=0x7f7a6415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a59d9b700 1 -- 172.21.15.92:0/1414521604 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6415e140 con 0x7f7a64159c60 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a61d9b700 1 -- 172.21.15.92:0/1414521604 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359015708 0 0) 0x7f7a6415a160 con 0x7f7a64159890 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a59d9b700 1 --2- 172.21.15.92:0/1414521604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a6415d5d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098e0 tx=0x7f7a4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6c3b342d986f51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.640+0000 7f7a61d9b700 1 -- 172.21.15.92:0/1414521604 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c009e80 con 0x7f7a64159c60 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a61d9b700 1 -- 172.21.15.92:0/1414521604 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00b860 con 0x7f7a64159c60 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a61d9b700 1 -- 172.21.15.92:0/1414521604 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00bbe0 con 0x7f7a64159c60 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 -- 172.21.15.92:0/1414521604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 msgr2=0x7f7a6415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 --2- 172.21.15.92:0/1414521604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a6415d5d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0098e0 tx=0x7f7a4c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 -- 172.21.15.92:0/1414521604 shutdown_connections 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 --2- 172.21.15.92:0/1414521604 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a6415db10 0x7f7a64069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 --2- 172.21.15.92:0/1414521604 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a6415d5d0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 -- 172.21.15.92:0/1414521604 >> 172.21.15.92:0/1414521604 conn(0x7f7a64068e20 msgr2=0x7f7a640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:32.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 -- 172.21.15.92:0/1414521604 shutdown_connections 2023-10-06T03:13:32.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.641+0000 7f7a688f6700 1 -- 172.21.15.92:0/1414521604 wait complete. 2023-10-06T03:13:32.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.642+0000 7f7a688f6700 1 Processor -- start 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.642+0000 7f7a688f6700 1 -- start start 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a688f6700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a64159890 0x7f7a641522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a688f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a640ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a688f6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6415db10 0x7f7a640b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a688f6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640cdde0 con 0x7f7a64159c60 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a688f6700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641527f0 con 0x7f7a6415db10 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a688f6700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a640cd1d0 con 0x7f7a64159890 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a640ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a640ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41316/0 (socket says 172.21.15.92:41316) 2023-10-06T03:13:32.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 -- 172.21.15.92:0/2485445908 learned_addr learned my addr 172.21.15.92:0/2485445908 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a6259c700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a64159890 0x7f7a641522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a62d9d700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6415db10 0x7f7a640b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 -- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a64159890 msgr2=0x7f7a641522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a64159890 0x7f7a641522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 -- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6415db10 msgr2=0x7f7a640b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6415db10 0x7f7a640b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 -- 172.21.15.92:0/2485445908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009420 con 0x7f7a64159c60 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a6259c700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a64159890 0x7f7a641522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a62d9d700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6415db10 0x7f7a640b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.643+0000 7f7a59d9b700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a640ce270 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f7a640899c0 tx=0x7f7a4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.644+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011420 con 0x7f7a64159c60 2023-10-06T03:13:32.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.644+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0115c0 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.644+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a640cd4b0 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.644+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011980 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.644+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a640cda70 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.645+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6404c3c0 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.645+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7a4c0253f0 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.646+0000 7f7a5affd700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a44067320 0x7f7a440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.646+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f7a4c0868e0 con 0x7f7a64159c60 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.646+0000 7f7a6259c700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a44067320 0x7f7a440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.646+0000 7f7a6259c700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a44067320 0x7f7a440697d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7a540018d0 tx=0x7f7a54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.650+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a4c053420 con 0x7f7a64159c60 2023-10-06T03:13:32.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.833+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7a6406a5b0 con 0x7f7a64159c60 2023-10-06T03:13:32.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.833+0000 7f7a5affd700 1 -- 172.21.15.92:0/2485445908 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v203) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7a4c056a40 con 0x7f7a64159c60 2023-10-06T03:13:32.834 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2023-10-06T03:13:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a44067320 msgr2=0x7f7a440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a44067320 0x7f7a440697d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7a540018d0 tx=0x7f7a54009040 comp rx=0 tx=0).stop 2023-10-06T03:13:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 msgr2=0x7f7a640ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a640ce270 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f7a640899c0 tx=0x7f7a4c017000 comp rx=0 tx=0).stop 2023-10-06T03:13:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 shutdown_connections 2023-10-06T03:13:32.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7a44067320 0x7f7a440697d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7a6415db10 0x7f7a640b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7a64159890 0x7f7a641522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 --2- 172.21.15.92:0/2485445908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a64159c60 0x7f7a640ce270 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 >> 172.21.15.92:0/2485445908 conn(0x7f7a64068e20 msgr2=0x7f7a64152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 shutdown_connections 2023-10-06T03:13:32.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.835+0000 7f7a688f6700 1 -- 172.21.15.92:0/2485445908 wait complete. 2023-10-06T03:13:32.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:13:32.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.964+0000 7f04dcf3e700 1 Processor -- start 2023-10-06T03:13:32.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.964+0000 7f04dcf3e700 1 -- start start 2023-10-06T03:13:32.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04dcf3e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04d80b16d0 0x7f04d80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04dcf3e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f04d80b33b0 0x7f04d80b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04dcf3e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d80b2080 con 0x7f04d80b4e70 2023-10-06T03:13:32.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04dcf3e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d80cfb00 con 0x7f04d80b16d0 2023-10-06T03:13:32.967 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04dcf3e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d80b5c40 con 0x7f04d80b33b0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04d80b16d0 0x7f04d80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f04d80b16d0 0x7f04d80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41324/0 (socket says 172.21.15.92:41324) 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 -- 172.21.15.92:0/1761717722 learned_addr learned my addr 172.21.15.92:0/1761717722 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d7fff700 1 --2- 172.21.15.92:0/1761717722 >> v2:172.21.15.136:3300/0 conn(0x7f04d80b33b0 0x7f04d80b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 -- 172.21.15.92:0/1761717722 >> v2:172.21.15.136:3300/0 conn(0x7f04d80b33b0 msgr2=0x7f04d80b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 --2- 172.21.15.92:0/1761717722 >> v2:172.21.15.136:3300/0 conn(0x7f04d80b33b0 0x7f04d80b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 -- 172.21.15.92:0/1761717722 >> v1:172.21.15.136:6789/0 conn(0x7f04d80b4e70 legacy=0x7f04d80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.965+0000 7f04d6ffd700 1 -- 172.21.15.92:0/1761717722 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80b5f20 con 0x7f04d80b16d0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04d6ffd700 1 --2- 172.21.15.92:0/1761717722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80b1b40 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f04d8071380 tx=0x7f04cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528b702c36b68da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04d67fc700 1 -- 172.21.15.92:0/1761717722 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00b820 con 0x7f04d80b16d0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04d67fc700 1 -- 172.21.15.92:0/1761717722 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04cc00b980 con 0x7f04d80b16d0 2023-10-06T03:13:32.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04d67fc700 1 -- 172.21.15.92:0/1761717722 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00bd40 con 0x7f04d80b16d0 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 -- 172.21.15.92:0/1761717722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 msgr2=0x7f04d80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/1761717722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80b1b40 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f04d8071380 tx=0x7f04cc009250 comp rx=0 tx=0).stop 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 -- 172.21.15.92:0/1761717722 shutdown_connections 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/1761717722 >> v2:172.21.15.136:3300/0 conn(0x7f04d80b33b0 0x7f04d80b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/1761717722 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80b1b40 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 -- 172.21.15.92:0/1761717722 >> 172.21.15.92:0/1761717722 conn(0x7f04d80d10c0 msgr2=0x7f04d80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 -- 172.21.15.92:0/1761717722 shutdown_connections 2023-10-06T03:13:32.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.966+0000 7f04dcf3e700 1 -- 172.21.15.92:0/1761717722 wait complete. 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 Processor -- start 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 -- start start 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d80b33b0 0x7f04d80c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04d80b4e70 0x7f04d80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80cc300 con 0x7f04d80b16d0 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d80c9310 con 0x7f04d80b33b0 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04dcf3e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d816b1e0 con 0x7f04d80b4e70 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41330/0 (socket says 172.21.15.92:41330) 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.968+0000 7f04d77fe700 1 -- 172.21.15.92:0/2942128770 learned_addr learned my addr 172.21.15.92:0/2942128770 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:32.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d6ffd700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d80b33b0 0x7f04d80c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d7fff700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04d80b4e70 0x7f04d80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d77fe700 1 -- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04d80b4e70 msgr2=0x7f04d80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d77fe700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04d80b4e70 0x7f04d80affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d77fe700 1 -- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d80b33b0 msgr2=0x7f04d80c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d77fe700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d80b33b0 0x7f04d80c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d77fe700 1 -- 172.21.15.92:0/2942128770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0094f0 con 0x7f04d80b16d0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d6ffd700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d80b33b0 0x7f04d80c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04d77fe700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80cbdc0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c980 tx=0x7f04c8002760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800a7a0 con 0x7f04d80b16d0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c800a900 con 0x7f04d80b16d0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800acc0 con 0x7f04d80b16d0 2023-10-06T03:13:32.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d80c8d50 con 0x7f04d80b16d0 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.969+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d816ae70 con 0x7f04d80b16d0 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.970+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d816ace0 con 0x7f04d80b16d0 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.970+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f04c8012630 con 0x7f04d80b16d0 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.971+0000 7f04bf7fe700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f04c4067320 0x7f04c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.971+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f04c8085370 con 0x7f04d80b16d0 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.971+0000 7f04d6ffd700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f04c4067320 0x7f04c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:32.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.971+0000 7f04d6ffd700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f04c4067320 0x7f04c40697d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f04d8071380 tx=0x7f04cc016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:32.975 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:32.975+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04c8051ee0 con 0x7f04d80b16d0 2023-10-06T03:13:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.156+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04d80b14b0 con 0x7f04d80b16d0 2023-10-06T03:13:33.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.156+0000 7f04bf7fe700 1 -- 172.21.15.92:0/2942128770 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v203) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04c8055500 con 0x7f04d80b16d0 2023-10-06T03:13:33.157 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 32 2023-10-06T03:13:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f04c4067320 msgr2=0x7f04c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f04c4067320 0x7f04c40697d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f04d8071380 tx=0x7f04cc016000 comp rx=0 tx=0).stop 2023-10-06T03:13:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 msgr2=0x7f04d80cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80cbdc0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c980 tx=0x7f04c8002760 comp rx=0 tx=0).stop 2023-10-06T03:13:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 shutdown_connections 2023-10-06T03:13:33.159 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f04c4067320 0x7f04c40697d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d80b16d0 0x7f04d80cbdc0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04d80b4e70 0x7f04d80affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 --2- 172.21.15.92:0/2942128770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d80b33b0 0x7f04d80c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 >> 172.21.15.92:0/2942128770 conn(0x7f04d80d10c0 msgr2=0x7f04d80b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 shutdown_connections 2023-10-06T03:13:33.160 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.158+0000 7f04dcf3e700 1 -- 172.21.15.92:0/2942128770 wait complete. 2023-10-06T03:13:33.171 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:13:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.287+0000 7f6a9c558700 1 Processor -- start 2023-10-06T03:13:33.288 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.287+0000 7f6a9c558700 1 -- start start 2023-10-06T03:13:33.290 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.288+0000 7f6a9c558700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 0x7f6a9415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.288+0000 7f6a9c558700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 0x7f6a9415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.288+0000 7f6a9c558700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9415a160 con 0x7f6a9415db10 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.288+0000 7f6a9c558700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a940d13a0 con 0x7f6a94159890 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.288+0000 7f6a9c558700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9415e140 con 0x7f6a94159c60 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 0x7f6a9415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a9a2f4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 0x7f6a9415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a9a2f4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 0x7f6a9415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:49736/0 (socket says 172.21.15.92:49736) 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 0x7f6a9415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:47032/0 (socket says 172.21.15.92:47032) 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a9a2f4700 1 -- 172.21.15.92:0/3873436396 learned_addr learned my addr 172.21.15.92:0/3873436396 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:33.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a9aaf5700 1 --1- 172.21.15.92:0/3873436396 >> v1:172.21.15.136:6789/0 conn(0x7f6a9415db10 0x7f6a94069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.92:53730/0 (socket says 172.21.15.92:53730) 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 -- 172.21.15.92:0/3873436396 >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 msgr2=0x7f6a9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 --2- 172.21.15.92:0/3873436396 >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 0x7f6a9415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 -- 172.21.15.92:0/3873436396 >> v1:172.21.15.136:6789/0 conn(0x7f6a9415db10 legacy=0x7f6a94069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 -- 172.21.15.92:0/3873436396 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9415a3d0 con 0x7f6a94159c60 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a992f2700 1 -- 172.21.15.92:0/3873436396 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697053308 0 0) 0x7f6a9415a160 con 0x7f6a9415db10 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a9a2f4700 1 --2- 172.21.15.92:0/3873436396 >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 0x7f6a9415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a99af3700 1 --2- 172.21.15.92:0/3873436396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 0x7f6a9415d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a840098e0 tx=0x7f6a840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08874c53b3541cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a992f2700 1 -- 172.21.15.92:0/3873436396 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400b640 con 0x7f6a94159c60 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a992f2700 1 -- 172.21.15.92:0/3873436396 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8400b7a0 con 0x7f6a94159c60 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.289+0000 7f6a992f2700 1 -- 172.21.15.92:0/3873436396 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400bb40 con 0x7f6a94159c60 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 -- 172.21.15.92:0/3873436396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 msgr2=0x7f6a9415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.292 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3873436396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 0x7f6a9415d5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a840098e0 tx=0x7f6a840091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 -- 172.21.15.92:0/3873436396 shutdown_connections 2023-10-06T03:13:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3873436396 >> v2:172.21.15.136:3300/0 conn(0x7f6a94159890 0x7f6a9415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3873436396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159c60 0x7f6a9415d5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 -- 172.21.15.92:0/3873436396 >> 172.21.15.92:0/3873436396 conn(0x7f6a94068e20 msgr2=0x7f6a940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 -- 172.21.15.92:0/3873436396 shutdown_connections 2023-10-06T03:13:33.293 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.290+0000 7f6a9c558700 1 -- 172.21.15.92:0/3873436396 wait complete. 2023-10-06T03:13:33.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.291+0000 7f6a9c558700 1 Processor -- start 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.291+0000 7f6a9c558700 1 -- start start 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159890 0x7f6a9415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a94159c60 0x7f6a940ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 0x7f6a940ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9416e360 con 0x7f6a9415db10 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a94152c00 con 0x7f6a94159c60 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9416ded0 con 0x7f6a94159890 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 0x7f6a940ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 0x7f6a940ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41338/0 (socket says 172.21.15.92:41338) 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 -- 172.21.15.92:0/3580861299 learned_addr learned my addr 172.21.15.92:0/3580861299 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:33.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a99af3700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a94159c60 0x7f6a940ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9a2f4700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159890 0x7f6a9415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 -- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159890 msgr2=0x7f6a9415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159890 0x7f6a9415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 -- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a94159c60 msgr2=0x7f6a940ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a94159c60 0x7f6a940ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 -- 172.21.15.92:0/3580861299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a900094f0 con 0x7f6a9415db10 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a99af3700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a94159c60 0x7f6a940ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9a2f4700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159890 0x7f6a9415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9aaf5700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 0x7f6a940ce7c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00ccc0 tx=0x7f6a8c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c005df0 con 0x7f6a9415db10 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c00a900 con 0x7f6a9415db10 2023-10-06T03:13:33.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a84009420 con 0x7f6a9415db10 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.292+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00acc0 con 0x7f6a9415db10 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.293+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9415d560 con 0x7f6a9415db10 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.293+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9416e080 con 0x7f6a9415db10 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.294+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6a8c007300 con 0x7f6a9415db10 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.294+0000 7f6a8a7fc700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a78067320 0x7f6a780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.294+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f6a8c084f00 con 0x7f6a9415db10 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.294+0000 7f6a9a2f4700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a78067320 0x7f6a780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.295+0000 7f6a9a2f4700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a78067320 0x7f6a780697d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6a900094c0 tx=0x7f6a90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.299+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a8c051a40 con 0x7f6a9415db10 2023-10-06T03:13:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.480+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a9404c3c0 con 0x7f6a9415db10 2023-10-06T03:13:33.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.480+0000 7f6a8a7fc700 1 -- 172.21.15.92:0/3580861299 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v203) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a8c055060 con 0x7f6a9415db10 2023-10-06T03:13:33.481 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 64 2023-10-06T03:13:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a78067320 msgr2=0x7f6a780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a78067320 0x7f6a780697d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6a900094c0 tx=0x7f6a90008040 comp rx=0 tx=0).stop 2023-10-06T03:13:33.483 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 msgr2=0x7f6a940ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 0x7f6a940ce7c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00ccc0 tx=0x7f6a8c001a20 comp rx=0 tx=0).stop 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 shutdown_connections 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6a78067320 0x7f6a780697d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6a9415db10 0x7f6a940ce7c0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a94159890 0x7f6a9415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 --2- 172.21.15.92:0/3580861299 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a94159c60 0x7f6a940ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.482+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 >> 172.21.15.92:0/3580861299 conn(0x7f6a94068e20 msgr2=0x7f6a94153420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.483+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 shutdown_connections 2023-10-06T03:13:33.484 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.483+0000 7f6a9c558700 1 -- 172.21.15.92:0/3580861299 wait complete. 2023-10-06T03:13:33.495 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:13:33.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.609+0000 7fc1c473d700 1 Processor -- start 2023-10-06T03:13:33.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.609+0000 7fc1c473d700 1 -- start start 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c473d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1bc159c60 0x7fc1bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c473d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc1bc15db10 0x7fc1bc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c473d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc15a160 con 0x7fc1bc159890 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c473d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc15df50 con 0x7fc1bc159c60 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c473d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0d13a0 con 0x7fc1bc15db10 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1bc159c60 0x7fc1bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc1bc159c60 0x7fc1bc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41344/0 (socket says 172.21.15.92:41344) 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/2279217905 learned_addr learned my addr 172.21.15.92:0/2279217905 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:33.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c24d9700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc1bc159890 0x7fc1bc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37114/0 (socket says 172.21.15.92:37114) 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c2cda700 1 --2- 172.21.15.92:0/2279217905 >> v2:172.21.15.136:3300/0 conn(0x7fc1bc15db10 0x7fc1bc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/2279217905 >> v2:172.21.15.136:3300/0 conn(0x7fc1bc15db10 msgr2=0x7fc1bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 --2- 172.21.15.92:0/2279217905 >> v2:172.21.15.136:3300/0 conn(0x7fc1bc15db10 0x7fc1bc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/2279217905 >> v1:172.21.15.92:6789/0 conn(0x7fc1bc159890 legacy=0x7fc1bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/2279217905 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc15e140 con 0x7fc1bc159c60 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c14d7700 1 -- 172.21.15.92:0/2279217905 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644023427 0 0) 0x7fc1bc15a160 con 0x7fc1bc159890 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c1cd8700 1 --2- 172.21.15.92:0/2279217905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc15d5d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06a890 tx=0x7fc1ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b9f0119b9c1235 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.610+0000 7fc1c14d7700 1 -- 172.21.15.92:0/2279217905 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00b860 con 0x7fc1bc159c60 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c14d7700 1 -- 172.21.15.92:0/2279217905 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac00b9c0 con 0x7fc1bc159c60 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c14d7700 1 -- 172.21.15.92:0/2279217905 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00bd60 con 0x7fc1bc159c60 2023-10-06T03:13:33.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 -- 172.21.15.92:0/2279217905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 msgr2=0x7fc1bc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 --2- 172.21.15.92:0/2279217905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc15d5d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc06a890 tx=0x7fc1ac009250 comp rx=0 tx=0).stop 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 -- 172.21.15.92:0/2279217905 shutdown_connections 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 --2- 172.21.15.92:0/2279217905 >> v2:172.21.15.136:3300/0 conn(0x7fc1bc15db10 0x7fc1bc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 --2- 172.21.15.92:0/2279217905 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc15d5d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 -- 172.21.15.92:0/2279217905 >> 172.21.15.92:0/2279217905 conn(0x7fc1bc068e20 msgr2=0x7fc1bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 -- 172.21.15.92:0/2279217905 shutdown_connections 2023-10-06T03:13:33.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.611+0000 7fc1c473d700 1 -- 172.21.15.92:0/2279217905 wait complete. 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.612+0000 7fc1c473d700 1 Processor -- start 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.612+0000 7fc1c473d700 1 -- start start 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c473d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1bc159890 0x7fc1bc14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c473d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c473d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1bc15db10 0x7fc1bc151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c473d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0b1fb0 con 0x7fc1bc159c60 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c473d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c5ff0 con 0x7fc1bc159890 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c473d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0c56c0 con 0x7fc1bc15db10 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.617 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc0cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41358/0 (socket says 172.21.15.92:41358) 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/1893520354 learned_addr learned my addr 172.21.15.92:0/1893520354 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c24d9700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1bc159890 0x7fc1bc14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c2cda700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1bc15db10 0x7fc1bc151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1bc15db10 msgr2=0x7fc1bc151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1bc15db10 0x7fc1bc151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1bc159890 msgr2=0x7fc1bc14b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1bc159890 0x7fc1bc14b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.613+0000 7fc1c1cd8700 1 -- 172.21.15.92:0/1893520354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0094f0 con 0x7fc1bc159c60 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.614+0000 7fc1c1cd8700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc0cd610 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac006f50 tx=0x7fc1ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.618 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.614+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac024440 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.614+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac0245e0 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.614+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc0c59a0 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.614+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0249a0 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.614+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc14bdd0 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.615+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc0c6160 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.615+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc1ac024d50 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.615+0000 7fc1b27fc700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1a0067320 0x7fc1a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.616+0000 7fc1c24d9700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1a0067320 0x7fc1a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.616+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fc1ac086b80 con 0x7fc1bc159c60 2023-10-06T03:13:33.619 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.616+0000 7fc1c24d9700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1a0067320 0x7fc1a00697d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4001560 tx=0x7fc1b4005d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.620 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.620+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1ac053640 con 0x7fc1bc159c60 2023-10-06T03:13:33.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.847+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1bc04c3c0 con 0x7fc1bc159c60 2023-10-06T03:13:33.848 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.848+0000 7fc1b27fc700 1 -- 172.21.15.92:0/1893520354 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fc1ac02b340 con 0x7fc1bc159c60 2023-10-06T03:13:33.849 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:33.849 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":137,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:11:20.409587+0000","modified":"2023-10-06T03:11:24.052115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3455480640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3455480640},{"type":"v1","addr":"172.21.15.92:6837","nonce":3455480640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T03:13:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1a0067320 msgr2=0x7fc1a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1a0067320 0x7fc1a00697d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4001560 tx=0x7fc1b4005d10 comp rx=0 tx=0).stop 2023-10-06T03:13:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 msgr2=0x7fc1bc0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc0cd610 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac006f50 tx=0x7fc1ac017000 comp rx=0 tx=0).stop 2023-10-06T03:13:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 shutdown_connections 2023-10-06T03:13:33.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc1a0067320 0x7fc1a00697d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1bc15db10 0x7fc1bc151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1bc159890 0x7fc1bc14b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 --2- 172.21.15.92:0/1893520354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1bc159c60 0x7fc1bc0cd610 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 >> 172.21.15.92:0/1893520354 conn(0x7fc1bc068e20 msgr2=0x7fc1bc1573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 shutdown_connections 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.850+0000 7fc1c473d700 1 -- 172.21.15.92:0/1893520354 wait complete. 2023-10-06T03:13:33.852 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 137 2023-10-06T03:13:33.863 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:13:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.976+0000 7f4576488700 1 Processor -- start 2023-10-06T03:13:33.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.976+0000 7f4576488700 1 -- start start 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4576488700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159890 0x7f457015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4576488700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4570159c60 0x7f457015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4576488700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f457015db10 0x7f4570069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4576488700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457015a160 con 0x7f457015db10 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4576488700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45700d13a0 con 0x7f4570159c60 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4576488700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457015e140 con 0x7f4570159890 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f457015db10 0x7f4570069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f457015db10 0x7f4570069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41368/0 (socket says 172.21.15.92:41368) 2023-10-06T03:13:33.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 -- 172.21.15.92:0/2408627368 learned_addr learned my addr 172.21.15.92:0/2408627368 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f456f7fe700 1 --2- 172.21.15.92:0/2408627368 >> v2:172.21.15.136:3300/0 conn(0x7f4570159c60 0x7f457015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f456ffff700 1 --2- 172.21.15.92:0/2408627368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159890 0x7f457015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 -- 172.21.15.92:0/2408627368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159890 msgr2=0x7f457015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 --2- 172.21.15.92:0/2408627368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159890 0x7f457015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 -- 172.21.15.92:0/2408627368 >> v2:172.21.15.136:3300/0 conn(0x7f4570159c60 msgr2=0x7f457015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 --2- 172.21.15.92:0/2408627368 >> v2:172.21.15.136:3300/0 conn(0x7f4570159c60 0x7f457015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 -- 172.21.15.92:0/2408627368 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457015a3d0 con 0x7f457015db10 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f456f7fe700 1 --2- 172.21.15.92:0/2408627368 >> v2:172.21.15.136:3300/0 conn(0x7f4570159c60 0x7f457015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f4574a25700 1 --2- 172.21.15.92:0/2408627368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f4570069560 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f45700650f0 tx=0x7f4568009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdf10e71e23d649 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f456effd700 1 -- 172.21.15.92:0/2408627368 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800e070 con 0x7f457015db10 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.977+0000 7f456effd700 1 -- 172.21.15.92:0/2408627368 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800ba50 con 0x7f457015db10 2023-10-06T03:13:33.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f456effd700 1 -- 172.21.15.92:0/2408627368 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680092e0 con 0x7f457015db10 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 -- 172.21.15.92:0/2408627368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 msgr2=0x7f4570069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 --2- 172.21.15.92:0/2408627368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f4570069560 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f45700650f0 tx=0x7f4568009840 comp rx=0 tx=0).stop 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 -- 172.21.15.92:0/2408627368 shutdown_connections 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 --2- 172.21.15.92:0/2408627368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159890 0x7f457015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 --2- 172.21.15.92:0/2408627368 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f4570069560 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 --2- 172.21.15.92:0/2408627368 >> v2:172.21.15.136:3300/0 conn(0x7f4570159c60 0x7f457015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 -- 172.21.15.92:0/2408627368 >> 172.21.15.92:0/2408627368 conn(0x7f4570068e20 msgr2=0x7f45700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 -- 172.21.15.92:0/2408627368 shutdown_connections 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.978+0000 7f4576488700 1 -- 172.21.15.92:0/2408627368 wait complete. 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.979+0000 7f4576488700 1 Processor -- start 2023-10-06T03:13:33.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.979+0000 7f4576488700 1 -- start start 2023-10-06T03:13:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4576488700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4570159890 0x7f45701491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4576488700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159c60 0x7f45700b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4576488700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f45700b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4576488700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45700b4f70 con 0x7f457015db10 2023-10-06T03:13:33.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4576488700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4570155bb0 con 0x7f4570159890 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4576488700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45700b46d0 con 0x7f4570159c60 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f45700b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f45700b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41382/0 (socket says 172.21.15.92:41382) 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 -- 172.21.15.92:0/1861740248 learned_addr learned my addr 172.21.15.92:0/1861740248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f456f7fe700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159c60 0x7f45700b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f456ffff700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4570159890 0x7f45701491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 -- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159c60 msgr2=0x7f45700b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159c60 0x7f45700b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 -- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4570159890 msgr2=0x7f45701491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4570159890 0x7f45701491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:33.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f4574a25700 1 -- 172.21.15.92:0/1861740248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568009ae0 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f456f7fe700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159c60 0x7f45700b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.980+0000 7f456ffff700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4570159890 0x7f45701491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.981+0000 7f4574a25700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f45700b1f60 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4570070e20 tx=0x7f4568018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.981+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680032f0 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.981+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4568008e40 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.981+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45700b49b0 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.981+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680256b0 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.981+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45700ce710 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.982+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45700ce570 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.982+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4568003040 con 0x7f457015db10 2023-10-06T03:13:33.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.982+0000 7f4557fff700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f455c067320 0x7f455c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.983+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f4568086520 con 0x7f457015db10 2023-10-06T03:13:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.983+0000 7f456ffff700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f455c067320 0x7f455c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.983+0000 7f456ffff700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f455c067320 0x7f455c0697d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4564006fd0 tx=0x7f4564008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:33.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:33.987+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45700ce570 con 0x7f457015db10 2023-10-06T03:13:34.173 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.173+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45700ce570 con 0x7f457015db10 2023-10-06T03:13:34.174 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.174+0000 7f4557fff700 1 -- 172.21.15.92:0/1861740248 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+17072 (secure 0 0 0) 0x7f45700ce570 con 0x7f457015db10 2023-10-06T03:13:34.175 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:34.175 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":203,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:12:26.653173+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:11:18.064170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-06T03:11:19.378751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"35.1","mappings":[{"from":4,"to":1}]},{"pgid":"35.5","mappings":[{"from":4,"to":1}]},{"pgid":"35.1b","mappings":[{"from":6,"to":1}]},{"pgid":"35.24","mappings":[{"from":4,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:13:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.175+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f455c067320 msgr2=0x7f455c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.176 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.175+0000 7f4576488700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f455c067320 0x7f455c0697d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4564006fd0 tx=0x7f4564008040 comp rx=0 tx=0).stop 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.175+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 msgr2=0x7f45700b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.175+0000 7f4576488700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f45700b1f60 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4570070e20 tx=0x7f4568018000 comp rx=0 tx=0).stop 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 shutdown_connections 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f455c067320 0x7f455c0697d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f457015db10 0x7f45700b1f60 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4570159890 0x7f45701491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 --2- 172.21.15.92:0/1861740248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4570159c60 0x7f45700b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 >> 172.21.15.92:0/1861740248 conn(0x7f4570068e20 msgr2=0x7f457015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 shutdown_connections 2023-10-06T03:13:34.177 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.176+0000 7f4576488700 1 -- 172.21.15.92:0/1861740248 wait complete. 2023-10-06T03:13:34.188 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:13:34.189 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:13:34.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.303+0000 7f9c6cdc4700 1 Processor -- start 2023-10-06T03:13:34.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.303+0000 7f9c6cdc4700 1 -- start start 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6cdc4700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9c68159890 0x7f9c6815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6cdc4700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9c68159c60 0x7f9c6815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6cdc4700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6815a160 con 0x7f9c6815db10 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6cdc4700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6815df50 con 0x7f9c68159890 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6cdc4700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c680d13a0 con 0x7f9c68159c60 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9c68159890 0x7f9c6815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9c68159890 0x7f9c6815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41390/0 (socket says 172.21.15.92:41390) 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 -- 172.21.15.92:0/3448668149 learned_addr learned my addr 172.21.15.92:0/3448668149 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 -- 172.21.15.92:0/3448668149 >> v2:172.21.15.136:3300/0 conn(0x7f9c68159c60 msgr2=0x7f9c6815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c5dd9b700 1 --2- 172.21.15.92:0/3448668149 >> v2:172.21.15.136:3300/0 conn(0x7f9c68159c60 0x7f9c6815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 --2- 172.21.15.92:0/3448668149 >> v2:172.21.15.136:3300/0 conn(0x7f9c68159c60 0x7f9c6815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 -- 172.21.15.92:0/3448668149 >> v1:172.21.15.136:6789/0 conn(0x7f9c6815db10 legacy=0x7f9c68069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 -- 172.21.15.92:0/3448668149 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6815e140 con 0x7f9c68159890 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.304+0000 7f9c6659c700 1 --2- 172.21.15.92:0/3448668149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159890 0x7f9c6815abd0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f9c680b67f0 tx=0x7f9c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c5e24357fe5836 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c65d9b700 1 -- 172.21.15.92:0/3448668149 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800b820 con 0x7f9c68159890 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c65d9b700 1 -- 172.21.15.92:0/3448668149 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5800b980 con 0x7f9c68159890 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c65d9b700 1 -- 172.21.15.92:0/3448668149 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800bd20 con 0x7f9c68159890 2023-10-06T03:13:34.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/3448668149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159890 msgr2=0x7f9c6815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/3448668149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159890 0x7f9c6815abd0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f9c680b67f0 tx=0x7f9c58009250 comp rx=0 tx=0).stop 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/3448668149 shutdown_connections 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/3448668149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159890 0x7f9c6815abd0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/3448668149 >> v2:172.21.15.136:3300/0 conn(0x7f9c68159c60 0x7f9c6815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/3448668149 >> 172.21.15.92:0/3448668149 conn(0x7f9c68068e20 msgr2=0x7f9c680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/3448668149 shutdown_connections 2023-10-06T03:13:34.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.305+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/3448668149 wait complete. 2023-10-06T03:13:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.306+0000 7f9c6cdc4700 1 Processor -- start 2023-10-06T03:13:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.306+0000 7f9c6cdc4700 1 -- start start 2023-10-06T03:13:34.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c6cdc4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c68159890 0x7f9c680b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c6cdc4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 0x7f9c680ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c6cdc4700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c6815db10 0x7f9c68152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c6cdc4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680caa70 con 0x7f9c68159c60 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c6cdc4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68152dd0 con 0x7f9c68159890 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c6cdc4700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680c7940 con 0x7f9c6815db10 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 0x7f9c680ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 0x7f9c680ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41406/0 (socket says 172.21.15.92:41406) 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 -- 172.21.15.92:0/719969500 learned_addr learned my addr 172.21.15.92:0/719969500 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:34.311 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 -- 172.21.15.92:0/719969500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c6815db10 msgr2=0x7f9c68152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c6815db10 0x7f9c68152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 -- 172.21.15.92:0/719969500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c68159890 msgr2=0x7f9c680b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c68159890 0x7f9c680b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.307+0000 7f9c5dd9b700 1 -- 172.21.15.92:0/719969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580094f0 con 0x7f9c68159c60 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.308+0000 7f9c5dd9b700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 0x7f9c680ca530 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c7d0 tx=0x7f9c50008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.308+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50011600 con 0x7f9c68159c60 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.308+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50011760 con 0x7f9c68159c60 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.308+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c680c7ec0 con 0x7f9c68159c60 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.308+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50011b20 con 0x7f9c68159c60 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.308+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c680b5070 con 0x7f9c68159c60 2023-10-06T03:13:34.312 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.309+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c680b5260 con 0x7f9c68159c60 2023-10-06T03:13:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.309+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9c500193f0 con 0x7f9c68159c60 2023-10-06T03:13:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.310+0000 7f9c5effd700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c48067320 0x7f9c480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.310+0000 7f9c6659c700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c48067320 0x7f9c480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.310+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f9c50085330 con 0x7f9c68159c60 2023-10-06T03:13:34.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.310+0000 7f9c6659c700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c48067320 0x7f9c480697d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9c680b67f0 tx=0x7f9c58016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:34.314 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.314+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c50051ce0 con 0x7f9c68159c60 2023-10-06T03:13:34.537 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.537+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c6804c3c0 con 0x7f9c68159c60 2023-10-06T03:13:34.538 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.537+0000 7f9c5effd700 1 -- 172.21.15.92:0/719969500 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f9c50055300 con 0x7f9c68159c60 2023-10-06T03:13:34.538 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:34.538 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":137,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:11:20.409587+0000","modified":"2023-10-06T03:11:24.052115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3455480640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3455480640},{"type":"v1","addr":"172.21.15.92:6837","nonce":3455480640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T03:13:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c48067320 msgr2=0x7f9c480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c48067320 0x7f9c480697d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9c680b67f0 tx=0x7f9c58016000 comp rx=0 tx=0).stop 2023-10-06T03:13:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 msgr2=0x7f9c680ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 0x7f9c680ca530 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c7d0 tx=0x7f9c50008040 comp rx=0 tx=0).stop 2023-10-06T03:13:34.540 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 shutdown_connections 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9c48067320 0x7f9c480697d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c6815db10 0x7f9c68152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c68159890 0x7f9c680b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 --2- 172.21.15.92:0/719969500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c68159c60 0x7f9c680ca530 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 >> 172.21.15.92:0/719969500 conn(0x7f9c68068e20 msgr2=0x7f9c681530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 shutdown_connections 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.539+0000 7f9c6cdc4700 1 -- 172.21.15.92:0/719969500 wait complete. 2023-10-06T03:13:34.541 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 137 2023-10-06T03:13:34.552 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:13:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.665+0000 7fdc833cf700 1 Processor -- start 2023-10-06T03:13:34.666 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.665+0000 7fdc833cf700 1 -- start start 2023-10-06T03:13:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc833cf700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc7c159b10 0x7fdc7c0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc833cf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c15d730 0x7fdc7c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc833cf700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0c74b0 con 0x7fdc7c159ee0 2023-10-06T03:13:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc833cf700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c0c8700 con 0x7fdc7c159b10 2023-10-06T03:13:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc833cf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0c9070 con 0x7fdc7c15d730 2023-10-06T03:13:34.668 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc8116b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc7c159b10 0x7fdc7c0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc8116b700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fdc7c159b10 0x7fdc7c0cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41412/0 (socket says 172.21.15.92:41412) 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.666+0000 7fdc8116b700 1 -- 172.21.15.92:0/3434244818 learned_addr learned my addr 172.21.15.92:0/3434244818 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc8116b700 1 -- 172.21.15.92:0/3434244818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c15d730 msgr2=0x7fdc7c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc8096a700 1 --2- 172.21.15.92:0/3434244818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c15d730 0x7fdc7c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc8116b700 1 --2- 172.21.15.92:0/3434244818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c15d730 0x7fdc7c15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc8116b700 1 -- 172.21.15.92:0/3434244818 >> v1:172.21.15.136:6789/0 conn(0x7fdc7c159ee0 legacy=0x7fdc7c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc8116b700 1 -- 172.21.15.92:0/3434244818 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c15e0e0 con 0x7fdc7c159b10 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc8116b700 1 --2- 172.21.15.92:0/3434244818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159b10 0x7fdc7c0cb3e0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009f20 tx=0x7fdc78009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636d615e3672ed67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc73fff700 1 -- 172.21.15.92:0/3434244818 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800e070 con 0x7fdc7c159b10 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc73fff700 1 -- 172.21.15.92:0/3434244818 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7800bad0 con 0x7fdc7c159b10 2023-10-06T03:13:34.669 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc73fff700 1 -- 172.21.15.92:0/3434244818 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780092e0 con 0x7fdc7c159b10 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc833cf700 1 -- 172.21.15.92:0/3434244818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159b10 msgr2=0x7fdc7c0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc833cf700 1 --2- 172.21.15.92:0/3434244818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159b10 0x7fdc7c0cb3e0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009f20 tx=0x7fdc78009840 comp rx=0 tx=0).stop 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc833cf700 1 -- 172.21.15.92:0/3434244818 shutdown_connections 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc833cf700 1 --2- 172.21.15.92:0/3434244818 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159b10 0x7fdc7c0cb3e0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc833cf700 1 --2- 172.21.15.92:0/3434244818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c15d730 0x7fdc7c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.667+0000 7fdc833cf700 1 -- 172.21.15.92:0/3434244818 >> 172.21.15.92:0/3434244818 conn(0x7fdc7c0c9870 msgr2=0x7fdc7c0c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:34.670 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.668+0000 7fdc833cf700 1 -- 172.21.15.92:0/3434244818 shutdown_connections 2023-10-06T03:13:34.671 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.668+0000 7fdc833cf700 1 -- 172.21.15.92:0/3434244818 wait complete. 2023-10-06T03:13:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.669+0000 7fdc833cf700 1 Processor -- start 2023-10-06T03:13:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.669+0000 7fdc833cf700 1 -- start start 2023-10-06T03:13:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.669+0000 7fdc833cf700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c159b10 0x7fdc7c0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.669+0000 7fdc833cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 0x7fdc7c0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.669+0000 7fdc833cf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc7c15d730 0x7fdc7c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.674 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc833cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0b1c40 con 0x7fdc7c159ee0 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc833cf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c15bd50 con 0x7fdc7c15d730 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc833cf700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c0af5f0 con 0x7fdc7c159b10 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 0x7fdc7c0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 0x7fdc7c0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41426/0 (socket says 172.21.15.92:41426) 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 -- 172.21.15.92:0/4010282720 learned_addr learned my addr 172.21.15.92:0/4010282720 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8116b700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c159b10 0x7fdc7c0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8196c700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc7c15d730 0x7fdc7c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 -- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c159b10 msgr2=0x7fdc7c0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.675 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c159b10 0x7fdc7c0b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 -- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc7c15d730 msgr2=0x7fdc7c0b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc7c15d730 0x7fdc7c0b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 -- 172.21.15.92:0/4010282720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009ae0 con 0x7fdc7c159ee0 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc8096a700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 0x7fdc7c0af0b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c880 tx=0x7fdc6c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c011600 con 0x7fdc7c159ee0 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c011760 con 0x7fdc7c159ee0 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c15b420 con 0x7fdc7c159ee0 2023-10-06T03:13:34.676 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c011b00 con 0x7fdc7c159ee0 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.670+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c15b9e0 con 0x7fdc7c159ee0 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.671+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c15bee0 con 0x7fdc7c159ee0 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.672+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdc6c018360 con 0x7fdc7c159ee0 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.672+0000 7fdc717fa700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc60067320 0x7fdc600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.672+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7fdc6c084e10 con 0x7fdc7c159ee0 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.672+0000 7fdc8116b700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc60067320 0x7fdc600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:34.677 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.673+0000 7fdc8116b700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc60067320 0x7fdc600697d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009880 tx=0x7fdc78017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:34.678 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.676+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc6c088030 con 0x7fdc7c159ee0 2023-10-06T03:13:34.904 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.903+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc7c04c3c0 con 0x7fdc7c159ee0 2023-10-06T03:13:34.905 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.904+0000 7fdc717fa700 1 -- 172.21.15.92:0/4010282720 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fdc6c015090 con 0x7fdc7c159ee0 2023-10-06T03:13:34.905 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:34.905 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":137,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:11:20.409587+0000","modified":"2023-10-06T03:11:24.052115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9198},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9198":{"gid":9198,"name":"c","rank":0,"incarnation":132,"state":"up:active","state_seq":3,"addr":"172.21.15.92:6837/3455480640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":3455480640},{"type":"v1","addr":"172.21.15.92:6837","nonce":3455480640}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[35],"metadata_pool":34,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc60067320 msgr2=0x7fdc600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc60067320 0x7fdc600697d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdc78009880 tx=0x7fdc78017000 comp rx=0 tx=0).stop 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 msgr2=0x7fdc7c0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 0x7fdc7c0af0b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c880 tx=0x7fdc6c008040 comp rx=0 tx=0).stop 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 shutdown_connections 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fdc60067320 0x7fdc600697d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.907 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc7c15d730 0x7fdc7c0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc7c159b10 0x7fdc7c0b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 --2- 172.21.15.92:0/4010282720 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc7c159ee0 0x7fdc7c0af0b0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 >> 172.21.15.92:0/4010282720 conn(0x7fdc7c0c9870 msgr2=0x7fdc7c155ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 shutdown_connections 2023-10-06T03:13:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:34.906+0000 7fdc833cf700 1 -- 172.21.15.92:0/4010282720 wait complete. 2023-10-06T03:13:34.908 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 137 2023-10-06T03:13:34.919 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:13:35.032 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.032+0000 7f3476474700 1 Processor -- start 2023-10-06T03:13:35.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.032+0000 7f3476474700 1 -- start start 2023-10-06T03:13:35.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3476474700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3470159420 0x7f34700b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3476474700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f34701597f0 0x7f34700b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3476474700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34700b11d0 con 0x7f347015d230 2023-10-06T03:13:35.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3476474700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34700b14b0 con 0x7f3470159420 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3476474700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34700b1790 con 0x7f34701597f0 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3470159420 0x7f34700b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3470159420 0x7f34700b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41442/0 (socket says 172.21.15.92:41442) 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475c73700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f347015d230 0x7f347015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37116/0 (socket says 172.21.15.92:37116) 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 -- 172.21.15.92:0/833069949 learned_addr learned my addr 172.21.15.92:0/833069949 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3474c71700 1 --2- 172.21.15.92:0/833069949 >> v2:172.21.15.136:3300/0 conn(0x7f34701597f0 0x7f34700b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 -- 172.21.15.92:0/833069949 >> v2:172.21.15.136:3300/0 conn(0x7f34701597f0 msgr2=0x7f34700b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 --2- 172.21.15.92:0/833069949 >> v2:172.21.15.136:3300/0 conn(0x7f34701597f0 0x7f34700b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 -- 172.21.15.92:0/833069949 >> v1:172.21.15.92:6789/0 conn(0x7f347015d230 legacy=0x7f347015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3467fff700 1 -- 172.21.15.92:0/833069949 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784747733 0 0) 0x7f34700b11d0 con 0x7f347015d230 2023-10-06T03:13:35.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 -- 172.21.15.92:0/833069949 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34700b1a70 con 0x7f3470159420 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3474c71700 1 --2- 172.21.15.92:0/833069949 >> v2:172.21.15.136:3300/0 conn(0x7f34701597f0 0x7f34700b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.033+0000 7f3475472700 1 --2- 172.21.15.92:0/833069949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700b5f40 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3470089750 tx=0x7f3468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46843a320c35012 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3467fff700 1 -- 172.21.15.92:0/833069949 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800b860 con 0x7f3470159420 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3467fff700 1 -- 172.21.15.92:0/833069949 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346800b9c0 con 0x7f3470159420 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3467fff700 1 -- 172.21.15.92:0/833069949 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800bd60 con 0x7f3470159420 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 -- 172.21.15.92:0/833069949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 msgr2=0x7f34700b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 --2- 172.21.15.92:0/833069949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700b5f40 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f3470089750 tx=0x7f3468009250 comp rx=0 tx=0).stop 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 -- 172.21.15.92:0/833069949 shutdown_connections 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 --2- 172.21.15.92:0/833069949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700b5f40 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 --2- 172.21.15.92:0/833069949 >> v2:172.21.15.136:3300/0 conn(0x7f34701597f0 0x7f34700b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 -- 172.21.15.92:0/833069949 >> 172.21.15.92:0/833069949 conn(0x7f34700d0660 msgr2=0x7f34700d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:35.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 -- 172.21.15.92:0/833069949 shutdown_connections 2023-10-06T03:13:35.038 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.034+0000 7f3476474700 1 -- 172.21.15.92:0/833069949 wait complete. 2023-10-06T03:13:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.035+0000 7f3476474700 1 Processor -- start 2023-10-06T03:13:35.040 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.035+0000 7f3476474700 1 -- start start 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3476474700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3476474700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34701597f0 0x7f347016a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3476474700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f347015d230 0x7f3470156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3476474700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347016add0 con 0x7f3470159420 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3476474700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3470156a50 con 0x7f34701597f0 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3476474700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700c5cb0 con 0x7f347015d230 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3475472700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3475472700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41452/0 (socket says 172.21.15.92:41452) 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3475472700 1 -- 172.21.15.92:0/1365779239 learned_addr learned my addr 172.21.15.92:0/1365779239 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3474c71700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34701597f0 0x7f347016a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.041 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3475c73700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f347015d230 0x7f3470156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3475472700 1 -- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f347015d230 msgr2=0x7f3470156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.036+0000 7f3475472700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f347015d230 0x7f3470156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3475472700 1 -- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34701597f0 msgr2=0x7f347016a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3475472700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34701597f0 0x7f347016a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3475472700 1 -- 172.21.15.92:0/1365779239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680094f0 con 0x7f3470159420 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3474c71700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34701597f0 0x7f347016a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3475472700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700c56d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3470089340 tx=0x7f3468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468026450 con 0x7f3470159420 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680265d0 con 0x7f3470159420 2023-10-06T03:13:35.042 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34700c5f90 con 0x7f3470159420 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468026990 con 0x7f3470159420 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.037+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34700c6550 con 0x7f3470159420 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.038+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347004c3c0 con 0x7f3470159420 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.038+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f346801a420 con 0x7f3470159420 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.038+0000 7f34657fa700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3454067320 0x7f34540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.039+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f3468086ef0 con 0x7f3470159420 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.039+0000 7f3474c71700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3454067320 0x7f34540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.039+0000 7f3474c71700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3454067320 0x7f34540697d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f34700c9e60 tx=0x7f3460008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.043 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.043+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3468053a30 con 0x7f3470159420 2023-10-06T03:13:35.225 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.224+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34700b1d50 con 0x7f3470159420 2023-10-06T03:13:35.226 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.225+0000 7f34657fa700 1 -- 172.21.15.92:0/1365779239 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+17072 (secure 0 0 0) 0x7f3468015090 con 0x7f3470159420 2023-10-06T03:13:35.226 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:35.226 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":203,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:12:26.653173+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":34,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:11:18.064170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":35,"pool_name":"cephfs_data","create_time":"2023-10-06T03:11:19.378751+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"35.1","mappings":[{"from":4,"to":1}]},{"pgid":"35.5","mappings":[{"from":4,"to":1}]},{"pgid":"35.1b","mappings":[{"from":6,"to":1}]},{"pgid":"35.24","mappings":[{"from":4,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:13:35.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3454067320 msgr2=0x7f34540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3454067320 0x7f34540697d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f34700c9e60 tx=0x7f3460008040 comp rx=0 tx=0).stop 2023-10-06T03:13:35.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 msgr2=0x7f34700c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700c56d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f3470089340 tx=0x7f3468017000 comp rx=0 tx=0).stop 2023-10-06T03:13:35.228 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 shutdown_connections 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f3454067320 0x7f34540697d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3470159420 0x7f34700c56d0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f347015d230 0x7f3470156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 --2- 172.21.15.92:0/1365779239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34701597f0 0x7f347016a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 >> 172.21.15.92:0/1365779239 conn(0x7f34700d0660 msgr2=0x7f34700cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 shutdown_connections 2023-10-06T03:13:35.229 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.227+0000 7f3476474700 1 -- 172.21.15.92:0/1365779239 wait complete. 2023-10-06T03:13:35.240 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:13:35.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.355+0000 7f24e8949700 1 Processor -- start 2023-10-06T03:13:35.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.355+0000 7f24e8949700 1 -- start start 2023-10-06T03:13:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e8949700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f24e4159420 0x7f24e40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e8949700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e8949700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e40b11d0 con 0x7f24e415d230 2023-10-06T03:13:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e8949700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40b14b0 con 0x7f24e4159420 2023-10-06T03:13:35.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e8949700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40b1790 con 0x7f24e41597f0 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e359e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f24e415d230 0x7f24e40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37130/0 (socket says 172.21.15.92:37130) 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e359e700 1 -- 172.21.15.92:0/569116992 learned_addr learned my addr 172.21.15.92:0/569116992 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e2d9d700 1 --2- 172.21.15.92:0/569116992 >> v2:172.21.15.136:3300/0 conn(0x7f24e4159420 0x7f24e40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e259c700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e1d9b700 1 -- 172.21.15.92:0/569116992 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004163077 0 0) 0x7f24e40b11d0 con 0x7f24e415d230 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e1d9b700 1 -- 172.21.15.92:0/569116992 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d0003570 con 0x7f24e415d230 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e2d9d700 1 -- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 msgr2=0x7f24e40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e2d9d700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e2d9d700 1 -- 172.21.15.92:0/569116992 >> v1:172.21.15.92:6789/0 conn(0x7f24e415d230 legacy=0x7f24e40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e2d9d700 1 -- 172.21.15.92:0/569116992 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40b1a70 con 0x7f24e4159420 2023-10-06T03:13:35.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.356+0000 7f24e259c700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e2d9d700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e4159420 0x7f24e40b5f40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0098e0 tx=0x7f24dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccfe1ff44bd27b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e1d9b700 1 -- 172.21.15.92:0/569116992 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc011630 con 0x7f24e4159420 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e1d9b700 1 -- 172.21.15.92:0/569116992 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc011790 con 0x7f24e4159420 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e1d9b700 1 -- 172.21.15.92:0/569116992 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc011b30 con 0x7f24e4159420 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 -- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e4159420 msgr2=0x7f24e40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e4159420 0x7f24e40b5f40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0098e0 tx=0x7f24dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 -- 172.21.15.92:0/569116992 shutdown_connections 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e4159420 0x7f24e40b5f40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 --2- 172.21.15.92:0/569116992 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 -- 172.21.15.92:0/569116992 >> 172.21.15.92:0/569116992 conn(0x7f24e40d0660 msgr2=0x7f24e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:35.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 -- 172.21.15.92:0/569116992 shutdown_connections 2023-10-06T03:13:35.361 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.357+0000 7f24e8949700 1 -- 172.21.15.92:0/569116992 wait complete. 2023-10-06T03:13:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.358+0000 7f24e8949700 1 Processor -- start 2023-10-06T03:13:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 -- start start 2023-10-06T03:13:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 0x7f24e415a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e415d230 0x7f24e4157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e415c770 con 0x7f24e4159420 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40c9470 con 0x7f24e415d230 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e8949700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e415ad00 con 0x7f24e41597f0 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 0x7f24e415a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e2d9d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 0x7f24e415a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41460/0 (socket says 172.21.15.92:41460) 2023-10-06T03:13:35.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.359+0000 7f24e2d9d700 1 -- 172.21.15.92:0/2275036395 learned_addr learned my addr 172.21.15.92:0/2275036395 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e259c700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e359e700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e415d230 0x7f24e4157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e2d9d700 1 -- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 msgr2=0x7f24e415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e2d9d700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e2d9d700 1 -- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e415d230 msgr2=0x7f24e4157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e2d9d700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e415d230 0x7f24e4157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e2d9d700 1 -- 172.21.15.92:0/2275036395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc009420 con 0x7f24e4159420 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e359e700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e415d230 0x7f24e4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e2d9d700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 0x7f24e415a620 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f24e4089340 tx=0x7f24dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc011b10 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc0253f0 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc0257b0 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24dc009420 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.360+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e415b2c0 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.361+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40c92d0 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.361+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f24dc025b60 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.362+0000 7f24daffd700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c4067320 0x7f24c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.362+0000 7f24e259c700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c4067320 0x7f24c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.362+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6174+0+0 (secure 0 0 0) 0x7f24dc086720 con 0x7f24e4159420 2023-10-06T03:13:35.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.362+0000 7f24e259c700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c4067320 0x7f24c40697d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f24e40cda80 tx=0x7f24cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.366+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24e40c92d0 con 0x7f24e4159420 2023-10-06T03:13:35.590 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.589+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f24e40c92d0 con 0x7f24e4159420 2023-10-06T03:13:35.590 INFO:tasks.ceph.mon.a.smithi092.stderr:2023-10-06T03:13:35.590+0000 7f7035024700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:13:35.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.594+0000 7f24daffd700 1 -- 172.21.15.92:0/2275036395 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v138) v1 ==== 161+0+0 (secure 0 0 0) 0x7f24e40c92d0 con 0x7f24e4159420 2023-10-06T03:13:35.596 INFO:tasks.ceph.mds.c.smithi092.stderr: -3289> 2023-10-06T03:11:25.322+0000 7ff9c7f28700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c4067320 msgr2=0x7f24c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c4067320 0x7f24c40697d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f24e40cda80 tx=0x7f24cc008040 comp rx=0 tx=0).stop 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 msgr2=0x7f24e415a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 0x7f24e415a620 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f24e4089340 tx=0x7f24dc017000 comp rx=0 tx=0).stop 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 shutdown_connections 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f24c4067320 0x7f24c40697d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f24e4159420 0x7f24e415a620 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24e415d230 0x7f24e4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 --2- 172.21.15.92:0/2275036395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24e41597f0 0x7f24e415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 >> 172.21.15.92:0/2275036395 conn(0x7f24e40d0660 msgr2=0x7f24e40b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:35.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 shutdown_connections 2023-10-06T03:13:35.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.598+0000 7f24e8949700 1 -- 172.21.15.92:0/2275036395 wait complete. 2023-10-06T03:13:35.601 INFO:teuthology.orchestra.run.smithi092.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:13:35.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:13:35.620 INFO:tasks.ceph.mds.c.smithi092.stdout:starting mds.c at 2023-10-06T03:13:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.727+0000 7f3408ac5700 1 Processor -- start 2023-10-06T03:13:35.728 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.727+0000 7f3408ac5700 1 -- start start 2023-10-06T03:13:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3408ac5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3404157680 0x7f34040cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3408ac5700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3404157a50 0x7f34040cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3408ac5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.730 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3408ac5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040cc700 con 0x7f3404157680 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3408ac5700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040cb900 con 0x7f3404157a50 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3408ac5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040cd240 con 0x7f34040b1450 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f34037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3404157680 0x7f34040cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f34037fe700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3404157680 0x7f34040cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41488/0 (socket says 172.21.15.92:41488) 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f34037fe700 1 -- 172.21.15.92:0/2874306146 learned_addr learned my addr 172.21.15.92:0/2874306146 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3402ffd700 1 --2- 172.21.15.92:0/2874306146 >> v2:172.21.15.136:3300/0 conn(0x7f3404157a50 0x7f34040cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f34037fe700 1 -- 172.21.15.92:0/2874306146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 msgr2=0x7f34040c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.728+0000 7f3403fff700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34037fe700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34037fe700 1 -- 172.21.15.92:0/2874306146 >> v2:172.21.15.136:3300/0 conn(0x7f3404157a50 msgr2=0x7f34040cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:35.731 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34037fe700 1 --2- 172.21.15.92:0/2874306146 >> v2:172.21.15.136:3300/0 conn(0x7f3404157a50 0x7f34040cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34037fe700 1 -- 172.21.15.92:0/2874306146 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040c9a20 con 0x7f3404157680 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3402ffd700 1 --2- 172.21.15.92:0/2874306146 >> v2:172.21.15.136:3300/0 conn(0x7f3404157a50 0x7f34040cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3403fff700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34037fe700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157680 0x7f34040cae80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f34040cbc70 tx=0x7f33f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fead17f0b14d1609 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34027fc700 1 -- 172.21.15.92:0/2874306146 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400e070 con 0x7f3404157680 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34027fc700 1 -- 172.21.15.92:0/2874306146 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400ba50 con 0x7f3404157680 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f34027fc700 1 -- 172.21.15.92:0/2874306146 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40092e0 con 0x7f3404157680 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 -- 172.21.15.92:0/2874306146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157680 msgr2=0x7f34040cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157680 0x7f34040cae80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f34040cbc70 tx=0x7f33f4009840 comp rx=0 tx=0).stop 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 -- 172.21.15.92:0/2874306146 shutdown_connections 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.732 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2874306146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157680 0x7f34040cae80 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2874306146 >> v2:172.21.15.136:3300/0 conn(0x7f3404157a50 0x7f34040cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.729+0000 7f3408ac5700 1 -- 172.21.15.92:0/2874306146 >> 172.21.15.92:0/2874306146 conn(0x7f340406baa0 msgr2=0x7f340406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.730+0000 7f3408ac5700 1 -- 172.21.15.92:0/2874306146 shutdown_connections 2023-10-06T03:13:35.733 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.730+0000 7f3408ac5700 1 -- 172.21.15.92:0/2874306146 wait complete. 2023-10-06T03:13:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 Processor -- start 2023-10-06T03:13:35.735 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 -- start start 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404157680 0x7f3404069e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 0x7f340414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340415b920 con 0x7f3404157a50 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404158840 con 0x7f3404157680 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.731+0000 7f3408ac5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340406a6c0 con 0x7f34040b1450 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 0x7f340414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 0x7f340414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41502/0 (socket says 172.21.15.92:41502) 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 -- 172.21.15.92:0/2724465796 learned_addr learned my addr 172.21.15.92:0/2724465796 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:35.736 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3402ffd700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404157680 0x7f3404069e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f34037fe700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 -- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 msgr2=0x7f34040b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 -- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404157680 msgr2=0x7f3404069e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404157680 0x7f3404069e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 -- 172.21.15.92:0/2724465796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009ae0 con 0x7f3404157a50 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3403fff700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 0x7f340414ffa0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c8b0 tx=0x7f33fc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011600 con 0x7f3404157a50 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc011760 con 0x7f3404157a50 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011b20 con 0x7f3404157a50 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340406a9a0 con 0x7f3404157a50 2023-10-06T03:13:35.737 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.732+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340415bb70 con 0x7f3404157a50 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.733+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340415bd90 con 0x7f3404157a50 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.734+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f33fc021350 con 0x7f3404157a50 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.734+0000 7f33eb7fe700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33f0067320 0x7f33f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.734+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f33fc084d30 con 0x7f3404157a50 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.734+0000 7f34037fe700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33f0067320 0x7f33f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.734+0000 7f34037fe700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33f0067320 0x7f33f00697d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009880 tx=0x7f33f4018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:35.738 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.738+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33fc051680 con 0x7f3404157a50 2023-10-06T03:13:35.965 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:35.964+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f34040cc910 con 0x7f3404157a50 2023-10-06T03:13:36.595 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.594+0000 7f33eb7fe700 1 -- 172.21.15.92:0/2724465796 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v139) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33fc054ca0 con 0x7f3404157a50 2023-10-06T03:13:36.597 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33f0067320 msgr2=0x7f33f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33f0067320 0x7f33f00697d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009880 tx=0x7f33f4018000 comp rx=0 tx=0).stop 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 msgr2=0x7f340414ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 0x7f340414ffa0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00c8b0 tx=0x7f33fc008040 comp rx=0 tx=0).stop 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 shutdown_connections 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f33f0067320 0x7f33f00697d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3404157a50 0x7f340414ffa0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34040b1450 0x7f34040b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 --2- 172.21.15.92:0/2724465796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3404157680 0x7f3404069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 >> 172.21.15.92:0/2724465796 conn(0x7f340406baa0 msgr2=0x7f3404151f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:36.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 shutdown_connections 2023-10-06T03:13:36.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.597+0000 7f3408ac5700 1 -- 172.21.15.92:0/2724465796 wait complete. 2023-10-06T03:13:36.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:13:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.769+0000 7f002335d700 1 Processor -- start 2023-10-06T03:13:36.770 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.769+0000 7f002335d700 1 -- start start 2023-10-06T03:13:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f002335d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f001c0cd910 0x7f001c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f002335d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c0b5b00 con 0x7f001c0b55a0 2023-10-06T03:13:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f002335d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c0b5da0 con 0x7f001c159ec0 2023-10-06T03:13:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f002335d700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c0b6080 con 0x7f001c0cd910 2023-10-06T03:13:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f00208f8700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f001c0cd910 0x7f001c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:36.772 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f00218fa700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f001c0b55a0 0x7f001c0ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37146/0 (socket says 172.21.15.92:37146) 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.770+0000 7f00218fa700 1 -- 172.21.15.92:0/4162809119 learned_addr learned my addr 172.21.15.92:0/4162809119 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f00208f8700 1 -- 172.21.15.92:0/4162809119 >> v1:172.21.15.136:6789/0 conn(0x7f001c159ec0 legacy=0x7f001c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f00208f8700 1 -- 172.21.15.92:0/4162809119 >> v1:172.21.15.92:6789/0 conn(0x7f001c0b55a0 legacy=0x7f001c0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f00208f8700 1 -- 172.21.15.92:0/4162809119 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0cc100 con 0x7f001c0cd910 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f00208f8700 1 --2- 172.21.15.92:0/4162809119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c0cd910 0x7f001c0ce340 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f000c009910 tx=0x7f000c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612f2e8c3c76c466 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f0013fff700 1 -- 172.21.15.92:0/4162809119 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00b7e0 con 0x7f001c0cd910 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f0013fff700 1 -- 172.21.15.92:0/4162809119 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c00b940 con 0x7f001c0cd910 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f0013fff700 1 -- 172.21.15.92:0/4162809119 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00bce0 con 0x7f001c0cd910 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f002335d700 1 -- 172.21.15.92:0/4162809119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c0cd910 msgr2=0x7f001c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f002335d700 1 --2- 172.21.15.92:0/4162809119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c0cd910 0x7f001c0ce340 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f000c009910 tx=0x7f000c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.773 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f002335d700 1 -- 172.21.15.92:0/4162809119 shutdown_connections 2023-10-06T03:13:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f002335d700 1 --2- 172.21.15.92:0/4162809119 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c0cd910 0x7f001c0ce340 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f002335d700 1 -- 172.21.15.92:0/4162809119 >> 172.21.15.92:0/4162809119 conn(0x7f001c0999f0 msgr2=0x7f001c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.771+0000 7f002335d700 1 -- 172.21.15.92:0/4162809119 shutdown_connections 2023-10-06T03:13:36.774 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.772+0000 7f002335d700 1 -- 172.21.15.92:0/4162809119 wait complete. 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 Processor -- start 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 -- start start 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f001c0b55a0 0x7f001c0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f001c159ec0 0x7f001c150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 0x7f001c15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c0c8e20 con 0x7f001c15a320 2023-10-06T03:13:36.776 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c15a7b0 con 0x7f001c159ec0 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.773+0000 7f002335d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c15aa90 con 0x7f001c0b55a0 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 0x7f001c15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 0x7f001c15b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41516/0 (socket says 172.21.15.92:41516) 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 -- 172.21.15.92:0/3239977862 learned_addr learned my addr 172.21.15.92:0/3239977862 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00210f9700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f001c0b55a0 0x7f001c0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00208f8700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f001c159ec0 0x7f001c150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 -- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f001c0b55a0 msgr2=0x7f001c0c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f001c0b55a0 0x7f001c0c99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 -- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f001c159ec0 msgr2=0x7f001c150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f001c159ec0 0x7f001c150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:36.777 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 -- 172.21.15.92:0/3239977862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c009420 con 0x7f001c15a320 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00208f8700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f001c159ec0 0x7f001c150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00218fa700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 0x7f001c15b850 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f001400c560 tx=0x7f0014007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014005b30 con 0x7f001c15a320 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0014005c90 con 0x7f001c15a320 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c15ad70 con 0x7f001c15a320 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00140072e0 con 0x7f001c15a320 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.774+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c15b330 con 0x7f001c15a320 2023-10-06T03:13:36.778 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.775+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c063c90 con 0x7f001c15a320 2023-10-06T03:13:36.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.778+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0014007690 con 0x7f001c15a320 2023-10-06T03:13:36.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.778+0000 7f00117fa700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:36.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.778+0000 7f00210f9700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:36.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.778+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6260+0+0 (secure 0 0 0) 0x7f00140845b0 con 0x7f001c15a320 2023-10-06T03:13:36.779 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.779+0000 7f00210f9700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f001800f430 tx=0x7f0018002ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:36.782 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.782+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00140510b0 con 0x7f001c15a320 2023-10-06T03:13:36.963 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:36.963+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f001c15b520 con 0x7f001c15a320 2023-10-06T03:13:37.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.790+0000 7f00117fa700 1 -- 172.21.15.92:0/3239977862 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v205) v1 ==== 181+0+0 (secure 0 0 0) 0x7f00140546d0 con 0x7f001c15a320 2023-10-06T03:13:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 msgr2=0x7f00000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f001800f430 tx=0x7f0018002ad0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 msgr2=0x7f001c15b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 0x7f001c15b850 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f001400c560 tx=0x7f0014007f60 comp rx=0 tx=0).stop 2023-10-06T03:13:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 shutdown_connections 2023-10-06T03:13:37.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0000067320 0x7f00000697d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f001c15a320 0x7f001c15b850 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f001c0b55a0 0x7f001c0c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 --2- 172.21.15.92:0/3239977862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f001c159ec0 0x7f001c150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 >> 172.21.15.92:0/3239977862 conn(0x7f001c0999f0 msgr2=0x7f001c0c55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 shutdown_connections 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.793+0000 7f002335d700 1 -- 172.21.15.92:0/3239977862 wait complete. 2023-10-06T03:13:37.796 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:13:37.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:13:37.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.925+0000 7f5ff3816700 1 Processor -- start 2023-10-06T03:13:37.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.925+0000 7f5ff3816700 1 -- start start 2023-10-06T03:13:37.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff3816700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec159ec0 0x7f5fec0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:37.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff3816700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 0x7f5fec0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff3816700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0b5b00 con 0x7f5fec0b53d0 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff3816700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec0b5da0 con 0x7f5fec0cf350 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff3816700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0b6080 con 0x7f5fec159ec0 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff1db3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 0x7f5fec0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff15b2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec159ec0 0x7f5fec0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff1db3700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 0x7f5fec0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.92:49878/0 (socket says 172.21.15.92:49878) 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff1db3700 1 -- 172.21.15.92:0/1993206355 learned_addr learned my addr 172.21.15.92:0/1993206355 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff15b2700 1 -- 172.21.15.92:0/1993206355 >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 msgr2=0x7f5fec0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff15b2700 1 --2- 172.21.15.92:0/1993206355 >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 0x7f5fec0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff15b2700 1 -- 172.21.15.92:0/1993206355 >> v1:172.21.15.136:6789/0 conn(0x7f5fec0b53d0 legacy=0x7f5fec0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:37.929 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff15b2700 1 -- 172.21.15.92:0/1993206355 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0cc100 con 0x7f5fec159ec0 2023-10-06T03:13:37.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.926+0000 7f5ff1db3700 1 --2- 172.21.15.92:0/1993206355 >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 0x7f5fec0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:37.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff15b2700 1 --2- 172.21.15.92:0/1993206355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec159ec0 0x7f5fec0b4e90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40098d0 tx=0x7f5fe4009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8acd5e3336cbd6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:37.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5febfff700 1 -- 172.21.15.92:0/1993206355 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4005c10 con 0x7f5fec159ec0 2023-10-06T03:13:37.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5febfff700 1 -- 172.21.15.92:0/1993206355 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4005d70 con 0x7f5fec159ec0 2023-10-06T03:13:37.930 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5febfff700 1 -- 172.21.15.92:0/1993206355 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4006280 con 0x7f5fec159ec0 2023-10-06T03:13:37.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff3816700 1 -- 172.21.15.92:0/1993206355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec159ec0 msgr2=0x7f5fec0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1993206355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec159ec0 0x7f5fec0b4e90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40098d0 tx=0x7f5fe4009120 comp rx=0 tx=0).stop 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff3816700 1 -- 172.21.15.92:0/1993206355 shutdown_connections 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1993206355 >> v2:172.21.15.136:3300/0 conn(0x7f5fec0cf350 0x7f5fec0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1993206355 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec159ec0 0x7f5fec0b4e90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.927+0000 7f5ff3816700 1 -- 172.21.15.92:0/1993206355 >> 172.21.15.92:0/1993206355 conn(0x7f5fec0999f0 msgr2=0x7f5fec0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.928+0000 7f5ff3816700 1 -- 172.21.15.92:0/1993206355 shutdown_connections 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.928+0000 7f5ff3816700 1 -- 172.21.15.92:0/1993206355 wait complete. 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 Processor -- start 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 -- start start 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 0x7f5fec068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:37.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fec0cf350 0x7f5fec06a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec159ec0 0x7f5fec06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec06b350 con 0x7f5fec0cf350 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec09d250 con 0x7f5fec159ec0 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.929+0000 7f5ff3816700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0692a0 con 0x7f5fec0b53d0 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fec0cf350 0x7f5fec06a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff15b2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 0x7f5fec068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff15b2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 0x7f5fec068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.92:47192/0 (socket says 172.21.15.92:47192) 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff15b2700 1 -- 172.21.15.92:0/1202058559 learned_addr learned my addr 172.21.15.92:0/1202058559 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff1db3700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec159ec0 0x7f5fec06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 -- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 msgr2=0x7f5fec068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:37.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 0x7f5fec068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 -- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec159ec0 msgr2=0x7f5fec06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec159ec0 0x7f5fec06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 -- 172.21.15.92:0/1202058559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0094f0 con 0x7f5fec0cf350 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff15b2700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 0x7f5fec068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff1db3700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec159ec0 0x7f5fec06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff0db1700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fec0cf350 0x7f5fec06a090 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c910 tx=0x7f5fe00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0012630 con 0x7f5fec0cf350 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00127b0 con 0x7f5fec0cf350 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe40093c0 con 0x7f5fec0cf350 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.930+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0012b70 con 0x7f5fec0cf350 2023-10-06T03:13:37.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.931+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec068ce0 con 0x7f5fec0cf350 2023-10-06T03:13:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.932+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5fe0003030 con 0x7f5fec0cf350 2023-10-06T03:13:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.932+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec09d020 con 0x7f5fec0cf350 2023-10-06T03:13:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.932+0000 7f5fe97fa700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fd8067350 0x7f5fd8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.932+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5822+0+0 (secure 0 0 0) 0x7f5fe0084c00 con 0x7f5fec0cf350 2023-10-06T03:13:37.936 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.932+0000 7f5ff15b2700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fd8067350 0x7f5fd8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:37.937 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.932+0000 7f5ff15b2700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fd8067350 0x7f5fd8069800 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009160 tx=0x7f5fe4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:37.938 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:37.936+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fec09d020 con 0x7f5fec0cf350 2023-10-06T03:13:38.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.118+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fec09d020 con 0x7f5fec0cf350 2023-10-06T03:13:38.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.797+0000 7f5fe97fa700 1 -- 172.21.15.92:0/1202058559 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v206) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5fec09d020 con 0x7f5fec0cf350 2023-10-06T03:13:38.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fd8067350 msgr2=0x7f5fd8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fd8067350 0x7f5fd8069800 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009160 tx=0x7f5fe4016000 comp rx=0 tx=0).stop 2023-10-06T03:13:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fec0cf350 msgr2=0x7f5fec06a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fec0cf350 0x7f5fec06a090 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c910 tx=0x7f5fe00016f0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 shutdown_connections 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fd8067350 0x7f5fd8069800 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec159ec0 0x7f5fec06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fec0b53d0 0x7f5fec068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 --2- 172.21.15.92:0/1202058559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fec0cf350 0x7f5fec06a090 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.800+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 >> 172.21.15.92:0/1202058559 conn(0x7f5fec0999f0 msgr2=0x7f5fec0c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.801+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 shutdown_connections 2023-10-06T03:13:38.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.801+0000 7f5ff3816700 1 -- 172.21.15.92:0/1202058559 wait complete. 2023-10-06T03:13:38.804 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'cephfs_data' removed 2023-10-06T03:13:38.816 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:13:38.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.938+0000 7f72fbfff700 1 Processor -- start 2023-10-06T03:13:38.940 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.938+0000 7f72fbfff700 1 -- start start 2023-10-06T03:13:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fbfff700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f72fc0b4e70 0x7f72fc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fbfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 0x7f72fc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fbfff700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72fc0b2080 con 0x7f72fc15d240 2023-10-06T03:13:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fbfff700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72fc0cfb00 con 0x7f72fc0b4e70 2023-10-06T03:13:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fbfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0b5c40 con 0x7f72fc0b16d0 2023-10-06T03:13:38.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fb7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f72fc15d240 0x7f72fc15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37158/0 (socket says 172.21.15.92:37158) 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fb7fe700 1 -- 172.21.15.92:0/975047577 learned_addr learned my addr 172.21.15.92:0/975047577 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727973687 0 0) 0x7f72fc0b2080 con 0x7f72fc15d240 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e8003570 con 0x7f72fc15d240 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72f27fc700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 0x7f72fc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.939+0000 7f72faffd700 1 --2- 172.21.15.92:0/975047577 >> v2:172.21.15.136:3300/0 conn(0x7f72fc0b4e70 0x7f72fc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421728388 0 0) 0x7f72e8003570 con 0x7f72fc15d240 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 msgr2=0x7f72fc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 0x7f72fc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 >> v2:172.21.15.136:3300/0 conn(0x7f72fc0b4e70 msgr2=0x7f72fc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:38.947 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 --2- 172.21.15.92:0/975047577 >> v2:172.21.15.136:3300/0 conn(0x7f72fc0b4e70 0x7f72fc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc0b5f20 con 0x7f72fc15d240 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f72f40017f0 con 0x7f72fc15d240 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 >> v1:172.21.15.92:6789/0 conn(0x7f72fc15d240 legacy=0x7f72fc15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72e8003a60 0x7f72e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 0x7f72e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8004d50 0x7f72e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0cfb00 con 0x7f72e8003a60 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0b5c40 con 0x7f72e8004d50 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e800bb90 con 0x7f72fc0b16d0 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72e8003a60 0x7f72e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.948 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72f27fc700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 0x7f72e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fb7fe700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8004d50 0x7f72e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 -- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 msgr2=0x7f72e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc0b16d0 0x7f72e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 -- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8004d50 msgr2=0x7f72e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8004d50 0x7f72e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 -- 172.21.15.92:0/975047577 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8003570 con 0x7f72e8003a60 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72faffd700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72e8003a60 0x7f72e8003e70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bb90 tx=0x7f72ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a515a9e3ce3a9f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00d5d0 con 0x7f72e8003a60 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec00d730 con 0x7f72e8003a60 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.940+0000 7f72fa7fc700 1 -- 172.21.15.92:0/975047577 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00daf0 con 0x7f72e8003a60 2023-10-06T03:13:38.949 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fbfff700 1 -- 172.21.15.92:0/975047577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72e8003a60 msgr2=0x7f72e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fbfff700 1 --2- 172.21.15.92:0/975047577 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72e8003a60 0x7f72e8003e70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bb90 tx=0x7f72ec008040 comp rx=0 tx=0).stop 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fb7fe700 1 -- 172.21.15.92:0/975047577 reap_dead start 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fbfff700 1 -- 172.21.15.92:0/975047577 shutdown_connections 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fbfff700 1 -- 172.21.15.92:0/975047577 >> 172.21.15.92:0/975047577 conn(0x7f72fc0d10c0 msgr2=0x7f72fc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fbfff700 1 -- 172.21.15.92:0/975047577 shutdown_connections 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.941+0000 7f72fbfff700 1 -- 172.21.15.92:0/975047577 wait complete. 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.942+0000 7f72fbfff700 1 Processor -- start 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.942+0000 7f72fbfff700 1 -- start start 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fbfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8003a60 0x7f72fc0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fbfff700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc15fa60 0x7f72fc146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 0x7f72fc147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.950 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fbfff700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc16e620 con 0x7f72fc15fed0 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fbfff700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc147660 con 0x7f72e8003a60 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fbfff700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0cbb80 con 0x7f72fc15fa60 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 0x7f72fc147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 0x7f72fc147120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41548/0 (socket says 172.21.15.92:41548) 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 -- 172.21.15.92:0/2283549998 learned_addr learned my addr 172.21.15.92:0/2283549998 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72faffd700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8003a60 0x7f72fc0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72f27fc700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc15fa60 0x7f72fc146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 -- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc15fa60 msgr2=0x7f72fc146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc15fa60 0x7f72fc146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.951 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 -- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8003a60 msgr2=0x7f72fc0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8003a60 0x7f72fc0c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 -- 172.21.15.92:0/2283549998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc16e340 con 0x7f72fc15fed0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.943+0000 7f72fb7fe700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 0x7f72fc147120 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f72f4001350 tx=0x7f72f4001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.944+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f400eb10 con 0x7f72fc15fed0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.944+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f400ec90 con 0x7f72fc15fed0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.944+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc0cbe20 con 0x7f72fc15fed0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.944+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4013420 con 0x7f72fc15fed0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.944+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc0cc3e0 con 0x7f72fc15fed0 2023-10-06T03:13:38.952 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.945+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72fc088990 con 0x7f72fc15fed0 2023-10-06T03:13:38.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.945+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f72f4013850 con 0x7f72fc15fed0 2023-10-06T03:13:38.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.945+0000 7f72f37fe700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72dc067320 0x7f72dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:38.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.945+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f72f4084f50 con 0x7f72fc15fed0 2023-10-06T03:13:38.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.945+0000 7f72faffd700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72dc067320 0x7f72dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:38.953 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.946+0000 7f72faffd700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72dc067320 0x7f72dc0697d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bfd0 tx=0x7f72ec01b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:38.954 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:38.950+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72f4051d80 con 0x7f72fc15fed0 2023-10-06T03:13:39.135 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.133+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f72fc04bb80 con 0x7f72fc15fed0 2023-10-06T03:13:39.138 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.137+0000 7f72f37fe700 1 -- 172.21.15.92:0/2283549998 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f72f40553a0 con 0x7f72fc15fed0 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72dc067320 msgr2=0x7f72dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72dc067320 0x7f72dc0697d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00bfd0 tx=0x7f72ec01b000 comp rx=0 tx=0).stop 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 msgr2=0x7f72fc147120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 0x7f72fc147120 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f72f4001350 tx=0x7f72f4001800 comp rx=0 tx=0).stop 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 shutdown_connections 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f72dc067320 0x7f72dc0697d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f72fc15fed0 0x7f72fc147120 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72e8003a60 0x7f72fc0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 --2- 172.21.15.92:0/2283549998 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72fc15fa60 0x7f72fc146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.141 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 >> 172.21.15.92:0/2283549998 conn(0x7f72fc0d10c0 msgr2=0x7f72fc069710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:39.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 shutdown_connections 2023-10-06T03:13:39.142 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.139+0000 7f72fbfff700 1 -- 172.21.15.92:0/2283549998 wait complete. 2023-10-06T03:13:39.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-06T03:13:39.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.266+0000 7f8b544cb700 1 Processor -- start 2023-10-06T03:13:39.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.266+0000 7f8b544cb700 1 -- start start 2023-10-06T03:13:39.269 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b544cb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8b4c069560 0x7f8b4c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b544cb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c069ee0 con 0x7f8b4c159890 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b544cb700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c15a160 con 0x7f8b4c159c60 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b544cb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c0d13a0 con 0x7f8b4c069560 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52267700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8b4c159890 0x7f8b4c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:37168/0 (socket says 172.21.15.92:37168) 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52a68700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8b4c069560 0x7f8b4c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52267700 1 -- 172.21.15.92:0/596473166 learned_addr learned my addr 172.21.15.92:0/596473166 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52a68700 1 -- 172.21.15.92:0/596473166 >> v1:172.21.15.136:6789/0 conn(0x7f8b4c159c60 legacy=0x7f8b4c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52a68700 1 -- 172.21.15.92:0/596473166 >> v1:172.21.15.92:6789/0 conn(0x7f8b4c159890 legacy=0x7f8b4c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52a68700 1 -- 172.21.15.92:0/596473166 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c15e140 con 0x7f8b4c069560 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b51265700 1 -- 172.21.15.92:0/596473166 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698127262 0 0) 0x7f8b4c069ee0 con 0x7f8b4c159890 2023-10-06T03:13:39.270 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b52a68700 1 --2- 172.21.15.92:0/596473166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0699a0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009e60 tx=0x7f8b44009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974528209d88a19a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b51265700 1 -- 172.21.15.92:0/596473166 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4400e070 con 0x7f8b4c069560 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b51265700 1 -- 172.21.15.92:0/596473166 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4400bc30 con 0x7f8b4c069560 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.267+0000 7f8b51265700 1 -- 172.21.15.92:0/596473166 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44012430 con 0x7f8b4c069560 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 -- 172.21.15.92:0/596473166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 msgr2=0x7f8b4c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 --2- 172.21.15.92:0/596473166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0699a0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009e60 tx=0x7f8b44009780 comp rx=0 tx=0).stop 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 -- 172.21.15.92:0/596473166 shutdown_connections 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 --2- 172.21.15.92:0/596473166 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0699a0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.271 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 -- 172.21.15.92:0/596473166 >> 172.21.15.92:0/596473166 conn(0x7f8b4c068e20 msgr2=0x7f8b4c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 -- 172.21.15.92:0/596473166 shutdown_connections 2023-10-06T03:13:39.272 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.268+0000 7f8b544cb700 1 -- 172.21.15.92:0/596473166 wait complete. 2023-10-06T03:13:39.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.269+0000 7f8b544cb700 1 Processor -- start 2023-10-06T03:13:39.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.269+0000 7f8b544cb700 1 -- start start 2023-10-06T03:13:39.274 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b4c159890 0x7f8b4c140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b4c159c60 0x7f8b4c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b4170 con 0x7f8b4c069560 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c16dac0 con 0x7f8b4c159890 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c16dda0 con 0x7f8b4c159c60 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:41580/0 (socket says 172.21.15.92:41580) 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 -- 172.21.15.92:0/712181307 learned_addr learned my addr 172.21.15.92:0/712181307 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b51a66700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b4c159890 0x7f8b4c140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52a68700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b4c159c60 0x7f8b4c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:39.275 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 -- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b4c159c60 msgr2=0x7f8b4c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b4c159c60 0x7f8b4c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 -- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b4c159890 msgr2=0x7f8b4c140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b4c159890 0x7f8b4c140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 -- 172.21.15.92:0/712181307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44009a20 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b52267700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0b3c30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800c7e0 tx=0x7f8b48007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48005b30 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48005c90 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c16e080 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48007300 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.270+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c0cd6b0 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.271+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c0cd8d0 con 0x7f8b4c069560 2023-10-06T03:13:39.276 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.272+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b4801b400 con 0x7f8b4c069560 2023-10-06T03:13:39.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.272+0000 7f8b427fc700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b30067320 0x7f8b300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:39.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.272+0000 7f8b51a66700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b30067320 0x7f8b300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:39.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.272+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f8b48084560 con 0x7f8b4c069560 2023-10-06T03:13:39.277 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.273+0000 7f8b51a66700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b30067320 0x7f8b300697d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c16d4b0 tx=0x7f8b3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:39.278 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.276+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b48051490 con 0x7f8b4c069560 2023-10-06T03:13:39.536 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.535+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7f8b4c04c3c0 con 0x7f8b4c069560 2023-10-06T03:13:39.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.798+0000 7f8b427fc700 1 -- 172.21.15.92:0/712181307 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v579) v1 ==== 131+0+0 (secure 0 0 0) 0x7f8b48054ab0 con 0x7f8b4c069560 2023-10-06T03:13:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b30067320 msgr2=0x7f8b300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b30067320 0x7f8b300697d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c16d4b0 tx=0x7f8b3c009040 comp rx=0 tx=0).stop 2023-10-06T03:13:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 msgr2=0x7f8b4c0b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0b3c30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800c7e0 tx=0x7f8b48007df0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 shutdown_connections 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8b30067320 0x7f8b300697d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8b4c069560 0x7f8b4c0b3c30 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8b4c159c60 0x7f8b4c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 --2- 172.21.15.92:0/712181307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8b4c159890 0x7f8b4c140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 >> 172.21.15.92:0/712181307 conn(0x7f8b4c068e20 msgr2=0x7f8b4c15aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 shutdown_connections 2023-10-06T03:13:39.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:39.800+0000 7f8b544cb700 1 -- 172.21.15.92:0/712181307 wait complete. 2023-10-06T03:13:39.816 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used ... ERROR 2023-10-06T03:13:39.816 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.816 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-06T03:13:39.817 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-06T03:13:39.818 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-06T03:13:39.819 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner:Ran 12 tests in 688.198s 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-06T03:13:39.820 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-06T03:13:39.821 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-06T03:13:39.822 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.823 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-06T03:13:39.824 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.824 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-06T03:13:39.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-06T03:13:39.824 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-06T03:13:39.824 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-06T03:13:39.824 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-06T03:13:39.825 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi092 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-06T03:13:39.826 INFO:tasks.cephfs_test_runner: 2023-10-06T03:13:39.827 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:13:39.893 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5f6d5b77ee1140afb4eb547cc433c1fa Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-06T03:13:39.897 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T03:13:39.909 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T03:13:39.955 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T03:13:39.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:39.956 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:13:39.983 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:13:39.983 INFO:teuthology.orchestra.run.smithi092.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:13:39.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:39.984 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:13:40.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.059 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:13:40.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T03:13:40.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.231 DEBUG:teuthology.orchestra.run.smithi092:> ip netns list 2023-10-06T03:13:40.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo ip link delete ceph-brx 2023-10-06T03:13:40.314 INFO:teuthology.orchestra.run.smithi092.stderr:Cannot find device "ceph-brx" 2023-10-06T03:13:40.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:13:40.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.316 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:13:40.334 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T03:13:40.335 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:13:40.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.335 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:13:40.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.415 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:13:40.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.494 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link delete ceph-brx 2023-10-06T03:13:40.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.604 DEBUG:teuthology.orchestra.run.smithi136:> ip netns list 2023-10-06T03:13:40.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:13:40.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo ip link delete ceph-brx 2023-10-06T03:13:40.687 INFO:teuthology.orchestra.run.smithi136.stderr:Cannot find device "ceph-brx" 2023-10-06T03:13:40.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:13:40.689 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T03:13:40.699 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T03:13:40.699 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:13:40.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.814+0000 7f5fe8eb9700 1 Processor -- start 2023-10-06T03:13:40.816 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.814+0000 7f5fe8eb9700 1 -- start start 2023-10-06T03:13:40.819 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe8eb9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe8eb9700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fe4069560 0x7f5fe40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe8eb9700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe415a160 con 0x7f5fe4159c60 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe8eb9700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe4069f20 con 0x7f5fe4069560 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe8eb9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe40d13a0 con 0x7f5fe4159890 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fe4069560 0x7f5fe40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fd9d9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5fe4159c60 0x7f5fe415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43978/0 (socket says 172.21.15.92:43978) 2023-10-06T03:13:40.820 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe2d9d700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fe4069560 0x7f5fe40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40108/0 (socket says 172.21.15.92:40108) 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fd9d9b700 1 -- 172.21.15.92:0/1246983382 learned_addr learned my addr 172.21.15.92:0/1246983382 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.815+0000 7f5fe259c700 1 --2- 172.21.15.92:0/1246983382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe2d9d700 1 -- 172.21.15.92:0/1246983382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 msgr2=0x7f5fe415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe2d9d700 1 --2- 172.21.15.92:0/1246983382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe2d9d700 1 -- 172.21.15.92:0/1246983382 >> v1:172.21.15.92:6789/0 conn(0x7f5fe4159c60 legacy=0x7f5fe415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe2d9d700 1 -- 172.21.15.92:0/1246983382 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe415e140 con 0x7f5fe4069560 2023-10-06T03:13:40.821 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe1d9b700 1 -- 172.21.15.92:0/1246983382 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481001600 0 0) 0x7f5fe415a160 con 0x7f5fe4159c60 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe2d9d700 1 --2- 172.21.15.92:0/1246983382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe40699e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009e60 tx=0x7f5fdc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218c0bf05c97e915 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe1d9b700 1 -- 172.21.15.92:0/1246983382 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00e070 con 0x7f5fe4069560 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe1d9b700 1 -- 172.21.15.92:0/1246983382 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc00bb60 con 0x7f5fe4069560 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe1d9b700 1 -- 172.21.15.92:0/1246983382 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc012430 con 0x7f5fe4069560 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/1246983382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 msgr2=0x7f5fe40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/1246983382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe40699e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009e60 tx=0x7f5fdc009780 comp rx=0 tx=0).stop 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/1246983382 shutdown_connections 2023-10-06T03:13:40.822 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/1246983382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:40.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/1246983382 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe40699e0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:40.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/1246983382 >> 172.21.15.92:0/1246983382 conn(0x7f5fe4068e20 msgr2=0x7f5fe40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:40.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/1246983382 shutdown_connections 2023-10-06T03:13:40.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.816+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/1246983382 wait complete. 2023-10-06T03:13:40.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 Processor -- start 2023-10-06T03:13:40.823 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 -- start start 2023-10-06T03:13:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fe4159c60 0x7f5fe4154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4155770 con 0x7f5fe4069560 2023-10-06T03:13:40.826 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.818+0000 7f5fe8eb9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe415efc0 con 0x7f5fe4159c60 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe8eb9700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe4154cc0 con 0x7f5fe4159890 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe415d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40122/0 (socket says 172.21.15.92:40122) 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 -- 172.21.15.92:0/655182248 learned_addr learned my addr 172.21.15.92:0/655182248 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fd9d9b700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe2d9d700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fe4159c60 0x7f5fe4154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 -- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 msgr2=0x7f5fe40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:40.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 -- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fe4159c60 msgr2=0x7f5fe4154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fe4159c60 0x7f5fe4154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 -- 172.21.15.92:0/655182248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc009a20 con 0x7f5fe4069560 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe259c700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe415d890 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400ecb0 tx=0x7f5fd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400c920 con 0x7f5fe4069560 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd400ca80 con 0x7f5fe4069560 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe40b3fe0 con 0x7f5fe4069560 2023-10-06T03:13:40.828 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400ce20 con 0x7f5fe4069560 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.819+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe415b8a0 con 0x7f5fe4069560 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.820+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe40b4500 con 0x7f5fe4069560 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.820+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5fd4013680 con 0x7f5fe4069560 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.821+0000 7f5fdaffd700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fc4067320 0x7f5fc40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.821+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f5fd4084d30 con 0x7f5fe4069560 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.821+0000 7f5fd9d9b700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fc4067320 0x7f5fc40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:40.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.821+0000 7f5fd9d9b700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fc4067320 0x7f5fc40697d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40b5810 tx=0x7f5fcc007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:40.830 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:40.825+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fe40b4500 con 0x7f5fe4069560 2023-10-06T03:13:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.008+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fe415ed90 con 0x7f5fe4069560 2023-10-06T03:13:41.010 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.009+0000 7f5fdaffd700 1 -- 172.21.15.92:0/655182248 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+13789 (secure 0 0 0) 0x7f5fe415ed90 con 0x7f5fe4069560 2023-10-06T03:13:41.011 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:41.011 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":206,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:13:38.787978+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6837/3455480640":"2023-10-07T03:13:35.590557+0000","172.21.15.92:6835/3455480640":"2023-10-07T03:13:35.590557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:13:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fc4067320 msgr2=0x7f5fc40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fc4067320 0x7f5fc40697d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40b5810 tx=0x7f5fcc007d30 comp rx=0 tx=0).stop 2023-10-06T03:13:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 msgr2=0x7f5fe415d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.013 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe415d890 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400ecb0 tx=0x7f5fd40091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 shutdown_connections 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f5fc4067320 0x7f5fc40697d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe4069560 0x7f5fe415d890 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fe4159c60 0x7f5fe4154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 --2- 172.21.15.92:0/655182248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fe4159890 0x7f5fe40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 >> 172.21.15.92:0/655182248 conn(0x7f5fe4068e20 msgr2=0x7f5fe41580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.014 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 shutdown_connections 2023-10-06T03:13:41.015 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.011+0000 7f5fe8eb9700 1 -- 172.21.15.92:0/655182248 wait complete. 2023-10-06T03:13:41.025 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T03:13:41.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:13:41.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.138+0000 7f440a98b700 1 Processor -- start 2023-10-06T03:13:41.140 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.138+0000 7f440a98b700 1 -- start start 2023-10-06T03:13:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f440a98b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f440a98b700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440415a160 con 0x7f4404159890 2023-10-06T03:13:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f440a98b700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44040d13a0 con 0x7f440415db10 2023-10-06T03:13:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f440a98b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440415e140 con 0x7f4404159c60 2023-10-06T03:13:41.143 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f4403fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f4404159890 0x7f440415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43986/0 (socket says 172.21.15.92:43986) 2023-10-06T03:13:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f4403fff700 1 -- 172.21.15.92:0/4054947775 learned_addr learned my addr 172.21.15.92:0/4054947775 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.139+0000 7f44037fe700 1 --2- 172.21.15.92:0/4054947775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459308913 0 0) 0x7f440415a160 con 0x7f4404159890 2023-10-06T03:13:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43ec003570 con 0x7f4404159890 2023-10-06T03:13:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299406628 0 0) 0x7f44040d13a0 con 0x7f440415db10 2023-10-06T03:13:41.144 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440415a160 con 0x7f440415db10 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f44037fe700 1 -- 172.21.15.92:0/4054947775 >> v1:172.21.15.136:6789/0 conn(0x7f440415db10 legacy=0x7f4404069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f44037fe700 1 -- 172.21.15.92:0/4054947775 >> v1:172.21.15.92:6789/0 conn(0x7f4404159890 legacy=0x7f440415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f44037fe700 1 -- 172.21.15.92:0/4054947775 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440415a3d0 con 0x7f4404159c60 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f44037fe700 1 --2- 172.21.15.92:0/4054947775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440415d5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009910 tx=0x7f43f80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46cb7bc297279cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8011630 con 0x7f4404159c60 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f8011790 con 0x7f4404159c60 2023-10-06T03:13:41.145 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.140+0000 7f4402ffd700 1 -- 172.21.15.92:0/4054947775 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f8011b30 con 0x7f4404159c60 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 -- 172.21.15.92:0/4054947775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 msgr2=0x7f440415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 --2- 172.21.15.92:0/4054947775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440415d5d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009910 tx=0x7f43f80091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 -- 172.21.15.92:0/4054947775 shutdown_connections 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 --2- 172.21.15.92:0/4054947775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440415d5d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 -- 172.21.15.92:0/4054947775 >> 172.21.15.92:0/4054947775 conn(0x7f4404068e20 msgr2=0x7f44040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 -- 172.21.15.92:0/4054947775 shutdown_connections 2023-10-06T03:13:41.146 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.141+0000 7f440a98b700 1 -- 172.21.15.92:0/4054947775 wait complete. 2023-10-06T03:13:41.147 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.142+0000 7f440a98b700 1 Processor -- start 2023-10-06T03:13:41.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.142+0000 7f440a98b700 1 -- start start 2023-10-06T03:13:41.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 0x7f4404152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.149 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f440415db10 0x7f440415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404153690 con 0x7f4404159890 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404151630 con 0x7f440415db10 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4404151910 con 0x7f4404159c60 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 0x7f4404152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 0x7f4404152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40124/0 (socket says 172.21.15.92:40124) 2023-10-06T03:13:41.150 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 -- 172.21.15.92:0/4274537893 learned_addr learned my addr 172.21.15.92:0/4274537893 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f44037fe700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4408f28700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f440415db10 0x7f440415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 -- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 msgr2=0x7f440416a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440416a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 -- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f440415db10 msgr2=0x7f440415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f440415db10 0x7f440415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.151 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 -- 172.21.15.92:0/4274537893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8009420 con 0x7f4404159890 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f4403fff700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 0x7f4404152f90 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c790 tx=0x7f43f4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4005b30 con 0x7f4404159890 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f4005c90 con 0x7f4404159890 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007300 con 0x7f4404159890 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4404151bf0 con 0x7f4404159890 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.143+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440415cc30 con 0x7f4404159890 2023-10-06T03:13:41.152 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.144+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440404c3c0 con 0x7f4404159890 2023-10-06T03:13:41.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.145+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f43f401b400 con 0x7f4404159890 2023-10-06T03:13:41.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.145+0000 7f43ebfff700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43f0067320 0x7f43f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.145+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f43f4084620 con 0x7f4404159890 2023-10-06T03:13:41.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.145+0000 7f44037fe700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43f0067320 0x7f43f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.145+0000 7f44037fe700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43f0067320 0x7f43f00697d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f440415a090 tx=0x7f43f8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.153 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.149+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43f4055010 con 0x7f4404159890 2023-10-06T03:13:41.332 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.330+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f440406a5b0 con 0x7f43f0067320 2023-10-06T03:13:41.335 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.333+0000 7f43ebfff700 1 -- 172.21.15.92:0/4274537893 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73115 (secure 0 0 0) 0x7f440406a5b0 con 0x7f43f0067320 2023-10-06T03:13:41.336 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:41.337 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43f0067320 msgr2=0x7f43f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43f0067320 0x7f43f00697d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f440415a090 tx=0x7f43f8017000 comp rx=0 tx=0).stop 2023-10-06T03:13:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 msgr2=0x7f4404152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 0x7f4404152f90 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c790 tx=0x7f43f4007ea0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.338 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 shutdown_connections 2023-10-06T03:13:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f43f0067320 0x7f43f00697d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4404159890 0x7f4404152f90 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f440415db10 0x7f440415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.339 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 --2- 172.21.15.92:0/4274537893 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4404159c60 0x7f440416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 >> 172.21.15.92:0/4274537893 conn(0x7f4404068e20 msgr2=0x7f440415aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 shutdown_connections 2023-10-06T03:13:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.335+0000 7f440a98b700 1 -- 172.21.15.92:0/4274537893 wait complete. 2023-10-06T03:13:41.340 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:13:41.349 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":564,"stamp":"2023-10-06T03:13:39.335598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2432660,"kb_used_data":21588,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542060,"statfs":{"total":773094113280,"available":770603069440,"internally_reserved":0,"allocated":22106112,"data_stored":11135255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001051"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.302359+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:13:38.302359+0000","last_peered":"2023-10-06T03:13:38.302359+0000","last_clean":"2023-10-06T03:13:38.302359+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:13:38.302359+0000","last_undegraded":"2023-10-06T03:13:38.302359+0000","last_fullsized":"2023-10-06T03:13:38.302359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792855+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:13:37.792855+0000","last_peered":"2023-10-06T03:13:37.792855+0000","last_clean":"2023-10-06T03:13:37.792855+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:13:37.792855+0000","last_undegraded":"2023-10-06T03:13:37.792855+0000","last_fullsized":"2023-10-06T03:13:37.792855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301764+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:13:38.301764+0000","last_peered":"2023-10-06T03:13:38.301764+0000","last_clean":"2023-10-06T03:13:38.301764+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:13:38.301764+0000","last_undegraded":"2023-10-06T03:13:38.301764+0000","last_fullsized":"2023-10-06T03:13:38.301764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301153+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:13:38.301153+0000","last_peered":"2023-10-06T03:13:38.301153+0000","last_clean":"2023-10-06T03:13:38.301153+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:13:38.301153+0000","last_undegraded":"2023-10-06T03:13:38.301153+0000","last_fullsized":"2023-10-06T03:13:38.301153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":320,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301253+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:13:38.301253+0000","last_peered":"2023-10-06T03:13:38.301253+0000","last_clean":"2023-10-06T03:13:38.301253+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:13:38.301253+0000","last_undegraded":"2023-10-06T03:13:38.301253+0000","last_fullsized":"2023-10-06T03:13:38.301253+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791888+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:13:37.791888+0000","last_peered":"2023-10-06T03:13:37.791888+0000","last_clean":"2023-10-06T03:13:37.791888+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:13:37.791888+0000","last_undegraded":"2023-10-06T03:13:37.791888+0000","last_fullsized":"2023-10-06T03:13:37.791888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.300512+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:13:38.300512+0000","last_peered":"2023-10-06T03:13:38.300512+0000","last_clean":"2023-10-06T03:13:38.300512+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:13:38.300512+0000","last_undegraded":"2023-10-06T03:13:38.300512+0000","last_fullsized":"2023-10-06T03:13:38.300512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791600+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:13:37.791600+0000","last_peered":"2023-10-06T03:13:37.791600+0000","last_clean":"2023-10-06T03:13:37.791600+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:13:37.791600+0000","last_undegraded":"2023-10-06T03:13:37.791600+0000","last_fullsized":"2023-10-06T03:13:37.791600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792425+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:13:37.792425+0000","last_peered":"2023-10-06T03:13:37.792425+0000","last_clean":"2023-10-06T03:13:37.792425+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:13:37.792425+0000","last_undegraded":"2023-10-06T03:13:37.792425+0000","last_fullsized":"2023-10-06T03:13:37.792425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.60799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":6.0110000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.316}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85299999999999998,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52800000000000002,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70099999999999996,"5min":1.264,"15min":1.264},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.752,"5min":2.286,"15min":2.286},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52800000000000002},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77300000000000002,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":2.504,"15min":2.504},"last":0.504}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83399999999999996,"5min":1.141,"15min":1.141},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84799999999999998,"5min":1.639,"15min":1.639},"last":0.47799999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.91400000000000003,"15min":1.6200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.48199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80700000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.53400000000000003},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.73199999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":1.6359999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":0.77200000000000002,"15min":0.85099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.57999999999999996,"5min":0.81200000000000006,"15min":0.88700000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":0.873,"15min":0.873},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.313,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":0.76900000000000002,"15min":0.85299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.85699999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.26000000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.54600000000000004,"5min":0.55600000000000005,"15min":3.1139999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.24399999999999999},"max":{"1min":0.71799999999999997,"5min":0.98899999999999999,"15min":3.6520000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.28499999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47899999999999998,"5min":0.57999999999999996,"15min":1.282},"last":0.31},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34699999999999998},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":1.25},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":1.1060000000000001,"15min":1.302},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":1.127,"15min":1.3240000000000001},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78800000000000003,"5min":0.81599999999999995,"15min":0.96299999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":5.0270000000000001},"last":0.498}]},{"osd":6,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.3360000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.35199999999999998},"max":{"1min":0.91400000000000003,"5min":1.054,"15min":1.3520000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.59799999999999998,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.087,"15min":1.379},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304512,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067328,"statfs":{"total":96636764160,"available":96324943872,"internally_reserved":0,"allocated":3203072,"data_stored":1832838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.313},"min":{"1min":0.23499999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.317},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.32800000000000001},"min":{"1min":0.25900000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34100000000000003},"min":{"1min":0.28699999999999998,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35199999999999998},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.34499999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.47799999999999998,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.99199999999999999},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.92100000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94799999999999995},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.375},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.93700000000000006},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":0.89000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23999999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.33000000000000002},"min":{"1min":0.28999999999999998,"5min":0.23300000000000001,"15min":0.20499999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.249}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.33800000000000002},"min":{"1min":0.311,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.24399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.36499999999999999,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.333,"15min":1.333},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.999,"15min":0.999},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.34499999999999997},"max":{"1min":0.82899999999999996,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.442}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.57699999999999996},"min":{"1min":0.48299999999999998,"5min":0.42199999999999999,"15min":0.34399999999999997},"max":{"1min":0.85599999999999998,"5min":1.216,"15min":1.216},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.874,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.878,"5min":1.2789999999999999,"15min":1.903},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.376},"max":{"1min":0.82499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.46700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303928,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067912,"statfs":{"total":96636764160,"available":96325541888,"internally_reserved":0,"allocated":2605056,"data_stored":1239307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20100000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.65200000000000002},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.216},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.72299999999999998},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34200000000000003,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33400000000000002},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.216},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.69699999999999995},"last":0.316}]},{"osd":2,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"min":{"1min":0.35399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.56000000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.379,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.67600000000000005},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.73799999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.43099999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.379,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.997},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.315},"max":{"1min":0.78300000000000003,"5min":0.86099999999999999,"15min":0.86499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.44600000000000001,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.98499999999999999},"last":0.47299999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":1.0660000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.35599999999999998},"max":{"1min":0.78400000000000003,"5min":1.446,"15min":1.446},"last":0.48499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54500000000000004,"15min":0.52100000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.307},"max":{"1min":0.872,"5min":1.069,"15min":1.4410000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":1.0009999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":0.81899999999999995,"5min":0.92600000000000005,"15min":3},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56100000000000005,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":10.084},"last":0.498}]},{"osd":3,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89100000000000001,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.503},"min":{"1min":0.32800000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.83999999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.76200000000000001,"5min":0.97599999999999998,"15min":1.54},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":1.167,"15min":1.167},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.69499999999999995,"5min":0.79100000000000004,"15min":2.3599999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.58599999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.28100000000000003},"max":{"1min":0.72799999999999998,"5min":0.83499999999999996,"15min":2.371},"last":0.63}]},{"osd":1,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.82399999999999995,"15min":2.3410000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.83999999999999997,"15min":4.1219999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.92400000000000004,"15min":6.532},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":5.6500000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.34799999999999998,"15min":0.315},"max":{"1min":0.70199999999999996,"5min":0.85299999999999998,"15min":5.4630000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70899999999999996,"5min":0.86099999999999999,"15min":5.524},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.50700000000000001},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.27100000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76400000000000001,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":1.1990000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.44800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.95399999999999996,"15min":1.052},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.175},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.93899999999999995,"15min":1.145},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:13:41.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:13:41.466 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.464+0000 7fc79152f700 1 Processor -- start 2023-10-06T03:13:41.467 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.464+0000 7fc79152f700 1 -- start start 2023-10-06T03:13:41.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.465+0000 7fc79152f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 0x7fc78c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.465+0000 7fc79152f700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c15a160 con 0x7fc78c15db10 2023-10-06T03:13:41.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.465+0000 7fc79152f700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c0d13a0 con 0x7fc78c159890 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.465+0000 7fc79152f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c15e140 con 0x7fc78c159c60 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.465+0000 7fc78b7fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc78c15db10 0x7fc78c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43988/0 (socket says 172.21.15.92:43988) 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.465+0000 7fc78b7fe700 1 -- 172.21.15.92:0/2548481947 learned_addr learned my addr 172.21.15.92:0/2548481947 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317876526 0 0) 0x7fc78c15a160 con 0x7fc78c15db10 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc7827fc700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 0x7fc78c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc778003570 con 0x7fc78c15db10 2023-10-06T03:13:41.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 450274394 0 0) 0x7fc778003570 con 0x7fc78c15db10 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 msgr2=0x7fc78c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 0x7fc78c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 >> v1:172.21.15.136:6789/0 conn(0x7fc78c159890 legacy=0x7fc78c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c15a3d0 con 0x7fc78c15db10 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7fc7840017f0 con 0x7fc78c15db10 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 >> v1:172.21.15.92:6789/0 conn(0x7fc78c15db10 legacy=0x7fc78c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 0x7fc778003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc778003fd0 0x7fc778004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc778004d50 0x7fc7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c15e140 con 0x7fc778004d50 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77800bb40 con 0x7fc778003fd0 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77800bd00 con 0x7fc78c159c60 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78b7fe700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc778004d50 0x7fc7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc7827fc700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc778003fd0 0x7fc778004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.473 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78affd700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 0x7fc778003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78b7fe700 1 -- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 msgr2=0x7fc778003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78b7fe700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c159c60 0x7fc778003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78b7fe700 1 -- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc778003fd0 msgr2=0x7fc778004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78b7fe700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc778003fd0 0x7fc778004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.466+0000 7fc78b7fe700 1 -- 172.21.15.92:0/2548481947 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778003570 con 0x7fc778004d50 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc78b7fe700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc778004d50 0x7fc7780051c0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc78400a950 tx=0x7fc7840021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa3fdeb505978c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.474 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7840070d0 con 0x7fc778004d50 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784007230 con 0x7fc778004d50 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc78a7fc700 1 -- 172.21.15.92:0/2548481947 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7840075f0 con 0x7fc778004d50 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc79152f700 1 -- 172.21.15.92:0/2548481947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc778004d50 msgr2=0x7fc7780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc79152f700 1 --2- 172.21.15.92:0/2548481947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc778004d50 0x7fc7780051c0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc78400a950 tx=0x7fc7840021f0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc78b7fe700 1 -- 172.21.15.92:0/2548481947 reap_dead start 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc79152f700 1 -- 172.21.15.92:0/2548481947 shutdown_connections 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc79152f700 1 -- 172.21.15.92:0/2548481947 >> 172.21.15.92:0/2548481947 conn(0x7fc78c068e20 msgr2=0x7fc78c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.475 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc79152f700 1 -- 172.21.15.92:0/2548481947 shutdown_connections 2023-10-06T03:13:41.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.467+0000 7fc79152f700 1 -- 172.21.15.92:0/2548481947 wait complete. 2023-10-06T03:13:41.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 Processor -- start 2023-10-06T03:13:41.476 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 -- start start 2023-10-06T03:13:41.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c0b05c0 0x7fc78c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.478 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 0x7fc78c0b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78c0b1340 0x7fc78c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0cd0c0 con 0x7fc78c0b0990 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0cdff0 con 0x7fc78c0b1340 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc79152f700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0cff50 con 0x7fc78c0b05c0 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc7827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 0x7fc78c0b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc7827fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 0x7fc78c0b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40152/0 (socket says 172.21.15.92:40152) 2023-10-06T03:13:41.479 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc7827fc700 1 -- 172.21.15.92:0/2871359081 learned_addr learned my addr 172.21.15.92:0/2871359081 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.469+0000 7fc78affd700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c0b05c0 0x7fc78c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc78b7fe700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78c0b1340 0x7fc78c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7827fc700 1 -- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c0b05c0 msgr2=0x7fc78c0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7827fc700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c0b05c0 0x7fc78c0cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7827fc700 1 -- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78c0b1340 msgr2=0x7fc78c151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7827fc700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78c0b1340 0x7fc78c151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7827fc700 1 -- 172.21.15.92:0/2871359081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c0b1b40 con 0x7fc78c0b0990 2023-10-06T03:13:41.480 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7827fc700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 0x7fc78c0b0e00 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc774011ba0 tx=0x7fc7740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400f8e0 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77400fa40 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c0cd2d0 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77400fdc0 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.470+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c0b17b0 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.471+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c151060 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.471+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc774014650 con 0x7fc78c0b0990 2023-10-06T03:13:41.481 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.472+0000 7fc7837fe700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc76c067320 0x7fc76c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.472+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc774018030 con 0x7fc78c0b0990 2023-10-06T03:13:41.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.472+0000 7fc78affd700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc76c067320 0x7fc76c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.472+0000 7fc78affd700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc76c067320 0x7fc76c0697d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc77c003c80 tx=0x7fc77c007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.482 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.476+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc78c151060 con 0x7fc78c0b0990 2023-10-06T03:13:41.657 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.656+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc78c151060 con 0x7fc76c067320 2023-10-06T03:13:41.660 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.658+0000 7fc7837fe700 1 -- 172.21.15.92:0/2871359081 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73115 (secure 0 0 0) 0x7fc78c151060 con 0x7fc76c067320 2023-10-06T03:13:41.661 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc76c067320 msgr2=0x7fc76c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.662 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc76c067320 0x7fc76c0697d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc77c003c80 tx=0x7fc77c007d10 comp rx=0 tx=0).stop 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 msgr2=0x7fc78c0b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 0x7fc78c0b0e00 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fc774011ba0 tx=0x7fc7740091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 shutdown_connections 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc76c067320 0x7fc76c0697d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78c0b1340 0x7fc78c151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc78c0b05c0 0x7fc78c0cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.663 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 --2- 172.21.15.92:0/2871359081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc78c0b0990 0x7fc78c0b0e00 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.660+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 >> 172.21.15.92:0/2871359081 conn(0x7fc78c068e20 msgr2=0x7fc78c153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.661+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 shutdown_connections 2023-10-06T03:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.661+0000 7fc79152f700 1 -- 172.21.15.92:0/2871359081 wait complete. 2023-10-06T03:13:41.664 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:13:41.674 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":565,"stamp":"2023-10-06T03:13:41.335938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2432660,"kb_used_data":21588,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542060,"statfs":{"total":773094113280,"available":770603069440,"internally_reserved":0,"allocated":22106112,"data_stored":11135255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001124"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.302359+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:13:38.302359+0000","last_peered":"2023-10-06T03:13:38.302359+0000","last_clean":"2023-10-06T03:13:38.302359+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:13:38.302359+0000","last_undegraded":"2023-10-06T03:13:38.302359+0000","last_fullsized":"2023-10-06T03:13:38.302359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792855+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:13:37.792855+0000","last_peered":"2023-10-06T03:13:37.792855+0000","last_clean":"2023-10-06T03:13:37.792855+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:13:37.792855+0000","last_undegraded":"2023-10-06T03:13:37.792855+0000","last_fullsized":"2023-10-06T03:13:37.792855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301764+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:13:38.301764+0000","last_peered":"2023-10-06T03:13:38.301764+0000","last_clean":"2023-10-06T03:13:38.301764+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:13:38.301764+0000","last_undegraded":"2023-10-06T03:13:38.301764+0000","last_fullsized":"2023-10-06T03:13:38.301764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301153+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:13:38.301153+0000","last_peered":"2023-10-06T03:13:38.301153+0000","last_clean":"2023-10-06T03:13:38.301153+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:13:38.301153+0000","last_undegraded":"2023-10-06T03:13:38.301153+0000","last_fullsized":"2023-10-06T03:13:38.301153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":320,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301253+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:13:38.301253+0000","last_peered":"2023-10-06T03:13:38.301253+0000","last_clean":"2023-10-06T03:13:38.301253+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:13:38.301253+0000","last_undegraded":"2023-10-06T03:13:38.301253+0000","last_fullsized":"2023-10-06T03:13:38.301253+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791888+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:13:37.791888+0000","last_peered":"2023-10-06T03:13:37.791888+0000","last_clean":"2023-10-06T03:13:37.791888+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:13:37.791888+0000","last_undegraded":"2023-10-06T03:13:37.791888+0000","last_fullsized":"2023-10-06T03:13:37.791888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.300512+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:13:38.300512+0000","last_peered":"2023-10-06T03:13:38.300512+0000","last_clean":"2023-10-06T03:13:38.300512+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:13:38.300512+0000","last_undegraded":"2023-10-06T03:13:38.300512+0000","last_fullsized":"2023-10-06T03:13:38.300512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791600+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:13:37.791600+0000","last_peered":"2023-10-06T03:13:37.791600+0000","last_clean":"2023-10-06T03:13:37.791600+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:13:37.791600+0000","last_undegraded":"2023-10-06T03:13:37.791600+0000","last_fullsized":"2023-10-06T03:13:37.791600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792425+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:13:37.792425+0000","last_peered":"2023-10-06T03:13:37.792425+0000","last_clean":"2023-10-06T03:13:37.792425+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:13:37.792425+0000","last_undegraded":"2023-10-06T03:13:37.792425+0000","last_fullsized":"2023-10-06T03:13:37.792425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.60799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":6.0110000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.316}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85299999999999998,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52800000000000002,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70099999999999996,"5min":1.264,"15min":1.264},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.752,"5min":2.286,"15min":2.286},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52800000000000002},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77300000000000002,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":2.504,"15min":2.504},"last":0.504}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83399999999999996,"5min":1.141,"15min":1.141},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84799999999999998,"5min":1.639,"15min":1.639},"last":0.47799999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.91400000000000003,"15min":1.6200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.48199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80700000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.53400000000000003},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.73199999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":1.6359999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":0.77200000000000002,"15min":0.85099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.57999999999999996,"5min":0.81200000000000006,"15min":0.88700000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":0.873,"15min":0.873},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.313,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":0.76900000000000002,"15min":0.85299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.85699999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.26000000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.54600000000000004,"5min":0.55600000000000005,"15min":3.1139999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.24399999999999999},"max":{"1min":0.71799999999999997,"5min":0.98899999999999999,"15min":3.6520000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.28499999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47899999999999998,"5min":0.57999999999999996,"15min":1.282},"last":0.31},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34699999999999998},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":1.25},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":1.1060000000000001,"15min":1.302},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":1.127,"15min":1.3240000000000001},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78800000000000003,"5min":0.81599999999999995,"15min":0.96299999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":5.0270000000000001},"last":0.498}]},{"osd":6,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.3360000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.35199999999999998},"max":{"1min":0.91400000000000003,"5min":1.054,"15min":1.3520000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.59799999999999998,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.087,"15min":1.379},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304512,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067328,"statfs":{"total":96636764160,"available":96324943872,"internally_reserved":0,"allocated":3203072,"data_stored":1832838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.313},"min":{"1min":0.23499999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.317},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.32800000000000001},"min":{"1min":0.25900000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34100000000000003},"min":{"1min":0.28699999999999998,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35199999999999998},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.34499999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.47799999999999998,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.99199999999999999},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.92100000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94799999999999995},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.375},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.93700000000000006},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":0.89000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23999999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.33000000000000002},"min":{"1min":0.28999999999999998,"5min":0.23300000000000001,"15min":0.20499999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.249}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.33800000000000002},"min":{"1min":0.311,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.24399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.36499999999999999,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.333,"15min":1.333},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.999,"15min":0.999},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.34499999999999997},"max":{"1min":0.82899999999999996,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.442}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.57699999999999996},"min":{"1min":0.48299999999999998,"5min":0.42199999999999999,"15min":0.34399999999999997},"max":{"1min":0.85599999999999998,"5min":1.216,"15min":1.216},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.874,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.878,"5min":1.2789999999999999,"15min":1.903},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.376},"max":{"1min":0.82499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.46700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303928,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067912,"statfs":{"total":96636764160,"available":96325541888,"internally_reserved":0,"allocated":2605056,"data_stored":1239307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20100000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.65200000000000002},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.216},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.72299999999999998},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34200000000000003,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33400000000000002},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.216},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.69699999999999995},"last":0.316}]},{"osd":2,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"min":{"1min":0.35399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.56000000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.379,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.67600000000000005},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.73799999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.43099999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.379,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.997},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.315},"max":{"1min":0.78300000000000003,"5min":0.86099999999999999,"15min":0.86499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.44600000000000001,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.98499999999999999},"last":0.47299999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":1.0660000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.35599999999999998},"max":{"1min":0.78400000000000003,"5min":1.446,"15min":1.446},"last":0.48499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54500000000000004,"15min":0.52100000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.307},"max":{"1min":0.872,"5min":1.069,"15min":1.4410000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":1.0009999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":0.81899999999999995,"5min":0.92600000000000005,"15min":3},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56100000000000005,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":10.084},"last":0.498}]},{"osd":3,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89100000000000001,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.503},"min":{"1min":0.32800000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.83999999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.76200000000000001,"5min":0.97599999999999998,"15min":1.54},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":1.167,"15min":1.167},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.69499999999999995,"5min":0.79100000000000004,"15min":2.3599999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.58599999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.28100000000000003},"max":{"1min":0.72799999999999998,"5min":0.83499999999999996,"15min":2.371},"last":0.63}]},{"osd":1,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.82399999999999995,"15min":2.3410000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.83999999999999997,"15min":4.1219999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.92400000000000004,"15min":6.532},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":5.6500000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.34799999999999998,"15min":0.315},"max":{"1min":0.70199999999999996,"5min":0.85299999999999998,"15min":5.4630000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70899999999999996,"5min":0.86099999999999999,"15min":5.524},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.50700000000000001},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.27100000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76400000000000001,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":1.1990000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.44800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.95399999999999996,"15min":1.052},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.175},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.93899999999999995,"15min":1.145},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:13:41.676 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T03:13:41.677 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:13:41.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.789+0000 7fc3a6ad7700 1 Processor -- start 2023-10-06T03:13:41.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.789+0000 7fc3a6ad7700 1 -- start start 2023-10-06T03:13:41.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a6ad7700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3a0159420 0x7fc3a00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.793 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a6ad7700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a00b2980 con 0x7fc3a015d230 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a6ad7700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a00b11d0 con 0x7fc3a01597f0 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a6ad7700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a00b14b0 con 0x7fc3a0159420 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a62d6700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fc3a015d230 0x7fc3a015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:43990/0 (socket says 172.21.15.92:43990) 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a62d6700 1 -- 172.21.15.92:0/2301465923 learned_addr learned my addr 172.21.15.92:0/2301465923 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a5ad5700 1 --2- 172.21.15.92:0/2301465923 >> v2:172.21.15.136:3300/0 conn(0x7fc3a0159420 0x7fc3a00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690733996 0 0) 0x7fc3a00b2980 con 0x7fc3a015d230 2023-10-06T03:13:41.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc38c003570 con 0x7fc3a015d230 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881161384 0 0) 0x7fc3a00b11d0 con 0x7fc3a01597f0 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a00b2980 con 0x7fc3a01597f0 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.790+0000 7fc3a5ad5700 1 -- 172.21.15.92:0/2301465923 >> v1:172.21.15.136:6789/0 conn(0x7fc3a01597f0 legacy=0x7fc3a00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a5ad5700 1 -- 172.21.15.92:0/2301465923 >> v1:172.21.15.92:6789/0 conn(0x7fc3a015d230 legacy=0x7fc3a015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a5ad5700 1 -- 172.21.15.92:0/2301465923 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a00b1790 con 0x7fc3a0159420 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a5ad5700 1 --2- 172.21.15.92:0/2301465923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a0159420 0x7fc3a00b5f40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0098e0 tx=0x7fc39c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7044d228ab71349c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.795 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c011630 con 0x7fc3a0159420 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c011790 con 0x7fc3a0159420 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a4ad3700 1 -- 172.21.15.92:0/2301465923 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c011b30 con 0x7fc3a0159420 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2301465923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a0159420 msgr2=0x7fc3a00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2301465923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a0159420 0x7fc3a00b5f40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0098e0 tx=0x7fc39c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2301465923 shutdown_connections 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2301465923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a0159420 0x7fc3a00b5f40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.796 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2301465923 >> 172.21.15.92:0/2301465923 conn(0x7fc3a00d0660 msgr2=0x7fc3a00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.791+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2301465923 shutdown_connections 2023-10-06T03:13:41.797 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.792+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2301465923 wait complete. 2023-10-06T03:13:41.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 Processor -- start 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 -- start start 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a0159420 0x7fc3a0143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a01597f0 0x7fc3a0068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 0x7fc3a00688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0069bf0 con 0x7fc3a015d230 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0169ca0 con 0x7fc3a01597f0 2023-10-06T03:13:41.800 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a6ad7700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0155830 con 0x7fc3a0159420 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a62d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 0x7fc3a00688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a62d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 0x7fc3a00688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40164/0 (socket says 172.21.15.92:40164) 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.793+0000 7fc3a62d6700 1 -- 172.21.15.92:0/2113475771 learned_addr learned my addr 172.21.15.92:0/2113475771 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a52d4700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a01597f0 0x7fc3a0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a5ad5700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a0159420 0x7fc3a0143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a62d6700 1 -- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a0159420 msgr2=0x7fc3a0143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.801 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a62d6700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a0159420 0x7fc3a0143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a62d6700 1 -- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a01597f0 msgr2=0x7fc3a0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a62d6700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a01597f0 0x7fc3a0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a62d6700 1 -- 172.21.15.92:0/2113475771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009420 con 0x7fc3a015d230 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a52d4700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a01597f0 0x7fc3a0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a62d6700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 0x7fc3a00688d0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fc398001460 tx=0x7fc398001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39800a910 con 0x7fc3a015d230 2023-10-06T03:13:41.802 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39800aa70 con 0x7fc3a015d230 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a0068f70 con 0x7fc3a015d230 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39800ae30 con 0x7fc3a015d230 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.794+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a0069530 con 0x7fc3a015d230 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.795+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a0069d30 con 0x7fc3a015d230 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.795+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc398007300 con 0x7fc3a015d230 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.796+0000 7fc395ffb700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc384067320 0x7fc3840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:41.803 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.796+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fc398084e60 con 0x7fc3a015d230 2023-10-06T03:13:41.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.796+0000 7fc3a5ad5700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc384067320 0x7fc3840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:41.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.796+0000 7fc3a5ad5700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc384067320 0x7fc3840697d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0093f0 tx=0x7fc39c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:41.804 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.800+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc398051d10 con 0x7fc3a015d230 2023-10-06T03:13:41.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.978+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3a004c3c0 con 0x7fc384067320 2023-10-06T03:13:41.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.981+0000 7fc395ffb700 1 -- 172.21.15.92:0/2113475771 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73115 (secure 0 0 0) 0x7fc3a004c3c0 con 0x7fc384067320 2023-10-06T03:13:41.983 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc384067320 msgr2=0x7fc3840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc384067320 0x7fc3840697d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0093f0 tx=0x7fc39c017000 comp rx=0 tx=0).stop 2023-10-06T03:13:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 msgr2=0x7fc3a00688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:41.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 0x7fc3a00688d0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fc398001460 tx=0x7fc398001a20 comp rx=0 tx=0).stop 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 shutdown_connections 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fc384067320 0x7fc3840697d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc3a015d230 0x7fc3a00688d0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a0159420 0x7fc3a0143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 --2- 172.21.15.92:0/2113475771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a01597f0 0x7fc3a0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 >> 172.21.15.92:0/2113475771 conn(0x7fc3a00d0660 msgr2=0x7fc3a01576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:41.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 shutdown_connections 2023-10-06T03:13:41.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:41.983+0000 7fc3a6ad7700 1 -- 172.21.15.92:0/2113475771 wait complete. 2023-10-06T03:13:41.987 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:13:41.997 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":565,"stamp":"2023-10-06T03:13:41.335938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2432660,"kb_used_data":21588,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752542060,"statfs":{"total":773094113280,"available":770603069440,"internally_reserved":0,"allocated":22106112,"data_stored":11135255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001124"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.302359+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:13:38.302359+0000","last_peered":"2023-10-06T03:13:38.302359+0000","last_clean":"2023-10-06T03:13:38.302359+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:13:38.302359+0000","last_undegraded":"2023-10-06T03:13:38.302359+0000","last_fullsized":"2023-10-06T03:13:38.302359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792855+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:13:37.792855+0000","last_peered":"2023-10-06T03:13:37.792855+0000","last_clean":"2023-10-06T03:13:37.792855+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:13:37.792855+0000","last_undegraded":"2023-10-06T03:13:37.792855+0000","last_fullsized":"2023-10-06T03:13:37.792855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301764+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:13:38.301764+0000","last_peered":"2023-10-06T03:13:38.301764+0000","last_clean":"2023-10-06T03:13:38.301764+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:13:38.301764+0000","last_undegraded":"2023-10-06T03:13:38.301764+0000","last_fullsized":"2023-10-06T03:13:38.301764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301153+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:13:38.301153+0000","last_peered":"2023-10-06T03:13:38.301153+0000","last_clean":"2023-10-06T03:13:38.301153+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:13:38.301153+0000","last_undegraded":"2023-10-06T03:13:38.301153+0000","last_fullsized":"2023-10-06T03:13:38.301153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":320,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301253+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:13:38.301253+0000","last_peered":"2023-10-06T03:13:38.301253+0000","last_clean":"2023-10-06T03:13:38.301253+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:13:38.301253+0000","last_undegraded":"2023-10-06T03:13:38.301253+0000","last_fullsized":"2023-10-06T03:13:38.301253+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791888+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:13:37.791888+0000","last_peered":"2023-10-06T03:13:37.791888+0000","last_clean":"2023-10-06T03:13:37.791888+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:13:37.791888+0000","last_undegraded":"2023-10-06T03:13:37.791888+0000","last_fullsized":"2023-10-06T03:13:37.791888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.300512+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:13:38.300512+0000","last_peered":"2023-10-06T03:13:38.300512+0000","last_clean":"2023-10-06T03:13:38.300512+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:13:38.300512+0000","last_undegraded":"2023-10-06T03:13:38.300512+0000","last_fullsized":"2023-10-06T03:13:38.300512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791600+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:13:37.791600+0000","last_peered":"2023-10-06T03:13:37.791600+0000","last_clean":"2023-10-06T03:13:37.791600+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:13:37.791600+0000","last_undegraded":"2023-10-06T03:13:37.791600+0000","last_fullsized":"2023-10-06T03:13:37.791600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792425+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:13:37.792425+0000","last_peered":"2023-10-06T03:13:37.792425+0000","last_clean":"2023-10-06T03:13:37.792425+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:13:37.792425+0000","last_undegraded":"2023-10-06T03:13:37.792425+0000","last_fullsized":"2023-10-06T03:13:37.792425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.60799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":6.0110000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.316}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85299999999999998,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52800000000000002,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70099999999999996,"5min":1.264,"15min":1.264},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.752,"5min":2.286,"15min":2.286},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52800000000000002},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77300000000000002,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":2.504,"15min":2.504},"last":0.504}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83399999999999996,"5min":1.141,"15min":1.141},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84799999999999998,"5min":1.639,"15min":1.639},"last":0.47799999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.54100000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.91400000000000003,"15min":1.6200000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.48199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80700000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.53400000000000003},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.73199999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":1.6359999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":0.77200000000000002,"15min":0.85099999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.57999999999999996,"5min":0.81200000000000006,"15min":0.88700000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":0.873,"15min":0.873},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.313,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":0.76900000000000002,"15min":0.85299999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.85699999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.61599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.26000000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.54600000000000004,"5min":0.55600000000000005,"15min":3.1139999999999999},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.24399999999999999},"max":{"1min":0.71799999999999997,"5min":0.98899999999999999,"15min":3.6520000000000001},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.28499999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47899999999999998,"5min":0.57999999999999996,"15min":1.282},"last":0.31},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34699999999999998},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":1.25},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":1.1060000000000001,"15min":1.302},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":1.127,"15min":1.3240000000000001},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78800000000000003,"5min":0.81599999999999995,"15min":0.96299999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":5.0270000000000001},"last":0.498}]},{"osd":6,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.3360000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.35199999999999998},"max":{"1min":0.91400000000000003,"5min":1.054,"15min":1.3520000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.59799999999999998,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.087,"15min":1.379},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304512,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067328,"statfs":{"total":96636764160,"available":96324943872,"internally_reserved":0,"allocated":3203072,"data_stored":1832838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.32000000000000001,"15min":0.313},"min":{"1min":0.23499999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.317},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.32800000000000001},"min":{"1min":0.25900000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.34100000000000003},"min":{"1min":0.28699999999999998,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35199999999999998},"min":{"1min":0.25700000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.20899999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.39400000000000002,"15min":0.38500000000000001},"min":{"1min":0.34499999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.47799999999999998,"5min":0.377,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.45000000000000001,"5min":0.377,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.99199999999999999},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.48399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.92100000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59099999999999997,"15min":0.58699999999999997},"min":{"1min":0.42699999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94799999999999995},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.39100000000000001,"15min":0.375},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.93700000000000006},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83099999999999996,"15min":1.0089999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:12:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.49199999999999999,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":0.89000000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.60099999999999998,"15min":0.60799999999999998},"min":{"1min":0.45800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.63}]}]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23999999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.33000000000000002},"min":{"1min":0.28999999999999998,"5min":0.23300000000000001,"15min":0.20499999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.249}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.33800000000000002},"min":{"1min":0.311,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.24399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.31},{"interface":"front","average":{"1min":0.44,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.36499999999999999,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.379}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.333,"15min":1.333},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.999,"15min":0.999},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.34499999999999997},"max":{"1min":0.82899999999999996,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.442}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.57699999999999996},"min":{"1min":0.48299999999999998,"5min":0.42199999999999999,"15min":0.34399999999999997},"max":{"1min":0.85599999999999998,"5min":1.216,"15min":1.216},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.874,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.42999999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.878,"5min":1.2789999999999999,"15min":1.903},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.376},"max":{"1min":0.82499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.46700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303928,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067912,"statfs":{"total":96636764160,"available":96325541888,"internally_reserved":0,"allocated":2605056,"data_stored":1239307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20100000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.65200000000000002},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.216},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.72299999999999998},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34200000000000003,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33400000000000002},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.216},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.69699999999999995},"last":0.316}]},{"osd":2,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"min":{"1min":0.35399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.56000000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.379,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.67600000000000005},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.73799999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.43099999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.379,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.997},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.315},"max":{"1min":0.78300000000000003,"5min":0.86099999999999999,"15min":0.86499999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.44600000000000001,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.98499999999999999},"last":0.47299999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":1.0660000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.35599999999999998},"max":{"1min":0.78400000000000003,"5min":1.446,"15min":1.446},"last":0.48499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54500000000000004,"15min":0.52100000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.307},"max":{"1min":0.872,"5min":1.069,"15min":1.4410000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":1.0009999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":0.81899999999999995,"5min":0.92600000000000005,"15min":3},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56100000000000005,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":10.084},"last":0.498}]},{"osd":3,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89100000000000001,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.63900000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.503},"min":{"1min":0.32800000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.83999999999999997},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.76200000000000001,"5min":0.97599999999999998,"15min":1.54},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":1.167,"15min":1.167},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1242470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.69499999999999995,"5min":0.79100000000000004,"15min":2.3599999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.58599999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.28100000000000003},"max":{"1min":0.72799999999999998,"5min":0.83499999999999996,"15min":2.371},"last":0.63}]},{"osd":1,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.82399999999999995,"15min":2.3410000000000002},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.83999999999999997,"15min":4.1219999999999999},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.92400000000000004,"15min":6.532},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":5.6500000000000004},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.34799999999999998,"15min":0.315},"max":{"1min":0.70199999999999996,"5min":0.85299999999999998,"15min":5.4630000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70899999999999996,"5min":0.86099999999999999,"15min":5.524},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.50700000000000001},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.27100000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76400000000000001,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":1.1990000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.44800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.95399999999999996,"15min":1.052},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.175},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.93899999999999995,"15min":1.145},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:13:42.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:13:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.111+0000 7fe562788700 1 Processor -- start 2023-10-06T03:13:42.113 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.111+0000 7fe562788700 1 -- start start 2023-10-06T03:13:42.114 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe562788700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe55c1574e0 0x7fe55c0af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe562788700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c0689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe562788700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c0b0bc0 con 0x7fe55c1578b0 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe562788700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c06c700 con 0x7fe55c1574e0 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe562788700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0affb0 con 0x7fe55c0aef50 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe560f85700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe55c1578b0 0x7fe55c0ae840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44006/0 (socket says 172.21.15.92:44006) 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe560f85700 1 -- 172.21.15.92:0/4261644402 learned_addr learned my addr 172.21.15.92:0/4261644402 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:42.115 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe561f87700 1 --2- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c0689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.112+0000 7fe561786700 1 --2- 172.21.15.92:0/4261644402 >> v2:172.21.15.136:3300/0 conn(0x7fe55c1574e0 0x7fe55c0af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe553fff700 1 -- 172.21.15.92:0/4261644402 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684105466 0 0) 0x7fe55c0b0bc0 con 0x7fe55c1578b0 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe553fff700 1 -- 172.21.15.92:0/4261644402 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe548003570 con 0x7fe55c1578b0 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe561786700 1 -- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 msgr2=0x7fe55c0689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe561786700 1 --2- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c0689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe561786700 1 -- 172.21.15.92:0/4261644402 >> v1:172.21.15.92:6789/0 conn(0x7fe55c1578b0 legacy=0x7fe55c0ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe561786700 1 -- 172.21.15.92:0/4261644402 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0b0290 con 0x7fe55c1574e0 2023-10-06T03:13:42.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe561786700 1 --2- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c0af9d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe558009e60 tx=0x7fe558009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a628281a9423dec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe553fff700 1 -- 172.21.15.92:0/4261644402 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800e070 con 0x7fe55c1574e0 2023-10-06T03:13:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe553fff700 1 -- 172.21.15.92:0/4261644402 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55800bab0 con 0x7fe55c1574e0 2023-10-06T03:13:42.117 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.113+0000 7fe553fff700 1 -- 172.21.15.92:0/4261644402 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800be70 con 0x7fe55c1574e0 2023-10-06T03:13:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 -- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 msgr2=0x7fe55c0af9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 --2- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c0af9d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe558009e60 tx=0x7fe558009780 comp rx=0 tx=0).stop 2023-10-06T03:13:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 -- 172.21.15.92:0/4261644402 shutdown_connections 2023-10-06T03:13:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 --2- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 --2- 172.21.15.92:0/4261644402 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c0af9d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 -- 172.21.15.92:0/4261644402 >> 172.21.15.92:0/4261644402 conn(0x7fe55c06ddb0 msgr2=0x7fe55c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 -- 172.21.15.92:0/4261644402 shutdown_connections 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.114+0000 7fe562788700 1 -- 172.21.15.92:0/4261644402 wait complete. 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.115+0000 7fe562788700 1 Processor -- start 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.115+0000 7fe562788700 1 -- start start 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:42.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 0x7fe55c156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c16b9d0 con 0x7fe55c1578b0 2023-10-06T03:13:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c167cf0 con 0x7fe55c1574e0 2023-10-06T03:13:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c167440 con 0x7fe55c0aef50 2023-10-06T03:13:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 0x7fe55c156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 0x7fe55c156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40170/0 (socket says 172.21.15.92:40170) 2023-10-06T03:13:42.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 -- 172.21.15.92:0/2977713370 learned_addr learned my addr 172.21.15.92:0/2977713370 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561786700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe560f85700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 -- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 msgr2=0x7fe55c144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 -- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 msgr2=0x7fe55c155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 -- 172.21.15.92:0/2977713370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe558009a20 con 0x7fe55c1578b0 2023-10-06T03:13:42.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe561f87700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 0x7fe55c156120 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fe55400ea50 tx=0x7fe5540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55400c8c0 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55400ca40 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55400ce00 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.116+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c16b6f0 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.117+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c167980 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.117+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c167e80 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.118+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe554013680 con 0x7fe55c1578b0 2023-10-06T03:13:42.124 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.118+0000 7fe5517fa700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540067320 0x7fe5400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:42.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.118+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe554084f30 con 0x7fe55c1578b0 2023-10-06T03:13:42.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.118+0000 7fe561786700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540067320 0x7fe5400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:42.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.119+0000 7fe561786700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540067320 0x7fe5400697d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe5580097c0 tx=0x7fe558019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:42.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.122+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe554051d20 con 0x7fe55c1578b0 2023-10-06T03:13:42.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.304+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe55c04c3c0 con 0x7fe55c1578b0 2023-10-06T03:13:42.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.305+0000 7fe5517fa700 1 -- 172.21.15.92:0/2977713370 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+13789 (secure 0 0 0) 0x7fe554055340 con 0x7fe55c1578b0 2023-10-06T03:13:42.307 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:42.307 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":206,"fsid":"e5cba154-9a64-4fb6-add9-5855ed5b5b5a","created":"2023-10-06T03:01:17.654095+0000","modified":"2023-10-06T03:13:38.787978+0000","last_up_change":"2023-10-06T03:01:28.649993+0000","last_in_change":"2023-10-06T03:01:20.063609+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":35,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T03:01:29.224553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T03:01:35.968472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b6c8c1e3-7b81-48e7-968a-7d3f12021a11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":90688},{"type":"v1","addr":"172.21.15.92:6809","nonce":90688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":90688},{"type":"v1","addr":"172.21.15.92:6814","nonce":90688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":90688},{"type":"v1","addr":"172.21.15.92:6825","nonce":90688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6817","nonce":90688},{"type":"v1","addr":"172.21.15.92:6819","nonce":90688}]},"public_addr":"172.21.15.92:6809/90688","cluster_addr":"172.21.15.92:6814/90688","heartbeat_back_addr":"172.21.15.92:6825/90688","heartbeat_front_addr":"172.21.15.92:6819/90688","state":["exists","up"]},{"osd":1,"uuid":"fd5b7adc-fe7a-4b54-9099-2d40b4fabbfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":90685},{"type":"v1","addr":"172.21.15.92:6801","nonce":90685}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":90685},{"type":"v1","addr":"172.21.15.92:6803","nonce":90685}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":90685},{"type":"v1","addr":"172.21.15.92:6807","nonce":90685}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":90685},{"type":"v1","addr":"172.21.15.92:6805","nonce":90685}]},"public_addr":"172.21.15.92:6801/90685","cluster_addr":"172.21.15.92:6803/90685","heartbeat_back_addr":"172.21.15.92:6807/90685","heartbeat_front_addr":"172.21.15.92:6805/90685","state":["exists","up"]},{"osd":2,"uuid":"14ab28bc-0084-45c5-bb6c-4a7b469ce151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":90689},{"type":"v1","addr":"172.21.15.92:6816","nonce":90689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":90689},{"type":"v1","addr":"172.21.15.92:6824","nonce":90689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":90689},{"type":"v1","addr":"172.21.15.92:6831","nonce":90689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6827","nonce":90689},{"type":"v1","addr":"172.21.15.92:6829","nonce":90689}]},"public_addr":"172.21.15.92:6816/90689","cluster_addr":"172.21.15.92:6824/90689","heartbeat_back_addr":"172.21.15.92:6831/90689","heartbeat_front_addr":"172.21.15.92:6829/90689","state":["exists","up"]},{"osd":3,"uuid":"54c6fa79-9c4a-4035-afec-92597ecdc604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":90687},{"type":"v1","addr":"172.21.15.92:6813","nonce":90687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":90687},{"type":"v1","addr":"172.21.15.92:6818","nonce":90687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":90687},{"type":"v1","addr":"172.21.15.92:6828","nonce":90687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6821","nonce":90687},{"type":"v1","addr":"172.21.15.92:6823","nonce":90687}]},"public_addr":"172.21.15.92:6813/90687","cluster_addr":"172.21.15.92:6818/90687","heartbeat_back_addr":"172.21.15.92:6828/90687","heartbeat_front_addr":"172.21.15.92:6823/90687","state":["exists","up"]},{"osd":4,"uuid":"c338a9f5-ade8-40aa-b512-dbe9e4c9d29b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":90892},{"type":"v1","addr":"172.21.15.136:6817","nonce":90892}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":90892},{"type":"v1","addr":"172.21.15.136:6819","nonce":90892}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":90892},{"type":"v1","addr":"172.21.15.136:6823","nonce":90892}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":90892},{"type":"v1","addr":"172.21.15.136:6821","nonce":90892}]},"public_addr":"172.21.15.136:6817/90892","cluster_addr":"172.21.15.136:6819/90892","heartbeat_back_addr":"172.21.15.136:6823/90892","heartbeat_front_addr":"172.21.15.136:6821/90892","state":["exists","up"]},{"osd":5,"uuid":"23217f93-1916-413e-8251-9258ed175816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":90874},{"type":"v1","addr":"172.21.15.136:6801","nonce":90874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":90874},{"type":"v1","addr":"172.21.15.136:6803","nonce":90874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":90874},{"type":"v1","addr":"172.21.15.136:6807","nonce":90874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":90874},{"type":"v1","addr":"172.21.15.136:6805","nonce":90874}]},"public_addr":"172.21.15.136:6801/90874","cluster_addr":"172.21.15.136:6803/90874","heartbeat_back_addr":"172.21.15.136:6807/90874","heartbeat_front_addr":"172.21.15.136:6805/90874","state":["exists","up"]},{"osd":6,"uuid":"d5701781-828b-4f78-ace3-f8445084bfeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":90875},{"type":"v1","addr":"172.21.15.136:6809","nonce":90875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":90875},{"type":"v1","addr":"172.21.15.136:6811","nonce":90875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":90875},{"type":"v1","addr":"172.21.15.136:6815","nonce":90875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":90875},{"type":"v1","addr":"172.21.15.136:6813","nonce":90875}]},"public_addr":"172.21.15.136:6809/90875","cluster_addr":"172.21.15.136:6811/90875","heartbeat_back_addr":"172.21.15.136:6815/90875","heartbeat_front_addr":"172.21.15.136:6813/90875","state":["exists","up"]},{"osd":7,"uuid":"d78abc86-39b3-454c-b242-09e42328acc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":90905},{"type":"v1","addr":"172.21.15.136:6825","nonce":90905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":90905},{"type":"v1","addr":"172.21.15.136:6827","nonce":90905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":90905},{"type":"v1","addr":"172.21.15.136:6831","nonce":90905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":90905},{"type":"v1","addr":"172.21.15.136:6829","nonce":90905}]},"public_addr":"172.21.15.136:6825/90905","cluster_addr":"172.21.15.136:6827/90905","heartbeat_back_addr":"172.21.15.136:6831/90905","heartbeat_front_addr":"172.21.15.136:6829/90905","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284107+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.186603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.223046+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.284524+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.287073+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.293455+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.272326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:01:25.264928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.92:6837/3455480640":"2023-10-07T03:13:35.590557+0000","172.21.15.92:6835/3455480640":"2023-10-07T03:13:35.590557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540067320 msgr2=0x7fe5400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540067320 0x7fe5400697d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe5580097c0 tx=0x7fe558019000 comp rx=0 tx=0).stop 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 msgr2=0x7fe55c156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 0x7fe55c156120 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fe55400ea50 tx=0x7fe5540091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 shutdown_connections 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fe540067320 0x7fe5400697d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe55c1578b0 0x7fe55c156120 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe55c0aef50 0x7fe55c144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 --2- 172.21.15.92:0/2977713370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe55c1574e0 0x7fe55c155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:42.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 >> 172.21.15.92:0/2977713370 conn(0x7fe55c06ddb0 msgr2=0x7fe55c1528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:42.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 shutdown_connections 2023-10-06T03:13:42.310 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:42.307+0000 7fe562788700 1 -- 172.21.15.92:0/2977713370 wait complete. 2023-10-06T03:13:43.322 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T03:13:43.322 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:43.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.446+0000 7fa6a8fe1700 1 Processor -- start 2023-10-06T03:13:43.449 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.446+0000 7fa6a8fe1700 1 -- start start 2023-10-06T03:13:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a8fe1700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa6a4157540 0x7fa6a40c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a8fe1700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6a40c92f0 0x7fa6a40cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a8fe1700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a40c9710 con 0x7fa6a4157910 2023-10-06T03:13:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a8fe1700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a40cbbe0 con 0x7fa6a40c92f0 2023-10-06T03:13:43.451 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a8fe1700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a415b560 con 0x7fa6a4157540 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6a40c92f0 0x7fa6a40cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a2ffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa6a4157910 0x7fa6a40cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44022/0 (socket says 172.21.15.92:44022) 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a3fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa6a40c92f0 0x7fa6a40cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40174/0 (socket says 172.21.15.92:40174) 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a2ffd700 1 -- 172.21.15.92:0/3468250387 learned_addr learned my addr 172.21.15.92:0/3468250387 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a37fe700 1 --2- 172.21.15.92:0/3468250387 >> v2:172.21.15.136:3300/0 conn(0x7fa6a4157540 0x7fa6a40c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3468250387 >> v2:172.21.15.136:3300/0 conn(0x7fa6a4157540 msgr2=0x7fa6a40c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a3fff700 1 --2- 172.21.15.92:0/3468250387 >> v2:172.21.15.136:3300/0 conn(0x7fa6a4157540 0x7fa6a40c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.452 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3468250387 >> v1:172.21.15.92:6789/0 conn(0x7fa6a4157910 legacy=0x7fa6a40cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a27fc700 1 -- 172.21.15.92:0/3468250387 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521508129 0 0) 0x7fa6a40c9710 con 0x7fa6a4157910 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.447+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3468250387 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a415b840 con 0x7fa6a40c92f0 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a3fff700 1 --2- 172.21.15.92:0/3468250387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a40c92f0 0x7fa6a40cb4e0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa6a406f350 tx=0x7fa69c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd67d03c7f948a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a27fc700 1 -- 172.21.15.92:0/3468250387 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00b6b0 con 0x7fa6a40c92f0 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a27fc700 1 -- 172.21.15.92:0/3468250387 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c00b810 con 0x7fa6a40c92f0 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a27fc700 1 -- 172.21.15.92:0/3468250387 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c00bbb0 con 0x7fa6a40c92f0 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3468250387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a40c92f0 msgr2=0x7fa6a40cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.453 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 --2- 172.21.15.92:0/3468250387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a40c92f0 0x7fa6a40cb4e0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa6a406f350 tx=0x7fa69c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3468250387 shutdown_connections 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 --2- 172.21.15.92:0/3468250387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a40c92f0 0x7fa6a40cb4e0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 --2- 172.21.15.92:0/3468250387 >> v2:172.21.15.136:3300/0 conn(0x7fa6a4157540 0x7fa6a40c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3468250387 >> 172.21.15.92:0/3468250387 conn(0x7fa6a406baf0 msgr2=0x7fa6a406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3468250387 shutdown_connections 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.448+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3468250387 wait complete. 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 Processor -- start 2023-10-06T03:13:43.454 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 -- start start 2023-10-06T03:13:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6a40c92f0 0x7fa6a41491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6a4157540 0x7fa6a40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a4157910 0x7fa6a41592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a416bec0 con 0x7fa6a4157910 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a4151d50 con 0x7fa6a4157540 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a8fe1700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a416b2b0 con 0x7fa6a40c92f0 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a4157910 0x7fa6a41592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a4157910 0x7fa6a41592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40180/0 (socket says 172.21.15.92:40180) 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.450+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3315090208 learned_addr learned my addr 172.21.15.92:0/3315090208 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:43.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3315090208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6a40c92f0 msgr2=0x7fa6a41491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a37fe700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6a40c92f0 0x7fa6a41491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a2ffd700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6a4157540 0x7fa6a40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a3fff700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6a40c92f0 0x7fa6a41491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3315090208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6a4157540 msgr2=0x7fa6a40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a3fff700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6a4157540 0x7fa6a40b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3315090208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c009420 con 0x7fa6a4157910 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a3fff700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a4157910 0x7fa6a41592d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa6a406ce10 tx=0x7fa69c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011630 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c011790 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a416b590 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c011b30 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.451+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a416bb50 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.452+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6a404c3c0 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.452+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa69c02e440 con 0x7fa6a4157910 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa68b7fe700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa690067320 0x7fa6900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa69c0866a0 con 0x7fa6a4157910 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa6a37fe700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa690067320 0x7fa6900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa68b7fe700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7fa69006d790 0x7fa69006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 --> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa690070270 con 0x7fa69006d790 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa69c088c00 con 0x7fa6a4157910 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa6a2ffd700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7fa69006d790 0x7fa69006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa6a37fe700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa690067320 0x7fa6900697d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa694007460 tx=0x7fa69400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.453+0000 7fa6a2ffd700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7fa69006d790 0x7fa69006fba0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.454+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== osd.0 v2:172.21.15.92:6808/90688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa690070270 con 0x7fa69006d790 2023-10-06T03:13:43.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 --> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6a40c8db0 con 0x7fa69006d790 2023-10-06T03:13:43.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa68b7fe700 1 -- 172.21.15.92:0/3315090208 <== osd.0 v2:172.21.15.92:6808/90688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa6a40c8db0 con 0x7fa69006d790 2023-10-06T03:13:43.470 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:43.470 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:43.470 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7fa69006d790 msgr2=0x7fa69006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6808/90688,v1:172.21.15.92:6809/90688] conn(0x7fa69006d790 0x7fa69006fba0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa690067320 msgr2=0x7fa6900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa690067320 0x7fa6900697d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa694007460 tx=0x7fa69400b040 comp rx=0 tx=0).stop 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a4157910 msgr2=0x7fa6a41592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 --2- 172.21.15.92:0/3315090208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa6a4157910 0x7fa6a41592d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa6a406ce10 tx=0x7fa69c017000 comp rx=0 tx=0).stop 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a3fff700 1 -- 172.21.15.92:0/3315090208 reap_dead start 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 shutdown_connections 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.468+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 >> 172.21.15.92:0/3315090208 conn(0x7fa6a406baf0 msgr2=0x7fa6a414a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.469+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 shutdown_connections 2023-10-06T03:13:43.472 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.469+0000 7fa6a8fe1700 1 -- 172.21.15.92:0/3315090208 wait complete. 2023-10-06T03:13:43.481 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T03:13:43.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:13:43.527 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:13:43.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.599+0000 7f56cf750700 1 Processor -- start 2023-10-06T03:13:43.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.599+0000 7f56cf750700 1 -- start start 2023-10-06T03:13:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56cf750700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56c806c0a0 0x7f56c80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56cf750700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f56c815df50 0x7f56c815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.603 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56cf750700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c815d7e0 con 0x7f56c806bcd0 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56cf750700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c815dac0 con 0x7f56c806c0a0 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56cf750700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c815aa30 con 0x7f56c815df50 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56c806c0a0 0x7f56c80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f56c806c0a0 0x7f56c80caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40194/0 (socket says 172.21.15.92:40194) 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 -- 172.21.15.92:0/4116580586 learned_addr learned my addr 172.21.15.92:0/4116580586 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 -- 172.21.15.92:0/4116580586 >> v2:172.21.15.136:3300/0 conn(0x7f56c815df50 msgr2=0x7f56c815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:43.604 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 --2- 172.21.15.92:0/4116580586 >> v2:172.21.15.136:3300/0 conn(0x7f56c815df50 0x7f56c815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 -- 172.21.15.92:0/4116580586 >> v1:172.21.15.136:6789/0 conn(0x7f56c806bcd0 legacy=0x7f56c815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 -- 172.21.15.92:0/4116580586 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c815ad10 con 0x7f56c806c0a0 2023-10-06T03:13:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56ccceb700 1 --2- 172.21.15.92:0/4116580586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c806c0a0 0x7f56c80caa90 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f56c815d720 tx=0x7f56b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0bee752c7ad4ed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.600+0000 7f56bffff700 1 -- 172.21.15.92:0/4116580586 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800b820 con 0x7f56c806c0a0 2023-10-06T03:13:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56bffff700 1 -- 172.21.15.92:0/4116580586 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b800b980 con 0x7f56c806c0a0 2023-10-06T03:13:43.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56bffff700 1 -- 172.21.15.92:0/4116580586 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800bd40 con 0x7f56c806c0a0 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 -- 172.21.15.92:0/4116580586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c806c0a0 msgr2=0x7f56c80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 --2- 172.21.15.92:0/4116580586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c806c0a0 0x7f56c80caa90 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f56c815d720 tx=0x7f56b8009250 comp rx=0 tx=0).stop 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 -- 172.21.15.92:0/4116580586 shutdown_connections 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 --2- 172.21.15.92:0/4116580586 >> v2:172.21.15.136:3300/0 conn(0x7f56c815df50 0x7f56c815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 --2- 172.21.15.92:0/4116580586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c806c0a0 0x7f56c80caa90 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 -- 172.21.15.92:0/4116580586 >> 172.21.15.92:0/4116580586 conn(0x7f56c80ce600 msgr2=0x7f56c80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:43.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 -- 172.21.15.92:0/4116580586 shutdown_connections 2023-10-06T03:13:43.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.601+0000 7f56cf750700 1 -- 172.21.15.92:0/4116580586 wait complete. 2023-10-06T03:13:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.602+0000 7f56cf750700 1 Processor -- start 2023-10-06T03:13:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.602+0000 7f56cf750700 1 -- start start 2023-10-06T03:13:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cf750700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56c806bcd0 0x7f56c80c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cf750700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56c806c0a0 0x7f56c80b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cf750700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 0x7f56c816a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cf750700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c6430 con 0x7f56c815df50 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cf750700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c87d0 con 0x7f56c806bcd0 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cf750700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c77f0 con 0x7f56c806c0a0 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 0x7f56c816a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 0x7f56c816a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40208/0 (socket says 172.21.15.92:40208) 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 -- 172.21.15.92:0/3778567762 learned_addr learned my addr 172.21.15.92:0/3778567762 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56ccceb700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56c806c0a0 0x7f56c80b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.611 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cd4ec700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56c806bcd0 0x7f56c80c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 -- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56c806c0a0 msgr2=0x7f56c80b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56c806c0a0 0x7f56c80b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 -- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56c806bcd0 msgr2=0x7f56c80c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56c806bcd0 0x7f56c80c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.603+0000 7f56cdced700 1 -- 172.21.15.92:0/3778567762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80094f0 con 0x7f56c815df50 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.604+0000 7f56cdced700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 0x7f56c816a2c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f56c000bc30 tx=0x7f56c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.612 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.604+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c00092e0 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.604+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c000c7a0 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.604+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c816adc0 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.604+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c000cb80 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.604+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c80c7d70 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.605+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c804c3c0 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.605+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f56c0012070 con 0x7f56c815df50 2023-10-06T03:13:43.613 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.606+0000 7f56bd7fa700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56ac067320 0x7f56ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.606+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f56c0084790 con 0x7f56c815df50 2023-10-06T03:13:43.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.606+0000 7f56cd4ec700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56ac067320 0x7f56ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.606+0000 7f56cd4ec700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56ac067320 0x7f56ac0697d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006fd0 tx=0x7f56c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.614 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.610+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56c00515c0 con 0x7f56c815df50 2023-10-06T03:13:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.792+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c815aff0 con 0x7f56ac067320 2023-10-06T03:13:43.794 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.792+0000 7f56bd7fa700 1 -- 172.21.15.92:0/3778567762 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f56c815aff0 con 0x7f56ac067320 2023-10-06T03:13:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56ac067320 msgr2=0x7f56ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56ac067320 0x7f56ac0697d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006fd0 tx=0x7f56c4008040 comp rx=0 tx=0).stop 2023-10-06T03:13:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 msgr2=0x7f56c816a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 0x7f56c816a2c0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f56c000bc30 tx=0x7f56c0009840 comp rx=0 tx=0).stop 2023-10-06T03:13:43.798 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 shutdown_connections 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f56ac067320 0x7f56ac0697d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f56c815df50 0x7f56c816a2c0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56c806bcd0 0x7f56c80c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 --2- 172.21.15.92:0/3778567762 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56c806c0a0 0x7f56c80b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.794+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 >> 172.21.15.92:0/3778567762 conn(0x7f56c80ce600 msgr2=0x7f56c80c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.795+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 shutdown_connections 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.795+0000 7f56cf750700 1 -- 172.21.15.92:0/3778567762 wait complete. 2023-10-06T03:13:43.799 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T03:13:43.808 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T03:13:43.808 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:43.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.966+0000 7f8f021fd700 1 Processor -- start 2023-10-06T03:13:43.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.966+0000 7f8f021fd700 1 -- start start 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8f021fd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8efc153f50 0x7f8efc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8f021fd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc0b4570 con 0x7f8efc154320 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8f021fd700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc15a730 con 0x7f8efc06d840 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8f021fd700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc15aa10 con 0x7f8efc153f50 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efaffd700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8efc154320 0x7f8efc06d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44034/0 (socket says 172.21.15.92:44034) 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efaffd700 1 -- 172.21.15.92:0/3532928879 learned_addr learned my addr 172.21.15.92:0/3532928879 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:43.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efb7fe700 1 --2- 172.21.15.92:0/3532928879 >> v2:172.21.15.136:3300/0 conn(0x7f8efc153f50 0x7f8efc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479616412 0 0) 0x7f8efc0b4570 con 0x7f8efc154320 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee4003570 con 0x7f8efc154320 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365179667 0 0) 0x7f8ee4003570 con 0x7f8efc154320 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 >> v2:172.21.15.136:3300/0 conn(0x7f8efc153f50 msgr2=0x7f8efc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efa7fc700 1 --2- 172.21.15.92:0/3532928879 >> v2:172.21.15.136:3300/0 conn(0x7f8efc153f50 0x7f8efc0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.967+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 >> v1:172.21.15.136:6789/0 conn(0x7f8efc06d840 legacy=0x7f8efc069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc15acf0 con 0x7f8efc154320 2023-10-06T03:13:43.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f8ef0001470 con 0x7f8efc154320 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 >> v1:172.21.15.92:6789/0 conn(0x7f8efc154320 legacy=0x7f8efc06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003a60 0x7f8ee4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4004810 0x7f8ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee40051c0 0x7f8ee400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc15aa10 con 0x7f8ee4004810 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee400c0d0 con 0x7f8ee40051c0 2023-10-06T03:13:43.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee400c290 con 0x7f8ee4003a60 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4004810 0x7f8ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efb7fe700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003a60 0x7f8ee4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efbfff700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee40051c0 0x7f8ee400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 -- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003a60 msgr2=0x7f8ee4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ee4003a60 0x7f8ee4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 -- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee40051c0 msgr2=0x7f8ee400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:43.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee40051c0 0x7f8ee400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efbfff700 1 -- 172.21.15.92:0/3532928879 reap_dead start 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 -- 172.21.15.92:0/3532928879 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee4003570 con 0x7f8ee4004810 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efaffd700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4004810 0x7f8ee4004c80 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000c3c0 tx=0x7f8ef0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57adfabf5f9f2fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000cc90 con 0x7f8ee4004810 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef000cdf0 con 0x7f8ee4004810 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.968+0000 7f8efa7fc700 1 -- 172.21.15.92:0/3532928879 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0005b50 con 0x7f8ee4004810 2023-10-06T03:13:43.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 -- 172.21.15.92:0/3532928879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4004810 msgr2=0x7f8ee4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4004810 0x7f8ee4004c80 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000c3c0 tx=0x7f8ef0001e50 comp rx=0 tx=0).stop 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 -- 172.21.15.92:0/3532928879 shutdown_connections 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 --2- 172.21.15.92:0/3532928879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ee4004810 0x7f8ee4004c80 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 -- 172.21.15.92:0/3532928879 >> 172.21.15.92:0/3532928879 conn(0x7f8efc06baf0 msgr2=0x7f8efc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 -- 172.21.15.92:0/3532928879 shutdown_connections 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.969+0000 7f8f021fd700 1 -- 172.21.15.92:0/3532928879 wait complete. 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.970+0000 7f8f021fd700 1 Processor -- start 2023-10-06T03:13:43.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.970+0000 7f8f021fd700 1 -- start start 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8f021fd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004810 0x7f8efc169970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8f021fd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc15b880 0x7f8efc15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8f021fd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8efc16d150 0x7f8efc16d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8f021fd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0ceb50 con 0x7f8efc16d150 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8f021fd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc15c230 con 0x7f8ee4004810 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8f021fd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc16a8f0 con 0x7f8efc15b880 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8efc16d150 0x7f8efc16d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.984 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8efc16d150 0x7f8efc16d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40228/0 (socket says 172.21.15.92:40228) 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 -- 172.21.15.92:0/1330785540 learned_addr learned my addr 172.21.15.92:0/1330785540 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efb7fe700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004810 0x7f8efc169970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 -- 172.21.15.92:0/1330785540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc15b880 msgr2=0x7f8efc15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc15b880 0x7f8efc15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 -- 172.21.15.92:0/1330785540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004810 msgr2=0x7f8efc169970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004810 0x7f8efc169970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 -- 172.21.15.92:0/1330785540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc16abb0 con 0x7f8efc16d150 2023-10-06T03:13:43.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efb7fe700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ee4004810 0x7f8efc169970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8efbfff700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8efc16d150 0x7f8efc16d5f0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b2e30 tx=0x7f8ef4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.971+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4006db0 con 0x7f8efc16d150 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.972+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400c5a0 con 0x7f8efc16d150 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.972+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc16a050 con 0x7f8efc16d150 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.972+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400c960 con 0x7f8efc16d150 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.972+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc16a610 con 0x7f8efc16d150 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.972+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8efc0718b0 con 0x7f8efc16d150 2023-10-06T03:13:43.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.973+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8ef400cd10 con 0x7f8efc16d150 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.973+0000 7f8ee37fe700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ee8067320 0x7f8ee80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.973+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f8ef40843c0 con 0x7f8efc16d150 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.973+0000 7f8efb7fe700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ee8067320 0x7f8ee80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.973+0000 7f8ee37fe700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f8ee806d790 0x7f8ee806fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.974+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 --> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ee8070270 con 0x7f8ee806d790 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.974+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ef40868d0 con 0x7f8efc16d150 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.974+0000 7f8efaffd700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f8ee806d790 0x7f8ee806fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:43.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.974+0000 7f8efb7fe700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ee8067320 0x7f8ee80697d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0093f0 tx=0x7f8eec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.974+0000 7f8efaffd700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f8ee806d790 0x7f8ee806fba0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:43.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.974+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== osd.1 v2:172.21.15.92:6800/90685 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8ee8070270 con 0x7f8ee806d790 2023-10-06T03:13:43.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 --> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8efc15afd0 con 0x7f8ee806d790 2023-10-06T03:13:43.990 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8ee37fe700 1 -- 172.21.15.92:0/1330785540 <== osd.1 v2:172.21.15.92:6800/90685 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8efc15afd0 con 0x7f8ee806d790 2023-10-06T03:13:43.990 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:43.990 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:43.991 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f8ee806d790 msgr2=0x7f8ee806fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6800/90685,v1:172.21.15.92:6801/90685] conn(0x7f8ee806d790 0x7f8ee806fba0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ee8067320 msgr2=0x7f8ee80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8ee8067320 0x7f8ee80697d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0093f0 tx=0x7f8eec008040 comp rx=0 tx=0).stop 2023-10-06T03:13:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8efc16d150 msgr2=0x7f8efc16d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:43.992 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 --2- 172.21.15.92:0/1330785540 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8efc16d150 0x7f8efc16d5f0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b2e30 tx=0x7f8ef4001640 comp rx=0 tx=0).stop 2023-10-06T03:13:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8efbfff700 1 -- 172.21.15.92:0/1330785540 reap_dead start 2023-10-06T03:13:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 shutdown_connections 2023-10-06T03:13:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.988+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 >> 172.21.15.92:0/1330785540 conn(0x7f8efc06baf0 msgr2=0x7f8efc06d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.989+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 shutdown_connections 2023-10-06T03:13:43.993 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:43.989+0000 7f8f021fd700 1 -- 172.21.15.92:0/1330785540 wait complete. 2023-10-06T03:13:44.002 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T03:13:44.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.114+0000 7f595716d700 1 Processor -- start 2023-10-06T03:13:44.116 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.114+0000 7f595716d700 1 -- start start 2023-10-06T03:13:44.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f595716d700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5950069560 0x7f59500699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f595716d700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950069ee0 con 0x7f5950159890 2023-10-06T03:13:44.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f595716d700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595015a160 con 0x7f5950159c60 2023-10-06T03:13:44.119 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f595716d700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500d13a0 con 0x7f5950069560 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f5954f09700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f5950159890 0x7f595015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44042/0 (socket says 172.21.15.92:44042) 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f5954f09700 1 -- 172.21.15.92:0/2516130823 learned_addr learned my addr 172.21.15.92:0/2516130823 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.115+0000 7f595570a700 1 --2- 172.21.15.92:0/2516130823 >> v2:172.21.15.136:3300/0 conn(0x7f5950069560 0x7f59500699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214239991 0 0) 0x7f5950069ee0 con 0x7f5950159890 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5938003570 con 0x7f5950159890 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543004673 0 0) 0x7f595015a160 con 0x7f5950159c60 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5950069ee0 con 0x7f5950159c60 2023-10-06T03:13:44.120 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f595570a700 1 -- 172.21.15.92:0/2516130823 >> v1:172.21.15.136:6789/0 conn(0x7f5950159c60 legacy=0x7f595015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286431540 0 0) 0x7f5938003570 con 0x7f5950159890 2023-10-06T03:13:44.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f595570a700 1 -- 172.21.15.92:0/2516130823 >> v1:172.21.15.92:6789/0 conn(0x7f5950159890 legacy=0x7f595015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f595570a700 1 -- 172.21.15.92:0/2516130823 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595015e140 con 0x7f5950069560 2023-10-06T03:13:44.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f595570a700 1 --2- 172.21.15.92:0/2516130823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950069560 0x7f59500699a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f59480098e0 tx=0x7f59480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62d58a79d7cc319 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0017f0 con 0x7f5950069560 2023-10-06T03:13:44.121 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948011420 con 0x7f5950069560 2023-10-06T03:13:44.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.116+0000 7f59477fe700 1 -- 172.21.15.92:0/2516130823 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480117a0 con 0x7f5950069560 2023-10-06T03:13:44.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 -- 172.21.15.92:0/2516130823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950069560 msgr2=0x7f59500699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 --2- 172.21.15.92:0/2516130823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950069560 0x7f59500699a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f59480098e0 tx=0x7f59480091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 -- 172.21.15.92:0/2516130823 shutdown_connections 2023-10-06T03:13:44.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 --2- 172.21.15.92:0/2516130823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950069560 0x7f59500699a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.122 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 -- 172.21.15.92:0/2516130823 >> 172.21.15.92:0/2516130823 conn(0x7f5950068e20 msgr2=0x7f59500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 -- 172.21.15.92:0/2516130823 shutdown_connections 2023-10-06T03:13:44.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.117+0000 7f595716d700 1 -- 172.21.15.92:0/2516130823 wait complete. 2023-10-06T03:13:44.123 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.118+0000 7f595716d700 1 Processor -- start 2023-10-06T03:13:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.118+0000 7f595716d700 1 -- start start 2023-10-06T03:13:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 0x7f59501441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950159890 0x7f59500b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.125 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595016a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015c110 con 0x7f5950069560 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015b570 con 0x7f5950159890 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595015b7e0 con 0x7f5950159c60 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 0x7f59501441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 0x7f59501441c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40236/0 (socket says 172.21.15.92:40236) 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 -- 172.21.15.92:0/3319599924 learned_addr learned my addr 172.21.15.92:0/3319599924 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:44.126 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5947fff700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950159890 0x7f59500b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595570a700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595016a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 -- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 msgr2=0x7f595016a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595016a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 -- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950159890 msgr2=0x7f59500b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950159890 0x7f59500b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 -- 172.21.15.92:0/3319599924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009420 con 0x7f5950069560 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5954f09700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 0x7f59501441c0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f594c00cc70 tx=0x7f594c007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.127 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00ce80 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c004dc0 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00abb0 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595015bac0 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.119+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5950144700 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.120+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595004c3c0 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.121+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f594c007150 con 0x7f5950069560 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.121+0000 7f5944ff9700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067320 0x7f593c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.128 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.121+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f594c0843c0 con 0x7f5950069560 2023-10-06T03:13:44.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.121+0000 7f5947fff700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067320 0x7f593c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.122+0000 7f5947fff700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067320 0x7f593c0697d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f59500cd0e0 tx=0x7f5940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.129 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.125+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f594c0511f0 con 0x7f5950069560 2023-10-06T03:13:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.311+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f595015c250 con 0x7f593c067320 2023-10-06T03:13:44.313 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.311+0000 7f5944ff9700 1 -- 172.21.15.92:0/3319599924 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5940013420 con 0x7f593c067320 2023-10-06T03:13:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067320 msgr2=0x7f593c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.316 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067320 0x7f593c0697d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f59500cd0e0 tx=0x7f5940009040 comp rx=0 tx=0).stop 2023-10-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 msgr2=0x7f59501441c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 0x7f59501441c0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f594c00cc70 tx=0x7f594c007d20 comp rx=0 tx=0).stop 2023-10-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 shutdown_connections 2023-10-06T03:13:44.317 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f593c067320 0x7f593c0697d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5950069560 0x7f59501441c0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5950159c60 0x7f595016a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 --2- 172.21.15.92:0/3319599924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5950159890 0x7f59500b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.318 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 >> 172.21.15.92:0/3319599924 conn(0x7f5950068e20 msgr2=0x7f5950157210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 shutdown_connections 2023-10-06T03:13:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.313+0000 7f595716d700 1 -- 172.21.15.92:0/3319599924 wait complete. 2023-10-06T03:13:44.319 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T03:13:44.327 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T03:13:44.328 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:44.447 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.445+0000 7f695b1d2700 1 Processor -- start 2023-10-06T03:13:44.448 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.445+0000 7f695b1d2700 1 -- start start 2023-10-06T03:13:44.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f695b1d2700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69540b42c0 0x7f69540ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.456 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f695b1d2700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f69540ceb60 0x7f69540cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f695b1d2700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69540cefd0 con 0x7f6954155360 2023-10-06T03:13:44.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f695b1d2700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695415c320 con 0x7f69540b42c0 2023-10-06T03:13:44.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f695b1d2700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695415b710 con 0x7f69540ceb60 2023-10-06T03:13:44.457 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f6953fff700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f69540b42c0 0x7f69540ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f6958f6e700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f6954155360 0x7f69540b3bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44056/0 (socket says 172.21.15.92:44056) 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.446+0000 7f6958f6e700 1 -- 172.21.15.92:0/3596847113 learned_addr learned my addr 172.21.15.92:0/3596847113 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f6953fff700 1 -- 172.21.15.92:0/3596847113 >> v2:172.21.15.136:3300/0 conn(0x7f69540ceb60 msgr2=0x7f69540cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f6953fff700 1 --2- 172.21.15.92:0/3596847113 >> v2:172.21.15.136:3300/0 conn(0x7f69540ceb60 0x7f69540cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f6953fff700 1 -- 172.21.15.92:0/3596847113 >> v1:172.21.15.92:6789/0 conn(0x7f6954155360 legacy=0x7f69540b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f69537fe700 1 -- 172.21.15.92:0/3596847113 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039439068 0 0) 0x7f6944001490 con 0x7f6954155360 2023-10-06T03:13:44.458 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f6953fff700 1 -- 172.21.15.92:0/3596847113 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695415b9f0 con 0x7f69540b42c0 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f6953fff700 1 --2- 172.21.15.92:0/3596847113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540b42c0 0x7f69540ce620 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f695406f5b0 tx=0x7f69480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73ae09277f29a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f69537fe700 1 -- 172.21.15.92:0/3596847113 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800b6b0 con 0x7f69540b42c0 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f69537fe700 1 -- 172.21.15.92:0/3596847113 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694800b810 con 0x7f69540b42c0 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f69537fe700 1 -- 172.21.15.92:0/3596847113 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694800bbb0 con 0x7f69540b42c0 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f695b1d2700 1 -- 172.21.15.92:0/3596847113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540b42c0 msgr2=0x7f69540ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f695b1d2700 1 --2- 172.21.15.92:0/3596847113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540b42c0 0x7f69540ce620 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f695406f5b0 tx=0x7f69480091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.459 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f695b1d2700 1 -- 172.21.15.92:0/3596847113 shutdown_connections 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f695b1d2700 1 --2- 172.21.15.92:0/3596847113 >> v2:172.21.15.136:3300/0 conn(0x7f69540ceb60 0x7f69540cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f695b1d2700 1 --2- 172.21.15.92:0/3596847113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540b42c0 0x7f69540ce620 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.447+0000 7f695b1d2700 1 -- 172.21.15.92:0/3596847113 >> 172.21.15.92:0/3596847113 conn(0x7f695406baf0 msgr2=0x7f695406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.448+0000 7f695b1d2700 1 -- 172.21.15.92:0/3596847113 shutdown_connections 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.448+0000 7f695b1d2700 1 -- 172.21.15.92:0/3596847113 wait complete. 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 Processor -- start 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 -- start start 2023-10-06T03:13:44.460 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69540b42c0 0x7f6954167f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540ceb60 0x7f695416ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6954155360 0x7f69540b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69540adee0 con 0x7f69540ceb60 2023-10-06T03:13:44.461 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69540ae130 con 0x7f69540b42c0 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.449+0000 7f695b1d2700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69540ae410 con 0x7f6954155360 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540ceb60 0x7f695416ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540ceb60 0x7f695416ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40262/0 (socket says 172.21.15.92:40262) 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 -- 172.21.15.92:0/795772094 learned_addr learned my addr 172.21.15.92:0/795772094 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6958f6e700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69540b42c0 0x7f6954167f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f695976f700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6954155360 0x7f69540b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.462 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 -- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6954155360 msgr2=0x7f69540b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6954155360 0x7f69540b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 -- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69540b42c0 msgr2=0x7f6954167f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69540b42c0 0x7f6954167f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 -- 172.21.15.92:0/795772094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6948009420 con 0x7f69540ceb60 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6958f6e700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69540b42c0 0x7f6954167f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6953fff700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540ceb60 0x7f695416ac40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f6948013000 tx=0x7f6948017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.463 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948011420 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6948011580 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69540ae6f0 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6948011940 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.450+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695416b180 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.451+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69540718b0 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.451+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6948011cb0 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6950ff9700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6940067320 0x7f69400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f69480864c0 con 0x7f69540ceb60 2023-10-06T03:13:44.464 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6958f6e700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6940067320 0x7f69400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6950ff9700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f694006d790 0x7f694006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 --> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6940070270 con 0x7f694006d790 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6948088a20 con 0x7f69540ceb60 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f695976f700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f694006d790 0x7f694006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f6958f6e700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6940067320 0x7f69400697d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f694400e280 tx=0x7f6944009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.452+0000 7f695976f700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f694006d790 0x7f694006fba0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.465 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.453+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== osd.2 v2:172.21.15.92:6812/90689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6940070270 con 0x7f694006d790 2023-10-06T03:13:44.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.466+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 --> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f695415bcd0 con 0x7f694006d790 2023-10-06T03:13:44.468 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f6950ff9700 1 -- 172.21.15.92:0/795772094 <== osd.2 v2:172.21.15.92:6812/90689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f695415bcd0 con 0x7f694006d790 2023-10-06T03:13:44.468 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:44.469 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:44.469 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:44.469 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f694006d790 msgr2=0x7f694006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6812/90689,v1:172.21.15.92:6816/90689] conn(0x7f694006d790 0x7f694006fba0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6940067320 msgr2=0x7f69400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f6940067320 0x7f69400697d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f694400e280 tx=0x7f6944009040 comp rx=0 tx=0).stop 2023-10-06T03:13:44.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540ceb60 msgr2=0x7f695416ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 --2- 172.21.15.92:0/795772094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f69540ceb60 0x7f695416ac40 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f6948013000 tx=0x7f6948017000 comp rx=0 tx=0).stop 2023-10-06T03:13:44.470 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695976f700 1 -- 172.21.15.92:0/795772094 reap_dead start 2023-10-06T03:13:44.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 shutdown_connections 2023-10-06T03:13:44.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 >> 172.21.15.92:0/795772094 conn(0x7f695406baf0 msgr2=0x7f695406d130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 shutdown_connections 2023-10-06T03:13:44.471 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.467+0000 7f695b1d2700 1 -- 172.21.15.92:0/795772094 wait complete. 2023-10-06T03:13:44.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T03:13:44.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.594+0000 7f035cdeb700 1 Processor -- start 2023-10-06T03:13:44.596 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.594+0000 7f035cdeb700 1 -- start start 2023-10-06T03:13:44.598 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035cdeb700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0358159890 0x7f035815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035cdeb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358069560 0x7f03580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035cdeb700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0358069ee0 con 0x7f0358159c60 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035cdeb700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035815a160 con 0x7f0358159890 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035cdeb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580d13a0 con 0x7f0358069560 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0358159890 0x7f035815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f034dd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0358159c60 0x7f035815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44062/0 (socket says 172.21.15.92:44062) 2023-10-06T03:13:44.599 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035659c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0358159890 0x7f035815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40272/0 (socket says 172.21.15.92:40272) 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f034dd9b700 1 -- 172.21.15.92:0/936697191 learned_addr learned my addr 172.21.15.92:0/936697191 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035659c700 1 -- 172.21.15.92:0/936697191 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358069560 msgr2=0x7f03580699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f0356d9d700 1 --2- 172.21.15.92:0/936697191 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358069560 0x7f03580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035659c700 1 --2- 172.21.15.92:0/936697191 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358069560 0x7f03580699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035659c700 1 -- 172.21.15.92:0/936697191 >> v1:172.21.15.92:6789/0 conn(0x7f0358159c60 legacy=0x7f035815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f0355d9b700 1 -- 172.21.15.92:0/936697191 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219410727 0 0) 0x7f0358069ee0 con 0x7f0358159c60 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.595+0000 7f035659c700 1 -- 172.21.15.92:0/936697191 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035815e140 con 0x7f0358159890 2023-10-06T03:13:44.600 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035659c700 1 --2- 172.21.15.92:0/936697191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f035815abd0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f03480098e0 tx=0x7f03480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cd4ef1d2c06802 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f0355d9b700 1 -- 172.21.15.92:0/936697191 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348009e80 con 0x7f0358159890 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f0355d9b700 1 -- 172.21.15.92:0/936697191 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034800b860 con 0x7f0358159890 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f0355d9b700 1 -- 172.21.15.92:0/936697191 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800bbe0 con 0x7f0358159890 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 -- 172.21.15.92:0/936697191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 msgr2=0x7f035815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 --2- 172.21.15.92:0/936697191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f035815abd0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f03480098e0 tx=0x7f03480091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 -- 172.21.15.92:0/936697191 shutdown_connections 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 --2- 172.21.15.92:0/936697191 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358069560 0x7f03580699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.601 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 --2- 172.21.15.92:0/936697191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f035815abd0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 -- 172.21.15.92:0/936697191 >> 172.21.15.92:0/936697191 conn(0x7f0358068e20 msgr2=0x7f03580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 -- 172.21.15.92:0/936697191 shutdown_connections 2023-10-06T03:13:44.602 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.596+0000 7f035cdeb700 1 -- 172.21.15.92:0/936697191 wait complete. 2023-10-06T03:13:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 Processor -- start 2023-10-06T03:13:44.605 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 -- start start 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0358069560 0x7f035814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f0358155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358159c60 0x7f0358156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580ceb40 con 0x7f0358159890 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580cdea0 con 0x7f0358069560 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f035cdeb700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580ce180 con 0x7f0358159c60 2023-10-06T03:13:44.606 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.598+0000 7f034dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f0358155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f0358155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40278/0 (socket says 172.21.15.92:40278) 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 -- 172.21.15.92:0/57361371 learned_addr learned my addr 172.21.15.92:0/57361371 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f035659c700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0358069560 0x7f035814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f0356d9d700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358159c60 0x7f0358156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 -- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358159c60 msgr2=0x7f0358156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358159c60 0x7f0358156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 -- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0358069560 msgr2=0x7f035814dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.607 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0358069560 0x7f035814dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 -- 172.21.15.92:0/57361371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348009420 con 0x7f0358159890 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034dd9b700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f0358155ed0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f034000c7e0 tx=0x7f0340007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340005b30 con 0x7f0358159890 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0340005c90 con 0x7f0358159890 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03580ce4f0 con 0x7f0358159890 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340007300 con 0x7f0358159890 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.599+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03581517c0 con 0x7f0358159890 2023-10-06T03:13:44.608 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.600+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035804c3c0 con 0x7f0358159890 2023-10-06T03:13:44.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.600+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f034001b3b0 con 0x7f0358159890 2023-10-06T03:13:44.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.601+0000 7f034effd700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0338067320 0x7f03380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.601+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0340083ff0 con 0x7f0358159890 2023-10-06T03:13:44.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.601+0000 7f035659c700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0338067320 0x7f03380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.609 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.601+0000 7f035659c700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0338067320 0x7f03380697d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f03480091e0 tx=0x7f0348017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.610 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.605+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0340050e20 con 0x7f0358159890 2023-10-06T03:13:44.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.786+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f035815a6a0 con 0x7f0338067320 2023-10-06T03:13:44.788 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.787+0000 7f034effd700 1 -- 172.21.15.92:0/57361371 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f035815a6a0 con 0x7f0338067320 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0338067320 msgr2=0x7f03380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0338067320 0x7f03380697d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f03480091e0 tx=0x7f0348017000 comp rx=0 tx=0).stop 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 msgr2=0x7f0358155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f0358155ed0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f034000c7e0 tx=0x7f0340007df0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 shutdown_connections 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f0338067320 0x7f03380697d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.791 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358159c60 0x7f0358156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0358069560 0x7f035814dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 --2- 172.21.15.92:0/57361371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0358159890 0x7f0358155ed0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 >> 172.21.15.92:0/57361371 conn(0x7f0358068e20 msgr2=0x7f0358158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 shutdown_connections 2023-10-06T03:13:44.792 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.789+0000 7f035cdeb700 1 -- 172.21.15.92:0/57361371 wait complete. 2023-10-06T03:13:44.793 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T03:13:44.803 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T03:13:44.803 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:44.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.920+0000 7f4a56e61700 1 Processor -- start 2023-10-06T03:13:44.922 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.920+0000 7f4a56e61700 1 -- start start 2023-10-06T03:13:44.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56e61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a500b03c0 0x7f4a500cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56e61700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a500b0790 0x7f4a500b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56e61700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4a501591e0 0x7f4a50159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.924 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56e61700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a500cfa00 con 0x7f4a500b0790 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56e61700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a500cb580 con 0x7f4a501591e0 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56e61700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500b2de0 con 0x7f4a500b03c0 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a500b0790 0x7f4a500b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4a500b0790 0x7f4a500b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40286/0 (socket says 172.21.15.92:40286) 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 -- 172.21.15.92:0/754009407 learned_addr learned my addr 172.21.15.92:0/754009407 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a55e5f700 1 --2- 172.21.15.92:0/754009407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a500b03c0 0x7f4a500cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a56660700 1 --2- 172.21.15.92:0/754009407 >> v2:172.21.15.136:3300/0 conn(0x7f4a501591e0 0x7f4a50159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.925 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 -- 172.21.15.92:0/754009407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a500b03c0 msgr2=0x7f4a500cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 --2- 172.21.15.92:0/754009407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a500b03c0 0x7f4a500cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 -- 172.21.15.92:0/754009407 >> v2:172.21.15.136:3300/0 conn(0x7f4a501591e0 msgr2=0x7f4a50159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 --2- 172.21.15.92:0/754009407 >> v2:172.21.15.136:3300/0 conn(0x7f4a501591e0 0x7f4a50159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.921+0000 7f4a5565e700 1 -- 172.21.15.92:0/754009407 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500cf160 con 0x7f4a500b0790 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a5565e700 1 --2- 172.21.15.92:0/754009407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a500b28a0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cb4f0 tx=0x7f4a40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f09cf805e1a14f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a54e5d700 1 -- 172.21.15.92:0/754009407 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000b640 con 0x7f4a500b0790 2023-10-06T03:13:44.926 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a54e5d700 1 -- 172.21.15.92:0/754009407 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000b7a0 con 0x7f4a500b0790 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a54e5d700 1 -- 172.21.15.92:0/754009407 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000bb60 con 0x7f4a500b0790 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 -- 172.21.15.92:0/754009407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 msgr2=0x7f4a500b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 --2- 172.21.15.92:0/754009407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a500b28a0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cb4f0 tx=0x7f4a40009250 comp rx=0 tx=0).stop 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 -- 172.21.15.92:0/754009407 shutdown_connections 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 --2- 172.21.15.92:0/754009407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a500b03c0 0x7f4a500cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 --2- 172.21.15.92:0/754009407 >> v2:172.21.15.136:3300/0 conn(0x7f4a501591e0 0x7f4a50159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 --2- 172.21.15.92:0/754009407 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a500b28a0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 -- 172.21.15.92:0/754009407 >> 172.21.15.92:0/754009407 conn(0x7f4a50063230 msgr2=0x7f4a50063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.927 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 -- 172.21.15.92:0/754009407 shutdown_connections 2023-10-06T03:13:44.928 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.922+0000 7f4a56e61700 1 -- 172.21.15.92:0/754009407 wait complete. 2023-10-06T03:13:44.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.923+0000 7f4a56e61700 1 Processor -- start 2023-10-06T03:13:44.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.923+0000 7f4a56e61700 1 -- start start 2023-10-06T03:13:44.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56e61700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a500b03c0 0x7f4a500c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56e61700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a501489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.931 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56e61700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a501591e0 0x7f4a5015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56e61700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a50148ef0 con 0x7f4a500b0790 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56e61700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5016a490 con 0x7f4a500b03c0 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56e61700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5016a770 con 0x7f4a501591e0 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a501489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a501489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40290/0 (socket says 172.21.15.92:40290) 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 -- 172.21.15.92:0/2916857733 learned_addr learned my addr 172.21.15.92:0/2916857733 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a55e5f700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a500b03c0 0x7f4a500c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.932 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a56660700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a501591e0 0x7f4a5015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 -- 172.21.15.92:0/2916857733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a501591e0 msgr2=0x7f4a5015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a501591e0 0x7f4a5015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 -- 172.21.15.92:0/2916857733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a500b03c0 msgr2=0x7f4a500c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a500b03c0 0x7f4a500c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.924+0000 7f4a5565e700 1 -- 172.21.15.92:0/2916857733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400094f0 con 0x7f4a500b0790 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.925+0000 7f4a5565e700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a501489b0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cfe40 tx=0x7f4a40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.925+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000b910 con 0x7f4a500b0790 2023-10-06T03:13:44.933 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.925+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4000ba70 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.925+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5016aa50 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.925+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40011630 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.925+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5016b010 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.926+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4a5004c3c0 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.926+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4a400119a0 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a45ffb700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a34067320 0x7f4a340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4a40085cb0 con 0x7f4a500b0790 2023-10-06T03:13:44.934 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a55e5f700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a34067320 0x7f4a340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a45ffb700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7f4a3406d790 0x7f4a3406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 --> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a34070270 con 0x7f4a3406d790 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4008a220 con 0x7f4a500b0790 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a56660700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7f4a3406d790 0x7f4a3406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a55e5f700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a34067320 0x7f4a340697d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009d10 tx=0x7f4a4c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.927+0000 7f4a56660700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7f4a3406d790 0x7f4a3406fba0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:44.935 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.928+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== osd.3 v2:172.21.15.92:6810/90687 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4a34070270 con 0x7f4a3406d790 2023-10-06T03:13:44.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.941+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 --> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4a500cad50 con 0x7f4a3406d790 2023-10-06T03:13:44.943 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a45ffb700 1 -- 172.21.15.92:0/2916857733 <== osd.3 v2:172.21.15.92:6810/90687 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4a500cad50 con 0x7f4a3406d790 2023-10-06T03:13:44.943 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:44.943 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:44.944 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7f4a3406d790 msgr2=0x7f4a3406fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6810/90687,v1:172.21.15.92:6813/90687] conn(0x7f4a3406d790 0x7f4a3406fba0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a34067320 msgr2=0x7f4a340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f4a34067320 0x7f4a340697d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009d10 tx=0x7f4a4c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 msgr2=0x7f4a501489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 --2- 172.21.15.92:0/2916857733 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4a500b0790 0x7f4a501489b0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cfe40 tx=0x7f4a40017000 comp rx=0 tx=0).stop 2023-10-06T03:13:44.945 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56660700 1 -- 172.21.15.92:0/2916857733 reap_dead start 2023-10-06T03:13:44.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 shutdown_connections 2023-10-06T03:13:44.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 >> 172.21.15.92:0/2916857733 conn(0x7f4a50063230 msgr2=0x7f4a500b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:44.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 shutdown_connections 2023-10-06T03:13:44.946 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:44.942+0000 7f4a56e61700 1 -- 172.21.15.92:0/2916857733 wait complete. 2023-10-06T03:13:44.955 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T03:13:45.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.070+0000 7f391d46c700 1 Processor -- start 2023-10-06T03:13:45.072 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.070+0000 7f391d46c700 1 -- start start 2023-10-06T03:13:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f391d46c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3918159890 0x7f391815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f391d46c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918069560 0x7f39180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f391d46c700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3918069ee0 con 0x7f3918159c60 2023-10-06T03:13:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f391d46c700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391815a160 con 0x7f3918159890 2023-10-06T03:13:45.075 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f391d46c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39180d13a0 con 0x7f3918069560 2023-10-06T03:13:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3918159890 0x7f391815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f390e7fc700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f3918159c60 0x7f391815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44070/0 (socket says 172.21.15.92:44070) 2023-10-06T03:13:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3918159890 0x7f391815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40298/0 (socket says 172.21.15.92:40298) 2023-10-06T03:13:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f390e7fc700 1 -- 172.21.15.92:0/408647 learned_addr learned my addr 172.21.15.92:0/408647 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f39177fe700 1 --2- 172.21.15.92:0/408647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918069560 0x7f39180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.076 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 -- 172.21.15.92:0/408647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918069560 msgr2=0x7f39180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 --2- 172.21.15.92:0/408647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918069560 0x7f39180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 -- 172.21.15.92:0/408647 >> v1:172.21.15.92:6789/0 conn(0x7f3918159c60 legacy=0x7f391815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 -- 172.21.15.92:0/408647 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391815e140 con 0x7f3918159890 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f39167fc700 1 -- 172.21.15.92:0/408647 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674706198 0 0) 0x7f3918069ee0 con 0x7f3918159c60 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f3916ffd700 1 --2- 172.21.15.92:0/408647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391815abd0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f39080098e0 tx=0x7f39080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f028b91d42836382 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.071+0000 7f39167fc700 1 -- 172.21.15.92:0/408647 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908009e80 con 0x7f3918159890 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f39167fc700 1 -- 172.21.15.92:0/408647 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390800b860 con 0x7f3918159890 2023-10-06T03:13:45.077 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f39167fc700 1 -- 172.21.15.92:0/408647 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800bbe0 con 0x7f3918159890 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 -- 172.21.15.92:0/408647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 msgr2=0x7f391815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 --2- 172.21.15.92:0/408647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391815abd0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f39080098e0 tx=0x7f39080091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 -- 172.21.15.92:0/408647 shutdown_connections 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 --2- 172.21.15.92:0/408647 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918069560 0x7f39180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 --2- 172.21.15.92:0/408647 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391815abd0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 -- 172.21.15.92:0/408647 >> 172.21.15.92:0/408647 conn(0x7f3918068e20 msgr2=0x7f39180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 -- 172.21.15.92:0/408647 shutdown_connections 2023-10-06T03:13:45.078 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.072+0000 7f391d46c700 1 -- 172.21.15.92:0/408647 wait complete. 2023-10-06T03:13:45.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.073+0000 7f391d46c700 1 Processor -- start 2023-10-06T03:13:45.079 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.073+0000 7f391d46c700 1 -- start start 2023-10-06T03:13:45.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f391d46c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3918069560 0x7f391814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f391d46c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391816d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f391d46c700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918159c60 0x7f39180cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.081 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f391d46c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391816dee0 con 0x7f3918159890 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f391d46c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39180b3730 con 0x7f3918069560 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f391d46c700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39180b3a10 con 0x7f3918159c60 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391816d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391816d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40314/0 (socket says 172.21.15.92:40314) 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 -- 172.21.15.92:0/2822796617 learned_addr learned my addr 172.21.15.92:0/2822796617 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 -- 172.21.15.92:0/2822796617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918159c60 msgr2=0x7f39180cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.082 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918159c60 0x7f39180cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 -- 172.21.15.92:0/2822796617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3918069560 msgr2=0x7f391814ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3918069560 0x7f391814ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.074+0000 7f390e7fc700 1 -- 172.21.15.92:0/2822796617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009420 con 0x7f3918159890 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.075+0000 7f390e7fc700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391816d9a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f390000c7e0 tx=0x7f3900007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.075+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900005b30 con 0x7f3918159890 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.075+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900005c90 con 0x7f3918159890 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.075+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39180cd1e0 con 0x7f3918159890 2023-10-06T03:13:45.083 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.075+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900007300 con 0x7f3918159890 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.075+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39180cfb90 con 0x7f3918159890 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.076+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391804c3c0 con 0x7f3918159890 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.076+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f390001b400 con 0x7f3918159890 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.076+0000 7f390f7fe700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38f8067320 0x7f38f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.076+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f3900084650 con 0x7f3918159890 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.076+0000 7f3916ffd700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38f8067320 0x7f38f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.084 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.077+0000 7f3916ffd700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38f8067320 0x7f38f80697d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f39080091e0 tx=0x7f3908017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.085 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.080+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3900051500 con 0x7f3918159890 2023-10-06T03:13:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.262+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f39180cd680 con 0x7f38f8067320 2023-10-06T03:13:45.264 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.262+0000 7f390f7fe700 1 -- 172.21.15.92:0/2822796617 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f39180cd680 con 0x7f38f8067320 2023-10-06T03:13:45.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38f8067320 msgr2=0x7f38f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38f8067320 0x7f38f80697d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f39080091e0 tx=0x7f3908017000 comp rx=0 tx=0).stop 2023-10-06T03:13:45.266 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 msgr2=0x7f391816d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391816d9a0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f390000c7e0 tx=0x7f3900007df0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 shutdown_connections 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f38f8067320 0x7f38f80697d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3918159c60 0x7f39180cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3918069560 0x7f391814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 --2- 172.21.15.92:0/2822796617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3918159890 0x7f391816d9a0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.267 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 >> 172.21.15.92:0/2822796617 conn(0x7f3918068e20 msgr2=0x7f39180b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 shutdown_connections 2023-10-06T03:13:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.264+0000 7f391d46c700 1 -- 172.21.15.92:0/2822796617 wait complete. 2023-10-06T03:13:45.268 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T03:13:45.278 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T03:13:45.278 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.395+0000 7ff904c0e700 1 Processor -- start 2023-10-06T03:13:45.397 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.395+0000 7ff904c0e700 1 -- start start 2023-10-06T03:13:45.399 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff904c0e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff900159ec0 0x7ff9000b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff904c0e700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff9000cf350 0x7ff9000cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff904c0e700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9000b5b00 con 0x7ff9000b53d0 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff904c0e700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9000b5850 con 0x7ff900159ec0 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff904c0e700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9000b5da0 con 0x7ff9000cf350 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff900159ec0 0x7ff9000b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff900159ec0 0x7ff9000b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40316/0 (socket says 172.21.15.92:40316) 2023-10-06T03:13:45.400 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1484843640 learned_addr learned my addr 172.21.15.92:0/1484843640 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fed9d700 1 --2- 172.21.15.92:0/1484843640 >> v2:172.21.15.136:3300/0 conn(0x7ff9000cf350 0x7ff9000cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1484843640 >> v2:172.21.15.136:3300/0 conn(0x7ff9000cf350 msgr2=0x7ff9000cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 --2- 172.21.15.92:0/1484843640 >> v2:172.21.15.136:3300/0 conn(0x7ff9000cf350 0x7ff9000cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1484843640 >> v1:172.21.15.136:6789/0 conn(0x7ff9000b53d0 legacy=0x7ff9000cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1484843640 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000b6080 con 0x7ff900159ec0 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.396+0000 7ff8fed9d700 1 --2- 172.21.15.92:0/1484843640 >> v2:172.21.15.136:3300/0 conn(0x7ff9000cf350 0x7ff9000cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:13:45.401 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff8fe59c700 1 --2- 172.21.15.92:0/1484843640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff900159ec0 0x7ff9000b4e90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ff9000cb4a0 tx=0x7ff8f0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd21026d8fccd395 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff8fdd9b700 1 -- 172.21.15.92:0/1484843640 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0005c10 con 0x7ff900159ec0 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff8fdd9b700 1 -- 172.21.15.92:0/1484843640 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0005d70 con 0x7ff900159ec0 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff8fdd9b700 1 -- 172.21.15.92:0/1484843640 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f000b840 con 0x7ff900159ec0 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 -- 172.21.15.92:0/1484843640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff900159ec0 msgr2=0x7ff9000b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 --2- 172.21.15.92:0/1484843640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff900159ec0 0x7ff9000b4e90 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7ff9000cb4a0 tx=0x7ff8f0009220 comp rx=0 tx=0).stop 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 -- 172.21.15.92:0/1484843640 shutdown_connections 2023-10-06T03:13:45.402 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 --2- 172.21.15.92:0/1484843640 >> v2:172.21.15.136:3300/0 conn(0x7ff9000cf350 0x7ff9000cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 --2- 172.21.15.92:0/1484843640 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff900159ec0 0x7ff9000b4e90 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 -- 172.21.15.92:0/1484843640 >> 172.21.15.92:0/1484843640 conn(0x7ff9000999f0 msgr2=0x7ff9000b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 -- 172.21.15.92:0/1484843640 shutdown_connections 2023-10-06T03:13:45.403 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.397+0000 7ff904c0e700 1 -- 172.21.15.92:0/1484843640 wait complete. 2023-10-06T03:13:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 Processor -- start 2023-10-06T03:13:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 -- start start 2023-10-06T03:13:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9000b53d0 0x7ff900145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9000cf350 0x7ff90009c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.407 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff900159ec0 0x7ff90009cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90009e220 con 0x7ff9000b53d0 2023-10-06T03:13:45.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff900145ab0 con 0x7ff900159ec0 2023-10-06T03:13:45.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff904c0e700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90009dd90 con 0x7ff9000cf350 2023-10-06T03:13:45.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff8fe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9000b53d0 0x7ff900145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff8fe59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9000b53d0 0x7ff900145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40322/0 (socket says 172.21.15.92:40322) 2023-10-06T03:13:45.408 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.399+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1502781967 learned_addr learned my addr 172.21.15.92:0/1502781967 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:45.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fed9d700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff900159ec0 0x7ff90009cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8f5d9b700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9000cf350 0x7ff90009c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9000cf350 msgr2=0x7ff90009c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.409 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fe59c700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9000cf350 0x7ff90009c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff900159ec0 msgr2=0x7ff90009cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fe59c700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff900159ec0 0x7ff90009cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fe59c700 1 -- 172.21.15.92:0/1502781967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0009480 con 0x7ff9000b53d0 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fed9d700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff900159ec0 0x7ff90009cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8fe59c700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9000b53d0 0x7ff900145570 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ff900089390 tx=0x7ff8f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0011b30 con 0x7ff9000b53d0 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0011cb0 con 0x7ff9000b53d0 2023-10-06T03:13:45.410 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90009d180 con 0x7ff9000b53d0 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f001d680 con 0x7ff9000b53d0 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.400+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90009d740 con 0x7ff9000b53d0 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.401+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff900063c90 con 0x7ff9000b53d0 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.401+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff8f001da30 con 0x7ff9000b53d0 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f6ffd700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e0067320 0x7ff8e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ff8f0086e40 con 0x7ff9000b53d0 2023-10-06T03:13:45.411 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f5d9b700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e0067320 0x7ff8e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f6ffd700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7ff8e006d7e0 0x7ff8e006fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 --> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e00702c0 con 0x7ff8e006d7e0 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8f00893a0 con 0x7ff9000b53d0 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8fed9d700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7ff8e006d7e0 0x7ff8e006fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.402+0000 7ff8f5d9b700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e0067320 0x7ff8e00697d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006fd0 tx=0x7ff8e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.403+0000 7ff8fed9d700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7ff8e006d7e0 0x7ff8e006fbf0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.412 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.403+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== osd.4 v2:172.21.15.136:6816/90892 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff8e00702c0 con 0x7ff8e006d7e0 2023-10-06T03:13:45.418 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 --> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9000718b0 con 0x7ff8e006d7e0 2023-10-06T03:13:45.419 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff8f6ffd700 1 -- 172.21.15.92:0/1502781967 <== osd.4 v2:172.21.15.136:6816/90892 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff9000718b0 con 0x7ff8e006d7e0 2023-10-06T03:13:45.419 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:45.419 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:45.421 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7ff8e006d7e0 msgr2=0x7ff8e006fbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.136:6816/90892,v1:172.21.15.136:6817/90892] conn(0x7ff8e006d7e0 0x7ff8e006fbf0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e0067320 msgr2=0x7ff8e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7ff8e0067320 0x7ff8e00697d0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006fd0 tx=0x7ff8e8008040 comp rx=0 tx=0).stop 2023-10-06T03:13:45.422 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9000b53d0 msgr2=0x7ff900145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 --2- 172.21.15.92:0/1502781967 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9000b53d0 0x7ff900145570 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7ff900089390 tx=0x7ff8f0015000 comp rx=0 tx=0).stop 2023-10-06T03:13:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff8fed9d700 1 -- 172.21.15.92:0/1502781967 reap_dead start 2023-10-06T03:13:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.417+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 shutdown_connections 2023-10-06T03:13:45.423 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.418+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 >> 172.21.15.92:0/1502781967 conn(0x7ff9000999f0 msgr2=0x7ff9000c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.418+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 shutdown_connections 2023-10-06T03:13:45.424 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.418+0000 7ff904c0e700 1 -- 172.21.15.92:0/1502781967 wait complete. 2023-10-06T03:13:45.431 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T03:13:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.544+0000 7f768c5bd700 1 Processor -- start 2023-10-06T03:13:45.546 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.544+0000 7f768c5bd700 1 -- start start 2023-10-06T03:13:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768c5bd700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7684159c60 0x7f768415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768c5bd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f7684069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.552 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768c5bd700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768415a160 con 0x7f7684159890 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768c5bd700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768415df50 con 0x7f7684159c60 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768c5bd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76840d13a0 con 0x7f768415db10 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768a359700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f7684159890 0x7f768415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44078/0 (socket says 172.21.15.92:44078) 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.545+0000 7f768a359700 1 -- 172.21.15.92:0/1904713297 learned_addr learned my addr 172.21.15.92:0/1904713297 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768ab5a700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f7684069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689b58700 1 --2- 172.21.15.92:0/1904713297 >> v2:172.21.15.136:3300/0 conn(0x7f7684159c60 0x7f768415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.553 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938996705 0 0) 0x7f768415a160 con 0x7f7684159890 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7670003570 con 0x7f7684159890 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 201088006 0 0) 0x7f7670003570 con 0x7f7684159890 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 msgr2=0x7f7684069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f7684069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 >> v2:172.21.15.136:3300/0 conn(0x7f7684159c60 msgr2=0x7f768415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 --2- 172.21.15.92:0/1904713297 >> v2:172.21.15.136:3300/0 conn(0x7f7684159c60 0x7f768415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768415e140 con 0x7f7684159890 2023-10-06T03:13:45.554 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f76800017f0 con 0x7f7684159890 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689b58700 1 --2- 172.21.15.92:0/1904713297 >> v2:172.21.15.136:3300/0 conn(0x7f7684159c60 0x7f768415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768ab5a700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f7684069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 >> v1:172.21.15.92:6789/0 conn(0x7f7684159890 legacy=0x7f768415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7670003a60 0x7f7670003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7670004810 0x7f7670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f76700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.555 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76840d13a0 con 0x7f7670003a60 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767000ba60 con 0x7f7670004810 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767000bba0 con 0x7f768415db10 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768a359700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7670003a60 0x7f7670003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689b58700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7670004810 0x7f7670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768ab5a700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f76700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768a359700 1 -- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 msgr2=0x7f76700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.556 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768a359700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f768415db10 0x7f76700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768a359700 1 -- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7670004810 msgr2=0x7f7670004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768a359700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7670004810 0x7f7670004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f768a359700 1 -- 172.21.15.92:0/1904713297 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c009420 con 0x7f7670003a60 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.546+0000 7f7689b58700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7670004810 0x7f7670004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768a359700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7670003a60 0x7f7670003e70 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f768000a970 tx=0x7f7680002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d20b04d85b115a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800070d0 con 0x7f7670003a60 2023-10-06T03:13:45.557 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7680007270 con 0x7f7670003a60 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f7689357700 1 -- 172.21.15.92:0/1904713297 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680007630 con 0x7f7670003a60 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768c5bd700 1 -- 172.21.15.92:0/1904713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7670003a60 msgr2=0x7f7670003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1904713297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7670003a60 0x7f7670003e70 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f768000a970 tx=0x7f7680002210 comp rx=0 tx=0).stop 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768ab5a700 1 -- 172.21.15.92:0/1904713297 reap_dead start 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768c5bd700 1 -- 172.21.15.92:0/1904713297 shutdown_connections 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768c5bd700 1 -- 172.21.15.92:0/1904713297 >> 172.21.15.92:0/1904713297 conn(0x7f7684068e20 msgr2=0x7f76840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.558 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768c5bd700 1 -- 172.21.15.92:0/1904713297 shutdown_connections 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.547+0000 7f768c5bd700 1 -- 172.21.15.92:0/1904713297 wait complete. 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.548+0000 7f768c5bd700 1 Processor -- start 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.548+0000 7f768c5bd700 1 -- start start 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768c5bd700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7670003a60 0x7f768415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768c5bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 0x7f7684153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768c5bd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7684157510 0x7f7684153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768c5bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684157080 con 0x7f768414d3f0 2023-10-06T03:13:45.559 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768c5bd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768415d8b0 con 0x7f7684157510 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768c5bd700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7684156470 con 0x7f7670003a60 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 0x7f7684153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 0x7f7684153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40340/0 (socket says 172.21.15.92:40340) 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 -- 172.21.15.92:0/1960214042 learned_addr learned my addr 172.21.15.92:0/1960214042 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768a359700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7670003a60 0x7f768415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f768ab5a700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7684157510 0x7f7684153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.560 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 -- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7670003a60 msgr2=0x7f768415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7670003a60 0x7f768415d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 -- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7684157510 msgr2=0x7f7684153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7684157510 0x7f7684153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.549+0000 7f7689b58700 1 -- 172.21.15.92:0/1960214042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7684156750 con 0x7f768414d3f0 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.550+0000 7f7689b58700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 0x7f7684153860 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f7674009fd0 tx=0x7f7674008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.550+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674011ec0 con 0x7f768414d3f0 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.550+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767400f860 con 0x7f768414d3f0 2023-10-06T03:13:45.561 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.550+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7684156a30 con 0x7f768414d3f0 2023-10-06T03:13:45.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.550+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400fbe0 con 0x7f768414d3f0 2023-10-06T03:13:45.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.550+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7684155ed0 con 0x7f768414d3f0 2023-10-06T03:13:45.562 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.551+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7684156ed0 con 0x7f768414d3f0 2023-10-06T03:13:45.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.551+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7674015460 con 0x7f768414d3f0 2023-10-06T03:13:45.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.551+0000 7f767a7fc700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7668067320 0x7f76680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.552+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f76740852a0 con 0x7f768414d3f0 2023-10-06T03:13:45.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.552+0000 7f768a359700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7668067320 0x7f76680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.552+0000 7f768a359700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7668067320 0x7f76680697d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f768000a970 tx=0x7f7680015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.563 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.556+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7674052090 con 0x7f768414d3f0 2023-10-06T03:13:45.739 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.738+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f768404c3c0 con 0x7f7668067320 2023-10-06T03:13:45.740 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.738+0000 7f767a7fc700 1 -- 172.21.15.92:0/1960214042 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f768404c3c0 con 0x7f7668067320 2023-10-06T03:13:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7668067320 msgr2=0x7f76680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7668067320 0x7f76680697d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f768000a970 tx=0x7f7680015000 comp rx=0 tx=0).stop 2023-10-06T03:13:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 msgr2=0x7f7684153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 0x7f7684153860 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f7674009fd0 tx=0x7f7674008040 comp rx=0 tx=0).stop 2023-10-06T03:13:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 shutdown_connections 2023-10-06T03:13:45.742 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f7668067320 0x7f76680697d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7684157510 0x7f7684153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7670003a60 0x7f768415d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 --2- 172.21.15.92:0/1960214042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f768414d3f0 0x7f7684153860 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 >> 172.21.15.92:0/1960214042 conn(0x7f7684068e20 msgr2=0x7f76840cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 shutdown_connections 2023-10-06T03:13:45.743 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.740+0000 7f768c5bd700 1 -- 172.21.15.92:0/1960214042 wait complete. 2023-10-06T03:13:45.744 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T03:13:45.753 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T03:13:45.754 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:45.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.871+0000 7f8f88458700 1 Processor -- start 2023-10-06T03:13:45.873 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.871+0000 7f8f88458700 1 -- start start 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f88458700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f80159ec0 0x7f8f800b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f88458700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800b5b00 con 0x7f8f800cf350 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f88458700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800b5da0 con 0x7f8f800b53d0 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f88458700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800b6080 con 0x7f8f80159ec0 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f869f5700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8f800cf350 0x7f8f800cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44084/0 (socket says 172.21.15.92:44084) 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f869f5700 1 -- 172.21.15.92:0/1083110558 learned_addr learned my addr 172.21.15.92:0/1083110558 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:45.876 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.872+0000 7f8f861f4700 1 --2- 172.21.15.92:0/1083110558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f80159ec0 0x7f8f800b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849246510 0 0) 0x7f8f800b5b00 con 0x7f8f800cf350 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f6c003570 con 0x7f8f800cf350 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601750637 0 0) 0x7f8f800b5da0 con 0x7f8f800b53d0 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f800b5b00 con 0x7f8f800b53d0 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f861f4700 1 -- 172.21.15.92:0/1083110558 >> v1:172.21.15.136:6789/0 conn(0x7f8f800b53d0 legacy=0x7f8f800cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f861f4700 1 -- 172.21.15.92:0/1083110558 >> v1:172.21.15.92:6789/0 conn(0x7f8f800cf350 legacy=0x7f8f800cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f861f4700 1 -- 172.21.15.92:0/1083110558 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800cc100 con 0x7f8f80159ec0 2023-10-06T03:13:45.877 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f861f4700 1 --2- 172.21.15.92:0/1083110558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f80159ec0 0x7f8f800b4e90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009900 tx=0x7f8f7c009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef070047ad51d60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c005c10 con 0x7f8f80159ec0 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c005d70 con 0x7f8f80159ec0 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.873+0000 7f8f851f2700 1 -- 172.21.15.92:0/1083110558 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c006260 con 0x7f8f80159ec0 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 -- 172.21.15.92:0/1083110558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f80159ec0 msgr2=0x7f8f800b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 --2- 172.21.15.92:0/1083110558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f80159ec0 0x7f8f800b4e90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009900 tx=0x7f8f7c009120 comp rx=0 tx=0).stop 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 -- 172.21.15.92:0/1083110558 shutdown_connections 2023-10-06T03:13:45.878 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 --2- 172.21.15.92:0/1083110558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f80159ec0 0x7f8f800b4e90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 -- 172.21.15.92:0/1083110558 >> 172.21.15.92:0/1083110558 conn(0x7f8f800999f0 msgr2=0x7f8f800b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 -- 172.21.15.92:0/1083110558 shutdown_connections 2023-10-06T03:13:45.879 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.874+0000 7f8f88458700 1 -- 172.21.15.92:0/1083110558 wait complete. 2023-10-06T03:13:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.875+0000 7f8f88458700 1 Processor -- start 2023-10-06T03:13:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.875+0000 7f8f88458700 1 -- start start 2023-10-06T03:13:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800b53d0 0x7f8f80142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f800cf350 0x7f8f80068d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f80159ec0 0x7f8f80152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.882 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80069790 con 0x7f8f800b53d0 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80069a30 con 0x7f8f80159ec0 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f80069d10 con 0x7f8f800cf350 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800b53d0 0x7f8f80142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800b53d0 0x7f8f80142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40354/0 (socket says 172.21.15.92:40354) 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 -- 172.21.15.92:0/1372717899 learned_addr learned my addr 172.21.15.92:0/1372717899 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f869f5700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f80159ec0 0x7f8f80152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.883 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f859f3700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f800cf350 0x7f8f80068d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 -- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f800cf350 msgr2=0x7f8f80068d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f800cf350 0x7f8f80068d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 -- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f80159ec0 msgr2=0x7f8f80152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f80159ec0 0x7f8f80152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 -- 172.21.15.92:0/1372717899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0093c0 con 0x7f8f800b53d0 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f869f5700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f80159ec0 0x7f8f80152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:45.884 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f861f4700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800b53d0 0x7f8f80142210 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009160 tx=0x7f8f7c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c01d910 con 0x7f8f800b53d0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c01da90 con 0x7f8f800b53d0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f80069ff0 con 0x7f8f800b53d0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c01de50 con 0x7f8f800b53d0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.876+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f80142750 con 0x7f8f800b53d0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.877+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f80063c90 con 0x7f8f800b53d0 2023-10-06T03:13:45.885 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8f7c01b690 con 0x7f8f800b53d0 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f767fc700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8f64067320 0x7f8f640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f8f7c086f10 con 0x7f8f800b53d0 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f859f3700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8f64067320 0x7f8f640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f767fc700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f8f6406d790 0x7f8f6406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 --> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f64070270 con 0x7f8f6406d790 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.878+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f7c087220 con 0x7f8f800b53d0 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.879+0000 7f8f869f5700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f8f6406d790 0x7f8f6406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:45.886 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.879+0000 7f8f859f3700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8f64067320 0x7f8f640697d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8f70007fd0 tx=0x7f8f70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.879+0000 7f8f869f5700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f8f6406d790 0x7f8f6406fba0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:45.887 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.880+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== osd.5 v2:172.21.15.136:6800/90874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f8f64070270 con 0x7f8f6406d790 2023-10-06T03:13:45.894 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.893+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 --> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8f800718b0 con 0x7f8f6406d790 2023-10-06T03:13:45.895 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f767fc700 1 -- 172.21.15.92:0/1372717899 <== osd.5 v2:172.21.15.136:6800/90874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8f800718b0 con 0x7f8f6406d790 2023-10-06T03:13:45.895 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:45.896 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:45.896 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f8f6406d790 msgr2=0x7f8f6406fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.136:6800/90874,v1:172.21.15.136:6801/90874] conn(0x7f8f6406d790 0x7f8f6406fba0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8f64067320 msgr2=0x7f8f640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f8f64067320 0x7f8f640697d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8f70007fd0 tx=0x7f8f70009040 comp rx=0 tx=0).stop 2023-10-06T03:13:45.897 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800b53d0 msgr2=0x7f8f80142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 --2- 172.21.15.92:0/1372717899 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f800b53d0 0x7f8f80142210 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009160 tx=0x7f8f7c016000 comp rx=0 tx=0).stop 2023-10-06T03:13:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f869f5700 1 -- 172.21.15.92:0/1372717899 reap_dead start 2023-10-06T03:13:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 shutdown_connections 2023-10-06T03:13:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 >> 172.21.15.92:0/1372717899 conn(0x7f8f800999f0 msgr2=0x7f8f800cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 shutdown_connections 2023-10-06T03:13:45.898 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:45.894+0000 7f8f88458700 1 -- 172.21.15.92:0/1372717899 wait complete. 2023-10-06T03:13:45.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T03:13:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.021+0000 7f9e29415700 1 Processor -- start 2023-10-06T03:13:46.023 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e29415700 1 -- start start 2023-10-06T03:13:46.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e29415700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e2415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e29415700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e24159c60 0x7f9e2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e29415700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2415a160 con 0x7f9e2415db10 2023-10-06T03:13:46.026 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e29415700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240d13a0 con 0x7f9e24159c60 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e29415700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2415e140 con 0x7f9e24159890 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e1a7fc700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e24159c60 0x7f9e2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.022+0000 7f9e237fe700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9e2415db10 0x7f9e24069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44094/0 (socket says 172.21.15.92:44094) 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e237fe700 1 -- 172.21.15.92:0/1106111892 learned_addr learned my addr 172.21.15.92:0/1106111892 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e22ffd700 1 --2- 172.21.15.92:0/1106111892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e2415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e1a7fc700 1 -- 172.21.15.92:0/1106111892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 msgr2=0x7f9e2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.027 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e1a7fc700 1 --2- 172.21.15.92:0/1106111892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e2415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e1a7fc700 1 -- 172.21.15.92:0/1106111892 >> v1:172.21.15.92:6789/0 conn(0x7f9e2415db10 legacy=0x7f9e24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e1a7fc700 1 -- 172.21.15.92:0/1106111892 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2415a3d0 con 0x7f9e24159c60 2023-10-06T03:13:46.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e227fc700 1 -- 172.21.15.92:0/1106111892 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444016724 0 0) 0x7f9e2415a160 con 0x7f9e2415db10 2023-10-06T03:13:46.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e1a7fc700 1 --2- 172.21.15.92:0/1106111892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24159c60 0x7f9e2415d5d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0098e0 tx=0x7f9e0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4ce9dfe7edb6f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e227fc700 1 -- 172.21.15.92:0/1106111892 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00b640 con 0x7f9e24159c60 2023-10-06T03:13:46.028 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e227fc700 1 -- 172.21.15.92:0/1106111892 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c00b7a0 con 0x7f9e24159c60 2023-10-06T03:13:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.023+0000 7f9e227fc700 1 -- 172.21.15.92:0/1106111892 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00bb40 con 0x7f9e24159c60 2023-10-06T03:13:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 -- 172.21.15.92:0/1106111892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24159c60 msgr2=0x7f9e2415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 --2- 172.21.15.92:0/1106111892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24159c60 0x7f9e2415d5d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0098e0 tx=0x7f9e0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 -- 172.21.15.92:0/1106111892 shutdown_connections 2023-10-06T03:13:46.029 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 --2- 172.21.15.92:0/1106111892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e2415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 --2- 172.21.15.92:0/1106111892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e24159c60 0x7f9e2415d5d0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 -- 172.21.15.92:0/1106111892 >> 172.21.15.92:0/1106111892 conn(0x7f9e24068e20 msgr2=0x7f9e240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 -- 172.21.15.92:0/1106111892 shutdown_connections 2023-10-06T03:13:46.030 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.024+0000 7f9e29415700 1 -- 172.21.15.92:0/1106111892 wait complete. 2023-10-06T03:13:46.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.025+0000 7f9e29415700 1 Processor -- start 2023-10-06T03:13:46.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.025+0000 7f9e29415700 1 -- start start 2023-10-06T03:13:46.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e240c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e24159c60 0x7f9e24149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.033 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 0x7f9e24148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240c6160 con 0x7f9e2415db10 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240c6440 con 0x7f9e24159c60 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240c6720 con 0x7f9e24159890 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 0x7f9e24148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 0x7f9e24148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40368/0 (socket says 172.21.15.92:40368) 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 -- 172.21.15.92:0/2109611873 learned_addr learned my addr 172.21.15.92:0/2109611873 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e1a7fc700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e24159c60 0x7f9e24149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.034 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e22ffd700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e240c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 -- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 msgr2=0x7f9e240c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e240c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 -- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e24159c60 msgr2=0x7f9e24149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e24159c60 0x7f9e24149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 -- 172.21.15.92:0/2109611873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c009420 con 0x7f9e2415db10 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e237fe700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 0x7f9e24148fe0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00cbf0 tx=0x7f9e1c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.035 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00a8b0 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00aa10 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00add0 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.026+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e240c6a00 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.027+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e240c5d70 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.028+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e240c6f20 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.028+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9e1c007300 con 0x7f9e2415db10 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.028+0000 7f9e1b7fe700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e04067320 0x7f9e040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.036 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.028+0000 7f9e22ffd700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e04067320 0x7f9e040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.028+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9e1c084d40 con 0x7f9e2415db10 2023-10-06T03:13:46.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.029+0000 7f9e22ffd700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e04067320 0x7f9e040697d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009db0 tx=0x7f9e140091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.037 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.033+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e240c6f20 con 0x7f9e2415db10 2023-10-06T03:13:46.217 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.216+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e240c6f20 con 0x7f9e04067320 2023-10-06T03:13:46.218 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.216+0000 7f9e1b7fe700 1 -- 172.21.15.92:0/2109611873 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9e240c6f20 con 0x7f9e04067320 2023-10-06T03:13:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e04067320 msgr2=0x7f9e040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e04067320 0x7f9e040697d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009db0 tx=0x7f9e140091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.220 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 msgr2=0x7f9e24148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 0x7f9e24148fe0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00cbf0 tx=0x7f9e1c0019b0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 shutdown_connections 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e04067320 0x7f9e040697d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e2415db10 0x7f9e24148fe0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e24159890 0x7f9e240c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 --2- 172.21.15.92:0/2109611873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e24159c60 0x7f9e24149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 >> 172.21.15.92:0/2109611873 conn(0x7f9e24068e20 msgr2=0x7f9e2415fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.221 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 shutdown_connections 2023-10-06T03:13:46.222 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.218+0000 7f9e29415700 1 -- 172.21.15.92:0/2109611873 wait complete. 2023-10-06T03:13:46.222 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T03:13:46.232 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T03:13:46.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:46.355 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.350+0000 7f2f951ce700 1 Processor -- start 2023-10-06T03:13:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.350+0000 7f2f951ce700 1 -- start start 2023-10-06T03:13:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f951ce700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2f90069a50 0x7f2f90169730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f951ce700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f900af2f0 0x7f2f90169c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f951ce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f90069e20 0x7f2f9006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.356 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f951ce700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f9016a1b0 con 0x7f2f900af2f0 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f951ce700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f900c7a50 con 0x7f2f90069a50 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f951ce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f900c7190 con 0x7f2f90069e20 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f8e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f900af2f0 0x7f2f90169c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f8e59c700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f2f900af2f0 0x7f2f90169c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40378/0 (socket says 172.21.15.92:40378) 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.351+0000 7f2f8e59c700 1 -- 172.21.15.92:0/1407436801 learned_addr learned my addr 172.21.15.92:0/1407436801 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.352+0000 7f2f8ed9d700 1 --2- 172.21.15.92:0/1407436801 >> v2:172.21.15.136:3300/0 conn(0x7f2f90069a50 0x7f2f90169730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.357 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.352+0000 7f2f8f59e700 1 --2- 172.21.15.92:0/1407436801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f90069e20 0x7f2f9006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8e59c700 1 -- 172.21.15.92:0/1407436801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f90069e20 msgr2=0x7f2f9006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8e59c700 1 --2- 172.21.15.92:0/1407436801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f90069e20 0x7f2f9006d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8e59c700 1 -- 172.21.15.92:0/1407436801 >> v2:172.21.15.136:3300/0 conn(0x7f2f90069a50 msgr2=0x7f2f90169730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8e59c700 1 --2- 172.21.15.92:0/1407436801 >> v2:172.21.15.136:3300/0 conn(0x7f2f90069a50 0x7f2f90169730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8e59c700 1 -- 172.21.15.92:0/1407436801 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9006e480 con 0x7f2f900af2f0 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8e59c700 1 --2- 172.21.15.92:0/1407436801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f900af2f0 0x7f2f90169c70 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f2f9006c580 tx=0x7f2f8400a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad34e712238bc00f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.358 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8dd9b700 1 -- 172.21.15.92:0/1407436801 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400ae60 con 0x7f2f900af2f0 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8dd9b700 1 -- 172.21.15.92:0/1407436801 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400b750 con 0x7f2f900af2f0 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.353+0000 7f2f8dd9b700 1 -- 172.21.15.92:0/1407436801 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400baf0 con 0x7f2f900af2f0 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 -- 172.21.15.92:0/1407436801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f900af2f0 msgr2=0x7f2f90169c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 --2- 172.21.15.92:0/1407436801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f900af2f0 0x7f2f90169c70 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f2f9006c580 tx=0x7f2f8400a330 comp rx=0 tx=0).stop 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 -- 172.21.15.92:0/1407436801 shutdown_connections 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 --2- 172.21.15.92:0/1407436801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f90069e20 0x7f2f9006d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.359 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 --2- 172.21.15.92:0/1407436801 >> v2:172.21.15.136:3300/0 conn(0x7f2f90069a50 0x7f2f90169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 --2- 172.21.15.92:0/1407436801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f900af2f0 0x7f2f90169c70 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 -- 172.21.15.92:0/1407436801 >> 172.21.15.92:0/1407436801 conn(0x7f2f900cc520 msgr2=0x7f2f900cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 -- 172.21.15.92:0/1407436801 shutdown_connections 2023-10-06T03:13:46.360 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.354+0000 7f2f951ce700 1 -- 172.21.15.92:0/1407436801 wait complete. 2023-10-06T03:13:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.355+0000 7f2f951ce700 1 Processor -- start 2023-10-06T03:13:46.363 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.355+0000 7f2f951ce700 1 -- start start 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f90069a50 0x7f2f900c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f90069e20 0x7f2f900c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f900af2f0 0x7f2f90167d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f900c1910 con 0x7f2f90069e20 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f900ae2b0 con 0x7f2f90069a50 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f900adfd0 con 0x7f2f900af2f0 2023-10-06T03:13:46.364 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f90069e20 0x7f2f900c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f90069e20 0x7f2f900c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40384/0 (socket says 172.21.15.92:40384) 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 -- 172.21.15.92:0/2527788665 learned_addr learned my addr 172.21.15.92:0/2527788665 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8f59e700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f900af2f0 0x7f2f90167d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8ed9d700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f90069a50 0x7f2f900c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 -- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f900af2f0 msgr2=0x7f2f90167d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f900af2f0 0x7f2f90167d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.365 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 -- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f90069a50 msgr2=0x7f2f900c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f90069a50 0x7f2f900c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 -- 172.21.15.92:0/2527788665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8400a590 con 0x7f2f90069e20 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8f59e700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f900af2f0 0x7f2f90167d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8ed9d700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f90069a50 0x7f2f900c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f8e59c700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f90069e20 0x7f2f900c3530 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f2f9006c2e0 tx=0x7f2f84019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8401f070 con 0x7f2f90069e20 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400bad0 con 0x7f2f90069e20 2023-10-06T03:13:46.366 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840196a0 con 0x7f2f90069e20 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.356+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f900c1b40 con 0x7f2f90069e20 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.357+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f900c2100 con 0x7f2f90069e20 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.357+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f900718b0 con 0x7f2f90069e20 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f84019a10 con 0x7f2f90069e20 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f76ffd700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f7c067320 0x7f2f7c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2f84085a10 con 0x7f2f90069e20 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f8ed9d700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f7c067320 0x7f2f7c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.367 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f76ffd700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7f2f7c06d790 0x7f2f7c06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 --> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f7c070270 con 0x7f2f7c06d790 2023-10-06T03:13:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.358+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f8408a220 con 0x7f2f90069e20 2023-10-06T03:13:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.359+0000 7f2f8f59e700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7f2f7c06d790 0x7f2f7c06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.359+0000 7f2f8ed9d700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f7c067320 0x7f2f7c0697d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c570 tx=0x7f2f80010040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.359+0000 7f2f8f59e700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7f2f7c06d790 0x7f2f7c06fba0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.368 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.360+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== osd.6 v2:172.21.15.136:6808/90875 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2f7c070270 con 0x7f2f7c06d790 2023-10-06T03:13:46.374 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.373+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 --> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2f900c1910 con 0x7f2f7c06d790 2023-10-06T03:13:46.375 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f76ffd700 1 -- 172.21.15.92:0/2527788665 <== osd.6 v2:172.21.15.136:6808/90875 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2f900c1910 con 0x7f2f7c06d790 2023-10-06T03:13:46.375 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:46.376 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:46.376 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7f2f7c06d790 msgr2=0x7f2f7c06fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.136:6808/90875,v1:172.21.15.136:6809/90875] conn(0x7f2f7c06d790 0x7f2f7c06fba0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f7c067320 msgr2=0x7f2f7c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2f7c067320 0x7f2f7c0697d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c570 tx=0x7f2f80010040 comp rx=0 tx=0).stop 2023-10-06T03:13:46.377 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f90069e20 msgr2=0x7f2f900c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 --2- 172.21.15.92:0/2527788665 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f90069e20 0x7f2f900c3530 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f2f9006c2e0 tx=0x7f2f84019000 comp rx=0 tx=0).stop 2023-10-06T03:13:46.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f8f59e700 1 -- 172.21.15.92:0/2527788665 reap_dead start 2023-10-06T03:13:46.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 shutdown_connections 2023-10-06T03:13:46.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 >> 172.21.15.92:0/2527788665 conn(0x7f2f900cc520 msgr2=0x7f2f90169500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 shutdown_connections 2023-10-06T03:13:46.378 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.374+0000 7f2f951ce700 1 -- 172.21.15.92:0/2527788665 wait complete. 2023-10-06T03:13:46.387 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T03:13:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.500+0000 7f9e80328700 1 Processor -- start 2023-10-06T03:13:46.502 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.500+0000 7f9e80328700 1 -- start start 2023-10-06T03:13:46.504 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e80328700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e78140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e80328700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e7806c0a0 0x7f9e78141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e80328700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9e780c8e00 0x7f9e780c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e80328700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7815c2a0 con 0x7f9e7806c0a0 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e80328700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7815b690 con 0x7f9e780c8e00 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e80328700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7815b970 con 0x7f9e7806bcd0 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e7d8c3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e7806c0a0 0x7f9e78141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e7d8c3700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9e7806c0a0 0x7f9e78141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40398/0 (socket says 172.21.15.92:40398) 2023-10-06T03:13:46.505 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e7d8c3700 1 -- 172.21.15.92:0/2543618015 learned_addr learned my addr 172.21.15.92:0/2543618015 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e7e0c4700 1 --2- 172.21.15.92:0/2543618015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e78140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.501+0000 7f9e7e8c5700 1 --2- 172.21.15.92:0/2543618015 >> v2:172.21.15.136:3300/0 conn(0x7f9e780c8e00 0x7f9e780c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d8c3700 1 -- 172.21.15.92:0/2543618015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 msgr2=0x7f9e78140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d8c3700 1 --2- 172.21.15.92:0/2543618015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e78140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d8c3700 1 -- 172.21.15.92:0/2543618015 >> v2:172.21.15.136:3300/0 conn(0x7f9e780c8e00 msgr2=0x7f9e780c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d8c3700 1 --2- 172.21.15.92:0/2543618015 >> v2:172.21.15.136:3300/0 conn(0x7f9e780c8e00 0x7f9e780c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.506 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d8c3700 1 -- 172.21.15.92:0/2543618015 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7815bc50 con 0x7f9e7806c0a0 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d8c3700 1 --2- 172.21.15.92:0/2543618015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e7806c0a0 0x7f9e78141330 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f9e7815b530 tx=0x7f9e68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abe47b779b4f4cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d0c2700 1 -- 172.21.15.92:0/2543618015 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800b640 con 0x7f9e7806c0a0 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d0c2700 1 -- 172.21.15.92:0/2543618015 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6800b7a0 con 0x7f9e7806c0a0 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e7d0c2700 1 -- 172.21.15.92:0/2543618015 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6800bb60 con 0x7f9e7806c0a0 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 -- 172.21.15.92:0/2543618015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e7806c0a0 msgr2=0x7f9e78141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 --2- 172.21.15.92:0/2543618015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e7806c0a0 0x7f9e78141330 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f9e7815b530 tx=0x7f9e68009250 comp rx=0 tx=0).stop 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 -- 172.21.15.92:0/2543618015 shutdown_connections 2023-10-06T03:13:46.507 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 --2- 172.21.15.92:0/2543618015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e78140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 --2- 172.21.15.92:0/2543618015 >> v2:172.21.15.136:3300/0 conn(0x7f9e780c8e00 0x7f9e780c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 --2- 172.21.15.92:0/2543618015 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e7806c0a0 0x7f9e78141330 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.502+0000 7f9e80328700 1 -- 172.21.15.92:0/2543618015 >> 172.21.15.92:0/2543618015 conn(0x7f9e780ce600 msgr2=0x7f9e780cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.503+0000 7f9e80328700 1 -- 172.21.15.92:0/2543618015 shutdown_connections 2023-10-06T03:13:46.508 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.503+0000 7f9e80328700 1 -- 172.21.15.92:0/2543618015 wait complete. 2023-10-06T03:13:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 Processor -- start 2023-10-06T03:13:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 -- start start 2023-10-06T03:13:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e780c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e7806c0a0 0x7f9e780c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 0x7f9e7813ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.511 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e780c6080 con 0x7f9e780c8e00 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78167ef0 con 0x7f9e7806c0a0 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e80328700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e780c62b0 con 0x7f9e7806bcd0 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e7e8c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 0x7f9e7813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e7e8c5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 0x7f9e7813ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40408/0 (socket says 172.21.15.92:40408) 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e7e8c5700 1 -- 172.21.15.92:0/3217681904 learned_addr learned my addr 172.21.15.92:0/3217681904 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.504+0000 7f9e7e0c4700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e780c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7d8c3700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e7806c0a0 0x7f9e780c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.512 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7e8c5700 1 -- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 msgr2=0x7f9e780c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7e8c5700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e780c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7e8c5700 1 -- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e7806c0a0 msgr2=0x7f9e780c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7e8c5700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e7806c0a0 0x7f9e780c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7e8c5700 1 -- 172.21.15.92:0/3217681904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e680094f0 con 0x7f9e780c8e00 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7d8c3700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e7806c0a0 0x7f9e780c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e7e8c5700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 0x7f9e7813ef10 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e8d0 tx=0x7f9e700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70013680 con 0x7f9e780c8e00 2023-10-06T03:13:46.513 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700137e0 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e780c6590 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70013ba0 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.505+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e781681b0 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.506+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7804c3c0 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.506+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9e700123d0 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.507+0000 7f9e6e7fc700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e5c067320 0x7f9e5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.507+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9e70084c20 con 0x7f9e780c8e00 2023-10-06T03:13:46.514 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.507+0000 7f9e7e0c4700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e5c067320 0x7f9e5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.507+0000 7f9e7e0c4700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e5c067320 0x7f9e5c0697d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400f570 tx=0x7f9e7400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.515 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.511+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e70051a10 con 0x7f9e780c8e00 2023-10-06T03:13:46.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.692+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e781683a0 con 0x7f9e5c067320 2023-10-06T03:13:46.694 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.693+0000 7f9e6e7fc700 1 -- 172.21.15.92:0/3217681904 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9e781683a0 con 0x7f9e5c067320 2023-10-06T03:13:46.696 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e5c067320 msgr2=0x7f9e5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e5c067320 0x7f9e5c0697d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400f570 tx=0x7f9e7400b040 comp rx=0 tx=0).stop 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 msgr2=0x7f9e7813ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 0x7f9e7813ef10 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e8d0 tx=0x7f9e700091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 shutdown_connections 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f9e5c067320 0x7f9e5c0697d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9e780c8e00 0x7f9e7813ef10 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e7806bcd0 0x7f9e780c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.697 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 --2- 172.21.15.92:0/3217681904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e7806c0a0 0x7f9e780c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 >> 172.21.15.92:0/3217681904 conn(0x7f9e780ce600 msgr2=0x7f9e780cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 shutdown_connections 2023-10-06T03:13:46.698 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.695+0000 7f9e80328700 1 -- 172.21.15.92:0/3217681904 wait complete. 2023-10-06T03:13:46.698 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T03:13:46.708 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T03:13:46.708 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T03:13:46.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.826+0000 7f2db4d41700 1 Processor -- start 2023-10-06T03:13:46.827 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.826+0000 7f2db4d41700 1 -- start start 2023-10-06T03:13:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2db4d41700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0153f50 0x7f2db00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.831 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2db4d41700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db00b4570 con 0x7f2db0154320 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2db4d41700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db015a730 con 0x7f2db006d840 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2db4d41700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015aa10 con 0x7f2db0153f50 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dadd9b700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f2db0154320 0x7f2db006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44108/0 (socket says 172.21.15.92:44108) 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dadd9b700 1 -- 172.21.15.92:0/2394788362 learned_addr learned my addr 172.21.15.92:0/2394788362 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dae59c700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0153f50 0x7f2db00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498246835 0 0) 0x7f2db00b4570 con 0x7f2db0154320 2023-10-06T03:13:46.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d98003570 con 0x7f2db0154320 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340410446 0 0) 0x7f2db015a730 con 0x7f2db006d840 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db00b4570 con 0x7f2db006d840 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1517453694 0 0) 0x7f2d98003570 con 0x7f2db0154320 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0153f50 msgr2=0x7f2db00b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0153f50 0x7f2db00b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 >> v1:172.21.15.136:6789/0 conn(0x7f2db006d840 legacy=0x7f2db0069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db015acf0 con 0x7f2db0154320 2023-10-06T03:13:46.833 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f2da4001470 con 0x7f2db0154320 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 >> v1:172.21.15.92:6789/0 conn(0x7f2db0154320 legacy=0x7f2db006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d98003c20 0x7f2d98004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2d98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d980051c0 0x7f2d9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db015aa10 con 0x7f2d98004810 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9800c0b0 con 0x7f2d98003c20 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9800c270 con 0x7f2d980051c0 2023-10-06T03:13:46.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dae59c700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0153f50 0x7f2db00b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.827+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2d98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2daed9d700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d980051c0 0x7f2d9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dae59c700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d98003c20 0x7f2d98004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dadd9b700 1 -- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d980051c0 msgr2=0x7f2d9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d980051c0 0x7f2d9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dadd9b700 1 -- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d98003c20 msgr2=0x7f2d98004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d98003c20 0x7f2d98004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dadd9b700 1 -- 172.21.15.92:0/2394788362 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d98003570 con 0x7f2d98004810 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2daed9d700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d980051c0 0x7f2d9800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2d98004c80 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2da4007fd0 tx=0x7f2da4001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50406dedaae1a811 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da400cae0 con 0x7f2d98004810 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da400cc60 con 0x7f2d98004810 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.828+0000 7f2dad59a700 1 -- 172.21.15.92:0/2394788362 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4005920 con 0x7f2d98004810 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2db4d41700 1 -- 172.21.15.92:0/2394788362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 msgr2=0x7f2d98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2db4d41700 1 --2- 172.21.15.92:0/2394788362 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2d98004c80 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2da4007fd0 tx=0x7f2da4001d40 comp rx=0 tx=0).stop 2023-10-06T03:13:46.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2daed9d700 1 -- 172.21.15.92:0/2394788362 reap_dead start 2023-10-06T03:13:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2db4d41700 1 -- 172.21.15.92:0/2394788362 shutdown_connections 2023-10-06T03:13:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2db4d41700 1 -- 172.21.15.92:0/2394788362 >> 172.21.15.92:0/2394788362 conn(0x7f2db006baf0 msgr2=0x7f2db006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2db4d41700 1 -- 172.21.15.92:0/2394788362 shutdown_connections 2023-10-06T03:13:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.829+0000 7f2db4d41700 1 -- 172.21.15.92:0/2394788362 wait complete. 2023-10-06T03:13:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.830+0000 7f2db4d41700 1 Processor -- start 2023-10-06T03:13:46.837 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.830+0000 7f2db4d41700 1 -- start start 2023-10-06T03:13:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.830+0000 7f2db4d41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2db015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.830+0000 7f2db4d41700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0157600 0x7f2db0157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2db4d41700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2db00ce8f0 0x7f2db00ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.840 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2db4d41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db016b500 con 0x7f2d98004810 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2db4d41700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00cf290 con 0x7f2db00ce8f0 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2db4d41700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00cdb30 con 0x7f2db0157600 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2db015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2db015c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40436/0 (socket says 172.21.15.92:40436) 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 -- 172.21.15.92:0/1173100152 learned_addr learned my addr 172.21.15.92:0/1173100152 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2daed9d700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2db00ce8f0 0x7f2db00ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0157600 0x7f2db0157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.841 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 -- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0157600 msgr2=0x7f2db0157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2db0157600 0x7f2db0157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 -- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2db00ce8f0 msgr2=0x7f2db00ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2db00ce8f0 0x7f2db00ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 -- 172.21.15.92:0/1173100152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00cde10 con 0x7f2d98004810 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2dae59c700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2db015c9f0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2da0011910 tx=0x7f2da0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0011e50 con 0x7f2d98004810 2023-10-06T03:13:46.842 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da000f8e0 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da000fca0 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db00ce0f0 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.831+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db015cf30 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.832+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2db00718b0 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.832+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2da0015670 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2d9e7fc700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d90067320 0x7f2d900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2da0085040 con 0x7f2d98004810 2023-10-06T03:13:46.843 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d90067320 0x7f2d900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2d9e7fc700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f2d9006d790 0x7f2d9006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 --> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d90070270 con 0x7f2d9006d790 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2da0087540 con 0x7f2d98004810 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2daed9d700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f2d9006d790 0x7f2d9006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.833+0000 7f2dadd9b700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d90067320 0x7f2d900697d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2da4007ee0 tx=0x7f2da4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.834+0000 7f2daed9d700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f2d9006d790 0x7f2d9006fba0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.844 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.834+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== osd.7 v2:172.21.15.136:6824/90905 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2d90070270 con 0x7f2d9006d790 2023-10-06T03:13:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 --> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2db015afd0 con 0x7f2d9006d790 2023-10-06T03:13:46.849 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2d9e7fc700 1 -- 172.21.15.92:0/1173100152 <== osd.7 v2:172.21.15.136:6824/90905 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2db015afd0 con 0x7f2d9006d790 2023-10-06T03:13:46.850 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2023-10-06T03:13:46.850 INFO:teuthology.orchestra.run.smithi092.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T03:13:46.850 INFO:teuthology.orchestra.run.smithi092.stdout:} 2023-10-06T03:13:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f2d9006d790 msgr2=0x7f2d9006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.136:6824/90905,v1:172.21.15.136:6825/90905] conn(0x7f2d9006d790 0x7f2d9006fba0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d90067320 msgr2=0x7f2d900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.851 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f2d90067320 0x7f2d900697d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2da4007ee0 tx=0x7f2da4015000 comp rx=0 tx=0).stop 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 msgr2=0x7f2db015c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 --2- 172.21.15.92:0/1173100152 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d98004810 0x7f2db015c9f0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2da0011910 tx=0x7f2da0008040 comp rx=0 tx=0).stop 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2daed9d700 1 -- 172.21.15.92:0/1173100152 reap_dead start 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 shutdown_connections 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.848+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 >> 172.21.15.92:0/1173100152 conn(0x7f2db006baf0 msgr2=0x7f2db00c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.849+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 shutdown_connections 2023-10-06T03:13:46.852 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.849+0000 7f2db4d41700 1 -- 172.21.15.92:0/1173100152 wait complete. 2023-10-06T03:13:46.862 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T03:13:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.975+0000 7fa732a79700 1 Processor -- start 2023-10-06T03:13:46.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.975+0000 7fa732a79700 1 -- start start 2023-10-06T03:13:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa732a79700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c159890 0x7fa72c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa732a79700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa72c069560 0x7fa72c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa732a79700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c15a160 con 0x7fa72c159c60 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa732a79700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c069f20 con 0x7fa72c069560 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa732a79700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0d13a0 con 0x7fa72c159890 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa731016700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa72c069560 0x7fa72c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa723fff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa72c159c60 0x7fa72c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:44112/0 (socket says 172.21.15.92:44112) 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa723fff700 1 -- 172.21.15.92:0/1560264055 learned_addr learned my addr 172.21.15.92:0/1560264055 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa731016700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa72c069560 0x7fa72c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40450/0 (socket says 172.21.15.92:40450) 2023-10-06T03:13:46.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa730815700 1 --2- 172.21.15.92:0/1560264055 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c159890 0x7fa72c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa731016700 1 -- 172.21.15.92:0/1560264055 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c159890 msgr2=0x7fa72c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.976+0000 7fa731016700 1 --2- 172.21.15.92:0/1560264055 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c159890 0x7fa72c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa731016700 1 -- 172.21.15.92:0/1560264055 >> v1:172.21.15.92:6789/0 conn(0x7fa72c159c60 legacy=0x7fa72c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa731016700 1 -- 172.21.15.92:0/1560264055 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c15e140 con 0x7fa72c069560 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa7237fe700 1 -- 172.21.15.92:0/1560264055 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492892349 0 0) 0x7fa72c15a160 con 0x7fa72c159c60 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa731016700 1 --2- 172.21.15.92:0/1560264055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c069560 0x7fa72c0699e0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa7240098e0 tx=0x7fa7240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea07c63e20b4ca1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa7237fe700 1 -- 172.21.15.92:0/1560264055 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724009e80 con 0x7fa72c069560 2023-10-06T03:13:46.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa7237fe700 1 -- 172.21.15.92:0/1560264055 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72400b860 con 0x7fa72c069560 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa7237fe700 1 -- 172.21.15.92:0/1560264055 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400bbe0 con 0x7fa72c069560 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 -- 172.21.15.92:0/1560264055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c069560 msgr2=0x7fa72c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 --2- 172.21.15.92:0/1560264055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c069560 0x7fa72c0699e0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fa7240098e0 tx=0x7fa7240091a0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 -- 172.21.15.92:0/1560264055 shutdown_connections 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 --2- 172.21.15.92:0/1560264055 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c159890 0x7fa72c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 --2- 172.21.15.92:0/1560264055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c069560 0x7fa72c0699e0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 -- 172.21.15.92:0/1560264055 >> 172.21.15.92:0/1560264055 conn(0x7fa72c068e20 msgr2=0x7fa72c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:46.982 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 -- 172.21.15.92:0/1560264055 shutdown_connections 2023-10-06T03:13:46.983 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.977+0000 7fa732a79700 1 -- 172.21.15.92:0/1560264055 wait complete. 2023-10-06T03:13:46.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.978+0000 7fa732a79700 1 Processor -- start 2023-10-06T03:13:46.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 -- start start 2023-10-06T03:13:46.985 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c069560 0x7fa72c15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa72c159890 0x7fa72c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 0x7fa72c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0cd940 con 0x7fa72c159c60 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0cdc00 con 0x7fa72c159890 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa732a79700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c15c8c0 con 0x7fa72c069560 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.979+0000 7fa731016700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 0x7fa72c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.986 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 0x7fa72c0c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40452/0 (socket says 172.21.15.92:40452) 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 -- 172.21.15.92:0/3607061644 learned_addr learned my addr 172.21.15.92:0/3607061644 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa730815700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c069560 0x7fa72c15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa723fff700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa72c159890 0x7fa72c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 -- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c069560 msgr2=0x7fa72c15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c069560 0x7fa72c15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 -- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa72c159890 msgr2=0x7fa72c0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:46.987 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa72c159890 0x7fa72c0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 -- 172.21.15.92:0/3607061644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724009420 con 0x7fa72c159c60 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa731016700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 0x7fa72c0c6c50 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0899c0 tx=0x7fa724017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400baa0 con 0x7fa72c159c60 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724025420 con 0x7fa72c159c60 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c15cba0 con 0x7fa72c159c60 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7240257e0 con 0x7fa72c159c60 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.980+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c15d160 con 0x7fa72c159c60 2023-10-06T03:13:46.988 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.981+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c04c3c0 con 0x7fa72c159c60 2023-10-06T03:13:46.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.981+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa724023070 con 0x7fa72c159c60 2023-10-06T03:13:46.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.982+0000 7fa720ff9700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa718067320 0x7fa7180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:46.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.982+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fa724086240 con 0x7fa72c159c60 2023-10-06T03:13:46.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.982+0000 7fa730815700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa718067320 0x7fa7180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:46.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.982+0000 7fa730815700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa718067320 0x7fa7180697d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa728009fd0 tx=0x7fa728009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:46.989 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:46.986+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa724053070 con 0x7fa72c159c60 2023-10-06T03:13:47.167 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.166+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa72c0cdd40 con 0x7fa718067320 2023-10-06T03:13:47.168 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.166+0000 7fa720ff9700 1 -- 172.21.15.92:0/3607061644 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa72c0cdd40 con 0x7fa718067320 2023-10-06T03:13:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa718067320 msgr2=0x7fa7180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa718067320 0x7fa7180697d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa728009fd0 tx=0x7fa728009880 comp rx=0 tx=0).stop 2023-10-06T03:13:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 msgr2=0x7fa72c0c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.170 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 0x7fa72c0c6c50 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fa72c0899c0 tx=0x7fa724017000 comp rx=0 tx=0).stop 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 shutdown_connections 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fa718067320 0x7fa7180697d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa72c159c60 0x7fa72c0c6c50 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa72c069560 0x7fa72c15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 --2- 172.21.15.92:0/3607061644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa72c159890 0x7fa72c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 >> 172.21.15.92:0/3607061644 conn(0x7fa72c068e20 msgr2=0x7fa72c157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 shutdown_connections 2023-10-06T03:13:47.171 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.168+0000 7fa732a79700 1 -- 172.21.15.92:0/3607061644 wait complete. 2023-10-06T03:13:47.172 INFO:teuthology.orchestra.run.smithi092.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T03:13:47.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:13:47.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.294+0000 7fb64bbe5700 1 Processor -- start 2023-10-06T03:13:47.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.294+0000 7fb64bbe5700 1 -- start start 2023-10-06T03:13:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb64bbe5700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb644159c60 0x7fb64415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:47.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb64bbe5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb64415db10 0x7fb644069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb64bbe5700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64415a160 con 0x7fb644159890 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb64bbe5700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64415df50 con 0x7fb644159c60 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb64bbe5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440d13a0 con 0x7fb64415db10 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb649180700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb644159c60 0x7fb64415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb649180700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb644159c60 0x7fb64415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40456/0 (socket says 172.21.15.92:40456) 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb649180700 1 -- 172.21.15.92:0/397763048 learned_addr learned my addr 172.21.15.92:0/397763048 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.295+0000 7fb64a182700 1 --2- 172.21.15.92:0/397763048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb64415db10 0x7fb644069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:47.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb649180700 1 -- 172.21.15.92:0/397763048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb64415db10 msgr2=0x7fb644069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb649180700 1 --2- 172.21.15.92:0/397763048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb64415db10 0x7fb644069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb649180700 1 -- 172.21.15.92:0/397763048 >> v1:172.21.15.136:6789/0 conn(0x7fb644159890 legacy=0x7fb64415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb649180700 1 -- 172.21.15.92:0/397763048 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64415e140 con 0x7fb644159c60 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb649180700 1 --2- 172.21.15.92:0/397763048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb644159c60 0x7fb64415d5d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb6340099b0 tx=0x7fb634009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6c991349b998f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64897f700 1 -- 172.21.15.92:0/397763048 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400b820 con 0x7fb644159c60 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64897f700 1 -- 172.21.15.92:0/397763048 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63400b980 con 0x7fb644159c60 2023-10-06T03:13:47.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64897f700 1 -- 172.21.15.92:0/397763048 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400bd20 con 0x7fb644159c60 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 -- 172.21.15.92:0/397763048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb644159c60 msgr2=0x7fb64415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/397763048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb644159c60 0x7fb64415d5d0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb6340099b0 tx=0x7fb634009250 comp rx=0 tx=0).stop 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 -- 172.21.15.92:0/397763048 shutdown_connections 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/397763048 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb64415db10 0x7fb644069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/397763048 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb644159c60 0x7fb64415d5d0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 -- 172.21.15.92:0/397763048 >> 172.21.15.92:0/397763048 conn(0x7fb644068e20 msgr2=0x7fb6440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.296+0000 7fb64bbe5700 1 -- 172.21.15.92:0/397763048 shutdown_connections 2023-10-06T03:13:47.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.297+0000 7fb64bbe5700 1 -- 172.21.15.92:0/397763048 wait complete. 2023-10-06T03:13:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 Processor -- start 2023-10-06T03:13:47.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 -- start start 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb644159890 0x7fb644147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb644159c60 0x7fb6440b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 0x7fb6440c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440b64c0 con 0x7fb64415db10 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440cdcd0 con 0x7fb644159890 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64bbe5700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6440cd3a0 con 0x7fb644159c60 2023-10-06T03:13:47.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64a182700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 0x7fb6440c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64a182700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 0x7fb6440c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40460/0 (socket says 172.21.15.92:40460) 2023-10-06T03:13:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb64a182700 1 -- 172.21.15.92:0/66573371 learned_addr learned my addr 172.21.15.92:0/66573371 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:13:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.298+0000 7fb649180700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb644159c60 0x7fb6440b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb649981700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb644159890 0x7fb644147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64a182700 1 -- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb644159c60 msgr2=0x7fb6440b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64a182700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb644159c60 0x7fb6440b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64a182700 1 -- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb644159890 msgr2=0x7fb644147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64a182700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb644159890 0x7fb644147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64a182700 1 -- 172.21.15.92:0/66573371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340094f0 con 0x7fb64415db10 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb649981700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb644159890 0x7fb644147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64a182700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 0x7fb6440c67b0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c880 tx=0x7fb63c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c011600 con 0x7fb64415db10 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c011760 con 0x7fb64415db10 2023-10-06T03:13:47.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6440cd680 con 0x7fb64415db10 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c011b00 con 0x7fb64415db10 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.299+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb644147750 con 0x7fb64415db10 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.300+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6440cf1e0 con 0x7fb64415db10 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.300+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb63c021350 con 0x7fb64415db10 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.301+0000 7fb639ffb700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb628067320 0x7fb6280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.301+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb63c0848f0 con 0x7fb64415db10 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.301+0000 7fb649981700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb628067320 0x7fb6280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:13:47.308 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.301+0000 7fb649981700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb628067320 0x7fb6280697d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb640007fd0 tx=0x7fb640009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:13:47.309 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.305+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb63c051610 con 0x7fb64415db10 2023-10-06T03:13:47.485 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.482+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb64404c3c0 con 0x7fb628067320 2023-10-06T03:13:47.486 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.485+0000 7fb639ffb700 1 -- 172.21.15.92:0/66573371 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72898 (secure 0 0 0) 0x7fb64404c3c0 con 0x7fb628067320 2023-10-06T03:13:47.487 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:13:47.488 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb628067320 msgr2=0x7fb6280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb628067320 0x7fb6280697d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb640007fd0 tx=0x7fb640009040 comp rx=0 tx=0).stop 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 msgr2=0x7fb6440c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 0x7fb6440c67b0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c880 tx=0x7fb63c008040 comp rx=0 tx=0).stop 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 shutdown_connections 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7fb628067320 0x7fb6280697d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64415db10 0x7fb6440c67b0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb644159890 0x7fb644147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.489 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 --2- 172.21.15.92:0/66573371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb644159c60 0x7fb6440b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:13:47.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 >> 172.21.15.92:0/66573371 conn(0x7fb644068e20 msgr2=0x7fb644157270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:13:47.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 shutdown_connections 2023-10-06T03:13:47.490 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:13:47.487+0000 7fb64bbe5700 1 -- 172.21.15.92:0/66573371 wait complete. 2023-10-06T03:13:47.490 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:13:47.500 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":568,"stamp":"2023-10-06T03:13:47.336770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2432744,"kb_used_data":21672,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752541976,"statfs":{"total":773094113280,"available":770602983424,"internally_reserved":0,"allocated":22192128,"data_stored":11168288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.546267"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.302359+0000","last_change":"2023-10-06T03:01:37.252387+0000","last_active":"2023-10-06T03:13:38.302359+0000","last_peered":"2023-10-06T03:13:38.302359+0000","last_clean":"2023-10-06T03:13:38.302359+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:13:38.302359+0000","last_undegraded":"2023-10-06T03:13:38.302359+0000","last_fullsized":"2023-10-06T03:13:38.302359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:10.408485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792855+0000","last_change":"2023-10-06T03:01:37.252269+0000","last_active":"2023-10-06T03:13:37.792855+0000","last_peered":"2023-10-06T03:13:37.792855+0000","last_clean":"2023-10-06T03:13:37.792855+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:13:37.792855+0000","last_undegraded":"2023-10-06T03:13:37.792855+0000","last_fullsized":"2023-10-06T03:13:37.792855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301764+0000","last_change":"2023-10-06T03:01:38.143171+0000","last_active":"2023-10-06T03:13:38.301764+0000","last_peered":"2023-10-06T03:13:38.301764+0000","last_clean":"2023-10-06T03:13:38.301764+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:13:38.301764+0000","last_undegraded":"2023-10-06T03:13:38.301764+0000","last_fullsized":"2023-10-06T03:13:38.301764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:15.092565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301153+0000","last_change":"2023-10-06T03:01:37.253832+0000","last_active":"2023-10-06T03:13:38.301153+0000","last_peered":"2023-10-06T03:13:38.301153+0000","last_clean":"2023-10-06T03:13:38.301153+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:13:38.301153+0000","last_undegraded":"2023-10-06T03:13:38.301153+0000","last_fullsized":"2023-10-06T03:13:38.301153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":320,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.301253+0000","last_change":"2023-10-06T03:01:30.689208+0000","last_active":"2023-10-06T03:13:38.301253+0000","last_peered":"2023-10-06T03:13:38.301253+0000","last_clean":"2023-10-06T03:13:38.301253+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:13:38.301253+0000","last_undegraded":"2023-10-06T03:13:38.301253+0000","last_fullsized":"2023-10-06T03:13:38.301253+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:29.651552+0000","last_clean_scrub_stamp":"2023-10-06T03:01:29.651552+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:13:49.016871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791888+0000","last_change":"2023-10-06T03:01:37.253374+0000","last_active":"2023-10-06T03:13:37.791888+0000","last_peered":"2023-10-06T03:13:37.791888+0000","last_clean":"2023-10-06T03:13:37.791888+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:13:37.791888+0000","last_undegraded":"2023-10-06T03:13:37.791888+0000","last_fullsized":"2023-10-06T03:13:37.791888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:38.300512+0000","last_change":"2023-10-06T03:01:37.381537+0000","last_active":"2023-10-06T03:13:38.300512+0000","last_peered":"2023-10-06T03:13:38.300512+0000","last_clean":"2023-10-06T03:13:38.300512+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:13:38.300512+0000","last_undegraded":"2023-10-06T03:13:38.300512+0000","last_fullsized":"2023-10-06T03:13:38.300512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:28.355923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":188,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.791600+0000","last_change":"2023-10-06T03:01:37.253577+0000","last_active":"2023-10-06T03:13:37.791600+0000","last_peered":"2023-10-06T03:13:37.791600+0000","last_clean":"2023-10-06T03:13:37.791600+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:13:37.791600+0000","last_undegraded":"2023-10-06T03:13:37.791600+0000","last_fullsized":"2023-10-06T03:13:37.791600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:19.389366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-06T03:13:37.792425+0000","last_change":"2023-10-06T03:01:38.142988+0000","last_active":"2023-10-06T03:13:37.792425+0000","last_peered":"2023-10-06T03:13:37.792425+0000","last_clean":"2023-10-06T03:13:37.792425+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:13:37.792425+0000","last_undegraded":"2023-10-06T03:13:37.792425+0000","last_fullsized":"2023-10-06T03:13:37.792425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:01:36.233628+0000","last_clean_scrub_stamp":"2023-10-06T03:01:36.233628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:37:16.946421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.60799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":6.0110000000000001},"last":0.438},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.35599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.435,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.86399999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85299999999999998,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52800000000000002,"15min":0.54200000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.70099999999999996,"5min":1.264,"15min":1.264},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56200000000000006,"15min":0.53900000000000003},"min":{"1min":0.40899999999999997,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.79700000000000004,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.752,"5min":2.286,"15min":2.286},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39500000000000002},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.377}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.52800000000000002},"min":{"1min":0.27900000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.77300000000000002,"5min":4.0289999999999999,"15min":4.0289999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78900000000000003,"5min":2.504,"15min":2.504},"last":0.36699999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.40799999999999997,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.83399999999999996,"5min":1.141,"15min":1.141},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.56999999999999995},"min":{"1min":0.33100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.84799999999999998,"5min":1.639,"15min":1.639},"last":0.40799999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.59599999999999997},"min":{"1min":0.432,"5min":0.35299999999999998,"15min":0.32300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.36899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82799999999999996,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":0.314}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57799999999999996,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.30099999999999999,"15min":0.28999999999999998},"max":{"1min":0.79800000000000004,"5min":0.91400000000000003,"15min":1.6200000000000001},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.57899999999999996},"min":{"1min":0.48199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.80700000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":0.754,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.55400000000000005,"15min":0.53400000000000003},"min":{"1min":0.46500000000000002,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.73199999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.74199999999999999,"5min":0.75,"15min":1.6359999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55300000000000005,"15min":0.53200000000000003},"min":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.55700000000000005},"min":{"1min":0.41999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.74099999999999999,"5min":0.77200000000000002,"15min":0.85099999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.438,"5min":0.375,"15min":0.375},"max":{"1min":0.77500000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.373}]},{"osd":4,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.5,"15min":0.52200000000000002},"min":{"1min":0.35599999999999998,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.57999999999999996,"5min":0.81200000000000006,"15min":0.88700000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67300000000000004,"5min":0.873,"15min":0.873},"last":0.40400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52400000000000002,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.313,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":0.76900000000000002,"15min":0.85299999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.379,"5min":0.377,"15min":0.32100000000000001},"max":{"1min":0.76800000000000002,"5min":0.85199999999999998,"15min":0.85699999999999998},"last":0.41499999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.39800000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.435},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.26000000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.54600000000000004,"5min":0.55600000000000005,"15min":3.1139999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.39100000000000001,"15min":0.38700000000000001},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.24399999999999999},"max":{"1min":0.71799999999999997,"5min":0.98899999999999999,"15min":3.6520000000000001},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.33800000000000002},"min":{"1min":0.28499999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47899999999999998,"5min":0.57999999999999996,"15min":1.282},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.34699999999999998,"15min":0.33800000000000002},"min":{"1min":0.24299999999999999,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.26500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.36599999999999999,"15min":0.34699999999999998},"min":{"1min":0.28000000000000003,"5min":0.218,"15min":0.20699999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":1.25},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.39400000000000002,"15min":0.375},"min":{"1min":0.30099999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.30399999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.56599999999999995},"min":{"1min":0.437,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.82499999999999996,"5min":1.1060000000000001,"15min":1.302},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78100000000000003,"5min":1.127,"15min":1.3240000000000001},"last":0.38400000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78800000000000003,"5min":0.81599999999999995,"15min":0.96299999999999997},"last":0.377},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":5.0270000000000001},"last":0.441}]},{"osd":6,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.499,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.3360000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.59299999999999997},"min":{"1min":0.44600000000000001,"5min":0.39900000000000002,"15min":0.35199999999999998},"max":{"1min":0.91400000000000003,"5min":1.054,"15min":1.3520000000000001},"last":0.41599999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.59799999999999998,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.625,"15min":0.60399999999999998},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.087,"15min":1.379},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33400000000000002,"15min":0.317},"min":{"1min":0.20399999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.33200000000000002},"min":{"1min":0.27500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.373}]},{"osd":2,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.34499999999999997},"min":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":0.53000000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.371,"15min":0.35599999999999998},"min":{"1min":0.29299999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55400000000000005,"5min":0.627,"15min":0.627},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.374,"15min":0.34899999999999998},"min":{"1min":0.28699999999999998,"5min":0.255,"15min":0.20899999999999999},"max":{"1min":0.59899999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.40600000000000003,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.251},"max":{"1min":0.60999999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42799999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.35499999999999998},"max":{"1min":0.76000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.90200000000000002,"15min":0.99199999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.92100000000000004},"last":0.753},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77400000000000002,"5min":0.94499999999999995,"15min":0.94799999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.51800000000000002,"5min":0.40500000000000003,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.84899999999999998,"15min":0.93700000000000006},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":1.0089999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82499999999999996,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.65400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.32600000000000001,"15min":0.31900000000000001},"min":{"1min":0.23999999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.33000000000000002},"min":{"1min":0.28999999999999998,"5min":0.23300000000000001,"15min":0.20499999999999999},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.26200000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.35499999999999998,"15min":0.33800000000000002},"min":{"1min":0.311,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.25700000000000001,"15min":0.24399999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.33600000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"min":{"1min":0.30499999999999999,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.38900000000000001,"15min":0.379},"min":{"1min":0.36499999999999999,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.34699999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.42199999999999999,"5min":0.38500000000000001,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.435,"15min":0.374},"max":{"1min":0.90000000000000002,"5min":1.333,"15min":1.333},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.55300000000000005},"min":{"1min":0.436,"5min":0.40300000000000002,"15min":0.35799999999999998},"max":{"1min":0.78900000000000003,"5min":0.999,"15min":0.999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.34499999999999997},"max":{"1min":0.82899999999999996,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61399999999999999,"15min":0.57699999999999996},"min":{"1min":0.48299999999999998,"5min":0.42199999999999999,"15min":0.34399999999999997},"max":{"1min":0.85599999999999998,"5min":1.216,"15min":1.216},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.47099999999999997,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.874,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.62}]},{"osd":7,"last update":"Fri Oct 6 03:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.878,"5min":1.2789999999999999,"15min":1.903},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.376},"max":{"1min":0.82499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.34000000000000002,"15min":0.31900000000000001},"min":{"1min":0.20899999999999999,"5min":0.20100000000000001,"15min":0.19600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.65200000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.32500000000000001,"5min":0.26700000000000002,"15min":0.216},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.72299999999999998},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.34200000000000003,"15min":0.31900000000000001},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.222},"max":{"1min":0.47999999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.34300000000000003,"15min":0.33400000000000002},"min":{"1min":0.23999999999999999,"5min":0.218,"15min":0.216},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.69699999999999995},"last":0.374}]},{"osd":2,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.38400000000000001,"15min":0.35999999999999999},"min":{"1min":0.35399999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.56000000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.379,"15min":0.36299999999999999},"min":{"1min":0.27100000000000002,"5min":0.24399999999999999,"15min":0.23999999999999999},"max":{"1min":0.57599999999999996,"5min":0.60499999999999998,"15min":0.67600000000000005},"last":0.33300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.32200000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.73799999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.46400000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.379,"15min":0.34000000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.997},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57699999999999996,"15min":0.56299999999999994},"min":{"1min":0.42699999999999999,"5min":0.38100000000000001,"15min":0.315},"max":{"1min":0.78300000000000003,"5min":0.86099999999999999,"15min":0.86499999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.44600000000000001,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.98499999999999999},"last":0.47399999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.441,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":1.0660000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.57899999999999996},"min":{"1min":0.48799999999999999,"5min":0.40999999999999998,"15min":0.35599999999999998},"max":{"1min":0.78400000000000003,"5min":1.446,"15min":1.446},"last":0.499}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54500000000000004,"15min":0.52100000000000002},"min":{"1min":0.374,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.36899999999999999,"15min":0.307},"max":{"1min":0.872,"5min":1.069,"15min":1.4410000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":0.74399999999999999,"5min":0.76600000000000001,"15min":1.0669999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.44800000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":1.0009999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.56100000000000005},"min":{"1min":0.41699999999999998,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":0.81899999999999995,"5min":0.92600000000000005,"15min":3},"last":0.622},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.56100000000000005,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.34699999999999998,"15min":0.34399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":10.084},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.90500000000000003,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.63},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.89100000000000001,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.503},"min":{"1min":0.32800000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71499999999999997,"5min":0.749,"15min":0.83999999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.76200000000000001,"5min":0.97599999999999998,"15min":1.54},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.38200000000000001,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.70299999999999996,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.45900000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.377,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":1.167,"15min":1.167},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.35699999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38300000000000001,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.69499999999999995,"5min":0.79100000000000004,"15min":2.3599999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57099999999999995,"15min":0.58599999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.28100000000000003},"max":{"1min":0.72799999999999998,"5min":0.83499999999999996,"15min":2.371},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.70199999999999996,"5min":0.82399999999999995,"15min":2.3410000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.83999999999999997,"15min":4.1219999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.92400000000000004,"15min":6.532},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.53900000000000003,"15min":0.64300000000000002},"min":{"1min":0.441,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":5.6500000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.54800000000000004,"15min":0.627},"min":{"1min":0.47999999999999998,"5min":0.34799999999999998,"15min":0.315},"max":{"1min":0.70199999999999996,"5min":0.85299999999999998,"15min":5.4630000000000001},"last":0.435},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70899999999999996,"5min":0.86099999999999999,"15min":5.524},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48099999999999998,"15min":0.50700000000000001},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.27100000000000002},"max":{"1min":0.746,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.76400000000000001,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49199999999999999,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":1.1990000000000001},"last":0.624},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.44800000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77900000000000003,"5min":0.95399999999999996,"15min":1.052},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.59299999999999997},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.175},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54700000000000004,"15min":0.58399999999999996},"min":{"1min":0.40999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70499999999999996,"5min":0.93899999999999995,"15min":1.145},"last":0.64900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:13:47.504 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.505 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.505 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.505 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.505 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T03:01:29.651552+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=29, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.505 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.506 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.506 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.506 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T03:01:36.233628+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=1, tm_sec=36, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=13, tm_sec=42, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T03:13:47.506 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T03:14:07.507 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:14:07.637 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.637+0000 7f80cb991700 1 Processor -- start 2023-10-06T03:14:07.638 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.637+0000 7f80cb991700 1 -- start start 2023-10-06T03:14:07.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80cb991700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40cbf30 0x7f80c40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.640 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80cb991700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80c40b1420 0x7f80c40c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80cb991700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c40cc300 con 0x7f80c40ccf90 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80cb991700 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c4156bf0 con 0x7f80c40b1420 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80cb991700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40ae950 con 0x7f80c40cbf30 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c9f2e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80c40b1420 0x7f80c40c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c8f2c700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f80c40ccf90 0x7f80c40b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48592/0 (socket says 172.21.15.92:48592) 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c9f2e700 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f80c40b1420 0x7f80c40c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55280/0 (socket says 172.21.15.92:55280) 2023-10-06T03:14:07.641 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c8f2c700 1 -- 172.21.15.92:0/947009337 learned_addr learned my addr 172.21.15.92:0/947009337 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c972d700 1 --2- 172.21.15.92:0/947009337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40cbf30 0x7f80c40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c9f2e700 1 -- 172.21.15.92:0/947009337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40cbf30 msgr2=0x7f80c40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c9f2e700 1 --2- 172.21.15.92:0/947009337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40cbf30 0x7f80c40ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c9f2e700 1 -- 172.21.15.92:0/947009337 >> v1:172.21.15.92:6789/0 conn(0x7f80c40ccf90 legacy=0x7f80c40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80c9f2e700 1 -- 172.21.15.92:0/947009337 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40ad370 con 0x7f80c40b1420 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.638+0000 7f80bbfff700 1 -- 172.21.15.92:0/947009337 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549718972 0 0) 0x7f80c40cc300 con 0x7f80c40ccf90 2023-10-06T03:14:07.642 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80c9f2e700 1 --2- 172.21.15.92:0/947009337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40b1420 0x7f80c40c9740 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009e90 tx=0x7f80bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94d012a37de852c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80bbfff700 1 -- 172.21.15.92:0/947009337 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00e070 con 0x7f80c40b1420 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80bbfff700 1 -- 172.21.15.92:0/947009337 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc00bbf0 con 0x7f80c40b1420 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80bbfff700 1 -- 172.21.15.92:0/947009337 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc012430 con 0x7f80c40b1420 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 -- 172.21.15.92:0/947009337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40b1420 msgr2=0x7f80c40c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 --2- 172.21.15.92:0/947009337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40b1420 0x7f80c40c9740 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009e90 tx=0x7f80bc009780 comp rx=0 tx=0).stop 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 -- 172.21.15.92:0/947009337 shutdown_connections 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 --2- 172.21.15.92:0/947009337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40cbf30 0x7f80c40ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.643 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 --2- 172.21.15.92:0/947009337 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40b1420 0x7f80c40c9740 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 -- 172.21.15.92:0/947009337 >> 172.21.15.92:0/947009337 conn(0x7f80c406a320 msgr2=0x7f80c40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:14:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 -- 172.21.15.92:0/947009337 shutdown_connections 2023-10-06T03:14:07.644 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.639+0000 7f80cb991700 1 -- 172.21.15.92:0/947009337 wait complete. 2023-10-06T03:14:07.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.640+0000 7f80cb991700 1 Processor -- start 2023-10-06T03:14:07.646 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 -- start start 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40b1420 0x7f80c416b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 0x7f80c4168760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c40ccf90 0x7f80c4154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4168ca0 con 0x7f80c40cbf30 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41549d0 con 0x7f80c40ccf90 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80cb991700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c416b8a0 con 0x7f80c40b1420 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.641+0000 7f80c8f2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 0x7f80c4168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.647 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 0x7f80c4168760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55290/0 (socket says 172.21.15.92:55290) 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 -- 172.21.15.92:0/3494203489 learned_addr learned my addr 172.21.15.92:0/3494203489 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 -- 172.21.15.92:0/3494203489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40b1420 msgr2=0x7f80c416b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40b1420 0x7f80c416b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 -- 172.21.15.92:0/3494203489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c40ccf90 msgr2=0x7f80c4154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c40ccf90 0x7f80c4154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 -- 172.21.15.92:0/3494203489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009a20 con 0x7f80c40cbf30 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80c8f2c700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 0x7f80c4168760 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f80b400c5b0 tx=0x7f80b4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:07.648 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4005b30 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4005c90 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c416bb80 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4007300 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.642+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c416c140 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.643+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c404c3c0 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.643+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f80b401b3b0 con 0x7f80c40cbf30 2023-10-06T03:14:07.649 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.644+0000 7f80b97fa700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80a8067320 0x7f80a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.644+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f80b4084000 con 0x7f80c40cbf30 2023-10-06T03:14:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.644+0000 7f80c972d700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80a8067320 0x7f80a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.644+0000 7f80c972d700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80a8067320 0x7f80a80697d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009a60 tx=0x7f80c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:07.650 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.648+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80b4050e30 con 0x7f80c40cbf30 2023-10-06T03:14:07.829 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.829+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 --> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80c40ad650 con 0x7f80a8067320 2023-10-06T03:14:07.832 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.831+0000 7f80b97fa700 1 -- 172.21.15.92:0/3494203489 <== mgr.4104 v2:172.21.15.92:6832/90454 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73211 (secure 0 0 0) 0x7f80c40ad650 con 0x7f80a8067320 2023-10-06T03:14:07.833 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:14:07.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.833+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80a8067320 msgr2=0x7f80a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.834 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.833+0000 7f80cb991700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80a8067320 0x7f80a80697d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009a60 tx=0x7f80c0008040 comp rx=0 tx=0).stop 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.833+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 msgr2=0x7f80c4168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.833+0000 7f80cb991700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 0x7f80c4168760 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f80b400c5b0 tx=0x7f80b4001070 comp rx=0 tx=0).stop 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.833+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 shutdown_connections 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.833+0000 7f80cb991700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f80a8067320 0x7f80a80697d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.834+0000 7f80cb991700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80c40ccf90 0x7f80c4154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.834+0000 7f80cb991700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80c40b1420 0x7f80c416b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.835 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.834+0000 7f80cb991700 1 --2- 172.21.15.92:0/3494203489 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f80c40cbf30 0x7f80c4168760 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.834+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 >> 172.21.15.92:0/3494203489 conn(0x7f80c406a320 msgr2=0x7f80c40cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:14:07.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.834+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 shutdown_connections 2023-10-06T03:14:07.836 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.834+0000 7f80cb991700 1 -- 172.21.15.92:0/3494203489 wait complete. 2023-10-06T03:14:07.836 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2023-10-06T03:14:07.846 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":578,"stamp":"2023-10-06T03:14:07.340967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2432744,"kb_used_data":21672,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752541976,"statfs":{"total":773094113280,"available":770602983424,"internally_reserved":0,"allocated":22192128,"data_stored":11168288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002616"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":175,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:46.722166+0000","last_change":"2023-10-06T03:13:46.722138+0000","last_active":"2023-10-06T03:13:46.722166+0000","last_peered":"2023-10-06T03:13:46.722166+0000","last_clean":"2023-10-06T03:13:46.722166+0000","last_became_active":"2023-10-06T03:01:37.252074+0000","last_became_peered":"2023-10-06T03:01:37.252074+0000","last_unstale":"2023-10-06T03:13:46.722166+0000","last_undegraded":"2023-10-06T03:13:46.722166+0000","last_fullsized":"2023-10-06T03:13:46.722166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:46.722018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:46.722018+0000","last_clean_scrub_stamp":"2023-10-06T03:13:46.722018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:10:10.944606+0000","scrub_duration":0.001923931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":183,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:44.979687+0000","last_change":"2023-10-06T03:13:44.979666+0000","last_active":"2023-10-06T03:13:44.979687+0000","last_peered":"2023-10-06T03:13:44.979687+0000","last_clean":"2023-10-06T03:13:44.979687+0000","last_became_active":"2023-10-06T03:01:37.252005+0000","last_became_peered":"2023-10-06T03:01:37.252005+0000","last_unstale":"2023-10-06T03:13:44.979687+0000","last_undegraded":"2023-10-06T03:13:44.979687+0000","last_fullsized":"2023-10-06T03:13:44.979687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:44.979583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:44.979583+0000","last_clean_scrub_stamp":"2023-10-06T03:13:44.979583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:30:57.102086+0000","scrub_duration":0.0019708680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":178,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:54.106177+0000","last_change":"2023-10-06T03:13:54.106150+0000","last_active":"2023-10-06T03:13:54.106177+0000","last_peered":"2023-10-06T03:13:54.106177+0000","last_clean":"2023-10-06T03:13:54.106177+0000","last_became_active":"2023-10-06T03:01:38.142873+0000","last_became_peered":"2023-10-06T03:01:38.142873+0000","last_unstale":"2023-10-06T03:13:54.106177+0000","last_undegraded":"2023-10-06T03:13:54.106177+0000","last_fullsized":"2023-10-06T03:13:54.106177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:54.106037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:54.106037+0000","last_clean_scrub_stamp":"2023-10-06T03:13:54.106037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:55.705177+0000","scrub_duration":0.0022444359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:52.086053+0000","last_change":"2023-10-06T03:13:52.086016+0000","last_active":"2023-10-06T03:13:52.086053+0000","last_peered":"2023-10-06T03:13:52.086053+0000","last_clean":"2023-10-06T03:13:52.086053+0000","last_became_active":"2023-10-06T03:01:37.253628+0000","last_became_peered":"2023-10-06T03:01:37.253628+0000","last_unstale":"2023-10-06T03:13:52.086053+0000","last_undegraded":"2023-10-06T03:13:52.086053+0000","last_fullsized":"2023-10-06T03:13:52.086053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:52.085911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:52.085911+0000","last_clean_scrub_stamp":"2023-10-06T03:13:52.085911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:31:04.208414+0000","scrub_duration":0.002058519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"206'119","reported_seq":333,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:50.055811+0000","last_change":"2023-10-06T03:13:50.055770+0000","last_active":"2023-10-06T03:13:50.055811+0000","last_peered":"2023-10-06T03:13:50.055811+0000","last_clean":"2023-10-06T03:13:50.055811+0000","last_became_active":"2023-10-06T03:01:30.688339+0000","last_became_peered":"2023-10-06T03:01:30.688339+0000","last_unstale":"2023-10-06T03:13:50.055811+0000","last_undegraded":"2023-10-06T03:13:50.055811+0000","last_fullsized":"2023-10-06T03:13:50.055811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"206'119","last_scrub_stamp":"2023-10-06T03:13:50.055636+0000","last_deep_scrub":"206'119","last_deep_scrub_stamp":"2023-10-06T03:13:50.055636+0000","last_clean_scrub_stamp":"2023-10-06T03:13:50.055636+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:38:35.136396+0000","scrub_duration":0.0053024049999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:46.289260+0000","last_change":"2023-10-06T03:13:46.289231+0000","last_active":"2023-10-06T03:13:46.289260+0000","last_peered":"2023-10-06T03:13:46.289260+0000","last_clean":"2023-10-06T03:13:46.289260+0000","last_became_active":"2023-10-06T03:01:37.253099+0000","last_became_peered":"2023-10-06T03:01:37.253099+0000","last_unstale":"2023-10-06T03:13:46.289260+0000","last_undegraded":"2023-10-06T03:13:46.289260+0000","last_fullsized":"2023-10-06T03:13:46.289260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:46.289123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:46.289123+0000","last_clean_scrub_stamp":"2023-10-06T03:13:46.289123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:25.230622+0000","scrub_duration":0.0018054119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":179,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:51.622813+0000","last_change":"2023-10-06T03:13:51.622779+0000","last_active":"2023-10-06T03:13:51.622813+0000","last_peered":"2023-10-06T03:13:51.622813+0000","last_clean":"2023-10-06T03:13:51.622813+0000","last_became_active":"2023-10-06T03:01:37.381239+0000","last_became_peered":"2023-10-06T03:01:37.381239+0000","last_unstale":"2023-10-06T03:13:51.622813+0000","last_undegraded":"2023-10-06T03:13:51.622813+0000","last_fullsized":"2023-10-06T03:13:51.622813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:51.622678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:51.622678+0000","last_clean_scrub_stamp":"2023-10-06T03:13:51.622678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:43:12.894522+0000","scrub_duration":0.0017691250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":199,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:48.288746+0000","last_change":"2023-10-06T03:13:48.288719+0000","last_active":"2023-10-06T03:13:48.288746+0000","last_peered":"2023-10-06T03:13:48.288746+0000","last_clean":"2023-10-06T03:13:48.288746+0000","last_became_active":"2023-10-06T03:01:37.253223+0000","last_became_peered":"2023-10-06T03:01:37.253223+0000","last_unstale":"2023-10-06T03:13:48.288746+0000","last_undegraded":"2023-10-06T03:13:48.288746+0000","last_fullsized":"2023-10-06T03:13:48.288746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:48.288591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:48.288591+0000","last_clean_scrub_stamp":"2023-10-06T03:13:48.288591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:25.556829+0000","scrub_duration":0.0021249009999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":183,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T03:13:45.947097+0000","last_change":"2023-10-06T03:13:45.947068+0000","last_active":"2023-10-06T03:13:45.947097+0000","last_peered":"2023-10-06T03:13:45.947097+0000","last_clean":"2023-10-06T03:13:45.947097+0000","last_became_active":"2023-10-06T03:01:38.142696+0000","last_became_peered":"2023-10-06T03:01:38.142696+0000","last_unstale":"2023-10-06T03:13:45.947097+0000","last_undegraded":"2023-10-06T03:13:45.947097+0000","last_fullsized":"2023-10-06T03:13:45.947097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:13:45.946983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:13:45.946983+0000","last_clean_scrub_stamp":"2023-10-06T03:13:45.946983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:47.546123+0000","scrub_duration":0.0016744640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51800000000000002,"15min":0.58799999999999997},"min":{"1min":0.29499999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73899999999999999,"5min":0.78400000000000003,"15min":6.0110000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54000000000000004,"15min":0.52600000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.78100000000000003,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.622}]},{"osd":1,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.51800000000000002},"min":{"1min":0.35599999999999998,"5min":0.34100000000000003,"15min":0.316},"max":{"1min":0.73099999999999998,"5min":0.86399999999999999,"15min":1.0409999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74099999999999999,"5min":0.85299999999999998,"15min":1.5049999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.54200000000000004},"min":{"1min":0.35099999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":1.264},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.40000000000000002,"5min":0.34699999999999998,"15min":0.31900000000000001},"max":{"1min":0.751,"5min":0.79800000000000004,"15min":1.0960000000000001},"last":0.68000000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.56299999999999994},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77700000000000002,"5min":1.103,"15min":2.286},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":1.0349999999999999,"15min":1.1890000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.30199999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63900000000000001,"5min":0.77300000000000002,"15min":4.0289999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56899999999999995,"15min":0.59599999999999997},"min":{"1min":0.36699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":2.504},"last":0.41699999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68700000000000006,"5min":0.88500000000000001,"15min":1.141},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55600000000000005,"15min":0.56799999999999995},"min":{"1min":0.372,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70999999999999996,"5min":0.84799999999999998,"15min":1.639},"last":0.439}]},{"osd":6,"last update":"Fri Oct 6 03:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32300000000000001},"max":{"1min":0.68300000000000005,"5min":0.88400000000000001,"15min":1.5649999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.314,"5min":0.307,"15min":0.307},"max":{"1min":0.69399999999999995,"5min":0.82799999999999996,"15min":2.3359999999999999},"last":0.38}]}]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.56200000000000006},"min":{"1min":0.33700000000000002,"5min":0.30099999999999999,"15min":0.28999999999999998},"max":{"1min":0.69499999999999995,"5min":0.91400000000000003,"15min":1.6200000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.60399999999999998,"15min":0.57999999999999996},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.67200000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53500000000000003,"15min":0.51200000000000001},"min":{"1min":0.36299999999999999,"5min":0.35499999999999998,"15min":0.318},"max":{"1min":0.67300000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56000000000000005,"15min":0.53500000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32100000000000001},"max":{"1min":0.67000000000000004,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34200000000000003},"max":{"1min":0.70999999999999996,"5min":0.75,"15min":1.6359999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.315},"max":{"1min":0.67800000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.55600000000000005},"min":{"1min":0.41599999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68100000000000005,"5min":0.77200000000000002,"15min":0.85099999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.76100000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48599999999999999,"15min":0.51300000000000001},"min":{"1min":0.33900000000000002,"5min":0.28799999999999998,"15min":0.28499999999999998},"max":{"1min":0.55100000000000005,"5min":0.80700000000000005,"15min":0.88700000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.52200000000000002,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63700000000000001,"5min":0.873,"15min":0.873},"last":0.36499999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51200000000000001,"15min":0.53000000000000003},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.28199999999999997},"max":{"1min":0.71199999999999997,"5min":0.73799999999999999,"15min":0.85299999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.55700000000000005,"15min":0.57399999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.32100000000000001},"max":{"1min":0.65000000000000002,"5min":0.85199999999999998,"15min":0.85699999999999998},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.55900000000000005,"15min":0.57699999999999996},"min":{"1min":0.38200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.35999999999999999,"15min":0.32400000000000001},"max":{"1min":0.73399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.34799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.35699999999999998,"15min":0.34300000000000003},"min":{"1min":0.23400000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":3.1139999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.30199999999999999,"5min":0.255,"15min":0.24399999999999999},"max":{"1min":0.61699999999999999,"5min":0.71799999999999997,"15min":3.6520000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.36499999999999999,"15min":0.34200000000000003},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.20200000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":1.282},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35399999999999998,"15min":0.34000000000000002},"min":{"1min":0.26500000000000001,"5min":0.222,"15min":0.20000000000000001},"max":{"1min":0.50700000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38,"15min":0.34899999999999998},"min":{"1min":0.27900000000000003,"5min":0.223,"15min":0.20699999999999999},"max":{"1min":0.52900000000000003,"5min":0.69999999999999996,"15min":1.25},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.40699999999999997,"15min":0.38},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.56699999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63}]},{"osd":4,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.34699999999999998,"15min":0.34200000000000003},"max":{"1min":0.80000000000000004,"5min":1.1060000000000001,"15min":1.302},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":1.3240000000000001},"last":0.90900000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.96299999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.441,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":1.0629999999999999,"15min":5.0270000000000001},"last":0.92300000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.45600000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89700000000000002,"5min":0.97699999999999998,"15min":1.3360000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61399999999999999,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":1.052,"15min":1.3520000000000001},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63700000000000001,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.35899999999999999},"max":{"1min":0.91100000000000003,"5min":1.0820000000000001,"15min":1.379},"last":0.875}]}]},{"osd":0,"up_from":13,"seq":55834575002,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1838812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.33400000000000002,"15min":0.317},"min":{"1min":0.20399999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.52400000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.33200000000000002},"min":{"1min":0.27500000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.36499999999999999,"15min":0.34499999999999997},"min":{"1min":0.24099999999999999,"5min":0.23999999999999999,"15min":0.23200000000000001},"max":{"1min":0.53000000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.371,"15min":0.35599999999999998},"min":{"1min":0.29299999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55400000000000005,"5min":0.627,"15min":0.627},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.374,"15min":0.34899999999999998},"min":{"1min":0.28699999999999998,"5min":0.255,"15min":0.20899999999999999},"max":{"1min":0.59899999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.40600000000000003,"15min":0.38700000000000001},"min":{"1min":0.27100000000000002,"5min":0.252,"15min":0.251},"max":{"1min":0.60999999999999999,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58699999999999997,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.39800000000000002,"15min":0.35499999999999998},"max":{"1min":0.76000000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.44800000000000001,"5min":0.377,"15min":0.371},"max":{"1min":0.85999999999999999,"5min":0.90200000000000002,"15min":0.99199999999999999},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59899999999999998,"15min":0.59099999999999997},"min":{"1min":0.49299999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.92100000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77400000000000002,"5min":0.94499999999999995,"15min":0.94799999999999995},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.51800000000000002,"5min":0.40500000000000003,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.84899999999999998,"15min":0.93700000000000006},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":1.0089999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.628,"15min":0.60599999999999998},"min":{"1min":0.501,"5min":0.40999999999999998,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.94699999999999995,"15min":0.96099999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82499999999999996,"5min":0.93500000000000005,"15min":0.97999999999999998},"last":0.69799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.23899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51400000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.33400000000000002},"min":{"1min":0.249,"5min":0.23300000000000001,"15min":0.20499999999999999},"max":{"1min":0.53900000000000003,"5min":0.624,"15min":0.624},"last":0.438}]},{"osd":1,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.34000000000000002},"min":{"1min":0.28299999999999997,"5min":0.23599999999999999,"15min":0.23000000000000001},"max":{"1min":0.55200000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.372,"15min":0.35399999999999998},"min":{"1min":0.26800000000000002,"5min":0.25700000000000001,"15min":0.24399999999999999},"max":{"1min":0.502,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.35499999999999998,"15min":0.33500000000000002},"min":{"1min":0.28899999999999998,"5min":0.23799999999999999,"15min":0.222},"max":{"1min":0.53000000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.38100000000000001},"min":{"1min":0.308,"5min":0.28399999999999997,"15min":0.249},"max":{"1min":0.56100000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57999999999999996,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.38},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.57399999999999995},"min":{"1min":0.45800000000000002,"5min":0.435,"15min":0.374},"max":{"1min":0.74399999999999999,"5min":1.333,"15min":1.333},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.48099999999999998,"5min":0.42599999999999999,"15min":0.35799999999999998},"max":{"1min":0.79200000000000004,"5min":0.999,"15min":0.999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60599999999999998,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.34499999999999997},"max":{"1min":0.85599999999999998,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.58299999999999996},"min":{"1min":0.48799999999999999,"5min":0.42199999999999999,"15min":0.34399999999999997},"max":{"1min":0.76600000000000001,"5min":1.216,"15min":1.216},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.42999999999999999,"5min":0.41999999999999998,"15min":0.36599999999999999},"max":{"1min":0.83699999999999997,"5min":1.2569999999999999,"15min":1.8740000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.46800000000000003,"5min":0.41299999999999998,"15min":0.39200000000000002},"max":{"1min":0.82699999999999996,"5min":1.2789999999999999,"15min":1.903},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61799999999999999,"15min":0.58299999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.376},"max":{"1min":0.87,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.73999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34499999999999997,"15min":0.32200000000000001},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.19600000000000001},"max":{"1min":0.46899999999999997,"5min":0.56000000000000005,"15min":0.65200000000000002},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36199999999999999,"15min":0.34200000000000003},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.216},"max":{"1min":0.45500000000000002,"5min":0.52300000000000002,"15min":0.72299999999999998},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34899999999999998,"15min":0.32100000000000001},"min":{"1min":0.23699999999999999,"5min":0.23300000000000001,"15min":0.222},"max":{"1min":0.51000000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.36099999999999999,"15min":0.33900000000000002},"min":{"1min":0.28199999999999997,"5min":0.218,"15min":0.216},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.69699999999999995},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"min":{"1min":0.27300000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52800000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38700000000000001,"15min":0.36499999999999999},"min":{"1min":0.313,"5min":0.253,"15min":0.23999999999999999},"max":{"1min":0.51100000000000001,"5min":0.60499999999999998,"15min":0.67600000000000005},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.55400000000000005},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58299999999999996,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.47199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":1.2629999999999999,"15min":5.5220000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.43099999999999999,"5min":0.379,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.89900000000000002,"15min":0.997},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.56699999999999995},"min":{"1min":0.40200000000000002,"5min":0.39700000000000002,"15min":0.315},"max":{"1min":0.76200000000000001,"5min":0.86099999999999999,"15min":0.86499999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.47299999999999998,"5min":0.41699999999999998,"15min":0.378},"max":{"1min":0.81100000000000005,"5min":0.82599999999999996,"15min":0.98499999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.44900000000000001,"5min":0.38500000000000001,"15min":0.34599999999999997},"max":{"1min":0.79000000000000004,"5min":0.91100000000000003,"15min":1.0660000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.58499999999999996},"min":{"1min":0.48499999999999999,"5min":0.40999999999999998,"15min":0.35599999999999998},"max":{"1min":0.88200000000000001,"5min":1.446,"15min":1.446},"last":0.70799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.52900000000000003},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.30099999999999999},"max":{"1min":0.78000000000000003,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61199999999999999,"15min":0.58099999999999996},"min":{"1min":0.51600000000000001,"5min":0.39500000000000002,"15min":0.307},"max":{"1min":0.79300000000000004,"5min":1.069,"15min":1.4410000000000001},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.434,"5min":0.32100000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":1.0669999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.52800000000000002},"min":{"1min":0.42799999999999999,"5min":0.34100000000000003,"15min":0.307},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.0009999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55900000000000005},"min":{"1min":0.442,"5min":0.36499999999999999,"15min":0.315},"max":{"1min":0.76500000000000001,"5min":0.92600000000000005,"15min":3},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57599999999999996,"15min":0.64800000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.80300000000000005,"5min":0.84999999999999998,"15min":10.084},"last":0.628}]},{"osd":3,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.45000000000000001,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":0.72999999999999998,"5min":1.1160000000000001,"15min":1.5029999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.60299999999999998,"15min":0.621},"min":{"1min":0.501,"5min":0.38700000000000001,"15min":0.34599999999999997},"max":{"1min":0.76200000000000001,"5min":0.94899999999999995,"15min":4.9939999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48199999999999998,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66100000000000003,"5min":0.749,"15min":0.83999999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52400000000000002,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.312,"15min":0.29099999999999998},"max":{"1min":0.67800000000000005,"5min":0.97599999999999998,"15min":1.54},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.53100000000000003,"15min":0.54000000000000004},"min":{"1min":0.34599999999999997,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.72399999999999998,"5min":0.89700000000000002,"15min":0.90500000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.33100000000000002,"15min":0.32600000000000001},"max":{"1min":0.70999999999999996,"5min":0.86599999999999999,"15min":0.90300000000000002},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.38700000000000001,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.70599999999999996,"5min":1.167,"15min":1.167},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59099999999999997,"15min":0.59999999999999998},"min":{"1min":0.40600000000000003,"5min":0.35699999999999998,"15min":0.32900000000000001},"max":{"1min":0.76000000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1248444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.49199999999999999,"5min":0.30099999999999999,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":2.3599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28100000000000003},"max":{"1min":0.752,"5min":0.83499999999999996,"15min":2.371},"last":0.621}]},{"osd":1,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.316,"15min":0.30599999999999999},"max":{"1min":0.73899999999999999,"5min":0.82399999999999995,"15min":2.3410000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.45700000000000002,"5min":0.33100000000000002,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.83999999999999997,"15min":4.1219999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54400000000000004,"15min":0.57699999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.33400000000000002},"max":{"1min":0.70199999999999996,"5min":0.92400000000000004,"15min":6.532},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55500000000000005,"15min":0.629},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.315},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":5.6500000000000004},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55000000000000004,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.34799999999999998,"15min":0.315},"max":{"1min":0.74099999999999999,"5min":0.85299999999999998,"15min":5.4630000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55700000000000005,"15min":0.56799999999999995},"min":{"1min":0.42899999999999999,"5min":0.38,"15min":0.36899999999999999},"max":{"1min":0.77000000000000002,"5min":0.86099999999999999,"15min":5.524},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47599999999999998,"15min":0.503},"min":{"1min":0.33500000000000002,"5min":0.316,"15min":0.27100000000000002},"max":{"1min":0.78300000000000003,"5min":0.88900000000000001,"15min":0.92900000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51000000000000001,"15min":0.54700000000000004},"min":{"1min":0.33100000000000002,"5min":0.29399999999999998,"15min":0.29299999999999998},"max":{"1min":0.78800000000000003,"5min":0.90300000000000002,"15min":0.97299999999999998},"last":0.373}]},{"osd":6,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.48899999999999999,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":1.1990000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.61099999999999999},"min":{"1min":0.42299999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.873,"5min":0.95399999999999996,"15min":1.052},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54800000000000004,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.34699999999999998},"max":{"1min":0.81799999999999995,"5min":0.878,"15min":1.175},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54800000000000004,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.84699999999999998,"5min":0.93899999999999995,"15min":1.145},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:14:07.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T03:14:07.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.963+0000 7f34d1722700 1 Processor -- start 2023-10-06T03:14:07.964 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.963+0000 7f34d1722700 1 -- start start 2023-10-06T03:14:07.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.964+0000 7f34d1722700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 0x7f34cc0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.964+0000 7f34d1722700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc0c9310 con 0x7f34cc1579c0 2023-10-06T03:14:07.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.964+0000 7f34d1722700 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc157d90 con 0x7f34cc0c97a0 2023-10-06T03:14:07.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.964+0000 7f34d1722700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0c8700 con 0x7f34cc068aa0 2023-10-06T03:14:07.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.964+0000 7f34cbfff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f34cc1579c0 0x7f34cc06b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48594/0 (socket says 172.21.15.92:48594) 2023-10-06T03:14:07.968 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.964+0000 7f34cbfff700 1 -- 172.21.15.92:0/1484861980 learned_addr learned my addr 172.21.15.92:0/1484861980 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34d0f21700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 0x7f34cc0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735216153 0 0) 0x7f34cc0c9310 con 0x7f34cc1579c0 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34b4003570 con 0x7f34cc1579c0 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873264983 0 0) 0x7f34b4003570 con 0x7f34cc1579c0 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 msgr2=0x7f34cc0b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 0x7f34cc0b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.969 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 >> v1:172.21.15.136:6789/0 conn(0x7f34cc0c97a0 legacy=0x7f34cc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0c89e0 con 0x7f34cc1579c0 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f34bc001470 con 0x7f34cc1579c0 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 >> v1:172.21.15.92:6789/0 conn(0x7f34cc1579c0 legacy=0x7f34cc06b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34b4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34b4004810 0x7f34b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 0x7f34b40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.970 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0c8700 con 0x7f34b4003a90 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b400bb60 con 0x7f34b4004810 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b400bd20 con 0x7f34cc068aa0 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34b4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cb7fe700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34b4004810 0x7f34b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34d0f21700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 0x7f34b40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 -- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 msgr2=0x7f34b40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:07.971 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc068aa0 0x7f34b40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 -- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34b4004810 msgr2=0x7f34b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34b4004810 0x7f34b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 -- 172.21.15.92:0/1484861980 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4003570 con 0x7f34b4003a90 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.965+0000 7f34cbfff700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34b4003ea0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00c3c0 tx=0x7f34bc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9cce820d0348e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00cc90 con 0x7f34b4003a90 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc00cdf0 con 0x7f34b4003a90 2023-10-06T03:14:07.972 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34caffd700 1 -- 172.21.15.92:0/1484861980 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc005b50 con 0x7f34b4003a90 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d1722700 1 -- 172.21.15.92:0/1484861980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 msgr2=0x7f34b4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d1722700 1 --2- 172.21.15.92:0/1484861980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34b4003ea0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00c3c0 tx=0x7f34bc001e50 comp rx=0 tx=0).stop 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d0f21700 1 -- 172.21.15.92:0/1484861980 reap_dead start 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d1722700 1 -- 172.21.15.92:0/1484861980 shutdown_connections 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d1722700 1 -- 172.21.15.92:0/1484861980 >> 172.21.15.92:0/1484861980 conn(0x7f34cc0cddc0 msgr2=0x7f34cc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d1722700 1 -- 172.21.15.92:0/1484861980 shutdown_connections 2023-10-06T03:14:07.973 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.966+0000 7f34d1722700 1 -- 172.21.15.92:0/1484861980 wait complete. 2023-10-06T03:14:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.967+0000 7f34d1722700 1 Processor -- start 2023-10-06T03:14:07.974 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.967+0000 7f34d1722700 1 -- start start 2023-10-06T03:14:07.976 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d1722700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34cc151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d1722700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc16cef0 0x7f34cc16b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d1722700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34cc16b9b0 0x7f34cc0c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d1722700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc16be50 con 0x7f34b4003a90 2023-10-06T03:14:07.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d1722700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc16c130 con 0x7f34cc16b9b0 2023-10-06T03:14:07.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d1722700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc16c410 con 0x7f34cc16cef0 2023-10-06T03:14:07.977 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34cbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34cc151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34cbfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34cc151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55294/0 (socket says 172.21.15.92:55294) 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34cbfff700 1 -- 172.21.15.92:0/3640610335 learned_addr learned my addr 172.21.15.92:0/3640610335 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34d0f21700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34cc16b9b0 0x7f34cc0c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.968+0000 7f34cb7fe700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc16cef0 0x7f34cc16b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cbfff700 1 -- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc16cef0 msgr2=0x7f34cc16b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cbfff700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc16cef0 0x7f34cc16b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.978 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cbfff700 1 -- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34cc16b9b0 msgr2=0x7f34cc0c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cbfff700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34cc16b9b0 0x7f34cc0c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cbfff700 1 -- 172.21.15.92:0/3640610335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc16c6f0 con 0x7f34b4003a90 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34d0f21700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34cc16b9b0 0x7f34cc0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cb7fe700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc16cef0 0x7f34cc16b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34cbfff700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34cc151210 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f34b400bb30 tx=0x7f34bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00cc90 con 0x7f34b4003a90 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc0072e0 con 0x7f34b4003a90 2023-10-06T03:14:07.979 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0076a0 con 0x7f34b4003a90 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc16c9d0 con 0x7f34b4003a90 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.969+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc151750 con 0x7f34b4003a90 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.970+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc16cd40 con 0x7f34b4003a90 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.970+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f34bc019030 con 0x7f34b4003a90 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.971+0000 7f34b3fff700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f34b8067320 0x7f34b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.971+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f34bc085eb0 con 0x7f34b4003a90 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.971+0000 7f34cb7fe700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f34b8067320 0x7f34b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:07.980 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.971+0000 7f34cb7fe700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f34b8067320 0x7f34b80697d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f34c0007fd0 tx=0x7f34c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:07.981 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:07.975+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34bc052ce0 con 0x7f34b4003a90 2023-10-06T03:14:08.157 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.157+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f34cc0c8cc0 con 0x7f34b4003a90 2023-10-06T03:14:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.161+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f34bc052ce0 con 0x7f34b4003a90 2023-10-06T03:14:08.162 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.161+0000 7f34b3fff700 1 -- 172.21.15.92:0/3640610335 <== mon.0 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v34)=0 v34) v1 ==== 123+0+0 (secure 0 0 0) 0x7f34bc056300 con 0x7f34b4003a90 2023-10-06T03:14:08.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f34b8067320 msgr2=0x7f34b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f34b8067320 0x7f34b80697d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f34c0007fd0 tx=0x7f34c0009040 comp rx=0 tx=0).stop 2023-10-06T03:14:08.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 msgr2=0x7f34cc151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34cc151210 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f34b400bb30 tx=0x7f34bc015000 comp rx=0 tx=0).stop 2023-10-06T03:14:08.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 shutdown_connections 2023-10-06T03:14:08.165 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f34b8067320 0x7f34b80697d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34b4003a90 0x7f34cc151210 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34cc16b9b0 0x7f34cc0c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 --2- 172.21.15.92:0/3640610335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34cc16cef0 0x7f34cc16b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 >> 172.21.15.92:0/3640610335 conn(0x7f34cc0cddc0 msgr2=0x7f34cc0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:14:08.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 shutdown_connections 2023-10-06T03:14:08.166 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.164+0000 7f34d1722700 1 -- 172.21.15.92:0/3640610335 wait complete. 2023-10-06T03:14:08.176 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:14:08.177 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:14:08.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.290+0000 7f84cda76700 1 Processor -- start 2023-10-06T03:14:08.291 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.290+0000 7f84cda76700 1 -- start start 2023-10-06T03:14:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cda76700 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cda76700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 0x7f84c815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cda76700 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80b2980 con 0x7f84c81597f0 2023-10-06T03:14:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cda76700 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c815dbb0 con 0x7f84c8159420 2023-10-06T03:14:08.295 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cda76700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b11d0 con 0x7f84c815d230 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84bffff700 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f84c81597f0 0x7f84c80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:48610/0 (socket says 172.21.15.92:48610) 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84bffff700 1 -- 172.21.15.92:0/2697931420 learned_addr learned my addr 172.21.15.92:0/2697931420 (peer_addr_for_me v1:172.21.15.92:0/0) 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cd275700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 0x7f84c815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84cca74700 1 --2- 172.21.15.92:0/2697931420 >> v2:172.21.15.136:3300/0 conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 <== mon.0 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202052550 0 0) 0x7f84c80b2980 con 0x7f84c81597f0 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.291+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b0003570 con 0x7f84c81597f0 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 <== mon.0 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530379809 0 0) 0x7f84b0003570 con 0x7f84c81597f0 2023-10-06T03:14:08.296 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 msgr2=0x7f84c815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 0x7f84c815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 >> v2:172.21.15.136:3300/0 conn(0x7f84c8159420 msgr2=0x7f84c80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 --2- 172.21.15.92:0/2697931420 >> v2:172.21.15.136:3300/0 conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80b14b0 con 0x7f84c81597f0 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 <== mon.0 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2576432426 0 0) 0x7f84b8001470 con 0x7f84c81597f0 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 >> v1:172.21.15.92:6789/0 conn(0x7f84c81597f0 legacy=0x7f84c80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b0003a60 0x7f84b0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.297 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 0x7f84b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b0004d50 0x7f84b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c815dbb0 con 0x7f84b0004d50 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80b11d0 con 0x7f84b0003a60 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b000ba60 con 0x7f84c815d230 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cca74700 1 --2- 172.21.15.92:0/2697931420 >> v2:172.21.15.136:3300/0 conn(0x7f84c8159420 0x7f84c80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b0004d50 0x7f84b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.298 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bffff700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 0x7f84b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cca74700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b0003a60 0x7f84b0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 -- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 msgr2=0x7f84b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c815d230 0x7f84b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 -- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b0003a60 msgr2=0x7f84b0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84b0003a60 0x7f84b0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 -- 172.21.15.92:0/2697931420 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0003570 con 0x7f84b0004d50 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84cd275700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b0004d50 0x7f84b00051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009c00 tx=0x7f84c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14a34d1787de03a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:08.299 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000e6b0 con 0x7f84b0004d50 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f84c000e810 con 0x7f84b0004d50 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.292+0000 7f84bf7fe700 1 -- 172.21.15.92:0/2697931420 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000ebf0 con 0x7f84b0004d50 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cda76700 1 -- 172.21.15.92:0/2697931420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b0004d50 msgr2=0x7f84b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cda76700 1 --2- 172.21.15.92:0/2697931420 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84b0004d50 0x7f84b00051c0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009c00 tx=0x7f84c00091a0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cd275700 1 -- 172.21.15.92:0/2697931420 reap_dead start 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cda76700 1 -- 172.21.15.92:0/2697931420 shutdown_connections 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cda76700 1 -- 172.21.15.92:0/2697931420 >> 172.21.15.92:0/2697931420 conn(0x7f84c80d0660 msgr2=0x7f84c80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:14:08.300 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cda76700 1 -- 172.21.15.92:0/2697931420 shutdown_connections 2023-10-06T03:14:08.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.293+0000 7f84cda76700 1 -- 172.21.15.92:0/2697931420 wait complete. 2023-10-06T03:14:08.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.294+0000 7f84cda76700 1 Processor -- start 2023-10-06T03:14:08.301 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.294+0000 7f84cda76700 1 -- start start 2023-10-06T03:14:08.303 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cda76700 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c806dbd0 0x7f84c8069880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cda76700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 0x7f84c806b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cda76700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84c806dfa0 0x7f84c806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cda76700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c815aca0 con 0x7f84c806c170 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cda76700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8154620 con 0x7f84c806dfa0 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cda76700 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c80693f0 con 0x7f84c806dbd0 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 0x7f84c806b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.304 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 0x7f84c806b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:55312/0 (socket says 172.21.15.92:55312) 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 -- 172.21.15.92:0/2819968784 learned_addr learned my addr 172.21.15.92:0/2819968784 (peer_addr_for_me v2:172.21.15.92:0/0) 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cca74700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c806dbd0 0x7f84c8069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cd275700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84c806dfa0 0x7f84c806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 -- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c806dbd0 msgr2=0x7f84c8069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c806dbd0 0x7f84c8069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 -- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84c806dfa0 msgr2=0x7f84c806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84c806dfa0 0x7f84c806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.305 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84bffff700 1 -- 172.21.15.92:0/2819968784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c806bbf0 con 0x7f84c806c170 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.295+0000 7f84cca74700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c806dbd0 0x7f84c8069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84cd275700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84c806dfa0 0x7f84c806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84bffff700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 0x7f84c806b6b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f84b8007390 tx=0x7f84b8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8018030 con 0x7f84c806c170 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f84b800f8d0 con 0x7f84c806c170 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c815aeb0 con 0x7f84c806c170 2023-10-06T03:14:08.306 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b800fcd0 con 0x7f84c806c170 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.296+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c8068da0 con 0x7f84c806c170 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.297+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c804c3c0 con 0x7f84c806c170 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.297+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f84b801b070 con 0x7f84c806c170 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.297+0000 7f84bcff9700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f84b4067320 0x7f84b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.297+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f84b8019070 con 0x7f84c806c170 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.298+0000 7f84cca74700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f84b4067320 0x7f84b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.298+0000 7f84cca74700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f84b4067320 0x7f84b40697d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f84c4007760 tx=0x7f84c400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:14:08.307 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.301+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84b80519a0 con 0x7f84c806c170 2023-10-06T03:14:08.524 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.524+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c8069530 con 0x7f84c806c170 2023-10-06T03:14:08.525 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.524+0000 7f84bcff9700 1 -- 172.21.15.92:0/2819968784 <== mon.0 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+3537 (secure 0 0 0) 0x7f84b8054fc0 con 0x7f84c806c170 2023-10-06T03:14:08.525 INFO:teuthology.orchestra.run.smithi092.stdout: 2023-10-06T03:14:08.526 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":139,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8068,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6834/4099408757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":4099408757},{"type":"v1","addr":"172.21.15.136:6834","nonce":4099408757}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":8071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3782820103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6833","nonce":3782820103},{"type":"v1","addr":"172.21.15.136:6835","nonce":3782820103}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9195,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6836/3173875082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":3173875082},{"type":"v1","addr":"172.21.15.92:6836","nonce":3173875082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131},{"gid":9480,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/462678056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6835","nonce":462678056},{"type":"v1","addr":"172.21.15.92:6837","nonce":462678056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[]} 2023-10-06T03:14:08.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f84b4067320 msgr2=0x7f84b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f84b4067320 0x7f84b40697d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f84c4007760 tx=0x7f84c400b040 comp rx=0 tx=0).stop 2023-10-06T03:14:08.527 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 msgr2=0x7f84c806b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 0x7f84c806b6b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f84b8007390 tx=0x7f84b8001e90 comp rx=0 tx=0).stop 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 shutdown_connections 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:6832/90454,v1:172.21.15.92:6833/90454] conn(0x7f84b4067320 0x7f84b40697d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84c806dfa0 0x7f84c806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84c806dbd0 0x7f84c8069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 --2- 172.21.15.92:0/2819968784 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f84c806c170 0x7f84c806b6b0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:14:08.528 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 >> 172.21.15.92:0/2819968784 conn(0x7f84c80d0660 msgr2=0x7f84c80cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:14:08.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 shutdown_connections 2023-10-06T03:14:08.529 INFO:teuthology.orchestra.run.smithi092.stderr:2023-10-06T03:14:08.526+0000 7f84cda76700 1 -- 172.21.15.92:0/2819968784 wait complete. 2023-10-06T03:14:08.529 INFO:teuthology.orchestra.run.smithi092.stderr:dumped fsmap epoch 139 2023-10-06T03:14:08.539 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T03:14:08.539 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T03:14:08.539 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:14:08.539 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.547 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:14:08.547 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:14:08.548 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.596 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:14:08.596 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:14:08.596 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.607 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:14:08.607 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:14:08.608 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.658 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:14:08.658 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T03:14:08.658 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T03:14:08.658 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.693 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T03:14:08.693 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T03:14:08.693 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.764 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T03:14:08.764 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T03:14:08.765 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.828 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T03:14:08.829 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T03:14:08.829 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.895 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T03:14:08.895 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T03:14:08.895 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.941 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T03:14:08.942 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T03:14:08.942 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:08.982 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T03:14:08.982 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T03:14:08.982 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.054 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T03:14:09.055 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T03:14:09.055 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.126 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T03:14:09.127 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T03:14:09.127 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T03:14:09.127 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.151 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T03:14:09.152 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T03:14:09.152 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.191 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T03:14:09.191 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T03:14:09.191 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T03:14:09.191 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.202 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T03:14:09.202 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T03:14:09.203 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.242 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T03:14:09.243 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T03:14:09.243 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:14:09.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-10-06T03:14:09.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-10-06T03:14:09.295 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T03:14:09.295 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T03:14:09.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T03:14:09.325 INFO:teuthology.orchestra.run.smithi092.stdout:1696561297.2393353 mon.a (mon.0) 192 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T03:14:09.325 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T03:14:09.325 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:14:09.325 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T03:14:09.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:14:09.654 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T03:14:09.746 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:14:09.747 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T03:14:09.832 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:14:09.832 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T03:14:09.891 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:14:09.891 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T03:14:10.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:14:10.114 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T03:14:10.194 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:14:10.194 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T03:14:10.265 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:14:10.265 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T03:14:10.339 INFO:tasks.ceph:Archiving mon data... 2023-10-06T03:14:10.340 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/data/mon.a.tgz 2023-10-06T03:14:10.341 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2023-10-06T03:14:10.355 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.w68mFBKU3T 2023-10-06T03:14:10.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.w68mFBKU3T 2023-10-06T03:14:10.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.w68mFBKU3T 2023-10-06T03:14:10.621 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.w68mFBKU3T is 623KB 2023-10-06T03:14:10.719 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.w68mFBKU3T 2023-10-06T03:14:10.733 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/data/mon.b.tgz 2023-10-06T03:14:10.733 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2023-10-06T03:14:10.750 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.aUfF0A9GFJ 2023-10-06T03:14:10.751 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aUfF0A9GFJ 2023-10-06T03:14:10.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.aUfF0A9GFJ 2023-10-06T03:14:11.036 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.aUfF0A9GFJ is 633KB 2023-10-06T03:14:11.106 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.aUfF0A9GFJ 2023-10-06T03:14:11.120 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/data/mon.c.tgz 2023-10-06T03:14:11.120 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2023-10-06T03:14:11.173 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.V67xSuZMaJ 2023-10-06T03:14:11.174 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.V67xSuZMaJ 2023-10-06T03:14:11.347 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.V67xSuZMaJ 2023-10-06T03:14:11.428 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.V67xSuZMaJ is 633KB 2023-10-06T03:14:11.527 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.V67xSuZMaJ 2023-10-06T03:14:11.541 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T03:14:11.542 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T03:14:11.544 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T03:14:11.605 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T03:14:11.605 DEBUG:teuthology.orchestra.run.smithi092:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T03:14:11.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T03:14:11.633 INFO:teuthology.orchestra.run.smithi092.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T03:14:11.669 INFO:teuthology.orchestra.run.smithi136.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T03:14:11.675 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T03:14:11.677 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/remote/smithi092/crash 2023-10-06T03:14:11.678 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T03:14:11.706 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/remote/smithi136/crash 2023-10-06T03:14:11.707 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T03:14:11.738 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T03:14:11.738 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:14:11.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:14:11.868 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T03:14:11.891 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T03:14:11.915 INFO:tasks.ceph:Compressing logs... 2023-10-06T03:14:11.915 DEBUG:teuthology.orchestra.run.smithi092:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T03:14:11.934 DEBUG:teuthology.orchestra.run.smithi136:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T03:14:14.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-10-06T03:14:14.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-10-06T03:14:23.561 INFO:tasks.ceph:Archiving logs... 2023-10-06T03:14:23.562 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/remote/smithi092/log 2023-10-06T03:14:23.563 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T03:14:25.459 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/remote/smithi136/log 2023-10-06T03:14:25.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T03:14:25.967 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T03:14:25.982 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T03:14:25.982 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T03:14:25.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T03:14:26.061 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T03:14:26.072 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T03:14:26.072 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T03:14:26.075 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T03:14:26.087 INFO:teuthology.orchestra.run.smithi092.stderr:bash: ntpq: command not found 2023-10-06T03:14:26.090 INFO:teuthology.orchestra.run.smithi092.stdout:210 Number of sources = 4 2023-10-06T03:14:26.090 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T03:14:26.090 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2023-10-06T03:14:26.090 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 -327us[ -257us] +/- 95ms 2023-10-06T03:14:26.090 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -331us[ -260us] +/- 52ms 2023-10-06T03:14:26.091 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +653us[ +724us] +/- 58ms 2023-10-06T03:14:26.091 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:14:26.091 INFO:teuthology.orchestra.run.smithi136.stderr:bash: ntpq: command not found 2023-10-06T03:14:26.093 INFO:teuthology.orchestra.run.smithi136.stdout:210 Number of sources = 4 2023-10-06T03:14:26.093 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T03:14:26.094 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2023-10-06T03:14:26.094 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 -267us[ -267us] +/- 95ms 2023-10-06T03:14:26.094 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -262us[ -257us] +/- 52ms 2023-10-06T03:14:26.094 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +722us[ +722us] +/- 58ms 2023-10-06T03:14:26.094 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:14:26.095 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T03:14:26.104 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T03:14:26.105 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T03:14:26.141 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T03:14:26.195 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T03:14:26.293 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T03:14:26.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:14:26.320 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T03:14:26.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:14:26.351 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T03:14:26.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T03:14:26.438 INFO:teuthology.task.internal:Duration was 1447.499542 seconds 2023-10-06T03:14:26.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T03:14:26.475 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T03:14:26.476 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T03:14:26.478 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T03:14:26.520 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T03:14:26.522 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T03:14:26.772 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T03:14:26.772 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2023-10-06T03:14:26.773 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T03:14:26.791 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2023-10-06T03:14:26.791 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T03:14:26.811 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T03:14:26.811 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T03:14:26.833 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T03:14:26.891 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T03:14:26.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T03:14:26.893 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T03:14:27.007 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T03:14:27.017 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T03:14:27.018 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T03:14:27.042 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T03:14:27.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T03:14:27.077 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T03:14:27.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T03:14:27.106 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2023-10-06T03:14:27.130 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2023-10-06T03:14:27.144 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T03:14:27.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:14:27.173 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T03:14:27.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:14:27.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T03:14:27.209 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T03:14:27.210 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/remote/smithi092 2023-10-06T03:14:27.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T03:14:27.248 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176/remote/smithi136 2023-10-06T03:14:27.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T03:14:27.284 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T03:14:27.284 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T03:14:27.286 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T03:14:27.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T03:14:27.344 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T03:14:27.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T03:14:27.375 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T03:14:27.375 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T03:14:27.377 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T03:14:27.389 INFO:teuthology.orchestra.run.smithi092.stdout: 263007 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 03:14 /home/ubuntu/cephtest 2023-10-06T03:14:27.391 INFO:teuthology.orchestra.run.smithi136.stdout: 263007 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 03:14 /home/ubuntu/cephtest 2023-10-06T03:14:27.393 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T03:14:27.417 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T03:14:27.502 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T03:14:27.527 DEBUG:teuthology.nuke:shortname: smithi092 2023-10-06T03:14:27.528 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T03:14:27.549 DEBUG:teuthology.nuke:shortname: smithi136 2023-10-06T03:14:27.550 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T03:14:27.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:43:07.402300', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNySbKwKtbn23kaOpdUEXi+78qW6CwALxFpT/RCDrTx+bL/kB6CcHKIvUERPjiPH16MiOBLWYiifw3LJjrmqQLo='} 2023-10-06T03:14:27.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412176', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 02:43:07.401011', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKH/OkfSWXHIbERePHsgBONEAbvU0eMC2nwTK72hhDG2ASmy/zmAyTPj/6R828c1bvdoAMZ2gsV8tNbd6Z76RR4='} 2023-10-06T03:14:27.604 INFO:teuthology.orchestra.console.smithi092:Power off 2023-10-06T03:14:27.604 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T03:14:27.627 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2023-10-06T03:14:27.628 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:27.648 INFO:teuthology.orchestra.console.smithi136:Power off 2023-10-06T03:14:27.648 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T03:14:27.657 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-10-06T03:14:27.673 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2023-10-06T03:14:27.673 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:27.696 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2023-10-06T03:14:31.658 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:31.698 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:31.769 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-10-06T03:14:31.809 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2023-10-06T03:14:35.772 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:35.810 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:35.882 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2023-10-06T03:14:35.920 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2023-10-06T03:14:36.021 INFO:teuthology.orchestra.console.smithi136:Power off completed 2023-10-06T03:14:39.884 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T03:14:40.065 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2023-10-06T03:14:40.166 INFO:teuthology.orchestra.console.smithi092:Power off completed 2023-10-06T03:14:40.270 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} duration: 1447.4995422363281 failure_reason: 'Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5f6d5b77ee1140afb4eb547cc433c1fa status: fail success: false 2023-10-06T03:14:40.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T03:14:40.344 INFO:teuthology.run:FAIL